Documentation Index Fetch the complete documentation index at: https://docs.semantictest.dev/llms.txt
Use this file to discover all available pages before exploring further.
Overview Assertions are the final step in your test pipeline. After all blocks execute, assertions check if the data in the DataBus matches your expectations.{ "pipeline" : [ ], "assertions" : { "response.status" : 200 , "user.id" : 1 , "validation.passed" : true } }
Simple Assertions (Equality) The simplest form checks for exact equality:
{ "assertions" : { "response.status" : 200 , "user.name" : "John Doe" , "data.count" : 10 , "validation.passed" : true } }
Rules:
String values must match exactly
Numbers must match exactly
Booleans must match exactly
Null values must match exactly
Operator Assertions For more complex validation, use operator objects with 20+ powerful operators.
Equality Operators { "assertions" : { "response.status" : { "equals" : 200 }, "user.id" : { "notEquals" : 0 } } }
Numeric Comparisons { "assertions" : { "user.age" : { "gt" : 18 }, "user.age" : { "gte" : 18 }, "response.time" : { "lt" : 1000 }, "data.count" : { "lte" : 100 } } }
gt: Greater thangte: Greater than or equallt: Less thanlte: Less than or equal
String/Array Operations { "assertions" : { "response.message" : { "contains" : "success" }, "user.roles" : { "contains" : "admin" }, "response.error" : { "notContains" : "fatal" }, "user.name" : { "minLength" : 1 }, "user.bio" : { "maxLength" : 500 } } }
For text: Checks if the string contains the substring or meets length requirementsFor arrays: Checks if the array contains the value or meets length requirementsPattern Matching { "assertions" : { "user.email" : { "matches" : ".*@example \\ .com" }, "data.phone" : { "matches" : "^ \\ d{3}- \\ d{3}- \\ d{4}$" }, "response.id" : { "notMatches" : "^temp-.*" } } }
Uses JavaScript regex syntax.
Type Checks { "assertions" : { "data.value" : { "isNull" : true }, "user.name" : { "isNotNull" : true }, "config.setting" : { "isDefined" : true }, "optional.field" : { "isUndefined" : true } } }
Boolean Checks { "assertions" : { "validation.passed" : { "isTrue" : true }, "hasError" : { "isFalse" : true } } }
Empty Checks { "assertions" : { "user.name" : { "isNotEmpty" : true }, "errors" : { "isEmpty" : true } } }
Works with strings, arrays, and objects.
Complete Example { "name" : "User API Test" , "tests" : [{ "id" : "test-user" , "pipeline" : [ { "block" : "HttpRequest" , "input" : { "url" : "https://api.example.com/users/1" , "method" : "GET" }, "output" : "response" }, { "block" : "JsonParser" , "input" : "${response.body}" , "output" : { "parsed" : "user" } }, { "block" : "LLMJudge" , "input" : { "text" : "${user}" , "expected" : { "expectedBehavior" : "Should have complete user profile" } }, "output" : "judgement" } ], "assertions" : { "response.status" : { "gte" : 200 , "lt" : 300 }, "user.id" : { "gt" : 0 }, "user.email" : { "matches" : ".*@.* \\ ..*" , "isNotEmpty" : true }, "user.age" : { "gte" : 0 , "lte" : 150 }, "user.roles" : { "contains" : "user" , "isNotEmpty" : true }, "user.name" : { "minLength" : 1 , "maxLength" : 100 }, "judgement.score" : { "gte" : 0.7 }, "judgement.reasoning" : { "isNotEmpty" : true } } }] }
Multiple Operators You can combine multiple operators on the same field - all must pass:
{ "assertions" : { "user.age" : { "gte" : 18 , "lte" : 100 }, "response.message" : { "contains" : "success" , "matches" : "^[A-Z].*" , "isNotEmpty" : true }, "user.email" : { "matches" : ".*@.* \\ ..*" , "isNotEmpty" : true , "minLength" : 5 } } }
Example explanations:
user.age: Must be between 18 and 100response.message: Must contain “success” AND start with capital letteruser.email: Must be valid email format AND not empty AND at least 5 characters
All operators in a group are evaluated with AND logic - every operator must pass for the assertion to succeed.
Common Patterns HTTP Status
AI Judge
Data Validation
Tool Validation
{ "assertions" : { "response.status" : { "gte" : 200 , "lt" : 300 }, "response.body" : { "isNotEmpty" : true } } }
Check for success status range (200-299) and non-empty response body. { "assertions" : { "judgement.score" : { "gte" : 0.7 , "lte" : 1.0 }, "judgement.reasoning" : { "isNotEmpty" : true } } }
Ensure quality score is between 0.7 and 1.0 and reasoning is provided. { "assertions" : { "user.email" : { "matches" : ".*@.* \\ ..*" , "isNotEmpty" : true }, "user.id" : { "gt" : 0 }, "user.name" : { "minLength" : 1 , "maxLength" : 100 }, "user.bio" : { "maxLength" : 500 }, "user.createdAt" : { "isDefined" : true }, "user.nickname" : { "maxLength" : 50 } } }
Validate email format, required fields, and field length constraints. { "assertions" : { "toolValidation.passed" : true , "toolValidation.actualTools" : { "contains" : "search_users" } } }
Ensure tool validation passed and expected tools were called. Assertion Failures When an assertion fails, you get detailed error messages:
❌ test-user: User API Test (234ms) ✅ response.status = 200 ❌ user.age: Expected 25 to be > 30 ✅ user.email matches ".*@.* \\ ..*" ❌ judgement.score: Expected 0.6 to be > 0.7
Best Practices
Test Both Success and Failure Cases
Test successful response: { "id" : "success-case" , "pipeline" : [], "assertions" : { "response.status" : 200 , "validation.passed" : true } }
Test error handling: { "id" : "error-case" , "pipeline" : [], "assertions" : { "response.status" : 400 , "validation.passed" : false } }
Use Ranges Instead of Exact Values
Bad - too specific: { "judgement.score" : 0.85 }
Good - allows for variance: { "judgement.score" : { "gt" : 0.7 } }
Validate Structure and Content
{ "assertions" : { "user.id" : { "gt" : 0 }, "user.email" : { "isDefined" : true , "isNotEmpty" : true }, "user.name" : { "isNotEmpty" : true }, "user.email" : { "matches" : ".*@.* \\ ..*" }, "user.name" : { "minLength" : 1 , "maxLength" : 100 } } }
Structure checks: Required fields exist and are not emptyContent checks: Data values are valid (format, length, etc.)
Bad - too many assertions make tests fragile: { "user.name" : "John Doe" , "user.age" : 30 , "user.email" : "john@example.com" , "user.address.street" : "123 Main St" , "user.address.city" : "Springfield" }
Good - test what matters: { "user.id" : 1 , "user.email" : { "matches" : ".*@.* \\ ..*" }, "response.status" : 200 }
If any specific detail changes, overly specific tests will break unnecessarily. Operator Reference Equality Operators Operator Description Example equalsValue equals expected { "equals": 200 }notEqualsValue does not equal expected { "notEquals": 404 }
Numeric Operators Operator Description Example gtGreater than { "gt": 10 }gteGreater than or equal to { "gte": 18 }ltLess than { "lt": 100 }lteLess than or equal to { "lte": 200 }
String/Array Operators Operator Description Example containsString/array contains value { "contains": "text" }notContainsString/array does not contain value { "notContains": "error" }minLengthString/array length >= value { "minLength": 1 }maxLengthString/array length <= value { "maxLength": 100 }
Pattern Operators Operator Description Example matchesString matches regex pattern { "matches": "^\\d+$" }notMatchesString does not match regex { "notMatches": "^temp-.*" }
Type Operators Operator Description Example isNullValue is null { "isNull": true }isNotNullValue is not null { "isNotNull": true }isDefinedValue is defined (not undefined) { "isDefined": true }isUndefinedValue is undefined { "isUndefined": true }
Boolean Operators Operator Description Example isTrueValue is boolean true { "isTrue": true }isFalseValue is boolean false { "isFalse": true }
Empty Operators Operator Description Example isEmptyString/array/object is empty { "isEmpty": true }isNotEmptyString/array/object is not empty { "isNotEmpty": true }
Next Steps
Browse Blocks See what data each block produces
View Examples See assertions in real tests
