Skip to main content

​
Overview

Assertions are the final step in your test pipeline. After all blocks execute, assertions check if the data in the DataBus matches your expectations. 
{
  "pipeline": [

  ],
  "assertions": {
    "response.status": 200,
    "user.id": 1,
    "validation.passed": true
  }
}

​
Simple Assertions (Equality)

The simplest form checks for exact equality: 
{
  "assertions": {
    "response.status": 200,
    "user.name": "John Doe",
    "data.count": 10,
    "validation.passed": true
  }
}
Rules:
  • String values must match exactly
  • Numbers must match exactly
  • Booleans must match exactly
  • Null values must match exactly

​
Operator Assertions

For more complex validation, use operator objects with 20+ powerful operators.

​
Equality Operators

{
  "assertions": {
    "response.status": { "equals": 200 },
    "user.id": { "notEquals": 0 }
  }
}

​
Numeric Comparisons

{
  "assertions": {
    "user.age": { "gt": 18 },
    "user.age": { "gte": 18 },
    "response.time": { "lt": 1000 },
    "data.count": { "lte": 100 }
  }
}
  • gt: Greater than
  • gte: Greater than or equal
  • lt: Less than
  • lte: Less than or equal

​
String/Array Operations

{
  "assertions": {
    "response.message": { "contains": "success" },
    "user.roles": { "contains": "admin" },
    "response.error": { "notContains": "fatal" },
    "user.name": { "minLength": 1 },
    "user.bio": { "maxLength": 500 }
  }
}
For text: Checks if the string contains the substring or meets length requirements For arrays: Checks if the array contains the value or meets length requirements

​
Pattern Matching

{
  "assertions": {
    "user.email": { "matches": ".*@example\\.com" },
    "data.phone": { "matches": "^\\d{3}-\\d{3}-\\d{4}$" },
    "response.id": { "notMatches": "^temp-.*" }
  }
}
Uses JavaScript regex syntax.

​
Type Checks

{
  "assertions": {
    "data.value": { "isNull": true },
    "user.name": { "isNotNull": true },
    "config.setting": { "isDefined": true },
    "optional.field": { "isUndefined": true }
  }
}

​
Boolean Checks

{
  "assertions": {
    "validation.passed": { "isTrue": true },
    "hasError": { "isFalse": true }
  }
}

​
Empty Checks

{
  "assertions": {
    "user.name": { "isNotEmpty": true },
    "errors": { "isEmpty": true }
  }
}
Works with strings, arrays, and objects.

​
Complete Example

{
  "name": "User API Test",
  "tests": [{
    "id": "test-user",
    "pipeline": [
      {
        "block": "HttpRequest",
        "input": {
          "url": "https://api.example.com/users/1",
          "method": "GET"
        },
        "output": "response"
      },
      {
        "block": "JsonParser",
        "input": "${response.body}",
        "output": {
          "parsed": "user"
        }
      },
      {
        "block": "LLMJudge",
        "input": {
          "text": "${user}",
          "expected": {
            "expectedBehavior": "Should have complete user profile"
          }
        },
        "output": "judgement"
      }
    ],
    "assertions": {
      "response.status": { "gte": 200, "lt": 300 },
      "user.id": { "gt": 0 },
      "user.email": { "matches": ".*@.*\\..*", "isNotEmpty": true },
      "user.age": { "gte": 0, "lte": 150 },
      "user.roles": { "contains": "user", "isNotEmpty": true },
      "user.name": { "minLength": 1, "maxLength": 100 },
      "judgement.score": { "gte": 0.7 },
      "judgement.reasoning": { "isNotEmpty": true }
    }
  }]
}

​
Multiple Operators

You can combine multiple operators on the same field - all must pass: 
{
  "assertions": {
    "user.age": {
      "gte": 18,
      "lte": 100
    },
    "response.message": {
      "contains": "success",
      "matches": "^[A-Z].*",
      "isNotEmpty": true
    },
    "user.email": {
      "matches": ".*@.*\\..*",
      "isNotEmpty": true,
      "minLength": 5
    }
  }
}
Example explanations:
  • user.age: Must be between 18 and 100
  • response.message: Must contain “success” AND start with capital letter
  • user.email: Must be valid email format AND not empty AND at least 5 characters
All operators in a group are evaluated with AND logic - every operator must pass for the assertion to succeed.

​
Common Patterns

  • HTTP Status
  • AI Judge
  • Data Validation
  • Tool Validation
{
  "assertions": {
    "response.status": { "gte": 200, "lt": 300 },
    "response.body": { "isNotEmpty": true }
  }
}
Check for success status range (200-299) and non-empty response body.

​
Assertion Failures

When an assertion fails, you get detailed error messages: 
❌ test-user: User API Test (234ms)
   ✅ response.status = 200
   ❌ user.age: Expected 25 to be > 30
   ✅ user.email matches ".*@.*\\..*"
   ❌ judgement.score: Expected 0.6 to be > 0.7

​
Best Practices

Test successful response:
{
  "id": "success-case",
  "pipeline": [],
  "assertions": {
    "response.status": 200,
    "validation.passed": true
  }
}
Test error handling:
{
  "id": "error-case",
  "pipeline": [],
  "assertions": {
    "response.status": 400,
    "validation.passed": false
  }
}
Bad - too specific:
{
  "judgement.score": 0.85
}
Good - allows for variance:
{
  "judgement.score": { "gt": 0.7 }
}

{
  "assertions": {
    "user.id": { "gt": 0 },
    "user.email": { "isDefined": true, "isNotEmpty": true },
    "user.name": { "isNotEmpty": true },
    "user.email": { "matches": ".*@.*\\..*" },
    "user.name": { "minLength": 1, "maxLength": 100 }
  }
}
Structure checks: Required fields exist and are not emptyContent checks: Data values are valid (format, length, etc.)
Bad - too many assertions make tests fragile:
{
  "user.name": "John Doe",
  "user.age": 30,
  "user.email": "john@example.com",
  "user.address.street": "123 Main St",
  "user.address.city": "Springfield"
}
Good - test what matters:
{
  "user.id": 1,
  "user.email": { "matches": ".*@.*\\..*" },
  "response.status": 200
}
If any specific detail changes, overly specific tests will break unnecessarily.

​
Operator Reference

​
Equality Operators

OperatorDescriptionExample
equalsValue equals expected{ "equals": 200 }
notEqualsValue does not equal expected{ "notEquals": 404 }

​
Numeric Operators

OperatorDescriptionExample
gtGreater than{ "gt": 10 }
gteGreater than or equal to{ "gte": 18 }
ltLess than{ "lt": 100 }
lteLess than or equal to{ "lte": 200 }

​
String/Array Operators

OperatorDescriptionExample
containsString/array contains value{ "contains": "text" }
notContainsString/array does not contain value{ "notContains": "error" }
minLengthString/array length >= value{ "minLength": 1 }
maxLengthString/array length <= value{ "maxLength": 100 }

​
Pattern Operators

OperatorDescriptionExample
matchesString matches regex pattern{ "matches": "^\\d+$" }
notMatchesString does not match regex{ "notMatches": "^temp-.*" }

​
Type Operators

OperatorDescriptionExample
isNullValue is null{ "isNull": true }
isNotNullValue is not null{ "isNotNull": true }
isDefinedValue is defined (not undefined){ "isDefined": true }
isUndefinedValue is undefined{ "isUndefined": true }

​
Boolean Operators

OperatorDescriptionExample
isTrueValue is boolean true{ "isTrue": true }
isFalseValue is boolean false{ "isFalse": true }

​
Empty Operators

OperatorDescriptionExample
isEmptyString/array/object is empty{ "isEmpty": true }
isNotEmptyString/array/object is not empty{ "isNotEmpty": true }

​
Next Steps

Browse Blocks

See what data each block produces

View Examples

See assertions in real tests
⌘I