Skip to main content

​
What are Blocks?

Blocks are the building pieces of test pipelines. Each block does one thing well and can be combined with other blocks to create complex test scenarios.

​
Available Blocks

HttpRequest

Make HTTP requests to APIs with full control over method, headers, and body

JsonParser

Parse JSON strings into structured data

StreamParser

Parse streaming SSE responses from OpenAI, Vercel AI SDK, and custom formats

ValidateContent

Validate text content with contains, regex, length checks, and more

ValidateTools

Validate AI tool calls, order, arguments, and constraints

LLMJudge

Use AI to evaluate responses semantically with configurable criteria

Loop

Loop back to previous blocks with max iteration control

MockData

Return mock data for testing without external dependencies

​
Block Categories

​
HTTP & Data Fetching

  • HttpRequest: Make API calls
  • MockData: Test without external dependencies

​
Parsing & Transformation

  • JsonParser: Parse JSON responses
  • StreamParser: Handle streaming data

​
Validation

  • ValidateContent: Text and content validation
  • ValidateTools: AI tool call validation
  • LLMJudge: Semantic validation with AI

​
Control Flow

  • Loop: Retry logic and iterations

​
Common Patterns

​
API Testing

{
  "pipeline": [
    { "block": "HttpRequest" },   // Call API
    { "block": "JsonParser" },    // Parse response
    { "block": "ValidateContent" } // Validate content
  ]
}

​
AI Agent Testing

{
  "pipeline": [
    { "block": "HttpRequest" },    // Call AI API
    { "block": "StreamParser" },   // Parse streaming response
    { "block": "ValidateTools" },  // Check tool calls
    { "block": "LLMJudge" }        // Semantic validation
  ]
}

​
Retry Logic

{
  "pipeline": [
    { "id": "request", "block": "HttpRequest" },
    { "block": "ValidateContent" },
    {
      "block": "Loop",
      "config": {
        "target": "request",
        "maxIterations": 3
      }
    }
  ]
}

​
Creating Custom Blocks

Don’t see what you need? Create custom blocks: 
import { Block } from '@blade47/semantic-test';

export class MyCustomBlock extends Block {
  static get inputs() {
    return {
      required: ['data'],
      optional: ['config']
    };
  }

  static get outputs() {
    return {
      produces: ['result']
    };
  }

  async process(inputs, context) {
    // Your logic here
    return { result: processedData };
  }
}

Learn More About Custom Blocks

Complete guide to creating and registering custom blocks

​
Block Composition

Blocks are designed to be composable. Here are common combinations: 
HttpRequest → JsonParser → ValidateContent → Assertions

HttpRequest → StreamParser → ValidateTools → LLMJudge → Assertions

HttpRequest → ValidateContent → Loop (if failed) → Assertions

MockData → ValidateContent → LLMJudge → Assertions

​
Next Steps

Explore Each Block

Click on any block above to see detailed documentation

View Examples

See blocks in action with real examples

Create Custom Blocks

Extend SemanticTest with your own blocks
⌘I