yapi

CLI-first API testing for HTTP, GraphQL, gRPC, and TCP.

The Workflow

yapi enables test-driven API development. Write the test first, then implement until it passes:

1. Write the test - Create a .yapi.yml file with the expected behavior
2. Run it - yapi run file.yapi.yml (it will fail)
3. Implement/fix - Build the API endpoint
4. Iterate - Refine assertions, add edge cases

This loop is the core of agentic API development with yapi.

---

Environment Setup (Do This First)

Before writing any tests, set up your environments. Create yapi.config.yml in your project root:

yapi: v1
default_environment: local

environments:
  local:
    url: http://localhost:3000
    vars:
      API_KEY: dev_key_123

  staging:
    url: https://staging.example.com
    vars:
      API_KEY: ${STAGING_API_KEY}  # from shell env

  prod:
    url: https://api.example.com
    vars:
      API_KEY: ${PROD_API_KEY}
    env_files:
      - .env.prod  # load secrets from file

Now your tests use ${url} and ${API_KEY} - same test, any environment:

yapi run get-users.yapi.yml              # uses local (default)
yapi run get-users.yapi.yml --env staging
yapi run get-users.yapi.yml --env prod

Variable resolution order (highest priority first):

1. Shell environment variables
2. Environment-specific vars
3. Environment-specific env_files
4. Default vars
5. Default env_files

---

A) Smoke Testing

Quick health checks to verify endpoints are alive.

HTTP

yapi: v1
url: ${url}/health
method: GET
expect:
  status: 200

GraphQL

yapi: v1
url: ${url}/graphql
graphql: |
  query { __typename }
expect:
  status: 200
  assert:
    - .data.__typename != null

gRPC

yapi: v1
url: grpc://${host}:${port}
service: grpc.health.v1.Health
rpc: Check
plaintext: true
body:
  service: ""
expect:
  status: 200

TCP

yapi: v1
url: tcp://${host}:${port}
data: "PING\n"
encoding: text
expect:
  status: 200

---

B) Integration Testing

Multi-step workflows with data passing between requests. Use chains when steps depend on each other.

Authentication Flow

yapi: v1
chain:
  - name: login
    url: ${url}/auth/login
    method: POST
    body:
      email: test@example.com
      password: ${TEST_PASSWORD}
    expect:
      status: 200
      assert:
        - .token != null

  - name: get_profile
    url: ${url}/users/me
    method: GET
    headers:
      Authorization: Bearer ${login.token}
    expect:
      status: 200
      assert:
        - .email == "test@example.com"

CRUD Flow

yapi: v1
chain:
  - name: create
    url: ${url}/posts
    method: POST
    body:
      title: "Test Post"
      content: "Hello World"
    expect:
      status: 201
      assert:
        - .id != null

  - name: read
    url: ${url}/posts/${create.id}
    method: GET
    expect:
      status: 200
      assert:
        - .title == "Test Post"

  - name: update
    url: ${url}/posts/${create.id}
    method: PATCH
    body:
      title: "Updated Post"
    expect:
      status: 200

  - name: delete
    url: ${url}/posts/${create.id}
    method: DELETE
    expect:
      status: 204

Running Integration Tests

Name test files with .test.yapi.yml suffix:

tests/
  auth.test.yapi.yml
  posts.test.yapi.yml
  users.test.yapi.yml

Run all tests:

yapi test ./tests                    # sequential
yapi test ./tests --parallel 4       # concurrent
yapi test ./tests --env staging      # against staging
yapi test ./tests --verbose          # detailed output

---

C) Uptime Monitoring

Create test suites for monitoring your services in production.

Monitor Suite Structure

monitors/
  api-health.test.yapi.yml
  auth-service.test.yapi.yml
  database-check.test.yapi.yml
  graphql-schema.test.yapi.yml

Health Check with Timeout

yapi: v1
url: ${url}/health
method: GET
timeout: 5s  # fail if response takes longer
expect:
  status: 200
  assert:
    - .status == "healthy"
    - .database == "connected"

Run Monitoring Suite

# Check all monitors in parallel
yapi test ./monitors --parallel 10 --env prod

# With verbose output for debugging
yapi test ./monitors --parallel 10 --env prod --verbose

CI/CD Integration (GitHub Actions)

name: API Health Check
on:
  schedule:
    - cron: '*/5 * * * *'  # every 5 minutes
  workflow_dispatch:

jobs:
  monitor:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install yapi
        run: curl -fsSL https://yapi.run/install/linux.sh | bash

      - name: Run health checks
        env:
          PROD_API_KEY: ${{ secrets.PROD_API_KEY }}
        run: yapi test ./monitors --env prod --parallel 5

Load Testing

Stress test endpoints or entire workflows:

# 1000 requests, 50 concurrent
yapi stress api-flow.yapi.yml -n 1000 -p 50

# Run for 30 seconds
yapi stress api-flow.yapi.yml -d 30s -p 25

# Against production (with confirmation)
yapi stress api-flow.yapi.yml -e prod -n 500 -p 10

---

D) Async Job Polling with wait_for

For endpoints that process data asynchronously, use wait_for to poll until conditions are met.

Fixed Period Polling

yapi: v1
url: ${url}/jobs/${job_id}
method: GET

wait_for:
  until:
    - .status == "completed" or .status == "failed"
  period: 2s
  timeout: 60s

expect:
  assert:
    - .status == "completed"

Exponential Backoff

Better for rate-limited APIs or long-running jobs:

yapi: v1
url: ${url}/jobs/${job_id}
method: GET

wait_for:
  until:
    - .status == "completed"
  backoff:
    seed: 1s       # Initial wait
    multiplier: 2  # 1s -> 2s -> 4s -> 8s...
  timeout: 300s

Async Workflow Chain

Complete example: create job, poll until done, download result:

yapi: v1
chain:
  - name: create_job
    url: ${url}/jobs
    method: POST
    body:
      type: "data_export"
      filters:
        date_range: "last_30_days"
    expect:
      status: 202
      assert:
        - .job_id != null

  - name: wait_for_job
    url: ${url}/jobs/${create_job.job_id}
    method: GET
    wait_for:
      until:
        - .status == "completed" or .status == "failed"
      period: 2s
      timeout: 300s
    expect:
      assert:
        - .status == "completed"
        - .download_url != null

  - name: download_result
    url: ${wait_for_job.download_url}
    method: GET
    output_file: ./export.csv

Webhook/Callback Waiting

Wait for a webhook to be received:

yapi: v1
chain:
  - name: trigger_action
    url: ${url}/payments/initiate
    method: POST
    body:
      amount: 100
    expect:
      status: 202

  - name: wait_for_webhook
    url: ${url}/webhooks/received
    method: GET
    wait_for:
      until:
        - . | length > 0
        - .[0].event == "payment.completed"
      period: 1s
      timeout: 30s

---

E) Integrated Test Server

Automatically start your dev server, wait for health checks, run tests, and clean up. Configure in yapi.config.yml:

yapi: v1

test:
  start: "npm run dev"
  wait_on:
    - "http://localhost:3000/healthz"
    - "grpc://localhost:50051"
  timeout: 60s
  parallel: 8
  directory: "./tests"

environments:
  local:
    url: http://localhost:3000

Running with Integrated Server

# Automatically starts server, waits for health, runs tests, kills server
yapi test

# Skip server startup (server already running)
yapi test --no-start

# Override config from CLI
yapi test --start "npm start" --wait-on "http://localhost:4000/health"

# See server stdout/stderr
yapi test --verbose

Health Check Protocols

| Protocol | URL Format | Behavior | |----------|------------|----------| | HTTP/HTTPS | http://localhost:3000/healthz | Poll until 2xx response | | gRPC | grpc://localhost:50051 | Uses grpc.health.v1.Health/Check | | TCP | tcp://localhost:5432 | Poll until connection succeeds |

Local vs CI Parity

The same workflow works locally and in CI:

Local development:

yapi test  # starts server, runs tests, cleans up

GitHub Actions:

- uses: jamierpond/yapi/action@main
  with:
    start: npm run dev
    wait-on: http://localhost:3000/healthz
    command: yapi test -a

---

Commands Reference

| Command | Description | |---------|-------------| | yapi run file.yapi.yml | Execute a request | | yapi run file.yapi.yml --env prod | Execute against specific environment | | yapi test ./dir | Run all *.test.yapi.yml files | | yapi test ./dir --all | Run all *.yapi.yml files (not just tests) | | yapi test ./dir --parallel 4 | Run tests concurrently | | yapi validate file.yapi.yml | Check syntax without executing | | yapi watch file.yapi.yml | Re-run on every file save | | yapi stress file.yapi.yml | Load test with concurrency | | yapi list | List all yapi files in directory |

---

Assertion Syntax

Assertions use JQ expressions that must evaluate to true.

Body Assertions

expect:
  status: 200
  assert:
    - .id != null                    # field exists
    - .name == "John"                # exact match
    - .age > 18                      # comparison
    - . | length > 0                 # array not empty
    - .[0].email != null             # first item has email
    - .users | length == 10          # exactly 10 users
    - .type == "admin" or .type == "user"  # alternatives
    - .tags | contains(["api"])      # array contains value

Header Assertions

expect:
  status: 200
  assert:
    headers:
      - .["Content-Type"] | contains("application/json")
      - .["X-Request-Id"] != null
      - .["Cache-Control"] == "no-cache"
    body:
      - .data != null

Status Code Options

expect:
  status: 200           # exact match
  status: [200, 201]    # any of these

---

Protocol Examples

HTTP with Query Params and Headers

yapi: v1
url: ${url}/api/users
method: GET
headers:
  Authorization: Bearer ${API_KEY}
  Accept: application/json
query:
  limit: "10"
  offset: "0"
  sort: "created_at"
expect:
  status: 200

HTTP POST with JSON Body

yapi: v1
url: ${url}/api/users
method: POST
body:
  name: "John Doe"
  email: "john@example.com"
  roles:
    - admin
    - user
expect:
  status: 201
  assert:
    - .id != null

HTTP Form Data

yapi: v1
url: ${url}/upload
method: POST
content_type: multipart/form-data
form:
  name: "document.pdf"
  description: "Q4 Report"
expect:
  status: 200

GraphQL with Variables

yapi: v1
url: ${url}/graphql
graphql: |
  query GetUser($id: ID!) {
    user(id: $id) {
      id
      name
      email
    }
  }
variables:
  id: "123"
expect:
  status: 200
  assert:
    - .data.user.id == "123"

gRPC with Metadata

yapi: v1
url: grpc://${host}:${port}
service: users.UserService
rpc: GetUser
plaintext: true
headers:
  authorization: Bearer ${API_KEY}
body:
  user_id: "123"
expect:
  status: 200
  assert:
    - .user.id == "123"

TCP Raw Connection

yapi: v1
url: tcp://${host}:${port}
data: |
  GET / HTTP/1.1
  Host: example.com

encoding: text
read_timeout: 5
expect:
  status: 200

---

File Organization

Recommended project structure:

project/
  yapi.config.yml          # environments
  .env                     # local secrets (gitignored)
  .env.example             # template for secrets

  tests/
    auth/
      login.test.yapi.yml
      logout.test.yapi.yml
    users/
      create-user.test.yapi.yml
      get-user.test.yapi.yml

  monitors/
    health.test.yapi.yml
    critical-endpoints.test.yapi.yml

---

Tips

• Start simple: Begin with status code checks, add body assertions as needed
• Use watch mode: yapi watch file.yapi.yml for rapid iteration
• Validate before running: yapi validate file.yapi.yml catches syntax errors
• Keep tests focused: One logical flow per file
• Name steps clearly: In chains, use descriptive names like create_user, verify_email
• Reference previous steps: Use ${step_name.field} to pass data between chain steps