Batch requests

Send many texts in one HTTP call. Each item is answered, billed and logged exactly like a single request, so a batch costs the same as sending the items one by one. It saves round trips, not money.

Endpoints

POST https://layahost.com/v1/decide/batch
POST https://layahost.com/v1/systemone/batch

The same decision for many texts

/v1/decide/batch takes the body of POST /v1/decide, with texts (a list of up to 64 strings) instead of text. The template or question applies to every text.

POST /v1/decide/batch

curl https://layahost.com/v1/decide/batch \
  -H "Authorization: Bearer $LAYAHOST_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "template": "spam",
    "texts": [
      "Congratulations! You won a free iPhone, claim it at bit.ly/xyz",
      "Hi, I was charged twice for my subscription this month."
    ]
  }'

POST /v1/decide/batch

import os
import requests

res = requests.post(
    "https://layahost.com/v1/decide/batch",
    headers={"Authorization": f"Bearer {os.environ['LAYAHOST_API_KEY']}"},
    json={"template": "spam", "texts": messages},
    timeout=30,
)
res.raise_for_status()
for item in res.json()["results"]:
    if item["status"] == 200:
        print(item["index"], item["decision"], item["confidence"])
    else:
        print(item["index"], "failed:", item["detail"])

POST /v1/decide/batch

const res = await fetch("https://layahost.com/v1/decide/batch", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.LAYAHOST_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ template: "spam", texts: messages }),
});

const { results, usage } = await res.json();
for (const item of results) {
  console.log(item.index, item.status === 200 ? item.decision : item.detail);
}

Response · 200

{
  "results": [
    {
      "index": 0,
      "status": 200,
      "decision": "spam",
      "confidence": 0.3328,
      "probabilities": { "spam": 0.8258, "legit": 0.1742 },
      "template": "spam",
      "model": "laya-english",
      "usage": { "decisions": 1, "input_tokens": 77, "cost_micros": 5, "cost_usd": 5.0e-6 },
      "latency_ms": 325.72,
      "request_id": "req_yyc8n70ajnyo5rwgkhcwmceb_0"
    },
    {
      "index": 1,
      "status": 200,
      "decision": "legit",
      "confidence": 0.0806,
      "probabilities": { "legit": 0.6656, "spam": 0.3344 },
      "template": "spam",
      "model": "laya-english",
      "usage": { "decisions": 1, "input_tokens": 72, "cost_micros": 5, "cost_usd": 5.0e-6 },
      "latency_ms": 231.02,
      "request_id": "req_yyc8n70ajnyo5rwgkhcwmceb_1"
    }
  ],
  "usage": { "decisions": 2, "failed": 0, "cost_micros": 10, "cost_usd": 1.0e-5 },
  "request_id": "req_yyc8n70ajnyo5rwgkhcwmceb"
}

Each item in results is the /v1/decide response plus its index and status. Results are in the order of texts.

Many Jev requests at once

/v1/systemone/batch takes requests, a list of up to 64 ordinary /v1/systemone bodies, each with its own text, model and questions. At most 256 questions in total per batch.

POST /v1/systemone/batch

curl https://layahost.com/v1/systemone/batch \
  -H "Authorization: Bearer $LAYAHOST_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "requests": [
      {"state": "Our checkout is down!", "model": "jev-latest",
       "questions": {"urgent": {"type": "noul", "instructions": "Is this urgent?"}}},
      {"state": "Love the new dashboard.", "model": "jev-latest",
       "questions": {"urgent": {"type": "noul", "instructions": "Is this urgent?"}}}
    ]
  }'

Response · 200

{
  "responses": [
    {
      "index": 0,
      "status": 200,
      "body": {
        "model": "laya-english",
        "answers": { "urgent": { "type": "noul", "noul": 0.733, "confidence": 0.733 } },
        "usage": { "input_tokens": 35, "output_tokens": 0 },
        "meta": { "request_id": "req_halhpd064uorzjutdn2mwwpo_0", "decisions": 1, "cost_micros": 5, "cost_usd": 5.0e-6, "latency_ms": 85.52 }
      }
    },
    {
      "index": 1,
      "status": 200,
      "body": {
        "model": "laya-english",
        "answers": { "urgent": { "type": "noul", "noul": 0.0392, "confidence": 0.9608 } },
        "usage": { "input_tokens": 35, "output_tokens": 0 },
        "meta": { "request_id": "req_halhpd064uorzjutdn2mwwpo_1", "decisions": 1, "cost_micros": 5, "cost_usd": 5.0e-6, "latency_ms": 73.45 }
      }
    }
  ],
  "request_id": "req_halhpd064uorzjutdn2mwwpo"
}

Each body is exactly what /v1/systemone would have returned for that request, success or error.

Partial failures and billing

A batch answers 200 once its envelope is valid, even if some items fail. Check each item's status.
A bad item gets its own 422 and does not stop the rest. It is not billed.
Items run in order and are charged one by one. If your credit runs out halfway, the remaining items get 402 and are not billed.
An invalid envelope (no list, too many items, an unknown template) gets 422 for the whole batch and nothing runs.
Every item appears in Logs with the batch request ID plus _0, _1 and so on.
A batch counts as one request against your rate limit.

Items are answered one after another, so a full batch takes about as long as its items added up: typically 2 to 5 seconds for 64 short texts. Set your HTTP timeout to at least 30 seconds.