Thai ASR APIReading Score APITeam API keysShared LLM APITTS APILTX23 Video API

Public Developer Docs

Integration docs for KaleidoVid speech, TTS, video, and local LLM APIs.

This page documents the current customer-facing Thai ASR streaming API, Reading Score upload API, shared OpenAI-compatible LLM API, TTS API, and LTX23 image-to-video API exposed by this SaaS site. The examples below are built from the actual routes and backend contracts in this repository.

Thai ASR base URL

https://www.kaleidovid.com/api/thai-asr

Reading Score URL

https://www.kaleidovid.com/api/reading-score

https://www.kaleidovid.com/api/reading-score/task/{taskId}

Shared LLM base URL

https://www.kaleidovid.com/api/llm/v1

TTS base URL

https://www.kaleidovid.com/api/tts/v1

LTX23 Video base URL

https://www.kaleidovid.com/api/ltx23/v1

Quick start

  1. 1. Create or copy a team API key from Dashboard → API Keys.
  2. 2. Use x-api-key for HTTP calls. Browser WebSocket clients should append ws_url to the returned api_key.
  3. 3. Use Thai ASR for live streaming transcripts, Reading Score for Thai reading evaluation, Shared LLM for OpenAI-compatible local model inference, TTS for API-key authenticated speech synthesis, and LTX23 Video for image-to-video generation.

Authentication

Use team-scoped API keys.

These APIs are meant to be called with a KaleidoVid team API key. The SaaS dashboard manages key lifecycle, while your integration sends the raw key on every request.

Where to create keys

Create, revoke, restore, and inspect quotas in Dashboard → API Keys. Team owners and admins can manage keys; team members can still inspect existing key policies and usage.

Raw keys are only shown once at creation time. Store them in your own secret manager immediately.

Header and WebSocket auth

Sample
x-api-key: kvid_your_prefix_your_secret

Reading Score, Shared LLM, TTS, and LTX23 Video also accept:
Authorization: Bearer kvid_your_prefix_your_secret

Browser WebSocket clients should append `api_key=<YOUR_API_KEY>`
to the `ws_url` returned by POST /api/thai-asr/sessions.

Thai ASR API

Streaming Thai speech-to-text over HTTP + WebSocket.

The Thai ASR product is a three-step flow: inspect defaults, create a session, then stream PCM audio to the returned WebSocket URL. The service is Thai-only and currently reports the `typhoon` engine in responses.

GET/api/thai-asr/config

Inspect defaults and limits

Returns the current backend name, decode defaults, tenant information, and request/session/audio quota limits for the calling key.

POST/api/thai-asr/sessions

Create a streaming session

Returns a fresh `session_id`, the resolved `ws_url`, the accepted config, and current SLA target hints for the low-latency benchmark surface.

WS/api/thai-asr/stream

Stream audio and receive transcripts

After the server emits `ready`, send a `start` message, then raw PCM16LE mono audio chunks. Watch `partial`, `stabilized_partial`, and `final` events.

GET /api/thai-asr/config

Sample
curl -sS \
  -H "x-api-key: YOUR_API_KEY" \
  "https://www.kaleidovid.com/api/thai-asr/config"

Config response example

Sample
{
  "language": "th",
  "mode": "benchmark_spike",
  "backend": "persistent_decoder",
  "engine": "typhoon",
  "defaults": {
    "sample_rate": 16000,
    "frame_ms": 20,
    "stream_chunk_ms": 80,
    "partial_interval_ms": 40,
    "min_decode_audio_ms": 240,
    "decode_window_ms": 1600,
    "vad": true
  },
  "notes": [
    "This spike measures websocket, VAD, and partial transcript latency for Thai ASR on the 8x4090 host."
  ],
  "tenant": {
    "team_id": "team_cuid",
    "team_name": "Acme Team",
    "key_prefix": "abcd1234"
  },
  "limits": {
    "requests_per_minute": 120,
    "concurrent_sessions": 3,
    "daily_audio_seconds": 3600,
    "today_audio_seconds_used": 420
  }
}

Create a session

The session creation payload is JSON. You can omit fields to use the server defaults, but most clients should send the same values they expect to use on the WebSocket start message so there is no mismatch between planning and runtime.

FieldTypeRequiredNotes
sample_rateintegerNo8,000 to 48,000. Use 16,000 for the current Thai ASR defaults.
frame_msintegerNo10 to 200. The built-in smoke test uses 20 ms.
partial_interval_msintegerNo20 to 1,000. Controls how often the service tries to emit new partials.
min_decode_audio_msintegerNo100 to 5,000. Minimum buffered audio before partial decoding starts.
decode_window_msintegerNo200 to 15,000. Rolling audio window used for decode requests.
vadbooleanNoDefaults to true. Emits `speech_start` when voice activity is detected.
benchmark_labelstringNoOptional label up to 200 characters.

POST /api/thai-asr/sessions

Sample
curl -sS \
  -X POST \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "sample_rate": 16000,
    "frame_ms": 20,
    "partial_interval_ms": 40,
    "min_decode_audio_ms": 240,
    "decode_window_ms": 1600,
    "vad": true
  }' \
  "https://www.kaleidovid.com/api/thai-asr/sessions"

Session response example

Sample
{
  "session_id": "e6a4c4f6c69f4ca1aa8f9b3ad8d515f0",
  "mode": "benchmark_spike",
  "language": "th",
  "engine": "typhoon",
  "backend": "persistent_decoder",
  "ws_url": "wss://www.kaleidovid.com/api/thai-asr/stream?session_id=e6a4c4f6c69f4ca1aa8f9b3ad8d515f0",
  "config": {
    "sample_rate": 16000,
    "frame_ms": 20,
    "partial_interval_ms": 40,
    "min_decode_audio_ms": 240,
    "decode_window_ms": 1600,
    "vad": true,
    "benchmark_label": null
  },
  "sla_targets": {
    "speech_detected_p95_ms": 60,
    "first_partial_p95_ms": 250,
    "final_after_eou_p95_ms": 800
  },
  "tenant": {
    "team_id": "team_cuid",
    "team_name": "Acme Team",
    "key_prefix": "abcd1234"
  }
}

WebSocket flow

Use the exact ws_url returned by session creation. Browser clients typically append api_key to that URL because custom WebSocket headers are harder to set in the browser. Binary audio frames are the preferred transport.

Production caption rule: render one live caption from stabilized_partial when available, otherwise fall back to partial. Only persist the transcript after receiving final.

Client messages

FieldTypeRequiredNotes
startJSON messageYesSend once after the server emits `ready`. Carries the same config fields used in session creation.
binary audio frameraw bytesYesRecommended path. Send PCM16LE mono audio bytes only, without WAV headers.
audioJSON messageNoFallback JSON shape: `{ "type": "audio", "audio_b64": "..." }` where the payload is base64-encoded PCM16LE audio.
flushJSON messageNoForces the service to emit a best-effort partial from current buffered audio.
end / end_utteranceJSON messageYesFinalizes the utterance and triggers the `final` event.
resetJSON messageNoClears buffered state so another utterance can run on the same socket.

Server events

FieldTypeRequiredNotes
readyserver eventAlwaysFirst event on a successful connection. Includes team info, backend, limits, and sample rate.
startedserver eventAlwaysConfirms the stream is configured and ready for audio frames.
speech_startserver eventOptionalEmitted when VAD first detects speech. Includes `offset_ms` and `latency_ms`.
partialserver eventOptionalBest-effort rolling transcript. Includes `text`, `segments`, `audio_ms`, `latency_ms`, `queue_ms`, and `decode_ms`.
stabilized_partialserver eventOptionalA partial that repeated enough times to look stable. Use this for live captions when available.
finalserver eventAlways on successFinal transcript with segments and latency metrics. Persist only this event in production workflows.
errorserver eventOn failureCarries `code` and `error` for issues such as timeout, decode failures, or stream failures.
reset_okserver eventOptionalAcknowledges a successful `reset` command.

JavaScript integration example

Sample
const API_KEY = "YOUR_API_KEY";
const HTTP_BASE = "https://www.kaleidovid.com";

const config = await fetch(`${HTTP_BASE}/api/thai-asr/config`, {
  headers: { "x-api-key": API_KEY },
}).then(async (response) => {
  if (!response.ok) throw new Error(await response.text());
  return response.json();
});

const session = await fetch(`${HTTP_BASE}/api/thai-asr/sessions`, {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "x-api-key": API_KEY,
  },
  body: JSON.stringify({
    sample_rate: 16000,
    frame_ms: 20,
    partial_interval_ms: 40,
    min_decode_audio_ms: 240,
    decode_window_ms: 1600,
    vad: true,
  }),
}).then(async (response) => {
  if (!response.ok) throw new Error(await response.text());
  return response.json();
});

const wsUrl = `${session.ws_url}&api_key=${encodeURIComponent(API_KEY)}`;
const socket = new WebSocket(wsUrl);

socket.onmessage = (event) => {
  const payload = JSON.parse(event.data);
  switch (payload.type) {
    case "ready":
      socket.send(JSON.stringify({
        type: "start",
        sample_rate: 16000,
        frame_ms: 20,
        partial_interval_ms: 40,
        min_decode_audio_ms: 240,
        decode_window_ms: 1600,
        vad: true,
      }));
      break;
    case "partial":
    case "stabilized_partial":
      console.log("live", payload.text);
      break;
    case "final":
      console.log("final", payload.text, payload.segments);
      break;
    case "error":
      console.error(payload.code, payload.error);
      break;
  }
};

// `pcmChunks` must contain raw PCM16LE mono audio frames.
// Do not send WAV headers after the socket is started.
for (const chunk of pcmChunks) {
  socket.send(chunk);
}

socket.send(JSON.stringify({ type: "flush" }));
socket.send(JSON.stringify({ type: "end" }));

`final` event example

Sample
{
  "type": "final",
  "session_id": "e6a4c4f6c69f4ca1aa8f9b3ad8d515f0",
  "engine": "typhoon",
  "backend": "persistent_decoder",
  "text": "สวัสดีครับ นี่คือการทดสอบระบบถอดเสียงภาษาไทยแบบหน่วงต่ำ",
  "segments": [
    {
      "start": 0.0,
      "end": 2.84,
      "text": "สวัสดีครับ นี่คือการทดสอบระบบถอดเสียงภาษาไทยแบบหน่วงต่ำ"
    }
  ],
  "audio_ms": 2840,
  "latency_ms": 134,
  "queue_ms": 0,
  "decode_ms": 134,
  "processing_time_ms": 133,
  "audio_duration_ms": 2840
}

Reading Score API

Upload a student recording and get Thai best-attempt scoring with structured feedback data.

The Reading Score API is a canonical HTTP upload endpoint on this Next.js app. Send multipart form data with a student recording and either reference text or reference audio. The service selects the best-matching reading attempt from the recording and exposes structured comparison fields that callers can render however they want. Most requests return the final score JSON directly with HTTP 200. Longer jobs may return HTTP 202 with `taskId` and `statusPath`; poll that status route with the same API key until `ready=true`. The default ASR engine is `typhoon`, and callers can override it with `asr_engine=whisper` or the equivalent `scoring_options` field. The current service only supports Thai.

Migration note for existing callers

  • Use /api/reading-score as the canonical public route for new integrations. The older /api/low-latency-asr/reading-score path is still accepted for compatibility.
  • Handle two success modes: HTTP 200 returns the final reading-score payload; HTTP 202 returns a queued job with taskId and statusPath.
  • When you receive HTTP 202, call GET /api/reading-score/task/{taskId} with the same x-api-key until ready=true. If successful=true, read the final score from result.
  • Update your response parsing to read student.selectedAttempt, feedback.*, and comparison.wordResults[] instead of treating the API as transcript-only.
  • Displayed scores.* values are usually scaled into the `80-100` band, but VAD-confirmed no-speech detections now return `0.0` with assessment.status=no_speech. Use diagnostics.rawScores.* if you need the underlying raw metrics.
  • If you stay on Typhoon, optionally handle student.meta.silenceRemovedOnTyphoon, student.meta.speechRegionCount, student.meta.windowedRecoveryOnTyphoon and student.meta.windowedRecoveryWindowCount when Typhoon recovery paths activate.
POST/api/reading-score

Canonical public route

Use this route for new integrations. It validates the API key, normalizes form fields, forwards the upload into the reading-score backend stack, and records usage under the calling team.

POST/api/low-latency-asr/reading-score

Legacy compatibility alias

Older clients may still call this path. New integrations should prefer `/api/reading-score`.

GET/api/reading-score/task/{taskId}

Canonical queued-job status route

Use this only after the upload route returns HTTP 202. It validates the same API key and returns task state, readiness, success/failure, progress when available, and the final score under `result` when complete.

GET/api/low-latency-asr/reading-score/task/{taskId}

Legacy status alias

Older clients using the legacy upload path may poll this matching status path. New integrations should prefer `/api/reading-score/task/{taskId}`.

Multipart request fields

FieldTypeRequiredNotes
student_audiofileYesStudent recording to score. This is the canonical field name.
reference_textstringOne of twoReference sentence or passage. Required when `reference_audio` is not sent.
reference_audiofileOne of twoReference recording. If provided without `reference_text`, the service first transcribes this audio.
languagestringNoDefaults to `th`. The current service only accepts Thai.
asr_enginestringNoOptional ASR override. Supported values are `typhoon` and `whisper`. Defaults to `typhoon`. You can also send this inside `scoring_options`.
student_idstringNoOptional caller-supplied student identifier returned in `request.studentId`.
lesson_idstringNoOptional caller-supplied lesson identifier returned in `request.lessonId`.
scoring_optionsJSON stringNoSupported keys today include `{ "include_pronunciation": true }` and `{ "asr_engine": "whisper" }`.
include_pronunciationboolean-like stringNoAlternative top-level form for pronunciation scoring. `true` is the default behavior.

Canonical field names and accepted aliases

The canonical request fields are student_audio, reference_text, reference_audio, student_id, lesson_id, asr_engine, and scoring_options. The backend also accepts a few compatibility aliases such as studentAudio, audio_file, referenceText, referenceAudio, studentId, lessonId, and asrEngine

curl upload example

Sample
curl -sS \
  -H "x-api-key: YOUR_API_KEY" \
  -F "student_audio=@student.wav" \
  -F "reference_text=สวัสดีครับ วันนี้เราจะเรียนภาษาไทย" \
  -F "student_id=student_001" \
  -F "lesson_id=lesson_01" \
  -F "language=th" \
  -F "include_pronunciation=true" \
  "https://www.kaleidovid.com/api/reading-score"

Python upload example

Sample
# pip install requests

import json
import time
import requests

ENDPOINT_URL = "https://www.kaleidovid.com/api/reading-score"
BASE_URL = ENDPOINT_URL.removesuffix("/api/reading-score")
API_KEY = "YOUR_API_KEY"

def read_result_or_poll(response):
    if response.status_code != 202:
        response.raise_for_status()
        return response.json()

    queued = response.json()
    status_path = queued.get("statusPath")
    if not status_path:
        return queued

    while True:
        status_response = requests.get(
            f"{BASE_URL}{status_path}",
            headers={"x-api-key": API_KEY},
            timeout=30,
        )
        status_response.raise_for_status()
        status_payload = status_response.json()
        if not status_payload.get("ready"):
            time.sleep(2)
            continue
        if status_payload.get("successful"):
            return status_payload.get("result")
        raise RuntimeError(status_payload.get("error", "Reading score failed"))

with open("student.wav", "rb") as student_audio:
    response = requests.post(
        ENDPOINT_URL,
        headers={"x-api-key": API_KEY},
        data={
            "reference_text": "สวัสดีครับ วันนี้เราจะเรียนภาษาไทย",
            "student_id": "student_001",
            "lesson_id": "lesson_01",
            "language": "th",
            "include_pronunciation": "true",
        },
        files={
            "student_audio": ("student.wav", student_audio, "audio/wav"),
        },
        timeout=300,
    )

payload = read_result_or_poll(response)
print(json.dumps(payload, ensure_ascii=False, indent=2))

Response shape

HTTP 200 responses return the normalized request summary, including the resolved ASR engine, the full student transcript plus the best selected attempt, displayed student-facing scores, token-by-token comparison, structured feedback fields, raw diagnostics, and optional forced-alignment metadata. HTTP 202 responses mean the request is still running in the reading-score worker queue; poll the returned `statusPath` with the same auth until `ready=true`, then read `result` when `successful=true`.

Display-score formula: after bounding the raw score to `0-100`, the public `scores.*` value is usually scaled as `80 + raw * 0.2`. VAD-confirmed no-speech detections are the exception: `assessment.status` becomes `no_speech` and `scores.*` are forced to `0.0`.

FieldTypeRequiredNotes
assessment.passedbooleanAlwaysCustomer-facing assessment flag. Typical reads return `true`, while VAD-confirmed no-speech detections return `false`.
assessment.statusstringAlwaysCustomer-facing status string. Typical successful reads return `passed`; no-speech is only returned after the silence detector and VAD both find no usable voice activity.
request.asrEnginestringAlwaysThe resolved ASR engine that actually handled the request. Use this instead of assuming the default from your client code.
student.selectedAttemptobject | nullMaybeDescribes the best-matching reading attempt selected from the full recording before scoring.
scores.overallnumber | nullAlwaysDisplayed student-facing score. The service usually scales raw scores into an encouragement band of `80-100`, but VAD-confirmed no-speech detections return `0.0`. Raw unclamped values live under `diagnostics.rawScores`.
scores.textAccuracynumberAlwaysDisplayed text-accuracy score for the selected best attempt.
scores.pronunciationnumber | nullMaybeDisplayed pronunciation score derived from alignment confidence. `null` means pronunciation scoring was unavailable.
scores.levenshteinSimilaritynumberAlwaysDisplayed edit-distance similarity for the selected attempt.
feedback.pronunciationFocusWords[]arrayAlwaysReference words whose pronunciation confidence was weak enough that you may want to flag them in your own UI.
feedback.missingWords[] / feedback.extraWords[]arrayAlwaysStructured token lists for omitted reference words and extra spoken words.
feedback.substitutions[]arrayAlwaysStructured expected/actual pairs for substitution mismatches.
diagnostics.rawScoresobjectAlwaysUnderlying unclamped metrics for internal review, analytics, or debugging.
student.meta.noSpeechDetected / student.meta.audioDurationboolean / numberMaybeOptional no-speech metadata. When `noSpeechDetected` is `true`, both the silence pass and VAD found no usable student voice, `assessment.status` becomes `no_speech`, and `scores.*` are forced to `0.0`. `audioDuration` reports the analyzed clip length.
student.meta.voiceActivityChecked / voiceActivityDetectedboolean / booleanMaybeOptional VAD metadata for no-speech decisions. `voiceActivityChecked=true` means the service ran the follow-up VAD pass; `voiceActivityDetected=false` is required before the service returns `assessment.status=no_speech`.
student.meta.voiceActivityDuration / voiceActivityDetectornumber / stringMaybeOptional VAD detail. `voiceActivityDuration` reports the estimated voiced duration in seconds, and `voiceActivityDetector` currently reports `torchaudio_vad`.
student.meta.silenceRemovedOnTyphoon / student.meta.speechRegionCountboolean / numberMaybeOptional Typhoon-only metadata for long-gap recovery. `silenceRemovedOnTyphoon` means the service retried on a silence-removed copy, and `speechRegionCount` reports how many speech regions were detected.
student.meta.windowedRecoveryOnTyphoon / student.meta.windowedRecoveryWindowCountboolean / numberMaybeOptional Typhoon-only metadata for reference-aware boundary recovery. `windowedRecoveryOnTyphoon` means the service retried short overlapping windows after the first transcript looked like a strict boundary-truncated slice of the expected text, and `windowedRecoveryWindowCount` reports how many windows were tested.
comparison.wordResults[]arrayAlwaysToken-by-token breakdown with `status` = `correct`, `missing`, `extra`, or `substitution`.
alignment.usedbooleanAlwaysIndicates whether pronunciation alignment was strong enough to contribute to the score.

Successful response example

Sample
{
  "success": true,
  "assessment": {
    "passed": true,
    "status": "passed",
    "displayScoreMin": 80.0,
    "usedBestAttemptSelection": true
  },
  "request": {
    "language": "th",
    "asrEngine": "typhoon",
    "studentId": "student_001",
    "lessonId": "lesson_01",
    "includePronunciation": true,
    "referenceSource": "referenceText"
  },
  "reference": {
    "text": "สวัสดีครับ วันนี้เราจะเรียนภาษาไทย",
    "normalizedText": "สวัสดีครับ วันนี้เราจะเรียนภาษาไทย",
    "tokens": ["สวัสดีครับ", "วันนี้", "เรา", "จะ", "เรียน", "ภาษาไทย"],
    "tokenCount": 6,
    "meta": {}
  },
  "student": {
    "transcript": "สวัสดีครับ วันนี้ เรียน ภาษาใจ",
    "fullTranscript": "สวัสดีครับ วันนี้เราจะเรียนภาษาไทย สวัสดีครับ วันนี้ เรียน ภาษาใจ",
    "normalizedText": "สวัสดีครับ วันนี้ เรียน ภาษาใจ",
    "tokens": ["สวัสดีครับ", "วันนี้", "เรียน", "ภาษาใจ"],
    "tokenCount": 4,
    "selectedAttempt": {
      "mode": "best_token_window",
      "applied": true,
      "candidateCount": 32,
      "multipleAttemptsDetected": true,
      "start": 2.84,
      "end": 4.96,
      "durationSeconds": 2.12,
      "tokenStartIndex": 6,
      "tokenEndIndex": 9,
      "hasTiming": true
    },
    "meta": {
      "model": "scb10x/typhoon-asr-realtime",
      "device": "cuda",
      "speechRegionCount": 2,
      "silenceRemovedOnTyphoon": true
    }
  },
  "scores": {
    "overall": 89.83,
    "textAccuracy": 90.0,
    "pronunciation": 89.45,
    "levenshteinSimilarity": 90.0
  },
  "comparison": {
    "correctTokenCount": 3,
    "referenceTokenCount": 6,
    "studentTokenCount": 4,
    "missingTokenCount": 2,
    "extraTokenCount": 0,
    "substitutionCount": 1,
    "wordResults": [
      {
        "referenceIndex": 0,
        "studentIndex": 0,
        "status": "correct",
        "referenceToken": "สวัสดีครับ",
        "studentToken": "สวัสดีครับ",
        "pronunciationConfidence": 0.82
      },
      {
        "referenceIndex": 2,
        "studentIndex": null,
        "status": "missing",
        "referenceToken": "เรา",
        "studentToken": null,
        "pronunciationConfidence": 0.33
      },
      {
        "referenceIndex": 5,
        "studentIndex": 3,
        "status": "substitution",
        "referenceToken": "ภาษาไทย",
        "studentToken": "ภาษาใจ",
        "pronunciationConfidence": 0.41
      }
    ]
  },
  "feedback": {
    "pronunciationFocusWords": ["เรา", "จะ", "ภาษาไทย"],
    "missingWords": ["เรา", "จะ"],
    "extraWords": [],
    "substitutions": [
      {
        "expected": "ภาษาไทย",
        "actual": "ภาษาใจ"
      }
    ]
  },
  "diagnostics": {
    "rawScores": {
      "overall": 49.17,
      "textAccuracy": 50.0,
      "pronunciation": 47.25,
      "levenshteinSimilarity": 50.0
    }
  },
  "alignment": {
    "used": true,
    "averageConfidence": 0.4725,
    "words": [
      {
        "index": 0,
        "word": "สวัสดีครับ",
        "start": 0.0,
        "end": 0.34,
        "confidence": 0.82
      },
      {
        "index": 2,
        "word": "เรา",
        "start": 0.62,
        "end": 0.84,
        "confidence": 0.33
      },
      {
        "index": 5,
        "word": "ภาษาไทย",
        "start": 1.55,
        "end": 1.93,
        "confidence": 0.41
      }
    ]
  }
}

No-speech response example

Sample
{
  "success": true,
  "assessment": {
    "passed": false,
    "status": "no_speech",
    "displayScoreMin": 0.0,
    "usedBestAttemptSelection": false
  },
  "request": {
    "language": "th",
    "asrEngine": "typhoon",
    "studentId": null,
    "lessonId": null,
    "includePronunciation": true,
    "referenceSource": "referenceText"
  },
  "reference": {
    "text": "สวัสดีครับ",
    "normalizedText": "สวัสดีครับ",
    "tokens": ["สวัสดี", "ครับ"],
    "tokenCount": 2,
    "meta": {}
  },
  "student": {
    "transcript": "",
    "fullTranscript": "",
    "normalizedText": "",
    "tokens": [],
    "tokenCount": 0,
    "selectedAttempt": {
      "mode": "no_speech_detected",
      "applied": false,
      "candidateCount": 0,
      "multipleAttemptsDetected": false,
      "start": null,
      "end": null,
      "durationSeconds": null,
      "tokenStartIndex": null,
      "tokenEndIndex": null,
      "hasTiming": false
    },
    "meta": {
      "noSpeechDetected": true,
      "speechRegionCount": 0,
      "audioDuration": 1.0,
      "voiceActivityChecked": true,
      "voiceActivityDetected": false,
      "voiceActivityDuration": 0.0,
      "voiceActivityDetector": "torchaudio_vad"
    }
  },
  "scores": {
    "overall": 0.0,
    "textAccuracy": 0.0,
    "pronunciation": 0.0,
    "levenshteinSimilarity": 0.0
  },
  "comparison": {
    "correctTokenCount": 0,
    "referenceTokenCount": 2,
    "studentTokenCount": 0,
    "missingTokenCount": 2,
    "extraTokenCount": 0,
    "substitutionCount": 0,
    "wordResults": [
      {
        "referenceIndex": 0,
        "studentIndex": null,
        "status": "missing",
        "referenceToken": "สวัสดี",
        "studentToken": null,
        "pronunciationConfidence": null
      },
      {
        "referenceIndex": 1,
        "studentIndex": null,
        "status": "missing",
        "referenceToken": "ครับ",
        "studentToken": null,
        "pronunciationConfidence": null
      }
    ]
  },
  "feedback": {
    "pronunciationFocusWords": [],
    "missingWords": ["สวัสดี", "ครับ"],
    "extraWords": [],
    "substitutions": []
  },
  "diagnostics": {
    "rawScores": {
      "overall": 0.0,
      "textAccuracy": 0.0,
      "pronunciation": 0.0,
      "levenshteinSimilarity": 0.0
    }
  },
  "alignment": {
    "used": false,
    "averageConfidence": null,
    "words": [],
    "meta": {
      "reason": "no_speech_detected"
    }
  }
}

Queued upload response example

Sample
{
  "success": true,
  "queued": true,
  "taskId": "a1f4387a-09fd-4ac9-8cd5-4eac34f6f6cc",
  "state": "PENDING",
  "ready": false,
  "statusPath": "/api/reading-score/task/a1f4387a-09fd-4ac9-8cd5-4eac34f6f6cc"
}

Completed status response example

Sample
{
  "taskId": "a1f4387a-09fd-4ac9-8cd5-4eac34f6f6cc",
  "state": "SUCCESS",
  "ready": true,
  "successful": true,
  "result": {
    "success": true,
    "assessment": {
      "passed": true,
      "status": "passed",
      "displayScoreMin": 80.0,
      "usedBestAttemptSelection": true
    },
    "scores": {
      "overall": 89.83,
      "textAccuracy": 90.0,
      "pronunciation": 89.45,
      "levenshteinSimilarity": 90.0
    }
  }
}
If alignment.used is false, pronunciation scoring was unavailable or too weak to trust. On non-silent reads, scores.pronunciation may become null, while the customer-facing scores.* fields usually still stay in the encouragement range. If the service confirms no speech with VAD, assessment.status becomes no_speech, assessment.passed becomes false, and scores.* return 0.0. Check student.meta.voiceActivity* for the VAD decision and diagnostics.rawScores when you need the underlying unclamped metrics or the raw text-only calculation.

Shared LLM API

Use one OpenAI-compatible base URL for pooled local models.

The shared LLM API exposes a stable OpenAI-compatible surface under this SaaS domain. Today it is backed by the broker-managed Qwen3 pool, and future local models can be added behind the same `/api/llm/v1` base URL so callers do not need to re-integrate.

GET/api/llm/v1/models

List live models

Returns the models currently exposed by the shared local LLM stack. Use this when you want runtime discovery instead of hard-coding one model forever.

POST/api/llm/v1/chat/completions

Create a chat completion

Accepts OpenAI-style `model`, `messages`, `temperature`, and `max_tokens` fields and forwards the request into the pooled local model router.

GET /api/llm/v1/models

Sample
curl -sS \
  -H "x-api-key: YOUR_API_KEY" \
  "https://www.kaleidovid.com/api/llm/v1/models"

Models response example

Sample
{
  "object": "list",
  "data": [
    {
      "id": "qwen3-4b",
      "object": "model",
      "owned_by": "kaleidovid-local"
    }
  ]
}

POST /api/llm/v1/chat/completions

Sample
curl -sS \
  -X POST \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "model": "qwen3-4b",
    "messages": [
      { "role": "system", "content": "You are a helpful assistant." },
      { "role": "user", "content": "Explain why a shared local LLM endpoint is useful." }
    ],
    "temperature": 0.4,
    "max_tokens": 256
  }' \
  "https://www.kaleidovid.com/api/llm/v1/chat/completions"

Python OpenAI SDK example

Sample
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://www.kaleidovid.com/api/llm/v1",
)

response = client.chat.completions.create(
    model="qwen3-4b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain the API in one short paragraph."},
    ],
    temperature=0.4,
    max_tokens=256,
)

print(response.choices[0].message.content)
The current production model exposed by this shared route is qwen3-4b. The important contract is the base URL, not the model family: as more local models are added, callers should discover them through /api/llm/v1/models and then choose a model id dynamically.

TTS API

Create speech audio with selectable language and model controls.

The TTS API exposes an asynchronous text-to-speech surface under `/api/tts/v1`. Submit text plus language/model options, receive a `taskId`, then poll the status endpoint until the job returns a generated WAV URL or a failure reason.

POST/api/tts/v1/custom-voices

Upload clone audio

Registers a short reference WAV or public HTTPS audio URL and returns a `custom_voices/...` path that can be reused across speech requests.

POST/api/tts/v1/audio/speech

Start speech synthesis

Creates a TTS generation task. The public route validates the API key, records usage, and forwards JSON options to the existing TTS worker stack.

GET/api/tts/v1/audio/speech/{taskId}

Check synthesis status

Returns Celery task state, readiness, progress when available, success/failure, and the generated audio under `result.url` when complete.

JSON request fields

Speech synthesis accepts JSON. Clone/reference audio can be passed as `clone_audio`, uploaded first through `/api/tts/v1/custom-voices`, or supplied as a public `https://` audio URL for automatic registration. Treat the returned `custom_voices/...` path as an opaque reusable reference and send it back exactly as returned.

FieldTypeRequiredNotes
textstringYesText to synthesize. Maximum length is 200 characters.
languagestringNoSupported values are `en`, `zh`, `vi`, `th`, `ms`, `lo`, `ja`, and `ko`. Defaults to `en`; Malay and Lao must use `omnivoice`.
modelstringNoSupported public models are `f5`, `vieneu_v2`, `cosytts`, `qwen_viet_tts`, and `omnivoice`. If omitted, the service chooses a default from the language.
seedintegerNoOptional deterministic seed from `0` to `2147483647`. Omit or send a negative value for random generation.
f5_samplestringNoOptional reference voice path for `f5` or `vieneu_v2`. Accepted forms include `/uploads/...`, `F5-TTS/...`, and `custom_voices/...`.
f5_thai_variantstringNoOnly used when `model=f5` and `language=th`. Supported values are `v1` and `v2`; the dashboard defaults to `v2`.
vieneu_expressionstringNoOnly used when `model=vieneu_v2` and `language=vi`. Supported values are `natural` and `storytelling`.
speakerstringNoOptional built-in speaker for `cosytts` or `qwen_viet_tts` when no clone audio is supplied. Examples: `Ryan`, `Vivian`, `yen_nhi`, `my_van`.
instructstringNoOptional voice/style instruction for `cosytts` and OmniVoice non-clone requests, such as `Warm, clear narration`. OmniVoice clone requests with `clone_audio` must omit `instruct`.
clone_audiostringNoOptional clone/reference audio for `cosytts`, `qwen_viet_tts`, or `omnivoice`. Use a returned `custom_voices/...` path from `/api/tts/v1/custom-voices`, an accepted saved path (`/uploads/...`, `F5-TTS/...`, `custom_voices/...`), or a public `https://` audio URL. Legacy alias: `qwen3_clone_audio`.
clone_textstringNoOptional transcript for `clone_audio`. Provide it when known; uploaded or hosted references are auto-transcribed when the ASR service is configured. For OmniVoice it is recommended for consistency but not required. Legacy alias: `qwen3_clone_text`.

POST /api/tts/v1/custom-voices

Sample
curl -sS \
  -X POST \
  -H "x-api-key: YOUR_API_KEY" \
  -F "audio_file=@thai-ref-female-4s.wav;type=audio/wav" \
  -F "name=thai_flashcard_female" \
  -F "model=omnivoice" \
  -F "language=th" \
  -F "transcript=reference transcript here" \
  "https://www.kaleidovid.com/api/tts/v1/custom-voices"

Custom voice response

Sample
{
  "success": true,
  "voice": {
    "id": "cv_123",
    "name": "thai_flashcard_female",
    "model": "omnivoice",
    "language": "th",
    "audioPath": "custom_voices/user-1/thai_flashcard_female.wav",
    "transcript": "reference transcript here"
  },
  "clone_audio": "custom_voices/user-1/thai_flashcard_female.wav",
  "clone_text": "reference transcript here",
  "ttsRequest": {
    "clone_audio": "custom_voices/user-1/thai_flashcard_female.wav",
    "clone_text": "reference transcript here"
  }
}

Register hosted reference

Sample
curl -sS \
  -X POST \
  -H "x-api-key: YOUR_API_KEY" \
  -F "audio_url=https://example.com/tts_refs/thai-ref-female-4s.wav" \
  -F "name=thai_flashcard_female" \
  -F "model=omnivoice" \
  -F "language=th" \
  "https://www.kaleidovid.com/api/tts/v1/custom-voices"

Cloned OmniVoice request

Sample
curl -sS \
  -X POST \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "text": "สวัสดีครับ",
    "language": "th",
    "model": "omnivoice",
    "clone_audio": "custom_voices/user-1/thai_flashcard_female.wav",
    "clone_text": "reference transcript here",
    "seed": 12345
  }' \
  "https://www.kaleidovid.com/api/tts/v1/audio/speech"

POST /api/tts/v1/audio/speech

Sample
curl -sS \
  -X POST \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "text": "Hello from the KaleidoVid TTS API.",
    "language": "en",
    "model": "cosytts",
    "speaker": "Ryan",
    "instruct": "Warm, clear narration",
    "seed": 12345
  }' \
  "https://www.kaleidovid.com/api/tts/v1/audio/speech"

Queued response example

Sample
{
  "success": true,
  "taskId": "a97bf5a8-16d4-44c7-b45b-c8a5f94c9e8f",
  "message": "TTS generation task started with COSYTTS",
  "statusPath": "/api/tts/v1/audio/speech/a97bf5a8-16d4-44c7-b45b-c8a5f94c9e8f"
}

GET /api/tts/v1/audio/speech/{taskId}

Sample
curl -sS \
  -H "x-api-key: YOUR_API_KEY" \
  "https://www.kaleidovid.com/api/tts/v1/audio/speech/a97bf5a8-16d4-44c7-b45b-c8a5f94c9e8f"

Completed status response example

Sample
{
  "taskId": "a97bf5a8-16d4-44c7-b45b-c8a5f94c9e8f",
  "state": "SUCCESS",
  "ready": true,
  "successful": true,
  "result": {
    "success": true,
    "url": "/uploads/generations/tts/tts_cosytts_20260605_1234.wav",
    "metadata": {
      "text": "Hello from the KaleidoVid TTS API.",
      "language": "en",
      "model": "cosytts",
      "duration": 2.14,
      "seed": 12345
    }
  }
}

Status response fields

FieldTypeRequiredNotes
taskIdstringAlways on successCelery task id returned by the start request. Use it to poll the status endpoint.
statusPathstringAlways on successRelative polling path for the public TTS status endpoint.
statestringOn status responsesCelery state such as `PENDING`, `STARTED`, `SUCCESS`, or `FAILURE`.
ready / successfulboolean / boolean | nullOn status responsesWhen `ready=true` and `successful=true`, read the generated audio from `result.url`.
result.urlstringWhen successfulRelative or absolute URL of the generated WAV file.
result.metadataobjectWhen successfulIncludes the resolved `text`, `language`, `model`, `duration`, optional `seed`, and model-specific metadata.
errorstringOn failureFailure reason returned when the job fails.

Python polling example

Sample
# pip install requests

import time
import requests

API_KEY = "YOUR_API_KEY"
BASE_URL = "https://www.kaleidovid.com/api/tts/v1"

start = requests.post(
    f"{BASE_URL}/audio/speech",
    headers={"x-api-key": API_KEY},
    json={
        "text": "Hello from the KaleidoVid TTS API.",
        "language": "en",
        "model": "cosytts",
        "speaker": "Ryan",
        "instruct": "Warm, clear narration",
    },
    timeout=180,
)
start.raise_for_status()
task_id = start.json()["taskId"]

while True:
    status = requests.get(f"{BASE_URL}/audio/speech/{task_id}", headers={"x-api-key": API_KEY}, timeout=60)
    status.raise_for_status()
    payload = status.json()
    if payload.get("ready"):
        break
    time.sleep(3)

if not payload.get("successful"):
    raise RuntimeError(payload.get("error", "TTS generation failed"))

print(payload["result"]["url"])

Failed response example

Sample
{
  "taskId": "a97bf5a8-16d4-44c7-b45b-c8a5f94c9e8f",
  "state": "FAILURE",
  "ready": true,
  "successful": false,
  "error": "TTS generation failed: upstream message"
}
Recommended model pairings: `f5` for Thai, `vieneu_v2` or `qwen_viet_tts` for Vietnamese, `cosytts` for English/Chinese/Japanese/Korean, and `omnivoice` for Malay/Lao or multilingual fallback. For cloned OmniVoice flashcard batches, reuse the same `clone_audio`, `clone_text` when available, and `seed` across every short word and sentence, and omit `instruct`. The reference audio anchors speaker identity; the seed makes sampling more repeatable, but it is not a substitute for a stable reference. The 200-character limit is per request, so split longer learning sentences on phrase or sentence boundaries and stitch WAVs client-side; chunks generated with the same clone reference and seed are expected to remain voice-consistent.

LTX23 Video API

Create Standard AV image-to-video jobs with the LTX23 video API.

The LTX23 Video API exposes a small asynchronous image-to-video surface under `/api/ltx23/v1`. Submit a prompt and input image, receive a `request_id`, then poll the status endpoint until the job returns a generated MP4 URL or a failure reason.

POST/api/ltx23/v1/videos/generations

Start image-to-video generation

Creates a Standard AV job. The public route validates the API key, records usage, and forwards the request to the Django task queue backed by the broker-managed LTX23 worker pool.

GET/api/ltx23/v1/videos/{request_id}

Check generation status

Returns `queued`, `processing`, `done`, `failed`, or `canceled`. When the job is done, the response includes `video.url`.

JSON request fields

The endpoint accepts JSON only. For direct file uploads from your own server or browser, convert the image to a base64 data URL and send it as image.url. Remote image URLs must be publicly reachable.

FieldTypeRequiredNotes
modelstringNoDefaults to `ltx23-standard`. Accepted aliases include `ltx23-standard`, `ltx23-standard-fast`, `ltx23-standard-quality`, `ltx23-standard-full`, `ltx23-standard-distilled`, `ltx23`, and `ltx-2.3`.
promptstringYesText instruction for how the input image should animate.
image.urlstringYesA base64 data URL, a same-site `/uploads/...` path, or a public `http(s)` image URL. JPG, PNG, and WEBP are accepted up to 25 MB. Private network hosts and redirects are rejected for remote URLs.
durationintegerNoDefaults to 5. Must be between 2 and 10 seconds.
aspect_ratiostringNoDefaults to `16:9`. Supported values are `16:9`, `9:16`, and `1:1`. The camelCase alias `aspectRatio` is also accepted.
resolutionstringNoDefaults to `720p`. Supported values are `480p` and `720p`.
seedintegerNoUse `-1` or omit the field for a random seed. Non-negative seeds are clamped to the service-safe range.

POST /api/ltx23/v1/videos/generations

Sample
curl -sS \
  -X POST \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "model": "ltx23-standard",
    "prompt": "A cinematic product shot with gentle camera motion.",
    "image": {
      "url": "https://example.com/input.png"
    },
    "duration": 5,
    "aspect_ratio": "16:9",
    "resolution": "720p",
    "seed": -1
  }' \
  "https://www.kaleidovid.com/api/ltx23/v1/videos/generations"

Queued response example

Sample
{
  "request_id": "6bcd2f29-4b74-4479-a3b3-f71a50a77dab",
  "status": "queued",
  "model": "ltx23-standard"
}

GET /api/ltx23/v1/videos/{request_id}

Sample
curl -sS \
  -H "x-api-key: YOUR_API_KEY" \
  "https://www.kaleidovid.com/api/ltx23/v1/videos/6bcd2f29-4b74-4479-a3b3-f71a50a77dab"

Done response example

Sample
{
  "request_id": "6bcd2f29-4b74-4479-a3b3-f71a50a77dab",
  "status": "done",
  "model": "ltx23-standard",
  "video": {
    "url": "/uploads/generations/ltx23/ltx23_av_20260426_abcdef.mp4"
  }
}

Status response fields

FieldTypeRequiredNotes
request_idstringAlwaysCelery task id used to poll the status endpoint.
statusstringAlwaysOne of `queued`, `processing`, `done`, `failed`, or `canceled`.
modelstringAlwaysThe public model alias associated with the request.
video.urlstringWhen doneRelative or absolute URL of the generated MP4. Present only when `status=done`.
progressobjectMaybeBest-effort task progress metadata while the job is processing.
errorstringOn failureFailure reason returned when the job fails.

Python polling example

Sample
# pip install requests

import base64
import mimetypes
import time
from pathlib import Path

import requests

API_KEY = "YOUR_API_KEY"
BASE_URL = "https://www.kaleidovid.com/api/ltx23/v1"
image_path = Path("input.png")
mime_type = mimetypes.guess_type(image_path.name)[0] or "image/png"
image_data_url = (
    f"data:{mime_type};base64,"
    + base64.b64encode(image_path.read_bytes()).decode("ascii")
)

start = requests.post(
    f"{BASE_URL}/videos/generations",
    headers={"x-api-key": API_KEY},
    json={
        "model": "ltx23-standard",
        "prompt": "A cinematic product shot with gentle camera motion.",
        "image": {"url": image_data_url},
        "duration": 5,
        "aspect_ratio": "16:9",
        "resolution": "720p",
        "seed": -1,
    },
    timeout=180,
)
start.raise_for_status()
request_id = start.json()["request_id"]

while True:
    status = requests.get(f"{BASE_URL}/videos/{request_id}", headers={"x-api-key": API_KEY}, timeout=60)
    status.raise_for_status()
    payload = status.json()
    if payload["status"] in {"done", "failed", "canceled"}:
        break
    time.sleep(5)

print(payload)

Failed response example

Sample
{
  "request_id": "6bcd2f29-4b74-4479-a3b3-f71a50a77dab",
  "status": "failed",
  "model": "ltx23-standard",
  "error": "Remote LTX image-to-video failed: upstream message"
}
The public model aliases map to the same Standard AV backend. `ltx23-standard`, `ltx23-standard-fast`, and `ltx23-standard-distilled` use the distilled variant; `ltx23-standard-quality` and `ltx23-standard-full` request the full variant. Use the status endpoint instead of holding a long HTTP connection open.

Errors And Limits

Expect explicit auth, quota, and upstream failures.

These APIs share the same team API key system, but their runtime error surfaces differ. Thai ASR enforces request and concurrent-session limits directly; Reading Score mainly returns validation or upstream-transcription failures; Shared LLM can surface model saturation; TTS can return model/language validation or synthesis runtime failures; LTX23 Video can return validation, upstream timeout, or async job failure responses.

FieldTypeRequiredNotes
401HTTPThai ASR + Reading Score + Shared LLM + TTS + LTX23 VideoMissing or invalid API key.
400HTTPTTS + LTX23 VideoInvalid JSON body, missing required text/prompt/image fields, unsupported model or language pairing, invalid seed, duration, aspect ratio, resolution, or media source.
429HTTP / WS closeThai ASRRequest-per-minute limit or concurrent-session limit exceeded. WebSocket clients can see close code `4429`.
4401WS closeThai ASRWebSocket authentication failed.
503HTTPThai ASR + Reading Score + Shared LLMPolicy lookup or the upstream ASR / transcription / LLM runtime was unavailable.
502HTTPTTS + LTX23 VideoThe Next.js public route could not reach the Django/TTS/LTX23 upstream, or the upstream timed out.
500HTTPTTSThe backend accepted the TTS request but the synthesis task or runtime returned an immediate server-side failure.
429HTTPShared LLMThe shared LLM runtime or upstream worker was saturated.
1011WS closeThai ASRUnexpected server-side streaming failure.

Thai ASR quota notes

  • The HTTP config and session routes can reject requests when the team request bucket is exhausted.
  • The WebSocket service can reject or close sessions when concurrent-session limits are exceeded.
  • Daily audio quota is checked during streaming as audio accumulates.

Reading Score validation notes

  • student_audio is required.
  • You must send at least one of reference_text or reference_audio.
  • language must resolve to Thai. Other values are rejected.

TTS validation notes

  • text is required and capped at 200 characters.
  • Vietnamese must use vieneu_v2, qwen_viet_tts, or omnivoice.
  • Malay and Lao must use omnivoice.

Legacy Routes

Prefer the canonical routes for new integrations.

Compatibility paths still exist for older callers, but new integrations should standardize on the canonical endpoints below.

CanonicalLegacy AliasNotes
https://www.kaleidovid.com/api/thai-asrhttps://www.kaleidovid.com/v2/asr/low-latency/thThai ASR streaming surface. Prefer `/api/thai-asr/*`.
https://www.kaleidovid.com/api/reading-scorehttps://www.kaleidovid.com/api/low-latency-asr/reading-scoreReading Score upload API. Prefer `/api/reading-score`.
https://www.kaleidovid.com/api/reading-score/task/{taskId}https://www.kaleidovid.com/api/low-latency-asr/reading-score/task/{taskId}Reading Score queued-job status API. Use after an upload returns HTTP 202.