{"slug":"assemblyai-transcription","title":"AssemblyAI Transcription — Models, Audio Intelligence, LeMUR","tags":["assemblyai","transcription","sentiment","entity-detection","lemur","post-call"],"agent_summary":"Transcribe call recordings with Universal-2 / Universal-3-Pro models. Sentiment analysis, entity detection (44 types), PII redaction, speaker diarization, summarization. LeMUR applies LLMs across batches of up to 200 hours. Node SDK + REST contract.","trigger_phrases":["AssemblyAI transcription","Universal-2 model","LeMUR audio analysis","speaker diarization AssemblyAI","sentiment_analysis: true","entity_detection","PII redaction transcript","AssemblyAI realtime"],"runnable":true,"markdown":"\n# AssemblyAI — Transcription and Audio Intelligence\n\nUsed in BirdsEyeROI's lead pipeline to transcribe recorded calls and extract structured intent/sentiment data. Pairs naturally with SignalWire's `record_call` output (URL → AssemblyAI → analysis → DB).\n\n## Install (Node)\n\n```bash\nnpm install assemblyai\n```\n\n```javascript\nimport { AssemblyAI } from 'assemblyai';\nconst client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });\n```\n\n## Basic transcription\n\n```javascript\nconst transcript = await client.transcripts.transcribe({\n audio_url: 'https://your.api/recordings/abc123.mp3',\n // speech_model defaults to universal-2; explicit value not required\n});\nconsole.log(transcript.text);\n```\n\n## Full audio intelligence call\n\n```javascript\nconst transcript = await client.transcripts.transcribe({\n audio_url: 'https://your.api/recordings/abc123.mp3',\n sentiment_analysis: true,\n entity_detection: true,\n speaker_labels: true, // diarization\n redact_pii: true,\n redact_pii_policies: ['phone_number', 'email_address', 'credit_card_number'],\n redact_pii_sub: 'entity_name', // or 'hash'\n iab_categories: true,\n auto_chapters: true,\n summarization: true,\n});\n\ntranscript.sentiment_analysis_results.forEach(r => {\n console.log(r.text, r.sentiment, r.confidence);\n // sentiment: \"POSITIVE\" | \"NEGATIVE\" | \"NEUTRAL\"\n});\n\ntranscript.entities.forEach(e => {\n console.log(e.text, e.entity_type);\n // 44 entity types: person, company, email, phone, date, location, etc.\n});\n```\n\n## Models\n\n| Model | Languages | Price | Notes |\n|---|---|---|---|\n| **Universal-2** (default) | 99 | $0.15/hr | High accuracy, low latency. 200-word keyterms. Multichannel. Auto language detect. Code switching. Diarization. |\n| **Universal-3-Pro** | 6 (optimized) | $0.21/hr | State-of-the-art accuracy. 1000-word keyterms. Natural-language prompting. |\n| **Universal-Streaming** | 99 | real-time | Real-time with immutable results. Low latency. |\n| ~~Nano~~ | ~~—~~ | ~~deprecated~~ | **Replaced by Universal-2** — 53% more accurate at the same price. Remove `speech_model: 'nano'` from existing code. |\n\n## Audio Intelligence features\n\n| Feature | Param | Cost | Output |\n|---|---|---|---|\n| **Sentiment Analysis** | `sentiment_analysis: true` | $0.02/hr | Per-sentence POSITIVE/NEGATIVE/NEUTRAL + confidence |\n| **Entity Detection** | `entity_detection: true` | $0.08/hr | 44 entity types with confidence scores |\n| **PII Redaction** | `redact_pii: true` | included | Replace PII with hashes or entity labels |\n| **Topic Detection** | `iab_categories: true` | varies | IAB content-taxonomy categories |\n| **Auto Chapters** | `auto_chapters: true` | varies | Timestamped chapter summaries |\n| **Summarization** | `summarization: true` | varies | Paragraph / bullets / gist / headline |\n| **Speaker Labels** | `speaker_labels: true` | included | Speaker A / B / C diarization |\n\n## LeMUR — LLMs over transcripts\n\nApply an LLM (Claude, GPT-4, Gemini, etc.) across one or many transcripts. Batch up to 200 hours of audio per call.\n\n```javascript\nconst { response } = await client.lemur.task({\n transcript_ids: ['transcript-id-1', 'transcript-id-2'],\n prompt: 'What were the main complaints mentioned across these calls?',\n final_model: 'anthropic/claude-sonnet-4-6',\n});\n```\n\nSupported via the LeMUR LLM Gateway (Claude Sonnet/Opus 4.6, OpenAI GPT-4 family, Google Gemini, 15+ models total).\n\n### Use cases\n\n- Batch analyze calls: `\"What are the top 3 pain points?\"`\n- Auto-score leads: `\"Is this caller a qualified lead? Why?\"`\n- Generate summaries: `\"Summarize this call in 3 bullet points\"`\n- Extract action items: `\"What follow-ups were promised?\"`\n\n## Real-time streaming\n\n```javascript\nconst transcriber = client.realtime.transcriber({ sampleRate: 16000 });\ntranscriber.on('transcript', (t) => {\n if (t.message_type === 'FinalTranscript') console.log(t.text);\n});\nawait transcriber.connect();\n// transcriber.sendAudio(audioBuffer)\n```\n\nUse real-time when you need live transcripts in a UI. For SignalWire calls, prefer `live_transcribe` (Deepgram) for in-call streaming and AssemblyAI for post-call deep analysis.\n\n## REST API\n\n### Submit\n\n```bash\nPOST https://api.assemblyai.com/v2/transcript\nAuthorization: YOUR_API_KEY\nContent-Type: application/json\n\n{\n \"audio_url\": \"https://example.com/audio.mp3\",\n \"sentiment_analysis\": true,\n \"entity_detection\": true\n}\n```\n\n### Status\n\n```bash\nGET https://api.assemblyai.com/v2/transcript/{id}\nAuthorization: YOUR_API_KEY\n```\n\nStatus sequence: `queued` → `processing` → `completed` (or `error`).\n\n### Delete\n\n```bash\nDELETE https://api.assemblyai.com/v2/transcript/{id}\n```\n\n## Pairing with SignalWire `record_call`\n\nThe canonical pipeline:\n\n1. SWML `record_call` produces a recording URL (callback to your `status_url` webhook).\n2. Your webhook handler enqueues an AssemblyAI transcription job with `audio_url` set to the recording URL.\n3. AssemblyAI calls your `webhook_url` (or you poll) when complete.\n4. Store transcript + sentiment + entities + LeMUR-derived structured fields in your DB.\n\n```javascript\n// In your status_url webhook handler:\nconst transcript = await client.transcripts.transcribe({\n audio_url: req.body.params.url, // from SignalWire status_url\n sentiment_analysis: true,\n entity_detection: true,\n speaker_labels: true,\n});\n\nawait db.calls.update({\n call_id: req.body.params.call_id,\n}, {\n transcript: transcript.text,\n sentiment: aggregateSentiment(transcript.sentiment_analysis_results),\n entities: extractEntities(transcript.entities),\n});\n```\n\n## Anti-patterns\n\n- Leaving `speech_model: 'nano'` in code — deprecated. Remove the param to default to Universal-2.\n- Skipping `speaker_labels: true` when callers and agents need separation — sentiment per side gets mixed.\n- Running `entity_detection: true` on every call indiscriminately — costs $0.08/hr. Only enable when you actually use the entities.\n- Sending audio_url that requires auth without including credentials — AssemblyAI can't fetch it.\n- Calling LeMUR on a single transcript when AssemblyAI's built-in summarization would do — LeMUR is for batch / cross-transcript analysis.\n\n## See also\n\n- [SignalWire Call Intelligence pipeline](/topic/signalwire-call-intelligence)\n- [SWML record and transcribe](/topic/swml-record-and-transcribe)\n- [Fish Audio TTS](/topic/fish-audio-tts)\n","html":"

AssemblyAI — Transcription and Audio Intelligence

\n

Used in BirdsEyeROI's lead pipeline to transcribe recorded calls and extract structured intent/sentiment data. Pairs naturally with SignalWire's record_call output (URL → AssemblyAI → analysis → DB).

\n

Install (Node)

\n
npm install assemblyai\n
\n
import { AssemblyAI } from 'assemblyai';\nconst client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });\n
\n

Basic transcription

\n
const transcript = await client.transcripts.transcribe({\n  audio_url: 'https://your.api/recordings/abc123.mp3',\n  // speech_model defaults to universal-2; explicit value not required\n});\nconsole.log(transcript.text);\n
\n

Full audio intelligence call

\n
const transcript = await client.transcripts.transcribe({\n  audio_url: 'https://your.api/recordings/abc123.mp3',\n  sentiment_analysis: true,\n  entity_detection: true,\n  speaker_labels: true,            // diarization\n  redact_pii: true,\n  redact_pii_policies: ['phone_number', 'email_address', 'credit_card_number'],\n  redact_pii_sub: 'entity_name',   // or 'hash'\n  iab_categories: true,\n  auto_chapters: true,\n  summarization: true,\n});\n\ntranscript.sentiment_analysis_results.forEach(r => {\n  console.log(r.text, r.sentiment, r.confidence);\n  // sentiment: \"POSITIVE\" | \"NEGATIVE\" | \"NEUTRAL\"\n});\n\ntranscript.entities.forEach(e => {\n  console.log(e.text, e.entity_type);\n  // 44 entity types: person, company, email, phone, date, location, etc.\n});\n
\n

Models

\n

| Model | Languages | Price | Notes |\n|---|---|---|---|\n| Universal-2 (default) | 99 | $0.15/hr | High accuracy, low latency. 200-word keyterms. Multichannel. Auto language detect. Code switching. Diarization. |\n| Universal-3-Pro | 6 (optimized) | $0.21/hr | State-of-the-art accuracy. 1000-word keyterms. Natural-language prompting. |\n| Universal-Streaming | 99 | real-time | Real-time with immutable results. Low latency. |\n| ~~Nano~~ | ~~—~~ | ~~deprecated~~ | Replaced by Universal-2 — 53% more accurate at the same price. Remove speech_model: 'nano' from existing code. |

\n

Audio Intelligence features

\n

| Feature | Param | Cost | Output |\n|---|---|---|---|\n| Sentiment Analysis | sentiment_analysis: true | $0.02/hr | Per-sentence POSITIVE/NEGATIVE/NEUTRAL + confidence |\n| Entity Detection | entity_detection: true | $0.08/hr | 44 entity types with confidence scores |\n| PII Redaction | redact_pii: true | included | Replace PII with hashes or entity labels |\n| Topic Detection | iab_categories: true | varies | IAB content-taxonomy categories |\n| Auto Chapters | auto_chapters: true | varies | Timestamped chapter summaries |\n| Summarization | summarization: true | varies | Paragraph / bullets / gist / headline |\n| Speaker Labels | speaker_labels: true | included | Speaker A / B / C diarization |

\n

LeMUR — LLMs over transcripts

\n

Apply an LLM (Claude, GPT-4, Gemini, etc.) across one or many transcripts. Batch up to 200 hours of audio per call.

\n
const { response } = await client.lemur.task({\n  transcript_ids: ['transcript-id-1', 'transcript-id-2'],\n  prompt: 'What were the main complaints mentioned across these calls?',\n  final_model: 'anthropic/claude-sonnet-4-6',\n});\n
\n

Supported via the LeMUR LLM Gateway (Claude Sonnet/Opus 4.6, OpenAI GPT-4 family, Google Gemini, 15+ models total).

\n

Use cases

\n\n

Real-time streaming

\n
const transcriber = client.realtime.transcriber({ sampleRate: 16000 });\ntranscriber.on('transcript', (t) => {\n  if (t.message_type === 'FinalTranscript') console.log(t.text);\n});\nawait transcriber.connect();\n// transcriber.sendAudio(audioBuffer)\n
\n

Use real-time when you need live transcripts in a UI. For SignalWire calls, prefer live_transcribe (Deepgram) for in-call streaming and AssemblyAI for post-call deep analysis.

\n

REST API

\n

Submit

\n
POST https://api.assemblyai.com/v2/transcript\nAuthorization: YOUR_API_KEY\nContent-Type: application/json\n\n{\n  \"audio_url\": \"https://example.com/audio.mp3\",\n  \"sentiment_analysis\": true,\n  \"entity_detection\": true\n}\n
\n

Status

\n
GET https://api.assemblyai.com/v2/transcript/{id}\nAuthorization: YOUR_API_KEY\n
\n

Status sequence: queued → processing → completed (or error).

\n

Delete

\n
DELETE https://api.assemblyai.com/v2/transcript/{id}\n
\n

Pairing with SignalWire record_call

\n

The canonical pipeline:

\n
    \n
  1. SWML record_call produces a recording URL (callback to your status_url webhook).
    2. \n
  2. Your webhook handler enqueues an AssemblyAI transcription job with audio_url set to the recording URL.
    3. \n
  3. AssemblyAI calls your webhook_url (or you poll) when complete.
    4. \n
  4. Store transcript + sentiment + entities + LeMUR-derived structured fields in your DB.
    5. \n
\n
// In your status_url webhook handler:\nconst transcript = await client.transcripts.transcribe({\n  audio_url: req.body.params.url,           // from SignalWire status_url\n  sentiment_analysis: true,\n  entity_detection: true,\n  speaker_labels: true,\n});\n\nawait db.calls.update({\n  call_id: req.body.params.call_id,\n}, {\n  transcript: transcript.text,\n  sentiment: aggregateSentiment(transcript.sentiment_analysis_results),\n  entities: extractEntities(transcript.entities),\n});\n
\n

Anti-patterns

\n\n

See also

\n\n"}