AssemblyAI — Transcription and Audio Intelligence

Used in BirdsEyeROI's lead pipeline to transcribe recorded calls and extract structured intent/sentiment data. Pairs naturally with SignalWire's record_call output (URL → AssemblyAI → analysis → DB).

Install (Node)

npm install assemblyai

import { AssemblyAI } from 'assemblyai';
const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });

Basic transcription

const transcript = await client.transcripts.transcribe({
  audio_url: 'https://your.api/recordings/abc123.mp3',
  // speech_model defaults to universal-2; explicit value not required
});
console.log(transcript.text);

Full audio intelligence call

const transcript = await client.transcripts.transcribe({
  audio_url: 'https://your.api/recordings/abc123.mp3',
  sentiment_analysis: true,
  entity_detection: true,
  speaker_labels: true,            // diarization
  redact_pii: true,
  redact_pii_policies: ['phone_number', 'email_address', 'credit_card_number'],
  redact_pii_sub: 'entity_name',   // or 'hash'
  iab_categories: true,
  auto_chapters: true,
  summarization: true,
});

transcript.sentiment_analysis_results.forEach(r => {
  console.log(r.text, r.sentiment, r.confidence);
  // sentiment: "POSITIVE" | "NEGATIVE" | "NEUTRAL"
});

transcript.entities.forEach(e => {
  console.log(e.text, e.entity_type);
  // 44 entity types: person, company, email, phone, date, location, etc.
});

Models

| Model | Languages | Price | Notes | |---|---|---|---| | Universal-2 (default) | 99 | $0.15/hr | High accuracy, low latency. 200-word keyterms. Multichannel. Auto language detect. Code switching. Diarization. | | Universal-3-Pro | 6 (optimized) | $0.21/hr | State-of-the-art accuracy. 1000-word keyterms. Natural-language prompting. | | Universal-Streaming | 99 | real-time | Real-time with immutable results. Low latency. | | ~~Nano~~ | ~~—~~ | ~~deprecated~~ | Replaced by Universal-2 — 53% more accurate at the same price. Remove speech_model: 'nano' from existing code. |

Audio Intelligence features

| Feature | Param | Cost | Output | |---|---|---|---| | Sentiment Analysis | sentiment_analysis: true | $0.02/hr | Per-sentence POSITIVE/NEGATIVE/NEUTRAL + confidence | | Entity Detection | entity_detection: true | $0.08/hr | 44 entity types with confidence scores | | PII Redaction | redact_pii: true | included | Replace PII with hashes or entity labels | | Topic Detection | iab_categories: true | varies | IAB content-taxonomy categories | | Auto Chapters | auto_chapters: true | varies | Timestamped chapter summaries | | Summarization | summarization: true | varies | Paragraph / bullets / gist / headline | | Speaker Labels | speaker_labels: true | included | Speaker A / B / C diarization |

LeMUR — LLMs over transcripts

Apply an LLM (Claude, GPT-4, Gemini, etc.) across one or many transcripts. Batch up to 200 hours of audio per call.

const { response } = await client.lemur.task({
  transcript_ids: ['transcript-id-1', 'transcript-id-2'],
  prompt: 'What were the main complaints mentioned across these calls?',
  final_model: 'anthropic/claude-sonnet-4-6',
});

Supported via the LeMUR LLM Gateway (Claude Sonnet/Opus 4.6, OpenAI GPT-4 family, Google Gemini, 15+ models total).

Use cases

• Batch analyze calls: "What are the top 3 pain points?"
• Auto-score leads: "Is this caller a qualified lead? Why?"
• Generate summaries: "Summarize this call in 3 bullet points"
• Extract action items: "What follow-ups were promised?"

Real-time streaming

const transcriber = client.realtime.transcriber({ sampleRate: 16000 });
transcriber.on('transcript', (t) => {
  if (t.message_type === 'FinalTranscript') console.log(t.text);
});
await transcriber.connect();
// transcriber.sendAudio(audioBuffer)

Use real-time when you need live transcripts in a UI. For SignalWire calls, prefer live_transcribe (Deepgram) for in-call streaming and AssemblyAI for post-call deep analysis.

REST API

Submit

POST https://api.assemblyai.com/v2/transcript
Authorization: YOUR_API_KEY
Content-Type: application/json

{
  "audio_url": "https://example.com/audio.mp3",
  "sentiment_analysis": true,
  "entity_detection": true
}

Status

GET https://api.assemblyai.com/v2/transcript/{id}
Authorization: YOUR_API_KEY

Status sequence: queued → processing → completed (or error).

Delete

DELETE https://api.assemblyai.com/v2/transcript/{id}

Pairing with SignalWire record_call

The canonical pipeline:

1. SWML record_call produces a recording URL (callback to your status_url webhook).
2. Your webhook handler enqueues an AssemblyAI transcription job with audio_url set to the recording URL.
3. AssemblyAI calls your webhook_url (or you poll) when complete.
4. Store transcript + sentiment + entities + LeMUR-derived structured fields in your DB.

// In your status_url webhook handler:
const transcript = await client.transcripts.transcribe({
  audio_url: req.body.params.url,           // from SignalWire status_url
  sentiment_analysis: true,
  entity_detection: true,
  speaker_labels: true,
});

await db.calls.update({
  call_id: req.body.params.call_id,
}, {
  transcript: transcript.text,
  sentiment: aggregateSentiment(transcript.sentiment_analysis_results),
  entities: extractEntities(transcript.entities),
});

Anti-patterns

• Leaving speech_model: 'nano' in code — deprecated. Remove the param to default to Universal-2.
• Skipping speaker_labels: true when callers and agents need separation — sentiment per side gets mixed.
• Running entity_detection: true on every call indiscriminately — costs $0.08/hr. Only enable when you actually use the entities.
• Sending audio_url that requires auth without including credentials — AssemblyAI can't fetch it.
• Calling LeMUR on a single transcript when AssemblyAI's built-in summarization would do — LeMUR is for batch / cross-transcript analysis.

See also

• SignalWire Call Intelligence pipeline
• SWML record and transcribe
• Fish Audio TTS