Welcome to Anam

Anam uses WebRTC technology to deliver real-time video and audio streaming of AI personas. This enables sub-second response times and natural conversation flow that feels truly interactive.

How Streaming Works

Connection Establishment

Your client establishes a WebRTC peer-to-peer connection using the session token

Media Stream Setup

Anam begins streaming the persona’s video feed to your specified video element

Bidirectional Communication

Your application can send messages while receiving real-time video and audio responses

Synchronized Output

The persona’s speech is automatically synchronized with lip movements and facial expressions

Stream Architecture

Understanding the streaming architecture helps you optimize performance and handle edge cases:

Basic Streaming Setup

Simple Video Streaming

The most basic streaming setup connects a persona to a video element:

import { createClient } from '@anam-ai/js-sdk';

async function initializeStreaming() {
  // Get session token from your server
  const sessionToken = await getSessionToken();
  
  // Create client
  const anamClient = createClient(sessionToken);
  
  // Start streaming to video element
  await anamClient.streamToVideoElement('persona-video');
  
  console.log('Streaming started successfully');
}

// HTML video element
// <video id="persona-video" autoplay playsinline muted></video>

Advanced Streaming Configuration

For more control over the streaming experience:

class PersonaStreaming {
  constructor(sessionToken) {
    this.client = createClient(sessionToken);
    this.isConnected = false;
    this.streamQuality = 'high';
  }

  async initialize(videoElementId, options = {}) {
    const defaultOptions = {
      autoplay: true,
      muted: true, // Start muted to comply with browser policies
      controls: false,
      playsInline: true
    };

    const streamOptions = { ...defaultOptions, ...options };

    try {
      // Set up event listeners before connecting
      this.setupEventListeners();
      
      // Configure stream quality
      await this.client.setStreamQuality(this.streamQuality);
      
      // Start streaming
      await this.client.streamToVideoElement(videoElementId, streamOptions);
      
      this.isConnected = true;
      console.log('Streaming initialized with options:', streamOptions);
      
    } catch (error) {
      console.error('Failed to initialize streaming:', error);
      throw error;
    }
  }

  setupEventListeners() {
    this.client.on('stream_started', () => {
      console.log('Video stream started');
      this.handleStreamStart();
    });

    this.client.on('stream_ended', () => {
      console.log('Video stream ended');
      this.handleStreamEnd();
    });

    this.client.on('stream_error', (error) => {
      console.error('Stream error:', error);
      this.handleStreamError(error);
    });

    this.client.on('quality_changed', (quality) => {
      console.log('Stream quality changed to:', quality);
      this.streamQuality = quality;
    });
  }

  handleStreamStart() {
    // Update UI to show streaming state
    document.getElementById('status').textContent = 'Connected';
    document.getElementById('persona-video').classList.add('streaming');
  }

  handleStreamEnd() {
    // Update UI and potentially reconnect
    document.getElementById('status').textContent = 'Disconnected';
    this.isConnected = false;
  }

  handleStreamError(error) {
    // Implement retry logic or user notification
    if (error.code === 'NETWORK_ERROR') {
      this.retryConnection();
    } else {
      this.showErrorToUser(error.message);
    }
  }

  async retryConnection() {
    const maxRetries = 3;
    let retryCount = 0;

    while (retryCount < maxRetries && !this.isConnected) {
      try {
        await new Promise(resolve => setTimeout(resolve, 1000 * (retryCount + 1)));
        await this.client.reconnect();
        break;
      } catch (error) {
        retryCount++;
        console.warn(`Retry ${retryCount} failed:`, error);
      }
    }
  }
}

Stream Quality Management

Adaptive Quality

Anam automatically adjusts stream quality based on network conditions, but you can also control it manually:

class AdaptiveStreaming {
  constructor(anamClient) {
    this.client = anamClient;
    this.networkMonitor = new NetworkQualityMonitor();
  }

  async optimizeStreamQuality() {
    // Monitor network conditions
    this.networkMonitor.on('quality_change', async (networkQuality) => {
      let streamQuality;
      
      switch (networkQuality) {
        case 'excellent':
          streamQuality = 'ultra';
          break;
        case 'good':
          streamQuality = 'high';
          break;
        case 'fair':
          streamQuality = 'medium';
          break;
        case 'poor':
          streamQuality = 'low';
          break;
        default:
          streamQuality = 'auto';
      }

      try {
        await this.client.setStreamQuality(streamQuality);
        console.log(`Stream quality adjusted to: ${streamQuality}`);
      } catch (error) {
        console.error('Failed to adjust stream quality:', error);
      }
    });
  }

  // Manual quality control
  async setQuality(quality) {
    const validQualities = ['auto', 'low', 'medium', 'high', 'ultra'];
    
    if (!validQualities.includes(quality)) {
      throw new Error(`Invalid quality setting: ${quality}`);
    }

    await this.client.setStreamQuality(quality);
  }
}

// Usage
const adaptiveStreaming = new AdaptiveStreaming(anamClient);
await adaptiveStreaming.optimizeStreamQuality();

// Manual quality adjustment
await adaptiveStreaming.setQuality('high');

Quality Settings

Ultra Quality

1080p, 60fps - Best visual quality for high-end applications

High Quality

720p, 30fps - Great balance of quality and performance

Medium Quality

480p, 30fps - Good quality with lower bandwidth usage

Low Quality

360p, 24fps - Optimized for poor network conditions

Audio Management

Audio Controls

Handle audio playback and user interactions:

class AudioManager {
  constructor(anamClient) {
    this.client = anamClient;
    this.isMuted = true; // Start muted for browser compliance
    this.volume = 1.0;
  }

  async enableAudio() {
    try {
      // Request user interaction first (required by browsers)
      await this.requestUserInteraction();
      
      // Unmute the stream
      await this.client.setMuted(false);
      this.isMuted = false;
      
      console.log('Audio enabled');
      this.updateAudioUI();
      
    } catch (error) {
      console.error('Failed to enable audio:', error);
      throw error;
    }
  }

  async requestUserInteraction() {
    return new Promise((resolve) => {
      const button = document.getElementById('enable-audio-btn');
      const handleClick = () => {
        button.removeEventListener('click', handleClick);
        button.style.display = 'none';
        resolve();
      };
      button.addEventListener('click', handleClick);
      button.style.display = 'block';
      button.textContent = 'Click to Enable Audio';
    });
  }

  async toggleMute() {
    this.isMuted = !this.isMuted;
    await this.client.setMuted(this.isMuted);
    this.updateAudioUI();
  }

  async setVolume(volume) {
    this.volume = Math.max(0, Math.min(1, volume));
    await this.client.setVolume(this.volume);
    this.updateAudioUI();
  }

  updateAudioUI() {
    const muteButton = document.getElementById('mute-button');
    const volumeSlider = document.getElementById('volume-slider');
    
    muteButton.textContent = this.isMuted ? '🔇' : '🔊';
    volumeSlider.value = this.volume;
  }
}

// HTML controls
/*
<button id="enable-audio-btn" style="display: none;">Enable Audio</button>
<button id="mute-button" onclick="audioManager.toggleMute()">🔇</button>
<input id="volume-slider" type="range" min="0" max="1" step="0.1" 
       onchange="audioManager.setVolume(this.value)">
*/

Performance Optimization

Connection Optimization

Optimize streaming performance for different scenarios:

class StreamOptimizer {
  constructor(anamClient) {
    this.client = anamClient;
    this.performanceMetrics = {
      latency: 0,
      frameRate: 0,
      bandwidth: 0
    };
  }

  async optimizeForMobile() {
    // Reduce quality for mobile devices
    await this.client.setStreamQuality('medium');
    
    // Enable battery optimization
    await this.client.setConfig({
      powerSaveMode: true,
      adaptiveFrameRate: true,
      reducedMotion: true
    });
    
    console.log('Mobile optimizations applied');
  }

  async optimizeForDesktop() {
    // Higher quality for desktop
    await this.client.setStreamQuality('high');
    
    // Full feature set
    await this.client.setConfig({
      powerSaveMode: false,
      adaptiveFrameRate: false,
      reducedMotion: false
    });
    
    console.log('Desktop optimizations applied');
  }

  monitorPerformance() {
    setInterval(async () => {
      try {
        const stats = await this.client.getStreamStats();
        this.performanceMetrics = {
          latency: stats.latency,
          frameRate: stats.frameRate,
          bandwidth: stats.bandwidth
        };
        
        this.adjustBasedOnPerformance();
      } catch (error) {
        console.error('Failed to get stream stats:', error);
      }
    }, 5000); // Check every 5 seconds
  }

  adjustBasedOnPerformance() {
    const { latency, frameRate, bandwidth } = this.performanceMetrics;
    
    // Auto-adjust quality based on performance
    if (latency > 500 || frameRate < 20) {
      this.client.setStreamQuality('low');
      console.log('Quality reduced due to performance issues');
    } else if (latency < 100 && frameRate > 28 && bandwidth > 2000) {
      this.client.setStreamQuality('high');
      console.log('Quality increased due to good performance');
    }
  }
}

Bandwidth Management

// Optimize for poor network conditions
await anamClient.setConfig({
  streamQuality: 'low',
  audioQuality: 'compressed',
  frameRate: 15,
  adaptiveBitrate: true
});

// Optimize for poor network conditions
await anamClient.setConfig({
  streamQuality: 'low',
  audioQuality: 'compressed',
  frameRate: 15,
  adaptiveBitrate: true
});

// Optimize for excellent network conditions
await anamClient.setConfig({
  streamQuality: 'ultra',
  audioQuality: 'high',
  frameRate: 60,
  adaptiveBitrate: false
});

// Let Anam handle optimization automatically
await anamClient.setConfig({
  streamQuality: 'auto',
  audioQuality: 'auto',
  adaptiveBitrate: true,
  networkAdaptation: true
});

Handling Stream Events

Connection Lifecycle

Monitor and respond to streaming events:

class StreamEventHandler {
  constructor(anamClient) {
    this.client = anamClient;
    this.connectionState = 'disconnected';
    this.setupEventListeners();
  }

  setupEventListeners() {
    // Connection events
    this.client.on('connecting', () => {
      this.connectionState = 'connecting';
      this.showStatus('Connecting to persona...', 'connecting');
    });

    this.client.on('connected', () => {
      this.connectionState = 'connected';
      this.showStatus('Connected', 'success');
      this.enableInteraction();
    });

    this.client.on('disconnected', () => {
      this.connectionState = 'disconnected';
      this.showStatus('Disconnected', 'error');
      this.disableInteraction();
    });

    // Stream quality events
    this.client.on('stream_quality_changed', (quality) => {
      console.log(`Stream quality changed to: ${quality}`);
      this.updateQualityIndicator(quality);
    });

    // Performance events
    this.client.on('poor_connection', (metrics) => {
      console.warn('Poor connection detected:', metrics);
      this.showBandwidthWarning();
    });

    this.client.on('connection_recovered', () => {
      console.log('Connection quality improved');
      this.hideBandwidthWarning();
    });

    // Error events
    this.client.on('stream_error', (error) => {
      this.handleStreamError(error);
    });
  }

  showStatus(message, type) {
    const statusElement = document.getElementById('connection-status');
    statusElement.textContent = message;
    statusElement.className = `status ${type}`;
  }

  enableInteraction() {
    document.getElementById('message-input').disabled = false;
    document.getElementById('send-button').disabled = false;
  }

  disableInteraction() {
    document.getElementById('message-input').disabled = true;
    document.getElementById('send-button').disabled = true;
  }

  updateQualityIndicator(quality) {
    const indicator = document.getElementById('quality-indicator');
    indicator.textContent = quality.toUpperCase();
    indicator.className = `quality-indicator ${quality}`;
  }

  showBandwidthWarning() {
    const warning = document.getElementById('bandwidth-warning');
    warning.style.display = 'block';
    warning.textContent = 'Poor network connection detected. Stream quality may be reduced.';
  }

  hideBandwidthWarning() {
    document.getElementById('bandwidth-warning').style.display = 'none';
  }

  handleStreamError(error) {
    console.error('Stream error:', error);
    
    switch (error.code) {
      case 'MEDIA_ERROR':
        this.showError('Video playback error. Please refresh the page.');
        break;
      case 'NETWORK_ERROR':
        this.showError('Network connection lost. Attempting to reconnect...');
        this.attemptReconnection();
        break;
      case 'PERMISSION_ERROR':
        this.showError('Browser permissions required for video playback.');
        break;
      default:
        this.showError('An unexpected error occurred. Please try again.');
    }
  }

  async attemptReconnection() {
    try {
      await this.client.reconnect();
    } catch (error) {
      this.showError('Failed to reconnect. Please refresh the page.');
    }
  }

  showError(message) {
    const errorElement = document.getElementById('error-message');
    errorElement.textContent = message;
    errorElement.style.display = 'block';
  }
}

Common Streaming Issues

Video Not Appearing

Symptoms: Black screen or no video element content

Common Causes & Solutions:

Missing autoplay attribute: Add autoplay to video element
Browser autoplay policy: Require user interaction before streaming
Invalid session token: Check token creation and expiration
Network connectivity: Verify internet connection and firewall settings

// Ensure proper video element setup
const video = document.getElementById('persona-video');
video.autoplay = true;
video.playsInline = true;
video.muted = true; // Required for autoplay in most browsers

// Handle autoplay policy
anamClient.on('autoplay_blocked', async () => {
  // Show user interaction prompt
  await showPlayButton();
});

Audio Not Playing

Symptoms: Video appears but no sound

Common Causes & Solutions:

Browser autoplay policy: Audio requires user interaction
Muted by default: Browsers often start videos muted
Audio permissions: Check browser audio permissions
Volume settings: Verify system and application volume

// Handle audio enablement
const enableAudioButton = document.getElementById('enable-audio');
enableAudioButton.addEventListener('click', async () => {
  await anamClient.setMuted(false);
  enableAudioButton.style.display = 'none';
});

Poor Video Quality

Symptoms: Pixelated, blurry, or low-resolution video

Common Causes & Solutions:

Bandwidth limitations: Stream quality auto-reduced
Device performance: Lower quality for mobile/older devices
Network congestion: Temporary quality reduction
Manual quality setting: Check if quality is manually set to low

// Check and adjust quality
const currentQuality = await anamClient.getStreamQuality();
console.log('Current quality:', currentQuality);

// Force higher quality if network allows
if (currentQuality === 'low') {
  try {
    await anamClient.setStreamQuality('medium');
  } catch (error) {
    console.log('Cannot increase quality due to network constraints');
  }
}

High Latency / Lag

Symptoms: Delayed responses, out-of-sync audio/video

Common Causes & Solutions:

Network latency: Check internet connection speed
Server distance: Use CDN or edge servers
Device performance: Reduce quality on lower-end devices
Browser optimization: Use hardware acceleration when available

// Monitor and optimize for latency
setInterval(async () => {
  const stats = await anamClient.getStreamStats();
  if (stats.latency > 300) { // 300ms threshold
    console.warn('High latency detected:', stats.latency);
    await anamClient.setStreamQuality('low'); // Reduce quality
  }
}, 5000);

Browser Compatibility

Supported Browsers

Chrome/Edge

Full Support - Best performance and feature compatibility

Firefox

Full Support - Excellent WebRTC implementation

Safari

Partial Support - Some limitations on iOS, works well on macOS

Mobile Browsers

Good Support - Optimized quality and features for mobile

Browser-Specific Optimizations

class BrowserOptimizer {
  constructor(anamClient) {
    this.client = anamClient;
    this.browser = this.detectBrowser();
  }

  detectBrowser() {
    const userAgent = navigator.userAgent;
    if (userAgent.includes('Chrome')) return 'chrome';
    if (userAgent.includes('Firefox')) return 'firefox';
    if (userAgent.includes('Safari')) return 'safari';
    if (userAgent.includes('Edge')) return 'edge';
    return 'unknown';
  }

  async applyOptimizations() {
    switch (this.browser) {
      case 'safari':
        // Safari-specific optimizations
        await this.client.setConfig({
          preferH264: true, // Safari prefers H.264
          reducedFrameRate: true // Better performance on Safari
        });
        break;
        
      case 'firefox':
        // Firefox optimizations
        await this.client.setConfig({
          preferVP9: true, // Firefox handles VP9 well
          enableHardwareAcceleration: true
        });
        break;
        
      case 'chrome':
      case 'edge':
        // Chromium-based optimizations
        await this.client.setConfig({
          enableAdvancedCodecs: true,
          preferAV1: true // Latest Chrome supports AV1
        });
        break;
    }
  }
}

Next Steps

Event Handling

Learn about all streaming and conversation events

Performance Best Practices

Optimize streaming performance for production

Troubleshooting Guide

Solve common streaming and connection issues

Production Deployment

Deploy streaming personas at scale

Getting Started

Core Concepts

Examples

SDK Reference

Resources

Real-time Streaming

How Streaming Works

Stream Architecture

Basic Streaming Setup

Simple Video Streaming

Advanced Streaming Configuration

Stream Quality Management

Adaptive Quality

Quality Settings

Ultra Quality

High Quality

Medium Quality

Low Quality

Audio Management

Audio Controls

Performance Optimization

Connection Optimization

Bandwidth Management

Handling Stream Events

Connection Lifecycle

Common Streaming Issues

Browser Compatibility

Supported Browsers

Chrome/Edge

Firefox

Safari

Mobile Browsers

Browser-Specific Optimizations

Next Steps

Event Handling

Performance Best Practices

Troubleshooting Guide

Production Deployment

Getting Started

Core Concepts

Examples

SDK Reference

Resources

​How Streaming Works

​Stream Architecture

​Basic Streaming Setup

​Simple Video Streaming

​Advanced Streaming Configuration

​Stream Quality Management

​Adaptive Quality

​Quality Settings

Ultra Quality

High Quality

Medium Quality

Low Quality

​Audio Management

​Audio Controls

​Performance Optimization

​Connection Optimization

​Bandwidth Management

​Handling Stream Events

​Connection Lifecycle

​Common Streaming Issues

​Browser Compatibility

​Supported Browsers

Chrome/Edge

Firefox

Safari

Mobile Browsers

​Browser-Specific Optimizations

​Next Steps

Event Handling

Performance Best Practices

Troubleshooting Guide

Production Deployment

How Streaming Works

Stream Architecture

Basic Streaming Setup

Simple Video Streaming

Advanced Streaming Configuration

Stream Quality Management

Adaptive Quality

Quality Settings

Audio Management

Audio Controls

Performance Optimization

Connection Optimization

Bandwidth Management

Handling Stream Events

Connection Lifecycle

Common Streaming Issues

Browser Compatibility

Supported Browsers

Browser-Specific Optimizations

Next Steps