SDK Swift

SDK ElevenAgents: implementa agenti vocali interattivi e personalizzati nelle tue applicazioni Swift.

Consulta il nostro progetto quickstart Swift completo per iniziare rapidamente con un esempio completo e funzionante.

Installazione

Aggiungi l’SDK Swift di ElevenLabs al tuo progetto usando Swift Package Manager:

Aggiungi la dipendenza del pacchetto

dependencies: [ .package(url: "https://github.com/elevenlabs/elevenlabs-swift-sdk.git",
from: "2.0.0") ]

Oppure usa Xcode:

Apri il tuo progetto in Xcode
Vai a File > Add Package Dependencies...
Inserisci l’URL del repository: https://github.com/elevenlabs/elevenlabs-swift-sdk.git
Seleziona la versione 2.0.0 o successiva

Importa l'SDK

import ElevenLabs

Assicurati di aggiungere NSMicrophoneUsageDescription al tuo Info.plist per spiegare agli utenti l’accesso al microfono. L’SDK richiede iOS 14.0+ / macOS 11.0+ e Swift 5.9+.

Guida rapida

Inizia una semplice conversazione in poche righe. Facoltativamente, ti consigliamo di passare i tuoi ID utente finali per associare le conversazioni ai tuoi utenti.

import ElevenLabs
// Start a conversation with your agent
let conversation = try await ElevenLabs.startConversation(
    agentId: "your-agent-id",
    userId: "your-end-user-id",
    config: ConversationConfig()
)
// Observe conversation state and messages
conversation.$state
    .sink { state in
        print("Connection state: \(state)")
    }
    .store(in: &cancellables)
conversation.$messages
    .sink { messages in
        for message in messages {
            print("\(message.role): \(message.content)")
        }
    }
    .store(in: &cancellables)
// Send messages and control the conversation
try await conversation.sendMessage("Hello!")
try await conversation.toggleMute()
await conversation.endConversation()

Autenticazione

Esistono due modi per autenticarsi e avviare una conversazione:

Agenti pubblici

Agenti privati

Per gli agenti pubblici, usa direttamente l’ID dell’agente:

let conversation = try await ElevenLabs.startConversation(
    agentId: "your-public-agent-id",
    config: ConversationConfig()
)

Funzionalità principali

Gestione reattiva delle conversazioni

L’SDK offre una moderna classe Conversation con proprietà @Published per aggiornamenti reattivi dell’interfaccia:

@MainActor
class ConversationManager: ObservableObject {
    @Published var conversation: Conversation?
    private var cancellables = Set<AnyCancellable>()
    func startConversation(agentId: String) async throws {
        let config = ConversationConfig(
            conversationOverrides: ConversationOverrides(textOnly: false)
        )
        conversation = try await ElevenLabs.startConversation(
            agentId: agentId,
            config: config
        )
        setupObservers()
    }
    private func setupObservers() {
        guard let conversation else { return }
        // Monitor connection state
        conversation.$state
            .sink { state in print("State: \(state)") }
            .store(in: &cancellables)
        // Monitor messages
        conversation.$messages
            .sink { messages in print("Messages: \(messages.count)") }
            .store(in: &cancellables)
    }
}

Modalità vocale e testo

// Voice conversation (default)
let voiceConfig = ConversationConfig(
    conversationOverrides: ConversationOverrides(textOnly: false)
)
// Text-only conversation
let textConfig = ConversationConfig(
    conversationOverrides: ConversationOverrides(textOnly: true)
)

Controlli audio

// Microphone control
try await conversation.toggleMute()
try await conversation.setMuted(true)
// Check microphone state
let isMuted = conversation.isMuted
// Access audio tracks for advanced use cases
let inputTrack = conversation.inputTrack
let agentAudioTrack = conversation.agentAudioTrack

Strumenti client

Gli strumenti client ti consentono di registrare funzioni personalizzate che il tuo agente IA può chiamare durante le conversazioni. Il nuovo SDK offre una gestione migliorata dei parametri e degli errori.

Gestione delle chiamate agli strumenti

Gestisci le chiamate agli strumenti dal tuo agente con supporto completo dei parametri:

private func handleToolCall(_ toolCall: ClientToolCallEvent) async {
    do {
        let parameters = try toolCall.getParameters()
        let result = await executeClientTool(
            name: toolCall.toolName,
            parameters: parameters
        )
        if toolCall.expectsResponse {
            try await conversation?.sendToolResult(
                for: toolCall.toolCallId,
                result: result
            )
        } else {
            conversation?.markToolCallCompleted(toolCall.toolCallId)
        }
    } catch {
        // Handle tool execution errors
        if toolCall.expectsResponse {
            try? await conversation?.sendToolResult(
                for: toolCall.toolCallId,
                result: ["error": error.localizedDescription],
                isError: true
            )
        }
    }
}
// Example tool implementation
func executeClientTool(name: String, parameters: [String: Any]) async -> [String: Any] {
    switch name {
    case "get_weather":
        guard let location = parameters["location"] as? String else {
            return ["error": "Missing location parameter"]
        }
        // Fetch weather data
        return ["temperature": "22°C", "condition": "Sunny"]
    case "send_email":
        guard let recipient = parameters["recipient"] as? String,
              let subject = parameters["subject"] as? String else {
            return ["error": "Missing required parameters"]
        }
        // Send email logic
        return ["status": "sent", "messageId": "12345"]
    default:
        return ["error": "Unknown tool: \(name)"]
    }
}

Ricorda di configurare il tuo agente con gli strumenti client nella UI di ElevenLabs. Consulta la documentazione degli strumenti client per le istruzioni di configurazione.

Gestione dello stato della connessione

Monitora lo stato della conversazione per gestire le diverse fasi di connessione:

conversation.$state
    .sink { state in
        switch state {
        case .idle:
            // Not connected
            break
        case .connecting:
            // Show connecting indicator
            break
        case .active(let callInfo):
            // Connected to agent: \(callInfo.agentId)
            break
        case .ended(let reason):
            // Handle disconnection: \(reason)
            break
        case .error(let error):
            // Handle error: \(error)
            break
        }
    }
    .store(in: &cancellables)

Monitoraggio dello stato dell’agente

Tieni traccia di quando l’agente ascolta o parla:

conversation.$agentState
    .sink { state in
        switch state {
        case .listening:
            // Agent is listening, show listening indicator
            break
        case .speaking:
            // Agent is speaking, show speaking indicator
            break
        }
    }
    .store(in: &cancellables)

Gestione dei messaggi

Invia messaggi di testo e monitora la conversazione:

// Send a text message
try await conversation.sendMessage("Hello, how can you help me today?")
// Monitor all messages in the conversation
conversation.$messages
    .sink { messages in
        for message in messages {
            switch message.role {
            case .user:
                print("User: \(message.content)")
            case .agent:
                print("Agent: \(message.content)")
            }
        }
    }
    .store(in: &cancellables)

Allineamento audio

Monitora i dati temporali a livello di carattere per una visualizzazione del testo sincronizzata:

// Using the callback
let config = ConversationConfig(
    onAudioAlignment: { alignment in
        // Character-level timing data
        for (index, char) in alignment.chars.enumerated() {
            let startMs = alignment.charStartTimesMs[index]
            let durationMs = alignment.charDurationsMs[index]
            print("'\(char)' at \(startMs)ms for \(durationMs)ms")
        }
    }
)
// Or observe the published property
conversation.$latestAudioAlignment
    .compactMap { $0 }
    .sink { alignment in
        // Handle alignment updates
    }
    .store(in: &cancellables)

Gestione delle sessioni

// End the conversation
await conversation.endConversation()
// Check if conversation is active
let isActive = conversation.state.isActive

Integrazione con SwiftUI

Ecco un esempio SwiftUI completo che usa il nuovo SDK:

import SwiftUI
import ElevenLabs
import Combine
struct ConversationView: View {
    @StateObject private var viewModel = ConversationViewModel()
    var body: some View {
        VStack(spacing: 20) {
            // Connection status
            Text(viewModel.connectionStatus)
                .font(.headline)
                .foregroundColor(viewModel.isConnected ? .green : .red)
            // Chat messages
            ScrollView {
                LazyVStack(alignment: .leading, spacing: 8) {
                    ForEach(viewModel.messages, id: \.id) { message in
                        MessageBubble(message: message)
                    }
                }
            }
            .frame(maxHeight: 400)
            // Controls
            HStack(spacing: 16) {
                Button(viewModel.isConnected ? "End" : "Start") {
                    Task {
                        if viewModel.isConnected {
                            await viewModel.endConversation()
                        } else {
                            await viewModel.startConversation()
                        }
                    }
                }
                .buttonStyle(.borderedProminent)
                Button(viewModel.isMuted ? "Unmute" : "Mute") {
                    Task { await viewModel.toggleMute() }
                }
                .buttonStyle(.bordered)
                .disabled(!viewModel.isConnected)
                Button("Send Message") {
                    Task { await viewModel.sendTestMessage() }
                }
                .buttonStyle(.bordered)
                .disabled(!viewModel.isConnected)
            }
            // Agent state indicator
            if viewModel.isConnected {
                HStack {
                    Circle()
                        .fill(viewModel.agentState == .speaking ? .blue : .gray)
                        .frame(width: 10, height: 10)
                    Text(viewModel.agentState == .speaking ? "Agent speaking" : "Agent listening")
                        .font(.caption)
                }
            }
        }
        .padding()
    }
}
struct MessageBubble: View {
    let message: Message
    var body: some View {
        HStack {
            if message.role == .user { Spacer() }
            VStack(alignment: .leading) {
                Text(message.role == .user ? "You" : "Agent")
                    .font(.caption)
                    .foregroundColor(.secondary)
                Text(message.content)
                    .padding()
                    .background(message.role == .user ? Color.blue : Color.gray.opacity(0.3))
                    .foregroundColor(message.role == .user ? .white : .primary)
                    .cornerRadius(12)
            }
            if message.role == .agent { Spacer() }
        }
    }
}
@MainActor
class ConversationViewModel: ObservableObject {
    @Published var messages: [Message] = []
    @Published var isConnected = false
    @Published var isMuted = false
    @Published var agentState: AgentState = .listening
    @Published var connectionStatus = "Disconnected"
    private var conversation: Conversation?
    private var cancellables = Set<AnyCancellable>()
    func startConversation() async {
        do {
            conversation = try await ElevenLabs.startConversation(
                agentId: "your-agent-id",
                config: ConversationConfig()
            )
            setupObservers()
        } catch {
            print("Failed to start conversation: \(error)")
            connectionStatus = "Failed to connect"
        }
    }
    func endConversation() async {
        await conversation?.endConversation()
        conversation = nil
        cancellables.removeAll()
    }
    func toggleMute() async {
        try? await conversation?.toggleMute()
    }
    func sendTestMessage() async {
        try? await conversation?.sendMessage("Hello from the app!")
    }
    private func setupObservers() {
        guard let conversation else { return }
        conversation.$messages
            .assign(to: &$messages)
        conversation.$state
            .map { state in
                switch state {
                case .idle: return "Disconnected"
                case .connecting: return "Connecting..."
                case .active: return "Connected"
                case .ended: return "Ended"
                case .error: return "Error"
                }
            }
            .assign(to: &$connectionStatus)
        conversation.$state
            .map { $0.isActive }
            .assign(to: &$isConnected)
        conversation.$isMuted
            .assign(to: &$isMuted)
        conversation.$agentState
            .assign(to: &$agentState)
    }
}