Run details

aiWorkspace

CI Pipeline Executions
- Current executionSucceeded
  docs-ai-memory-readme
- Runs and Agents
- TimelineBeta
- Configuration
- Self-Healing CI
- Project Graph
- Circular DependenciesEnterprise
- Resource Usage
- Analysis
- ConformanceEnterprise
Runs

Insights

Compare tasks
Analytics

feat(ai): add a generateVoice activity and refresh the ElevenLabs models (#1390) * feat(ai): add generateVoice activity Voice creation gets a first-class activity alongside generateSpeech. Providers create voices two ways - design one from a text description, or clone one from reference audio - and both return voice ids that generateSpeech() accepts as `voice`. ElevenLabs implements both modes, xAI custom voices and fal-hosted clone endpoints cover the clone half, so the contract is deliberately wider than any single provider's API. ElevenLabs' two-step design-then-create collapses inside its adapter rather than leaking into the activity. Voice events reuse the devtools speech bucket - same family, and the panel renders activity events generically by name. Refs #1389 * feat(ai-elevenlabs): refresh models and add the voice-design adapter Bumps @elevenlabs/elevenlabs-js 2.44 -> 2.68 and @elevenlabs/client 1.3 -> 1.25. The bump is a prerequisite, not housekeeping: 2.44 hard-typed the music request as modelId?: 'music_v1' and the speech-to-text request as a two-member union, so the new ids would not have type-checked. Models, against https://elevenlabs.io/docs/models: - TTS gains eleven_v3_conversational - Scribe gains scribe_v2_medical and scribe_v2_realtime - Music gains music_v2_5 and music_v2 - Voice design is a new family: eleven_ttv_v3, eleven_multilingual_ttv_v2 Deprecated ids stay accepted with @deprecated JSDoc so pinned callers survive the upgrade. The exception is eleven_text_to_sound_v1, which ElevenLabs dropped from the catalog and from the SDK's SfxModelId - keeping it would have meant a cast that lies about what the API takes. isElevenLabsMusicModel now matches the music_ prefix instead of the single music_v1 literal, so the dispatcher stops needing a branch per release. elevenlabsVoiceDesign() implements the new generateVoice activity. ElevenLabs' design-then-create two-step is hidden: pass a name to get a saved voice back, leave it off to audition previews. Reference audio is normalized to base64 in the adapter, and remote URLs are rejected rather than downloaded and inlined. Refs #1389 * test(e2e): cover voice design end to end Adds a voice-design feature to the matrix, a harness page, an /api/voice route, and the two specs. generateVoice() is Promise-based, so the page is a single fetch with no connection-adapter modes - the same shape as embedding. aimock 1.34 has no /v1/text-to-voice routes, so elevenlabsVoiceMount serves both halves. Like the BytePlus mounts it validates rather than drains: it 400s on a missing voice_description, an unexpected model_id, or a generated_voice_id that did not come from a preview it handed out. Without that, an adapter regression would still be answered 200 and the spec would pass green. Refs #1389 * docs(ai): document voice creation, plus a changeset Adds docs/media/voice-creation.md: why you would create a voice, designing one from a description, speaking with it, keeping it, and cloning from reference audio. Cross-linked from text-to-speech, the generations activity table, and the ElevenLabs adapter page. The ElevenLabs page gains a Voice Design section and a Models table, and its samples move off the ids ElevenLabs deprecated. Updates the ai-core/media-generation agent skill in the same change, since agents read it as authoritative and would otherwise keep generating code for an activity list that no longer matches the package. Fixes a switch-exhaustiveness lint error in client.ts by enumerating every generation kind, so a new kind has to declare the field that identifies its request body rather than silently inheriting `prompt`. Aligns examples/ts-react-chat on the bumped ElevenLabs SDK, which sherif flagged. Closes #1389 * ci: apply automated fixes * docs: correct the BytePlus row in the voice-creation provider table BytePlus does have voice cloning. Seed Speech Voice Replication trains a clone with POST /api/v3/tts/voice_clone and polls POST /api/v3/tts/get_voice. That is a different thing from the references array on /api/v3/tts/create, which attaches a clip to one synthesis call and persists nothing - reading only the synthesis adapter is what produced the wrong claim. Two facts an implementer needs: the speaker_id is bought in the console, so it is an input to training rather than a result, and training is async, so the adapter polls internally within the activity timeout / abortSignal. Refs #1389 * feat(ai): keep async voice training async A provider that trains a clone in the background now says so instead of the adapter hiding it behind a blocking poll. generateVoice() returns the voice with status: 'training', and getVoiceStatus({ adapter, voiceId }) polls it to 'ready' or 'failed'. getVoiceStatus is OPTIONAL on VoiceAdapter, unlike generateVideo's three-method job protocol. Video can require it because every video provider is async; voice is mixed - ElevenLabs and xAI hand back a finished voice in one call and have nothing to poll. Forcing them to write a stub that returns a fake 'ready' would be worse than leaving the method off, so calling getVoiceStatus() on a single-call adapter throws and the message says why. status is absent rather than 'ready' on the single-call path, so no existing adapter has to be touched and a missing status reads as usable. Voice events gain requestType ('create' | 'status') so a poll is distinguishable from a creation, plus trainingStatus on the completed event. Refs #1389 * ci: apply automated fixes * feat(ai): add listVoices for per-account voice catalogs generateVoice() hands back an id you are expected to store. listVoices() reads the catalog back when you did not, filtered by origin (premade, generated, cloned, professional). It hangs off TTSAdapter rather than VoiceAdapter: `voice` is a generateSpeech() option, so that is where the id gets consumed, and putting it on VoiceAdapter would make anyone using only elevenlabsSpeech construct a voice-design adapter just to enumerate. Optional, and deliberately not a new activity. Every activity under activities/ is a generation; a catalog query is provider metadata, which the repo already hangs off the adapter (VideoAdapter.availableDurations, snapDuration) rather than giving its own kind, events and client.ts entry. Only providers with a per-account catalog implement it. Where the voice list is fixed the package already exports a const - GEMINI_TTS_VOICES, OpenAI's voice union, BytePlus preset speaker ids - which is a type union rather than a network call, so calling listVoices() there throws and names the const. elevenlabsSpeech implements it against GET /v1/voices. ElevenLabs has no origin filter on that endpoint, so the adapter narrows in memory, and its famous / high_quality categories both read as professional. Refs #1389 * ci: apply automated fixes * docs: correct how async BytePlus voice training actually is Voice Enrollment (POST /api/v3/tts/voice_clone) returns a training status in the same response, and the documented example returns 2 (success) with the demo audio URL. A voice is usable at status 2 or 4, so the normal path is not a poll at all - the earlier wording claimed it was. Status 4 (activated) is reached by the FIRST synthesis call, which is also when the voice slot is billed. It is not a state training produces, which the skill now says so nobody writes a poll loop waiting for it. Source: docs.byteplus.com/en/docs/byteplusvoice/voicereplication-v3-voice-training, read with access rather than inferred. Refs #1389 * fix(ai): address review findings on the generateVoice PR Cut the async-training contract, which had no adapter behind it anywhere in the repo: `getVoiceStatus`, `VoiceStatusResult`, `VoiceStatusOptions`, the `'status'` request type and `trainingStatus` event fields, the polling docs and skill sections, and the four tests that only exercised a mock. The signature was also half-designed — no slot for `abortSignal`, `timeout`, or `middleware`, unlike `getVideoJobStatus` — so the first async provider would have reshaped it. `GeneratedVoice.status` stays, now required, because it is cheap and forward-compatible. Correctness: - Forward `abortSignal` to both ElevenLabs SDK calls. `raceWithAbort` rejected the caller while `textToVoice.create` kept running, so an aborted request could still persist and bill a voice. - Drop `scribe_v2_realtime` from the transcription list. It is a WebSocket model in SDK 2.68 (`wrapper/realtime/scribe.d.ts`); the adapter calls `speechToText.convert()`, whose `modelId` is `string`, so it type-checked and then failed against `POST /v1/speech-to-text`. - Throw instead of returning `voices: []` when the design call comes back with no previews, which reported success for a request that produced nothing. - Make `isElevenLabsMusicModel` check membership as well as the `music_` prefix; it narrowed to a closed union any `music_*` string could enter. - Pin `ELEVENLABS_AUDIO_MODELS` to `MusicModelId | SfxModelId` with `satisfies`, so an id the SDK drops fails the build, not the request. - Make `saved` required alongside `status`. Docs and comments that did not match the code: - `GEMINI_TTS_VOICES` is exported as `GeminiTTSVoices`, and OpenAI's is `OpenAITTSVoice`; three files named identifiers that cannot be imported. - `@deprecated` on a string literal inside an `as const` array produces no editor deprecation at all, so the "marked `@deprecated` in your editor" claim was false. Use plain comments and drop the claim. - "Exactly one of `prompt` / `referenceAudio`" was wrong in three directions; the rule is at least one, and ElevenLabs needs both for a reference. - ElevenLabs reference audio is a design reference weighted by `promptStrength`, not a clone: the SDK's clone API (`voices.ivc`) is unused. - There is no automated ElevenLabs model pipeline; the sync script excludes it. - Remove the xAI/BytePlus/fal behavioural claims, including a present-tense description of an adapter that does not exist. - `clip.buffer` is the whole backing store of a possibly-pooled Buffer; use `new Blob([clip])`. - Nothing renders `speechEvents`, so the devtools comment overstated it. Tests: - Add the missing `voice` row to the otel activity-name table. - Cover middleware start/usage/finish ordering, the result-transform hook, and error-vs-abort classification, none of which the activity had. - Replace an abort assertion that passed by object identity — and so would have passed with the plumbing deleted — with real abort and timeout cases. - Assert the streamed `generation:result` payload, not just chunk types. - Cover ElevenLabs option passthrough (including zero values), outputFormat derivation, Blob input, non-base64 data URLs, abort forwarding, and the empty-preview path. Also move the new e2e mock below `byteplusTTSMount`, whose long doc comment it had been inserted in front of. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

nx run @tanstack/ai-grok-build:build

Succeeded

4 CPU cores

read access token used

30fc1bcb1417

100%

Cache hits

9 of 9 tasks used cache.

Get faster results

Learn how to enable distribution.

Atomizer enabled

9 groups of tasks optimized.

© 2026 - Nx Cloud

Terms of Service Privacy Policy Changelog Status Docs Contact Nx Cloud Pricing Company @NxDevTools

aiWorkspace