Overview - Naxodev/mar

naxodev/mar

7 days30 days

Latest CI Pipeline Executions

Fuzzy

Succeeded
main
467d7eb7 fix: zero executor not found
5 days ago by NachoVazquez
Succeeded
naxo/mae-52-extract-vessel-management-subdomain
e43f4639 feat: integrate with the client v1 and react
6 days ago by NachoVazquez
Succeeded
naxo/mae-51-extract-connection-management-subdomain
b0f22849 fix: type issues
6 days ago by NachoVazquez
Failed
naxo/mae-51-extract-connection-management-subdomain
Fix ready→
a180cdc7 refactor: more state renamed to status
6 days ago by NachoVazquez
Succeeded
naxo/mae-51-extract-connection-management-subdomain
0dc889b8 docs: add jsdocs to connection manager
7 days ago by NachoVazquez
Succeeded
naxo/mae-51-extract-connection-management-subdomain
387cf172 feat: remove the setupFn and cleanupFn
7 days ago by NachoVazquez
Succeeded
main
a4b09c2a refactor: create message processor event loop in client v2 (#11) ## PR Type What kind of change does this PR introduce?  - [ ] Bugfix - [x] Feature - [ ] Code style update (formatting, local variables) - [ ] Refactoring (no functional changes, no api changes) - [ ] Build related changes - [ ] CI related changes - [ ] Documentation content changes - [ ] Other... Please describe: ## What is the current behavior?  **Client Architecture Issues:** - The orchestrator was a monolithic module with all logic in a single file - Legacy action execution infrastructure (runner-registry, uow) was difficult to maintain - Message processing lacked proper state management and type safety - No separation between voice detection and orchestration concerns **Voice Orb:** - Audio level visualization was based on FFT frequency data (`AnalyserNode.getByteFrequencyData()`) - Responded to any sound including TTS playback, background noise, and non-speech audio - No client-side Voice Activity Detection (VAD) Issue Number: N/A ## What is the new behavior? ### Client v2 Architecture **New Package: `packages/maelstrom/client-v2/`** - `MessageProcessor` - Modern message processing with event listeners and state management - `ActionManager` - Full action lifecycle management with waiting capabilities - `LayoutManager` - View/layout state management - Comprehensive test suite with 400+ tests **Orchestrator Refactor:** - Split monolithic orchestrator into modular components: - `engine-communication/` - Engine processor, lifecycle, update queue - `orchestrator/` - Main orchestration logic with API contract tests - `registry/` - Vessel registry management - `state-management/` - Store manager, event bus, typed store - `vad/` - Voice Activity Detection (Silero VAD integration) - `types/` - Comprehensive type definitions (split from single file) **Strict Type System:** - `ActionRunner<V>` with full parameter inference from vessel definitions - `defineVessel()` helper to preserve literal types - Support for partial action subsets per view - Type-safe action runners in React hooks ### Voice Activity Detection (VAD) **New VAD Module (`packages/maelstrom/react/src/lib/vad/`):** - Uses **Silero VAD v5 + ONNX Runtime Web (WASM)** - `silero-vad.ts` - Low-level ONNX Runtime wrapper - `vad-processor.ts` - Speech segment detection with configurable thresholds - `use-vad.ts` - React hook for VAD integration - Processes 512-sample windows (32ms @ 16kHz) **Integration:** - VAD probability drives voice orb visualization (0-1 scale) - Playback mode raises threshold to 0.9 during TTS for barge-in support - Browser Acoustic Echo Cancellation (AEC) prevents feedback **Assets:** - Silero VAD model: `apps/mar/public/models/silero_vad.v5.onnx` (2.3MB) - ONNX Runtime WASM loaded from CDN ## Does this PR introduce a breaking change? - [x] Yes - [ ] No  **Breaking Changes:** - `registerRunners()` changed to `registerRunner()` with object-based API - `ActionRunner` now requires Vessel type parameter for full inference - Old `message-queue.ts` removed (use `MessageProcessor` from client-v2) - Legacy action execution infrastructure removed **Migration:** - Update vessel definitions to use `defineVessel()` helper - Replace `registerRunners()` calls with `registerRunner()` - Update React components to use new typed `useVesselActions` hook ## Other information **Stats:** 111 files changed, +7107/-3835 lines (net +3272 lines) **Commits:** 22 commits including: - VAD implementation with Silero - Orchestrator refactor into subdomains - Client-v2 with MessageProcessor and ActionManager - Strict ActionRunner type inference - Full integration between client v1 and v2 **New Dependencies:** - `onnxruntime-web` - ONNX Runtime for browser-based VAD **Key Architectural Improvements:** - Modular client architecture with clear separation of concerns - Type-safe action system with full inference - Comprehensive test coverage (400+ tests in client-v2) - Client-side VAD for better voice UX --------- Co-authored-by: nx-cloud[bot] <71083854+nx-cloud[bot]@users.noreply.github.com> Co-authored-by: NachoVazquez <NachoVazquez@users.noreply.github.com>
7 days ago by NachoVazquez
Failed
naxo/mae-51-extract-connection-management-subdomain
Fix ready→
62d92f18 feat: remove the setupFn and cleanupFn
7 days ago by NachoVazquez
Succeeded
naxo/mae-46-rewrite-message-processing-following-the-uow-unit-of-work
ae653ae7 refactor: rename the state change method to status_changed
7 days ago by NachoVazquez
Succeeded
naxo/mae-51-extract-connection-management-subdomain
ed540ca6 feat: remove the setupFn and cleanupFn
9 days ago by NachoVazquez
Succeeded
naxo/mae-46-rewrite-message-processing-following-the-uow-unit-of-work
9f52fff1 fix: rebase issues
9 days ago by NachoVazquez
Failed
naxo/mae-51-extract-connection-management-subdomain
Environment issue→
d44935f1 fix: type issues
9 days ago by NachoVazquez
Failed
naxo/mae-46-rewrite-message-processing-following-the-uow-unit-of-work
Fix ready→
c7a9b9b8 fix: type issues
9 days ago by NachoVazquez
Succeeded
main
96d68173 chore: lock zero version
9 days ago by NachoVazquez
Succeeded
main
57e88171 feat: implement effective barge-in for voice interruption (#14) Description Implement client and server-side barge-in to enable users to interrupt TTS playback by speaking. ### Changes - **Client**: Re-chunk incoming TTS into 512-sample chunks (~32ms) for responsive interruption - **Client**: Add bargeIn handler to stop playback when VAD detects speech - **Server**: Handle vad.speech_start by interrupting TTS provider if currently speaking ### Result Users can now interrupt AI speech with sub-50ms latency, providing a natural conversational experience.
9 days ago by NachoVazquez
Succeeded
naxo/mae-51-extract-connection-management-subdomain
dfb4958b feat: remove the setupFn and cleanupFn
9 days ago by NachoVazquez
Succeeded
naxo/mae-46-rewrite-message-processing-following-the-uow-unit-of-work
ed422a2e fix: type issues
10 days ago by NachoVazquez
Succeeded
naxo/mae-51-extract-connection-management-subdomain
c1b5730e fix: type issues
10 days ago by NachoVazquez
Succeeded
naxo/mae-51-extract-connection-management-subdomain
c1b5730e fix: type issues
10 days ago by NachoVazquez
Succeeded
naxo/mae-51-extract-connection-management-subdomain
e323fba0 refactor: move engine connector to the protocol package
10 days ago by NachoVazquez