Overview - Ghost

7 days30 days

Latest CI Pipeline Executions

Fuzzy

Failed
30472
06b12e5e [WIP, NO MERGE]: Added recurring job to sync automation runs to Tinybird ref https://linear.app/ghost/issue/NY-1559/ghost-backend-send-events-to-traffic-analytics-service-with-recurring This cannot be merged as-is because it includes database migrations, so it will need to be broken up further before review. - Replaces the boot.js proof of concept with a tinybird-sync service scheduled through the jobs service every five minutes, with a per-process random offset - Adds a tinybird_syncs table storing a per-table watermark (last_synced_updated_at), plus updated_at indexes on automation_runs and automation_run_steps so each run only scans recently changed rows - Sends only the columns the Tinybird materialized views read; member id, member email, and lock bookkeeping never leave MySQL - Reads rows in keyset-paginated batches and splits requests by UTF-8 byte size under Tinybird's Events API limit - Uses wait=true so Tinybird acknowledges committed rows, fails the job on quarantined rows or HTTP errors without advancing the watermark, and aborts stalled requests after a timeout - Holds back rows updated within the last minute so a still-open transaction cannot be skipped by the watermark - Syncs both tables concurrently and guards against overlapping runs - Adds a backgroundJobs.tinybirdSync config flag to disable the job - Reverts the WIP changes that read stats from Tinybird; that work belongs to NY-1560 - Covers the sync logic with unit tests against an in-memory SQLite database and a fake Tinybird Express server
by Troy Ciesco