Rebuilding a Kafka pipeline without downtime
Consumer groups made the migration dull, which is exactly what we wanted: new consumers joined the new topic, lag was compared per partition until it matched, then traffic was cut over during a maintenance window that nobody outside the team noticed.
We kept the old topic writable for eleven days. The dual-write layer was a small producer wrapper, four lines of configuration, not a new service. The lesson: the migration plan matters more than the tooling; every step had a measurement that said whether to continue.