CDC

Change Data Capture (CDC) Summary Purpose: Track and propagate data changes (insert, update, delete) from a source to downstream systems. Write-Ahead Log (WAL): In relational databases (e.g., PostgreSQL), all changes are logged before being applied. CDC reads from the WAL to capture changes. NoSQL: In systems like DynamoDB, item-level streams capture changes in a similar fashion. Pipeline Actions: Detect changes. Send to downstream systems. Examples: replicate to read replicas, sync to a search index (Elasticsearch), or forward to Kafka. Key Considerations: ...

Learning 🎬 Video Upload & Moderation System (TikTok-style) 1. Functional Requirements Creators can upload videos Users can view videos Reviewers can moderate videos Users get notified if content is banned 2. Non-Functional Requirements High availability (eventual consistency acceptable) High throughput for uploads and reads Low latency for video playback Scalable for viral (hot) content 3. High-Level Architecture Upload Flow Client → API Gateway → UploadService → S3 ↓ Kafka Read Flow User → CDN → S3 ...