https://yutongjin.com/tags/cdc/Recent content in CDC on Yutong JinHugoen-usThu, 09 Apr 2026 00:00:00 +0000https://yutongjin.com/posts/cdc/Thu, 09 Apr 2026 00:00:00 +0000https://yutongjin.com/posts/cdc/<h3 id="change-data-capture-cdc-summary">Change Data Capture (CDC) Summary</h3> <ul> <li> <p><strong>Purpose</strong>: Track and propagate data changes (insert, update, delete) from a source to downstream systems.</p> </li> <li> <p><strong>Write-Ahead Log (WAL)</strong>: In relational databases (e.g., PostgreSQL), all changes are logged before being applied. CDC reads from the WAL to capture changes.</p> </li> <li> <p><strong>NoSQL</strong>: In systems like DynamoDB, item-level streams capture changes in a similar fashion.</p> </li> <li> <p><strong>Pipeline Actions</strong>:</p> <ul> <li>Detect changes.</li> <li>Send to downstream systems.</li> <li>Examples: replicate to read replicas, sync to a search index (Elasticsearch), or forward to Kafka.</li> </ul> </li> <li> <p><strong>Key Considerations</strong>:</p>https://yutongjin.com/posts/49_learning/Thu, 09 Apr 2026 00:00:00 +0000https://yutongjin.com/posts/49_learning/<h3 id="learning">Learning</h3> <h1 id="-video-upload--moderation-system-tiktok-style">🎬 Video Upload &amp; Moderation System (TikTok-style)</h1> <h2 id="1-functional-requirements">1. Functional Requirements</h2> <ul> <li>Creators can upload videos</li> <li>Users can view videos</li> <li>Reviewers can moderate videos</li> <li>Users get notified if content is banned</li> </ul> <hr> <h2 id="2-non-functional-requirements">2. Non-Functional Requirements</h2> <ul> <li>High availability (eventual consistency acceptable)</li> <li>High throughput for uploads and reads</li> <li>Low latency for video playback</li> <li>Scalable for viral (hot) content</li> </ul> <hr> <h2 id="3-high-level-architecture">3. High-Level Architecture</h2> <h3 id="upload-flow">Upload Flow</h3> <p>Client → API Gateway → UploadService → S3<br> ↓<br> Kafka</p> <h3 id="read-flow">Read Flow</h3> <p>User → CDN → S3</p>