Responsibilities:

• Audit Trail & Version History

  • Centralized Audit Trail Mechanism - Implemented a centralized Audit Trail mechanism using Database Triggers to automatically and consistently log every data change, enabling reliable record version restoration.
  • Performance & Retention Management - Designed the Audit system for high-performance and managed a Retention Policy to clean up old versions based on defined criteria (e.g., number of latest versions or maximum age).
  • JSON Change Identification - Developed a JSONBDEEPDIFF function to accurately identify changes, including in nested JSON values, and present them in a readable format (Path, Old Value, New Value).

• Kafka Backup & Replay System

  • Kafka Backup Service - Developed a Kafka Backup Service that reads events, temporarily stores them in dedicated Buffers, and efficiently writes them to the Database while maintaining event order and using Checkpoints for failure recovery.
  • Managed Scheduler Service - Designed and developed a Scheduler Service for automated background task management, featuring Cron-based configuration and handling of Task Dependencies.
  • Controlled Replay System - Built a comprehensive Replay Control System for reprocessing and resending historical events to parallel Kafka queues, featuring real-time monitoring via WebSockets. The system utilizes a Priority Semaphore for task prioritization, Cancellation Tokens for safe cancel-in-flight functionality, and Checkpointing to ensure reliable recovery after failure or cancellation.
  • Range Splitter Algorithm - Implemented the RangeSplitter algorithm for optimization by resolving overlapping ranges ('Last-Wins' logic), ensuring every Timestamp is processed exactly once by the most recent source.
  • Event Order Synchronization in Replay - Researched and adopted a Synchronization mechanism to resolve event order challenges in the Replay process (e.g., 'Timestamp Ties' and 'Clock Skew'), ensuring deterministic processing.