Overview
Distinguished Data Engineer and Director at Capital One — an elite technical leadership role held by fewer than 0.5% of associates. 19+ years of experience architecting modern cloud-native data platforms that power intelligent products, decisioning systems, and analytics at enterprise scale.
M.Tech from the Indian Institute of Technology (IIT) Kharagpur (GPA: 9.11/10). Fellow of the British Computer Society. Senior Member, IEEE. Fellow, IETE. Forbes Technology Council Member.
Career
Capital One | Distinguished Data Engineer & Director | 2022–Present Leading data architecture across Capital One’s most critical platforms — from Enterprise Data Tech to Card Tech. Architected the ML Zone for production-scale model training, drove $10M+ in annual cloud cost savings, and led architecture of the enterprise-scale data lake (300+ PB) and multi-tenant cloud warehouse (50+ PB). Represents Card Tech in Capital One’s Distinguished Engineer community, shaping enterprise-wide platform direction.
EY | Senior Manager, Data & Analytics, Financial Services | 2019–2022 Led high-impact data architecture engagements across Banking, Capital Markets, Wealth Management, and Insurance. Architected cloud-based enterprise data lakes and analytics hubs on AWS and Snowflake for major U.S. banks. Designed a graph-based resiliency framework for a global investment bank using Neo4j. Led architecture reviews presented at board level.
IBM | Sr. Managing Consultant, Big Data & Analytics | 2015–2019 Architected hybrid cloud data lakes integrating 100+ source systems and 5B+ daily records for a major U.S. insurer. Designed a Kafka-based transport platform handling 4B+ daily events. Built Spark/MapReduce pipelines supporting 10,000+ daily workloads. Deployed predictive models that automated 1.5M+ IVR actions and reduced call volumes.
Persistent Systems | Software Engineer → Architect | 2007–2015 Led architecture of next-gen hybrid data lake platforms in partnership with IBM. Ported Apache Hadoop to Windows, enabling the launch of Microsoft HDInsight on Azure. Engineered Microsoft’s ODBC Driver for Apache Hive. Built bi-directional Sqoop connectors for SQL Server / PDW integration. Built Persistent’s Big Data Analytics Library (PEBAL).
NVIDIA | Software QA Engineer Trainee | 2006–2007 Quality assurance and validation of GPU device drivers and media decoders. DirectX and OpenGL compliance testing across hardware configurations and rendering pipelines.
Research & Publications
Five peer-reviewed journal articles published in 2025, with work spanning data-centric AI, composable architectures, human-in-the-loop systems, and the ethics of data engineering.
Peer-Reviewed Journal Articles (2025):
- Making Banking Platforms AI-Ready: The Data Engineering Foundation — International Journal of Computing and Engineering, Vol. 7, Issue 7, pp. 25-38
- Human-in-the-Loop AI in Financial Services: Data Engineering That Enables Judgment at Scale — Journal of Computer Science and Technology Studies, Vol. 7, Issue 7, pp. 228-236
- Composable Data Architectures: Moving Beyond the Monolithic Platform — Sarcouncil Journal of Engineering and Computer Sciences, pp. 232-240
- Data-Centric AI: Engineering Platforms for Pre-Model Intelligence — Sarcouncil Journal of Multidisciplinary, pp. 48-54
- The Invisible Hand of Data Engineering: How Platform Decisions Impact People’s Lives — European Modern Studies Journal
Conference Paper:
- Displacement Based Unsupervised Metric for Evaluating Rank Aggregation — Springer PReMI 2011. Co-authored with M.S. Desarkar and S. Sarkar, IIT Kharagpur.
Capital One Tech Blog:
- Delta Lake Transaction Logs Explained
- Lakehouse Convergence: Delta Lake & Iceberg
- Understanding the Evolution of Data Lakes
LinkedIn Articles — Databricks Data + AI Summit 2025:
- The Future of Data Platforms is Federated, Governed, and AI Ready — Day 1
- The Data Intelligence Platform Goes Operational, App-Ready, and Agent-Aware — Day 2
- The Lakehouse Is Evolving: Spark Declarative Pipelines, Efficient Interoperable Tables, and a Unified User Experience — Day 3
Professional Memberships
- Fellow, British Computer Society (BCS)
- Fellow, Institution of Electronics and Telecommunication Engineers (IETE)
- Senior Member, IEEE
- Member, Forbes Technology Council
- Full Member, Sigma Xi — The Scientific Research Honor Society
Awards & Recognition
- Peer Reviewer, IEEE Conferences
- Judge, Globee Awards for Technology (2025)
- Service Excellence Award, IBM (2016)
- Eminence and Excellence Award, IBM (2015)
- Star Performer, Persistent Systems (2012)
- Best Outgoing Student, Computer Engineering Department (2006)
Certifications
- AWS Certified Solutions Architect – Associate
- AWS Certified Cloud Practitioner
- SnowPro Core Certification (Snowflake)
- Enterprise Design Thinking Practitioner, IBM
Education
Indian Institute of Technology (IIT) Kharagpur | M.Tech, Computer Science & Engineering | 2009–2011 GPA: 9.11/10. Research in information retrieval, machine learning, and rank aggregation.
Pune University | B.E., Computer Engineering | 2002–2006 First Class with Distinction. Best Outgoing Student.
Connect
- GitHub
- Email: visit2rahul@gmail.com