AS Aniket Surve
Aniket Surve

Tech Lead (Big Data) · Jersey City, NJ

Hi, I'm Aniket.

I'm a data engineer with 10+ years building scalable, cloud-native data pipelines end to end — from ingestion to analytics — using Python, Apache Spark, Databricks, SQL, and Hive on AWS and Azure.

Read the blog Try the apps

About

I lead data engineering efforts building ingestion pipelines that process millions of records daily, designing schemas and partitioning strategies for efficient analytics, and automating deployments across cloud and on-prem environments. I care about translating messy business requirements into reliable, well-tested data infrastructure — and about automating away the manual toil around it.

Python Apache Spark Databricks Hive Kafka SQL AWS Azure Docker

Experience

Tech Lead Jun 2022 – Present
Synechron · Jersey City, USA
  • Led a team of 5 building ingestion pipelines processing millions of records daily across multiple source systems
  • Designed Hive table schemas and partitioning strategy for efficient analytics querying
  • Automated deployments with TeamCity, UDeploy, and Harness/OpenShift, cutting deployment time from hours to minutes
Application Development Senior Analyst Aug 2019 – Jun 2022
Accenture · Pune, India
  • Built Databricks pipelines ingesting data from S3, RDS, and other sources
  • Developed Spark Structured Streaming jobs for real-time ingestion from 5 Kafka topics
  • Optimized SQL query performance, reducing runtime by up to 80%
Software Engineer Aug 2016 – Jul 2019
Tech Mahindra · Pune, India
  • Built a Python log-analyzer tool that automated manual log inspection, cutting triage time by up to 50%
  • Automated routine server maintenance with Python scripts
  • Integrated SOAP and REST APIs for data exchange with external systems

Education

Master of Science, Computer Science May 2023
Liverpool John Moores University, Liverpool
Post Graduate Diploma, Big Data Feb 2022
Indian Institutes of Information Technology, Bangalore
Bachelor of Engineering, Information Technology Jul 2016
Savitribai Phule Pune University, Lonavala

Explore

Blog

Notes and write-ups, stored in DynamoDB and served through the API.

Guestbook

Leave a note — say hi, ask a question, whatever you like.

Apps

Small tools and experiments, running entirely in your browser.