Bengaluru, India · 5+ years experience

Partha Mehta

Cloud/AI Data Engineer · Gen/Agentic AI · AWS SA/CCP · TensorFlow Cert
Ex-Cisco Talos · Ex-PwC · Intern at Amazon · Python · Go · Scala · LLMs · RAG · MCP

LinkedIn → GitHub Blog Email
About

5+ years. 8 companies. One constant — shipping.

I'm a results-driven Cloud/AI Data Engineer with 5+ years of industry experience, having contributed to the Cisco Talos Threat Analytics Platform Core Dev Team. I specialize in Python, Go, PySpark, and Scala, with a proven track record of designing and scaling multi-terabyte data pipelines, distributed systems, and cloud-native services across AWS, Azure, Terraform, and Databricks.

Proficient in Gen/Agentic AI, LLMs, RAG, MCP, OpenAI APIs, and Claude Code. My expertise spans data engineering, platform reliability, and AI — delivering high-availability microservices, CLI tooling, and observability solutions in Go, alongside Spark/Databricks pipelines that power large-scale cybersecurity analytics.

Previously, I worked at PwC SDC and completed internships at Amazon, where I built a strong foundation in data engineering, distributed computing, and scalable infrastructure.

🏢
Current Role
Data Engineer — Globus Systems
📍
Location
Bengaluru, Karnataka, India
🎓
Education
M.S. Data Science · NJIT · 2023
🧠
Specialization
LLMs · RAG · MCP · Agentic AI · Data Pipelines
🌐
Languages
English · Hindi
Experience

Where I've worked

Data Engineer — ETL Development
Globus Systems
Jan 2026 – Present
  • Built SQL-centric ETL pipelines for non-profit fundraising systems, loading constituent, gift, and marketing data into DMI/System5
  • Architected end-to-end donor data pipeline (DMI/System5 → DBT → Python → HubSpot), reducing data latency from hours to near real-time
  • Designed modular DBT models with hash-based change detection, reducing redundant processing and API calls by ~60%
  • Engineered robust retry and rate-limiting mechanisms (HTTP 429 backoff), achieving >99% API success rate
  • Applied agent-oriented prompt engineering and context modeling for AI agents (Claude, IDE copilots)
SQLDBTPythonHubSpotETLClaude
Cloud Engineer — Talos Intelligence Security
Cisco
Oct 2023 – Oct 2025
  • Built and maintained multi-terabyte ETL pipelines using Python, PySpark, and Databricks for large-scale threat intelligence ingestion and analytics
  • Designed high-availability microservices in Go deployed as AWS Lambdas/ECS tasks, powering real-time ingestion pipelines across multi-region environments
  • Implemented SCD Type 2 pipeline achieving 70%+ storage reduction through de-duplication and S3 retention optimization
  • Leveraged Go concurrency to parallelize data processing, reducing pipeline latency by 40%+
  • Deployed Spark jobs, Go services, Lambdas, and Step Functions across multi-region AWS using Terraform
  • Designed and deployed API MCP Servers for real-time hash, IP, and domain reputation services
  • Developed full-stack AI-powered chatbot using Streamlit + RAG + LangChain
GoPythonPySparkDatabricksAWSTerraformDelta LakeClickHouseMCP
Data Scientist
SyntacTech
Jan 2023 – May 2023
  • Built ML models (XGBoost, Random Forest, SVM) and time-series models (Prophet, ARIMA) achieving <3% monthly error on revenue models
  • Designed data pipelines using Airflow, PySpark, Databricks, and SQL to process large datasets
  • Worked on NLP and document intelligence models with >95% precision/recall
  • Deployed ML microservices using Docker, Kubernetes, supporting both CPU/GPU workloads
PythonPySparkAirflowMLNLPDockerK8s
Data Engineer
GEP Worldwide
Jul 2022 – Dec 2022
  • Automated Data Ingestion & Parsing Guide generation using Azure Databricks and Data Factory
  • Cut down 47% of architecture's cash burn and achieved 32% revenue increase
  • Increased data processing efficiency by 40% through optimized code in Azure Databricks and Kafka
  • Revamped ETL process with automated framework, resulting in 25% increase in data accuracy
AzureDatabricksData FactoryKafkaPySparkPower BI
Data Scientist NLP — Summer Intern
Amazon
Jun 2022 – Jul 2022
  • Developed 5-year demand forecasting model — 20% reduction in excess inventory costs
  • Led AB test on customer engagement emails for Prime Video — 10% increase in user engagement
  • Enhanced search relevance using NLP and deep learning — 15% improvement in search rankings
  • Spearheaded supply chain optimization — 30% reduction in delivery timeframes
PythonNLPMLDeep LearningA/B TestingAWS
Graduate Research Data Scientist NLP Assistant
New Jersey Institute of Technology
Sep 2021 – Dec 2021
  • Worked on COVID misinformation spread analysis using DistillBERT and Graph Neural Networks
  • Proposed GCN-based Deep Learning model for accident severity prediction, outperforming standard NN models
  • Built end-to-end pipeline for knowledge-based text generation using NLP, AMR Graphs, and PyTorch
PythonNLPGNNBERTPyTorch
Technology Consultant / Data Engineer
PwC Acceleration Centers
Aug 2019 – Aug 2021
  • Built and optimized large-scale data and ML pipelines using PySpark, Scala, SQL across 10TB+ datasets
  • Developed streaming pipelines on AWS (Kinesis, Spark Streaming, DynamoDB) processing millions of events/day
  • Exposed ML systems via scalable Go-based REST/gRPC services, using Protobuf and concurrency patterns
  • Deployed systems on AWS with performance tuning, achieving 30% faster processing
  • Automated delivery with Docker, Kubernetes, Terraform, CI/CD (GitHub Actions/Jenkins)
GoPySparkScalaAWSKafkaDockerK8sTerraformGrafana
Data Engineer Intern
Medloc
Oct 2017 – Dec 2017
  • Developed strategy for cloud migration using AWS DMS and Server Migration Service
  • Set up AWS infrastructure using VPC, EC2, S3, DynamoDB, IAM, CloudFormation templates
  • Implemented Kubernetes with Docker for auto-scaling and continuous integration
AWSDockerK8sRedshiftCloudFormation
Machine Learning Engineer Intern
A & I Infotech
Sep 2016 – Dec 2016
  • Developed ML model to predict insurance premium pricing with 80% accuracy
  • Implemented unsupervised ML techniques for time series anomaly detection on 3TB unstructured data
  • Wrangled 10TB of trading data in Hadoop using Scala, enabling 500+ end clients to track trade impact
PythonScalaMLHadoopDeep Learning
Skills

Technical toolkit

Languages
Python · Go · Scala
Shell Scripting · Java · SQL
Data Engineering
Apache Spark · PySpark
Databricks · Delta Lake
Airflow · DBT · Kafka
HiveQL · SparkSQL
Cloud & Infra
AWS (S3, EMR, Glue, Lambda,
Step Functions, ECS, EKS,
Redshift, SageMaker, Bedrock)
Azure · Terraform
DevOps
Docker · Kubernetes
GitHub Actions · CI/CD
Git · Helm
AI & ML
TensorFlow · PyTorch
LangChain · LangGraph
RAG · MCP · LLMs
Agentic AI · HuggingFace
Observability
Prometheus · Grafana
PagerDuty · CloudWatch
Structured Logging · Tracing
Databases
MySQL · DynamoDB
MongoDB · Redshift
ClickHouse · Delta Lake
Frameworks & Tools
Flask · Django · FastAPI
gRPC · REST APIs · Swagger
Tableau · Power BI
Jira · Confluence · Agile
Key Projects

Highlight projects

🤖
GitHub
AI-Powered Chatbot — Cisco Talos
Cisco Internal · 2024

Full-stack internal AI-powered chatbot built with Python (Streamlit + backend APIs + RAG + LangChain, vector stores) supporting AI/LLM initiatives at Cisco Talos. Designed and deployed API MCP Servers for 2 data pipelines to deliver real-time hash, IP, and domain reputation services, with bulk support, timestamp filtering, and integration into Claude/Amazon Q/VSC clients.

PythonStreamlitRAG LangChainMCPVector Stores
📊
Donor Data Pipeline — DMI to HubSpot
Globus Systems · 2026

Architected end-to-end donor data pipeline (DMI/System5 → DBT → Python → HubSpot) enabling hourly synchronization and reducing data latency from hours to near real-time. Built Python-based incremental sync service with idempotent upsert logic, ensuring zero-duplicate contact creation.

DBTPythonHubSpot API SQLETLIncremental Sync
Certifications

Credentials

AWS Solutions Architect Associate
Amazon Web Services
AWS Certified Cloud Practitioner
Amazon Web Services
TensorFlow Developer Certificate
Google
Core Java Certification
Certification
Education

Academic background

🎓
Master's Degree in Data Science
New Jersey Institute of Technology
Aug 2021 – May 2023
🏛️
Bachelor of Engineering in Computer Science
Nitte Institute of Technology, Bangalore
Aug 2015 – Apr 2019
Publication

Research

📝
Smart College Camera Security System Using IOT
Published Research

Published research paper on an IoT-based smart surveillance system designed for college campus security, leveraging embedded systems and computer vision for real-time monitoring and alert generation.

IoTComputer VisionEmbedded SystemsSecurity
Contact

Let's connect

Open to conversations about data engineering, cloud infrastructure, Agentic AI, LLMs, or collaboration opportunities.

Send an email → LinkedIn GitHub Medium