Hi, I'm Ashish Vishwakarma.I build data products that help businesses make smart decisions._

Senior Data Engineer, Global Payments

Jan 2026 - Present  ·  San Jose, CA

Continuation of the Global Payments data engineering role below. The same team and responsibilities transferred into TikTok U.S. Data Security (USDS), the U.S. data-security joint venture, in January 2026.

Senior Data Engineer, Global Payments

Dec 2024 - Jan 2026  ·  San Jose, CA

• Built and own the single source of truth for Refund and Payout data across TikTok Shop, TikTok Ads, and other non-TikTok apps, consolidating fragmented payment flows into trusted, business-critical datasets used by analytics, product, and operations teams.
• Designed payments data models and pipelines that power reporting, AI/LLM data preparation, PSR reporting, and campaign recap analytics across the Global Payments domain.
• Optimized high-priority task execution by migrating critical Dorado workloads to a dedicated YARN queue, reducing resource contention and improving reliability for time-sensitive pipelines.
• Implemented an SLA monitoring and alerting framework for core payment datasets using Aeolus and internal metadata, shifting reliability from reactive incident response to proactive monitoring.
• Partner with upstream e-commerce, product, and data platform teams to improve trust in payments datasets across US-TTP, ROW, and related regional flows.

Cloud Software Engineer

Nov 2021 - Oct 2024  ·  Folsom, CA

• Designed and implemented Intel's foundational telemetry data model, scalable to 100+ GB of daily data, serving as the base layer for two analytics data marts and Superset dashboards.
• Built end-to-end AWS data pipelines following medallion architecture across ingestion, transformation, curated modeling, and analytics consumption.
• Designed hyperscaler customer pipelines using AWS Glue, Lambda, Kinesis Data Streams, Step Functions, S3, and Redshift to process 100+ packages concurrently.
• Led AI dataset reconciliation pipelines for pre-silicon data, collaborating with teams in China, Poland, and India to curate datasets for data science using PySpark and Databricks.
• Processed unstructured PDF/HTML data with Unstructured.io to build high-quality datasets for LLM fine-tuning, an approach adopted by multiple Intel teams.
• Led migration from legacy Hive tables to Apache Iceberg, enabling lakehouse capabilities: time travel, faster queries, schema evolution, and reliable historical data management.
• Led design and code reviews focused on AWS cost optimization and Spark/Python performance tuning across Glue jobs, RDS workloads, and Kinesis streams.

Data Engineer Tech Lead

May 2021 - Oct 2021  ·  Hartford, CT

• Led a team of 5 data engineers migrating 3 big data projects from on-prem Hadoop to AWS, improving scalability and cloud readiness for insurance analytics workloads.
• Designed and built an advanced analytics pipeline processing 10+ GB of business insurance data using PySpark on Databricks, AWS, and Snowflake.
• Directed a SAS-to-PySpark migration for legacy analytics with 1:1 validation, improving processing efficiency 10x.
• Optimized critical PySpark pipelines for international insurance reporting, cutting runtime from 3 hours to 20 minutes through Spark tuning and refactoring.

Senior Data Engineer

Aug 2019 - May 2021  ·  Hartford, CT

• Built and maintained production pipelines ingesting from Teradata, SQL Server, and Oracle using Sqoop and PySpark on YARN for pre-issuance insurance analytics.
• Reduced PySpark job runtime from 2 hours to 15 minutes by optimizing executor configuration, file distribution, partitioning, and parallelism.
• Developed reusable PySpark utilities for parsing complex XML and JSON, improving consistency and speed of semi-structured ingestion.
• Partnered with technical architects and business teams to improve data interfaces, pipeline performance, and reliability of analytics-ready data.

Senior Software Engineer

Dec 2016 - Jul 2019  ·  Bangalore, India

• Created big data proofs of concept using Hadoop, Hive, Spark, and Sqoop to demonstrate scalable ingestion and transformation for telecom requirements.
• Re-engineered a critical ingestion pipeline processing 100+ GB of data, improving processing efficiency and reliability.
• Authored complex Hive SQL transformations with embedded business logic and optimized query patterns for large-scale telecom analysis.
• Used HBase and Apache Phoenix for NoSQL storage and fast-retrieval use cases.

Software Engineer / Associate Software Engineer

Nov 2013 - Nov 2016  ·  Bangalore, India

• Implemented Sqoop-based ingestion from legacy source systems into the data lake and built Oozie workflows to automate recurring Hadoop tasks.
• Built ETL with Revenue Assurance ETL tooling and automated UNIX shell scripts for duplicate detection, data validation, and operational alerting.
• Delivered Hadoop, Hive, and HBase training sessions to grow team capability.