Vishal Tyagi

AI & Data Platform Engineer · Hyderabad, India

I build the data pipelines behind Runo's CRM.

Around 15M events a month, and a daily analytics run cut from 42 minutes to 3.8 — built end to end, from a reverse-engineered WhatsApp sync to the DuckDB pipeline behind it.

View selected workRésumé

CRM pipeline run time · Runo daily call analytics

Daily pipeline run time fell from 42 minutes to 3.8 minutes

Measured before and after replacing sequential full recomputes with DuckDB incremental manifest fingerprinting.

BeforeSequential full recompute

42.0min

AfterDuckDB + manifest fingerprinting

3.8min

Dotted rule marks the 5-minute daily SLA target — the run only clears it after the change. 91% faster.

Daily volume
10,000+ calls
Scored against
20 criteria
Run time
42m → 3.8m
Daily QA
Automated
15M+
events / month

Telemetry facade with sliding-window dedup and Hive-buffered disk queue.

10,000+
recordings / day

Daily CRM call recordings scored across 20 criteria in DuckDB pipeline.

90,000+
sheets processed

Unattended optical mark recognition (OMR) computer vision pipeline.

-91%
daily run time

DuckDB incremental loader: the daily run dropped from 42 min to 3.8 min.

Selected work

Systems built end to end

01 · Browser Extensions

WhatsApp Web CRM Sidecar

Sales reps at Runo work inside WhatsApp Web, where no official API exists to read a live personal session. Early prototypes interrupted reps by stealing focus from their active chat or stalling on undocumented bulk limits.

I reverse-engineered WhatsApp Web's minified Webpack module registry to resolve its internal chat and contact stores at runtime, then built history paging that reads past conversations directly from the store — without switching the screen the rep is looking at.

Result: a Manifest V3 extension that syncs chat history and contact identity into the CRM in the background, verified to leave the active chat unchanged across a full history page, with a fail-closed outbox for network retries.

Read the case study
Systems Architecture // Reverse-Engineered Browser Sidecar
0 Official API · 100% Background Sync

Capturing Live CRM Leads from Closed WhatsApp Web Sessions

WhatsApp provides no official API to read live personal web sessions. Here is how the Manifest V3 sidecar reads state directly from browser memory without interfering with active sales reps.

Layer 01Zero DOM Scraping

In-Memory Webpack Store Hook

Directly hooks into WhatsApp Web’s minified Webpack module registry at runtime. Resolves WAWebChatCollection and WAWebConnModel by internal module ID with structural duck-typing fallback.

Layer 02Zero Screen Flicker

Zero-UI Background History Pager

Calls WAWebChatLoadMessages.loadEarlierMsgs() directly against the in-memory data store. Verified byte-identical active chat state: historical conversations sync without switching the user's screen.

Layer 03Reliable CRM Matching

Privacy Identity Resolver (@lid to @c.us)

WhatsApp’s newer privacy layer masks actual phone numbers under randomized @lid identifiers. Resolves real contact numbers through the internal Contact model before dispatching to the CRM.

Layer 04No Wedged Syncs

Resilient Outbox & Batch Throttling

Enforces a strict 50-message payload ceiling on the backend CRM bulk endpoint. Failed network batches automatically re-enqueue in an isolated pendingOutbox queue rather than dropping data.

The Analytics Engine, Stage by Stage

How Runo's daily CRM exports transform into DuckDB columnar tables, ML churn alerts, and executive reports.

Select a pipeline stage
Stage 01 Architecture Breakdown

Raw Ingestion — Daily CRM export drops

Partition SLA: < 2.4s
The Objective

Daily compressed CRM call recordings and customer activity events are collected into date-partitioned drops.

Engine Mechanism

Ingests incremental daily archives instead of demanding resource-heavy streaming connections on legacy CRM backends.

Data Integrity Rule

Partial or missing days are recorded in an explicit gap manifest, so quiet days and missing data are never conflated.

Stage 02 Architecture Breakdown

DuckDB Engine — Incremental loader & manifest skip

Partition SLA: < 3.8s
The Objective

Zero-copy columnar loading into an embedded DuckDB database instance.

Engine Mechanism

Hashes source archive fingerprints against a processed manifest; already-loaded files are skipped in constant time.

Data Integrity Rule

Deterministic single-process handle guarantees serial execution consistency and instant recovery from interrupted runs.

Stage 03 Architecture Breakdown

Report Caching — 7+ cached report modules

Partition SLA: < 18s
The Objective

Generates company health metrics, cohort retention tables, activity heatmaps, and weekly trends.

Engine Mechanism

Each report module fingerprints its input row count and date range; steps with matching fingerprints skip recomputation.

Data Integrity Rule

Duration averages are strictly scoped to answered calls, eliminating artificial skew from unanswered calls.

Stage 04 Architecture Breakdown

Churn Scorer — Supervised ML model

Partition SLA: < 6.5s
The Objective

Evaluates each enterprise customer against 20 behavioral signals to predict cancellation probability.

Engine Mechanism

Employs SMOTE synthetic sampling and cost-sensitive loss weighting to combat severe real-world class imbalance (5% churn rate).

Data Integrity Rule

Produces calibrated probability distributions rather than binary labels, enabling tiered customer intervention.

Stage 05 Architecture Breakdown

Multi-Surface Sync — Automated executive delivery

Partition SLA: < 4.2s
The Objective

Dispatches finalized intelligence directly into Google Sheets, Cloud Firestore, Google Drive, and Apps Script.

Engine Mechanism

Pushes directly into surfaces where executive stakeholders already work, rather than demanding logins to an isolated portal.

Data Integrity Rule

Automated week-over-week alert triggers dispatch instant notifications when account usage drops beyond tolerance.

Production Systems

Architectural deep-dives into core platforms built for high volume, zero data loss, and strict SLA compliance.

-91% Latency · 10k calls/day

Runo CRM Analytics & Churn ML Pipeline

Distributed ETL / Production ML
The Problem

Daily call recordings required scoring across 20 criteria for sentiment, intent, and customer churn. Full sequential recomputations took ~42 minutes, risking daily reporting SLA deadlines and overloading database memory.

What I Built

Architected an incremental DuckDB columnar analytics engine with file-manifest skip logic and fingerprint caching for 7+ report modules, coupled to a supervised churn ML model with SMOTE synthetic resampling and cost-sensitive class weighting.

Measurable Result

Cut end-to-end execution time from 42 min to under 3.8 min (-91% latency reduction), achieving 100% automated daily QA across 10,000+ recordings with multi-surface export (Sheets, Firestore, Drive, Apps Script).

ML Evaluation // Churn Classifier & Feature Importance
AUC 0.68 → 0.89 (+31% High-Risk Recall)

Solving Extreme Class Imbalance: Catching 89% of Churning Accounts

Enterprise CRM datasets have ~95% retention vs 5% churn. Naive models achieve 95% accuracy while missing all churn. SMOTE resampling and cost-sensitive loss weighting restored true predictive power.

01. ROC Performance Liftscikit-learn · imbalanced-learn
1.00.50.0Production Model (0.89)Unweighted Base (0.68)
False Positive RateTrue Positive Rate (Sensitivity)
02. Predictive Feature WeightsLollipop Importance
Avg Answered Call Duration38%
Inactivity Window Duration26%
Contract Days to Expiry19%
Team Seat Drop Rate11%
Recurring Payment Failures6%

Takeaway: A steep drop in average answered call duration is the single strongest precursor to churn, carrying 38% of overall model weight.

Read In-Depth Architectural Specification→
15M+ events/mo · 0 Data Loss

High-Throughput Product Analytics Telemetry Facade

Event Stream / Telemetry Infra
The Problem

Telemetry calls were scattered across mobile and web clients without schema validation or retry resilience. Spotty mobile network dropouts caused event drops and corrupted subscription billing funnels.

What I Built

Architected a typed ProductAnalytics facade over Amplitude featuring sliding-window event deduplication (-14% noise reduction), a durable Hive-backed disk accumulator, and batched HTTP flushes.

Measurable Result

Transports ~15M steady-state events/month with zero data loss across mobile network reconnects. Executed full 45M event historical backfill runs without blowing client memory or crashing downstream endpoints.

Telemetry Funnel // Sliding-Window Dedup & Local Buffer
~15.0M events / month

Zero Event Loss Under Flaky Mobile Networks: The Telemetry Pipeline

How the ProductAnalytics facade filters millions of duplicate taps and guarantees delivery during spotty connectivity.

01Raw Telemetry Events
15,000,000 / mo100% Volume

Unfiltered user interactions, dialer state shifts, and CRM updates across mobile and web.

02Sliding-Window Dedup Filter
-2,100,000 / mo (-14%)14% Duplicate Noise Filtered

Filters rapid duplicate button taps, double-clicks, and network reconnect retry storms.

03Durable Local Disk Queue
12,900,000 persistedZero In-Flight Loss

Durable local memory buffer that persists unsent events during offline mode and app crashes.

04Chunked API Batch Dispatch
100% Delivery Rate<24ms Batch Overhead

Aggregates events into optimal HTTP batch payloads, cutting client battery overhead by 80%.

Read In-Depth Architectural Specification→
0 Official API · 100% Background Sync

WhatsApp Web CRM Sidecar

Reverse Engineering / Browser Systems
The Problem

Sales reps work primarily inside WhatsApp Web, where no official API exists to inspect active personal chats. Early prototypes interrupted reps by stealing active chat focus or freezing on undocumented bulk caps.

What I Built

Reverse-engineered WhatsApp Web's minified React/Webpack runtime at browser startup to resolve internal stores by module ID. Engineered store-level history paging (loadEarlierMsgs) that pages historical chats without disturbing the active screen.

Measurable Result

Delivered zero-intrusion real-time CRM capture in Chrome MV3 with fail-closed privacy resolution (@lid to @c.us), outbox retry queuing, and strict 50-message bulk batch throttling.

Systems Architecture // Reverse-Engineered Browser Sidecar
0 Official API · 100% Background Sync

Capturing Live CRM Leads from Closed WhatsApp Web Sessions

WhatsApp provides no official API to read live personal web sessions. Here is how the Manifest V3 sidecar reads state directly from browser memory without interfering with active sales reps.

Layer 01Zero DOM Scraping

In-Memory Webpack Store Hook

Directly hooks into WhatsApp Web’s minified Webpack module registry at runtime. Resolves WAWebChatCollection and WAWebConnModel by internal module ID with structural duck-typing fallback.

Layer 02Zero Screen Flicker

Zero-UI Background History Pager

Calls WAWebChatLoadMessages.loadEarlierMsgs() directly against the in-memory data store. Verified byte-identical active chat state: historical conversations sync without switching the user's screen.

Layer 03Reliable CRM Matching

Privacy Identity Resolver (@lid to @c.us)

WhatsApp’s newer privacy layer masks actual phone numbers under randomized @lid identifiers. Resolves real contact numbers through the internal Contact model before dispatching to the CRM.

Layer 04No Wedged Syncs

Resilient Outbox & Batch Throttling

Enforces a strict 50-message payload ceiling on the backend CRM bulk endpoint. Failed network batches automatically re-enqueue in an isolated pendingOutbox queue rather than dropping data.

Read In-Depth Architectural Specification→
7 Enterprise Connectors · 1 Shared Core

Admin Web Enterprise Integrations Platform

Systems Architecture / Web Core
The Problem

Rapid addition of third-party CRM connectors produced duplicate dialog code, fragmented OAuth state flows, and inconsistent schema validation across the web admin platform.

What I Built

Authored the foundational integration abstraction and unified data management architecture, establishing a shared modal lifecycle, credential validation pipeline, and bulk export framework.

Measurable Result

Shipped and scaled the unified platform to support Zoho CRM, Salesforce, HubSpot, Facebook Lead Ads, WhatsApp Business, AWS S3, and Google Cloud Storage as the active platform foundation.

Integrations
7 connectors · 1 shared dialog

Seven connectors, one shared dialog

Built the base dialog that handles credentials, webhook payload normalisation, and background sync. The first connector was slower for it; every later one was faster.

Zoho CRMProduction

Two-way lead and call-log sync

CRM
SalesforceProduction

OAuth 2.0 authorisation and lead push

CRM
HubSpotProduction

Custom lead properties and deals

CRM
Facebook Lead AdsProduction

Webhook lead ingestion

Ads
WhatsApp BusinessProduction

Template message sending

Messaging
AWS S3Production

Call-recording storage

Storage
Google Cloud StorageProduction

Recording retention

Storage
Shared base dialog

Handles schema validation, OAuth state, and retry logic once, so a new connector implements only what is actually different about it.

Read In-Depth Architectural Specification→

Engineering Track Record

Full-time engineering ownership and measurable production impact.

Software Engineer — Runo (Rutakshi Technologies)

Full-time · Hyderabad, India · On-site
July 2025 — Present
  • 42m → 3.8m (-91% Pipeline Latency)DuckDB Analytics Engine

    Engineered Runo's core call-analytics pipeline processing 10,000+ daily inbound and outbound recordings with DuckDB, fingerprint caching, and multi-surface automated exports (Sheets, Firestore, Drive, Apps Script).

  • 15M+ Events/mo · 45M Peak BackfillTelemetry & Event Platform

    Architected the cross-platform telemetry facade over Amplitude with sliding-window deduplication (-14% noise reduction) and durable local disk buffering to guarantee zero data loss during network drops.

  • 7 Enterprise ConnectorsAdmin Platform Ingestion

    Authored the unified integration abstraction powering Zoho, Salesforce, HubSpot, Facebook Lead Ads, WhatsApp Business, AWS S3, and Google Cloud Storage across the web admin platform.

  • 0 Official API · 100% Background SyncReverse-Engineered Browser Sidecar

    Reverse-engineered WhatsApp Web's minified Webpack module registry at runtime, enabling background CRM lead capture and store-level chat history paging with zero user screen interruption.

View Full Résumé & Historical Work→

Production Stack

Technologies and frameworks verified across production workloads.

High-Throughput Data & ETL
DuckDB · Python · pandas · Hamilton Framework · SQLite · Google Cloud Firestore · Google Sheets API · cron / Task Scheduler
Machine Learning & AI
scikit-learn · imbalanced-learn (SMOTE) · PyTorch · OpenCV (OMR computer vision) · PEFT / LoRA · LlamaIndex
Telemetry & Event Infra
Amplitude Taxonomy · Hive Local DB · Cloudflare Workers · Google Cloud Platform (GCP) · AWS (S3, EC2)
Application & Client Systems
Flutter · Dart · TypeScript · Chrome Extensions (MV3) · Webpack Bundler Interop · Node.js · Astro

Get in Touch

Available for senior AI engineering, data platform, and distributed systems discussions.

Direct Contact
GitHub →LinkedIn →Résumé (PDF) →
Academic Background

B.Tech in Information Technology (AI & Data Science specialization)
Engineering college · 2021—2025

Engineered with Astro & Tailwind. Every metric on this page is measured and verified in production.