Tyagi // Engineering Log
← Back to Projects
E-commerce Automation shipped

ENTHRAL.IN Supplier Price Change Detection

Built a supplier scraping and comparison workflow that flags pricing/listing differences against an OpenCart storefront and prepares margin-based updates.

Date 2023-07
Reading Time 3 min read
Difficulty
Stack
Python,OpenCart,Web Scraping
Reported

Electronics / CCTV e-commerce

Domain

Problem & Context

ENTHRAL.IN, an electronics and CCTV e-commerce business, needed its OpenCart product listings and prices to stay aligned with supplier-side changes without manually re-checking every product page.

Solution & Architecture

Diagram source (mermaid)
flowchart LR
  A[Supplier site] -->|scrape| B[Product data + pricing]
  C[OpenCart storefront] -->|export| D[Current listings]
  B --> E[Compare]
  D --> E
  E --> F[Margin-based price adjustment]
  F --> G[Import file for OpenCart]

The tool scraped supplier product details and pricing, compared them against the client’s existing OpenCart listings, and generated import-ready updates that applied predefined margin rules — turning a manual price-check task into a repeatable comparison-and-import workflow.

Key Decisions & Tradeoffs

Scraping a third-party supplier site instead of relying on a feed or API meant the comparison step had to assume the source layout could change without notice — the safer design treats every scrape as a possibly-stale snapshot and surfaces differences for review rather than auto-publishing every detected change straight to the storefront. That trades full automation for a margin of human review on anything outside expected bounds, which matters more for a small electronics catalog where one bad import can mis-price a product for every visitor until caught.

Applying margin rules as a separate step from comparison (rather than baking margin logic into the scraper) also meant OpenCart-specific import formatting stayed isolated from the comparison logic, so either side could change — a different supplier source, a different margin policy — without forcing a rewrite of the other.

Lessons Learned

Treating “detect a difference” and “decide what to do about it” as separate steps (compare first, then apply margin rules) made the tool easier to adjust later — margin rules could change without touching the scraping or comparison logic.

A scraper-based integration is also a maintenance commitment in a way an API integration isn’t: it needs to keep working as the supplier’s site changes, with no contract guaranteeing stability. For a small catalog where the supplier relationship is informal and no official feed exists, that tradeoff was acceptable — the alternative was no automation at all, not a cleaner integration. The lesson generalizes: scraping is often the only option available for small-scale, informal data sources, and the right response isn’t to avoid it but to design the downstream pipeline (compare → review → apply) so a broken scrape produces an obviously-wrong diff rather than a silently bad price update.

In practice that meant erring toward over-flagging rather than under-flagging differences: a comparison step that’s too sensitive produces a noisier review queue, which is annoying but safe, while one that’s too lenient can silently let a bad price through, which is the failure mode that actually costs the business money.

Share: X LinkedIn

Keep exploring