Skip to content
Rachad Ghanem
All work
LiveLive with the launch client · tenant rollout starting

Smart Scale

Camera-based produce recognition for retail scales.

  • Python
  • MNN
  • Electron
  • FastAPI
  • OpenCV
  • SQLite
  • .NET 8
  • PostgreSQL + pgvector
  • Next.js
  • Docker

Closed source · case study

01

The problem

At a produce scale, the cashier has to find the right item in a long list, for every customer. The scales shipped with the vendor's own app; this replaces it with one that recognizes the produce and learns from every confirmed sale.

It has to keep selling with no network, never hold up a sale when it's unsure, and cope with produce that looks different from shop to shop and season to season, without retraining a model each time.

02

My role

I pushed for the recognition approach and owned its design.

  • The case for embeddings over a local classifier or a paid vision API
  • Reusing the scale's own embedding model instead of shipping a new one
  • The learning loop: a confirmed pick becomes a reference for its category
  • How references move between scales: through the Portal API, gated by approval and pipeline signature
03

Architecture

SCALE · PYTHON AGENT + ELECTRON UItriggerssavedapproved refssame pipelineown refswith each saleTenant dashboardNext.jsOwner portalNext.jsapproves new categoriesShared databaseall tenantsPortal API.NET 8Every other scalesame loopWeighing scaleCOM1 · stable weightCameraon the scaleEmbedderscale's own modelSimilarity searchvs all categories' refsRanked liston screenUser confirmson screenReferencesper categoryown + sharedNew referencefor that category
Recognition on the scale, learning across the network. The highlighted path is the loop that makes it better with use.

ALSO IN THE SYSTEM

  • A Python agent on each scale owns the weighing scale, camera, model, index and label printer; the Electron kiosk only draws the screen
  • Only 1,024-number fingerprints travel between scales, never photos; ranking happens on the scale, and Postgres with pgvector only stores them
  • Two pools per category, own and shared, with the oldest dropped first: a mislabelled photo is the least like the rest, so the rule that drops redundant references could never drop it
  • Labels are drawn as bitmaps because the printer's text mode can't shape Arabic, and carry an EAN-13 with product code and weight
  • Licensing: one sign-in per shop day, then the scale sells offline all day; clock rollback is detected
04

Key decisions

DECISION 01

Embeddings, not a classifier

CHOSE
Embed each photo and rank it against stored references for every category.
REJECTED
Training a local classifier, or paying for a vision API.
WHY
Teaching the system a new category means adding references, not running a training job. And everything runs on the scale, so the till keeps working offline.
DECISION 02

Keep the scale's own model, after measuring

CHOSE
The MobileNetV2-based embedding model that shipped with the scale, run on-device.
REJECTED
Six open models: CLIP, SigLIP, ResNet-50, EfficientNet-B0, MobileNetV2 and MobileNetV3.
WHY
On the shop's own produce photos it ranked first once a category had three references, and it was already on the hardware. The same benchmark showed its weak spot: with a single reference, two open models did better.
DECISION 03

A shortcut, never a gate

CHOSE
Suggest up to three categories, and only let one lead when it clearly wins. The cashier picks the exact product, and only labels a person confirmed are learned.
REJECTED
Auto-selecting the top match.
WHY
A wrong guess costs a tap, not a wrong sale. And a corrected pick is worth more than any other reference: it's the case the model got wrong, labelled by someone looking at the item.
DECISION 04

Share fingerprints, gate what spreads

CHOSE
A new category needs approval with real photos. A scale only accepts references made by the same pipeline, checked by a signature of model, size, crop and normalisation.
REJECTED
Every scale learning alone, which was the first design, or sharing everything unchecked.
WHY
One mislabeled reference shouldn't spread to every shop, and a changed camera or crop setting must be detected instead of silently ruining every score. A QA pass caught exactly that: one setting change had cut off every approved reference.
05

Results

Live

with the launch client, tenant rollout starting

1st of 7

embedding models benchmarked on the shop's own produce, at three references

1,274

automated tests on the scale agent

1,153

automated tests on the .NET API

Recognition quality is measured live: the tenant dashboard shows 7-day top-3 accuracy against the previous period, and the pairs the camera confuses.

06

What I'd do next

  1. 01

    Shrink the on-scale index's memory, so shops with 150+ categories fit on the same hardware.

  2. 02

    Record which shop each shared reference came from, so a bad one can be traced and removed.

  3. 03

    Keep source photos, so a pipeline change re-embeds references instead of every shop teaching from scratch.