Krowd Merchant Intelligence Graph
A merchant graph that knows Dishoom Shoreditch and Dishoom King's Cross are the same business.
I designed and built Krowd's structured Merchant Directory for the UK restaurant market: a Neo4j graph of ~20,000 restaurant outlets modelled as merchant-to-outlet relationships, fed by an incremental, entity-resolving ingestion pipeline in Python. It went into production as the data layer behind Krowd's campaign platform and later grew to ~70K nodes as the company expanded into Ireland and the US.
- Role
- SDE-1
- Timeline
- 2019 — 2021
- Company
- Krowd R&D India Pvt Ltd
- Data store
- Neo4j
~20K
UK restaurant records
In the initial merchant directory
~70K
Graph nodes
After expansion to Ireland and the US
3
Markets
UK, Ireland, United States
6
Targeting modes
All, selected, geographic, new, underperforming, excluded outlets
0
Rebuilds
Incremental, identity-preserving ingestion
Live
In production
Data layer behind Krowd's campaign platform
01The problem
A flat list of restaurants loses the business behind the outlets.
Restaurant businesses run across many locations. Treating every outlet as an independent restaurant throws away the fact that London Soho, Canary Wharf, Shoreditch, Birmingham and Manchester all belong to one merchant. For Krowd's campaign system that relationship was the whole point: merchants needed to run a campaign across every outlet, a chosen subset, a geographic slice, newly opened sites or underperforming ones.
The hard part was never storing 20,000 records. It was deciding what each incoming record represented, an independent merchant, an existing one, a new outlet of a known brand, a known outlet, or a duplicate, and keeping those identities stable as new data kept arriving.
Raw data should not directly power intelligent applications. Turn records into entities, relationships and context first; product actions follow.
02My role
Every layer, explicitly.
No hand-waving about “contributed to”. These are the parts I designed, built and operated.
- Data model
- Designed the merchant → HAS_OUTLET → outlet graph in Neo4j around the real hierarchy of restaurant businesses.
- Ingestion pipeline
- Python workflows for cleaning, normalisation, validation, duplicate detection, parent identification and outlet matching.
- Entity resolution
- Name and business-information matching to decide create, update, link or reject for every record.
- Incremental architecture
- Continuous ingestion that preserves existing identities and relationships instead of rebuilding.
- Graph queries
- Relationship queries for outlets by merchant, location, recency and campaign criteria.
- Campaign integration
- Exposed the directory so merchant users target all, selected or underperforming outlets.
03What it does
Raw restaurant data in, a reliable merchant graph out.
- 01Normaliseclean & validate
- 02Searchthe existing graph
- 03Dedupereject duplicates
- 04Resolve merchantexisting or new
- 05Resolve outletmatch or create
- 06LinkHAS_OUTLET in Neo4j
Merchant graph model
Merchants as business entities, physical locations as outlet entities, connected by HAS_OUTLET relationships in Neo4j. Extensible as new markets and relationship types arrived.
- Neo4j
- merchant
- HAS_OUTLET
- outlet
Directory ingestion
Python workflows turning raw restaurant information into reliable entities for ~20,000 UK locations.
Normalisation & validation
Cleaning and normalising names and business information, then validating each record before it can touch the graph.
Entity resolution
Deciding whether a record is an independent merchant, an existing merchant, a new outlet of a known brand, a known outlet or a duplicate.
Duplicate prevention
Incoming records checked against existing merchants and outlets and routed to create, update, link or reject, so campaigns never target the same outlet twice.
Incremental ingestion
New record → normalise → validate → search graph → duplicate check → resolve merchant → resolve or create outlet → establish relationships. No rebuilds.
- open
- close
- expand
- update
- new market
Campaign targeting
Merchant users pick all outlets, selected outlets, a geographic subset, new sites or underperforming ones directly from the directory.
Graph queries
Which outlets belong to this merchant, how many locations a brand runs, which were added recently, which merchant a location belongs to, whether an outlet already exists.
04Architecture
A merchant intelligence layer between raw data and the product.
External restaurant data flows through normalisation, validation, entity resolution and duplicate detection into the Neo4j merchant graph. Krowd's campaign and personalization systems read the graph instead of re-deriving who owns which outlet.
Client
Krowd campaign platform
Merchant users choose the part of their business to target
API
Python ingestion & graph services
- Cleaning & normalisation
- Entity validation
- Duplicate detection
- Parent (merchant) identification
- Outlet matching
- Graph creation & update
- Relationship queries
- Outlet selection for campaigns
Neo4j
merchant → outlet graph, ~70K nodes
External restaurant data
raw records, continuously arriving
Python pipelines
incremental ingestion
Campaign system
all / selected / underperforming outlets
Personalization
outlet-level data foundation
- Python
- Neo4j
- Cypher
- Graph data modelling
- Entity resolution
- Deduplication
- Data normalisation
- Incremental pipelines
05Skills used
Everything the build touched, by layer.
Graph & data
- Neo4j
- Graph data modelling
- Entity relationships
- Parent / outlet hierarchies
- Cypher queries
Data engineering
- Python
- Data processing
- Data normalisation
- Validation
- Incremental ingestion
Entity resolution
- Duplicate detection
- Merchant identification
- Outlet matching
- Identity preservation
Product applications
- Merchant intelligence
- Restaurant directory
- Campaign targeting
- Outlet selection
Scale & operations
- 20K → 70K nodes
- UK, Ireland, US
- Continuous growth without rebuilds
Architecture
- Intelligence layer separation
- Entities → relationships → context
- Extensible data model
06Hard problems
Where the engineering actually lived.
- 01
Deciding what a record actually is
Every incoming restaurant could be an independent merchant, an existing one, a new outlet of a known brand, a known outlet or a duplicate. Normalisation plus matching on names and business information had to make that call reliably before anything was written.
- 02
Keeping identities stable while the data keeps changing
Restaurants open, close, expand and change details constantly. Incremental ingestion resolves each change against the existing graph so merchant and outlet identities that campaigns already depend on never shift underneath them.
- 03
Duplicates that would misfire campaigns
A duplicated outlet means wrong targeting and repeated actions against one location. Validation and deduplication run before the graph is touched, routing each record to create, update, link or reject.
- 04
Modelling the business, not the list
Campaigns needed to run at merchant level, at outlet level and at arbitrary subsets in between. Making the parent-outlet hierarchy explicit in the graph made all of those first-class queries rather than application-side reconstruction.
- 05
Growing past the original market
The model built for ~20K UK records carried Krowd's expansion into Ireland and the US to ~70K nodes without a redesign, because relationships rather than a flat schema were the foundation.
07Engineering decisions
Why it is built this way.
Why a graph database instead of relational tables?
The questions the campaign platform asked were all about relationships: which outlets belong to this merchant, which were added recently, which meet a criterion. In Neo4j those relationships are first-class elements of the model rather than joins reconstructed in application code, and the model extended naturally as Krowd added markets and new business relationships.
Why make parent and outlet separate entities?
A merchant is a business; an outlet is a place. Campaigns, performance and personalization operate at both levels, and performance differs a lot between outlets of the same brand. Separating them let a merchant run one campaign across everything or only its underperforming sites without the data model getting in the way.
Why resolve entities before writing, rather than cleaning up afterwards?
Duplicated outlets in a campaign system mean incorrect targeting and repeated campaign actions against the same location. Every record passes normalisation, validation and a search of the existing graph before it is created, updated, linked or rejected, so the directory never has to be de-duplicated after the fact.
Why incremental ingestion instead of periodic rebuilds?
Merchant data changes constantly as restaurants open, close, expand and update details. A rebuild would reset identities that campaigns already referenced. Incremental ingestion keeps the merchant and outlet identities stable while the directory keeps growing.
How does the pipeline decide what an incoming restaurant is?
Normalise the name and business information, validate the record, then search the graph: an existing outlet becomes an update, a new location of a known brand becomes a new outlet linked to its parent, an unknown business becomes a new merchant, and a near-identical record is rejected as a duplicate. Dishoom Covent Garden, Shoreditch and King's Cross resolve to one merchant with three outlets.
Why keep the intelligence layer separate from the campaign system?
Campaign creation should not have to work out whether two restaurants belong to the same business every time. That knowledge lives in the graph once, and campaign targeting, personalization and analytics all read from it. The directory grew from ~20K UK records to ~70K nodes across three markets without the merchant model being redesigned.
08What I'd tell another builder
Relationships are the data.
In an entity-centric system the links between records matter as much as the records themselves. Store them as first-class elements.
Resolve before you write.
The engineering challenge was identity, not storage. Deciding what each record represents up front keeps the whole system trustworthy downstream.
Build the intelligence layer once.
Entities → relationships → context → product actions. Once that layer exists, targeting, recommendations, analytics and personalization all get easier.
09Ownership
From raw restaurant records to a production merchant graph: data model, pipeline, resolution and campaign integration.
- Neo4j data model
- Python ingestion pipeline
- Normalisation & validation
- Entity resolution
- Duplicate prevention
- Incremental ingestion
- Graph queries
- Campaign targeting integration
Contribution
- Graph data modelling
- Ingestion pipeline
- Entity resolution
- Duplicate prevention
- Incremental ingestion
- Campaign integration
Built at Krowd
Want this kind of ownership on your team?