One reviewer decision has to apply correctly to every duplicate and related document, across millions of items.
What I built
Duplicate-aware tag propagation, a safe fallback for missing data, and AI-assisted tag recommendations, shipped behind feature flags.
Result
An estimated 1,500+ hours of manual tagging saved for users.
Simplified conceptual visualization. No proprietary Microsoft code, customer data, internal architecture, or confidential implementation details are shown. All diagrams are my own.
The problem
One decision, many documents
Review sets hold large collections of documents. Many are exact duplicates, and many belong to families, like an email and its attachments. When a reviewer tags one document, that decision has to reach every related copy, or the review ends up inconsistent.
Simplified conceptual representation
Nothing tagged yet. Apply the tag to see how far one decision has to travel.
Tagging one document should give its duplicates and family members the same tag, without the reviewer finding and tagging each one by hand.
Scale
Applying a tag is easy. Keeping it correct at 5 million documents is the hard part.
With ten documents, a mistake is visible and cheap to fix. At production scale, a small inconsistency repeats across thousands of duplicate groups, and nobody can check them by eye. Correctness has to be designed into the system.
Drawn to scale · each dot = 5,000 documents
10
documents in a quick local test
5,000,000
documents in the largest review sets
At 10 documents the whole set fits on one row of squares. At 5 million, a single dot stands for 5,000 documents, and it still takes a thousand dots.
System map
What happens between a click and a consistent result
Each step answers a question the previous one leaves open. Select a step to see what it does and why it is there.
Simplified conceptual representation
If duplicate data is missing or incomplete
The workflow continues safely instead of failing.
Step 1 of 7
What it does
Why it is there
Engineering decisions
Four decisions that shaped the system
Expand relationships before tagging
Problem
A reviewer selects one item, but that item can stand for a whole document family.
Decision
Expand every selected ID into its members before any tag is written.
Why it mattered
Families are tagged as a unit, so related documents never drift apart.
Treat IDs case-insensitively
Problem
The same document can arrive with identifiers that differ only in letter case.
Decision
Merge ID sets case-insensitively before propagation.
Why it mattered
No document is tagged twice or skipped because of formatting.
Fail predictably, not loudly
Problem
Duplicate data can be missing or incomplete when a tag is applied.
Decision
Define a deterministic fallback path instead of letting the request error out.
Why it mattered
The reviewer's decision still completes, and downstream pipelines don't fail.
Ship behind feature flags
Problem
A change to tagging behavior affects live reviews immediately.
Decision
Gate the feature with flighting so it can be turned on gradually.
Why it mattered
Controlled rollout, and rollback without downtime if something looks wrong.
Reliability
What if the data isn't there?
Switch the input and compare the two paths.
Simplified conceptual representation
Is duplicate data available for the tagged document?
✓ Active path
Normal resolution
Look up duplicate groups
Merge related IDs
Propagate the tag to every copy
Result: consistent tags across the group.
Not used
Deterministic fallback
Detect incomplete duplicate data
Take the defined fallback path
Tagging continues
Result: the workflow is preserved and nothing downstream fails.
Duplicate data available. Normal resolution path is active.
Both paths end in a usable result. The difference is whether propagation can run, not whether the reviewer's work survives.
Reliable systems are designed for imperfect inputs.
AI-assisted recommendations
From one document to a suggested tag
Propagation handles copies. Recommendations help with documents that are similar but not identical, by suggesting tags that related content already carries.
Simplified conceptual representation
inputDocument
A document in the review set.
representEmbedding
Its content becomes a vector.
[0.12, −0.48, 0.91, …]
searchSimilarity
Find the documents whose vectors sit closest.
contextRelated content
See which tags those neighbors carry.
outputTag recommendation
Suggest those tags for this document.
Each stage narrows the question, from “what is this document?” to “what would a reviewer likely tag it?”＋ Engineering detail: keeping it fast at scale
I integrated an embedding service and similarity orchestration into the tagging workflow.
The pipeline is backed by Cosmos DB, with cache-based embedding reuse so a document's vector doesn't have to be recomputed each time it is needed.
Computing embeddings is the expensive step, so reuse is what keeps recommendations practical across review sets of millions of documents.
Impact
What it changed for reviewers
5M
documents in the largest review sets the feature supports
Scale
1,500+
hours of manual tagging saved for users
Estimate
1 → all
one reviewer decision applied across every duplicate copy
Behavior
Reflection
What this project changed about how I engineer
01
Think in systems
A feature doesn't stop at its own code. It touches data, dependencies, failures, users, and the workflows that run after it.
02
Design for imperfect inputs
Production data is sometimes missing or inconsistent. A system has to stay predictable when its assumptions fail.
03
Measure engineering through impact
Engineering quality matters because of what it lets users do, like finishing a review without tagging the same email over and over.
Closing
Engineering is not just writing code. It is turning complexity into reliable outcomes.