TikTok Visual Search Pipeline

Visual search relevance is deceptively complex—a red dress query matching a red dress image seems obvious until you encounter edge cases: same dress different angle, similar dress different brand, dress in a lifestyle context. We built a 7-dimensional annotation framework that captures visual relevance, content relevance, query/doc quality drops, functional parity, category granularity, and visual similarity.

The pipeline iterated through 10+ prompt versions, each informed by automated comparison tools and reflection analysis that mined systematic error patterns. Rate limiting, checkpoint recovery, and concurrent processing made it production-ready. The final v2.2 prompt handles medical safety edge cases and clothing-specific rules that earlier versions missed entirely.