← The Thread: notes on shopping an outfit from a photo

How it works

How does outfit search work, from screenshot to buy link?

Snagfit turns a photo into a list of garments, turns each garment into a short text search, and runs that search against shopping listings. Here is every step, in order.

Save to Pinterest

Snagfit takes a photo of an outfit and gives you back a list of garments with somewhere to buy each one. Between those two things sit three steps, and none of them work the way most people assume. Here they are in order.

Step one: how does the AI read your photo?

The photo goes to a vision model, which is an AI that can look at an image and describe it in words. Snagfit asks it one specific question: what is the main person wearing?

The model answers with a list. For every garment it can see, it writes down five things. The name of the item, its category, its main colour, a short note on the material or pattern or cut, and a short search query built from all of that.

That last field is the important one. It is four to eight words long, and it is what actually goes to the shops.

Step two: how does a garment become a search?

Here is a real example of what comes out of step one:

Name: chunky retro runner sneakers. Category: Footwear. Colour: cream and gum. Details: suede overlays, dad-shoe silhouette. Search query: chunky cream retro runner sneakers.

Snagfit then searches shopping listings for chunky cream retro runner sneakers. It does not search for your photo. It does not search for pages that used your photo. It searches for the words that describe the shoe.

This is the part people find surprising, so it is worth being blunt about it. By the time the shopping search runs, your picture is gone. What is left is a sentence about a shoe.

Step three: why do results arrive one item at a time?

Snagfit shows you the detected items first. Nothing has been searched for yet at that point, which is why the first screen appears quickly.

When you tap Snag it on one item, that is when its search runs. Snagfit returns up to six listings for that garment, each with the store name, the price and the picture from the listing.

Doing it this way means you wait a few seconds once, for the item you care about, instead of waiting for eight searches you never asked for.

What happens when you tap buy?

Nothing is bought on Snagfit. There is no checkout, no cart and no payment screen.

The buy link resolves the retailer's own product page and redirects you there. Everything after that point, including price, stock, shipping and returns, is between you and that retailer. Snagfit may earn a commission on some of those links, which is written out in full on the privacy, terms and disclosure page.

Why describe the garment instead of matching the image?

Describing the garment before searching for it is a deliberate choice, and it has one clear tradeoff.

The cost is that Snagfit will sometimes miss the exact piece. A description of a black leather jacket matches many black leather jackets, and the specific one in your photo may not come first.

The benefit is that Snagfit can find something for a garment that has no listing anywhere. A vintage coat, a sold-out drop, a piece from a brand that never sold online. An image-matching search returns nothing for those. A description search returns the closest thing you can actually buy, which is a more useful answer than an empty screen. There is a longer post on why close matches happen if you want the detail.

What happens to your photo before the search starts?

There is a step before step one, and it happens on your own phone.

When you pick a photo, your browser reads it and checks it is actually an image. Then it draws it onto a canvas at a smaller size: 1280 pixels on the longest side, saved as a JPEG at quality 0.9. Only that smaller copy is sent anywhere.

This is worth knowing for two reasons. It means the upload is fast even on a poor connection, because you are sending a few hundred kilobytes rather than a multi-megabyte original. And it means resolution is almost never your problem. A screenshot from any phone made in the last several years already carries more detail than 1280 pixels, so the picture gets smaller before the search ever sees it.

The server accepts an image up to 12 megabytes. In practice you will never reach that, because the downscale happens first.

What this does not fix is blur, bad framing or a garment that is hidden. Those survive the resize perfectly. Detail that was never in the photo cannot be added back, which is why cropping and choosing a still frame matter so much.

What is a vision model, and why use one?

A vision model is an AI that can look at an image and write about it in words. That is exactly the capability the first step needs, and it is a recent one.

Snagfit sends the photo with a fixed set of instructions: identify every distinct clothing item, footwear and notable accessory on the main subject, and return them in a strict structure. The model is required to answer as data, not as prose, so there is no paragraph to parse and no chance of it replying "the person appears to be wearing something stylish".

The instructions also carry specific corrections for mistakes these models make by default. The clearest one is sleepwear. Left alone, a model looking at a flannel pajama set will often call it a sweater, because a sweater is the nearest familiar category. Snagfit tells it not to do that, and to separate sweaters, hoodies, sweatshirts, robes and pajamas by their actual construction: a matching top and bottom, a drawstring or elastic waist, a button-front sleep shirt, a satin or fleece texture.

Prints get the same protection. A candy cane print pajama pant has to stay a candy cane print pajama pant. If the model flattens it to "red and white pants", the shopping search that follows is looking for the wrong thing entirely, and the print was the most searchable feature it had.

What happens if the AI is down?

Snagfit calls Gemini first. If that request fails, it falls back to Anthropic's model rather than returning an error.

You will not notice which one answered, and that is the point. An API can be slow or briefly unavailable, and a shopper who took a screenshot of a video that has already scrolled away should not lose their result because a service had a bad minute.

Both are given the same instructions and the same required output structure, so the shape of what comes back does not change.

How does the buy link find the shop's own page?

The listings Snagfit shows come from shopping search results, and those results do not always link straight to the retailer's own page. Some of them point at an intermediate page first.

So when you tap the buy link, Snagfit takes the identifier attached to that listing and resolves it to the retailer's real product URL, then redirects you there. What you land on is the shop's own page for that item, where you can check the price, the size and the stock yourself before deciding anything.

If the merchant link can be wrapped for affiliate tracking, it is. That is how Snagfit makes money, it costs you nothing, and it never changes which results you see or what order they appear in. The full disclosure is on the privacy, terms and disclosure page.

What Snagfit does not do at any point is take a payment. There is no cart, no checkout and no card field. Everything after the redirect is between you and that retailer, including delivery, returns and anything that goes wrong, which is why it is worth running a couple of checks on a shop you do not recognise.

Does Snagfit save the shops it found?

Snagfit saves your photo and the garments it found, so you can reopen a search later. It deliberately does not save the listings.

Those are fetched fresh every time. A price stored three weeks ago would be wrong more often than right, and a saved link to a sold-out listing sends you to a dead page with a number on it that no longer applies. The garments stay the same because a garment is a description; the shops change because shops change.

What are the limits of an outfit search?

Snagfit returns at most eight items per photo, and it puts the most prominent first. So a group shot of five people does not return forty garments. It returns the main pieces on the main subject.

If Snagfit cannot see a person or any clothing, it returns nothing and says so. That is on purpose. Inventing an item would send you shopping for something that was never in your picture.

Common questions

How does Snagfit find clothes from a photo?

Snagfit sends the photo to a vision model, which identifies each garment and writes a short description of it. Each description becomes a text search of four to eight words, and that search runs against current shopping listings. So the search is built on what the garment is, not on the pixels of your picture.

Does Snagfit search for the picture or for the clothes?

For the clothes. Snagfit never looks for other copies of your photo. It identifies the garment first, describes it in words, then searches shopping listings using that description. A reverse image search does the opposite, looking for other pages that used the same picture, which tells you nothing about what a garment is.

How many items can Snagfit find in one photo?

Between one and eight. Snagfit is instructed to return at most eight items and to put the most prominent ones first, so a busy photo returns the main pieces rather than every strap and button. A photo with no visible person or clothing returns nothing at all.

What information does Snagfit record about each garment?

Five things for every item it finds. The name, the category, the main colour, a short note on material, pattern or cut, and the search query built from all of that. You can see the first four on screen when you tap an item, and the search query is the four to eight word phrase that actually goes to the shops.

What categories can Snagfit put a garment in?

Eleven. Top, Bottom, Outerwear, Footwear, Dress, Sleepwear, Headwear, Bag, Accessory, Jewelry and Eyewear. Every item Snagfit finds is filed under one of them, which is what lets the search stay specific instead of returning general clothing, and it is why a pajama set gets filed as Sleepwear rather than as a plain sweater.

Does Snagfit buy anything or take payment?

No. Snagfit has no checkout, no cart and no card field. When you tap the buy link, Snagfit resolves the retailer's own product page and sends you there, and the purchase happens on that retailer's site under their terms. Snagfit never sees your card, and delivery and returns are between you and that shop.

How long does a search usually take?

A few seconds for the detection step, then a few more when you tap Snag it on an item. Product matches are fetched per item rather than all at once, so the first screen appears before any shopping search has run, and you only wait on the items you actually tap.

What happens if Snagfit cannot see any clothing?

It returns an empty result and tells you so. Snagfit is instructed to return nothing rather than guess when there is no person or no visible garment in the photo, because a made-up item would send you shopping for something that was never in your picture.

Keep reading

How it works

Why do two people in a photo confuse a search?

Group photos are the most common upload that comes back wrong. The search reads every garment it can see and has no way of knowing which person you meant.

8 min read
How it works

Why does the search pick the wrong garment?

You wanted the jacket and it described the shirt underneath. That is a detection problem rather than a matching problem, and the two have completely different fixes.

7 min read

← Back to Snagfit