System design

• First a simple working solution, then criticism, then the full one. A complex system designed from scratch does not work and cannot be patched into working.
• Goal setting cannot be skipped. A crude mistake here will not be rescued by any model below it — and skipping it is always tempting.
• The size of the catalogue determines the architecture. Up to 75 000 candidates no funnel is needed at all, and refusing one is the right decision, not a simplification.
• A rare target event is measured through a proxy not out of stubbornness. Catching the same relative effect at a conversion of 0.5% instead of 40% costs 133 times more traffic.

They are part of the life of any engineer and any runtime. They are best accepted with an understanding of what the long-term correct solution is — so as not to do something that fundamentally contradicts it.

An honest caveat: sometimes merely working out the correct solution is extremely expensive. Then it is better to trust experience — some working solution is in any case better than none.

Formulate the key actors — most often the user, the item and the context. Then invent ideas for candidate generation and for features:

• for each actor separately — three cells;
• for each pair of actors — three more.

The result is a 3 + 3 grid that almost always covers everything essential. And it is convenient to speak through out loud as the structure of the answer — the interviewer sees a system rather than an enumeration.

Product ones. Recommendations on very many surfaces, which actively cannibalise each other. The user is well known: a large share of regular customers, nothing works without a login — cold start is less relevant. And the user already has an intent: almost nobody comes in «just to scroll». The task is not to get in the way too much while selling something else along the road.

Technical ones. The catalogue at a location is most often up to 3000 items and well structured. There are tens of millions of users and hundreds of requests per second. The response budget, counting prefetches, is up to 300 ms. Goods become available and unavailable instantly, and they spoil, so new arrivals have to appear in the output quickly.

It is useful to name that yourself, without waiting for the question.

• A user may assemble a basket over a long time. Then actions inside the session become important — especially for those we know less about. Which means a runtime loop for the user part is needed.
• The solutions against cannibalisation will require a full shared feature layer.
• Growth of the catalogue will require splitting candidate generation and features into separate services.

Re-ranking: the hygienic minimum is softmax sampling against the feedback loop, plus categorical diversity and separate work with new arrivals through bandit blending.

The experiment: the split is by user; the key metric is integral turnover; the auxiliary ones are margin and the conversion of the surface; the guardrails are turnover by category (did we skew the output) and conversion by position (did we break the top).

1. Pick a list of base metrics with a plausible long-term effect: watching series (they have better retention), discovery and the diversity of what is consumed, frequency of use.
2. Build a dozen candidate combinations out of them.
3. Work out retrospectively which of them show the long-term effect better — by matching similar users or by adjusted correlations. A model can be trained too, but interpretability is needed.
4. Validate on a golden set of past experiments — those A/Bs whose long-term outcome is already known.

The fourth point is the one usually forgotten, and it is the real validation: without it the proxy remains a hypothesis.

This is the main difference of the case, and it changes the formulation. The author has their own interests and expectations: authors count on impressions, and their absence produces complaints. The system stops being two-sided «user — item» and becomes three-sided.

The typical approaches: item exploration built into the model, PID controllers to stabilise the volume of impressions, explicit quotas.

And the consequence people forget: authors are paid for impressions, so fraud begins — bots are run, CTR is inflated. In a good system the target is antifraud-aware, as is the objective metric, and the teams work closely together.

1. Which business metric? And immediately — how often the target event happens. If it is rare, the conversation about a proxy starts here, not at the end.
2. The size of the catalogue and the number of users? That decides whether a funnel is needed at all.
3. Does the user have an intent? A feed and a search are different problems, though formally one.
4. The response budget? 50 ms and 300 ms are different architectures.
5. Who is known: users, items, both? Hence which cold start is critical.
6. Is there a third side — authors, sellers, advertisers? That changes the formulation entirely.
7. Which filters are mandatory? Legal restrictions are designed in from the very beginning, not bolted on.

Nineteen chapters followed the path of a request: from the statement of the problem and its measurement — through candidate generation, ranking, work with sequences and with the slate — to engineering and back to design.

If one thought is worth taking away from all of it, let it be this: in recommendations almost every technical decision is derived from a constraint rather than from fashion. The funnel comes from the latency budget. Factorisation comes from sparsity. Two towers come from the number of candidates. The stochasticity of the output comes from the need to measure. A good interview answer differs from a bad one not in the set of names but in whether the person names the constraint before the solution.

Next come the revision questions, the widgets and the walkthrough of the open-sourced code of a real feed, where all of this is visible in working form.