Multi-task, debiasing and distillation

Because there are always several signals: engagement (clicks, time), satisfaction (likes, ratings), long-term metrics. A separate model per signal costs several times the resources and is harder to maintain.

Shared bottom gives regularisation, transfer of knowledge and a saving. The trouble arises on conflicting tasks: the gradients of the heads pull the shared representation in different directions. In a simple geometric model where the directions of the tasks differ by an angle α, each gets cos(α/2): at 90° that is 0.71, a loss of 29%, and at 180° zero.

The blunt cure is to widen the trunk so that there is room for both directions. The more careful one is MMoE.

MoE: \(y = \sum_i g_i(x) f_i(x)\), where \(f_i\) are experts and \(g\) is a gate; the key idea is conditional computation, with part of the experts working on an object.

MMoE is multi-gate: every task has its own gate, that is, its own expert weights. If the tasks are unrelated, each learns its own subset of experts and no conflict arises.

An important note for an honest comparison: the gates are of the order of one and a half percent of the parameters, everything else is simply the capacity of the experts. So MMoE has to be compared not with a narrow shared bottom but with one widened to the same capacity — otherwise it is unclear what produced the gain.

The problem is a mismatch between the slice of training and the slice of application. Training P(purchase | impression) directly is blocked by sparsity: at a CTR of 5% and a CVR of 3% the imbalance is 1 to 666, and the signal is too complex besides. Training P(purchase | click) is easier, but such a head is trained on 5% of the data (clicks only) and applied to every impression — 20 times more objects.

Why that breaks ranking: pCVR is a probability conditional on a click. A niche product with a pCTR of 0.01 and a pCVR of 0.40 is the champion by the conditional probability, while by expected purchase it is 0.004 against 0.0072 for a popular one — a loss of 44% at the top position.

ESMM: model the chain impression → click → purchase, P(purchase|impression) = P(click|impression)·P(purchase|click), two heads. And the key technique — the second head is trained on a loss over P(purchase|impression), that is, over the entire space of impressions; the gradient reaches it through the product.

For two different reasons, and they have to be told apart: users click the top more often because they see it sooner and trust it more (that is a bias), and the top really does hold more relevant items (that is the model working). Mixing them means subtracting real relevance along with the bias.

Concretely, the bias reverses the order: an item with a relevance of 0.12 at position 5 gives an observed CTR of 0.039, while an item with a relevance of 0.10 at position 1 gives 0.100. A naive model confidently puts the worse one higher.

Four cures: train on the first position only (with a slate of 20 that leaves 5% of the data), position as a feature with pos=1 substituted at inference (the model can hide relevance in the position feature), IPW (needs random traffic, and a spread of weights of 8 times inflates the variance) and a bias tower — the production standard.

A separate small tower fed with all the context useful for modelling the bias — not only the position but also, for instance, the device: fewer positions are visible on a phone than on a desktop.

During training the outputs are added in the logit: the main model plus the tower. A mandatory element is dropout on the tower's output, otherwise it explains too much by position, pulls the relevance signal onto itself, and the main model is left undertrained. At inference the tower is thrown away and a clean score remains.

The elegance of the idea is that the bias is modelled explicitly and separably: we do not subtract it after the fact, we give it a separate channel to be written off to, and then cut that channel away.

Two reasons. First: a label is binary, a prediction is not. «There was no click» looks identical for an object with a probability of 0.40 and one with 0.02, while the teacher distinguishes those cases. This is dark knowledge — what is passed on is not the answer but the shape of the distribution.

Second: the teacher is a smoothed version of the data. At a true click probability of 0.05 and ten impressions, the standard error of the estimate is 0.069, that is, 138% of the quantity itself: the label says almost nothing about the particular object. The teacher averaged that noise over the whole set, and the student gets a less noisy target — and noise is the main source of overfitting on the tail.

The motivation for the whole technique is trading quality for latency: the teacher is computed offline, the student fits the ranker's budget.

Because the teacher has biases of its own: it was trained on the logs of the same system and inherited its positional bias, its skew towards the popular and its feedback loop. Distilling it directly, we distil that too.

The solution: one head learns from the target labels, the other from the teacher's predictions, and the first is the one used for ranking. The distillation head works as an auxiliary task: it forces the shared representation to absorb the teacher's knowledge without dictating the order. The knowledge passes through the shared trunk, the biases stay in the other head.

The second motive is calibration: the teacher's predictions break it, a head on real labels preserves it. For advertising, where the score is multiplied by a bid, that settles it.

Four places. The teacher drifts when retrained, and the student's target changes abruptly — and monitoring will not show it, since each model on its own looks fine. The teacher has to be computed over the whole training stream — distillation moves the cost from inference into training rather than removing it. The student inherits the teacher's systematic errors and will never surpass it on that subset, though on the real labels it could have. And a closed circle of versions: the teacher is trained on the logs of the previous student, and after a few iterations the system distils itself.