Softmax and loss lab

Change logits and temperature. The target changes the loss, not the predicted probabilities.

Inputs

Token A logit Token B logit Token C logit Temperature Correct next token

Try making the correct token's logit low. Watch its negative log-likelihood rise.

Entropy measures uncertainty in the model's distribution. Cross-entropy for a one-hot target is −ln(probability of the correct token).