Do loss functions have to be differentiable?

Yes and no. If the loss function is only used for evaluating the quality of the prediction (on an evaluation set), then it does not have to be differentiable. However, if the loss function is the objective of a gradient descent algorithm, then yes, it has to be differentiable to do that.

Example of loss functions that don’t need to be differentiable to be useful?

F1 score, BLEU score, rewards in reinforcement learning. Those losses are for evaluation of how good a certain model performs on a set of data. Therefore, only used in comparing models, but not in directly training models.