Speed review timer reset not consistent

When I’m doing a speed review, the timer is reset whenever I make a mistake and lose a :heart:.
However, when I don’t reply quickly enough, the timer is not reset.

That seems inconsistent to me. Is there a particular reason for the speed review working like this?