After a brief "spike" in the connectivity that causes buffering (often at twitch end) there is no way to "recover" the buffer.
Reducing delays between streamer and viewer is obviously a good thing that Twitch has worked to minimise, however, being suddenly thrown tens of seconds behind "real time" because of a spike offers no means of recovery - one can refresh the page and lose those 30 seconds of content, however, it would be nice if there was an option to "speed up" playback by 1-5% so that the delay is eroded until we catch back up with minimum buffer sizes and drop back to pure 100% speed.

Think the way NTP works and the way it correct for clock drift - slowly bring us back into sync (or give the option to).