Ahhhh my bad! I really should’ve researched a bit further before posting my question.
It is indeed a browser thing so, for anyone interested in avoiding this situation when it’s unwanted, simply use this bit of JS on your dismiss/close JS call:
// Remove navigation prompt
window.onbeforeunload = null;
Source: https://stackoverflow.com/questions/1119289/how-to-show-the-are-you-sure-you-want-to-navigate-away-from-this-page-when-ch
