Yes, that was it! 🙂
I have a custom fields box in the sidebar that appears for all post categories. I set the Order No. to 5 and the problem is now solved.
I do think however you should make it so that hiding takes priority over not hiding, for the options.
