I had begun the decrease rounds when I admitted that no amount of elastic in the band was going to save this hat. This hat was huge and there was no hiding that fact any longer. I ripped it out.
I did deep breaths and cast on again, with the correct needle size. I finished the band and have begun the increase section. A normal person will be able to wear this hat, fingers crossed.