This isn't finished, and I haven't tried this pattern yet
This is just to give the general idea; a few folks have asked for the pattern, but I don't currently have the time.
I haven't yet worked out how to sew the eye hole, and still be able to turn the thing inside-out.
I've drawn it this way, because this is how I thought of it first.
Perhaps it's easier if you cut two bits and sew across QP. Or hand-sew the eye hole, maybe.
You can see how the hat relates to this.
You can also introduce rectangles into QP, so you get more of a neck on this.
I suppose you needn't actually cut two pieces at all - one might be enough, but I'm not convinced.