I too normally wear XL, but sometimes need XXL. I have this shirt in XL, and I'm comfortable in it. Just roomy enough, but not baggy. Trimmer than some of my other XL shirts, but not what I would call a slim fit, like some western shirts are. Being nylon, it does't shrink either.
Go up a size!