I have owned many Glocks over the years, as well as a few M&P's. Both are great guns, but the P320 is, to me, the most comfortable.
I'd get the X-Model because it CAN take a red dot, you don't have to use one (best to have it and not need it, than to need it and not have it). Also, the trigger is one of the best striker-fired triggers out there...the CZ P-10 and HK VP series also have great striker-fired triggers out of the box.
Sig discontinued the X-Carry pistol, but you can still buy the frame size...they just don't sell a complete gun in that size. The X-Carry pistols are getting very difficult to find....I was lucky to find one a few weeks ago and grabbed it. I'm a fan of the full sized grip with the shorter slide....it's basically if Sig made a 19X.