I’ve been using my custom domain with iCloud email for a couple weeks. Truth be told I hardly notice any difference from when I used Gmail. That’s largely due to using Apple Mail app as my primary interface. I haven’t had any reliability issues and the only real difference I’ve noticed is the email size limit is 20MB as opposed to Gmail’s 25MB. The primary benefit I’ve seen is that calendar invites end up on my iCloud calendar, which is where I prefer them.
Your mileage may vary, but its working out for me thus far.