In the end, my conclusion is that while troubleshooting another issue, I ran the fix my network wizard which deleted my receive connector for external email. There should be 3 receive connectors by default - one for sharepoint, one for internal users to use for sending mail and internal mail, and one for receiving external email from the outside.
This is a proper looking set of receive connectors:
This is what the internal connector should look like for internal usage:
This is what a proper external/internet receive connector should look like: