Please check SiteController.php: actionContact(). It's using PHP mail() function. If that doesn't work for you, you may need to check you PHP configuration or use some other mail functions/packages.
Thanks Qiang! The form works, looks like my issue was sleep deprivation. I had two setups a remote and local. The issue on the remote server was I'd forgotten to set the admin email in the config/main.php. The issue on the local server was the mail server was setup incorrectly.
FYI if you're running a local dev server on a linux box, you need to relay through a public mail sever that you have SMTP credentials for, as it turns out most dynamically assigned ip's are on the Spamhaus Policy Block List (PBL).