I needed to be able to send emails from a stored procedure using something like

EXECUTE msdb.dbo.sp_send_dbmail @recipients='address@domainname.com',
@subject = 'Test email',
@body = 'Test email body'

The solution was to add the user I was using to run the above query to the msdb database and to assign the role DatabaseMailUserRole to the user:

EXEC msdb.dbo.sp_addrolemember @rolename = 'DatabaseMailUserRole'
    ,@membername = 'myUserName';