As part of Project Rhino, I’ve been setting up Hadoop along with Oozie to run our ETL pipeline.
Oozie has a cool feature that will send you an email as part of a job flow. However the SMTP setup does not seem to support TLS (PK encryption for SMTP) which GMail and Outlook.com / Live.com require.
What I did was setup a Postfix email relay on one of the servers. This allows for Oozie to communicate unencrypted with the local SMTP server. Then Postfix sends the mail on to the actual SMTP server encrypted.
The team uses outlook.com to host the email for our domain (it’s free!). However this setup should work for any email provider that requires TLS.
Install postfix:
apt-get install postfix
Then make a backup of your configuration (/etc/postfix/main.cf) and change it
to:
/etc/postfix/main.cf
# The first hop server (change to smtp.gmail.com for GMail)
relayhost = [smtp.live.com]:587
smtp_sasl_auth_enable = yes
# Location of the password database.
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
# CAs to trusted when verifying server certificate
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
# This trick is from
# http://mhawthorne.net/posts/postfix-configuring-gmail-as-relay.html
smtp_sasl_security_options =
Next we need to setup our authentication. We use oozie@ could be changed to
any valid account you have. Make sure this matches the from field you set in the
Oozie config later.
/etc/postfix/sasl_passwd:
[smtp.live.com]:587 [email protected]:supersecret
Next we need to run this command to build the password DB:
postmap /etc/postfix/sasl_passwd
Then we can reload postfix:
/etc/init.d/postfix reload
You may also need to change the permissions of the password files.
sudo chown postfix /etc/postfix/sasl_passwd*
When you are looking at the Oozie config, you’ll need to set the
oozie.email.from.address to match the one you put in the Postfix
configuration.
Good luck!