SMTP Email Relay for GMail (TLS) with Oozie Using Postfix

As part of Project Rhino, I’ve been setting up Hadoop along with Oozie to run our ETL pipeline.

Oozie has a cool feature that will send you an email as part of a job flow. However the SMTP setup does not seem to support TLS (PK encryption for SMTP) which GMail and Outlook.com / Live.com require.

What I did was setup a Postfix email relay on one of the servers. This allows for Oozie to communicate unencrypted with the local SMTP server. Then Postfix sends the mail on to the actual SMTP server encrypted.

The team uses outlook.com to host the email for our domain (it’s free!). However this setup should work for any email provider that requires TLS.

Postfix Setup

Install postfix:

apt-get install postfix

Then make a backup of your configuration (/etc/postfix/main.cf) and change it to:

/etc/postfix/main.cf

# The first hop server (change to smtp.gmail.com for GMail)
relayhost = [smtp.live.com]:587
smtp_sasl_auth_enable = yes 

# Location of the password database.
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd

# CAs to trusted when verifying server certificate
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt

# This trick is from
# http://mhawthorne.net/posts/postfix-configuring-gmail-as-relay.html
smtp_sasl_security_options =

Next we need to setup our authentication. We use [email protected] could be changed to any valid account you have. Make sure this matches the from field you set in the Oozie config later.

/etc/postfix/sasl_passwd:

[smtp.live.com]:587  [email protected]:supersecret

Next we need to run this command to build the password DB:

postmap /etc/postfix/sasl_passwd

Then we can reload postfix:

/etc/init.d/postfix reload

You may also need to change the permissions of the password files.

sudo chown postfix /etc/postfix/sasl_passwd*

Configuring Oozie

When you are looking at the Oozie config, you’ll need to set the oozie.email.from.address to match the one you put in the Postfix configuration.

Good luck!