Downloading
the Mail
Downloading
the mail is perhaps the most complex part of this exercise but take
it one step at a time, Gentle Reader, and it becomes quite logical.
First step is to download and setup the required SSL certificates,
then to setup fetchmail and
finally to setup procmail.
Download the SSL Certificates
The
debacle of the Gmail "expired certificate" saga of August
2008 has prompted me to completely rewrite this section and to
reconsider the method I have advocated in the past. I will
demonstrate a method to create your own certificate pack directly
from the Mozilla's source tree using Daniel Stenberg's amazing perl
script and then extract the required certificates from this
certificate pack. This script is part of the newest version of cURL
but I give the directions here to download it from gitweb for those
like myself who are on older versions. To run this script you will
need perl installed as well as the perl-libwww modules
which can be easily picked up by installing Bundle::LWP:
$ mkdir -pv $HOME/.certs
$ cd $HOME/.certs
$ touch Thawte_Premium_Server_CA.pem
$ touch Equifax_Secure_CA.pem
$ wget http://github.com/bagder/curl/raw/master/lib/mk-ca-bundle.pl
$ perl mk-ca-bundle.pl
This will generate a full certificate pack named ca-bundle.crt. We need to extract two certificates, the first is the "Thawte Premium Server CA" certificate which should be saved in the file we have just created:Thawte_Premium_Server_CA.pem. I show the certificate here for any who has trouble with the perl script, although it is always best to generate the certificate yourself:
-----BEGIN CERTIFICATE-----
MIIDJzCCApCgAwIBAgIBATANBgkqhkiG9w0BAQQFADCBzjELMAkGA1UEBhMCWkExFTATBgNVBAgT
DFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3duMR0wGwYDVQQKExRUaGF3dGUgQ29uc3Vs
dGluZyBjYzEoMCYGA1UECxMfQ2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjEhMB8GA1UE
AxMYVGhhd3RlIFByZW1pdW0gU2VydmVyIENBMSgwJgYJKoZIhvcNAQkBFhlwcmVtaXVtLXNlcnZl
ckB0aGF3dGUuY29tMB4XDTk2MDgwMTAwMDAwMFoXDTIwMTIzMTIzNTk1OVowgc4xCzAJBgNVBAYT
AlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEdMBsGA1UEChMU
VGhhd3RlIENvbnN1bHRpbmcgY2MxKDAmBgNVBAsTH0NlcnRpZmljYXRpb24gU2VydmljZXMgRGl2
aXNpb24xITAfBgNVBAMTGFRoYXd0ZSBQcmVtaXVtIFNlcnZlciBDQTEoMCYGCSqGSIb3DQEJARYZ
cHJlbWl1bS1zZXJ2ZXJAdGhhd3RlLmNvbTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA0jY2
aovXwlue2oFBYo847kkEVdbQ7xwblRZH7xhINTpS9CtqBo87L+pW46+GjZ4X9560ZXUCTe/LCaIh
Udib0GfQug2SBhRz1JPLlyoAnFxODLz6FVL88kRu2hFKbgifLy3j+ao6hnO2RlNYyIkFvYMRuHM/
qgeN9EJN50CdHDcCAwEAAaMTMBEwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQQFAAOBgQAm
SCwWwlj66BZ0DKqqX1Q/8tfJeGBeXm43YyJ3Nn6yF8Q0ufUIhfzJATj/Tb7yFkJD57taRvvBxhEf
8UqwKEbJw8RCfbz6q1lu1bdRiBHjpIUZa4JMpAwSremkrj/xw0llmozFyD4lt5SZu5IycQfwhl7t
UCemDaYj+bvLpgcUQg==
-----END CERTIFICATE-----
The second is the "Equifax Secure CA" certificate which should be saved in the file we have just created:Equifax_Secure_CA.pem. Again I show the certificate here for any who has struggled with the perl script:
-----BEGIN CERTIFICATE-----
MIIDIDCCAomgAwIBAgIENd70zzANBgkqhkiG9w0BAQUFADBOMQswCQYDVQQGEwJVUzEQMA4GA1UE
ChMHRXF1aWZheDEtMCsGA1UECxMkRXF1aWZheCBTZWN1cmUgQ2VydGlmaWNhdGUgQXV0aG9yaXR5
MB4XDTk4MDgyMjE2NDE1MVoXDTE4MDgyMjE2NDE1MVowTjELMAkGA1UEBhMCVVMxEDAOBgNVBAoT
B0VxdWlmYXgxLTArBgNVBAsTJEVxdWlmYXggU2VjdXJlIENlcnRpZmljYXRlIEF1dGhvcml0eTCB
nzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAwV2xWGcIYu6gmi0fCG2RFGiYCh7+2gRvE4RiIcPR
fM6fBeC4AfBONOziipUEZKzxa1NfBbPLZ4C/QgKO/t0BCezhABRP/PvwDN1Dulsr4R+AcJkVV5MW
8Q+XarfCaCMczE1ZMKxRHjuvK9buY0V7xdlfUNLjUA86iOe/FP3gx7kCAwEAAaOCAQkwggEFMHAG
A1UdHwRpMGcwZaBjoGGkXzBdMQswCQYDVQQGEwJVUzEQMA4GA1UEChMHRXF1aWZheDEtMCsGA1UE
CxMkRXF1aWZheCBTZWN1cmUgQ2VydGlmaWNhdGUgQXV0aG9yaXR5MQ0wCwYDVQQDEwRDUkwxMBoG
A1UdEAQTMBGBDzIwMTgwODIyMTY0MTUxWjALBgNVHQ8EBAMCAQYwHwYDVR0jBBgwFoAUSOZo+SvS
spXXR9gjIBBPM5iQn9QwHQYDVR0OBBYEFEjmaPkr0rKV10fYIyAQTzOYkJ/UMAwGA1UdEwQFMAMB
Af8wGgYJKoZIhvZ9B0EABA0wCxsFVjMuMGMDAgbAMA0GCSqGSIb3DQEBBQUAA4GBAFjOKer89961
zgK5F7WF0bnj4JXMJTENAKaSbn+2kmOeUJXRmm/kEd5jhW6Y7qj/WsjTVbJmcVfewCHrPSqnI0kB
BIZCe/zuf6IWUrVnZ9NA2zsmWLIodz2uFHdh1voqZiegDfqnc1zqcPGUIWVEX/r87yloqaKHee95
70+sB3c4
-----END CERTIFICATE-----
And finally to rehash the certificates so Openssl knows how to find them:
$ c_rehash $HOME/.certs/
I should mention as well that many distros have a bundle of CA certificates with a package name something like "ca-certificates" that can also be used in place of these reasonably complex instructions. I shall point out how this bundle can be utilised further on in the guide. This completes the SSL configuration and now onto lighter matters with the required steps to setup fetchmail.
Setting up fetchmail
Fetchmail is
fairly easy to use and setup, particularly as the hard yards have
already been done with the SSL certificates. You will need to create
the file $HOME/.fetchmailrc and then add the following
information for accessing the Gmail
server:
poll pop.gmail.com
with proto POP3
user 'shankar.example'
there with password 'ishvar'
is 'shankar' here
mda "/usr/bin/procmail -d %T"
options
no keep
ssl
sslcertck
sslcertpath /home/shankar/.certs/
It is almost too obvious to point out, Gentle Reader, that you will need to substitute your own details for password, username, email address etc. Those who have installed a CA certificate bundle through their distro's repository will need to change the sslcertpath setting to somehing like /etc/ssl/certs according to the installation path of the certificate bundle. There remains a final touch, since the username and password are openly in this file you should make the file readable only by the file owner:
$ chmod 600 ~/.fetchmailrc
Now would be a good to time also to make sure you have POP forwarding enabled in your Gmail account. You will find this in: Settings - Forwarding and POP at the web interface of Gmail. Note as well that you cannot remove mail from Gmail servers via POP3 but you can choose to have your messages archived, kept or deleted once they have been downloaded via POP3. This is a Gmail setting hidden in Settings - Forwarding and POP: "When messages are accessed with POP..."
poll pop.gmail.com
with proto POP3
user 'shankar.example'
there with password 'ishvar'
is 'shankar' here
mda "/usr/bin/procmail -d %T"
options
no keep
ssl
sslcertck
sslcertpath /home/shankar/.certs/
It is almost too obvious to point out, Gentle Reader, that you will need to substitute your own details for password, username, email address etc. Those who have installed a CA certificate bundle through their distro's repository will need to change the sslcertpath setting to somehing like /etc/ssl/certs according to the installation path of the certificate bundle. There remains a final touch, since the username and password are openly in this file you should make the file readable only by the file owner:
$ chmod 600 ~/.fetchmailrc
Now would be a good to time also to make sure you have POP forwarding enabled in your Gmail account. You will find this in: Settings - Forwarding and POP at the web interface of Gmail. Note as well that you cannot remove mail from Gmail servers via POP3 but you can choose to have your messages archived, kept or deleted once they have been downloaded via POP3. This is a Gmail setting hidden in Settings - Forwarding and POP: "When messages are accessed with POP..."
Setting up procmail
procmail
is the final link in the chain for downloading mail. Before setting
it up there is a little bit of outstanding business to attend to:
setting the $MAIL environmental
variable so that all software
that deals with mail knows where the delivery point is. Add the
following to ~/.bashrc making the
obvious substitution for username:
# Sets the Mail Environment Variable
MAIL=/var/spool/mail/shankar && export MAIL
procmail will now know the location of the default mail spool and will deliver all mail to there that has not been sorted to other locations. But the creation of a $HOME/.procmailrc is still required and I give an example below. I have also added a sample filtering recipe for the mutt user mailing list to give the very beginning of what can be a complex process:
PATH=/bin:/usr/bin:/usr/local/bin
VERBOSE=off
DEFAULT=/var/spool/mail/shankar
MAILDIR=$HOME/mail
LOGFILE=$HOME/.procmaillog
# Recipes below this comment:
:0:
* ^TOmutt-user
mutt
The single filtering recipe means that procmail delivers all email addressed to "mutt-user" directly to $HOME/mail/mutt. Allother mail goes to the default location /var/spool/mail/username as specified by $MAIL environmental variable. If for some reason you are averse to adding this to ~/.bashrc, Gentle Reader, I have demonstrated above how to add this to~/.procmailrc with the DEFAULT setting. For the definitive guide to further recipes, and guidance on much more complex recipes, don't forget to run man procmailex, it all starts to make sense after a while.
# Sets the Mail Environment Variable
MAIL=/var/spool/mail/shankar && export MAIL
procmail will now know the location of the default mail spool and will deliver all mail to there that has not been sorted to other locations. But the creation of a $HOME/.procmailrc is still required and I give an example below. I have also added a sample filtering recipe for the mutt user mailing list to give the very beginning of what can be a complex process:
PATH=/bin:/usr/bin:/usr/local/bin
VERBOSE=off
DEFAULT=/var/spool/mail/shankar
MAILDIR=$HOME/mail
LOGFILE=$HOME/.procmaillog
# Recipes below this comment:
:0:
* ^TOmutt-user
mutt
The single filtering recipe means that procmail delivers all email addressed to "mutt-user" directly to $HOME/mail/mutt. Allother mail goes to the default location /var/spool/mail/username as specified by $MAIL environmental variable. If for some reason you are averse to adding this to ~/.bashrc, Gentle Reader, I have demonstrated above how to add this to~/.procmailrc with the DEFAULT setting. For the definitive guide to further recipes, and guidance on much more complex recipes, don't forget to run man procmailex, it all starts to make sense after a while.
Sending the Mail
I
have formerly been an advocate for the simple MTA ssmtp, which some
would call a Mail Sending Agent (MSA). However I believe that
development of ssmtp has ceased and I have moved with some regret on
to msmtp. A single configuration file is required for
msmtp: $HOME/.msmtprc and the following section gives the
required details to access Gmail and reference the required
certificate:
account default
host smtp.gmail.com
port 587
from shankar.example@gmail.com
tls on
tls_starttls on
# tls_trust_file /home/shankar/.certs/Thawte_Premium_Server_CA.pem
tls_trust_file /home/shankar/.certs/Equifax_Secure_CA.pem
auth on
user shankar.example
password ishvar
logfile ~/.msmtp.log
I need not mention again, Gentle Reader, that there should be some fairly obvious changes here to substitute your ownusername, password and email address? For those who have installed a bundle of CA certificates from their distro's repository the tls_trust_file setting will need to be changed to something like /etc/ssl/certs/ca-certificates.crtdepending on the exact installation path of the certificate bundle. And then the final touch, since the username and password are openly in this file, you should make the file readable only by the file owner:
$ chmod 600 ~/.msmtprc
msmtp is a great program that has many features that quite frankly I am still exploring, feel free to point out anything that I have missed, there is an email link at the base of this page for that purpose. But now finally to Mutt:
account default
host smtp.gmail.com
port 587
from shankar.example@gmail.com
tls on
tls_starttls on
# tls_trust_file /home/shankar/.certs/Thawte_Premium_Server_CA.pem
tls_trust_file /home/shankar/.certs/Equifax_Secure_CA.pem
auth on
user shankar.example
password ishvar
logfile ~/.msmtp.log
I need not mention again, Gentle Reader, that there should be some fairly obvious changes here to substitute your ownusername, password and email address? For those who have installed a bundle of CA certificates from their distro's repository the tls_trust_file setting will need to be changed to something like /etc/ssl/certs/ca-certificates.crtdepending on the exact installation path of the certificate bundle. And then the final touch, since the username and password are openly in this file, you should make the file readable only by the file owner:
$ chmod 600 ~/.msmtprc
msmtp is a great program that has many features that quite frankly I am still exploring, feel free to point out anything that I have missed, there is an email link at the base of this page for that purpose. But now finally to Mutt:
Reading the Mail
June
14th 2009 saw the release of a "Development" version of
Mutt: 1.5.20 and it is this version that I have used for this page.
Although labelled as a pre-release version I have found
it very stable
and I suspect you will as well. Mutt is driven by the
file$HOME/.muttrc and I spent some time
building this file from scratch but for you, Gentle Reader, I include
here a morebasic version,
similar to the one I started from. Some parts of this, such as
aliases and colors, are sourced from their own file so don't forget
to create these
files.
#======================================================#
# Boring details
set realname = "shankar"
set from = "shankar.example@gmail.com"
set use_from = yes
set envelope_from ="yes"
# Use a signature
set signature="~/.signature"
# Use msmtp rather than sendmail. Check that
# the path is correct for your system:
set sendmail="/usr/bin/msmtp"
# If not set in ~/.bashrc:
set spoolfile = /var/spool/mail/shankar
#======================================================#
# Folders
set folder="$HOME/mail" # Local mailboxes stored here
set record="+sent" # Where to store sent messages
set postponed="+postponed" # Where to store draft messages
set mbox_type=mbox # Mailbox type
set move=no # Don't move mail from spool
#======================================================#
# Watch these mailboxes for new mail, useful only if
# Procmail or Maildrop is used to sort mail.
mailboxes ! +slrn +fetchmail +mutt
set sort_browser=alpha # Sort mailboxes by alpha(bet)
#======================================================#
# What to show and order of headers
ignore *
unignore Date: From: User-Agent: X-Mailer X-Operating-System To: \
Cc: Reply-To: Subject: Mail-Followup-To:
hdr_order Date: From: User-Agent: X-Mailer X-Operating-System To: \
Cc: Reply-To: Subject: Mail-Followup-To:
#======================================================#
# which editor do you want to use?
# vim of course!
set editor="vim -c 'set tw=70 et' '+/^$' "
set edit_headers=yes # See the headers when editing
#======================================================#
# Aliases
set alias_file = ~/mail/mutt_aliases # In their own file
source ~/mail/mutt_aliases # Source them
set sort_alias=alias # Sort alphabetically
#======================================================#
# Colours: defaults are a little bleak so experiment!
source ~/mutt/mutt_colors # In their own file
#======================================================#
# Lists: An example using the mutt-users list:
lists mutt-users
subscribe mutt-users
set followup_to=yes # Sets 'Mail-Followup-To' header
set honor_followup_to=yes
fcc-hook mutt-user +mutt # See your own posts using fcc
#======================================================#
# Odds and ends
set markers # mark wrapped lines of text in the pager with a +
set smart_wrap # Don't wrap mid-word
set pager_context=5 # Retain 5 lines of previous page when scrolling.
set status_on_top # Status bar on top.
push <show-version> # Shows mutt version at startup
I have been taken to task somewhat by one reader of this page who felt there should be a little more information about Mutt and colors and so ...
#======================================================#
# Boring details
set realname = "shankar"
set from = "shankar.example@gmail.com"
set use_from = yes
set envelope_from ="yes"
# Use a signature
set signature="~/.signature"
# Use msmtp rather than sendmail. Check that
# the path is correct for your system:
set sendmail="/usr/bin/msmtp"
# If not set in ~/.bashrc:
set spoolfile = /var/spool/mail/shankar
#======================================================#
# Folders
set folder="$HOME/mail" # Local mailboxes stored here
set record="+sent" # Where to store sent messages
set postponed="+postponed" # Where to store draft messages
set mbox_type=mbox # Mailbox type
set move=no # Don't move mail from spool
#======================================================#
# Watch these mailboxes for new mail, useful only if
# Procmail or Maildrop is used to sort mail.
mailboxes ! +slrn +fetchmail +mutt
set sort_browser=alpha # Sort mailboxes by alpha(bet)
#======================================================#
# What to show and order of headers
ignore *
unignore Date: From: User-Agent: X-Mailer X-Operating-System To: \
Cc: Reply-To: Subject: Mail-Followup-To:
hdr_order Date: From: User-Agent: X-Mailer X-Operating-System To: \
Cc: Reply-To: Subject: Mail-Followup-To:
#======================================================#
# which editor do you want to use?
# vim of course!
set editor="vim -c 'set tw=70 et' '+/^$' "
set edit_headers=yes # See the headers when editing
#======================================================#
# Aliases
set alias_file = ~/mail/mutt_aliases # In their own file
source ~/mail/mutt_aliases # Source them
set sort_alias=alias # Sort alphabetically
#======================================================#
# Colours: defaults are a little bleak so experiment!
source ~/mutt/mutt_colors # In their own file
#======================================================#
# Lists: An example using the mutt-users list:
lists mutt-users
subscribe mutt-users
set followup_to=yes # Sets 'Mail-Followup-To' header
set honor_followup_to=yes
fcc-hook mutt-user +mutt # See your own posts using fcc
#======================================================#
# Odds and ends
set markers # mark wrapped lines of text in the pager with a +
set smart_wrap # Don't wrap mid-word
set pager_context=5 # Retain 5 lines of previous page when scrolling.
set status_on_top # Status bar on top.
push <show-version> # Shows mutt version at startup
I have been taken to task somewhat by one reader of this page who felt there should be a little more information about Mutt and colors and so ...
Coloring in Mutt
If
your terminal supports color, and I believe it is a rarity these days
for this not to be case, you can
color almost any aspect of the Mutt window. All the fine details are
in the Mutt manual: "Section 3: Configuration 8: Using color and
mono video attributes". But I can tell you, Gentle Reader, that
the basic usage is:
color object foreground background
This can be made a lot more compilcated but a simple start is a good start. The basic colors are white, black, green, magenta, blue, cyan, yellow, red and default but you can also prefix a foreground color with "bright" to make the color bold. Now I personally use a Terminal with a white background so my own colors are:
#---- Mutt Colors for White Background -------
color hdrdefault black default
color quoted red default
color signature brightblack default
color indicator brightwhite red
color attachment black default
color error red default
color message blue default
color search brightwhite magenta
color status brightyellow blue
color tree red default
color normal blue default
color tilde green default
color bold brightyellow default
color markers red default
There should also be a file called colors.default installed as part of your Mutt installation which is intended for Terminals with white backgrounds, this might also furnish a starting point as it did once for me. Just to balance out the equation a little I will also give the colors for those who prefer a black background. This example is taken directly from another sample filecolors.linux which should be installed along with Mutt:
#---- Mutt Colors for Black Background -------
color hdrdefault blue black
color quoted blue black
color signature blue black
color attachment red black
color message brightred black
color error brightred black
color indicator black red
color status brightgreen blue
color tree white black
color normal white black
color markers red black
color search white black
color tilde brightmagenta black
color index blue black ~F
color index red black "~N|~O"
color object foreground background
This can be made a lot more compilcated but a simple start is a good start. The basic colors are white, black, green, magenta, blue, cyan, yellow, red and default but you can also prefix a foreground color with "bright" to make the color bold. Now I personally use a Terminal with a white background so my own colors are:
#---- Mutt Colors for White Background -------
color hdrdefault black default
color quoted red default
color signature brightblack default
color indicator brightwhite red
color attachment black default
color error red default
color message blue default
color search brightwhite magenta
color status brightyellow blue
color tree red default
color normal blue default
color tilde green default
color bold brightyellow default
color markers red default
There should also be a file called colors.default installed as part of your Mutt installation which is intended for Terminals with white backgrounds, this might also furnish a starting point as it did once for me. Just to balance out the equation a little I will also give the colors for those who prefer a black background. This example is taken directly from another sample filecolors.linux which should be installed along with Mutt:
#---- Mutt Colors for Black Background -------
color hdrdefault blue black
color quoted blue black
color signature blue black
color attachment red black
color message brightred black
color error brightred black
color indicator black red
color status brightgreen blue
color tree white black
color normal white black
color markers red black
color search white black
color tilde brightmagenta black
color index blue black ~F
color index red black "~N|~O"
If you don't like either of these feel free to experiment a little and come up with your own, most Mutt users will alter these basic colors. I prefer mine simple but if you pull out the manual you will see that you can spend many hours getting it exactly as you want.
No comments:
Post a Comment