The Ultimate Guide to Mutt

Introduction to Email System

On the picture below we can see basic overview we'll be setting up in this tutorial. When working with Mutt, we have to take care of sending and receiving emails separately, since different servers are used for those. The overview can be seen on the picture below.

mutt_overview.png

When sending an email to the SMTP server, we have to use SMTP client, which is msmtp in our case; it authenticates to the SMTP server and sends the email. We need to be careful to authenticate over secure TLS connection, which we'll configure later on. Mutt can either send the email by itself or contact an external program like msmtp to do the job for him.

When receiving an email, we can use two protocols for the job: POP3 or IMAP. We really should use IMAP, which copies the messages to local computer and leaves the copy on the server intact. POP3 protocol downloads the messages to local computer and deletes the messages on upstream mail server. When using IMAP protocol we can use any of the following external programs: offlineimap, getmail or fetchmail. In our case we'll be using offlineimap program, which needs to be running in the background to constantly connect to the IMAP server and check whether any new emails are available. Note that IMAP client must also connect over the secure TLS connection prior to authentication. The IMAP client basically copies the email messages to local Maildir directory for each defined email account.

At the end of the day, we also want to have a notification system that informs us about the arrival of a new email. Since the offlineimap doesn't inform us when new email arrives, we need some kind of notification system letting us know about new email messages. We'll see about this later on in the article.

Let's also describe the terminology used when talking about mail systems:

  • MUA (Mail User Agent): interacts with the MDA to receive emails, which it presents to the user. It also receives user interactions and executes them appropriately. A MUA can be installed locally on the client's machine as email client like Mozilla Thunderbird, Microsoft Outlook or Mutt, or it can be accessible through web interface like Gmail, Outlook, Yahoo, etc.
  • MTA (Mail Transfer Agent): used for sending the email through the Internet. A MTA agent communicates with other MTA agents by using SMTP protocol. The MTA agent listens on port 25 (unencrypted) or port 587 (encrypted) where it receives SMTP messages from other MTA agents. We need to properly protect our MTA agent (SMTP server) so only authenticated requests are processed.
  • MDA (Mail Delivery Agent): used for receiving the email through the Internet. A MDA agent receives emails from MTAs and stores them locally waiting for users to access them. A user can access emails by using either POP3 or IMAP protocol. The MDA is needed so clients don't need to be connected all the time to receive emails. Imagine what would happen if Internet connection goes down because of ISP maintenance and a client will not be able to download emails for a couple of hours. When the Internet connection is reestablished, the client still won't be able to receive the emails, because they were not stored anywhere: this is what MDA is for; it stores messages locally, so the client can access them at any time.

When sending an email, the MTA agents looks for MX records of the domain to determine the SMTP server to which it must forward the sending email. We can execute the following dig command to determine the MX servers to which we have to send email for gmail.com domain.

# dig -t MX google.com

; <<>> DiG 9.9.3-P2 <<>> -t MX google.com
;; global options: +cmd
;; ->>HEADER< ;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0,
ADDITIONAL: 5

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;google.com. IN MX

;; ANSWER SECTION:
google.com. 600 IN MX 10 aspmx.l.google.com.
google.com. 600 IN MX 20 alt1.aspmx.l.google.com.
google.com. 600 IN MX 50 alt4.aspmx.l.google.com.
google.com. 600 IN MX 40 alt3.aspmx.l.google.com.
google.com. 600 IN MX 30 alt2.aspmx.l.google.com.

;; Query time: 27 msec
;; SERVER: 10.10.100.1#53(10.10.100.1)
;; WHEN: Sat Jan 11 14:05:02 CET 2014
;; MSG SIZE rcvd: 147

Notice that DNS request returned five MX records with priority numbers 10, 20, 30, 40 and 50. The MX record with the lower priority number has the maximum priority and will be used first in email delivery process.

Introduction to Gnome Keyring

For those of you that don't know what Gnome Keyring is, it's a daemon, which holds credentials in encrypted form, so we don't need to store them in plaintext files. The applications that have support for Gnome Keyring can use it to provide passwords when needed, for example: when msmtp wants to send an email, it must connect to SMTP server by providing the correct username and password, after which it can send the email. The msmtp can use a plaintext password it has stored in a configuration file or request the password from Gnome Keyring. Most of the time we don't want to store passwords in configuration files, so if the application supports the Gnome Keyring interaction, we should use it.

The first thing that we need to know is that we can use seahorse as a GUI for managing Gnome Keyring. On the picture below, we can see stored passwords, ssh and pgp keys.

seahorse1.png

Introduction to Seahorse

The seahorse is used to store user credentials in an encrypted database called keyring. If we would like to create a new password, we must click on the File - New - Password Keyring as presented on the picture below.

seahorse2.png

We need to choose to name the keyring, where we entered the string test after which we must provide a password for the keyring. That is presented on the picture below.

seahorse3.png

Once completed, we can notice a new test entry in the menu on the left. The keyring is currently empty.

seahorse4.png

We can add a new password to the keyring by pressing on the New button and selecting the Stored Password and filling in the keyring to which we would like to add the password as well as description and the actual password.

seahorse5.png

After adding the password, we can see it in the seahorse and by double-clicking on it, where the password is shown, because the keyring has already been unlocked.

seahorse6.png

Sending the Email

Msmtp Introduction

When we would like to send an email, we usually want to use SMTP client, which in our case is msmtp. First we need to install msmtp, which we should do with our default package manager. The msmtp program reads the ~/.msmtprc configuration file, where the SMTP server, username, password and other stuff are defined.

Normal configuration file contains the configuration variables as presented below. The first part of the configuration file defines default account that uses smtp.gmail.com SMTP server and port 587. It authenticates the user name.surname with password mypassword to the SMTP server. After that we're defining that we want to use TLS connection, so credentials are send over encrypted tunnel. At last, there are some logging options in case something goes wrong, so we can debug the error further.

# Account
account default
host smtp.gmail.com
port 587
auth on
user name.surname@gmail.com
from name.surname@gmail.com
password mypassword

# Enable TLS
tls on
tls_certcheck on
tls_trust_file /etc/ssl/certs/ca-certificates.crt

# Logging
syslog LOG_MAIL
logfile /home/user/msmtp.log

Sending the Email

In this example, we'll be sending an email to ourselves. To do that, we must define a new file mail.txt with the following contents.

To: <name.surname@gmail.com>
From: "Name Surname" <name.surname@gmail.com>
Subject: Test

Testing if msmtp works.

To actually send the email to the SMTP server, which will relay it to where it needs to go, we have to pipe the content of the mail.txt file to msmtp program.

# cat mail.txt | msmtp -a default name.surname@gmail.com

Notice that we used the default account, which we previously defined in ~/.msmtprc configuration file? At the same time as sending the mail, we can also tail the contents of the /home/user/msmtp.log log file to see what's going on.

# tail -f /home/user/msmtp.log

When piping the email to the msmtp command, a new entry should appear in the msmtp.log at the time of sending the email, which looks like presented below. The message has existed with a status EX_OK, which means everything is ok and the message was delivered successfully. If we login to our own mailbox, we should see this email in our Inbox.

Jan 06 20:20:53 host=smtp.gmail.com tls=on auth=on user=name.surname@gmail.com from=name.surname@gmail.com recipients=name.surname@gmail.com mailsize=112 smtpstatus=250 smtpmsg='250 2.0.0 OK 1389036172 n1sm173566001eep.20 - gsmtp' exitcode=EX_OK

This is all fine and all, but we have a password stored in a configuration file in clear-text form. This is something we don't want, since it's not very secure and anyone gaining access to our computer would also be able to read the password. Later on, we'll see how we can secure this process by using Gnome Keyring.

If you ever need to debug the msmtp, you can run the msmtp program with --debug=masmtlog parameter as follows. It will display whole communication between the MX server, which can shed some light into the problem when something is not working.

# cat mail.txt | msmtp --debug=masmtlog -a default name.surname@gmail.com

Multiple Accounts

We can also use multiple accounts with msmtp client. All we need to do is define another account section on the ~/.msmtprc configuration file. The code below defines two email accounts first and second.

account first
host smtp.gmail.com
port 587
auth on
user name1@gmail.com

account second
host smtp.gmail.com
port 587
auth on
user name2@gmail.com

When we want to send an email by using either of the defined accounts, we need to pass the account name with the -a option to msmtp program. Instead of using default we can use first and second as follows.

# msmtp -a first name.surname@gmail.com
# msmtp -a second name.surname@gmail.com

Receiving the Email

When we would like to setup the receiving process with Mutt, we can choose between multiple clients, which are outlined below. In the brackets I presented the protocols we can use with the each of the programs.

  • offlineimap (IMAP)
  • getmail (IMAP, POP3)
  • fetchmail (IMAP, POP3)

Introduction to OfflineIMAP

In this tutorial we'll use offlineimap, which can be easily installed with our default package manager. It can be used to synchronize remote IMAP server with local maildir. We need to copy the offlineimap.conf configuration file from /usr/share/offlineimap/ directory to ~/.offlineimaprc. A simple configuration file is presented below.

[general]
# List accounts to be synced.
accounts = main

[mbnames]
enabled = no
filename = ~/Mutt/muttrc.mailboxes
header = "mailboxes "
sep = " "
footer = "n"

#
# Accounts
# localrepository : the maildir to be synced via IMAP (local).
# remoterepository : the remote IMAP to be synced (remote).
# status_backend : status cache: plain, sqlite.
#
[Account main]
localrepository = local_main
remoterepository = remote_main
status_backend = sqlite

#
# Repository
# type : local repository can be: Maildir, IMAP, Gmail.
# localfolders : the folder where to save the emails.
# remotehost : the remote host of IMAP server.
# remoteuser : the remote user of IMAP server.
# ssl : use SSL connection on port 993, otherwise 143.
[Repository local_main]
type = Maildir
localfolders = ~/Mail/

[Repository remote_main]
type = Gmail
ssl = yes
remoteuser = name.surname@gmail.com
remotepass = mypassword
cert_fingerprint = 89091347184d41768bfc0da9fad94bfe882dd358

Most of the configuration variables are self-explanatory, but there's also an interesting option cert_fingerprint, which specifies the SHA1 hash of IMAP public server certificate. To calculate the SHA1 of the imap.gmail.com certificate, we have to run the gnutls-cli command as follows.

# gnutls-cli -p 993 imap.gmail.com
Resolving 'imap.gmail.com'...
Connecting to '173.194.70.108:993'...
- Certificate type: X.509
- Got a certificate list of 3 certificates.
- Certificate[0] info:
- subject `C=US,ST=California,L=Mountain View,O=Google
Inc,CN=imap.gmail.com', issuer `C=US,O=Google Inc,CN=Google Internet
Authority G2', RSA key 2048 bits, signed using RSA-SHA
1, activated `2013-09-10 07:59:51 UTC', expires `2014-09-10
07:59:51 UTC', SHA-1 fingerprint
`89091347184d41768bfc0da9fad94bfe882dd358'
- Certificate[1] info:
- subject `C=US,O=Google Inc,CN=Google Internet Authority G2',
issuer `C=US,O=GeoTrust Inc.,CN=GeoTrust Global CA', RSA key 2048 bits,
signed using RSA-SHA1, activated `2013-04-0
5 15:15:55 UTC', expires `2015-04-04 15:15:55 UTC', SHA-1
fingerprint `d83c1a7f4d0446bb2081b81a1670f8183451ca24'
- Certificate[2] info:
- subject `C=US,O=GeoTrust Inc.,CN=GeoTrust Global CA', issuer
`C=US,O=Equifax,OU=Equifax Secure Certificate Authority', RSA key 2048
bits, signed using RSA-SHA1, activated `2002
-05-21 04:00:00 UTC', expires `2018-08-21 04:00:00 UTC', SHA-1
fingerprint `7359755c6df9a0abc3060bce369564c8ec4542a3'
- The hostname in the certificate matches 'imap.gmail.com'.
- Peer's certificate issuer is unknown
- Peer's certificate is NOT trusted
- Version: TLS1.2
- Key Exchange: RSA
- Cipher: ARCFOUR-128
- MAC: SHA1
- Compression: NULL
- Handshake was completed
- Simple Client Mode

Notice that the gnutls-cli printed the SHA1 hashes. If we would like to be able to connect to the Gmail IMAP server, we need to include the cert_fingerprint line with the correct SHA1 in the configuration file. Otherwise offlineimap will terminate the connection, because the validity of the server cannot be verified.

cert_fingerprint = 89091347184d41768bfc0da9fad94bfe882dd358

After providing the right SHA1 hash, we need to synchronize the remote and local email addresses by running offlineimap.

# offlineimap

If everything is setup correctly, the offlineimap should synchronize the local and remote end of communication. Note that offlineimap merely downloads the emails from the IMAP server, but doesn't actually do anything with them. If we would like to read the synchronized emails, we can do so from local maildir: we can use any mail client for that, like mutt or gnus.

Notifications

When we switch from a GUI mail client to text-based mail client, we have to use a notification system that's letting us know about the unread emails.

We can use mail-notification program, which we can install by using our default package manager. After installation we can start the program by running /usr/bin/mail-notification and configure the Mailbox accounts by adding them into the mail-notification. This looks like presented on the picture below.

notification1.png

We defined three Mailboxes, where I've intentionally hidden the Mailbox paths, but you can get the point. You should define as many Mailboxes as you use on your system. On the picture you can also see the "Message Popups" tab where we can configure different things, like whether we would like to have message popups enabled and the expiration time for the message. This can be seen on the picture below.

notification2.png

Using Different Password Stores

Rather than storing a clear-text password in a configuration file, we can use different methods of providing the password to the email daemons/programs. We can use any of the following authentication mechanisms.

Plaintext Password

The passwords are stored at appropriate program configuration files in clear-text. This is what we've been using so far, and is the easiest to configure, but provides least security.

Netrc

We can store passwords in ~/.netrc file, where we need to specify every username with keyword login and password with keyword password. An example of specifying login credentials for the Gmail account name.surname@gmail.com can be seen below.

machine mail.google.com
login name.surname@gmail.com
password mypass

When using this option, don't forget to properly secure ~/.netrc file with chmod 600, so it's only readable and writeable by the current user.

# chmod 600 ~/.netrc

Python Keyring

When we want to use this option, we first need to install the right package for the python keyring. On my system I had to install gnome-keyring-python, but on your system it might have a different name like python-keyring or something else. Search your package repository for appropriate name and install it. Alternatively we can also install it with easy_install or pip.

# easy_install keyring
# pip install keyring

After that, we need to add the ~/.mutt/scripts/pythonkeyring.py and add import keyring to it.

# vim ~/.mutt/scripts/pythonkeyring.py
import keyring

This needs to be done in order to make the keyring available to offlineimap. Offlineimap supports the pythonfile configuration option, where we can specify a file that will be imported and used in offlineimap; since we only need access to the keyring, we only import that in pythonkeyring.py, but we could use any Python code. After that we need to edit the ~/.offlineimap and configure the pythonfile accordingly.

pythonfile = ~/.mutt/scripts/pythonkeyring.py
remotepasseval = keyring.get_password("offlineimap", "name.surname@gmail.com")

Notice that we're referencing the offlineimap keyring by asking for the password of name.surname@gmail.com account. After that we also need to add the password to the offlineimap keyring, which we can simply do by opening Python interpreter and calling the set_password function

# python
>>> import keyring
>>> keyring.set_password("offlineimap", "name.surname@gmail.com", "mypass")
Please enter password for encrypted keyring:
>>> keyring.get_password("offlineimap", "name.surname@gmail.com") u'mypass'

The module will store keyring config file as well as keyring itself in it's working directory, which can be found by executing the following command [7].

# python -c "import keyring.util.platform_; print(keyring.util.platform_.data_root())"
/home/user/.local/share/python_keyring

# ls /home/user/.local/share/python_keyring
crypted_pass.cfg

We can see only the keyring, but not the configuration file keyringrc.cfg. The crypted_pass.cfg contains the following after adding the password:

# cat /home/user/.local/share/python_keyring/crypted_pass.cfg
[offlineimap]
name_2esurname_40gmail_2ecom = eyJwYXNzd29yZF9lbmNyeXB0ZWQiOiAiNkxneXp6OXBTQ0ZpXG4iLCAic2FsdCI6ICJ3WWk4N2lZ UEp4Sncxa1ZiMnZwTXRISFpyUVgyL3ptayt4UDNaSGVMTExFPVxuIiwgIklWIjogIml0cHV1T2NR
enZnTUI3TmFOWWdkL1E9PVxuIn0=

Notice that the value of name.surname@gmail.com was base64 encoded and encrypted and only the master password can be used to decrypt it.

There are multiple backends that we can use for safely storing our passwords; note that each backend keyring must subclass KeyringBackend [7].

  • EncryptedFileKeyring : keyring.backends.file.BaseKeyring()
  • PlaintextKeyring : keyring.backends.pyfs.PlaintextKeyring()
  • KeyczarKeyring : keyring.backends.pyfs.KeyczarKeyring()
  • GnomeKeyring : keyring.backends.Gnome.Keyring()
  • GoogleKeyring : keyring.backends.Google.DocsKeyring()
  • KDEWallet : keyring.backends.kwallet.Keyring()
  • OSXKeychain : keyring.backends.OS_X.Keyring()
  • SecretServiceKeyring : keyring.backends.SecretService.Keyring()
  • WinVaultKeyring : keyring.backends.Windows.WinVaultKeyring()
  • RegistryKeyring : keyring.backends.Windows.RegistryKeyring()

The default keyring is the file keyring as we've already seen. We can confirm that by calling the get_keyring function as seen below.

# python
>>> import keyring
>>> keyring.get_keyring()

In case we're already using Gnome Keyring and would like to use that instead of the file keyring, we can call set_keyring to set the Gnome Keyring. After that we can call set_password and get_password the same way as before.

>>> keyring.set_keyring(keyring.backends.Gnome.Keyring())
>>> keyring.set_password("offlineimap", "name.surname@gmail.com", "mypass")
>>> keyring.get_password("offlineimap", "name.surname@gmail.com") u'mypass'

After setting the password in our keyring, we need to call offlineimap to download the messages from the server. In case we're using the Python Keyring, we'll be asked for the keyring password every time we run offlineimap as seen below.

# offlineimap
Account sync gmailname:
*** Processing account gmailname
Establishing connection to imap.gmail.com:993
Please enter password for encrypted keyring:

The offlineimap is asking us for the password of encrypted keyring, not for the password of our Gmail account. Once we enter the password of the keyring, it will start syncing the emails from the server. Since offlineimap will ask for the keyring password every time we restart it, this option is unacceptable for everyday usage. Besides that, there's also a problem of offlineimap not poping up a GUI window to input a password, which makes running offlineimap in crontab very hard.

But if we're using Gnome Keyring, a GUI popup is shown automatically, which requests a password to be inputted to unlock the keyring. Below we can see that offlineimap started syncing all the email messages automatically, because the keyring has already been unlocked.

# offlineimap
OfflineIMAP 6.5.4
Licensed under the GNU GPL v2+ (v2 or any later version)
Account sync gmailname:
*** Processing account gmailname
Establishing connection to imap.gmail.com:993
Folder [Gmail]/All Mail [acc: gmailname]:
Syncing [Gmail]/All Mail: IMAP -> Maildir

There's also an entry stored in the Gnome Keyring, which we can observe by starting Seahorse.

seahorse_offlineimap.png

Gnome Keyring

Despite talking about Gnome Keyring in the previous section, I think it deserves a special section of it's own, because it's very hard to get everything right. Besides, we also have to set-up the msmtp program to use Gnome Keyring for sending emails. If we don't, then we'll have to enter the password for our email account every time we'll want to send an email, which is a nuisance.

To add a password for msmtp into Gnome Keyring, we can use the msmtp-gnome-tool.py script, which is part of the msmtp package and is located in the msmtp-1.4.30/scripts/ directory. Note that you have to download the source code in order to get that script. When I downloaded the script and wanted to save my credentials into Gnome keyring I stumbled upon the following error:

# ./msmtp-gnome-tool.py --help
Traceback (most recent call last):
File "./msmtp-gnome-tool.py", line 171, in if main():
File "./msmtp-gnome-tool.py", line 134, in main km = keyringManager()
File "./msmtp-gnome-tool.py", line 46, in __init__ self.keyring = gk.get_default_keyring_sync()
gnomekeyring.IOError

The script is failing and I've spent quite a while figuring out why, but then I found [9]. The error happens because I'm using Slim login manager, which doesn't automatically start DBus when setting up the desktop. Because of that we have to edit the /etc/slim.conf and edit the login_cmd line to match the following, which will automatically start DBus.

login_cmd exec ck-launch-session dbus-launch /bin/bash -login ~/.xinitrc %session >~/.xsession-errors 2>&1

Besides starting DBus, we also have to start Gnome Keyring properly, which we can do by copying the following into ~/.xinitrc:

eval $(/usr/bin/gnome-keyring-daemon --start --components=secrets)
export GNOME_KEYRING_CONTROL GNOME_KEYRING_PID

Those lines will start gnome-keyring-daemon and export the GNOME_KEYRING_CONTROL and GNOME_KEYRING_PID environmental variables. Those variables will then be accessible to every instance of the our shell environment.

# env | grep GNOME
GNOME_KEYRING_CONTROL=/home/user/.cache/keyring-6wDCJn
GNOME_KEYRING_PID=4063

After that we can use the msmtp-gnome-tool.py without a problem. To set the password, we have to issue the command below.

# python2.7 msmtp-gnome-tool.py --set-password --username "name.surname@gmail.com" --server smtp.gmail.com
Password for user 'name.surname@gmail.com' in server 'smtp.gmail.com'
?
Confirmation ?
Password successfully set

If we launch Seahorse again, we can see the password stored in Gnome Keyring and can also be seen below.

seahorse_msmtp.png

After that we can send email without entering a password. To test whether everything works, we can issue the following command.

# cat mail.txt | msmtp -a default name.surname@gmail.com

Prior to launching msmtp command, we should also delete the password line from ~/.msmtprc, so the program will use Python Keyring automatically.

Address Book

When pressing m to create a new email, Mutt will first ask us for the recipient to sent the email to. The problem is that we have to type in the recipient's email address manually, which is quite a hindrance. We can use the abook external address book with mutt, which we can install with our default package manager.

# apt-get install abook

Then we can run abook to enter the ncurses interface, which can be seen below.

abook1.png

We can use '?' to display all commands, which are available to use when we're in abook. But the problem is that the address book is empty, which is why we must populate it with the contacts from our Gmail/Outlook accounts.

In Gmail, we have to click on the Gmail Icon and select Contacts where we can export it in vCard format as presented below.

gmail_addressbook1.png

After that we have to use the vcard2abook.py script we can download with wget.

# wget https://raw.github.com/yaroot/scripts/master/vcard2abook.py

To convert the exported contacts contacts.vcf into the abook address book, we can run the command below.

# python3.3 vcard2abook.py -f contacts.vcf -o
/home/user/.abook/addressbook

After that we can simply run the abook command and we'll be able to see all the imported contacts.

# abook

We can also add the /home/user/.abook/abookrc configuration file and add some configuration options. The most important is that we enable the add_email_prevent_duplicates, which disables adding duplicate entries into the address book.

set www_command=elinks
set add_email_prevent_duplicates=true

We also have to add the configuration options into ~/.muttrc, which enable Mutt to communicate with abook. Those configuration options will enable us to press Q to query the address book and A to add a new contact to address book.

set query_command="abook --mutt-query '%s'"
macro index,pager A "|abook --add-email-quietn" 'add sender to
abook'

To test whether the address book is working, we run Mutt, and type Q at a pager. This will popup a Query: line where we need to enter our search string.

gmail_addressbook2.png

After entering the search string, Mutt will call the "abook --mutt-query '<search-string>'" to display all the matching email addresses. In our cases I entered proteansec, which should display my own email address. We can see that below:

gmail_addressbook3.png

After selecting the email address, the To: email header will automatically be populated as can be seen below.

gmail_addressbook4.png

After that we can send email as we normally would. The great thing about integrating abook with Mutt is that we don't need to write the whole email addresses manually, but can use abook to do it for us.

Attachements

When starting Mutt and selecting a particular email, we can press the v key to view a list of attachments. On the picture below we can see that we're dealing with the attachment picture.png.

attachements1.png

When we select the attachment we would like to see and press Enter on it, the following is displayed.

attachements2.png

Obviously the picture is not displayed properly, because Mutt doesn't know how to handle images; it treats it just as it was a text document. In order to change that, we have to add the mailcap_path configuration option to the ~/.muttrc, which specifies the programs to use when trying to display certain MIME types.

set mailcap_path = ~/.mutt/mailcap

In the ~/.mutt/mailcap configuration we have to save the following, which instructs Mutt to use gpicview when trying to open image MIME types.

image/jpg; gpicview %s
image/jpeg; gpicview %s
image/pjpeg; gpicview %s
image/png; gpicview %s
image/gif; gpicview %s

If we view the same attached picture again and select it, this time the picture will be displayed normally by using the gpicview program, which can be seen below.

attachements3.png

We showed how we can use external program to view picture attachments in Mutt, but we can just as easily open any other attachment MIME types with appropriate programs.

Mu/Mairix

Mu

While Mutt supports regular expressions when searching over email messages by using the / shortcut, it's limited to current mailbox only. A better approach is using Mu or Mairix program, which indexes and searches the Maildir folders. First we have to install the program, which we won't go into here.

To index the maildirs, we need to run the command below.

# mu index --quiet --maildir /home/user/Mail

We can also add that command to our local crontab, so it will be run automatically every one in a while. If we would like to run it once in a day, we can use the following command, which runs it every day at 10 am.

0 10 * * * mu index --quiet --maildir /home/user/Mail

Then we can use mu find command to actually search for email messages:

# mu find wordtosearch

Mairix

To use the mairix, we first need to create the mairix folder inside the /home/user/Mail, which will be used to hold the Mairix specific files. Then we need to create the /home/user/.mairixrc configuration file, which holds the following:

# the email location
base=~/Mail

# which maildirs to index (separated by colons)
mformat=maildir
maildir=name.surname@gmail.com...

# search results will be copied to mairix folder inside base
mfolder=mairix

# path to the mairix index
database=~/Mail/mairix/.mairixdb

After that we need to run the "mairix -v -F" command to build the index, which is saved into the ~/Mail/mairix/mairixdb file. To use the search function from Mutt then, we have to add the following shortcut to ~/.muttrc:

macro index / "<shell-escape>mairix " "Search with Mairix."

When we search for some string by pressing / in Mutt, Mairix will create directories cur/, new/ and tmp/ in mfolder, where the search results are placed. This means that mairix folder is just another maildir, where the messages are stored and we can enter it the same was as the others. To actually jump to the search results use the ( shortcut:

macro index,pager ( "<change-folder>=mairixn" "Search results"

If we would like to combine the / and ( shortcuts into one shortcut, we can first jump to the mairix maildir and then execute a search, which will result in results being showed immediately. The following command can do what we want in one go.

macro index / "<change-folder>=mairix<enter><shell-escape>mairix " "Change the folder with search results and search with Mairix."

We can look at how to conduct a more versatile search by using built-in commands at [14].

GnuPG

One very important topic we need to talk about is GPG, which is used to encrypt and sign email messages. The first thing that we need to do is create a new private/public key pair with gpg.

# gpg --gen-key

After filling in all the required information like name and email address, the new GPG key will be created and automatically imported into the local keyring. All the public keys are contained in public keyring, which can be displayed by using gpg --list-keys command. Whenever we would like to send an email to a friend, we must first have his public key in our local keyring, so we can encrypt the message. Our friend can later use his private key to decrypt and read the message.

# gpg --list-keys
pub 4096R/12FEF905 2014-01-20
uid Dejan Lukan <dejan@proteansec.com>
sub 4096R/D8E68E25 2014-01-20

If we would like others to have access to our public key, we have to submit it to the PGP servers. Also, if we would like to fetch the public keys, we have to configure the keyservers. We can do that by adding the keyserver configuration lines into the ~/.gnupg/gpg.conf. GPG can send and receive keys from a keyserver, which can be HKP, email or LDAP. We can put the following into gpg.conf to use the GPG server.

keyserver hkp://keys.gnupg.net

After that we can send the key to the specified keyserver with the command presented below, where the 12FEF905 is the public ID of our key.

# gpg --send-key 12FEF905

To get a public key from the GPG server, we can either query it by public key ID or email address as presented below.

# gpg --recv-keys 12FEF905
# gpg --search-keys dejan@proteansec.com

Sometimes, we have to download the public key from a website and store it into a file. In such cases, we can import the key into our public store by using the following command.

# gpg --import publickey.pub

The GnuPG has various configuration files stored in ~/.gnupg/ directory and are presented below.

  • secring.gpg: the secret keyring, which stores private keys.
  • secring.gpg.lock: the lock file for the secret keyring.
  • pubring.gpg: the public keyring, which stores public keys.
  • pubring.gpg.lock: the lock file for the public key ring.
  • trustdb.gpg: the trust database.
  • trustdb.gpg.lock: the lock file for the trust database.
  • random_seed: a file used to provide a random pool.

We also have to talk about signing the messages we send to our friends. The key objective is that we should sign all our messages to prove that the mail was actually sent by us and wasn't forged. Typically, the message signature is appended to the email, so it can be easily checked for authenticity.

When Mutt encrypts email with a public key of the recipient, even we cannot read the email, because we don't have the private key of the recipient. Mutt has two ways of overcoming this limitation:

  • Save the plaintext copy of the message in our Sent folder.
  • Encrypt the copy of the messages with our public key and save it in the Sent folder.

Let's send a signed/encrypted message now. First we can send a message like we normally would and after editing the message body, the following will be presented to us.

gpg0.png

At the top of the page, we have the following option that we can choose from.

y:Send q:Abort t:To c:CC s:Subj a:Attach file d:Descrip ?:Help

At that time we have to press the letter p to edit the Security settings. At that timw the options presented below will be shown.

gpg1.png

We can press e to encrypt the message, which will now be signed as well as encrypted. After that we can hit y to send the message to ourselves. At this point Mutt will use the recipient's public key to encrypt the message. If we go to my Inbox in web interface, we can see that the message is encrypted: it's contained in the msg.asc attachment.

gpg2.png

To decrypt the message, we can download the attachment and run the gpg command below, which will ask us for the password of our private key in order to decrypt the message.

# gpg /home/user/msg.asc

After entering the right password, the original message will be saved to /home/user/msg text file.

# cat /home/user/msg
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline

This is an encrypted email message.

We can see that we were able to read the email message, which was encrypted.

Whenever you would like to share your public key on your website, so other people can use it to send encrypted messages to you, you can simply export the public key with the --export parameter and copy it to the web page you own.

# gpg --output dejan.gpg --armor --export dejan@proteansec.com

The --armor configuration setting is present so the key is exported in ascii format, not in binary. After that you should upload the dejan.gpg public certificate to a website and post a link to it on a web page.

Mutt Usage

We've seen that there's a lot going on with sending and receiving emails, making sure notifications work, securing password by keeping them in gnome-keyring, but despite all that, we also need to talk about mutt, which connects everything together.

The very first thing I would like to talk about are mailbox shortcuts, which refer to specific mailboxes [6]. Mutt supports four different mailbox formats: mbox, MMDF, MH and Maildir; we can specify which format we would like to use by specifying the mbox_type configuration option.

  • mbox: all messages are stored in a single file.
  • mmdf: similar to the mbox format, but each message is surrounded with four control-A's.
  • mh: each messages is stored in a separate file in mailbox directory.
  • maildir: the newest format, which adds three subdirectories to the mailbox directory: tmp, new and cur. Each messages is a stored in a separate file.

Now that we've cleared the confusion about different mailbox formats, we also need to talk about mailbox shortcuts. In configuration files, we often need to reference the specific mailbox where the shortcuts can come very handy, so we don't need to provide full path to the maildir. The shortcuts are the following [6]:

  • ! : the $spoolfile incoming mailbox
  • > : the $mbox file
  • < : the $record file
  • ~-*: our home directory
  • *: the last visited file
    • : the $folder directory
  • = : the $folder directory

Let's present an example of how a shortcut can be used. The folder configuration option defined the mailbox location, while the spoolfile defines our default inbox location. Normally we would have to write the options with full paths as presented below

set folder = "~/Mail"
set spoolfile = "~/Mail/name.surname/Inbox"

But if we use the '+' shortcut, which defines the $folder directory, we can shorten the same information as presented below. Notice that we used the '+' shortcut, which will automatically be replaced with "~/Mail" when needed.

set folder = "~/Mail"
set spoolfile = "+name.surname/Inbox"

We've just seen that using a shortcuts can be a great help when defining paths in configuration file.

The folder-hook options in our .muttrc configuration file are used to change settings based on which mailbox we're interacting with. The folder-hook executes arbitrary command based on the used regular expression, which matches the path to the mailbox.

Mutt also contains different configuration options regarding different folders:

  • spoolfile: the startup folder where all the new email is kept: this can be thought as an Inbox folder.
  • mbox: a folder where we can move the read emails, so they don't show up in spoolfile anymore (but we also don't want to delete those emails). This can be thought as an Archive folder.
  • record: a folder where sent messages are moved, so we have them archived somewhere in case we need it sometime in the future. This can be thought as a Sent Mail folder.
  • postponed: a folder where emails that we've started to write, but haven't finished yet are stored. This can be thought as a Drafts folder.

Now I would like to talk to you about how to actually do something once we had executed the command mutt and we're inside mutt MUA.

# mutt

The shortcuts that we can use in Mutt are the presented below.

  • Up (next-entry) : move to the next entry
  • Down (previous-entry) : move to the previous entry
  • = (first-entry) : move to the first entry
  • * (last-entry) : move to the last entry
  • ? (help) : print help
  • q (quit) : quit mutt
  • c (change-dir) : change directories
  • d (delete) : delete the current message
  • <num> : jump to message with index number <num>
  • / (search) : search for a regular expression

We must also present status flags used in Mutt to represent the status of emails, which are presented below [8]:

  • D: message is marked for deletion
  • d: message has attachments marked for deletion
  • K: message contains a PGP public key
  • N: new message
  • O: old message
  • P: PGP encrypted message
  • r: we've replied to the message
  • S: PGP signed message with a valid signature
  • s: PGP signed message with invalid signature
  • !: flagged message
  • *: tagged message
  • T: message was also sent to CC subscribers
  • C: message was delivered to us, because we're in CC subscriber list
  • F: message was sent by us
  • L: message was sent to mailing list

There are also other tips that can greatly enhance the usage of Mutt.

# When reading email with Mutt, the status of the email is not synchronized with the web interface, which causes different IMAP clients to make notifications about unread mail when in fact the email has already been read. We can solve this by pressing $ (sync-mailbox) in Mutt, which will upload our local changes to the server. # When in Mutt, we can change folders with the c letter, but it's very cumbersome. Rather than that, it's better to redefined the keys to go to next/previous folder to Down/Up keys and to enter a folder to Right key. This tip was taken from [10].

bind index,pager sidebar-next
bind index,pager sidebar-prev
bind index,pager sidebar-open

To conclude, we must say that Mutt is definitely not for the faint-hearted, but if you take the time to customize and learn it, then you'll be given a powerful and lightweight tool for your day to day usage.

The most prevalent reason why I turned to Mutt was because I had quite a few email addresses and keeping up to date with them was a nightmare. But when I got a very important email that I didn't see for two weeks, because I simply hadn't had time to login to the web interface, I had enough. I switched to Thunderbird, but it was very cumbersome and very slow especially when I was running a lot of different things at the same time, like virtualized virtual machines. It also consumed a fair amount of memory, so my system was often slow and unresponsive. At that time I thought that it must be a better alternative, which is when I found Mutt and quickly fell in love: after the first few stitches and glitches.

Now I'm a satisfied user of Mutt, which brought order to my emailing system that I love to use on a daily basis. But what's more important, it doesn't slow down my computer one single bit.

References:

[1] A unix style mail setup, http://dev.gentoo.org/~tomka/mail.html.

[2] Msmtp, https://wiki.archlinux.org/index.php/msmtp.

[3] OfflineIMAP, https://wiki.archlinux.org/index.php/OfflineIMAP.

[4] Mailcheck, http://mailcheck.sourceforge.net/.

[5] My Gmail workflow with mutt, offlineimap, mailcheck, wmbiff, some shell scripts and notify-send, http://crmafra.blogspot.com/2012/01/my-gmail-email-with-mutt-offlineimap.html.

[6] Mutt Advanced Usage, http://www.mutt.org/doc/manual/manual-4.html.

[7] Installing and Using Python Keyring Lib, https://pypi.python.org/pypi/keyring

[8] Mutt Getting Started, http://www.mutt.org/doc/manual/manual-2.html.

[9] [network-manager-applet] Unable to access secrets service, https://bugs.archlinux.org/task/18930.

[10] The Homely Mutt, http://stevelosh.com/blog/2012/10/the-homely-mutt/#running.

[11] abook(1) - Linux man page, http://linux.die.net/man/1/abook.

[12] Everything You Need To Know To Start Using GnuPG with Mutt, [http://codesorcery.net/old/mutt/mutt-gnupg-howto].

[13] mutt and friends, tools in the Email Toolchain, [https://dgoodmaniii.wordpress.com/tag/muttprint/ ].

[14] Mairix, http://pbrisbin.com/posts/mairix/.

Comments