Introduction to Email System
On the picture below we can see basic overview we'll be setting up in this tutorial. When working with Mutt, we have to take care of sending and receiving emails separately, since different servers are used for those. The overview can be seen on the picture below.
When sending an email to the SMTP server, we have to use SMTP client, which is msmtp in our case; it authenticates to the SMTP server and sends the email. We need to be careful to authenticate over secure TLS connection, which we'll configure later on. Mutt can either send the email by itself or contact an external program like msmtp to do the job for him.
When receiving an email, we can use two protocols for the job: POP3 or IMAP. We really should use IMAP, which copies the messages to local computer and leaves the copy on the server intact. POP3 protocol downloads the messages to local computer and deletes the messages on upstream mail server. When using IMAP protocol we can use any of the following external programs: offlineimap, getmail or fetchmail. In our case we'll be using offlineimap program, which needs to be running in the background to constantly connect to the IMAP server and check whether any new emails are available. Note that IMAP client must also connect over the secure TLS connection prior to authentication. The IMAP client basically copies the email messages to local Maildir directory for each defined email account.
At the end of the day, we also want to have a notification system that informs us about the arrival of a new email. Since the offlineimap doesn't inform us when new email arrives, we need some kind of notification system letting us know about new email messages. We'll see about this later on in the article.
Let's also describe the terminology used when talking about mail systems:
- MUA (Mail User Agent): interacts with the MDA to receive emails, which it presents to the user. It also receives user interactions and executes them appropriately. A MUA can be installed locally on the client's machine as email client like Mozilla Thunderbird, Microsoft Outlook or Mutt, or it can be accessible through web interface like Gmail, Outlook, Yahoo, etc.
- MTA (Mail Transfer Agent): used for sending the email through the Internet. A MTA agent communicates with other MTA agents by using SMTP protocol. The MTA agent listens on port 25 (unencrypted) or port 587 (encrypted) where it receives SMTP messages from other MTA agents. We need to properly protect our MTA agent (SMTP server) so only authenticated requests are processed.
- MDA (Mail Delivery Agent): used for receiving the email through the Internet. A MDA agent receives emails from MTAs and stores them locally waiting for users to access them. A user can access emails by using either POP3 or IMAP protocol. The MDA is needed so clients don't need to be connected all the time to receive emails. Imagine what would happen if Internet connection goes down because of ISP maintenance and a client will not be able to download emails for a couple of hours. When the Internet connection is reestablished, the client still won't be able to receive the emails, because they were not stored anywhere: this is what MDA is for; it stores messages locally, so the client can access them at any time.
When sending an email, the MTA agents looks for MX records of the domain to determine the SMTP server to which it must forward the sending email. We can execute the following dig command to determine the MX servers to which we have to send email for gmail.com domain.
# dig -t MX google.com ; <<>> DiG 9.9.3-P2 <<>> -t MX google.com ;; global options: +cmd ;; ->>HEADER< ;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 5 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;google.com. IN MX ;; ANSWER SECTION: google.com. 600 IN MX 10 aspmx.l.google.com. google.com. 600 IN MX 20 alt1.aspmx.l.google.com. google.com. 600 IN MX 50 alt4.aspmx.l.google.com. google.com. 600 IN MX 40 alt3.aspmx.l.google.com. google.com. 600 IN MX 30 alt2.aspmx.l.google.com. ;; Query time: 27 msec ;; SERVER: 10.10.100.1#53(10.10.100.1) ;; WHEN: Sat Jan 11 14:05:02 CET 2014 ;; MSG SIZE rcvd: 147
Notice that DNS request returned five MX records with priority numbers 10, 20, 30, 40 and 50. The MX record with the lower priority number has the maximum priority and will be used first in email delivery process.
Introduction to Gnome Keyring
For those of you that don't know what Gnome Keyring is, it's a daemon, which holds credentials in encrypted form, so we don't need to store them in plaintext files. The applications that have support for Gnome Keyring can use it to provide passwords when needed, for example: when msmtp wants to send an email, it must connect to SMTP server by providing the correct username and password, after which it can send the email. The msmtp can use a plaintext password it has stored in a configuration file or request the password from Gnome Keyring. Most of the time we don't want to store passwords in configuration files, so if the application supports the Gnome Keyring interaction, we should use it.
The first thing that we need to know is that we can use seahorse as a GUI for managing Gnome Keyring. On the picture below, we can see stored passwords, ssh and pgp keys.
Introduction to Seahorse
The seahorse is used to store user credentials in an encrypted database called keyring. If we would like to create a new password, we must click on the File - New - Password Keyring as presented on the picture below.
We need to choose to name the keyring, where we entered the string test after which we must provide a password for the keyring. That is presented on the picture below.
Once completed, we can notice a new test entry in the menu on the left. The keyring is currently empty.
We can add a new password to the keyring by pressing on the New button and selecting the Stored Password and filling in the keyring to which we would like to add the password as well as description and the actual password.
After adding the password, we can see it in the seahorse and by double-clicking on it, where the password is shown, because the keyring has already been unlocked.
Sending the Email
When we would like to send an email, we usually want to use SMTP client, which in our case is msmtp. First we need to install msmtp, which we should do with our default package manager. The msmtp program reads the ~/.msmtprc configuration file, where the SMTP server, username, password and other stuff are defined.
Normal configuration file contains the configuration variables as presented below. The first part of the configuration file defines default account that uses smtp.gmail.com SMTP server and port 587. It authenticates the user name.surname with password mypassword to the SMTP server. After that we're defining that we want to use TLS connection, so credentials are send over encrypted tunnel. At last, there are some logging options in case something goes wrong, so we can debug the error further.
# Account account default host smtp.gmail.com port 587 auth on user firstname.lastname@example.org from email@example.com password mypassword # Enable TLS tls on tls_certcheck on tls_trust_file /etc/ssl/certs/ca-certificates.crt # Logging syslog LOG_MAIL logfile /home/user/msmtp.log
Sending the Email
In this example, we'll be sending an email to ourselves. To do that, we must define a new file mail.txt with the following contents.
To: <firstname.lastname@example.org> From: "Name Surname" <email@example.com> Subject: Test Testing if msmtp works.
To actually send the email to the SMTP server, which will relay it to where it needs to go, we have to pipe the content of the mail.txt file to msmtp program.
# cat mail.txt | msmtp -a default firstname.lastname@example.org
Notice that we used the default account, which we previously defined in ~/.msmtprc configuration file? At the same time as sending the mail, we can also tail the contents of the /home/user/msmtp.log log file to see what's going on.
# tail -f /home/user/msmtp.log
When piping the email to the msmtp command, a new entry should appear in the msmtp.log at the time of sending the email, which looks like presented below. The message has existed with a status EX_OK, which means everything is ok and the message was delivered successfully. If we login to our own mailbox, we should see this email in our Inbox.
Jan 06 20:20:53 host=smtp.gmail.com tls=on auth=on email@example.com firstname.lastname@example.org email@example.com mailsize=112 smtpstatus=250 smtpmsg='250 2.0.0 OK 1389036172 n1sm173566001eep.20 - gsmtp' exitcode=EX_OK
This is all fine and all, but we have a password stored in a configuration file in clear-text form. This is something we don't want, since it's not very secure and anyone gaining access to our computer would also be able to read the password. Later on, we'll see how we can secure this process by using Gnome Keyring.
If you ever need to debug the msmtp, you can run the msmtp program with --debug=masmtlog parameter as follows. It will display whole communication between the MX server, which can shed some light into the problem when something is not working.
# cat mail.txt | msmtp --debug=masmtlog -a default firstname.lastname@example.org
We can also use multiple accounts with msmtp client. All we need to do is define another account section on the ~/.msmtprc configuration file. The code below defines two email accounts first and second.
account first host smtp.gmail.com port 587 auth on user email@example.com account second host smtp.gmail.com port 587 auth on user firstname.lastname@example.org
When we want to send an email by using either of the defined accounts, we need to pass the account name with the -a option to msmtp program. Instead of using default we can use first and second as follows.
# msmtp -a first email@example.com # msmtp -a second firstname.lastname@example.org
Receiving the Email
When we would like to setup the receiving process with Mutt, we can choose between multiple clients, which are outlined below. In the brackets I presented the protocols we can use with the each of the programs.
- offlineimap (IMAP)
- getmail (IMAP, POP3)
- fetchmail (IMAP, POP3)
Introduction to OfflineIMAP
In this tutorial we'll use offlineimap, which can be easily installed with our default package manager. It can be used to synchronize remote IMAP server with local maildir. We need to copy the offlineimap.conf configuration file from /usr/share/offlineimap/ directory to ~/.offlineimaprc. A simple configuration file is presented below.
[general] # List accounts to be synced. accounts = main [mbnames] enabled = no filename = ~/Mutt/muttrc.mailboxes header = "mailboxes " sep = " " footer = "n" # # Accounts # localrepository : the maildir to be synced via IMAP (local). # remoterepository : the remote IMAP to be synced (remote). # status_backend : status cache: plain, sqlite. # [Account main] localrepository = local_main remoterepository = remote_main status_backend = sqlite # # Repository # type : local repository can be: Maildir, IMAP, Gmail. # localfolders : the folder where to save the emails. # remotehost : the remote host of IMAP server. # remoteuser : the remote user of IMAP server. # ssl : use SSL connection on port 993, otherwise 143. [Repository local_main] type = Maildir localfolders = ~/Mail/ [Repository remote_main] type = Gmail ssl = yes remoteuser = email@example.com remotepass = mypassword cert_fingerprint = 89091347184d41768bfc0da9fad94bfe882dd358
Most of the configuration variables are self-explanatory, but there's also an interesting option cert_fingerprint, which specifies the SHA1 hash of IMAP public server certificate. To calculate the SHA1 of the imap.gmail.com certificate, we have to run the gnutls-cli command as follows.
# gnutls-cli -p 993 imap.gmail.com Resolving 'imap.gmail.com'... Connecting to '18.104.22.168:993'... - Certificate type: X.509 - Got a certificate list of 3 certificates. - Certificate info: - subject `C=US,ST=California,L=Mountain View,O=Google Inc,CN=imap.gmail.com', issuer `C=US,O=Google Inc,CN=Google Internet Authority G2', RSA key 2048 bits, signed using RSA-SHA 1, activated `2013-09-10 07:59:51 UTC', expires `2014-09-10 07:59:51 UTC', SHA-1 fingerprint `89091347184d41768bfc0da9fad94bfe882dd358' - Certificate info: - subject `C=US,O=Google Inc,CN=Google Internet Authority G2', issuer `C=US,O=GeoTrust Inc.,CN=GeoTrust Global CA', RSA key 2048 bits, signed using RSA-SHA1, activated `2013-04-0 5 15:15:55 UTC', expires `2015-04-04 15:15:55 UTC', SHA-1 fingerprint `d83c1a7f4d0446bb2081b81a1670f8183451ca24' - Certificate info: - subject `C=US,O=GeoTrust Inc.,CN=GeoTrust Global CA', issuer `C=US,O=Equifax,OU=Equifax Secure Certificate Authority', RSA key 2048 bits, signed using RSA-SHA1, activated `2002 -05-21 04:00:00 UTC', expires `2018-08-21 04:00:00 UTC', SHA-1 fingerprint `7359755c6df9a0abc3060bce369564c8ec4542a3' - The hostname in the certificate matches 'imap.gmail.com'. - Peer's certificate issuer is unknown - Peer's certificate is NOT trusted - Version: TLS1.2 - Key Exchange: RSA - Cipher: ARCFOUR-128 - MAC: SHA1 - Compression: NULL - Handshake was completed - Simple Client Mode
Notice that the gnutls-cli printed the SHA1 hashes. If we would like to be able to connect to the Gmail IMAP server, we need to include the cert_fingerprint line with the correct SHA1 in the configuration file. Otherwise offlineimap will terminate the connection, because the validity of the server cannot be verified.
cert_fingerprint = 89091347184d41768bfc0da9fad94bfe882dd358
After providing the right SHA1 hash, we need to synchronize the remote and local email addresses by running offlineimap.
If everything is setup correctly, the offlineimap should synchronize the local and remote end of communication. Note that offlineimap merely downloads the emails from the IMAP server, but doesn't actually do anything with them. If we would like to read the synchronized emails, we can do so from local maildir: we can use any mail client for that, like mutt or gnus.
When we switch from a GUI mail client to text-based mail client, we have to use a notification system that's letting us know about the unread emails.
We can use mail-notification program, which we can install by using our default package manager. After installation we can start the program by running /usr/bin/mail-notification and configure the Mailbox accounts by adding them into the mail-notification. This looks like presented on the picture below.
We defined three Mailboxes, where I've intentionally hidden the Mailbox paths, but you can get the point. You should define as many Mailboxes as you use on your system. On the picture you can also see the "Message Popups" tab where we can configure different things, like whether we would like to have message popups enabled and the expiration time for the message. This can be seen on the picture below.
Using Different Password Stores
Rather than storing a clear-text password in a configuration file, we can use different methods of providing the password to the email daemons/programs. We can use any of the following authentication mechanisms.
The passwords are stored at appropriate program configuration files in clear-text. This is what we've been using so far, and is the easiest to configure, but provides least security.
We can store passwords in ~/.netrc file, where we need to specify every username with keyword login and password with keyword password. An example of specifying login credentials for the Gmail account firstname.lastname@example.org can be seen below.
machine mail.google.com login email@example.com password mypass
When using this option, don't forget to properly secure ~/.netrc file with chmod 600, so it's only readable and writeable by the current user.
# chmod 600 ~/.netrc
When we want to use this option, we first need to install the right package for the python keyring. On my system I had to install gnome-keyring-python, but on your system it might have a different name like python-keyring or something else. Search your package repository for appropriate name and install it. Alternatively we can also install it with easy_install or pip.
# easy_install keyring # pip install keyring
After that, we need to add the ~/.mutt/scripts/pythonkeyring.py and add import keyring to it.
# vim ~/.mutt/scripts/pythonkeyring.py import keyring
This needs to be done in order to make the keyring available to offlineimap. Offlineimap supports the pythonfile configuration option, where we can specify a file that will be imported and used in offlineimap; since we only need access to the keyring, we only import that in pythonkeyring.py, but we could use any Python code. After that we need to edit the ~/.offlineimap and configure the pythonfile accordingly.
pythonfile = ~/.mutt/scripts/pythonkeyring.py remotepasseval = keyring.get_password("offlineimap", "firstname.lastname@example.org")
Notice that we're referencing the offlineimap keyring by asking for the password of email@example.com account. After that we also need to add the password to the offlineimap keyring, which we can simply do by opening Python interpreter and calling the set_password function
# python >>> import keyring >>> keyring.set_password("offlineimap", "firstname.lastname@example.org", "mypass") Please enter password for encrypted keyring: >>> keyring.get_password("offlineimap", "email@example.com") u'mypass'
The module will store keyring config file as well as keyring itself in it's working directory, which can be found by executing the following command .
# python -c "import keyring.util.platform_; print(keyring.util.platform_.data_root())" /home/user/.local/share/python_keyring # ls /home/user/.local/share/python_keyring crypted_pass.cfg
We can see only the keyring, but not the configuration file keyringrc.cfg. The crypted_pass.cfg contains the following after adding the password:
# cat /home/user/.local/share/python_keyring/crypted_pass.cfg [offlineimap] name_2esurname_40gmail_2ecom = eyJwYXNzd29yZF9lbmNyeXB0ZWQiOiAiNkxneXp6OXBTQ0ZpXG4iLCAic2FsdCI6ICJ3WWk4N2lZ UEp4Sncxa1ZiMnZwTXRISFpyUVgyL3ptayt4UDNaSGVMTExFPVxuIiwgIklWIjogIml0cHV1T2NR enZnTUI3TmFOWWdkL1E9PVxuIn0=
Notice that the value of firstname.lastname@example.org was base64 encoded and encrypted and only the master password can be used to decrypt it.
There are multiple backends that we can use for safely storing our passwords; note that each backend keyring must subclass KeyringBackend .
- EncryptedFileKeyring : keyring.backends.file.BaseKeyring()
- PlaintextKeyring : keyring.backends.pyfs.PlaintextKeyring()
- KeyczarKeyring : keyring.backends.pyfs.KeyczarKeyring()
- GnomeKeyring : keyring.backends.Gnome.Keyring()
- GoogleKeyring : keyring.backends.Google.DocsKeyring()
- KDEWallet : keyring.backends.kwallet.Keyring()
- OSXKeychain : keyring.backends.OS_X.Keyring()
- SecretServiceKeyring : keyring.backends.SecretService.Keyring()
- WinVaultKeyring : keyring.backends.Windows.WinVaultKeyring()
- RegistryKeyring : keyring.backends.Windows.RegistryKeyring()
The default keyring is the file keyring as we've already seen. We can confirm that by calling the get_keyring function as seen below.
# python >>> import keyring >>> keyring.get_keyring()
In case we're already using Gnome Keyring and would like to use that instead of the file keyring, we can call set_keyring to set the Gnome Keyring. After that we can call set_password and get_password the same way as before.
>>> keyring.set_keyring(keyring.backends.Gnome.Keyring()) >>> keyring.set_password("offlineimap", "email@example.com", "mypass") >>> keyring.get_password("offlineimap", "firstname.lastname@example.org") u'mypass'
After setting the password in our keyring, we need to call offlineimap to download the messages from the server. In case we're using the Python Keyring, we'll be asked for the keyring password every time we run offlineimap as seen below.
# offlineimap Account sync gmailname: *** Processing account gmailname Establishing connection to imap.gmail.com:993 Please enter password for encrypted keyring:
The offlineimap is asking us for the password of encrypted keyring, not for the password of our Gmail account. Once we enter the password of the keyring, it will start syncing the emails from the server. Since offlineimap will ask for the keyring password every time we restart it, this option is unacceptable for everyday usage. Besides that, there's also a problem of offlineimap not poping up a GUI window to input a password, which makes running offlineimap in crontab very hard.
But if we're using Gnome Keyring, a GUI popup is shown automatically, which requests a password to be inputted to unlock the keyring. Below we can see that offlineimap started syncing all the email messages automatically, because the keyring has already been unlocked.
# offlineimap OfflineIMAP 6.5.4 Licensed under the GNU GPL v2+ (v2 or any later version) Account sync gmailname: *** Processing account gmailname Establishing connection to imap.gmail.com:993 Folder [Gmail]/All Mail [acc: gmailname]: Syncing [Gmail]/All Mail: IMAP -> Maildir
There's also an entry stored in the Gnome Keyring, which we can observe by starting Seahorse.
Despite talking about Gnome Keyring in the previous section, I think it deserves a special section of it's own, because it's very hard to get everything right. Besides, we also have to set-up the msmtp program to use Gnome Keyring for sending emails. If we don't, then we'll have to enter the password for our email account every time we'll want to send an email, which is a nuisance.
To add a password for msmtp into Gnome Keyring, we can use the msmtp-gnome-tool.py script, which is part of the msmtp package and is located in the msmtp-1.4.30/scripts/ directory. Note that you have to download the source code in order to get that script. When I downloaded the script and wanted to save my credentials into Gnome keyring I stumbled upon the following error:
# ./msmtp-gnome-tool.py --help Traceback (most recent call last): File "./msmtp-gnome-tool.py", line 171, in if main(): File "./msmtp-gnome-tool.py", line 134, in main km = keyringManager() File "./msmtp-gnome-tool.py", line 46, in __init__ self.keyring = gk.get_default_keyring_sync() gnomekeyring.IOError
The script is failing and I've spent quite a while figuring out why, but then I found . The error happens because I'm using Slim login manager, which doesn't automatically start DBus when setting up the desktop. Because of that we have to edit the /etc/slim.conf and edit the login_cmd line to match the following, which will automatically start DBus.
login_cmd exec ck-launch-session dbus-launch /bin/bash -login ~/.xinitrc %session >~/.xsession-errors 2>&1
Besides starting DBus, we also have to start Gnome Keyring properly, which we can do by copying the following into ~/.xinitrc:
eval $(/usr/bin/gnome-keyring-daemon --start --components=secrets) export GNOME_KEYRING_CONTROL GNOME_KEYRING_PID
Those lines will start gnome-keyring-daemon and export the GNOME_KEYRING_CONTROL and GNOME_KEYRING_PID environmental variables. Those variables will then be accessible to every instance of the our shell environment.
# env | grep GNOME GNOME_KEYRING_CONTROL=/home/user/.cache/keyring-6wDCJn GNOME_KEYRING_PID=4063
After that we can use the msmtp-gnome-tool.py without a problem. To set the password, we have to issue the command below.
# python2.7 msmtp-gnome-tool.py --set-password --username "email@example.com" --server smtp.gmail.com Password for user 'firstname.lastname@example.org' in server 'smtp.gmail.com' ? Confirmation ? Password successfully set
If we launch Seahorse again, we can see the password stored in Gnome Keyring and can also be seen below.
After that we can send email without entering a password. To test whether everything works, we can issue the following command.
# cat mail.txt | msmtp -a default email@example.com
Prior to launching msmtp command, we should also delete the password line from ~/.msmtprc, so the program will use Python Keyring automatically.
When pressing m to create a new email, Mutt will first ask us for the recipient to sent the email to. The problem is that we have to type in the recipient's email address manually, which is quite a hindrance. We can use the abook external address book with mutt, which we can install with our default package manager.
# apt-get install abook
Then we can run abook to enter the ncurses interface, which can be seen below.
We can use '?' to display all commands, which are available to use when we're in abook. But the problem is that the address book is empty, which is why we must populate it with the contacts from our Gmail/Outlook accounts.
In Gmail, we have to click on the Gmail Icon and select Contacts where we can export it in vCard format as presented below.
After that we have to use the vcard2abook.py script we can download with wget.
# wget https://raw.github.com/yaroot/scripts/master/vcard2abook.py
To convert the exported contacts contacts.vcf into the abook address book, we can run the command below.
# python3.3 vcard2abook.py -f contacts.vcf -o /home/user/.abook/addressbook
After that we can simply run the abook command and we'll be able to see all the imported contacts.
We can also add the /home/user/.abook/abookrc configuration file and add some configuration options. The most important is that we enable the add_email_prevent_duplicates, which disables adding duplicate entries into the address book.
set www_command=elinks set add_email_prevent_duplicates=true
We also have to add the configuration options into ~/.muttrc, which enable Mutt to communicate with abook. Those configuration options will enable us to press Q to query the address book and A to add a new contact to address book.
set query_command="abook --mutt-query '%s'" macro index,pager A "|abook --add-email-quietn" 'add sender to abook'
To test whether the address book is working, we run Mutt, and type Q at a pager. This will popup a Query: line where we need to enter our search string.
After entering the search string, Mutt will call the "abook --mutt-query '<search-string>'" to display all the matching email addresses. In our cases I entered proteansec, which should display my own email address. We can see that below:
After selecting the email address, the To: email header will automatically be populated as can be seen below.
After that we can send email as we normally would. The great thing about integrating abook with Mutt is that we don't need to write the whole email addresses manually, but can use abook to do it for us.
When starting Mutt and selecting a particular email, we can press the v key to view a list of attachments. On the picture below we can see that we're dealing with the attachment picture.png.
When we select the attachment we would like to see and press Enter on it, the following is displayed.
Obviously the picture is not displayed properly, because Mutt doesn't know how to handle images; it treats it just as it was a text document. In order to change that, we have to add the mailcap_path configuration option to the ~/.muttrc, which specifies the programs to use when trying to display certain MIME types.
set mailcap_path = ~/.mutt/mailcap
In the ~/.mutt/mailcap configuration we have to save the following, which instructs Mutt to use gpicview when trying to open image MIME types.
image/jpg; gpicview %s image/jpeg; gpicview %s image/pjpeg; gpicview %s image/png; gpicview %s image/gif; gpicview %s
If we view the same attached picture again and select it, this time the picture will be displayed normally by using the gpicview program, which can be seen below.
We showed how we can use external program to view picture attachments in Mutt, but we can just as easily open any other attachment MIME types with appropriate programs.
While Mutt supports regular expressions when searching over email messages by using the / shortcut, it's limited to current mailbox only. A better approach is using Mu or Mairix program, which indexes and searches the Maildir folders. First we have to install the program, which we won't go into here.
To index the maildirs, we need to run the command below.
# mu index --quiet --maildir /home/user/Mail
We can also add that command to our local crontab, so it will be run automatically every one in a while. If we would like to run it once in a day, we can use the following command, which runs it every day at 10 am.
0 10 * * * mu index --quiet --maildir /home/user/Mail
Then we can use mu find command to actually search for email messages:
# mu find wordtosearch
To use the mairix, we first need to create the mairix folder inside the /home/user/Mail, which will be used to hold the Mairix specific files. Then we need to create the /home/user/.mairixrc configuration file, which holds the following:
# the email location base=~/Mail # which maildirs to index (separated by colons) mformat=maildir firstname.lastname@example.org... # search results will be copied to mairix folder inside base mfolder=mairix # path to the mairix index database=~/Mail/mairix/.mairixdb
After that we need to run the "mairix -v -F" command to build the index, which is saved into the ~/Mail/mairix/mairixdb file. To use the search function from Mutt then, we have to add the following shortcut to ~/.muttrc:
macro index / "<shell-escape>mairix " "Search with Mairix."
When we search for some string by pressing / in Mutt, Mairix will create directories cur/, new/ and tmp/ in mfolder, where the search results are placed. This means that mairix folder is just another maildir, where the messages are stored and we can enter it the same was as the others. To actually jump to the search results use the ( shortcut:
macro index,pager ( "<change-folder>=mairixn" "Search results"
If we would like to combine the / and ( shortcuts into one shortcut, we can first jump to the mairix maildir and then execute a search, which will result in results being showed immediately. The following command can do what we want in one go.
macro index / "<change-folder>=mairix<enter><shell-escape>mairix " "Change the folder with search results and search with Mairix."
We can look at how to conduct a more versatile search by using built-in commands at .
One very important topic we need to talk about is GPG, which is used to encrypt and sign email messages. The first thing that we need to do is create a new private/public key pair with gpg.
# gpg --gen-key
After filling in all the required information like name and email address, the new GPG key will be created and automatically imported into the local keyring. All the public keys are contained in public keyring, which can be displayed by using gpg --list-keys command. Whenever we would like to send an email to a friend, we must first have his public key in our local keyring, so we can encrypt the message. Our friend can later use his private key to decrypt and read the message.
# gpg --list-keys pub 4096R/12FEF905 2014-01-20 uid Dejan Lukan <email@example.com> sub 4096R/D8E68E25 2014-01-20
If we would like others to have access to our public key, we have to submit it to the PGP servers. Also, if we would like to fetch the public keys, we have to configure the keyservers. We can do that by adding the keyserver configuration lines into the ~/.gnupg/gpg.conf. GPG can send and receive keys from a keyserver, which can be HKP, email or LDAP. We can put the following into gpg.conf to use the GPG server.
After that we can send the key to the specified keyserver with the command presented below, where the 12FEF905 is the public ID of our key.
# gpg --send-key 12FEF905
To get a public key from the GPG server, we can either query it by public key ID or email address as presented below.
# gpg --recv-keys 12FEF905 # gpg --search-keys firstname.lastname@example.org
Sometimes, we have to download the public key from a website and store it into a file. In such cases, we can import the key into our public store by using the following command.
# gpg --import publickey.pub
The GnuPG has various configuration files stored in ~/.gnupg/ directory and are presented below.
- secring.gpg: the secret keyring, which stores private keys.
- secring.gpg.lock: the lock file for the secret keyring.
- pubring.gpg: the public keyring, which stores public keys.
- pubring.gpg.lock: the lock file for the public key ring.
- trustdb.gpg: the trust database.
- trustdb.gpg.lock: the lock file for the trust database.
- random_seed: a file used to provide a random pool.
We also have to talk about signing the messages we send to our friends. The key objective is that we should sign all our messages to prove that the mail was actually sent by us and wasn't forged. Typically, the message signature is appended to the email, so it can be easily checked for authenticity.
When Mutt encrypts email with a public key of the recipient, even we cannot read the email, because we don't have the private key of the recipient. Mutt has two ways of overcoming this limitation:
- Save the plaintext copy of the message in our Sent folder.
- Encrypt the copy of the messages with our public key and save it in the Sent folder.
Let's send a signed/encrypted message now. First we can send a message like we normally would and after editing the message body, the following will be presented to us.
At the top of the page, we have the following option that we can choose from.
y:Send q:Abort t:To c:CC s:Subj a:Attach file d:Descrip ?:Help
At that time we have to press the letter p to edit the Security settings. At that timw the options presented below will be shown.
We can press e to encrypt the message, which will now be signed as well as encrypted. After that we can hit y to send the message to ourselves. At this point Mutt will use the recipient's public key to encrypt the message. If we go to my Inbox in web interface, we can see that the message is encrypted: it's contained in the msg.asc attachment.
To decrypt the message, we can download the attachment and run the gpg command below, which will ask us for the password of our private key in order to decrypt the message.
# gpg /home/user/msg.asc
After entering the right password, the original message will be saved to /home/user/msg text file.
# cat /home/user/msg Content-Type: text/plain; charset=utf-8 Content-Disposition: inline This is an encrypted email message.
We can see that we were able to read the email message, which was encrypted.
Whenever you would like to share your public key on your website, so other people can use it to send encrypted messages to you, you can simply export the public key with the --export parameter and copy it to the web page you own.
# gpg --output dejan.gpg --armor --export email@example.com
The --armor configuration setting is present so the key is exported in ascii format, not in binary. After that you should upload the dejan.gpg public certificate to a website and post a link to it on a web page.
We've seen that there's a lot going on with sending and receiving emails, making sure notifications work, securing password by keeping them in gnome-keyring, but despite all that, we also need to talk about mutt, which connects everything together.
The very first thing I would like to talk about are mailbox shortcuts, which refer to specific mailboxes . Mutt supports four different mailbox formats: mbox, MMDF, MH and Maildir; we can specify which format we would like to use by specifying the mbox_type configuration option.
- mbox: all messages are stored in a single file.
- mmdf: similar to the mbox format, but each message is surrounded with four control-A's.
- mh: each messages is stored in a separate file in mailbox directory.
- maildir: the newest format, which adds three subdirectories to the mailbox directory: tmp, new and cur. Each messages is a stored in a separate file.
Now that we've cleared the confusion about different mailbox formats, we also need to talk about mailbox shortcuts. In configuration files, we often need to reference the specific mailbox where the shortcuts can come very handy, so we don't need to provide full path to the maildir. The shortcuts are the following :
- ! : the $spoolfile incoming mailbox
- > : the $mbox file
- < : the $record file
- ~-*: our home directory
- *: the last visited file
- : the $folder directory
- = : the $folder directory
Let's present an example of how a shortcut can be used. The folder configuration option defined the mailbox location, while the spoolfile defines our default inbox location. Normally we would have to write the options with full paths as presented below
set folder = "~/Mail" set spoolfile = "~/Mail/name.surname/Inbox"
But if we use the '+' shortcut, which defines the $folder directory, we can shorten the same information as presented below. Notice that we used the '+' shortcut, which will automatically be replaced with "~/Mail" when needed.
set folder = "~/Mail" set spoolfile = "+name.surname/Inbox"
We've just seen that using a shortcuts can be a great help when defining paths in configuration file.
The folder-hook options in our .muttrc configuration file are used to change settings based on which mailbox we're interacting with. The folder-hook executes arbitrary command based on the used regular expression, which matches the path to the mailbox.
Mutt also contains different configuration options regarding different folders:
- spoolfile: the startup folder where all the new email is kept: this can be thought as an Inbox folder.
- mbox: a folder where we can move the read emails, so they don't show up in spoolfile anymore (but we also don't want to delete those emails). This can be thought as an Archive folder.
- record: a folder where sent messages are moved, so we have them archived somewhere in case we need it sometime in the future. This can be thought as a Sent Mail folder.
- postponed: a folder where emails that we've started to write, but haven't finished yet are stored. This can be thought as a Drafts folder.
Now I would like to talk to you about how to actually do something once we had executed the command mutt and we're inside mutt MUA.
The shortcuts that we can use in Mutt are the presented below.
- Up (next-entry) : move to the next entry
- Down (previous-entry) : move to the previous entry
- = (first-entry) : move to the first entry
- * (last-entry) : move to the last entry
- ? (help) : print help
- q (quit) : quit mutt
- c (change-dir) : change directories
- d (delete) : delete the current message
- <num> : jump to message with index number <num>
- / (search) : search for a regular expression
We must also present status flags used in Mutt to represent the status of emails, which are presented below :
- D: message is marked for deletion
- d: message has attachments marked for deletion
- K: message contains a PGP public key
- N: new message
- O: old message
- P: PGP encrypted message
- r: we've replied to the message
- S: PGP signed message with a valid signature
- s: PGP signed message with invalid signature
- !: flagged message
- *: tagged message
- T: message was also sent to CC subscribers
- C: message was delivered to us, because we're in CC subscriber list
- F: message was sent by us
- L: message was sent to mailing list
There are also other tips that can greatly enhance the usage of Mutt.
# When reading email with Mutt, the status of the email is not synchronized with the web interface, which causes different IMAP clients to make notifications about unread mail when in fact the email has already been read. We can solve this by pressing $ (sync-mailbox) in Mutt, which will upload our local changes to the server. # When in Mutt, we can change folders with the c letter, but it's very cumbersome. Rather than that, it's better to redefined the keys to go to next/previous folder to Down/Up keys and to enter a folder to Right key. This tip was taken from .
bind index,pager sidebar-next bind index,pager sidebar-prev bind index,pager sidebar-open
To conclude, we must say that Mutt is definitely not for the faint-hearted, but if you take the time to customize and learn it, then you'll be given a powerful and lightweight tool for your day to day usage.
The most prevalent reason why I turned to Mutt was because I had quite a few email addresses and keeping up to date with them was a nightmare. But when I got a very important email that I didn't see for two weeks, because I simply hadn't had time to login to the web interface, I had enough. I switched to Thunderbird, but it was very cumbersome and very slow especially when I was running a lot of different things at the same time, like virtualized virtual machines. It also consumed a fair amount of memory, so my system was often slow and unresponsive. At that time I thought that it must be a better alternative, which is when I found Mutt and quickly fell in love: after the first few stitches and glitches.
Now I'm a satisfied user of Mutt, which brought order to my emailing system that I love to use on a daily basis. But what's more important, it doesn't slow down my computer one single bit.
 A unix style mail setup, http://dev.gentoo.org/~tomka/mail.html.
 Msmtp, https://wiki.archlinux.org/index.php/msmtp.
 OfflineIMAP, https://wiki.archlinux.org/index.php/OfflineIMAP.
 Mailcheck, http://mailcheck.sourceforge.net/.
 My Gmail workflow with mutt, offlineimap, mailcheck, wmbiff, some shell scripts and notify-send, http://crmafra.blogspot.com/2012/01/my-gmail-email-with-mutt-offlineimap.html.
 Mutt Advanced Usage, http://www.mutt.org/doc/manual/manual-4.html.
 Installing and Using Python Keyring Lib, https://pypi.python.org/pypi/keyring
 Mutt Getting Started, http://www.mutt.org/doc/manual/manual-2.html.
 [network-manager-applet] Unable to access secrets service, https://bugs.archlinux.org/task/18930.
 The Homely Mutt, http://stevelosh.com/blog/2012/10/the-homely-mutt/#running.
 abook(1) - Linux man page, http://linux.die.net/man/1/abook.
 Everything You Need To Know To Start Using GnuPG with Mutt, [http://codesorcery.net/old/mutt/mutt-gnupg-howto].
 mutt and friends, tools in the Email Toolchain, [https://dgoodmaniii.wordpress.com/tag/muttprint/ ].
 Mairix, http://pbrisbin.com/posts/mairix/.