E-mail redirection and sorting

Mail Delivery Agent - MDA - Procmail is used to deliver e-mail on Unix mail servers at FIT and FEEC. Procmail can filter incomming messages using other programs, store to arbitrary mailboxes, resend to other recepients etc. Procmail follows the rules stored in file $HOME/.procmailrc (variable $HOME is users' directory on Unix server - Eva for FIT students, Fest for FEEC students). File .procmailrc may contain variable defintions as well. If there is no .procmailrc in user's home directory all incomming mail messages are stored into file /var/mail/login where login is user's login name.

Note: Mail redirection via .forward file should be avoided since it may cause mail loops.

Attention: email accounts outside the BUT organization (e.g. Gmail or Yahoo) now require a valid DMARC, i.e. at least one of the SPF and DKIM data must be valid. Messages must therefore be sent by a server that is valid for the sender's domain or that has a valid domain digital signature. So if the sender does not have an @vut.cz email address (but e.g. @fme.vutbr.cz) and the message is forwarded to somewhere other than @vutbr.cz/@vut.cz, it may not be delivered to the forwarded email address or to the original recipient - it depends on the definition of the forwarding rule! In case of using mail forwarding to external email addresses, we recommend to define FIT mail as an additional server in the client.

Email structure

Email comprises of headers and a body. The contents of mail message is stored in the body. Headers section is group of lines in the beginning of message which contain legible email information. The format of headers is strictly defined. Many header lines are just informative and some are meaningfull for user (e.g. Subject or From headers). Email clinets usually display headers intended for user only. Each header begins with keyword followed by colon, e.g. From:. Simple email message may lok like this:

Message-ID: <20051005174105.GA4290@stud.fit.vutbr.cz>
From: "Jan Opicka" <opicka@fit.vutbr.cz>
To: <novak@fit.vutbr.cz>
Subject: Test
Date: Tue, 30 Mar 2004 10:35:47 +0200

This is mail body.

Delivery rule for procmail

The rules consist of flags, conditions and a line of action. The rule is generaly of form:

:0 [flags] [: [LockfileName] ]
zero, one or more conditions (each condition on separate line
required action (one line)

The beginning of the rule

Each rule begins with :0. Digit zero may be followed by flags. If digit zero or flags is followed by another colon procmail should use local lock file. Lock file name should be specified. If there is none the name of mailbox from action line is used followed by $LOCKEXT (default .lock). Name of implicit lock file is not possible to determine when forwarding or processing using pipe.

Locking is required for delivering to mailbox (i.e. writing file) and for processing action by external program which writes to file. Locking is not necessary and should not be used for forwarding mail to another address.

Why should we lock? Procmail may be invoked for delivering several messages for one recepient. In such situation several processes may try to write to the same file (e.g. mailbox). Without locking this would result in mixed messages and corrupted mailbox structure. With locking the first process following given rule creates locking file. When another process trie to follow the same rule an attempt to create lock file fails. The second process then waits while the first one finishes it's task and deletes locking file.

Conditions

Conditions are optional. If there is none specified action is executed on all delivered emails. If there are more than one conditions (i.e. more lines) logical operator AND is applied. An action is executed if all conditions are fulfilled (true). Conditions are extended regular expressions compatible with egrep(1). Besides egrep regural expressions there are some other special conditions which have to begin with one of the following characters:

!	Negation of condition
?	Use return code of program specified
<	True if overall message size is less than given decadic number
>	True if overall message size is bigger than given decadic number
`env` ??	Compares the rest of condition to environment variable value `env`.

Note 1: Conditions are evaluated for headers only by default. This may be changed by flags B and H. With flag B only the body is searched. With flag BH both body and headers are searched.
Note 2: By default procmail does not distinguish lower and upper case letters. This may be changed with flag D.

Action

Action line specifies what to do with message when all conditions are met. To store a message to mailbox action line contains mailbox name. To forward mail action line begins with exclamation point ! followed by destination email address. To pass a message to external program action line begins with pipe character |.

There are two types of rules - delivering and non-delivering. If delivering rule is executed procmail stops proccessing of following rules and email is considered delivered. This behaviour may be changed using c flag. Delivering rules either save messag (or just it's headers or body) or pass the message to external program or resend message to another address. If non-delivering rule is executed procmail continues with next rule. Non-delivering rules pass the message through program or filter and the result is returned back to procmail for further processing.

If the conditions are not met action is not executed (mail is not delivered) and procmail continues with next rule. If all rules in .procmailrc are processed (i.e. action in no delivering rule was executed) email is delivered to mailbox defined by $DEFAULT variable. The variable $DEFAULT is set to /var/mail/login on servers at FIT and FEEC.

Example: mail sorting rule:

# Messages from user opicka to mailbox opicka
:0:
* ^From: opicka@fit\.vutbr\.cz
mail/opicka

Description:

The first line is comment. Everything after # character to the end of line is considered to be comment and therefore ignored.
The second line begins with :0 which denotes the beginning of the rule. Colon after zero indicates lockin file should be used.
The third line starts with an asterisk which means it is a condition. ^ is metacharacter which specifies the beginning of the line. Characters \ in front of dots specify it is a character dot and not metacharacter dot representign any single character. The condition is met if the line begins with From: followed by one space and string opicka@fit.vutbr.cz.
The fourth line is an action - store message to mailbox (file) opicka in $HOME/mail directory.

Example: forward mail to another server:

:0
# do not resend errors
* !^FROM_MAILER		
! petr.novak@email.cz

Description:

The first line consits of :0 only which represents the beginning of the rule without use of any lock file (the second colon is missing). Since it is mail redirection and not the file (mailbox) write operation lock file is not needed and should not be used.
The second line is a condition - it begins with *. The firs character of the condition - ! - means the condition is reversed (negated). FROM_MAILER is not an ordinary string. It is replaced with quite complicated regular expression by procmail (see man prcmailrc). This condition ensures emails from daemons (email servers, from address like postmaster, daemon, mmdf, uucp etc.) will not be resent. Without such condition your mail may end in an infinite loop quite easilly: if a mail resent to 'petr.novak@email.cz' fails for any reason the error message is returned. It is however resent again to 'petr.novak@email.cz' which is rejected again and again until disk space is exhausted. If you intend to redirect emails don't forget to include the condition * !^FROM_MAILER.
The third line is an action. The first character ! means resend mail to specified address.

With this redirection rule any errors (e.g. mailbox full, attachement too large) are not redirected and they are stored in your local mailbox. Original sender of the message is not notified at all. This means you should check your local mailbox quite often.

The other option is to use a bit more complex rule which returns an error message to original sender. The drawback of this rule is the original sender may in - case of delivery problem - receive an error message concerning address which he/she has not used. Even worse if the mail was sent to several recepients it may not be clear which one the error is related to. But by adding the Resend-From header when sending mail, the advanced email user will find address in the headers.

:0
# don't resend errors - just in case
* !^FROM_MAILER
{
  :0 h
  ORIG=| formail -x Return-path

  :0
  |formail -a 'Resend-From: <xnovak9z@stud.fit.vut.cz>'|$SENDMAIL -oi -f "$ORIG" novak@email.cz
}

Example: resend mail to another server and store it locally:

:0 c
# don't send errors
* !^FROM_MAILER
! petr.novak@email.cz

Use of flag c instructs procmail to execute delivery rule and then continue with processing .procmailrc file. If no other delivery rule is met mail is stored to default incomming mailbox.

Example: resend email to mobile phone:

(Note: quite unreliable, depends on quailty and restrictions on mail-to-sms gateway of your mobile operator, think twice before using it, no damages compensations may be claimed)

:0
# don't send errors
* !^FROM_MAILER			
# just small messages to mobile
* < 1000			
! petr.novak@sms.oscar.cz

Similar to previous rule except second condition < 1000, which prevents resending messages larger than 1000 characters.

Regular expressions (RE)

Elementary regular expression is one character or string of characters. More complex regural expressions contain metacharacters with this meaning:

Metacharacter	Meaning in regular expression
.	Any single character except newline.
*	Any (even zero) count of repetition of preceding character or item.
?	Zero or one repetition of previous character or item.
+	At least one (or more) repetitions of previous character or item.
^	Beginning of line.
$	End of line.
[list of characters]	Any single character from the list.
[^list of characters]	Any single character except those in the list.
(string)	String is considered to be one item.
\|	OR. E.g. (abc\|cde) represents either string abc or string cde.
\<string\>	Beginning or end of word. The word is string consisting of letters, digits and underscores only. All other characters mark the boundaries of a word.
\metacharacter	Backslash suppress special meaning of metacharacter. E.g. \. represents single character dot (.), not a "backslash followed by any single character". Does not apply to character < - it's special meaning must be suppressed in a different way.
\newline	Expression continues on the next line. Spaces on the beginning of the next line are ignored.

Some regular expressions examples:

a*	Any number of characters a (even zero).
a+	One or more characters a.
a?	No or single character a.
[abuv12]	Any single character of a,b,u,v,1,2.
[0-9]	Digit.
[^abx12]	Any single character except a,b,x,1,2
[^-a-d]	Any single character which is neither hyphen (-) nor letter a,b,c,d
de\|abc	Either string 'de' or 'abc'.
(abc)*	Zero or more repetition of a string 'abc'.

Internal procmail variables H and B

Teh environment variable H contains headers of a mail while variable B contains a body.

Exaples of usage:

:0:
* H ?? ^Subject:.*money
* B ?? call toll free
mail/trashbin

The condition ?? searches the specified variable for the string described by regural expression. This rules moves into mailbox named trashbin all messages with Subject: header containing string 'money' and the body contains string 'call toll free'.

Special meaning of strings ^TO_, ^FROM_DAEMON, ^FROM_MAILER

The strings mentioned above are a sort of makro definitions which are replaced by quite complex regural expressions. String ^TO_ (both letters uppercase) checks headers containing destination of a message. Most common headers are To, Cc, Resent-To. The string ^FROM_DEAMON selects messages from most deamons and theč string ^FROM_MAILER selects messages from most of the mailservers.

^TO_ is replaced with:

(^((Original-)?(Resent-)?(To |Cc |Bcc) |(X-Envelope |Apparently(-Resent)?)-To)
:(.*[^-a-zA-Z0-9_.])?)

This expression searches headers beginning with To, Cc, Resent-To, or some other recepient specification, character : follows and then any number of characters and email address starting with character -a-zA-Z0-9_..

^FROM_DEAMON is replaced with:

(^(Mailing-List : |Precedence :.*(junk |bulk |list) |
To : Multiple recipients of |(((Resent-)?(From |Sender) |X-Envelope-From) : |
>?From )([^>]*[^(.%@a-z0-9])?(Post(ma?(st(e?r)? |n) |office) |(send)?Mail(er)? |
daemon |m(mdf |ajordomo) |n?uucp |LIST(SERV |proc) |NETSERV |o(wner |ps) |
r(e(quest |sponse) |oot) |b(ounce |bs\.smtp) |echo |mirror |s(erv(ices? |er) |
mtp(error)? |ystem) |A(dmin(istrator)? |MMGR |utoanswer))
(([^).! :a-z0-9][-_a-z0-9]*)?[%@>\t ][^<)]*(\(.*\).*)?)?$([^>] |$)))

^FROM_MAILER is replaced with:

(^(((Resent-)?(From |Sender) |X-Envelope-From) : |
>?From )([^>]*[^(.%@a-z0-9])?(Post(ma(st(er)? |n) |office) |(send)?Mail(er)? |
daemon |mmdf |n?uucp |ops |r(esponse |oot) |(bbs\.)?smtp(error)? |
s(erv(ices? |er) |ystem) |A(dmin(istrator)? |MMGR))
(([^).! :a-z0-9][-_a-z0-9]*)?[%@>\t ][^<)]*(\(.*\).*)?)?$([^>] |$))

Links to further procmail information

More detailed description of .procmailrc file may be found in manual page "man procmailrc". Many examples of procmail rules are described in manual page "man procmailex". Further reading concerning procmai program and rules construction can be found at Proctut: Procmail Tutorials or Nancy McGough's "Procmail Quick Start".

Is procmail still secure?

The procmail program is no longer actively developed. However, all known potential security issues (CVEs) are fixed in the version in use.

Back to guides

Send comments to lampa@fit.vutbr.cz