Bash Shell and Beyond Applied

Deleting Spam on a POP3 Server

This article will illustrate the use of my extended 'case' and 'read' Bash shell builtins (See my other articles in issues 108, 109 and 110) to delete Spam on my ISP's POP3 mail server before it gets downloaded into my local mail system. The example scripts use these extended functions, so they require that you have my shell extensions installed.

On average, I get 1 MB of spam per hour on my Yahoo account. The most troublesome of these, both in size and number, are Microsoft Swen and Netsky worms. Fortunately, they are easy to identify, and can be deleted right on the POP3 server.

Swen worms are usually 150kB in size and use all lowercase letters (with optional '-' prefix) as the MIME boundary pattern, ie.
```
    boundary="-*[a-z]+"
```

Netsky worms are about 42kB in size and use 3 different patterns for MIME boundary pattern, namely

    boundary="----=_NextPart_000_0016----=_NextPart_000_0016"
    boundary="----=_NextPart_000_001B_01C0CA80.6B015D10"
    boundary="----=_NextPart_000_001B_01C0CA81.7B015D10"

Telnet to POP3 server

In order to understand the shell script, you should first log in to your POP3 server using Telnet, because a shell script only automates what you type on the command line. So, let's do that:

    telnet pop.your.isp 110
    user username
    pass password

will connect to remote POP3 server (port 110), and log in using your 'username' and 'password'.

    stat
    top 1 10

Here, stat returns the number of messages and total size, and top 1 10 prints the header of the 1st email plus the top 10 lines of the body. For our purpose, we are only interested in the header, specifically the 'boundary' parameter; so, top 1 0 is what we need for our script. Note that a single '.' (dot) on a line by itself signals the end of output.

    dele 1
    quit

dele 1 marks the 1st message to be deleted, and quit ends the POP3 session upon which the server removes all messages marked for deletion.

Shell script

```
read3 ()                # Usage: read3
{
    read -r -u3 -D              # read from fd=3
}

send3 ()                # Usage: send3 [cmd...]
{
    echo -D "$*" 1>&3            # write to fd=3
    read3
    echo "$* --> $REPLY"
    [[ $REPLY == +OK* ]]  ||  exit 1
}
```
For this to work, you have to read (read -D) and send (echo -D) DOS lines, since the POP3 protocol specification (RFC1939) requires CRLF (\r\n) line termination. The POP3 protocol is extremely simple, in that there are only 2 possible responses from the remote server:
- +OK (with optional text), if there is no error
- -ERR (with optional text), if there was an error

check ()                # Usage: check server username password
{
    local ok n size i

    exec 3<>/dev/tcp/$1/pop3  ||  exit 1
    read3
    send3 user $2
    send3 pass $3
    send3 stat                  # +OK 11 1504321
    read ok n size <<< "$REPLY"        
    for ((i = 1; i <= n; i++)); do
        send3 top $i 0
        case `until read3; [ "$REPLY" = . ]; do echo "${REPLY#.}"; done` in
	     'boundary="-*[a-z]+"' ))
	       echo swen.0 ;;
	     'boundary="(----=_NextPart_000_0016){2}"' ))
	       echo netsky.1 ;;
	     'boundary="----=_NextPart_000_001B_01C0CA8(0\.6|1\.7)B015D10"'
	     ))
	       echo netsky.2 ;;
        esac then
            send3 dele $i
        fi
    done
    send3 quit
}

This is the main program loop. It logs in and checks for the above boundary patterns using regex(7). If there is match, then it deletes that message from the POP3 server. The type of spam is also printed to stdout. You'll notice that the exit condition of the extended 'case' statement is used here.

Usage

You can source the 3 functions and run

    check pop.your.isp username password

from the command line or in a script. However, if you use Fetchmail to download emails (like I do), then you already have servers, usernames, and passwords in ~/.fetchmailrc. You can extract these data using fetchmail --configdump directly:

    (
    fetchmail --configdump
    cat << EOF
    for server in fetchmailrc['servers']:
        if server['protocol'] == 'POP3':
            for user in server['users']:
                print server['pollname'], user['remote'], user['password']
    EOF
    ) | python | while read server user pass; do
        # use (...) to prevent 'exit' terminating entire script
        check "$server" "$user" "$pass"
    done

The entire script is available from popcheck.bash, and should be run just before Fetchmail,

    popcheck.bash && fetchmail

usually from crontab.

Summary

Although the script deals with Microsoft Swen/Netsky worms, you can add your own patterns. For example,

     'boundary="=+[0-9]+=+"' ))
       echo TAG.spam ;;
     '(Subject|From): =\?[A-Za-z0-9_-]+\?' ))
       echo non.English ;;
     'charset="(ks_c_5601-1987|euc-kr|big5|gb2312|iso-2022-jp|shift-jis)"'
     ))
       echo APIC.charset ;;
     '&lt;(5[89]|6[01]|20[23]|21[0189]|22[012])(\.[0-9]{1,3}){3}&gt;'
     ))
       echo APIC.IP ;;
     'Content-Type: text/html' ))
       echo HTML.header ;;

[BIO] I learned Unix using the original Bourne shell. And, after my journey through language wilderness, I have come full-circle back to shell. Recently, I've been patching features into Bash, giving other scripting languages a run for their money. Slackware has been my primary distribution since the beginning, because I can type. In my toolbox, I have Vim, Bash, Mutt, Tin, TeX/LaTeX, Python, Awk, Sed. Even my shell command line is in Vi-mode.

Published in Issue 110 of Linux Gazette, January 2005

<-- prev | next -->