Friday, December 29, 2006

Current Spam Techniques and Trends


Everytime I look at the spam folder of my e-mail account, I observe that existing spamming techniques have evolved rapidly - faster than I would have liked!

I would like to express my observations regarding the current scenario of e-mail spam.

Previously (a couple of years back) we had spam e-mail as a regular visitor to our e-mail accounts. However, detecting these mails were a lot easier since they used to be composed of almost the same set of keywords that were present in almost every other junk mail. Our old junk mail filter used to check incoming mails for the frequency of all words in it. It would then draw a comparison between the frequencies of the words in the incoming mail against the frequencies of the words in standard spam mail. So the spam filters were constructed using a Bayesian probabilistic mechanism in which the incoming e-mail was classified as spam if the frequency of certain words in it followed the frequency of the same words in standard spam mail. There were certain other variations to this technique; however the practiced mode still remained probabilistic measures and word frequency analysis.

Coming back to the point I was trying to place, I opened my spam folder in my e-mail account to check for some false positives (genuine mail or "ham" that has been incorrectly classified as "spam") that I was expecting. The following are my observations from the nature of mail in the spam folder:

Standard spam e-mails come from legitimate looking addresses and he sender names are not some cryptic business or company names - they are names that are commonly used. (E.g. Mary Sanders): The sender e-mail addresses are often genuine e-mail addresses that have been hacked or infested with a worm that has resulted in compromising the account (converted it to a spam sender). A couple of years back; we could have easily identified spam by looking at the sender name or the sender e-mail address. However, in recent times, spam is being transformed into a form that represents a genuine e-mail. Indeed it has become difficult to distinguish between a standard spam and a ham today even by visual inspection. Imagine the difficulties faced by an automated tool that is supposed to classify the messages coming in daily into your inbox as spam or ham.

These mails have a friendly subject line (often prefixed by a Re: to indicate that the e-mail was a reply from one of your sent mails): I was surprised to see that some of the mails prefixed by the subject line Re: contain reply attachments which indicate that I sent a mail to that address and the junk mail is simply a reply to what I sent. I am yet to comprehend whether my account has been compromised as in the previous observation and the Re: prefixed e-mails I receive are simple challenge responses sent by the receiving e-mail address server. Alternatively, this might be another twist in the tale from the spammers where they construct the spam mail by disguising it as a reply to a genuine e-mail.

The content of these e-mails generally contain strings of literary works (mostly quoted from popular literary works such as Macbeth, Hamlet, etc): Spammers have now started sending fragments of well known literary works along with the e-mails in order to damage the frequency database that standard anti-spam utilities use. If we are using one of the dated Bayesian anti-spam utilities and we start classifying modern e-mail as spam, then it is very likely that the spam-word-frequency database will be unnecessarily adulterated (Bayesian poisoning) with the wrong frequencies and result in more false positives than warranted. The solution lies in updating your anti-spam filter with one which uses a more of a heuristic approach rather than a simple probabilistic estimate.

It is possible for the spammer to disguise certain words that the spam filter recognizes as spam mail text: For example, it is common for a spam filter to filter out mail containing the word viagra, for example. Unfortunately, it has become common for spammers to disguise these words with intermediate special characters so that they do not appear as the words under the scanner (for example, the word viagr@ can be used instead). With html mails becoming commonplace, invalid html tags are also used in between these words for the purpose of throwing the spam filter off-track. Ironically, since the html interpreter ignores invalid tags, the text appears to the recipient as a normal message which serves the spammer's purpose.

Some spam mails contain an image attachment that contains the advertisement: Image attachments sent along with spam mails are very difficult to detect and classify as spam. Spammers have resorted to sending their advertisements as images that contain the ad. This is indeed difficult for an automated system to detect. Even if we used an optical character recognition utility, the process would not be successful as I have noted that the characters in the images are often scrawled or blurred and the background littered with misleading pixels (as in standard Image verification text that appear on web-forms to prevent automated form submissions). These result in the image being almost indecipherable to the OCR utility, but at the same time the characters remain visually discernable to the person viewing it.

It is not always a requirement that a person has to fill up his e-mail address in a web-form in order to receive spam mail: We might receive spam even if we use an account that has not been given away in a web-form that compromises your mail account to spammers. Nowadays spammers have also resorted to generating lists of arbitrary e-mail addresses that contain commonly used names or phrases. These addresses become the targets of spam mail. The solution would lie in judicious use of the special characters in your address that the account provider permits. For example, it might be wise to use Sanjit_11_b@xyz.com rather than using Sanjit@xyz.com. But the drawback of this technique might be that people will tend to forget your e-mail addresses more often.

All said and done, spam is becoming a major problem for us all. I usually get around 20 messages a day in my spam folder (people might get less if they are luckier). Some false negatives (spam which is not detected) also creep in sometimes in my inbox and a few false positives into my spam folder as well. Cyber laws will hardly be a deterrent to this nuisance. And as we all know cyber laws mostly reside in cyberspace where they are known to exist but hardly ever exercised, leaving it all to the authorities would result in getting stuck with this nuisance for God knows how long. Getting a proper solution to this problem would perhaps be something that we’ll forever dream of as spamming techniques are clearly outrunning the anti-spam filter technologies. We could go on and change the existing e-mail system and come up with a new one which enforces security, digital signatures and so on, but even then newer techniques to counter the security measures are sure to crop up and leave us baffled.

For now, the best we can do is hope for something to develop while we keep on clicking the “Report as Spam” & “Delete” buttons for the spam that we receive.


1 comment:

Arnab said...

Ha ha ha sanjit you posted against spam and just see you have a spam comment...
You are true that it is really difficult.
But amazingly google's spam filter works perfectly for me.... Just great...and exact... I get 50 spams a day TRUE but they are all safe in my spam folder..... Go to that folder and delete it.....
Nice information sanjit loved it......