Recegnition of Repeating SMS Patterns

With the advances in e-mail spam recognition and user awareness, spammers are moving towards less researched media. One of those is the short messaging system (SMS), which boasts high availability and open rates. Those characteristics are also attractive to legitimate businesses that need to send short, bulk messages to their clients. However, while these messages might be solicited by the end-user, they might represent a loss for the SMS service provider, as these businesses often misuse unlimited SMS plans meant for regular customers to avoid paying for more expensive solutions designated for them. It is therefore desirable to be able to recognize both unsolicited and solicited bulk messages. Bulk messages are generally generated from a template. The goal of this work is to design a clustering algorithm that treats a message as a sequence of lexical units (words), and evaluate it's effectiveness compared to a locality sensitivity hashing method that treats the message as a string of symbols. The work evaluates the suitability of the Smith-Waterman alignment algorithm for this task. The work details why Smith-Waterman (and other local alignment techniques) is unsuitable, and how it can be replaced by Needleman-Wunsch (global alignment) to produce much better results. The resulting algorithm is able to cluster real messages into campaigns satisfactorily, and performs well even in situations where the benchmark locality sensitivity hashing method fragments campaigns.

Smith-Waterman, Needleman-Wunsch, SMS, spam, sequence alignment, string clustering