Had a spare hour last Thursday and decided to write a little twitter bot. There he is above. His name is Grammer_Man and he corrects other twitter users’ misspellings, using data scraped from these Wikipedia pages.

Responses have been pouring in already, some agitated, some confused, but most positive — which was a pleasant surprise. In any event, the minimal amount of effort in coding has paid off many times over in entertainment.

You can see who’s responding at the moment by searching for @grammer_man, and also by checking his list of favourites.

Here is the (somewhat slapdash) code that powers our fearless spelling Nazi:

grabber.py

This module grabs the spelling data from Wikipedia.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os
import pickle

import requests
from BeautifulSoup import BeautifulSoup

def grab(letter):
    '''
    Grabs spellings from wikipedia
    '''
    url = 'http://en.wikipedia.org/wiki/Wikipedia:Lists_of_common_misspellings/%s' % letter
    html = requests.get(url).content
    soup = BeautifulSoup(html)
    bullets = soup.findAll('li')
    retval = {}
    for bullet in bullets:
        if 'plainlinks' in repr(bullet):
            values = bullet.text.split('(')
            if len(values) == 2:
                retval[values[0]] = values[1][:-1] # shave off the ) at end
    return retval

def get_spellings():
    '''
    Returns a dictionary of {false: correct} spellings
    '''
    if not os.path.exists('words.pkl'):
        retval = {}
        for c in 'ABCDEFGHIJKLMNOPQRSTUVWXYZ':
            print 'Getting typos - %s' % c
            retval.update(grab(c))
        print 'Dumping...'
        f = open('words.pkl', 'w')
        pickle.dump(retval, f)
        f.close()
        return retval
    else:
        f = open('words.pkl', 'r')
        retval = pickle.load(f)
        f.close()
        return retval
    
if __name__ == '__main__':
    get_spellings()

bot.py

The bot. Selects misspellings at random, searches for them, responds to them, while also taking breaks between tweets and longer breaks every few hours.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os
import random
import time
import pickle

import twitter

from grabber import get_spellings

API = twitter.Api(consumer_key='XXX',
                  consumer_secret='XXX',
                  access_token_key='XXX',
                  access_token_secret='XXX')

MESSAGES = u'''
$USERNAME sooo you might wanna spell $CORRECT the right way next time!! Not your fault bro.
#
# All messages stored in here, one per line. 
# Edited out in order to save space in this blog post.
#
'''.split('\n')

def compose_message(twitter_post, mistake, correct):
    '''
    Choose a message from MESSAGES at random, substitute fields to personalise it and 
    check if it exceeds the twitter message limit. Try this 100 times before failing.
    '''
    retries = 0
    while retries < 100:
        retries += 1
        message = MESSAGES[random.randint(0, len(MESSAGES) - 1)]
        message = message.replace('$USERNAME', '@%s' % twitter_post.user.screen_name)
        message = message.replace('$MISTAKE', '"%s"' % mistake).replace('$CORRECT', '"%s"' % correct)
        if message and len(message) < 141:
            return message
    return None

def correct_spelling(twitter_post, mistake, correct):
    '''
    Correct someone's spelling in a twitter_post
    '''
    print u'Correcting @%s for using %s...' %(twitter_post.user.screen_name,
                                            mistake)
    message = compose_message(twitter_post, mistake, correct)
    if not message:
        print u'All messages were too long... Aborting...'
        return None
    else:
        API.PostUpdate(message, in_reply_to_status_id=twitter_post.id)
        return True

def search(word):
    '''
    Search twitter for uses of a word, return one if it's been used recently.
    Otherwise return None.
    
    TODO: Add time awareness.
    '''
    print 'Searching for uses of %s...' % word
    results = API.GetSearch(word)
    if results:
        for result in results:
            if not check_if_done(result.id) and not result.user.screen_name == 'grammer_man' and word in result.text:
                return result
    return None

def check_if_done(id):
    '''
    Checks if a tweet has already been responded to
    '''
    if os.path.exists('done.pkl'):
        f = open('done.pkl', 'r')
        done = pickle.load(f)
        f.close()
        if id in done:
            return True
    return False

def update_done(id):
    '''
    Updates a list of tweets that've been replied to
    '''
    if os.path.exists('done.pkl'):
        f = open('done.pkl', 'r')
        done = pickle.load(f)
        f.close()
    else:
        done = []

    done.append(id)

    f = open('done.pkl', 'w')
    pickle.dump(done, f)
    f.close()

def main():
    '''
    Main program flow
    '''
    words = get_spellings()
    counter = 0
    while True:
        word = random.choice(words.keys())
        post = search(word)
        if counter > 100:
            rand_time = random.randint(120*60, 240*60)
            print 'Done %s tweets, sleeping for %s minutes' % (counter, rand_time/60)
            time.sleep(rand_time)
            counter = 0
        # TODO: PROPERLY PRUNE THE MISTAKES/CORRECTIONS FROM WIKIPEDIA AND REMOVE THIS:
        if not u',' in word + words[word] and not u';' in word + words[word]:
            if post:
                result = correct_spelling(post, word, words[word])
                if result:
                    counter += 1
                    print '#%s Done' % counter
                    update_done(post.id)
                    time.sleep(random.randint(300,500))


if __name__ == '__main__':
    main()

Grammer_Man uses the following libraries:

• python-twitter (Be warned: no proxy support)
• requests
• BeautifulSoup

I rather like the “A”, “Amer-kawh”, “and”, “for” and “great” bits.

500 / 200.

From a collective of people including the man behind King Lud’s Revenge.

The text, although written in Classical Chinese, can be easily comprehended by most educated readers. However, changes in pronunciation over 2,500 years resulted in a large degree of homophony in Classical Chinese, so the poem becomes completely incomprehensible when spoken out in Standard Mandarin or when written romanized in Standard Mandarin.

Pretty remarkable. Its Wikipedia page, linked above, is rather detailed and well worth a read.

Here’s a video of the poem being read aloud in Standard Mandarin:

More: A List of Linguistics Example Sentences

Thanks to Hugh for bringing this to my attention.

Please Say Something is a 10 minute short concerning a troubled relationship between a Cat and Mouse set in the distant Future. The final film was completed in January 2009 and contains 23 episodes of exactly 25 seconds each.

The film won the Golden Bear for best short at the 2009 Berlinale, the Cartoon D’or and several other awards. In 2010 it was given a distinction of cultural significance by the German ratings agency FBW (Prädikat Besonderes Wertvoll).

Please Say Something – Full Length from David O’Reilly.

Photo: Reuters

A select few quotations from a BBC article on Hitler’s bizarre popularity in India:

Latest reports say Bollywood is now planning to cash in. A film – Dear Friend Hitler – is due to be released by the end of the year, focusing on the dictator’s relationship with his mistress Eva Braun.

…

It’s hard to narrow down what makes the dictator popular in India, but some young people say they are attracted by his “discipline and patriotism”.

Most of them are, however, quick to add that they do not approve of his racial prejudices and the Holocaust in which millions of Jews were killed.

…

Nearly all the booksellers and publishers contacted in India say it is mainly young people who read Mein Kampf. It’s not just the autobiography – books on the Nazi leader, T-shirts, bags, bandanas and key-rings are also in demand. A shop in Pune, called Teens, says it sells nearly 100 T-shirts a month with Hitler’s image on them.

Dimple Kumari, a research associate in Pune, has not read Mein Kampf but she would wear the Hitler T-shirt out of admiration for him. She calls him “a legend” and tries to put her admiration for him in perspective: “The killing of Jews was not good, but everybody has a positive and negative side.”

I have to say, I find this peculiar naivety fascinating. I also can’t imagine what it must be like for a Western traveller to be walking down a street in, say, Bangalore, spotting a few people coming towards him clad in Hitler Apparel. Indeed, staying with Bangalore, since it’s such a huge IT hub… Should we expect to see originally well-meaning and innocuous (to Indians, that is) photographs of young IT workers on their IBM or Microsoft campus, posing happily with their corporate swipe-cards dangling from from their neck, the strap perfectly framing a portrait of their “Dear Friend Hitler”? Indeed, do such places, renowned for their lack of dress-code in the West, already have a strict dress-code in places like India, in order to prevent such embarrassments? I wonder.

And, before I go, here’s another great article from Der Spiegel on the same phenomenon, only this time in Pakistan. Yep, they’re at it too. Who knows – perhaps this new-found love for the 20th century’s most hated, genocidal dictator will only serve to foster a new friendship of shared values between India and Pakistan, leading to a stable peace! Surely no harm could come of future generations of two of the world’s most antagonistic and unstable nuclear-countries worshipping a genocidal, maniacal, militaristic dictator!

Brings a whole new sense to that Vonnegut quote…