Had a spare hour last Thursday and decided to write a little twitter bot. There he is above. His name is Grammer_Man and he corrects other twitter users’ misspellings, using data scraped from these Wikipedia pages.

Responses have been pouring in already, some agitated, some confused, but most positive — which was a pleasant surprise. In any event, the minimal amount of effort in coding has paid off many times over in entertainment.

You can see who’s responding at the moment by searching for @grammer_man, and also by checking his list of favourites.

Here is the (somewhat slapdash) code that powers our fearless spelling Nazi:

grabber.py

This module grabs the spelling data from Wikipedia.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os
import pickle

import requests
from BeautifulSoup import BeautifulSoup

def grab(letter):
    '''
    Grabs spellings from wikipedia
    '''
    url = 'http://en.wikipedia.org/wiki/Wikipedia:Lists_of_common_misspellings/%s' % letter
    html = requests.get(url).content
    soup = BeautifulSoup(html)
    bullets = soup.findAll('li')
    retval = {}
    for bullet in bullets:
        if 'plainlinks' in repr(bullet):
            values = bullet.text.split('(')
            if len(values) == 2:
                retval[values[0]] = values[1][:-1] # shave off the ) at end
    return retval

def get_spellings():
    '''
    Returns a dictionary of {false: correct} spellings
    '''
    if not os.path.exists('words.pkl'):
        retval = {}
        for c in 'ABCDEFGHIJKLMNOPQRSTUVWXYZ':
            print 'Getting typos - %s' % c
            retval.update(grab(c))
        print 'Dumping...'
        f = open('words.pkl', 'w')
        pickle.dump(retval, f)
        f.close()
        return retval
    else:
        f = open('words.pkl', 'r')
        retval = pickle.load(f)
        f.close()
        return retval
    
if __name__ == '__main__':
    get_spellings()

bot.py

The bot. Selects misspellings at random, searches for them, responds to them, while also taking breaks between tweets and longer breaks every few hours.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os
import random
import time
import pickle

import twitter

from grabber import get_spellings

API = twitter.Api(consumer_key='XXX',
                  consumer_secret='XXX',
                  access_token_key='XXX',
                  access_token_secret='XXX')

MESSAGES = u'''
$USERNAME sooo you might wanna spell $CORRECT the right way next time!! Not your fault bro.
#
# All messages stored in here, one per line. 
# Edited out in order to save space in this blog post.
#
'''.split('\n')

def compose_message(twitter_post, mistake, correct):
    '''
    Choose a message from MESSAGES at random, substitute fields to personalise it and 
    check if it exceeds the twitter message limit. Try this 100 times before failing.
    '''
    retries = 0
    while retries < 100:
        retries += 1
        message = MESSAGES[random.randint(0, len(MESSAGES) - 1)]
        message = message.replace('$USERNAME', '@%s' % twitter_post.user.screen_name)
        message = message.replace('$MISTAKE', '"%s"' % mistake).replace('$CORRECT', '"%s"' % correct)
        if message and len(message) < 141:
            return message
    return None

def correct_spelling(twitter_post, mistake, correct):
    '''
    Correct someone's spelling in a twitter_post
    '''
    print u'Correcting @%s for using %s...' %(twitter_post.user.screen_name,
                                            mistake)
    message = compose_message(twitter_post, mistake, correct)
    if not message:
        print u'All messages were too long... Aborting...'
        return None
    else:
        API.PostUpdate(message, in_reply_to_status_id=twitter_post.id)
        return True

def search(word):
    '''
    Search twitter for uses of a word, return one if it's been used recently.
    Otherwise return None.
    
    TODO: Add time awareness.
    '''
    print 'Searching for uses of %s...' % word
    results = API.GetSearch(word)
    if results:
        for result in results:
            if not check_if_done(result.id) and not result.user.screen_name == 'grammer_man' and word in result.text:
                return result
    return None

def check_if_done(id):
    '''
    Checks if a tweet has already been responded to
    '''
    if os.path.exists('done.pkl'):
        f = open('done.pkl', 'r')
        done = pickle.load(f)
        f.close()
        if id in done:
            return True
    return False

def update_done(id):
    '''
    Updates a list of tweets that've been replied to
    '''
    if os.path.exists('done.pkl'):
        f = open('done.pkl', 'r')
        done = pickle.load(f)
        f.close()
    else:
        done = []

    done.append(id)

    f = open('done.pkl', 'w')
    pickle.dump(done, f)
    f.close()

def main():
    '''
    Main program flow
    '''
    words = get_spellings()
    counter = 0
    while True:
        word = random.choice(words.keys())
        post = search(word)
        if counter > 100:
            rand_time = random.randint(120*60, 240*60)
            print 'Done %s tweets, sleeping for %s minutes' % (counter, rand_time/60)
            time.sleep(rand_time)
            counter = 0
        # TODO: PROPERLY PRUNE THE MISTAKES/CORRECTIONS FROM WIKIPEDIA AND REMOVE THIS:
        if not u',' in word + words[word] and not u';' in word + words[word]:
            if post:
                result = correct_spelling(post, word, words[word])
                if result:
                    counter += 1
                    print '#%s Done' % counter
                    update_done(post.id)
                    time.sleep(random.randint(300,500))


if __name__ == '__main__':
    main()

Grammer_Man uses the following libraries:

• python-twitter (Be warned: no proxy support)
• requests
• BeautifulSoup

Dearest creature in creation
Studying English pronunciation,
   I will teach you in my verse
   Sounds like corpse, corps, horse and worse.

I will keep you, Susy, busy,
Make your head with heat grow dizzy;
   Tear in eye, your dress you’ll tear;
   Queer, fair seer, hear my prayer.

Pray, console your loving poet,
Make my coat look new, dear, sewit!
   Just compare heart, hear and heard,
   Dies and diet, lord and word.

Sword and sward, retain and Britain
(Mind the latter how it’s written).
   Made has not the sound of bade,
   Say-said, pay-paid, laid but plaid.

Now I surely will not plague you
With such words as vague and ague,
   But be careful how you speak,
   Say: gush, bush, steak, streak, break, bleak ,

Previous, precious, fuchsia, via
Recipe, pipe, studding-sail, choir;
   Woven, oven, how and low,
   Script, receipt, shoe, poem, toe.

Say, expecting fraud and trickery:
Daughter, laughter and Terpsichore,
   Branch, ranch, measles, topsails, aisles,
   Missiles, similes, reviles.

Wholly, holly, signal, signing,
Same, examining, but mining,
   Scholar, vicar, and cigar,
   Solar, mica, war and far.

From “desire”: desirable-admirable from “admire”,
Lumber, plumber, bier, but brier,
   Topsham, brougham, renown, but known,
   Knowledge, done, lone, gone, none, tone,

One, anemone, Balmoral,
Kitchen, lichen, laundry, laurel.
   Gertrude, German, wind and wind,
   Beau, kind, kindred, queue, mankind,

Tortoise, turquoise, chamois-leather,
Reading, Reading, heathen, heather.
   This phonetic labyrinth
   Gives moss, gross, brook, brooch, ninth, plinth.

Have you ever yet endeavoured
To pronounce revered and severed,
   Demon, lemon, ghoul, foul, soul,
   Peter, petrol and patrol?

Billet does not end like ballet;
Bouquet, wallet, mallet, chalet.
   Blood and flood are not like food,
   Nor is mould like should and would.

Banquet is not nearly parquet,
Which exactly rhymes with khaki.
   Discount, viscount, load and broad,
   Toward, to forward, to reward,

Ricocheted and crocheting, croquet?
Right! Your pronunciation’s OK.
   Rounded, wounded, grieve and sieve,
   Friend and fiend, alive and live.

Is your r correct in higher?
Keats asserts it rhymes Thalia.
   Hugh, but hug, and hood, but hoot,
   Buoyant, minute, but minute.

Say abscission with precision,
Now: position and transition;
   Would it tally with my rhyme
   If I mentioned paradigm?

Twopence, threepence, tease are easy,
But cease, crease, grease and greasy?
   Cornice, nice, valise, revise,
   Rabies, but lullabies.

Of such puzzling words as nauseous,
Rhyming well with cautious, tortious,
   You’ll envelop lists, I hope,
   In a linen envelope.

Would you like some more? You’ll have it!
Affidavit, David, davit.
   To abjure, to perjure. Sheik
   Does not sound like Czech but ache.

Liberty, library, heave and heaven,
Rachel, loch, moustache, eleven.
   We say hallowed, but allowed,
   People, leopard, towed but vowed.

Mark the difference, moreover,
Between mover, plover, Dover.
   Leeches, breeches, wise, precise,
   Chalice, but police and lice,

Camel, constable, unstable,
Principle, disciple, label.
   Petal, penal, and canal,
   Wait, surmise, plait, promise, pal,

Suit, suite, ruin. Circuit, conduit
Rhyme with “shirk it” and “beyond it”,
   But it is not hard to tell
   Why it’s pall, mall, but PallMall.

Muscle, muscular, gaol, iron,
Timber, climber, bullion, lion,
   Worm and storm, chaise, chaos, chair,
   Senator, spectator, mayor,

Ivy, privy, famous; clamour
Has the a of drachm and hammer.
   Pussy, hussy and possess,
   Desert, but desert, address.

Golf, wolf, countenance, lieutenants
Hoist in lieu of flags left pennants.
   Courier, courtier, tomb, bomb, comb,
   Cow, but Cowper,some and home.

“Solder, soldier! Blood is thicker“,
Quoth he, “than liqueur or liquor“,
   Making, it is sad but true,
   In bravado, much ado.

Stranger does not rhyme with anger,
Neither does devour with clangour.
   Pilot, pivot, gaunt, but aunt,
   Font, front, wont, want, grand and grant.

Arsenic, specific, scenic,
Relic, rhetoric, hygienic.
   Gooseberry, goose, and close, but close,
   Paradise, rise, rose, and dose.

Say inveigh, neigh, but inveigle,
Make the latter rhyme with eagle.
   Mind! Meandering but mean,
   Valentine and magazine.

And I bet you, dear, a penny,
You say mani-(fold) like many,
   Which is wrong. Say rapier, pier,
   Tier (one who ties), but tier.

Arch, archangel; pray, does erring
Rhyme with herring or with stirring?
   Prison, bison, treasure trove,
   Treason, hover, cover, cove,

Perseverance, severance. Ribald
Rhymes (but piebald doesn’t) with nibbled.
   Phaeton, paean, gnat, ghat, gnaw,
   Lien, psychic, shone, bone, pshaw.

Don’t be down, my own, but rough it,
And distinguish buffet, buffet;
   Brood, stood, roof, rook, school, wool, boon,
   Worcester, Boleyn, to impugn.

Say in sounds correct and sterling
Hearse, hear, hearken, year and yearling.
   Evil, devil, mezzotint,
   Mind the z! (A gentle hint.)

Now you need not pay attention
To such sounds as I don’t mention,
   Sounds like pores, pause, pours and paws,
   Rhyming with the pronoun yours;

Nor are proper names included,
Though I often heard, as you did,
   Funny rhymes to unicorn,
   Yes, you know them, Vaughan and Strachan.

No, my maiden, coy and comely,
I don’t want to speak of Cholmondeley.
   No. Yet Froude compared with proud
   Is no better than McLeod.

But mind trivial and vial,
Tripod, menial, denial,
   Troll and trolley, realm and ream,
   Schedule, mischief, schism, and scheme.

Argil, gill, Argyll, gill. Surely
May be made to rhyme with Raleigh,
   But you’re not supposed to say
   Piquet rhymes with sobriquet.

Had this invalid invalid
Worthless documents? How pallid,
   How uncouth he, couchant, looked,
   When for Portsmouth I had booked!

Zeus, Thebes, Thales, Aphrodite,
Paramour, enamoured, flighty,
   Episodes, antipodes,
   Acquiesce, and obsequies.

Please don’t monkey with the geyser,
Don’t peel ‘taters with my razor,
   Rather say in accents pure:
   Nature, stature and mature.

Pious, impious, limb, climb, glumly,
Worsted, worsted, crumbly, dumbly,
   Conquer, conquest, vase, phase, fan,
   Wan, sedan and artisan.

The th will surely trouble you
More than r, ch or w.
   Say then these phonetic gems:
   Thomas, thyme, Theresa, Thames.

Thompson, Chatham, Waltham, Streatham,
There are more but I forget ‘em-
   Wait! I’ve got it: Anthony,
   Lighten your anxiety.

The archaic word albeit
Does not rhyme with eight-you see it;
   With and forthwith, one has voice,
   One has not, you make your choice.

Shoes, goes, does *. Now first say: finger;
Then say: singer, ginger, linger.
   Real, zeal, mauve, gauze and gauge,
   Marriage, foliage, mirage, age,

Hero, heron, query, very,
Parry, tarry fury, bury,
   Dost, lost, post, and doth, cloth, loth,
   Job, Job, blossom, bosom, oath.

Faugh, oppugnant, keen oppugners,
Bowing, bowing, banjo-tuners
   Holm you know, but noes, canoes,
   Puisne, truism, use, to use?

Though the difference seems little,
We say actual, but victual,
   Seat, sweat, chaste, caste, Leigh, eight, height,
   Put, nut, granite, and unite.

Reefer does not rhyme with deafer,
Feoffer does, and zephyr, heifer.
   Dull, bull, Geoffrey, George, ate, late,
   Hint, pint, senate, but sedate.

Gaelic, Arabic, pacific,
Science, conscience, scientific;
   Tour, but our, dour, succour, four,
   Gas, alas, and Arkansas.

Say manoeuvre, yacht and vomit,
Next omit, which differs from it
   Bona fide, alibi
   Gyrate, dowry and awry.

Sea, idea, guinea, area,
Psalm, Maria, but malaria.
   Youth, south, southern, cleanse and clean,
   Doctrine, turpentine, marine.

Compare alien with Italian,
Dandelion with battalion,
   Rally with ally; yea, ye,
   Eye, I, ay, aye, whey, key, quay!

Say aver, but ever, fever,
Neither, leisure, skein, receiver.
   Never guess-it is not safe,
   We say calves, valves, half, but Ralf.

Starry,granary, canary,
Crevice, but device, and eyrie,
   Face, but preface, then grimace,
   Phlegm, phlegmatic, ass, glass, bass.

Bass, large, target, gin, give, verging,
Ought, oust,joust, and scour, but scourging;
   Ear, but earn; and ere and tear
   Do not rhyme with here but heir.

Mind the o of off and often
Which may be pronounced as orphan,
   With the sound of saw and sauce;
   Also soft, lost, cloth and cross.

Pudding, puddle, putting. Putting?
Yes: at golf it rhymes with shutting.
   Respite, spite, consent, resent.
   Liable, but Parliament.

Seven is right, but so is even,
Hyphen, roughen, nephew, Stephen,
   Monkey, donkey, clerk and jerk,
   Asp, grasp, wasp, demesne, cork, work.

A of valour, vapid vapour,
S of news (compare newspaper),
   G of gibbet, gibbon, gist,
   I of antichrist and grist,

Differ like diverse and divers,
Rivers, strivers, shivers, fivers.
   Once, but nonce, toll, doll, but roll,
   Polish, Polish, poll and poll.

Pronunciation-think of Psyche!-
Is a paling, stout and spiky.
   Won’t it make you lose your wits
   Writing groats and saying “grits”?

It’s a dark abyss or tunnel
Strewn with stones like rowlock, gunwale,
   Islington, and Isle of Wight,
   Housewife, verdict and indict.

Don’t you think so, reader, rather,
Saying lather, bather, father?
   Finally, which rhymes with enough,
   Though, through, bough, cough, hough, sough, tough??

Hiccough has the sound of sup…
My advice is: GIVE IT UP!

Thanks to Hugh for bringing this to my attention.

500 / 200.

From a collective of people including the man behind King Lud’s Revenge.

Thanks to Hugh for bringing this to my attention.

Later on, in the evening, I had been walking about the town, looking for a suitable place to have dinner when I came across an Irish pub which I decided to have a few drinks in later that night, after having eaten. A match was on that night between Manchester Utd and Besiktas for which they had the projector screen out and all. Upon entering, I attempted to take a seat at the bar, since I had absolutely no interest in the match, only to be chaperoned to the pub audience and told I must be seated with everyone else, in front of the projector screen.

It quickly became apparant that I was the only Irishman in the building: the staff were all Hungarian, there was a group of Americans closest to the screen, then directly in front of me a group of about four Englishmen, and to my left a group of about eight Turks, men and women, who were occasionally chatting to three Danes seated beside them, having dinner. Those Turks immediately to my left were rather friendly and chatty, and after a while we had exchanged pleasantries and stories explaining why and how we had wound up in an Irish pub in Budapest of all places.

One hour and many beers later, and not a goal had been scored. I grew more and more impatient, and the Turks (for whom this game seemed to mean an awful lot) grew more and more raucous. Then, out of nowhere, a shot on-target rebounded off the goal-posts. As it seems, the drink had affected my prior apathy towards the whole event, and I let an annoyed roar of “JESUS!” out of me. One of the Turks turned to me and said with a smile, “Don’t you mean Mohammad?” I responded, “Ah yeah, he’s pretty good too, just don’t draw any funny pictures of him, ye?”

The Danes exploded in laughter.

The Turks went completely silent, staring straight ahead at the projector screen.

I was at Glastonbury when Jacko died. That’s not a factual statement, but a T-shirt slogan. The day after his death, souvenir tops with “I was at Glasto 09 when Jacko died” printed on them were already on sale around the site. In fact, when Jacko died, I was at home playing Grand Theft Auto: Chinatown Wars on a Nintendo DSi. I am 38 years old.

…

The next day he was still dead, but somehow deader than the day before. He was all over the radio and papers. The TV had clips of Thriller on heavy rotation, which seemed a tad inappropriate, what with him playing a decomposing corpse in it. If Bruce Willis died falling from a skyscraper, I doubt they’d illustrate his life story by repeatedly showing that bit from Die Hard where he ties a firehose round his waist and jumps off the building.

Across all the networks, a million talking heads shared their thoughts and feelings on his death. They had rung everyone in the universe and invited them on the show. On This Morning, a Coronation Street actor revealed he had once had tickets for a Michael Jackson concert but couldn’t go because of the traffic. It was a sad day indeed. At 3pm, his death was still “BREAKING NEWS” according to Sky, which has to be some kind of record. Even 9/11 didn’t “break” that long.

He ends with a well-expressed sentiment I share completely:

But the news is not the place to “celebrate” Jackson’s music. The Glastonbury stage, the pub, the club, the office stereo, the arts documentary: that’s the place. The news should report his death, then piss off out of the way, leaving people to moonwalk and raise a toast in peace.