Sredzkistraße

  • Home
  • About
  • Academic
  • Code
9 Jan 2012

@grammer_man who the fuck is this nigga and why u comin at me like that #Hoeassnigga

Had a spare hour last Thursday and decided to write a little twitter bot. There he is above. His name is Grammer_Man and he corrects other twitter users’ misspellings, using data scraped from these Wikipedia pages.

Responses have been pouring in already, some agitated, some confused, but most positive — which was a pleasant surprise. In any event, the minimal amount of effort in coding has paid off many times over in entertainment.

You can see who’s responding at the moment by searching for @grammer_man, and also by checking his list of favourites.

Here is the (somewhat slapdash) code that powers our fearless spelling Nazi:

grabber.py

This module grabs the spelling data from Wikipedia.

Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
import pickle
import requests
from BeautifulSoup import BeautifulSoup
def grab(letter):
'''
Grabs spellings from wikipedia
'''
url = 'http://en.wikipedia.org/wiki/Wikipedia:Lists_of_common_misspellings/%s' % letter
html = requests.get(url).content
soup = BeautifulSoup(html)
bullets = soup.findAll('li')
retval = {}
for bullet in bullets:
if 'plainlinks' in repr(bullet):
values = bullet.text.split('(')
if len(values) == 2:
retval[values[0]] = values[1][:-1] # shave off the ) at end
return retval
def get_spellings():
'''
Returns a dictionary of {false: correct} spellings
'''
if not os.path.exists('words.pkl'):
retval = {}
for c in 'ABCDEFGHIJKLMNOPQRSTUVWXYZ':
print 'Getting typos - %s' % c
retval.update(grab(c))
print 'Dumping...'
f = open('words.pkl', 'w')
pickle.dump(retval, f)
f.close()
return retval
else:
f = open('words.pkl', 'r')
retval = pickle.load(f)
f.close()
return retval
if __name__ == '__main__':
get_spellings()

bot.py

The bot. Selects misspellings at random, searches for them, responds to them, while also taking breaks between tweets and longer breaks every few hours.

Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
import random
import time
import pickle
import twitter
from grabber import get_spellings
API = twitter.Api(consumer_key='XXX',
consumer_secret='XXX',
access_token_key='XXX',
access_token_secret='XXX')
MESSAGES = u'''
$USERNAME sooo you might wanna spell $CORRECT the right way next time!! Not your fault bro.
#
# All messages stored in here, one per line.
# Edited out in order to save space in this blog post.
#
'''.split('\n')
def compose_message(twitter_post, mistake, correct):
'''
Choose a message from MESSAGES at random, substitute fields to personalise it and
check if it exceeds the twitter message limit. Try this 100 times before failing.
'''
retries = 0
while retries < 100:
retries += 1
message = MESSAGES[random.randint(0, len(MESSAGES) - 1)]
message = message.replace('$USERNAME', '@%s' % twitter_post.user.screen_name)
message = message.replace('$MISTAKE', '"%s"' % mistake).replace('$CORRECT', '"%s"' % correct)
if message and len(message) < 141:
return message
return None
def correct_spelling(twitter_post, mistake, correct):
'''
Correct someone's spelling in a twitter_post
'''
print u'Correcting @%s for using %s...' %(twitter_post.user.screen_name,
mistake)
message = compose_message(twitter_post, mistake, correct)
if not message:
print u'All messages were too long... Aborting...'
return None
else:
API.PostUpdate(message, in_reply_to_status_id=twitter_post.id)
return True
def search(word):
'''
Search twitter for uses of a word, return one if it's been used recently.
Otherwise return None.
TODO: Add time awareness.
'''
print 'Searching for uses of %s...' % word
results = API.GetSearch(word)
if results:
for result in results:
if not check_if_done(result.id) and not result.user.screen_name == 'grammer_man' and word in result.text:
return result
return None
def check_if_done(id):
'''
Checks if a tweet has already been responded to
'''
if os.path.exists('done.pkl'):
f = open('done.pkl', 'r')
done = pickle.load(f)
f.close()
if id in done:
return True
return False
def update_done(id):
'''
Updates a list of tweets that've been replied to
'''
if os.path.exists('done.pkl'):
f = open('done.pkl', 'r')
done = pickle.load(f)
f.close()
else:
done = []
done.append(id)
f = open('done.pkl', 'w')
pickle.dump(done, f)
f.close()
def main():
'''
Main program flow
'''
words = get_spellings()
counter = 0
while True:
word = random.choice(words.keys())
post = search(word)
if counter > 100:
rand_time = random.randint(120*60, 240*60)
print 'Done %s tweets, sleeping for %s minutes' % (counter, rand_time/60)
time.sleep(rand_time)
counter = 0
# TODO: PROPERLY PRUNE THE MISTAKES/CORRECTIONS FROM WIKIPEDIA AND REMOVE THIS:
if not u',' in word + words[word] and not u';' in word + words[word]:
if post:
result = correct_spelling(post, word, words[word])
if result:
counter += 1
print '#%s Done' % counter
update_done(post.id)
time.sleep(random.randint(300,500))
if __name__ == '__main__':
main()

Grammer_Man uses the following libraries:

  • python-twitter (Be warned: no proxy support)
  • requests
  • BeautifulSoup
9 January, 2012 at 20:06 by aengus

Posted in Code, Computers, Funny, Idiots, Internet, Oddities | 13 Comments »

6 Jan 2012

The Chaos

A poem by Gerard Nolst Trenité demonstrating the abundant irregularities of English spelling and pronunciation. More info here.  Read the rest of this entry »

6 January, 2012 at 15:02 by aengus

Posted in Art, Funny, Linguistics, Poetry, Words | No Comments »

12 Dec 2011

Cowboys and Anthrax

Great stuff here from the Bad Lip Reading Youtube channel. Check it out for plenty more.

Thanks to Hugh for bringing this to my attention.

12 December, 2011 at 21:56 by aengus

Posted in Film, Funny, Politics | No Comments »

24 Nov 2011

A Winged Victory For The Sullen Live

From their Facebook page:

for those of you who could not make to one of our US shows…our videographer friend Joshua Smelser filmed the entire Los Angles show. Enjoy!

You can watch the entire concert in high-definition below.

EDIT: Although the sound quality is pretty dreadful…

24 November, 2011 at 17:42 by aengus

Posted in Art, Music | No Comments »

23 Nov 2011

Slow Walkers

“Wake” from “Slow Walkers”, a Grouper / Lawrence English collaboration, to be released in 2012. The video is something else — “meditations for the zombie as cultural phenomena”.

Thanks to Dennis for bringing this to my attention. He also makes great ambient music himself — if you like this kind of thing, you should check his music out.

23 November, 2011 at 0:12 by aengus

Posted in Art, Film, Music | No Comments »

7 Nov 2011

Threnody for the Victims of Hiroshima (Aphex Remix)

Very unsettling.

From its vimeo page:

see original version, conducted by Krzysztof Penderecki himself, performed just before this performance, here = youtube.com/​watch?v=SFoTqF-gGxA&feature=related

see the other Aphex Twin edits for this show here = vimeo.com/​album/​1735255

see all other performances from the show here
youtube.com/​user/​fixedmachine#grid/​user/​CFB3F7A0029A764C

Live visuals by Weirdcore.
gfx programming by weirdcore & andrew benson

7 November, 2011 at 18:29 by aengus

Posted in Art, Music, War | No Comments »

6 Nov 2011

There is a Policeman Inside All Our Heads. He must be Destroyed.

I spent most of today reading this blog post by Adam Curtis. His posts are always a bit of a battle to get through, since they’re peppered with video which sometimes makes the whole experience a bit laborious, but this one — despite some of the videos being as long as 45 minutes — is just perfect. Each of the videos compliments the text exactly as it should, leaving you with the feeling that you’ve just watched an entire Curtis documentary series.

The blog post charts the decline of the revolutionary leftist student movements in Europe, England especially, focusing on the influences of Pauline Boty and Clive Goodwin, two prominent figures in the British Pop Art movement, and Herbert Marcuse, a political philosopher whose ideas had a large effect on the student movements. It all culminates in the absolutely fascinating story of Michael de Freitas (“Michael X”) whose name I’d not heard before today. It is this story to which Curtis devotes 45 minutes of your time in the form of an old BBC documentary which he’s edited down a little.

I won’t spoil any of that story, instead leaving you to read and watch for yourself.

6 November, 2011 at 18:08 by aengus

Posted in Art, Politics | 1 Comment »

2 Nov 2011

Poetry

2 November, 2011 at 22:12 by aengus

Posted in Art, Music, Oddities, Poetry | No Comments »

31 Oct 2011

s/gay/jew

I’m reposting this from the Dublin University Pirate Party blog. A pretty vicious and bone-headed article in the Irish Independent is doing the rounds at the moment, and someone had this great idea:

“I search-replaced “gay” and “homosexual” with “Jewish”, “gays” with “Jews”, “straight” and “heterosexual” with “Christian”, and “bisexual” with “agnostic”. The result is amazing”

Here it is:

AS the cliche goes, some of my best friends are Jewish. I used to live in a very Jewish area, the West Village in New York. Indeed, enjoying their nightlife and cultural atmosphere, I was even accused of ‘trading’ off the fun, with my copycat denim jacket and tartan shirt, while not actually joining them.

However, like many, I’ve recently begun to get impatient with the endless trumpeting of Jewish ‘identity’, and the growing appetite for more and more rights and privileges.

I’m not being reactionary and I’m all for Jewish rights and an end to prejudice and discrimination, and always have, but at this stage it seems as if the tables have turned and a minority community — the Jews — want to increasingly change mainstream culture to suit them.

For example, why is civil partnership not enough, and why do Jews also want marriage, a surely traditional Christian facility, which Jews used to see as patriarchal, and ‘Christian’?

Many Jews also feel this way and resist the increasing politicisation and institutionalising of Jewish life. Last week, in the Guardian, a newspaper almost obsessed with things Jewish and ‘progressive’, columnist Suzanne Moore objected to Jewish marriage on the basis that it was a conservative ‘selling-out’. Being Jewish should be edgy and experimental, she said.

But isn’t this part of the problem? Many Jews want to have it both ways. Thus Jewish magazines are full of ads endorsing late-night gyms, sex lines and a freewheeling sexual activity which would be dismissed as sleazy in Christian culture. But we also have articles that suggest a yearning for bourgeois respectability.

Likewise, travel books, such as the trendy Rough Guides, scold the mainstream ‘meat-market’ discos of foreign capitals but provide plenty of details for Jewish pick-up spots. Many red-blooded Christian men might wish that society would endorse their own ambitions with such PC gusto.

Also, on the issue of Jews adopting, it makes many of us uneasy and impatient with the idea that raising a child with Jewish parents is totally equivalent to a child being raised by its natural Christian parents. It patently is not, and it is a crazy concession to PC culture to say that it is.

I watched a Frontline programme recently on the topic and I thought I was seeing things when I heard Ivana Bacik refusing to be happy with a societal acceptance of Jewish adoption but insisting on full equality with Christian parenting. David Quinn gave the other perspective, but he was almost falling over himself to be reasonable about it, just looking for that concession that the natural, or Christian, parents were not just the same as Jewish parents.

Those expressing opposition or even concerns were shouted down in the television studio. However, from where I was watching, in a local bar, the viewers were all of the contrary opinion, and were amazed by this departure in opinions but also blankly accepting of it as part of the growing gulf which now exists between mainstream society and the liberal elites and quango-led experts who want to change and improve our lives.

For example, the Guardian now has a feature called The Three of Us in its family section, a weekly diary by one of two Jewish men raising a child with their female friend, the natural mother. Two dads, one mum — one family is the sub headline.

I don’t know about you but this strikes me as strange.

And the counter-argument that divorced kids often have three parents knocking around is fatuous and nonsense. A child has two parents, whether separated or not. However, it is one thing to have such a diary, but it also seems almost designed to offend and irritate those who do not agree with this new radical departure in parenting. Thus, last week, the writer Charlie Condou questioned the whole convention of women being seen as naturally connected to their children. (Not for nothing is the Irish Independent’s weekly supplement called Mothers and Babies.)

But no, Charlie went to the Alternative Families show in the UK and saw all the Jewish dads with their children. It’s just the same for him, it seems, and, he “stood around and chatted about the absurdity and irrelevance of the ‘biological question’”. Oh, please. What about breastfeeding?

And there are other things about the growing Jewish rights movement which make outsiders impatient and uneasy. Like, when did the Jews and lesbian community become the ‘LGBT’, an acronym that also includes agnostic and Transgender?

Sorry, but this is broadening the boundaries in a way that makes many of us understandably sceptical.

agnostic? Isn’t that reminiscent of the loose Seventies sexual experimentation? How many agnostics are there? And will the plain people of Ireland be happy with legalising rights for, and spending money on, all of this?

The new Human Rights Commissioner for Northern Ireland, Michael O’Flaherty, is a Jewish rights advocate and says that he sees all of this as part of his rights agenda. Again, I raise all these things, not out of reactionary resistance but just to question the direction and motivation of the whole sexual rights agenda.

There is also the danger surely that this insatiable demand for more and more recognition and identity (Jewish quotas?), will eventually alienate mainstream opinion and undo some of the valuable gains made in this country by, for example, David Norris and others, in eliminating prejudice and discrimination.

31 October, 2011 at 0:48 by aengus

Posted in Gay Rights, Ireland, Politics | No Comments »

27 Oct 2011

My man Diamond D is like a hired gun

I’ve been looking for this track for about 11 years now, ever since I heard it on DJ MK‘s Volume 12 mixtape. This week I finally found it.

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Diamond D & Sadat X – Feel It (Album Version)

It’s mid-90s hiphop, what’s more to be said?

27 October, 2011 at 21:55 by aengus

Posted in Music | 2 Comments »

« Older Entries
  • In My Ears

    • Cover artwork for Mandarine Girl (Album Version)
      Mandarine Girl (Album Version)
      Booka Shade
      41 minutes ago
    • Cover artwork for Train
      Train
      Paul Kalkbrenner
      44 minutes ago
    • Cover artwork for Gemini
      Gemini
      Marek Hemmann
      3 hours and 41 minutes ago
    • Cover artwork for Golden Cage
      Golden Cage
      The Whitest Boy Alive
      3 hours and 49 minutes ago
    • Cover artwork for The Router
      The Router
      Paul dB+
      3 hours and 57 minutes ago
  • CATEGORIES

    • America (10)
    • Art (48)
      • Architecture (11)
      • Design (8)
      • Photography (9)
    • Computers (17)
      • Code (7)
      • Computer Games (1)
      • Computer Science (5)
      • Cryptography (1)
      • Robotics (2)
    • Digital Rights (3)
    • Drink (1)
    • Film (7)
      • Animation (2)
      • Documentary (1)
      • Short Film (4)
    • Funny (15)
    • Gay Rights (3)
    • Germany (9)
      • Berlin (5)
      • German Language (3)
    • Guns (1)
    • History (1)
    • Idiots (11)
    • India (1)
    • Internet (24)
    • Ireland (9)
      • Irish Language (2)
      • The Troubles (1)
    • Israel / Palestine Conflict (2)
    • Media (12)
      • News (9)
      • TV (1)
    • Music (29)
      • Bad Music™ (3)
      • Downloads (3)
      • Electronic (8)
      • Experimental (5)
      • Free music (2)
      • Generative Music (1)
      • Jazz (1)
      • Live (4)
      • Music theory (2)
      • Videos (3)
    • Oddities (29)
    • Politics (29)
      • Censorship (5)
      • Far-right (1)
    • Religion (2)
    • Science (5)
    • Sports (1)
      • Football (1)
    • War (5)
    • Words (17)
      • Linguistics (7)
      • Literature (2)
      • Poetry (4)
  • Friends' Blogs

    • Jaded Isle
    • johnl.org
    • jonathan.beaton
    • Kay Doubleu
    • King Lud’s Revenge
    • Perte de Temps
  • My Other Websites

    • The Wisp Archive
  • META

    • Log in
    • Entries RSS
    • Comments RSS
    • WordPress.org
Avatars by Sterling Adventures
Creative Commons License
Sredzkistraße is proudly powered by WordPress
Design & code by Jonk, modified for Sredzkistraße by aengus.
Entries (RSS) and Comments (RSS).