The Engadget Bot … it’s ALIIIIIIVE!

Okay, I’ve been sitting on my hands for a day or two, itching to say something about this … but I resisted until Jason let the cat out of the bag, first. I consider this a mental health project, something I could hack on for fun. So, when he asked about an Engadget AIM Bot back on December 29, I decided to start hacking on it. Four days later, at the end of my New Years vacation on January 2, it was up and running and mostly functional.

For folks who aren’t sure what I’m talking about, let me back up a step. Engadget is a gadget blog that is part of the Weblogs, Inc. Network. A number of people contribute articles to it, covering all sorts of news about gadgets and technology. Traditionally, you might subscribe to its syndication feeds through an aggregator and keep track of updates to the site that way. But, what if notification of updates could be pushed to you via instant messaging instead? That’s one of the things the Engadget Bot does — it allows you to subscribe to any number of categories at Engadget and receive IM alerts when new entries are posted to those categories. For example, there’s lots of new entries being posted in the CES category because the CES tradeshow is going on right now in Vegas. To subscribe, you’d would send an IM to the screenname EngadgetBot with the message subscribe ces. Also, it seems that the bot has trouble sending IMs back to you if you don’t have it on your Buddy List, so it might be a good idea to add it to your Buddy List first.

Here’s a screenshot of an example interaction with the bot, receiving IM alerts and querying it for the latest headlines:

(That’s a screenshot of a Trililan window. Trillian is a multi-IM application for Wintel which I use regularly for my IM needs — so much so that I wrote a plugin for it called Tcllian which embeds my favorite scripting language, Tcl, so I can write scripts for it in Tcl that run inside Trillian.)

For the geeks in the audience, the Engadget Bot is written in Tcl … roughly 2,000 lines at this moment. For persistent data storage, I opted to use the light-weight SQLite 3, which has a really convenient Tcl binding. The source for the bot isn’t available, but it might be someday. A lot of it has to do with the fact that the code is embarassingly simple and I’d honestly be embarassed to have folks looking at it until I can clean it up and make it presentable.

Anyway, I really enjoyed hacking on this and feel really proud to have gotten it working. It’s very simple, but I think it’s already very useful if you’re interested in Engadget’s content and keeping up to date. I get to build something really simple because all the hard stuff — feed syndication of Engadget content, the AIM messaging network, etc. — are already all in place. This is just another example of what Web 2.0 mash-ups can enable folks to build.


UPDATE: My friend Og Maciel just blogged about the bot after I told him about it. Considering he’s on the Ubuntu team doing the translation to Brazilian Portuguese, it’s only natural that his entry is in Portugese. Way cool!

Dear Lazyweb: Please implement TagSoup in pure Tcl

Dear Lazyweb,

I find myself needing to create excerpts of HTML text with full markup, which sounds like an easy problem to solve incorrectly but is actually quite interesting and difficult to do correctly. The brief description of the problem is: take arbitrary HTML and produce an excerpt containing the first N characters of the text, including the markup but only counting the characters in the text. In other words, while “<b>bold</b>” is 11 bytes long, it only represents 4 characters of text towards the N character excerpt. This is why taking a simple substring of N bytes doesn’t work: it counts any markup as characters and worse, could break/unbalance tags, since the closing tags are likely to be truncated away.

My first thought to solving this problem simply — in under 5 minutes, would be my definition of simple — would be to write some code that walks the string, counting characters and stepping over markup, and truncating after the first N characters. This at least solves the “excerpt of N characters” portion of the problem. However, this leaves two problems unsolved: (1) what if the 100th character falls in the middle of a word, and (2) what do I do about any unclosed tags? Solving the word-boundary problem is simple: if the 100th character falls in the middle of the word, back up until the start of the word and truncate there. This does mean our excerpt will be less than N characters, but only by a word fragment which is acceptable in most cases, since I’m guessing most words are shorter than 15 characters. But, closing unclosed tags … in the odd edge case, this can get messy. If you’re starting out with well-formed HTML or XHTML, perhaps it’s a simple problem. But, in the general case, we know the world’s HTML is far from clean — plenty of it is invalid, which means creating rules to close tags based on the assumption that you’re working with well-formed input is not going to work.

There’s a well-known solution to cleaning up HTML called TagSoup and it does a magnificent job and has been packaged for easy use from the command line, which is an added bonus. The only downside is that it’s in Java and I primarily work in Tcl. Now, I can execute stuff from Tcl and grab the output, but that’s far from desirable from a performance standpoint — firing up a JVM every time I need to sanitize some HTML string would be insane. Sure, I could go through the gyrations to write a simple TCP server and make TagSoup available via network RPC, but that’d mean writing a mound of Java code and that’s a deep rathole that I want to avoid (it sure won’t take me 5 minutes). So, here’s my plea: Lazyweb, please implement TagSoup in Tcl, please.

In the meantime, I’m going to work on a simple implementation that’s based around a whole lot of assumptions about the input data I currently need to work with, but a solid, robust solution in the general case for Tcl would be really useful.

del.icio.us/dossy links since December 26, 2005 at 09:05 AM

del.icio.us/dossy (RSS) links since December 26, 2005 at 09:05 AM:

del.icio.us/dossy links since December 19, 2005 at 10:52 PM

del.icio.us/dossy (RSS) links since December 19, 2005 at 10:52 PM:

del.icio.us/dossy links since December 12, 2005 at 09:05 AM

del.icio.us/dossy (RSS) links since December 12, 2005 at 09:05 AM:

Nigerian 4-1-9 scam, now via Yahoo! IM

I’ve heard of the Nigerian 4-1-9 scam via email — numerous variations on the Advance Fee Fraud scheme — but this is the first time I’ve seen it carried out via instant messaging! Tonight, I received an IM from some random name on Yahoo! IM. Here’s the transcript:

[21:00] samuelmylove2008: hi
[21:00] samuelmylove2008: hello asl
[21:00] samuelmylove2008: <ding>
[21:01] samuelmylove2008: were are u from sweeti
[21:05] samuelmylove2008: <ding>
[21:05] dossy: ?
[21:05] samuelmylove2008: were are u from
[21:06] dossy: who are you and why are you IM’ing me?
[21:06] samuelmylove2008: are u there what can i do for u
[21:06] samuelmylove2008: am mary am from florida
[21:06] samuelmylove2008: u
[21:06] samuelmylove2008: ?
[21:07] dossy: how’s the weather in florida?
[21:07] samuelmylove2008: well am not right now in florida am in africa
[21:08] dossy: right.
[21:08] dossy: and I bet you work for a bank. or you’re the son of royalty. right.
[21:09] samuelmylove2008: what do u do?
[21:09] dossy: I avoid scams like this. Thanks.
[21:10] samuelmylove2008: what do u mean
[21:10] samuelmylove2008: i dont no much about u
[21:13] dossy: Where in Africa are you?
[21:14] samuelmylove2008: am in nigeria were the meeting was hold
[21:14] samuelmylove2008: i hope u dont mind?
[21:14] dossy: What are you doing in Nigeria?
[21:15] samuelmylove2008: a littel host by our boss
[21:17] dossy: Who’s your boss?
[21:18] samuelmylove2008: oh my managing director
[21:19] samuelmylove2008: we went there to cash a chekq but the chekq was not from nigeria is made from usa
[21:20] samuelmylove2008: sweety can u help me cash the chek only that my boss was not with us and even we dont have money to come back
[21:20] dossy: that’s a pity. Florida will miss you.
[21:20] samuelmylove2008: sweety can u help me to cash the chekq
[21:21] dossy: The Nigerian scam works better if you spell things right.
[21:21] dossy: And claim to be some Nigerian royalty. People really dig royalty.
[21:22] dossy: And you have to use really big numbers, like “TWO HUNDRED AND FIFTY MILLION (250,000,000) DOLLARS” — need lots of zeros.
[21:23] dossy: You want to go create a new screen name and try it again? You should practice more and get better before you try this on anyone else, you know?
[21:24] samuelmylove2008: what do u mean
[21:26] dossy: *beep beep* That’s the horn of the turnip truck, leaving you behind. You’d better run if you want to get back on it!
[21:27] samuelmylove2008: what do u mean by that
[21:28] samuelmylove2008: <ding>
[21:28] dossy: So, where were you born?
[21:28] samuelmylove2008: am born in usa but my mom and dad bass in africa
[21:29] dossy: Where did you grow up?
[21:30] samuelmylove2008: i grow up frm usa
[21:33] dossy: Where did you go to school?
[21:40] dossy: You know, right now, I bet there’s a Nigerian village that’s missing an idiot.

I couldn’t make this kinda stuff up. I’m still laughing.

del.icio.us/dossy links since December 5, 2005 at 09:05 AM

del.icio.us/dossy (RSS) links since December 5, 2005 at 09:05 AM:

This I believe: Everything is a manifestation of God

For those who have been reading my blog for a while might remember me mentioning that we attended church a few years back. Fast forward through a few more holidays where we continued to attend church … then to more regular attendance of church … then to our kids attending Sunday school … and then now. By the time I sat down to write this, the day has past and I’ve finished cleaning up around the house, but yesterday (December 11th), my kids and I were baptized at The Evangelical Lutheran Church of Our Savior, nearby in Pompton Plains, NJ. For those that know me, this might come as a surprise: did Dossy really find religion? What was I thinking? Let me try to explain …

As I said, we’ve been attending church on a pretty regular basis, once every other week or so, for the last few months. I’ve been really enjoying the sermons delivered by Pastor Rossman, and often find crying over them. Strangely, it’s a good feeling; after so many years of bitter angst and cynicism, finding something that moves me enough to actually cry helps me reconnect with these important feelings I’ve suffocated for so long. We’ve discovered that some of the people we meet, say, at our daughter’s school, who we thought we’d like and would get along with, we find out later also attend the same church! It really attracts the kind of people we really like, and that’s a very good sign. The people we’ve met at church are fantastic people, all very loving, caring and supportive — exactly the kind of people we need to surround ourselves with, struggling to raise two very bright, challenging and spirited daughters. We are starting to build healthy and positive relationships with people who are good for us mentally and emotionally. Without sounding corny, Kelly, our babysitter, who introduced us to her church, is the best thing to ever happen to us in our lives (next to the birth of our two daughters, of course). She has touched our lives in a way that will stay with us forever and we will always be thankful for that.

But, what of all that atheistic cynicism I used to hold onto so dearly? Why give all that comfort up and look to God? Well, around the time we started contemplating getting the girls baptized — as well as myself, since I don’t believe I ever was — I came across Penn Jillette’s fantastic essay from NPR, titled This I Believe: There Is No God (which inspired the title for this blog entry). While Penn (of Penn & Teller fame) is a professional funny-man who I’ve enjoyed for many years, the essay feels nothing but absolutely serious. Apparently his essay has stirred a lot of controversy. Why is this all relevant? Well, just as I was getting cozy with the idea of being baptized and joining a church and all, along comes Penn’s essay which very strongly called out to my former self: the religion-disparaging, God-denying nihilist. What was I to do?

Clearly, I agreed with what Penn wrote, so did that mean I should forget about church and being baptized and all that? Was that a necessary conclusion to agreeing with the essay? At first, I thought it might, but after reading it over a few times, I realized that the essay was just a written form of his talent: the misdirection and slight-of-hand (or, word in this case) that fools you into perceiving what isn’t really there. The truth is, if you read his words and what they really mean, as opposed to what semantics people tend to apply to the words he chose to use, he isn’t rejecting God, just refusing to believe — or, merely have faith — that there is a God. Look in the dictionary at the definition of “believe” and then ask yourself if it really is healthy behavior to believe in God? To have faith, to depend on, to accept as true or real, to expect, to trust? I think there’s a word that describes people who believe something is real that doesn’t exist: crazy. Oh, sure, they’ll swear up and down that it’s real, it’s real to them, and others might even collaborate their delusions, but does that make it any more real, or just make them more insane? After all this, how can I believe in God?

Maybe I don’t, well, not the way you might believe in God, or the way others do. But, with certainty, I know I do believe in God. Maybe I don’t believe there is a God, or a singular figure that lives somewhere and is responsible for everything that’s happened or will happen or does happen. But, then, what do I believe? How can I consider myself sane but believe in something that doesn’t really exist in a way that can be proven to exist? Well, let me ask you: do you believe in the concept of “infinity?” Of course, you probably do. You probably learned about it in school, or someone else explained it to you, or you just figured it out all a priori because you’re so smart like that. But, has have you ever found an infinity? Or counted to it? Or measured something infinite? Of course not. You can’t, it doesn’t exist. But you believe in the concept, and you consider yourself sane, right? Why should believing in God be so different? It isn’t. It’s that simple.

So then, what does it mean to believe in God? It’s exactly what Penn’s essay says: we own our actions, our thoughts, our relationships, our mistakes, our failures, our successes, everything. Everything is a manifestation of God. At the deepest level, our individual conscience is God’s voice, speaking to us. In healthy people, it makes you feel bad when you do something wrong. It guides your thoughts, your actions, and your perception of the world around you. Clearly, the conscience doesn’t control people, because we see people doing unconscionable things every day; maybe that’s the influence of the Devil, or maybe just the refusal to listen to the spirit of God’s influence, or maybe it’s just people choosing to be jerks — I don’t know. What I know is, God wants us to be loved, to feel loved and to share love with others — and listening to God, our conscience, will bring us closer to that. As Penn says, it’s the reason to “be more thoughtful” and “treat people right the first time around”. It’s why we should listen to each other, learn from each other and share ourselves with each other. It’s why we should do things to lessen other people’s suffering and not like it’s just some unjust cosmic punishment. It’s why we shouldn’t just believe there’s a God, but to prove it by listening to our conscience and living life in a way that we can be proud of. Only believing there is a God is not enough. Actions speak louder than words, for sure.

And so, yesterday, I was baptized along with my two daughters. I was welcomed into a new family, one of God, of our church and its people. In a sense, I am truly re-born, a child of God, learning for the first time that the life I knew will be different from the life I will get to know. I want to know God’s love — the love of everything — and to learn to enjoy this wonderful life I’ve been given, rather than just getting by, passing time until it’s over. Just like Penn, I want to “make this life the best life I will ever have.” He’s so very right. He’s a smart guy.

del.icio.us/dossy links since November 28, 2005 at 09:05 AM

del.icio.us/dossy (RSS) links since November 28, 2005 at 09:05 AM:

Ensign Crusher, set phasers to “All In”!

(Alternate title: OMG, Wil Wheaton blogs and plays poker online!)

Wil Wheaton, who most people know as either Gordie in Stand By Me (IMDb) or Wesley Crusher on Star Trek, has long since left those roles behind. These days, he’s got his own blog (which, after experiencing technical difficulties, has been moved here, to TypePad). He’s also written two books (Just a Geek and Dancing Barefoot), and actively plays online poker at PokerStars.com. He’s even started a podcast recently called Radio Free Burrito Episode Zero (MP3 [27.3MB]) where he answers some questions asked by his blog readers.

Why am I blogging about him? Mostly out of awe and admiration, I guess. I mean, he’s a father like me, he finds the inspiration to write, he finds the time for his geeky interests and to play poker online. On top of it all, he seems like a really nice person, someone I wish I could be friends with in meatspace. (If you knew me, you’d know why that’s a compliment, considering I’m patently misanthropic in nature.) Alas, he lives out on the West Coast and I’m all the way on the East Coast.

Anyway, I say he’s cool and if you’re curious as to what he’s been up to since he disappeared from the Hollywood limelight, you should go check him out. He’s been busy and it all looks like good stuff.