Evaluate Tcl scripts as a UDF in MySQL

In the previous blog entry, I mentioned evaluating Tcl scripts from within MySQL using the User-Defined Function (UDF) capability already in MySQL. For example, it would allow you to do this:

CREATE FUNCTION TCL RETURNS STRING SONAME 'libtcl-mysql-udf.so';

SELECT TCL('expr {2 + 2}');

Well, after a few hours of productive coding and hacking, I want to share tcl-mysql-udf-0.1.tar.gz with you.

I’ve done very limited testing on this and on very little sleep, so I wholly expect there to be really stupid mistakes in there–but, it does “work” on my personal Debian 4.0 Linux and MySQL 5.1.20 setup with Tcl 8.4.12. If you want to play around with this, I strongly urge you to read over the README included in the tarball, as it should have all the relevant bits of information necessary to get you started.

A quick demonstration of what’s currently possible with this version:

mysql> SELECT TCL('info patchlevel') AS script;
+--------+
| script |
+--------+
| 8.4.12 |
+--------+
1 row in set (0.00 sec)

mysql> SELECT TCL('info commands') AS script \G
*************************** 1. row ***************************
script: tell socket subst open eof pwd glob list exec pid time eval lrange fblocked lsearch gets case lappend proc break variable llength return linsert error catch clock info split array if fconfigure concat join lreplace source fcopy global switch update close
1 row in set (0.00 sec)

Currently, Tcl errors result in the function returning NULL–I haven’t figured out how to propagate the error string back up, yet.

mysql> SELECT TCL('error foo') AS script;
+--------+
| script |
+--------+
| NULL   |
+--------+
1 row in set (0.01 sec)

Also, the maximum length string that the function can return is currently hard-coded to 255 characters.

mysql> SELECT LENGTH(TCL('string repeat x 1024')) AS script;
+--------+
| script |
+--------+
|    255 |
+--------+
1 row in set (0.07 sec)

I implemented the function as accepting a variable number of arguments, the first being the Tcl script, followed by values that get appended to the $args Tcl variable–as though it were a Tcl proc call, in a sense.

mysql> SELECT TCL('concat [llength $args] $args', 'abc', 123, NOW()) AS script;
+---------------------------------+
| script                          |
+---------------------------------+
| 3 abc 123 {2007-08-25 08:50:35} |
+---------------------------------+
1 row in set (0.05 sec)

If you’re familiar with Tcl and MySQL and aren’t afraid of compiling something and fooling around with it in your dev playground, I’d love to hear what you think of this code. Can you see any use for it? What else would it need to be able to do before it would be useful to you?

Tags: , , ,

No happiness using C-Store as a MySQL storage engine

As the second day of MySQL Camp II comes to an end, all my hacking on a new MySQL pluggable storage engine using C-Store meets a dead-end.

Chad Miller and I worked together to try and make this happen, but the C-Store C++ is just not written to be usable as a library within a larger application codebase. The C-Store 0.2 reference implementation’s C++ doesn’t come with even command-line programs to execute queries. It comes with a pretty comprehensive collection of automated unit tests, which is fantastic, but that’s the only executable it currently builds.

The bigger concern is whether the code is thread-safe, and the fact that the C-Store code requires C++ exceptions, I’m doubtful. If it isn’t thread-safe, it’ll make the C-Store reference implementation a non-starter for any serious work, which is unfortunate.

If I get more ambitious, maybe I’ll take a stab at a fresh implementation in fully thread-safe C, as a library to be embedded in other applications. I’m just sad right now because I was hoping to have this working to show off to the other campers, today.

Perhaps I’ll work on using Tcl as a User-Defined Function (UDF) inside MySQL–which I’m betting nobody would ever take seriously–just for laughs. I imagine making something like this possible:

CREATE FUNCTION TCL RETURNS STRING SONAME 'libtcl-mysql-udf.so';

SELECT TCL('expr {2 + 2}');

Whether there are actually interesting non-trivial uses of this remains to be seen, but at least it’ll allow folks to start playing with it, once it’s ready.

Tags: , ,

Is Vertica’s product just Sybase IQ?

Emmanuelle Skala (Sr. Director, Commercial Sales at Vertica) emailed me a heads-up this morning that Vertica Systems, Inc. earned an honorable mention for this year’s Computerworld Horizon Award.

I first heard and blogged about Vertica back in January 2005 and was very excited to see what Michael Stonebraker was going to introduce to the world this time. He’s pretty much the father of the modern relational database, having been a part of many companies in the industry throughout the years.

His latest work has been on C-Store, a “column-oriented DBMS” developed amongst the Massachusetts tech. universities. It is being released as open source software under one of my favorite licenses, the BSD license.

It’s exciting to finally see a column-oriented DBMS released as open source. The concept is nothing new.  Sybase IQ was doing this commercially over 10 years ago, before 1995 when it acquired Expressway:

Sybase IQ is based on technology that Sybase acquired when it purchased Expressway in 1995. However, it was not until the release of version 12.0 in 1999 that the product really came to its present level of maturity.

It’s clearly in the Sybase IQ documentation, “When you load data into a table, Sybase IQ stores data by column rather than by row, for each column in the table.” I’m guessing, without having looked at their implementation, that C-Store does things differently than IQ, but the underlying premise sounds the same.

Perhaps C-Store can be hooked up to MySQL as a pluggable storage engine. That could be very cool and possibly even useful! Maybe I’ll work on that this week at MySQL Camp II.

Tags: , , , , ,

Twitter, haiku-style

Twitter‘s limitation of 140 characters is no stranger to folks who use the service. In response to William Hartz, I wrote this:

twitter haiku style
hundred-fourty characters
wisdom in small bites.

He found it interesting enough to repost it in his own blog, too.

Tags: , ,

“I” before “E” except after “C” …

Kids are taught a cute little tip to help them spell words that goes like this: “I” before “E” except after “C”. But, then, how do you explain this:

“It is weird how the inadequacies of science weigh so heavily on our society.”

Yeah.

Tags: , ,

Father Nature makes everything “man”-made

Tonight, driving home from dinner, the subject of Mother Nature came up. Charlie, my 7-year-old daughter, asked, “What about houses?” We explained that houses are man-made, to which my daughter explained without hesitation, “Oh, that must be Father Nature, then”–which logically follows since fathers are men.

I am constantly reminded of how I want to be as smart as she is when I grow up.

del.icio.us/dossy links since August 13, 2007 at 09:00 AM

del.icio.us/dossy (RSS) links since August 13, 2007 at 09:00 AM:

Jason sees value in Mahalo for the long tail of users

I love that Jason comes out and says “Mahalo is not for you” to the geeks:

“The fact is, if folks who are in the .001% of the internet population are in love with our service that’s probably a BAD SIGN.”

He goes on to say:

“[…] what the vast majority of people want from search is to type a word into a box and get an organized list of high quality links. That’s it. It’s that simple, and that’s what we’re doing.”

There’s a reason why Jason was a good fit for AOL–and a tremendous loss for AOL when he left–because Jason naturally “gets” the ideas that the AOL founders have known for almost 20 years now.

AOL Keyword dialog screenshot

It goes back to something I’ve argued with user interface designers on and off for years: users don’t want to browse the web, they want to find information. Getting more than one result back means having to make a decision and most people aren’t educated or informed enough to make the decision as to which path will result in finding what they’re looking for. They don’t have time to navigate down until they reach a dead-end and back-button their way to the decision point and try again.

Sometimes, I wonder if Mahalo’s going to miss the mark because they still give users too much to choose from. This is why Google’s “I’m Feeling Lucky” button is such brilliant genius–if only Google’s search result quality were much, much higher. Once Mahalo’s data set grows sufficiently large, it might be useful to just redirect people to the top link instead of serving them a results page. Yes, this means you can’t monetize on eyeballs with paid display ads, but perhaps you could revenue share with the destination that you send users to.

This is why I kept wanting to launch a little skunk-works project inside AOL to build out keyword.aol.com … which would just reuse their existing keyword database and redirect users, so if you went to keyword.aol.com/sports it would redirect you to the page that is already programmed as the destination for KW: Sports.  No real additional “work” would be done by the editorial team–just repurposing the existing data.

Oh well, AOL may have missed the boat on this one but maybe Mahalo has a chance to win here. I can’t wait to see how things turn out.

Tags: , ,

August 13th Hackfest was teh awesome

As an off-shoot of the monthly MySQL meetup that I co-organize with Steve, we started a weekly Hackfest meetup that meets on Monday nights. Last night’s Hackfest had a good turnout–7 of us at the start, 6 at the end–and there was lots of activity, even some coding!  Here’s some pictures I took half-way through:

August 13th Hackfest August 13th Hackfest

After Mary came and presented Rayt at the August MySQL meetup, I suggested she come to our Hackfest and I’d work on it with her. She was working up the design comp for the user interface and I started working on the bookmarklet and portal and soon we’ll have enough done to take some reasonably interesting screenshots.

I hope we can keep the momentum up and get more folks coming to work on their projects at the Hackfest. It would be nice to get someone to provide us a better venue–we currently meet at the Panera Bread in Paramus, NJ, which offers free wi-fi and is very accomodating of us, but it can be quite noisy and we usually can’t get the private room you see in the picture behind us. If you know of any organizations that would be willing to give us space with tables and chairs, a few electrical outlets, and some form of Internet connectivity for an evening every Monday night, let me know!

Tags: , ,

del.icio.us/dossy links since August 6, 2007 at 09:00 AM

del.icio.us/dossy (RSS) links since August 6, 2007 at 09:00 AM: