Today, Zoran Vasiljevic confirmed that there is a leak in the threaded Tcl memory allocator [1] (affectionately known as “Zippy” by the AOLserver crowd). I unfortunately already knew about this (see comments) because Qiong Wei told me about it back in December 2004, but didn’t really act on it because I originally thought that it was something in his application code that may have been causing the leak, not something in the Tcl core itself. Today, I verified that it was indeed a leak in the Tcl core, introduced in Tcl 8.4.7 [2].
Here’s my test run against Tcl 8.4.6 (no leak):
$ LD_LIBRARY_PATH=`pwd` ./tclsh % info patchlevel 8.4.6 % lappend auto_path /usr/lib/tcl8.4; package require Thread 2.6.1 % exec ps -p [pid] -o pid,vsz,rss,args PID VSZ RSS COMMAND 8317 12704 1996 ./tclsh % for {set i 0} {$i < 1000} {incr i} { thread::join [thread::create -joinable {}] } % exec ps -p [pid] -o pid,vsz,rss,args PID VSZ RSS COMMAND 8317 21192 2272 ./tclsh
Here's my test run against Tcl 8.4.7 (has leak):
$ LD_LIBRARY_PATH=`pwd` ./tclsh % info patchlevel 8.4.7 % lappend auto_path /usr/lib/tcl8.4; package require Thread 2.6.1 % exec ps -p [pid] -o pid,vsz,rss,args PID VSZ RSS COMMAND 9358 12692 2000 ./tclsh % for {set i 0} {$i < 1000} {incr i} { thread::join [thread::create -joinable {}] } % exec ps -p [pid] -o pid,vsz,rss,args PID VSZ RSS COMMAND 9358 48272 29432 ./tclsh
As I said in email to the AOLSERVER mailing list, if you're using Tcl >= 8.4.7, I strongly suggest you roll back to Tcl 8.4.6, and hope that the yet-to-be-released Tcl 8.4.10 contains the necessary fix for this leak issue.
So this is what caused my memory leak – also thought to be an application problem – a few months back. I recompiled as instructed by Zoran on the morning the story broke and instead of my process growing to 70+ MB in 24 hours, it’s been at an expected 9MB ever since.
Zoran filed Tcl Bug #1178445 for this. Unfortunately, he didn’t attach the patch that closes the bug to the tracker — he directly committed the code to the core-8-4-branch. So, the fix is in CVS.