After seeing strange crashes in xmlFreeTextWriter() on only one platform oldag and me dove into wonders of shared linking.
<oldag> just sit right back, and you'll hear a tale
<oldag> a tale of a faitful trip
<oldag> if not for the courage of the fearless crew (oldag and jan), the Minnow would be lost!
<eric> :)
<oldag> so, who LOVES unix shared lib dynamic linking!
<eric> you know you can sing that to the tune of "Amazing Grace" ?
<oldag> who loves it!
<oldag> come on....
* eric jumps up and down
<eric> Me!
<eric> Me!
<eric> Me!
<eric> not really.
<oldag> so get this
<eric> mmm, hmm?
<oldag> my xml "writer" was being alloc'd by one xml lib
<oldag> and then trying to be freed by another
<oldag> and they disagreed on size
<mark> With different sized structs :P
One libxml was the system libxml in Mac OS X 10.5 (2.6.22), the other one from MacPorts (2.6.31).
<eric> OMG
<eric> I get it.
<oldag> somehow, it resolved create in the first, and free in the second
The system libxml was brought in by the IOKit framework, the second one was loaded as part of loading plugin. dtruss for the win
<eric> That's going to be *very* interesting to untangle.
Oldag was using his secret gdb skills and magic from libgmalloc to see that the allocated struct was 40 bytes and the xmlFreeTextWriter() was expecting a 44 byte version from 2.6.31.
<oldag> take heed, all ye in unix land! this could happen to YOU!
<oldag> eric: it works in production (not sure why it doesn't have the same dual lib problem)
So the solution:
<oldag> but i fixed it local by axe'ing libxml2 from macports
<eric> okay.
<eric> let me look at my macports ... btw, at the end of last night I *still* hadn't gotten an agent build locally :(
<oldag> then, it takes a little configure-fu to get it to build, with system libxml
<eric> okay so would your recommendation be to blow away my libxml2 from macports, too, I guess?
<oldag> if you have no dependencies on it, i would deactivate it
<oldag> that was good enough for me
Part of this hacking session we tried to move the system lib out of the way to see if a new libxml (the 2.6.31 one from MacPorts) would fix the problem. We double checked with lsof that no one is using the system-libxml right now and pressed the button:
<kroepke> so, we throw out libxml on os x?
<oldag> kroepke: yeah, don't try
<oldag> i (by jan's rquest) mv /usr/lib/libxml* x.orig
<oldag> then nothing would run
<kroepke> no way
<oldag> sudo would lock up a terminal
<oldag> it goes through a security framework
<oldag> that reads in xml libs
<oldag> reboot to single user mode (cmd-s at boot)
<oldag> a litte cmd line love
<oldag> and we're back in b'ness
You know, I'm always glade to help.
<oldag> jan said "next time, have a root term open already"
<oldag> thanks for the LATE INFO, jan!
<oldag> (and he owes me whiskey)
<oldag> :)
Comments