Tue, 31 May 2005

GCJ linker problem?

gcj-4 (ld?) appears to have a problem passing arguments correctly embedding library paths — it’s ignoring -rpath arguments.

I’ve avoided it until now, but the mission critical procedures application I’ve been working on finally needed to move up to depending on Java 1.4.2. That’s no problem as far as Real Java VMs go, but I’ve worked hard to ensure xseq runs under Free Java, and so having raised the Java version requirement means depending on the newly released GCJ-4. It runs great under GCJ’s gij Java runtime, but I’ve run into a serious problem trying to use gcj to build the app to a native executable.

My system compiler is GCC 3.4.3 - that’s pretty modern, but I run Gentoo and so I really need gcc to work and upgrading the system compiler to >= GCC 4.0.0 just isn’t prudent yet. So I built and installed GCJ-4 and put it in /opt/gcj-4.

GCJ-4, of course, has a new runtime, libgcj.so.6, so if I wanted to build xseq with GCJ-4, I couldn’t use the java-gnome libraries that were installed on the system that had been compiled by GCJ-3.4.x. So I quickly built the current java-gnome from CVS stable with a prefix of /opt/java-gnome and installed it.

When I configured xseq and build it to native with the GCJ-4 and the new java-gnome, it segfaulted. Great. Long story short (and many iterations hidden for brevity) I ran ldd on the xseq binary and was really confused:

linux-gate.so.1 =>  (0xffffe000)
libgtkjava-2.6.so => /usr/lib/libgtkjava-2.6.so (0xb7c09000)
libgladejava-2.10.so => /usr/lib/libgladejava-2.10.so (0xb7bf0000)
libgcc_s.so.1 => /opt/gcj-4/lib/libgcc_s.so.1 (0xb7be5000)
libgcj.so.6 => /opt/gcj-4/lib/libgcj.so.6 (0xb6d25000)
libm.so.6 => /lib/libm.so.6 (0xb6ced000)
libpthread.so.0 => /lib/libpthread.so.0 (0xb6cdb000)
libdl.so.2 => /lib/libdl.so.2 (0xb6cd8000)
libc.so.6 => /lib/libc.so.6 (0xb6bc2000)
libgcj.so.5 => /usr/lib/libgcj.so.5 (0xb62a0000)
libz.so.1 => /usr/lib/libz.so.1 (0xb6292000)
libgnomejava-2.10.so => /usr/lib/libgnomejava-2.10.so (0xb61e8000)
/lib/ld-linux.so.2 (0xb7feb000)

Why the hell was libglade-java being sourced from /usr/lib and not /opt/java-gnome/lib? (and note that the “wrong” libgtkjava-2.6.so pulled wants to pull in libgcj.so.5, which clearly conflicts with .6).

Before you start telling me to use LD_LIBRARY_PATH to solve all my issues, I’m ahead of that. I used -rpath to embed the library paths. Read the Makefile fragment that built that executable:

xseq: build/native-libs build/native-dist build/native-tests
    $(GCJ) \
        -Wl,-rpath=/opt/gcj-4/lib \
        -Wl,-rpath=/opt/java-gnome/lib \
        -L/opt/gcj-4/lib \
        -L/opt/java-gnome/lib \
        -lgtkjava-2.6 -lgladejava-2.10 \
        -classpath $(CLASSPATH):lib:src:tests \
        -fjni -O \
        --main=xseq.ui.WindowRunner -o xseq \
        [all the .o files]

Here’s the rub: the first -rpath argument took, as it found the right libgcj.so.6 but the second -rpath argument appears to have been ignored. According the the ld man page, multiple rpath arguments should be aggregated together to compose a library search path. This seems to be not happening.

I sat with Dalibor Topic and Mark Wielaard for several hours working through this, and we discovered that its not just my app — when libgladejava-2.10.so was linked during the build of java-gnome’s libglade-java, it would only take 0 or 1 -rpath arguments, and also appears to have ignored subsequent ones. Note, however, that having reviewed carefully the output of --verbose (passed to the linker with -Wl,), ld does find the correct libraries - it’s just not embedding the paths properly. If the java-gnome libraries you’re using are installed to a prefix of /usr (ie, in /usr/lib), then things “just work”, but if you install to a non-standard (ie not in ld.so.conf) place things thunder in.

I’ve asked Dalibor to pick this up from here, as I really don’t know what to do about it. My code is available, and you can try this yourself, though. See the downloads page to see how to get the code. Once you’ve got it, and installed gcj and java-gnome somewhere non-standard, then do something like:

$ ./configure gcj_home=/opt/gcj-4 java-gnome_home=/opt/java-gnome
$ V=1 make

Then compare the output of

$ ldd ./xseq

against

$ LD_LIBRARY_PATH=/opt/gcj-4/lib:/opt/java-gnome/lib ldd ./xseq

and wonder why the first one isn’t correct.

Oh, lastly, even with LD_LIBRARY_PATH set to avoid the -rpath bug, when I ran the program it said (as windows were freezing, and other exceptions were blowing out) this:

GObject:finalize() of: org.gnu.gdk.Pixbuf
  Handle32Bits - pointer = 136749256
  nativeFinalize - Start.
    appending 136749256
  nativeFinalize - End.
GObject:finalize() completed.

I have no idea what that menas, but that error does not occur when .class files are run by gij, even though it was the same libraries and the same libgcj. It only happens when the natively compiled executable is run — which is why Mark suggested that miscompilation may also be an issue.

AfC

gcj-4.0.0-linking-problem, posted by Andrew Cowie in /andrew/software/java-gnome at 14:58, 31 May 2005.

Tue, 24 May 2005

Movie Review: Revenge of the Sith

The trend from pretty much everyone across the blogosphere over the last few days about the final installment of the Star Wars prequel trilogy has been words to the effect of “Yawn…”

Sorry, guys, but it’s rather obvious that everyone is just trying to be cool by acting like they didn’t enjoy it. Forget that! Episode III rocked! “Save your money for another film?” Huh? Who are you kidding? To anyone who hasn’t seen it yet, you’re in for a treat.

The film critics generally said “it’s better than the first two” and “more in the style of the original trilogy.” Easy enough — frankly, being an improvement over Episodes I and II wasn’t going to be that taxing, but Revenge of the Sith is quite a ride. Any movie where the introduction starts off with “War!” and whose opening sequence is a massive space battle where cruisers are slugging it out with each other can’t be all bad. The difference this time is that the action pretty much keeps going the whole film — and that put it in the tradition of the original movies.

It was notable this time that when something amusing happened, you had a chuckle, but it was just that, fun, without being corny. (No, don’t worry, I’m not a closet Jar-jar fan. Wait till you see R2D2 kick some other droid’s ass. Good stuff). Unfortunately, George Lucas just can’t seem to get the hang of writing romantic lines terribly well. The few “I love you”s between Anakin and Padme still have the familiar oh god, please make it stop feel that made us all cringe in Episode II. It’s not so much that the lines are bad (nothing wrong with saying I love you, after all) but it comes across a bit, well, abrupt. Maybe it’s because we missed out on the tender interludes (I rather suspect it’ll be a while before the X rated directors cut comes out. Oh well).

It is absolutely amazing how far the animation technology has come. In The Phantom Menace, everything had a bit of a plastic look to it. I mean, the background green hills they used for the final battle between the gungans and the droids looked like the stupid default image from the Windows XP desktop. Lame! In Attack of the Clones the realism was up, but it was all still a bit faux neon. The clone troopers were awesome, but they looked more like characters from “Finding Nemo” than men dressed in armour. The technology has obviously finally come into its own, though. The progress that they’ve made in Revenge of the Sith, especially with textures, is incredible. One place it jumped out early on was the destroyer droids (remember the wheel shaped battle droids that roll themselves down the corridors and then unfold into scorpion looking creatures with shields and laser guns?) — they were always cool, but this time they had a certain dirty grit about them - in short, they looked real. Wow.

Lucas describes his approach to movie making as “going out and getting elements” and then knitting those elements together. Blue screen footage of a character jumping off a platform… which turns out to be some impossibly high bridge (perhaps a painting by a matte artist or generated from pure CGI). I think this obsession with the individual details led him to somewhat loose the forest for the trees. I think that’s what got him in trouble and might explain in some measure why we were all a bit let down by the first two films. They were choppy. Almost every sequence was visually stunning — but at the end of the movie you were left a bit confused because although there was a story there, it wasn’t really told as well as it could have been.

Finally, however, almost all the loose ends are tied up. Episode III was a terrific movie, and I think worthy of lot more praise than all us jaded pundits have been giving it. For those who are inevitably going to disagree with me, all I have to say is this: remember that Luke guy? You’re trying to tell me that you didn’t find his lines as annoying as Jar-jar’s? Shit, I fast forward past almost every scene Luke is in when I watch Star Wars. No. As a cinematic accomplishment, Revenge of the Sith represents the culmination of a story over 30 years in the making. In any frame of reference, that’s quite an accomplishment, and I’m glad to have to have seen its conclusion.

AfC

star-wars-III, posted by Andrew Cowie in /andrew/reviews/movies at 19:58, 24 May 2005.

Fri, 13 May 2005

Whither GUIDs?

I’ve been trying to understand the impact of updating or moving a blog entry within Blosxom.

The RSS 2.0 schema provides two fields for each article that relate to links and identity. First is the <link> field which, reasonably enough, is expected to be a link to an online (presumably HTML) version of the article. The second is the <guid> field, intended to be a globally unique string which can be used by a reader to identify an article.

<guid> has one attribute, isPermaLink, defaulting to true. If true, the field can be interpreted as a usable permanent URL to the story.

If the date changes but GUID stays the same, my reader marks the article as updated, which is cool. And on a planet, well, it just gets reslotted and the post shows up wherever its new modification time indicates it should be. So updates are all good.

But what happens if I ever want to move entries around to make a more sensible category tree?

Recategorizing entries

Until now I’ve been doing what everyone else seems to do, that is, sticking the link in as the guid and allowing the isPermaLink="true" default to work.

One of the problems of using a hierarchical blog store (ie a set of files in a directory hierarchy) is that category is defined by what directory you’re in. That’s exactly what you want, but it creates a problem when you’re trying to figure out what a good GUID would be - binding the GUID (and implicitly the permalink) to the file’s name and position in the directory hierarchy means that if you move the file up or down, or rename it, (regardless of whether you preserve the modification time) you change the guid, making readers think the article is new. That’s no good.

Since I want to be able to recategorize postings at some point in the future, the alternative would seem to be changing my rss20 plugin and flavour template to say <guid isPermaLink="false"> and use something actually unique for the globally unique identifier. But what?

Generate persistent and unique IDs

Anyone who has ever done any work in the enterprise application server or core database world has come across the design problem of generating primary keys. There are several approaches to try. The AUTO_INCREMENT or SEQUENCE mechanisms in relational databases are an obvious source of sequential numbers, but this runs into problems when there are multiply redundant masters which have to co-ordinate handing out unique IDS. Blogging is certainly a far simpler case, but we still have to find a way to form GUIDs that will be stable.

The approach that Blosxom takes, as shipped, is to use a permalink of the form /2005/03/08#posting-name (which it serves as a virtual and autogenerated category). The trouble with using this string for the GUID field (even assuming isPermaLink="false") is “what happens when I make an update to the post and the date has changed?”. The whole point is to have a GUID field that survives updates.

Another idea would be to use the inode number as that would survive a file move. Inode number would actually be perfect except that, unfortunately, I publish my blog by rsyncing my local draft copy to our R&D server where the feed is hosted. rsync regards a moved file as a combination of a delete of the original and an addition of a file in new location, and a new file means a new inode number server side. (Damn)

Finally, of course, are any number of options which involve generating a sequential UIDs and storing them in the plugin cache somewhere. That is do-able but is really rather complicated, and would imply some kind of item-to-uid mapping that would then need to be maintained internally — and synced up to the server. Ick.

Keep it simple

At this point, I gave up and decided that posting-name itself would probably cut it. This implies that I’m imposing on myself the requirement that story names be unique across the entire blog, but that’s not so terribly bad. And it means that if I do move a file, I don’t rename it and I have to ensure I preserve mtime. But that’s easy, because that’s exactly what mv does! And mving makes perfect sense for a Blosxom blog because that’s how you recategorize something.

So, the RSS entry for this story now contains <guid isPermaLink="false">reverse-understanding-rss20</guid> which is fine. What I wanted was a GUID that would survive category moves. The file’s basename is good enough.

[Yes, there is the obvious problem of stale links floating around out there in people’s readers and on Planets and whatnot, but having decided on unique filenames, a 404 page which goes hunting for the file within the blog tree and redirects to the new location accordingly will be easy to implement]

I’ve updated my rss20 flavour to do the guids this way.

AfC

Sorry to anyone whose reader (correctly) interprets this as if all my posts are new entries — that’s what I get for mucking with the guid field. It won’t happen again … well, not until someone tells me a better way to generate a GUID for a Blosxom feed :)

reverse-understanding-rss20, posted by Andrew Cowie in /andrew/meta/blosxom at 15:52, 13 May 2005.

Sun, 08 May 2005

Getting Blosxom to work…

So I’ve finally joined the blogosphere. About time. I kept putting it off, being relatively content with the posts I was sending to email lists.

I began to notice that about once a day I found myself writing some really long email (or hammering away at length in some IRC channel) about a topic which I inevitably wanted to refer to again. But forwarded emails are usually a pain, and having to rummage around in your own email folders in order to then hunt down the message in an email archive so as to then send someone a link gets really old in a hurry. So, why not. Might as well blog the stuff up.

Picking a blog tool took a lot of thought. WordPress is awesome, especially the multi-categories feature. But what I really wanted was something that would work off of simple text files that I could work at offline. I tend to spend a lot of time in places with no internet coverage (airplanes, coffee shops by the beach, banana republics, that sort of thing) and so anything that was a web app was really a non-starter. I needed something I could publish by running rsync.

People had recommended pyBlosxom, but I couldn’t quite get it running, and in any case I don’t speak Python so that wasn’t terribly appealing. I’d come across Blosxom of course, and seen it used for a number of blog pages that I’d been following for a while, and on that basis alone it seemed like it might be alright. Blosxom is in Perl so I figured I’d have no problem getting it to behave.

Boy, was I in for a shock. First things first, however…

Text markup

Five or six years ago there was something called Simple Document Format which was blindingly simple to use, and from simple text documents generated man pages and web pages. Yeah, sure, so does LeX, but this was text. When wikis appeared on the scene a few years back I wasn’t perhaps as impressed as I might have been, other than “well, it’s about time.” But I quickly noted that the lack of any coherent driver to standardize wiki markup meant that as each person created their own new wiki, they came up with a syntax that they liked. Which sucks for users because we have to learn yet another wiki markup language each time we sign into one.

I came across Markdown, and at last found a text markup tool that I liked. It’s awesome. Writing about the philosophy behind the design, the author, John Gruber, states,

“Markdown is intended to be as easy-to-read and easy-to-write as is feasible. Readability, however, is emphasized above all else… the single biggest source of inspiration for Markdown’s syntax is the format of plain text email

To this end, Markdown’s syntax is comprised entirely of punctuation characters, which punctuation characters have been carefully chosen so as to look like what they mean. E.g., asterisks around a word actually look like emphasis. Markdown lists look like, well, lists. Even blockquotes look like quoted passages of text, assuming you’ve ever used email.

Markdown is not a replacement for HTML, or even close to it. Its syntax is very small, corresponding only to a very small subset of HTML tags. The idea is not to create a syntax that makes it easier to insert HTML tags. In my opinion, HTML tags are already easy to insert. The idea for Markdown is to make it easy to read, write, and edit prose. HTML is a publishing format; Markdown is a writing format.”

Reminded me of SDF. Combined with Blosxom, which had the premise of being simple and text file based I felt sure I’d have my blog up in no time.

Yeah, right. Who was I kidding?

Blosxom Plugins

You quickly realize that to do anything with Blosxom you’ll be using a variety of plugins. That didn’t seem like it would pose a problem as there was a well organized plugin directory to work from.

And then you realize that there are 4 different plugins that all purport to do the same thing. The one you decide to try turns out to have been written over two years ago, and that it hasn’t been touched since. Never a good sign. And then you try to download it, only to discover that the site that it was posted on is long gone.

Sadly, this is not an uncommon story in the Open Source world. Just look at your average search on SourceForge or Freshmeat - let alone the endless shady PHP classes debacle. But I kept coming across people who were using Blosxom as their blog tool, and so I figured it couldn’t be that bad, and persisted.

I ended up using the following plugins:

• breadcrumbs
• config
• whoami
• calendar
• file
• page_titler

They all had POD documentation in their files, but it was usually rather sparse, and just getting to the point where you knew whether or not a plugin was even running took quite a while.

In proper Free Software fashion, I finally figured out what was going on when I delved into the source. Yes I’d read about the difference between head.flavour and story.flavour, but it didn’t quite sunk in until I was trying to use the username from the whoami plugin in the blog header, and finally clued in that no, that wasn’t available yet because the $whoami::username variable that whoami populated was based on the individual item, not “the page as a whole” even though it was “my” blog. Fair enough, but like I said, a bit of a learning curve.

Syndication

Blosxom has a MultiViews notion it calls flavours. You ask for index.html, get a web page; ask for index.rss and ta-da you get a different stylesheet view of the blog data that’s an RSS feed. So I figured that getting an RSS feed out of it was going to be a snap.

Except for one thing: the rss flavour built into Blosxom doesn’t validate, at least not without fudging the content type and taking out all the HTML. (props to Chris Deigan for pointing me at that tool).

But hang on - everyone else was is using HTML in their feeds. So I thought “Maybe it’s because my feed is RSS 0.91”, and tried upgrading to RSS 1.0. I had a hint something was wrong when I came across these plugins:

• rss10
• atomfeed

Why should I need a plugin to fix the RSS flavour to be 1.0 compliant? Whatever. I grabbed the plugin, figured out how to use it… and it still wouldn’t validate. So I started looking at it, and discovered RSS 1.0 it was nothing even remotely close to 0.9 - (its full of namespaces on attributes. Yuk!) - and that Blosxom’s idea of RSS 1.0 wasn’t remotely close to validating either. So I hacked on the templates, and eventually got a RSS 1.0 feed that passed.

Except for one thing. It didn’t grok HTML in descriptions either. WTF?

So then I tried Atom, had to hack it a touch to validate, and finally I had a valid feed with HTML in it. My feed reader liferea groked it perfectly as well. Yeay!

The coming of wisdom

At that point, exhausted, I figured that RSS 2.0 would be a further nightmare, but being too punch drunk to know when to stop, I glanced at the spec anyway. And to my amazement, it was Really Simple. Imagine. With all I’d learned, I had a valid feed in no time, creating my own

• rss20

plugin and flavour template.

But wait, you say, embedded XHTML isn’t a valid part of RSS 2.0! Well, you’re right. In the end I finally discovered the technique of wrapping the description content with <?[CDATA[ ... ]]>. Slap that around the body text and you escape any and all non RSS entities. Damn and other comments. If I’d thought of that earlier I could have saved myself a lot of trouble.

Download

You can download my fixes to the atomfeed plugin (it’s self contained), or grab my rss20 plugin and rss20 flavour files, which have been working really well.

Sigh. Look at me. RSS guru. Yup. You know those little boxes on government forms that ask you how long it took to complete the process? … You don’t want to know.

AfC

blosxom-colophon, posted by Andrew Cowie in /andrew/meta/blosxom at 09:45, 08 May 2005.

Tue, 03 May 2005

xseq 0.2.0 released!

Last night I finally closed out the last bug scheduled for 0.2, and decided to release it as xseq-0.2.0.

It’s still pretty alpha, but the essential UI is there, and at least people can run it and try the basic functionality.

Anyone intereseted in the code is still best off getting the source via Darcs so they can keep up to date with the code (and contribute fixes back, hint hint) but I thought completing a milestone was worth a tarball :)

On with 0.4…

xseq-0.2.0-released, posted by Andrew Cowie in /andrew/software/xseq at 12:40, 03 May 2005.

---

Category Specific Feeds. Use these links for an RSS or ATOM feed limited to this category and its descendants. Technorati Profile

---