Tue, 31 Jul 2007

java-gnome 4.0.3 released!

As promised at GUADEC, the landmark release of Java bindings for GTK and GNOME based on generated internals has indeed been made in July! (with like five and half minutes to spare) As is traditional now, this blog post is the release note from the NEWS file which you can read online … or in the sources, of course!

---

java-gnome 4.0.3 (31 Jul 2007)

The code generator has landed!

Work has been underway for several months to develop the next stage of the new java-gnome: the code generator that will output the tedious translation and native layers that allow us to glue our public API to the native GNOME libraries. With this release we’re pleased to announce that the code generator is a reality!

Generated translation and JNI layers

The primary goal of the java-gnome 4.0 re-engineering effort has been to switch to an architecture whereby we could generate the bulk of the machinery necessary to take make native calls into the GNOME libraries from Java.

Extensive prototyping was done to establish the detailed design and to validate the architecture we had developed. Releases 4.0.0 through 4.0.2 contained this work along with mockups of the “translation” layer (the Java code that downshifts from our Proxy objects to primitives suitable to pass over the JNI boundary, along with the native declarations necessary in order to call methods actually written in C) and of the “JNI layer” (the C code that implements the methods declared in the translation layer which in turn coverts parameters into GLib terms and then makes the actual function call into the appropriate GNOME library).

With a solid foundation proving that our design was sound, we subsequently began the long effort to implement a code generator which would output these Java and C layers, allowing us to replace the mockup/ directory and at last leave behind the shackles of entirely hand written bindings. Over the past five months, the java-gnome hackers have been steadily working on the ‘codegen’ branch. The nature of the challenge meant that we had to have most of the code in place before any of it would be useful — never an enviable task to be working on. Thanks to the hard work of Andrew Cowie, Vreixo Formoso Lopes, and Srichand Pendyala, we reached the point where the output Java code compiled in May, and the output C code successfully compiled in by the end of June. Tremendous.

We’ve been bug hunting and refining since then, pushing towards the point where we could merge back to ‘mainline’, at last replacing the hand written mockup code. We are today pleased to announce the culmination of that work with the release of java-gnome 4.0.3.

This post on the development of the java-gnome code generator contains further details should you be interested; the file 5a-Architecture.txt in the doc/design/ directory of the source code explains the rationale and origin of the engineering design.

New coverage

Although our focus has evidently been on getting the generator into working order, there have nevertheless been a few minor coverage additions along the way which are largely to the credit of Sebastian Mancke for having submitted them and Srichand Pendyala for having fixed up their JavaDoc. There have also been steady improvements to a number of other classes; notably further signals and utility methods along with some preliminary coverage of the lower level GDK event machinery.

As is reasonable given our focus on writing the actual translation and JNI layer generators, most of these present only one or two methods from the underlying native class. Coverage will steadily improve as people contribute their knowledge and experience in documentation form.

People upgrading from 4.0.2 will actually notice that there are stubs for all of the public API classes; this was necessary to make the generated code compile. Most of these are empty as yet.

GList and friends

GList and GSList are the native GLib data structures used to represent lists and are the return type from quite a few methods across GTK. Vreixo Formoso Lopes worked out how to handle and working our wrap/unwrap functions for us to use in the JNI layer made a significant contribution to reducing our blacklisted method count.

Flags

We’ve also worked out handling of native types that, while type defined as enums, are actually bit fields.

Flags subclasses present an or() function allowing you to combine individual Flags constants into a new composite Flags object. Yes it’s a lot of machinery to do a logical |, but being strongly typed is a hallmark of java-gnome.

Build improvements

No release would be complete without mentioning that the code builds on more systems than it did before! Thanks to Maciej Piechotka and Srichand Pendyala for fixing problems resulting from Debian and Ubuntu strangeness.

The build internally now uses an optimized script that takes into account that even though the code generator may have run the translation and jni files may not actually be different. This was causing problems as Make only looks at file modification time. Instead, build/faster (great name, huh?) will only rebuild a target when source contents have changed. This was necessary for bindings hackers working in Eclipse; every time a file was saved Eclipse would merrily spawn off an auto-build which sooner or later would block the IDE UI. Yuk. Addressing this has also resulted in a faster build for everyone; all good.

The internal build script should be transparent; you still run ./configure and make as before. If you experience problems let us know.

Thread safety

In addition to the code generator, java-gnome 4.0.3 incorporates a comprehensive thread safety strategy.

None of the major Java graphical toolkits out there let you make GUI calls from threads other than the “main” one; they’re all single threaded. Even if all you want to do is a quick worker thread to carry out some input validation in the background after the user presses “OK”, you have to jump through horrific contortions to do so safely, resulting in cumbersome, clunky code.

By contrast, the new Java bindings of GTK presented in java-gnome are transparently thread safe, the first and only graphical user interface widget toolkit for Java to be so! We integrate properly with the underlying GDK thread lock and as a result you can safely make calls to various GTK methods from worker threads! This has been a long sought after goal and we hope a significant contribution to helping developers write elegant code.

Every call made to the native libraries is protected by entering the “GDK lock” [that’s gdk_threads_enter/leave() for those familiar with the C side of things]. The lock used is actually a Java side synchronized monitor and therefore reentrant; nested calls all behave properly. When in a signal handler callback the GDK lock is already held (you’re “in” the main loop when a callback happens), but since it just works transparently you don’t need to worry about it. If you do find a need to take the lock into account explicitly in your own code, see Gdk.lock in org.gnome.gdk.

It is worth noting that we have been warned that there are certain to be places in the underlying libraries that do not yet live up to the requirements of the GDK threads model — thus we will likely end up tripping over such things as we slowly add API coverage. We regard such inevitable instances as an opportunity to help contribute to improving the stability of the underlying libraries and will actively work with their maintainers to identify and resolve such issues. Nevertheless, in testing thus far our multi-threaded use of GTK has been rock solid. See these posts on GTK thread “awareness” and java-gnome’s thread strategy if you wish further details on our approach to the thread safety question.

Thanks in particular to Owen Taylor for having helped us navigate these waters!

Looking ahead

Taken together, these innovations represent the culmination of an immense amount of work towards realizing java-gnome as a viable platform for GTK and GNOME development.

Obviously with the generated translation layer in place the opportunity at last exists to start dramatically improving our coverage level, and we welcome contributions to this end. Prospective hackers are cautioned, however, that simply wrapping generated methods is insufficient — public API will only be added when it is clearly documented and meets the approachability criterion.

There are still areas where the code generator needs to be improved; we need to improve our handling for arrays, lists, and out-parameters — there are numerous permutations with all sorts of ugly corner cases.

Now that Free Java with support for generics is becoming widely available, 4.0.3 will be the last release holding the language level to Java 1.4; starting the next cycle 1.5 will be the minimum language requirement and we will be leveraging generics and other 1.5 features from here on.

Most importantly, the primary focus of the next few months will be developing a quality binding for the backbone of many applications: the TreeView Widget and the underlying TreeModel which powers it. The APIs in the native library is hideously complicated and has long been the source of confusion and pain for developers in C; it has long been a major goal amongst the java-gnome hackers to present a public API with as friendly and usable an interface as possible. It’ll be a good challenge.

---

You can download java-gnome from ftp.gnome.org or easily checkout a branch from mainline using Bazaar:

$ bzr branch http://research.operationaldynamics.com/bzr/java-gnome/mainline java-gnome

Back to work!

AfC

java-gnome-4.0.3-release, posted by Andrew Cowie in /andrew/software/java-gnome at 14:25, 31 Jul 2007.

Fri, 20 Jul 2007

Version control that doesn’t make your eyes bleed

At the GUADEC closing, Federico remarked:

trying to learn git felt just like trying to learn cvs the first time 10 years ago felt — why is this so hard?

That was exactly how I felt about git for the 4-5 months I spent dogfooding it — despite my respect for the underlying notions in Git, I was constantly perplexed at why basic things seemed so complicated — and in the back of my mind was “I’ve been at this for a while; how on earth is someone new to Linux supposed to learn this?”

And then I tried bzr!

We chose bzr (Bazaar-NG, as it was briefly known) as the VCS for the new Java bindings for GNOME for Bazaar’s relative straight-forwardness and because of our faith in the ethos and extreme competence of its developers. Anyone experienced with the old world 1st generation centralized VCS tools like CVS or Subversion will be able to make sense of it and you can learn from there. Bazaar is constantly improving in performance terms, has a vibrant developer community, is widely portable, and most of all the fact that they actually follow test-driven-development practices (unit test suite has over 7266 tests) to keep them honest biases in their favour.

In production use for the last year, we have found Bazaar to be reliable, amazingly easy for newcomers to the Open Source world to learn, and a big contributor to our goal of reducing barriers to entry. One of my most treasured emails this year has been when someone wrote me saying:

Nice HACKING instructions, really easy to follow, even though I never used bzr before. Attached are 2 small fixes for the configure script… builds on OpenSolaris now.

Newcomer to contributed patch in less than an hour. Wow.

Some people have argued that they should wait to see which 3rd generation decentralized version control system will “win” before migrating away from svn or god forbid cvs. This is, in my view, foolish — none of Bazaar, Git or Mercurial are going to go away; the determining factor, rather, is usability — and that makes the choice an easy one.

Congratulations to the Bazaar team on the release of 0.18!

AfC

git-is-like-cvs, posted by Andrew Cowie in /andrew/software/version-control at 15:56, 20 Jul 2007.

Mon, 16 Jul 2007

Bootstrapping Gentoo Linux on a new laptop with a LiveUSB key … with an ext2 filesystem!

Been looking for a new laptop for a while now, and I found what appears to be a good one from ASUS (an “A8S”; Core 2 Duo T7100 chip with nice new 800MHz front side bus, an NVidia 8400M G as the graphics chip, 802.11a,b,g,n (what the hell is n?) wireless and bluetooth, and a nice glosy 1280x800 WXGA screen). Not only are the specs nice, and I found it at an awesome price (in London, would you believe?). Sweet!

But as I wasn’t expecting to be buying a new laptop, I wasn’t carrying any install CDs — and being on the road have no way to burn one. I know I’ll be around lots of hackers this week at GUADEC, but when you’ve got a brand new laptop, who wants to be patient? :)

I do have a small 256 MB usb key with me, though. So on a whim, I thought I’d see if I could bootstrap the laptop that way. None of the guys at the Gentoo conference I was at Saturday had heard of anyone doing it, but we thought it might be possible.

More than possible: it didn’t take me very long to come across an unofficial page in the online Gentoo documentation about boostrapping a Gentoo install from a LiveUSB. Sweet!

They talked about doing it from a CD sized image on the stick, but I didn’t have room for that. Gentoo also has a “minimal” install image which is only 57 MB or so (which you can use if you’ve got good net access); apparently the current release’s minimal installer is missing a needed kernel module for the root filesystem to mount — something to do with FAT codepages, whatever those are (and certainly no reason that Gentoo’s minimal bootstrap media would have it). Bummer.

The workaround is to use a much older release, but the point was to verify that the laptop I’d just bought would actually work with Linux, and that sorta implied using a really current kernel etc.

So me being stubborn, I tried it anyway.

Off I went following the brief LiveUSB instructions to put install-x86-minimal-2007.0-r1.iso onto my usbkey. You have to muck around for a minute or so to switch it from isolinux to syslinux, but no big deal. Then 2 seconds to fix the BIOS boot order, and wow! The syslinux bootloader came up. So cool. Predictably enough, however, I hit the bug and the root filesystem wouldn’t boot. Oh well.

Then I had a bit of a brainwave and thought, “well, if FAT16 is the problem, what about just using a real Linux filesystem instead?” So I went through the steps again, this time just whacking ext2 onto the usbkey (as I normally have there). But when I went to run syslinux to install the bootloader onto the usb key, it failed saying that the target wasn’t a FAT filesystem. “Ah”, I say. So that’s what the difference between isolinux and syslinux is. Oh well.

What I needed, obviously, was a bootloader that would work for a USB key with an ext2 filesystem. I did a brief search, and suddenly came across something rather new called extlinux, from the same person who (it turns out) does all the *linux bootloaders.

extlinux? That sounds promising… and ta-da, it worked!

    livecd ~ #

Awesome!

At the moment bootstrapping from USB is not supported by the Gentoo Release engineers, but maybe the fact that you don’t have to mess around with foreign filesystems will change their minds. A usbkey is, after all, a lot easier for users to “burn” than a CD is, and 1 GB usb devices are suprisingly affordable these days.

AfC

usb-bootstrap, posted by Andrew Cowie in /andrew/software/gentoo-linux at 09:44, 16 Jul 2007.

Fri, 13 Jul 2007

Long flight, long airport

For some ungodly reason it was hard to find flights this week, so I travelled on Thai Airways a few days ago. They’re Star Alliance (good), but their in-flight amenities leave something to be desired (bad). I mean, they’re nice and all, but they have no personal video systems individual to each seat, and that makes them barbaric next to Singapore Airlines; giving the ladies an orchid just before landing as a present doesn’t really make up for it.

Worse there was no in-seat AC power, so after my laptop ran dry I had little to do and only a stupid movie to observe off in the distance; by some cruel twist of fate I had neglected to bring a novel with me and so was reduced to reading the duty-free catalogue for entertainment. Torture.

Bangkok’s new airport

Into Bangkok. My first time in their new airport. It is huge. Naively I thought that leaving the lounge 35 minutes before my connecting flight would be overkill. Instead, yikes! I conservatively estimate that I walked at least 1.5 kilometers from the lounge (somewhere on G where I arrived) to where my gate was (at the very tip of C). And I was not dawdling! Brutal. Only way I made it on time was because it was 01:00 and there were was no lineup at the checkpoint.

It’s unreal how big the place is. Searching around a bit came up with this photo dating from when it was being constructed:

Seeing the internal structure and the distance it encompasses is really something. So be warned! On the other hand, the entire D stretch is one monster duty free mall. Probably some good shopping to be had if you had a day or so between flights.

AfC

long-flight-long-airport, posted by Andrew Cowie in /andrew/travel/thailand at 11:48, 13 Jul 2007.

Tue, 10 Jul 2007

So much rain it’s almost British

Australia in general and Sydney in particular have been in the grips of a long term “drought”. One way this is measured here on the coast is by considering the water levels available in the dams that provide Sydney’s water supply. When I first became conscious that water levels were a concern the percentage capacity available was in the mid 40s; this past Christmas it was down as low as 33%.

While this still represents 4-5 odd years of drinking water, it nevertheless is the kind of long term trend that is pretty worrying for urban planners, leading to talk of desalinization plants and immediate opposition owing to the energy cost of such efforts. (Strangely, forcing the local utilities to reduce the massive leakage from their pipes never seems to come up in public debates)

This year has been a bit different, though. To the great chagrin of those doing agriculture inland (whose farms, somewhat unfortunately, are in the middle of large deserts and really are devastated by the continuing drought), it has been bloody pouring here on the coast. While I understand the federal political necessity to keep saying “we’re in a drought”, it raises a certain amount of cogitative dissonance every time one looks out the widow. Our summer this year was terrible, supposedly due to a) el nino [you can blame el nino for anything — it’s great! who can prove you wrong?] which caused to b) it raining almost every day since December.

This has, however, finally been good for the dams. Water levels went from 33% to 38%, and people were pretty excited.

And then June happened

I was out of Australia for much of June. Thank God: in just one month look at what happened to the dam levels:

Now that’s net inflow!

Put in perspective, over the last 5 years the plot looks like this:

This information from the amusingly named iliveinsydney.com website, a source of not just great information about what goes on with Sydney’s water supply, but practical advice about rainwater tanks and irrigation that anyone with gardens, lawns, or urban forests in their care would be interested in.

AfC

so-much-rain, posted by Andrew Cowie in /andrew/travel/australia at 08:51, 10 Jul 2007.

---

Category Specific Feeds. Use these links for an RSS or ATOM feed limited to this category and its descendants. Technorati Profile

---