In a recent thread on developer.db4o.com, several people were whining about how the GPL is too tough for them, and that they wanted db4o for free without neither the obligations of mutual software freedom nor the requirement to have to pay for a commercial licence.

Midway through, Christof Wittig, their CEO, posted a reply that I thought really captured the essence of what they are doing as a company. Here’s my favourite part:

I think most of you know that db4o is a dual-licensed product — with [the same code available in] an open source version and a commercial version. The commercial version is distributed by db4objects, a company that is for-profit and — believe it or not — we are here to make money.

If you are open source/GPL, we are. If you don’t want to comply with the GPL, we’re not open source and not free either — in this case we’re as commercial as you are, though perhaps a little bit more affordable than others. It is as simple as this.

All discussions on this topic are usually by people who want to use the free/open source versions for their commercial/closed source businesses without giving back to the community — either with code or with money. For me that’s just free-riding.

The forbearance of CEOs

It never ceases to amaze me what the CEOs go through. Running any company means endlessly answering the same questions, and I can only imagine what the clamour must be like for the CEO of an open source software company. Under that sort of constant barrage, it’s all the more impressive that someone like Christof continues to keep such an even keel. Well mostly…

Some of you use the value-approach for pricing (which we promote, too: Usually we try to capture 3-5% of your software value). Some claim, though, that a few thousand bucks [that would be db4o’s licence cost] would be 50% of their total cost. If this was right, then the entire business was only 2× a few thousand dollars. But how can this be? Even if you live in Siberia, you cannot make a living on this as a company. So you are either not saying the truth or, more likely, you exclude complementary revenue streams, e.g. from services or add-ons, that you cross-subsidize with low cost or free software. These revenue streams, though, should be included in your value approach, and then you’ll soon find yourself in the low digit ballpark with the db4o license fees.

Or you can release your own software under the GPL. Take your pick.

Why Open Source is not communism

I advise clients about Open Source, and one of the questions that inevitably comes up in the discussion about people who dual-licence their software is, “but it’s GPL! How dare they charge for it! How can they? %#@$^&!” Sadly, the people asking that question in such an arrogant and demanding tone have missed the point. Open Source is not communism! There is no obligation to contribute what you own and no one taking it from you by force, stealing it for their own benefit while justifying their actions in the Holy Name of the People.

“No,” replies the author, “It’s my property. I wrote it. I have the copyright on it, I own it. I can do whatever I like with it. I have chosen to release my code, without patent encumberment, and you can use it, but only under very strict conditions. If you’re willing to agree to those limitations and in so doing contribute to the raising up of the whole world through the apotheosis of software freedom, then you’re welcome to use their code for whatever you’d like. People and organizations who have chosen the dual-licence model to fund their enterprise then say, “Not willing to meet the terms the GPL? Then if you want a licence to use my property in your product, start forking over the cash.”

One way or the other, FOSS is about contribution. Don’t whine about why you can’t have it for nothing.

AfC

Those of us heavily involved in mondial libré take it for granted that our best work happens when we carry it out in an environment of openness and honesty. Few companies, however, even ones who release their code under an open source licence, truly embrace this idea and continue the bulk of their activities in secret.

So this week I was really impressed to see the developers of db4o invite a semi-public audience to their weekly conference call. They’re still feeling their way (which is why they started with a limited group) but this is a company determined to do things better — db4objects has already made the web forums their developers use to communicate public, then went one further and granted privileges to some of their more involved community members to contribute as well.

One of my colleagues noted:

It makes me feel good to listen to the people who are developing the database of my choice, because a database isn’t just software, a database is trustware! And it needs a big amount of trust to drop data into such an “exotic” and “small” object-database like db4o … but they seem to take the right approach: by open-sourcing their database and their development-model db4o gains lots of trust. And that’s what really counts in database-driven business.

— Maik Jablonski

In a rational world, I’d like to think I have better things to do than sitting up until 03:00 in order to listen to a bunch of database hackers talk about what they’re working on. The exciting thing about open source, though, is that its about more than cold hard rationality — it’s about the fire and passion that drives people and it’s always terrific to spend time watching a talented team being creative as they hash out hard problems.

Nothing is ever easy: I imagine that in the future as the spotlight moves onto them the temptation will be to turn their conference call into a marketing event. If they can resist that and continue to have frank discussions about contentious topics in the open, then I think they will have taken a significant step forward. Indeed, having their meeting in a more public forum will force them to stay on topic and use their time wisely, and I think that’s useful in its own right.

AfC

For those that have no idea what I’m talking about, or why I find it interesting, here’s a brief extract from a blog posting I made four months ago:

I’ve been working for a while now with db4o, an object-oriented database. It’s native Java (ie you use it from and feed it Java objects, as opposed to translating to some third-party pseudo or meta object representation as most of the original OODBMSes required you to do). It does a really straight forward job of just persisting Plain Old Java Objects. No mucking around with bytecode enhancers (like JDO) and certainly none of the self-mutilation that goes with EJB. db4o has proved fairly easy to use once you wrap your head around the fact that instead of foreign keys you just use a plain old Java variable … because of course all variables (ok, instance fields) are references in Java. Neat.

To say that I have been nervous about contemplating db4o would be an understatement — I have lots of experience with data evolution in enterprise settings and using a OODBMS rather than a more traditional RDBMS scares the heck out of me. On the other hand, db4o gives me almost totally transparent persistence and has certainly massively accelerated development by allowing me to have a functional persistence layer in one of my applications from the get-go rather than wasting eons fighting through the Object-Relational impedance mismatch. So all in all it’s been a very good experience. Thus far, anyway, db4o has proved to be reliable and robust. Of course, you have to use it properly…

Most of the other Java people* I’ve introduced db4o to have, of course, at first greeted me with “what the fuck” but usually the next morning I get a ping saying “hey, this is pretty good”, so it would seem I’m not alone in appreciating this library. The fact that it’s GPL is fantastic, but more than that it is tiny — it’s only a 600kB jar! We just dropped it into the lib/ directory of one of our (yes, GPL) projects and ship it there. Ta-da, a complete object persistence mechanism. Given that Java is the platform I work on and that the resources we can devote to development are limited, having something like db4o to use has been great; the least I can do is mention them once in a while.

AfC

* I’m a Java on Linux guy, right? [Actually, I’m a management consultant focusing on crisis management, but the systems side of what I do is advising people who run Linux enterprise platforms, especially ones running large and complex Java applications; and programming wise I write Java apps for Linux. (Yes, Linux only; I write to the Free Java platform and use java-gnome to get to GTK — I couldn’t care less if my work ever runs on Windows)]. So no need to tell me how my using Java is horribly wrong and that I should be using some 19th Generation Language instead :)

I’ve been working for a while now with db4o, an object-oriented database. It’s native Java (ie you use it from and feed it Java objects, as opposed to translating to some third-party pseudo or meta object representation as most of the original OODBMSes required you to do). It does a really straight forward job of just persisting Plain Old Java Objects. No mucking around with bytecode enhancers (like JDO) and certainly none of the self-mutilation that goes with EJB. db4o has proved fairly easy to use once you wrap your head around the fact that instead of foreign keys you just use a plain old Java variable … because of course all variables (ok, instance fields) are references in Java. Neat.

To say that I am nervous about contemplating db4o would be an understatement — I have lots of experience with data evolution in enterprise settings and using a OODBMS rather than a more traditional RDBMS scares the heck out of me. On the other hand, db4o gives me almost totally transparent persistence and has certainly massively accelerated development by allowing me to have a functional persistence layer in one of my applications from the get-go rather than wasting eons fighting through the Object-Relational impedance mismatch. So all in all it’s been a very good experience. Thus far, anyway, db4o has proved to be reliable and robust. Of course, you have to use it properly…

[I’m not worried about it loosing data so much as being worried about schema migration and being able to ensure I can always get my data out. db4o is actually pretty good about data upgrades across schema migration, but there are lots of good reasons why the SQL crowd have long asserted that loose coupling to data is a better idea and that abstracting data design and storage from domain layers is a good thing. {shrug} I won’t be surprised if I end up switching to Hibernate and PostgreSQL, but in the mean time, I’ve been able to completely avoid the ORM nightmare, and that makes me rather happy. Will I trust db4o it to actually store accounting data when I go live? Eeek. We’ll see.]

Aside: As it stands right now, I’ve deliberately architected things so that you have to talk to the application [think app server] to talk to the data. All the data integrity is in the domain objects layer. So the usual reasons to translate it into SQL (a. “so that other applications can work with it”, b. “because the data is already there and your app is secondary”, and c. “because that’s what an RDBMS is for — ensuring data integrity”) don’t apply. It’s like your local bank - the ATM talks to the enterprise application middleware and THAT and only that talks to the database. Customers don’t write SQL to get their account balance. Yes, that’s contrived, but more to the point application programmers at the bank don’t talk to the database either. They ask the app server for a finder to get them account objects, and get one handed to them which they can then operate on. The actual database is quite shielded from them, thank you very much — at which point I make the inference that since the nature of the datastore is transparent to the application writer anyway, it doesn’t much matter what mode that database is. Yes, I understand that RDBMSes do a perfectly good job of ensuring data integrity (in fact, some argue to me that it’s their only real job) but I’m a Java guy and that’s where I’ve got my application and data integrity logic at the moment. Which is why object persistence is of interest to me. The enduring reason to go to an RDBMS — to protect me from my own screwups — remains.

Querying

I’ve recently started writing some finders — things that would help me quickly look up [persisted] objects by some commonly used set of constraints. This week it was getting a specific Ledger given a fragment of a [parent] Account title and a fragment of the Ledger name. Right now Accounts contain a Set of Ledger objects in a field called ledgers.

db4o does query through Collections — transparently, in fact, but I had trouble descending through the Collection to constrain the result set to just members of the class which was contained in that Collection. To search for the Ledger whose name is “At Cost” within an Account titled “Furniture” using some prototypes to constrain the searches:

String title = "Furniture";
String name = "At Cost"
...
Query query = container.query();

Account a = new Account();
a.setTitle(title);
query.constrain(a);

Query subquery = query.descend("ledgers");

Ledger r = new Ledger();
r.setName(name);
subquery.constrain(r);

ObjectSet results = subquery.execute();

Returned me a single LinkedHashSet! That sorta makes sense given that the query node was at ledgers but if the constraints are applied transparently through Collection types, how come I can’t descend below the Collection to get a subsubquery object constrained to Ledger.class that would be the one I could execute?

I went through countless renditions of the above and had the same problem. One variation allowed me to do substring searching, (ie LIKE):

String titleFrag = "Furn";
String nameFrag = "Cost";
...
Query query = container.query();

query.constrain(Account.class);
query.descend("name").constrain(titleFrag).contains();

Query subquery = query.descend("ledgers");

subquery.constrain(Ledger.class);
subquery.descend("title").constrain(nameFrag).contains();

ObjectSet results = subquery.execute();

Variations on this theme would either return that single LinkedHashSet or worse would return all the Ledgers in the datastore. Ick.

Workaround

What ended up fixing the problem was to add a navigation reference from Ledger to the parent Account. I already had that pattern in various other places (for instance in Entries which are the bridge between transactions and accounts/ledgers, have references to both their parent Transaction and their parent Ledger) so adding a parentAccount reference to Ledger was just a few lines of code.

[This of course adds a whole field and object-to-object relationship so in persistence terms this means that all the objects of type Ledger already in the data store would need to be updated. This is what makes me nervous about db4o. At the moment it’s all just mockup data that I’m recreating each run, but in production use? They provide some really neat hooks to deal with this sort of thing but it strikes me that this would be really easy to screw up]

Now to get the Ledgers I want I just do Ledger.class first and descend “up” the object graph:

Query query = container.query();

query.constrain(Ledger.class);
query.descend("name").constrain(nameFrag).contains();

subquery.descend("parentAccount");

subquery.constrain(Account.class);
subquery.descend("title").constrain(titleFrag).contains();

Query subquery = query.descend("ledgers");

ObjectSet results = query.execute();

So the thing thats bugging me is adding & maintaining this kind of navigation reference just to get around what I view as a weakness in the query mechanism provided by the persistence store.

Actually, the thing bugs me even more is the fact that to navigate through the graph you have to naming the private fields of the classes you want to go to in String form. So much for getting the compiler to type check for you. Of course, with something like Hibernate you have to name, in string form in an XML file, bloody everything by exact name that is in both your classes and in your database. But back to this problem, if I just assumed everything [that needs to be] was in memory, and used appropriate classes (ie, TreeSets for large sets that need to be sorted, or ArrayLists for things that need to be iterated through) then I could probably avoid needing db4o’s query mechanism entirely.

Which would actually be just fine. It’s not like there are hundreds of thousands of these things — there are just hundreds, max. Sigh. It really is easy to over-engineer things. :)

It’s been a good exercise of their API, however, and out-of-band querying is still pretty cool. I’ll certainly need it when it comes time to allow users to do searches to isolating specific Transactions they might have interest in.

Strongly Typed arrays versus Collections

I did see some indication in the db4o forums to the effect that a strongly typed array would be easier to use than Java Collections of them, ie Hobbit[] array versus (say) a Set of them. Certainly, this whole descend() thing would descend you into an array of the Type you want, not into a Collection where you get stuck.

I did some looking around on the net to see what people thought of using arrays versus Collections — not so much from a performance standpoint but rather from an API usability standpoint.

Broadly, the consensus out there seems to be that while strongly typed arrays are speedy things indeed [especially care of System.arraycopy()] and “nicer” to deal with because of the compile time Type safety, almost any use case imaginable would end up needing all the various methods that Java >= 1.2’s Collections provide so you’ve got a lot of reimplementation to do.

One suggestion I saw in a Java forum somewhere was to store in arrays for steady state, but to switch to (say) a HashSet while you’re mucking with it and then ultimately export back to an array with Collection.toArray(). That seems interesting, but since the Collections are all internally implemented on top of Map instances and they in turn are implemented on top of Object[] you really aren’t buying much be writing all that machinery yourself.

Anyway, with respect to my adding a parent reference, now all I have to decide is whether this is an elegant solution and something I’ll need anyway, or a dirty hack which needs to be expunged before it sees the light of day. :)

And, of course, I need to decide whether to keep using db4o or to switch to a ORM tool on top of an SQL database. The debate continues.

/me runs off to write some more unit tests.

AfC