Imperative by day and functional by night

The Dynamic Discovery Paradox

ClassLoaders can facilitate dynamic discovery and make it impossible.

A recent article about logging abstractions has inspired me to revisit the topic of class loaders and dynamic discovery.

Java class loaders are responsible for finding and loading Java classes and resources for applications at runtime. As application developers, we typically develop code that is oblivious to their existence. This is because we develop Java applications that target Java platforms, and as such expect Java to successfully perform all our necessary class loading tasks. One thing we are not oblivious to however, is how we package our applications into reusable and deployable Java archives (JARs, WAR,s EARs, etc..). It is very important that we do this correctly, especially when targeting container environments.

In our development environments, we iteratively develop and test our applications until completion. When we are confident that our tested applications are ready, we package and deploy them into their target environments and execute them. In these moments of truth, our applications either execute successfully or crash spectacularly. Given that we have already executed them successfully in our development environments, we would reasonably expect that they would also execute successfully in our production environments. Right? Yes, no, maybe? Lets consider the case where they do spectacularly crash due to unexpected class loading problems. For example, when they crash with the dreaded java.lang.NoClassDefFoundError.

How is it possible that a packaged class or resource in the classpath can fail to load? The answer lies in the implementation of the class loading mechanism in the hosting Java environment. Container environments in particular, often employ complex and sophisticated class loading mechanisms. They consist of many class loader objects arranged into hierarchically delegating tree structures. Their relationships to one another are determined by their individually implemented delegation models (parent-first, parent-last, etc..). Container providers publish technical documents that specify their exact class loading structures and strategies. Some also provide configurable isolation modes that control visibility (application, module, etc..). By studying these strategies and tweaking these modes, it is possible to derive alternative packaging solutions. Although these solutions may work, they are often undesirable and can lead to unconventional deployment practices. Repackaging can introduce negative side effects and can also compromise application portability and modularity.

There are times when it seems that class loading problems cannot be solved at all and all forms of tweaking and repackaging consistently fail. Consider an application that contributes resources to a dynamic discovery API based on class loaders. The class loader that loads the API may or may not have visibility to the resources of the application. The application, through packaging, may or may not have control over which class loader loads the API. The API itself could be provided by the container or packaged within the application, or both. Either way, it will only have limited class loader accessibility. For these reasons, class loaders make dynamic discovery both possibly possible and possibly impossible.

ClassLoaders can drive you mad. Just when you think you understand them they baffle you. The use of the commons logging API in multi ClassLoader environments is a classic case.


Written by warpedjavaguy

September 14, 2007 at 10:55 am

Posted in java, programming

7 Responses

Subscribe to comments with RSS.

  1. Running with you logging example that spawned the whole entry …

    … how then do you address the very real issue of providing a library that does not assume your end user is using the same logging mechanism you are? Especially, when you clients are probably consuming libraries from other vendors as well.

    As a library vendor I have to ask myself if I’d like people who use complex environments to read and understand the complex class loading mechanism they have chosen to employ? Or do I want all users to configure 4 different logging mechanisms; all of which write to different files and don’t cooperate?


    September 14, 2007 at 4:49 pm

  2. Are classloaders realy such a horrible thing … or are they a good concept with just a screwed up implementation? Coz, from what i’ve read, they seem to be a very powerful concept!


    September 14, 2007 at 8:09 pm

  3. @AngryAss,
    It is unfortunate for both vendors and developers that the Java Logging API is not compatible with the more popular open source logging API’s. If only it wasn’t so.

    Classloaders are a fundamental Java concept. I for one am constantly being challenged to better my understanding of them.


    September 14, 2007 at 11:01 pm

  4. Can’t help but think that OSGi might be able to solve some of these issues, I know it works pretty well in the Eclipse RCP Application environment….


    September 15, 2007 at 1:02 am

  5. +1 for OSGi.

    Classloader hierarchies in J2EE servers are the Achilles heel of J2EE, as they prove to be the greatest obstacle to application portability. The ability to manipulate classloader hierarchies is a desperate attempt to prolong the life of a limited and ultimately dying platform design.

    OSGi offers a federated classloader model that supports package-level versioning, whereby issues such as the aforementioned logging abstraction problem just don’t exist.

    Combined with a lifecycle that supports hot-deployment (i.e. no more restarting the server), and simplified deployment model (just JARs – no EARs, WARs, RARs, etc), OSGi is positioned to inherit the Enterprise Java world. 🙂


    September 15, 2007 at 8:14 pm

  6. OSGi does sound very interesting 🙂

    What’s interesting too is that JSR 277 and JSR 291 aim to bring to Java a more robust dynamic discovery mechanism and an OSGi based dynamic component framework. Click here for the latest developments and discussions in that space.


    September 17, 2007 at 11:14 am

  7. Well, OSGi can help solve these problems….but it’s still tricky. Eclipse actually has the concept of “buddy classloaders” where the example given is often the installation of a logging framework where people are registering custom handlers. In this case, you need the log framework to be at a low point in the dependency hierarchy but it needs to use handlers registered by things dependent on it. An article I keep returning to for this is here.

    You might be interested in a couple articles I wrote on post delegation classloaders, debugging classloader problems, and a rare problem.

    Alex Miller

    October 6, 2007 at 11:21 pm

Comments are closed.

%d bloggers like this: