As Ken and Sebastian have already announced it OpenJDK integration into Jalimo was finished. However there was a bit of work to do to not only compile OpenJDK but to also package it nicely. This work is now also completed.
Additionally – and this was not told anywhere else yet – we can now also offer Cacao+OpenJDK. So anyone who is in need of a decent JIT for its target platform can now build this combination, too.
Those who do not know OpenEmbedded may wonder what is so special about the work I have done in the last weeks. Well, the special thing is that we are cross-compiling the OpenJDK. That means the machine on which the JDK is built is of a different kind than the one on which we want to run it later on. The difficulty stems from the fact that the OpenJDK build system is not designed for this (in contrast to the one used by PhoneME btw).
Before I will tell you about the guts of cross-compiling OpenJDK lets enjoy some screenshots:
People may have seen these apps on their desktops and as such they are not very exciting. However for me they have a special meaning: As a contributor to GNU Classpath it was always my wish to be able to run any Java (=J2SE) program on any device running a Free operating system. I was contributing to the Free Swing implementation and together with the enormous work done by Roman, Thomas, Lillian and many others we were able to run a few Swing and AWT programs. Still performance, completeness and correctness of our implementation where limited in many places and would require more years of dedicated effort. Thanks to Sun releasing Java as free software and the important work done by the IcedTea team we can now have the real thing on our devices and suddenly get 100% compatibility. 
Now on to the cross-compilation guts:
First of all compared to the work that would have been required with plain OpenJDK IcedTea made the effort of cross-compiling the thing a breeze. The best thing that IcedTea provides is the ability to use a GNU Classpath-based toolchain to build OpenJDK. On the major distros GCJ is used as a runtime and ECJ as the bootstrap Java compiler. In OpenEmbedded we have JamVM, Cacao and plain GNU Classpath as runtime options which work equally well. JamVM does not understand some of the “-X”options so I had to patch their use away.
Although IcedTea is a nice environment I had to patch a few things to get things started. E.g. IcedTea requires you to point to a GCJ home directory. It will then create symlinks to libraries and headers files contained within them. The problems with IcedTea are: When cross-compiling the libraries provided by your system’s GCJ do not make sense for your target (e.g. your system GCJ has AMD64 binaries while your target requires those for ARM). Secondly in a cross-compilation environment like OpenEmbedded you do not want to rely on stuff outside the environment. As such I had to modify this behavior to link to header files (jni.h and likes) which are provided by the OE-built GNU Classpath.
There where other problems like IcedTea not respecting the –with-openjdk-src-dir option. It will still try to download the OpenJDK sources itself (which is not allowed in OE, because this is done through the environment).
All in all I collected my changes in patches which I called ‘build hacks’. I consider these hacks fixable and will work together with the IcedTeam team to resolve those for future releases in order to make cross-compiling OpenJDK even easier in the future.
Besides IcedTea I had to patch OpenJDK itself to get things compiled properly. The first thing that causes trouble is that you cannot chose the compiler being used. OpenJDK contains some complicated makefiles that check your system environment and then decide which compiler to use. I used good old sed to replace the respective part of the makefile. I believe this behavior can be added to IcedTea without causing any harm to the other IcedTea users.
The next problem is the sanitizing step. This one will compile a few binaries (for the target platform!), run them (impossible) and decide on their output whether the build can continue. These checks are done for CUPS and Alsa. The fix is to patch them away. I think an optional –disable-sanity-checks would be OK for IcedTea.
The final big problem with OpenJDK’s build system is that it uses the result of ‘uname -m’ for its decision. This is troublesome because ‘uname -m’ will only tell you the architecture of your build machine not the target one. I solved this by replacing all these calls with a variable and allowed IcedTea’s makefile to provide a value for this.
The remaining issues where minor: Some unsuitable paths here, a sizer.32 binary that I had to take from a previous non-cross build there.
Apropos non-cross build. This was the really ambitious undertaking. But before I explain this here are some cross-compilation basics that I learned in the years contributing to OpenEmbedded:
Non-cross compilation aware projects are troublesome when they create binaries whose output is going to be used directly in the build. The general approach to this problem is that you first compile this particular binary for your build system and use it in the cross-build. In OpenEmbedded this will often result in a separate build recipe. If your project is cross-compilation aware it will allow you to specify the location of such binaries. E.g. Cacao 0.99.x has the –with-cacaoh option to point to the location of the non-cross-compiled header generator. In non-cross-compilation aware projects you need to patch the makefiles accordingly.
The IcedTea build normally consists of two builds of the JDK. The first one creates a bootstrap JDK. This one is heavily patched, stripped down and can be build using GNU Classpath-based software. In the second step the bootstrap JDK’s java, javac and a few other binaries are run to build the final JDK. For our cross-compilation effort this means we need to get this bootstrap JDK built for the build machine and as said above this is where things get troublesome.
The reason for this is that OpenJDK depends on a few libraries like CUPS, Alsa, libjpeg and giflib which I either had to provide in their native (= for the build system) form or patch their use away. Remember that I cannot just take e.g. Debian’s libcupsys-dev because that would be outside the OpenEmbedded build environment (and would make people unhappy who use OE on a different distro).
I decided to go the way of patching the bootstrap JDK build since printing and sound would not be needed for bootstrap purposes anyway. Originally I also disabled most of the AWT (and as such the libxt, xproto dependencies). However it turned out that at one point the OpenJDK build converts a bunch of GIF pictures into Java byte array source code using javax.imageio which requires the headless variant of the AWT.
A few lines above I told you that the bootstrap JDK is compiled separately for the build machine. In a simple project we could now skip this part from the cross-compilation build. However the bootstrap JDK build also creates libraries (for the target machine) to which the final JDK links against. That is why we cannot skip this part.
So cross-compiling OpenJDK consists of the major steps:
- build a GNU Classpath-based toolchain (gjar, gjavah), ECJ and Ant
- build the bootstrap JDK for the build machine (’make icedtea-against-ecj’)
- build the bootstrap JDK for the target machine
- replace the binaries in bootstrap/icedtea/bin with those of your native boostrap JDK
- build the final JDK
and as I have written in the first part of this posting: You need to patch everything heavily to get some problems out of the way which stem from the fact that OpenJDK is not cross-compilation aware.
Finally a screenshot showing Cacao+OpenJDK:
The next step will be integrating Hotspot-Shark into OpenEmbedded and of course getting as many of my patches upstream as possible.
Compared to GNU Classpath + Cacao/JamVM the OpenJDK packages are ~6 times larger. I am curious to see how the modularization efforts of Java7 works out. With the work done now it will be much easier to follow those developments.
Edit: Fixed images links.
