Since JDK 6 that Java provides the Java Compiler API. It allows one to compile Java source from Java applications. By default, it generates a set of class files with the generated bytecode. This post shows how to build the necessary support to generate that bytecode in-memory using the Java Compiler API.
Why would I need this? A while ago I had to build an expression evaluation engine that works in a way pretty much similar to Apache JXPath. JXPath allows you to navigate in a Java objects graph using XPath-like expressions through the use of reflection. If you are building a library like this, it should have a structure similar to the following:
A lexer, that translates the input text into input tokens
A parser, that takes the list of input tokens and generates a semantically valid AST (Abstract Syntax Tree).
Once we add the ability to navigate in a Java objects graph dynamically, we need to transverse the tree and generate something. We have two options:
We use reflection to understand the graph anatomy and navigate through it to generate the calculated value. That forces you to go through reflection to get the expression result
We use reflection to understand the graph anatomy and navigate through it to generate the java source code out of it, so it can be compiled to be evaluated later as if it was part of the application.
So remaining of this post shows how to get rid of this class files and use the Java Compiler API to generate bytecode into memory so it can be executed later as part of the application.
Let’s say we already built it. Here’s how to use it.
The CompilationPackage assembles a list of CompilationUnit instances, each one representing a class that we are compiling. Bear in mind that (anonymous) inner classes inside a class generate isolated classes by themselves, meaning that if we added an inline comparator implementation to our Collections.sort call, that anonymous inner class would generate a separate compilation unit for the resulting compilation package.
Yep, we still execute the main method using reflection this way. But that’s one global call instead of one call for each step in the object graph.
If we can tell the Java Compiler the application’s classpath, we can drive our generated code by interfaces and use them instead.
How to build it?
What we will need:
An in-memory compiler that instructs the Java Compiler API to generate bytecode in-memory instead of class files
A class loading mechanism that include those classes into the execution’s classpath.
The in-memory compiler
The default JavaFileManager implementation class uses a simple implementation of type JavaFileObject to read/write bytecode into class files. This InMemoryClassManager extends the standard JavaFileManager to read/write bytecode into memory using a custom implementation of the JavaFileObject
The JavaSourceFromString class stores Java source code from a String into a JavaFileObject and can be built as follows:
The in-memory class loader
Now that we have the ability to compile our source into memory in the form of a CompilationPackage, we need a way to load it in the form of executable classes into our application’s classpath. For that CompilationPackageLoader was built, as below:
The real trick is at the ByteArrayClassLoader, that is able to build a Class<?> instance out of an array of bytes.
The bytecode wrappers
Now we are just missing the implementation of CompilationPackage and CompilationUnit, which are simple immutable objects holding the bytecode.
Right now we have everything we need to compile Java source into memory and execute it.
It was seen what it takes to implement an In-memory Java Compiler using the Java Compiler API.
In the case you need to do some kind of trick like compiling source code into memory in a long running application, feel free to give it a try. The full source code can be found here.