Heating up the Code Generator

It has been a couple of years since I used a code generator in one of my projects. It must have been 2004 or 2005, when MDA was a big buzzword. Back then, we used a self-made code generator, which was written by a very smart developer who I used to work with at Aperto AG in Berlin. He even contributed the code generator to the open source community later on. Those were the days.

Since then, there wasn't much use for a code generator anymore. IDE's getting better and better at helping you with code generation and auto-completion etc. And of course, adding a generator to your build process is always a bit of work. So it is often faster to write the boilerplate code yourself, if that doesn't takes forever. However, last week it was time to bring code generation back from the grave. The game we are currently developing for EA here in Norway, has a mechanism where the game client sends game events to the server. In our domain we call these events also audit changes. A typical audit change can be that the player has found a treasure, that the player has consumed food or that the player has discovered a new scenery. On the client side, the audit change is implemented in ActionScript 3. On the server side the audit change is implemented in Java. There is a transport layer in between which serializes the AS3 object, sends it over the network and deserializes it back into a Java object. For us server developers, this meant that every new audit change also needed a transport definition about how to serialize and deserialize the audit change. This definition was always wrapped into a new audit change type class. The type definition class was written manually, which sort of was okay until we had more than 20 audit changes in the game. Thats when I started to look into generating the transport layer on the server side.

In Java 5 along with the new Annotation language feature, Sun added a command-line utility called Annotation Processing Tool (apt). This was later merged into the standard javac compiler with the release of Java 6. There is also the apt-jelly project which provides an interface to apt and can be used to generate code artifacts based on templates written with Freemarker or Jelly. Finally to glue everything together, there is the maven-apt-plugin which can be used to execute a AnnotationProcessorFactory during your build and therefore integrate apt into your project. The maven-apt-plugin looks sort of dead however. I think nowadays even the standard maven-compiler-plugin or the maven-annotation-plugin can be used to process your annotations and generate code artifacts. Since I got our generator working using the maven-apt-plugin, I did not bother looking at the other 2 plugins. If someone has a working example on how they are used with a AnnotationProcessorFactory, I would be really happy to see it. Let's look into some code now.

Here is an example of an audit change as it could exist in the game:



/**
* Audit change telling that the {@link User} has bought a {@link House}.
*/
@DatatypeDefinition(minSize = 7)
public class BoughtHouseForGold implements AuditChange {

private int itemId;

@DatatypeCollection(elementType = Integer.class)
private List<Integer> sceneries;

@DatatypeIgnore
private User friend;

... other stuff not relevant ...
}


The transport class that we want to generate need to serialize and deserialize every field of the audit change. As you can guess from above, we do not want to transport the friend field in the example. The meta data, that isn't accessible via reflection within the template which we will write later, needs to be given to the generator in another way - for instance via Annotations. Thats why I created a bunch of Annotations just to instruct the generator.


/**
* Marks the type annotated by this annotation as something that can be
* serialized and deserialized using a Datatype.
*/
@Retention(RetentionPolicy.SOURCE)
@Target({ElementType.TYPE})
public @interface DatatypeDefinition {
int minSize() default 0;
}


/**
* Any {@link Field} annotated this way, will be rendered as a Collection of the
* specified type when the Datatype is generated.
*/
@Retention(RetentionPolicy.SOURCE)
@Target({ElementType.FIELD})
public @interface DatatypeCollection {
Class<?> elementType();
}


/**
* Any {@link Field} annotated this way, will be ignored in when the Datatype is generated.
*/
@Retention(RetentionPolicy.SOURCE)
@Target({ElementType.FIELD})
public @interface DatatypeIgnore {
}


Now for the hardest part, the template. I can recommend, that you start writing the code for the first class (the one that should be generated later) manually before you work on the template. Add the class into the default location in Maven, i.e. src/main/java/com/whatever/package. The generated classes will end up in a different location later (under target/generated-sources/) so it will be easy to compare the expected outcome and the generated outcome while working on the template. Here is a template example in which I use Freemarker directives.


<#-- for each type annotated with DatatypeDefinition -->
<@forAllTypes var="type" annotationVar="datatypeDefinition" annotation="package.DatatypeDefinition">
<#-- tell apt-jelly that the outcome will be a java source artifact -->
<@javaSource name="package.types.${type.simpleName}Type">
package package.types;

<#-- all imports go here -->
import java.io.IOException;

/**
* This class contains the {@link Datatype} for {@link ${type.simpleName}}.
*/
public class ${type.simpleName}Type extends AbstractAuditableType<${type.simpleName}> { <#-- class name based on type that was annotated with DatatypeDefinition -->
public ${type.simpleName}Type() {
super(
<#-- replace camel case with underscores -->
TypeCodes.${type.simpleName?replace("(?<=[a-z0-9])[A-Z]|(?<=[a-zA-Z])[0-9]|(?<=[A-Z])[A-Z](?=[a-z])", "_$0", 'r')?upper_case}_TYPE_CODE,
new Datatype<${type.simpleName}>(${type.simpleName}.class, ${datatypeDefinition.minSize}) {

@Override
public ${type.simpleName} read(final DatatypeInput in) throws DataFormatException {
final ${type.simpleName} value = new ${type.simpleName}();
<@forAllFields var="field">
<#assign useField = true>
<#-- do not do anything if field is a constant -->
<#if field.static = true><#assign useField = false></#if>
<#-- do not do anything if annotated with @DatatypeIgnore -->
<@ifHasAnnotation declaration=field annotation="package.DatatypeIgnore"><#assign useField = false></@ifHasAnnotation>
<#if useField = true>
<#-- build name of the setter method -->
<#assign setter = "set${field?cap_first}">
<#assign useCollection = false>
<@ifHasAnnotation var="datatypeCollectionAnnotation" declaration=field annotation="package.DatatypeCollection"><#assign useCollection = true></@ifHasAnnotation>
<#if useCollection = true>
<#if datatypeCollectionAnnotation.elementType = "java.lang.Integer">
value.${setter}(in.readList(Datatype.uintvar31));
<#else>
System.out.println('Cannot read collections of type: ${datatypeCollectionAnnotation.elementType}. Extend auditable-type.fmt');
</#if>
<#else>
<#-- Handling for fields without extra annotations -->
<#if field.type = "int" || field.type = "java.lang.Integer">
value.${setter}(in.readUintvar31());
<#elseif field.type = "boolean" || field.type = "java.lang.Boolean">
value.${setter}(in.readBoolean());
<#elseif field.type = "java.lang.String">
value.${setter}(in.readString());
</#if>
</#if>
<#assign useCollection = false>
</#if>
<#assign useField = false>
</@forAllFields>
return value;
}

@Override
public void write(final DatatypeOutput out, final ${type.simpleName} value) throws IOException {
<@forAllFields var="field">
<#assign useField = true>
<#-- do not do anything if field is a constant -->
<#if field.static = true><#assign useField = false></#if>
<#-- do not do anything if annotated with @DatatypeIgnore -->
<@ifHasAnnotation declaration=field annotation="DatatypeIgnore"><#assign useField = false></@ifHasAnnotation>
<#if useField>
<#-- build name of the getter method -->
<#assign getter = "get${field?cap_first}">
<#if field.type = "boolean" || field.type = "java.lang.Boolean">
<#-- boolean getter starts with is -->
<#assign getter = "is${field?cap_first}">
</#if>
<#assign useCollection = false>
<@ifHasAnnotation var="datatypeCollectionAnnotation" declaration=field annotation="DatatypeCollection"><#assign useCollection = true></@ifHasAnnotation>
<#if useCollection = true>
<#if datatypeCollectionAnnotation.elementType = "java.lang.Integer">
out.writeCollection(Datatype.uintvar31, value.${getter}());
<#else>
System.out.println('Cannot write collections of type: ${datatypeCollectionAnnotation.elementType}. Extend auditable-type.fmt');
</#if>
<#else>
<#if field.type = "int" || field.type = "java.lang.Integer">
out.writeUintvar31(value.${getter}());
<#elseif field.type = "boolean" || field.type = "java.lang.Boolean">
out.writeBoolean(value.${getter}());
<#elseif field.type = "java.lang.String">
out.writeString(value.${getter}());
</#if>
</#if>
<#assign useCollection = false>
</#if>
<#assign useField = false>
</@forAllFields>
}
}
);
}
}
</@javaSource>
</@forAllTypes>


My apologies, this is incredibly hard to read here on the blog. It helps to click the "view source" button in the upper right corner of the code above and copy everything to a text editor. I also added comments in the template to explain what I am doing.

Finally here is the configuration for the maven-apt-plugin, so that it will generate your code artifacts before compiling your project (note that target/generated-sources will be merged with the real sources during compile time).


<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>apt-maven-plugin</artifactId>
<version>1.0-alpha-4</version>
<configuration>
<factory>net.sf.jelly.apt.freemarker.FreemarkerProcessorFactory</factory>
<options>
<option>template=${basedir}/src/main/resources/apt/auditable-type.fmt
</option>
</options>
<fork>true</fork>
</configuration>
<dependencies>
<dependency>
<groupId>net.sf.apt-jelly</groupId>
<artifactId>apt-jelly-core</artifactId>
<version>2.14</version>
</dependency>
<dependency>
<groupId>net.sf.apt-jelly</groupId>
<artifactId>apt-jelly-freemarker</artifactId>
<version>2.14</version>
</dependency>
</dependencies>
<executions>
<execution>
<goals>
<goal>process</goal>
</goals>
</execution>
</executions>
</plugin>


And voilà, here is our generated BoughtHouseForGoldType class fresh out of the oven:


package package.types;

import java.io.IOException;

/**
* This class contains the {@link Datatype} for {@link BoughtHouseForGold}.
*/
public class BoughtHouseForGoldType extends AbstractAuditableType<BoughtHouseForGold> {
public BoughtHouseForGoldType() {
super(
TypeCodes.BOUGHT_HOUSE_FOR_GOLD_TYPE_CODE,
new Datatype<BoughtHouseForGold>(BoughtHouseForGold.class, 7) {

@Override
public BoughtHouseForGold read(final DatatypeInput in) throws DataFormatException {
final BoughtHouseForGold value = new BoughtHouseForGold();
value.setItemId(in.readUintvar31());
value.setSceneries(in.readList(Datatype.uintvar31));
return value;
}

@Override
public void write(final DatatypeOutput out, final BoughtHouseForGold value) throws IOException {
out.writeUintvar31(value.getItemId());
out.writeCollection(Datatype.uintvar31, value.getSceneries());
}
}
);
}
}