Last month Google added Java support to the
Google App Engine. This means, that you can from now on host Java based applications on the Google infrastructure. Even better is the fact, that you can do it for free. Well, it is free for starters. There are quotas like 5 million page views per day, 6,5 hours CPU time per day, 1GB traffic per day etc. For the everyday "lets-do-something in Java" app, this is more than enough, so I decided to run a simple
Wicket web-application on Googles appengine.
I am still a part-time student at
Fernuni Hagen in Germany and in the current semester I am in a course called “
Web 2.0 and social software”. The course is organized so that 3 students have to pick a topic from the web 2.0 bubble and prepare a presentation about it.
Recently I have read the book “Collective Intelligence in Action” so I thought I might as well talk about Tag, Tagging and TagClouds. It is always better to not just talk in your presentations but show some real action. Therefore I created a little web-application, based on Satnam Alag's domain model from the book and powered with Apache Wicket and Spring to be fully functional.
The non Google App Enginefied version uses an in-memory HSQL database to persist Tags, Users and Items. This means, if you restart the application, everything will be gone. Everything is visualized using a TagCloud embedded in a Apache Wicket WebPage. The page will also contain entry forms to add new Users, Items and Tags. So here is the domain model which consists of five classes. The Entity class is an abstract base class for Item, Tag, TaggedItem and User.
public abstract class Entity implements Serializable
{
private static final long serialVersionUID = 1L;
public int id;
public String name;
public int getId()
{
return id;
}
public void setId(int id)
{
this.id = id;
}
public String getName()
{
return name;
}
public void setName(String name)
{
this.name = name;
}
@Override
public boolean equals(Object o)
{
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
Entity entity = (Entity) o;
if (id != entity.id) return false;
return true;
}
@Override
public int hashCode()
{
return new Integer(id).hashCode();
}
}
public class User extends Entity
{
}
public class Tag extends Entity
{
}
public class Item extends Entity
{
}
public class TaggedItem extends Entity
{
private int _userId;
private int _itemId;
private int _tagId;
private int _weight;
public int getUserId()
{
return _userId;
}
public void setUserId(final int userId)
{
_userId = userId;
}
public int getItemId()
{
return _itemId;
}
public void setItemId(final int itemId)
{
_itemId = itemId;
}
public int getTagId()
{
return _tagId;
}
public void setTagId(final int tagId)
{
_tagId = tagId;
}
public int getWeight()
{
return _weight;
}
public void setWeight(final int weight)
{
_weight = weight;
}
}
The CRUD operations for this domain model are implemented using the DAO pattern (surprise, surprise). On top of the hierarchy sits the GenericDao interface which contains methods for read, write, delete. Then I have created interfaces for each individual DAO class that is connected to one entity in the domain model, ie. UserDao or ItemDao. These individual DAO interfaces extend the GenericDao, specifying the Entity to persist and the primary key type. The concrete implementation is now done in classes like UserJdbcDao or ItemJdbcDao. These are implementing the appropriate interface and extend a helper class called AbstractJdbcDao, which contains a reference to the Spring SimpleJdbcTemplate. Here is an example of the class structure for the User class.
public interface GenericDao<T, PK extends Serializable>
{
/**
* Persist the newInstance object into database
*/
PK create(T newInstance);
/**
* Retrieve an object that was previously persisted to the database using
* the indicated id as primary key
*/
T read(PK id);
/**
* Retrieves all objects that were previously persisted to the database.
*/
List<T> readAll();
/**
* Save changes made to a persistent object.
*/
void update(T transientObject);
/**
* Remove an object from persistent storage in the database
*/
void delete(T persistentObject);
}
public interface UserDao extends GenericDao<User, Long>
{
/**
* Returns the {@link User} whose name matches the given String.
* @param name a users name
* @return User or <code>null</code>
*/
User readByName(final String name);
}
public abstract class AbstractJdbcDao<T extends Entity>
{
private SimpleJdbcTemplate m_simpleJdbcTemplate;
private String m_identityQuery;
protected SimpleJdbcTemplate getSimpleJdbcTemplate()
{
return m_simpleJdbcTemplate;
}
public void setDataSource(DataSource dataSource)
{
m_simpleJdbcTemplate = new SimpleJdbcTemplate(dataSource);
}
public String getIdentityQuery()
{
return m_identityQuery;
}
public void setIdentityQuery(final String identityQuery)
{
m_identityQuery = identityQuery;
}
abstract ParameterizedRowMapper<T> getMapper();
}
public class UserJdbcDao extends AbstractJdbcDao<User> implements UserDao
{
private static final ParameterizedRowMapper<User> MAPPER = new ParameterizedRowMapper<User>()
{
public User mapRow(ResultSet rs, int rowNum) throws SQLException
{
final User user = new User();
user.setId(rs.getLong("user_id"));
user.setName(rs.getString("user_name"));
return user;
}
};
public Long create(User newInstance)
{
final SimpleJdbcTemplate template = getSimpleJdbcTemplate();
template.update("INSERT INTO user(user_name) VALUES(?)", newInstance.getName());
return template.queryForLong(getIdentityQuery());
}
public User read(Long id)
{
final SimpleJdbcTemplate template = getSimpleJdbcTemplate();
final List<User> users = template.query("SELECT * FROM user WHERE user_id = ?", getMapper(), id);
return users.isEmpty() ? null : users.get(0);
}
public User readByName(final String name)
{
final SimpleJdbcTemplate template = getSimpleJdbcTemplate();
final List<User> users = template.query("SELECT * FROM user WHERE user_name = ?", getMapper(), name);
return users.isEmpty() ? null : users.get(0);
}
public List<User> readAll()
{
final SimpleJdbcTemplate template = getSimpleJdbcTemplate();
return template.query("SELECT * FROM user", getMapper());
}
public void update(User transientObject)
{
throw new UnsupportedOperationException("Not implemented yet.");
}
public void delete(User persistentObject)
{
final SimpleJdbcTemplate template = getSimpleJdbcTemplate();
template.update("DELETE FROM user WHERE user_id = ?", persistentObject.getId());
}
ParameterizedRowMapper<User> getMapper()
{
return MAPPER;
}
}
Everything is wired together using Spring beans. This is where I define, that I want to use an in-memory database. All standard Spring stuff so far, no magic involved.
<?xml version="1.0" encoding="UTF-8"?>
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:aop="http://www.springframework.org/schema/aop"
xmlns:tx="http://www.springframework.org/schema/tx"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.5.xsd
http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.5.xsd">
<!-- HSQL DS -->
<bean id="dataSource" class="org.springbyexample.jdbc.datasource.InitializingBasicDataSource" destroy-method="close">
<property name="driverClassName" value="org.hsqldb.jdbcDriver"/>
<property name="url" value="jdbc:hsqldb:mem:."/>
<property name="username" value="sa"/>
<property name="password" value=""/>
<property name="sqlScriptProcessor">
<bean class="org.springbyexample.jdbc.core.SqlScriptProcessor">
<property name="sqlScripts">
<list>
<value>classpath:/db-create.sql</value>
</list>
</property>
</bean>
</property>
</bean>
<!-- DAO base -->
<bean id="daoWithDataSource" abstract="true">
<property name="dataSource" ref="dataSource" />
<property name="identityQuery" value="CALL IDENTITY();" /> <!-- HSQL specific -->
</bean>
<!-- JDBC DAO's -->
<bean id="userDao" class="com.fernuni.db.jdbc.UserJdbcDao" parent="daoWithDataSource" />
<bean id="itemDao" class="com.fernuni.db.jdbc.ItemJdbcDao" parent="daoWithDataSource" />
<bean id="tagDao" class="com.fernuni.db.jdbc.TagJdbcDao" parent="daoWithDataSource" />
<bean id="taggedItemDao" class="com.fernuni.db.jdbc.TaggedItemJdbcDao" parent="daoWithDataSource" />
</beans>
Also note that I used some special DataSource which will always execute a database initializer script. To use this
InitializingBasicDataSource, you have to reference the SpringByExample library. In Maven powered projects you would add this to your POM:
<repositories>
<repository>
<id>springbyexample.org</id>
<name>Spring by Example</name>
<url>http://www.springbyexample.org/maven/repo</url>
</repository>
</repositories>
<dependency>
<groupId>org.springbyexample</groupId>
<artifactId>spring-by-example-jdbc</artifactId>
<version>1.0.3</version>
</dependency>
Finally I have created a single page using Apache Wicket which will visualize the TagCloud and also contain form fields to create and delete Users, Items and Tags. The page class is called HomePage and contains references to all DAO beans. Later on, it will replace the DAO beans with the ones I have to use in a Google App Engine environment. Here is my HomePage. I have removed a lot of code for better readability as the class is complex. Get the source code to have a look at the full implementation.
public class HomePage extends WebPage
{
private static final long serialVersionUID = 1L;
private String m_newUserName = "";
private String m_newItemName = "";
private String m_newTagName = "";
private User m_taggingUser = new User();
private Item m_taggedItem = new Item();
@SpringBean
private UserDao m_userDao;
@SpringBean
private ItemDao m_itemDao;
@SpringBean
private TagDao m_tagDao;
@SpringBean
private TaggedItemDao m_taggedItemDao;
public HomePage(final PageParameters parameters)
{
addUserFields();
addItemFields();
addTagFields();
displayTagCloud();
...
}
private void displayTagCloud()
{
...
}
private void addTagFields()
{
final Form tagForm = new Form("newTags");
final RequiredTextField newTagName = new RequiredTextField("newTagName", new PropertyModel(this, "newTagName"));
tagForm.add(newTagName);
final IModel userChoices = new LoadableDetachableModel()
{
protected Object load()
{
return m_userDao.readAll();
}
};
final IChoiceRenderer userChoiceRenderer = new IChoiceRenderer()
{
public Object getDisplayValue(Object object)
{
final User user = (User) object;
return user.getName();
}
public String getIdValue(Object object, int index)
{
final User user = (User) object;
return user.getId() + "";
}
};
final ListChoice userListChoices = new ListChoice("taggingUser", new PropertyModel(this, "taggingUser"), userChoices, userChoiceRenderer);
userListChoices.setRequired(true);
tagForm.add(userListChoices);
final IModel itemChoices = new LoadableDetachableModel()
{
protected Object load()
{
return m_itemDao.readAll();
}
};
final IChoiceRenderer itemChoiceRenderer = new IChoiceRenderer()
{
public Object getDisplayValue(Object object)
{
final Item item = (Item) object;
return item.getName();
}
public String getIdValue(Object object, int index)
{
final Item item = (Item) object;
return item.getId() + "";
}
};
final ListChoice itemListChoices = new ListChoice("taggedItem", new PropertyModel(this, "taggedItem"), itemChoices, itemChoiceRenderer);
itemListChoices.setRequired(true);
tagForm.add(itemListChoices);
final Button saveNewTagButton = new Button("saveNewTagButton")
{
@Override
public void onSubmit()
{
super.onSubmit();
Tag existingTag = m_tagDao.readByText(m_newTagName);
if (existingTag == null)
{
final Tag newTag = new Tag();
newTag.setName(m_newTagName);
m_tagDao.create(newTag);
existingTag = m_tagDao.readByText(m_newTagName);
assert existingTag != null;
}
final Long taggingUserId = m_taggingUser.getId();
final Long taggedItemId = m_taggedItem.getId();
final TaggedItem taggedItem = new TaggedItem();
taggedItem.setItemId(taggedItemId);
taggedItem.setUserId(taggingUserId);
taggedItem.setTagId(existingTag.getId());
m_taggedItemDao.create(taggedItem);
}
};
tagForm.add(saveNewTagButton);
add(tagForm);
// Existing Tags
final Form existingTagsForm = new Form("existingTagsForm");
final IModel taggedItemsModel = new LoadableDetachableModel()
{
protected Object load()
{
return m_taggedItemDao.readAll();
}
};
final ListView existingTaggedItems = new ListView("existingTaggedItems", taggedItemsModel)
{
protected void populateItem(ListItem item)
{
final TaggedItem taggedItem = (TaggedItem) item.getModelObject();
final User taggingUser = m_userDao.read(taggedItem.getUserId());
final Item itemContainingTag = m_itemDao.read(taggedItem.getItemId());
final Tag tag = m_tagDao.read(taggedItem.getTagId());
final Label taggedText = new Label("taggedText", new PropertyModel(tag, "name"));
item.add(taggedText);
final Label taggedBy = new Label("taggedBy", new PropertyModel(taggingUser, "name"));
item.add(taggedBy);
final Label taggedAt = new Label("taggedAt", new PropertyModel(itemContainingTag, "name"));
item.add(taggedAt);
final Button deleteTaggedItem = new Button("deleteTaggedItem")
{
@Override
public void onSubmit()
{
super.onSubmit();
m_taggedItemDao.delete(taggedItem);
}
};
item.add(deleteTaggedItem);
}
};
existingTagsForm.add(existingTaggedItems);
add(existingTagsForm);
}
...
}
So far so good. Lets now Google App Enginefy this application. First of all, since this application uses Apache Wicket, you have to change some settings regarding Threads. Google Appengine does not support Threads today. Step 1: run the Wicket application in
deployment mode to disable a thread checking for modifications in the background. Step 2: override the method newSessionStore() in your WebApplication class like this:
@Override
protected ISessionStore newSessionStore()
{
return new HttpSessionStore(this);
}
This will eliminate another thread that is used by the default DiskPageStore class. Step 3:
enable sessions in Google App Engine as they are turned off by default. Wicket uses Http Session heavily. I got all this from
Alastair Maw’s blog, thanks a lot!
Now you have to add all the libraries that come with the Google App Engine SDK to your project. In my case the project is based on Maven, so I added the following to my pom.xml.
<repositories>
<repository>
<id>appengine</id>
<name>Google App Engine Libraries</name>
<url>http://www.mvnsearch.org/maven2</url>
</repository>
<repository>
<id>datanucleus</id>
<name>Datanucleus Libraries</name>
<url>http://www.datanucleus.org/downloads/maven2</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>com.google.appengine</groupId>
<artifactId>jdo2-api</artifactId>
<version>2.3-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>com.google.appengine</groupId>
<artifactId>datanucleus-appengine</artifactId>
<version>1.0.1.final</version>
</dependency>
<dependency>
<groupId>org.datanucleus</groupId>
<artifactId>datanucleus-core</artifactId>
<version>${datanucleus.version}</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.datanucleus</groupId>
<artifactId>datanucleus-jpa</artifactId>
<version>${datanucleus.version}</version>
</dependency>
<dependency>
<groupId>com.google.appengine</groupId>
<artifactId>appengine-api-1.0-sdk</artifactId>
<version>1.2.1</version>
</dependency>
</dependencies>
<properties>
<datanucleus.version>1.1.0</datanucleus.version>
</properties>
Unfortunately some dependencies cannot be resolved, even with the extra repositories I added manually. You have to add these jar file manually to your local Maven repository. I will not write about it here but check
Dan Walmsleys Blog or
Shalins Blog, they explain what you need to do.
Alright, now that we have all set up to Google App Enginefly our Java application - let's start. The App Engine SDK ships with a Servlet Container which you can start and test your application in an App Engine like environment.
Download and unzip the SDK, then go into the bin folder. To enable remote debugging in the appengine environment, open the startup script (dev_appserver.sh in Linux) and make it look like this:
#!/bin/bash
# Launches the development AppServer
[ -z "${DEBUG}" ] || set -x # trace if $DEBUG env. var. is non-zero
SDK_BIN=`dirname $0 | sed -e "s#^\\([^/]\\)#${PWD}/\\1#"` # sed makes absolute
SDK_LIB=$SDK_BIN/../lib
SDK_CONFIG=$SDK_BIN/../config/sdk
java -ea -cp "$SDK_LIB/appengine-tools-api.jar" \
com.google.appengine.tools.KickStart \
--jvm_flag=-Xdebug \
--jvm_flag=-Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n \
com.google.appengine.tools.development.DevAppServerMain $*
Next time you start your local appengine, you will be able to start a remote debugger on port 8000. Nice!
So at this point, the application would run on Google App Engine without problems. However, the data is not persistent and will be gone every time you restart. So let's use a real database. Each App Enginefied application can access something what Google calls a datastore. The datastore may be accessed using either JDO or JPA. Neither plain JDBC nor O/R mappers like Hibernate or Toplink will work at this stage. Appengine uses something called
datanucleus for datastore access, which I think is another abstraction for different ways of persistence. So datanucleus will process JDO or JPA instrumented classes and do some persistence magic.
I decided to use JDO to access the appengine datastore from my application. In the first step let's add a PersistenceManagerFactory and some new DAO beans to the current Spring configuration.
<bean id="persistenceManagerFactory" class="org.springframework.orm.jdo.LocalPersistenceManagerFactoryBean">
<property name="jdoProperties">
<props>
<prop key="javax.jdo.PersistenceManagerFactoryClass">
org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManagerFactory
</prop>
<prop key="javax.jdo.option.ConnectionURL">appengine</prop>
<prop key="javax.jdo.option.NontransactionalRead">true</prop>
<prop key="javax.jdo.option.NontransactionalWrite">true</prop>
<prop key="javax.jdo.option.RetainValues">true</prop>
<prop key="datanucleus.appengine.autoCreateDatastoreTxns">true</prop>
<prop key="datanucleus.DetachOnClose">true</prop>
</props>
</property>
</bean>
<bean id="daoWithPersistenceManagerFactory" abstract="true">
<property name="persistenceManagerFactory" ref="persistenceManagerFactory" />
</bean>
<bean id="userDao" class="com.fernuni.db.jdo.UserJdoDao" parent="daoWithPersistenceManagerFactory" />
<bean id="itemDao" class="com.fernuni.db.jdo.ItemJdoDao" parent="daoWithPersistenceManagerFactory" />
<bean id="tagDao" class="com.fernuni.db.jdo.TagJdoDao" parent="daoWithPersistenceManagerFactory" />
<bean id="taggedItemDao" class="com.fernuni.db.jdo.TaggedItemJdoDao" parent="daoWithPersistenceManagerFactory" />
Next I wrote a set of new DAO classes based on the JdoTemplate from Spring. When compared to JDBC, it is the AbstractJdoDao this time, which contains most code. The concreted DAO classes like UserJdoDao are rather small and reuse almost everything in the AbstractJdoDao.
public abstract class AbstractJdoDao<T extends Entity>
{
private JdoTemplate m_jdoTemplate;
public JdoTemplate getJdoTemplate()
{
return m_jdoTemplate;
}
public void setPersistenceManagerFactory(final PersistenceManagerFactory persistenceManagerFactory)
{
m_jdoTemplate = new JdoTemplate(persistenceManagerFactory);
}
public T readEntityByName(final String name, final Class<T> clazz)
{
final JdoTemplate jdoTemplate = getJdoTemplate();
final Collection found = jdoTemplate.find(
clazz, "m_name == value", "String value", new Object[] {name}
);
return (found != null && !found.isEmpty()) ? (T) found.toArray()[0] : null;
}
public Long createEntity(final T item)
{
final JdoTemplate jdoTemplate = getJdoTemplate();
final T persistentItem = (T) jdoTemplate.makePersistent(item);
return persistentItem.getId();
}
public T readEntity(final Long id, final Class<T> clazz)
{
final JdoTemplate jdoTemplate = getJdoTemplate();
return (T) jdoTemplate.getObjectById(clazz, id);
}
public List<T> readAllEntities(final Class<T> clazz)
{
final JdoTemplate jdoTemplate = getJdoTemplate();
return new ArrayList<T>(jdoTemplate.find(clazz));
}
public void updateEntity(final T transientObject)
{
final JdoTemplate jdoTemplate = getJdoTemplate();
jdoTemplate.refresh(transientObject);
}
public void deleteEntity(final T persistentObject)
{
final JdoTemplate jdoTemplate = getJdoTemplate();
jdoTemplate.deletePersistent(persistentObject);
}
}
public class UserJdoDao extends AbstractJdoDao<User> implements UserDao
{
public User readByName(final String name)
{
return readEntityByName(name, User.class);
}
public Long create(final User user)
{
return createEntity(user);
}
public User read(final Long id)
{
return readEntity(id, User.class);
}
public List<User> readAll()
{
return readAllEntities(User.class);
}
public void update(final User transientObject)
{
updateEntity(transientObject);
}
public void delete(final User persistentObject)
{
deleteEntity(persistentObject);
}
}
Using JDO requires that the bytecode in your domain model classes is instrumented with the persistence information. So first of all, lets put down this information. To be least intrusive, I decided to provide separate .jdo files instead of using annotations. Here is the package.jdo file which I put in the package along with my domain model classes.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE jdo PUBLIC
"-//Sun Microsystems, Inc.//DTD Java Data Objects Metadata 2.0//EN"
"http://java.sun.com/dtd/jdo_2_0.dtd">
<jdo>
<package name="com.fernuni.domain">
<class name="Entity" detachable="true" identity-type="application">
<inheritance strategy="complete-table"/>
<field name="id" primary-key="true" value-strategy="identity">
<column name="ENTITY_ID"/>
</field>
<field name="name">
<column name="ENTITY_NAME"/>
</field>
</class>
<class name="User" detachable="true" identity-type="application">
</class>
<class name="Item" detachable="true" identity-type="application">
</class>
<class name="Tag" detachable="true" identity-type="application">
</class>
<class name="TaggedItem" detachable="true" identity-type="application">
<field name="m_userId">
<column name="TAGGED_USER_ID"/>
</field>
<field name="m_itemId">
<column name="TAGGED_ITEM_ID"/>
</field>
<field name="m_tagId">
<column name="TAGGED_TAG_ID"/>
</field>
</class>
</package>
</jdo>
Unfortunately datanucleus does not support primary keys of type Integer and all Id's in my JDBC based applications were Integers. So I had to change from Integer to Long to get it working. Quite intrusive but something I can live with. As a last step, Maven needs to instrument the domain classes using the JDO instructions from my package.jdo file. I added the maven-datanucleus-plugin in my Maven build process.
<build>
<plugins>
<plugin>
<groupId>org.datanucleus</groupId>
<artifactId>maven-datanucleus-plugin</artifactId>
<version>${datanucleus.version}</version>
<configuration>
<mappingIncludes>**/*.class</mappingIncludes>
<verbose>true</verbose>
<enhancerName>ASM</enhancerName>
<api>JDO</api>
</configuration>
<executions>
<execution>
<phase>compile</phase>
<goals>
<goal>enhance</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
Unfortunately it turned out the Google App Engine has huge difficulties when you have a hierarchy of persistence capable subclasses. In my case each User, Item, Tag and TaggedItem all derive from Entity. The Entity class contains the persistent fields id and name. For some reason in the current version of the datanucleus-appengine plugin, the fields from a persistence capable superclass are not accessible within the subclass. In other words, you cannot use inheritance in your domain model if you want to use JDO and Google App Engine. I have started
two threads in the App Engine community and Max Ross from Google promised that Inheritance will receive a higher priority in upcoming versions of the datanucleus-appengine plugin.
Let's sum this up. Google App Engine is a really great thing Google came up with. It is also a Google example that Tim O'Reilly is right about the
perpetual Beta, which is what I think we see with appengine right now. It has been put into public earlier than you would usually expect it from other Google products, probably to leverage the “power of the community” in making it better and more stable. On the other hand, I do not think that Google App Engine has reached a state where it can be used for real life enterprise applications.
