Howto manage Cron Jobs and Task Queues using Google AppEngine

In the process of discovering Google AppEngine's list of available features, I recently came across Cron Jobs and Task Queues. During my fairly extensive web development carrier, these terms always seemed like far away concepts that will not likely be a part of my average web development toolkit. At first, seeing these in AppEngine reminded me of those old mainframe systems that you read about in computer science manuals. One day I had some time to actually look into it, and was surprised and inspired by simplicity and usability of these nifty features.

In short, Cron is a time-based job scheduler. For web development, it could have variety of uses. For example:

- collecting data from a remote source and caching it locally
- processing local data to derive an intensive computation
- processing some data and sending email confirmations upon completion
etc... or basically anything that can facilitate app's performance that can run as a background process.

In AppEngine, it is very simple to declare a Cron Job. First, there is a cron.xml that specifies how you'd like to execute your jobs, for example:

saving it into /war/WEB-INF/cron.xml on Java SDK should do the trick. Notice url that points to web handler, time schedule that specifies how often this cron is run, a description of this cron, and a timezone. Cron jobs use Web Hooks to call a specified url at specified intervals. Sounds simple enough.. now we'll just have to define a handler for this request at /weather/storeData


Here is what AppEngine's dashboard looks like once a cron job is defined:

AppEngine SDK also provides a separate command to upload cron file:
appcfg update_cron war


Now, imagine that your cron job needs to spawn a number of processes to handle the request. This could be easily done using Tasks API. Here is an example queuing a task and passing a parameter to it:

A task handler could be any program/script that accepts HTTP requests. It would fetch passed parameters using familiar request.parameter('stationName'). The rest is up to the task :)


Notice that, in this case, we are calling a default queue. It is also possible to define a number of custom queues or change default queue, like this:

Here is what task dashboard looks like when tasks are executed:

Here is what regularly scheduled task executions look like using Request/Second Dashboard interface:

Finally, using both Cron Jobs and Task Queues adds a good bit to the overall collection of available tools. Using these tools appropriately significantly improves application's performance and simplifies overall code structure. Not to mention, it also looks very cool when browsing app's performance using AppEngine's dashboard interface :)

Thank you for reading and Aloha!

Ikayzo - Design • Build • Localize | Web • Desktop • Mobile