Scaling All the Way: Running Scala Applications on GigaSpaces XAP Platform

Scala, the “scalable language”, is the hottest thing in the programming languages world these days. You may have thought that innovation these days only happens in dynamic scripting languages. Or you may have thought that Java and .Net cover all angles for static languages. Or that object-oriented programming is the answer to every large-scale system. So here comes Scala and shuffles the cards. It offers a fascinating fusion of object-oriented and functional paradigms, an elegant and compact syntax, full compatibility with Java and .Net, and most notably – it is easily extensible for defining Domain-Specific Languages (DSL). Several major players already recognized the potential of the language and switched to Scala, including Twitter, Foursquare, and Sony Pictures. A fascinating example of the power of the language is the adoption of the Actor Model for high-level concurrency inspired by the Erlang language.

On one of my recent consulting sessions with a customer in the Financial Services industry in the UK, the customer expressed a desire to develop in Scala on top of the GigaSpaces XAP platform. XAP officially does not support the Scala language, so the customer consulted me on how to construct such a solution on top of the product. Well, I’m a solutions architect, so that’s just the kind of challenges I live for…

My first thought was that as Scala declares full compatibility with Java and complies with Java classes (with very few exceptions), and as any Java program can run on XAP, there should be a way to make it work. However, the customer reported that initial experiments had failed. While visiting the customer onsite, I investigated the problems together with the customer, and came up with some interesting conclusions about inter-operability with Scala that I’d like to share in this post.

The customer had domain model objects written in Scala that he wished to write into the Space. The Space is GigaSpaces implementation of an In-Memory Data Grid. Interestingly, the space functions as a general-purpose data store and exposes an Open Interfacing Layer that enables access to the data using various languages (Java, .Net, C++, scripting) and standard APIs (JPA, JDBC, REST, Map, Document, Memcached, Spring, and more). So having Scala as yet another API for interacting with data in the space would be a natural extension, wouldn’t it?

In order to store instances of a given Java domain class (POJO) in the Space, the class needs to be decorated with metadata to determine things such as which of its properties would serve as the primary key, which properties need to be indexed for efficient querying, etc. GigaSpaces offers two standard methods to specify this metadata: using Java annotations in the class source definition, or using XML alongside the class definition. I wanted to check if any of these methods can serve to similarly decorate Scala classes in preparation for storage in the Space. Experimenting with the customer’s use case, I was able to make the domain model of the customer work with both methods, namely the Scala objects were properly written into the data grid and replicated across the data grid after being annotated using the standard GigaSpaces Java annotations, or using the standard GigaSpaces XML.

That indeed solved the needs of the customer. But I myself only got more of an appetite: What else can be done with Scala on top of XAP? GigaSpaces XAP offers more than just an in-memory data grid, it also offers services on top of the data grid, such as remote invocation using the space as the transport layer, event-driven containers, executing distributed tasks in a Map/Reduce pattern, and more. Would these also work so neatly with Scala in virtue of Scala’s compatibility with Java? So I worked with my colleague Dan Kilman to port the GigaSpaces “Hello World” application to Scala. This application is the beginner’s example application (packaged with the XAP distribution), and it utilizes the basic features of the platform. Porting turned out to be straight-forward, and worked smoothly. The example is published as an open-source project at OpenSpaces.org.

This little experiment showed that programming in Scala on top of XAP is a viable option that deserves further investigation. I should add a proper disclaimer that the XAP platform offers a vast array of features, and that the Scala language offers a vast array of constructs, very few of which have been covered in this experiment. Similarly, I should also state that Scala is not officially supported by the XAP product, which means that there is no official support or test coverage of Scala for the product. However, as a solutions architect exploring possible solutions on top of the product I can now say that this is an exciting option worth exploring, and who knows, if Scala becomes predominant it may one day become an official feature of the product.

1311765722_picons03
Follow Dotan on Twitter!

About these ads

4 Comments

Filed under Programming Languages

4 responses to “Scaling All the Way: Running Scala Applications on GigaSpaces XAP Platform

  1. Pingback: Running Scala Applications on GigaSpaces XAP Platform - async I/O News

  2. Pingback: GigaSpaces on Application Scalability | Open Source PaaS and More - Scaling All the Way: Running Scala Applications on GigaSpaces XAP Platform

  3. Pingback: Scaling All The Way: Welcoming Scala as a First-Level Citizen in GigaSpaces XAP | horovits

  4. Pingback: Scaling All The Way: Welcoming Scala as a First-Level Citizen in GigaSpaces XAP | GigaSpaces on Application Scalability | Open Source PaaS and More

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s