Scala is a hot topic in the programming world. I’ve been following Scala for quite a while, and about 2 years ago I endeavored programming in Scala on top of GigaSpaces XAP (disclaimer: I’m solution architect for GigaSpaces). XAP had no built-in support for Scala, but I leveraged XAP’s native support for Java together with Scala’s compatibility with Java.
And it worked like a charm!
You can read the full details in my previous blog post. I concluded my blog post by saying that
… programming in Scala on top of XAP is a viable notion that deserves further investigation.
However, I also added a disclaimer that
… XAP platform offers a vast array of features, and that the Scala language offers a vast array of constructs, very few of which have been covered on this experiment. Similarly, I should also state that Scala is not officially supported by the XAP product, which means that there is no official support or test coverage of Scala in the product.
Further explorations of more advanced Scala usage, also together with concrete customer use cases, showed that although possible, the resulting code is less intuitive for Scala users, and does not fully utilize the elegant constucts of the Scala language.
Two years went by, and now we decided to take our relationship with Scala to the next level, and make Scala programming on XAP much more intuitive with things such as better support for immutable objects, functional querying using predicates, Scala script execution and enhanced REPL shell. XAP now also exposes some of the platform’s powerful mechanisms for distributed and scalable processing, such as remote script execution and Map/Reduce pattern, over native Scala. These goodies have just been unveiled as part of the latest release of XAP (XAP 9.6). Let’s give you a taste of some of the goodies.
Predicate-based queries
You can now run queries based on Scala predicates just as you’re used to for functional querying:
val pGigaSpace = gigaSpace.predicate
val person = pGigaSpace.read { person: Person =>
person.age > 25 || person.name == personName }
This will be compiled into the XAP native SQL query mechanism, so no runtime overhead and you get all the optimizations available on the platform’s SQL query engine and indexing.
Scatter/Gather and Map/Reduce patterns
XAP contains a mechanism called Task Execution which provides easy implementation of the Scatter/Gather and Map/Reduce patterns.
Scatter:
Gather:
You can now define both the Scatter and the Gather as Scala functions and dispatch onto a cluster of nodes with a simple invocation:
val asyncFuture2 = gigaSpace.execute(
{ gigaSpace: GigaSpace => gigaSpace.read(Data()).data },
{ results: Seq[AsyncResult[String]] => results.map { _.getResult() } mkString } )
Remote and parallel execution of Scala scripts over a cluster
What if you want to execute your Scala script across a cluster of compute nodes (a-la compute grid)? Maybe even colocated with the data found on these nodes (a-la in-memory data grid)? This is now easily achievable using XAP’s Dynamic Language Tasks (which also supports other dynamic language scripts such as JavaScript and Groovy):
val script = new ScalaTypedStaticScript("myScript", "scala", code)
.parameter("someNumber", 1)
.parameter("someString", "str")
.parameter("someSet", Set(1,2,3), classOf[Set[_]])
val result = executor.execute(script)
Final words
XAP is a multi-language and multi-interface platform. You can write in Java, .NET, C++;you can use standards such as SQL, JPA and Spring; you can use it as a key-value store; you may even choose to store your data in document format instead of objects, to support semi-structured model. So enhancing XAP to support Scala was but a natural move.
In my blog post two years ago I concluded saying that
… [Scala] is an exciting option worth exploring, and who knows, if Scala becomes predominant it may one day become an official feature of the product.
Finally the day has come and Scala made its first steps in becoming a first-level citizen in GigaSpaces XAP. What I described above is just part of it. You can read the full listing here. There are yet many more things to do in order to fully expose XAP platform’s rich functionality through Scala functional language, such as full support for immutable types. Now it’s time for the user community to check it out. So go ahead and play with it and let us know what you think you need to make your application scale all the way with XAP and Scala.
Follow Horovits on Twitter!
horovits 