From Java to Clojure Followup

Interest in the From Java to Clojure article has been pretty big, rarely before have we seen a PNEHM!-article getting so much attention!

I have also gotten some great feedback, in particular Stuart Halloway suggests that we use Clojure-contrib's seq-utils/frequencies to improve the code. Since we use Apache Commons Collections in the Java version of the code, its only fair for us to dive into the goodness of Clojure-contrib to see what can be of use there.

Let us start by a quick review of what our final version of the Clojure code looked like:

(ns step4.pnehm.clojure-orderer)

(defn count-words [coll]
   (reduce #(merge-with + %1 {%2 1}) {} coll))

(defn cmpr [[val1 freq1] [val2 freq2]]
   (let [freq (compare freq2 freq1)]
     (if-not (zero? freq) freq (compare val1 val2))))

(defn order-by-freq [coll]
   (keys (sort cmpr (count-words coll))))

As it turns out, clojure.contrib.seq-utils/frequencies does exactly what our function count-words does, as such we can use it as a drop-in replacement. A version which uses contrib now looks like this:

(ns withcontrib.pnehm.clojure-orderer
   (:use clojure.contrib.seq-utils))

(defn cmpr [[val1 freq1] [val2 freq2]]
   (let [freq (compare freq2 freq1)]
   (if-not (zero? freq) freq (compare val1 val2))))

(defn order-by-freq [coll]
  (keys (sort cmpr (frequencies coll))))

We have to make sure we :use clojure.contrib.seq-utils, and then we can replace our call to count-words, with a call to frequencies.

Now, for extra credits, let's look inside the frequencies function in Clojure-contrib, to see what i looks like:

(defn frequencies
  "Returns a map from distinct items in coll to the number of times
  they appear."
  [coll]
  (reduce (fn [counts x]
              (assoc counts x (inc (get counts x 0))))
          {} coll))

The implementation is quite different from our own, it feels a bit more straight-forward and intuitive. Initially, an empty map is created. As the collection is reduced a copy of the map is created for each processed item and the item is added with an incremented count (if the item already is in the map) or added to the new map with a count of one (if it is the first time the item is processed). get, gets a value from the map given a key, if there is no match the default, '0', is returned. inc increments the value, and assoc associates the value to a key in the map.

Not only is this version simpler than our own (which is good), it's also much faster (also good). Using seq-utils/frequencies a sample run with our micro-benchmark now looks like this (sorting 100 characters with 10000 samples):

Java: 120 ms
Groovy: 538 ms
Time Clojure: 563 ms

Excellent!

So, Joshua Bloch's item 47 in Effective Java (2 ed) applies as always: Know and use the libraries. If you get the feeling that someone must have done what you are about to do before you, someone most probably have.

Many thanks to Stuart Halloway (who will be at Øredev next week, don't miss it!).

The sources at http://code.google.com/p/pnehm-java-to-cool-language-x/ have been updated with the alternative version.

From Java to Clojure

In this article, published in Citerus’ newsletter PNEHM!, I introduce Clojure to Java developers by converting a piece of Java code to Clojure. The article examines how to call Java code from Clojure and Clojure code from Java, complete with source code.

Citerus’ consultant Peter Backlund took the same journey some months ago with his article Från Java till Groovy (in Swedish).

Read From Java to Clojure.

DDDSample Presentation (free event)

On Thursday evening, October 22, Sweden Spring User Group is hosting an even in Stockholm where Peter Backlund (mostly) and I (less) will present the current state of the DDDSample application.

Agenda

DDDSample är en Spring-applikation som utvecklats för att påvisa hur idéer och koncept från Domain-Driven Design kan implementeras i en modern utvecklingsstack. Applikationen utvecklas i samarbete med Eric Evans, författare till boken Domain-Driven Design, och syftar till att visa hur mönster från DDD konkret kan användas för att skapa en systemarkitektur för att lösa problem i en komplex domän, utan att skapa onödig teknisk komplexitet.

Sedan applikationen först introducerades för drygt ett år sedan har den kontinuerligt utvecklats, mycket tack vara den feedback som kommit från olika communities. Det är snart dags för en ny release och under denna träff kommer vi kika närmare på applikationens olika delar och se hur dessa mappar mot koncept inom Domain-Driven Design.

Vi kommer att visa hur Spring (och i viss mån andra ramverk) hjälper oss att programmera domändrivet genom att dels låta oss bygga en ren och rik modell i Java, och dels genom att ta hand om infrastrukturell kod och låta oss fokusera på affärsnytta (focus on the core domain).

The presentation will be in Swedish and the event is free to attend.

Sign up at http://www.eventbrite.com/event/445416252

Welcome!

Mocking to the Rescue!

I am not really a big fan of mocking. My experience is that if you design your code to be testable, you only rarely have to rely on mocks, instead you'll be fine just by stubbing an interface or two. But even as a proponent of what Martin Fowler calls classic TDD, I find it there are times when a mocking tool can come in handy. One situation is when you have to work with API:s that lack good interfaces, in these cases a good mocking tool with support for mocking concrete classes can be of great help.

Mockito is what I would like to consider a next-generation mocking tool, with support for both mocking and stubbing. Mockito fully and naturally uses recent language enhancements in Java, such as generics, static imports and annotations, to make the tool easy to use. Writing clean and elegant test code that is easy to understand can actually be pretty simple. TheMockito API is straightforward and well designed, the need for infrastructure code is kept to a minimum. If you are an EasyMock user, this comparison may be helpful when following along in the code below: http://code.google.com/p/mockito/wiki/MockitoVSEasyMock

Let's look at a few examples!

Stub an interface

A nice property of systems built using dependency injection is that you get loosely coupled systems with well-defined interfaces that are easy to stub for testing. It is easy enough to provide stub implementations of these interface directly in the test classes, e.g. as anonymous implementations or inner classes. But despite this, I have found myself more and more starting to rely on Mockito for these situations, it's fewer lines of code, and very convenient!

import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
import static org.mockito.Mockito.verify;
import static org.junit.Assert.assertEquals;

[...]

ClientRepository repository = mock(ClientRepository.class);
ClientId clientId = new ClientId(123);
Client client = new Client(clientId, new Name("Test", "User"));
when(repository.findById(clientId)).thenReturn(client);

Now, as an example, we can use our ClientRepository instance to test a service class:

FancyService service = new FancyServiceImpl();
service.setRepository(repository);
assertEquals(client, service.findClientAndDoSomethingTrulyAwesome(123));

When stubbing like this we are usually not that interested in verifying behaviour of the stub. But if we, for some reason, would like to explicitly make sure that the method findById(...) was called, we can do this with the following line of code:

verify(repository).findById(clientId);

Stubbing concrete classes

Now and then you encounter API:s that were not as designed for testing that you could wish for. In these cases, the possibilities of stubbing concrete classes can be very helpful.

Restlet

Restlet is a Java framework for creating RESTful web service. Reslet consists of quite a big API, and the authors have, among other things, decide to create their own implementation of things like Request, Response, Status.SUCCESS_OK (HTTP response 200) etc. Unfortunately, many parts of the API consists of concrete classes, instead of interfaces. One downside of this is that it is sometimes hard to write tests for code that uses the Restlet APIs.

Stubbing the concrete class org.restlet.data.Response is easily done in the same way we stubbed the ClientRepository interface above:

import static org.mockito.Mockito.*;
import org.restlet.data.Response;
import org.restlet.data.Status;
import org.restlet.resource.Representation;

[...]

Response response = mock(Response.class);
Representation entity = mock(Representation.class);
when(response.getStatus()).thenReturn(Status.SUCCESS_OK);
when(response.getEntity()).thenReturn(entity);

Quartz

Quartz is a scheduling component that is supported and also used by many popular application development frameworks and application servers, including Spring and Seam. When using Quartz you define jobs that execute according to a schedule. Every time a job is triggered a new instance of the job class is created, and executed. If state is to be saved between job executions it has to be stored in some kind data structure outside of the job. Quartz makes a context available, org.quartz.JobExecutionContext, for this and other purposes. The context is handed to every newly created job instance. The context can, a bit simplified, be viewed as Map where the job can store and retrieve data.

If we would like to make data available to a job or read the result from a job execution we have to create an instance of JobExecutionContext and hand to our job. Creating this instance is however quite complicated, and since Quartz also tends to favor concrete classes over interfaces, it makes it hard for us to provide our own implementation. Fortunately, Mockito (or another mocking tool that can mock concrete classes) can help us out here as well!

Use Mockito to setup the context and set a fictional indexCount parameter as input value to the job:

import static org.mockito.Mockito.*;
import org.quartz.JobDataMap;
import org.quartz.JobDetail;
import org.quartz.JobExecutionContext;

[...]

JobExecutionContext ctx = mock(JobExecutionContext.class);
JobDetail detail = mock(JobDetail.class);
JobDataMap map = new JobDataMap();
map.put("indexCount", 145);
when(detail.getJobDataMap()).thenReturn(map);
when(ctx.getJobDetail()).thenReturn(detail);

Now we can use the context in our test:

IndexUpdaterJob job = new IndexUpdater();
job.execute(ctx);

And verify that the index was updated:

map = ctx.getJobDetail().getJobDataMap();
assertEquals(146, map.get("indexCount"));

Mocking

As the name suggests, Mockito can of course be used for different kinds of mocking as well. For more examples, have a look at the Mockito documentation.

Follow Citerus on Twitter

What the title says: @citerus_se.

Geeky Fact of the Day

I read this tweet by Joshua Bloch:

"Geeky fact of the day: the sum of the first n cubes equals the square
of the sum of the first n integers"

And decide to throw some Clojure at it:

(use 'clojure.contrib.math)

(defn geeky-fact [n]
  (=
    (reduce + (map #(expt % 3) (take n (iterate inc 1))))
    (expt (reduce + (take n (iterate inc 1))) 2)))

Clojure Crash Course For Java Developers

Updated, fixed bug in test-vec def

Looking at Clojure code coming from a Java background may be quite a daunting experience at first. So I have put together a short introduction that I hope can facilitate going from Java to Clojure.

Clojure is a functional programming language. As such we use functions to manipulate data, we avoid mutable state and side-effects. In a way it can be viewed as programming Java with only static methods and immutable objects. At first this may sound like a huge limitation, but in contrast to Java, functions in Clojure are are first-class citizens and Clojure fully support higher-order functions. That is, we can treat functions like any other piece of data, functions can be assigned to variables, functions can be passed as arguments to other functions and we can write functions that return functions. This can be very powerful and by keeping data immutable and strive for side-effect free behavior the idea is that we can write more robust programs that  more easily can scale to multiple cores.

Clojure is a homoiconic language, that is, data and program instructions are represented the same way. This may look weird at first, and could need some time getting used to. Clojure programs typically starts out as text (in a text file) and is converted by a reader into data structures used by the compiler. The reader reads the text form by form. A form is a chunk of data that can be translated into a Clojure data structure. Numbers, like 123, are forms, so are lists, (+ 1 2) and vectors, [1 2 3].

Program and data are expressed as s-expressions, symbolic expressions, in a parenthesized prefix-notation. I.e. the function comes first and the data follows. The Clojure expression for adding the numbers 1, 2, and 3 is: (+ 1 2 3) This is a list and as such it will be evaluated as a function call. The function is + and the data to apply the function on comes after that.

Expressions can be, and typically are nested: (+ 1 2 3 (- 2 3)) => 5

REPL
The Read Evaluate Print Loop is an excellent tool to use when experimenting with Clojure. It is part of the Clojure distribution, so download and set it up before we proceed, and you will be able to try out things live as we look closer at the language:

1) Download Clojure (examples below are for Clojure 1.0.0).
2) Extract the zip and run java -cp clojure-1.0.0.jar clojure.lang.Repl

You should see something like:

Clojure 1.0.0-
user=>

Where user indicates the current namespace.

For a somewhat more comfortable REPL experience, add the JLine ConsoleRunner:

java -cp jline-0.9.94.jar:clojure-1.0.0.jar jline.ConsoleRunner clojure.lang.Repl

Forms
We have already met the numeric form, which is a literal, the vector, and the list but there are other forms as well:

Symbols:
Symbols are used to name things such as functions, namespaces (a Clojure equivalent of Java packages) and Java classes. We have already seen one: the + operator, which is a function.

Literals:
Strings, numbers, characters, nil (Java null), booleans (true, false) and keywords. Keywords are like symbols but begins with a colon e.g. :color.

Map:
Much like Java Maps, Clojure maps are zero or more key/value pairs enclosed in braces: {:language "Clojure", :kind "Functional"}
(commas are treated like witespace, leave them out if you like)

Set:
Much like Java Set, Clojure sets are zero or more forms enclosed in braces with a leading #: #{:red :green :orange}

These forms are translated in to data structures by the reader and compiler. More on Clojure data structures, their characteristics, related functions and relation to Java can be found on the Clojure Data Structures page.

Special Forms
In addition to the forms we have mentioned so far we have what are called "special forms", these are forms evaluated in a special way to do things normal forms can not do. There are a few particularily important ones:

(def symbol init?)
Binds a symbol to a global var in the current namespace, this is used to define things we need access to, e.g. a function or a value. Try it out using the REPL:

user=> (def a 42)
> #'user/a

user=> a
> 42

(if test then else?)
Evaluate test, if not nil or false, evaluate then.

user=> (if (> 3 2) "TRUE" "FALSE")
> "TRUE"

(let [bindings* ] exprs*)
Create local bindings used when evaluating exprs*

user=> (let [a 2] (> a 3))
> false

(fn name? [params* ] exprs*)
Defines a function. If we don't need to refer to the function, we can make it anonymous by not supplying a name:

Create a function that adds two values, and immediately call it on 1 and 2:

user=> ((fn [a b] (+ a b)) 1 2)
> 3

More special forms: http://clojure.org/special_forms

Macros
Clojure macros make it possible to add new functionality to the language, for instance by combining primitive forms. As a Clojure beginner you can safely ignore Clojure's macro functionality until you feel more confident in the language. But it can be good to know that many constructs in Clojure are macros, but they look just like ordinary functions. One of the most commonly used ones are probably (defn name doc-string? attr-map? [params*] body), used to create and name functions globally:

user=> (defn add [a b] (+ a b))
> #'user/add

user=> (add 2 3)
> 5

Using the function macroexpand we can see that defn is infact a combination of def and fn:

user=> (macroexpand '(defn add [a b] (+ a b)))
> (def add (clojure.core/fn ([a b] (+ a b))))

Immutable data.
The set of data structures in Clojure is extensive. We met a few of the structures when we talked about forms above. An important of property of Clojure data structures is that they are immutable. For example, operations on a list returns a new list with the result of the operation. The original list is untouched. We'll see examples of this below.

Functions
There is quite an extensive set of functions in the core Clojure libraries. Mathematical functions are named after their common symbols, as you would expect:

user=> (+ 1 2)
> 3

user=> (* 2 5)
> 10

user=> (> 1 4)
> false

There are also many functions to work on collections:

user=> (def test-vec [1 2 3 4 5 6 7 8 9 10])
#'user/test-vec

conj adds an element to the collection:

user=> (conj test-vec 11)
> [1 2 3 4 5 6 7 8 9 10 11]

map applies a given function to all elements of a collection, returning a new collection with the result:

(user=> (map #(* % 10) test-vec)      
> (10 20 30 40 50 60 70 80 90 100)

The construct #(* % 10) is a reader macro for anonymous functions, the % is short for %1 and is bound to the first (and here, only) argument. We could also have written this using the fn form:

user=> (map (fn [a] (* a 10)) test-vec)
> (10 20 30 40 50 60 70 80 90 100)

filter will keep all elements for which the given function evaluates to true:
user=> (filter even? test-vec)
(2 4 6 8 10)

Again, all these functions return a new collection, our initial test-vec is untouched.

See the complete Clojure APIs for all core functions. In addition to Clojure core there is also the core clojure-contrib, a user maintained library that also acts as an incubator for Clojure core.

Java integration
Clojure runs on the JVM and is designed to make it easy to call Java code from Clojure, and Clojure code from Java. This is a big benefit as we can always drop down to Java if there is something we need that is missing from the Clojure libraries.

Calling Java is done using the . (dot) special form:

Accessing a static field:

user=> (. Math PI)
> 3.141592653589793

Calling a static method:

user=> (. Math min 3 4)
> 3

Creating a new instance is done using new:

user=> (def l (new java.util.ArrayList))
> #'user/l

Accessing an instance method, again using the . form:

user=> (. l size)
> 0

user=> (. l add "Cheese")
> true

user=> (. l get 0)
> "Cheese"

Note here that as we access the ArrayList instance it behaves as expected in Java. Clojure's immutability is not extended to Java objects, they keep their original behavior.

This is the basics of Java interoperability, but there is a lot of syntactic sugar available to make Java easier to work with from Clojure:

Direct static member access can be written with /:

user=> Math/PI
> 3.141592653589793

If it's a static method, make it a call:

user=> (Math/min 3 4)
> 3

Object creation can be done with Classname.:

user=> (def l (java.util.ArrayList.))
> #'user/l

If it feels annoying having to fully qualify ArrayList, we can import it (Math, as used above, is in java.lang and is imported by default):

user=> (import '(java.util ArrayList))
> java.util.ArrayList

user=> (def l (ArrayList.))
> #'user/l

The ' (quote) used above is another reader macro preventing the list to be evaluated as a function call.

Instance calls can be shoretened by placing the thing we want to call, the method, first, prefix Clojure style:

user=> (.add l "Cheese")
> true

Chaining Java calls in Clojure looks a bit messy by default:

user=> (. (. (. l get 0)  (subSequence 1 3)) length)
> 2

Using the .. notation we can do this:

user=> (.. l (get 0) (subSequence 1 3) length)
> 2

The .. is, again, a Clojure macro, that expands to our first version above.

More on Java interoperability.

Tooling
REPL is all good for trying out things at the promt, but for writing larger Clojure programs you'd typicall want to use something else. Comming from a Java background all three major IDE:s have some kind of Clojure support at this point:

IntelliJ/IDEA: La Clojure plug-in
NetBeans: enclojure plug-in
Eclipse: clojure-dev

Hopefully this will get you started exploring Clojure. Have fun!