I enjoyed Peter Bourgon's How I Start: Go article. Writing Go seemed easy and fun. It made me wonder: how fun would it be in OCaml? There's only one way to find out, read on! (This post assumes a basic knowledge of OCaml.)
Note: I've copied a lot from the original post, ideas and words. Any mistake are my own.
OCaml belongs to the ML language family (whose roots go back to the 70's), it is used in compilers, static analysis tools, and proof systems. Ocaml is a statically typed, functional language with some imperative features, it has a powerful module system, and an object system.
OCaml is not very trendy. Yet recently, a great introductory book came out, and a new website launched. It also make the news, sometimes (Coq, Hack and Flow).
Like Peter, we will build a backend service for a web app.
The first step is, of course, to install OCaml and OPAM (a package manager for OCaml).
Here's how to go about it, on a recent Ubuntu:
$ sudo apt-get install software-properties-common
$ sudo apt-get install m4
$ sudo apt-get install make
$ sudo add-apt-repository ppa:avsm/ppa
$ sudo apt-get update
$ sudo apt-get install ocaml ocaml-native-compilers camlp4-extra opam
$ opam init
Once you're done, this should work:
$ ocaml -version
The OCaml toplevel, version 4.02.1
There are a few options for editing OCaml code (Emacs, Vim, SublimeText, and more). Personally, I use Emacs with tuareg and merlin
OCaml includes a bare bones top level, but you can use utop, for a more capable one.
Create a new file, hello.ml, this will be our simplest OCaml program.
let _ = print_endline "hello!"
Compile and run:
$ ocamlbuild hello.native
$ ./hello.native
hello!
Easy! All code
Let's turn our "hello" program into a web server.
Because the HTTP library I'm using (CoHTTP) is asynchronous, we'll go straight into the asynchronous web server. With CoHTTP, we'll be using Lwt, a cooperative threads library.
to install these two, do :
$ apt-get install libssl-dev
$ opam install lwt.2.4.6 cohttp.0.12.0
This should pull in all required dependencies
Here's the full program for a simple 'hello' server.
open Lwt
open Cohttp
open Cohttp_lwt_unix
let make_server () =
let callback conn_id req body =
let uri = Request.uri req in
match Uri.path uri with
| "/" -> Server.respond_string ~status:`OK ~body:"hello!\n" ()
| _ -> Server.respond_string ~status:`Not_found ~body:"Route not found" ()
in
let conn_closed conn_id () = () in
Server.create { Server.callback; Server.conn_closed }
let _ =
Lwt_unix.run (make_server ())
First, we open 3 modules, so we can use unqualified identifiers (for instance, Request.uri instead of Cohttp.Request.uri):
In make_server(), we create a server configured by 2 handler functions: callback handles the request, conn_closed handles the end of the connection (here, we chose to do nothing).
callback takes a connection id, an HTTP request and a body. It dispatches on the request uri: "/" gets a hello response, anything else gets a 404.
the last expression is the program main entry, make_server returns an Lwt thread.
Lwt_unix.run runs this thread until it terminates.
To compile this program, we first create the following _tags file:
true: package(lwt), package(cohttp), package(cohttp.lwt)
It lists all the packages we're using. Now ocamlbuild can find our dependencies, we can compile and run:
$ ocamlbuild -use-ocamlfind hello_server.native
$ ./hello_server.native
To interact with the server, in another terminal or your browser, make an HTTP request:
$ curl http://localhost:8080
hello!
That wasn't too bad! All code
We can do something more interesting than just say hello.
Let's take a city as input, call out to a weather API, and forward a response with the temperature. The OpenWeatherMap provides a simple and free API for current forecast info, which we can query by city. It returns responses like this (partially redacted):
{
"name": "Tokyo",
"coord": {
"lon": 139.69,
"lat": 35.69
},
"weather": [
{
"id": 803,
"main": "Clouds",
"description": "broken clouds",
"icon": "04n"
}
],
"main": {
"temp": 296.69,
"pressure": 1014,
"humidity": 83,
"temp_min": 295.37,
"temp_max": 298.15
}
}
To stay close to the original Go code, we'll use atdgen, a tool that generates OCaml code to (de)serialize JSON (an alternative would be to use a JSON library like yojson.
First, let's install atdgen:
$ opam install atdgen
Next, in openweathermap.atd, we'll define the following types to specify what we want to deserialize from the JSON response:
type main = { temp: float }
type weather = { main: main; name: string }
We can now use atdgen to generate the OCaml code:
atdgen -t openweathermap.atd
atdgen -j openweathermap.atd
This creates 2 OCaml modules: Openweathermap_t and Openweathermap_j.
Which we can use like this:
let weather = Openweathermap_j.weather_of_string response in ...
Now, in a new file weather.ml, let's write a function to query the API, and return a weather record:
let query city =
let open Openweathermap_j in
Client.get (Uri.of_string
("http://api.openweathermap.org/data/2.5/weather?q=" ^ city))
>>= fun (_, body) -> Cohttp_lwt_body.to_string body
>>= fun s -> return (string_of_weather (weather_of_string s))
Writing asynchronous code with LWT is done by combining operations with the >>= operator.
In f >>= fun x -> g x, g is a "callback" which runs after f has completed, with the result of f (here bound to x) as input".
For example, when in OCaml we write:
(Client.get uri) >>= fun (_, body) -> to_string body
in Javascript, we would write:
Client.get(uri, function(_, body){ return to_string(body); })
See this LWT tutorial for more.
Here's the complete program:
open Lwt
open Cohttp
open Cohttp_lwt_unix
let query city =
let open Openweathermap_j in
Client.get (Uri.of_string
("http://api.openweathermap.org/data/2.5/weather?q=" ^ city))
>>= fun (_, body) -> Cohttp_lwt_body.to_string body
>>= fun s -> return (string_of_weather (weather_of_string s))
let make_server () =
let callback conn_id req body =
let uri = Request.uri req in
match Re_str.(split_delim (regexp_string "/") (Uri.path uri)) with
| ""::"weather"::city::_ -> query city >>= fun json ->
let headers =
Header.init_with "content-type" "application/json; charset=utf-8" in
Server.respond_string ~headers ~status:`OK ~body:json ()
| _ ->
Server.respond_string ~status:`Not_found ~body:"Route not found" ()
in
let conn_closed conn_id () = () in
Server.create { Server.callback; Server.conn_closed }
let _ =
Lwt_unix.run (make_server ())
In our server handler, we match uri of the form "/weather/city" to an API call, and returns the temperature as JSON (Note that we specify a JSON specific Content-Type header).
Now we'll update the _tags file with our new dependencies:
true: package(lwt), package(cohttp), package(cohttp.lwt), package(atdgen), package(yojson), package(re.str)
Build and run, as before.
$ ocamlbuild -use-ocamlfind weather.native
$ ./weather.native
$ curl http://localhost:8080/weather/tokyo
{"main":{"temp":285.92},"name":"Tokyo"}
Maybe we can build a more accurate temperature for a city, by querying and averaging multiple weather APIs. Unfortunately for us, most weather APIs require authentication. So, get yourself an API key for Weather Underground.
All of our weather providers will expose a function to query an API and return a temperature. In OCaml, a weather provider could be a simple function, an object or a module.
To follow to the Go code, we'll use objects first.
In an new weather.ml file, we write the code for OpenWeatherMap:
(** OpenWeatherMap Provider *)
let open_weather_map = object
method temperature city =
let open Openweathermap_j in
Client.get (Uri.of_string
("http://api.openweathermap.org/data/2.5/weather?q=" ^ city))
>>= fun (_, body) -> Cohttp_lwt_body.to_string body
>>= fun s -> return (string_of_weather (weather_of_string s))
end
This defines an object that queries the OpenWeatherMap API (we've renamed our query function, temperature).
OCaml automatically infers the type of this object as :
val open_weather_map : < temperature : string -> float Lwt.t > = obj
Which in English, reads: "an object with a temperature method, which takes a string and returns a float asynchronously" (Lwt.t is the type of an LWT thread).
Similarly to Go, OCaml's objects are typed by the names and types of their methods.
Now, let's turn to the Weather Underground API.
Here's how the response looks (partially redacted):
{
"response": {
"version": "0.1",
"termsofService": "http://www.wunderground.com/weather/api/d/terms.html",
"features": {
"conditions": 1
}
},
"current_observation": {
...
"estimated": {},
"station_id": "KCASANFR58",
"observation_time": "Last Updated on June 27, 5:27 PM PDT",
"observation_time_rfc822": "Wed, 27 Jun 2012 17:27:13 -0700",
"observation_epoch": "1340843233",
"local_time_rfc822": "Wed, 27 Jun 2012 17:27:14 -0700",
"local_epoch": "1340843234",
"local_tz_short": "PDT",
"local_tz_long": "America/Los_Angeles",
"local_tz_offset": "-0700",
"weather": "Partly Cloudy",
"temperature_string": "66.3 F (19.1 C)",
"temp_f": 66.3,
"temp_c": 19.1,
...
}
}
First we define a new atdgen specification in weatherunderground.atd, just to get at temp_c:
type current_observation = { temp_c: float }
type conditions = { current_observation: current_observation }
and like before, we generate the OCaml code:
$ atdgen -t weatherunderground.atd
$ atdgen -j weatherunderground.atd
We need to provide a key to use this API (the key is used in the URI)
(Note that the Weather Underground doesn't disambiguate cities quite as nicely as OpenWeatherMap. We're skipping some important logic to handle ambiguous city names for the purposes of the example.)
(** WeatherUnderground Provider *)
let weather_underground key = object
method temperature ~city =
let open Weatherunderground_j in
let kelvin_of_celsius t = t +. 273.15 in
let uri = "http://api.wunderground.com/api/" ^ key ^
"/conditions/q/" ^ city ^ ".json" in
Client.get (Uri.of_string uri)
>>= fun (_, body) -> Cohttp_lwt_body.to_string body
>>= fun s -> let c = conditions_of_string s in
let temp = kelvin_of_celsius c.current_observation.temp_c in
Lwt_io.printf "%s: %s: %.2f\n" "WeatherUnderground" city temp >>=
fun _ -> return temp
end
Now that we have a couple of weather providers, let's write a function to query them all, and return the average temperature.
First we'll install core, to use its Time module (Core is a modernized OCaml standard library from Jane Street). Be aware that installing it takes a few minutes...
$ opam install core
Next, in response.atd, we'll define a type to describe the JSON response.
type response = { city: string; temp: float; took: string; }
Again, we generate the boilerplate OCaml code:
$ atdgen -t response.atd
$ atdgen -j response.atd
Now let's write a function multi_providers which create an object from a list of providers.
let multi_providers ps = object
method temperature ~city =
let average xs =
let sum = List.fold_left (+.) 0. xs in
(sum /. float_of_int (List.length xs))
in
let open Response_j in
let open Core.Std in
let t0 = Time.now () in
Lwt_list.map_p (fun p -> p#temperature ~city) ps >>=
fun temps ->
let t1 = Time.now () in
let response = { city = city; temp = average temps;
took = Core.Span.to_string (Time.diff t1 t0); }
in return (string_of_response response)
end
We use Lwt_list.map_p to fire off queries in parallel. Once the longest queriy is finished, we return the average temperature and the time it took.
Now, we can wire that up to our HTTP server. Note that we pass our multi_providers temperature method to make_server, so we can use it to handle requests.
(** web server *)
let make_server temperature =
let callback conn_id req body =
let uri = Request.uri req in
match Re_str.(split_delim (regexp_string "/") (Uri.path uri)) with
| ""::"weather"::city::_ -> temperature ~city >>= fun json ->
let headers =
Header.init_with "content-type" "application/json; charset=utf-8" in
Server.respond_string ~headers ~status:`OK ~body:json ()
| _ ->
Server.respond_string ~status:`Not_found ~body:"Route not found" ()
in
let conn_closed conn_id () = () in
Server.create { Server.callback; Server.conn_closed }
let _ =
let ps = multi_providers [
open_weather_map;
weather_underground "..." (* your API key*) ] in
Lwt_unix.run (make_server ps#temperature)
Now we'll update our _tags file:
true: package(core), package(lwt), package(cohttp), package(cohttp.lwt), package(atdgen), package(yojson), package(re.str), thread
Finally we compile, run, and GET, just as before.
$ ocamlbuild -use-ocamlfind weather.native
$ ./weather.native
openWeatherMap: tokyo: 287.11
WeatherUnderground: tokyo: 288.15
In addition to the JSON response, you'll see the following output.
$ curl http://localhost/weather/tokyo
{"city":"tokyo","temp":287.63,"took":"611.787ms"}
Another way to implement our weather providers is via OCaml's modules.
Let's create a new file api.ml to write our API code.
Each provider module will need to implement the following signature (module interface):
module type WeatherProvider =
sig
val temperature: string -> float Lwt.t
end
Let's write the OpenWeatherMap module:
module OpenWeatherMap : WeatherProvider =
struct
let name = "OpenWeatherMap"
let uri city =
"http://api.openweathermap.org/data/2.5/weather?q=" ^ city
let temperature city =
Client.get (Uri.of_string (uri city))
>>= fun (_, body) -> Cohttp_lwt_body.to_string body
>>= fun s ->
let open Openweathermap_j in
let w = weather_of_string s in
let temp = w.main.temp in
Lwt_io.printf "%s: %.2f\n" w.name temp
>>= fun _ -> return temp
end
and the WeatherUnderground module:
module WeatherUnderground : WeatherProvider =
struct
let kelvin_of_celsius t = t +. 273.15
let key = "..." (* your API key *)
let name = "WeatherUnderground"
let uri city = "http://api.wunderground.com/api/" ^
key ^ "/conditions/q/" ^ city ^ ".json"
let temperature city =
Client.get (Uri.of_string (uri city))
>>= fun (_, body) -> Cohttp_lwt_body.to_string body
>>= fun s ->
let open Weatherunderground_j in
let c = conditions_of_string s in
let temp = kelvin_of_celsius c.current_observation.temp_c in
let name = c.current_observation.display_location.full in
Lwt_io.printf "%s: %.2f\n" name temp
>>= fun _ -> return temp
end
To implement multiple providers, we'll use OCaml's functors.
MultipleWeather is a functor parameterized by 2 providers M1 and M2.
Like before, our temperature function averages the temperatures returned by the other providers:
module MultipleWeather (M1 : WeatherProvider)
(M2 : WeatherProvider) : WeatherProvider =
struct
let average xs =
let sum = List.fold_left (+.) 0. xs in
(sum /. float_of_int (List.length xs))
let temperature city =
Lwt_list.map_p (fun gt -> gt city) [M1.temperature; M2.temperature] >>=
fun temps -> return (average temps)
end
Finally, we apply our MultipleWeather functor to OpenWeatherMap and WeatherUnderground, to create MW:
module MW = MultipleWeather (OpenWeatherMap) (WeatherUnderground)
In weather.ml, we'll write our web server:
open Lwt
open Cohttp
open Cohttp_lwt_unix
open Api
let make_server temperature =
let callback conn_id req body =
let uri = Request.uri req in
match Re_str.(split_delim (regexp_string "/") (Uri.path uri)) with
| ""::"weather"::city::_ ->
let open Response_j in
let open Core.Std in
let t0 = Time.now () in
temperature city >>= fun temp ->
let t1 = Time.now () in
let response = { city = city; temp = temp;
took = Core.Span.to_string (Time.diff t1 t0); } in
let json = string_of_response response in
let headers =
Header.init_with "content-type" "application/json; charset=utf-8" in
Server.respond_string ~headers ~status:`OK ~body:json ()
| _ ->
Server.respond_string ~status:`Not_found ~body:"Route not found" ()
in
let conn_closed conn_id () = () in
Server.create { Server.callback; Server.conn_closed }
let _ =
Lwt_unix.run (make_server MW.temperature)
Here we pass our averaging temperature function, MW.temperature, to make_server
So in the end, was it as easy as Go ?
Well, I have to admit that OCaml is much less "batteries included" than Go: we had to install libraries for the http server, the asynchronous library, JSON serialization, and even time operations.
Also building the program is more involved (atdgen, the _tags file).
Writing concurrent code with LWT takes some time to get used to, but I am not familiar enough with Go's concurrency to judge which one is easier.
So probably not as easy overall...
I did have fun though :)
Fork the final code on github.
Can you add some error handling? For instance, can you prevent the failure of one weather provider from aborting the whole computation ? (Hint: use Lwt.catch and change the type of temperature to : string -> float option Lwt.t) .
Can you add another weather provider? (Hint: forecast.io is a good one).
Can you implement a timeout, to cancel a query that is taking to long? (Hint: see Cancelling).