I had a lot of trouble connecting to an MS SQL Server at work via Clojure. Java 6 comes with a JDBC-ODBC bridge which worked fine from a Clojure REPL at a command prompt, or from inferior-lisp in Emacs, but in SLIME it would hang every time I tried to connect and I'd have to kill Java. Couldn't for the life of me figure out why.

I got it to work eventually by using Microsoft's own JDBC driver, which you can download here.

Once you put the downloaded .jar file on your CLASSPATH (in my case, sqljdbc4.jar) you can connect like this:

user> (def db {:classname "com.microsoft.sqlserver.jdbc.SQLServerDriver"
               :subprotocol "sqlserver"
               :subname "//server_hostname;database=SomeDatabase;user=SomeUser;password=SomePassword"})
#'user/db
user> (use 'clojure.contrib.sql)
nil
user> (with-connection db 
        (with-query-results rs ["SELECT * FROM whatever"] (prn rs)))
... results ...

Posted for the sake of Googlebot and for my own future sanity.

This infamous blog post suggests that someone familiar with a language should be able to name five things they hate about it. "Hate" is a strong word, but I decided to think of five things I find mildyly annoying about Clojure, my favorite language of the moment.

Hashing integers

Clojure automatically converts integers between Integer, Long and BigInteger as needed to prevent overflow. This is good. Integers of the various classes test as equal too. This is also good.

user> (= 123 (int 123) (long 123) (bigint 123))
true

So would you expect this?

user> (hash-map (int 123) :foo (long 123) :bar (bigint 123) :baz)
{123 :foo, 123 :bar, 123 :baz}

Yes, each of the integer classes, though equal via =, do not have the same hash value when put into a hash-map. This is because:

user> (.equals (int 123) (long 123))
false

This is a wart inherited from the JVM. See here for discussion and explanation.

What's more, if you print this map and then try to read it back in, the integers will be read as int, long or bigint arbitrarily depending how big they are. This means you may not get the same class of object back that you output originally.

user> (def x {(bigint 123) :foo})
#'user/x
user> (= x x)
true
user> (def y (read-string (pr-str x)))
#'user/y
user> (= x y)
false
user> (class (first (keys y)))
java.lang.Integer
user> (class (first (keys x)))
java.math.BigInteger

This means that if you ever use integers as hash keys, you must be very careful to cast them all to the same integer type manually.

Metadata doesn't work on everything

Clojure lets you stick arbitrary metadata on various objects. This is higly useful; you can decorate objects with information that doesn't affect the value of the object. However metadata doesn't work everywhere.

user> (with-meta "foo" {:bar :baz})
java.lang.ClassCastException: java.lang.String cannot be cast to clojure.lang.IObj (NO_SOURCE_FILE:0)

You can only stick metadata on certain Clojure objects like Symbols, Vars, Refs, Agents, all of the Clojure collections and so on. You can't stick metadata on, say, a String or an Integer, because those are closed Java classes and can't be touched. It would be nice if you could.

use vs. require vs. import vs. load vs. ...

There are a startling number of ways to import a library into your code in Clojure. You have to choose from load, import, require, use, refer, and so on. Some work on Java classes, some work on Clojure libs. Some of them import symbols into your namespace, some of them don't. Some of them take strings as arguments, some take symbols, some take quoted lists of symbols, some take quoted lists of symbols with sub-lists of arguments. And all of these can be and usually are weirdly inlined into a namespace declaration, with a completely different list-quoting style.

So in Ruby you can do this:

require 'util'
require 'config'
require 'whatever'

Whether it's a gem, or a Ruby source file sitting locally, it all works the same as long as the load path is set up right.

But in Clojure, you do this (actual code from an IMAP library I wrote):

(ns qt4-mailtray.mail
  (:import (java.util Properties)
           (javax.mail Session Store Folder Message Flags Flags$Flag FetchProfile FetchProfile$Item)
           (javax.mail.internet InternetAddress))
  (:use clojure.contrib.str-utils))

This can quickly become unwieldy, especially if you start using the :as or :only or :rename arguments. It's made worse by Java's insane API's full of a billion classes that you need to import to do simple things. (And those things with dollar signs are mangled Java inner class names.) Clojure also lacks the ability to import a whole package worth of classes at once using java.io.* syntax, so you must name all of the classes explicitly.

every? vs some.

This is such a trite pet-peeve that it's barely worth mentioning, but it seems to be brought up repeatedly and endlessly on the Clojure mailing list so at least I'm not the only one bugged by it.

Clojure has a function (every? pred coll) which tests whether every item in a collection tests true via some predicate. To test whether every item in a collection tests false, we have not-any?. And we have a not-every? which tests whether any item tests false.

user> (every? even? [2 4 6])
true
user> (not-every? even? [2 4 6])
false
user> (not-any? even? [2 4 6])
false

Now what would you expect a function to be called which tests whether any item in a collection tests true via some predicate? If you said any? you are wrong! It's some.

Note that some isn't a predicate (hence no question mark in the name); it doesn't return true or false, as above, but rather returns the result of running pred on an item in coll.

user> (some identity [nil 1 2 3])
1

any? is pretty easy to write so it doesn't matter that much. Probably many people have an identical function sitting in some utils.clj file on their systems.

(defn any? [pred coll]
  (when (seq coll)
    (if (pred (first coll))
      true
      (recur pred (next coll)))))

Stack trace madness

Give this function:

(defn foo []
  (throw (Exception. "BARFED")))

What does the stack trace look like in SLIME when you call foo? Like this:

java.lang.Exception: BARFED (NO_SOURCE_FILE:0)
  [Thrown class clojure.lang.Compiler$CompilerException]

Restarts:
 0: [ABORT] Return to SLIME's top level.
 1: [CAUSE] Throw cause of this exception

Backtrace:
  2: swank.commands.basic$eval_region__729.invoke(basic.clj:36)
  3: swank.commands.basic$listener_eval__738.invoke(basic.clj:50)
  4: clojure.lang.Var.invoke(Var.java:346)
  5: user$eval__1506.invoke(NO_SOURCE_FILE)
  6: clojure.lang.Compiler.eval(Compiler.java:4580)
  7: clojure.core$eval__4016.invoke(core.clj:1728)
  8: swank.core$eval_in_emacs_package__336.invoke(core.clj:55)
  9: swank.core$eval_for_emacs__413.invoke(core.clj:123)
 10: clojure.lang.Var.invoke(Var.java:354)
 11: clojure.lang.AFn.applyToHelper(AFn.java:179)
 12: clojure.lang.Var.applyTo(Var.java:463)
 13: clojure.core$apply__3269.doInvoke(core.clj:390)
 14: clojure.lang.RestFn.invoke(RestFn.java:428)
 15: swank.core$eval_from_control__339.invoke(core.clj:62)
 16: swank.core$eval_loop__342.invoke(core.clj:67)
 17: swank.core$spawn_repl_thread__474$fn__505$fn__507.invoke(core.clj:173)
 18: clojure.lang.AFn.applyToHelper(AFn.java:171)
 19: clojure.lang.AFn.applyTo(AFn.java:164)
 20: clojure.core$apply__3269.doInvoke(core.clj:390)
 21: clojure.lang.RestFn.invoke(RestFn.java:428)
 22: swank.core$spawn_repl_thread__474$fn__505.doInvoke(core.clj:170)
 23: clojure.lang.RestFn.invoke(RestFn.java:402)
 24: clojure.lang.AFn.run(AFn.java:37)
 25: java.lang.Thread.run(Thread.java:619)

Yeouch. Now imagine that the above error is coming not from a simple function, but from some random line among hundreds of lines of source code.

Stack traces in Clojure will often tell you little to nothing about what is causing the error, or more importantly, where it's coming from in your code. Clojure functions are translated into Java classes when they're run through the JVM. Often can't even see the name of the function that's throwing the error; names are mangled into things like user$eval__1473.invoke, which is really really confusing when you use anonymous functions.

Per Jason Wolfe and Randall Schulz sometimes you can get a better stack trace if you dig a bit deeper:

user> (.printStackTrace (.getCause *e))

java.lang.Exception: BARFED
    at user$foo__1503.invoke(NO_SOURCE_FILE:1)
    at user$eval__1509.invoke(NO_SOURCE_FILE:1)
    at clojure.lang.Compiler.eval(Compiler.java:4580)
    at clojure.core$eval__4016.invoke(core.clj:1728)
    at swank.commands.basic$eval_region__729.invoke(basic.clj:36)
    at swank.commands.basic$listener_eval__738.invoke(basic.clj:50)
    at clojure.lang.Var.invoke(Var.java:346)
    at user$eval__1506.invoke(NO_SOURCE_FILE)
    at clojure.lang.Compiler.eval(Compiler.java:4580)
    at clojure.core$eval__4016.invoke(core.clj:1728)
    at swank.core$eval_in_emacs_package__336.invoke(core.clj:55)
    at swank.core$eval_for_emacs__413.invoke(core.clj:123)
    at clojure.lang.Var.invoke(Var.java:354)
    at clojure.lang.AFn.applyToHelper(AFn.java:179)
    at clojure.lang.Var.applyTo(Var.java:463)
    at clojure.core$apply__3269.doInvoke(core.clj:390)
    at clojure.lang.RestFn.invoke(RestFn.java:428)
    at swank.core$eval_from_control__339.invoke(core.clj:62)
    at swank.core$eval_loop__342.invoke(core.clj:67)
    at swank.core$spawn_repl_thread__474$fn__505$fn__507.invoke(core.clj:173)
    at clojure.lang.AFn.applyToHelper(AFn.java:171)
    at clojure.lang.AFn.applyTo(AFn.java:164)
    at clojure.core$apply__3269.doInvoke(core.clj:390)
    at clojure.lang.RestFn.invoke(RestFn.java:428)
    at swank.core$spawn_repl_thread__474$fn__505.doInvoke(core.clj:170)
    at clojure.lang.RestFn.invoke(RestFn.java:402)
    at clojure.lang.AFn.run(AFn.java:37)
    at java.lang.Thread.run(Thread.java:619)

This one at least mentions foo by name but you're still going to have a headache after a few hours of those stack traces.

Conclusion

So that's five things. You will notice a common theme. Most of these issues are inherited from the JVM. This is to be expected, I suppose. There's no way you can wrap one language in another without a few compromises.

But these things aren't show-stoppers. They are minor annoyances compared to the benefits you get from using the JVM, i.e. the good performance, tons of libraries, cross-platformness, and so on. Clojure is fun enough to work with and wart-less enough that it took me well over two weeks to write this post.

(If you were expecting me to mention loop/recur and the lack of native TCO in the JVM, you were PAINFULLY WRONG. No one who uses Clojure loses sleep over native TCO. It's largely a non-issue that's endlessly repeated by people looking for an excuse to pass up Clojure in favor of $their_pet_language. To each his own, but I have never found myself caring the slightest about loop/recur.)

I learned recently of ABLE, which is a minimalistic GUI editor + REPL for Common Lisp. You download it and run it and there you go, Common Lisp with lots of included libraries, no Emacs trial-by-fire necessary.

This is a really great idea in my opinion. I wish I'd had this a year or two ago. Common Lisp has too many barriers for newbies to get started. Anything that removes a few of them can only benefit the Lisp world.

After my brief stint in the world of fractal geometry and Clojure, I decided to make a real Mandelbrot set viewer. The resulting source code is here. Here's a simple output (click for bigger version):

It's a pretty naive implementation, barely 100 lines of code, but even with my brute-force approach, given a liberal sprinkling of type hints it runs fast enough. Programming Swing from Clojure couldn't be easier (though I doubt programming Swing from any language is ever really enjoyable, it's a painful bunch of libraries).

There's a discussion of different coloring algorithms on Wikipedia, but even after reading that, getting this thing to look good was difficult. I don't know enough math for it. I ended up cheating and I colored a couple of them in the GIMP, so I could use them as desktop wallpapers.

There are some more PNGS over here including one that's 16000x16000 (producing it almost melted my CPU last night).

My blog is still working, in spite of my best efforts to crash it. So that's good. But lately I've been thinking that an SQL database is a lot of overkill just to run a little blog like this.

My blog only has around 450 posts total (over the course of many years), and about an equal number of user comments (thanks to all commenters!). Why do I need a full-blown database for that? All of my posts plus comments plus all meta-data is only 2 MB as a flat text file, 700k gzipped.

By far the most complicated part of my blog engine is the part that stuffs data into the database and gets it back out again in a sane manner (translating Clojure data to SQL values, and back again; splitting up my Clojure data structures into rows for different tables, and then re-combining values joined from multiple tables into one data structure). Eliminating that mess would be nice.

Inevitably I ended up with some logic in the database too: enforcing uniqueness of primary keys, marking some fields as NOT NULL, giving default values and so on. But a lot of other logic was in my Clojure code, e.g. higher-level semantic checking, and some things I wanted to set as default values were impossible to implement in SQL.

Wouldn't it be nice for all the logic to be in Clojure? And the data store on disk to be a simple dump of a Clojure data structure? I can (and did) write a few macros to give me SQL-like field declaration and data validation, for uniqueness of IDs and data types etc. For my limited needs it works OK.

The next question is what format to use for dumping to disk. Happily Clojure is Lisp, so dumping it as a huge s-exp via pr-str works fine, and reading it back in later via read-string is trivial.

Some Java data types can't be printed readably by default, for example java.util.Dates, which print like this:

#<Date Wed May 20 22:39:00 PDT 2009>

The #<> reader macro deliberately throws an error if you try to read that back in, because the reader isn't smart enough to craft Date objects from strings by default. But Clojure is extensible; you can specify a readable-print method for any data type like this:

(defmethod clojure.core/print-method java.util.Date [o w]
  (.write w (str "#=" `(java.util.Date. ~(.getTime o)))))

Now dates print as

#=(java.util.Date. 1242884415044)

and if you try to read that via read-string, it'll create a Date object like you'd expect.

user> (def x (read-string "#=(java.util.Date. 1242884415044)"))
#'user/x
user> (class x)
java.util.Date
user> (str x)
"Wed May 20 22:40:15 PDT 2009"

Storing data in a plain file has another benefit of letting me grep my data from a command line, or even edit the data in a text editor and re-load it into the blog (God help me if that's ever necessary).

Having multiple threads banging on a single file on disk is a horrible idea, but Clojure refs and agents and transactions handle that easily. But I do have to work out how not to lose all my data in case the server crashes in the middle of a file update. (I've lost data (in a recoverable way) due to a server crash in the middle of a MySQL update too, so this is a problem for everyone.) Perhaps I'll keep a running history of my data, each update being a new timestamped file, so old files can't possibly be corrupted. Or use the old write-to-tmp-file-and-rename-to-real-file routine. Or heck, I could keep my data in Git and use Git commands from Clojure. It'd be nice to have a history of edits.

If this idea works out I'll upload code for everything to github, as usual.

Did you know there's this neat Lisp message board where from time to time someone posts a short problem similar in spirit to the infamous RubyQuiz?

Not a lot of people have participated so far, hopefully that changes. I participated this time; the problem is to render the Mandelbrot Set in ASCII. Here's my Clojure version (based loosely on this one).

(ns mandelbrot
  (:refer-clojure :exclude [+ * <])
  (:use (clojure.contrib complex-numbers)
        (clojure.contrib.generic [arithmetic :only [+ *]]
                                 [comparison :only [<]]
                                 [math-functions :only [abs]])))

(defn- mandelbrot-seq [x y]
  (let [z (complex x y)]
    (iterate #(+ z (* % %)) z)))

(defn- mandelbrot-char [x y]
  (loop [c 126
         m (mandelbrot-seq x y)]
    (if (and (< (abs (first m)) 2)
             (> c 32))
      (recur (dec c) (rest m))
      (char c))))

(defn- mandelbrot-line [xs y]
  (apply str (map #(mandelbrot-char % y) xs)))

(defn- m-range [min max num-steps]
  (range min
         max
         (/ (+ (abs min)
               (abs max))
            num-steps)))

(defn mandelbrot [rmin rmax imin imax]
  (let [rows 30
        cols 50
        xs (m-range rmin rmax cols)
        ys (m-range imin imax rows)]
    (dorun (map #(println (mandelbrot-line xs %)) ys))))

(comment
  ;Example run:
  (mandelbrot -2.0 1.0 -1.5 1.5)
"
~~~~~~~~~~~~}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}
~~~~~~~~~~~~}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}
~~~~~~~~~~}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}
~~~~~~~~~}}}}}}}}}}}}|||||||||}}}}}}}}}}}}}}}}}}}}
~~~~~~~}}}}}}}}|||||||||||||||||||||}}}}}}}}}}}}}}
~~~~~~}}}}}||||||||||||||{{{{zlxz{{{||||}}}}}}}}}}
~~~~~}}}}|||||||||||||{{{{{zzyxpvlz{{{||||}}}}}}}}
~~~~}}}|||||||||||||{{{{{{zzyxvnpwyzz{{{||||}}}}}}
~~~}}|||||||||||||{{{{{{zzyyws   .vyzzz{{|||||}}}}
~~~}||||||||||||{{{{{zzxwwwvus   muvxyywz{|||||}}}
~~}|||||||||||{{{zzzzyyu= p         oteqpz{|||||}}
~~||||||||||{zzzzzzyyyvtm              oxz{{|||||}
~}|||||{{{zyvwxxxxxxxwrG                vuz{|||||}
~||{{{{{zzzywsMsqRovvs                  pxz{{|||||
~|{{{{{zzzyxsq      pj                  `xz{{|||||
~{{{{yyyxwsrp                           wyz{{|||||
~?:3 3 #                              ovxzz{{|||||
~{{{{yyyxwsrp                           wyz{{|||||
~|{{{{{zzzyxsq      pj                  `xz{{|||||
~||{{{{{zzzywsMsqRovvs                  pxz{{|||||
~}|||||{{{zyvwxxxxxxxwrG                vuz{|||||}
~~||||||||||{zzzzzzyyyvtm              oxz{{|||||}
~~}|||||||||||{{{zzzzyyu= p         oteqpz{|||||}}
~~~}||||||||||||{{{{{zzxwwwvus   muvxyywz{|||||}}}
~~~}}|||||||||||||{{{{{{zzyyws   .vyzzz{{|||||}}}}
~~~~}}}|||||||||||||{{{{{{zzyxvnpwyzz{{{||||}}}}}}
~~~~~}}}}|||||||||||||{{{{{zzyxpvlz{{{||||}}}}}}}}
~~~~~~}}}}}||||||||||||||{{{{zlxz{{{||||}}}}}}}}}}
~~~~~~~}}}}}}}}|||||||||||||||||||||}}}}}}}}}}}}}}
~~~~~~~~~}}}}}}}}}}}}|||||||||}}}}}}}}}}}}}}}}}}}}
~~~~~~~~~~}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}
"
)

And here's some obligatory Jonathan Coulton.

Finally I found a good Git tutorial that starts from the absolute basics and goes steadily through more advanced things. I highly recommend it.