Synching your enviroment around

Ubuntu 9.10 has just came out & iv rushed to upgrade, upgrading my 9.04 to 9.10 via the upgrade manager is a working option but usually id rather do a clean install & get a fresh system to poke around.
The main issue that clean install brings is restoring your old environment which includes installed programs, settings files (.bashrc, .vimrc etc..), non packaged applications (usually the latest & greatest versions of Clojure, Groovy etc..).
Another issue is the need to restore these settings on multiple machines while keeping it all consistent between them, there must a better way than working it all manually!

Well the solution that iv come with involve two tools, Dropbox & AutomateIt, Dropbox is no more than a fancy folder to the cloud synchronizer which enables me to access all my settings file from any machine, lsym-ing my .bashrc, .vimrc & any other file that id like to share across machines into the Dropbox folder is all that is required.

The second part involves AutomateIt, which is a configuration management framework written in Ruby, an easy way of re-creating symlinks from my Dropbox folder onto a new system is using the following AutomateIt script:

HOME = ENV['HOME']
DROP = "#{HOME}/Private/Dropbox"

def link(src,dst)
   ln_s("#{DROP}/#{src}","#{HOME}/#{dst}")
end

# Vim
link "vim/.vimrc",".vimrc"
link "vim/.vim",".vim"
link "vim/.vimclojure-2.1.2",".vimclojure"
# Bash
link "BashEnv/.bashrc",".bashrc"
link "BashEnv/.inputrc",".inputrc"
# Languages
link "prog-langs/.clojure",".clojure"
link "prog-langs/.groovy",".groovy"
link "prog-langs/.jruby",".jruby"

The nice thing about it is that it hides all the nitty gritty details that id have to figure out when using plain Ruby (or any other environment for that matter), AutomateIt wraps common tasks with a bash like DSL, unlike bash this DSL is portable across multiple unix systems & it hides many of the complexities that follow.
AutomateIt performs actions only when they are required, this solves annoying cases that rise when scripts run more than once on a system (e.g. appending text to files).

Another cool feature is package management, AutomateIt is capable of interacting with multiple packaging systems (yum, apt, gem etc..) in a transparent way, its easy to replicate installed packages onto multiple systems:

# bash
%w(terminator rlwrap).each{ |p| package_manager.install p}

# programming
%w(vim git-core).each{ |p| package_manager.install p}

# communication
%w(deluge openssh-server).each{ |p| package_manager.install p}

# misc
%w(gpodder gnome-do).each{ |p| package_manager.install p}

# installing dropbox
download_manager.download 'http://www.getdropbox.com/download?dl=packages/nautilus-dropbox_0.6.1_i386_ubuntu_9.10.deb' , :to => '/tmp/nautilus-dropbox_0.6.1_i386_ubuntu_9.10.deb'
package_manager.install ({'nautilus-dropbox' => '/tmp/nautilus-dropbox_0.6.1_i386_ubuntu_9.10.deb'} , :with => :dpkg)

Using these tools has made configuration nirvana a bit closer to me & hopefully to you :)

The easiest singleton around

Iv just finished watching stuart-clojure-presentation where the issue of singletons is mentioned (global & thread local ones), a note from the audience mentions delay as a solution, REPL-ing a bit shows how this works:

user=> (defn singleton-factory []
         (println "creating object")
         (+ 1 2))
#'user/singleton-factory
user=> (def singleton (delay (singleton-factory)))
#'user/singleton
user=> (defn usage []
         (force singleton))
#'user/usage
user=> (usage)
creating object
3
user=> (usage)
3

Stuart does mention contrib singleton which has also per-thread-singleton.

user=> (use 'clojure.contrib.singleton)
nil
user=> (def t-singleton (per-thread-singleton singleton-factory)) 
#'user/t-singleton
user=> (defn use-twice [] (+ 1 (t-singleton)) (+ 1 (t-singleton)))
#'user/use-twice
user=> (defn use-twice-no-singleton [] (+ 1 (singleton-factory)) (+ 1 (singleton-factory)))
#'user/use-twice-no-singleton
user=> (. (Thread. use-twice) start) ; each Clojure Fn implements Runnable
nil
user=> creating object ; REPL prints on the wrong line

user=> (. (Thread. use-twice-no-singleton) start)
user=> creating object ; same thing
creating object

user=> 

This should come useful when working with all those Java mutable objects.

Dont overflow yourself!

Clojure has many nice less highlighted features, one of these is overflow protection, this can be a true life saver:

user=> (unchecked-add 1 2); not safe but might be faster
3
user=> (unchecked-add (. Integer MAX_VALUE) 2); oops
-2147483647
user=> (+ (. Integer MAX_VALUE) 2) ; using safe arithmetic again
2147483649
user=> (+ (. Integer MAX_VALUE) 23)
2147483670
user=> (class (+ (. Integer MAX_VALUE) 23)); Clojure does the right thing for us
java.lang.Long

It does pack more than just concurrency up its sleeve.

Genericsfaction, getting some generics satisfaction

Java generics have a lot of short shortcoming, most of which are the result of the religious backward compatibility that Java is known for.
One of the most annoying "features" of generics is type erasure in which the compiler removes (almost) any trace of generic types meta data.

Iv been pondering about this issue & had a simple idea, what if we could add back in the meta data that the erasing process removed?
So, iv decided to create a proof of concept in order to show how this might work and named it genericsfaction.

Using genericsfaction requires an additional code manipulation stage in which the source code is scanned & gets its AST manipulated (meta data is introduced), each manipulated source file is dumped into a separate folder (named gen), all that is left is to compile the files under gen (instead of the original src).

At the moment there are only two enrichments implemented, the first is method input parameters annotation enrichment:

public void twoTypeParamsMet(@GenMeta(stringRep = "((Object ()) (Object ()))", classes = { Object.class, Object.class }) Map <Object, Object> map) {
   // method code
}

The second is variable initialization enrichment:

public static void main(String [] args) {
   List<Integer> list = (List <Integer>)MetaProxy.newInstance(new ArrayList<Integer>(),"(Integer ())",new Class []{Integer.class});
 }

The reason that annotations aren't used in this case is due to this limitation.

There is no release to play with yet, however the source is up on github (mostly Clojure), I am planing on making a release as soon as ill finish to implement a validation module that uses this meta data in runtime.

Newify your Object

Update a generous Anonymous has left a comment that the following special macro form does the same as the newify function that i present later on:

(def nested-object (-> GrandFather. Father. GrandSon.)) 

Feel free to skip this entry (unless your interested in an alternative implementation).

Instantiating large nested object graphs can be tedious in any language Clojure included:

(def nested-object 
  (new Grandson 
    (new Father 
      (new Grandfather)))) 

All those new methods call get old pretty quickly and are not DRY, to remedy this will take advantage on a cool property of Clojure Homoiconicity.
Homoiconicity basically means that the code is represented by a basic data structure of the language itself (lists in the case of Clojure), this means that manipulating the AST is easy as pie!
It would be nice if we could write something like the following:

(def nested-object 
  (def nested-object (newify '(Grandson (Father (Grandfather))))) 

The newify function takes a nested list of objects & adds new before each object, post that the function evaluates the resulting list:

(defn newify [e] (eval (newify-imp e)))
; required since used before defined 
(defn- newify-rest [e] )

(defn- newify-imp [e]
  (let [add-new (partial cons 'new) add-rest (partial cons (first e))]
    (if (not (empty? e))
      (-> (newify-rest e) add-rest add-new) e)))

(defn- newify-rest [e]
  (map #(if (list? %) (newify-impl %) %) (rest e)))

The newify-imp & newify-rest functions are mutually recursive, newify-imp adds the new symbol & the first list element to the result of the newify-rest function result, the newify-rest maps each list in the rest of the list to its newified value.

This implementation could be transformed into a recursive macro that will spare us from the list escaping:

  (def nested-object (newify '(Grandson (Father (Grandfather))))) 
  ; no escaping here
  (def nested-object (newify (Grandson (Father (Grandfather))))) 

Ill have to dig that option later on.

NILFS on Jaunty

The latest release of Ubuntu includes the long awaited Ext4 FS (works flawlessly on my system).
Ext4 is faster & more secure but still lacks the ability to manage FS snapshots (ZFS excels in that, but runs only in FUSE on linux).
An interesting alternative is NILFS:

NILFS is a log-structured file system supporting versioning of the entire file system and continuous snapshotting which allows users to even restore files mistakenly overwritten or destroyed just a few seconds ago.

NILFS maintains a repo for Hardy, the Jaunty repos contain only the user land tools which won't do us much good since we need the kernel module as well, this leaves us only with the option of installing it from source (still quite easy).

# a perquisite
$ sudo aptitude install uuid-dev  
# installing kernel module, result module resides in /lib/modules/2.6.28-11-generic/kernel/fs/nilfs2/nilfs2.ko
$ wget http://www.nilfs.org/download/nilfs-2.0.12.tar.bz2
$ tar jxf nilfs-2.0.12.tar.bz2
$ cd nilfs-2.0.12
$ make
$ sudo make install
# installing user land tools
$ wget http://www.nilfs.org/download/nilfs-utils-2.0.11.tar.bz2
$ tar jxf nilfs-utils-2.0.11.tar.bz2
$ cd nilfs-utils-2.0.11
$ ./configure
$ make 
$ sudo make install

Creating a file system on a file (ideal for playing around):

$ dd if=/dev/zero of=mynilfs bs=512M count=1
$ mkfs.nilfs2 mynilfs

The FS is only a mount away:

# mounting the file as a loop device
$ sudo losetup /dev/loop0 mynilfs
$ sudo mkdir /media/nilfs
$ sudo mount -t nilfs2 /dev/loop0 /media/nilfs/

Now lets create a couple of files:

$ cd /media/nilfs
$ touch 1 2 3 
# listing all checkpoints & snapshots, on your system list should vary 
$ lscp
                  CNO        DATE     TIME  MODE  FLG   NBLKINC       ICNT
                   7  2009-05-01 01:08:09   ss    -         12          6
                  13  2009-05-01 19:05:34   cp    i          8          3
                  14  2009-05-01 19:05:59   cp    i          8          3
                  15  2009-05-01 19:07:09   cp    -         12          6
# creating a snapshot
$ sudo mkcp -s
# 15 is the new snapshot (mode is ss)
$ lscp 
                 CNO        DATE     TIME  MODE  FLG   NBLKINC       ICNT
                   7  2009-05-01 01:08:09   ss    -         12          6
                  13  2009-05-01 19:05:34   cp    i          8          3
                  14  2009-05-01 19:05:59   cp    i          8          3
                  15  2009-05-01 19:07:09   ss    -         12          6
                  16  2009-05-01 19:08:59   cp    i          8          6

# our post snapshot file
$ touch 4

Now lets go back in time into our snapshot, NILFS enables us to mount old snapshots as read only FS (while the original FS is still mounted):

$ sudo mkdir /media/nilfs-snapshot
$ sudo mount.nilfs2  -r  /dev/loop0 /media/nilfs-snapshot/ -o cp=15 # only snapshots works!
$ cd /media/nilfs-snapshot
# as we might expect
$ ls 
 1 2 3 

NILFS has some interesting features, its not production ready yet however it sure worth looking after its development.

MetaClasses can bite!

Groovy has MOP classes built in like the MetaClass class, this class enables (among other things) objects introspection, we could for example use its getProperties() method in order to get the list of all the meta properties of an object and copy their values into another object:

class ParentModel {
    def String parentProp

    /**
     * Copying the current object properties into the other object
     */
    def copyInto(other){
        this.metaClass.getProperties().findAll{it.getSetter()!=null}.each{metaProp->
            if(metaProp.getProperty(this)!=null){
                metaProp.setProperty(other,metaProp.getProperty(this))
            }
        }
        other
    }
}

The usage is quite simple:

 def parent = new ParentModel(parentProp:"parent")
 def copiedParent = parent.copyInto(new ParentModel())
 assert copiedParent.parentProp.equals(parent.parentProp)

While this code runs perfectly fine with object that have identical classes it has some unexpected side effects when one of the classes derives from another:

class DerivingModel extends ParentModel{
    def derivedProp
}

def parent = new ParentModel(parentProp:"parent")
def derived = new DerivingModel(derivedProp:"derived")
def copiedDerived = parent.copyInto(derived)
assert copiedDerived.parentProp.equals(parent.parentProp)
assert copiedDerived.derivedProp.equals("derived")// throws groovy.lang.MissingPropertyException:
                                                  // No such property: derived

How can this be? the DerivingModel class has the derivedProp but after the copyInto invocation its missing!
Lets see what the getProperties method actually returns:

def parent = new ParentModel(parentProp:"parent")
parent.metaClass.getProperties().each{println it.name}// prints: class, parentProp, metaClass

It seems that we don't only get the parentProp that we defined but also the metaClass and class properties, this means that in the derived case the copyInto method has set the metaClass and class values of ParentModel into the DerivedModel instance (causing it to lose access to its derivedProp).
The fix is quite simple:

class ParentModel {
    def String parentProp
    def excludes = ['metaClass']
    def copyInto(other){
        this.metaClass.getProperties().findAll{it.getSetter()!=null}.each{metaProp->
            // we must not set the metaClass of the current object into the other
            if(metaProp.getProperty(this)!=null && !excludes.contains(metaProp.name)){
                metaProp.setProperty(other,metaProp.getProperty(this))
            }
        }
        other
    }
}

There is no need to exclude the class property since it keeps the original even if set.