Development for the masses

A blog about software development and other software related matters

Monday, June 29, 2009

Genericsfaction, getting some generics satisfaction

Java generics have a lot of short shortcoming, most of which are the result of the religious backward compatibility that Java is known for.
One of the most annoying "features" of generics is type erasure in which the compiler removes (almost) any trace of generic types meta data.

Iv been pondering about this issue & had a simple idea, what if we could add back in the meta data that the erasing process removed?
So, iv decided to create a proof of concept in order to show how this might work and named it genericsfaction.

Using genericsfaction requires an additional code manipulation stage in which the source code is scanned & gets its AST manipulated (meta data is introduced), each manipulated source file is dumped into a separate folder (named gen), all that is left is to compile the files under gen (instead of the original src).

At the moment there are only two enrichments implemented, the first is method input parameters annotation enrichment:


public void twoTypeParamsMet(@GenMeta(stringRep = "((Object ()) (Object ()))", classes = { Object.class, Object.class }) Map <Object, Object> map) {
// method code
}


The second is variable initialization enrichment:

public static void main(String [] args) {
List<Integer> list = (List <Integer>)MetaProxy.newInstance(new ArrayList<Integer>(),"(Integer ())",new Class []{Integer.class});
}

The reason that annotations aren't used in this case is due to this limitation.

There is no release to play with yet, however the source is up on github (mostly Clojure), I am planing on making a release as soon as ill finish to implement a validation module that uses this meta data in runtime.

Saturday, May 23, 2009

Newify your Object

Update a generous Anonymous has left a comment that the following special macro form does the same as the newify function that i present later on:


(def nested-object (-> GrandFather. Father. GrandSon.))

Feel free to skip this entry (unless your interested in an alternative implementation).

Instantiating large nested object graphs can be tedious in any language Clojure included:

(def nested-object
(new Grandson
(new Father
(new Grandfather))))

All those new methods call get old pretty quickly and are not DRY, to remedy this will take advantage on a cool property of Clojure Homoiconicity.
Homoiconicity basically means that the code is represented by a basic data structure of the language itself (lists in the case of Clojure), this means that manipulating the AST is easy as pie!
It would be nice if we could write something like the following:

(def nested-object
(def nested-object (newify '(Grandson (Father (Grandfather)))))

The newify function takes a nested list of objects & adds new before each object, post that the function evaluates the resulting list:

(defn newify [e] (eval (newify-imp e)))
; required since used before defined
(defn- newify-rest [e] )

(defn- newify-imp [e]
(let [add-new (partial cons 'new) add-rest (partial cons (first e))]
(if (not (empty? e))
(-> (newify-rest e) add-rest add-new) e)))

(defn- newify-rest [e]
(map #(if (list? %) (newify-impl %) %) (rest e)))

The newify-imp & newify-rest functions are mutually recursive, newify-imp adds the new symbol & the first list element to the result of the newify-rest function result, the newify-rest maps each list in the rest of the list to its newified value.

This implementation could be transformed into a recursive macro that will spare us from the list escaping:

(def nested-object (newify '(Grandson (Father (Grandfather)))))
; no escaping here
(def nested-object (newify (Grandson (Father (Grandfather)))))

Ill have to dig that option later on.

Thursday, April 30, 2009

NILFS on Jaunty

The latest release of Ubuntu includes the long awaited Ext4 FS (works flawlessly on my system).
Ext4 is faster & more secure but still lacks the ability to manage FS snapshots (ZFS excels in that, but runs only in FUSE on linux).
An interesting alternative is NILFS:

NILFS is a log-structured file system supporting versioning of the entire file system and continuous snapshotting which allows users to even restore files mistakenly overwritten or destroyed just a few seconds ago.


NILFS maintains a repo for Hardy, the Jaunty repos contain only the user land tools which won't do us much good since we need the kernel module as well, this leaves us only with the option of installing it from source (still quite easy).


# a perquisite
$ sudo aptitude install uuid-dev
# installing kernel module, result module resides in /lib/modules/2.6.28-11-generic/kernel/fs/nilfs2/nilfs2.ko
$ wget http://www.nilfs.org/download/nilfs-2.0.12.tar.bz2
$ tar jxf nilfs-2.0.12.tar.bz2
$ cd nilfs-2.0.12
$ make
$ sudo make install
# installing user land tools
$ wget http://www.nilfs.org/download/nilfs-utils-2.0.11.tar.bz2
$ tar jxf nilfs-utils-2.0.11.tar.bz2
$ cd nilfs-utils-2.0.11
$ ./configure
$ make
$ sudo make install

Creating a file system on a file (ideal for playing around):

$ dd if=/dev/zero of=mynilfs bs=512M count=1
$ mkfs.nilfs2 mynilfs

The FS is only a mount away:

# mounting the file as a loop device
$ sudo losetup /dev/loop0 mynilfs
$ sudo mkdir /media/nilfs
$ sudo mount -t nilfs2 /dev/loop0 /media/nilfs/

Now lets create a couple of files:

$ cd /media/nilfs
$ touch 1 2 3
# listing all checkpoints & snapshots, on your system list should vary
$ lscp
CNO DATE TIME MODE FLG NBLKINC ICNT
7 2009-05-01 01:08:09 ss - 12 6
13 2009-05-01 19:05:34 cp i 8 3
14 2009-05-01 19:05:59 cp i 8 3
15 2009-05-01 19:07:09 cp - 12 6
# creating a snapshot
$ sudo mkcp -s
# 15 is the new snapshot (mode is ss)
$ lscp
CNO DATE TIME MODE FLG NBLKINC ICNT
7 2009-05-01 01:08:09 ss - 12 6
13 2009-05-01 19:05:34 cp i 8 3
14 2009-05-01 19:05:59 cp i 8 3
15 2009-05-01 19:07:09 ss - 12 6
16 2009-05-01 19:08:59 cp i 8 6

# our post snapshot file
$ touch 4

Now lets go back in time into our snapshot, NILFS enables us to mount old snapshots as read only FS (while the original FS is still mounted):

$ sudo mkdir /media/nilfs-snapshot
$ sudo mount.nilfs2 -r /dev/loop0 /media/nilfs-snapshot/ -o cp=15 # only snapshots works!
$ cd /media/nilfs-snapshot
# as we might expect
$ ls
1 2 3

NILFS has some interesting features, its not production ready yet however it sure worth looking after its development.

Friday, March 6, 2009

MetaClasses can bite!

Groovy has MOP classes built in like the MetaClass class, this class enables (among other things) objects introspection, we could for example use its getProperties() method in order to get the list of all the meta properties of an object and copy their values into another object:


class ParentModel {
def String parentProp

/**
* Copying the current object properties into the other object
*/
def copyInto(other){
this.metaClass.getProperties().findAll{it.getSetter()!=null}.each{metaProp->
if(metaProp.getProperty(this)!=null){
metaProp.setProperty(other,metaProp.getProperty(this))
}
}
other
}
}

The usage is quite simple:

def parent = new ParentModel(parentProp:"parent")
def copiedParent = parent.copyInto(new ParentModel())
assert copiedParent.parentProp.equals(parent.parentProp)

While this code runs perfectly fine with object that have identical classes it has some unexpected side effects when one of the classes derives from another:

class DerivingModel extends ParentModel{
def derivedProp
}

def parent = new ParentModel(parentProp:"parent")
def derived = new DerivingModel(derivedProp:"derived")
def copiedDerived = parent.copyInto(derived)
assert copiedDerived.parentProp.equals(parent.parentProp)
assert copiedDerived.derivedProp.equals("derived")// throws groovy.lang.MissingPropertyException:
// No such property: derived

How can this be? the DerivingModel class has the derivedProp but after the copyInto invocation its missing!
Lets see what the getProperties method actually returns:

def parent = new ParentModel(parentProp:"parent")
parent.metaClass.getProperties().each{println it.name}// prints: class, parentProp, metaClass

It seems that we don't only get the parentProp that we defined but also the metaClass and class properties, this means that in the derived case the copyInto method has set the metaClass and class values of ParentModel into the DerivedModel instance (causing it to lose access to its derivedProp).
The fix is quite simple:

class ParentModel {
def String parentProp
def excludes = ['metaClass']
def copyInto(other){
this.metaClass.getProperties().findAll{it.getSetter()!=null}.each{metaProp->
// we must not set the metaClass of the current object into the other
if(metaProp.getProperty(this)!=null && !excludes.contains(metaProp.name)){
metaProp.setProperty(other,metaProp.getProperty(this))
}
}
other
}
}

There is no need to exclude the class property since it keeps the original even if set.

Sunday, March 1, 2009

Clojure on youtube

Iv really enjoyed watching the Intro to Clojure series:



If your interested what the fuss is about check it out!

Saturday, February 28, 2009

GroovyShell bindings validation

Groovy 1.6 got released not long a go with a load of new features, one of these is AST transformations which basically lets developers manipulate/access Groovy's AST before it gets loaded into the JVM.
This feature enables some really cool MOP options to mind however iv decided to try it in solving a non MOP-ish issue.

The GroovyShell is a very versatile and useful tool that enables us to evaluate any Groovy script file at runtime:


def bindings = [lastDay:2]
def script = """
import java.util.Calendar
Calendar cal = Calendar.instance
cal.set(2009,Calendar.FEBRUARY,2)
def ghday = cal.time
cal.set(2009,Calendar.MARCH,lastDay)
def spring = cal.time
def days = (ghday..spring).size()
println days
"""
def shell = new GroovyShell(new Binding(bindings))
shell.evaluate(script)

The script get its input parameters from the bindings hash, forgeting to pass an input variable (lastDay in this case) will cause a groovy.lang.MissingPropertyException to be thrown.

This is quite reasonable during development time but not in other scenarios in which the script is written in one machine & evaluated on a remote machine.
The exception gets thrown only post its evaluation which might be too late (deep in the execution flow), wouldn't it be nice if it was possible to validate this at will on the client side?

We could scan the script for suspect variables that might not have been initialised, such scanning might be possible by inspecting the script AST, looking after variables that weren't declared in the current script, Groovy code visitors to the rescue!

The following implementation matches the bill perfectly:

package com.jdftm.script.valid
import org.codehaus.groovy.ast.ClassCodeVisitorSupport
import org.codehaus.groovy.ast.expr.MethodCallExpression
import org.codehaus.groovy.control.SourceUnit
import org.codehaus.groovy.ast.expr.VariableExpression
import org.codehaus.groovy.ast.expr.DeclarationExpression

class ClassCodeVisitorSupportImp extends ClassCodeVisitorSupport {

def declared = []

def existsByDefault=['context','args']

def bindings

def notInBindings = {name -> bindings?.keySet().find{it.equals(name)} == null}

def notDeclared = {name -> declared?.find{it.text.equals(name)} == null && !existsByDefault.contains(name)}

def notValid = {notDeclared(it.text) && !it.hasInitialExpression() && notInBindings(it.text)}

def suspectVariables = []

public void visitMethodCallExpression(MethodCallExpression methodCall) {
def variables=methodCall.arguments.expressions.findAll{(it instanceof VariableExpression)}
suspectVariables << variables.findAll{notValid(it)}
}

public void visitDeclarationExpression(final DeclarationExpression decleration){
declared << decleration.variableExpression
}

protected SourceUnit getSourceUnit() {
return null;
}
}

This visitor keeps a suspects list of all the variables that are either not in the declared variables or in the bindings hash.
The following class encapsulates the visitor usage:

package com.jdftm.script.valid
import org.codehaus.groovy.ast.ClassCodeVisitorSupport
import org.codehaus.groovy.ast.ModuleNode
import org.codehaus.groovy.control.SourceUnit
import org.codehaus.groovy.control.CompilationUnit
import org.codehaus.groovy.ast.ClassNode
import org.codehaus.groovy.control.Phases;

class GroovyScriptValidator {

def validate(script,bindings){
def ast=getAST(script)
def impl=new ClassCodeVisitorSupportImp()
impl.bindings=bindings
impl.visitClass((ClassNode)ast.getClasses().get(0));
impl.suspectVariables.flatten()
}

def ModuleNode getAST(String source) {
SourceUnit unit = SourceUnit.create("Test",source);
CompilationUnit compUnit = new CompilationUnit();
compUnit.addSource(unit);
compUnit.compile(Phases.SEMANTIC_ANALYSIS);
return unit.getAST();
}
}

Validating the script is now simple:
 
package com.jdftm.script.valid

/**
*
* @author ronen
*/
class InParamValidationTest {

public static void main(String[] args) {
def bindings = [lastDay:2]
def script = """
import java.util.Calendar

Calendar cal = Calendar.instance
cal.set(2009,Calendar.FEBRUARY,2)
def ghday = cal.time
cal.set(2009,Calendar.MARCH,lastDay)
def spring = cal.time
def days = (ghday..spring).size()
println days
"""
def validator = new GroovyScriptValidator()
def suspects = validator.validate(script,bindings)
suspects.each{println it.text}
if(suspects.size()==0){
def shell = new GroovyShell(new Binding(bindings))
shell.evaluate(script)
}
}
}

AST introspection is one cool feature to have!

Monday, January 19, 2009

Injecting Closures with Groovy BeanBuilder

Programmatic Spring configuration has always appealed to me (can't wait for Spring JavaConfig to come out), the clarity and ability to cut down repetitive configuration meta data is the way to go.

Groovy eco system has got a lot of builders to configure any thing from build to you'v guessed it right Spring beans.
The BeanBuilder is a Groovy builder (google the pattern) class that enables us to define Spring beans in pure Groovy code, for example:


class AnyPojo {
def c
}

def builder=new BeanBuilder()
def value="yummy"
builder.anyPojo(AnyPojo){
c=value
}
def context = builder.createApplicationContext()
assert context.getBean("anyPojo").c.equals("yummy")

Note that an external value is injected into the bean (its not a spring bean), this might useful in cases that we want to pass behavior into our Spring beans like so:

def builder=new BeanBuilder()
def value={"yummy"}// returns "yummy"
builder.anyPojo(AnyPojo){
c=value
}
def context = builder.createApplicationContext()
assert context.getBean("anyPojo").c.call().equals("yummy")

This should work!, however upon running this code we get this exception:

.. Error creating bean with name '(inner bean)': Instantiation of bean failed; nested exception is java.lang.IllegalStateException: No bean class specified on bean definition

The builder thinks that we are trying to created a nested inner bean:

builder.anyPojo(AnyPojo){
c={ SomePojo p->
//..
}
}

No matter what as soon as the builder gets a closure as value for a property it fails.
The best solution that iv found (without altering the BeanBuilder code) is to wrap the closure in a delegating class like so:

class ClosureDelegate {
def c

ClosureDelegate(c){
this.c=c
}

def call(){
c.call()
}
}

def builder=new BeanBuilder()
def value=new ClosureDelegate({"yummy"})// the diff
builder.anyPojo(AnyPojo){
c=value
}
def context = builder.createApplicationContext()
assert context.getBean("anyPojo").c.call().equals("yummy")// this passes as well

This way we avoid the call to inner bean builder method (=(Closure c)).
This solution is partial (sometimes closures are applied without a call the the call method), but than again it should do it for most cases (oh and BTW you can get BeanBuilder without Grails from spring spring jar).

Blog Archive