Testing those black sheep

One of the common reasonings for not writing tests is that writing tests for some code types isn't always possible nor cost effective, iv nicked them as the BDD black sheep, lets take for example command line API's, such API's are rarely tested & for some good(?) reasons like:

• Its so simple, why do we need to test it?


• Its hard to access, how can we assert anything in such an environment?


• I hate it! let me get over this and get back to the safety zone of my UI!

While these are all compelling (>.<) reasons they are only plain excuses, writing command line API's is very tricky, it requires many validations and care in order to function properly, as for the hating part id replace it with deep lovin the sooner the better, command line is the most efficient user API and is here to stay (like it or not).
There are some technical challenges when performing assertions and mimicking user input however they can be easily overcome with two techniques, the first is mocking and its great for mimicking user input, now i know what your thinking, is he bulshitting me or what? how can a simple mock type the damn keyboard!?
Well the simple answer is that it don't have to, lets think for a minute what do we actually need to test (its not stdin!), we don't need to test that the keyboard works and that the typed data was placed correctly into ARGV, we need test how this data is handled, which means that a mocked data structure has to be created when writing tests, the reason that im using a mock (and not a fixture) is that im assuming that your using some sort of an input parsing framework (your not re inventing the wheel are you??) such as optiflag (Ruby) or JOpt (Java) and that this parser has to be mocked in order to provide some nice input into our program entry point.
The second technique is stream redirecting (who?), well its not that complicated lets recap for a minute, the user types in data and gets feedback in the form of text which is spat onto the screen via (youv guessed it write) stdout, this means that in order to assert the functionality of our program all that we need to do is to assert what ever is printed out in different scenarios, this can be achieved by using stream redirection into some sort of buffer, in Ruby such an approach might look somthing like this:

def redirect
  orig_defout = $stdout
  $stdout = StringIO.new
  yield
  $stdout.string
ensure
  $stdout = orig_defout
end

Which can be used like so:

  it "should say hello if -f flag is set" do
    ARGV.flags.should_receive(:h).and_return(true)
    outputStr = redirect { @bash.process }
    outputStr.should eql('Hello!')  
end

Don't forget that if there are other wanted side effects (file changes, table updates, etc..) they should be asserted like in any other code that your testing.

Quick and dirty code search

There are some situations in which i need to query some code folder quick and dirty on my hard drive.
There are existing tools like desktop search applications (such as Google desktop) but it usually takes some time for them to index new folders and they are also quite resources hungry.
Bash find utility is also an alternative how ever this tool works best only on Unix machines (NTFS & cygwin's find aren't the best pair for such a use case).

A nice quick and dirty solution it to use Ferret which is a Ruby text indexing framework inspired by Apache Lucence to quickly index the code & search it up, all by using two small snippets of Ruby code!
The code following code is based on this entry which in turn is based on this one, first is the indexing code (index.rb):

require 'rubygems'
require 'ferret'
require 'find'
include Ferret

index = Index::Index.new(:default_field => 'content', :path => '/tmp/index_folder')# creating the index
ini = Time.now
numFiles=0
IndexedExts=['.java','.properties']# the list of file extensions that we wish to index
Find.find('/code/to/index') do |path|

 if(IndexedExts.find {|ext| path.include?(ext)}==nil)
   next # this file is ignored
 end 
 puts "Indexing: #{path}"
 numFiles=numFiles+1

 if FileTest.file? path
  File.open(path) do |file|
   index.add_document(:file => path, :content => file.readlines)
  end
 end
end

elapsed = Time.now - ini
puts "Files: #{numFiles}"
puts "Elapsed time: #{elapsed} secs\n"

This code is quite simple, first we are creating an index folder then we use the Find module and iterate all the files which are contained within the 'code/to/search' folder, the files which don't have a matching extension are rejected the rest have their content added to the index, next is the query code (search.rb):

require 'rubygems'
require 'ferret'
require 'find'

wot = ARGV[0]
if wot.nil?
 puts "use: search.rb "
 exit
end

index = Ferret::Index::Index.new(:default_field => 'content', :path => '/tmp/index_folder')
ini = Time.now
puts "Searching.."
docs=0

index.search_each(wot, options={:limit=>:all}) do |doc, score|
 
 res= < -------------------------------------------------------
 #{File.basename(index[doc]['file'])} :
 #{index.highlight(wot, doc,:field => :content,:pre_tag => "->>",:post_tag => "<<-")}
STRING_END
 puts res              
 docs+=1
end

elapsed = Time.now - ini
puts "Elapsed time: #{elapsed} secs\n"
puts "Documents found: #{docs}"

In this code we first load up the index with the default query field content, the query itself takes its value from the wot parameter, after the query execution the code block prints out the matching highlighting of the found match in the file.
That all about there is to it, see ya.

Building made easy..

Well this post is devoted to Buildr a Maven like build tool intended for Java projects which aims to make the whole building process much simpler to write and maintain.
It Actually uses similar Maven idioms & concepts like:

• Usage of remote Repositories.
• Artifacts.
• Different Life cycle stages.

Still it differs in its infer structure which is based upon Rake instead of Ant (no more XML and thank god for that!).
One of the positive things that i feel that Ruby can bring into the Java sphere is simplicity and Buildr is no exception, its really easy to start get going in a matter of hours not days!.
With that last statement in mind iv had some troubles with the installation of Buildr under Ubuntu, which lead me to the following conclusions:

• download and install gem manually (don't use synaptic).
• sudo apt-get install ruby1.8-dev, build-essential.
• choose ruby versions when ever gem asks you.

Now lets go ahead and create our simple build example with the following structure (looks familiar?):


We want our build to supply us with the following simple services:

• Compile our project.
• Create an Intellij project (with all the dependencies already defined).
• Package our project into a jar.
• Deploy our project into a folder ready to run.
• To clean up our acts.
• Run some basic tests.

Buildr requires a single file (named buildfile) to be placed at the base folder of our project, this file will contain the project's tasks.
Our first step will be to define the artifacts that we depend upon, Buildr uses Ruby data structures (arrays, structs and even hashes!) to define them.
Its up to us to if we want to place these definitions in a separate file or not (usually we do), here is our artifacts.rb file:

#group:id:type:version
DUMMY_ARTIFACTS=['commons-collections:commons-collections:jar:3.1'] 

Now will go ahead and define our project and its repositories:

require 'artifacts'
repositories.remote << "http://www.ibiblio.org/maven2/"
repositories.local = "home/myuser/.m2/repository/"      
desc "our dummy project" 
define "Dummy" do               
  project.version = '1.0'           
  project.group = "Dummy"            
  manifest["Main-Class"] = "main.Main"# defining the jar's main class       
   
  desc "dummy single module"              
   define "dummy-module" do              
    # our tasks ..            
   end    
  end# the end of the dummy single module  end 
end

Buildr provides us with all the expected predefined tasks that you might expect:

 desc 'compiling'
 compile.with DUMMY_ARTIFACTS

 desc 'testing'
 test.with DUMMY_ARTIFACTS

 desc 'packing and including classpath in the manifest'
 package(:jar).with(:manifest=>
 manifest.merge("Class-Path" =>
 compile.classpath.collect{|dep| 
 "  "+File.basename(dep.inspect)}.join("\n").strip))

Don't be alarmed by the package task code it's quite simple (remember its all Ruby code!), each and every task can be referenced by using task(:name) (compile is equivalent to task(:compile)).
We are also referencing the manifest property which is a predefined data member of our project and setting the Class-Path property with all the artifacts that our project depends upon (the spacing is required due to weird manifest requirements), in order to run one of these tasks (at the project's base dir):

 bla@bla-desktop:~/workspaces/Dummy$ buildr taskname
 #in order to build an Intellij project simply use the idea task.

Now will head on to deploying our application into a folder in with the following structure:


This is a custom task that iv cooked up:

 desc 'deploying the application'
 task :deploy_app => [:package,:build_app_folders,:build_bin] do      
  FileUtils.cp(path_to('target')+'/'+File.basename(project.packages[0].inspect),lib_path)
  compile.classpath.each{|dep|FileUtils.cp(dep.inspect.gsub('Buildr::Artifact:','').strip,lib_path)}
 end

This task depends upon the package, build_app_folders and build_bin tasks, it basicly copies the project jar and all the artifacts the project depends upon into the application folder.
Take special notice to the usage of Buildr path_to method which returns the path to our module's target folder, since its a custom task will need to run it by specifying its full name:

bla@bla-desktop:~/workspaces/Dummy$ 
buildr Dummy:dummy-module:deploy_app

Here are build_app_folders and build_bin tasks on which deploy_app depends upon:

 desc 'building the application folders'
  task :build_app_folders do
  unless File.exists?(app_path)
   FileUtils.mkdir app_path ,:mode=>0777
  end
  ...
 end

 desc 'building the application launcher'
 task :build_bin do
  launch_cmd='java -Dfile.encoding=UTF-8 -jar ./lib/'+File.basename(project.packages[0].inspect)
  unless File.exists?(bin_path+'/'+'run.sh')
   f = File.new(bin_path+'/'+'run.sh',"w+")
   f.puts '#!/bin/bash'
   f.puts 'cd ..'
   f.puts launch_cmd
   FileUtils.chmod 0755,bin_path+'/'+'run.sh'
  end
 end

Now all that is left is to extend the clean task so that it will remove the application folder, in order to extend an existing buildr task will use the enhance method:

 desc 'cleaning up our act'
  clean.enhance {
   if File.exists?(app_path)  
   FileUtils.remove_dir app_path 
  end
 }

Thats all, happy building to us all :)

A glass of Watir to quench your thirst

A couple of years ago (long before rails was in focus) Ruby has caught my eyes , to be exact it was Watir which is a Ruby based web testing framework that took my attention , at the time i didnt know how much useful it can be.
Watir is quite interesting since it drives the actual web browser UI (IE) during testing procedures , this makes it extremely simple to use , you don't have to sniff any http posts or use low level API , just select your DOM element and the operation that you wish the browser to perform on it:

 ie = IE.new # our IE window
 ie.goto("http://mytestsite")
 ie.text_field(:name, "typeinme").clear # clearing out a text field that carries the name typeinme

What can you do with it?
Well There are many applications (besides basic testing) that can utilize it such as:
• Web site monitors that scans web sites at certain intervals and look for errors.
• Web crawlers and Form data extractors.
• Applications that automatically recreate bugs (using saved user input data).

The fact that Watir is Ruby based makes its very extensible , you can use it in conjunction with Rails to create a web based management console for your application , fetching of input data can be done easily with ActiveRecord from you favorite DB.

Watir has its downsides too , its not very efficient since it requires an entire machine (as long your program keeps on running) , also since it works at the UI level (IE is its main resource) it takes a large amount of computing power (especially when there are a couple of them open).
Watir largest drawback is that its only supports IE at the current moment (support for Mozilla is planned on the next release) which is quite limiting (at least you can use this hack for running it on Linux) , still trust me Watir is one of the most useful frameworks youl ever stumble upon.