A blog about software development and other software related matters

Blog Archive

Showing posts with label Testing. Show all posts
Showing posts with label Testing. Show all posts

Monday, June 23, 2008

Testing those black sheep

One of the common reasonings for not writing tests is that writing tests for some code types isn't always possible nor cost effective, iv nicked them as the BDD black sheep, lets take for example command line API's, such API's are rarely tested & for some good(?) reasons like:


  • Its so simple, why do we need to test it?

  • Its hard to access, how can we assert anything in such an environment?

  • I hate it! let me get over this and get back to the safety zone of my UI!


While these are all compelling (>.<) reasons they are only plain excuses, writing command line API's is very tricky, it requires many validations and care in order to function properly, as for the hating part id replace it with deep lovin the sooner the better, command line is the most efficient user API and is here to stay (like it or not).
There are some technical challenges when performing assertions and mimicking user input however they can be easily overcome with two techniques, the first is mocking and its great for mimicking user input, now i know what your thinking, is he bulshitting me or what? how can a simple mock type the damn keyboard!?
Well the simple answer is that it don't have to, lets think for a minute what do we actually need to test (its not stdin!), we don't need to test that the keyboard works and that the typed data was placed correctly into ARGV, we need test how this data is handled, which means that a mocked data structure has to be created when writing tests, the reason that im using a mock (and not a fixture) is that im assuming that your using some sort of an input parsing framework (your not re inventing the wheel are you??) such as optiflag (Ruby) or JOpt (Java) and that this parser has to be mocked in order to provide some nice input into our program entry point.
The second technique is stream redirecting (who?), well its not that complicated lets recap for a minute, the user types in data and gets feedback in the form of text which is spat onto the screen via (youv guessed it write) stdout, this means that in order to assert the functionality of our program all that we need to do is to assert what ever is printed out in different scenarios, this can be achieved by using stream redirection into some sort of buffer, in Ruby such an approach might look somthing like this:

def redirect
orig_defout = $stdout
$stdout = StringIO.new
yield
$stdout.string
ensure
$stdout = orig_defout
end

Which can be used like so:

it "should say hello if -f flag is set" do
ARGV.flags.should_receive(:h).and_return(true)
outputStr = redirect { @bash.process }
outputStr.should eql('Hello!')
end

Don't forget that if there are other wanted side effects (file changes, table updates, etc..) they should be asserted like in any other code that your testing.

Monday, April 21, 2008

Making FunFx Flex 2 compatible

Moving to the latest and greatest software version in the enterprise takes time, Flex 3 came out not long ago and so did FunFx which a really cool tool for Flex integration testing.
FunFx got compiled with Flex 3 SDK which means that it won't run with Flex 2 projects out of the box, the developer does states that its possible to compile it from source with Flex 2 SDK however the procedure isn't trivial and took me some trial and error that i hope to save for the rest of you out there.
In order to get a Flex 2 compatible FunFx will need to compile the FunFxAdapter which is a swc library that any testable RIA has to depended upon, go ahead and grab its source from here, create a swc project within Flex builder called FunFxAdapter and add the source.
Now open the project properties and add in the following dependencies to the compiler options (your paths may vary):

 
-include-libraries "/Applications/eclipse 3.2.2/Adobe Flex Builder 3 Plug-in/sdks/2.0.1/frameworks/libs/automation.swc" "/Applications/eclipse 3.2.2/Adobe Flex Builder 3 Plug-in/sdks/2.0.1/frameworks/libs/automation_agent.swc"

The compiler options should look like:



Unfortunately the FunFxAdapter project depends upon the automation.swc library which was not a part of the Flex 2 SDK (it was introduced in Flex 3 SDK but this version isn't binary compilable with Flex 2 SDK), if your project is a legacy Flex 2 project than its most likely that you have access to this swc.
Compile the project and grab the compiled swc from the bin folder of the project, when adding it to the test target application you'll need to add some more dependencies in order to make it work with the compiled FunFxAdapter swc:
 
-include-libraries "/Users/ronen/Documents/workspace/FunFXAdapter/bin/FunFXAdapter.swc" "/Applications/eclipse 3.2.2/Adobe Flex Builder 3 Plug-in/sdks/2.0.1/frameworks/libs/automation.swc" "/Applications/eclipse 3.2.2/Adobe Flex Builder 3 Plug-in/sdks/2.0.1/frameworks/libs/automation_agent.swc" "/Applications/eclipse 3.2.2/Adobe Flex Builder 3 Plug-in/sdks/2.0.1/frameworks/locale/ja_JP/automation_agent_rb.swc"

hopefully iv saved your valuable time.
After publish comment:
Iv forgot to mention that you should add the AutomationGenericEnv.xml file to the src directory of the main project module and to the application folder under tomcat (a standard FunFx requirements also in the Flex 3 version).

Tuesday, December 4, 2007

Taking back control on your tests with TestNG

I'm truly a testing fanatic, in fact i think that for each component coded tests should be an integral part the process.
The thing is that as your code grows it gets harder to maintain the tests that you've written in addition JUnit adds pain to misery due to its bad design.
Its true that latest 4.* versions had fixed some really annoying practices like the naming convention of test methods and the need to extend a test base class but still it lacks in many other areas.

One of the major problems in JUnit is the fact that the main grouping point of tests is the test class itself which isn't fined grained enough, TestNG on the other hand allows to include any test method within any group no matter in which class its defined, for example:


package tests;

public class SomeCases {

@BeforeMethod(groups = {"restore", "authorization"})
public void cleanup() {
// some common cleanup code
}

@Test(groups = {"restore"})
public void getSingleModel() {
// ..
}

@Test(groups = {"restore"})
public void getManyModels() {
// ..
}

@Test(groups = {"authorization"})
public void authorizationService() {
// ..
}
}

This class holds a bunch of test methods which are divided to two major groups restore and authorization, this fine grained division enables all sorts of new possibilities like the handling of cross cutting concerns such as the cleanup method (we could easily move it to a super class to enable it on multiple classes).

Another possibility is to mix and match test suites that include test methods scattered around many classes, for example the next test method which belongs to the restore group that we've seen above:

package tests;

public class MoreSomeCases {
@Test(groups = {"restore"})
public void validateModelRestore() {
// ..
}
}

is made executable (with all the other restore methods) by defining the following suite (XML):

⟨suite name="BASIC_FUNCTIONALITY" verbose="1"⟩
⟨test name="BASIC_FUNCTIONALITY"⟩
⟨groups⟩
⟨run⟩
⟨include name="restore"/⟩
⟨/run⟩
⟨/groups⟩
⟨packages⟩
⟨package name="tests"/⟩
⟨/packages⟩
⟨/test⟩
⟨/suite⟩

this will enable the running of all the restore methods no matter in which class they are defined (this is an easy way to run integration tests).

TestNG has a lot more to offer (only a partial list):
- Defining input parameters of test methods and injecting them with data providers.
- Defining dependencies among tests and groups.
- Run tests in parallel.

Head on and check it out!