A blog about software development and other software related matters

Blog Archive

Showing posts with label Java. Show all posts
Showing posts with label Java. Show all posts

Saturday, August 30, 2008

AMF serialization from Java to Flex and back

There are some scenarios in which you want to pass objects to and from a Flex application to a Java back end, if your using an Adobe framework (LCDS or BlazeDS) than you get this functionality out of the box however there are some cases that you wish to escape Adobe's grip and uses some third party technology.
Some solutions that may come to mind are JSon or XML, the problem with these solutions is that they may require a lot of bandwidth and even lots of computing power for complex objects graphs, AMF to the rescue.
AMF is a binary protocol which is used natively by the flash player, with the release of BlazeDS it was made accessible to any Java client that wishes to use it, in order to use AMF we need to serialize an object into bytes and pass it as the payload of any protocol that we choose, the easiest way to do so is to encode the resulting AMF bytes into BASE64 encoding (a format that transforms byte arrays into a readable string form) and append it to the existing payload.
Ill start with the Java side first:


import com.google.inject.Inject;
import flex.messaging.endpoints.BaseHTTPEndpoint;
import flex.messaging.io.SerializationContext;
import flex.messaging.io.amf.Amf3Input;
import flex.messaging.io.amf.Amf3Output;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import sun.misc.BASE64Decoder;
import sun.misc.BASE64Encoder;

/**
*
* @author ronen
*/
public class AmfSerializer {

@Inject
private SerializationContext context;

public <T> String toAmf(final T source) throws IOException {
final StringBuffer buffer = new StringBuffer();
final ByteArrayOutputStream bout = new ByteArrayOutputStream();
final Amf3Output amf3Output = new Amf3Output(context);
amf3Output.setOutputStream(bout);
amf3Output.writeObject(source);
amf3Output.flush();
amf3Output.close();
final BASE64Encoder encoder = new BASE64Encoder();
return encoder.encode(bout.toByteArray());
}

public <T> T fromAmf(final String amf) throws ClassNotFoundException, IOException {
final BASE64Decoder decoder = new BASE64Decoder();
byte[] input = decoder.decodeBuffer(amf);
InputStream bIn = new ByteArrayInputStream(input);
Amf3Input amf3Input = new Amf3Input(context);
amf3Input.setInputStream(bIn);
return (T) amf3Input.readObject();
}
}


This code uses BlazeDS API (the messaging-core, messaging-common and messaging-remoting jars) in order to serialize an object of type T into AMF byte array the bytes are encoded to a Base64 string, encoding result of an instance of the following VO:


package com.jdftm.vo;

import java.util.Date;

public class CurrentDayVO {

private Date now;

public CurrentDayVO() {
}

public void setNow(Date now) {
this.now = now;
}

public Date getNow() {
return now;
}
}


might look something like ChMzY29tLmpkZnRtLnZvLkN1cnJlbnREYXlWTwdub3cIAUJrjUw54AAA.

Now lets turn to the Flex side:


package com.jdftm.stomp.interop {
import flash.utils.ByteArray;
import mx.utils.Base64Encoder;
import mx.utils.Base64Decoder;

public class AMFSerializer {
public function serializeToString(value:Object):String{
if(value==null){
throw new Error("null isn't a legal serialization candidate");
}
var bytes:ByteArray = new ByteArray();
bytes.writeObject(value);
bytes.position = 0;
var be:Base64Encoder = new Base64Encoder();
be.encodeBytes(bytes);
var res:String = be.toString();
be.reset();
return res;
}

public function readObjectFromStringBytes(value:String):Object{
var dec:Base64Decoder=new Base64Decoder();
dec.decode(value);
var result:ByteArray=dec.drain();
result.position=0;
return result.readObject();
}
}
}


The logic is quite similar to the Java side code however when using the AMFSerializer we must register the serialized classes prior to serializing them:


registerClassAlias("com.jdftm.vo.CurrentDayVO", CurrentDayVO);


Failing to so will cause the de-serialization process (on both sides) to fail since it uses the alias in order to create the resulting instance object, its also required that the serialized classes on both ends to be in the same package as implemented in the CurrentDayVO Flex class:


package com.jdftm.vo{
[Bindable]
[RemoteClass(alias="com.jdftm.vo.CurrentDayVO")]
public class CurrentDayVO{
private var _now:Date;

public function get now():Date{
return _now;
}

public function set now(value:Date):void{
_now=value;
}
}
}


As youv seen AMF isn't to hard to use and may prove to be a powerful contender in the crowded integration protocols market.

Updated 2/09/08
Here is the SerializationContext implementation which is used in the Java serialization class


import com.google.inject.Provider;
import flex.messaging.io.SerializationContext;

public class SerializationContextProvider implements Provider<SerializationContext> {

@Override
public SerializationContext get() {
SerializationContext serializationContext = SerializationContext.getSerializationContext();// Threadlocal SerializationContent
serializationContext.enableSmallMessages = true;
serializationContext.instantiateTypes = true;
serializationContext.supportRemoteClass = true;// use _remoteClass field
serializationContext.legacyCollection = false;// false Legacy Flex 1.5 behavior was to return a java.util.Collection for Array, New Flex 2+ behavior is to return Object[] for AS3 Array
serializationContext.legacyMap = false;// false Legacy flash.xml.XMLDocument Type
serializationContext.legacyXMLDocument = false;// true New E4X XML Type
serializationContext.legacyXMLNamespaces = false;// determines whether the constructed Document is name-space aware
serializationContext.legacyThrowable = false;
serializationContext.legacyBigNumbers = false;
serializationContext.restoreReferences = false;
serializationContext.logPropertyErrors = false;
serializationContext.ignorePropertyErrors = true;
return serializationContext;

/*
serializationContext.enableSmallMessages = serialization.getPropertyAsBoolean(ENABLE_SMALL_MESSAGES, true);
serializationContext.instantiateTypes = serialization.getPropertyAsBoolean(INSTANTIATE_TYPES, true);
serializationContext.supportRemoteClass = serialization.getPropertyAsBoolean(SUPPORT_REMOTE_CLASS, false);
serializationContext.legacyCollection = serialization.getPropertyAsBoolean(LEGACY_COLLECTION, false);
serializationContext.legacyMap = serialization.getPropertyAsBoolean(LEGACY_MAP, false);
serializationContext.legacyXMLDocument = serialization.getPropertyAsBoolean(LEGACY_XML, false);
serializationContext.legacyXMLNamespaces = serialization.getPropertyAsBoolean(LEGACY_XML_NAMESPACES, false);
serializationContext.legacyThrowable = serialization.getPropertyAsBoolean(LEGACY_THROWABLE, false);
serializationContext.legacyBigNumbers = serialization.getPropertyAsBoolean(LEGACY_BIG_NUMBERS, false);
boolean showStacktraces = serialization.getPropertyAsBoolean(SHOW_STACKTRACES, false);
if (showStacktraces && Log.isWarn())
log.warn("The " + SHOW_STACKTRACES + " configuration option is deprecated and non-functional. Please remove this from your configuration file.");
serializationContext.restoreReferences = serialization.getPropertyAsBoolean(RESTORE_REFERENCES, false);
serializationContext.logPropertyErrors = serialization.getPropertyAsBoolean(LOG_PROPERTY_ERRORS, false);
serializationContext.ignorePropertyErrors = serialization.getPropertyAsBoolean(IGNORE_PROPERTY_ERRORS, true);
*/
}
}



This is the guice provider which is used when creating such contexts.

Monday, June 23, 2008

Testing those black sheep

One of the common reasonings for not writing tests is that writing tests for some code types isn't always possible nor cost effective, iv nicked them as the BDD black sheep, lets take for example command line API's, such API's are rarely tested & for some good(?) reasons like:


  • Its so simple, why do we need to test it?

  • Its hard to access, how can we assert anything in such an environment?

  • I hate it! let me get over this and get back to the safety zone of my UI!


While these are all compelling (>.<) reasons they are only plain excuses, writing command line API's is very tricky, it requires many validations and care in order to function properly, as for the hating part id replace it with deep lovin the sooner the better, command line is the most efficient user API and is here to stay (like it or not).
There are some technical challenges when performing assertions and mimicking user input however they can be easily overcome with two techniques, the first is mocking and its great for mimicking user input, now i know what your thinking, is he bulshitting me or what? how can a simple mock type the damn keyboard!?
Well the simple answer is that it don't have to, lets think for a minute what do we actually need to test (its not stdin!), we don't need to test that the keyboard works and that the typed data was placed correctly into ARGV, we need test how this data is handled, which means that a mocked data structure has to be created when writing tests, the reason that im using a mock (and not a fixture) is that im assuming that your using some sort of an input parsing framework (your not re inventing the wheel are you??) such as optiflag (Ruby) or JOpt (Java) and that this parser has to be mocked in order to provide some nice input into our program entry point.
The second technique is stream redirecting (who?), well its not that complicated lets recap for a minute, the user types in data and gets feedback in the form of text which is spat onto the screen via (youv guessed it write) stdout, this means that in order to assert the functionality of our program all that we need to do is to assert what ever is printed out in different scenarios, this can be achieved by using stream redirection into some sort of buffer, in Ruby such an approach might look somthing like this:

def redirect
orig_defout = $stdout
$stdout = StringIO.new
yield
$stdout.string
ensure
$stdout = orig_defout
end

Which can be used like so:

it "should say hello if -f flag is set" do
ARGV.flags.should_receive(:h).and_return(true)
outputStr = redirect { @bash.process }
outputStr.should eql('Hello!')
end

Don't forget that if there are other wanted side effects (file changes, table updates, etc..) they should be asserted like in any other code that your testing.

Wednesday, February 27, 2008

Yet another informational post (yaip)

After not writing for some time in this blog iv decided that i should post more regularly and share cool ideas/technologies that i encounter on daily/weekly bases, my agenda these days revolves around some of the following:


  • gookup - a Google services backup utility which takes your valuable online data and stores it localy on your hard drive for safe keeping.
    Iv been working on it for the past couple of months, the nice thing about it (besides its usefulness i hope) is the fact that its fully JRuby based & uses Buildr as its build tool.
    This matching allows some intresting combination like 'auto' Jar requiring in our JRuby code via the usage of the artifacts that we define in our build (ill elaborate on this on some other post).

  • Another subject that will keep me busy in the couple of following weeks is Flex, this nifty piece of technology will be a main part of projects that ill take part in the near future.
    My plan is to study the following subjects:Flex Basics, Flex & Java/Tomcat integration, Cairngorm.
    So far iv taken a look on Flex Builder (current idea support should get better on 7.0.3) and some online resources such as thirty-minute Flex test drive .
    Still the best way to study a new technology is by getting your hands dirty and actually building something with it (ill figure something up).

  • Last but not least is Adobe AIR and JavaFX, im considering using one of the two for gookup front end (AIR dosn't support linux yet), still i might just use swing and profligacy instead if the integration with JRuby will prove to be impossible.

Tuesday, December 4, 2007

Taking back control on your tests with TestNG

I'm truly a testing fanatic, in fact i think that for each component coded tests should be an integral part the process.
The thing is that as your code grows it gets harder to maintain the tests that you've written in addition JUnit adds pain to misery due to its bad design.
Its true that latest 4.* versions had fixed some really annoying practices like the naming convention of test methods and the need to extend a test base class but still it lacks in many other areas.

One of the major problems in JUnit is the fact that the main grouping point of tests is the test class itself which isn't fined grained enough, TestNG on the other hand allows to include any test method within any group no matter in which class its defined, for example:


package tests;

public class SomeCases {

@BeforeMethod(groups = {"restore", "authorization"})
public void cleanup() {
// some common cleanup code
}

@Test(groups = {"restore"})
public void getSingleModel() {
// ..
}

@Test(groups = {"restore"})
public void getManyModels() {
// ..
}

@Test(groups = {"authorization"})
public void authorizationService() {
// ..
}
}

This class holds a bunch of test methods which are divided to two major groups restore and authorization, this fine grained division enables all sorts of new possibilities like the handling of cross cutting concerns such as the cleanup method (we could easily move it to a super class to enable it on multiple classes).

Another possibility is to mix and match test suites that include test methods scattered around many classes, for example the next test method which belongs to the restore group that we've seen above:

package tests;

public class MoreSomeCases {
@Test(groups = {"restore"})
public void validateModelRestore() {
// ..
}
}

is made executable (with all the other restore methods) by defining the following suite (XML):

⟨suite name="BASIC_FUNCTIONALITY" verbose="1"⟩
⟨test name="BASIC_FUNCTIONALITY"⟩
⟨groups⟩
⟨run⟩
⟨include name="restore"/⟩
⟨/run⟩
⟨/groups⟩
⟨packages⟩
⟨package name="tests"/⟩
⟨/packages⟩
⟨/test⟩
⟨/suite⟩

this will enable the running of all the restore methods no matter in which class they are defined (this is an easy way to run integration tests).

TestNG has a lot more to offer (only a partial list):
- Defining input parameters of test methods and injecting them with data providers.
- Defining dependencies among tests and groups.
- Run tests in parallel.

Head on and check it out!

Thursday, November 1, 2007

Intellij 7.01 on Ubuntu => Java 6 upgrade

Intellij now makes use of JDK 6 which means that one should upgrade his version (using synaptic you know the drill) and set IDEA_JDK to /usr/lib/jvm/java-6-sun at the .bashrc file.

But wait! thats not all!!
The upgrade that youv just made has changed the default Java version symlink (under /usr/bin) which is used in your bash!, this means that any bash invoked application (maven, JBoss) will use the new JDK instead the older one (a big no no if your still developing in 1.5 like me).

sudo update-alternatives --config java

Another pointer that i have found to be useful is to make sure to delete the older .IntelliJIdea70 hidden folder, that in case that you have used the M2 idea version prior to the final release.

Thats all, Idea Rock on!

Monday, August 20, 2007

Generics shall risen again!

Here is a possible solution for cases in which some libraries that you depend upon and are not under your control (Spring, Apache Commons etc..) don't use generics (usually this is due to 1.42 backward compatibility), in such cases you'd usually write lines that might resemble something like this:


JdbcTemplate jdbcTemplate=new JdbcTemplate();
List⟨String⟩ result=jdbcTemplate.queryForList(/*query*/,/*params*/);

This code will result in unchecked assignment warning during compilation however the most annoying thing about it is that the IDE will not auto complete the generics types for us(=> more typing for us!).
My solution is based upon JRetrofit a framework that enables us to add interfaces to classes during runtime, first will create an interface that should contain all the commonly used JdbcTemplate methods:


public interface JdbcDynamicWrapper {
 List queryForList(String sql,Object[] args)throws DataAccessException;
}

And a factory method that will be used to get jdbcTemplate instances:

class JdbcTemplateFactory {
 public ⟨T⟩ JdbcDynamicWrapper⟨T⟩ getTemplateWraper(Class⟨T⟩ clazz){
  JdbcTemplate jdbcTemplate = new JdbcTemplate();
  return(JdbcDynamicWrapper⟨T⟩)
   Retrofit.partial(jdbcTemplate,JdbcDynamicWrapper.class);
 }
}

All that is left is to use the factory method:

JdbcTemplateFactory jdbcTemplateFactory=new JdbcTemplateFactory();
JdbcDynamicWrapper⟨String⟩templateWraper=jdbcTemplateFactory.getTemplateWraper(String.class);
List⟩String⟨ names=templateWraper.queryForList(/*query*/,/*params*/);

Now i know that this solution has its down sides (the need to add each method to the interface is one of them), but for methods which are very commonly used i think that its worth its price, don't you?

Friday, August 3, 2007

Building made easy..


Well this post is devoted to Buildr a Maven like build tool intended for Java projects which aims to make the whole building process much simpler to write and maintain.
It Actually uses similar Maven idioms & concepts like:
  • Usage of remote Repositories.
  • Artifacts.
  • Different Life cycle stages.
Still it differs in its infer structure which is based upon Rake instead of Ant (no more XML and thank god for that!).
One of the positive things that i feel that Ruby can bring into the Java sphere is simplicity and Buildr is no exception, its really easy to start get going in a matter of hours not days!.
With that last statement in mind iv had some troubles with the installation of Buildr under Ubuntu, which lead me to the following conclusions:
  • download and install gem manually (don't use synaptic).
  • sudo apt-get install ruby1.8-dev, build-essential.
  • choose ruby versions when ever gem asks you.

Now lets go ahead and create our simple build example with the following structure (looks familiar?):


We want our build to supply us with the following simple services:
  • Compile our project.
  • Create an Intellij project (with all the dependencies already defined).
  • Package our project into a jar.
  • Deploy our project into a folder ready to run.
  • To clean up our acts.
  • Run some basic tests.

Buildr requires a single file (named buildfile) to be placed at the base folder of our project, this file will contain the project's tasks.
Our first step will be to define the artifacts that we depend upon, Buildr uses Ruby data structures (arrays, structs and even hashes!) to define them.
Its up to us to if we want to place these definitions in a separate file or not (usually we do), here is our artifacts.rb file:

#group:id:type:version
DUMMY_ARTIFACTS=['commons-collections:commons-collections:jar:3.1']

Now will go ahead and define our project and its repositories:

require 'artifacts'
repositories.remote << "http://www.ibiblio.org/maven2/"
repositories.local = "home/myuser/.m2/repository/"
desc "our dummy project"
define "Dummy" do
project.version = '1.0'
project.group = "Dummy"
manifest["Main-Class"] = "main.Main"# defining the jar's main class

desc "dummy single module"
define "dummy-module" do
# our tasks ..
end
end# the end of the dummy single module end
end

Buildr provides us with all the expected predefined tasks that you might expect:

desc 'compiling'
compile.with DUMMY_ARTIFACTS

desc 'testing'
test.with DUMMY_ARTIFACTS

desc 'packing and including classpath in the manifest'
package(:jar).with(:manifest=>
manifest.merge("Class-Path" =>
compile.classpath.collect{|dep|
" "+File.basename(dep.inspect)}.join("\n").strip))

Don't be alarmed by the package task code it's quite simple (remember its all Ruby code!), each and every task can be referenced by using task(:name) (compile is equivalent to task(:compile)).
We are also referencing the manifest property which is a predefined data member of our project and setting the Class-Path property with all the artifacts that our project depends upon (the spacing is required due to weird manifest requirements), in order to run one of these tasks (at the project's base dir):

bla@bla-desktop:~/workspaces/Dummy$ buildr taskname
#in order to build an Intellij project simply use the idea task.


Now will head on to deploying our application into a folder in with the following structure:


This is a custom task that iv cooked up:

desc 'deploying the application'
task :deploy_app => [:package,:build_app_folders,:build_bin] do
  FileUtils.cp(path_to('target')+'/'+File.basename(project.packages[0].inspect),lib_path)
  compile.classpath.each{|dep|FileUtils.cp(dep.inspect.gsub('Buildr::Artifact:','').strip,lib_path)}
 end

This task depends upon the package, build_app_folders and build_bin tasks, it basicly copies the project jar and all the artifacts the project depends upon into the application folder.
Take special notice to the usage of Buildr path_to method which returns the path to our module's target folder, since its a custom task will need to run it by specifying its full name:

bla@bla-desktop:~/workspaces/Dummy$
buildr Dummy:dummy-module:deploy_app

Here are build_app_folders and build_bin tasks on which deploy_app depends upon:

desc 'building the application folders'
task :build_app_folders do
unless File.exists?(app_path)
FileUtils.mkdir app_path ,:mode=>0777
end
...
end

desc 'building the application launcher'
task :build_bin do
launch_cmd='java -Dfile.encoding=UTF-8 -jar ./lib/'+File.basename(project.packages[0].inspect)
unless File.exists?(bin_path+'/'+'run.sh')
f = File.new(bin_path+'/'+'run.sh',"w+")
f.puts '#!/bin/bash'
f.puts 'cd ..'
f.puts launch_cmd
FileUtils.chmod 0755,bin_path+'/'+'run.sh'
end
end

Now all that is left is to extend the clean task so that it will remove the application folder, in order to extend an existing buildr task will use the enhance method:

 desc 'cleaning up our act'
  clean.enhance {
   if File.exists?(app_path)
   FileUtils.remove_dir app_path
  end
 }

Thats all, happy building to us all :)

Wednesday, May 2, 2007

Dicsrete math & Java

In my Bachelor's degree i had the pleasure to study discrete math, discrete math handles group theory which address well hmm... groups :).
A group is a math entity that has a proper definition and a collection of operations which are operable on them such as: intersection, union, disjunction, subtraction.

At this point you must be thinking "well this all doesn't matter to us Java folks ..", well in fact we can think of Java collections and sets as groups (not exactly since we wont insist on a binary operation), thinking of them this way can result in some clean code implementation.
For example imagine that we need to find which changes were made to an arbitrary list of uniquely identifiable objects (list which are new, removed or old), the simplistic approach will look like:


public void printChanges(List older, List newer) {
for(Object value:older){
if(newer.contains(value)){
log.info(value.toString()+" is old");
} else {
log.info(value.toString()+" was removed");
}
}

for(Object value:newer){
if(!older.contains(value)){
log.info(value.toString()+" was added");
}
}
}

Its not the most elegant code since it contains loops and conditionals, in simple cases this might not be so bad but in more complex cases keeping trace on this kind of code is not easy, as for run time its about o(n).

Now lets see how the groups approach might work, our input consists of two object groups and we are seeking for three other groups that contain elements from these two, the most easy one to detect is the intersection of the two which match the old objects.
Finding the removed and the new is the same symmetric problem which is to find the objects that exists in one group but doesn't exist on the other, in groups lingo the operation that finds such objects is called subtraction.
Now lets take a look at the group oriented implementation:


//making use of org.apache.commons.collections
public void printChanges(Collection older, Collection newer) {
final Collection removed = CollectionUtils.subtract(older, newer);
final Collection added = CollectionUtils.subtract(newer, older);
final Collection old = CollectionUtils.intersection(older, newer);
log.info("removed: "+removed);
log.info("added: "+added);
log.info("old: "+old);
}

Its easy to see that this implementation is much more easy to follow since there are no loops or conditionals (code complexity is lower), as for runtime its also o(n).

Tuesday, April 3, 2007

JBoss PermGen , same old error

A couple of weeks ago iv stumbled upon the following post which carried the title "Good Riddance, PermGen OutOfMemoryError !", the thing is that this error happens quite alot when you are redeploying an application under JBoss for a couple of times.
The only way to recover from it is to restart JBoss which is quite annoying (especially when it takes 2min to load it up again).
Iv promised to myself to check the proposed post configuration and mailed the link to all my coworkers , obviously i forgot it totally :).
That was until one of my coworkers who read my email asked me if this thing actually works , his question had led me to perform the following test:

  • Iv compared two configuration of /bin/run.conf and counted how many times ill be able to redeploy an application without getting the error.

  • In the first (standard) configuration iv set JAVA_OPTS to "-Xms32m -Xmx64m -XX:MaxPermSize=64m .. " (the rest unchanged ) , this limits the memory size which is allocated to the JVM and to the perm gen memory section.

  • In the second (modified) configuration iv appended "-XX:+UseConcMarkSweepGC -XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled" to JAVA_OPTS.

The results were quite disappointing , no matter how iv tried to set these parameters the results were identical , in fact the only thing that made a positive effect was setting "-XX:MaxPermSize=64m" to "-XX:MaxPermSize=128m".
Frustrated a bit iv went back to the post and noticed that it was updated , a link to another
post was added , this one with the more realistic title "PremGen strikes back" :).