Hi all, I have drawed up draft version of GSoC 2012 project "Refactor OModel and provide intelligent migration feature for Apache ODE" project proposal, you guys can find the details here[1].
This is only initial idea of Tammo and me, if you guys have any good advises about this project, let's discuss here, and i will update the proposal according to the discussion result. As GSoC's student application period will close on April 6, we have still a week to improve the proposal. Thank you all :-) [1] http://code.google.com/p/tigergui-proposal-temp/wiki/Dynamic_OModel?ts=1333094366&updated=Dynamic_OModel ===============Refactor OModel and provide intelligent migration feature for Apache ODE==================== Introduction Apache ODE <http://ode.apache.org/> (Orchestration Director Engine) executes business processes written following the WS-BPEL<http://ode.apache.org/ws-bpel-20.html> standard. It talks to web services, sending and receiving messages, handling data manipulation and error recovery as described by your process definition. It supports both long and short living process executions to orchestrate all the services that are part of your application. Current ODE's OModel has a static structure and is serialized using Java's built-in mechanisms, which are not very tolerant against class changes and could even struggle with JDK upgrades. We want to create a GSoC project to refactor OModel to provide intelligent migration feature for ODE. Details DescriptionCurrent ODE OModel Design Let's begin with Apache ODE's architectural<http://ode.apache.org/architectural-overview.html> : When a BPEL process is deployed to ODE, it first runs through the compiler, which parses the BPEL file into a BPEL Object Model (BOM), which basically aims at abstracting from different BPEL versions (1.1, 2.0 draft, 2.0 final). ODE Compiler compile and create OModel instances(such as OProcess,OAssign,OInvoke etc) from .bpel, .wsdl and deploy files. Then it can use Java Object Serializable technology to serialize OModel objects into a .cbp file (Compiled Process Definitions part in the diagram, and we can see OBase implements interface java.io.Serializable, and all the OModel classes are the sub-class of OBase). ODE runtime module can load a compiled BPEL model(the .cbp file in local disk) by de-serialize the .cbp file back into OModel classes. When a process instance is executed, the navigator operates on the OModel, i.e. it is loaded from disk into memory during execution and is unloaded afterwards, depending on hydration/dehydration policies to reduce the memory footprint. The execution state of the OModel keeps in-memory references to the OModel, but when the state is persisted, it does not store a copy of the OModel but only the IDs mentioned above. When the state is loaded, the IDs are looked up and replaced by proper references. Where Is The Problem Once you read OModel's serializer and de-serializer methods in class org.apache.ode.bpel.o.Serializer: public void writeOProcess(OProcess process, OutputStream os) throws IOException { DataOutputStream out = new DataOutputStream(os); out.write(MAGIC_NUMBER); out.writeShort(format); out.writeLong(compileTime); out.writeUTF(process.guid); out.writeUTF(process.targetNamespace); out.writeUTF(process.processName); out.flush(); ObjectOutputStream oos = new CustomObjectOutputStream(os); oos.writeObject(process); oos.flush(); } public OProcess readOProcess() throws IOException, ClassNotFoundException { ObjectInputStream ois = new CustomObjectInputStream(_inputStream); OProcess oprocess; try { oprocess = (OProcess) ois.readObject(); } catch (ClassNotFoundException e) { throw new IOException("DataStream Error"); } return oprocess; } You will find the problem with the current OModel is that it has a static structure and is serialized using Java's built-in mechanisms, which are not very tolerant against class changes and could even struggle with JDK upgrades. For example, there are these attributes in current OProcess: public final String version; public OConstants constants; public String uuid; public String targetNamespace; public String processName; public OScope procesScope; After ODE Compiler serialize it into .cbp file, the .cbp file will include these 6 attributes for OProcess instance. One day, we add another attribute for OProcess class, for example: pubic String newAttribute; If we try to de-serialize the old .cbp file into the new OProcess class, we will get a error. This is a problem because BPEL processes and their instances are potentially long-running,they usually run over years. Thus, a serialized OModel should survive ODE and JDK upgrades. Our problem is that even simple changes (that are needed to implement new features or even to fix some bugs) break backward compatibility in the OModel. We need a good solution to address that problem. What Shall We Do To Solve This Problem This project want to solve this problem, we will change .cbp file's format and use a brand new serialize solution to keep compatibility. After investigate possible approaches, we would like to use JSON to represent OModel(It means new .cbp file will be JSON format). About the above example, the old JSON format .cbp file can be de-serialized into the new OProcess class with a special migrate condition description string. Json Format Jackson <http://jackson.codehaus.org/> is actually a JSON processor (JSON parser + JSON generator) written in Java. Beyond basic JSON reading/writing (parsing, generating), it also offers full node-based Tree Model, as well as full OJM (Object/Json Mapper) data binding functionality. More important, it has good performance, even faster than Java's built-in means. So we decided to use Json format file store OModel's serialize result, and Jackson 2.0 is the potential solution to reading/writing OModel instance from/to Json format .cbp file. We can use Jackson to store attributes of Current OProcess class<http://svn.apache.org/viewvc/ode/trunk/bpel-obj/src/main/java/org/apache/ode/bpel/o/OProcess.java?view=markup> to Json format file like this: Map<String,Object> userData1 = new HashMap<String,Object>(); userData1.put("owner", process.getOwner()); userData1.put("name", process.getName()); userData1.put("targetNameSpace", process.targetNamespace); userData1.put("processName", process.processName); userData1.put("children", process.getChildren()); ObjectMapper mapper = new ObjectMapper(); StringWriter sw = new StringWriter(); org.codehaus.jackson.JsonGenerator gen; try { gen = new JsonFactory().createJsonGenerator(sw); mapper.writeValue(gen, userData1); gen.close(); String json = sw.toString(); System.out.println(json); } catch (IOException e) { e.printStackTrace(); } The target json file will be: {"targetNameSpace":"http://ode/bpel/unit-test","name":"HelloWorld2","processName":"HelloWorld2"} But not all the OModel instance can be serialized only using Jacksons mixin feature, we have to extend Jackson and would probably need to write custom (de)serializers for WSDL4J etc. OModel Migrate Mechanism Here we use a simple example proposed by my potential mentor Tammo van Lessen to describe the migrate mechanism. Simple Example OProcess<http://svn.apache.org/viewvc/ode/trunk/bpel-obj/src/main/java/org/apache/ode/bpel/o/OProcess.java?view=markup> stores currently targetNamespace and processName as Strings. Now imagine we change OProcess so that it stores a QName instead. When loading an old OModel, the QName field would be null and there would be the two String field in the "unknown fields" map. Migrate Solution Suppose the old version of OProcess is stored in following format: {"targetNameSpace":"http://ode/bpel/unit-test","name":"HelloWorld2","processName":"HelloWorld2"} and the corresponding new version of OProcess's json .cbp file will be: {"name":"HelloWorld2","namespace":"{http://ode/bpel/unit-test}HelloWorld2"} The migrate job is transform from one JSON to another JSON in a specific rule, We should consider how to define the migrate description language or just re-use some existing JSON migrate DSL. This migrate DSL may be the most important part of this project. In this simple demo, maybe looks like this: { "from":{ "name":"$value0", "targetNameSpace":"$value1", "processName":"$value2" }, "to":{ "name":"$value0", "namespace":"{$value1}$value2" } } Use $value to represent migrate attribute values. Domain-specific language(DSL) Once we update OModel, and want to migrate the old version to new version. If we have both the old version Json format .cbp file and the specificed migrate rule, we can get the new version Json format .cbp file, then load as new OProcess object, finish the migrate job easily. We are still survey for exist tools like XSLT, working on JSON Trees. If can not find such a tool, have both the old OModel and the new OModel, we can write a tool to generate migrate DSL string. Of course, we must specific the migrate rule for different version change, such as v1.0->v1.1, v1.1->v1.2 and v1.0->v1.2. -- Best Regards From Tiger Gui -------------------------------------- Open source is some kind of life attitude
