Senaka, I didn't execute the code yet, but I did a quick review and it looks already very good. I would like to make the following comments to improve this still further:
* In WrappedTextNodeStreamReader, the character data is returned in chunks in order to avoid loading the entire data into memory (typically the data comes from a temporary file). I don't think that this is necessary for the map values, and could even introduce unnecessary overhead. They should simply be converted to a String and returned as a single chunk. Getting rid of the java.io.Reader would also simplify the code. * The right way to represent a byte[] value is to produce a DataHandler (which is equivalent to having the binary data encoded as Base64). Note that this is not directly supported by the StAX API, but rather an extension introduced by AXIOM to handle binary data efficiently. Please have a look at StAXBuilder#createOMText(OMContainer, int) to see how this magic works. * For the moment the key of a map entry is represented using a "key" attribute but also used for the element name. I guess this is a mistake. Since a map key is not necessarily a valid XML element name, I think we should prefer the representation using an attribute. Regards, Andreas On Sun, Nov 9, 2008 at 11:40, Senaka Fernando <[EMAIL PROTECTED]> wrote: > Andreas, > > I did go through your suggested implementation, and [1]'s what I'm planning > to do. Please do let me know whether I've made the correct choices. As of > now, the getElementText() method is perhaps not quite correct and I have not > yet added a mechanism to represent a byte[]. > > [1] > http://sci-flex.googlecode.com/svn/sci-flex/trunk/java/axiom/src/main/java/org/apache/axiom/om/util/WrappedMapNodeStreamReader.java > > Regards, > Senaka > > On Sat, Nov 8, 2008 at 1:36 AM, Sanjiva Weerawarana > <[EMAIL PROTECTED]>wrote: > >> +1 Andreas. This should be written so that the OM is created IFF XML >> navigation is done. Otherwise the map message should remain in Java and then >> just get piped thru - that's critical for Synapse performance. >> >> Sanjiva. >> >> >> Andreas Veithen wrote: >> >>> Senaka, >>> >>> The AXIOM tree is built twice because of the following piece of code: >>> >>> public XMLStreamReader getReader() throws XMLStreamException { >>> return getUnderlyingElement().getXMLStreamReader(); >>> } >>> >>> The getUnderlyingElement method will build an AXIOM tree representing >>> the Map(Message), but when the OMSourcedElement is expanded, AXIOM >>> will build another tree based on the events pulled from the >>> XMLStreamReader. There are two options then: >>> >>> 1. One considers that in the vast majority of cases, the content will >>> be accessed anyway. Then it would make more sense to construct the >>> AXIOM tree directly when the message is received (i.e. no need for an >>> OMSourcedElement). >>> 2. Don't build an AXIOM tree inside the OMDataSource but construct an >>> XMLStreamReader implementation that returns the sequence of StAX >>> events corresponding to the desired XML representation. >>> >>> I used the technique behind option 2 in the following piece of code: >>> >>> >>> http://svn.apache.org/repos/asf/synapse/trunk/java/modules/core/src/main/java/org/apache/synapse/util/WrappedTextNodeStreamReader.java >>> >>> The XMLStreamReader implementation shown in this link is used to >>> transform character data (provided by a java.io.Reader) into an >>> OMSourcedElement that wraps this data, i.e. the resulting tree would >>> be an element with a text node as child. That doesn't sound very >>> useful at first glance, but in case of very long character data, it >>> allows to stream the data almost directly from the source to the >>> destination without ever building the OMText nodes (which would >>> consume a large amount of memory). >>> >>> Given that reading the data source is non destructive, option 2 has >>> the advantage that the AXIOM tree >>> * will be built exactly once if somebody queries the child OMNodes; >>> * will not be built at all when somebody serializes the content into a >>> byte stream. >>> >>> While this is the optimal solution, it is also much more difficult to >>> implement. It is certainly an interesting challenge to do that. >>> >>> Finally, for the type problem, it is indeed sufficient to add a "type" >>> attribute to the element that represents the key-value pair: >>> >>> <price type="double">12.456</price> >>> >>> Andreas >>> >>> On Thu, Oct 30, 2008 at 10:34, Senaka Fernando <[EMAIL PROTECTED]> >>> wrote: >>> >>>> Hi Andreas, >>>> I agree with your observations here. Also, I would like to understand >>>> what >>>> you mean by "it will build the AXIOM tree twice when the content >>>> is accessed", can this be corrected? As far as Map Messages found in the >>>> jms >>>> transport are concerned, the key is of type string, and the value is a >>>> primitive java type, I believe that a slight modification option 2 >>>> discussed >>>> here should work. WDYT? >>>> >>>> Regards, >>>> Senaka >>>> >>>> On Thu, Oct 30, 2008 at 2:10 AM, Andreas Veithen >>>> <[EMAIL PROTECTED]>wrote: >>>> >>>> Having alternative strategies that map between MapMessages and XML >>>>> might be interesting, but to start with we should have at least one >>>>> implementation that meets all of the following requirements: >>>>> >>>>> 1. Highly optimized and having the least possible overhead (even if >>>>> the AXIOM tree is build). >>>>> 2. The XML representation must be simple so that it can be easily used >>>>> with XSLT and XPath. >>>>> 3. The mapping must be two way and lossless. That is important if you >>>>> want to switch from JMS to another protocol and then back again to >>>>> JMS. >>>>> >>>>> In my opinion, the XMLEncoder based solution doesn't satisfy the first >>>>> two requirements, but will meet the last one. >>>>> >>>>> The other implementation you propose >>>>> - partially satisfies requirement 1 (partially because - as far as I >>>>> can see - it will build the AXIOM tree twice when the content is >>>>> accessed); >>>>> - satisfies requirement 2; >>>>> - doesn't satisfy requirement 3 because it looses information about >>>>> the property types, i.e. you will not be able to recreate an >>>>> equivalent MapMessage from the XML representation. >>>>> >>>>> Andreas >>>>> >>>>> >>>>> On Tue, Oct 28, 2008 at 04:44, Senaka Fernando <[EMAIL PROTECTED]> >>>>> wrote: >>>>> >>>>>> Hi Andreas, >>>>>> >>>>>> The scenario here was to have an implementation that will support Map >>>>>> Messages "as well as" hierarchical Maps, and any generic use of Maps >>>>>> with >>>>>> OM. And as you have mentioned here Map Messages can only have primitive >>>>>> types on it. Therefore, in theory MapMessage support would only require >>>>>> a >>>>>> subset of provisions made by this implementation. >>>>>> >>>>>> Also, if you have tried the implementation I have at the moment, it >>>>>> >>>>> supports >>>>> >>>>>> alternative strategies (so you may use whatever type of serializer you >>>>>> want). >>>>>> >>>>>> Regards, >>>>>> Senaka >>>>>> >>>>>> On Tue, Oct 28, 2008 at 5:34 AM, Andreas Veithen >>>>>> <[EMAIL PROTECTED]>wrote: >>>>>> >>>>>> Senaka, >>>>>>> >>>>>>> How does your question actually relate to the MapMessage support you >>>>>>> are working on? AFAIK MapMessages can't contain arbitrary Java >>>>>>> objects. >>>>>>> >>>>>>> Andreas >>>>>>> >>>>>>> On Sun, Oct 26, 2008 at 22:19, Senaka Fernando <[EMAIL PROTECTED]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Andreas, >>>>>>>> >>>>>>>> Here you go: >>>>>>>> >>>>>>>> <map> >>>>>>>> <java version="1.6.0_06" class="java.beans.XMLDecoder"> >>>>>>>> <object class="java.util.TreeMap"> >>>>>>>> <void method="put"> >>>>>>>> <string>KeyStr</string> >>>>>>>> <string>five</string> >>>>>>>> </void> >>>>>>>> <void method="put"> >>>>>>>> <string>Test</string> >>>>>>>> <float>5.5</float> >>>>>>>> </void> >>>>>>>> <void method="put"> >>>>>>>> <string>SomeKey</string> >>>>>>>> <int>5</int> >>>>>>>> </void> >>>>>>>> <void method="put"> >>>>>>>> <string>nested</string> >>>>>>>> <object class="java.util.TreeMap"> >>>>>>>> <void method="put"> >>>>>>>> <string>me</string> >>>>>>>> <float>2.0</float> >>>>>>>> </void> >>>>>>>> <void method="put"> >>>>>>>> <string>more</string> >>>>>>>> <int>100</int> >>>>>>>> </void> >>>>>>>> <void method="put"> >>>>>>>> <string>moreNested</string> >>>>>>>> <object class="java.util.TreeMap"> >>>>>>>> <void method="put"> >>>>>>>> <string>String</string> >>>>>>>> <string>ten</string> >>>>>>>> </void> >>>>>>>> </object> >>>>>>>> </void> >>>>>>>> </object> >>>>>>>> </void> >>>>>>>> </object> >>>>>>>> </java> >>>>>>>> </map> >>>>>>>> >>>>>>>> This is the serialization for a TreeMap having {<KeyStr, five>, >>>>>>>> <Test, >>>>>>>> >>>>>>> 5.5>, >>>>>>> >>>>>>>> <someKey, 5>, <nested, {<me, 2.0>, <more, 100>, <moreNested, >>>>>>>> {<String, >>>>>>>> ten>}>}>} >>>>>>>> >>>>>>>> Regards, >>>>>>>> Senaka >>>>>>>> >>>>>>>> On Mon, Oct 27, 2008 at 1:52 AM, Andreas Veithen >>>>>>>> <[EMAIL PROTECTED]>wrote: >>>>>>>> >>>>>>>> Senaka, >>>>>>>>> >>>>>>>>> Just a quick question: what does the serialization of a Map looks >>>>>>>>> >>>>>>>> like >>>>> >>>>>> with XMLEncoder? >>>>>>>>> >>>>>>>>> Andreas >>>>>>>>> >>>>>>>>> On Sat, Oct 25, 2008 at 20:01, Senaka Fernando <[EMAIL PROTECTED] >>>>>>>>> > >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> I'm working on a mechanism to attach a java.util.Map onto an Axiom >>>>>>>>>> >>>>>>>>> Tree. >>>>>>> >>>>>>>> So >>>>>>>>> >>>>>>>>>> far, I have been able to attach the java.util.Map onto the OM Tree >>>>>>>>>> >>>>>>>>> with >>>>>>> >>>>>>>> the >>>>>>>>> >>>>>>>>>> help of a specialized data source I have created. This >>>>>>>>>> >>>>>>>>> implementation >>>>> >>>>>> features on-demand building of the XML payload and I believe the >>>>>>>>>> >>>>>>>>> broader >>>>>>> >>>>>>>> usefulness of this would be to serve as a mechanism to store a >>>>>>>>>> >>>>>>>>> java.util.Map >>>>>>>>> >>>>>>>>>> as a part of the OM Tree and perform XML operations (ex:- XPath) to >>>>>>>>>> >>>>>>>>> extract >>>>>>>>> >>>>>>>>>> data if needed. However, there can be situations where one would >>>>>>>>>> >>>>>>>>> require >>>>>>> >>>>>>>> to >>>>>>>>> >>>>>>>>>> serialize the internal Map payload and obtain an XML >>>>>>>>>> >>>>>>>>> representation. >>>>> >>>>>> This >>>>>>> >>>>>>>> can be achieved either through a custom serializer or through a >>>>>>>>>> >>>>>>>>> built-in >>>>>>> >>>>>>>> serializer that will convert the Map into an XML representation. I >>>>>>>>>> >>>>>>>>> have >>>>>>> >>>>>>>> as >>>>>>>>> >>>>>>>>>> of present added two serializers to the implementation. >>>>>>>>>> >>>>>>>>>> 1. A simple serializer i I wrote that can handle primitive types, >>>>>>>>>> >>>>>>>>> and >>>>> >>>>>> Maps >>>>>>>>> >>>>>>>>>> (supports hierarchical maps) >>>>>>>>>> 2. The Java XML encoder/decoder for beans java.beans.XMLEncoder / >>>>>>>>>> java.beans.XMLDecoder (Apache Harmony has an implementation of this >>>>>>>>>> >>>>>>>>> if >>>>> >>>>>> you >>>>>>>>> >>>>>>>>>> are interested in digging deeper into what happens, [1], [2]) >>>>>>>>>> >>>>>>>>>> Now, after having a word with Paul on this setup I decided to make >>>>>>>>>> >>>>>>>>> this >>>>>>> >>>>>>>> implementation more generic, and capable of supporting any type of >>>>>>>>>> >>>>>>>>> object >>>>>>> >>>>>>>> attached to the Map, which eventually drops the 1st implementation >>>>>>>>>> >>>>>>>>> above. >>>>>>> >>>>>>>> The second works fine, but, is a highly Java specific way of doing >>>>>>>>>> >>>>>>>>> things >>>>>>> >>>>>>>> (but there is another point here, java.util.Map is Java anyway so >>>>>>>>>> >>>>>>>>> this >>>>> >>>>>> might >>>>>>>>> >>>>>>>>>> not be an issue) and make no sense in a non-Java context, and can >>>>>>>>>> >>>>>>>>> also >>>>> >>>>>> be >>>>>>> >>>>>>>> memory consuming and inefficient. >>>>>>>>>> >>>>>>>>>> I have investigated the possibility to make use of, >>>>>>>>>> >>>>>>>>>> 3. org.apache.axis2.databinding.utils.BeanUtil >>>>>>>>>> - This is a sample source code portion that i used, >>>>>>>>>> >>>>>>>>>> XMLStreamReader xtr = BeanUtil.getPullParser(map); >>>>>>>>>> StAXOMBuilder builder = new StAXOMBuilder(xtr); >>>>>>>>>> OMElement ele = builder.getDocumentElement(); >>>>>>>>>> >>>>>>>>>> However, for some reason this doesn't work and I run into an NPE. >>>>>>>>>> >>>>>>>>>> org.apache.axiom.om.OMException: java.lang.NullPointerException >>>>>>>>>> at >>>>>>>>>> >>>>>>>>>> org.apache.axiom.om.impl.builder.StAXOMBuilder.next(StAXOMBuilder.java:251) >>>>> >>>>>> at >>>>>>>>>> >>>>>>>>>> org.apache.axiom.om.impl.llom.OMDocumentImpl.getOMDocumentElement(OMDocumentImpl.java:132) >>>>> >>>>>> at >>>>>>>>>> >>>>>>>>>> org.apache.axiom.om.impl.builder.StAXOMBuilder.getDocumentElement(StAXOMBuilder.java:526) >>>>> >>>>>> at my.package.MyClass.myMethod(MyClass.java:127) >>>>>>>>>> Caused by: java.lang.NullPointerException >>>>>>>>>> at >>>>>>>>>> >>>>>>>>>> org.apache.axiom.om.impl.builder.StAXOMBuilder.endElement(StAXOMBuilder.java:508) >>>>> >>>>>> at >>>>>>>>>> >>>>>>>>>> org.apache.axiom.om.impl.builder.StAXOMBuilder.next(StAXOMBuilder.java:222) >>>>> >>>>>> ... 35 more >>>>>>>>>> >>>>>>>>>> I spoke to Chinthaka on this matter, and was told that there >>>>>>>>>> >>>>>>>>> might >>>>> >>>>>> be >>>>>>> >>>>>>>> an >>>>>>>>> >>>>>>>>>> assumption that the BeanUtil can only handle Bean Classes, or >>>>>>>>>> >>>>>>>>> Classes >>>>> >>>>>> that >>>>>>>>> >>>>>>>>>> are not Maps, which might have lead to this situation. I believe it >>>>>>>>>> >>>>>>>>> wont >>>>>>> >>>>>>>> be >>>>>>>>> >>>>>>>>>> easy to fix these issues. This is the rationale: I might be able to >>>>>>>>>> >>>>>>>>> get >>>>>>> >>>>>>>> this >>>>>>>>> >>>>>>>>>> to work for java.util.Map, but the whole idea is to make use of it >>>>>>>>>> >>>>>>>>> to >>>>> >>>>>> serialize any type of object, where I can't anticipate the >>>>>>>>>> >>>>>>>>> stability. >>>>> >>>>>> 4. PayloadHelper in Apache Synapse >>>>>>>>>> This is a robust implementation that will work for primitive Maps >>>>>>>>>> >>>>>>>>> (based >>>>>>> >>>>>>>> on org.apache.synapse.util.SimpleMap) like option 1. above. >>>>>>>>>> >>>>>>>>> However, >>>>> >>>>>> it >>>>>>> >>>>>>>> lacks some aspects. >>>>>>>>>> a. It is still a part of Synapse and needs to be ported to Axiom >>>>>>>>>> >>>>>>>>> (this >>>>>>> >>>>>>>> is do-able as the system has clear and loosely coupled interfaces). >>>>>>>>>> b. It is an extension of HashMap and thus will not work with >>>>>>>>>> >>>>>>>>> other >>>>> >>>>>> Map >>>>>>> >>>>>>>> types, such as TreeMap which can be an issue when element ordering >>>>>>>>>> >>>>>>>>> comes >>>>>>> >>>>>>>> into play. >>>>>>>>>> c. It wont support Hierarchical Maps (please correct me if I >>>>>>>>>> >>>>>>>>> made a >>>>> >>>>>> mistake here). >>>>>>>>>> d. It still doesn't serve the purpose of supporting more generic >>>>>>>>>> >>>>>>>>> Maps >>>>>>> >>>>>>>> with any types of objects in it. >>>>>>>>>> >>>>>>>>>> 5. A serialization/de-serialization mechanism found in Axis1 seems >>>>>>>>>> interesting as well. >>>>>>>>>> - test/soap12/TestDeser.java, test/soap12/TestSer.java explains >>>>>>>>>> >>>>>>>>> this >>>>>>> >>>>>>>> fact. >>>>>>>>>> >>>>>>>>>> In here, we have several advantages >>>>>>>>>> a. Uniform representation of any primitive type as well as >>>>>>>>>> >>>>>>>>> complex >>>>> >>>>>> types >>>>>>>>> >>>>>>>>>> as composites of primitive types >>>>>>>>>> b. Good performance >>>>>>>>>> c. Ability to nest >>>>>>>>>> d. Highly customizable >>>>>>>>>> >>>>>>>>>> But, there are disadvantages >>>>>>>>>> a. This scheme is not capable of storing information about the >>>>>>>>>> underlying object unless it being explicitly told. Thus, unless we >>>>>>>>>> >>>>>>>>> know >>>>>>> >>>>>>>> what >>>>>>>>> >>>>>>>>>> is going on, the Vector class or an extension of a Vector class is >>>>>>>>>> represented in the very same way. This is not the case in the java >>>>>>>>>> serializer mechanism as object type information is automatically >>>>>>>>>> >>>>>>>>> encoded. >>>>>>> >>>>>>>> b. Assume that we came up with a modification to this scheme >>>>>>>>>> >>>>>>>>> that >>>>> >>>>>> makes >>>>>>>>> >>>>>>>>>> it possible to encode object types, still the implementor will have >>>>>>>>>> >>>>>>>>> to >>>>> >>>>>> perhaps write his own Type Table for a type that we did not >>>>>>>>>> >>>>>>>>> anticipate. >>>>>>> >>>>>>>> c. Implementation can be complicated as the complexity of the >>>>>>>>>> >>>>>>>>> types >>>>> >>>>>> of >>>>>>> >>>>>>>> objects representable increases >>>>>>>>>> d. Additional maintenance overhead >>>>>>>>>> >>>>>>>>>> Therefore, each scheme seem to have pros and cons, and are not >>>>>>>>>> >>>>>>>>> perfectly >>>>>>> >>>>>>>> fitting in. IMHO, the Java serializer might be the best scheme if >>>>>>>>>> >>>>>>>>> we >>>>> >>>>>> are >>>>>>> >>>>>>>> to >>>>>>>>> >>>>>>>>>> consider a single scheme. However, modifications to a certain >>>>>>>>>> >>>>>>>>> scheme >>>>> >>>>>> to >>>>>>> >>>>>>>> have >>>>>>>>> >>>>>>>>>> a combination of schemes to yield a useful result can prove to be >>>>>>>>>> advantages. Also, I might have missed some other possibilities. >>>>>>>>>> >>>>>>>>> Your >>>>> >>>>>> input >>>>>>>>> >>>>>>>>>> is highly appreciated, and will serve as means for the approach I >>>>>>>>>> >>>>>>>>> should >>>>>>> >>>>>>>> be >>>>>>>>> >>>>>>>>>> taking. >>>>>>>>>> >>>>>>>>>> The current implementation is not as yet a part of Axiom and is >>>>>>>>>> >>>>>>>>> available >>>>>>> >>>>>>>> at, [3]. The source includes a maven build system, and please note >>>>>>>>>> >>>>>>>>> that >>>>>>> >>>>>>>> if >>>>>>>>> >>>>>>>>>> you may run into some test failures due to an issue in the Axiom >>>>>>>>>> >>>>>>>>> forceExpand >>>>>>>>> >>>>>>>>>> logic. I'm looking forward to have this fixed on the Axiom trunk. >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> >>>>>>>>>> >>>>> http://svn.apache.org/repos/asf/harmony/enhanced/classlib/trunk/modules/beans/src/main/java/java/beans/XMLEncoder.java >>>>> >>>>>> [2] >>>>>>>>>> >>>>>>>>>> >>>>> http://svn.apache.org/repos/asf/harmony/enhanced/classlib/trunk/modules/beans/src/main/java/java/beans/XMLDeccoder.java >>>>> >>>>>> [3] http://sci-flex.googlecode.com/svn/sci-flex/trunk/java/axiom >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Senaka >>>>>>>>>> >>>>>>>>>> >> >> -- >> Sanjiva Weerawarana, Ph.D. >> Founder & Director; Lanka Software Foundation; http://www.opensource.lk/ >> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/ >> Member; Apache Software Foundation; http://www.apache.org/ >> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/ >> >> Blog: http://sanjiva.weerawarana.org/ >> >
