Hi Andreas, I did some modifications to the source and committed it minutes ago. My previous post to the thread shows a sample output. Seems that your last post and my last post were sent almost at the same time. :-).. So in addition to what I've said in the previous post, i have added some comments to this post, inline.
On Mon, Nov 10, 2008 at 12:06 AM, Andreas Veithen <[EMAIL PROTECTED] > wrote: > Senaka, > > I didn't execute the code yet, but I did a quick review and it looks > already very good. I would like to make the following comments to > improve this still further: Thanks, and I will add some tests for this code, shortly. I tweaked the present test source to observe the sample output, which I have not committed. > > * In WrappedTextNodeStreamReader, the character data is returned in > chunks in order to avoid loading the entire data into memory > (typically the data comes from a temporary file). I don't think that > this is necessary for the map values, and could even introduce > unnecessary overhead. They should simply be converted to a String and > returned as a single chunk. Getting rid of the java.io.Reader would > also simplify the code. Sounds logical. But, what made me go for this approach is that I assumed that at times a typical Map MIGHT have data that is too large to fit in memory. WDYT? > > * The right way to represent a byte[] value is to produce a > DataHandler (which is equivalent to having the binary data encoded as > Base64). Note that this is not directly supported by the StAX API, but > rather an extension introduced by AXIOM to handle binary data > efficiently. Please have a look at > StAXBuilder#createOMText(OMContainer, int) to see how this magic > works. Thanks for the pointer, I will try to add this logic as well. > > * For the moment the key of a map entry is represented using a "key" > attribute but also used for the element name. I guess this is a > mistake. Since a map key is not necessarily a valid XML element name, > I think we should prefer the representation using an attribute. I corrected this. Now, element names are "value", and the key is an attribute. The logic limits Map keys to types that can be represented as Strings, and an exception is thrown if it is of any other type. Regards, Senaka > > Regards, > > Andreas > > On Sun, Nov 9, 2008 at 11:40, Senaka Fernando <[EMAIL PROTECTED]> wrote: > > Andreas, > > > > I did go through your suggested implementation, and [1]'s what I'm > planning > > to do. Please do let me know whether I've made the correct choices. As of > > now, the getElementText() method is perhaps not quite correct and I have > not > > yet added a mechanism to represent a byte[]. > > > > [1] > > > http://sci-flex.googlecode.com/svn/sci-flex/trunk/java/axiom/src/main/java/org/apache/axiom/om/util/WrappedMapNodeStreamReader.java > > > > Regards, > > Senaka > > > > On Sat, Nov 8, 2008 at 1:36 AM, Sanjiva Weerawarana > > <[EMAIL PROTECTED]>wrote: > > > >> +1 Andreas. This should be written so that the OM is created IFF XML > >> navigation is done. Otherwise the map message should remain in Java and > then > >> just get piped thru - that's critical for Synapse performance. > >> > >> Sanjiva. > >> > >> > >> Andreas Veithen wrote: > >> > >>> Senaka, > >>> > >>> The AXIOM tree is built twice because of the following piece of code: > >>> > >>> public XMLStreamReader getReader() throws XMLStreamException { > >>> return getUnderlyingElement().getXMLStreamReader(); > >>> } > >>> > >>> The getUnderlyingElement method will build an AXIOM tree representing > >>> the Map(Message), but when the OMSourcedElement is expanded, AXIOM > >>> will build another tree based on the events pulled from the > >>> XMLStreamReader. There are two options then: > >>> > >>> 1. One considers that in the vast majority of cases, the content will > >>> be accessed anyway. Then it would make more sense to construct the > >>> AXIOM tree directly when the message is received (i.e. no need for an > >>> OMSourcedElement). > >>> 2. Don't build an AXIOM tree inside the OMDataSource but construct an > >>> XMLStreamReader implementation that returns the sequence of StAX > >>> events corresponding to the desired XML representation. > >>> > >>> I used the technique behind option 2 in the following piece of code: > >>> > >>> > >>> > http://svn.apache.org/repos/asf/synapse/trunk/java/modules/core/src/main/java/org/apache/synapse/util/WrappedTextNodeStreamReader.java > >>> > >>> The XMLStreamReader implementation shown in this link is used to > >>> transform character data (provided by a java.io.Reader) into an > >>> OMSourcedElement that wraps this data, i.e. the resulting tree would > >>> be an element with a text node as child. That doesn't sound very > >>> useful at first glance, but in case of very long character data, it > >>> allows to stream the data almost directly from the source to the > >>> destination without ever building the OMText nodes (which would > >>> consume a large amount of memory). > >>> > >>> Given that reading the data source is non destructive, option 2 has > >>> the advantage that the AXIOM tree > >>> * will be built exactly once if somebody queries the child OMNodes; > >>> * will not be built at all when somebody serializes the content into a > >>> byte stream. > >>> > >>> While this is the optimal solution, it is also much more difficult to > >>> implement. It is certainly an interesting challenge to do that. > >>> > >>> Finally, for the type problem, it is indeed sufficient to add a "type" > >>> attribute to the element that represents the key-value pair: > >>> > >>> <price type="double">12.456</price> > >>> > >>> Andreas > >>> > >>> On Thu, Oct 30, 2008 at 10:34, Senaka Fernando <[EMAIL PROTECTED]> > >>> wrote: > >>> > >>>> Hi Andreas, > >>>> I agree with your observations here. Also, I would like to understand > >>>> what > >>>> you mean by "it will build the AXIOM tree twice when the content > >>>> is accessed", can this be corrected? As far as Map Messages found in > the > >>>> jms > >>>> transport are concerned, the key is of type string, and the value is a > >>>> primitive java type, I believe that a slight modification option 2 > >>>> discussed > >>>> here should work. WDYT? > >>>> > >>>> Regards, > >>>> Senaka > >>>> > >>>> On Thu, Oct 30, 2008 at 2:10 AM, Andreas Veithen > >>>> <[EMAIL PROTECTED]>wrote: > >>>> > >>>> Having alternative strategies that map between MapMessages and XML > >>>>> might be interesting, but to start with we should have at least one > >>>>> implementation that meets all of the following requirements: > >>>>> > >>>>> 1. Highly optimized and having the least possible overhead (even if > >>>>> the AXIOM tree is build). > >>>>> 2. The XML representation must be simple so that it can be easily > used > >>>>> with XSLT and XPath. > >>>>> 3. The mapping must be two way and lossless. That is important if you > >>>>> want to switch from JMS to another protocol and then back again to > >>>>> JMS. > >>>>> > >>>>> In my opinion, the XMLEncoder based solution doesn't satisfy the > first > >>>>> two requirements, but will meet the last one. > >>>>> > >>>>> The other implementation you propose > >>>>> - partially satisfies requirement 1 (partially because - as far as I > >>>>> can see - it will build the AXIOM tree twice when the content is > >>>>> accessed); > >>>>> - satisfies requirement 2; > >>>>> - doesn't satisfy requirement 3 because it looses information about > >>>>> the property types, i.e. you will not be able to recreate an > >>>>> equivalent MapMessage from the XML representation. > >>>>> > >>>>> Andreas > >>>>> > >>>>> > >>>>> On Tue, Oct 28, 2008 at 04:44, Senaka Fernando <[EMAIL PROTECTED]> > >>>>> wrote: > >>>>> > >>>>>> Hi Andreas, > >>>>>> > >>>>>> The scenario here was to have an implementation that will support > Map > >>>>>> Messages "as well as" hierarchical Maps, and any generic use of Maps > >>>>>> with > >>>>>> OM. And as you have mentioned here Map Messages can only have > primitive > >>>>>> types on it. Therefore, in theory MapMessage support would only > require > >>>>>> a > >>>>>> subset of provisions made by this implementation. > >>>>>> > >>>>>> Also, if you have tried the implementation I have at the moment, it > >>>>>> > >>>>> supports > >>>>> > >>>>>> alternative strategies (so you may use whatever type of serializer > you > >>>>>> want). > >>>>>> > >>>>>> Regards, > >>>>>> Senaka > >>>>>> > >>>>>> On Tue, Oct 28, 2008 at 5:34 AM, Andreas Veithen > >>>>>> <[EMAIL PROTECTED]>wrote: > >>>>>> > >>>>>> Senaka, > >>>>>>> > >>>>>>> How does your question actually relate to the MapMessage support > you > >>>>>>> are working on? AFAIK MapMessages can't contain arbitrary Java > >>>>>>> objects. > >>>>>>> > >>>>>>> Andreas > >>>>>>> > >>>>>>> On Sun, Oct 26, 2008 at 22:19, Senaka Fernando < > [EMAIL PROTECTED]> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Hi Andreas, > >>>>>>>> > >>>>>>>> Here you go: > >>>>>>>> > >>>>>>>> <map> > >>>>>>>> <java version="1.6.0_06" class="java.beans.XMLDecoder"> > >>>>>>>> <object class="java.util.TreeMap"> > >>>>>>>> <void method="put"> > >>>>>>>> <string>KeyStr</string> > >>>>>>>> <string>five</string> > >>>>>>>> </void> > >>>>>>>> <void method="put"> > >>>>>>>> <string>Test</string> > >>>>>>>> <float>5.5</float> > >>>>>>>> </void> > >>>>>>>> <void method="put"> > >>>>>>>> <string>SomeKey</string> > >>>>>>>> <int>5</int> > >>>>>>>> </void> > >>>>>>>> <void method="put"> > >>>>>>>> <string>nested</string> > >>>>>>>> <object class="java.util.TreeMap"> > >>>>>>>> <void method="put"> > >>>>>>>> <string>me</string> > >>>>>>>> <float>2.0</float> > >>>>>>>> </void> > >>>>>>>> <void method="put"> > >>>>>>>> <string>more</string> > >>>>>>>> <int>100</int> > >>>>>>>> </void> > >>>>>>>> <void method="put"> > >>>>>>>> <string>moreNested</string> > >>>>>>>> <object class="java.util.TreeMap"> > >>>>>>>> <void method="put"> > >>>>>>>> <string>String</string> > >>>>>>>> <string>ten</string> > >>>>>>>> </void> > >>>>>>>> </object> > >>>>>>>> </void> > >>>>>>>> </object> > >>>>>>>> </void> > >>>>>>>> </object> > >>>>>>>> </java> > >>>>>>>> </map> > >>>>>>>> > >>>>>>>> This is the serialization for a TreeMap having {<KeyStr, five>, > >>>>>>>> <Test, > >>>>>>>> > >>>>>>> 5.5>, > >>>>>>> > >>>>>>>> <someKey, 5>, <nested, {<me, 2.0>, <more, 100>, <moreNested, > >>>>>>>> {<String, > >>>>>>>> ten>}>}>} > >>>>>>>> > >>>>>>>> Regards, > >>>>>>>> Senaka > >>>>>>>> > >>>>>>>> On Mon, Oct 27, 2008 at 1:52 AM, Andreas Veithen > >>>>>>>> <[EMAIL PROTECTED]>wrote: > >>>>>>>> > >>>>>>>> Senaka, > >>>>>>>>> > >>>>>>>>> Just a quick question: what does the serialization of a Map looks > >>>>>>>>> > >>>>>>>> like > >>>>> > >>>>>> with XMLEncoder? > >>>>>>>>> > >>>>>>>>> Andreas > >>>>>>>>> > >>>>>>>>> On Sat, Oct 25, 2008 at 20:01, Senaka Fernando < > [EMAIL PROTECTED] > >>>>>>>>> > > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Hi all, > >>>>>>>>>> > >>>>>>>>>> I'm working on a mechanism to attach a java.util.Map onto an > Axiom > >>>>>>>>>> > >>>>>>>>> Tree. > >>>>>>> > >>>>>>>> So > >>>>>>>>> > >>>>>>>>>> far, I have been able to attach the java.util.Map onto the OM > Tree > >>>>>>>>>> > >>>>>>>>> with > >>>>>>> > >>>>>>>> the > >>>>>>>>> > >>>>>>>>>> help of a specialized data source I have created. This > >>>>>>>>>> > >>>>>>>>> implementation > >>>>> > >>>>>> features on-demand building of the XML payload and I believe the > >>>>>>>>>> > >>>>>>>>> broader > >>>>>>> > >>>>>>>> usefulness of this would be to serve as a mechanism to store a > >>>>>>>>>> > >>>>>>>>> java.util.Map > >>>>>>>>> > >>>>>>>>>> as a part of the OM Tree and perform XML operations (ex:- XPath) > to > >>>>>>>>>> > >>>>>>>>> extract > >>>>>>>>> > >>>>>>>>>> data if needed. However, there can be situations where one would > >>>>>>>>>> > >>>>>>>>> require > >>>>>>> > >>>>>>>> to > >>>>>>>>> > >>>>>>>>>> serialize the internal Map payload and obtain an XML > >>>>>>>>>> > >>>>>>>>> representation. > >>>>> > >>>>>> This > >>>>>>> > >>>>>>>> can be achieved either through a custom serializer or through a > >>>>>>>>>> > >>>>>>>>> built-in > >>>>>>> > >>>>>>>> serializer that will convert the Map into an XML representation. > I > >>>>>>>>>> > >>>>>>>>> have > >>>>>>> > >>>>>>>> as > >>>>>>>>> > >>>>>>>>>> of present added two serializers to the implementation. > >>>>>>>>>> > >>>>>>>>>> 1. A simple serializer i I wrote that can handle primitive > types, > >>>>>>>>>> > >>>>>>>>> and > >>>>> > >>>>>> Maps > >>>>>>>>> > >>>>>>>>>> (supports hierarchical maps) > >>>>>>>>>> 2. The Java XML encoder/decoder for beans java.beans.XMLEncoder > / > >>>>>>>>>> java.beans.XMLDecoder (Apache Harmony has an implementation of > this > >>>>>>>>>> > >>>>>>>>> if > >>>>> > >>>>>> you > >>>>>>>>> > >>>>>>>>>> are interested in digging deeper into what happens, [1], [2]) > >>>>>>>>>> > >>>>>>>>>> Now, after having a word with Paul on this setup I decided to > make > >>>>>>>>>> > >>>>>>>>> this > >>>>>>> > >>>>>>>> implementation more generic, and capable of supporting any type > of > >>>>>>>>>> > >>>>>>>>> object > >>>>>>> > >>>>>>>> attached to the Map, which eventually drops the 1st > implementation > >>>>>>>>>> > >>>>>>>>> above. > >>>>>>> > >>>>>>>> The second works fine, but, is a highly Java specific way of > doing > >>>>>>>>>> > >>>>>>>>> things > >>>>>>> > >>>>>>>> (but there is another point here, java.util.Map is Java anyway so > >>>>>>>>>> > >>>>>>>>> this > >>>>> > >>>>>> might > >>>>>>>>> > >>>>>>>>>> not be an issue) and make no sense in a non-Java context, and > can > >>>>>>>>>> > >>>>>>>>> also > >>>>> > >>>>>> be > >>>>>>> > >>>>>>>> memory consuming and inefficient. > >>>>>>>>>> > >>>>>>>>>> I have investigated the possibility to make use of, > >>>>>>>>>> > >>>>>>>>>> 3. org.apache.axis2.databinding.utils.BeanUtil > >>>>>>>>>> - This is a sample source code portion that i used, > >>>>>>>>>> > >>>>>>>>>> XMLStreamReader xtr = BeanUtil.getPullParser(map); > >>>>>>>>>> StAXOMBuilder builder = new StAXOMBuilder(xtr); > >>>>>>>>>> OMElement ele = builder.getDocumentElement(); > >>>>>>>>>> > >>>>>>>>>> However, for some reason this doesn't work and I run into an > NPE. > >>>>>>>>>> > >>>>>>>>>> org.apache.axiom.om.OMException: java.lang.NullPointerException > >>>>>>>>>> at > >>>>>>>>>> > >>>>>>>>>> > org.apache.axiom.om.impl.builder.StAXOMBuilder.next(StAXOMBuilder.java:251) > >>>>> > >>>>>> at > >>>>>>>>>> > >>>>>>>>>> > org.apache.axiom.om.impl.llom.OMDocumentImpl.getOMDocumentElement(OMDocumentImpl.java:132) > >>>>> > >>>>>> at > >>>>>>>>>> > >>>>>>>>>> > org.apache.axiom.om.impl.builder.StAXOMBuilder.getDocumentElement(StAXOMBuilder.java:526) > >>>>> > >>>>>> at my.package.MyClass.myMethod(MyClass.java:127) > >>>>>>>>>> Caused by: java.lang.NullPointerException > >>>>>>>>>> at > >>>>>>>>>> > >>>>>>>>>> > org.apache.axiom.om.impl.builder.StAXOMBuilder.endElement(StAXOMBuilder.java:508) > >>>>> > >>>>>> at > >>>>>>>>>> > >>>>>>>>>> > org.apache.axiom.om.impl.builder.StAXOMBuilder.next(StAXOMBuilder.java:222) > >>>>> > >>>>>> ... 35 more > >>>>>>>>>> > >>>>>>>>>> I spoke to Chinthaka on this matter, and was told that there > >>>>>>>>>> > >>>>>>>>> might > >>>>> > >>>>>> be > >>>>>>> > >>>>>>>> an > >>>>>>>>> > >>>>>>>>>> assumption that the BeanUtil can only handle Bean Classes, or > >>>>>>>>>> > >>>>>>>>> Classes > >>>>> > >>>>>> that > >>>>>>>>> > >>>>>>>>>> are not Maps, which might have lead to this situation. I believe > it > >>>>>>>>>> > >>>>>>>>> wont > >>>>>>> > >>>>>>>> be > >>>>>>>>> > >>>>>>>>>> easy to fix these issues. This is the rationale: I might be able > to > >>>>>>>>>> > >>>>>>>>> get > >>>>>>> > >>>>>>>> this > >>>>>>>>> > >>>>>>>>>> to work for java.util.Map, but the whole idea is to make use of > it > >>>>>>>>>> > >>>>>>>>> to > >>>>> > >>>>>> serialize any type of object, where I can't anticipate the > >>>>>>>>>> > >>>>>>>>> stability. > >>>>> > >>>>>> 4. PayloadHelper in Apache Synapse > >>>>>>>>>> This is a robust implementation that will work for primitive > Maps > >>>>>>>>>> > >>>>>>>>> (based > >>>>>>> > >>>>>>>> on org.apache.synapse.util.SimpleMap) like option 1. above. > >>>>>>>>>> > >>>>>>>>> However, > >>>>> > >>>>>> it > >>>>>>> > >>>>>>>> lacks some aspects. > >>>>>>>>>> a. It is still a part of Synapse and needs to be ported to > Axiom > >>>>>>>>>> > >>>>>>>>> (this > >>>>>>> > >>>>>>>> is do-able as the system has clear and loosely coupled > interfaces). > >>>>>>>>>> b. It is an extension of HashMap and thus will not work with > >>>>>>>>>> > >>>>>>>>> other > >>>>> > >>>>>> Map > >>>>>>> > >>>>>>>> types, such as TreeMap which can be an issue when element > ordering > >>>>>>>>>> > >>>>>>>>> comes > >>>>>>> > >>>>>>>> into play. > >>>>>>>>>> c. It wont support Hierarchical Maps (please correct me if I > >>>>>>>>>> > >>>>>>>>> made a > >>>>> > >>>>>> mistake here). > >>>>>>>>>> d. It still doesn't serve the purpose of supporting more > generic > >>>>>>>>>> > >>>>>>>>> Maps > >>>>>>> > >>>>>>>> with any types of objects in it. > >>>>>>>>>> > >>>>>>>>>> 5. A serialization/de-serialization mechanism found in Axis1 > seems > >>>>>>>>>> interesting as well. > >>>>>>>>>> - test/soap12/TestDeser.java, test/soap12/TestSer.java > explains > >>>>>>>>>> > >>>>>>>>> this > >>>>>>> > >>>>>>>> fact. > >>>>>>>>>> > >>>>>>>>>> In here, we have several advantages > >>>>>>>>>> a. Uniform representation of any primitive type as well as > >>>>>>>>>> > >>>>>>>>> complex > >>>>> > >>>>>> types > >>>>>>>>> > >>>>>>>>>> as composites of primitive types > >>>>>>>>>> b. Good performance > >>>>>>>>>> c. Ability to nest > >>>>>>>>>> d. Highly customizable > >>>>>>>>>> > >>>>>>>>>> But, there are disadvantages > >>>>>>>>>> a. This scheme is not capable of storing information about the > >>>>>>>>>> underlying object unless it being explicitly told. Thus, unless > we > >>>>>>>>>> > >>>>>>>>> know > >>>>>>> > >>>>>>>> what > >>>>>>>>> > >>>>>>>>>> is going on, the Vector class or an extension of a Vector class > is > >>>>>>>>>> represented in the very same way. This is not the case in the > java > >>>>>>>>>> serializer mechanism as object type information is automatically > >>>>>>>>>> > >>>>>>>>> encoded. > >>>>>>> > >>>>>>>> b. Assume that we came up with a modification to this scheme > >>>>>>>>>> > >>>>>>>>> that > >>>>> > >>>>>> makes > >>>>>>>>> > >>>>>>>>>> it possible to encode object types, still the implementor will > have > >>>>>>>>>> > >>>>>>>>> to > >>>>> > >>>>>> perhaps write his own Type Table for a type that we did not > >>>>>>>>>> > >>>>>>>>> anticipate. > >>>>>>> > >>>>>>>> c. Implementation can be complicated as the complexity of the > >>>>>>>>>> > >>>>>>>>> types > >>>>> > >>>>>> of > >>>>>>> > >>>>>>>> objects representable increases > >>>>>>>>>> d. Additional maintenance overhead > >>>>>>>>>> > >>>>>>>>>> Therefore, each scheme seem to have pros and cons, and are not > >>>>>>>>>> > >>>>>>>>> perfectly > >>>>>>> > >>>>>>>> fitting in. IMHO, the Java serializer might be the best scheme if > >>>>>>>>>> > >>>>>>>>> we > >>>>> > >>>>>> are > >>>>>>> > >>>>>>>> to > >>>>>>>>> > >>>>>>>>>> consider a single scheme. However, modifications to a certain > >>>>>>>>>> > >>>>>>>>> scheme > >>>>> > >>>>>> to > >>>>>>> > >>>>>>>> have > >>>>>>>>> > >>>>>>>>>> a combination of schemes to yield a useful result can prove to > be > >>>>>>>>>> advantages. Also, I might have missed some other possibilities. > >>>>>>>>>> > >>>>>>>>> Your > >>>>> > >>>>>> input > >>>>>>>>> > >>>>>>>>>> is highly appreciated, and will serve as means for the approach > I > >>>>>>>>>> > >>>>>>>>> should > >>>>>>> > >>>>>>>> be > >>>>>>>>> > >>>>>>>>>> taking. > >>>>>>>>>> > >>>>>>>>>> The current implementation is not as yet a part of Axiom and is > >>>>>>>>>> > >>>>>>>>> available > >>>>>>> > >>>>>>>> at, [3]. The source includes a maven build system, and please > note > >>>>>>>>>> > >>>>>>>>> that > >>>>>>> > >>>>>>>> if > >>>>>>>>> > >>>>>>>>>> you may run into some test failures due to an issue in the Axiom > >>>>>>>>>> > >>>>>>>>> forceExpand > >>>>>>>>> > >>>>>>>>>> logic. I'm looking forward to have this fixed on the Axiom > trunk. > >>>>>>>>>> > >>>>>>>>>> [1] > >>>>>>>>>> > >>>>>>>>>> > >>>>> > http://svn.apache.org/repos/asf/harmony/enhanced/classlib/trunk/modules/beans/src/main/java/java/beans/XMLEncoder.java > >>>>> > >>>>>> [2] > >>>>>>>>>> > >>>>>>>>>> > >>>>> > http://svn.apache.org/repos/asf/harmony/enhanced/classlib/trunk/modules/beans/src/main/java/java/beans/XMLDeccoder.java > >>>>> > >>>>>> [3] http://sci-flex.googlecode.com/svn/sci-flex/trunk/java/axiom > >>>>>>>>>> > >>>>>>>>>> Thanks, > >>>>>>>>>> Senaka > >>>>>>>>>> > >>>>>>>>>> > >> > >> -- > >> Sanjiva Weerawarana, Ph.D. > >> Founder & Director; Lanka Software Foundation; > http://www.opensource.lk/ > >> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/ > >> Member; Apache Software Foundation; http://www.apache.org/ > >> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/ > >> > >> Blog: http://sanjiva.weerawarana.org/ > >> > > >
