Hi Anshuk, Thanks for digging through the emails and finding this earlier discussion, and for linking it in this thread. You can actually do what you (and Luca) want much more easily than in the earlier example, though. The trick is to treat your repeating objects as a "virtual" collection (meaning it's not physically present in memory, or at least not in the form being used) rather than a real collection.
The binding <collection> element lets you define a collection in several ways. It's most common to just use a java.util.List or an array with the <collection>, but you can instead just specify methods to be used for accessing the values directly - either size-method and load-method, or iter-method for marshalling, and either store-method or add-method for marshalling. If you know the number of results you want to return the size-method/load-method combination give you an easy way to handle marshalling a virtual collection; if you don't know the number of results you can instead use iter-method and construct your own Iterator implementation to handle providing the values for marshalling. add-method works fine directly for unmarshalling a virtual collection - you just get the values in order, one at a time. Either way, JiBX does not hold a reference to the object (well, not unless you're using ID links, anyway) once it's done marshalling/unmarshalling, so you can work with collections much larger than physical memory. - Dennis Dennis M. Sosnoski SOA and Web Services in Java Training and Consulting http://www.sosnoski.com - http://www.sosnoski.co.nz Seattle, WA +1-425-939-0576 - Wellington, NZ +64-4-298-6117 Anshuk Pal Chaudhuri wrote: > > Hi, > > > > I think I have got some light and help in one of the earlier posts, > and I think ideally it should work out while marshalling the objects > to XML. This link > http://www.mail-archive.com/jibx-users@lists.sourceforge.net/msg02202.html > does provide some good pointers for what I am trying to achieve. > > > > Thanks. > > > > Regards, > > Anshuk Pal Chaudhuri > > > > *From:* Luca Buraggi [mailto:lbura...@mate.it] > *Sent:* Wednesday, May 13, 2009 2:09 PM > *To:* JiBX users > *Subject:* Re: [jibx-users] Object Streaming using JIBX > > > > Hi, > I also am facing with a similar problem. More precisely it is the > symmetric problem: I must unmarshall some huge (sometimes greater than > 500 Mbytes) XML files composed by a sequence of the same complex > element. Something similar to this: > --- > <list> > <element> ... </element> > <element> ... </element> > <element> ... </element> > ... > ... > </list> > --- > While unmarshalling each single <element> is perfectly feasible, any > attempt to unmarshall the whole file fails due to lack of heap space. > I am not an expert JIBX user, I am still exploring the framework. But > I examined other popular XML binding framework (JAXB and Castor) and I > think JIBX architecture is better and more powerful if applied to such > a task. > I am now prototyping and testing a solution based on using the API of > the UnmarshallingContext class (I can't use the IUnmarshallingContext > interface because it hides some methods that are needed, and this is > definitely not an optimal solution, but I have no chance) > This is the scenario: > --- > IBindingFactory bfact = BindingDirectory.getFactory(Element.class); > // Very ugly, but ... > UnmarshallingContext uctx = (UnmarshallingContext) > bfact.createUnmarshallingContext(); > uctx.setDocument(new FileInputStream("HugeDataFile.xml"), null); > > boolean done = false; > // This loop scans the whole input data file > while (done == false) { > if (uctx.isAt(null, "element") { > // The current event is a <element> start tag > Element el = (Element) uctx.unmarshalElement(); > // Now process the el object > ... > ... > } else { > // End of file ? > if (uctx.next() == IXMLReader.END_DOCUMENT) { > done = true; > } > } > } > --- > > I also will of course appreciate any suggestion or comment ... > > Regards, > > Luca Buraggi > > ------------------------------------ > Anshuk Pal Chaudhuri ha scritto: > > Hi, > > > > I have been working on jibx for quite some time and really appreciate > the philosophy went into building it and how the byte code > enhancements helps in faster marshalling of the objects into XML. I > was thinking to evangelize more into it and was trying to figure out > how could the java objects be marshaled/streamed one at a time rather > than marshalling the whole list of objects. I have defined the > scenario details in the below section: > > > > A query is fetched to hit the database which would fetch say 10000 > records (in some defined batches say 500); each record when fetched is > populated into a custom object (some custom value object). Rather than > setting the objects (10000 in this case) into a parent object’s some > list or some array list, I was thinking whether I can marshal the > custom value object straight way into the xml and as more records are > fetched, every time the populated object is marshaled into the same xml. > > So in a way, the complete list of objects (one big parent object) > which could have been passed into the jibx marshal only in one time > (which would have consume more memory), every a single object would > be marshaled and append in the same file every time it is being > fetched from the object. > > > > I think there are some apis which can be used to achieve something > like the above scenario; like *IMarshallingContext. startDocument, > IXMLWriter. startTagClosed, IXMLWriter. endTag* > > > > Can anybody let me know your thoughts on this. > > > > Any help is appreciated. > > > > Regards, > > Anshuk Pal Chaudhuri > > **************** CAUTION - Disclaimer ***************** > This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely > for the use of the addressee(s). If you are not the intended recipient, > please > notify the sender by e-mail and delete the original message. Further, you are > not > to copy, disclose, or distribute this e-mail or its contents to any other > person and > any such actions are unlawful. This e-mail may contain viruses. Infosys has > taken > every reasonable precaution to minimize this risk, but is not liable for any > damage > you may sustain as a result of any virus in this e-mail. You should carry out > your > own virus checks before opening the e-mail or attachment. Infosys reserves > the > right to monitor and review the content of all messages sent to or from this > e-mail > address. Messages sent to or from this e-mail address may be stored on the > Infosys e-mail system. > ***INFOSYS******** End of Disclaimer ********INFOSYS*** > > > ------------------------------------------------------------------------ > > > > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > > ------------------------------------------------------------------------ > > > > > _______________________________________________ > jibx-users mailing list > jibx-users@lists.sourceforge.net <mailto:jibx-users@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/jibx-users > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > ------------------------------------------------------------------------ > > _______________________________________________ > jibx-users mailing list > jibx-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/jibx-users > ------------------------------------------------------------------------------ Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://www.creativitycat.com _______________________________________________ jibx-users mailing list jibx-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jibx-users
