[dom4j-user] Large XML Parsing in a non-forward way

[João Tiago Ferreira]

Hi

I need to parse a large XML file and need to insert its contents to a database. 
Suppose the xml has the following structure.



Lots of  entities


Lots of  entities


Lots of  entities



Since I need to import this to a DB and there are referencial dependencies 
between the entities (let's say I have to insert A,B,C in this order) and I 
can't guarantee that the nodes appear in the correct order, what would be the 
"best" approach? 

I'm currently using the suggestion for large files 
(http://dom4j.sourceforge.net/dom4j-1.6.1/faq.html#large-doc) on the assumption 
that the nodes appear in the correct order.

I was thinking if I could order the nodes with a transformation, but I'm not 
sure if I can do it, or if it is efficient.


Thanks in advance
João Tiago Ferreira



--
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
___
dom4j-user mailing list
dom4j-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dom4j-user

[dom4j-user] large xml parsing

[Kamalakar Anala]

>
> Hi,
>
> I am working on a task where i need  to parse a large xml file (230  MB)
> and need to save into database. All the values are attributes so i am using
> the  below code to parse  and pull out attributes from child nodes and then
> save it using hibernate session.save and as  they are part of content and
> are not accesible to hibernate mapping if i put the direct
> path(//childelement/@attribute)



> Is the below correct/best way? of doing it for now its working fine but for
> large no of rows it  slows down considerably even though initially its runs
> very fast  5k per min
> After going thru some threads i doubt if its the detach method that is
> taking up time and memory or may be since i am adding up all the attributes
> to the root element is that causing te spike in javax memory
> Please let me know your thoughts.
>
> public class ParseLargeXML {
>
> public static void main(String args[])
> {
> try
> {
>
> ParseLargeXML plXML = new ParseLargeXML();
>  plXML.parseLargeXML();
> }
>
>protected List loanList = null;
> protected Session dom4jSession;
> protected Session session;
> protected Transaction tx;
>
>
> public void parseLargeXML() throws DocumentException, Exception
> {
>
> SAXReader reader = new SAXReader();
>
> try {
> session =
> HibernateUtil.getSessionFactory().openSession();
> }
>  catch(Exception e)
>  {
>  e.printStackTrace();
>  }
>
> reader.addHandler( "/LoanData/Loan",
> new ElementHandler() {
>
> public void onStart(ElementPath path) {
> // do nothing  count++
> }
>
> public void onEnd(ElementPath path) {
>
>   try {
>
>Element row = path.getCurrent();
>
>   Session dom4jSession =
> session.getSession(EntityMode.DOM4J);
>   tx = session.beginTransaction();
>   Document loandoc = row.getDocument();
>
>  loanList = loandoc.selectNodes("//Loan");
>  Element e = (Element) loanList.get(0);
>
> //parse and add child node Attributes for
> dom4jsession element e
>
>  e.addAttribute("CurntActivityPeriod","row
> .element("childelement").attributeValue(qname ));
>  .
>  .
>  . upto 100 attributes i am adding
> dom4jSession.saveOrUpdate("Loan", e);
>


> row.detach();
> loanList = null;
>
//batch insert

>   if ((count % 50) ==0) {
>
> if (tx != null)
> tx.commit();
> }
> }
> }
> );
>
> Document doc = reader.read(new
> File("c://largeDocument.xml"));
>
> }
>
--
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference___
dom4j-user mailing list
dom4j-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dom4j-user