I made an experimental SAX filter some time ago. This might help you!
Instead of getting the result as a w3c document, you can make your own contenthandler that makes the file or what ever you want to do!
From my first presentation of the filter:
------------------------------------------------------------------------------------------------
I have made an experimental SAX XMLFilter. It allows you to "filter" out the information in an xml document that you want to work with - using xpath - and skip the rest. You can place the filter anywhere in your application where a XMLFilter can be used.
- I don't know if this has already been done by others?
The whole idea is to "filter" out the fragments from the xml document that you specifies using an xpath expression. ex. SaxXPathFragmentFilter(saxparser, "/cellphone/*/[EMAIL PROTECTED]'1234']", "result"). Build a dom tree from the result, or why not feed the sax event into a xslt transformer and do some xslt transformations.
The big win is that you don't have to build a large dom tree, if you only needs part of the information in a large xml document. You just specify what fragments you want using xpath and the result will be a much smaller dom tree, witch requires less processing, memory etc.
Let us say that you have a large document with spare parts to Volvo vehicles. You want to do a list of engine parts for the S80 car model. What you do is specify the xpath (locationpath) that you want to cut out from the document ex. "/catalog/cars/s70/parts/engine".
// your sax parser here
XMLReader parser =
XMLReaderFactory.createXMLReader(
"org.apache.xerces.parsers.SAXParser");
// Get instances of your handlers
SAXHandler jdomsaxhandler = new SAXHandler();String xpath = "/catalog/cars/s70/parts/engine";
String rootName = "s70engineparts"; // this will be the new root.
// set SaxXPathFragmentFilter
SaxXPathFragmentFilter xpathfilter =
new SaxXPathFragmentFilter(parser, xpath, resultrootname);
xpathfilter.setContentHandler(jdomsaxhandler);
// Parse the document
xpathfilter.parse(uri);
// get the Document
Document doc = jdomsaxhandler.getDocument();This SaxXPathFragmentFilter is pure experimental. It is spaghetti code. I just sat down with an idea and started to code, and the code is not very pretty. It needs to be rewritten.
The xpath support is very limited for now. Here is the xpath you can do today with this filter:
"/a/b" - An absolute path.
"/a/*/c" - An absolute path but where element no 2 "*" could be any element.
"/a/*/[EMAIL PROTECTED]'value']" - If element c has an attribute with 'value'.
"/a/*/c[contains='value']" - If element c first child node is a text node that contains 'value'.
"/a/*/c[starts-with='value']" - If element c first child node is a text node that starts with 'value'.
"/a/*/c[ends-with='value']" - If element c first child node is a text node that ends with 'value'.
"/a/*/c['value']" - If element c first child node is a text node that is 'value'.
"/a/*/c[is='value']" - As above.
As you can see the xpath options is very limited. But I think that when I find a way to implement the "//" pattern, the filter will be even more powerful.
I have problems with building a dom tree from the result using xerces and saxon. But with jdom it works great. This needs to be fixed.
You can not rely on that the result is allways correct, so don't use this in any application, just use if for expermentation.
You can find the code at: http://www.npstrandberg.com/projects/saxxpathfragmentfilter/saxxpathfragmentfilter.
tar.gz
My goal with this filter is to keep it realiable, simple, fast and clean. If you want to contribute to this project, then you will be wellcome. The filter will be realeased under som kind of opensource license (if we get that fare!).
Test it an give me some feedback, on what you think.
-----------------------------------------------------------------------------------------------
Regards, Niels Peter Strandberg
On onsdag, februari 6, 2002, at 04:44 , Raghavendra, Karthik wrote:
Hi,
I have a XML file containing approximately 14800 product catalog information records and a file size of 80 MB. I am using Xerces 1.4.4 to validate and parse the XML document and create a DOM tree. I am using a Solaris box with 2 GB of RAM and so memory is not an issue. I am using the XPath capabilities of Xalan 2 to traverse the DOM tree and access required data. I am noticing that the performance degrades over time. For example, initial processing (first hundred records) returns within 800 milliseconds while the time for the last set of records (14000 and above) is about 77 seconds. I am using the CachedXPathAPI instead of the XPathAPI since the XPathAPI resulted in significantly greater response times.
The code snippets below:
Main Method: ------------- DOMParser parser = new DOMParser(); parser.parse(XMLFile); Document doc = parser.getDocument(); NodeList list = doc.getElementsByTagName("PRODUCT"); CachedXPathAPI xPath = new CachedXPathAPI(); Node prdNode = null;
for (int index=0;index<list.getlength();index++) { prdNode = list.item(index); processProduct(prdNode,xPath); ... ... }
ProcessProduct Method (this method uses XPath to get the relevant information. The product node for the current product is passed as a parameter): ------------------------------------------------------------------------ ------------------------------------------------------------------------ String prdNum = getText(xPath.selectSingleNode(prdNode,"child::ProductNumber")); String title = getText(xPath.selectSingleNode(prdNode,"child::Title")); String language = getText(xPath.selectSingleNode(prdNode,"child::Language")); String publisher = getText(xPath.selectSingleNode(prdNode,"child::Rights/child::Publisher") ); ... ... ...
getText Method (this method concatenates all the text node information and returns it): ------------------------------------------------------------------------ ------------- StringBuffer text = new StringBuffer(); NodeList list = node.getChildNodes(); for (int index=0;index < list.getLength();index++) { if (list.item(index).getNodeType() == Node.TEXT_NODE) text.append(list.item(index).getNodeValue()); } return (text.toString());
I excuted the main method as indicated in the code snippet and observed that the processing time increased gradually. I then reversed the loop in the main method to start from the last product record (14800) and noticed that the processing time was 77 seconds. However, it dropped significantly after a few iterations (100 records) to about 25 seconds.
What I do not understand is the reason for this increase in processing time. I have compared the XML records to confirm that there is nothing wrong with the data. The structure and content for all 14800 records is the same. Am I doing something wrong? Is there an issue with using XPath for large DOMs? Is there a XPath bug? I am passing the product node in the DOM to be traversed and I would think that the XPath lookup should be the same for product #14000 as it is for product #1. However, the pattern suggests that there is some kind of processing overhead when using XPath and going deeper in the DOM to retrieve data.
Any and all help is greatly appreciated.
Thanks in advance, Karthik
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
