[
https://issues.apache.org/jira/browse/AXIOM-478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086795#comment-15086795
]
Andreas Veithen edited comment on AXIOM-478 at 1/8/16 10:11 AM:
----------------------------------------------------------------
I tested getTextAsStream() but it still consume large memory. Here's the test
code:
{noformat}
InputStream is = getClass().getResourceAsStream("sample2.txt");
OMXMLParserWrapper builder =
OMXMLBuilderFactory.createOMBuilder(is);
OMElement document = builder.getDocumentElement();
Iterator<OMElement> entryIt = document.getChildElements();
Iterator<OMElement> contentIt = null;
OMElement contentElement = null;
OMElement base64Element = null;
Reader reader = null;
while (entryIt.hasNext()) {
contentElement = entryIt.next();
if ("content".equals(contentElement.getLocalName())) {
contentIt = contentElement.getChildElements();
while (contentIt.hasNext()) {
base64Element = contentIt.next();
if
("base64".equals(base64Element.getLocalName())) {
reader =
base64Element.getTextAsStream(false);
int byteCount = 0;
for (int character =
reader.read(); character != -1; character = reader.read()) {
byteCount++;
if (byteCount == 1) {
System.out.println("breakpoint");
}
if (byteCount ==
24553800) {
System.out.println("breakpoint");
}
}
System.out.println("Base 64
Encoded Bytes: " + byteCount);
}
}
}
}
{noformat}
was (Author: lujie):
I tested getTextAsStream() but it still consume large memory. Here's the test
code:
InputStream is = getClass().getResourceAsStream("sample2.txt");
OMXMLParserWrapper builder =
OMXMLBuilderFactory.createOMBuilder(is);
OMElement document = builder.getDocumentElement();
Iterator<OMElement> entryIt = document.getChildElements();
Iterator<OMElement> contentIt = null;
OMElement contentElement = null;
OMElement base64Element = null;
Reader reader = null;
while (entryIt.hasNext()) {
contentElement = entryIt.next();
if ("content".equals(contentElement.getLocalName())) {
contentIt = contentElement.getChildElements();
while (contentIt.hasNext()) {
base64Element = contentIt.next();
if
("base64".equals(base64Element.getLocalName())) {
reader =
base64Element.getTextAsStream(false);
int byteCount = 0;
for (int character =
reader.read(); character != -1; character = reader.read()) {
byteCount++;
if (byteCount == 1) {
System.out.println("breakpoint");
}
if (byteCount ==
24553800) {
System.out.println("breakpoint");
}
}
System.out.println("Base 64
Encoded Bytes: " + byteCount);
}
}
}
}
> Solution for parsing large XML
> ------------------------------
>
> Key: AXIOM-478
> URL: https://issues.apache.org/jira/browse/AXIOM-478
> Project: Axiom
> Issue Type: Question
> Reporter: LU Jie
>
> This is LU Jie from IBM. We use axiom to parse Atom in our project.
> One of our CMIS API will attach file content to the XML. If the file size is
> large, we will get a large atom.
> If we use Entry.getExtension(QName) to parse the content, it will allocate a
> large memory(around 5-6 times of the file size).
> We need you help to clarify if we can use DOM-like API of axiom to get the
> text of a certain element as stream. That is without allocating a large
> object in memory.
> Or is there an alternative solution for this use case?
> We DO know that we can use pull-parser to parse the XML as stream. But we
> need help to investigate if axiom has already provided an API or solution to
> avoid writing parser by ourselves.
> Here's the sample XML. We need to parse the text of cmisra:base64 element:
> {noformat}
> <atom:entry
> xmlns:atom="http://www.w3.org/2005/Atom";
> xmlns:cmisra="http://docs.oasis-open.org/ns/cmis/restatom/200908/";
> xmlns:chemistry="http://chemistry.apache.org/";
> xmlns:cmis="http://docs.oasis-open.org/ns/cmis/core/200908/";>
> <atom:id
>
> xmlns:atom="http://www.w3.org/2005/Atom";>urn:uuid:00000000-0000-0000-0000-00000000000
> </atom:id>
> <atom:title
> xmlns:atom="http://www.w3.org/2005/Atom";
> type="text">doucment1446016556658.txt
> </atom:title>
> <atom:updated
> xmlns:atom="http://www.w3.org/2005/Atom";>2015-10-28T07:15:57.594Z
> </atom:updated>
> <cmisra:content
> xmlns:cmisra="http://docs.oasis-open.org/ns/cmis/restatom/200908/";>
> <cmisra:mediatype
>
> xmlns:cmisra="http://docs.oasis-open.org/ns/cmis/restatom/200908/";>text/plain
> </cmisra:mediatype>
> <chemistry:filename
>
> xmlns:chemistry="http://chemistry.apache.org/";>doucment1446016556658.txt
> </chemistry:filename>
> <cmisra:base64
>
> xmlns:cmisra="http://docs.oasis-open.org/ns/cmis/restatom/200908/";>Base64
> encoded content of large file
> </cmisra:base64>
> </cmisra:content>
> <cmisra:object
> xmlns:cmisra="http://docs.oasis-open.org/ns/cmis/restatom/200908/";>
> <cmis:properties
> xmlns:cmis="http://docs.oasis-open.org/ns/cmis/core/200908/";>
> <cmis:propertyId
> xmlns:cmis="http://docs.oasis-open.org/ns/cmis/core/200908/";
> propertyDefinitionId="cmis:objectTypeId">
> <cmis:value
>
> xmlns:cmis="http://docs.oasis-open.org/ns/cmis/core/200908/";>snx:file
> </cmis:value>
> </cmis:propertyId>
> <cmis:propertyString
> xmlns:cmis="http://docs.oasis-open.org/ns/cmis/core/200908/";
> propertyDefinitionId="cmis:name">
> <cmis:value
>
> xmlns:cmis="http://docs.oasis-open.org/ns/cmis/core/200908/";>doucment1446016556658.txt
> </cmis:value>
> </cmis:propertyString>
> </cmis:properties>
> </cmisra:object>
> </atom:entry>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
