[
https://issues.apache.org/jira/browse/LUCENE-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241445#comment-13241445
]
Michael McCandless commented on LUCENE-3937:
--------------------------------------------
Note: I just run benchmark's conf/extractWikipedia.alg task on the XML
export... when XERCESJ-1257 strikes you get this:
{noformat}
...
[java] 936.83 sec --> main Wrote 2801000 line docs
[java] 937.04 sec --> main Wrote 2802000 line docs
[java] 937.27 sec --> main Wrote 2803000 line docs
[java] 937.53 sec --> main Wrote 2804000 line docs
[java] 937.79 sec --> main Wrote 2805000 line docs
[java] 938.04 sec --> main Wrote 2806000 line docs
[java] 938.35 sec --> main Wrote 2807000 line docs
[java] 938.65 sec --> main Wrote 2808000 line docs
[java] 938.88 sec --> main Wrote 2809000 line docs
[java] 939.09 sec --> main Wrote 2810000 line docs
[java] 939.09 sec --> main Wrote 2810000 line docs
[java] Exception in thread "Thread-0" java.lang.RuntimeException:
org.apache.xerces.impl.io.MalformedByteSequenceException: Invalid byte 2 of
4-byte UTF-8 sequence.
[java] at
org.apache.lucene.benchmark.byTask.feeds.EnwikiContentSource$Parser.run(EnwikiContentSource.java:198)
[java] at java.lang.Thread.run(Thread.java:619)
[java] ####################
[java] ### D O N E !!! ###
[java] Caused by:
org.apache.xerces.impl.io.MalformedByteSequenceException: Invalid byte 2 of
4-byte UTF-8 sequence.
[java] ####################
[java] at
org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown
Source)
[java] at
org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
[java] at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown
Source)
[java] at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown
Source)
[java] at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown
Source)
[java] at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
Source)
[java] at org.apache.xerces.parsers.XML11Configuration.parse(Unknown
Source)
[java] at org.apache.xerces.parsers.XML11Configuration.parse(Unknown
Source)
[java] at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
[java] at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown
Source)
[java] at
org.apache.lucene.benchmark.byTask.feeds.EnwikiContentSource$Parser.run(EnwikiContentSource.java:175)
[java] ... 1 more
[java] Caused by:
org.apache.xerces.impl.io.MalformedByteSequenceException: Invalid byte 2 of
4-byte UTF-8 sequence.
[java] at org.apache.xerces.impl.io.UTF8Reader.invalidByte(Unknown
Source)
[java] at org.apache.xerces.impl.io.UTF8Reader.read(Unknown Source)
[java] at org.apache.xerces.impl.XMLEntityScanner.load(Unknown Source)
[java] at org.apache.xerces.impl.XMLEntityScanner.scanContent(Unknown
Source)
[java] at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanContent(Unknown
Source)
[java] ... 8 more
{noformat}
> Workaround the XERCES-J bug in Benchmark
> ----------------------------------------
>
> Key: LUCENE-3937
> URL: https://issues.apache.org/jira/browse/LUCENE-3937
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Uwe Schindler
> Attachments: LUCENE-3937.patch
>
>
> In becnhmark we have a patched version of XERCES which is hard to compile
> from source. When looking at the code part patched and the source of
> EnwikiContentSource, to simply provide the XML parser a Reader instead of
> InputStream, so the broken code is not triggered. This assumes, that the
> XML-file is always UTF-8.... If not it will no longer work (because the XML
> parser cannot switch encoding, if it only has a Reader).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
