[
https://issues.apache.org/jira/browse/ANY23-324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336474#comment-16336474
]
ASF GitHub Bot commented on ANY23-324:
--------------------------------------
Github user HansBrende commented on the issue:
https://github.com/apache/any23/pull/58
@lewismc I wrote a quick little speed test. Not sure if this is the best
way to do it, but FWIW, here it is:
```
public static void main(String[] args) throws IOException {
String url =
"https://en.wikipedia.org/wiki/List_of_compositions_by_Johann_Sebastian_Bach";;
String encoding = "utf-8";
byte[] webpage = IOUtils.toString(new URL(url)).getBytes(encoding);
TagSoupParsingConfiguration neko = TagSoupParser.legacyConfig();
TagSoupParsingConfiguration jsoup = JsoupConfig.instance;
long totalTimeNeko = 0, totalTimeJsoup = 0;
for (int i = 0; i < 11000; i++) {
ByteArrayInputStream input = new ByteArrayInputStream(webpage);
boolean useNeko = i % 2 == 0;
TagSoupParsingConfiguration config = useNeko ? neko : jsoup;
long start = System.currentTimeMillis();
config.parse(input, url, encoding);
long duration = System.currentTimeMillis() - start;
long timeToAdd = i < 1000 ? 0 : duration; //SKIP FIRST 1000
ITERATIONS
if (useNeko) {
totalTimeNeko += timeToAdd;
} else {
totalTimeJsoup += timeToAdd;
}
if (i % 100 == 0) {
System.out.println(i);
}
}
System.out.println("total time neko: " + totalTimeNeko + " ms");
System.out.println("total time jsoup: " + totalTimeJsoup + " ms");
}
```
RESULTS:
<pre>
total time neko: 488787 ms
total time jsoup: 479211 ms
</pre>
> Replace net.sourceforge.nekohtml with jsoup
> --------------------------------------------
>
> Key: ANY23-324
> URL: https://issues.apache.org/jira/browse/ANY23-324
> Project: Apache Any23
> Issue Type: Improvement
> Components: core
> Reporter: Lewis John McGibbney
> Priority: Major
> Fix For: 2.2
>
>
> A long standing issue relates to the performance of the existing default
> [TagSoupParser.java|https://github.com/apache/any23/blob/master/core/src/main/java/org/apache/any23/extractor/html/TagSoupParser.java].
> There are a number of issues which now relate to limitations in the way
> nekohtml parses HTML5 for example
> [ANY23-317|https://issues.apache.org/jira/browse/ANY23-317],
> [ANY23-273|https://issues.apache.org/jira/browse/ANY23-273],
> [ANY23-267|https://issues.apache.org/jira/browse/ANY23-267]... there are
> several others.
> I propose to @Deprecate the TagSoupParser.java implementation for the next
> release (possibly making it configurable via
> default-configuration.properties). I also propose to replace it with
> https://jsoup.org/. AFAIK, Apache Tika also did this several years ago.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
