[
https://issues.apache.org/jira/browse/ANY23-336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16416234#comment-16416234
]
Hans Brende edited comment on ANY23-336 at 3/27/18 9:01 PM:
------------------------------------------------------------
[~lewismc] So I've dived into the source code of the {{BenchmarkTripleHandler}}
class, which is what is being used to report the statistics, and tested it out
myself.
I got approximately the same results you did.
Which means that the {{BenchmarkTripleHandler}} class is not reporting accurate
results for the extraction runtimes.
Indeed, looking at the {{SingleDocumentExtraction}} class, the
[TripleHandler.openContext()|https://github.com/apache/any23/blob/1867cc66de9a82cd98f1962fdabbd3a8680ff408/core/src/main/java/org/apache/any23/extractor/SingleDocumentExtraction.java#L720]
and
[TripleHandler.closeContext()|https://github.com/apache/any23/blob/1867cc66de9a82cd98f1962fdabbd3a8680ff408/core/src/main/java/org/apache/any23/extractor/SingleDocumentExtraction.java#L737]
methods (where the start and end times of the extraction interval are recorded
for statistical purposes) are only called *after the extraction has already
taken place*!!!
was (Author: hansbrende):
[~lewismc] So I've dived into the source code of the {{BenchmarkTripleHandler}}
class, which is what is being used to report the statistics, and tested it out
myself.
I got approximately the same results you did.
Which means that the {{BenchmarkTripleHandler}} class is not reporting accurate
results for the extraction runtimes.
Indeed, looking at the {{SingleDocumentExtraction}} class, the
{{TripleHandler.openContext()}} and {{TripleHandler.closeContext()}} methods
(where the start and end times of the extraction interval are recorded for
statistically purposes) are only called *after the extraction has already taken
place*!!!
> Parsing json-ld content takes prohibitively long time
> -----------------------------------------------------
>
> Key: ANY23-336
> URL: https://issues.apache.org/jira/browse/ANY23-336
> Project: Apache Any23
> Issue Type: Bug
> Components: core, extractors
> Affects Versions: 2.2
> Reporter: Hans Brende
> Priority: Critical
> Fix For: 2.3
>
> Attachments: Screen Shot 2018-03-27 at 2.52.15 PM.png, Screen Shot
> 2018-03-27 at 2.54.43 PM.png
>
>
> Using the page [https://www.guthriegreen.com|https://www.guthriegreen.com/]
> as a benchmark, a page fetch took about 100 ms, while simply *parsing* the
> json-ld content on that page took a *staggering 27400 ms*. For reference, I'm
> using Java 8, build 162, on a Macbook Pro (early 2015).
> The bad news is that this is not our fault.
> I've profiled this behavior down to the
> {{com.github.jsonldjava.utils.JsonUtils.fromURL(URL, CloseableHttpClient)}}
> function. 94% of the parsing time is spent there. This function is called
> when trying to load remote json-ld contexts.
> In order to avoid loading remote contexts repeatedly, this function tries to
> *cache* them by using a {{CachingHttpClient}} from the httpclient-osgi
> library.
> Unfortunately, that strategy is *not* working, as I have recorded exactly
> *zero* cache hits, meaning that *every* retrieval is a cache miss and a
> remote context is re-fetched via http every single time it's accessed.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
