[
https://issues.apache.org/jira/browse/SOLR-14537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151289#comment-17151289
]
Ishan Chattopadhyaya edited comment on SOLR-14537 at 7/4/20, 11:48 AM:
-----------------------------------------------------------------------
I thought I'll leave alone branch_8x for now. But it seems there's an ant
precommit failure there:
{code}
[jtidy] Checking for broken html (such as invalid tags)...
[delete] Deleting directory
/home/ishan/code/lucene-solr/lucene/build/jtidy_tmp
[echo] Checking for broken links...
[exec]
[exec] Crawl/parse...
[exec]
[exec] Verify...
[exec]
[exec]
file:///home/ishan/code/lucene-solr/solr/build/docs/solr-core/serialized-form.html
[exec] BROKEN LINK:
file:///home/ishan/code/lucene-solr/solr/build/docs/solr-core/org/apache/solr/handler/export/ExportBuffers.Buffer.html
[exec]
[exec]
file:///home/ishan/code/lucene-solr/solr/build/docs/solr-core/org/apache/solr/handler/export/ExportWriter.html
[exec] BROKEN LINK:
file:///home/ishan/code/lucene-solr/solr/build/docs/solr-core/org/apache/solr/handler/export/ExportBuffers.Buffer.html
[exec] BROKEN LINK:
file:///home/ishan/code/lucene-solr/solr/build/docs/solr-core/org/apache/solr/handler/export/ExportBuffers.Buffer.html
[exec]
[exec] Broken javadocs links were found! Common root causes:
[exec] * A typo of some sort for manually created links.
[exec] * Public methods referencing non-public classes in their signature.
{code}
Not sure if the same failure affects master as well.
Edit: Here's my ant version.
{code}
[ishan@x1extreme lucene-solr] $ ant -version
Apache Ant(TM) version 1.10.8 compiled on May 24 2020
{code}
was (Author: ichattopadhyaya):
I thought I'll leave alone branch_8x for now. But it seems there's an ant
precommit failure there:
{code}
[jtidy] Checking for broken html (such as invalid tags)...
[delete] Deleting directory
/home/ishan/code/lucene-solr/lucene/build/jtidy_tmp
[echo] Checking for broken links...
[exec]
[exec] Crawl/parse...
[exec]
[exec] Verify...
[exec]
[exec]
file:///home/ishan/code/lucene-solr/solr/build/docs/solr-core/serialized-form.html
[exec] BROKEN LINK:
file:///home/ishan/code/lucene-solr/solr/build/docs/solr-core/org/apache/solr/handler/export/ExportBuffers.Buffer.html
[exec]
[exec]
file:///home/ishan/code/lucene-solr/solr/build/docs/solr-core/org/apache/solr/handler/export/ExportWriter.html
[exec] BROKEN LINK:
file:///home/ishan/code/lucene-solr/solr/build/docs/solr-core/org/apache/solr/handler/export/ExportBuffers.Buffer.html
[exec] BROKEN LINK:
file:///home/ishan/code/lucene-solr/solr/build/docs/solr-core/org/apache/solr/handler/export/ExportBuffers.Buffer.html
[exec]
[exec] Broken javadocs links were found! Common root causes:
[exec] * A typo of some sort for manually created links.
[exec] * Public methods referencing non-public classes in their signature.
{code}
Not sure if the same failure affects master as well.
> Improve performance of ExportWriter
> -----------------------------------
>
> Key: SOLR-14537
> URL: https://issues.apache.org/jira/browse/SOLR-14537
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Export Writer
> Reporter: Andrzej Bialecki
> Assignee: Andrzej Bialecki
> Priority: Major
> Fix For: 8.7
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Retrieving, sorting and writing out documents in {{ExportWriter}} are three
> aspects of the /export handler that can be further optimized.
> SOLR-14470 introduced some level of caching in {{StringValue}}. Further
> options for caching and speedups should be explored.
> Currently the sort/retrieve and write operations are done sequentially, but
> they could be parallelized, considering that they block on different channels
> - the first is index reading & CPU bound, the other is bound by the receiving
> end because it uses blocking IO. The sorting and retrieving of values could
> be done in parallel with the operation of writing out the current batch of
> results.
> One possible approach here would be to use "double buffering" where one
> buffered batch that is ready (already sorted and retrieved) is being written
> out, while the other batch is being prepared in a background thread, and when
> both are done the buffers are swapped. This wouldn't complicate the current
> code too much but it should instantly give up to 2x higher throughput.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
