[
https://issues.apache.org/jira/browse/SOLR-5330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13793213#comment-13793213
]
Yonik Seeley edited comment on SOLR-5330 at 10/12/13 2:30 AM:
--------------------------------------------------------------
So I instrumented the faceting code like so:
{code}
seg.tempBR = seg.tenum.next();
if (seg.tempBR.bytes == val.bytes) {
System.err.println("##########SHARING DETECTED: val.offset="+val.offset + "
val.length="+val.length + " new.offset="+seg.tempBR.offset + "
new.length="+seg.tempBR.length);
if (val.offset == seg.tempBR.offset) {
System.err.println("!!!!!!SHARING USING SAME OFFSET");
}
{code}
And it detects tons of sharing (the returned bytesref still pointing to the
same byte[]) of course... but the thing is, it never generates an invalid
result. calling next() on the term enum never changes the bytes that were
previously pointed to... it simply points to a different part of the same byte
array. I can never detect a case where the original bytes are changed, thus
invalidating the shallow copy.
Example output:
{code}
##########SHARING DETECTED: val.offset=1 val.length=4 new.offset=6 new.length=4
{code}
was (Author: [email protected]):
So I instrumented the faceting code like so:
{code}
seg.tempBR = seg.tenum.next();
if (seg.tempBR.bytes == val.bytes) {
System.err.println("##########SHARING DETECTED: val.offset="+val.offset + "
val.length="+val.length + " new.offset="+seg.tempBR.offset + "
new.length="+seg.tempBR.length);
if (val.offset == seg.tempBR.offset) {
System.err.println("!!!!!!SHARING USING SAME OFFSET");
}
{code}
And it detects tons of sharing (the returned bytesref still pointing to the
same byte[]) of course... but the thing is, it never generates an invalid
result. calling next() on the term enum never changes the bytes that were
previously pointed to... it simply points to a different part of the same byte
array. I can never detect a case where the original bytes are changed, thus
invalidating the shallow copy.
> PerSegmentSingleValuedFaceting overwrites facet values
> ------------------------------------------------------
>
> Key: SOLR-5330
> URL: https://issues.apache.org/jira/browse/SOLR-5330
> Project: Solr
> Issue Type: Bug
> Affects Versions: 4.2.1
> Reporter: Michael Froh
> Assignee: Yonik Seeley
> Attachments: solr-5330.patch
>
>
> I recently tried enabling facet.method=fcs for one of my indexes and found a
> significant performance improvement (with a large index, many facet values,
> and near-realtime updates). Unfortunately, the results were also wrong.
> Specifically, some facet values were being partially overwritten by other
> facet values. (That is, if I expected facet values like "abcdef" and "123", I
> would get a value like "123def".)
> Debugging through the code, it looks like the problem was in
> PerSegmentSingleValuedFaceting, specifically in the getFacetCounts method,
> when BytesRef val is shallow-copied from the temporary per-segment BytesRef.
> The byte array assigned to val is shared with the byte array for seg.tempBR,
> and is overwritten a few lines down by the call to seg.tenum.next().
> I managed to fix it locally by replacing the shallow copy with a deep copy.
> While I encountered this problem on Solr 4.2.1, I see that the code is
> identical in 4.5. Unless the behavior of TermsEnum.next() has changed, I
> believe this bug still exists.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
