Martin -
I have to admit to still being unsure what the gist is here - is there a bug?
Apologies for not catching what you’re saying/showing here. Again, as far as I
can tell SpanPayloadCheckQuery is working as expected now.
I’m going to focus purely on SOLR-1485 this week to get it committed for 6.6.
If there is an issue to address with your work would you please open a JIRA and
include your patch there?
Thanks,
Erik
> On Apr 30, 2017, at 7:47 AM, Martin Gainty <[email protected]> wrote:
>
> Mornin' Erik
>
> there is a collectLeaf override in
> org.apache.lucene.queries.payloads.TestPayloadSpans ..but its never called:
>
> static class VerifyingCollector implements SpanCollector {
> List<BytesRef> payloads = new ArrayList<>();
> @Override
> public void collectLeaf(PostingsEnum postings, int position, Term term)
> throws IOException {
> ....
> }
> }
>
> the modification in org.apache.lucene.queries.payloads.TestPayloadCheckQuery
> tests collectLeaf for query
>
> //initialise term
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 231 before
> term1=new org.apache.lucene.index.Term('field','withPayload')");
> org.apache.lucene.index.Term term1=new org.apache.lucene.index.Term("field",
> "withPayload");
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 233
> term1="+term1);
>
> //assume position is 5
> int position=5;
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 235
> position="+position);
>
> BytesRef pay = new BytesRef("pos: " + position);
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 237
> pay="+pay);
>
> //build spanQuery with term parameter
> org.apache.lucene.search.spans.SpanQuery spanQuery1 = new
> SpanTermQuery(term1);
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 239
> spanQuery1="+spanQuery1);
>
> //add BytesRef to payloadToMatch list
> java.util.List<org.apache.lucene.util.BytesRef> payloadToMatch=new
> java.util.ArrayList<org.apache.lucene.util.BytesRef>();
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 241
> payloadToMatch="+payloadToMatch);
> payloadToMatch.add(pay);
>
> //build SpanPayloadCheckQuery
> query=new org.apache.lucene.queries.payloads.SpanPayloadCheckQuery(
> (org.apache.lucene.search.spans.SpanQuery)query,
> (java.util.List<org.apache.lucene.util.BytesRef>)payloadToMatch);
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 249
> query="+query);
>
> //build lucene Directory (TODO: we should use an existing directory with REAL
> test-data)
> org.apache.lucene.store.Directory ram =
> newDirectory(com.carrotsearch.randomizedtesting.RandomizedContext.current().getRandom());
>
> //build SegmentReader from Directory
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 251 ram="+ram);
> SegmentReader reader =
> getOnlySegmentReader(org.apache.lucene.index.DirectoryReader.open(ram));
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 253
> reader="+reader);
>
> //populate SlowCompositeReaderWrapper with SegmentReader
> org.apache.lucene.index.LeafReader sr =
> org.apache.lucene.index.SlowCompositeReaderWrapper.wrap(reader);
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 255 sr="+sr);
>
> //add term to SegmentReader postings
> org.apache.lucene.index.PostingsEnum postings = sr.postings(term1,
> PostingsEnum.PAYLOADS);
>
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 257 before
> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
> (int)position,(org.apache.lucene.index.Term)term1) where
> postings="+postings);
>
> //display the parameters for collectLeaf method for query:
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 258 before
> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
> (int)position,(org.apache.lucene.index.Term)term1) where
> position="+position);
>
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 259 before
> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
> (int)position,(org.apache.lucene.index.Term)term1) where term1="+term1);
> try
> { //public void collectLeaf(org.apache.lucene.index.PostingsEnum
> postings, int position, //org.apache.lucene.index.Term term) throws
> java.io.IOException {
> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
> (int)position,(org.apache.lucene.index.Term)term1);
> }
> catch(java.io.IOException ioe) {
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 264
> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
> (int)position,(org.apache.lucene.index.Term)term1) LINE 106 throws
> IOException ="+ioe.getMessage()); }
>
> collectLeaf is the only method that compares
> matches=payloadToMatch.get(upto).bytesEquals(payload)
> to determine "match"
>
> unless of course match between individual payload and payloadToMatch is
> tested elsewhere ?
>
> WDYT?
> Martin
> ______________________________________________
>
>
>
> From: Erik Hatcher <[email protected]>
> Sent: Saturday, April 29, 2017 7:58 PM
> To: Lucene/Solr dev
> Subject: Re: Release 6.6
>
> Martin -
>
> Interesting test but I couldn’t copy/paste it in to try it out as it uses
> some logging and APIs that aren’t already in the project and classpath of
> these lucene tests within that queries module (within IntelliJ, mind you).
>
> I was able to resolve the misunderstanding (and .equals/.hashCode bugs) so
> everything is working as expected with SpanPayloadCheckQuery for me now. Do
> your tests point out an issue? Or confirming that it is working properly at
> a lower level?
>
> Erik
>
>
>> On Apr 29, 2017, at 9:08 AM, Martin Gainty <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>> Martin Gainty has shared a OneDrive file with you. To view it, click the
>> link below.
>>
>> TestPayloadCheckQuery.java
>> <https://1drv.ms/u/s!AkpuiYcNg4cSgWRHc6DadiCIYaFN>
>> <https://1drv.ms/u/s!AkpuiYcNg4cSgWRHc6DadiCIYaFN>
>> TestPayloadCheckQuery.java <https://1drv.ms/u/s!AkpuiYcNg4cSgWRHc6DadiCIYaFN>
>> attached
>>
>> I coded this last nite so it is "quick and dirty"
>>
>> in any case let me know if testSpanPayloadCheck() method modification
>> properly addresses your situation
>>
>> Thanks!
>> Martin
>> ______________________________________________
>>
>>
>>
>> From: Erik Hatcher <[email protected] <mailto:[email protected]>>
>> Sent: Saturday, April 29, 2017 8:58 AM
>> To: [email protected] <mailto:[email protected]>
>> Subject: Re: Release 6.6
>>
>> Martin - there is a test class specifically for this query. See the recent
>> commits I've made to test it further fixing .equals/.hashCode and rewrite.
>>
>> Can you send your full test code?
>>
>> Erik
>>
>> On Apr 29, 2017, at 07:32, Martin Gainty <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>>> when Erik mentioned he couldnt get SpanPayloadCheckQuery to work I was
>>> about to reply just run that TestCase
>>> (until i discovered there wasnt any!)
>>>
>>> i'll start the bidding at 1 dinar for TestCase
>>> org.apache.lucene.queries.payloads.TestPayloadCheckQuery mod:
>>> @org.junit.Test
>>> public void testSpanPayloadCheck() throws Exception
>>>
>>> //now lets test the collectLeaf for query
>>> //of course calling Base Class SpanPayloadCheckQuery to construct query
>>>
>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 257 before
>>> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
>>> (int)position,(org.apache.lucene.index.Term)term1) where
>>> postings="+postings);
>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 258 before
>>> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
>>> (int)position,(org.apache.lucene.index.Term)term1) where
>>> position="+position);
>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 259 before
>>> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
>>> (int)position,(org.apache.lucene.index.Term)term1) where term1="+term1);
>>>
>>> try
>>> { //test public void collectLeaf(org.apache.lucene.index.PostingsEnum
>>> postings, int position, //org.apache.lucene.index.Term term)
>>> throws java.io.IOException {
>>> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
>>> (int)position,(org.apache.lucene.index.Term)term1);
>>> }
>>> catch(java.io.IOException ioe) {
>>> log.error("TestPayloadCheckQuery::testSpanPayloadCheck LINE 264
>>> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
>>> (int)position,(org.apache.lucene.index.Term)term1) LINE 106 throws
>>> IOException ="+ioe.getMessage()); }
>>>
>>> unless of course anyone has a better idea to test collectLeaf ?
>>> Martin
>>> ______________________________________________
>>>
>>>
>>>
>>> From: Erik Hatcher <[email protected] <mailto:[email protected]>>
>>> Sent: Tuesday, April 25, 2017 7:57 PM
>>> To: [email protected] <mailto:[email protected]>
>>> Subject: Re: Release 6.6
>>>
>>> Probably no bribe needed, but an updated patch would be a good start (or
>>> will the 2.5 year old patch still apply and be acceptable as-is?)
>>>
>>> Erik
>>>
>>>> On Apr 25, 2017, at 7:52 PM, Walter Underwood <[email protected]
>>>> <mailto:[email protected]>> wrote:
>>>>
>>>> Who do I have to bribe to get SOLR-629 included?
>>>>
>>>> https://issues.apache.org/jira/browse/SOLR-629
>>>> <https://issues.apache.org/jira/browse/SOLR-629>
>>>>
>>>> wunder
>>>> Walter Underwood
>>>> [email protected] <mailto:[email protected]>
>>>> http://observer.wunderwood.org/ <http://observer.wunderwood.org/> (my
>>>> blog)
>>>>
>>>>
>>>>> On Apr 25, 2017, at 10:46 AM, Ishan Chattopadhyaya
>>>>> <[email protected] <mailto:[email protected]>> wrote:
>>>>>
>>>>> Hi,
>>>>> We have lots of changes, optimizations, bug fixes for 6.6. I'd like to
>>>>> propose a 6.6 release (perhaps the last 6x non-bug-fix release before 7.0
>>>>> release).
>>>>>
>>>>> I can volunteer to release this, and I can cut the release branch around
>>>>> 4th May, in order to let some time for devs to finish current issues.
>>>>>
>>>>> What do you think?
>>>>>
>>>>> Regards,
>>>>> Ishan
