Re: Error using collapse parser with /export

[Rahul Goswami]

I checked again and looks like all documents with the same "id_field"
reside on the same shard, in which case I would expect collapse parser to
work. Here is my complete query:

http://localhost:8983/solr/mycollection/stream/?expr=search(mycollection
,sort="field1 asc,field2
asc",fl="fileld1,field2,field3",qt="/export",q="*:*",fq="((field4:1)
OR (field4:2))",fq="{!collapse field=id_field sort='field3 desc'}")

The same query with "select" handler does return the collapse result fine.
Looks like this might be a bug afterall (while working with /export)?

Thanks,
Rahul


On Sun, Jan 27, 2019 at 9:55 PM Rahul Goswami  wrote:

> Hi Joel,
>
> Thanks for responding to the query.
>
> Answers to your questions:
> 1) After collapsing is it not possible to use the /select handler?  - The
> collapsing itself is causing the failure (or did I not understand your
> question right?)
> 2) After exporting is it possible to unique the records using the
> unique  Streaming Expression?   (This can't be done since we require the
> unique document in a group subject to a sort order as in the query above.
> Looking at the Streaming API, 'unique' streaming expression doesn't give
> the capability to sort within a group. Or is there a way to do this?)
>
> I re-read the documentation
> 
> :
> "In order to use these features with SolrCloud, the documents must be
> located on the same shard."
>
> Looks like the "id_field"  in the collapse criteria above is coming from
> documents not present in the same shard. I'll verify this tomorrow and
> update the thread.
>
> Thanks,
> Rahul
>
> On Mon, Jan 21, 2019 at 2:26 PM Joel Bernstein  wrote:
>
>> I haven't had time to look into the details of this issue but it's not
>> clear that these two features will be able to be used together. Although
>> that it would be nice if the could.
>>
>> A couple of questions about your use case:
>>
>> 1) After collapsing is it not possible to use the /select handler?
>> 2) After exporting is it possible to unique the records using the unique
>> Streaming Expression?
>>
>> Either of those cases would be the typical uses of these features.
>>
>> Joel Bernstein
>> http://joelsolr.blogspot.com/
>>
>>
>> On Sun, Jan 20, 2019 at 10:13 PM Rahul Goswami 
>> wrote:
>>
>> > Hello,
>> >
>> > Following up on my query. I know this might be too specific an issue.
>> But I
>> > just want to know that it's a legitimate bug and the supported
>> operation is
>> > allowed with the /export handler. If someone has an idea about this and
>> > could confirm, that would be great.
>> >
>> > Thanks,
>> > Rahul
>> >
>> > On Thu, Jan 17, 2019 at 4:58 PM Rahul Goswami 
>> > wrote:
>> >
>> > > Hello,
>> > >
>> > > I am using SolrCloud on Solr 7.2.1.
>> > > I get the NullPointerException in the Solr logs (in ExportWriter.java)
>> > > when the /stream handler is invoked with a search() streaming
>> expression
>> > > with qt="/export" containing fq="{!collapse field=id_field sort="time
>> > > desc"} (among other fq's. I tried eliminating one fq at a time to find
>> > the
>> > > problematic one. The one with collapse parser is what makes it fail).
>> > >
>> > >
>> > > I see an open JIRA for this issue (with a submitted patch which has
>> not
>> > > yet been accepted):
>> > >
>> > > https://issues.apache.org/jira/browse/SOLR-8291
>> > >
>> > >
>> > >
>> > > In my case useFilterForSortedQuery=false
>> > >
>> > > org.apache.solr.servlet.HttpSolrCall
>> null:java.lang.NullPointerException
>> > > at
>> org.apache.lucene.util.BitSetIterator.(BitSetIterator.java:61)
>> > > at
>> org.apache.solr.handler.ExportWriter.writeDocs(ExportWriter.java:243)
>> > > at
>> > >
>> org.apache.solr.handler.ExportWriter.lambda$null$1(ExportWriter.java:222)
>> > > at
>> > >
>> >
>> org.apache.solr.response.JSONWriter.writeIterator(JSONResponseWriter.java:523)
>> > > at
>> > >
>> >
>> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:180)
>> > > at
>> org.apache.solr.response.JSONWriter$2.put(JSONResponseWriter.java:559)
>> > > at
>> > >
>> org.apache.solr.handler.ExportWriter.lambda$null$2(ExportWriter.java:222)
>> > > at
>> > >
>> org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:547)
>> > > at
>> > >
>> >
>> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:198)
>> > > at
>> org.apache.solr.response.JSONWriter$2.put(JSONResponseWriter.java:559)
>> > > at
>> > >
>> >
>> org.apache.solr.handler.ExportWriter.lambda$write$3(ExportWriter.java:220)
>> > > at
>> > >
>> org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:547)
>> > > at org.apache.solr.handler.ExportWriter.write(ExportWriter.java:218)
>> > > at org.apache.solr.core.SolrCore$3.write(SolrCore.java:2627)
>> > > at
>> > >
>> >
>> org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:49)
>> > > at
>> > >
>> 

Re: Error using collapse parser with /export

[Rahul Goswami]

Hi Joel,

Thanks for responding to the query.

Answers to your questions:
1) After collapsing is it not possible to use the /select handler?  - The
collapsing itself is causing the failure (or did I not understand your
question right?)
2) After exporting is it possible to unique the records using the
unique  Streaming Expression?   (This can't be done since we require the
unique document in a group subject to a sort order as in the query above.
Looking at the Streaming API, 'unique' streaming expression doesn't give
the capability to sort within a group. Or is there a way to do this?)

I re-read the documentation
:
"In order to use these features with SolrCloud, the documents must be
located on the same shard."

Looks like the "id_field"  in the collapse criteria above is coming from
documents not present in the same shard. I'll verify this tomorrow and
update the thread.

Thanks,
Rahul

On Mon, Jan 21, 2019 at 2:26 PM Joel Bernstein  wrote:

> I haven't had time to look into the details of this issue but it's not
> clear that these two features will be able to be used together. Although
> that it would be nice if the could.
>
> A couple of questions about your use case:
>
> 1) After collapsing is it not possible to use the /select handler?
> 2) After exporting is it possible to unique the records using the unique
> Streaming Expression?
>
> Either of those cases would be the typical uses of these features.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> On Sun, Jan 20, 2019 at 10:13 PM Rahul Goswami 
> wrote:
>
> > Hello,
> >
> > Following up on my query. I know this might be too specific an issue.
> But I
> > just want to know that it's a legitimate bug and the supported operation
> is
> > allowed with the /export handler. If someone has an idea about this and
> > could confirm, that would be great.
> >
> > Thanks,
> > Rahul
> >
> > On Thu, Jan 17, 2019 at 4:58 PM Rahul Goswami 
> > wrote:
> >
> > > Hello,
> > >
> > > I am using SolrCloud on Solr 7.2.1.
> > > I get the NullPointerException in the Solr logs (in ExportWriter.java)
> > > when the /stream handler is invoked with a search() streaming
> expression
> > > with qt="/export" containing fq="{!collapse field=id_field sort="time
> > > desc"} (among other fq's. I tried eliminating one fq at a time to find
> > the
> > > problematic one. The one with collapse parser is what makes it fail).
> > >
> > >
> > > I see an open JIRA for this issue (with a submitted patch which has not
> > > yet been accepted):
> > >
> > > https://issues.apache.org/jira/browse/SOLR-8291
> > >
> > >
> > >
> > > In my case useFilterForSortedQuery=false
> > >
> > > org.apache.solr.servlet.HttpSolrCall
> null:java.lang.NullPointerException
> > > at org.apache.lucene.util.BitSetIterator.(BitSetIterator.java:61)
> > > at
> org.apache.solr.handler.ExportWriter.writeDocs(ExportWriter.java:243)
> > > at
> > >
> org.apache.solr.handler.ExportWriter.lambda$null$1(ExportWriter.java:222)
> > > at
> > >
> >
> org.apache.solr.response.JSONWriter.writeIterator(JSONResponseWriter.java:523)
> > > at
> > >
> >
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:180)
> > > at
> org.apache.solr.response.JSONWriter$2.put(JSONResponseWriter.java:559)
> > > at
> > >
> org.apache.solr.handler.ExportWriter.lambda$null$2(ExportWriter.java:222)
> > > at
> > >
> org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:547)
> > > at
> > >
> >
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:198)
> > > at
> org.apache.solr.response.JSONWriter$2.put(JSONResponseWriter.java:559)
> > > at
> > >
> >
> org.apache.solr.handler.ExportWriter.lambda$write$3(ExportWriter.java:220)
> > > at
> > >
> org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:547)
> > > at org.apache.solr.handler.ExportWriter.write(ExportWriter.java:218)
> > > at org.apache.solr.core.SolrCore$3.write(SolrCore.java:2627)
> > > at
> > >
> >
> org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:49)
> > > at
> > >
> org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:788)
> > > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:525)
> > >
> > >
> > > Above is a smaller trace; I can provide the complete stacktrace if it
> > > helps. Before considering making a fix in ExportWriter.java and
> > rebuilding
> > > Solr as a last resort, I want to make sure I am not using something
> which
> > > is not supported on SolrCloud. Can anybody please help?
> > >
> > >
> > >
> >
>

Re: Error using collapse parser with /export

[Joel Bernstein]

I haven't had time to look into the details of this issue but it's not
clear that these two features will be able to be used together. Although
that it would be nice if the could.

A couple of questions about your use case:

1) After collapsing is it not possible to use the /select handler?
2) After exporting is it possible to unique the records using the unique
Streaming Expression?

Either of those cases would be the typical uses of these features.

Joel Bernstein
http://joelsolr.blogspot.com/


On Sun, Jan 20, 2019 at 10:13 PM Rahul Goswami 
wrote:

> Hello,
>
> Following up on my query. I know this might be too specific an issue. But I
> just want to know that it's a legitimate bug and the supported operation is
> allowed with the /export handler. If someone has an idea about this and
> could confirm, that would be great.
>
> Thanks,
> Rahul
>
> On Thu, Jan 17, 2019 at 4:58 PM Rahul Goswami 
> wrote:
>
> > Hello,
> >
> > I am using SolrCloud on Solr 7.2.1.
> > I get the NullPointerException in the Solr logs (in ExportWriter.java)
> > when the /stream handler is invoked with a search() streaming expression
> > with qt="/export" containing fq="{!collapse field=id_field sort="time
> > desc"} (among other fq's. I tried eliminating one fq at a time to find
> the
> > problematic one. The one with collapse parser is what makes it fail).
> >
> >
> > I see an open JIRA for this issue (with a submitted patch which has not
> > yet been accepted):
> >
> > https://issues.apache.org/jira/browse/SOLR-8291
> >
> >
> >
> > In my case useFilterForSortedQuery=false
> >
> > org.apache.solr.servlet.HttpSolrCall null:java.lang.NullPointerException
> > at org.apache.lucene.util.BitSetIterator.(BitSetIterator.java:61)
> > at org.apache.solr.handler.ExportWriter.writeDocs(ExportWriter.java:243)
> > at
> > org.apache.solr.handler.ExportWriter.lambda$null$1(ExportWriter.java:222)
> > at
> >
> org.apache.solr.response.JSONWriter.writeIterator(JSONResponseWriter.java:523)
> > at
> >
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:180)
> > at org.apache.solr.response.JSONWriter$2.put(JSONResponseWriter.java:559)
> > at
> > org.apache.solr.handler.ExportWriter.lambda$null$2(ExportWriter.java:222)
> > at
> > org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:547)
> > at
> >
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:198)
> > at org.apache.solr.response.JSONWriter$2.put(JSONResponseWriter.java:559)
> > at
> >
> org.apache.solr.handler.ExportWriter.lambda$write$3(ExportWriter.java:220)
> > at
> > org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:547)
> > at org.apache.solr.handler.ExportWriter.write(ExportWriter.java:218)
> > at org.apache.solr.core.SolrCore$3.write(SolrCore.java:2627)
> > at
> >
> org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:49)
> > at
> > org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:788)
> > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:525)
> >
> >
> > Above is a smaller trace; I can provide the complete stacktrace if it
> > helps. Before considering making a fix in ExportWriter.java and
> rebuilding
> > Solr as a last resort, I want to make sure I am not using something which
> > is not supported on SolrCloud. Can anybody please help?
> >
> >
> >
>

Re: Error using collapse parser with /export

[Rahul Goswami]

Hello,

Following up on my query. I know this might be too specific an issue. But I
just want to know that it's a legitimate bug and the supported operation is
allowed with the /export handler. If someone has an idea about this and
could confirm, that would be great.

Thanks,
Rahul

On Thu, Jan 17, 2019 at 4:58 PM Rahul Goswami  wrote:

> Hello,
>
> I am using SolrCloud on Solr 7.2.1.
> I get the NullPointerException in the Solr logs (in ExportWriter.java)
> when the /stream handler is invoked with a search() streaming expression
> with qt="/export" containing fq="{!collapse field=id_field sort="time
> desc"} (among other fq's. I tried eliminating one fq at a time to find the
> problematic one. The one with collapse parser is what makes it fail).
>
>
> I see an open JIRA for this issue (with a submitted patch which has not
> yet been accepted):
>
> https://issues.apache.org/jira/browse/SOLR-8291
>
>
>
> In my case useFilterForSortedQuery=false
>
> org.apache.solr.servlet.HttpSolrCall null:java.lang.NullPointerException
> at org.apache.lucene.util.BitSetIterator.(BitSetIterator.java:61)
> at org.apache.solr.handler.ExportWriter.writeDocs(ExportWriter.java:243)
> at
> org.apache.solr.handler.ExportWriter.lambda$null$1(ExportWriter.java:222)
> at
> org.apache.solr.response.JSONWriter.writeIterator(JSONResponseWriter.java:523)
> at
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:180)
> at org.apache.solr.response.JSONWriter$2.put(JSONResponseWriter.java:559)
> at
> org.apache.solr.handler.ExportWriter.lambda$null$2(ExportWriter.java:222)
> at
> org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:547)
> at
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:198)
> at org.apache.solr.response.JSONWriter$2.put(JSONResponseWriter.java:559)
> at
> org.apache.solr.handler.ExportWriter.lambda$write$3(ExportWriter.java:220)
> at
> org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:547)
> at org.apache.solr.handler.ExportWriter.write(ExportWriter.java:218)
> at org.apache.solr.core.SolrCore$3.write(SolrCore.java:2627)
> at
> org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:49)
> at
> org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:788)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:525)
>
>
> Above is a smaller trace; I can provide the complete stacktrace if it
> helps. Before considering making a fix in ExportWriter.java and rebuilding
> Solr as a last resort, I want to make sure I am not using something which
> is not supported on SolrCloud. Can anybody please help?
>
>
>