Dynamically Adding query parameters in my custom Request Handler class

[Mark Robinson]

Hi,
When I initially fire a query against my Solr instance using SOLRJ I pass
only, say q=*:*=(myfield:vaue1).

I have written a custom RequestHandler, which is what I call in my SolrJ
query.
Inside this custom request handler can I add more query params like say the
facets etc.. so that ultimately facets are also received back in my results
which were initially not specified when I invoked the Solr url using SolrJ.

In short, instead of constructing the query dynamically initially in SolrJ
I want to add the extra query params, adding a jar in Solr (a java code
that will check certain conditions and dynamically add the query params
after the initial SolrJ query is done). That is why I thought of a custom
RH which would help we write a java class and deploy in Solr.

Is this possible. Could some one get back please.

Thanks!
Mark.

Re: Dynamically Adding query parameters in my custom Request Handler class

[Mark Robinson]

Hi,

Ahmet, Jack, Thanks for the pointers.
My requirement is, I would not be having the facets or sort fields or its
order as static.
For example suppose for a particular scenario I need to show only 2 facets
and sort on only one field.
For another scenario I may have to do facet.field for a different set of
fields and sort on again another set of fields.
Consider there are some sort of user preferences for each query.

So I think I may not be able to store my parameters like facet fields, sort
fields etc preconfigured in solrconfig.xml.
Please correct me if I am wrong.

Based on Ahmet's reply I created a CustomSearchComponent with help from the
net.
I created a dummy RH and added this as the searchComponent in
SolrConfig.xml:-


  exampleComponent

  

  
  

..invoked it using:-
http://localhost:8984/solr/myexample?q=*:*
The o/p gave me that one record will ALL FIELDS fully in xml format.
*It did not give only the "id" field which was what I was trying as a test!*


my code for the custom Search Component shared below please:-
Before that I have these queries:-
1. In my code,Instead of hitting the server AGAIN using SolrJ to enforce my
params (just "fl" newly added) , is there anyway the query can be
executed with my additional fl param.
2. Just adding the input additional params is what I want to achieve. I
dont want to do anything on the response.
   Currently I am doing it:-
   --> builder.rsp.add( "example", doc.getFields());
  Note:- I removed this line and again when I ran this query NO OUTPUT came.
  So suppose I used it along with any of my existing RH by adding in
searchcomponent, I want it to only affect the input   querying by adding
additional params and should not influence the rendering of the o/p in any
way. How do I add this to one of my existing Request Handlers only to
influence the input for querying and NOT o/p format in any way.
3. Why is all fields being rendered for the one doc I selected to come back
in my "example"  variable, when I am actually restricting the fields to
only =id

Any help is greatly appreciated.


My console shows the following ie ... looks like the filtering happens but
what is /select doing here when I am calling
/myexample:-

154956 [qtp1856056345-18] INFO  org.apache.solr.core.SolrCore  û
[collection1] w
ebapp=/solr path=*/select* params=*{q=*:*=id=xml=2.2}* hits=4
status=0
 QTime=16
[stored,indexed,tokenized,omitNorms,indexOptions=DOCS_ONLY,numericType=INT,numer
icPrecisionStep=16,
org.apache.lucene.document.LazyDocument$LazyFi
eld@38585de9, org.apache.lucene.document.LazyDocument$LazyField@1b3edb96,
org.ap
ache.lucene.document.LazyDocument$LazyField@9692beb,
org.apache.lucene.document.
LazyDocument$LazyField@683f4dad,
org.apache.lucene.document.LazyDocument$LazyFie
ld@12f2e256, org.apache.lucene.document.LazyDocument$LazyField@7ffd69f5]
155080 [qtp1856056345-21] INFO  org.apache.solr.core.SolrCore  û
[collection1] w
ebapp=/solr path=*/myexample* params={*q=*:**} status=0 QTime=299


rough java class:-

public class ExampleSearchComponent extends SearchComponent {

@Override
public void prepare(ResponseBuilder builder) throws IOException {

}

@Override
public void process(ResponseBuilder builder) throws IOException {
SolrParams params = builder.req.getParams();
String q = params.get(CommonParams.Q);
ModifiableSolrParams params1 = new ModifiableSolrParams(params);
params1.add("fl", "id");
System.out.println("q is "+q);

QueryResponse response=null;

HttpSolrServer server = new HttpSolrServer( "
http://localhost:8984/solr/collection1; );
server.setParser(new XMLResponseParser());

try{
  response = server.query( params1 );
}catch(Exception e){}

SolrDocumentList results = new SolrDocumentList();
SolrIndexSearcher searcher = builder.req.getSearcher();
Document doc=searcher.doc(0);
System.out.println(doc.getFields());


builder.rsp.add( "example", doc.getFields());
}


@Override
public String getDescription() {
return "ExampleSearchComponent";
}

@Override
public String getSource() {
return "";
}

//@Override
public String getSourceId() {
return "";
}

@Override
public String getVersion() {
return "1.0";
}
}





Thanks and Rgds,
Mark.

On Sat, Jan 9, 2016 at 12:38 PM, Jack Krupansky 
wrote:

> Sure, you CAN do this, but why would you want to? I mean, what exactly is
> the motivation here? If you truly have custom code to execute, fine, but if
> all you are trying to do is set parameters, a custom request handler is
> hitting a tack with a sledge hammer. For example, why isn't setting
> defaults in solrconfig sufficient for your needs? At least then you can
> change parameters with a simple text edit rather than require a Java build
> and jar deploy.
>
> Can you share what some of the requirements are for 

Re: Dynamically Adding query parameters in my custom Request Handler class

[Erik Hatcher]

Woah, Mark….  you’re making a search request within a search component.  
Instead, let the built-in “query” component do the work for you.

I think one fix for you is to make your “components” be “first-components” 
instead (allowing the other default search components to come into play).  You 
don’t need to search within your component, just affect parameters, right?


—
Erik Hatcher, Senior Solutions Architect
http://www.lucidworks.com 



> On Jan 9, 2016, at 3:19 PM, Mark Robinson  wrote:
> 
> Hi,
> 
> Ahmet, Jack, Thanks for the pointers.
> My requirement is, I would not be having the facets or sort fields or its
> order as static.
> For example suppose for a particular scenario I need to show only 2 facets
> and sort on only one field.
> For another scenario I may have to do facet.field for a different set of
> fields and sort on again another set of fields.
> Consider there are some sort of user preferences for each query.
> 
> So I think I may not be able to store my parameters like facet fields, sort
> fields etc preconfigured in solrconfig.xml.
> Please correct me if I am wrong.
> 
> Based on Ahmet's reply I created a CustomSearchComponent with help from the
> net.
> I created a dummy RH and added this as the searchComponent in
> SolrConfig.xml:-
> 
>
>  exampleComponent
>
>  
> 
>   class="org.ExampleSearchComponent">
>  
> 
> ..invoked it using:-
> http://localhost:8984/solr/myexample?q=*:*
> The o/p gave me that one record will ALL FIELDS fully in xml format.
> *It did not give only the "id" field which was what I was trying as a test!*
> 
> 
> my code for the custom Search Component shared below please:-
> Before that I have these queries:-
> 1. In my code,Instead of hitting the server AGAIN using SolrJ to enforce my
> params (just "fl" newly added) , is there anyway the query can be
> executed with my additional fl param.
> 2. Just adding the input additional params is what I want to achieve. I
> dont want to do anything on the response.
>   Currently I am doing it:-
>   --> builder.rsp.add( "example", doc.getFields());
>  Note:- I removed this line and again when I ran this query NO OUTPUT came.
>  So suppose I used it along with any of my existing RH by adding in
> searchcomponent, I want it to only affect the input   querying by adding
> additional params and should not influence the rendering of the o/p in any
> way. How do I add this to one of my existing Request Handlers only to
> influence the input for querying and NOT o/p format in any way.
> 3. Why is all fields being rendered for the one doc I selected to come back
> in my "example"  variable, when I am actually restricting the fields to
> only =id
> 
> Any help is greatly appreciated.
> 
> 
> My console shows the following ie ... looks like the filtering happens but
> what is /select doing here when I am calling
> /myexample:-
> 
> 154956 [qtp1856056345-18] INFO  org.apache.solr.core.SolrCore  û
> [collection1] w
> ebapp=/solr path=*/select* params=*{q=*:*=id=xml=2.2}* hits=4
> status=0
> QTime=16
> [stored,indexed,tokenized,omitNorms,indexOptions=DOCS_ONLY,numericType=INT,numer
> icPrecisionStep=16,
> org.apache.lucene.document.LazyDocument$LazyFi
> eld@38585de9, org.apache.lucene.document.LazyDocument$LazyField@1b3edb96,
> org.ap
> ache.lucene.document.LazyDocument$LazyField@9692beb,
> org.apache.lucene.document.
> LazyDocument$LazyField@683f4dad,
> org.apache.lucene.document.LazyDocument$LazyFie
> ld@12f2e256, org.apache.lucene.document.LazyDocument$LazyField@7ffd69f5]
> 155080 [qtp1856056345-21] INFO  org.apache.solr.core.SolrCore  û
> [collection1] w
> ebapp=/solr path=*/myexample* params={*q=*:**} status=0 QTime=299
> 
> 
> rough java class:-
> 
> public class ExampleSearchComponent extends SearchComponent {
> 
> @Override
>public void prepare(ResponseBuilder builder) throws IOException {
> 
> }
> 
>@Override
>public void process(ResponseBuilder builder) throws IOException {
>SolrParams params = builder.req.getParams();
>String q = params.get(CommonParams.Q);
>ModifiableSolrParams params1 = new ModifiableSolrParams(params);
>params1.add("fl", "id");
>System.out.println("q is "+q);
> 
>QueryResponse response=null;
> 
>HttpSolrServer server = new HttpSolrServer( "
> http://localhost:8984/solr/collection1; );
>server.setParser(new XMLResponseParser());
> 
>try{
>  response = server.query( params1 );
>}catch(Exception e){}
> 
>SolrDocumentList results = new SolrDocumentList();
>SolrIndexSearcher searcher = builder.req.getSearcher();
>Document doc=searcher.doc(0);
>System.out.println(doc.getFields());
> 
> 
>builder.rsp.add( "example", doc.getFields());
>}
> 
> 
>@Override
>public String getDescription() {
>return "ExampleSearchComponent";
>}
> 
>@Override
>public String getSource() {
>

Re: Dynamically Adding query parameters in my custom Request Handler class

[Mark Robinson]

Thanks Eric!

Appreciate your valuable suggestions.
Now I am getting the concept of a search-component better!

So my custom class is just this after removing the SOLRJ part, as I just
need to modify the query by adding some parameters dynamically before the
query actually is executed by SOLR:-

public void process(ResponseBuilder builder) throws IOException {
SolrParams *params *= builder.req.getParams();
String q = params.get(CommonParams.Q);
ModifiableSolrParams *params1* = new ModifiableSolrParams(*params*);
*params1.add*("fl", "id");
//Added this line
*builder.req.setParams(params1);*

System.out.println("q is ### "+q);
}

Note:- Nothing inside prepare() method,

In my /select RH added the following in solrconfig.xml just bef close of
the   tag:-


 exampleComponent
 

 

Still it is not restricting o/p fields to only fl.
The console output shows the following:-
q is ### *:*
16140 [qtp1856056345-12] INFO  org.apache.solr.core.SolrCore  û
[collection1] we
bapp=/solr path=/select params={q=*:*} hits=4 status=0 QTime=4

Note:- the  "###" proves that it accessed the custom class. But the
ModifiableSolrParams params1 = new ModifiableSolrParams(params);
params1.add("fl", "id");

did not take effect.

I think I am near to developing my first dynamic query component. Could
some one please tell me where I am going wrong this time.

Appreciate any help.I am very eager to see my first dynamic query
implemented using a customized Search Component!!

Thanks!
Mark.

On Sat, Jan 9, 2016 at 3:38 PM, Erik Hatcher  wrote:

> Woah, Mark….  you’re making a search request within a search component.
> Instead, let the built-in “query” component do the work for you.
>
> I think one fix for you is to make your “components” be “first-components”
> instead (allowing the other default search components to come into play).
> You don’t need to search within your component, just affect parameters,
> right?
>
>
> —
> Erik Hatcher, Senior Solutions Architect
> http://www.lucidworks.com 
>
>
>
> > On Jan 9, 2016, at 3:19 PM, Mark Robinson 
> wrote:
> >
> > Hi,
> >
> > Ahmet, Jack, Thanks for the pointers.
> > My requirement is, I would not be having the facets or sort fields or its
> > order as static.
> > For example suppose for a particular scenario I need to show only 2
> facets
> > and sort on only one field.
> > For another scenario I may have to do facet.field for a different set of
> > fields and sort on again another set of fields.
> > Consider there are some sort of user preferences for each query.
> >
> > So I think I may not be able to store my parameters like facet fields,
> sort
> > fields etc preconfigured in solrconfig.xml.
> > Please correct me if I am wrong.
> >
> > Based on Ahmet's reply I created a CustomSearchComponent with help from
> the
> > net.
> > I created a dummy RH and added this as the searchComponent in
> > SolrConfig.xml:-
> > 
> >
> >  exampleComponent
> >
> >  
> >
> >   > class="org.ExampleSearchComponent">
> >  
> >
> > ..invoked it using:-
> > http://localhost:8984/solr/myexample?q=*:*
> > The o/p gave me that one record will ALL FIELDS fully in xml format.
> > *It did not give only the "id" field which was what I was trying as a
> test!*
> >
> >
> > my code for the custom Search Component shared below please:-
> > Before that I have these queries:-
> > 1. In my code,Instead of hitting the server AGAIN using SolrJ to enforce
> my
> > params (just "fl" newly added) , is there anyway the query can be
> > executed with my additional fl param.
> > 2. Just adding the input additional params is what I want to achieve. I
> > dont want to do anything on the response.
> >   Currently I am doing it:-
> >   --> builder.rsp.add( "example", doc.getFields());
> >  Note:- I removed this line and again when I ran this query NO OUTPUT
> came.
> >  So suppose I used it along with any of my existing RH by adding in
> > searchcomponent, I want it to only affect the input   querying by adding
> > additional params and should not influence the rendering of the o/p in
> any
> > way. How do I add this to one of my existing Request Handlers only to
> > influence the input for querying and NOT o/p format in any way.
> > 3. Why is all fields being rendered for the one doc I selected to come
> back
> > in my "example"  variable, when I am actually restricting the fields to
> > only =id
> >
> > Any help is greatly appreciated.
> >
> >
> > My console shows the following ie ... looks like the filtering happens
> but
> > what is /select doing here when I am calling
> > /myexample:-
> >
> > 154956 [qtp1856056345-18] INFO  org.apache.solr.core.SolrCore  û
> > [collection1] w
> > ebapp=/solr path=*/select* params=*{q=*:*=id=xml=2.2}*
> hits=4
> > status=0
> > QTime=16
> >
> [stored,indexed,tokenized,omitNorms,indexOptions=DOCS_ONLY,numericType=INT,numer
> > 

Re: Dynamically Adding query parameters in my custom Request Handler class

[Ahmet Arslan]

Hi Mark,

Try using set method instead of add method :  params1.set("fl", "id");

I also suggest to use static String for "fl" as you used  CommonParams.Q for "q"

Congrats for your first search component!

happy searching,


Ahmet



On Saturday, January 9, 2016 11:32 PM, Mark Robinson  
wrote:
Thanks Eric!

Appreciate your valuable suggestions.
Now I am getting the concept of a search-component better!

So my custom class is just this after removing the SOLRJ part, as I just
need to modify the query by adding some parameters dynamically before the
query actually is executed by SOLR:-

public void process(ResponseBuilder builder) throws IOException {
SolrParams *params *= builder.req.getParams();
String q = params.get(CommonParams.Q);
ModifiableSolrParams *params1* = new ModifiableSolrParams(*params*);
*params1.add*("fl", "id");
//Added this line
*builder.req.setParams(params1);*

System.out.println("q is ### "+q);
}

Note:- Nothing inside prepare() method,

In my /select RH added the following in solrconfig.xml just bef close of
the   tag:-


 exampleComponent




Still it is not restricting o/p fields to only fl.
The console output shows the following:-
q is ### *:*
16140 [qtp1856056345-12] INFO  org.apache.solr.core.SolrCore  û
[collection1] we
bapp=/solr path=/select params={q=*:*} hits=4 status=0 QTime=4

Note:- the  "###" proves that it accessed the custom class. But the
ModifiableSolrParams params1 = new ModifiableSolrParams(params);
params1.add("fl", "id");

did not take effect.

I think I am near to developing my first dynamic query component. Could
some one please tell me where I am going wrong this time.

Appreciate any help.I am very eager to see my first dynamic query
implemented using a customized Search Component!!

Thanks!
Mark.


On Sat, Jan 9, 2016 at 3:38 PM, Erik Hatcher  wrote:

> Woah, Mark….  you’re making a search request within a search component.
> Instead, let the built-in “query” component do the work for you.
>
> I think one fix for you is to make your “components” be “first-components”
> instead (allowing the other default search components to come into play).
> You don’t need to search within your component, just affect parameters,
> right?
>
>
> —
> Erik Hatcher, Senior Solutions Architect
> http://www.lucidworks.com 
>
>
>
> > On Jan 9, 2016, at 3:19 PM, Mark Robinson 
> wrote:
> >
> > Hi,
> >
> > Ahmet, Jack, Thanks for the pointers.
> > My requirement is, I would not be having the facets or sort fields or its
> > order as static.
> > For example suppose for a particular scenario I need to show only 2
> facets
> > and sort on only one field.
> > For another scenario I may have to do facet.field for a different set of
> > fields and sort on again another set of fields.
> > Consider there are some sort of user preferences for each query.
> >
> > So I think I may not be able to store my parameters like facet fields,
> sort
> > fields etc preconfigured in solrconfig.xml.
> > Please correct me if I am wrong.
> >
> > Based on Ahmet's reply I created a CustomSearchComponent with help from
> the
> > net.
> > I created a dummy RH and added this as the searchComponent in
> > SolrConfig.xml:-
> > 
> >
> >  exampleComponent
> >
> >  
> >
> >   > class="org.ExampleSearchComponent">
> >  
> >
> > ..invoked it using:-
> > http://localhost:8984/solr/myexample?q=*:*
> > The o/p gave me that one record will ALL FIELDS fully in xml format.
> > *It did not give only the "id" field which was what I was trying as a
> test!*
> >
> >
> > my code for the custom Search Component shared below please:-
> > Before that I have these queries:-
> > 1. In my code,Instead of hitting the server AGAIN using SolrJ to enforce
> my
> > params (just "fl" newly added) , is there anyway the query can be
> > executed with my additional fl param.
> > 2. Just adding the input additional params is what I want to achieve. I
> > dont want to do anything on the response.
> >   Currently I am doing it:-
> >   --> builder.rsp.add( "example", doc.getFields());
> >  Note:- I removed this line and again when I ran this query NO OUTPUT
> came.
> >  So suppose I used it along with any of my existing RH by adding in
> > searchcomponent, I want it to only affect the input   querying by adding
> > additional params and should not influence the rendering of the o/p in
> any
> > way. How do I add this to one of my existing Request Handlers only to
> > influence the input for querying and NOT o/p format in any way.
> > 3. Why is all fields being rendered for the one doc I selected to come
> back
> > in my "example"  variable, when I am actually restricting the fields to
> > only =id
> >
> > Any help is greatly appreciated.
> >
> >
> > My console shows the following ie ... looks like the filtering happens
> but
> > what is /select doing here 

Re: Dynamically Adding query parameters in my custom Request Handler class

[Ahmet Arslan]

Hi Mark,

Yes this is possible. Better, you can use a custom SearchComponent for this 
task too.
You retrieve solr parameters, wrap it into ModifiableSolrParams. Add extra 
parameters etc, then pass it to underlying search components.

Ahmet


On Saturday, January 9, 2016 3:59 PM, Mark Robinson  
wrote:
Hi,
When I initially fire a query against my Solr instance using SOLRJ I pass
only, say q=*:*=(myfield:vaue1).

I have written a custom RequestHandler, which is what I call in my SolrJ
query.
Inside this custom request handler can I add more query params like say the
facets etc.. so that ultimately facets are also received back in my results
which were initially not specified when I invoked the Solr url using SolrJ.

In short, instead of constructing the query dynamically initially in SolrJ
I want to add the extra query params, adding a jar in Solr (a java code
that will check certain conditions and dynamically add the query params
after the initial SolrJ query is done). That is why I thought of a custom
RH which would help we write a java class and deploy in Solr.

Is this possible. Could some one get back please.

Thanks!
Mark.

Re: Dynamically Adding query parameters in my custom Request Handler class

[Jack Krupansky]

Sure, you CAN do this, but why would you want to? I mean, what exactly is
the motivation here? If you truly have custom code to execute, fine, but if
all you are trying to do is set parameters, a custom request handler is
hitting a tack with a sledge hammer. For example, why isn't setting
defaults in solrconfig sufficient for your needs? At least then you can
change parameters with a simple text edit rather than require a Java build
and jar deploy.

Can you share what some of the requirements are for your custom request
handler, including the motivation? I'd hate to see you go off and invest
significant effort in a custom request handler when simpler techniques may
suffice.

-- Jack Krupansky

On Sat, Jan 9, 2016 at 12:08 PM, Ahmet Arslan 
wrote:

> Hi Mark,
>
> Yes this is possible. Better, you can use a custom SearchComponent for
> this task too.
> You retrieve solr parameters, wrap it into ModifiableSolrParams. Add extra
> parameters etc, then pass it to underlying search components.
>
> Ahmet
>
>
> On Saturday, January 9, 2016 3:59 PM, Mark Robinson <
> mark123lea...@gmail.com> wrote:
> Hi,
> When I initially fire a query against my Solr instance using SOLRJ I pass
> only, say q=*:*=(myfield:vaue1).
>
> I have written a custom RequestHandler, which is what I call in my SolrJ
> query.
> Inside this custom request handler can I add more query params like say the
> facets etc.. so that ultimately facets are also received back in my results
> which were initially not specified when I invoked the Solr url using SolrJ.
>
> In short, instead of constructing the query dynamically initially in SolrJ
> I want to add the extra query params, adding a jar in Solr (a java code
> that will check certain conditions and dynamically add the query params
> after the initial SolrJ query is done). That is why I thought of a custom
> RH which would help we write a java class and deploy in Solr.
>
> Is this possible. Could some one get back please.
>
> Thanks!
> Mark.
>