Re: solrj question

[Shawn Heisey]

On 3/26/2018 11:19 AM, Webster Homer wrote:
> You may say that the String in the constructor is "meant to be query
> syntax", nothing in the Javadoc says anything about the expected syntax.
> Since there is also a method to set the query, it seemed reasonable to
> expect that it would take the output of the toString method. (or some other
> serialization method)

You're right that the javadoc is not very specific.  It says this:

Parameters:
    q - query string

In general in Solr, "query string" is understood to be something you
would put in the "q" parameter when you send a query.  Or maybe the "fq"
parameter.  The javadoc could definitely be improved.

The javadoc for the toString specifically used here is a little more
specific.  (SolrQuery inherits from SolrParams, and that's where the
toString method is defined):

https://lucene.apache.org/solr/6_6_0/solr-solrj/org/apache/solr/common/params/SolrParams.html#toString--

It says "so that the URL may be unambiguously pasted back into a browser."

> So how would a user play back logged queries? This seems like an important
> use case. I can parse the toString output, It seems like the constructor
> should be able to take it.
> If not a constructor and toString, methods, I don't see methods to
> serialize and deserialize the query
> Being able to write the complete query to a log is important, but we also
> want to be able to read the log and submit the query to solr. Being able to
> playback the logs allows us to  trouble shoot search issues on our site. It
> also provides a way to create load tests.
>
> Yes I can and am going to create this functionality, it's not that
> complicated, but I don't think it's unreasonable to think that the existing
> API should handle it.

Yes, that would be great capability to have.  But it hasn't been written
yet.  A method like "parseUrlString" on SolrQuery would be a good thing
to have.

Thanks,
Shawn

Re: solrj question

[Webster Homer]

You may say that the String in the constructor is "meant to be query
syntax", nothing in the Javadoc says anything about the expected syntax.
Since there is also a method to set the query, it seemed reasonable to
expect that it would take the output of the toString method. (or some other
serialization method)
https://lucene.apache.org/solr/6_6_0/solr-solrj/org/apache/solr/client/solrj/SolrQuery.html#SolrQuery-java.lang.String-

So how would a user play back logged queries? This seems like an important
use case. I can parse the toString output, It seems like the constructor
should be able to take it.
If not a constructor and toString, methods, I don't see methods to
serialize and deserialize the query
Being able to write the complete query to a log is important, but we also
want to be able to read the log and submit the query to solr. Being able to
playback the logs allows us to  trouble shoot search issues on our site. It
also provides a way to create load tests.

Yes I can and am going to create this functionality, it's not that
complicated, but I don't think it's unreasonable to think that the existing
API should handle it.

Thanks,


On Fri, Mar 23, 2018 at 6:44 PM, Shawn Heisey  wrote:

> On 3/23/2018 3:24 PM, Webster Homer wrote:
> > I see this in the output:
> > Lexical error at line 1, column 1759.  Encountered:  after :
> > "/select?defType=edismax=0=25&...
> > It has basically the entire solr query which it obviously couldn't parse.
> >
> > solrQuery = new SolrQuery(log.getQuery());
>
> This isn't going to work.  The string in the constructor is expected to
> be query syntax -- so something like this:
>
> company:Google AND (city:"San Jose" OR state:WA)
>
> It has no idea what to do with a URL path and parameters.
>
> > Is there something I'm doing wrong, or is it that the SolrQuery class
> > cannot really take its toString output to make itself? Does it have a
> > different serialization method that could be used?
>
> I don't think there's any expectation that an object's toString() output
> can be used as input for anything.  This is the javadoc for
> Object.toString():
>
> https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html#toString--
>
> The emphasis there is on human readability.  It is not intended for
> deserialization.  You *could* be looking at a toString() output like
> SolrQuery@1d44bcfa instead of something you can actually read.
>
> For the incomplete string shown in the error message you mentioned, you
> could do:
>
> SolrQuery q = new SolrQuery();
> q.setRequestHandler("/select");
> // The default handler is /select, so
> // the above is actually not necessary.
>
> q.set("defType", "edismax");
> q.set("start", "0");
> q.set("rows","25");
> // sugar method: q.setStart(0);
> // sugar method: q.setRows(25);
>
> Thanks,
> Shawn
>
>

-- 


This message and any attachment are confidential and may be privileged or 
otherwise protected from disclosure. If you are not the intended recipient, 
you must not copy this message or attachment or disclose the contents to 
any other person. If you have received this transmission in error, please 
notify the sender immediately and delete the message and any attachment 
from your system. Merck KGaA, Darmstadt, Germany and any of its 
subsidiaries do not accept liability for any omissions or errors in this 
message which may arise as a result of E-Mail-transmission or for damages 
resulting from any unauthorized changes of the content of this message and 
any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its 
subsidiaries do not guarantee that this message is free of viruses and does 
not accept liability for any damages caused by any virus transmitted 
therewith.

Click http://www.emdgroup.com/disclaimer to access the German, French, 
Spanish and Portuguese versions of this disclaimer.

Re: solrj question

[Shawn Heisey]

On 3/23/2018 3:24 PM, Webster Homer wrote:
> I see this in the output:
> Lexical error at line 1, column 1759.  Encountered:  after :
> "/select?defType=edismax=0=25&...
> It has basically the entire solr query which it obviously couldn't parse.
>
> solrQuery = new SolrQuery(log.getQuery());

This isn't going to work.  The string in the constructor is expected to
be query syntax -- so something like this:

company:Google AND (city:"San Jose" OR state:WA)

It has no idea what to do with a URL path and parameters.

> Is there something I'm doing wrong, or is it that the SolrQuery class
> cannot really take its toString output to make itself? Does it have a
> different serialization method that could be used?

I don't think there's any expectation that an object's toString() output
can be used as input for anything.  This is the javadoc for
Object.toString():

https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html#toString--

The emphasis there is on human readability.  It is not intended for
deserialization.  You *could* be looking at a toString() output like
SolrQuery@1d44bcfa instead of something you can actually read.

For the incomplete string shown in the error message you mentioned, you
could do:

SolrQuery q = new SolrQuery();
q.setRequestHandler("/select");
// The default handler is /select, so
// the above is actually not necessary.

q.set("defType", "edismax");
q.set("start", "0");
q.set("rows","25");
// sugar method: q.setStart(0);
// sugar method: q.setRows(25);

Thanks,
Shawn

solrj question

[Webster Homer]

I am working on a program to play back queries from a log file. It seemed
straight forward. The log has the solr query written to it. via the
SolrQuery.toString method. The SolrQuery class has a constructor which
takes a  string. It does instantiate a SolrQuery object, however when I try
to actually use it in a search, Solr throws an error that it is not able to
parse the query.

I see this in the output:
Lexical error at line 1, column 1759.  Encountered:  after :
"/select?defType=edismax=0=25&...
It has basically the entire solr query which it obviously couldn't parse.

solrQuery = new SolrQuery(log.getQuery());

the log.getQuery method just returns the query that was written to the log
with the toString() method

Is there something I'm doing wrong, or is it that the SolrQuery class
cannot really take its toString output to make itself? Does it have a
different serialization method that could be used?

This would be very useful functionality.

-- 


This message and any attachment are confidential and may be privileged or 
otherwise protected from disclosure. If you are not the intended recipient, 
you must not copy this message or attachment or disclose the contents to 
any other person. If you have received this transmission in error, please 
notify the sender immediately and delete the message and any attachment 
from your system. Merck KGaA, Darmstadt, Germany and any of its 
subsidiaries do not accept liability for any omissions or errors in this 
message which may arise as a result of E-Mail-transmission or for damages 
resulting from any unauthorized changes of the content of this message and 
any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its 
subsidiaries do not guarantee that this message is free of viruses and does 
not accept liability for any damages caused by any virus transmitted 
therewith.

Click http://www.emdgroup.com/disclaimer to access the German, French, 
Spanish and Portuguese versions of this disclaimer.

SolrJ Question about Bad Request Root cause error

[roz dev]

Hi All

We are using SolrJ client (v 1.4.1) to integrate with our solr search
server.
We notice that whenever SolrJ request does not match with Solr schema, we
get Bad Request exception which makes sense.

org.apache.solr.common.SolrException: Bad Request

But, SolrJ Client does not provide any clue about the reason request is Bad.

Is there any way to get the root cause on client side?

Of Course, solr server logs have enough info to know that data is bad but it
would be great
to have the same info in the exception generated by SolrJ.

Any thoughts? Is there any plan to add this in future releases?

Thanks,
Saroj

Re: SolrJ Question about Bad Request Root cause error

[Savvas-Andreas Moysidis]

good point! that's an enhancement we would definitely welcome as well.

currently, we too have to remote desktop to the Sol machine and search
through the logs..

Any thoughts?

Cheers,
-- Savvas

On 11 January 2011 19:59, roz dev rozde...@gmail.com wrote:

 Hi All

 We are using SolrJ client (v 1.4.1) to integrate with our solr search
 server.
 We notice that whenever SolrJ request does not match with Solr schema, we
 get Bad Request exception which makes sense.

 org.apache.solr.common.SolrException: Bad Request

 But, SolrJ Client does not provide any clue about the reason request is
 Bad.

 Is there any way to get the root cause on client side?

 Of Course, solr server logs have enough info to know that data is bad but
 it
 would be great
 to have the same info in the exception generated by SolrJ.

 Any thoughts? Is there any plan to add this in future releases?

 Thanks,
 Saroj

Solrj Question

[Neil Lott]

Hi,

I'm a little confused on how either solrj is working or how solr is working.  

I'm using solr 1.4.

@Test (groups = {integration}, enabled = true)
   public void testDate() throws Exception
   {
  SolrServer solr = 
SolrServerFactory.getStreamingUpdateSolrServer(searchDataIngestConfiguration.getSolrURL(solrConfigurationName),
 1, 1);

  // LicenseWindowStart 2010-01-21
  String result = SolrDocumentHelper.convertLongToSolrDate(126405720L);
  System.out.println(result);

  SolrInputDocument document = new SolrInputDocument();
  document.addField(created, result);
  UpdateResponse updateResponse = solr.add(document);
  solr.commit();

  SolrQuery parameters = new SolrQuery();
  parameters.set(q, *.*);
  QueryResponse queryResponse = solr.query(parameters);
  SolrDocumentList list = queryResponse.getResults();
  java.util.Date date = 
(java.util.Date)(list.get(0).getFieldValue(created));
  System.out.println(date.getTime());
  System.out.println(new DateTime(date.getTime()));
   }

126405720L // What I pass to solr
2010-01-21T00:00:00Z // What I pass to solr
126403200 // What I get back -- 7 hour difference from my original time
2010-01-20T17:00:00.000-07:00 // What I get Back
2010-06-29T15:54:43.855-06:00 // My current time

So essentially solr is giving me a java.util.Date back converted to my local 
time zone but not really.

On my box if I run
 date
 Tue Jun 29 15:47:49 MDT 2010

So why is solr doing a -7 hour conversion when I'm -6?

I also tried running solr on a jdk 1.5 and I get the same result.

Thanks,

Neil

SolrJ question

[Paul Tomblin]

If I put an object into a SolrInputDocument and store it, how do I
query for it back?  For instance, I stored a java.net.URI in a field
called url, and I want to query for all the documents that match a
particular URI.  The query syntax only seems to allow Strings, and if
I just try query.setQuery(url: + uri.toString()) I get an error
because of the colon after http in the URI.

I'm really new to Solr, so please let me know if I'm missing something
basic here.

-- 
http://www.linkedin.com/in/paultomblin

RE: SolrJ question

[Harsch, Timothy J. (ARC-SC)[PEROT SYSTEMS]]

Assuming you have written the SolrInputDocument to the server, you would next 
query.  See ClientUtils.escapeQueryChars.  Also you need to be cognizant of 
URLEncoding at times.

-Original Message-
From: ptomb...@gmail.com [mailto:ptomb...@gmail.com] On Behalf Of Paul Tomblin
Sent: Monday, August 17, 2009 2:12 PM
To: solr-user@lucene.apache.org
Subject: SolrJ question

If I put an object into a SolrInputDocument and store it, how do I
query for it back?  For instance, I stored a java.net.URI in a field
called url, and I want to query for all the documents that match a
particular URI.  The query syntax only seems to allow Strings, and if
I just try query.setQuery(url: + uri.toString()) I get an error
because of the colon after http in the URI.

I'm really new to Solr, so please let me know if I'm missing something
basic here.

-- 
http://www.linkedin.com/in/paultomblin

RE: SolrJ question

[Ensdorf Ken]

You can escape the string with

org.apache.lucene.queryParser.QueryParser.escape(String query)

http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/queryParser/QueryParser.html#escape%28java.lang.String%29



 -Original Message-
 From: ptomb...@gmail.com [mailto:ptomb...@gmail.com] On Behalf Of Paul
 Tomblin
 Sent: Monday, August 17, 2009 5:12 PM
 To: solr-user@lucene.apache.org
 Subject: SolrJ question

 If I put an object into a SolrInputDocument and store it, how do I
 query for it back?  For instance, I stored a java.net.URI in a field
 called url, and I want to query for all the documents that match a
 particular URI.  The query syntax only seems to allow Strings, and if
 I just try query.setQuery(url: + uri.toString()) I get an error
 because of the colon after http in the URI.

 I'm really new to Solr, so please let me know if I'm missing something
 basic here.

 --
 http://www.linkedin.com/in/paultomblin

Re: SolrJ question

[Paul Tomblin]

On Mon, Aug 17, 2009 at 5:28 PM, Harsch, Timothy J. (ARC-SC)[PEROT
SYSTEMS]timothy.j.har...@nasa.gov wrote:
 Assuming you have written the SolrInputDocument to the server, you would next 
 query.

I'm sorry, I don't understand what you mean by you would next query.
 There appear to be some words missing from that sentence.



-- 
http://www.linkedin.com/in/paultomblin

Re: SolrJ question

[Paul Tomblin]

On Mon, Aug 17, 2009 at 5:30 PM, Ensdorf Kenensd...@zoominfo.com wrote:
 You can escape the string with

 org.apache.lucene.queryParser.QueryParser.escape(String query)

 http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/queryParser/QueryParser.html#escape%28java.lang.String%29


Does this mean I should have converted my objects to string before
writing them to the server?

-- 
http://www.linkedin.com/in/paultomblin

RE: SolrJ question

[Ensdorf Ken]

 Does this mean I should have converted my objects to string before
 writing them to the server?


I believe SolrJ takes care of that for you by calling toString(), but you would 
need to convert explicitly when you query (and then escape).

Re: SolrJ question

[Paul Tomblin]

On Mon, Aug 17, 2009 at 5:36 PM, Ensdorf Kenensd...@zoominfo.com wrote:
 Does this mean I should have converted my objects to string before
 writing them to the server?


 I believe SolrJ takes care of that for you by calling toString(), but you 
 would need to convert explicitly when you query (and then escape).


Hmmm.  It's not working right.  I've added a 5 documents, 3 with the
URL set to http://xcski.com/pharma/; and 2 with the URL set to
http://xcski.com/nano/;.  Doing other sorts of queries seems to be
pulling back the right data:

 [DEBUG] 34:20 (Solr.java:getForConcept:116)
 [java] search term = fribbet, concept = pharma
 [java]
 [java] Aug 17, 2009 5:34:20 PM org.apache.solr.core.SolrCore execute
 [java] INFO: [] webapp=null path=/select
params={q=fribbetfq=concept%3Apharma} hits=1 status=0 QTime=9
 [java] [DEBUG] 34:20 (Solr.java:getForConcept:130)
 [java] got doc SolrDocument[{id=2:http://xcski.com/pharma/,
concept=pharma, text=this is a third big long chunk of text containing
the word fribbet, title=this is the third title, keywords=pills,drugs,
origDoctype=html, chunkNum=2, url=http://xcski.com/pharma/}]

 But if I want to restrict it to a specific URL, I use

   SolrQuery query = new SolrQuery();
query.setQuery(url: + ClientUtils.escapeQueryChars(url));

and it's not returning anything.  Log4j output looks like:

 [java] [DEBUG] 34:20 (Solr.java:getAllForURL:89)
 [java] getting for URL: http://xcski.com/nano/
 [java]
 [java] Aug 17, 2009 5:34:20 PM org.apache.solr.core.SolrCore execute
 [java] INFO: [] webapp=null path=/select
params={q=url%3Ahttp%5C%3A%5C%2F%5C%2Fxcski%5C.com%5C%2Fnano%5C%2F}
hits=0 status=0 QTime=16
 [java] [DEBUG] 34:20 (Solr.java:main:229)
 [java] found: 0

Actually, looking at that, it looks like it's escaped the URL twice,
converting : into %3A, then converting that to %5C%3A.  Could
that be?



-- 
http://www.linkedin.com/in/paultomblin

Re: SolrJ question

[Paul Tomblin]

On Mon, Aug 17, 2009 at 5:47 PM, Paul Tomblinptomb...@xcski.com wrote:

 Hmmm.  It's not working right.  I've added a 5 documents, 3 with the
 URL set to http://xcski.com/pharma/; and 2 with the URL set to
 http://xcski.com/nano/;.  Doing other sorts of queries seems to be
 pulling back the right data:


Of course, It doesn't help that my url field was set to
indexed=false in the schema.  Changing it to true fixed it.

-- 
http://www.linkedin.com/in/paultomblin