Re: ConcurrentUpdateSolrServer update response doesn't reflect correct response status

[Lasitha Wattaladeniya]

Hi Shawn,

Thanks for the detailed explanation, really informative. However after
further analyzing I found that I can use HttpSolrServer instead of
ConcurrentUpdateSolrServer,
I'm handling the concurrency by my self. But the issue with
ConcurrentUpdateSolrServer
is still there, seem like an interesting topic to look into.

Regards,
Lasitha

Lasitha Wattaladeniya
Software Engineer

Mobile : +6593896893
Blog : techreadme.blogspot.com

On Mon, Feb 13, 2017 at 10:01 PM, Shawn Heisey <apa...@elyograg.org> wrote:

> On 2/12/2017 10:51 PM, Lasitha Wattaladeniya wrote:
> > Thanks for the reply.  But even if I add a single document,  the response
> > doesn't reflect the correct response status.  From the given solr console
> > ui,  it shows status 400 for the bad requests (when I try to index
> document
> > with no fields ). But when I do the same using solrj,
> response
> > is 0 eventhough the console prints it as a bad request with status 400.
>
> The concurrent client swallows all indexing exceptions.  This is how it
> is designed.  In fact, it's potentially even worse than you might expect
> -- the client not only always returns a success for updates sent, it
> returns immediately, often before any updates are sent to the server.
> The updates are sent in the background.  If another request type is
> sent, like a commit, then the client will wait for all queued updates to
> be sent before it sends that request and returns.  If an error is
> encountered during an explicit commit, I believe that the client *is*
> notified about that, but I have not tested this.
>
> If you want multi-threaded indexing *and* error detection, you're going
> to need to handle the multiple threads in your own program, using the
> Http or Cloud client object instead.  The Concurrent object is good for
> initial bulk indexing, but not for the kind of indexing where you want
> to be informed about errors.
>
> I'm told that if you override the "handleError" method, you might be
> able to make it work how you want, but I was unable to figure out how to
> modify that method to achieve that result.
>
> I filed an issue and even came up with an idea for a fix, but nobody was
> excited about it.  Because of the design of the client, it wasn't
> possible to guarantee that no further indexing had occurred after the
> first error encountered.
>
> https://issues.apache.org/jira/browse/SOLR-3284
>
> Thanks,
> Shawn
>
>

Re: ConcurrentUpdateSolrServer update response doesn't reflect correct response status

[Shawn Heisey]

On 2/12/2017 10:51 PM, Lasitha Wattaladeniya wrote:
> Thanks for the reply.  But even if I add a single document,  the response
> doesn't reflect the correct response status.  From the given solr console
> ui,  it shows status 400 for the bad requests (when I try to index document
> with no fields ). But when I do the same using solrj,  response
> is 0 eventhough the console prints it as a bad request with status 400.

The concurrent client swallows all indexing exceptions.  This is how it
is designed.  In fact, it's potentially even worse than you might expect
-- the client not only always returns a success for updates sent, it
returns immediately, often before any updates are sent to the server. 
The updates are sent in the background.  If another request type is
sent, like a commit, then the client will wait for all queued updates to
be sent before it sends that request and returns.  If an error is
encountered during an explicit commit, I believe that the client *is*
notified about that, but I have not tested this.

If you want multi-threaded indexing *and* error detection, you're going
to need to handle the multiple threads in your own program, using the
Http or Cloud client object instead.  The Concurrent object is good for
initial bulk indexing, but not for the kind of indexing where you want
to be informed about errors.

I'm told that if you override the "handleError" method, you might be
able to make it work how you want, but I was unable to figure out how to
modify that method to achieve that result.

I filed an issue and even came up with an idea for a fix, but nobody was
excited about it.  Because of the design of the client, it wasn't
possible to guarantee that no further indexing had occurred after the
first error encountered.

https://issues.apache.org/jira/browse/SOLR-3284

Thanks,
Shawn

Re: ConcurrentUpdateSolrServer update response doesn't reflect correct response status

[Lasitha Wattaladeniya]

Hi Eric,

Thanks for the reply.  But even if I add a single document,  the response
doesn't reflect the correct response status.  From the given solr console
ui,  it shows status 400 for the bad requests (when I try to index document
with no fields ). But when I do the same using solrj,  response
is 0 eventhough the console prints it as a bad request with status 400.

P.S: I'm using solrj 4.10.4

Regards,
Lasitha

On 13 Feb 2017 13:33, "Erick Erickson" <erickerick...@gmail.com> wrote:

Closest that I know of is:
https://issues.apache.org/jira/browse/SOLR-445, available since Solr
6.1

Best,
Erick

On Sun, Feb 12, 2017 at 8:05 PM, Lasitha Wattaladeniya
<watt...@gmail.com> wrote:
> Hi devs,
>
> I'm using the ConcurrentUpdateSolrServer to add some documents to solr.
> While I'm doing so I see some 'bad request" errors printed in the console
> due to incorrect solrinputdocuments.
>
> The issue is I can't capture this issue by reading  updateresonse status.
> I saw some one had stated that this is due to ConcurrentUpdateSolrServer
> uses a queue to buffer the documents and then later call update method.
>
> Isn't there any way to capture the correct  response?
>
> Regards,
> Lasitha

Re: ConcurrentUpdateSolrServer update response doesn't reflect correct response status

[Erick Erickson]

Closest that I know of is:
https://issues.apache.org/jira/browse/SOLR-445, available since Solr
6.1

Best,
Erick

On Sun, Feb 12, 2017 at 8:05 PM, Lasitha Wattaladeniya
 wrote:
> Hi devs,
>
> I'm using the ConcurrentUpdateSolrServer to add some documents to solr.
> While I'm doing so I see some 'bad request" errors printed in the console
> due to incorrect solrinputdocuments.
>
> The issue is I can't capture this issue by reading  updateresonse status.
> I saw some one had stated that this is due to ConcurrentUpdateSolrServer
> uses a queue to buffer the documents and then later call update method.
>
> Isn't there any way to capture the correct  response?
>
> Regards,
> Lasitha

ConcurrentUpdateSolrServer update response doesn't reflect correct response status

[Lasitha Wattaladeniya]

Hi devs,

I'm using the ConcurrentUpdateSolrServer to add some documents to solr.
While I'm doing so I see some 'bad request" errors printed in the console
due to incorrect solrinputdocuments.

The issue is I can't capture this issue by reading  updateresonse status.
I saw some one had stated that this is due to ConcurrentUpdateSolrServer
uses a queue to buffer the documents and then later call update method.

Isn't there any way to capture the correct  response?

Regards,
Lasitha

Re: Solr query - response status

[Shyam R]

Thanks Shawn for your insight!

On Fri, Jul 22, 2016 at 6:32 PM, Shawn Heisey  wrote:

> On 7/22/2016 12:41 AM, Shyam R wrote:
> > I see that SOLR returns status value as 0 for successful searches
> > org.apache.solr.core.SolrCore; [users_shadow_shard1_replica1]
> > webapp=/solr path=/user/ping params={} status=0 QTime=0 I do see that
> > the status come's back as 400 whenever the search is invalid (
> > invoking query with parameters that are not available in the target
> > collection ) What are the legitimate values of status and reason for
> > choosing 0?
>
> Solr (Jetty, really) sends back "200" for the HTTP status code when the
> request status is zero.
>
> The reason Solr uses a status of zero internally has its origins in the
> way most operating systems deal with program exit codes.  Almost
> universally, when a program exits with an exit code of 0, it tells the
> operating system that the exit was normal, no errors.  Any positive
> number indicates some kind of error.  The reason this is not reversed is
> simple -- unlike HTTP, which has multiple codes meaning success,
> operating systems must handle many different error codes, but only one
> success code.  So the success code is assigned to the number that's
> inherently different from the rest -- zero.
>
> Internally, Solr doesn't necessarily know that the response is going to
> use HTTP, although that is the most common method.  In the mind of a
> typical open source developer, an exit status of ANY positive number
> means there was an error, including 200.  Once control is handed off to
> Jetty, then the zero success status is translated to the most-used
> success code for HTTP.
>
> Any number could *potentially* be valid for the status in Solr logs, but
> I've only ever seen zero, 40x, and 50x.  The 40x series means there was
> a problem detected in the request, 50x means an error happened inside
> Solr itself after the request was determined to be good.  The ping
> handler will return a 503 statusif the health check is put into a
> disabledstate.
>
> Thanks,
> Shawn
>
>


-- 
Ph: 9845704792

Re: Solr query - response status

[Shawn Heisey]

On 7/22/2016 12:41 AM, Shyam R wrote:
> I see that SOLR returns status value as 0 for successful searches
> org.apache.solr.core.SolrCore; [users_shadow_shard1_replica1]
> webapp=/solr path=/user/ping params={} status=0 QTime=0 I do see that
> the status come's back as 400 whenever the search is invalid (
> invoking query with parameters that are not available in the target
> collection ) What are the legitimate values of status and reason for
> choosing 0? 

Solr (Jetty, really) sends back "200" for the HTTP status code when the
request status is zero.

The reason Solr uses a status of zero internally has its origins in the
way most operating systems deal with program exit codes.  Almost
universally, when a program exits with an exit code of 0, it tells the
operating system that the exit was normal, no errors.  Any positive
number indicates some kind of error.  The reason this is not reversed is
simple -- unlike HTTP, which has multiple codes meaning success,
operating systems must handle many different error codes, but only one
success code.  So the success code is assigned to the number that's
inherently different from the rest -- zero.

Internally, Solr doesn't necessarily know that the response is going to
use HTTP, although that is the most common method.  In the mind of a
typical open source developer, an exit status of ANY positive number
means there was an error, including 200.  Once control is handed off to
Jetty, then the zero success status is translated to the most-used
success code for HTTP.

Any number could *potentially* be valid for the status in Solr logs, but
I've only ever seen zero, 40x, and 50x.  The 40x series means there was
a problem detected in the request, 50x means an error happened inside
Solr itself after the request was determined to be good.  The ping
handler will return a 503 statusif the health check is put into a
disabledstate.

Thanks,
Shawn

Solr query - response status

[Shyam R]

All,

I see that SOLR returns status value as 0 for successful searches

org.apache.solr.core.SolrCore; [users_shadow_shard1_replica1] webapp=/solr
path=/user/ping params={} status=0 QTime=0

I do see that the status come's back as 400 whenever the search is invalid
( invoking query with parameters that are not available in the target
collection )

What are the legitimate values of status and reason for choosing 0?


Thanks
Shyam
-- 
Ph: 9845704792

Re: Response status

[Jens Ellenberg]

Hello,

Is there a reference to this status-codes?


Erik Hatcher wrote
 
 It means the request was successful.  If the status is non-zero (err,  
 1) then there was an error of some sort.
 
   Erik
 
 On Dec 4, 2008, at 9:32 AM, Robert Young wrote:
 
 In the standard response format, what does the status mean? It  
 always seems
 to be 0.

 Thanks
 Rob
 


--
View this message in context: 
http://lucene.472066.n3.nabble.com/Response-status-tp490876p3702747.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Response status

[Erik Hatcher]

On Jan 31, 2012, at 04:42 , Jens Ellenberg wrote:

 Hello,
 
 Is there a reference to this status-codes?

Just the source code.  SolrCore#setResponseHeaderValues, which predominately 
uses the codes specified in SolrException:

BAD_REQUEST( 400 ),
UNAUTHORIZED( 401 ),  // not currently used
FORBIDDEN( 403 ),
NOT_FOUND( 404 ),
SERVER_ERROR( 500 ),
SERVICE_UNAVAILABLE( 503 ),
UNKNOWN(0);

Erik


 
 
 Erik Hatcher wrote
 
 It means the request was successful.  If the status is non-zero (err,  
 1) then there was an error of some sort.
 
  Erik
 
 On Dec 4, 2008, at 9:32 AM, Robert Young wrote:
 
 In the standard response format, what does the status mean? It  
 always seems
 to be 0.
 
 Thanks
 Rob
 
 
 
 --
 View this message in context: 
 http://lucene.472066.n3.nabble.com/Response-status-tp490876p3702747.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: Response status

[Jens Ellenberg]

Thanks,

this helps a lot

greetings
Jens

Am 31.01.2012 13:53, schrieb Erik Hatcher-4 [via Lucene]:

 On Jan 31, 2012, at 04:42 , Jens Ellenberg wrote:

  Hello,
 
  Is there a reference to this status-codes?

 Just the source code.  SolrCore#setResponseHeaderValues, which 
 predominately uses the codes specified in SolrException:

 BAD_REQUEST( 400 ),
 UNAUTHORIZED( 401 ),  // not currently used
 FORBIDDEN( 403 ),
 NOT_FOUND( 404 ),
 SERVER_ERROR( 500 ),
 SERVICE_UNAVAILABLE( 503 ),
 UNKNOWN(0);

 Erik


 
 
  Erik Hatcher wrote
 
  It means the request was successful.  If the status is non-zero (err,
  1) then there was an error of some sort.
 
  Erik
 
  On Dec 4, 2008, at 9:32 AM, Robert Young wrote:
 
  In the standard response format, what does the status mean? It
  always seems
  to be 0.
 
  Thanks
  Rob
 
 
 
  --
  View this message in context: 
 http://lucene.472066.n3.nabble.com/Response-status-tp490876p3702747.html
  Sent from the Solr - User mailing list archive at Nabble.com.



 
 If you reply to this email, your message will be added to the 
 discussion below:
 http://lucene.472066.n3.nabble.com/Response-status-tp490876p3703172.html
 To unsubscribe from Response status, click here 
 http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=490876code=ZWxsZW5iZXJnQHNpbHBpb24uZGV8NDkwODc2fDc0MjIyMzQ1Nw==.
 NAML 
 http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
  




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Response-status-tp490876p3703708.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: response status: error 400

[Elaine Li]

Thanks.

I am pretty sure the url is no problem. The query syntax has a
problem, so it returns 400. I just want to grab that value.

On Thu, Oct 22, 2009 at 4:50 AM, James liu liuping.ja...@gmail.com wrote:
 Are you sure the url is correct?


 --
 regards
 j.L ( I live in Shanghai, China)

Re: response status: error 400

[Chris Hostetter]

: I am pretty sure the url is no problem. The query syntax has a
: problem, so it returns 400. I just want to grab that value.

When Solr encounters an error, it returns the status code and error 
message w/o specifying any sort of error page.  The servlet container 
(jetty, resin, tomcat, etc...) then decides how to format it.  So you 
could look into configureing a custom error page in your servlet container 
to generate it a format your client code can understand.

There is an open issue in Jira to make it possible for the response 
writers to format the errors, but it hasn't gotten a lot of attention over 
the years...

https://issues.apache.org/jira/browse/SOLR-141



-Hoss

response status: error 400

[Elaine Li]

Hi,

I am using Javascript and JSON to submit the solr query. I want to get
the reponse status so if there is a 400 http error, i will issue an
error instead of doing nothing which is very confusing to the user.
Hope I can even grab the message body.

How do i do that?

Thanks.

Elaine

Response status

[Robert Young]

In the standard response format, what does the status mean? It always seems
to be 0.

Thanks
Rob

Re: Response status

[Erik Hatcher]

It means the request was successful.  If the status is non-zero (err,  
1) then there was an error of some sort.


Erik

On Dec 4, 2008, at 9:32 AM, Robert Young wrote:

In the standard response format, what does the status mean? It  
always seems

to be 0.

Thanks
Rob

Re: Response status

[Robert Young]

Thanks

On Thu, Dec 4, 2008 at 2:53 PM, Erik Hatcher [EMAIL PROTECTED]wrote:

 It means the request was successful.  If the status is non-zero (err, 1)
 then there was an error of some sort.

Erik


 On Dec 4, 2008, at 9:32 AM, Robert Young wrote:

  In the standard response format, what does the status mean? It always
 seems
 to be 0.

 Thanks
 Rob