Re: Intermittent error 401 with JSON Facet query to retrieve count all collections
Hi Jason,
Yes. I am using the latest Solr 8.1.1.
The query which I'm using is the JSON Facet query which I faced the error
initially.
Regards,
Edwin
On Tue, 4 Jun 2019 at 20:15, Jason Gerlowski wrote:
> Hi Edwin,
>
> Thanks for the additional datapoint. It seemed to work for me, but we
> don't really understand the problem yet, so maybe it's not a solid
> work around like I'd hoped. I'm curious to hear whether it works for
> Colvin.
>
> To double check though: forwardCredentials is only supported in Solr >
> 8.0. You're using an 8.x version, right?
>
> Jason
>
> On Tue, Jun 4, 2019 at 2:45 AM Zheng Lin Edwin Yeo
> wrote:
> >
> > Hi Jason,
> >
> > Thanks for your reply.
> >
> > I have tried to add the "forwardCredentials": true in the security.json,
> > but I still get the same error.
> >
> > Regards,
> > Edwin
> >
> > On Mon, 3 Jun 2019 at 22:19, Colvin Cowie
> > wrote:
> >
> > > Hi, thanks I'll give that a go when I get a chance.
> > >
> > > I was trying to reply to an older thread (
> > >
> > >
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201904.mbox/%3CCAF2DzVXeVZqnixnkbzw0La1ui5N5-RG9PwfMBHG9vmkfBSMzJA%40mail.gmail.com%3E
> > > ),
> > > which I don't have in my mailbox, so obviously didn't reply to the
> right
> > > address to get my response threaded so mine has appeared on its own.
> Oops.
> > >
> > > A JIRA issue was raised on that thread
> > > https://issues.apache.org/jira/browse/SOLR-13421 but it's not had any
> > > attention.
> > >
> > >
> > > On Mon, 3 Jun 2019 at 14:46, Jason Gerlowski
> > > wrote:
> > >
> > > > Hi Colvin,
> > > >
> > > > We're still taking a look at fixing the bug, but as a workaround in
> > > > the meantime, you can look into adding a "forwardCredentials":true
> > > > property under the "authentication" section of security.json. That
> > > > seems to fix the issue in my reproduction at least.
> > > >
> > > > e.g.
> > > >
> > > > {
> > > > "authentication": {
> > > > "blockUnknown": true,
> > > > "class": "solr.BasicAuthPlugin",
> > > > "credentials": {
> > > > "solradmin": ""
> > > > },
> > > > "forwardCredentials": true
> > > > },
> > > > ...
> > > > }
> > > >
> > > > Jason
> > > >
> > > > On Mon, Jun 3, 2019 at 9:31 AM Jason Gerlowski <
> gerlowsk...@gmail.com>
> > > > wrote:
> > > > >
> > > > > One last note: as far as I can tell, nothing about this issue is
> > > > > specific to JSON Faceting or the JSON request API. It can be
> > > > > triggered just as easily with "/select?q=*:*".
> > > > >
> > > > > The bug created for this is: SOLR-13510
> > > > >
> > > > > On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski <
> gerlowsk...@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > I'm also able to reproduce this bug on master. A few more notes
> > > about
> > > > > > the bad behavior:
> > > > > >
> > > > > > - the behavior occurs regardless of the specific permissions
> > > > > > configured in security.json. (i.e. whether the top permission is
> > > > > > "all", or "security-edit", or there are no permissions at all.)
> > > > > > - I tried looking for a pattern in which requests saw the 401s,
> but
> > > > > > didn't have any luck. The 401 occurs when talking to the whole
> > > > > > collection or targeting individual cores directly. It occurs
> when
> > > > > > curl hits a host containing a replica for the collection in
> question,
> > > > > > and when it doesnt. etc. This distinguishes it from SOLR-13472,
> > > which
> > > > > > seems more specific to collection structure/layout.
> > > > > >
> > > > > > I'll create a bug for this in JIRA.
> > > > > >
> > > > > > On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie <
> > > > colvin.cowie@gmail.com> wrote:
> > > > > > >
> > > > > > > Hello. I encountered this issue too and wrote this up before I
> > > found
> > > > this
> > > > > > > thread, but I thought I might as well post it still, if it
> helps...
> > > > > > >
> > > > > > > Currently I'm trying to move our product on to Solr 8.1.1. We
> are
> > > > currently
> > > > > > > using 6.6.6, so things have definitely moved on.
> > > > > > >
> > > > > > > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to
> lock
> > > > down Solr
> > > > > > > (and we also secure our zookeeper). Here's an example for
> solradmin
> > > > as the
> > > > > > > user and password
> > > > > > >
> > > > > > > {
> > > > > > > "authentication": {
> > > > > > > "blockUnknown": true,
> > > > > > > "class": "solr.BasicAuthPlugin",
> > > > > > > "credentials": {
> > > > > > > "solradmin":
> > > > "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > > > > > > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> > > > > > > }
> > > > > > > },
> > > > > > > "authorization": {
> > > > > > > "class": "solr.RuleBasedAuthorizationPlugin",
> > > > > > > "permissions": [
> > > > > > > {
> > > > > > > "name": "all",
> > > > > > >
Re: Intermittent error 401 with JSON Facet query to retrieve count all collections
Hi Edwin,
Thanks for the additional datapoint. It seemed to work for me, but we
don't really understand the problem yet, so maybe it's not a solid
work around like I'd hoped. I'm curious to hear whether it works for
Colvin.
To double check though: forwardCredentials is only supported in Solr >
8.0. You're using an 8.x version, right?
Jason
On Tue, Jun 4, 2019 at 2:45 AM Zheng Lin Edwin Yeo wrote:
>
> Hi Jason,
>
> Thanks for your reply.
>
> I have tried to add the "forwardCredentials": true in the security.json,
> but I still get the same error.
>
> Regards,
> Edwin
>
> On Mon, 3 Jun 2019 at 22:19, Colvin Cowie
> wrote:
>
> > Hi, thanks I'll give that a go when I get a chance.
> >
> > I was trying to reply to an older thread (
> >
> > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201904.mbox/%3CCAF2DzVXeVZqnixnkbzw0La1ui5N5-RG9PwfMBHG9vmkfBSMzJA%40mail.gmail.com%3E
> > ),
> > which I don't have in my mailbox, so obviously didn't reply to the right
> > address to get my response threaded so mine has appeared on its own. Oops.
> >
> > A JIRA issue was raised on that thread
> > https://issues.apache.org/jira/browse/SOLR-13421 but it's not had any
> > attention.
> >
> >
> > On Mon, 3 Jun 2019 at 14:46, Jason Gerlowski
> > wrote:
> >
> > > Hi Colvin,
> > >
> > > We're still taking a look at fixing the bug, but as a workaround in
> > > the meantime, you can look into adding a "forwardCredentials":true
> > > property under the "authentication" section of security.json. That
> > > seems to fix the issue in my reproduction at least.
> > >
> > > e.g.
> > >
> > > {
> > > "authentication": {
> > > "blockUnknown": true,
> > > "class": "solr.BasicAuthPlugin",
> > > "credentials": {
> > > "solradmin": ""
> > > },
> > > "forwardCredentials": true
> > > },
> > > ...
> > > }
> > >
> > > Jason
> > >
> > > On Mon, Jun 3, 2019 at 9:31 AM Jason Gerlowski
> > > wrote:
> > > >
> > > > One last note: as far as I can tell, nothing about this issue is
> > > > specific to JSON Faceting or the JSON request API. It can be
> > > > triggered just as easily with "/select?q=*:*".
> > > >
> > > > The bug created for this is: SOLR-13510
> > > >
> > > > On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski
> > > wrote:
> > > > >
> > > > > I'm also able to reproduce this bug on master. A few more notes
> > about
> > > > > the bad behavior:
> > > > >
> > > > > - the behavior occurs regardless of the specific permissions
> > > > > configured in security.json. (i.e. whether the top permission is
> > > > > "all", or "security-edit", or there are no permissions at all.)
> > > > > - I tried looking for a pattern in which requests saw the 401s, but
> > > > > didn't have any luck. The 401 occurs when talking to the whole
> > > > > collection or targeting individual cores directly. It occurs when
> > > > > curl hits a host containing a replica for the collection in question,
> > > > > and when it doesnt. etc. This distinguishes it from SOLR-13472,
> > which
> > > > > seems more specific to collection structure/layout.
> > > > >
> > > > > I'll create a bug for this in JIRA.
> > > > >
> > > > > On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie <
> > > colvin.cowie@gmail.com> wrote:
> > > > > >
> > > > > > Hello. I encountered this issue too and wrote this up before I
> > found
> > > this
> > > > > > thread, but I thought I might as well post it still, if it helps...
> > > > > >
> > > > > > Currently I'm trying to move our product on to Solr 8.1.1. We are
> > > currently
> > > > > > using 6.6.6, so things have definitely moved on.
> > > > > >
> > > > > > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock
> > > down Solr
> > > > > > (and we also secure our zookeeper). Here's an example for solradmin
> > > as the
> > > > > > user and password
> > > > > >
> > > > > > {
> > > > > > "authentication": {
> > > > > > "blockUnknown": true,
> > > > > > "class": "solr.BasicAuthPlugin",
> > > > > > "credentials": {
> > > > > > "solradmin":
> > > "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > > > > > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> > > > > > }
> > > > > > },
> > > > > > "authorization": {
> > > > > > "class": "solr.RuleBasedAuthorizationPlugin",
> > > > > > "permissions": [
> > > > > > {
> > > > > > "name": "all",
> > > > > > "role": "admin"
> > > > > > }
> > > > > > ],
> > > > > > "user-role": {
> > > > > > "solradmin": "admin"
> > > > > > }
> > > > > > }
> > > > > > }
> > > > > >
> > > > > >
> > > > > > On Solr 8.1.1, using our previously working security.json, running
> > > queries
> > > > > > (through the admin UI currently) I non-deterministically get 401
> > > responses
> > > > > > on queries when a collection has more than 1 shard. Increasing the
> > > number
> > > > > > of shar
Re: Intermittent error 401 with JSON Facet query to retrieve count all collections
Hi Jason,
Thanks for your reply.
I have tried to add the "forwardCredentials": true in the security.json,
but I still get the same error.
Regards,
Edwin
On Mon, 3 Jun 2019 at 22:19, Colvin Cowie
wrote:
> Hi, thanks I'll give that a go when I get a chance.
>
> I was trying to reply to an older thread (
>
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201904.mbox/%3CCAF2DzVXeVZqnixnkbzw0La1ui5N5-RG9PwfMBHG9vmkfBSMzJA%40mail.gmail.com%3E
> ),
> which I don't have in my mailbox, so obviously didn't reply to the right
> address to get my response threaded so mine has appeared on its own. Oops.
>
> A JIRA issue was raised on that thread
> https://issues.apache.org/jira/browse/SOLR-13421 but it's not had any
> attention.
>
>
> On Mon, 3 Jun 2019 at 14:46, Jason Gerlowski
> wrote:
>
> > Hi Colvin,
> >
> > We're still taking a look at fixing the bug, but as a workaround in
> > the meantime, you can look into adding a "forwardCredentials":true
> > property under the "authentication" section of security.json. That
> > seems to fix the issue in my reproduction at least.
> >
> > e.g.
> >
> > {
> > "authentication": {
> > "blockUnknown": true,
> > "class": "solr.BasicAuthPlugin",
> > "credentials": {
> > "solradmin": ""
> > },
> > "forwardCredentials": true
> > },
> > ...
> > }
> >
> > Jason
> >
> > On Mon, Jun 3, 2019 at 9:31 AM Jason Gerlowski
> > wrote:
> > >
> > > One last note: as far as I can tell, nothing about this issue is
> > > specific to JSON Faceting or the JSON request API. It can be
> > > triggered just as easily with "/select?q=*:*".
> > >
> > > The bug created for this is: SOLR-13510
> > >
> > > On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski
> > wrote:
> > > >
> > > > I'm also able to reproduce this bug on master. A few more notes
> about
> > > > the bad behavior:
> > > >
> > > > - the behavior occurs regardless of the specific permissions
> > > > configured in security.json. (i.e. whether the top permission is
> > > > "all", or "security-edit", or there are no permissions at all.)
> > > > - I tried looking for a pattern in which requests saw the 401s, but
> > > > didn't have any luck. The 401 occurs when talking to the whole
> > > > collection or targeting individual cores directly. It occurs when
> > > > curl hits a host containing a replica for the collection in question,
> > > > and when it doesnt. etc. This distinguishes it from SOLR-13472,
> which
> > > > seems more specific to collection structure/layout.
> > > >
> > > > I'll create a bug for this in JIRA.
> > > >
> > > > On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie <
> > colvin.cowie@gmail.com> wrote:
> > > > >
> > > > > Hello. I encountered this issue too and wrote this up before I
> found
> > this
> > > > > thread, but I thought I might as well post it still, if it helps...
> > > > >
> > > > > Currently I'm trying to move our product on to Solr 8.1.1. We are
> > currently
> > > > > using 6.6.6, so things have definitely moved on.
> > > > >
> > > > > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock
> > down Solr
> > > > > (and we also secure our zookeeper). Here's an example for solradmin
> > as the
> > > > > user and password
> > > > >
> > > > > {
> > > > > "authentication": {
> > > > > "blockUnknown": true,
> > > > > "class": "solr.BasicAuthPlugin",
> > > > > "credentials": {
> > > > > "solradmin":
> > "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > > > > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> > > > > }
> > > > > },
> > > > > "authorization": {
> > > > > "class": "solr.RuleBasedAuthorizationPlugin",
> > > > > "permissions": [
> > > > > {
> > > > > "name": "all",
> > > > > "role": "admin"
> > > > > }
> > > > > ],
> > > > > "user-role": {
> > > > > "solradmin": "admin"
> > > > > }
> > > > > }
> > > > > }
> > > > >
> > > > >
> > > > > On Solr 8.1.1, using our previously working security.json, running
> > queries
> > > > > (through the admin UI currently) I non-deterministically get 401
> > responses
> > > > > on queries when a collection has more than 1 shard. Increasing the
> > number
> > > > > of shards in the collection makes the errors more likely.
> > > > >
> > > > > {
> > > > > "responseHeader":{
> > > > > "zkConnected":true,
> > > > > "status":401,
> > > > > "QTime":30,
> > > > > "params":{
> > > > > "q":"*:*",
> > > > > "_":"1559474550365"}},
> > > > > "error":{
> > > > > "metadata":[
> > > > >
> > > > >
> >
> "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
> > > > >
> > > > >
> >
> "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> > > > > "msg":"Error from server at null: Expected mime type
> > > > > application/octet-stre
Re: Intermittent error 401 with JSON Facet query to retrieve count all collections
Hi, thanks I'll give that a go when I get a chance.
I was trying to reply to an older thread (
http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201904.mbox/%3CCAF2DzVXeVZqnixnkbzw0La1ui5N5-RG9PwfMBHG9vmkfBSMzJA%40mail.gmail.com%3E),
which I don't have in my mailbox, so obviously didn't reply to the right
address to get my response threaded so mine has appeared on its own. Oops.
A JIRA issue was raised on that thread
https://issues.apache.org/jira/browse/SOLR-13421 but it's not had any
attention.
On Mon, 3 Jun 2019 at 14:46, Jason Gerlowski wrote:
> Hi Colvin,
>
> We're still taking a look at fixing the bug, but as a workaround in
> the meantime, you can look into adding a "forwardCredentials":true
> property under the "authentication" section of security.json. That
> seems to fix the issue in my reproduction at least.
>
> e.g.
>
> {
> "authentication": {
> "blockUnknown": true,
> "class": "solr.BasicAuthPlugin",
> "credentials": {
> "solradmin": ""
> },
> "forwardCredentials": true
> },
> ...
> }
>
> Jason
>
> On Mon, Jun 3, 2019 at 9:31 AM Jason Gerlowski
> wrote:
> >
> > One last note: as far as I can tell, nothing about this issue is
> > specific to JSON Faceting or the JSON request API. It can be
> > triggered just as easily with "/select?q=*:*".
> >
> > The bug created for this is: SOLR-13510
> >
> > On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski
> wrote:
> > >
> > > I'm also able to reproduce this bug on master. A few more notes about
> > > the bad behavior:
> > >
> > > - the behavior occurs regardless of the specific permissions
> > > configured in security.json. (i.e. whether the top permission is
> > > "all", or "security-edit", or there are no permissions at all.)
> > > - I tried looking for a pattern in which requests saw the 401s, but
> > > didn't have any luck. The 401 occurs when talking to the whole
> > > collection or targeting individual cores directly. It occurs when
> > > curl hits a host containing a replica for the collection in question,
> > > and when it doesnt. etc. This distinguishes it from SOLR-13472, which
> > > seems more specific to collection structure/layout.
> > >
> > > I'll create a bug for this in JIRA.
> > >
> > > On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie <
> colvin.cowie@gmail.com> wrote:
> > > >
> > > > Hello. I encountered this issue too and wrote this up before I found
> this
> > > > thread, but I thought I might as well post it still, if it helps...
> > > >
> > > > Currently I'm trying to move our product on to Solr 8.1.1. We are
> currently
> > > > using 6.6.6, so things have definitely moved on.
> > > >
> > > > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock
> down Solr
> > > > (and we also secure our zookeeper). Here's an example for solradmin
> as the
> > > > user and password
> > > >
> > > > {
> > > > "authentication": {
> > > > "blockUnknown": true,
> > > > "class": "solr.BasicAuthPlugin",
> > > > "credentials": {
> > > > "solradmin":
> "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > > > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> > > > }
> > > > },
> > > > "authorization": {
> > > > "class": "solr.RuleBasedAuthorizationPlugin",
> > > > "permissions": [
> > > > {
> > > > "name": "all",
> > > > "role": "admin"
> > > > }
> > > > ],
> > > > "user-role": {
> > > > "solradmin": "admin"
> > > > }
> > > > }
> > > > }
> > > >
> > > >
> > > > On Solr 8.1.1, using our previously working security.json, running
> queries
> > > > (through the admin UI currently) I non-deterministically get 401
> responses
> > > > on queries when a collection has more than 1 shard. Increasing the
> number
> > > > of shards in the collection makes the errors more likely.
> > > >
> > > > {
> > > > "responseHeader":{
> > > > "zkConnected":true,
> > > > "status":401,
> > > > "QTime":30,
> > > > "params":{
> > > > "q":"*:*",
> > > > "_":"1559474550365"}},
> > > > "error":{
> > > > "metadata":[
> > > >
> > > >
> "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
> > > >
> > > >
> "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> > > > "msg":"Error from server at null: Expected mime type
> > > > application/octet-stream but got text/html. \n\n > > > http-equiv=\"Content-Type\"
> > > > content=\"text/html;charset=utf-8\"/>\nError 401 require
> > > > authentication\n\nHTTP ERROR
> 401\nProblem
> > > > accessing /solr/gettingstarted_shard4_replica_n6/select.
> Reason:\n
> > > > require authentication\n\n\n",
> > > > "code":401}}
> > > >
> > > > The security stats indicate this is happening because the requests
> do not
> > > > have credentials with them, e.g.
> > > >
> http://localhost:8983
Re: Intermittent error 401 with JSON Facet query to retrieve count all collections
Hi Colvin,
We're still taking a look at fixing the bug, but as a workaround in
the meantime, you can look into adding a "forwardCredentials":true
property under the "authentication" section of security.json. That
seems to fix the issue in my reproduction at least.
e.g.
{
"authentication": {
"blockUnknown": true,
"class": "solr.BasicAuthPlugin",
"credentials": {
"solradmin": ""
},
"forwardCredentials": true
},
...
}
Jason
On Mon, Jun 3, 2019 at 9:31 AM Jason Gerlowski wrote:
>
> One last note: as far as I can tell, nothing about this issue is
> specific to JSON Faceting or the JSON request API. It can be
> triggered just as easily with "/select?q=*:*".
>
> The bug created for this is: SOLR-13510
>
> On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski wrote:
> >
> > I'm also able to reproduce this bug on master. A few more notes about
> > the bad behavior:
> >
> > - the behavior occurs regardless of the specific permissions
> > configured in security.json. (i.e. whether the top permission is
> > "all", or "security-edit", or there are no permissions at all.)
> > - I tried looking for a pattern in which requests saw the 401s, but
> > didn't have any luck. The 401 occurs when talking to the whole
> > collection or targeting individual cores directly. It occurs when
> > curl hits a host containing a replica for the collection in question,
> > and when it doesnt. etc. This distinguishes it from SOLR-13472, which
> > seems more specific to collection structure/layout.
> >
> > I'll create a bug for this in JIRA.
> >
> > On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie
> > wrote:
> > >
> > > Hello. I encountered this issue too and wrote this up before I found this
> > > thread, but I thought I might as well post it still, if it helps...
> > >
> > > Currently I'm trying to move our product on to Solr 8.1.1. We are
> > > currently
> > > using 6.6.6, so things have definitely moved on.
> > >
> > > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock down
> > > Solr
> > > (and we also secure our zookeeper). Here's an example for solradmin as the
> > > user and password
> > >
> > > {
> > > "authentication": {
> > > "blockUnknown": true,
> > > "class": "solr.BasicAuthPlugin",
> > > "credentials": {
> > > "solradmin": "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> > > }
> > > },
> > > "authorization": {
> > > "class": "solr.RuleBasedAuthorizationPlugin",
> > > "permissions": [
> > > {
> > > "name": "all",
> > > "role": "admin"
> > > }
> > > ],
> > > "user-role": {
> > > "solradmin": "admin"
> > > }
> > > }
> > > }
> > >
> > >
> > > On Solr 8.1.1, using our previously working security.json, running queries
> > > (through the admin UI currently) I non-deterministically get 401 responses
> > > on queries when a collection has more than 1 shard. Increasing the number
> > > of shards in the collection makes the errors more likely.
> > >
> > > {
> > > "responseHeader":{
> > > "zkConnected":true,
> > > "status":401,
> > > "QTime":30,
> > > "params":{
> > > "q":"*:*",
> > > "_":"1559474550365"}},
> > > "error":{
> > > "metadata":[
> > >
> > > "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
> > >
> > > "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> > > "msg":"Error from server at null: Expected mime type
> > > application/octet-stream but got text/html. \n\n > > http-equiv=\"Content-Type\"
> > > content=\"text/html;charset=utf-8\"/>\nError 401 require
> > > authentication\n\nHTTP ERROR 401\nProblem
> > > accessing /solr/gettingstarted_shard4_replica_n6/select. Reason:\n
> > > require authentication\n\n\n",
> > > "code":401}}
> > >
> > > The security stats indicate this is happening because the requests do not
> > > have credentials with them, e.g.
> > > http://localhost:8983/solr/#/gettingstarted_shard4_replica_n6/plugins?type=security&entry=org.apache.solr.security.BasicAuthPlugin
> > >
> > > org.apache.solr.security.BasicAuthPlugin
> > > class:
> > > org.apache.solr.security.BasicAuthPlugin
> > > description:
> > > Authentication Plugin org.apache.solr.security.BasicAuthPlugin
> > > stats
> > > SECURITY./authentication.authenticated:
> > > 182
> > > SECURITY./authentication.errors.count:
> > > 0
> > > SECURITY./authentication.failMissingCredentials:
> > > 58
> > > SECURITY./authentication.failWrongCredentials:
> > > 0
> > > SECURITY./authentication.passThrough:
> > > 0
> > > SECURITY./authentication.requestTimes.meanRate:
> > > 0.4183414110946125
Re: Intermittent error 401 with JSON Facet query to retrieve count all collections
One last note: as far as I can tell, nothing about this issue is
specific to JSON Faceting or the JSON request API. It can be
triggered just as easily with "/select?q=*:*".
The bug created for this is: SOLR-13510
On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski wrote:
>
> I'm also able to reproduce this bug on master. A few more notes about
> the bad behavior:
>
> - the behavior occurs regardless of the specific permissions
> configured in security.json. (i.e. whether the top permission is
> "all", or "security-edit", or there are no permissions at all.)
> - I tried looking for a pattern in which requests saw the 401s, but
> didn't have any luck. The 401 occurs when talking to the whole
> collection or targeting individual cores directly. It occurs when
> curl hits a host containing a replica for the collection in question,
> and when it doesnt. etc. This distinguishes it from SOLR-13472, which
> seems more specific to collection structure/layout.
>
> I'll create a bug for this in JIRA.
>
> On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie
> wrote:
> >
> > Hello. I encountered this issue too and wrote this up before I found this
> > thread, but I thought I might as well post it still, if it helps...
> >
> > Currently I'm trying to move our product on to Solr 8.1.1. We are currently
> > using 6.6.6, so things have definitely moved on.
> >
> > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock down Solr
> > (and we also secure our zookeeper). Here's an example for solradmin as the
> > user and password
> >
> > {
> > "authentication": {
> > "blockUnknown": true,
> > "class": "solr.BasicAuthPlugin",
> > "credentials": {
> > "solradmin": "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> > }
> > },
> > "authorization": {
> > "class": "solr.RuleBasedAuthorizationPlugin",
> > "permissions": [
> > {
> > "name": "all",
> > "role": "admin"
> > }
> > ],
> > "user-role": {
> > "solradmin": "admin"
> > }
> > }
> > }
> >
> >
> > On Solr 8.1.1, using our previously working security.json, running queries
> > (through the admin UI currently) I non-deterministically get 401 responses
> > on queries when a collection has more than 1 shard. Increasing the number
> > of shards in the collection makes the errors more likely.
> >
> > {
> > "responseHeader":{
> > "zkConnected":true,
> > "status":401,
> > "QTime":30,
> > "params":{
> > "q":"*:*",
> > "_":"1559474550365"}},
> > "error":{
> > "metadata":[
> >
> > "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
> >
> > "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> > "msg":"Error from server at null: Expected mime type
> > application/octet-stream but got text/html. \n\n > http-equiv=\"Content-Type\"
> > content=\"text/html;charset=utf-8\"/>\nError 401 require
> > authentication\n\nHTTP ERROR 401\nProblem
> > accessing /solr/gettingstarted_shard4_replica_n6/select. Reason:\n
> > require authentication\n\n\n",
> > "code":401}}
> >
> > The security stats indicate this is happening because the requests do not
> > have credentials with them, e.g.
> > http://localhost:8983/solr/#/gettingstarted_shard4_replica_n6/plugins?type=security&entry=org.apache.solr.security.BasicAuthPlugin
> >
> > org.apache.solr.security.BasicAuthPlugin
> > class:
> > org.apache.solr.security.BasicAuthPlugin
> > description:
> > Authentication Plugin org.apache.solr.security.BasicAuthPlugin
> > stats
> > SECURITY./authentication.authenticated:
> > 182
> > SECURITY./authentication.errors.count:
> > 0
> > SECURITY./authentication.failMissingCredentials:
> > 58
> > SECURITY./authentication.failWrongCredentials:
> > 0
> > SECURITY./authentication.passThrough:
> > 0
> > SECURITY./authentication.requestTimes.meanRate:
> > 0.4183414110946125
> > SECURITY./authentication.requests:
> > 240
> > SECURITY./authentication.totalTime:
> > 117791100
> >
> > I assume that this is connected to the changes around
> > https://issues.apache.org/jira/browse/SOLR-7896 and
> > https://issues.apache.org/jira/browse/SOLR-13344 I've tested with Solr
> > 7.6.0 and it appears to be unaffected
> >
> > Repro steps:
> ># Extract solr 8.1.1.
> ># bin\solr start -e cloud
> > 1 node / [default port] / [default collection name] / 4 shards / 1
> > replica / [_default configuration]
> ># server\scripts\cloud-scripts\zkcli -zkhost localhost:9983 -cmd putfile
> > /security.json
> >
> ># Execute repeated GETS to
> > http://localhost:8983/solr/gettingstarted/select?q=*%3A* - a lot
Re: Intermittent error 401 with JSON Facet query to retrieve count all collections
I'm also able to reproduce this bug on master. A few more notes about
the bad behavior:
- the behavior occurs regardless of the specific permissions
configured in security.json. (i.e. whether the top permission is
"all", or "security-edit", or there are no permissions at all.)
- I tried looking for a pattern in which requests saw the 401s, but
didn't have any luck. The 401 occurs when talking to the whole
collection or targeting individual cores directly. It occurs when
curl hits a host containing a replica for the collection in question,
and when it doesnt. etc. This distinguishes it from SOLR-13472, which
seems more specific to collection structure/layout.
I'll create a bug for this in JIRA.
On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie wrote:
>
> Hello. I encountered this issue too and wrote this up before I found this
> thread, but I thought I might as well post it still, if it helps...
>
> Currently I'm trying to move our product on to Solr 8.1.1. We are currently
> using 6.6.6, so things have definitely moved on.
>
> We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock down Solr
> (and we also secure our zookeeper). Here's an example for solradmin as the
> user and password
>
> {
> "authentication": {
> "blockUnknown": true,
> "class": "solr.BasicAuthPlugin",
> "credentials": {
> "solradmin": "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> }
> },
> "authorization": {
> "class": "solr.RuleBasedAuthorizationPlugin",
> "permissions": [
> {
> "name": "all",
> "role": "admin"
> }
> ],
> "user-role": {
> "solradmin": "admin"
> }
> }
> }
>
>
> On Solr 8.1.1, using our previously working security.json, running queries
> (through the admin UI currently) I non-deterministically get 401 responses
> on queries when a collection has more than 1 shard. Increasing the number
> of shards in the collection makes the errors more likely.
>
> {
> "responseHeader":{
> "zkConnected":true,
> "status":401,
> "QTime":30,
> "params":{
> "q":"*:*",
> "_":"1559474550365"}},
> "error":{
> "metadata":[
>
> "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
>
> "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> "msg":"Error from server at null: Expected mime type
> application/octet-stream but got text/html. \n\n http-equiv=\"Content-Type\"
> content=\"text/html;charset=utf-8\"/>\nError 401 require
> authentication\n\nHTTP ERROR 401\nProblem
> accessing /solr/gettingstarted_shard4_replica_n6/select. Reason:\n
> require authentication\n\n\n",
> "code":401}}
>
> The security stats indicate this is happening because the requests do not
> have credentials with them, e.g.
> http://localhost:8983/solr/#/gettingstarted_shard4_replica_n6/plugins?type=security&entry=org.apache.solr.security.BasicAuthPlugin
>
> org.apache.solr.security.BasicAuthPlugin
> class:
> org.apache.solr.security.BasicAuthPlugin
> description:
> Authentication Plugin org.apache.solr.security.BasicAuthPlugin
> stats
> SECURITY./authentication.authenticated:
> 182
> SECURITY./authentication.errors.count:
> 0
> SECURITY./authentication.failMissingCredentials:
> 58
> SECURITY./authentication.failWrongCredentials:
> 0
> SECURITY./authentication.passThrough:
> 0
> SECURITY./authentication.requestTimes.meanRate:
> 0.4183414110946125
> SECURITY./authentication.requests:
> 240
> SECURITY./authentication.totalTime:
> 117791100
>
> I assume that this is connected to the changes around
> https://issues.apache.org/jira/browse/SOLR-7896 and
> https://issues.apache.org/jira/browse/SOLR-13344 I've tested with Solr
> 7.6.0 and it appears to be unaffected
>
> Repro steps:
># Extract solr 8.1.1.
># bin\solr start -e cloud
> 1 node / [default port] / [default collection name] / 4 shards / 1
> replica / [_default configuration]
># server\scripts\cloud-scripts\zkcli -zkhost localhost:9983 -cmd putfile
> /security.json
>
># Execute repeated GETS to
> http://localhost:8983/solr/gettingstarted/select?q=*%3A* - a lot of them,
> but not all, will fail with 401s
>
>
> Also as a side note, because the authentication is now done through the
> form login rather than the browser basic auth, if you go directly to a non
> UI url (e.g. http://localhost:8983/solr/main_index/select?q=*%3A*) you have
> to authenticate to it using the browser's basic auth prompt. Which is
> slightly annoying since the query page in the Admin UI generates links to
> it for the queries you run, and you've already authenticated to get there.
> But it's no
Re: Intermittent error 401 with JSON Facet query to retrieve count all collections
Hello. I encountered this issue too and wrote this up before I found this
thread, but I thought I might as well post it still, if it helps...
Currently I'm trying to move our product on to Solr 8.1.1. We are currently
using 6.6.6, so things have definitely moved on.
We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock down Solr
(and we also secure our zookeeper). Here's an example for solradmin as the
user and password
{
"authentication": {
"blockUnknown": true,
"class": "solr.BasicAuthPlugin",
"credentials": {
"solradmin": "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
}
},
"authorization": {
"class": "solr.RuleBasedAuthorizationPlugin",
"permissions": [
{
"name": "all",
"role": "admin"
}
],
"user-role": {
"solradmin": "admin"
}
}
}
On Solr 8.1.1, using our previously working security.json, running queries
(through the admin UI currently) I non-deterministically get 401 responses
on queries when a collection has more than 1 shard. Increasing the number
of shards in the collection makes the errors more likely.
{
"responseHeader":{
"zkConnected":true,
"status":401,
"QTime":30,
"params":{
"q":"*:*",
"_":"1559474550365"}},
"error":{
"metadata":[
"error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
"root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
"msg":"Error from server at null: Expected mime type
application/octet-stream but got text/html. \n\n\nError 401 require
authentication\n\nHTTP ERROR 401\nProblem
accessing /solr/gettingstarted_shard4_replica_n6/select. Reason:\n
require authentication\n\n\n",
"code":401}}
The security stats indicate this is happening because the requests do not
have credentials with them, e.g.
http://localhost:8983/solr/#/gettingstarted_shard4_replica_n6/plugins?type=security&entry=org.apache.solr.security.BasicAuthPlugin
org.apache.solr.security.BasicAuthPlugin
class:
org.apache.solr.security.BasicAuthPlugin
description:
Authentication Plugin org.apache.solr.security.BasicAuthPlugin
stats
SECURITY./authentication.authenticated:
182
SECURITY./authentication.errors.count:
0
SECURITY./authentication.failMissingCredentials:
58
SECURITY./authentication.failWrongCredentials:
0
SECURITY./authentication.passThrough:
0
SECURITY./authentication.requestTimes.meanRate:
0.4183414110946125
SECURITY./authentication.requests:
240
SECURITY./authentication.totalTime:
117791100
I assume that this is connected to the changes around
https://issues.apache.org/jira/browse/SOLR-7896 and
https://issues.apache.org/jira/browse/SOLR-13344 I've tested with Solr
7.6.0 and it appears to be unaffected
Repro steps:
# Extract solr 8.1.1.
# bin\solr start -e cloud
1 node / [default port] / [default collection name] / 4 shards / 1
replica / [_default configuration]
# server\scripts\cloud-scripts\zkcli -zkhost localhost:9983 -cmd putfile
/security.json
# Execute repeated GETS to
http://localhost:8983/solr/gettingstarted/select?q=*%3A* - a lot of them,
but not all, will fail with 401s
Also as a side note, because the authentication is now done through the
form login rather than the browser basic auth, if you go directly to a non
UI url (e.g. http://localhost:8983/solr/main_index/select?q=*%3A*) you have
to authenticate to it using the browser's basic auth prompt. Which is
slightly annoying since the query page in the Admin UI generates links to
it for the queries you run, and you've already authenticated to get there.
But it's not a massive burden or anything... I guess I just preferred
having the browser BA prompt.
Thanks
Re: Intermittent error 401 with JSON Facet query to retrieve count all collections
Patches welcome!
> On Apr 30, 2019, at 8:17 PM, Zheng Lin Edwin Yeo wrote:
>
> Hi,
>
> Any expected timeline on when we can solve this bug?
>
> Regards,
> Edwin
>
> On Fri, 26 Apr 2019 at 00:21, Zheng Lin Edwin Yeo
> wrote:
>
>> Thanks Jan.
>>
>> I have created a bug in JIRA under
>> https://issues.apache.org/jira/browse/SOLR-13421
>>
>> Regards,
>> Edwin
>>
>> On Thu, 25 Apr 2019 at 19:53, Jan Høydahl wrote:
>>
>>> Please create a bug for this in JIRA. I think that HttpShardHandler needs
>>> to handle shards from shards param the same as the auto-computed shards it
>>> gets from ZK, but have not looked into it in detail yet.
>>>
>>> --
>>> Jan Høydahl, search solution architect
>>> Cominvent AS - www.cominvent.com
>>>
25. apr. 2019 kl. 09:15 skrev Zheng Lin Edwin Yeo >>> :
Regarding the issue, we have found that if we put only one collections
>>> in
the shards (can be any collection), there will not be error, and it can
always produce the correct output. If we put 2 collections, there will
>>> not
be error 90% of the time (only 10% of the time the issue will occur).
However, once we put 3 or more collections (can be any of the
>>> collections),
this issue will keep occurring.
This seems more like a bug, since the error mostly occurs when we put 3
>>> or
more collections, but it does not give error when we put only one or
>>> two of
the three collections, and we do not face this issue in the earlier
>>> version
of Solr.
Appreciate if anyone can advice on this weird scenario?
Thank you.
Regards,
Edwin
On Wed, 24 Apr 2019 at 15:20, Zheng Lin Edwin Yeo >>>
wrote:
> Hi,
>
>
> I am using the below JSON Facet to retrieve the count of all the
>>> different
> collections in one query.
>
>
>
>>> https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1,https://localhost:8983/solr/collection2,https://localhost:8983/solr/collection3,https://localhost:8983/solr/collection4,https://localhost:8983/solr/collection5,https://localhost:8983/solr/collection6&rows=0&json.facet={categories
>>> :
> {type : terms,field : content_type,limit : 100}}
>
>
> Previously, in Solr 7.6 and Solr 7.7, this query can work correctly
>>> and we
> are able to produce the correct output.
>
> {
> "responseHeader":
> { "zkConnected":true, "status":0, "QTime":24}
>
> ,
> "response":
> {"numFound":41200,"start":0,"maxScore":12.993215,"docs":[] }
>
> ,
> "facets":{
> "count":41200,
> "categories":{
> "buckets":[
> { "val":"collection1", "count":26213}
>
> ,
>
> { "val":"collection2", "count":12075}
>
> ,
>
> { "val":"collection3", "count":1947}
>
> ,
>
> { "val":"collection4", "count":850}
>
> ,
>
> { "val":"collection5", "count":111}
>
> ,
>
> { "val":"collection6", "count":4}
>
> ]}}}
>
>
> However, in the new Solr 8.0.0, this query can only work occasionally.
> Most of the time, we will get the following error of 'Error 401 require
> authentication':
>
> {
> "responseHeader":
> { "zkConnected":true, "status":401, "QTime":11}
>
> ,
> "error":{
> "metadata":[
>
>
>>> "error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException",
>
>
>>> "root-error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException"],
> "msg":"Error from server at null: Expected mime type
> application/octet-stream but got text/html. \n\n http-equiv=\"Content-Type\"
> content=\"text/html;charset=utf-8\"/>\nError 401 require
> authentication\n\nHTTP ERROR
>>> 401\nProblem
> accessing /solr/collection6/select. Reason:\nrequire
> authentication\n\n\n",
> "code":401}}
>
> This issue does not occur in Solr 7.6 and Solr 7.7, even though I have
>>> set
> up the same authentication for all the versions.
>
> What could be the issue that causes this?
>
>
> Below is the format of my security.json:
>
> {
> "authentication":
>
> {"blockUnknown": true,"class":"solr.BasicAuthPlugin",
> "credentials":
> {"user1":"hyHXXuJSqcZdNgdSTGUvrQZRpqrYFUQ2ffmlWQ4GUTk=
> E0w3/2FD+rlxulbPm2G7i9HZqT+2gMBzcyJCcGcMWwA="}
>
> },
> "authorization":
>
> {"class":"solr.RuleBasedAuthorizationPlugin","user-role":
> {"user1":"admin"}
>
> ,
> "permissions":[
> {"name":"security-edit", "role":"admin"}
>
> ]
> }}
>
>
>
Re: Intermittent error 401 with JSON Facet query to retrieve count all collections
Hi,
Any expected timeline on when we can solve this bug?
Regards,
Edwin
On Fri, 26 Apr 2019 at 00:21, Zheng Lin Edwin Yeo
wrote:
> Thanks Jan.
>
> I have created a bug in JIRA under
> https://issues.apache.org/jira/browse/SOLR-13421
>
> Regards,
> Edwin
>
> On Thu, 25 Apr 2019 at 19:53, Jan Høydahl wrote:
>
>> Please create a bug for this in JIRA. I think that HttpShardHandler needs
>> to handle shards from shards param the same as the auto-computed shards it
>> gets from ZK, but have not looked into it in detail yet.
>>
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>>
>> > 25. apr. 2019 kl. 09:15 skrev Zheng Lin Edwin Yeo > >:
>> >
>> > Regarding the issue, we have found that if we put only one collections
>> in
>> > the shards (can be any collection), there will not be error, and it can
>> > always produce the correct output. If we put 2 collections, there will
>> not
>> > be error 90% of the time (only 10% of the time the issue will occur).
>> >
>> > However, once we put 3 or more collections (can be any of the
>> collections),
>> > this issue will keep occurring.
>> >
>> > This seems more like a bug, since the error mostly occurs when we put 3
>> or
>> > more collections, but it does not give error when we put only one or
>> two of
>> > the three collections, and we do not face this issue in the earlier
>> version
>> > of Solr.
>> >
>> > Appreciate if anyone can advice on this weird scenario?
>> >
>> > Thank you.
>> >
>> > Regards,
>> > Edwin
>> >
>> >
>> >
>> > On Wed, 24 Apr 2019 at 15:20, Zheng Lin Edwin Yeo > >
>> > wrote:
>> >
>> >> Hi,
>> >>
>> >>
>> >> I am using the below JSON Facet to retrieve the count of all the
>> different
>> >> collections in one query.
>> >>
>> >>
>> >>
>> https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1,https://localhost:8983/solr/collection2,https://localhost:8983/solr/collection3,https://localhost:8983/solr/collection4,https://localhost:8983/solr/collection5,https://localhost:8983/solr/collection6&rows=0&json.facet={categories
>> :
>> >> {type : terms,field : content_type,limit : 100}}
>> >>
>> >>
>> >> Previously, in Solr 7.6 and Solr 7.7, this query can work correctly
>> and we
>> >> are able to produce the correct output.
>> >>
>> >> {
>> >> "responseHeader":
>> >> { "zkConnected":true, "status":0, "QTime":24}
>> >>
>> >> ,
>> >> "response":
>> >> {"numFound":41200,"start":0,"maxScore":12.993215,"docs":[] }
>> >>
>> >> ,
>> >> "facets":{
>> >>"count":41200,
>> >>"categories":{
>> >> "buckets":[
>> >> { "val":"collection1", "count":26213}
>> >>
>> >> ,
>> >>
>> >> { "val":"collection2", "count":12075}
>> >>
>> >> ,
>> >>
>> >> { "val":"collection3", "count":1947}
>> >>
>> >> ,
>> >>
>> >> { "val":"collection4", "count":850}
>> >>
>> >> ,
>> >>
>> >> { "val":"collection5", "count":111}
>> >>
>> >> ,
>> >>
>> >> { "val":"collection6", "count":4}
>> >>
>> >> ]}}}
>> >>
>> >>
>> >> However, in the new Solr 8.0.0, this query can only work occasionally.
>> >> Most of the time, we will get the following error of 'Error 401 require
>> >> authentication':
>> >>
>> >> {
>> >> "responseHeader":
>> >> { "zkConnected":true, "status":401, "QTime":11}
>> >>
>> >> ,
>> >> "error":{
>> >>"metadata":[
>> >>
>> >>
>> "error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException",
>> >>
>> >>
>> "root-error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException"],
>> >>"msg":"Error from server at null: Expected mime type
>> >> application/octet-stream but got text/html. \n\n> >> http-equiv=\"Content-Type\"
>> >> content=\"text/html;charset=utf-8\"/>\nError 401 require
>> >> authentication\n\nHTTP ERROR
>> 401\nProblem
>> >> accessing /solr/collection6/select. Reason:\nrequire
>> >> authentication\n\n\n",
>> >>"code":401}}
>> >>
>> >> This issue does not occur in Solr 7.6 and Solr 7.7, even though I have
>> set
>> >> up the same authentication for all the versions.
>> >>
>> >> What could be the issue that causes this?
>> >>
>> >>
>> >> Below is the format of my security.json:
>> >>
>> >> {
>> >> "authentication":
>> >>
>> >> {"blockUnknown": true,"class":"solr.BasicAuthPlugin",
>> >> "credentials":
>> >> {"user1":"hyHXXuJSqcZdNgdSTGUvrQZRpqrYFUQ2ffmlWQ4GUTk=
>> >> E0w3/2FD+rlxulbPm2G7i9HZqT+2gMBzcyJCcGcMWwA="}
>> >>
>> >> },
>> >> "authorization":
>> >>
>> >> {"class":"solr.RuleBasedAuthorizationPlugin","user-role":
>> >> {"user1":"admin"}
>> >>
>> >> ,
>> >> "permissions":[
>> >> {"name":"security-edit", "role":"admin"}
>> >>
>> >> ]
>> >> }}
>> >>
>> >>
>> >> Regards,
>> >> Edwin
>> >>
>> >>
>> >> On Mon, 22 Apr 2019 at 09:37, Zheng Lin Edwin Yeo <
>> edwinye...@gmail.com>
>> >> wrote:
>> >>
>> >>> Hi,
>> >>>
>> >>> Anyone has exp
Re: Intermittent error 401 with JSON Facet query to retrieve count all collections
Thanks Jan.
I have created a bug in JIRA under
https://issues.apache.org/jira/browse/SOLR-13421
Regards,
Edwin
On Thu, 25 Apr 2019 at 19:53, Jan Høydahl wrote:
> Please create a bug for this in JIRA. I think that HttpShardHandler needs
> to handle shards from shards param the same as the auto-computed shards it
> gets from ZK, but have not looked into it in detail yet.
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
> > 25. apr. 2019 kl. 09:15 skrev Zheng Lin Edwin Yeo >:
> >
> > Regarding the issue, we have found that if we put only one collections in
> > the shards (can be any collection), there will not be error, and it can
> > always produce the correct output. If we put 2 collections, there will
> not
> > be error 90% of the time (only 10% of the time the issue will occur).
> >
> > However, once we put 3 or more collections (can be any of the
> collections),
> > this issue will keep occurring.
> >
> > This seems more like a bug, since the error mostly occurs when we put 3
> or
> > more collections, but it does not give error when we put only one or two
> of
> > the three collections, and we do not face this issue in the earlier
> version
> > of Solr.
> >
> > Appreciate if anyone can advice on this weird scenario?
> >
> > Thank you.
> >
> > Regards,
> > Edwin
> >
> >
> >
> > On Wed, 24 Apr 2019 at 15:20, Zheng Lin Edwin Yeo
> > wrote:
> >
> >> Hi,
> >>
> >>
> >> I am using the below JSON Facet to retrieve the count of all the
> different
> >> collections in one query.
> >>
> >>
> >>
> https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1,https://localhost:8983/solr/collection2,https://localhost:8983/solr/collection3,https://localhost:8983/solr/collection4,https://localhost:8983/solr/collection5,https://localhost:8983/solr/collection6&rows=0&json.facet={categories
> :
> >> {type : terms,field : content_type,limit : 100}}
> >>
> >>
> >> Previously, in Solr 7.6 and Solr 7.7, this query can work correctly and
> we
> >> are able to produce the correct output.
> >>
> >> {
> >> "responseHeader":
> >> { "zkConnected":true, "status":0, "QTime":24}
> >>
> >> ,
> >> "response":
> >> {"numFound":41200,"start":0,"maxScore":12.993215,"docs":[] }
> >>
> >> ,
> >> "facets":{
> >>"count":41200,
> >>"categories":{
> >> "buckets":[
> >> { "val":"collection1", "count":26213}
> >>
> >> ,
> >>
> >> { "val":"collection2", "count":12075}
> >>
> >> ,
> >>
> >> { "val":"collection3", "count":1947}
> >>
> >> ,
> >>
> >> { "val":"collection4", "count":850}
> >>
> >> ,
> >>
> >> { "val":"collection5", "count":111}
> >>
> >> ,
> >>
> >> { "val":"collection6", "count":4}
> >>
> >> ]}}}
> >>
> >>
> >> However, in the new Solr 8.0.0, this query can only work occasionally.
> >> Most of the time, we will get the following error of 'Error 401 require
> >> authentication':
> >>
> >> {
> >> "responseHeader":
> >> { "zkConnected":true, "status":401, "QTime":11}
> >>
> >> ,
> >> "error":{
> >>"metadata":[
> >>
> >>
> "error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException",
> >>
> >>
> "root-error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException"],
> >>"msg":"Error from server at null: Expected mime type
> >> application/octet-stream but got text/html. \n\n >> http-equiv=\"Content-Type\"
> >> content=\"text/html;charset=utf-8\"/>\nError 401 require
> >> authentication\n\nHTTP ERROR
> 401\nProblem
> >> accessing /solr/collection6/select. Reason:\nrequire
> >> authentication\n\n\n",
> >>"code":401}}
> >>
> >> This issue does not occur in Solr 7.6 and Solr 7.7, even though I have
> set
> >> up the same authentication for all the versions.
> >>
> >> What could be the issue that causes this?
> >>
> >>
> >> Below is the format of my security.json:
> >>
> >> {
> >> "authentication":
> >>
> >> {"blockUnknown": true,"class":"solr.BasicAuthPlugin",
> >> "credentials":
> >> {"user1":"hyHXXuJSqcZdNgdSTGUvrQZRpqrYFUQ2ffmlWQ4GUTk=
> >> E0w3/2FD+rlxulbPm2G7i9HZqT+2gMBzcyJCcGcMWwA="}
> >>
> >> },
> >> "authorization":
> >>
> >> {"class":"solr.RuleBasedAuthorizationPlugin","user-role":
> >> {"user1":"admin"}
> >>
> >> ,
> >> "permissions":[
> >> {"name":"security-edit", "role":"admin"}
> >>
> >> ]
> >> }}
> >>
> >>
> >> Regards,
> >> Edwin
> >>
> >>
> >> On Mon, 22 Apr 2019 at 09:37, Zheng Lin Edwin Yeo >
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> Anyone has experienced this or have any insights of this?
> >>>
> >>> Regards,
> >>> Edwin
> >>>
> >>> On Thu, 18 Apr 2019 at 18:04, Zheng Lin Edwin Yeo <
> edwinye...@gmail.com>
> >>> wrote:
> >>>
> Is there possibility that this could be a bug in the new Solr 8.0.0?
>
> Since I do not face the issue in the earlier version, and I have not
>
Re: Intermittent error 401 with JSON Facet query to retrieve count all collections
Please create a bug for this in JIRA. I think that HttpShardHandler needs to
handle shards from shards param the same as the auto-computed shards it gets
from ZK, but have not looked into it in detail yet.
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
> 25. apr. 2019 kl. 09:15 skrev Zheng Lin Edwin Yeo :
>
> Regarding the issue, we have found that if we put only one collections in
> the shards (can be any collection), there will not be error, and it can
> always produce the correct output. If we put 2 collections, there will not
> be error 90% of the time (only 10% of the time the issue will occur).
>
> However, once we put 3 or more collections (can be any of the collections),
> this issue will keep occurring.
>
> This seems more like a bug, since the error mostly occurs when we put 3 or
> more collections, but it does not give error when we put only one or two of
> the three collections, and we do not face this issue in the earlier version
> of Solr.
>
> Appreciate if anyone can advice on this weird scenario?
>
> Thank you.
>
> Regards,
> Edwin
>
>
>
> On Wed, 24 Apr 2019 at 15:20, Zheng Lin Edwin Yeo
> wrote:
>
>> Hi,
>>
>>
>> I am using the below JSON Facet to retrieve the count of all the different
>> collections in one query.
>>
>>
>> https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1,https://localhost:8983/solr/collection2,https://localhost:8983/solr/collection3,https://localhost:8983/solr/collection4,https://localhost:8983/solr/collection5,https://localhost:8983/solr/collection6&rows=0&json.facet={categories
>> :
>> {type : terms,field : content_type,limit : 100}}
>>
>>
>> Previously, in Solr 7.6 and Solr 7.7, this query can work correctly and we
>> are able to produce the correct output.
>>
>> {
>> "responseHeader":
>> { "zkConnected":true, "status":0, "QTime":24}
>>
>> ,
>> "response":
>> {"numFound":41200,"start":0,"maxScore":12.993215,"docs":[] }
>>
>> ,
>> "facets":{
>>"count":41200,
>>"categories":{
>> "buckets":[
>> { "val":"collection1", "count":26213}
>>
>> ,
>>
>> { "val":"collection2", "count":12075}
>>
>> ,
>>
>> { "val":"collection3", "count":1947}
>>
>> ,
>>
>> { "val":"collection4", "count":850}
>>
>> ,
>>
>> { "val":"collection5", "count":111}
>>
>> ,
>>
>> { "val":"collection6", "count":4}
>>
>> ]}}}
>>
>>
>> However, in the new Solr 8.0.0, this query can only work occasionally.
>> Most of the time, we will get the following error of 'Error 401 require
>> authentication':
>>
>> {
>> "responseHeader":
>> { "zkConnected":true, "status":401, "QTime":11}
>>
>> ,
>> "error":{
>>"metadata":[
>>
>> "error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException",
>>
>> "root-error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException"],
>>"msg":"Error from server at null: Expected mime type
>> application/octet-stream but got text/html. \n\n> http-equiv=\"Content-Type\"
>> content=\"text/html;charset=utf-8\"/>\nError 401 require
>> authentication\n\nHTTP ERROR 401\nProblem
>> accessing /solr/collection6/select. Reason:\nrequire
>> authentication\n\n\n",
>>"code":401}}
>>
>> This issue does not occur in Solr 7.6 and Solr 7.7, even though I have set
>> up the same authentication for all the versions.
>>
>> What could be the issue that causes this?
>>
>>
>> Below is the format of my security.json:
>>
>> {
>> "authentication":
>>
>> {"blockUnknown": true,"class":"solr.BasicAuthPlugin",
>> "credentials":
>> {"user1":"hyHXXuJSqcZdNgdSTGUvrQZRpqrYFUQ2ffmlWQ4GUTk=
>> E0w3/2FD+rlxulbPm2G7i9HZqT+2gMBzcyJCcGcMWwA="}
>>
>> },
>> "authorization":
>>
>> {"class":"solr.RuleBasedAuthorizationPlugin","user-role":
>> {"user1":"admin"}
>>
>> ,
>> "permissions":[
>> {"name":"security-edit", "role":"admin"}
>>
>> ]
>> }}
>>
>>
>> Regards,
>> Edwin
>>
>>
>> On Mon, 22 Apr 2019 at 09:37, Zheng Lin Edwin Yeo
>> wrote:
>>
>>> Hi,
>>>
>>> Anyone has experienced this or have any insights of this?
>>>
>>> Regards,
>>> Edwin
>>>
>>> On Thu, 18 Apr 2019 at 18:04, Zheng Lin Edwin Yeo
>>> wrote:
>>>
Is there possibility that this could be a bug in the new Solr 8.0.0?
Since I do not face the issue in the earlier version, and I have not
changed any configuration in this new version. My data in Solr 8.0.0 is
freshly re-index directly in Solr 8.0.0, not upgraded from earlier version.
Regards,
Edwin
On Thu, 18 Apr 2019 at 10:10, Zheng Lin Edwin Yeo
wrote:
> Hi Jason,
>
> The same problem still persist after restarting my Solr nodes. The only
> time the problem didn't occur is when I disabled the basic authentication.
>
> I have tried wit
Re: Intermittent error 401 with JSON Facet query to retrieve count all collections
Regarding the issue, we have found that if we put only one collections in
the shards (can be any collection), there will not be error, and it can
always produce the correct output. If we put 2 collections, there will not
be error 90% of the time (only 10% of the time the issue will occur).
However, once we put 3 or more collections (can be any of the collections),
this issue will keep occurring.
This seems more like a bug, since the error mostly occurs when we put 3 or
more collections, but it does not give error when we put only one or two of
the three collections, and we do not face this issue in the earlier version
of Solr.
Appreciate if anyone can advice on this weird scenario?
Thank you.
Regards,
Edwin
On Wed, 24 Apr 2019 at 15:20, Zheng Lin Edwin Yeo
wrote:
> Hi,
>
>
> I am using the below JSON Facet to retrieve the count of all the different
> collections in one query.
>
>
> https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1,https://localhost:8983/solr/collection2,https://localhost:8983/solr/collection3,https://localhost:8983/solr/collection4,https://localhost:8983/solr/collection5,https://localhost:8983/solr/collection6&rows=0&json.facet={categories
> :
> {type : terms,field : content_type,limit : 100}}
>
>
> Previously, in Solr 7.6 and Solr 7.7, this query can work correctly and we
> are able to produce the correct output.
>
> {
> "responseHeader":
> { "zkConnected":true, "status":0, "QTime":24}
>
> ,
> "response":
> {"numFound":41200,"start":0,"maxScore":12.993215,"docs":[] }
>
> ,
> "facets":{
> "count":41200,
> "categories":{
> "buckets":[
> { "val":"collection1", "count":26213}
>
> ,
>
> { "val":"collection2", "count":12075}
>
> ,
>
> { "val":"collection3", "count":1947}
>
> ,
>
> { "val":"collection4", "count":850}
>
> ,
>
> { "val":"collection5", "count":111}
>
> ,
>
> { "val":"collection6", "count":4}
>
> ]}}}
>
>
> However, in the new Solr 8.0.0, this query can only work occasionally.
> Most of the time, we will get the following error of 'Error 401 require
> authentication':
>
> {
> "responseHeader":
> { "zkConnected":true, "status":401, "QTime":11}
>
> ,
> "error":{
> "metadata":[
>
> "error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException",
>
> "root-error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException"],
> "msg":"Error from server at null: Expected mime type
> application/octet-stream but got text/html. \n\n http-equiv=\"Content-Type\"
> content=\"text/html;charset=utf-8\"/>\nError 401 require
> authentication\n\nHTTP ERROR 401\nProblem
> accessing /solr/collection6/select. Reason:\nrequire
> authentication\n\n\n",
> "code":401}}
>
> This issue does not occur in Solr 7.6 and Solr 7.7, even though I have set
> up the same authentication for all the versions.
>
> What could be the issue that causes this?
>
>
> Below is the format of my security.json:
>
> {
> "authentication":
>
> {"blockUnknown": true,"class":"solr.BasicAuthPlugin",
> "credentials":
> {"user1":"hyHXXuJSqcZdNgdSTGUvrQZRpqrYFUQ2ffmlWQ4GUTk=
> E0w3/2FD+rlxulbPm2G7i9HZqT+2gMBzcyJCcGcMWwA="}
>
> },
> "authorization":
>
> {"class":"solr.RuleBasedAuthorizationPlugin","user-role":
> {"user1":"admin"}
>
> ,
>"permissions":[
> {"name":"security-edit", "role":"admin"}
>
> ]
> }}
>
>
> Regards,
> Edwin
>
>
> On Mon, 22 Apr 2019 at 09:37, Zheng Lin Edwin Yeo
> wrote:
>
>> Hi,
>>
>> Anyone has experienced this or have any insights of this?
>>
>> Regards,
>> Edwin
>>
>> On Thu, 18 Apr 2019 at 18:04, Zheng Lin Edwin Yeo
>> wrote:
>>
>>> Is there possibility that this could be a bug in the new Solr 8.0.0?
>>>
>>> Since I do not face the issue in the earlier version, and I have not
>>> changed any configuration in this new version. My data in Solr 8.0.0 is
>>> freshly re-index directly in Solr 8.0.0, not upgraded from earlier version.
>>>
>>> Regards,
>>> Edwin
>>>
>>> On Thu, 18 Apr 2019 at 10:10, Zheng Lin Edwin Yeo
>>> wrote:
>>>
Hi Jason,
The same problem still persist after restarting my Solr nodes. The only
time the problem didn't occur is when I disabled the basic authentication.
I have tried with a few "/select?q=*:*", and they do not exhibit the
same problem. Even the similar query with only 1 shard does not have the
problem.
https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1&rows=0&json.facet={categories
: {type : terms,field : content_type,limit : 100}}
It is only when there are 2 or more shards, that the problem occur.
https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1,https://localhost:89
Intermittent error 401 with JSON Facet query to retrieve count all collections
Hi,
I am using the below JSON Facet to retrieve the count of all the different
collections in one query.
https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1,https://localhost:8983/solr/collection2,https://localhost:8983/solr/collection3,https://localhost:8983/solr/collection4,https://localhost:8983/solr/collection5,https://localhost:8983/solr/collection6&rows=0&json.facet={categories
:
{type : terms,field : content_type,limit : 100}}
Previously, in Solr 7.6 and Solr 7.7, this query can work correctly and we
are able to produce the correct output.
{
"responseHeader":
{ "zkConnected":true, "status":0, "QTime":24}
,
"response":
{"numFound":41200,"start":0,"maxScore":12.993215,"docs":[] }
,
"facets":{
"count":41200,
"categories":{
"buckets":[
{ "val":"collection1", "count":26213}
,
{ "val":"collection2", "count":12075}
,
{ "val":"collection3", "count":1947}
,
{ "val":"collection4", "count":850}
,
{ "val":"collection5", "count":111}
,
{ "val":"collection6", "count":4}
]}}}
However, in the new Solr 8.0.0, this query can only work occasionally. Most
of the time, we will get the following error of 'Error 401 require
authentication':
{
"responseHeader":
{ "zkConnected":true, "status":401, "QTime":11}
,
"error":{
"metadata":[
"error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException",
"root-error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException"],
"msg":"Error from server at null: Expected mime type
application/octet-stream but got text/html. \n\n\nError 401 require
authentication\n\nHTTP ERROR 401\nProblem
accessing /solr/collection6/select. Reason:\nrequire
authentication\n\n\n",
"code":401}}
This issue does not occur in Solr 7.6 and Solr 7.7, even though I have set
up the same authentication for all the versions.
What could be the issue that causes this?
Below is the format of my security.json:
{
"authentication":
{"blockUnknown": true,"class":"solr.BasicAuthPlugin",
"credentials":
{"user1":"hyHXXuJSqcZdNgdSTGUvrQZRpqrYFUQ2ffmlWQ4GUTk=
E0w3/2FD+rlxulbPm2G7i9HZqT+2gMBzcyJCcGcMWwA="}
},
"authorization":
{"class":"solr.RuleBasedAuthorizationPlugin","user-role":
{"user1":"admin"}
,
"permissions":[
{"name":"security-edit", "role":"admin"}
]
}}
Regards,
Edwin
On Mon, 22 Apr 2019 at 09:37, Zheng Lin Edwin Yeo
wrote:
> Hi,
>
> Anyone has experienced this or have any insights of this?
>
> Regards,
> Edwin
>
> On Thu, 18 Apr 2019 at 18:04, Zheng Lin Edwin Yeo
> wrote:
>
>> Is there possibility that this could be a bug in the new Solr 8.0.0?
>>
>> Since I do not face the issue in the earlier version, and I have not
>> changed any configuration in this new version. My data in Solr 8.0.0 is
>> freshly re-index directly in Solr 8.0.0, not upgraded from earlier version.
>>
>> Regards,
>> Edwin
>>
>> On Thu, 18 Apr 2019 at 10:10, Zheng Lin Edwin Yeo
>> wrote:
>>
>>> Hi Jason,
>>>
>>> The same problem still persist after restarting my Solr nodes. The only
>>> time the problem didn't occur is when I disabled the basic authentication.
>>>
>>> I have tried with a few "/select?q=*:*", and they do not exhibit the
>>> same problem. Even the similar query with only 1 shard does not have the
>>> problem.
>>>
>>>
>>> https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1&rows=0&json.facet={categories
>>> : {type : terms,field : content_type,limit : 100}}
>>>
>>>
>>> It is only when there are 2 or more shards, that the problem occur.
>>>
>>>
>>> https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1,https://localhost:8983/solr/collection2&rows=0&json.facet={categories
>>> : {type : terms,field : content_type,limit : 100}}
>>>
>>>
>>> Regards,
>>> Edwin
>>>
>>>
>>> On Thu, 18 Apr 2019 at 01:15, Jason Gerlowski
>>> wrote:
>>>
Agreed, I'd be surprised if this behavior was specific to JSON
Faceting. Though I'm surprised it's happening at all, so...
Anyway, that's easy for you to test though. Try a few "/select?q=*:*"
queries and see whether they also exhibits this behavior. One other
question: does the behavior persist after restarting your Solr nodes?
Good luck,
Jason
On Wed, Apr 17, 2019 at 4:05 AM Zheng Lin Edwin Yeo
wrote:
>
> Hi,
>
> For your info, I have enabled basic authentication and SSL in all the
3
> versions, and I'm not sure if the issue is more on the authentication
side
> instead of the JSON Facet query?
>
> Regards,
> Edwin
>
> On Wed, 17 Apr 2019 at 06:54, Zheng Lin Edwin Yeo <
edwinye...@gmail.com>
> wrote:
>
