We have a problem on a 3.5gig collection running Solr7.4 (we will soon upgrade
to Solr8.5.2)
Users were often encountering timeout errors of the type shown below
My colleague found a blog post at
What works for us is having something like this at the bottom of security.json:
{
"name":"open_select",
"path":"/select/*",
"role":null,
"index":9},
{
"name":"catch-all-nocollection",
"collection":null,
"path":"/*",
We recently have had a few occasions when cores for one specific collection
were renamed (or more likely dropped and recreated, and thus ended up with a
different core name).
Is this a known phenomenon? Is there any explanation?
It may be relevant that we just recently started running this
ng a large collection (as
required in order to complete the upgrade to a new release), let me know.
-Original Message-----
From: Oakley, Craig (NIH/NLM/NCBI) [C]
Sent: Tuesday, November 24, 2020 1:56 PM
To: solr-user@lucene.apache.org
Subject: RE: disallowing delete through security.json
Users might submit an update that strips everything but "id"
from your documents. In many/most usecases that'd be equally
concerning. Just wondering what your usecase is - if it's generally
applicable this is probably worth a JIRA ticket.
Best,
Jason
On Thu, Nov 19, 2020 at 10:34 AM
-----
From: Oakley, Craig (NIH/NLM/NCBI) [C]
Sent: Monday, October 26, 2020 6:23 PM
To: solr-user@lucene.apache.org
Subject: disallowing delete through security.json
I am interested in disallowing delete through security.json
After seeing the "method" section in
lucene.apache.org/solr/
We are in the process of upgrading from Solr7.4.0 to Solr8.5.2, and we are
experiencing intermittent problem with log rotation
Our log4j2.xml file (both for Solr7.4.0 and for Solr8.5.2) includes the
following:
%d{-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection}
I am interested in disallowing delete through security.json
After seeing the "method" section in
lucene.apache.org/solr/guide/8_4/rule-based-authorization-plugin.html my first
attempt was as follows:
{"set-permission":{
"name":"NO_delete",
"path":["/update/*","/update"],
"collection":col_name,
reliable mechanism
for its purpose, solr cloud isnt going to replace standalone, if it does,
thats when I guess I stop upgrading or move to elastic
On Wed, Sep 30, 2020 at 2:58 PM Oakley, Craig (NIH/NLM/NCBI) [C]
wrote:
> Based on the thread below (reading "legacy" as meaning &quo
Based on the thread below (reading "legacy" as meaning "likely to be deprecated
in later versions"), we have been working to extract ourselves from
Master/Slave replication
Most of our collections need to be in two data centers (a read/write copy in
one local data center: the
, has been deprecated in 8.6.
It suffers from serious design flaws and it allows such things to happen
that you observe. While there may be workarounds, it is advisable to not
rely on CDCR in production.
Thanks,
Ishan
On Thu, 2 Jul, 2020, 1:12 am Oakley, Craig (NIH/NLM/NCBI) [C],
wrote
added after the missing records).
Does anyone yet have any suggestion how to get CDCR to work properly?
-Original Message-
From: Oakley, Craig (NIH/NLM/NCBI) [C]
Sent: Wednesday, June 24, 2020 9:46 AM
To: solr-user@lucene.apache.org
Subject: CDCR stress-test issues
In attempting
In attempting to stress-test CDCR (running Solr 7.4), I am running into a
couple of issues.
One is that the tlog files keep accumulating for some nodes in the CDCR system,
particularly for the non-Leader nodes in the Source SolrCloud. No quantity of
hard commits seem to cause any of these tlog
Trying think of a term that both fresh (and yet sort of standard already) and
appropriate: how about IndexFetcher instead of "Slave"? And then "Master" could
be "FetchedIndex" or "FetchedSource"
I think it could be beneficial to broaden the range of candidates.
ather than as INFO or WARN ?
Please advise
-Original Message-----
From: Oakley, Craig (NIH/NLM/NCBI) [C]
Sent: Wednesday, March 11, 2020 5:18 PM
To: solr-user@lucene.apache.org
Subject: RE: No files to download for index generation
I wanted to ask *again* whether anyone has any insight
flagged as an ERROR rather than as INFO or WARN ?
-Original Message-
From: Oakley, Craig (NIH/NLM/NCBI) [C]
Sent: Monday, June 10, 2019 9:57 AM
To: solr-user@lucene.apache.org
Subject: RE: No files to download for index generation
Does anyone yet have any insight on interpreting the seve
I have found that for admin commands you may need to include "collection":null
{
"name":"admin-info-system2",
"path":"/admin/*",
"collection":null,
"role":"*"}
-Original Message-
From: Jesús Roca
Sent: Friday, February 28, 2020 2:10 PM
To:
created one?
> On Dec 13, 2019, at 5:10 PM, Oakley, Craig (NIH/NLM/NCBI) [C]
> wrote:
>
> It looks as though I do not have an option under
> issues.apache.org/jira/projects/SOLR/issues by which to create an issue.
> Could you create one (and let me know its number)?
>
> T
@lucene.apache.org
Subject: Re: Solr8 changes how security.json restricts access to GUI
Ok, se should perhaps print a warning somewhere that IE is not supported. Can
you file a JIRA issue?
Jan Høydahl
> 13. des. 2019 kl. 21:43 skrev Oakley, Craig (NIH/NLM/NCBI) [C]
> :
>
> Well that is prog
st visit?
>
> Jan
>
>> 12. des. 2019 kl. 17:27 skrev Oakley, Craig (NIH/NLM/NCBI) [C]
>> :
>>
>> Below is the security.json (with password hashes redacted): in Solr7.4 it
>> prompts for a password and (if you get it right) lets you into the whole
>
_col",
"path":"/admin/*",
"role":"trgadmin"},
{
"name":"open_select",
"path":"/select/*",
"role":null},
{
"name":"open_search",
In Solr 7, we had clauses in our security.json saying
{
"name":"all-admin",
"collection":null,
"path":"/*",
"role":"allgen",
"index":15},
{
"name":"all-core-handlers",
"path":"/*",
"role":"allgen",
"index":16},
For the record, the solution was to edit solr.xml changing
${socketTimeout:0}
to
${socketTimeout:60}
-Original Message-
From: Oakley, Craig (NIH/NLM/NCBI) [C]
Sent: Tuesday, November 19, 2019 6:19 PM
To: solr-user@lucene.apache.org
Subject: RE: async BACKUP under Solr8.3
In some
In some collections I am having problems with Solr8.1.1 through 8.3; with other
collections it is fine in Solr8.1.1 through 8.3
I'm investigating what might be wrong with the collections which have the
problems.
Thanks
-Original Message-
From: Oakley, Craig (NIH/NLM/NCBI) [C]
Sent
FYI, I DO succeed in doing an async backup in Solr8.1
-Original Message-
From: Oakley, Craig (NIH/NLM/NCBI) [C]
Sent: Tuesday, November 19, 2019 9:03 AM
To: solr-user@lucene.apache.org
Subject: RE: async BACKUP under Solr8.3
This is on a test server: simple case: one node, one shard
representative, it just says
that some node (ok, it's overseer) fails to wait another one.
Ideally we need a log from overseer node and subordinate node during backup
operation.
Thanks.
On Tue, Nov 19, 2019 at 2:13 AM Oakley, Craig (NIH/NLM/NCBI) [C]
wrote:
> For Solr 8.3, when I attempt a comm
For Solr 8.3, when I attempt a command of the form
host:port/solr/admin/collections?action=BACKUP=snapshot1=col1=/tmp=bug
And then when I run /solr/admin/collections?action=REQUESTSTATUS=bug
I get "msg":"found [bug] in failed tasks"
The solr.log file has a stack trace like the following
currently works for you with any account and any password, including without
any login-and-password at all?
-Original Message-----
From: Oakley, Craig (NIH/NLM/NCBI) [C]
Sent: Thursday, August 01, 2019 10:58 AM
To: solr-user@lucene.apache.org
Subject: RE: Basic Authentication problem
The
The hash value
"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="
is based on both the plain text password AND the plain test login. Since
"solr" is not the same string as "solr-admin", the password will not work. If
the only authorization in
I am having problems with a SolrCloud where both nodes become unresponsive. I
am wondering whether there is some sort of memory leak. Attached is a portion
of the solr_gc.log from around the time that the problem starts. Have you any
suggestions how to diagnose and address this issue?
Thanks
Does anyone yet have any insight on interpreting the severity of this message?
-Original Message-
From: Oakley, Craig (NIH/NLM/NCBI) [C]
Sent: Tuesday, June 04, 2019 4:07 PM
To: solr-user@lucene.apache.org
Subject: No files to download for index generation
We have occasionally been
We have occasionally been seeing an error such as the following:
2019-06-03 23:32:45.583 INFO (indexFetcher-45-thread-1) [ ]
o.a.s.h.IndexFetcher Master's generation: 1424625
2019-06-03 23:32:45.583 INFO (indexFetcher-45-thread-1) [ ]
o.a.s.h.IndexFetcher Master's version: 1559619115480
We are wanting to tweak the logging levels of our Solr 7.4 nodes to see what
might be helpful to add to the solr.log for debugging purposes.
In investigating what is available, however, I run /solr/admin/info/logging and
I find that there is little consistency in what logging settings are
Thanks.
That resolves the issue.
Thanks again.
-Original Message-
From: Shawn Heisey
Sent: Tuesday, March 19, 2019 7:10 PM
To: solr-user@lucene.apache.org
Subject: Re: is df needed for SolrCloud replication?
On 3/19/2019 4:48 PM, Oakley, Craig (NIH/NLM/NCBI) [C] wrote:
> I recen
I recently noticed that my solr.log files have been getting the following error
message:
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: no field name
specified in query and no default specified via 'df' param
The timing of these messages coincide with pings to leader node of
> Can we take this thread back to the mailing list, please? It would be good to
> allow other people to weigh in!
Sure
-Original Message-
From: Matt Pearce
Sent: Friday, February 08, 2019 6:45 AM
To: Oakley, Craig (NIH/NLM/NCBI) [C]
Subject: Re: change in White Space when upg
We had a problem when upgrading from Solr 6.6 to Solr 7.4 in that a query
ceased to work.
The query was of the form
http://localhost:8983/solr/collection/select?indent=on=ABC4856.21%20AND%20-field1:ABC4856.21=json=0
Basically finding a count of those records where there is some field which
this a valid behavior? Unless I am missing something, this could be
> >> potentially fatal.
> >> >
> >> > Here's the query and the cluster state post split:
> >> >
> >>
> http://localhost:8983/solr/admin/collections?action=SPLITSHARD=gettingsta
-
From: Oakley, Craig (NIH/NLM/NCBI) [C]
Sent: Friday, August 10, 2018 12:54 PM
To: solr-user@lucene.apache.org
Subject: RE: sharding and placement of replicas
Note that I usually create collections with commands which contain (for example)
solr/admin/collections?action=CREATE=collectest=collectest
of the same shards)
-Original Message-
From: Oakley, Craig (NIH/NLM/NCBI) [C] [mailto:craig.oak...@nih.gov]
Sent: Thursday, August 09, 2018 5:08 PM
To: solr-user@lucene.apache.org
Subject: RE: sharding and placement of replicas
Okay, I've tried again with two nodes running Solr7.4
ne of them for each shard.
Best,
Erick
On Tue, Jul 31, 2018 at 12:24 PM, Oakley, Craig (NIH/NLM/NCBI) [C]
wrote:
> In my case, when trying on Solr7.4 (in response to Shawn Heisey's 6/19/18
> comment "If this is a provable and reproducible bug, and it's still a problem
> in the cu
In my case, when trying on Solr7.4 (in response to Shawn Heisey's 6/19/18
comment "If this is a provable and reproducible bug, and it's still a problem
in the current stable branch"), I had only installed Solr7.4 on one host, and
so I was testing with two nodes on the same host (different port
al Message-
From: Shawn Heisey [mailto:apa...@elyograg.org]
Sent: Tuesday, June 19, 2018 2:20 PM
To: solr-user@lucene.apache.org
Subject: Re: sharding and placement of replicas
On 6/15/2018 11:08 AM, Oakley, Craig (NIH/NLM/NCBI) [C] wrote:
> If I start with a collection X on two nodes with one
If I start with a collection X on two nodes with one shard and two replicas
(for redundancy, in case a node goes down): a node on host1 has
X_shard1_replica1 and a node on host2 has X_shard1_replica2: when I try
SPLITSHARD, I generally get X_shard1_0_replica1, X_shard1_1_replica1 and
I have a sharding question.
We have a collection (one shard, two replicas, currently running Solr6.6) which
sometimes becomes unresponsive on the non-leader node. It is 214 gigabytes, and
we were wondering whether there is a rule of thumb how large to allow a core to
grow before sharding.
My guess would be to edit server/resources/log4j.properties to have
log4j.appender.file.MaxBackupIndex=0
-Original Message-
From: Noriyuki TAKEI [mailto:nta...@sios.com]
Sent: Monday, October 02, 2017 10:39 AM
To: solr-user@lucene.apache.org
Subject: solr.log rotation
HI,All
When I
We usually end security.json with the permissions
{
"name":"open_select",
"path":"/select/*",
"role":null},
{
"name":"all-admin",
"collection":null,
"path":"/*",
FWIW, we now have a hypothetical suspect. We are getting these errors on three
CentOS7 hosts, each of which recently had antivirus software installed.
-Original Message-
From: Oakley, Craig (NIH/NLM/NCBI) [C] [mailto:craig.oak...@nih.gov]
Sent: Thursday, May 11, 2017 11:03 AM
To: solr
thing special so don't be surprised if there's nothing there.
Best,
Erick
On Wed, May 10, 2017 at 10:56 AM, Oakley, Craig (NIH/NLM/NCBI) [C]
<craig.oak...@nih.gov> wrote:
>> You need to look at all of your core.properties files and see if any of them
>> point t
u issue a "kill -9" you can leave write locks lingering.
Best,
Erick
On Thu, May 4, 2017 at 11:00 AM, Oakley, Craig (NIH/NLM/NCBI) [C]
<craig.oak...@nih.gov> wrote:
> We have been having problems with different collections on different
> SolrCloud clusters, all seemin
We have been having problems with different collections on different SolrCloud
clusters, all seeming to be related to the write.lock file with stack traces
similar to the following. Are there any suggestions what might be the cause and
what might be the solution? Thanks
We are considering using Cross Data Center Replication between SolrClouds in
different domains which have a firewall between them. Is it documented anywhere
how many firewall holes will be needed? From each source SolrCloud node to each
target SolrCloud node? From each target SolrCloud node to
to do it).
Unfortunately, that is the nature of open source - there's so many such
features that *could* be extended, they tend to get the feature when
someone actually needs it.
Upayavira
On Tue, Dec 29, 2015, at 06:14 PM, Oakley, Craig (NIH/NLM/NCBI) [C]
wrote:
> I do have authorizat
y need it, AFAIK you'll have to back it up whenever
you build it I'm afraid.
Best,
Erick
On Wed, May 11, 2016 at 8:30 AM, Oakley, Craig (NIH/NLM/NCBI) [C]
<craig.oak...@nih.gov> wrote:
> I have a client whose Solr installation creates a
> analyzingInfixSuggesterIndexDir directory besi
I have a client whose Solr installation creates a
analyzingInfixSuggesterIndexDir directory besides index and tlog. I notice that
this analyzingInfixSuggesterIndexDir is not included in backups (created by
replication?command=backup). Is there a way to include this? Or does it not
need to be
.
I would not recommend adding such complexity to the existing json backed user
list, although it has the benefit of beting 100% self contained.
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
> 18. mar. 2016 kl. 23.30 skrev Oakley, Craig (NIH/NLM/NCBI)
I am wondering whether this might be the bug of SOLR-8326, which is fixed in
Solr 5.4
That's my guess as a user who ran into the bug myself.
-Original Message-
From: Kelly, Frank [mailto:frank.ke...@here.com]
Sent: Wednesday, March 16, 2016 3:09 PM
To: solr-user@lucene.apache.org
When using security.json (in Solr 5.4.1 for instance), is there a recommended
method to allow users to change their own passwords? We certainly would not
want to grant blanket security-edit to all users; but requiring users to
divulge their intended passwords (in Email or by other means) to the
I'm planning to upgrade (from 5.4.0 to 5.4.1) a SolrCloud with two replicas
(one shard).
Am I correct in thinking I should be able simply to shutdown one node, change
it to using 5.4.1, restart the upgraded node, shutdown the other node and
upgrade it? Or are there caveats to consider?
Followup question:
If one has multiple instances on the same host (a host running basically
nothing except multiple instances of Solr), then the values specified as -Xmx
in the various instances should add up to 25% of the RAM of the host...
Is that correct?
-Original Message-
From:
in step 1 of
"Enabling Basic Authentication".
https://lucidworks.com/blog/2015/08/17/securing-solr-basic-auth--rules/
Is this what you're looking for?
Thanks,
Esther Quansah
> Le 29 déc. 2015 à 12:24, Oakley, Craig (NIH/NLM/NCBI) [C]
> <craig.oak...@nih.gov> a écrit :
&g
Or to put it another way, how does one get security.json to work with SOLR-5960?
Has anyone any suggestions?
-Original Message-
From: Oakley, Craig (NIH/NLM/NCBI) [C]
Sent: Thursday, December 24, 2015 2:12 PM
To: 'solr-user@lucene.apache.org' <solr-user@lucene.apache.org>
S
In the old jetty-based implementation of Basic Authentication, one could use
post.jar by running something like
java -Durl="http://user:pswd@host:8983/solr/corename/update;
-Dtype=application/xml -jar post.jar example.xml
By what mechanism does one pass in the user name and password to
d anything other than those patches
and whatever comes from 8355 to have a patched 5.3.1.
Any help in testing this out would be awesome and thanks for reporting and
following up on the issues!
On Tue, Dec 1, 2015 at 6:09 AM, Oakley, Craig (NIH/NLM/NCBI) [C] <
craig.oak...@nih.gov> wrote:
>
> wrote:
> > Please do not put any number. That number is used by the system to
> > optimize loading/reloading plugins. It is not relevant for the user.
> >
> > On Thu, Dec 10, 2015 at 11:52 PM, Oakley, Craig (NIH/NLM/NCBI) [C]
> > <craig.oak...@nih.gov> wrote
Subject: Re: Authorization API versus zkcli.sh
There's nothing cluster specific in security.json if you're using those
plugins. It is totally safe to just take the file from one cluster and
upload it for another for things to work.
On Sat, Dec 5, 2015 at 3:38 AM, Oakley, Craig (NIH/NLM/NCBI) [C]
Looking through
cwiki.apache.org/confluence/display/solr/Authentication+and+Authorization+Plugins
one notices that security.json is initially created by zkcli.sh, and then
modified by means of the Authentication API and the Authorization API. By and
large, this sounds like a good way to
ushed out) before the
official release.
Call it the first of the year for safety's sake, but that's a guess.
Best,
Erick
On Tue, Nov 24, 2015 at 10:22 AM, Oakley, Craig (NIH/NLM/NCBI) [C]
<craig.oak...@nih.gov> wrote:
> Thanks for the reply,
>
> I don't suppose there is an ETA for 5.4?
>
>
> Thanks again
>
> -Original Message-
...
humgupta.net>
wrote:
> To restart solr, you should instead use something like:
> bin/solr start -c -p 8983 -s "example/cloud/node1/solr" -z localhost:2181
> or
> bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z localhost:2181
>
> I've seen others rep
ADDREPLICA
Yes, it certainly is a PKI issue.
On Tue, Nov 24, 2015 at 7:59 AM, Oakley, Craig (NIH/NLM/NCBI) [C] <
craig.oak...@nih.gov> wrote:
> Thank you for the reply
>
> Trying those exact commands, I'm still getting the same issue
> tar xvzf /net/sybdev11/export/home/sybase/Distr/S
symptoms seem similar whether
I use the security.json from SOLR-8326 or the security.json from the Wiki (with
the comma repositioned).
-Original Message-
From: Oakley, Craig (NIH/NLM/NCBI) [C]
Sent: Friday, November 20, 2015 6:59 PM
To: 'solr-user@lucene.apache.org' <solr-user@luce
g that was done when
there were no Collections API.
On Thu, Nov 19, 2015 at 12:36 PM, Oakley, Craig (NIH/NLM/NCBI) [C] <
craig.oak...@nih.gov> wrote:
> I tried again with the following security.json, but the results were the
> same:
>
> {
> "authentication":{
>
{"solrread":["read"]}}'
After this, you should start having issues with ADDREPLICA.
Also, as you would at this point have a collection with a shard that has a
replication factor > 1 (remember the ADDREPLICA we did earlier), you would
have issues when you r
nshumgupta.net]
Sent: Friday, November 20, 2015 6:18 PM
To: solr-user@lucene.apache.org
Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
This seems unrelated and more like a user error somewhere. Can you just
follow the steps, without any security settings i.e. not even uploading
I note that the thread called "Security Problems" (most recent post by Nobel
Paul) seems like it may help with much of what I'm trying to do. I will see to
what extent that may help.
"role":"xmpladmin"},
{
"name":"xmpl_sel",
"collection":"xmpl",
"path":"/select/*",
"role":null},
{
"name":"all-admin",
Thank you for the reply.
What we are attempting is to require a password for practically everything, so
that even were a hacker to get within the firewall, they would have limited
access to the various services (the Security team even complained that, for
Solr 4.5 servers, attempts to access
Implementing security.json is breaking ADDREPLICA
I have been able to reproduce this issue with minimal changes from an
out-of-the-box Zookeeper (3.4.6) and Solr (5.3.1): loading
configsets/basic_configs/conf into Zookeeper, creating the security.json listed
below, creating two nodes (one with
78 matches
Mail list logo
