Re: KeywordTokenizerFactory and Standard Query Parser

ll not be split on whitespace before analysis. See https://lucidworks.com/2017/04/18/multi-word-synonyms-solr-adds-query-time-support/ ." On Tue, Apr 2, 2019 at 8:11 AM Chris Ulicny wrote: > Hi all, > > We have a multivalued field that has an integer at the beginning followed > by

KeywordTokenizerFactory and Standard Query Parser

Hi all, We have a multivalued field that has an integer at the beginning followed by a space, and the index analyzer chain extracts that value to search on testField:[ 34 blah blah blah 27 blah blah blah ... ] The query analyzer chain is just a keyword tokenizer factory since the clients

Re: Solr index slow response

71-189972 DBG1:doc_count: 10 , doc_size: 563 KB, Res > code: 200, QTime: 111 ms, Request time: 113 ms. > >>> 190318-162816.586-189972 DBG1:doc_count: 10 , doc_size: 554 KB, Res > code: 200, QTime: 110 ms, Request time: 111 ms. > >>> 190318-162816.716-189972 DBG1:doc_count: 10 , doc_size: 590 KB, Res >

Re: Solr index slow response

One other thing to look at besides the heap is your commit settings. We've experienced something similar, and changing commit settings alleviated the issue. Are you opening a search on every hardcommit? If so, you might want to reconsider and use the softcommit for the hourly creation of a new sea

Re: SPLITSHARD not working as expected

I'm not sure what the expected behavior is. However, as of 7.4.0, it doesn't seem like there is any attempt to prevent both the new leader and follower replicas from being created on the same instance. Sometimes for one of the sub-shards, the new leader and one of the new followers end up on the s

Re: Solr Cloud wiping all cores when restart without proper zookeeper directories

Out of curiosity, why are you manually deleting nodes in zookeeper? It's always seemed to me that the majority (definitely not all) of modifications needed during normal operations can usually be done through Solr's APIs. Thanks, Chris On Thu, Jan 10, 2019 at 12:04 AM Yogendra Kumar Soni < yogen

Re: Solr query response time analysis

Rajdeep, Not an external tool, but there is the option of using the "debug" parameter in the Solr query that can be used at least as a starting point for looking at the query timing. https://lucene.apache.org/solr/guide/6_6/common-query-parameters.html#CommonQueryParameters-ThedebugParameter Bes

Re: Overseer could not get tags

ot; default="81920" /> > After solr restart - no more annoying messages > > > -Original Message- > > From: Chris Ulicny [mailto:culicny@iq.media] > > Sent: Wednesday, October 31, 2018 7:40 PM > > To: solr-user > > Subject: Re: Overseer could not g

Re: Overseer could not get tags

mczi - 2018-09-18 13:07:55] at org.apache.solr.handler.admin.MetricsHistoryHandler.lambda$new$0(MetricsHistoryHandler.java:231) ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55] at java.util.concurrent.Executors$RunnableAdapter.ca

Re: Slow import from MsSQL and down cluster during process

Dan, Do you have any idea on the resource usage for the hosts when Solr starts to become unresponsive? It could be that you need more resources or better AWS instances for the hosts. We had what sounds like a similar scenario when attempting to move one of our solrcloud instances to a cloud compu

Re: Query to multiple collections

There weren't any particular problems we ran into since the client that makes the queries to multiple collections previously would query multiple cores using the 'shards' parameter before we moved to solrcloud. We didn't have any complicated sorting or scoring requirements fortunately. The one thi

Re: Query to multiple collections

Rohan, I do not remember where I came across it or what restrictions exist on it, but it works for our use case of querying multiple archived collections with identical schemas in the same SolrCloud cluster. The queries have the following form: http::/solr/current/select?collection=current,archiv

Overseer could not get tags

Hi all, Recently in a 7.4.0 test cluster, we ran into SOLR-12814 which we fixed by slightly increasing the request header size. However, there were some other log messages along with the "URI size >8192" message which we thought were related, but

Re: Realtime get not always returning existing data

> > Which version of SOLR are you running? I was thinking of maybe trying > another version to see if it fixes the issue. > > On Thu, Oct 11, 2018 at 8:11 AM Chris Ulicny wrote: > > > We've also run into that issue of not being able to reproduce it outside > of

Re: Realtime get not always returning existing data

> H. I wonder if a version conflict or perhaps other failure can > > > somehow cause this. It shouldn't be very hard to add that to my test > > > setup, just randomly add n _version_ field value. > > > > > > Erick > > > On Mon, Oct 1, 2018

Re: Realtime get not always returning existing data

In our case, we are heavily indexing in the collection while the /get requests are happening which is what we assumed was causing this very rare behavior. However, we have experienced the problem for a collection where the following happens in sequence with minutes in between them. 1. Document id=

Re: Realtime get not always returning existing data

I don't think I've much to add that Steve hasn't already covered, but we've also seen this "null doc" problem in one of our setups. In one of our Solr Cloud instances in production where the /get handler is hit very hard in bursts, the /get request will occasionally return "null" for a document th

Re: Json Facets in version 7

I missed the change previously when looking at v7.3 release notes. The fix for SOLR-12064 <https://issues.apache.org/jira/browse/SOLR-12064> should be the solution. Will upgrade to more recent release and retest. On Thu, Aug 9, 2018 at 8:39 AM Chris Ulicny wrote: > Additionally, the &

Re: Json Facets in version 7

Additionally, the "limit: -1" seems to work as expected if we are only doing a single aggregation in the nested facet: "...facet:{sum_val1:"sum(value1)"}}}" On Thu, Aug 9, 2018 at 8:24 AM Chris Ulicny wrote: > Hi all, > > We are in the process of upgrad

Json Facets in version 7

Hi all, We are in the process of upgrading from version 6.3.0 to version 7.2.1 and have found something that doesn't seem to work with the json facets any more. The query is below /select?q=*:*&wt=xml&indent=off&rows=0&fq=groupid:(4572 4573)+AND+gmttimestamp:[2018-08-05T00:00:00Z+TO+2018-08-07T00

Re: problems with while loop in indexing - no results given when I use that

The lucenenet project is a separate Apache project from Solr (and the Lucene project as well). You will have better luck getting helpful information on their mailing list (https://cwiki.apache.org/confluence/display/LUCENENET/Mailing+Lists). Best, Chris On Fri, Jul 27, 2018 at 8:40 AM - - wrote

Re: Solr objects consuming more GC (Garbage collector) on our application

SolrNet is not part of the Apache Solr project and isn't supported by them. I thought the SolrNet project was abandoned, but it seems like work's been completed on it recently. You might have better luck asking over at the SolrNet git page: https://github.com/SolrNet/SolrNet. Best, Chris On Fri,

Re: some solr replicas down

e everything falls in place. > > > > Yes, there are some other errors that there is a javabin character 2 > > expected and is returning 60 which is "<" . > > > > Thanks, > > Satya > > > > On Tue, Jun 19, 2018 at 8:01 AM Chris Ulicny

Re: some solr replicas down

Satya, There should be some other log messages that are probably relevant to the issue you are having. Something along the lines of "leader cannot communicate with follower...publishing replica as down." It's likely there also is a message of "expecting json/xml but got html" in another instance's

Collection Backup During Indexing

Hi all, Not sure if this is a bug or not, but in versions 6.3.0 and 7.2.1 it seems that an async backup task is put into "completed tasks" even though it seems to have failed during processing: Response: ... "testbackup213286829099257358":{ "responseHeader":{ "status":0, "QTime":0

Re: SolrCloud cluster does not accept new documents for indexing

I haven't worked with AWS, but recently we tried to move some of our solr instances to a cloud in Google's Cloud offering, and it did not go well. All of our problems ended up stemming from the fact that the I/O is throttled. Any complicated enough query would require too many disk reads to return

Re: SolrCloud design question

Beyond failure/maintenance concerns, the first setup is not necessarily a good distribution of the hosts' resources. Depending on the use case, it could be much more prone to hot-spots in the cluster, especially if routing or manual sharding is involved. If for some reason, there are documents on

Re: Basic Security Plugin and Collection Shard Distribution

[ ] o.a.s.s.HttpSolrCall USER_REQUIRED auth header Basic context : userPrincipal: [[principal: solrreader]] type: [READ], collections: [c2, c2,], Path: [/select] path : /select params :null On Thu, Apr 5, 2018 at 12:02 PM Chris Ulicny wrote: > Hi all, > > I've been periodica

Basic Security Plugin and Collection Shard Distribution

Hi all, I've been periodically running into a strange permissions issues and have finally some useful information on it. We've run into the issue on v6.3.0 and v7.X clusters. Assume we have 2 hosts (1 instance on each) with 2 collections. Collection c1 has 2 shards, and collection c2 has 1 shard.

Re: Implications of using implicit routing

it seemed that the implicit/manual routing might work for this new collection. Apparently not based on the requirement of the indexing processes to enforce uniqueness as well as distribution. Thanks for the help. Chris On Wed, Mar 14, 2018 at 11:39 AM Shawn Heisey wrote: > On 3/14/2018 9:26 AM,

Implications of using implicit routing

Hi all, We've been looking at using implicit for one of our collections, and there seems to be some weird behavior that we're not sure whether it was expected or not. Is it recommended to use a uniqueKey for implicit routing? Is the following behavior intended? We have encountered the following

Re: Continuing Saga of Authorization on 6.6.0

quot; file. > Clearly, solr is persistently keeping track of the > authentication/authorization information, but I don't see where. I > suppose it might be kept in zookeeper (which perhaps survives solr > restarts - but I don't know). Any insights on that? > > Terry > &

Re: Continuing Saga of Authorization on 6.6.0

> *failed to delete a user:* "delete-user" is expecting an array of users in the json, so the data should be: {"delete-user": ["lanny"]} > *failed to set a permission: * There are separate endpoints for authorization and authentication. You should use ".../solr/admin/authorization" for the perm

LatLonPointSpatialField

Hi all, I'm currently migrating a 4.9.0 solr standalone index to 7.2.1 cloud. In the schema, I've replaced the old LatLon type with the new LatLonPointSpatialField type as below In the 4.9.0 version, we could search based on a single point passed as an fq parameter ...&fq=doc_lat_lon:"37.779152

Problems with DocExpirationUpdateProcessor with Secured SolrCloud

Hi, We recently setup a 7.2.1 cloud with the intent to have the documents be automatically deleted from the collection using the DocExpirationUpdateProcessorFactory. We also have the cloud secured using the BasicAuthenticationPlugin. Our current config settings are below. The deployment is 3 node

Re: Clusterstatus Action

t 7:53 AM Chris Ulicny wrote: > Hi all, > > According to the documentation, the 'shard' parameter for the > CLUSTERSTATUS action should allow a comma delimited list of shards. > However, passing 'shard1,shard2' as the value results in a shard-not-found > e

Clusterstatus Action

Hi all, According to the documentation, the 'shard' parameter for the CLUSTERSTATUS action should allow a comma delimited list of shards. However, passing 'shard1,shard2' as the value results in a shard-not-found error where it was looking for 'shard1,shard2'. Not a search for 'shard1' and 'shard2

Re: Negative Core Node Numbers

the case that 7.1.1 needed to be released, but some change between that and the released 7.1.0 seems to have fixed the counter issue in my scenario. On Thu, Jan 4, 2018 at 8:14 PM Chris Ulicny wrote: > Thanks Anshum, > > They don't seem to be consistently numbered on any particular c

Re: Negative Core Node Numbers

;:{"core":"testcollection_shard170_replica_n-2080505220", "base_url":"http://host4:8080/solr";, "node_name":"host4:8080_solr","state":"active", "type":"NRT"}}} ... Is there a way to view the counter

Negative Core Node Numbers

Hi, In 7.1, how does solr determine the numbers that are assigned to the replicas? I'm familiar with the earlier naming conventions from 6.3, but I wanted to know if there was supposed to be any connection between the "_n##" suffix and the number assigned to the "core_node##" name since they don't

Authentication Plugin

Hi all, We've got a solrcloud cluster set up on 6.3.0 with the BasicAuthentication plugin enabled. All of the hosts are time synchronized using ntp and are on the same network switch. We're periodically experiencing issues where follower replicas are put into down states by the leader in the case

Re: SOLR-10272

Thanks Shawn, That's great to hear. I'm glad the overwriting behavior is being reverted. On Wed, Dec 6, 2017 at 10:33 AM Shawn Heisey wrote: > On 12/6/2017 6:43 AM, Chris Ulicny wrote: > > I am curious as to whether overwriting an _existing_ config with the > > "

SOLR-10272

Hi all, We are currently in the process of testing out solr 7.1, and I was running into some strange issues with config upload and collection creation. After some digging, I found SOLR-10272 which makes the configName parameter necessary for collection creation, but all of our scripts for 6.X used

Re: Inconsistent results for facet queries

rtant that there only be a single replica for > each shard active at this point. These two collection do _not_ need to > be part of the same SolrCloud, the fetchindex command just takes a URL > of the core to fetch from. > 4> add the replicas back and let them replicate. > >

Re: Inconsistent results for facet queries

> Consider indexed=true stored=false. After stemming, "running" can be > > indexed as "run". At merge time you have no way of knowing that > > "running" was the original term so you simply couldn't fix it on merge, > > not to mention that the perf

Re: Inconsistent results for facet queries

> This can propagate through all following segment merges IIUC. > > So my bet is that if you index into a new collection, everything will > be fine. You can also just delete everything first, but I usually > prefer a new collection so I'm absolutely and positively sure that the

Inconsistent results for facet queries

Hi, We've run into a strange issue with our deployment of solrcloud 6.3.0. Essentially, a standard facet query on a string field usually comes back empty when it shouldn't. However, every now and again the query actually returns the correct values. This is only affecting a single shard in our setu

Re: Querying a specific replica in SolrCloud

> You can also try adding "&shards.info=true" to the standart request like: > solr_node:port/solr/collection/query?shards.info=true&blah blah blah > > Best, > Erick > > On Wed, Oct 11, 2017 at 7:58 AM, Chris Ulicny wrote: > > Hi, > > > &g

Querying a specific replica in SolrCloud

Hi, We're trying to investigate a possible data issue between two replicas in our cloud setup. We have docValues enabled for a string field, and when we facet by it, the results come back with the expected numbers per value, or zero for all values. Is there a way to tell which replica is handling

Unauthorized Requests on Empty Solr Node

Hi all, I've run into an issue with using the basic authentication plugin that comes with solr 6.3.0 that seems to prevent requests from being processed in certain situations. Essentially, if we have a solr node as part of a cloud but contains no replicas for any collection, it cannot process sea

SplitShard Replica Placement

Hi all, I've run into an issue with solr 6.3.0 where the splitshard command placed both replicas of the new smaller shard on the same node, and I was curious as to whether the behavior should be expected or not. Without having dug into the source code, this is what I've observed splitshard doing

Re: SolrDispatchFilter Request Consumption

at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(Thread.java:745) On Thu, Aug 10, 2017 at 10:44 AM Shawn Heisey wrote: > On 8/10/2017 7:31 AM, Chris Ulicny wrote: > > I&#x

SolrDispatchFilter Request Consumption

Hi all, I've noticed that there are quite a few INFO log entries of the form: "...o.a.s.s.SolrDispatchFilter Could not consume full client request" Should we be worried about these, or is there something wrong with the requests we're making? We aren't noticing anything unexpected in what data m

Re: Get handler failure

would be useful. We're running Solr 6.3.0 on Ubuntu 16.04 virtual machines. On Thu, Aug 3, 2017 at 9:18 AM Shawn Heisey wrote: > On 8/3/2017 6:30 AM, Chris Ulicny wrote: > > I've run into an issue in a test environment where a document exists, but > > fails to be retrieved co

Get handler failure

Hi all, I've run into an issue in a test environment where a document exists, but fails to be retrieved consistently by /get requests. In a series of 10 requests for the specific document across a few minute timespan, one of the middle requests returned a null document. Currently, nothing is upda

Re: Problem in documentation -- authentication JSON fails validation

I haven't actually tested it, but I believe the JSON should probably be this: { "set-permission": {"name": "update", "role":"dev"}, "set-permission": {"name": "read", "role":"guest"} } It's missing closing double quotes for 'update' and 'read' and had an extra comma after the 'guest' entry.

Re: Live update the zookeeper for SOLR

> It seems SOLR cluster use one of the zookeeper instance Is solr configured to point to exactly one host or to a list of the hosts in the zookeeper ensemble? The zkHost value should contain all of the zk hosts. We had a similar issue where the solr instance was only pointing to zk1 and none of t

Re: Swapping indexes on disk

Are you physically swapping the disks to introduce the new index? Or having both disks mounted at the same time? If the disks are simultaneously available, can you just swap the cores and then delete the core on the old disk? https://cwiki.apache.org/confluence/display/solr/CoreAdmin+API#CoreAdmin

Re: Solr Admin Documents tab

If you pass the array of documents without the opening and closing square brackets it should work with the page defaults (at least in v6.3.0) { "id":"1",...},{"id":"2",...},... instead of [{ "id":"1",...},{"id":"2",...},...] Best, Chris On Tue, May 16, 2017 at 2:42 PM Rick Leir wrote: > Hi all

Re: Atomic Updates

On Thu, Apr 27, 2017 at 11:23 AM Chris Ulicny wrote: > I'm sending commit=true with every update while testing. I'll write up the > tests and see if someone else can reproduce it. > > On Thu, Apr 27, 2017 at 10:54 AM Erick Erickson > wrote: > >> bq: but i

Re: Atomic Updates

for some strange reason > > There better not be or it's a bug. Things will stick around until > you issue a commit, is there any chance that's the problem? > > If you can document the exact steps, maybe we can reproduce > the issue and raise a JIRA. > > Best, > Er

Re: Atomic Updates

an Hoxha wrote: > @Chris, > According to doc-link-above, only INC,SET are in-place-updates. And only > when they're not indexed/stored, while your 'integer-field' is. So still > shenanigans in there somewhere (docs,your-code,your-test,solr-code). > > On Thu, Apr 27, 20

Re: Atomic Updates

> On Wed, Apr 26, 2017 at 2:13 PM, Dorian Hoxha > > wrote: > > > There are In Place Updates, but according to docs they stll shouldn't > > work > > > in your case: > > > https://cwiki.apache.org/confluence/display/solr/ > > Updating+Parts+of+Documents &

Re: Atomic Updates

serve the non-stored text field. Thanks, Chris On Wed, Apr 26, 2017 at 4:07 PM Dorian Hoxha wrote: > You'll lose the data in that field. Try doing a commit and it should > happen. > > On Wed, Apr 26, 2017 at 9:50 PM, Chris Ulicny wrote: > > > Thanks Shawn, I didn't re

Re: Atomic Updates

non-Text field? However, that shouldn't apply to this since I'm concerned with a non-stored TextField without docValues enabled. Best, Chris On Wed, Apr 26, 2017 at 3:36 PM Shawn Heisey wrote: > On 4/25/2017 1:40 PM, Chris Ulicny wrote: > > Hello all, > > > > Suppose

Re: Atomic Updates

PM Erick Erickson wrote: > How is "otherText" getting values in the first place? If > it's the destination of a copyField directive, it'll be repopulated > if the source of the copyField is stored=true. > > Best, > Erick > > On Tue, Apr 25, 2017 at

Atomic Updates

Hello all, Suppose I have the following fields in a document and populate all 4 fields for every document. id: uniqueKey, indexed and stored integer_field: indexed and stored text_field: indexed and stored othertext_field: indexed but not stored No default values, multivalues, docvalues, copyfie

Re: Get handler not working

parameter somewhere that should be set to match the field you use, but is not. Regards, Alex. http://www.solr-start.com/ - Resources for Solr users, new and experienced On 16 March 2017 at 12:28, Chris Ulicny wrote: > I think I've figured out where the issue is, at least superf

Re: Get handler not working

fferent route field it's highly likely > that's the issue. > I was always against that "feature", and this thread demonstrates part > of the problem (complicating clients, including us human clients > trying to make sense of what's going on). > > -Yonik

Re: Get handler not working

ngs wrote: i still would like to see an experiment where you change the field to id instead of iqdocid, On Thu, Mar 16, 2017 at 9:33 AM, Yonik Seeley wrote: > Something to do with routing perhaps? (the mapping of ids to shards, > by default is based on hashes of the id) > -Yonik > >

Re: Get handler not working

might make the most sense, please let me know. Thanks again On Wed, Mar 15, 2017 at 5:49 PM Erick Erickson wrote: Wait... Is iqdocid set to the in your schema? That might be the missing thing. On Wed, Mar 15, 2017 at 11:20 AM, Chris Ulicny wrote: > Unless the behavior's changed on

Re: Get handler not working

David Hastings wrote: > from your previous email: > "There is no "id" > field defined in the schema." > > you need an id field to use the get handler > > On Wed, Mar 15, 2017 at 1:45 PM, Chris Ulicny wrote: > > > I thought that "id" and

Re: Get handler not working

originally wrote it. > > Regards, >Alex. > > http://www.solr-start.com/ - Resources for Solr users, new and experienced > > > On 15 March 2017 at 13:22, Chris Ulicny wrote: > > Sorry, that is a typo. The get is using the iqdocid field. There is no > "

Re: Get handler not working

t; Is this a typo or are you trying to use get with an "id" field and > your filter query uses "iqdocid"? > > Best, > Erick > > On Wed, Mar 15, 2017 at 8:31 AM, Chris Ulicny wrote: > > Yes, we're using a fixed schema with the iqdocid field set as t

Re: Get handler not working

rs, new and experienced > > > On 15 March 2017 at 11:24, Chris Ulicny wrote: > > Hi, > > > > I've been trying to use the get handler for a new solr cloud collection > we > > are using, and something seems to be amiss. > > > > We are running 6.3.0, so

Get handler not working

Hi, I've been trying to use the get handler for a new solr cloud collection we are using, and something seems to be amiss. We are running 6.3.0, so we did not explicitly define the request handler in the solrconfig since it's supposed to be implicitly defined. We also have the update log enabled

Re: Conditions for replication to copy full index

l replication, shut down the slave, > "rm -rf data". (data should be the parent of the "index" dir) and > restart solr. > > Best, > Erick > > On Mon, Mar 6, 2017 at 8:06 AM, Chris Ulicny wrote: > > Hi all, > > > > We've recently had some

Conditions for replication to copy full index

Hi all, We've recently had some issues with a 5.1.0 core copying the whole index when it was set to replicate from a master core. I've read that if there are documents that have been added to the slave core by mistake, it will do a full copy. Though we are still investigating, this is probably no

Facets and docValues

I've been trying to figure out how exactly docValues help with facet queries, and I only seem to find mention that they are beneficial to facet performance without many specifics. What I'd like to know is whether it applies to all fields used in the facet or just fields that are faceted on. For ex

Get Handler Returning Null

Recently started using the get handler on a solr cloud collection and it seems that it does not return any documents even when I can find those documents by filtering for their unique ids. I explicitly enabled the get handler, reindexed one of the documents, and it seems to work fine for that sing

Re: SolrCloud and LVM

to cache entirely in memory, and we haven't had any performance problems so far. So I'm not expecting that to change too much when moving the cloud architecture. Thanks again. On Thu, Jan 5, 2017 at 7:55 PM Shawn Heisey wrote: > On 1/5/2017 3:12 PM, Chris Ulicny wrote: > > Is

SolrCloud and LVM

Is there any known significant performance impact of running solrcloud with lvm on linux? While migrating to solrcloud we don't have the storage capacity for our expected final size, so we are planning on setting up the solrcloud instances on a logical volume that we can grow when hardware becomes