Re: Index download speed while replicating is fixed at 5.1 in replication.html

2020-06-16 Thread Shawn Heisey
On 6/14/2020 12:06 AM, Florin Babes wrote: While checking ways to optimize the speed of replication I've noticed that the index download speed is fixed at 5.1 in replication.html. There is a reason for that? If not, I would like to submit a patch with the fix. We are using solr 8.3.1. Looking

Re: How to determine why solr stops running?

2020-06-10 Thread Shawn Heisey
On 6/10/2020 12:13 PM, Ryan W wrote: People keep suggesting I check the logs for errors. What do those errors look like? Does anyone have examples of the text of a Solr oom error? Or the text of any other errors I should be looking for the next time solr fails? Are there phrases I should

Re: Fw: TolerantUpdateProcessorFactory not functioning

2020-06-09 Thread Shawn Heisey
On 6/9/2020 12:44 AM, Hup Chen wrote: Thanks for your reply, this is one of the example where it fail. POST by using charset=utf-8 or other charset didn't help that CTRL-CHAR "^" error found in the title field, I hope solr can simply skip this record and go ahead to index the rest data.

Re: How to determine why solr stops running?

2020-06-09 Thread Shawn Heisey
On 5/14/2020 7:22 AM, Ryan W wrote: I manage a site where solr has stopped running a couple times in the past week. The server hasn't been rebooted, so that's not the reason. What else causes solr to stop running? How can I investigate why this is happening? Any situation where Solr stops

Re: Solr admin error message - where are relevant log files?

2020-06-07 Thread Shawn Heisey
On 6/7/2020 10:16 AM, Jim Anderson wrote: The admin pages comes up with: SolrCore Initialization Failures I look in my .../solr/server/logs directory and cannot find and meaningful errors or warnings. Should I be looking elsewhere? That depends. Did you install Solr with the installer

Re: SolrClient.query take a 'collection' argument

2020-06-05 Thread Shawn Heisey
On 6/5/2020 4:24 PM, Jim Anderson wrote: I am running my first solrj program and it is crashing when I call the method client.query("coreName",queryParms) The API doc says the string should be a collection. I'm still not sure about the difference between a collection and a core, so what I am

Re: Solr takes time to warm up core with huge data

2020-06-05 Thread Shawn Heisey
On 6/5/2020 12:17 AM, Srinivas Kashyap wrote: q=*:*=PARENT_DOC_ID:100=MODIFY_TS:[1970-01-01T00:00:00Z TO *]=PHY_KEY2:"HQ012206"=PHY_KEY1:"JACK"=1000=MODIFY_TS desc,LOGICAL_SECT_NAME asc,TRACK_ID desc,TRACK_INTER_ID asc,PHY_KEY1 asc,PHY_KEY2 asc,PHY_KEY3 asc,PHY_KEY4 asc,PHY_KEY5 asc,PHY_KEY6

Re: Solr takes time to warm up core with huge data

2020-06-04 Thread Shawn Heisey
On 6/4/2020 9:51 PM, Srinivas Kashyap wrote: We are on solr 8.4.1 and In standalone server mode. We have a core with 497,767,038 Records indexed. It took around 32Hours to load data through DIH. The disk occupancy is shown below: 82G /var/solr/data//data/index When I restarted solr

Re: question about setup for maximizing solr performance

2020-06-01 Thread Shawn Heisey
On 6/1/2020 9:29 AM, Odysci wrote: Hi, I'm looking for some advice on improving performance of our solr setup. Does anyone have any insights on what would be better for maximizing throughput on multiple searches being done at the same time? thanks! In almost all cases, adding memory will

Re: +(-...) vs +(*:* -...) vs -(+...)

2020-05-21 Thread Shawn Heisey
On 5/21/2020 12:25 PM, Jochen Barth wrote: why does +(-x_ss:y) finds 0 docs, while -(+x_ss:y) finds many docs? Ok... +(*:* -x_ss:y) works, too, but I'm a bit surprised. Purely negative queries, if that is what ultimately makes it to Lucene, do not work. The basic problem is that if you

Re: Need help on handling large size of index.

2020-05-20 Thread Shawn Heisey
On 5/20/2020 11:43 AM, Modassar Ather wrote: Can you please help me with following few questions? - What is the ideal index size per shard? We have no way of knowing that. A size that works well for one index use case may not work well for another, even if the index size in both cases

Re: This IndexSchema is not mutable. Solr 7.3.1

2020-05-20 Thread Shawn Heisey
On 5/20/2020 4:30 PM, Vincenzo D'Amore wrote: another update. I think I found the problem. This error is generated when I have defined add-schema-fields in the updateRequestProcessorChain. In other words you can have ClassicIndexSchemaFactory but (and make sense) add-schema-fields has to be

Re: Solrcloud 6.6 becomes nuts

2020-05-17 Thread Shawn Heisey
On 5/17/2020 4:18 PM, Dominique Bejean wrote: I was not thinking that queries using facet with fields with high number of unique value but with low hits count can be the origin of this problem. Performance for most things does not depend on numFound (hit count) or the rows parameter. The

Re: Solrcloud 6.6 becomes nuts

2020-05-17 Thread Shawn Heisey
On 5/17/2020 2:05 AM, Dominique Bejean wrote: One or two hours before the nodes stop with OOM, we see this scenario on all six nodes during the same five minutes time frame : * a little bit more young gc : from one each second (duration<0.05secs) to one each two or three seconds (duration <0.15

Re: nested entities and DIH indexing time

2020-05-14 Thread Shawn Heisey
On 5/14/2020 3:14 PM, matthew sporleder wrote:> Can a non-nested entity write into existing docs, or do they always> have to produce document-per-entity? This is the only thing I found on this topic, and it is on a third-party website, so I can't say much about how accurate it is:

Re: 404 response from Schema API

2020-05-14 Thread Shawn Heisey
On 5/14/2020 1:13 PM, Mark H. Wood wrote: On Fri, Apr 17, 2020 at 10:11:40AM -0600, Shawn Heisey wrote: On 4/16/2020 10:07 AM, Mark H. Wood wrote: I need to ask Solr 4.10 for the name of the unique key field of a schema. So far, no matter what I've done, Solr is returning a 404. The Luke

Re: nested entities and DIH indexing time

2020-05-14 Thread Shawn Heisey
On 5/14/2020 9:36 AM, matthew sporleder wrote: It appears that adding entities to my entities in my data import config is slowing down my import process by a lot. Is there a good way to speed this up? I see the ID's are individually queried instead of using IN() or similar normal techniques to

Re: Response Time Diff between Collection with low deletes

2020-05-10 Thread Shawn Heisey
On 5/10/2020 4:48 PM, Ganesh Sethuraman wrote: The additional info is that when we execute the test for longer (20mins) we are seeing better response time, however for a short test (5mins) and rerun the test after an hour or so we are seeing slow response times again. Note that we don't update

Re: off-heap OOM

2020-04-29 Thread Shawn Heisey
On 4/29/2020 2:07 AM, Raji N wrote: Has anyone encountered off-heap OOM. We are thinking of reducing heap further and increasing the hardcommit interval . Any other suggestions? . Please share your thoughts. It sounds like it's not heap memory that's running out. When the OutOfMemoryError is

Re: stored=true what should I see from stem fields

2020-04-24 Thread Shawn Heisey
On 4/24/2020 5:48 PM, matthew sporleder wrote: Is what is shown in "analysis" the same as what is stored in a field? The stored data (what you see in search results) is always exactly what was sent to Solr, modified by any update processors that are in use. The index (what you are actually

Re: Solr performance using fq with multiple values

2020-04-18 Thread Shawn Heisey
On 4/18/2020 12:20 PM, Odysci wrote: We don't used this field for general queries (q:*), only for fq and faceting. Do you think making it indexed="true" would make a difference in fq performance? fq means "filter query". It's still a query. So yes, the field should be indexed. The query

Re: 404 response from Schema API

2020-04-17 Thread Shawn Heisey
On 4/16/2020 10:07 AM, Mark H. Wood wrote: I need to ask Solr 4.10 for the name of the unique key field of a schema. So far, no matter what I've done, Solr is returning a 404. This works: curl 'https://toolshed.wood.net:8443/isw6_3/solr/statistics/select' This gets a 404: curl

Re: Multiple versions of Solr nodes in the same cloud

2020-04-08 Thread Shawn Heisey
On 4/6/2020 11:13 AM, Kudrettin Güleryüz wrote: Can I run let's say, Solr 8.5.0 together with Solr 7.3.1. nodes while replacing older nodes with 8.5.0 one by one? We have a cloud with rather large disk space utilization. What is the suggested way to handle this kind of situation? Building

Re: A way to initialize existing core

2020-04-04 Thread Shawn Heisey
On 3/28/2020 8:02 AM, Marius Ghita wrote: I'm running Solr inside a VM and I'm mounting the core folder when the VM boots up. However, if I destroy the VM and reinstall Solr, it won't recognize the previously created core. How can I tell Solr in an automated way to register the core? When Solr

Re: how splitting more shards impact performance

2020-02-03 Thread Shawn Heisey
On 2/3/2020 5:17 PM, ChienHua wrote: What should we expect the query performance impacted by splitting one collection into more shards? We expect the query performance would degrade by splitting more shards since the overhead of merging results from several shards. However, the test result

Re: Performance comparison for wildcard searches

2020-02-03 Thread Shawn Heisey
On 2/3/2020 12:06 PM, Rahul Goswami wrote: I am working with Solr 7.2.1 and had a question regarding the performance of wildcard searches. q=*:* vs q=id:* vs q=id:[* TO *] Can someone please rank them in the order of performance with the underlying reason? The only one of those that is an

Re: Solr 8.4.1 error

2020-02-03 Thread Shawn Heisey
On 2/3/2020 5:16 AM, Srinivas Kashyap wrote: I'm trying to upgrade to solr 8.4.1 and facing below error while start up and my cores are not being listed in solr admin screen. I need your help. Caused by: java.nio.channels.OverlappingFileLockException at

Re: Importing Large CSV File into Solr Cloud Fails with 400 Bad Request

2020-02-02 Thread Shawn Heisey
On 2/2/2020 8:47 AM, Joseph Lorenzini wrote: 1000 1 That autoSoftCommit setting is far too aggressive, especially for bulk indexing. I don't know whether it's causing the specific problem you're asking about here, but it's still a setting

Re: Timeout occurred while waiting response from server

2020-01-30 Thread Shawn Heisey
On 1/30/2020 4:28 AM, Gell-Holleron, Daniel wrote: I'm having trouble when trying to back up my Solr collection. I send the request through and eventually I'll be prompted with a timeout error from. Is there a timeout setting that can be changed so I can increase the amount of time it takes

Re: Doubt in solr metrics

2020-01-30 Thread Shawn Heisey
On 1/30/2020 5:28 AM, akhil dutt wrote: What does this "gc.G1-Young-Generation.count" metric give ? number of times the garbage collector was called till that point of time or number of objects it holds? This number comes from Java. It is the number of times that the collector has been

Re: Solr Searcher 100% Latency Spike

2020-01-29 Thread Shawn Heisey
On 1/29/2020 2:48 PM, Karl Stoney wrote: I know the images didn't load btw so when I say spike I mean p95th response time going from 50ms to 100-120ms momentarily. I agree with Erick on looking at what users can actually notice. When the normal response time is 50 milliseconds, even if that

Re: How expensive is core loading?

2020-01-29 Thread Shawn Heisey
On 1/29/2020 3:01 PM, Rahul Goswami wrote: 1) How expensive is core loading if I am only getting stats like the total docs and size of the index (no expensive queries)? 2) Does the memory consumption on core loading depend on the index size ? 3) What is a reasonable value for transient cache

Re: Clarity on Stable Release

2020-01-29 Thread Shawn Heisey
On 1/29/2020 11:24 AM, Jeff wrote: Now, we are considering 8.2.0, 8.3.1, or 8.4.1 to use as they seem to be stable. But it is hard to determine if we should be using the bleeding edge or a few minor versions back since each of these includes many bug fixes. It is unclear to me why some fixes

Re: Can I create 1000 cores in SOLR CLOUD

2020-01-29 Thread Shawn Heisey
On 1/27/2020 4:59 AM, Vignan Malyala wrote: We are currently using solr without cloud with 500 cores. It works good. Now we are planning to expand it using solr cloud with 1000 cores, (2 cores for each of my client with different domain data). SolrCloud starts having scalability issues once

Re: Operation backup caused exception : AccessDeniedException

2020-01-29 Thread Shawn Heisey
On 1/29/2020 3:26 AM, Salmaan Rashid Syed wrote: I was trying to execute the backup command using curl command on my work computer to see why EC2 instance was giving the previous error. On my current computer, I have root privileges. But when I execute the command on my work computer, I have a

Re: Operation backup caused exception : AccessDeniedException

2020-01-27 Thread Shawn Heisey
On 1/27/2020 6:57 PM, Salmaan Rashid Syed wrote: I have Solr 7.6 and it is installed in an AWS ec2 (Ubuntu machine) instance. I am new to AWS. Devops team has configured it for me. When I check out the source for Solr 7.6, it intersects perfectly with the line numbers in the stacktrace that

Re: Operation backup caused exception : AccessDeniedException

2020-01-27 Thread Shawn Heisey
On 1/27/2020 1:58 AM, Salmaan Rashid Syed wrote: Collection: PANNA operation: backup failed:java.nio.file.AccessDeniedException: /opt/solr_Backup/solr-data-backup-27-01-2020/panna_backup at java.base/java.nio.file.Files.createDirectory(Files.java:689) While trying to create a directory,

Re: How to negate numeric range query - or - how to get records NOT matching a certain numeric range

2020-01-24 Thread Shawn Heisey
On 1/24/2020 9:04 AM, David Hastings wrote: just tried "fq":"NOT year:[1900 TO 2000]"}}, on my data et and also worked as expected, mind if i ask why: (u_lastLendingDate_combined_ls_ns:([8610134693 TO 8611935823])) there are ()'s around your range query? I think David is correct here about

Re: Solr cloud production set up

2020-01-18 Thread Shawn Heisey
On 1/18/2020 10:09 AM, Rajdeep Sahoo wrote: We are having 2.3 million documents and size is 2.5 gb. 10 core cpu and 24 gb ram . 16 slave nodes. Still some of the queries are taking 50 sec at solr end. As we are using solr 4.6 . Other thing is we are having 200 (avg) facet fields in a

Re: Solr cloud production set up

2020-01-18 Thread Shawn Heisey
On 1/18/2020 9:55 AM, Rajdeep Sahoo wrote: We do parallel indexing in production, What about search performance in solr cloud in comparison with master slave. And what about block join performance in solr cloud. Do we need to increase the infra for solr cloud as we would be

Re: Solr cloud production set up

2020-01-18 Thread Shawn Heisey
On 1/18/2020 1:05 AM, Rajdeep Sahoo wrote: Our Index size is huge and in master slave the full indexing time is almost 24 hrs. In future the no of documents will increase. So,please some one recommend about the no of nodes and configuration like ram and cpu core for solr cloud. Indexing is

Re: Querying multiple pages for same keyword at same time

2020-01-13 Thread Shawn Heisey
On 1/13/2020 11:53 AM, Gael Jourdan-Weil wrote: Just to clarify something, we are not returning 1000 docs per request, we are only returning 100. We get 10 requests to Solr querying for docs 1 to 100, then 101 to 200, ... until 901 to 1000. But all that in the exact same second. But I

Re: Search phrase not parsed properly

2020-01-10 Thread Shawn Heisey
On 1/10/2020 5:30 PM, chester wrote: I'm using solr 6.3 and am having an issue with a certain search phrase. When I search for the phrase "Perkins AND Will", the parsed query does not include "Will". See debug info below. select?q=firmname:(Perkins%20AND%20Will) "debug":{

Re: does copyFields increase indexe size ?

2019-12-27 Thread Shawn Heisey
On 12/26/2019 1:21 PM, Nicolas Paris wrote: Below a part of the managed-schema. There is 1k section* fields. The second experience, I removed the copyField, droped the collection and re-indexed the whole. To mesure the index size, I went to solr-cloud and looked in the cloud part: 40GO per

Re: does copyFields increase indexe size ?

2019-12-24 Thread Shawn Heisey
On 12/24/2019 5:11 PM, Nicolas Paris wrote: Do you mean "copy fields" is only an action of changing the schema ? I was thinking it was adding a new field and eventually a new index to the collection The copy that copyField does happens at index time. Reindexing is required after changing the

Re: does copyFields increase indexe size ?

2019-12-24 Thread Shawn Heisey
On 12/24/2019 10:45 AM, Nicolas Paris wrote: From my understanding, copy fields creates an new indexes from the copied fields. From my tests, I copied 1k textual fields into _text_ with copyFields. As a result there is no increase in the size of the collection. All the source fields are

Re: Help for importing large data (approx. 8GB) from old solr version to new solr version

2019-12-23 Thread Shawn Heisey
On 12/23/2019 9:58 PM, Ken Walker wrote: We are upgrading solr version from solr 8.2.0 version to solr 8.3.1 version but we have faced out of memory error while importing data and then we have extended memory in our server and then again start importing process but it has work too slowy for 8GB

Re: how to exclude path from being queried

2019-12-18 Thread Shawn Heisey
On 12/18/2019 1:21 PM, Nan Yu wrote:     I am trying to find all files containing a keyword in a directory (and many sub-directories).     I did a quick indexing using bin/post -c myCore /RootDir     When I query the index using "keyword", all files whose path containing the keyword

Re: Starting Solr automatically

2019-12-18 Thread Shawn Heisey
On 12/16/2019 9:48 PM, Anuj Bhargava wrote: Often solr stops working. We have to then go to the root directory and give the command *'service solr start*' Is there a way to automatically start solr when it stops. If Solr is stopping, then something went wrong. Something that will probably

Re: Updates via curl and json not showing in api

2019-12-13 Thread Shawn Heisey
On 12/13/2019 9:23 AM, rhys J wrote: When I do the following update: curl http://localhost:8983/solr/debt/update -d '[ {"id": "393291-18625", "orig_int_amt":{ "set" : "2.5"}, }]' and then: curl http://localhost:8983/solr/debt/get?id=393291-18625 I see the document is updated via the command

Re: Sometimes searching slow in Solr 6.1.0

2019-12-13 Thread Shawn Heisey
On 12/13/2019 12:29 AM, vishal patel wrote: We have 2 shards and 2 replicas in our live environment. Total of 26 collections. we give 64GB RAM for a single Solr instance. Are you saying that the machine has 64GB of memory, or that the Java heap for Solr is 64GB? Looking over the list

Re: Atomic solrj update

2019-12-12 Thread Shawn Heisey
On 12/12/2019 10:00 PM, Prem wrote: I am trying to partially update of 50M data in a collection from CSV using Atomic script(solrj).But it is taking 2 hrs for 1M records.is there anyway i can speed up my update. How many documents are you sending in one request? Using HTTPClient to establish

Re: A question of solr recovery

2019-12-12 Thread Shawn Heisey
On 12/12/2019 8:53 AM, Shawn Heisey wrote: I do not think the replication handler deals with tlog files at all. The transaction log capability did not exist when the replication handler was built. I may have mixed up your message with a different one. Looking back over this, I don't see any

Re: A question of solr recovery

2019-12-12 Thread Shawn Heisey
On 12/12/2019 3:37 AM, Hongxu Ma wrote: And I found my "full sync" log: "IndexFetcher Total time taken for download (fullCopy=true,bytesDownloaded=178161685180) : 4377 secs (40704063 bytes/sec) to NIOFSDirectory@..." A more question: Form the log, looks it downloaded all segment files

Re: user solr created by install not working with default password

2019-12-11 Thread Shawn Heisey
On 12/11/2019 9:17 AM, rhys J wrote: When I try to su solr, using the password mentioned here: https://lucidworks.com/post/securing-solr-basic-auth-permission-rules/, i get an authentication failure. That page talks about setting up authentication for HTTP access to the Solr API. It has

Re: Convert TLOG collection to NRT

2019-12-10 Thread Shawn Heisey
On 12/10/2019 11:25 AM, Dominique Bejean wrote: I would like to convert a collection (3 shards x 3 replicas) from TLOG to NRT. The only solution I imagine is something like : * with collection API, remove replicas in order to keep only 1 replica per 3 shard * update the collection state.json in

Re: native Thread - solr 8.2.0

2019-12-09 Thread Shawn Heisey
On 12/9/2019 2:23 PM, Joe Obernberger wrote: Getting this error on some of the nodes in a solr cloud during heavy indexing: Caused by: java.lang.OutOfMemoryError: unable to create new native thread Java was not able to start a new thread. Most likely this is caused by the operating

Re: Solr indexing performance

2019-12-05 Thread Shawn Heisey
On 12/5/2019 10:42 PM, Paras Lehana wrote: Can ulimit settings impact this? Review once. If the OS limits prevent Solr from opening a file or starting a thread, it is far more

Re: From solr to solr cloud

2019-12-05 Thread Shawn Heisey
On 12/5/2019 12:28 PM, Vignan Malyala wrote: I currently have 500 collections in my stand alone solr. Bcoz of day by day increase in Data, I want to convert it into solr cloud. Can you suggest me how to do it successfully. How many shards should be there? How many nodes should be there? Are so

Re: xms/xmx choices

2019-12-05 Thread Shawn Heisey
On 12/5/2019 12:57 PM, David Hastings wrote: That probably isnt enough data, so if youre interested: https://gofile.io/?c=rZQ2y4 The previous one was less than 4 minutes, so it doesn't reveal anything useful. This one is a little bit less than two hours. That's more useful, but still

Re: xms/xmx choices

2019-12-05 Thread Shawn Heisey
On 12/5/2019 11:58 AM, David Hastings wrote: as of now we do an xms of 8gb and xmx of 60gb, generally through the dashboard the JVM hangs around 16gb. I know Xms and Xmx are supposed to be the same so thats the change #1 on my end, I am just concerned of dropping it from 60 as thus far over the

Re: Solr indexing performance

2019-12-05 Thread Shawn Heisey
On 12/5/2019 10:28 AM, Rahul Goswami wrote: We have a Solr 7.2.1 Solr Cloud setup where the client is indexing in 5 parallel threads with 5000 docs per batch. This is a test setup and all documents are indexed on the same node. We are seeing connection timeout issues thereafter some time into

Re: upgrade solr from 7.2 to 74

2019-11-30 Thread Shawn Heisey
On 11/30/2019 9:48 AM, Sergio Herrero wrote: I am witing to you in order to be helped. I am trying to upgrade this solr software but It is impossible for me. I am doing it with this command: sudo bash ./install_solr_service.sh solr-7.7.2.tgz -f All seems to be nice but status show:

Re: problem using Http2SolrClient with solr 8.3.0

2019-11-28 Thread Shawn Heisey
On 11/28/2019 9:30 AM, Odysci wrote: No, I did nothing specific to Jetty. Should I? The http/2 Solr client uses a different http client than the previous ones do. It uses the client from Jetty, while the previous clients use the one from Apache. Achieving http/2 with the Apache client

Re: Cursor mark page duplicates

2019-11-28 Thread Shawn Heisey
On 11/28/2019 1:30 AM, Dwane Hall wrote: I asked a question on the forum a couple of weeks ago regarding cursorMark duplicates. I initially thought it may be due to HDFSCaching because I was unable replicate the issue on local indexes but unfortunately the dreaded duplicates have returned!!

Re: Solr master issue : IndexNotFoundException

2019-11-27 Thread Shawn Heisey
On 11/27/2019 6:28 AM, Akreeti Agarwal wrote: Caused by: org.apache.lucene.index.IndexNotFoundException: no segments* file found in LockValidatingDirectoryWrapper(NRTCachingDirectory(MMapDirectory@/solr-m/server/solr/sitecore_web_index/data/index

Re: Some newby questions ...

2019-11-26 Thread Shawn Heisey
On 11/26/2019 2:17 PM, Christian Dannemann wrote: Issue 1: I want to secure my server with basic authentication (that's why I'm running on port 10539 at the moment, but that's not security ... I've put a file security.json in /opt/solr/server/solr/configsets/_default/conf, but that doesn't do

Re: How to tell which core was used based on Json or XML response from Solr

2019-11-25 Thread Shawn Heisey
On 11/25/2019 8:30 AM, rhys J wrote: On Mon, Nov 25, 2019 at 2:10 AM Erik Hatcher wrote: add ==all and the parameter will be in the response header. Erik Thanks. I just tried this, and all I got was this response: http://localhost:8983/solr/dbtr/select?q=debtor_id%3A%20393291=all

Re: fq pfloat_field:* returns no documents, tfloat:* does

2019-11-21 Thread Shawn Heisey
On 11/21/2019 7:48 AM, Webster Homer wrote: Thank you. Why don't point fields get loaded by the Schema Browser's "Load Term Info" button? From what I've seen in the past on discussions, Point-based fields are missing Term data. There's literally nothing to load.

Re: How to change config set for some collection

2019-11-20 Thread Shawn Heisey
On 11/20/2019 1:34 PM, Pratik Patel wrote: Let's say I have a collection called "collection1" which uses config set "config_set_1". Now, using "upconfig" command, I upload a new configuration called "config_set_2". How can I make "collection1" use "config_set_2" instead of "config_set_1"? I

Re: When does Solr write in Zookeeper ?

2019-11-18 Thread Shawn Heisey
On 11/18/2019 8:39 AM, Dominique Bejean wrote: How Solr nodes know that something was changed in Zookeeper by an other node ? Is there any notification from ZK or do Solr nodes read systematically in ZK (without local caching) ? This is built-in functionality of ZooKeeper. The client allows

Re: using NOT or - to exclude results with a textField type

2019-11-15 Thread Shawn Heisey
On 11/15/2019 10:15 AM, rhys J wrote: I'm trying to exclude results based on the documentation about the boolean NOT symbol, but I keep getting errors. I've tried: http://localhost:8983/solr/debt/select?q=clt_ref_no:-”owl-2924-8” and

Re: using gt and lt in a query

2019-11-14 Thread Shawn Heisey
On 11/14/2019 11:09 AM, rhys J wrote: But I'm not doing a range, I'm doing a query on whether one field is greater than another field. Or am I missing something here? I am unaware of any way to do that with simple query syntax. Sometimes there is capability that I do not know about, though.

Re: using gt and lt in a query

2019-11-14 Thread Shawn Heisey
On 11/14/2019 10:49 AM, rhys J wrote: I am trying to duplicate this line from a db query: (debt.orig_princ_amt > 0 AND debt.princ_paid > 0 AND debt.orig_princ_amt > debt.princ_paid) I have the following, but it returns no results: Range queries are done with brackets and/or braces. A square

Re: daily SolrCloud collection wipes

2019-11-14 Thread Shawn Heisey
On 11/14/2019 9:17 AM, Werner Detter wrote: first, thanks for your response. By "reset" I mean: collection still exists but documents have been dropped (from actually round 50k to 0). It happened twice within the same timeframe early in the morning the last two days so I was wondering if

Re: Question about startup memory usage

2019-11-14 Thread Shawn Heisey
On 11/14/2019 1:46 AM, Hongxu Ma wrote: Thank you @Shawn Heisey<mailto:apa...@elyograg.org> , you help me many times. My -xms=1G When restart solr, I can see the progress of memory increasing (from 1G to 9G, took near 10s). I have a guess: maybe solr is loading some needed files int

Re: daily SolrCloud collection wipes

2019-11-14 Thread Shawn Heisey
On 11/14/2019 12:28 AM, Werner Detter wrote: I've got a SolrCloud instance with two collections running (Solr 7.7.2) on Debian Stretch VMs. Every morning round about 03:3* am the collection gets reset by $something and I have no clue what causes this and how to prevent it as there areeven no log

Re: date fields and invalid date string errors

2019-11-13 Thread Shawn Heisey
On 11/13/2019 10:45 AM, rhys J wrote: If you use DateRangeField instead of DatePointField for your field's class, then you can indeed use partial timestamps for both indexing and querying. This only works with DateRangeField. I don't see that as an option in the API? Do I need to change what

Re: Need some help on solr versions (LTS vs stable)

2019-11-13 Thread Shawn Heisey
On 11/6/2019 9:58 AM, suyog joshi wrote: So we can say its better to go with latest stable version (8.x) instead of 7.x, which is LTS right now, but can soon become EOL post launching of 9.x sometime early next year. I don't know where you got the idea that 7.x is LTS ... but I do not think

Re: date fields and invalid date string errors

2019-11-13 Thread Shawn Heisey
On 11/13/2019 10:07 AM, rhys J wrote: I have date fields in my documents that are just -MM-DD. and When I use the API to do a search and try: 2018-01-01 [2018-01-01 TO NOW] I get 'Invalid Date String'. Did I type my data wrong in the schema? Is there something I'm missing from the

Re: Question about startup memory usage

2019-11-13 Thread Shawn Heisey
On 11/13/2019 2:03 AM, Hongxu Ma wrote: I have a solr-cloud cluster with a big collection, after startup (no any search/index operations), its jvm memory usage is 9GB (via top: RES). Cluster and collection info: each host: total 64G mem, two solr nodes with -xmx=15G collection: total 9B

Re: EmbeddedSolrServer thread-safety

2019-11-13 Thread Shawn Heisey
On 11/13/2019 9:45 AM, Kaminski, Adi wrote: Can someone confirm that EmbeddedSolrServer calss is thread-safe (meaning from API perspective of add/delete/query operations by different threads)? It's not written in any documentation, and we have some different answers from different forums. The

Re: Full-text search for Solr manual

2019-11-13 Thread Shawn Heisey
On 11/12/2019 11:15 PM, Luke Miller wrote: Unfortunately the Solr source does not ship with the source of the manual. (Directory /docs only contains a link to the online manual.) The docs directory is not where the manual lives. In the source code download, the solr/docs directory has an

Re: Question about memory usage and file handling

2019-11-11 Thread Shawn Heisey
On 11/11/2019 1:40 PM, siddharth teotia wrote: I have a few questions about Lucene indexing and file handling. It would be great if someone can help with these. I had earlier asked these questions on gene...@lucene.apache.org but was asked to seek help here. This mailing list (solr-user) is

Re: Solr 7.2.1 - unexpected docvalues type

2019-11-08 Thread Shawn Heisey
On 11/8/2019 5:31 PM, Antony Alphonse wrote: I shared the collection and re-indexed the data with the same schema. But one of the field is throwing the below error. Any suggestions? ERROR (qtp672320506-32) [c: s:shard3 r:core_node01 x:_shard3_replica_n69] o.a.s.h.RequestHandlerBase

Re: ConcurrentModificationException in SolrInputDocument writeMap

2019-11-07 Thread Shawn Heisey
On 11/6/2019 8:17 AM, Tim Swetland wrote: I'm currently running into a ConcurrentModificationException ingesting data as we attempt to upgrade from Solr 8.1 to 8.2. It's not every document, but it definitely appears regularly in our logs. We didn't run into this problem in 8.1, so I'm not sure

Re: Solr 8.2.0 - Unable to write response

2019-11-01 Thread Shawn Heisey
On 11/1/2019 7:20 AM, Joe Obernberger wrote: Hi All - getting this error from only one server in a 45 node cluster when calling COLSTATUS.  Any ideas? 2019-11-01 13:17:32.556 INFO  (qtp694316372-44709) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections

Re: Solr v4.2.1: fields without associated documents

2019-10-29 Thread Shawn Heisey
On 10/29/2019 4:05 PM, Shawn Heisey wrote: I can ask on our dev list to see what I can learn. I should add something important to this. Even if we can implement an enhancement, it would only be added to an 8.x version at the earliest. It is not possible to take an index from 4.2.1 and use

Re: Solr v4.2.1: fields without associated documents

2019-10-29 Thread Shawn Heisey
On 10/29/2019 2:25 PM, Bridger Dyson-Smith wrote: A very helpful person on IRC suggested that the only way to get rid of these might be a clean rebuild of the index, and that's not out of the question for us; I hoped to get a bit more information here. I'm the one who you talked to on IRC.

Re: ant precommit fails on .adoc files

2019-10-29 Thread Shawn Heisey
On 10/8/2019 5:37 PM, Chris Hostetter wrote: This is strange -- I can't reproduce, and I can't see any evidence of a change to explain why this might have been failing 8 days ago but not any more. On the master branch that I just updated, "ant clean precommit" fails. Looks like it had a

Re: Solr Ref Guide Changes - now HTML only

2019-10-28 Thread Shawn Heisey
On 10/28/2019 3:51 PM, Nicolas Paris wrote: I am not very happy with the search engine embedded within the html documentation I admit. Hope this is not solr under the hood :S It's not Solr under the hood. It is done by a javascript library that runs in the browser. It only searches page

Re: Parts of the Json response to a curl query are arrays, and parts are hashes

2019-10-28 Thread Shawn Heisey
On 10/28/2019 9:30 AM, rhys J wrote: Will I break Solr if i change this to default to not multi-valued? If you are only indexing one value in those fields, then setting multiValued to false will not break anything. If an indexing request ever comes in that has more than one value for a

Re: merge policy & autocommit

2019-10-28 Thread Shawn Heisey
On 10/28/2019 7:23 AM, Danilo Tomasoni wrote: We have a solr instance with around 40MLN docs. In the bulk import phase we noticed a high IO and CPU load and it looks like it's related to autocommit because if I disable autocommit the load of the system is very low. I know that disabling

Re: Parts of the Json response to a curl query are arrays, and parts are hashes

2019-10-25 Thread Shawn Heisey
On 10/25/2019 2:30 PM, rhys J wrote: So I went back to one of the fields that is multi-valued, which I explicitly did not choose when I created the field, and I re-created it. It still made the field multi-valued as true. Why is this? Did you reload the core/collection or restart Solr so the

Re: cdcr replicator NPE errors

2019-10-25 Thread Shawn Heisey
On 10/25/2019 3:22 PM, Jay Potharaju wrote: Is there a solr slack channel? People with @apache.org email addresses can readily join the ASF workspace, I do not know whether it is possible for others. That workspace might be only for ASF members. https://the-asf.slack.com In that

Re: Parts of the Json response to a curl query are arrays, and parts are hashes

2019-10-25 Thread Shawn Heisey
On 10/25/2019 1:48 PM, rhys J wrote: Is there some reason that text_general fields are returned as arrays, and other fields are returned as hashes in the json response from a curl query? Here's the response: "dl2":["Great Plains"], "do_not_call":false, There are no

Re: solr configuration issue

2019-10-25 Thread Shawn Heisey
On 10/25/2019 5:44 AM, Danilo Tomasoni wrote: Another question, is softCommit sufficient to ensure visibility or should I call a commit to ensure a new searcher will be opened? softCommit automatically opens a new searcher? There would be little point to doing a soft commit with openSearcher

Re: NRT vs TLOG bulk indexing performances

2019-10-25 Thread Shawn Heisey
On 10/25/2019 1:16 AM, Dominique Bejean wrote: For collection created with all replicas as NRT * Indexing time : 22 minutes For collection created with all replicas as TLOG * Indexing time : 34 minutes NRT indexes simultaneously on all replicas. So when indexing is done on one, it is

Re: solr-user-subscribe

2019-10-25 Thread Shawn Heisey
On 10/24/2019 11:19 PM, Hafiz Muhammad Shafiq wrote: HI, I am using Solr 6.x version for search purposes. Now data has been increased into one shard. I have to create some additional shards and also have to balance base on number of documents. According to my search, solr does not provide

Re: Migration: SOLR8-Java8 -> SOLR8-JAVA11 indexing issue.

2019-10-24 Thread Shawn Heisey
On 10/24/2019 11:50 AM, Junagade, Anup wrote: * Attachment 1: OpenJDK 8 vs OpenJDK 8 key metrics * Attachment 2: OpenJDK 8 vs OpenJDK 8 waiting QTP Threads * Attachment 3: OpenJDK 11 Thread dump There are no attachments. Apache mailing lists swallow almost all attachments.

<    1   2   3   4   5   6   7   8   9   10   >