Re: Solr admin timeout

2018-09-20 Thread Shawn Heisey
On 9/19/2018 1:05 PM, Webster Homer wrote: I have a fairly complex query which I'm trying to debug. The query works as long as I don't try to return the field: [explain style=nl] Of course this is the data I'm really interested in. When I run it, the console acts busy, but then the screen

Re: Solr 7.X negative filter not working

2018-09-20 Thread Shawn Heisey
On 9/20/2018 2:04 AM, damian.pawski wrote: On the Solr 5.4.x below query works fine: * "q": "*:*", "_": "1537429094299", "wt": "json", "fq": "JobTitle:(NOT programmer)" ... * I'm surprised this works on any version. See this page:

Re: SolrCoreInitializationException after restart of one solr node

2018-09-20 Thread Shawn Heisey
On 9/19/2018 9:20 PM, Schaum Mallik wrote: The data and index get stored under /opt/solr/server/solr/articles_shard1_replica_n1. The config directory when the collection was created, that time the path to the config was given as '/opt/solr/server/solr/configsets/articles'. I didn't use the

Re: SolrCoreInitializationException after restart of one solr node

2018-09-19 Thread Shawn Heisey
On 9/19/2018 8:22 PM, Schaum Mallik wrote: I have a 3 zookeeper ensemble and 3 solr nodes running version 7.4.0. Recently I had to restart one node and after I did that it started throwing this exception. Caused by: org.apache.solr.common.SolrException: No coreNodeName for

Re: Unnecessary Components

2018-09-19 Thread Shawn Heisey
On 9/19/2018 1:48 PM, oddtyme wrote: I am helping implement solr for a "downloadable library" of sorts. The objective is that communities without internet access will be able to access a library's worth of information on a small, portable device. As such, I am working within strict space

Re: Is that solr supports multi version operations?

2018-09-19 Thread Shawn Heisey
On 9/18/2018 8:11 PM, zhenyuan wei wrote: Hi all, add solr document with overwrite=false will keepping multi version documents, My question is : 1. How to search newest documents?with what options? 2. How to delete old version < newest version documents? When Solr is

Re: SOLR 7.0 DIH out of memory issue with sqlserver

2018-09-18 Thread Shawn Heisey
On 9/18/2018 4:48 PM, Tanya Bompi wrote: I have the SOLR 7.0 setup with the DataImportHandler connecting to the sql server db. I keep getting OutOfMemory: Java Heap Space when doing a full import. The size of the records is around 3 million so not very huge. I tried the following steps and

Re: Command Line Indexer

2018-09-18 Thread Shawn Heisey
On 9/18/2018 2:21 PM, Christopher Schultz wrote: AIUI, Solr doesn't support updating a single field in a document. The document is replaced no matter how hard to try to be surgical about updating a single field. Solr does have Atomic Update functionality.  For this to work, the index must be

Re: Solr 7.2.1 Collection Backup Performance issue

2018-09-18 Thread Shawn Heisey
On 9/18/2018 11:00 AM, Ganesh Sethuraman wrote: We are using Solr 7.2.1 with SolrCloud with 35 collections with 1 node ZK ensemble (in lower environment, we will have 3 nodes ensemble) in AWS. We are testing to see if we have Async Solr Cloud backup (

Re: weird error for accessing solr

2018-09-18 Thread Shawn Heisey
On 9/18/2018 12:23 PM, Gu, Steve (CDC/OD/OADS) (CTR) wrote: I have set up my solr as a standalone service and the its url is http://solr.server:8983/solr. I opened 8983 on solr.server to anyone, and solr can be accessed from laptops/desktops. But when I tried to access the solr from some

Re: 20180917-Need Apache SOLR support

2018-09-18 Thread Shawn Heisey
On 9/18/2018 1:11 AM, zhenyuan wei wrote: I have 6 machines,and each machine run a solr server, each solr server use RAM 18GB. Total document number is 3.2billion,1.4TB , my collection‘s replica factor is 1。collection shard number is 60,currently each shard is 20~30GB。 15 fields per document。

Re: user field(uf) not working with Post filter

2018-09-18 Thread Shawn Heisey
On 9/5/2018 7:17 AM, shruti suri wrote: I am using a custom handler with edismax parser. I am using uf parameter in the handler to restrict some fields from search. But uf is not working with post filter(fq). I want to restricted same fields in fq, so that people could not filter on some

Re: Does solr support rollback or any method to do the same job?

2018-09-18 Thread Shawn Heisey
On 9/18/2018 4:03 AM, zhenyuan wei wrote: Does solr support rollback or any method to do the same job? Like update/add/delete a document, can I rollback them? With SolrCloud, rollback is not supported.  This is because a typical SolrCloud install spreads the index across multiple servers. 

Re: 20180917-Need Apache SOLR support

2018-09-17 Thread Shawn Heisey
On 9/17/2018 9:05 PM, zhenyuan wei wrote: Is that means: Small amount of shards gains better performance? I also have a usecase which contains 3 billion documents,the collection contains 60 shard now. Is that 10 shard is better than 60 shard? There is no definite answer to this question.  It

Re: Solr standalone health checks

2018-09-17 Thread Shawn Heisey
On 9/17/2018 3:01 PM, Christopher Schultz wrote: The basic questions I'd like to have answered on a regular basis are: 1. Is the JVM up (this can be done with a ping, of course) 2. Is the heap healthy? Any OOMEs? 3. Will a sample query return in a reasonable amount of time? 1 and 3 are quite

Re: OOM Solr 4.8.1

2018-09-17 Thread Shawn Heisey
On 9/17/2018 9:52 AM, Vincenzo D'Amore wrote: recently I had few Java OOM in my Solr 4.8.1 instance. Here the configuration I have. The only part of your commandline options that matters for OOM is the max heap. Which is 16GB for your server.  Note, you should set the min heap and max heap

Re: 20180917-Need Apache SOLR support

2018-09-17 Thread Shawn Heisey
On 9/17/2018 7:04 AM, KARTHICKRM wrote: Dear SOLR Team, We are beginners to Apache SOLR, We need following clarifications from you. Much of what I'm going to say is a mirror of what you were already told by Jan.  All of Jan's responses are good. 1. In SOLRCloud, How can we install

Re: Logging fails when starting Solr in Windows using solr.cmd

2018-09-17 Thread Shawn Heisey
On 9/16/2018 3:05 PM, marcostocch...@gmail.com wrote: I experienced the same issue on Windows 10 professional. I don't think the OS version is important. The solr version might. I have solr-7.4.0. The problem has been fixed in the source code and the next version of Solr (7.5.0) won't

Re: Need Support - Apache Solr - 20180915

2018-09-15 Thread Shawn Heisey
On 9/14/2018 9:51 PM, senthil wrote: We are beginners to Apache Solr and its implementations. We need the following basic clarifications regarding Apache Solr usage and implementing with MS-SQL server database. I don't know what you think of Erick's answers, but he's right on the money

Re: Sorting multi-valued fields

2018-09-14 Thread Shawn Heisey
On 9/14/2018 4:50 AM, richard.clarke wrote: What does it mean to sort documents by a multivalued field? If a field has multiple values, how can this be used to sort documents? e.g. if document 1 has a numeric field containing values 1,2,3,4,5 and document 2 has values in the same field of

Re: solrcloud configuration: solr failed to start with multiple zookeeper servers

2018-09-14 Thread Shawn Heisey
On 9/13/2018 6:47 AM, Gu, Steve (CDC/OD/OADS) (CTR) wrote: After zk servers have started, I got the following error when I tried to start solr: localhost:2182 was unexpected at this time. Where is this error seen?  Was there more to the message?  In addition to knowing that, we will need to

Re: start solr cloud at boot time

2018-09-13 Thread Shawn Heisey
On 9/13/2018 9:25 AM, Chuming Chen wrote: Is there any bash script that can start a solr cloud at boot time?  Use the service installer script.  This will install an init script and set it up so it starts at boot.

Re: 20180913 - Clarification about Limitation

2018-09-13 Thread Shawn Heisey
On 9/13/2018 2:07 AM, Rekha wrote: Hi Solr Team, I am new to SOLR. I need following clarification from you. How many documents can be stored in one core? Is there any limit for number of fields per document? How many Cores can be created in

Re: how to access solr in solrcloud

2018-09-13 Thread Shawn Heisey
On 9/13/2018 6:23 AM, Gu, Steve (CDC/OD/OADS) (CTR) wrote: Hi, Florian We need to pass zookeeper url to CloudSolrClient. Since there are multiple zk servers, is it the common practice to set a proxy server in front of zookeeper? ZooKeeper should not be behind a load balancer.  Your client

Re: how to access solr in solrcloud

2018-09-12 Thread Shawn Heisey
On 9/12/2018 7:38 AM, Gu, Steve (CDC/OD/OADS) (CTR) wrote: I am upgrading our solr to 7.4 and would like to set up solrcloud for failover and load balance. There are three zookeeper servers (zk1:2181, zk1:2182) and two solr instance solr1:8983, solr2:8983. So what will be the solr url

Re: 6.x to 7.x differences

2018-09-12 Thread Shawn Heisey
On 9/12/2018 8:12 AM, John Blythe wrote: shawn: at first, no. we rsynced data up after running it through the migration tool. we'd gotten errors when using WDF so updated all instances of it to WDGF (and subsequently added FlattenGraphFilterFactory to each index analyzer that used WDGF to avoid

Re: Docker and Solr Indexing

2018-09-12 Thread Shawn Heisey
On 9/12/2018 7:43 AM, Dominique Bejean wrote: Are you aware about issues in Java applications in Docker if java version is not 10 ? https://blog.docker.com/2018/04/improved-docker-container-integration-with-java-10/ Solr explicitly sets heap size when it starts, so Java is *NOT* determining

Re: switch query parser and solr cloud

2018-09-12 Thread Shawn Heisey
On 9/12/2018 5:47 AM, Dwane Hall wrote: Good afternoon Solr brains trust I'm seeking some community advice if somebody can spare a minute from their busy schedules. I'm attempting to use the switch query parser to influence client search behaviour based on a client specified request

Re: Error casting to PointField

2018-09-12 Thread Shawn Heisey
On 9/11/2018 10:15 PM, Zahra Aminolroaya wrote: Thanks Erick. We used to use TrieLongField for our unique id and in the document it is said that all Trie* fieldtypes are casting to *pointfieldtypes. What would be the alternative solution? I've never heard of Trie casting to Point. Point is

Re: 6.x to 7.x differences

2018-09-12 Thread Shawn Heisey
On 9/11/2018 8:32 PM, John Blythe wrote: we recently migrated to cloud. part of that migration jumped us from 6.1 to 7.4. one example query between our old solr instance and our new cloud instance produces 42 results and 19k results. the analyzer is the same aside from

Re: Docker and Solr Indexing

2018-09-11 Thread Shawn Heisey
On 9/11/2018 9:20 PM, solrnoobie wrote: So what we did is we upgraded the instances to 16 gigs and we rarely encounter this now. So what we did was to increase the batch size to 500 instead of 50 and it worked for our test data. But when we tried 1000 batch size, the invalid content type error

Re: parent/child rows in solr

2018-09-11 Thread Shawn Heisey
On 9/11/2018 8:35 PM, John Smith wrote: The problem is that the math isn't a simple case of adding up all the row counts. These are "left outer join"s. In sql, it would be this query: I think we'll just have to conclude that I do not understand what you are doing.  I have no idea what "left

Re: parent/child rows in solr

2018-09-11 Thread Shawn Heisey
On 9/11/2018 7:07 PM, John Smith wrote: header: 223,580 child1: 124,978 child2: 254,045 child3: 127,917 child4:1,009,030 child5: 225,311 child6: 381,561 child7: 438,315 child8: 18,850 Trying to index that into solr with a flatfile schema, blows up

Re: group.limit>1 and sorting is not working as expected

2018-09-11 Thread Shawn Heisey
On 9/11/2018 12:00 PM, Venkateswarlu Bommineni wrote: What i am expecting is (it might be silly) if i put group.main=true and sort by price then the results are: { "priceValueGLP_usd_double":32015.0, "sapRank_int":446}, {

Re: Solr RSIZE memory overusage

2018-09-11 Thread Shawn Heisey
On 9/11/2018 12:14 PM, Boris Pasko wrote: Run top, press shift-M to sort by memory usage, then grab a atop: http://oi68.tinypic.com/10pokkk.jpg top: http://oi63.tinypic.com/msbpfp.jpg Looking at the second one: The SHR value is showing 90GB. Your Java process is in actuality only using in

Re: Update partial document

2018-09-11 Thread Shawn Heisey
On 9/11/2018 10:23 AM, Vincenzo D'Amore wrote: I suppose to be able to remove attr_1 and add attr_3 with one atomic update. Like this: curl -X POST -H 'Content-Type: application/json' ' http://localhost:8983/solr/gettingstarted/update?versions=true=true' --data-binary ' [ { "id"

Re: Solr RSIZE memory overusage

2018-09-11 Thread Shawn Heisey
On 9/11/2018 11:07 AM, Boris Pasko wrote: Hi. We're running Solr 6.6.1 (SolrCloud, 3 nodes). Recently I noticed it became significantly slower to respond and did some basic checks on servers. There is little IO, a bit of CPU usage (110% user, 3090% idle), but one thing is very strange - the

Re: group.limit>1 and sorting is not working as expected

2018-09-11 Thread Shawn Heisey
On 9/11/2018 10:14 AM, Venkateswarlu Bommineni wrote: Please find the resonse and query when grouping and sorting by rank : I see no evidence of grouping happening in either of those responses.  They look like standard responses do when grouping is not enabled. Here's an example of a

Re: local "q.op=AND" ignored for edismax query

2018-09-11 Thread Shawn Heisey
On 9/10/2018 5:45 PM, dshih wrote: Based on what you said, is my query supposed to work as is if I set luceneMatchVersion=7.1.0? It does not appear to. It does look like a luceneMatchVersion check was added to the change in SOLR-11501, so I would expect that to work.  Setting

Re: Error while creating a new solr core

2018-09-11 Thread Shawn Heisey
On 9/10/2018 11:51 PM, Shalvak Mittal (UST, ) wrote: I have recently installed solr 7.2.1 in my ubuntu 16.04 system. While creating a new core, the solr logging shows an error saying " Caused by: org.apache.solr.common.SolrException: fips module was not loaded." I have never heard of a

Re: Nutch 1.15 Indexing

2018-09-11 Thread Shawn Heisey
On 9/11/2018 12:22 AM, Bineesh wrote: Need help on Nutch 1.15 indexing issues. We are using Nutch 1.15 and Solr 7.3.1 in our setup 1 : Is there a way i can mention multiple collections in the Nutch 1.15 indexwriters.xml file for the same ? I see collection works fine if i hardcoded the

Re: Date Query Using Local Params

2018-09-10 Thread Shawn Heisey
On 9/10/2018 1:21 PM, Antelmo Aguilar wrote: Hi, I have a question. I am trying to use the "within" op parameter in a Date Search. This works like I would expect: {!field f=collection_date_range op=Within}[2013-07-08 TO 2013-07-09] I would like to use an OR with the query though, something

Re: Unable to enable SSL with self-sign certs

2018-09-10 Thread Shawn Heisey
On 9/10/2018 12:47 PM, cyndefromva wrote: I installed solr 5.4.1 and java 1.8 on its own linux server and used the install_solr_service.sh file to install and setup solr. At this point I was able to start the process and access the dashboard from a browser. After shutting down solr I then

Re: group.limit>1 and sorting is not working as expected

2018-09-08 Thread Shawn Heisey
On 9/8/2018 8:34 PM, Venkateswarlu Bommineni wrote: Query ; https:///solr/default/select?fq=rank_int:[1%20TO%20*]=on=json=true=true= rank_int=3=*=code_string,sapRank_int,price=price+desc I am grouping on field *rank_int with group limit>3 and doing the sorting on price. sorting is happening

Re: parent/child rows in solr

2018-09-07 Thread Shawn Heisey
On 9/7/2018 7:44 PM, John Smith wrote: Thanks Shawn, for your comments. The reason why I don't want to go flat file structure, is due to all the wasted/duplicated data. If a department has 100 employees, then it's very wasteful in terms of disk space to repeat the header data over and over

Re: local "q.op=AND" ignored for edismax query

2018-09-07 Thread Shawn Heisey
On 9/7/2018 6:59 PM, dshih wrote: Query: /select?q={!q.op=AND}mysearchtext*=edismax=true Result: "querystring": "{!q.op=AND}mysearchtext*", "parsedquery": "+DisjunctionMaxQuery(((Synonym(text:q text:qopandmysearchtext text:{!q.op=and}mysearchtext*) text:op text:and text:mysearchtext)))" In the

Re: parent/child rows in solr

2018-09-07 Thread Shawn Heisey
On 9/7/2018 3:06 PM, John Smith wrote: Hi, I have a document structure like this (this is a made up schema, my data has nothing to do with departments and employees, but the structure holds true to my real data): department 1 employee 11 employee 12 employee 13 room 11

Re: Solr 7.4 and log4j2 JSONLayout

2018-09-07 Thread Shawn Heisey
On 9/6/2018 7:46 AM, Michael Aleythe, Sternwald wrote: I'm trying to edit the log4j2 logging configuration for solr. The goal is to get a log file in json format. I configured the the JSONLayout for this purpose inside the rollingFile appender in the log4j2.xml. After this solr stops logging

Re: Expected mime type application/octet-stream but got text/html

2018-09-07 Thread Shawn Heisey
On 9/6/2018 12:29 PM, nalsrini wrote: Here is the error message I am getting: https://screencast.com/t/XwEjA22jX Alexandre covered things quite well.  It looks like you're using an invalid URL. Your original message shows this code: SolrClient client = new

Re: Solr memory reqs for time-sorted data

2018-09-07 Thread Shawn Heisey
On 9/7/2018 8:39 AM, Pavel Micka wrote: I found on wiki (https://wiki.apache.org/solr/SolrPerformanceProblems#RAM) that optimal amount of RAM for SOLR is equal to index size. This is lets say the ideal case to have everything in memory. I wrote that page. We plan to have small installation

Re: Null Pointer Exception without details on Update in schemaless 7.4

2018-09-05 Thread Shawn Heisey
On 9/5/2018 8:32 PM, deniz wrote: I have set up a schemaless solr (cloud) and have been trying to test the updates. as DIH is not going through field guessing, I have wrote a small piece of code to query data from db and push the docs to solr... Once the client pushes the docs to solr, on

Re: Accessing multiple datasource from an entity

2018-09-05 Thread Shawn Heisey
On 9/5/2018 11:36 AM, Monique Monteiro wrote: I need to add a deltaQuery to a DIH configuration, but it should reference a different datasource. Example: * * ...* Is it possible? As far as I am aware, each entity can only reference one dataSource.  Both of

Re: Solr 6.4.1: : SolrException:nfs no locks available

2018-09-05 Thread Shawn Heisey
On 9/5/2018 6:55 AM, Imran Rajjad wrote: I am using Solr Cloud 6.4.1. After a hard restart the solr nodes are constantly showing to be in DOWN state and would not go into recovery. I have also deleted the write.lock files from all the replica folders, but the problem would not go away. The error

Re: Replicas do not come up after nodes are restarted in SOLR cloud

2018-09-05 Thread Shawn Heisey
On 9/5/2018 2:55 AM, Sudip Mukherjee wrote: I have a 2 node SOLR (7.x) cloud cluster on which I have collection with replicas ( replicationFactor = 2, shard = 1 ). I am seeing that the replicas do not come up ( state is "down") when both nodes are restarted. From the "legend" in Graph

Re: Expected mime type application/octet-stream but got text/html

2018-09-04 Thread Shawn Heisey
On 9/4/2018 3:22 PM, nalsrini wrote: Hi,I use Solr 5.3 and I don't have any collection. When I execute the following code, I got this error:Error from server at http://:/solr-master: Expected mime type application/octet-stream but got text/html.SolrClient client = new

Re: Replacing Double Quotes from a field

2018-09-03 Thread Shawn Heisey
On 9/3/2018 1:51 AM, Gopesh Sharma wrote: I am trying to remove the double quotes from a field and that's why written PatternReplaceCharFilterFactory, but it doesn't seem to be working. When you say it's not working, how precisely are you checking?  If you're looking at the field value in

Re: BUMP: Atomic updates and POST command?

2018-09-01 Thread Shawn Heisey
On 8/31/2018 7:18 PM, Scott Prentice wrote: Yup. That does the trick! Here's my command line ..     $ ./bin/post -c core01 -format solr /home/xtech/solrtest/test1b.json I saw that "-format solr" option, but it wasn't clear what it did. It's still not clear to me how that changes the endpoint

Re: Sorting by custom order

2018-09-01 Thread Shawn Heisey
On 9/1/2018 12:36 AM, Salvo Bonanno wrote: I need to sort a results set in a particolar way... the documents looks like this: There are two problems with what you want and your setup. I need to sort them by profile_txt value (it's a multiValue field but actually it contains just a single

Re: ZooKeeper issues with AWS

2018-09-01 Thread Shawn Heisey
On 9/1/2018 3:42 AM, Björn Häuser wrote: as far as I can see the required fix for this is finally in 3.4.13: - https://issues.apache.org/jira/browse/ZOOKEEPER-2184 Would be great to have this in the next solr update. Issue created.

Re: ZooKeeper issues with AWS

2018-08-31 Thread Shawn Heisey
On 8/31/2018 12:14 PM, Jack Schlederer wrote: Our working hypothesis is that Solr's JVM is caching the IP addresses for the ZK hosts' DNS names when it starts up, and doesn't re-query DNS for some reason when it finds that that IP address is no longer reachable (i.e., when a ZooKeeper node

Re: Solrcloud collection file location on zookeeper

2018-08-31 Thread Shawn Heisey
On 8/31/2018 8:40 AM, Sushant Vengurlekar wrote: Any idea where this database is stored on the file system. I don’t want to read it but just know where it resides. If you followed recommendations, your ZooKeeper ensemble is NOT using the ZK server that's embedded inside Solr.  In that case I

Re: change DocExpirationUpdateProcessorFactory deleteByQuery NOW parameter time zone

2018-08-31 Thread Shawn Heisey
On 8/30/2018 7:26 PM, Derek Poh wrote: Can the timezone of the NOW parameter in the |deleteByQuery| of the DocExpirationUpdateProcessorFactory be change to my timezone? I am in SG and using solr 6.5.1. I do not know what SG is. The timezone cannot be changed.  Solr *always* handles dates in

Re: Solrcloud collection file location on zookeeper

2018-08-31 Thread Shawn Heisey
On 8/30/2018 10:15 PM, Sushant Vengurlekar wrote: Where does zookeeper store the collection info on local filesystem on zookeeper? ZooKeeper doesn't store info in a way that you can find on the filesystem.  ZK data is stored in a file structure that's basically a database.   The way I

Re: if we can do that Inverted index and forward information stored on different disks by solr?

2018-08-31 Thread Shawn Heisey
On 8/31/2018 3:19 AM, shreck wrote: thanks , Elasticsearch also dosen't support the feature? in some situation,i think we can improve search performance through this way. I cannot see how it would improve performance. The best way to improve search performance is to add memory to the

Re: [EXTERNAL] - Re: join works with a core, doesn't work with a collection

2018-08-30 Thread Shawn Heisey
On 8/30/2018 9:49 AM, Steve Pruitt wrote: If you mean another running Solr server running, then no. I mean multiple Solr processes. The cloud example (started with bin/solr -e cloud) starts two Solr instances if you give it the defaults.  They are both running on the same machine, but if

Re: [EXTERNAL] - Re: join works with a core, doesn't work with a collection

2018-08-30 Thread Shawn Heisey
On 8/30/2018 9:17 AM, Steve Pruitt wrote: Single server. Localhost. I am using the simple setup and took all the defaults. Is there more than one Solr instance on that server? SolrCloud considers multiple instances to be completely separate, even if they're actually on the same hardware.

Re: join works with a core, doesn't work with a collection

2018-08-30 Thread Shawn Heisey
On 8/30/2018 9:00 AM, Steve Pruitt wrote: Is there something different I need to do for a query with a join for a Collection? Singular Collection, not across Collections. Initially, I used a Core for simple development. One of my queries uses a join. It works fine. I know very little

Re: SolrCore Initialization Failure Error loading class 'solr.IntField'

2018-08-30 Thread Shawn Heisey
On 8/30/2018 3:14 AM, Salvo Bonanno wrote: The solr version in both enviroment is 7.4.0 Looks that there was a problem using the intPointField type for a key field in my schema, I've changed the type to string and now everything works. Seeing that problem in 7.4.0 definitely sounds like

Re: Need Help on Solr Client connection Pooling

2018-08-30 Thread Shawn Heisey
On 8/30/2018 2:13 AM, Gembali Satish kumar wrote: *SolrClient client = new HttpSolrClient.Builder(* * SolrUtil.getSolrURL(tsConfigUtil.getClusterAdvertisedAddress(), aInCollectionName)).build();* after my job search done, I am closing my client. *client.close();* but from UI getting more

Re: Solr7.4 core node and shard replica numbering question

2018-08-29 Thread Shawn Heisey
On 8/29/2018 6:00 AM, Synet Spambox wrote: The core numbers and the replica numbers are up to 10 (double the amount of servers)? It is confusing for me... Does it matter? Do I miss something ? Something changed somewhere regarding core naming in SolrCloud, and it doesn't behave in the same

Re: SolrCore Initialization Failure Error loading class 'solr.IntField'

2018-08-29 Thread Shawn Heisey
On 8/29/2018 1:27 AM, Salvo Bonanno wrote: [error] corename: org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Could not load conf for core corename: Can't load schema /opt/solr/server/solr/corename/conf/managed-schema: Plugin init failure for [schema.xml] fieldType

Re: Migrate 4 Shards/0 Replica to 1 Shard/1 Replica

2018-08-29 Thread Shawn Heisey
On 8/29/2018 7:17 AM, Pure Host - Wolfgang Freudenberger wrote: I am currently restructuring a big-data cloud with 1000+ collections on a SOLRCloud. The datas are stored on 4 shards without a replica. This data are deprecated and readonly for some purpose, so I want to migrate them to a new

Re: Solr Indexing error

2018-08-28 Thread Shawn Heisey
On 8/28/2018 6:03 AM, kunhu0...@gmail.com wrote: possible analysis error: Document contains at least one immense term in field="content" (whose UTF8 encoding is longer than the max length 32766), It's telling you exactly what is wrong. The field named "content" is probably using a field class

Re: Solr LTS and EOL

2018-08-28 Thread Shawn Heisey
On 8/28/2018 2:59 AM, Dan Untenzu wrote: I would like to get some feedback about LTS & EOL timeframes in Solr. The Solr website states that "6.4.x" is a LTS version and "7.x" is the current mayor version (https://lucene.apache.org/solr/community.html). Question 1: Shouldn't it use "6.x", since

Re: “solr.data.dir” can only config a single directory

2018-08-27 Thread Shawn Heisey
On 8/27/2018 8:29 PM, zhenyuan wei wrote: I found the “solr.data.dir” can only config a single directory. I think it is necessary to be config multi dirs,such as ”solr.data.dir:/mnt/disk1,/mnt/disk2,/mnt/disk3" , due to one disk overload or capacity limitation. Any reason to support

Re: An exception when running Solr on HDFS,why a solr server can not recognize the write.lock file is created by itself before?

2018-08-27 Thread Shawn Heisey
On 8/26/2018 7:47 PM, zhenyuan wei wrote: I found an exception when running Solr on HDFS。The detail is: Running solr on HDFS,and update doc was running always, then,kill -9 solr JVM or reboot linux os/shutdown linux os,then restart all. If you use "kill -9" to stop a Solr instance, the

Re: solr crypto mining hack...

2018-08-26 Thread Shawn Heisey
On 8/25/2018 9:21 PM, Erick Erickson wrote: This is probably CVE-2017-12629, see SOLR-11482, SOLR-11477 for specific versions that have been patched and upgrade. You also need to, as Jan suggested, figure out a way to be absolutely sure that your installation is cleaned before you can be sure

Re: Multiple solr instances per host vs Multiple cores in same solr instance

2018-08-26 Thread Shawn Heisey
On 8/26/2018 12:00 AM, Wei wrote: I have a question about the deployment configuration in solr cloud. When we need to increase the number of shards in solr cloud, there are two options: 1. Run multiple solr instances per host, each with a different port and hosting a single core for one

Re: solr crypto mining hack...

2018-08-25 Thread Shawn Heisey
On 8/25/2018 12:59 PM, humanitarian wrote: I am struggling to fight an attack were the solr user is being used to crate files used for mining cryptocurrencies. The files are being created in the /var/tmp and /tmp folders. It will use 100% of the CPU. I am looking for help in stopping these

Re: Index Upgrader tool

2018-08-24 Thread Shawn Heisey
On 8/24/2018 12:44 AM, dami...@gmail.com wrote: Shawn, Is it possible to run optimize on the live collection? For example, /solr/collection/update?commit=true=true For all the reasons in the blog post that Erick referenced, we recommend that you do not do this. Something to note:  That

Re: How to hit filterCache?if filterQuery is a sub range query of another already cache range filterQuery

2018-08-24 Thread Shawn Heisey
On 8/24/2018 5:23 AM, zhenyuan wei wrote: I am confuse about How to hit filterCache? If filterQuery is range [3 to 100] , but not cache in FilterCache, and filterCache already exists filterQuery range [2 to 100], My question is " Dose this filterQuery range [3 to 100] will fetch DocSet

Re: How to trace one query?the debug/debugQuery info are not enough to find out why a query is slow

2018-08-24 Thread Shawn Heisey
On 8/24/2018 5:17 AM, zhenyuan wei wrote: Thanks again~ @Shawn @Jan Høydahl What is the recommend size of one shard?or how many docs per shard is recommend? My collection has 20 shard,each shard is 30~50GB。 There are no general rules.  Every situation will be different, and advice that works

Re: Not possible to use NOT queries with Solr Export Handler?

2018-08-23 Thread Shawn Heisey
On 8/23/2018 1:12 PM, Antelmo Aguilar wrote: I asked this question in the IRC channel, but had to leave so was not able to wait for a response. So sending it through here instead with the hopes that someone can give me some insight on the issue I am experiencing. So in our Solr setup, we use

Re: Still not seeing Solr listening on 8983 after 30 seconds!

2018-08-23 Thread Shawn Heisey
On 8/23/2018 2:02 PM, Abhijit Pawar wrote: Recently with no change in the configuration or code we are facing a slowdown of approx 3 minutes while restarting the SOLR instance.Earlier it used to come up in few seconds however now it takes so long. *Error Message Displayed:* Waiting up to 30

Re: Permission Denied when trying to connect to Solr running on a different server

2018-08-23 Thread Shawn Heisey
On 8/23/2018 1:36 PM, cyndefromva wrote: But when I try to access search through my web application I'm getting Errno::EACCES Permission denied -- connect(2) for port 8983. This sounds like an error message from your rails app.  You may need to ask whoever created the Solr client that you

Re: How to trace one query?the debug/debugQuery info are not enough to find out why a query is slow

2018-08-23 Thread Shawn Heisey
On 8/23/2018 5:19 AM, zhenyuan wei wrote: Thanks for your detail answer @Shawn Yes I run the query in SolrCloud mode, and my collection has 20 shards, each shard size is 30~50GB。 4 solr server, each solr JVM use 6GB, HDFS datanode are 4 too, each datanode JVM use 2.5GB。 Linux server host are 4

Re: How to trace one query?the debug/debugQuery info are not enough to find out why a query is slow

2018-08-23 Thread Shawn Heisey
On 8/23/2018 4:03 AM, Shawn Heisey wrote: Configuring caches cannot speed up the first time a query runs.  That speeds up later runs.  To speed up the first time will require two things: 1) Ensuring that there is enough memory in the system for the operating system to effectively cache

Re: How to trace one query?the debug/debugQuery info are not enough to find out why a query is slow

2018-08-23 Thread Shawn Heisey
On 8/23/2018 3:41 AM, zhenyuan wei wrote: Thank you very much to answer. @Jan Høydahl My query is simple, just wildcard last 2 char in this query(have more other query to optimize) curl " http://emr-worker-1:8983/solr/collection005/query?q=v10_s:YY*=10&=id=all " I think

Re: Solr unable to start up after setting up SSL in Solr 7.4.0

2018-08-23 Thread Shawn Heisey
On 8/23/2018 2:42 AM, Jan Høydahl wrote: Don't need a git checkout to pull a text file :) https://github.com/apache/lucene-solr/blob/branch_7x/solr/bin/solr.cmd

Re: Solr core corrupted for version 7.4.0, please help!

2018-08-22 Thread Shawn Heisey
On 8/22/2018 7:09 PM, mmb1234 wrote: The original post clarified the current state. "we have about 75 cores with "transientCacheSize" set to 32". If transientCacheSize is increased to match current cores, we'll differ the issue. It's going to hit 100's cores per solr instance shortly. Majority

Re: Solr unable to start up after setting up SSL in Solr 7.4.0

2018-08-22 Thread Shawn Heisey
On 8/22/2018 8:31 PM, Zheng Lin Edwin Yeo wrote: Hi Jan and Shawn, So far I am still getting the error from the workaround and quick fix methods. Not sure if it is good to continue to use the files from Solr 7.3.1 while waiting for the release of Solr 7.5.0? You could check out the source

Re: Solr core corrupted for version 7.4.0, please help!

2018-08-22 Thread Shawn Heisey
On 8/16/2018 7:14 PM, Michael Hu (CMBU) wrote: Environment: * solr 7.4.1 * all cores are vanilla cores with "loadOnStartUp" set to false, and "transient" set to true * we have about 75 cores with "transientCacheSize" set to 32 Issue: we have core corruption from time to time

Re: How to add solr admin ui

2018-08-22 Thread Shawn Heisey
On 8/21/2018 7:11 PM, Ahmed Musallam wrote: I'd like to build a UI plugin for solr, I can see all the ui related assets in `/server/solr-webapp` but is there a way to add UI plugins without modifying the ui assets under `/server/solr-webapp`? by plugin, I mean some way I can add a some form of

Re: Index Upgrader tool

2018-08-21 Thread Shawn Heisey
On 8/21/2018 2:29 AM, Artjoms Laivins wrote: We are running Solr cloud with 3 nodes v. 6.6.2 We started with version 5 so we have some old index that we need safely move over to v. 7 now. New data comes in several times per day. Our questions are: Should we run IndexUpgrader tool on one slave

Re: SOLR Issue

2018-08-21 Thread Shawn Heisey
On 8/21/2018 8:05 AM, Sandesh Ingawale wrote: We have ecommerce base web application and we used SOLR for search. Now we want to implement SOLR authentication and SOLR ranking feature with our application. We need to do this programmatically(C#), can you please suggest what should be

Re: 7.3.1: Query of death - all nodes ran out of memory and had to be shut down

2018-08-21 Thread Shawn Heisey
On 8/20/2018 9:55 PM, Ash Ramesh wrote: We ran a bunch of deep paginated queries (offset of 1,000,000) with a filter query. We set the timeout to 5 seconds and it did timeout. We aren't sure if this is what caused the irrecoverable failure, but by reading this -

Re: Dataimport not working on solrcloud

2018-08-21 Thread Shawn Heisey
On 8/20/2018 10:00 PM, Sushant Vengurlekar wrote: I have a dataimport working on standalone solr instance but the same doesn't work on solrcloud. I keep on hitting this error Full Import failed:java.lang.RuntimeException: java.lang.RuntimeException:

Re: SOLRJ 7.x library fails ejb deployment with weblogic

2018-08-21 Thread Shawn Heisey
On 8/21/2018 7:29 AM, Ganesh Kumar J wrote: Thank you so much for your reply. Unfortunately our production weblogic cluster runs in java 1.7 we are unable to upgrade to java 1.8 In this case do you have any idea how we can use solrj version below 7.x which can authenticate with kerberozied

Re: Solr unable to start up after setting up SSL in Solr 7.4.0

2018-08-21 Thread Shawn Heisey
On 8/20/2018 9:57 PM, Zheng Lin Edwin Yeo wrote: This is the error that I get: ERROR StatusLogger Unable to access file:/C:/Users/edwin/Desktop/edwin/solr- 7.4.0/server/scripts/cloud-scripts/file:C:/Users/edwin/Desktop/edwin/solr-7. 4.0/server/scripts/cloud-scripts/log4j2.xml

Re: Error with plugin in Solrcloud

2018-08-19 Thread Shawn Heisey
On 8/18/2018 11:03 PM, Zahra Aminolroaya wrote: Thanks Shawn. For Solr mode I should include all dependencies in lib, so I thought for Solr Cloud mode I should include the dependencies too. Even for standalone mode, you don't want those dependency jars in the lib folder.  The same problems

<    3   4   5   6   7   8   9   10   11   12   >