Re: Getting to grips with auto-scaling

from when you add nodes in each tier). > > Best regards, > Radu > > https://sematext.com > > vin., 5 iun. 2020, 21:59 Tom Evans a > scris: > > > Hi > > > > I'm trying to get a handle on the newer auto-scaling features in Solr. > > We're in

Indexing error when using Category Routed Alias

Hi all 1. Setup simple 1 node solrcloud test setup using docker-compose, solr:8.5.2, zookeeper:3.5.8. 2. Upload a configset 3. Create two collections, one standard collection, one CRA, both using the same configset legacy: action=CREATE=products_old=products=true=1=-1 CRA: { "create-alias":

Getting to grips with auto-scaling

Hi I'm trying to get a handle on the newer auto-scaling features in Solr. We're in the process of upgrading an older SolrCloud cluster from 5.5 to 8.5, and re-architecture it slightly to improve performance and automate operations. If I boil it down slightly, currently we have two collections,

Re: Provide suggestion on indexing performance

On Tue, Sep 12, 2017 at 4:06 AM, Aman Tandon wrote: > Hi, > > We want to know about the indexing performance in the below mentioned > scenarios, consider the total number of 10 string fields and total number > of documents are 10 million. > > 1) indexed=true, stored=true

Re: Solr returning same object in different page

On Tue, Sep 12, 2017 at 7:42 PM, ruby wrote: > I'm running into a issue where an object is appearing twice when we are > paging. My query is gives documents boost based on field values. First query > returns 50 object. Second query is exactly same as first query, except >

Re: Get results in multiple orders (multiple boosts)

On Fri, Aug 18, 2017 at 8:21 AM, Luca Dall'Osto wrote: > > Yes, of course, and excuse me for the misunderstanding. > > > In my scenario I have to display a list with hundreds of documents. > An user can show this documents in a particular order, this order is

Re: setup solrcloud from scratch vie web-ui

On Wed, May 17, 2017 at 6:28 AM, Thomas Porschberg wrote: > Hi, > > I did not manipulating the data dir. What I did was: > > 1. Downloaded solr-6.5.1.zip > 2. ensured no solr process is running > 3. unzipped solr-6.5.1.zip to ~/solr_new2/solr-6.5.1 > 3. started an external

Re: to handle expired documents: collection alias or delete by id query

On Thu, Mar 23, 2017 at 6:10 AM, Derek Poh wrote: > Hi > > I have collections of products. I am doing indexing 3-4 times daily. > Every day there are products that expired and I need to remove them from > these collectionsdaily. > > Ican think of 2 ways to do this. > 1.

Re: Simulating group.facet for JSON facets, high mem usage w/ sorting on aggregation...

Hi Mike Looks like you are trying to get a list of the distinct item ids in a result set, ordered by the most frequent item ids? Can you use collapsing qparser for this instead? Should be much quicker. https://cwiki.apache.org/confluence/display/solr/Collapse+and+Expand+Results Every document

Re: Interval Facets with JSON

On Wed, Feb 8, 2017 at 11:26 PM, deniz <denizdurmu...@gmail.com> wrote: > Tom Evans-2 wrote >> I don't think there is such a thing as an interval JSON facet. >> Whereabouts in the documentation are you seeing an "interval" as JSON >> facet type? >> >

Re: Interval Facets with JSON

On Tue, Feb 7, 2017 at 8:54 AM, deniz wrote: > Hello, > > I am trying to run JSON facets with on interval query as follows: > > > "json.facet":{"height_facet":{"interval":{"field":"height","set":["[160,180]","[180,190]"]}}} > > And related field is stored="true" /> > >

Re: Upgrade SOLR version - facets perfomance regression

On Tue, Jan 31, 2017 at 5:49 AM, SOLR4189 wrote: > But I can't run Json Facet API. I checked on SOLR-5.4.1. > If I write: > localhost:9001/solr/Test1_shard1_replica1/myHandler/q=*:*=5=*=json=true=someField > It works fine. But if I write: >

Re: Concat Fields in JSON Facet

On Mon, Jan 16, 2017 at 2:58 PM, Zheng Lin Edwin Yeo wrote: > Hi, > > I have been using JSON Facet, but I am facing some constraints in > displaying the field. > > For example, I have 2 fields, itemId and itemName. However, when I do the > JSON Facet, I can only get it to

Re: Has anyone used linode.com to run Solr | ??Best way to deliver PHP/Apache clients with Solr question

On Thu, Dec 15, 2016 at 12:37 PM, GW wrote: > While my client is all PHP it does not use a solr client. I wanted to stay > with he latest Solt Cloud and the PHP clients all seemed to have some kind > of issue being unaware of newer Solr Cloud versions. The client makes pure

Re: Using DIH FileListEntityProcessor with SolrCloud

On Fri, Dec 2, 2016 at 4:36 PM, Chris Rogers wrote: > Hi all, > > A question regarding using the DIH FileListEntityProcessor with SolrCloud > (solr 6.3.0, zookeeper 3.4.8). > > I get that the config in SolrCloud lives on the Zookeeper node (a different > server

Re: insert lat/lon from jpeg into solr

On Wed, Nov 30, 2016 at 1:36 PM, win harrington wrote: > I have jpeg files with latitude and longitudein separate fields. When I run > the post tool,it stores the lat/lon in separate fields. > For geospatial search, Solr wants themcombined into one field with

Re: Import from S3

On Fri, Nov 25, 2016 at 7:23 AM, Aniket Khare wrote: > You can use Solr DIH for indexing csv data into solr. > https://wiki.apache.org/solr/DataImportHandler > Seems overkill when you can simply post CSV data to the UpdateHandler, using either the post tool:

Re: Query formulation help

On Wed, Oct 26, 2016 at 4:00 PM, Prasanna S. Dhakephalkar wrote: > Hi, > > Thanks for reply, I did > > "q": "cost:[2 TO (2+5000)]" > > Got > > "error": { > "msg": "org.apache.solr.search.SyntaxError: Cannot parse 'cost:[2 to > (2+5000)]': Encountered

Re: Query formulation help

On Wed, Oct 26, 2016 at 8:03 AM, Prasanna S. Dhakephalkar wrote: > Hi, > > > > May be very rudimentary question > > > > There is a integer field in a core : "cost" > > Need to build a query that will return documents where 0 < > "cost"-given_number < 500 >

Re: OOM Error

On Wed, Oct 26, 2016 at 4:53 AM, Shawn Heisey wrote: > On 10/25/2016 8:03 PM, Susheel Kumar wrote: >> Agree, Pushkar. I had docValues for sorting / faceting fields from >> begining (since I setup Solr 6.0). So good on that side. I am going to >> analyze the queries to find

Re: indexing - offline

On Thu, Oct 20, 2016 at 5:38 PM, Rallavagu wrote: > Solr 5.4.1 cloud with embedded jetty > > Looking for some ideas around offline indexing where an independent node > will be indexed offline (not in the cloud) and added to the cloud to become > leader so other cloud nodes

min()/max() on date fields using JSON facets

Hi all I'm trying to replace a use of the stats module with JSON facets in order to calculate the min/max date range of documents in a query. For the same search, "stats.field=date_published" returns this: {u'date_published': {u'count': 86760, u'max':

Re: Node not recovering, leader elections not occuring

) This is with the "leader that is not the leader" shut down. Issuing a FORCELEADER via collections API doesn't in fact force a leader election to occur. Is there any other way to prompt Solr to have an election? Cheers Tom On Tue, Jul 19, 2016 at 5:10 PM, Tom Evans <tevans...@googlema

Re: Node not recovering, leader elections not occuring

s you > bring Solr up saying something like > "OutOfMemory error unable to create native thread" > > SOLR-7280 has patches for 6x and 7x, with a 5x one > being added momentarily. > > Best, > Erick > > On Tue, Jul 19, 2016 at 7:41 AM, Tom Evans <tevans...@googlemai

Node not recovering, leader elections not occuring

Hi all - problem with a SolrCloud 5.5.0, we have a node that has most of the collections on it marked as "Recovering" or "Recovery Failed". It attempts to recover from the leader, but the leader responds with: Error while trying to recover.

Strange highlighting on search

Hi all I'm investigating a bug where by every term in the highlighted field gets marked for highlighting instead of just the words that match the fulltext portion of the query. This is on Solr 5.5.0, but I didn't see any bug fixes related to highlighting in 5.5.1 or 6.0 release notes. The query

Re: result grouping in sharded index

Do you have to group, or can you collapse instead? https://cwiki.apache.org/confluence/display/solr/Collapse+and+Expand+Results Cheers Tom On Tue, Jun 14, 2016 at 4:57 PM, Jay Potharaju wrote: > Any suggestions on how to handle result grouping in sharded index? > > > On

Re: Import html data in mysql and map schemas using onlySolrCELL+TIKA+DIH [scottchu]

On Tue, May 24, 2016 at 3:06 PM, Scott Chu wrote: > p.s. There're really many many extensive, worthy stuffs in Solr. If the > project team can provide some "dictionary" of them, It would be a "Santa > Claus" > for we solr users. Ha! Just a X'mas wish! Sigh! I know it's

Re: SolrCloud increase replication factor

On Mon, May 23, 2016 at 10:37 AM, Hendrik Haddorp wrote: > Hi, > > I have a SolrCloud 6.0 setup and created my collection with a > replication factor of 1. Now I want to increase the replication factor > but would like the replicas for the same shard to be on different

Re: Creating a collection with 1 shard gives a weird range

On Tue, May 17, 2016 at 9:40 AM, John Smith wrote: > I'm trying to create a collection starting with only one shard > (numShards=1) using a compositeID router. The purpose is to start small > and begin splitting shards when the index grows larger. The shard > created gets

Re: Indexing 700 docs per second

On Tue, Apr 19, 2016 at 10:25 AM, Mark Robinson wrote: > Hi, > > I have a requirement to index (mainly updation) 700 docs per second. > Suppose I have a 128GB RAM, 32 CPU machine, with each doc size around 260 > byes (6 fields out of which only 2 will undergo updation at

Re: Verifying - SOLR Cloud replaces load balancer?

On Mon, Apr 18, 2016 at 3:52 PM, John Bickerstaff wrote: > Thanks all - very helpful. > > @Shawn - your reply implies that even if I'm hitting the URL for a single > endpoint via HTTP - the "balancing" will still occur across the Solr Cloud > (I understand the caveat

Re: Anticipated Solr 5.5.1 release date

Awesome, thanks :) On Fri, Apr 15, 2016 at 4:19 PM, Anshum Gupta <ans...@anshumgupta.net> wrote: > Hi Tom, > > I plan on getting a release candidate out for vote by Monday. If all goes > well, it'd be about a week from then for the official release. > > On Fri, Apr 15, 20

Anticipated Solr 5.5.1 release date

Hi all We're currently using Solr 5.5.0 and converting our regular old style facets into JSON facets, and are running in to SOLR-8155 and SOLR-8835. I can see these have already been back-ported to 5.5.x branch, does anyone know when 5.5.1 may be released? We don't particularly want to move to

SolrCloud no leader for collection

Hi all, I have an 8 node SolrCloud 5.5 cluster with 11 collections, most of them in a 1 shard x 8 replicas configuration. We have 5 ZK nodes. During the night, we attempted to reindex one of the larger collections. We reindex by pushing json docs to the update handler from a number of processes.

Re: Creating new cluster with existing config in zookeeper

On Wed, Mar 23, 2016 at 3:43 PM, Robert Brown wrote: > So I setup a new solr server to point to my existing ZK configs. > > When going to the admin UI on this new server I can see the shards/replica's > of the existing collection, and can even query it, even tho this new

Re: Re: Paging and cursorMark

On Wed, Mar 23, 2016 at 12:21 PM, Vanlerberghe, Luc wrote: > I worked on something similar a couple of years ago, but didn’t continue work > on it in the end. > > I've included the text of my original mail. > If you're interested, I could try to find the sources I

Paging and cursorMark

Hi all With Solr 5.5.0, we're trying to improve our paging performance. When we are delivering results using infinite scrolling, cursorMark is perfectly fine - one page is followed by the next. However, we also offer traditional paging of results, and this is where it gets a little tricky. Say

Re: Ping handler in SolrCloud mode

On Wed, Mar 16, 2016 at 4:10 PM, Shawn Heisey <apa...@elyograg.org> wrote: > On 3/16/2016 8:14 AM, Tom Evans wrote: >> The problem occurs when we attempt to query a node to see if products >> or items is active on that node. The balancer (haproxy) requests the >> ping

Re: Ping handler in SolrCloud mode

On Wed, Mar 16, 2016 at 2:14 PM, Tom Evans <tevans...@googlemail.com> wrote: > Hi all > > [ .. ] > > The option I'm trying now is to make two ping handler for skus that > join to one of items/products, which should fail on the servers which > do not sup

Ping handler in SolrCloud mode

Hi all I have a cloud setup with 8 nodes and 3 collections, products, items and skus. All collections have just one shard, products has 6 replicas, items has 2 replicas, skus has 8 replicas. No node has both products and items, all nodes have skus Some of our queries join from sku to either

mergeFactor/maxMergeDocs is deprecated

Hi all Updating to Solr 5.5.0, and getting these messages in our error log: Beginning with Solr 5.5, is deprecated, configure it on the relevant instead. Beginning with Solr 5.5, is deprecated, configure it on the relevant instead. However, mergeFactor is only mentioned in a commented out

Re: Separating cores from Solr home

Hmm, I've worked around this by setting the directory where the indexes should live to be the actual solr home, and symlink the files from the current release in to that directory, but it feels icky. Any better ideas? Cheers Tom On Thu, Mar 3, 2016 at 11:12 AM, Tom Evans <tev

Separating cores from Solr home

Hi all I'm struggling to configure solr cloud to put the index files and core.properties in the correct places in SolrCloud 5.5. Let me explain what I am trying to achieve: * solr is installed in /opt/solr * the user who runs solr only has read only access to that tree * the solr home files -

Re: docValues error

On Mon, Feb 29, 2016 at 11:43 AM, David Santamauro wrote: > You will have noticed below, the field definition does not contain > multiValues=true What version of the schema are you using? In pre 1.1 schemas, multiValued="true" is the default if it is omitted. Cheers

Re: Json faceting, aggregate numeric field by day?

On Wed, Feb 10, 2016 at 12:13 PM, Markus Jelsma wrote: > Hi Tom - thanks. But judging from the article and SOLR-6348 faceting stats > over ranges is not yet supported. More specifically, SOLR-6352 is what we > would need. > > [1]:

Re: Json faceting, aggregate numeric field by day?

On Wed, Feb 10, 2016 at 10:21 AM, Markus Jelsma wrote: > Hi - if we assume the following simple documents: > > > 2015-01-01T00:00:00Z > 2 > > > 2015-01-01T00:00:00Z > 4 > > > 2015-01-02T00:00:00Z > 3 > > > 2015-01-02T00:00:00Z > 7 > > > Can i

fq in SolrCloud

I have a small question about fq in cloud mode that I couldn't find an explanation for in confluence. If I specify a query with an fq, where is that cached, is it just on the nodes/replicas that process that specific query, or will it exist on all replicas? We have a sub type of queries that

Re: Shard allocation across nodes

ur own snitch: >> https://cwiki.apache.org/confluence/display/solr/Rule-based+Replica+Placement >> >> Or, it would be more annoying, but you can always add/remove replicas >> manually and juggle things yourself after you create the initial collection. >> >> >&

Shard allocation across nodes

Hi all We're setting up a solr cloud cluster, and unfortunately some of our VMs may be physically located on the same VM host. Is there a way of ensuring that all copies of a shard are not located on the same physical server? If they do end up in that state, is there a way of rebalancing them?

SolrCloud, DIH, and XPathEntityProcessor

Hi all, trying to move our Solr 4 setup to SolrCloud (5.4). Having some problems with a DIH config that attempts to load an XML file and iterate through the nodes in that file, it trys to load the file from disk instead of from zookeeper. The file exists in zookeeper, adjacent to the

Re: SolrCloud, DIH, and XPathEntityProcessor

On Tue, Jan 12, 2016 at 3:00 PM, Shawn Heisey <apa...@elyograg.org> wrote: > On 1/12/2016 7:45 AM, Tom Evans wrote: >> That makes no sense whatsoever. DIH loads the data_import.conf from ZK >> just fine, or is that provided to DIH from another module that

Re: SolrCloud, DIH, and XPathEntityProcessor

On Tue, Jan 12, 2016 at 2:32 PM, Shawn Heisey <apa...@elyograg.org> wrote: > On 1/12/2016 6:05 AM, Tom Evans wrote: >> Hi all, trying to move our Solr 4 setup to SolrCloud (5.4). Having >> some problems with a DIH config that attempts to load an XML file and >>

Re: Defining SOLR nested fields

On Sun, Dec 13, 2015 at 6:40 PM, santosh sidnal wrote: > Hi All, > > I want to define nested fileds in SOLR using schema.xml. we are using Apache > Solr 4.7.0. > > i see some links which says how to do, but not sure how can i do it in > schema.xml >

Moving to SolrCloud, specifying dataDir correctly

Hi all We're currently in the process of migrating our distributed search running on 5.0 to SolrCloud running on 5.4, and setting up a test cluster for performance testing etc. We have several cores/collections, and in each core's solrconfig.xml, we were specifying an empty , and specifying the

Re: Moving to SolrCloud, specifying dataDir correctly

On Mon, Dec 14, 2015 at 1:22 PM, Shawn Heisey <apa...@elyograg.org> wrote: > On 12/14/2015 10:49 AM, Tom Evans wrote: >> When I tried this in SolrCloud mode, specifying >> "-Dsolr.data.dir=/mnt/solr/" when starting each node, it worked fine >> for the

Re: Best way to track cumulative GC pauses in Solr

On Fri, Nov 13, 2015 at 4:50 PM, Walter Underwood wrote: > Also, what GC settings are you using? We may be able to make some suggestions. > > Cumulative GC pauses aren’t very interesting to me. I’m more interested in > the longest ones, 90th percentile, 95th, etc. > Any

Best way to track cumulative GC pauses in Solr

Hi all We have some issues with our Solr servers spending too much time paused doing GC. From turning on gc debug, and extracting numbers from the GC log, we're getting an idea of just how much of a problem. I'm currently doing this in a hacky, inefficient way: grep -h 'Total time for which

Re: org.apache.solr.common.SolrException: Document is missing mandatory uniqueKey field: id

On Mon, Nov 2, 2015 at 1:38 PM, fabigol wrote: > Thank > All works. > I have 2 last questions: > How can i put 0 by defaults " clean" during a indexation? > > To conclure, i wand to understand: > > > Requests: 7 (1/s), Fetched: 452447 (45245/s), Skipped: 0, Processed:

Re: Checking of Solr Memory and Disk usage

On Fri, Apr 24, 2015 at 8:31 AM, Zheng Lin Edwin Yeo edwinye...@gmail.com wrote: Hi, So has anyone knows what is the issue with the Heap Memory Usage reading showing the value -1. Should I open an issue in Jira? I have solr 4.8.1 and solr 5.0.0 servers, on the solr 4.8.1 servers the core

Re: Confusing SOLR 5 memory usage

requests. Again, sorry! Tom On Tue, Apr 21, 2015 at 11:54 AM, Tom Evans tevans...@googlemail.com wrote: We monitor them with munin, so I have charts if attachments are acceptable? Having said that, they have only been running for a day with this memory allocation.. Describing them, the master

Re: Confusing SOLR 5 memory usage

We monitor them with munin, so I have charts if attachments are acceptable? Having said that, they have only been running for a day with this memory allocation.. Describing them, the master consistently has 8GB used for apps, the 8GB used in cache, whilst the slave consistently only uses ~1.5GB

Confusing SOLR 5 memory usage

Hi all I have two SOLR 5 servers, one is the master and one is the slave. They both have 12 cores, fully replicated and giving identical results when querying them. The only difference between configuration on the two servers is that one is set to slave from the other - identical core configs and

Re: Setting up SOLR 5 from an RPM

On Wed, Mar 25, 2015 at 2:40 PM, Shawn Heisey apa...@elyograg.org wrote: I think you will only need to change the ownership of the solr home and the location where the .war file is extracted, which by default is server/solr-webapp. The user must be able to *read* the program data, but should

Re: Setting up SOLR 5 from an RPM

On Tue, Mar 24, 2015 at 4:00 PM, Tom Evans tevans...@googlemail.com wrote: Hi all We're migrating to SOLR 5 (from 4.8), and our infrastructure guys would prefer we installed SOLR from an RPM rather than extracting the tarball where we need it. They are creating the RPM file themselves

Setting up SOLR 5 from an RPM

Hi all We're migrating to SOLR 5 (from 4.8), and our infrastructure guys would prefer we installed SOLR from an RPM rather than extracting the tarball where we need it. They are creating the RPM file themselves, and it installs an init.d script and the equivalent of the tarball to /opt/solr.

Determining which field caused a document to not be imported

Hi all I recently rewrote our SOLR 4.8 dataimport to read from a set of denormalised DB tables, in an attempt to increase full indexing speed. When I tried it out however, indexing broke telling me that java.lang.Long cannot be cast to java.lang.Integer (full stack below, with the document

Re: Determining which field caused a document to not be imported

On Fri, Oct 3, 2014 at 2:24 PM, Shawn Heisey apa...@elyograg.org wrote: Can you give us the entire stacktrace, with complete details from any caused by sections? Also, is this 4.8.0 or 4.8.1? Thanks Shawn, this is SOLR 4.8.1 and here is the full traceback from the log: 95191 [Thread-21] INFO

Re: Determining which field caused a document to not be imported

On Fri, Oct 3, 2014 at 3:13 PM, Tom Evans tevans...@googlemail.com wrote: I tried converting the selected data to SIGNED INTEGER, eg CONVERT(country_id, SIGNED INTEGER) AS country_id, but this did not have the desired effect. However, changing them to be cast to CHAR changed the error message

Re: Determining which field caused a document to not be imported

On Fri, Oct 3, 2014 at 3:24 PM, Tom Evans tevans...@googlemail.com wrote: On Fri, Oct 3, 2014 at 3:13 PM, Tom Evans tevans...@googlemail.com wrote: I tried converting the selected data to SIGNED INTEGER, eg CONVERT(country_id, SIGNED INTEGER) AS country_id, but this did not have the desired