Re: Using of Streaming to join between shards

Hi Joel, Thanks for confirming that Streaming would be too costly for high qps loads. Regards, -- View this message in context: http://lucene.472066.n3.nabble.com/Using-of-Streaming-to-join-between-shards-tp4342563p4343104.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Tlogs not being deleted/truncated

We also have the same collections in our development and QA environments. In our Dev environment which is not using CDCR replication, but does have autoCommit set, we have 440 tlog files. The only difference in the configuration is that dev doesn't have the cdcr request handler configured. It does

Re: SolrJ 6.6.0 Connection pool shutdown

On 6/27/2017 6:50 AM, Markus Jelsma wrote: > We have a proces checking presence of many documents in a collection, just a > simple client.getById(id). It sometimes begins throwing lots of these > exceptions in a row: > > org.apache.solr.client.solrj.SolrServerException: >

Re: Tlogs not being deleted/truncated

It appears right how that we are not seeing an issue with the target collections, we definitely see a problem with the source collection. numRecordsToKeep and maxNumLogsToKeep are set to the default values of 100 and 10 respectively. We probably don't need 10 tlog files around. On Tue, Jun 27,

Re: Dynamic fields vs parent child

Do you have any close count of how many max dynamic fields you may have (1k, 2k or 3k etc.). In one of our index we have a total around 2K dynamic fields across all documents. My suggestion would be to try out dynamic fields for the use case you are describing and do some real performance test.

Sharding of index data takes long time.

I am just trying to shard my index data of size 22GB(1.7M documents) into three shards. The total time for splitting takes about 7 hours. In used the same query that is mentioned in solr collections API. Is there anyway to do that quicker. Can i use REBALANCE API . is that secured?? Is there

Re: Dynamic fields vs parent child

We have key-value pairs that need to be searchable. We are looking for best approach, both in terms of indexing (fast as well as space efficient) as well as retrieval (fast search). Right now, the two approaches that we have are: Nested docs or dynamic fields (myField_*_time:some date) The

Re: Solr PDF parsing failing with java error

Take a look at the solr logs, they'll give you a more explicit message. My guess: Someone went into the Solr admin UI, clicked "core admin" and then said "I wonder what this 'new core' button does?". The default name is, you guessed it, "new_core". And if you don't have the underlying directories

Re: Solr PDF parsing failing with java error

Erick Erickson wrote > Sure, someone changed the system variable "solr.install.dir" (i.e. > -Dsolr.install.dir=some other place). Or removed it. Or changed the > startup script. Or > > I've gotten very skeptical of "we didn't change anything but suddenly > it stopped working". Usually it's

Re: Index 0, Size 0 - hashJoin Stream function Error

Ok, I'll take a look. Thanks! Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Jun 27, 2017 at 10:01 AM, Susheel Kumar wrote: > Hi Joel, > > I have submitted a patch to handle this. Please review. > >

Re: solr /export handler - behavior during close()

Ok, I'll fix the ParallelStream to set the stream context though when creating the SolrStreams. Thanks for pointing this out. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Jun 27, 2017 at 1:46 PM, Susmit Shukla wrote: > Hi Joel, > > I was on solr 6.3 branch. I

Re: Using of Streaming to join between shards

I don't think the distributed joins are going to work for you in the ACL use case you describe. I think the overhead of streaming the documents will be too costly in this scenario. The distributed joins were designed more for OLAP data warehousing use cases rather then high QPS loads. Joel

Re: solr /export handler - behavior during close()

Hi Joel, I was on solr 6.3 branch. I see HttpClient deprecated methods are all fixed in master. I had forgot to mention that I used a custom SolrClientCache to have higher limits for maxConnectionPerHost settings thats why I saw difference in behavior. SolrClientCache also looks configurable with

Re: solr /export handler - behavior during close()

Ok, I see where it's not set the stream context. This needs to be fixed. I'm curious about where you're seeing deprecated methods in the HttpClientUtil? I was reviewing the master version of HttpClientUtil and didn't see any deprecations in my IDE. I'm wondering if you're using an older version

RE: Master/Slave out of sync

I figured the attachments would get stripped, but it was worth a shot! It was just a screenshot showing the version numbers off from each other. Here are the Master/Slave commit settings: 18 false 6 -Original Message- From: Erick Erickson

Re: Tlogs not being deleted/truncated

Commits were definitely not happening. We ran out of filesystem space. The admins deleted old tlogs and restartd. The collection in question was missing a lot of data. We reloaded it, and then we saw some commits. In Solrcloud they look like this: 2017-06-23 17:28:06.441 INFO

Solr 6.6 SSL Question

How does Solr find the correct certificate to use for handling inbound requests? Documentation shows the solr.in.sh settings but I only see the keystore locations and the passwords, there is no reference to the alias being used if the keystore contains more certificates ... However the

Re: Master/Slave out of sync

First, attachments are almost always stripped by the mail program, so we can't see anything. Hmmm, does look odd. What happens if you issue a commit against the slave via a url? I.e. http://server:port/solr/core/update?commit=true? And what are the autocommit settings on the slave? Best, Erick

Master/Slave out of sync

Hey guys... Does anyone else have a problem with the master/slave setup getting out of sync and staying that way until I either optimize the core or restart SOLR? It seems to be happening more and more frequently these days and I'm looking for a solution here. Running SOLR 6.2 on these

Re: Index 0, Size 0 - hashJoin Stream function Error

Hi Joel, I have submitted a patch to handle this. Please review. https://issues.apache.org/jira/secure/attachment/12874681/SOLR-10944.patch Thanks, Susheel On Fri, Jun 23, 2017 at 12:32 PM, Susheel Kumar wrote: > Thanks for confirming. Here is the JIRA > >

SolrJ 6.6.0 Connection pool shutdown

Hi, We have a proces checking presence of many documents in a collection, just a simple client.getById(id). It sometimes begins throwing lots of these exceptions in a row: org.apache.solr.client.solrj.SolrServerException: java.lang.IllegalStateException: Connection pool shut down Then, as

Re: Dynamic fields vs parent child

Can you describe your use case in terms of what business functionality you are looking to achieve. Thanks, Susheel On Mon, Jun 26, 2017 at 4:26 PM, Saurabh Sethi wrote: > Number of dynamic fields will be in thousands (millions of users + > thousands of events shared

Re: SOLR Suggester returns either the full field value or single terms only

Hi Alessandro, Thanks. I've experimented a bit more and here is what I have discovered - If my query is enclosed with quotes, then i get multi terms, if it is not enclosed in quotes, i only get single terms, Example: will only return single terms:

Re: async backup

yes. Requeststatus is returning state=completed prematurely. On Tuesday, 27 June 2017, Amrit Sarkar wrote: > Damien, > > then I poll with REQUESTSTATUS > > > REQUESTSTATUS is an API which provided you the status of the any API > (including other heavy duty apis like

Re: async backup

Damien, then I poll with REQUESTSTATUS REQUESTSTATUS is an API which provided you the status of the any API (including other heavy duty apis like SPLITSHARD or CREATECOLLECTION) associated with async_id at that current timestamp / moment. Does that give you "state"="completed"? Amrit Sarkar

Re: Boosting Documents using the field Value

Hi Erick, Finally Made it work. bf=if(exists(query($qqone)),one_score,0)=one_query:\"google cloud\" Thanks a lot for guiding, also reminding its not url escape. No analyzers used. Regards, Govind On Tue, Jun 27, 2017 at 11:01 AM, govind nitk wrote: > Hi Erick, > I

Re: Using of Streaming to join between shards

Hi Susheel, Thanks for your reply and as you suggested we will start with innerJoin. But what I want know is that, Is Streaming can be used instead of normal default Join ? For ex. currently we fire request for every user clicks on menu in the page to show list of his documents with default

Re: SOLR Suggester returns either the full field value or single terms only

Hi Angel, can you give me an example of query, a couple of documents of example, and the suggestions you get ( which you don't expect) ? The config seems fine ( I remember there were some tricky problems with the default separator, but a space should be fine there). Cheers -

Solr 5.5 - spatial intersects query returns results outside of search box

Hi all, I’m running on Solr 5.5 and have run into an issue where the Solr spatial search is returning results outside of the search rectangle parameters, and I’m not quite sure what is causing this to happen. Thank you in advance for any troubleshooting tips you can pass along. The spatial