Re: Using Solr as a Database?

2019-06-03 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Daniel, On 6/3/19 16:26, Davis, Daniel (NIH/NLM) [C] wrote: > I think the sweet spot of Cassandra and Solr should be mentioned > in this discussion. Cassandra is more scalable/clusterable than > an RDBMS, without losing all of the structure that

Re: Using Solr as a Database?

2019-06-03 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Ralph, On 6/2/19 16:32, Ralph Soika wrote: > The whole system is highly transactional as it runs on Java EE with > JPA and Session EJBs. And you write-through from your application -> RDBMS -> Lucene/Solr? How are you handling commits (both soft

Re: Solr Heap Usage

2019-06-03 Thread Shawn Heisey
On 6/2/2019 4:35 PM, John Davis wrote: If we assume there is no query load then effectively this boils down to most effective way for adding a large number of documents to the solr index. I've looked through SolrJ, DIH and others -- is the bottomline across all of them to "batch updates" and not

Re: where to see deleted document in Solr log

2019-06-03 Thread Shawn Heisey
On 6/3/2019 2:51 PM, Wendy2 wrote: Hi, I am using Solr 7.3.1 to index data via DIH. Solr admin panel indicated that 152160 documents got indexed, while 3944 documents were deleted. But DIH indicated that added/update: 662059 documents. Deleted 0 documents. I try to find the deleted documents,

where to see deleted document in Solr log

2019-06-03 Thread Wendy2
Hi, I am using Solr 7.3.1 to index data via DIH. Solr admin panel indicated that 152160 documents got indexed, while 3944 documents were deleted. But DIH indicated that added/update: 662059 documents. Deleted 0 documents. I try to find the deleted documents, but I don't see anywhere in the solr

Solr Learning to Rank - Trailing slash in path behavior differences, expected?

2019-06-03 Thread Doug Turnbull
Hello everyone, I encountered some surprising behavior that got be stuck on Solr LTR for a good hour. I wanted to share it, and you can decide if its a bug (I suspect it's a bug) I wanted to list all the feature stores on my Solr 7.7.1 instance. So I visited GET

RE: Using Solr as a Database?

2019-06-03 Thread Davis, Daniel (NIH/NLM) [C]
I think the sweet spot of Cassandra and Solr should be mentioned in this discussion. Cassandra is more scalable/clusterable than an RDBMS, without losing all of the structure that is desirable in an RDBMS. In contrast, if you use a full document store such as MongoDB, you lose some of the

Re: Using Solr as a Database?

2019-06-03 Thread Shawn Heisey
On 6/2/2019 7:28 AM, Ralph Soika wrote: This is not intended to contradict the other replies you've gotten, only supplement them. Now as far as I understand is solr a cluster enabled datastore which can be used to store also all the data form our document. The problem with relational

Re: alias read access impossible for anyone other than admin?

2019-06-03 Thread Sotiris Fragkiskos
it's 7.2.1. Thanks! On Mon, Jun 3, 2019 at 6:26 PM Jason Gerlowski wrote: > Hi Sotiris, > > What version of Solr are you running? The behavior has changed some > over time, both intentionally and due to bugs that have come and gone > over time. I (or someone else) can explain things and offer

Re: Solr 8.1.1 / Zookeeper 3.5.5 problem

2019-06-03 Thread Erick Erickson
See: https://issues.apache.org/jira/browse/SOLR-8346. We haven’t released anything yet that even purports to run against ZK 3.5.5 since it’s so new, but 8.2 should have the upgrade. Meanwhile, if you were ambitious you could apply the patch at the JIRA above and try it, or wait a few days and

Solr 8.1.1 / Zookeeper 3.5.5 problem

2019-06-03 Thread Paul Isaac's
Hi, when attempting to run the example to create a collection the client appears to timeout but has partially created the collection. [solr@vm-ckan-head ~]$ /opt/solr/bin/solr start -e cloud Welcome to the SolrCloud example! This interactive session will help you launch a SolrCloud cluster on

Re: SolrJ, CloudSolrClient and basic authentication

2019-06-03 Thread Kevin Risden
Chris - not sure if what you are seeing is related to basic auth credentials not being sent until a 401. There was report of this behavior with Apache Knox in front of Solr. https://issues.apache.org/jira/browse/KNOX-1066 The jira above has an example of how to preemptively send basic auth

Re: alias read access impossible for anyone other than admin?

2019-06-03 Thread Jason Gerlowski
Hi Sotiris, What version of Solr are you running? The behavior has changed some over time, both intentionally and due to bugs that have come and gone over time. I (or someone else) can explain things and offer you better help once we know your Solr version. Jason On Mon, Jun 3, 2019 at 12:13

Re: alias read access impossible for anyone other than admin?

2019-06-03 Thread Sotiris Fragkiskos
Hi again, I moved the "all" permission to the bottom as suggested, but it still doesn't work. Actually, i tried all possible combinations that I could think of, but I just can't get it to work. Could there be something else that I'm doing wrong? I'm a complete newbie, so pretty much anything is a

Re: Adding Multiple JSON Documents

2019-06-03 Thread Alexandre Rafalovitch
Hi John, This may be useful: https://www.slideshare.net/arafalov/json-in-solr-from-top-to-bottom (there is the video of the session at the end too). Basically, we have two ways to process JSON and sometimes they look very similar and you have to be very deliberate in indicating which one is the

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

2019-06-03 Thread Colvin Cowie
Hi, thanks I'll give that a go when I get a chance. I was trying to reply to an older thread ( http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201904.mbox/%3CCAF2DzVXeVZqnixnkbzw0La1ui5N5-RG9PwfMBHG9vmkfBSMzJA%40mail.gmail.com%3E), which I don't have in my mailbox, so obviously didn't

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

2019-06-03 Thread Jason Gerlowski
Hi Colvin, We're still taking a look at fixing the bug, but as a workaround in the meantime, you can look into adding a "forwardCredentials":true property under the "authentication" section of security.json. That seems to fix the issue in my reproduction at least. e.g. { "authentication":

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

2019-06-03 Thread Jason Gerlowski
One last note: as far as I can tell, nothing about this issue is specific to JSON Faceting or the JSON request API. It can be triggered just as easily with "/select?q=*:*". The bug created for this is: SOLR-13510 On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski wrote: > > I'm also able to

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

2019-06-03 Thread Jason Gerlowski
I'm also able to reproduce this bug on master. A few more notes about the bad behavior: - the behavior occurs regardless of the specific permissions configured in security.json. (i.e. whether the top permission is "all", or "security-edit", or there are no permissions at all.) - I tried looking

Re: Please help on pdate type during indexing

2019-06-03 Thread Shawn Heisey
On 6/2/2019 11:34 PM, derrick cui wrote: I spent whole day to indexing my data to solr(8.0), but there is one field which type is pdate always failed. error adding field 'UpdateDate'='org.apache.solr.common.SolrInputField:UpdateDate=2019-06-03T05:22:14.842Z' msg=Invalid Date in Date Math

Re: WG: SolrException: Can't determine a Sort Order with Solr 6.6

2019-06-03 Thread Shawn Heisey
On 6/3/2019 5:12 AM, Schwank, Désirée wrote: But how can it be protected. What can I do? What is to configure? Can you help me with an example. Place the Solr server in a network location such that only trusted systems and people can reach it. Sanitize all input in your application before

Re: Adding Multiple JSON Documents

2019-06-03 Thread Jason Gerlowski
Hi John, I believe the documentation there is correct. That is: those are two different "update" APIs. /update takes a JSON array of potentially multiple docs, /update/json/docs takes either a JSON array of multiple docs, or a single document not wrapped in the JSON array syntax. Best, Jason

WG: SolrException: Can't determine a Sort Order with Solr 6.6

2019-06-03 Thread Schwank , Désirée
In fact I am sending nothing, no sort order. The URl only contains a parameter q with the searchterm. Sortorder is only configured in solconfig. I agree with Walter, it is not safe that a bot can send values directly to Solr. But how can it be protected. What can I do? What is to configure?

highlighting not working as expected

2019-06-03 Thread Martin Frank Hansen (MHQ)
Hi, I am having some difficulties making highlighting work. For some reason the highlighting feature only works on some fields but not on other fields even though these fields are stored. An example of a request looks like this:

Re: Using Solr as a Database?

2019-06-03 Thread Ralph Soika
Thanks a lot again for your answers. I do now better understand the operation purpose of Solar Thanks for your help === Ralph On 02.06.19 23:27, Erick Erickson wrote: Not exactly. If I’m reading this right, you do now, and will continue, to have all the data in the RDBMS, correct? That’s