Re: Indexing Approach

2018-06-27 Thread solrnoobie
Thank you very much! This helped a lot in our estimates. One last thing though, will a queue based system work for us? Or are there better suitable implementations? -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Graph, GraphML, Gephi and Edge Labels

2018-06-27 Thread Heidi McClure
Hello, I am trying to export graph data from a Solr index (version 7.2) in a format that can be imported to Gephi for visualization. I'm getting close! Is there a way to add edge labels to the exports from this type of command (see curl command that follows and sample outputs)? Thanks in

Re: String concatenation in Streaming Expressions

2018-06-27 Thread Aroop Ganguly
I think there is a bug here in your query, the syntax is : concat(fields=\"fieldA,fieldB\", as=\"fieldABConcat\", delim=\"-\")" Try this: select( search(collection1,q="*:*",fl="conceptid",sort="conceptid asc",fq=storeid:"59c03d21d997b97bf47b3eeb",fq=schematype:"Article",fq=tags:"genetics",

Re: String concatenation in Streaming Expressions

2018-06-27 Thread Pratik Patel
Thanks Aroop, I tired following Streaming Expression but it doesn't work for me. select( search(collection1,q="*:*",fl="conceptid",sort="conceptid asc",fq=storeid:"59c03d21d997b97bf47b3eeb",fq=schematype:"Article",fq=tags:"genetics", qt="/export"), conceptid as conceptid, storeid as "test_",

Re: Storing documents with multiple primary keys

2018-06-27 Thread Alexandre Rafalovitch
eDisMax field aliasing? defType=edismax q=sysadmin name:Mike qf=title text last_name first_name f.name.qf=last_name first_name https://lucene.apache.org/solr/guide/7_3/the-extended-dismax-query-parser.html See also:

Re: String concatenation in Streaming Expressions

2018-06-27 Thread Aroop Ganguly
It seems like append is not available on 6.4, but concat is … Check this out on the 6.4 branch: https://github.com/apache/lucene-solr/blob/branch_6_4/solr/solrj/src/test/org/apache/solr/client/solrj/io/stream/ops/ConcatOperationTest.java

Re: String concatenation in Streaming Expressions

2018-06-27 Thread Aroop Ganguly
It should, but 6.6.* has some issues of things not working per documentation. Try using 7+. > On Jun 27, 2018, at 1:24 PM, Pratik Patel wrote: > > Thanks a lot for help! > > Looks like this is a recent addition? It doesn't work for me in version > 6.6.4 > > > > On Wed, Jun 27, 2018 at 4:18

Re: String concatenation in Streaming Expressions

2018-06-27 Thread Pratik Patel
Thanks a lot for help! Looks like this is a recent addition? It doesn't work for me in version 6.6.4 On Wed, Jun 27, 2018 at 4:18 PM, Aroop Ganguly wrote: > So it will become: > select( > search(..), > conceptid as foo, >storeid as bar > append(conceptid, storeid) as id > ) > >

Re: String concatenation in Streaming Expressions

2018-06-27 Thread Aroop Ganguly
So it will become: select( search(..), conceptid as foo, storeid as bar append(conceptid, storeid) as id ) Or select select( search(..), conceptid as foo, storeid as bar ), foo, bar, append(foo,bar) as id ) > On Jun 27, 2018, at 1:12 PM, Aroop Ganguly wrote: > > this test

Re: String concatenation in Streaming Expressions

2018-06-27 Thread Aroop Ganguly
this test case here will help in understanding the usage: https://github.com/apache/lucene-solr/blob/branch_7_2/solr/solrj/src/test/org/apache/solr/client/solrj/io/stream/eval/AppendEvaluatorTest.java

Re: String concatenation in Streaming Expressions

2018-06-27 Thread Aroop Ganguly
I think u can use the append evaluator https://github.com/apache/lucene-solr/blob/master/solr/solrj/src/java/org/apache/solr/client/solrj/io/eval/AppendEvaluator.java

String concatenation in Streaming Expressions

2018-06-27 Thread Pratik Patel
Hello, Is there a function which can be used in Streaming Expressions to concatenate two strings? I want to use it just like add(1,2) in a Streaming Expression. Essentially, I want to achieve something as follows. select( search(..), conceptid as foo, storeid as bar

Re: Storing documents with multiple primary keys

2018-06-27 Thread Erick Erickson
I have to ask "why can't you just store the second type with the same field names?" Or do they also have fields A, B and C? It's possible to store these in separate _collections_, configured as you see fit and use "collection aliasing" to search them both with a single query, but that won't solve

Storing documents with multiple primary keys

2018-06-27 Thread Grimes, Kevin M (398F)
Hi all, I have a unique situation that I’m not quite sure how to approach. In my current Solr collection, I have documents that look like this: { “fieldA”: 1, “fieldB”: 2, “fieldC”: 3 } We’re adding a new “type” of document, that looks like this: { “fieldD”: 1, “fieldE”: 2, “fieldF”: 3 } The

Storing documents with multiple equivalent field names

2018-06-27 Thread Kevin Grimes
Hi all, I have a unique situation that I’m not quite sure how to approach. In my current Solr collection, I have documents that look like this: { “fieldA”: 1, “fieldB”: 2, “fieldC”: 3 } We’re adding a new “type” of document, that looks like this: { “fieldD”: 1, “fieldE”: 2, “fieldF”: 3 }

Re: Java library for building Streaming Expressions

2018-06-27 Thread Aroop Ganguly
From my experience this is still limited and lot of things are broken if you still want to do it in a strongly typed manner. The only reliable way is to go right now is via a string-expression building route for the streaming expression. That being said, if your expressions are simple enough

Re: Total Collection Size in Solr 7

2018-06-27 Thread Aroop Ganguly
Ah ok ! > On Jun 27, 2018, at 8:53 AM, Erick Erickson wrote: > > Just sum up the sizes of all the files in your index directory. Clumsy > to be sure > > On Tue, Jun 26, 2018 at 3:12 PM, Aroop Ganguly > wrote: >> Hi Eric >> >> Thanks for the advice. >> One open question still, about

Java library for building Streaming Expressions

2018-06-27 Thread Pratik Patel
Hello Everyone, Is there any java library for building Streaming Expressions? Currently, I am using solr's java client and building Streaming Expressions as follows. StreamFactory factory = new StreamFactory().withCollectionZkHost( collName, zkHost )

Solr Update

2018-06-27 Thread Goston IV, Stanley W
All, I was wondering if it is possible to do a solr update by doing something like the following: JSON: { "add":{ "doc":{ "id_txt":"_query_:{q=title_txt:test.jpg fl=id_txt}", "title_txt": {"set":"Maybe"}, "location_geo": {"set":"POINT(-108.5491667

Re: Total Collection Size in Solr 7

2018-06-27 Thread Erick Erickson
Just sum up the sizes of all the files in your index directory. Clumsy to be sure On Tue, Jun 26, 2018 at 3:12 PM, Aroop Ganguly wrote: > Hi Eric > > Thanks for the advice. > One open question still, about point 1 below: how to get that magic number of > size in GBs :) ? > As I am mostly

Re: Sorting issue while using collection parameter

2018-06-27 Thread Erick Erickson
what kind of field is title? text_general or something? Sorting on a tokenized field is usually something you don't want to do. If a field has aardvard and zebra, how would it sort? There's usually something like alphaOnlySort. People often copyField from "title" to "title_sort" and search on

Re: Approach for Merge Database and Files

2018-06-27 Thread Erick Erickson
Seriously consider de-normalizing the data at index time. Your indexing client just accesses both the DB and the file system, selects the relevant data from each then indexes that data as a _single_ document. Then there's no joining necessary at query time. It's a common pattern to do extra work

Re: Solr 6.1 collection backup into HDFS

2018-06-27 Thread ankur.gpt28
Hi, were you able to figure out? I am struggling to use hdfs for backup for solr 6.1 running in local mode. -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr Analyzer for Vietnamese

2018-06-27 Thread bhavyajain
Dear @caomanhdat, Were you able to have a look into this i.e. updating the vietnamese Analyzer? Regards, Bhavya Jain -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: 7.4.0 changes in DocTransformer behaviour

2018-06-27 Thread Shawn Heisey
On 6/27/2018 8:29 AM, Markus Jelsma wrote: > I am attempting an upgrade to 7.4.0, besides a decent amount of thread leak > warnings i suddenly get, there are some strange failing tests, of which one > is one of four DocTransformer plugins. That plugin is really simple, it just > transformers

Re: solr basic authentication

2018-06-27 Thread Dinesh Sundaram
Thanks Chris for your help. I tried to find that solution but nothing is working out. it is not accepting the credentils, may be i'm trying with wrong base 64 algorithm. On Thu, Jun 21, 2018 at 12:25 PM, Christopher Schultz < ch...@christopherschultz.net> wrote: > Dinesh, > > On 6/21/18 11:40

Re: Solr basic auth

2018-06-27 Thread Dinesh Sundaram
yes, Thanks. I would like to whitelist one domain from the authlogin. On Thu, Jun 21, 2018 at 12:57 PM, Jan Høydahl wrote: > Hi, > > As I said there is not way to combine multiple authentication plugins at > the moment. > So your best shot is probably to create your own CustomAuthPlugin where

7.4.0 changes in DocTransformer behaviour

2018-06-27 Thread Markus Jelsma
Hello, I am attempting an upgrade to 7.4.0, besides a decent amount of thread leak warnings i suddenly get, there are some strange failing tests, of which one is one of four DocTransformer plugins. That plugin is really simple, it just transformers integer or long numerics into a binary string

Re: Approach for Merge Database and Files

2018-06-27 Thread Angel Addati
Thank you Erick. I readed a lot of in the web and I still a litle lost. In summarizze, I need perform a join (sql join). For the moment, I evaluated the following alternatives: 1. Alias (first response). I think it doesn't solve the problem, because I understand this is usefull to union

Re: Excluding facets - need help

2018-06-27 Thread Shawn Heisey
On 6/27/2018 6:14 AM, nc-tech-user wrote: Currently I am trying to rebuild the products filter so it will use Solr search facet result to show its attributes. I am having some difficulties with this solution. In most cases I'm using facet on about 60 fields (their types are float, integer

Re: Maximum number of SolrCloud collections in limited hardware resource

2018-06-27 Thread Shawn Heisey
On 6/27/2018 5:10 AM, Sharif Shahrair wrote: Now the problem is, when we create about 1400 collection(all of them are empty i.e. no document is added yet) the solr service goes down showing out of memory exception. We have few questions here- 1. When we are creating collections, each collection

Re: Drive Change for existing Solr Setup

2018-06-27 Thread Shawn Heisey
On 6/27/2018 4:13 AM, Srinivas Muppu (US) wrote: Below are the high level Installation steps used for ZK and Solr setup in Windows* D:\ drive*. Could you please go through once and suggest possible solution/steps needs to be done for the setup moving to the E:\ Drive(New drive) without any

Excluding facets - need help

2018-06-27 Thread nc-tech-user
​Dear Solr User, I develop an online shop which has possibility to search for products and filter the results. I've used Solr as an engine to search through our shop offer. Currently I am trying to rebuild the products filter so it will use Solr search facet result to show its attributes. I

Re: Suggestions for debugging performance issue

2018-06-27 Thread Susheel Kumar
Did you try to see where/which component like query, facet highlight... is taking time by debugQuery=on when performance is slow. Just to rule out any other component is not the culprit... Thnx On Mon, Jun 25, 2018 at 2:06 PM, Chris Troullis wrote: > FYI to all, just as an update, we rebuilt

Maximum number of SolrCloud collections in limited hardware resource

2018-06-27 Thread Sharif Shahrair
Hi Guys, We are in a use-case where we need to create a large number of collections(1000 to 1500) in a SolrCloud. Here most of collections will have a very limited number of documents(100 to 1000), even some collections are empty. We are using single shard and 2 replicas.For each replica we using

Re: Drive Change for existing Solr Setup

2018-06-27 Thread Srinivas Muppu (US)
Hi Shawn, Thanks for the response. Below are the high level Installation steps used for ZK and Solr setup in Windows* D:\ drive*. Could you please go through once and suggest possible solution/steps needs to be done for the setup moving to the E:\ Drive(New drive) without any impacting to the

Re: Indexing part of Binary Documents and not the entire contents

2018-06-27 Thread neotorand
Thanks Erick I already have gone through the link from tika example you shared. Please look at the code in bold. I believe still the entire contents is pushed to memory with handler object. sorry i copied lengthy code from tika site. Regards Neo *Streaming the plain text in chunks* Sometimes,

Re: SOLR migration

2018-06-27 Thread Jan Høydahl
Hi, Apache Solr is an open source project, with contributors and users doing our best to both document and answer questions, but sometimes it may take time, either due busy schedules or because the answer is unknown or an uncommon one that very few have tried. I suspect the last reason for the

[ANNOUNCE] Apache Solr 7.4.0 released

2018-06-27 Thread Adrien Grand
27th June 2018, Apache Solr™ 7.4.0 available The Lucene PMC is pleased to announce the release of Apache Solr 7.4.0 Solr is the popular, blazing fast, open source NoSQL search platform from the Apache Lucene project. Its major features include powerful full-text search, hit highlighting, faceted

RE: Change/Override Solrconfig.xml across collections

2018-06-27 Thread Markus Jelsma
https://lucene.apache.org/solr/guide/6_6/configuring-solrconfig-xml.html This page has a section about parameter substitution in config via command line overrides. -Original message- > From:Ganesh Sethuraman > Sent: Wednesday 27th June 2018 1:06 > To: solr-user@lucene.apache.org >

Sorting issue while using collection parameter

2018-06-27 Thread Vijay Tiwary
Hello Team, I have multiple collection on solr (5.4.1) cloud based on year content2107 content2018 Also I have a collection "content" which does not have any data. Now if I query them as follows http://host:port/solr/content/select?q=*:*=content2107, content2108=title asc Where title is string