Re: Java object binding not working

[Ray Niu]

did you use multi values field？

On Sun, Feb 3, 2019 at 8:22 PM Swapnil Katkar  wrote:

> Greetings!
>
> I am working on a requirement where I want to query the data and want to
> do the object mapping for the retrieved result using Solrj. For this, I am
> referring to the official document at 
> *https://lucene.apache.org/solr/guide/7_6/using-solrj.html#java-object-binding
> .*
>  I
> set-up the necessary class files and the collections.
>
> With the help of this document, I can create the documents in the Solr DB,
> but it is not working for fetching and mapping the fields to the Java POJO
> class. To do the mapping, I used @Field annotation.
>
> Details are as below:
> *1)* Solrj version: 7.6.0
> *2)* The line of code which is not working: *List employees =
> response.getBeans(Employee.class);*
> *3)* Exception stack trace:
> *Caused by: java.lang.IllegalArgumentException: Can not set
> java.lang.String field demo.apache.solr.vo.Employee.name
>  to java.util.ArrayList*
> * at
> sun.reflect.UnsafeFieldAccessorImpl.throwSetIllegalArgumentException(Unknown
> Source)*
> * at
> sun.reflect.UnsafeFieldAccessorImpl.throwSetIllegalArgumentException(Unknown
> Source)*
> * at sun.reflect.UnsafeObjectFieldAccessorImpl.set(Unknown Source)*
> * at java.lang.reflect.Field.set(Unknown Source)*
> *4)* Collection was created using
> *solr.cmd create -c employees -s 2 -rf 2*
>
> Please find the attached source code files. Also, I attached the stack
> trace file. Can you please help me on how to resolve them?
>
> Regards,
> Swapnil Katkar
>

Re: Question about elevations

[Ray Niu]

one more thing to add, if there are fqs, they will be evaluated as well.

Edward Ribeiro  于2018年11月19日周一 下午1:24写道：

> Just complementing Alessandro's answer:
> 1. the elevateIds are inserted into the query, server side (a query
> expansion indeed);
> 2. the query is executed;
> 3. elevatedIds (if found) are popped up to the top of the search results
> via boosting;
>
> Edward
>
> On Mon, Nov 19, 2018 at 3:41 PM Alessandro Benedetti  >
> wrote:
> >
> > As far as I remember the answer is no.
> > You could take a deep look into the code, but as far as I remember the
> > elevated doc Ids must be in the index to be elevated.
> > Those ids will be added to the query built, a sort of query expansion
> server
> > side.
> > And then the search executed.
> >
> > Cheers
> >
> >
> >
> >
> >
> > -
> > ---
> > Alessandro Benedetti
> > Search Consultant, R Software Engineer, Director
> > Sease Ltd. - www.sease.io
> > --
> > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>

Re: Sort across collapsed document is not working

[Ray Niu]

this is not sharded collection, it only had one shard. I want to use
collapse to replace current group query, but the result is not same, I feel
there are some function issue in collapse plugin

2017-09-07 14:59 GMT-07:00 Erick Erickson <erickerick...@gmail.com>:

> Is this a sharded collection? group.ngroups isn't supported (see the
> docs, "group.ngroups and group.facet require that all documents in
> each group must be co-located on the same shard") in sharded
> situations so it's not surprising that the results differ.
>
> Best,
> Erick
>
> On Thu, Sep 7, 2017 at 10:35 AM, Ray Niu <newry1...@gmail.com> wrote:
> > Hello:
> >I tried to use Collapsing Query Parser per following link:
> >
> > https://cwiki.apache.org/confluence/display/solr/
> Collapse+and+Expand+Results
> > here is the query I am using
> > http:///solr/collection/select?q=groupId:*&
> > fl=id,groupId,date=%7B!collapse%20field=groupId%
> 20sort=%27id%20asc%27%7D&
> > expand=true=3=date%20asc=id%20asc=3
> >
> > but I found the result is different from group query:
> > http:///solr/collection/select?q=groupId:*&
> > fl=id,date,groupId=true=groupId
> > limit=4=true=date%20asc=id%20asc=3
> >
> > it seems sort across collapsed document is not working.
> >
> > Can anyone help on this?
>

Sort across collapsed document is not working

[Ray Niu]

Hello:
   I tried to use Collapsing Query Parser per following link:

https://cwiki.apache.org/confluence/display/solr/Collapse+and+Expand+Results
here is the query I am using
http:///solr/collection/select?q=groupId:*&
fl=id,groupId,date=%7B!collapse%20field=groupId%20sort=%27id%20asc%27%7D&
expand=true=3=date%20asc=id%20asc=3

but I found the result is different from group query:
http:///solr/collection/select?q=groupId:*&
fl=id,date,groupId=true=groupId
limit=4=true=date%20asc=id%20asc=3

it seems sort across collapsed document is not working.

Can anyone help on this?

Re: Solr Join Failures

[Ray Niu]

I am using following one:
http://lucene.apache.org/solr/5_4_1/solr-core/org/apache/solr/search/join/ScoreJoinQParserPlugin.html
q={!join from=id to=id fromIndex=B}id:*

2017-06-12 20:05 GMT-07:00 Zheng Lin Edwin Yeo <edwinye...@gmail.com>:

> What is the query that you used to do the Join?
>
> There is the Streaming expression which has the various Join function, but
> it requires Solr version 6 onward.
> https://cwiki.apache.org/confluence/display/solr/Streaming+Expressions
>
>
> Regards,
> Edwin
>
>
> On 13 June 2017 at 05:25, Ray Niu <newry1...@gmail.com> wrote:
>
> > Hi:
> >We encounter an issue when using join query in Solr Cloud, our version
> > is 5.5.2. We will query collection A and join with Collection B in the
> > runtime, collection A and B always co-exist in the same node.
> >Sometimes we found collection B was down for some reason, while
> > collection A was still active, during that period if we issue join query,
> > we will see following errors:
> >  SolrCloud join: B has a local replica (B_shard1_replica1) on
> > solrCloud_node1:8080_solr, but it is down
> >   Can anyone provide any suggestions on such scenario?
> >
>

Solr Join Failures

[Ray Niu]

Hi:
   We encounter an issue when using join query in Solr Cloud, our version
is 5.5.2. We will query collection A and join with Collection B in the
runtime, collection A and B always co-exist in the same node.
   Sometimes we found collection B was down for some reason, while
collection A was still active, during that period if we issue join query,
we will see following errors:
 SolrCloud join: B has a local replica (B_shard1_replica1) on
solrCloud_node1:8080_solr, but it is down
  Can anyone provide any suggestions on such scenario?

Re: solrj5.50 query excepion

[Ray Niu]

Spellcheck reponse format was changed since 5.0,not backward compatible
alias <524839...@qq.com>于2017年2月13日 周一下午6:05写道：

> hi  I use solrj 5.5.0 to inquire solr3.6 reported the following error:
> Java.lang.ClassCastException: java.lang.Boolean can not be cast to
> org.apache.solr.common.util.NamedList
> At org.apache.solr.client.solrj.response.SpellCheckResponse.
>  (SpellCheckResponse.java:47)
> At
> org.apache.solr.client.solrj.response.QueryResponse.extractSpellCheckInfo
> (QueryResponse.java:179)
> At org.apache.solr.client.solrj.response.QueryResponse.setResponse
> (QueryResponse.java:153)
> At org.apache.solr.client.solrj.SolrRequest.process
> (SolrRequest.java:149)
> At org.apache.solr.client.solrj.SolrClient.query
> (SolrClient.java:942)
> At org.apache.solr.client.solrj.SolrClient.query
> (SolrClient.java:957)
> At com.vip.vipme.demo.utils.SolrTest.testCategoryIdPC
> (SolrTest.java:66)
> At com.vip.vipme.demo.SolrjServlet1.doGet (SolrjServlet1.java:33)
> At javax.servlet.http.HttpServlet.service (HttpServlet.java:707)
> At javax.servlet.http.HttpServlet.service (HttpServlet.java:820)
> At org.mortbay.jetty.servlet.ServletHolder.handle
> (ServletHolder.java:487)
> At org.mortbay.jetty.servlet.ServletHandler.handle
> (ServletHandler.java:362)
> At org.mortbay.jetty.security.SecurityHandler.handle
> (SecurityHandler.java:216)
> At org.mortbay.jetty.servlet.SessionHandler.handle
> (SessionHandler.java:181)
> At org.mortbay.jetty.handler.ContextHandler.handle
> (ContextHandler.java:712)
> At org.mortbay.jetty.webapp.WebAppContext.handle
> (WebAppContext.java:405)
>
>
> If you set the query.set ("spellcheck", Boolean.FALSE); can solve this
> problem,
> But I would like to know what the specific reasons for this problem
>
>
> thinks

Re: High CPU Usage in export handler

[Ray Niu]

Thanks Joel.

2016-11-08 11:43 GMT-08:00 Joel Bernstein <joels...@gmail.com>:

> It sounds like your scenario, is around 25 queries per second, each pulling
> entire results. This would be enough to drive up CPU usage as you have more
> concurrent requests then CPU's. Since there isn't much IO blocking
> happening, in the scenario you describe, I would expect some pretty busy
> CPU's.
>
> That being said I think it would useful to understand exactly where the
> hotspots are in Lucene to see if we can make this more efficient.
>
> Leading up to the 6.4 release I'll try to spend some time understanding the
> Lucene hotspots with /export. I'll report back to this thread when I have
> more info.
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Mon, Nov 7, 2016 at 3:44 PM, Ray Niu <newry1...@gmail.com> wrote:
>
> > Hello:
> >Any follow up?
> >
> > 2016-11-03 11:18 GMT-07:00 Ray Niu <newry1...@gmail.com>:
> >
> > > the soft commit is 15 seconds and hard commit is 10 minutes.
> > >
> > > 2016-11-03 11:11 GMT-07:00 Erick Erickson <erickerick...@gmail.com>:
> > >
> > >> Followup question: You say you're indexing 100 docs/second.  How often
> > >> are you _committing_? Either
> > >> soft commit
> > >> or
> > >> hardcommit with openSearcher=true
> > >>
> > >> ?
> > >>
> > >> Best,
> > >> Erick
> > >>
> > >> On Thu, Nov 3, 2016 at 11:00 AM, Ray Niu <newry1...@gmail.com> wrote:
> > >> > Thanks Joel
> > >> > here is the information you requested.
> > >> > Are you doing heavy writes at the time?
> > >> > we are doing write very frequently, but not very heavy, we will
> update
> > >> > about 100 solr document per second.
> > >> > How many concurrent reads are are happening?
> > >> > the concurrent reads are about 1000-2000 per minute per node
> > >> > What version of Solr are you using?
> > >> > we are using solr 5.5.2
> > >> > What is the field definition for the double, is it docValues?
> > >> > the field definition is
> > >> >  > >> > docValues="true"/>
> > >> >
> > >> >
> > >> > 2016-11-03 6:30 GMT-07:00 Joel Bernstein <joels...@gmail.com>:
> > >> >
> > >> >> Are you doing heavy writes at the time?
> > >> >>
> > >> >> How many concurrent reads are are happening?
> > >> >>
> > >> >> What version of Solr are you using?
> > >> >>
> > >> >> What is the field definition for the double, is it docValues?
> > >> >>
> > >> >>
> > >> >>
> > >> >>
> > >> >> Joel Bernstein
> > >> >> http://joelsolr.blogspot.com/
> > >> >>
> > >> >> On Thu, Nov 3, 2016 at 12:56 AM, Ray Niu <newry1...@gmail.com>
> > wrote:
> > >> >>
> > >> >> > Hello:
> > >> >> >We are using export handler in Solr Cloud to get some data, we
> > >> only
> > >> >> > request for one field, which type is tdouble, it works well at
> the
> > >> >> > beginning, but recently we saw high CPU issue in all the solr
> cloud
> > >> >> nodes,
> > >> >> > we took some thread dump and found following information:
> > >> >> >
> > >> >> >java.lang.Thread.State: RUNNABLE
> > >> >> >
> > >> >> > at java.lang.Thread.isAlive(Native Method)
> > >> >> >
> > >> >> > at
> > >> >> > org.apache.lucene.util.CloseableThreadLocal.purge(
> > >> >> > CloseableThreadLocal.java:115)
> > >> >> >
> > >> >> > - locked <0x0006e24d86a8> (a java.util.WeakHashMap)
> > >> >> >
> > >> >> > at
> > >> >> > org.apache.lucene.util.CloseableThreadLocal.maybePurge(
> > >> >> > CloseableThreadLocal.java:105)
> > >> >> >
> > >> >> > at
> > >> >> > org.apache.lucene.util.CloseableThreadLocal.get(
> > >> >> > CloseableThreadLocal.java:88)
> > >> >> >
> > >> >> > at
> > >> >> > org.apache.lucene.index.CodecReader.getNumericDocValues(
> > >> >> > CodecReader.java:143)
> > >> >> >
> > >> >> > at
> > >> >> > org.apache.lucene.index.FilterLeafReader.getNumericDocValues(
> > >> >> > FilterLeafReader.java:430)
> > >> >> >
> > >> >> > at
> > >> >> > org.apache.lucene.uninverting.UninvertingReader.
> > getNumericDocValues(
> > >> >> > UninvertingReader.java:239)
> > >> >> >
> > >> >> > at
> > >> >> > org.apache.lucene.index.FilterLeafReader.getNumericDocValues(
> > >> >> > FilterLeafReader.java:430)
> > >> >> >
> > >> >> > Is this a known issue for export handler? As we only fetch up to
> > 5000
> > >> >> > documents, it should not be data volume issue.
> > >> >> >
> > >> >> > Can anyone help on that? Thanks a lot.
> > >> >> >
> > >> >>
> > >>
> > >
> > >
> >
>

Re: High CPU Usage in export handler

[Ray Niu]

Hello:
   Any follow up?

2016-11-03 11:18 GMT-07:00 Ray Niu <newry1...@gmail.com>:

> the soft commit is 15 seconds and hard commit is 10 minutes.
>
> 2016-11-03 11:11 GMT-07:00 Erick Erickson <erickerick...@gmail.com>:
>
>> Followup question: You say you're indexing 100 docs/second.  How often
>> are you _committing_? Either
>> soft commit
>> or
>> hardcommit with openSearcher=true
>>
>> ?
>>
>> Best,
>> Erick
>>
>> On Thu, Nov 3, 2016 at 11:00 AM, Ray Niu <newry1...@gmail.com> wrote:
>> > Thanks Joel
>> > here is the information you requested.
>> > Are you doing heavy writes at the time?
>> > we are doing write very frequently, but not very heavy, we will update
>> > about 100 solr document per second.
>> > How many concurrent reads are are happening?
>> > the concurrent reads are about 1000-2000 per minute per node
>> > What version of Solr are you using?
>> > we are using solr 5.5.2
>> > What is the field definition for the double, is it docValues?
>> > the field definition is
>> > > > docValues="true"/>
>> >
>> >
>> > 2016-11-03 6:30 GMT-07:00 Joel Bernstein <joels...@gmail.com>:
>> >
>> >> Are you doing heavy writes at the time?
>> >>
>> >> How many concurrent reads are are happening?
>> >>
>> >> What version of Solr are you using?
>> >>
>> >> What is the field definition for the double, is it docValues?
>> >>
>> >>
>> >>
>> >>
>> >> Joel Bernstein
>> >> http://joelsolr.blogspot.com/
>> >>
>> >> On Thu, Nov 3, 2016 at 12:56 AM, Ray Niu <newry1...@gmail.com> wrote:
>> >>
>> >> > Hello:
>> >> >We are using export handler in Solr Cloud to get some data, we
>> only
>> >> > request for one field, which type is tdouble, it works well at the
>> >> > beginning, but recently we saw high CPU issue in all the solr cloud
>> >> nodes,
>> >> > we took some thread dump and found following information:
>> >> >
>> >> >java.lang.Thread.State: RUNNABLE
>> >> >
>> >> > at java.lang.Thread.isAlive(Native Method)
>> >> >
>> >> > at
>> >> > org.apache.lucene.util.CloseableThreadLocal.purge(
>> >> > CloseableThreadLocal.java:115)
>> >> >
>> >> > - locked <0x0006e24d86a8> (a java.util.WeakHashMap)
>> >> >
>> >> > at
>> >> > org.apache.lucene.util.CloseableThreadLocal.maybePurge(
>> >> > CloseableThreadLocal.java:105)
>> >> >
>> >> > at
>> >> > org.apache.lucene.util.CloseableThreadLocal.get(
>> >> > CloseableThreadLocal.java:88)
>> >> >
>> >> > at
>> >> > org.apache.lucene.index.CodecReader.getNumericDocValues(
>> >> > CodecReader.java:143)
>> >> >
>> >> > at
>> >> > org.apache.lucene.index.FilterLeafReader.getNumericDocValues(
>> >> > FilterLeafReader.java:430)
>> >> >
>> >> > at
>> >> > org.apache.lucene.uninverting.UninvertingReader.getNumericDocValues(
>> >> > UninvertingReader.java:239)
>> >> >
>> >> > at
>> >> > org.apache.lucene.index.FilterLeafReader.getNumericDocValues(
>> >> > FilterLeafReader.java:430)
>> >> >
>> >> > Is this a known issue for export handler? As we only fetch up to 5000
>> >> > documents, it should not be data volume issue.
>> >> >
>> >> > Can anyone help on that? Thanks a lot.
>> >> >
>> >>
>>
>
>

Re: High CPU Usage in export handler

[Ray Niu]

the soft commit is 15 seconds and hard commit is 10 minutes.

2016-11-03 11:11 GMT-07:00 Erick Erickson <erickerick...@gmail.com>:

> Followup question: You say you're indexing 100 docs/second.  How often
> are you _committing_? Either
> soft commit
> or
> hardcommit with openSearcher=true
>
> ?
>
> Best,
> Erick
>
> On Thu, Nov 3, 2016 at 11:00 AM, Ray Niu <newry1...@gmail.com> wrote:
> > Thanks Joel
> > here is the information you requested.
> > Are you doing heavy writes at the time?
> > we are doing write very frequently, but not very heavy, we will update
> > about 100 solr document per second.
> > How many concurrent reads are are happening?
> > the concurrent reads are about 1000-2000 per minute per node
> > What version of Solr are you using?
> > we are using solr 5.5.2
> > What is the field definition for the double, is it docValues?
> > the field definition is
> >  > docValues="true"/>
> >
> >
> > 2016-11-03 6:30 GMT-07:00 Joel Bernstein <joels...@gmail.com>:
> >
> >> Are you doing heavy writes at the time?
> >>
> >> How many concurrent reads are are happening?
> >>
> >> What version of Solr are you using?
> >>
> >> What is the field definition for the double, is it docValues?
> >>
> >>
> >>
> >>
> >> Joel Bernstein
> >> http://joelsolr.blogspot.com/
> >>
> >> On Thu, Nov 3, 2016 at 12:56 AM, Ray Niu <newry1...@gmail.com> wrote:
> >>
> >> > Hello:
> >> >We are using export handler in Solr Cloud to get some data, we only
> >> > request for one field, which type is tdouble, it works well at the
> >> > beginning, but recently we saw high CPU issue in all the solr cloud
> >> nodes,
> >> > we took some thread dump and found following information:
> >> >
> >> >java.lang.Thread.State: RUNNABLE
> >> >
> >> > at java.lang.Thread.isAlive(Native Method)
> >> >
> >> > at
> >> > org.apache.lucene.util.CloseableThreadLocal.purge(
> >> > CloseableThreadLocal.java:115)
> >> >
> >> > - locked <0x0006e24d86a8> (a java.util.WeakHashMap)
> >> >
> >> > at
> >> > org.apache.lucene.util.CloseableThreadLocal.maybePurge(
> >> > CloseableThreadLocal.java:105)
> >> >
> >> > at
> >> > org.apache.lucene.util.CloseableThreadLocal.get(
> >> > CloseableThreadLocal.java:88)
> >> >
> >> > at
> >> > org.apache.lucene.index.CodecReader.getNumericDocValues(
> >> > CodecReader.java:143)
> >> >
> >> > at
> >> > org.apache.lucene.index.FilterLeafReader.getNumericDocValues(
> >> > FilterLeafReader.java:430)
> >> >
> >> > at
> >> > org.apache.lucene.uninverting.UninvertingReader.getNumericDocValues(
> >> > UninvertingReader.java:239)
> >> >
> >> > at
> >> > org.apache.lucene.index.FilterLeafReader.getNumericDocValues(
> >> > FilterLeafReader.java:430)
> >> >
> >> > Is this a known issue for export handler? As we only fetch up to 5000
> >> > documents, it should not be data volume issue.
> >> >
> >> > Can anyone help on that? Thanks a lot.
> >> >
> >>
>

Re: High CPU Usage in export handler

[Ray Niu]

Thanks Joel
here is the information you requested.
Are you doing heavy writes at the time?
we are doing write very frequently, but not very heavy, we will update
about 100 solr document per second.
How many concurrent reads are are happening?
the concurrent reads are about 1000-2000 per minute per node
What version of Solr are you using?
we are using solr 5.5.2
What is the field definition for the double, is it docValues?
the field definition is



2016-11-03 6:30 GMT-07:00 Joel Bernstein <joels...@gmail.com>:

> Are you doing heavy writes at the time?
>
> How many concurrent reads are are happening?
>
> What version of Solr are you using?
>
> What is the field definition for the double, is it docValues?
>
>
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Thu, Nov 3, 2016 at 12:56 AM, Ray Niu <newry1...@gmail.com> wrote:
>
> > Hello:
> >We are using export handler in Solr Cloud to get some data, we only
> > request for one field, which type is tdouble, it works well at the
> > beginning, but recently we saw high CPU issue in all the solr cloud
> nodes,
> > we took some thread dump and found following information:
> >
> >java.lang.Thread.State: RUNNABLE
> >
> > at java.lang.Thread.isAlive(Native Method)
> >
> > at
> > org.apache.lucene.util.CloseableThreadLocal.purge(
> > CloseableThreadLocal.java:115)
> >
> > - locked <0x0006e24d86a8> (a java.util.WeakHashMap)
> >
> > at
> > org.apache.lucene.util.CloseableThreadLocal.maybePurge(
> > CloseableThreadLocal.java:105)
> >
> > at
> > org.apache.lucene.util.CloseableThreadLocal.get(
> > CloseableThreadLocal.java:88)
> >
> > at
> > org.apache.lucene.index.CodecReader.getNumericDocValues(
> > CodecReader.java:143)
> >
> > at
> > org.apache.lucene.index.FilterLeafReader.getNumericDocValues(
> > FilterLeafReader.java:430)
> >
> > at
> > org.apache.lucene.uninverting.UninvertingReader.getNumericDocValues(
> > UninvertingReader.java:239)
> >
> > at
> > org.apache.lucene.index.FilterLeafReader.getNumericDocValues(
> > FilterLeafReader.java:430)
> >
> > Is this a known issue for export handler? As we only fetch up to 5000
> > documents, it should not be data volume issue.
> >
> > Can anyone help on that? Thanks a lot.
> >
>

High CPU Usage in export handler

[Ray Niu]

Hello:
   We are using export handler in Solr Cloud to get some data, we only
request for one field, which type is tdouble, it works well at the
beginning, but recently we saw high CPU issue in all the solr cloud nodes,
we took some thread dump and found following information:

   java.lang.Thread.State: RUNNABLE

at java.lang.Thread.isAlive(Native Method)

at
org.apache.lucene.util.CloseableThreadLocal.purge(CloseableThreadLocal.java:115)

- locked <0x0006e24d86a8> (a java.util.WeakHashMap)

at
org.apache.lucene.util.CloseableThreadLocal.maybePurge(CloseableThreadLocal.java:105)

at
org.apache.lucene.util.CloseableThreadLocal.get(CloseableThreadLocal.java:88)

at
org.apache.lucene.index.CodecReader.getNumericDocValues(CodecReader.java:143)

at
org.apache.lucene.index.FilterLeafReader.getNumericDocValues(FilterLeafReader.java:430)

at
org.apache.lucene.uninverting.UninvertingReader.getNumericDocValues(UninvertingReader.java:239)

at
org.apache.lucene.index.FilterLeafReader.getNumericDocValues(FilterLeafReader.java:430)

Is this a known issue for export handler? As we only fetch up to 5000
documents, it should not be data volume issue.

Can anyone help on that? Thanks a lot.

Re: Whether solr can support 2 TB data?

[Ray Niu]

You can only put 2 billion documents in one core, I would suggest to use solr
cloud. you need calculate how many solr document in your data and then
decide how many shards to go. you can get many useful resource in website,
I just provide one here.
http://www.slideshare.net/anshumg/best-practices-for-highly-available-and-large-scale-solrcloud


2016-09-23 13:33 GMT-07:00 Jeffery Yuan :

> Hi, Dear all:
>
>   In our application, every data there is about 800mb raw data, we are
> going
> to store this data for 5 years, then it's about 1 or 2 TB data.
>
>   I am wondering whether solr can support this much data?
>   Usually how much data we store per node, how many nodes we can have in
> solr cloud, what hardware configuration each node should be?
>
> Thanks very much for your help.
>
>
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/Whether-solr-can-support-2-TB-data-tp4297790.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: help with field definition

[Ray Niu]

Just add q.op=OR to change default operator to OR and it should work

2016-09-16 12:44 GMT-07:00 Gandham, Satya :

> Hi Emir,
>
>Thanks for your reply. But I’m afraid I’m not seeing the
> expected response. I’ve included the query and the corresponding debug
> portion of the response:
>
> select?q=Justin\ Beiber=exactName_noAlias_en_US
>
>  Debug:
>
> "rawquerystring":"Justin\\ Beiber",
> "querystring":"Justin\\ Beiber",
> "parsedquery":"+((exactName_noAlias_en_US:justin
> exactName_noAlias_en_US:justin beiber)/no_coord) +exactName_noAlias_en_US:
> beiber",
> "parsedquery_toString":"+(exactName_noAlias_en_US:justin
> exactName_noAlias_en_US:justin beiber) +exactName_noAlias_en_US:beiber",
> "explain":{},
>
>
> Satya.
>
> On 9/16/16, 2:46 AM, "Emir Arnautovic" 
> wrote:
>
> Hi,
>
> I missed that you already did define field and you are having troubles
> with query (did not read stackoverflow). Added answer there, but just
> in
> case somebody else is having similar troubles, issue is how query is
> written - space has to be escaped:
>
>q=Justin\ Bieber
>
> Regards,
> Emir
>
> On 13.09.2016 23:27, Gandham, Satya wrote:
> > HI,
> >
> >I need help with defining a field ‘singerName’ with the
> right tokenizers and filters such that it gives me the below described
> behavior:
> >
> > I have a few documents as given below:
> >
> > Doc 1
> >singerName: Justin Beiber
> > Doc 2:
> >singerName: Justin Timberlake
> > …
> >
> >
> > Below is the list of quries and the corresponding matches:
> >
> > Query 1: “My fav artist Justin Beiber is very impressive”
> > Docs Matched : Doc1
> >
> > Query 2: “I have a Justin Timberlake poster on my wall”
> > Docs Matched: Doc2
> >
> > Query 3: “The name Bieber Justin is unique”
> > Docs Matched: None
> >
> > Query 4: “Timberlake is a lake of timber..?”
> > Docs Matched: None.
> >
> > I have this described a bit more detailed here:
> http://stackoverflow.com/questions/39399321/solr-shingle-query-matching-
> keyword-tokenized-field
> >
> > I’d appreciate any help in addressing this problem.
> >
> > Thanks !!
> >
>
> --
> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> Solr & Elasticsearch Support * http://sematext.com/
>
>
>
>

Re: Solr Rounding Issue On Float fields.

[Ray Niu]

I met simliar issue before，suggest to use double as field type for this case

Ray
2016年7月21日星期四，Nick Vasilyev  写道：

> Thanks Chris.
>
> Searching for both values and retrieving the documents would be alright as
> long as the data was correct. In this case, the data that I am indexing
> into Solr is not the same data that I am pulling out at query time. That is
> the real impact here.
>
> On Thu, Jul 21, 2016 at 6:12 PM, Chris Hostetter  >
> wrote:
>
> >
> > : Hi, I am running into a weird rounding issue on Solr 5.2.1. I have a
> > float
> > : field (also tried tfloat), I am indexing 154035.26 into it (confirmed
> in
> > : the data),  but at query time, I get back 154035.27 (.01 more).
> > : Additionally when I query for the document and include this number in
> > the q
> > : parameter, it comes up with both values, .26 and .27.
> >
> > Pretty sure what you are observing is just the normal consequences of
> IEEE
> > floats (as used by java) being base2 -- not every base10 decimal value
> > has a precise base2 representation.
> >
> > Quering for 154035.27 and 154035.26 will both match the same docs,
> because
> > the String->Float parsing in both cases will produce the closest *legal*
> > float value, which is identical for both inputs.
> >
> > If need precise decimal values in solr, you need to either use 2
> > ints/longs (ie num_base="154035", num_decimal="26") or use one int/long
> > and multiply/divide by a power of 10 corisponding to the number of
> > significant digits you want in the client (ie: "15403526" divide by 100)
> >
> >
> > Some good reading linked to from here...
> >
> > http://perlmonks.org/?node_id=203257
> >
> > And of course, if you really want to bang java against your head,
> > this is a classic (all of which is still appliable i believe) ...
> >
> > https://people.eecs.berkeley.edu/~wkahan/JAVAhurt.pdf
> >
> >
> >
> >
> >
> > -Hoss
> > http://www.lucidworks.com/
> >
>

Issues in export handler

[Ray Niu]

Hello:
I am seeing following Exception during call export handler, is anyone
familiar with it?
at org.apache.lucene.util.BitSetIterator.(BitSetIterator.java:58)
at
org.apache.solr.response.SortingResponseWriter.write(SortingResponseWriter.java:138)
at
org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:53)
at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:727)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:274)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:242)
at
com.stubhub.newplatform.http.filter.DyeHttpFilter.doFilter(DyeHttpFilter.java:33)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:274)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:242)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
at
org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:181)
at
org.jboss.modcluster.catalina.CatalinaContext$RequestListenerValve.event(CatalinaContext.java:285)
at
org.jboss.modcluster.catalina.CatalinaContext$RequestListenerValve.invoke(CatalinaContext.java:261)
at
org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:88)
at
org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:100)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:159)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:567)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.jboss.web.tomcat.service.request.ActiveRequestResponseCacheValve.invoke(ActiveRequestResponseCacheValve.java:53)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:362)
at org.apache.coyote.ajp.AjpAprProcessor.process(AjpAprProcessor.java:489)
at
org.apache.coyote.ajp.AjpAprProtocol$AjpConnectionHandler.process(AjpAprProtocol.java:452)
at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:2019)
at java.lang.Thread.run(Thread.java:745)

Re: Sort not working as expected

[Ray Niu]

you should use string type instead of text if you want to sort
alphabetically

2015-10-30 11:12 GMT-07:00 Brian Narsi :

> I have a fieldtype setup as
>
>  "100">   "solr.StandardTokenizerFactory"/>  "solr.LowerCaseFilterFactory"/>  minGramSize="3" maxGramSize="25"/>   <
> tokenizer class="solr.StandardTokenizerFactory"/>  "solr.LowerCaseFilterFactory"/>  
>
>
> When I sort on this field type in ascending order I am not getting results
> sorted alphabetically as expected.
>
> Why is that?
>
> What should I do to get the sort on?
>
> Thanks
>

Re: Query differently or change fieldtype

[Ray Niu]

I think this is how StandardTokenizerFactory works, if you want different
behavior, you should try to use a different tokenizer, also like Upayavira
said,use the analysis tab on the admin UI to see what analysis is doing to your
terms.

2015-10-26 12:33 GMT-07:00 Brian Narsi <bnars...@gmail.com>:

> That is right Ray, that is exactly what I found out and that is why I am
> asking the question.
>
> On Mon, Oct 26, 2015 at 2:19 PM, Ray Niu <newry1...@gmail.com> wrote:
>
> > I found the conf minGramSize="2"，which will only create index with at
> least
> > 2 chars，j will not match
> > also StandardTokenizerFactory will tokenize st j to st and j
> >
> > 2015年10月26日星期一，Brian Narsi <bnars...@gmail.com> 写道：
> >
> > > I have the following field type on a field ClientName:
> > >
> > >  > > positionIncrementGap="100">
> > > 
> > > 
> > > 
> > >  > > maxGramSize="25"/>
> > > 
> > > 
> > > 
> > > 
> > > 
> > >   
> > >
> > >
> > > For data where
> > >
> > > ClientName = st jude medical inc
> > >
> > > When querying I get the following:
> > >
> > > 1) st --> result = st jude medical inc (works correctly)
> > > 2) st j  --> No results are returned (NOT correct) - Expect to find st
> > jude
> > > medical inc
> > > 3) st ju m --> No results are returned (NOT correct) - Expect to find
> st
> > > jude medical inc
> > > 4) st ju me --> result = st jude medical inc (works correctly)
> > > 5) st ju inc --> No results are returned (NOT correct) - Expect to find
> > st
> > > jude medical inc
> > >
> > > Is my field type definition correct? Or do I need to query differently?
> > >
> > > Thanks
> > >
> >
>

Re: Query differently or change fieldtype

[Ray Niu]

I found the conf minGramSize="2"，which will only create index with at least
2 chars，j will not match
also StandardTokenizerFactory will tokenize st j to st and j

2015年10月26日星期一，Brian Narsi  写道：

> I have the following field type on a field ClientName:
>
>  positionIncrementGap="100">
> 
> 
> 
>  maxGramSize="25"/>
> 
> 
> 
> 
> 
>   
>
>
> For data where
>
> ClientName = st jude medical inc
>
> When querying I get the following:
>
> 1) st --> result = st jude medical inc (works correctly)
> 2) st j  --> No results are returned (NOT correct) - Expect to find st jude
> medical inc
> 3) st ju m --> No results are returned (NOT correct) - Expect to find st
> jude medical inc
> 4) st ju me --> result = st jude medical inc (works correctly)
> 5) st ju inc --> No results are returned (NOT correct) - Expect to find st
> jude medical inc
>
> Is my field type definition correct? Or do I need to query differently?
>
> Thanks
>

Re: Should I install 4.x or 5.x? Book recommendations?

[Ray Niu]

I would also suggest to use Solr5, as there are lots of new features. We
are using 5.2.1 now, which is pretty stable.

2015-10-23 16:32 GMT-07:00 Shawn Heisey :

> On 10/23/2015 12:22 PM, Robert Hume wrote:
> > I'm investigating installing a new Solr deployment to be able to search
> > about two million documents (mostly HTML and PDF).
> >
> > QUESTIONS:
> >
> > A. Should I use Solr 4.x or 5.x?  My concerns are mostly to do with
> > support.  Is 5.x too new to be able to get good answers and advice from
> the
> > community?  Or should I stick with the latest 4.x release?
>
> I think the books were covered pretty well by Alexandre, so I won't go
> into that.
>
> I agree that you should use 5.x.  Once you get a particular major
> version deployed, upgrading to a new major version can be challenging.
> Upgrading to new minor versions is usually pretty easy, so go with the
> latest version.
>
> Additionally, it is highly unlikely that you will see any bug fixes for
> 4.x.  If a serious enough bug is found, a new 4.10.x version may get
> released, but it will have to be a HUGE bug, without a workaround.  I
> can guarantee that you won't see any performance improvements in 4.x.
> There will never be a 4.11.0 version.  4.x development is effectively
> dead.  Most effort is focused on 5.x, and some of it is focused on 6.0.
>
> There is always the possibility that 4.x might be more stable than 5.3.1
> ... but this is a very slim possibility.  A lot of effort is spent
> trying to make sure that all releases are as stable as possible, and
> frequently, especially when running SolrCloud, the newest version will
> be more stable than anything in the previous major version.  Stability
> is likely to improve with each new minor release.
>
> If 5.x were still at the 5.0 or 5.1 version, this might be different
> advice, but there have been four minor releases in the 5.x series.  Most
> of the bugs that are inevitable in any x.0 release have been fixed.
>
> Thanks,
> Shawn
>
>