Re: jconsole over jmx - should threads be visible?

2012-12-19 Thread Bernd Fehling
Hi Shawn, actually I use munin for monitoring but just checked with jvisualvm which also runs fine for remote monitoring. You might try the following: http://www.codefactorycr.com/java-visualvm-to-profile-a-remote-server.html You have to: - generate a policy file on the server to be monitored -

Re: Faceting on Dynamic fields

2012-12-19 Thread Erick Erickson
The base problem is you don't know what they are in advance. You could simply attach facet.field=name_0facet.field=name_1 Or is name variable too? If it is, you could query the index up-front and get all of the dynamic field names (the admin page does this) and assemble the above from the

Re: Solr Cloud 4.0 Production Ready?

2012-12-19 Thread Anirudha Jadhav
I am curious to know what issues you are facing with 3.5 and large indices. We use 3.5 with 200g indices with no issues until now -Ani On Tuesday, December 18, 2012, Luis Cappa Banda wrote: Any idea about when Solr 4.1 will be released? 2012/12/18 Otis Gospodnetic otis.gospodne...@gmail.com

Re: Solr atomic update of multi-valued field

2012-12-19 Thread Erick Erickson
FIrst question: Is the name field stored (stored=true)? If it isn't, that would explain your problems with that field. _all_ relevant fields (i.e. everything not a destination of a copyField) need to be stored for atomic updates to work. Your second problem I'm not sure about. I remember some

Re: Strange data-loss problem on one of our cores

2012-12-19 Thread Erick Erickson
Thanks for letting us know, and do bring let us know if you see the problem again. Erick On Tue, Dec 18, 2012 at 7:39 AM, John Nielsen j...@mcb.dk wrote: I build a solr version from the solr-4x branch yesterday and so far am unable to replicate the problems i had before. I am cautiously

Re: jconsole over jmx - should threads be visible?

2012-12-19 Thread Shawn Heisey
On 12/19/2012 1:39 AM, Bernd Fehling wrote: Hi Shawn, actually I use munin for monitoring but just checked with jvisualvm which also runs fine for remote monitoring. You might try the following: http://www.codefactorycr.com/java-visualvm-to-profile-a-remote-server.html You have to: - generate

Re: Solr atomic update of multi-valued field

2012-12-19 Thread Dikchant Sahi
Hi Erick, The name field is stored. I experience problem only when I update multiValued field with multiple values like, * field name=skills update=setsolr/field* * field name=skills update=setlucene/field* * * It works perfect, when I set single value for multiValued field like, *field

Re: java.lang.OutOfMemoryError and shard can‘t work

2012-12-19 Thread Jam Luo
OK, I will try to upgrade Oracle java and the lastest solr. thanks 2012/12/19 Shawn Heisey s...@elyograg.org On 12/18/2012 8:18 PM, Jam Luo wrote: I deployed a solr-4.0-beta cluster, 4 shard, 2 peers in a shard. A peer catch exception: 十二月 18, 2012 7:56:31 下午

RE: order question on solr multi value field

2012-12-19 Thread Dyer, James
I was noticing when I was looking this up in LIA yesterday a mention was made that when using a field list to return just some of the stored fields, that it still takes time to seek over the fields you don't want. So, if it has this strict guarantee about order, could a user potentially

Re: order question on solr multi value field

2012-12-19 Thread Yonik Seeley
On Tue, Dec 18, 2012 at 8:24 PM, Robert Muir rcm...@gmail.com wrote: I agree with James. Actually lucene tests will fail if a codec violates this. Actually it goes much deeper than this. From the lucene apis, when you call IndexReader.document() with your storedfieldVisitor, it must visit

Invalid version (expected 2, but 60) or the data in not in 'javabin'

2012-12-19 Thread Shahar Davidson
Hi, I'm encountering this error randomly when running a distributed facet. (i.e. I'm sending the exact same request, yet this does not reproduce consistently) I have about 180 shards that are being queried. It seems that when Solr distributes the request to the shards one , or perhaps more,

RE: Ensuring SpellChecker returns corrections which satisfy fq params for default OR query

2012-12-19 Thread Dyer, James
Let me try and get a better idea of what you're after. Is it that your users might query a combination of irrelevant terms and misspelled terms, so you want the ability to ignore the irrelevant terms but still get suggestions for the misspelled terms? For instance if someone wanted

Re: Solr 4.0 with Solr Cloud 404

2012-12-19 Thread Mark Miller
The default host for urls is /solr. Looks like you want to run on the root context. In Solr 4.1 this is best done by setting the host to / (either in solr.xml or with the hostContext sys prop). For 4.0 you have to use a host context of . to run on the root context. - Mark On Dec 19, 2012, at

Solr Cloud Architecture and DIH

2012-12-19 Thread Mark
We're currently running Solr 3.5 and our indexing process works as follows: We have a master that has a cron job to run a delta import via DIH every 5 minutes. The delta-import takes around 75 minutes to full complete, most of that is due to optimization after each delta and then the slaves

Re: if I only need exact search, does frequency/score matter?

2012-12-19 Thread Jie Sun
Hi Otis, I customized the Similarity class and add it through the end of schema.xml: ... ... solrQueryParser defaultOperator=OR/ similarity class=mypackage.NoTfSimilarity/ /schema and mypackage.NoTfSimilarity.java is like: public class NoTfSimilarity extends DefaultSimilarity { public

Solrj- QueryResponse's getbeans throwing bindingexception

2012-12-19 Thread Sadhana
Hi, I have a state column defined in schema.xml as field name=state type=string indexed=true stored=true multiValued=true/ The getbeans method of Queryrespoinse using solj API is throwing binding exception as follows..Probably the special characters like [ : / - might be

Re: if I only need exact search, does frequency/score matter?

2012-12-19 Thread Jie Sun
Hi Otis, here is the debug output on the query... seems all tf and idf indeed return 1.0f as I customized... I did not overwrite queryNorm or weight etc... see below. but the bottom line is that if my purpose is to reduce the frq file size, customize similarity seems wont help on that. I guess

Re: Ensuring SpellChecker returns corrections which satisfy fq params for default OR query

2012-12-19 Thread Nalini Kartha
Hi James, Yup the example you gave about sums it up. Reason we use an OR query is that we want the flexibility of every term not having to match but when it comes to corrections we want to be sure that the ones we pick will actually return results (we message the user with the corrected query so

Re: Solrj- QueryResponse's getbeans throwing bindingexception

2012-12-19 Thread Sadhana
field name=state type=string indexed=true stored=true multiValued=true/ Though the type is String its storing the datatype as List because the mutlitvalued=true in the schema.xml. So the issue got resolved after I change my datatype of State variable in the POJO from String to List. Thanks

RE: dataimport.properties not created/updated with solrcloud

2012-12-19 Thread adm1n
Well, I saw, that when I ran the full/delta i,port process on the 2nd, 3rd etc times I didn't so this exception any more. So I checked in my mysql queries log what's going on in mysql when I was running delta import process and I saw, that the queries got correct times on each delta-import

RE: Ensuring SpellChecker returns corrections which satisfy fq params for default OR query

2012-12-19 Thread Dyer, James
Instead of using spellcheck.collateParam.mm, try just setting spellcheck.maxResultsForSuggest to a very high value (you can use up to Integer.MAX_VALUE here). So long as the user gets fewer results that whatever this is set for, you will get suggestions (and collations if desired). I was

RE: dataimport.properties not created/updated with solrcloud

2012-12-19 Thread Dyer, James
Someone with more zookeeper knowledge than I have can better answer this, but there is code in place specifically for using DIH with SolrCloud to save the dataimport.properties file in an appropriate place. The default path is: /configs/{collection}/dataimport.properties I'm not sure which

ICUTokenizer labels number as Han character?

2012-12-19 Thread Tom Burton-West
Hello, Don't know if the Solr admin panel is lying, or if this is a wierd bug. The string: 1986年 gets analyzed by the ICUTokenizer with 1986 being identified as type:NUM and script:Han. Then the CJKBigram filter identifies 1986 as type:Num and script:Han and 年 as type:Single and script: Common.

Solr stats.facet on TrieField doesn't work

2012-12-19 Thread jefferyyuan
This seems an known issue: http://wiki.apache.org/solr/StatsComponent TrieFields has to use a precisionStep of -1 to avoid using UnInvertedField.java. Consider using one field for doing stats, and one for doing range facetting on. To fix this problem. and support dacet search on this field, I

Re: ICUTokenizer labels number as Han character?

2012-12-19 Thread Robert Muir
Your attachment didnt come through: I think the list strips them. Maybe just open a JIRA and attach your screenshots? or put them elsewhere and just include a link? As far as the ultimate behavior, I think its correct. Keep in mind tokens don't really get a script value: runs of untokenized text

Re: Solr Cloud Architecture and DIH

2012-12-19 Thread Shawn Heisey
On 12/19/2012 11:50 AM, Mark wrote: We have a master that has a cron job to run a delta import via DIH every 5 minutes. The delta-import takes around 75 minutes to full complete, most of that is due to optimization after each delta and then the slaves sync up. Our index is around 30 gigs so

Re: Intersect Circle is matching points way outside the radius ( Solr 4 Spatial)

2012-12-19 Thread Javi Molina
Hi David, Your latest response was lost in my inbox, I just realised it was there. You are right, I am using Open Layers, and even though I use the mercator projection, there are elements that not adhere to that projection, in particular the polygon that generates the circle and the scale

Putting more weight on particular column.

2012-12-19 Thread Prachi Phatak
We configured Enhanced search in our environment and while testing noticed below behavior: Just searching for P returns Enterprise ID which has P (Pankaj) as well as Resource Entity which has P (Sowmya), and the result lists Sowmya above Pankaj. * We are looking to limit the search just

Using SolrJ to update index with array fields

2012-12-19 Thread leonildo
Hello, I´m using solr 1.4 How can I update solr index creating fields that must be updated with arrays ? My code is: long arrayOfLong[] = new long [] {1,2,3} ; SolrInputDocument doc = new SolrInputDocument(); doc.setField(field.getFieldName(), arrayOfLong); when updating the following error

edismax request handler doesn't use defined qf

2012-12-19 Thread Jose Yadao
Hi, I wanted to put the fields I want to be searched in qf but it looks like edismax is not using it. I have the ff request handler made for testing edismax, used the sample xml to populate my solr 4.0 requestHandler name=test class=solr.SearchHandler lst name=defaults str

SolrCloud: only partial results returned

2012-12-19 Thread Lili
We set up SolrCloud with 2 shards and separate multiple zookeepers. The data added using http post with json in tutorial sample are not completely returned in query.However, if you send the same http post request again or shutdown solr instance and restart, the complete results will be

Re: Putting more weight on particular column.

2012-12-19 Thread Otis Gospodnetic
Hi, If I understand correctly, you want to search against a specific field - enterprise id. To do that just use something like enterpriseid:(keywords). Yes, you can sort using sort URL parameter. This stuff id on the Wiki and you can search it, too. :) Otis -- SOLR Performance Monitoring -

Re: edismax request handler doesn't use defined qf

2012-12-19 Thread Jack Krupansky
The white space is probably messing things up. Change: str name=qf features^1.0 /str to str name=qffeatures^1.0/str It SHOULDN'T be a problem, but... -- Jack Krupansky -Original Message- From: Jose Yadao Sent: Wednesday, December 19, 2012 6:32 PM To: solr-user@lucene.apache.org

Re: edismax request handler doesn't use defined qf

2012-12-19 Thread Jose Yadao
Thanks but that didn't work. Parsedquery doesn't show DisjunctionMaxQuery it's using the default field of select str name=parsedquerytext:car/str On Thu, Dec 20, 2012 at 9:38 AM, Jack Krupansky j...@basetechnology.comwrote: The white space is probably messing things up. Change: str

Re: edismax request handler doesn't use defined qf

2012-12-19 Thread Jack Krupansky
It may not be using that request handler. You are using the qt parameter, but have you changed the compatibility setting to enable it? See the comments in solrconfig. -- Jack Krupansky -Original Message- From: Jose Yadao Sent: Wednesday, December 19, 2012 8:44 PM To:

Re: edismax request handler doesn't use defined qf

2012-12-19 Thread Jose Yadao
Thanks! so using select was a old approach and was not the default since 3.6. On Thu, Dec 20, 2012 at 9:47 AM, Jack Krupansky j...@basetechnology.comwrote: It may not be using that request handler. You are using the qt parameter, but have you changed the compatibility setting to enable it? See

Improving the speed of Solr query over 16 million tweets

2012-12-19 Thread naryad
I use Solr (SolrCloud) to index and search my tweets. There are about 16 million tweets and the index size is approximately 3 GB. The tweets are indexed in real time as they come so that real time search is enabled. Currently I use lowercase field type for my tweet body field. For a single search

Re: Solr Cloud Architecture and DIH

2012-12-19 Thread Mikhail Khludnev
Hello Mark some of these questions has been touched recently, see below. On Wed, Dec 19, 2012 at 10:50 PM, Mark static.void@gmail.com wrote: We're currently running Solr 3.5 and our indexing process works as follows: . I also have the following questions. Does DIH work with Solr

Re: Improving the speed of Solr query over 16 million tweets

2012-12-19 Thread Alexandre Rafalovitch
KeywordTokenizerFactory does not actually break the text into tokens. It makes the whole field one whole token. Is that what you actually want? I would have thought that at least tweet body would be broken into words/tokens. Regards, Alex. Personal blog: http://blog.outerthoughts.com/

Re: Improving the speed of Solr query over 16 million tweets

2012-12-19 Thread naryad
You are completely right. I realized this, so is the only way to fix this is to redeclare the field as text_en or text_en_splitting and then delete all the documents and recreate the index? Or is there any other easy way? -- View this message in context:

Dynamic modification of field value

2012-12-19 Thread AlexeyK
Hi, Suppose the document stored in the index has fields A and B. What would be the best way to alter the value of B after the result set is available? The modified value of B is influenced by the value of A and also by some custom logic based on (custom) SolrCache. Can it be a custom function

Finding the last committed record in SOLR 4

2012-12-19 Thread Joe
I'm using SOLR 4 for an application, where I need to search the index soon after inserting records. I'm using the solrj code below to get the last ID in the index. However, I noticed that the last id I see when I execute a query through the solr web admin is often lagging behind this. And that

SolrTestCaseJ4 and searcher initialization

2012-12-19 Thread AlexeyK
Hi, I've written a unit test for a custom search component, which naturally extends the SolrTestCaseJ4. beforeClass() has initCore(), assertU(adoc()) and assertU(commit()) inside. The test creates a SolrQueryRequest via req() and runs h.query(request). In other words, nothing special. I see a