Re: How to get a list of servers per collection in sorlcloud using java api?
On Thu, Nov 22, 2012 at 7:20 PM, joe.cohe...@gmail.com joe.cohe...@gmail.com wrote: Thanks Rakudten. I had my question mis-phrased. What I need is being able to get the solr servers storing a collection by giving the zookeeper server as an input. something like: // returns a list of solr servers in the zookeeper ensemble that store the given collection ListString getServers(String zkhost, String collectionName) You can use ZKStateReader (#getClusterState) to get this info. -- Sami Siren
Re: SolrCloud: Very strange behavior when doing atomic updates or documents reindexation.
I think the problem is that even though you were able to work around the bug in the client solr still uses the xml format internally so the atomic update (with multivalued field) fails later down the stack. The bug you filed needs to be fixed to get the problem solved. On Thu, Nov 22, 2012 at 8:19 PM, Luis Cappa Banda luisca...@gmail.comwrote: Hello everyone. I´ve starting to seriously worry about with SolrCloud due an strange behavior that I have detected. The situation is this the following: *1.* SolrCloud with one shard and two Solr instances. *2.* Indexation via SolrJ with CloudServer and a custom BinaryLBHttpSolrServer that uses BinaryRequestWriter to execute correctly atomic updates. Check JIRA-4080 https://issues.apache.org/jira/browse/SOLR-4080?focusedCommentId=13498055page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13498055 *3.* An asynchronous proccess updates partially some document fields. After that operation I automatically execute a commit, so the index must be reloaded. What I have checked is that both using atomic updates or complete document reindexations* aleatory documents are not updated* *even if I saw debugging how the add() and commit() operations were executed correctly* *and without errors*. Has anyone experienced a similar behavior? Is it posible that if an index update operation didn´t finish and CloudSolrServer receives a new one this second update operation doesn´t complete? Thank you in advance. Regards, -- - Luis Cappa
Re: SolrCloud: Very strange behavior when doing atomic updates or documents reindexation.
It might even depend on the cluster layout! Let's say you have 2 shards (no replicas) if the doc belongs to the node you send it to so that it does not get forwarded to another node then the update should work and in case where the doc gets forwarded to another node the problem occurs. With replicas it could appear even more strange: the leader might have the doc right and the replica not. I only briefly looked at the bits that deal with this so perhaps there's something more involved. On Thu, Nov 22, 2012 at 8:29 PM, Luis Cappa Banda luisca...@gmail.comwrote: Hi, Sami! But isn´t strange that some documents were updated (atomic updates) correctly and other ones not? Can´t it be a more serious problem like some kind of index writer lock, or whatever? Regards, - Luis Cappa. 2012/11/22 Sami Siren ssi...@gmail.com I think the problem is that even though you were able to work around the bug in the client solr still uses the xml format internally so the atomic update (with multivalued field) fails later down the stack. The bug you filed needs to be fixed to get the problem solved. On Thu, Nov 22, 2012 at 8:19 PM, Luis Cappa Banda luisca...@gmail.com wrote: Hello everyone. I´ve starting to seriously worry about with SolrCloud due an strange behavior that I have detected. The situation is this the following: *1.* SolrCloud with one shard and two Solr instances. *2.* Indexation via SolrJ with CloudServer and a custom BinaryLBHttpSolrServer that uses BinaryRequestWriter to execute correctly atomic updates. Check JIRA-4080 https://issues.apache.org/jira/browse/SOLR-4080?focusedCommentId=13498055page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13498055 *3.* An asynchronous proccess updates partially some document fields. After that operation I automatically execute a commit, so the index must be reloaded. What I have checked is that both using atomic updates or complete document reindexations* aleatory documents are not updated* *even if I saw debugging how the add() and commit() operations were executed correctly* *and without errors*. Has anyone experienced a similar behavior? Is it posible that if an index update operation didn´t finish and CloudSolrServer receives a new one this second update operation doesn´t complete? Thank you in advance. Regards, -- - Luis Cappa -- - Luis Cappa
Re: SolrJ: atomic updates.
On Thu, Nov 15, 2012 at 11:51 AM, Luis Cappa Banda luisca...@gmail.comwrote:
Thread update:
When I use a simple:
*Map operation = new HashMap();*
Instead of:
*MapString, ListString operation = new HashMapString,
ListString();*
The result looks better, but it´s still wrong:
fieldName: [
[Value1, Value2]
],
However, ListString value is received as a simple String [Value1,
Value2]. In other words, SolrJ is internally executing a toString()
operation to the ListSring. Is impossible to update atomically a
multivalued field with a List of values in just one atomic update
operation?
Seems to be working fine here with HttpSolrServer / BinaryRequestWriter;
HashMap editTags = new HashMap();
editTags.put(set, new String[]{tag1,tag2,tag3});
doc = new SolrInputDocument();
doc.addField(id, unique);
doc.addField(tags_ss, editTags);
server.add(doc);
server.commit(true, true);
resp = server.query(q);
System.out.println(resp.getResults().get(0).getFirstValue(tags_ss));
prints tag1
ArrayListString as a value works the same way as String[].
When using xml (RequestWriter) I can see the problem that you are
describing, can you add a jira for that?
--
Sami SIren
Regards,
- Luis Cappa.
2012/11/15 Luis Cappa Banda luisca...@gmail.com
Hello everyone,
I´ve tested atomic updates via Ajax calls and now I´m starting with
atomic
updates via SolrJ... but the way I´m proceeding doesn´t seem to work
well.
Here is the snippet:
*SolrInputDocument do = ne SolrInputDocument();*
*doc.addField(id, myId);*
*
*
*MapString, ListString operation = new HashMapString,
ListString();*
*operation.put(set, [[a list of String elements]]); // I want a set
operation to override field values.*
*doc.addField(fieldName, operation);*
*
*
*cloudSolrServer.add(doc); // Atomic update operation.*
And after updating the resultant doc is as follows:
*doc: {*
*
*
*...*
*
*
*fieldName: [ {set=values}*
*],*
*
*
*...*
*
*
*}*
In other words, the map which includes the set operation and the field
values is String formatted and that String is used to update the field,
:-/
What is the correct way to update just one or more fields with SolrJ?
Regards,
--
- Luis Cappa
--
- Luis Cappa
Re: SolrJ: atomic updates.
Actually it seems that xml/binary request writers only behave differently
when using array[] as the value. if I use ArrayList it also works with the
xml format (4.1 branch). Still it's annoying that the two request writers
behave differently so I guess it's worth adding the jira anyway.
The Affects version should be 4.0.
On Thu, Nov 15, 2012 at 1:42 PM, Luis Cappa Banda luisca...@gmail.comwrote:
Hello, Sami.
It will be the first issue that I open so, should I create it under Solr
4.0 version or in Solr 4.1.0 one?
Thanks,
- Luis Cappa.
2012/11/15 Sami Siren ssi...@gmail.com
On Thu, Nov 15, 2012 at 11:51 AM, Luis Cappa Banda luisca...@gmail.com
wrote:
Thread update:
When I use a simple:
*Map operation = new HashMap();*
Instead of:
*MapString, ListString operation = new HashMapString,
ListString();*
The result looks better, but it´s still wrong:
fieldName: [
[Value1, Value2]
],
However, ListString value is received as a simple String [Value1,
Value2]. In other words, SolrJ is internally executing a toString()
operation to the ListSring. Is impossible to update atomically a
multivalued field with a List of values in just one atomic update
operation?
Seems to be working fine here with HttpSolrServer / BinaryRequestWriter;
HashMap editTags = new HashMap();
editTags.put(set, new String[]{tag1,tag2,tag3});
doc = new SolrInputDocument();
doc.addField(id, unique);
doc.addField(tags_ss, editTags);
server.add(doc);
server.commit(true, true);
resp = server.query(q);
System.out.println(resp.getResults().get(0).getFirstValue(tags_ss));
prints tag1
ArrayListString as a value works the same way as String[].
When using xml (RequestWriter) I can see the problem that you are
describing, can you add a jira for that?
--
Sami SIren
Regards,
- Luis Cappa.
2012/11/15 Luis Cappa Banda luisca...@gmail.com
Hello everyone,
I´ve tested atomic updates via Ajax calls and now I´m starting with
atomic
updates via SolrJ... but the way I´m proceeding doesn´t seem to work
well.
Here is the snippet:
*SolrInputDocument do = ne SolrInputDocument();*
*doc.addField(id, myId);*
*
*
*MapString, ListString operation = new HashMapString,
ListString();*
*operation.put(set, [[a list of String elements]]); // I want a
set
operation to override field values.*
*doc.addField(fieldName, operation);*
*
*
*cloudSolrServer.add(doc); // Atomic update operation.*
And after updating the resultant doc is as follows:
*doc: {*
*
*
*...*
*
*
*fieldName: [ {set=values}*
*],*
*
*
*...*
*
*
*}*
In other words, the map which includes the set operation and the
field
values is String formatted and that String is used to update the
field,
:-/
What is the correct way to update just one or more fields with SolrJ?
Regards,
--
- Luis Cappa
--
- Luis Cappa
--
- Luis Cappa
Re: SolrJ: atomic updates.
Try setting Request writer to binary like this:
server.setParser(new BinaryResponseParser());
server.setRequestWriter(new BinaryRequestWriter());
Or then instead of string array use ArrayListString() that contains your
strings as the value for the map
On Thu, Nov 15, 2012 at 3:58 PM, Luis Cappa Banda luisca...@gmail.comwrote:
Hi, Sami.
Doing some tests I´ve used the same code as you and did a quick execution:
*HttpSolrServer server = new HttpSolrServer(
http://localhost:8080/solrserver/core1
http://localhost:10080/newscover_es/items_es
);*
* *
* try {*
* *
* HashMap editTags = new HashMap();*
* editTags.put(set, new String[]{tag1,tag2,tag3});*
* doc.addField(myField, editTags);*
* server.add(doc);*
* server.commit(true, true);*
* *
* } catch (SolrServerException e) {*
* e.printStackTrace();*
* } catch (IOException e) {*
* e.printStackTrace();*
* }*
*
*
*
*
And the resultant doc is wrong:
response
lst name=responseHeader
int name=status0/int
int name=QTime1/int
lst name=params
str name=q*:*/str
/lst
/lst
result name=response numFound=1 start=0
doc
str name=i_id50a0f1f90cf226fb5677fe13/str
arr name=myField
*str[Ljava.lang.String;@7d5e90cb/str --- toString() from String[]
array.*
/arr
long name=_version_1418710023780958208/long
/doc
/result
/response
So is definetely executing toString() method in both *HttpSolrServer *and *
CloudServer*. Tips:
- I´m using* Solrj 4.0.0 *artifact version in a *Maven *project.
- The fields to be updated are *dynamicFields*.
- I´m using Java jdk6.
Alternatives:
- I´m doing something wrong and I´m so stupid that I can´t see it, :-(
- The way I update fields is not the correct one.
- There is a general bug with atomic updates via SolrJ.
Regards,
- Luis Cappa.
2012/11/15 Luis Cappa Banda luisca...@gmail.com
I´ll have a look to Solr source code and try to fix the bug. If I succeed
I´ll update JIRA issue with it, :-)
2012/11/15 Sami Siren ssi...@gmail.com
Actually it seems that xml/binary request writers only behave
differently
when using array[] as the value. if I use ArrayList it also works with
the
xml format (4.1 branch). Still it's annoying that the two request
writers
behave differently so I guess it's worth adding the jira anyway.
The Affects version should be 4.0.
On Thu, Nov 15, 2012 at 1:42 PM, Luis Cappa Banda luisca...@gmail.com
wrote:
Hello, Sami.
It will be the first issue that I open so, should I create it under
Solr
4.0 version or in Solr 4.1.0 one?
Thanks,
- Luis Cappa.
2012/11/15 Sami Siren ssi...@gmail.com
On Thu, Nov 15, 2012 at 11:51 AM, Luis Cappa Banda
luisca...@gmail.com
wrote:
Thread update:
When I use a simple:
*Map operation = new HashMap();*
Instead of:
*MapString, ListString operation = new HashMapString,
ListString();*
The result looks better, but it´s still wrong:
fieldName: [
[Value1, Value2]
],
However, ListString value is received as a simple String
[Value1,
Value2]. In other words, SolrJ is internally executing a
toString()
operation to the ListSring. Is impossible to update atomically a
multivalued field with a List of values in just one atomic update
operation?
Seems to be working fine here with HttpSolrServer /
BinaryRequestWriter;
HashMap editTags = new HashMap();
editTags.put(set, new String[]{tag1,tag2,tag3});
doc = new SolrInputDocument();
doc.addField(id, unique);
doc.addField(tags_ss, editTags);
server.add(doc);
server.commit(true, true);
resp = server.query(q);
System.out.println(resp.getResults().get(0).getFirstValue(tags_ss));
prints tag1
ArrayListString as a value works the same way as String[].
When using xml (RequestWriter) I can see the problem that you are
describing, can you add a jira for that?
--
Sami SIren
Regards,
- Luis Cappa.
2012/11/15 Luis Cappa Banda luisca...@gmail.com
Hello everyone,
I´ve tested atomic updates via Ajax calls and now I´m starting
with
atomic
updates via SolrJ... but the way I´m proceeding doesn´t seem to
work
well.
Here is the snippet:
*SolrInputDocument do = ne SolrInputDocument();*
*doc.addField(id, myId);*
*
*
*MapString, ListString operation = new HashMapString,
ListString();*
*operation.put(set, [[a list of String elements]]); // I
want a
set
operation to override field values.*
*doc.addField(fieldName, operation);*
*
*
*cloudSolrServer.add(doc); // Atomic update operation.*
And after updating the resultant doc
Re: CloudSolrServer and LBHttpSolrServer: setting BinaryResponseParser and BinaryRequestWriter.
hi, did you try setting your values in a List, for example ArrayList it should work when you use that even without specifying reguest-/response writer. -- Sami Siren On Thu, Nov 15, 2012 at 4:56 PM, Luis Cappa Banda luisca...@gmail.comwrote: Hello, I´ve found what It seems to be a bug JIRA-SOLR4080 https://issues.apache.org/jira/browse/SOLR-4080?focusedCommentId=13498055page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13498055 with CloudSolrServer during atomic updates via SolrJ. Thanks to Sami I detected that the problem could be solved setting BinaryResponseParser as Parser and BinaryRequestWriter as Request writer. That workis with HttpSolrServer, but CloudSolrServer or LBHttpSolrServer hasn´t got any method to set them. Is there a way to set both parsers to a CloudSolrServer instance? Do you know if maybe it could be configured insed solrconfig.xml? Thanks a lot. -- - Luis Cappa
Re: SolrJ 4.0 Beta maxConnectionsPerHost
On Wed, Oct 10, 2012 at 12:02 AM, Briggs Thompson w.briggs.thomp...@gmail.com wrote: *Sami* The client IS instantiated only once and not for every request. I was curious if this was part of the problem. Do I need to re-instantiate the object for each request made? No, it is expensive if you instantiate the client every time. When the client seems to be hanging, can you still access the Solr instance normally and execute updates/searches from other clients? -- Sami Siren
Re: SolrJ 4.0 Beta maxConnectionsPerHost
On Wed, Oct 10, 2012 at 5:36 PM, Briggs Thompson w.briggs.thomp...@gmail.com wrote: There are other updates that happen on the server that do not fail, so the answer to your question is yes. The other updates are using solrj or something else? It would be helpful if you could prepare a simple java program that uses solrj to demonstrate the problem. Based on the available information it is really difficult try to guess what's happening. -- Sami Siren
Re: SolrJ 4.0 Beta maxConnectionsPerHost
On Tue, Oct 9, 2012 at 4:52 AM, Briggs Thompson w.briggs.thomp...@gmail.com wrote: I am running into an issue of a multithreaded SolrJ client application used for indexing is getting into a hung state. I responded to a separate thread earlier today with someone that had the same error, see http://lucene.472066.n3.nabble.com/SolrJ-IOException-td4010026.html I did some digging and experimentation and found something interesting. When starting up the application, I see the following in Solr logs: Creating new http client, config:maxConnections=200maxConnectionsPerHost=8 Do you see this in Solr log or at the client end, the default parameters for http client is the following: max total: 128 max per host: 32 The way I instantiate the HttpSolrServer through SolrJ is like the following HttpSolrServer solrServer = new HttpSolrServer(serverUrl); solrServer.setConnectionTimeout(1000); solrServer.setDefaultMaxConnectionsPerHost(100); solrServer.setMaxTotalConnections(100); solrServer.setParser(new BinaryResponseParser()); solrServer.setRequestWriter(new BinaryRequestWriter()); Do you instantiate the client just once and not for every request you make? If you, for some reason, need to recreate the client again and again you must call shutdown() when you're done with it to release the resources. It seems as though the maxConnections and maxConnectionsPerHost are not actually getting set. Anyone seen this problem or have an idea how to resolve? I did some experiments with the solrj and from what it looked like it seems to respect the values that you set. -- Sami Siren
Re: Change config to use port 8080 instead of port 8983
i just tried this with tomcat and the props work for me. Did you wipe out your zoo_data before starting with the additional system properties? here's how i ran it: JAVA_OPTS=-DzkRun -DnumShards=1 -Djetty.port=8080 -Dbootstrap_conf=true -Dhost=127.0.0.1 bin/catalina.sh run -- Sami Siren On Thu, Sep 27, 2012 at 9:47 PM, JesseBuesking jessebuesk...@gmail.com wrote: I've set the JAVA_OPTS you mentioned (Djetty.port and Dhost), but zookeeper still says that the node runs on port 8983 (clusterstate.json is the same). Would you happen to have any other suggestions that I could try? -- View this message in context: http://lucene.472066.n3.nabble.com/Change-config-to-use-port-8080-instead-of-port-8983-tp4010663p4010805.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Change config to use port 8080 instead of port 8983
On Thu, Sep 27, 2012 at 4:46 AM, JesseBuesking jessebuesk...@gmail.com wrote:
What configuration file sets the port to 8983 when running SolrCloud? I'd
like to change this to port 8080 since I'm running tomcat.
These are the JAVA_OPTS that I pass to tomcat when starting the server:
-DzkHost=/*xxx.xx.xxx.xxx*/:52001
-Dbootstrap_confdir=/usr/share/solr/collection-01/conf/
-Dcollection.configName=collection-01-config
-DshardId=collection-01-shard-01
This is the clusterstate.json that is generated by zookeeper:
{collection-01:{collection-01-shard-01:{dm-search-sr-01:8983_solr_core-01:{
shard:collection-01-shard-01,
roles:null,
leader:true,
state:active,
core:core-01,
collection:collection-01,
node_name:dm-search-sr-01:8983_solr,
base_url:http://dm-search-sr-01:8983/solr
Also, I see that the node_name/base_url are using a name dm-search-sr-01
instead of the ip address associated to the server. Is there a way to set
this to the ipaddress instead?
If you use the example configuration then take a look at solr.xml. In
it's current state it accepts port in system property called
jetty.port and the hostname or in your case the ip in host.
So adding -Djetty.port=xxx -Dhost=yyy to you JAVA_OPTS allow you to
set them when using the default solr.xml.
--
Sami Siren
Re: Solrcloud not reachable and after restart just a no servers hosting shard
hi, Can you share a little bit more about your configuration: how many shards, # of replicas, how does your clusterstate.json look like, anything suspicious in the logs? -- Sami Siren On Mon, Sep 24, 2012 at 11:13 AM, Daniel Brügge daniel.brue...@gmail.com wrote: Hi, I am running Solrcloud 4.0-BETA and during the weekend it 'crashed' somehow, so that it wasn't reachable. CPU load was 100%. After a restart i couldn't access the data it just telled me: no servers hosting shard Is there a way to get the data back? Thanks regards Daniel
Re: Nodes cannot recover and become unavailable
Hi, I am having troubles understanding the reason for that NPE. First you could try removing the line #102 in HttpClientUtility so that logging does not prevent creation of the http client in SyncStrategy. -- Sami Siren On Wed, Sep 19, 2012 at 5:29 PM, Markus Jelsma markus.jel...@openindex.io wrote: Hi, Since the 2012-09-17 11:10:41 build shards start to have trouble coming back online. When i restart one node the slices on the other nodes are throwing exceptions and cannot be queried. I'm not sure how to remedy the problem but stopping a node or restarting it a few times seems to help it. The problem is when i restart a node, and it happens, i must not restart another node because that may trigger other slices becoming unavailable. Here are some parts of the log: 2012-09-19 14:13:18,149 ERROR [solr.cloud.RecoveryStrategy] - [RecoveryThread] - : Recovery failed - trying again... core=oi_i 2012-09-19 14:13:25,818 WARN [solr.cloud.RecoveryStrategy] - [main-EventThread] - : Stopping recovery for zkNodeName=nl10.host:8080_solr_oi_icore=oi_i 2012-09-19 14:13:44,497 WARN [solr.cloud.RecoveryStrategy] - [Thread-4] - : Stopping recovery for zkNodeName=nl10.host:8080_solr_oi_jcore=oi_j 2012-09-19 14:14:00,321 ERROR [solr.cloud.RecoveryStrategy] - [RecoveryThread] - : Error while trying to recover. core=oi_i:org.apache.solr.common.SolrException: We are not the leader at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:402) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:182) at org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:199) at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:388) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:220) 2012-09-19 14:14:00,321 ERROR [solr.cloud.RecoveryStrategy] - [RecoveryThread] - : Recovery failed - trying again... core=oi_i 2012-09-19 14:14:00,321 ERROR [solr.cloud.RecoveryStrategy] - [RecoveryThread] - : Recovery failed - max retries exceeded. core=oi_i 2012-09-19 14:14:00,321 ERROR [solr.cloud.RecoveryStrategy] - [RecoveryThread] - : Recovery failed - I give up. core=oi_i 2012-09-19 14:14:00,333 WARN [solr.cloud.RecoveryStrategy] - [RecoveryThread] - : Stopping recovery for zkNodeName=nl10.host:8080_solr_oi_icore=oi_i ERROR [solr.cloud.SyncStrategy] - [main-EventThread] - : Sync request error: java.lang.NullPointerException ERROR [solr.cloud.SyncStrategy] - [main-EventThread] - : http://nl10.host:8080/solr/oi_i/: Could not tell a replica to recover:java.lang.NullPointerException at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:305) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:102) at org.apache.solr.client.solrj.impl.HttpSolrServer.init(HttpSolrServer.java:155) at org.apache.solr.client.solrj.impl.HttpSolrServer.init(HttpSolrServer.java:128) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:262) at org.apache.solr.cloud.SyncStrategy.requestRecovery(SyncStrategy.java:272) at org.apache.solr.cloud.SyncStrategy.syncToMe(SyncStrategy.java:203) at org.apache.solr.cloud.SyncStrategy.syncReplicas(SyncStrategy.java:125) at org.apache.solr.cloud.SyncStrategy.sync(SyncStrategy.java:87) at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:169) at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:158) at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:102) at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:275) at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:326) at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:159) at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:158) at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:102) at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:56) at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:131) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:526) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) ERROR [apache.zookeeper.ClientCnxn] - [main-EventThread] - : Error while calling watcher java.lang.NullPointerException at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:139) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:526) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) ERROR
Re: Nodes cannot recover and become unavailable
also, did you re create the cluster after upgrading to a newer version? I believe there were some changes made to the clusterstate.json recently that are not backwards compatible. -- Sami Siren On Wed, Sep 19, 2012 at 6:21 PM, Sami Siren ssi...@gmail.com wrote: Hi, I am having troubles understanding the reason for that NPE. First you could try removing the line #102 in HttpClientUtility so that logging does not prevent creation of the http client in SyncStrategy. -- Sami Siren On Wed, Sep 19, 2012 at 5:29 PM, Markus Jelsma markus.jel...@openindex.io wrote: Hi, Since the 2012-09-17 11:10:41 build shards start to have trouble coming back online. When i restart one node the slices on the other nodes are throwing exceptions and cannot be queried. I'm not sure how to remedy the problem but stopping a node or restarting it a few times seems to help it. The problem is when i restart a node, and it happens, i must not restart another node because that may trigger other slices becoming unavailable. Here are some parts of the log: 2012-09-19 14:13:18,149 ERROR [solr.cloud.RecoveryStrategy] - [RecoveryThread] - : Recovery failed - trying again... core=oi_i 2012-09-19 14:13:25,818 WARN [solr.cloud.RecoveryStrategy] - [main-EventThread] - : Stopping recovery for zkNodeName=nl10.host:8080_solr_oi_icore=oi_i 2012-09-19 14:13:44,497 WARN [solr.cloud.RecoveryStrategy] - [Thread-4] - : Stopping recovery for zkNodeName=nl10.host:8080_solr_oi_jcore=oi_j 2012-09-19 14:14:00,321 ERROR [solr.cloud.RecoveryStrategy] - [RecoveryThread] - : Error while trying to recover. core=oi_i:org.apache.solr.common.SolrException: We are not the leader at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:402) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:182) at org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:199) at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:388) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:220) 2012-09-19 14:14:00,321 ERROR [solr.cloud.RecoveryStrategy] - [RecoveryThread] - : Recovery failed - trying again... core=oi_i 2012-09-19 14:14:00,321 ERROR [solr.cloud.RecoveryStrategy] - [RecoveryThread] - : Recovery failed - max retries exceeded. core=oi_i 2012-09-19 14:14:00,321 ERROR [solr.cloud.RecoveryStrategy] - [RecoveryThread] - : Recovery failed - I give up. core=oi_i 2012-09-19 14:14:00,333 WARN [solr.cloud.RecoveryStrategy] - [RecoveryThread] - : Stopping recovery for zkNodeName=nl10.host:8080_solr_oi_icore=oi_i ERROR [solr.cloud.SyncStrategy] - [main-EventThread] - : Sync request error: java.lang.NullPointerException ERROR [solr.cloud.SyncStrategy] - [main-EventThread] - : http://nl10.host:8080/solr/oi_i/: Could not tell a replica to recover:java.lang.NullPointerException at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:305) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:102) at org.apache.solr.client.solrj.impl.HttpSolrServer.init(HttpSolrServer.java:155) at org.apache.solr.client.solrj.impl.HttpSolrServer.init(HttpSolrServer.java:128) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:262) at org.apache.solr.cloud.SyncStrategy.requestRecovery(SyncStrategy.java:272) at org.apache.solr.cloud.SyncStrategy.syncToMe(SyncStrategy.java:203) at org.apache.solr.cloud.SyncStrategy.syncReplicas(SyncStrategy.java:125) at org.apache.solr.cloud.SyncStrategy.sync(SyncStrategy.java:87) at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:169) at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:158) at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:102) at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:275) at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:326) at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:159) at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:158) at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:102) at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:56) at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:131) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:526) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) ERROR [apache.zookeeper.ClientCnxn] - [main-EventThread] - : Error while calling watcher
Re: Solr 4.0 BETA Replication problems on Tomcat
I opened SOLR-3789. As a workaround you can remove str name=compressioninternal/str from the config and it should work. -- Sami Siren On Wed, Sep 5, 2012 at 5:58 AM, Ravi Solr ravis...@gmail.com wrote: Hello, I have a very simple setup one master and one slave configured as below, but replication keeps failing with stacktrace as shown below. Note that 3.6 works fine on the same machines so I am thinking that Iam missing something in configuration with regards to solr 4.0...can somebody kindly let me know if Iam missing something ? I am running SOLR 4.0 on Tomcat-7.0.29 with Java6. FYI I never has any problem with SOLR on glassfish, this is the first time Iam using it on Tomcat On Master requestHandler name=/replication class=solr.ReplicationHandler lst name=master str name=replicateAftercommit/str str name=replicateAfteroptimize/str str name=confFilesschema.xml,stopwords.txt,synonyms.txt/str str name=commitReserveDuration00:00:10/str /lst /requestHandler On Slave requestHandler name=/replication class=solr.ReplicationHandler lst name=slave str name=masterUrlhttp://testslave:8080/solr/mycore/replication/str str name=pollInterval00:00:50/str str name=compressioninternal/str str name=httpConnTimeout5000/str str name=httpReadTimeout1/str /lst /requestHandler Error 22:44:10WARNING SnapPuller Error in fetching packets java.util.zip.ZipException: unknown compression method at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:147) at org.apache.solr.common.util.FastInputStream.readWrappedStream(FastInputStream.java:79) at org.apache.solr.common.util.FastInputStream.refill(FastInputStream.java:88) at org.apache.solr.common.util.FastInputStream.read(FastInputStream.java:124) at org.apache.solr.common.util.FastInputStream.readFully(FastInputStream.java:149) at org.apache.solr.common.util.FastInputStream.readFully(FastInputStream.java:144) at org.apache.solr.handler.SnapPuller$FileFetcher.fetchPackets(SnapPuller.java:1024) at org.apache.solr.handler.SnapPuller$FileFetcher.fetchFile(SnapPuller.java:985) at org.apache.solr.handler.SnapPuller.downloadIndexFiles(SnapPuller.java:627) at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:331) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:297) at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:175) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) 22:44:10SEVERE ReplicationHandler SnapPull failed :org.apache.solr.common.SolrException: Unable to download _3_Lucene40_0.tip completely. Downloaded 0!=170 at org.apache.solr.handler.SnapPuller$FileFetcher.cleanup(SnapPuller.java:1115) at org.apache.solr.handler.SnapPuller$FileFetcher.fetchFile(SnapPuller.java:999) at org.apache.solr.handler.SnapPuller.downloadIndexFiles(SnapPuller.java:627) at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:331) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:297) at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:175) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662)
Re: JIRA Issue : SOLR-3771
Based on the stack trace it seems that DIH uses URLConnection. You might want to try setting the proxy related system properties for the jvm that runs Solr: http://docs.oracle.com/javase/6/docs/technotes/guides/net/proxies.html -- Sami Siren On Fri, Aug 31, 2012 at 9:58 AM, Molala, Nagaraj (GE Capital, Non-GE) nagaraj.mol...@ge.com wrote: GE INTERNAL Hi, Is there any option in the configuration to add proxy setting in the rss-data-config.xml file or any configuration in solr?. Basically mentioned rss url is able to parse with standalone java program using proxy setting. while trying to index with the Solr using dataImport, its giving the UnknownHostException exception. I am attaching the config files. ?xml version=1.0 encoding=UTF-8 ? dataConfig dataSource type=URLDataSource / document entity name=IndustryMonitor pk=link url=company URL comes here .. processor=XPathEntityProcessor forEach=/channel | /channel/item transformer=DateFormatTransformer field column=title xpath=/rss/channel/item/title / field column=author xpath=/rss/channel/item/author / field column=link xpath=/rss/channel/item/link / /entity /document /dataConfig Warm Regards, Nagaraj Molala Cell: +91 8447016501 From: Jack Krupansky (JIRA) [j...@apache.org] Sent: Friday, August 31, 2012 4:28 AM To: Nagaraj M Subject: [jira] [Commented] (SOLR-3771) While using RSS indexing from Solr, we are getting error Caused by: java.net.UnknownHostException indexing fail. [ https://issues.apache.org/jira/browse/SOLR-3771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13445389#comment-13445389 ] Jack Krupansky commented on SOLR-3771: -- Any support questions should be directed to the Solr user mailing list: solr-user@lucene.apache.org It still doesn't sound as if there is any evidence being presented here of a bug in Solr. While using RSS indexing from Solr, we are getting error Caused by: java.net.UnknownHostException indexing fail. Key: SOLR-3771 URL: https://issues.apache.org/jira/browse/SOLR-3771 Project: Solr Issue Type: Bug Components: search Affects Versions: 3.6 Environment: Solr Search engine using RSS data feeding Reporter: Nagaraj Molala Attachments: rss-data-config.xml, schema.xml, solrconfig.xml we are getting below error. Please give us the solution as this is a show stopper for our application. Attached the config files for your reference. https://issues.apache.org/jira/browse/SOLR 2:51 PM Caused by: java.net.UnknownHostException: xx.abcd.abcd.com at java.net.PlainSocketImpl.connect(Unknown Source) at java.net.SocksSocketImpl.connect(Unknown Source) at java.net.Socket.connect(Unknown Source) at sun.net.NetworkClient.doConnect(Unknown Source) at sun.net.www.http.HttpClient.openServer(Unknown Source) at sun.net.www.http.HttpClient.openServer(Unknown Source) at sun.net.www.http.HttpClient.init(Unknown Source) at sun.net.www.http.HttpClient.New(Unknown Source) at sun.net.www.http.HttpClient.New(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Sour ce) at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown So urce) at org.apache.solr.handler.dataimport.URLDataSource.getData(URLDataSourc e.java:97) ... 13 more -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira Warm Regards, Nagaraj Molala Contractor GE Capital – Americas T : +91 120 4675233 M : +91 8447016501. E : nagaraj.mol...@ge.com D-195 Sector 63 Noida, UP 201306, INDIA
Re: Updating document with the Solr Java API
On Tue, Jul 31, 2012 at 5:16 PM, Jonatan Fournier jonatan.fourn...@gmail.com wrote: Hi, What is the Java syntax to create an update document? Currently to update a field in a document with solrj you need to do something like this: doc = new SolrInputDocument(); doc.addField(id, unique); doc.addField(_version_, version); // this is needed only if optimistic locking is to be used HashMapString, Object value = new HashMapString, Object(); value.put(set,100); doc.addField(price_f, value); -- Sami Siren
Re: Problem with Solr 4.0-ALPHA and JSON response
On Fri, Jul 27, 2012 at 6:32 PM, Federico Valeri fedeval...@gmail.com wrote: Hi all, Hi, I'm new to Solr, I have a problem with JSON format, this is my Java client code: The java client (SolrServer) can only operate with xml or javabin format. If you need to get the json response from Solr by using java you could just use a http client directly and bypass the solr client. Now the problem is that I recieve the response but it doesn't trigger the javascript callback function. I see wt=javabin in SolrCore.execute log, even if I set wt=json in paramters, is this normal? yes, to control the format used by the client there's a method HttpSolrServer#setParser that set's the client parser (that also overrides the wt param when the request is made) -- Sami Siren
Re: LeaderElection bugfix
On Wed, Jun 27, 2012 at 10:32 AM, Trym R. Møller t...@sigmat.dk wrote: Hi Hi, The behaviour of this can be verified using the below test in the org.apache.solr.cloud.LeaderElectionIntegrationTest Can you reproduce the failure in your test every time or just rarely? I added the test method to LeaderElectionIntegrationTest and ran it few times but I can't get it to fail. -- Sami Siren
Re: LeaderElection bugfix
ok,
I see what you mean. Looks to me that you're right. I am not too
familiar with the LeaderElector so I'll let Mark take a second look.
--
Sami Siren
On Wed, Jun 27, 2012 at 11:32 AM, Trym R. Møller t...@sigmat.dk wrote:
Hi Sami
Regarding 2) A simple way to inspect the number of watchers, is to add an
error log statement to the process method of the watcher
public void process(WatchedEvent event) {
log.error(seq + watcher received event: + event);
and see that the number of logs doubles for each call to
containerMap.get(replicaPort).getZkController().getZkClient().getSolrZooKeeper().pauseCnxn(2000);
Best regards Trym
Den 27-06-2012 10:14, Trym R. Møller skrev:
Hi Sami
Thanks for your rapid reply.
Regarding 1) This seems to be time dependent but it is seen on my local
windows running the unit test and on a linux server running Solr.
Regarding 2) The test does not show the number of Watchers are increasing,
but this can be observed either by dumping the memory from the jvm or by
looking at the debug statements (if debug is enabled).
I don't know how to make assert statements regarding the number of
watchers in zookeeper, so the test is not quite informative, but more
confirming that the fix doesn't destroy anything.
Best regards Trym
Den 27-06-2012 10:06, Sami Siren skrev:
On Wed, Jun 27, 2012 at 10:32 AM, Trym R. Møller t...@sigmat.dk
wrote:
Hi
Hi,
The behaviour of this can be verified using the below test in the
org.apache.solr.cloud.LeaderElectionIntegrationTest
Can you reproduce the failure in your test every time or just rarely?
I added the test method to LeaderElectionIntegrationTest and ran it
few times but I can't get it to fail.
--
Sami Siren
Re: SolrCloud and split-brain
On Sat, Jun 16, 2012 at 5:33 AM, Otis Gospodnetic otis_gospodne...@yahoo.com wrote: And here is one more Q. * Imagine a client is adding documents and, for simplicity, imagine SolrCloud routes all these documents to the same shard, call it S. * Imagine that both the 7-node and the 3-node partition end up with a complete index and thus both accept updates. According to comments from Mark this (having two functioning sides) is not possible... only one side _can_ continue functioning (taking in updates). Depending on how the shards are deployed over the nodes a side may still not accept updates (even if that side has a working zk setup). Now imagine if the client sending documents for indexing happened to be sending documents to 2 nodes, say in round-robin fashion. In my understanding all updates are routed through a shard leader. -- Sami Siren
Re: How to update one field without losing the others?
On Mon, Jun 18, 2012 at 5:03 PM, Kai Gülzau kguel...@novomind.com wrote:
I'm currently playing around with a branch 4x Version
(https://builds.apache.org/job/Solr-4.x/5/) but I don't get field updates to
work.
A simple GET testrequest
http://localhost:8983/solr/master/update/json?stream.body={add:{doc:{ukey:08154711,type:1,nbody:{set:mycontent
results in
{
ukey:08154711,
type:1,
nbody:{set=mycontent}}]
}
All fields are stored.
ukey is the unique key :-)
type is a required field.
nbody is a solr.TextField.
With the Solr example (4.x), the following seems to work:
URL=http://localhost:8983/solr/update
curl $URL?commit=true -H 'Content-type:application/json' -d '{ add:
{ doc: { id: id, title: test, price_f: 10 }}}'
curl $URL?commit=true -H 'Content-type:application/json' -d '{ add:
{ doc: { id: id, price_f: {set: 5'
If you are using solrj then there's a junit test method,
testUpdateField(), that does something similar:
http://svn.apache.org/viewvc/lucene/dev/trunk/solr/solrj/src/test/org/apache/solr/client/solrj/SolrExampleTests.java?view=markup
--
Sami Siren
Re: SolrJ indexing pdf documents
On Sat, Jun 16, 2012 at 5:59 PM, 12rad prama.an...@gmail.com wrote: Hi, I'm new to SolrJ. Hi and welcome! Here I are the steps I followed to write an application to index pdf documents to fresh solr3.6 1 -In Schema.xml: I added the fields I wanted indexed and changed stored = true. 2 - Started Solr using java -jar start.jar from the /example dir in Solr 3.6 3- In my application I start the server solrServer = new CommonsHttpSolrServer(url); solrServer.setParser(new XMLResponseParser()); 4 - i index the data like this: ContentStreamUpdateRequest index = new ContentStreamUpdateRequest(/update/extract); index.setParam(literal.id, doc); index.setParam(CommonParams.STREAM_FILE, /location/x.pdf); index.setParam(CommonParams.STREAM_CONTENTTYPE, application/pdf); index.setParam(UpdateParams.COMMIT, true); Are you sending the request to solr? ie something like solrServer.request(index); Here's a simple snippet that sends a pdf to be indexed in solr (3.6): SolrServer server = ... ContentStreamUpdateRequest req = new ContentStreamUpdateRequest( /update/extract); req.setParam(literal.id, doc); req.addFile(new File( solr/contrib/dataimporthandler-extras/src/test-files/dihextras/solr-word.pdf)); server.request(req); server.commit(); -- Sami Siren
Re: StreamingUpdateSolrServer Connection Timeout Setting
The api doc for version 3.6.0 is available here: http://lucene.apache.org/solr/api/org/apache/solr/client/solrj/impl/StreamingUpdateSolrServer.html I think the default is coming from your OS if you are not setting it explicitly. -- Sami Siren On Fri, Jun 15, 2012 at 8:22 PM, Kissue Kissue kissue...@gmail.com wrote: Hi, Does anybody know what the default connection timeout setting is for StreamingUpdateSolrServer? Can i explicitly set one and how? Thanks.
Re: solrj library requirements: slf4j-jdk14-1.5.5.jar
What is the version of solrj you are trying to get working? If you download version 3.6 of solr there's a directory dist/solrj-lib in the binary release artifact that includes the required dependencies. I would start with those. -- Sami Siren On Wed, Jun 6, 2012 at 5:34 PM, Welty, Richard rwe...@ltionline.com wrote: the section of the solrj wiki page on setting up the class path calls for slf4j-jdk14-1.5.5.jar which is supposed to be in a lib/ subdirectory. i don't see this jar or any like it with a different version anywhere in either the 3.5.0 or 3.6.0 distributions. is it really needed or is this just slightly outdated documentation? the top of the page (which references solr 1.4) suggests this is true, and i see other docs on the web suggesting this is the case, but the first result that pops out of google for solrj is the apparently outdated wiki page, so i imagine others will encounter the same issue. the other, more recent pages are not without issue as well, for example this page: http://lucidworks.lucidimagination.com/display/solr/Using+SolrJ references apache-solr-common which i'm not finding either. thanks, richard
Re: SolrJ dependencies
On Tue, Jun 12, 2012 at 4:22 PM, Thijs vonk.th...@gmail.com wrote: Hi I just checked out and build solrlucene from branches/lucene_4x I wanted to upgrade my custom client to this new version (using solrj). So I copied lucene/solr/dist/apache-solr-solrj-4.0-SNAPSHOT.jar lucene/solr/dist/apache-solr-core-4.0-SNAPSHOT.jar to my project and I updated the other libs from the libs in /solr/dist/solrj-lib However, when I wanted to run my client I got exceptions indicating that I was missing the HTTPClient jars. (httpclient, htpcore,httpmime) Shouldn't those go into lucene/solr/dist/solrj-lib as wel? Yes they should. Do I need to create a ticket for this? Please do so. -- Sami Siren
Re: Accent Characters
Vicente,
Are you using CommonsHttpSolrServer or HttpSolrServer? If the latter
then you are probably hitting this:
https://issues.apache.org/jira/browse/SOLR-3375
The remedy is to use CommonshHttpSolrServer.
--
Sami Siren
On Thu, May 31, 2012 at 7:52 AM, Vicente Couto couto.vice...@gmail.com wrote:
Hello, Jack.
Yeah, I'm screwed up.
Well, the documents are indexed with the accents.
I started a new clean solr 3.6 configuration, with as few changes as
possible; I'm running two cores, one for English and another one for French.
Here is where I am now: If I try to run queries by using solrJ, it does some
sort of encoding. For example, I can see into the logs that if I run one
query looking for pré, I got
INFO: [coreFR] webapp=/solr path=/select
params={fl=*,scoreq=content:préhl.fl=contenthl.maxAnalyzedChars=10hl=true}
hits=0 status=0 QTime=0
And I can't see any results. If I try by using encoding to UTF-8 it's not
works.
But if I simply put http calls into the browser address bar, for example, it
works perfectly!
So, how can I tell solrJ to not encode the queries?
Thank you
--
View this message in context:
http://lucene.472066.n3.nabble.com/Accent-Characters-tp3985931p3986970.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: upgrade to 3.6
Hi, If you're using non ascii data with solrj you might want to test that it works for you properly. See for example https://issues.apache.org/jira/browse/SOLR-3375 -- Sami Siren On Fri, May 25, 2012 at 10:11 AM, Cam Bazz camb...@gmail.com wrote: Hello, I have upgraded from 1.4 to 3.6 - it went quite smooth, using the same schema.xml I have done some testing, and I have not found any problems yet. Soon I will migrate the production system to 3.6 Any recomendations on this matter? Maybe I skipped something? Best Regards, C.B.
Re: Throws Null Pointer Exception Even Query is Correct in solr
What version of solr (solrj) are you using? -- Sami SIren On Thu, May 24, 2012 at 8:41 AM, in.abdul in.ab...@gmail.com wrote: Hi Dmitry , There is no out of memory execution in solr .. Thanks and Regards, S SYED ABDUL KATHER On Thu, May 24, 2012 at 1:14 AM, Dmitry Kan [via Lucene] ml-node+s472066n3985762...@n3.nabble.com wrote: do you also see out of memory exception in your tomcat logs? If so, try setting the JVM's -Xmx to something reasonable. -- Dmitry On Wed, May 23, 2012 at 10:09 PM, in.abdul [hidden email]http://user/SendEmail.jtp?type=nodenode=3985762i=0 wrote: Sorry i missed the point i am already using Method.Post Only .. Still i could not able to execute Thanks and Regards, S SYED ABDUL KATHER On Thu, May 24, 2012 at 12:19 AM, iorixxx [via Lucene] [hidden email] http://user/SendEmail.jtp?type=nodenode=3985762i=1 wrote: I have creteria where i am passing more than 10 ids in Query like q=(ROWINDEX:(1 2 3 4 )) using solrJ . i had increased the Max Boolean clause = 10500 and i had increased the Max Header Size in tomcat also in sufficient amount .. But still its is throwing Null Pointer Exception in Tomcat and in Eclipse while debugging i had seen Error as Error Executing Query . Please give me suggestion for this. If you are using GET method ( which is default) try POST method instead. See how to use it : http://search-lucene.com/m/34M4GTEIaD -- If you reply to this email, your message will be added to the discussion below: http://lucene.472066.n3.nabble.com/Throws-Null-Pointer-Exception-Even-Query-is-Correct-in-solr-tp3985736p3985746.html To unsubscribe from Lucene, click here . NAML http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml - THANKS AND REGARDS, SYED ABDUL KATHER -- View this message in context: http://lucene.472066.n3.nabble.com/Throws-Null-Pointer-Exception-Even-Query-is-Correct-in-solr-tp3985736p3985754.html Sent from the Solr - User mailing list archive at Nabble.com. -- Regards, Dmitry Kan -- If you reply to this email, your message will be added to the discussion below: http://lucene.472066.n3.nabble.com/Throws-Null-Pointer-Exception-Even-Query-is-Correct-in-solr-tp3985736p3985762.html To unsubscribe from Lucene, click herehttp://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=472066code=aW4uYWJkdWxAZ21haWwuY29tfDQ3MjA2NnwxMDczOTUyNDEw . NAMLhttp://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml - THANKS AND REGARDS, SYED ABDUL KATHER -- View this message in context: http://lucene.472066.n3.nabble.com/Throws-Null-Pointer-Exception-Even-Query-is-Correct-in-solr-tp3985736p3985834.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr error when querying.
Ron, Did you actually add new xslt file there or did you try to use the example one, if the latter I believe the filename is example.xsl not example.xslt -- Sami Siren On Wed, May 23, 2012 at 5:30 PM, watson ron.jagannat...@gmail.com wrote: Here is my query: http://127.0.0.1:/solr/JOBS/select/??q=Apachewt=xslttr=example.xslt The response I get is the following. I have example.xslt in the /conf/xslt path. What is wrong here? Thanks! HTTP ERROR 500 Problem accessing /solr/JOBS/select/. Reason: getTransformer fails in getContentType java.lang.RuntimeException: getTransformer fails in getContentType at org.apache.solr.response.XSLTResponseWriter.getContentType(XSLTResponseWriter.java:72) at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:326) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:261) at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:129) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:59) at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:122) at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:110) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) Caused by: java.io.IOException: Unable to initialize Templates 'example.xslt' at org.apache.solr.util.xslt.TransformerProvider.getTemplates(TransformerProvider.java:117) at org.apache.solr.util.xslt.TransformerProvider.getTransformer(TransformerProvider.java:77) at org.apache.solr.response.XSLTResponseWriter.getTransformer(XSLTResponseWriter.java:130) at org.apache.solr.response.XSLTResponseWriter.getContentType(XSLTResponseWriter.java:69) ... 23 more Caused by: javax.xml.transform.TransformerConfigurationException: Could not compile stylesheet at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl.newTemplates(Unknown Source) at org.apache.solr.util.xslt.TransformerProvider.getTemplates(TransformerProvider.java:110) ... 26 more -- View this message in context: http://lucene.472066.n3.nabble.com/solr-error-when-querying-tp3985677.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Arabic document analysis fails for HttpServer
Hi,
there are some serious issues with encoding of the data sent to Solr
in the released 3.6.0 version of Solrj (HttpSolrServer), for example:
https://issues.apache.org/jira/browse/SOLR-3375
I believe your issue should already be fixed in the 3.6.0 branch. The
contents from that branch will eventually become solr 3.6.1.
For now I recommend you use the Commons version of the solr server (if
You need to be on released version) or then just check out the fixed
version from the 3.6 branch.
--
Sami Siren
On Thu, May 24, 2012 at 6:23 PM, Shane Perry thry...@gmail.com wrote:
Hi,
Upgrading from 3.5 to 3.6, the CommonsHttpServer was deprecated in
favor of HttpServer. After updating my code and running my unit
tests, I have one test that fails. Digging into it I found that
FieldAnalysisRequest was return zero tokens for the test string (which
is Arabic). I am at a loss for what steps to take next and would
appreciate any direction that could be given. I have included a unit
test which demonstrates the behavior.
My field's schema is:
fieldType name=text class=solr.TextField positionIncrementGap=100
analyzer
charFilter class=solr.HTMLStripCharFilterFactory /
tokenizer class=solr.StandardTokenizerFactory /
filter class=solr.StandardFilterFactory /
filter class=solr.TrimFilterFactory /
filter class=solr.LowerCaseFilterFactory /
/analyzer
/fieldType
I've also tried using the LegacyHTMLStripCharFilterFactory but with
the same results.
Thanks,
Shane
=
public class TestServer {
private static final String HOST = http://localhost:8080/junit-master;;
private static final String ARABIC_TEXT = ﺐﻃﺎﻠﺒﻳ;
private static final String ARABIC_FIELD = text;
@Test
public void testArabicCommonsHttpServer() throws Exception {
CommonsHttpSolrServer server = null;
try {
server = new CommonsHttpSolrServer(HOST);
server.setParser(new XMLResponseParser());
} catch (MalformedURLException ex) {
}
assertTrue(server != null);
ListString tokens = analysis(analysis(server, ARABIC_FIELD,
ARABIC_TEXT));
assertTrue(!tokens.isEmpty());
}
@Test
public void testArabicHttpServer() throws Exception {
HttpSolrServer server = new HttpSolrServer(HOST);
server.setParser(new XMLResponseParser());
assertTrue(server != null);
ListString tokens = analysis(analysis(server, ARABIC_FIELD,
ARABIC_TEXT));
assertTrue(!tokens.isEmpty());
}
private static FieldAnalysisResponse analysis(SolrServer server,
String field, String text) {
FieldAnalysisResponse response = null;
FieldAnalysisRequest request = new FieldAnalysisRequest(/analysis/field).
addFieldName(field).
setFieldValue(text).
setQuery(text);
request.setMethod(METHOD.POST);
try {
response = request.process(server);
} catch (Exception ex) {
}
return response;
}
private static ListString analysis(FieldAnalysisResponse response) {
ListString token = new LinkedListString();
if (response == null) {
return token;
}
IteratorEntryString, Analysis iterator = response.
getAllFieldNameAnalysis().iterator();
if (iterator.hasNext()) {
EntryString, Analysis entry = iterator.next();
IteratorAnalysisPhase phaseIterator =
entry.getValue().getQueryPhases().
iterator();
ListTokenInfo tokens = null;
while (phaseIterator.hasNext()) {
tokens = phaseIterator.next().getTokens(); // Only need the last one
}
for (TokenInfo ti : tokens) {
token.add(ti.getText());
}
}
return token;
}
}
Re: Solr Shards multi core slower then single big core
Sharding is (nearly) always slower than using one big index with sufficient hardware resources. Only use sharding when your index is too huge to fit into one single machine. If you're not constrained by CPU or IO, in other words have plenty of CPU cores available together with for example separate hard discs for each shard splitting your index into smaller shards can in some cases make a huge difference in one box too. -- Sami Siren
Re: Dynamic core creation works in 3.5.0 fails in 3.6.0: At least one core definition required at run-time for Solr 3.6.0?
On Wed, May 2, 2012 at 9:35 PM, Emes, Matthew (US - Irvine)
me...@aaxiscommerce.com wrote:
Hi:
I have been working on an integration project involving Solr 3.5.0 that
dynamically registers cores as needed at run-time, but does not contain any
cores by default. The current solr.xml configuration file is:-
?xml version=1.0 encoding=UTF-8 ?
solr persistent=false sharedLib=lib
cores adminPath=/admin/cores/
/solr
This configuration does not include any cores as those are created
dynamically by each application that is using the Solr server. This is
working fine with Solr 3.5.0; the server starts and running web
applications can register a new core using SolrJ CoreAdminRequest and
everything is working correctly. However, I tried to update to Solr 3.6.0
and this configuration fails with a SolrException due to the following code
in CoreContainer.java (lines 171-173):-
if (cores.cores.isEmpty()){
throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, No cores
were created, please check the logs for errors);
}
This is a change from Solr 3.5.0 which has no such check. I have searched
but cannot find any ticket or notice that this is a planned change in
3.6.0, but before I file a ticket I am asking the community in case this is
an issue that has been discussed and this is a planned direction for Solr.
I believe that this particular change was part of
https://issues.apache.org/jira/browse/SOLR-1730. The ability to start
solr with no cores seems like a reasonable feature so I would classify
this as a bug. Not sure what others thing about it.
--
Sami Siren
Re: # open files with SolrCloud
On Sat, Apr 21, 2012 at 9:57 PM, Yonik Seeley yo...@lucidimagination.com wrote: I can reproduce some kind of searcher leak issue here, even w/o SolrCloud, and I've opened https://issues.apache.org/jira/browse/SOLR-3392 With the fix integrated. I do not see the leaking problem anymore with my setup so it seems to be working now. -- Sami Siren
Re: # open files with SolrCloud
On Thu, Apr 19, 2012 at 3:12 PM, Sami Siren ssi...@gmail.com wrote: I have a simple solrcloud setup from trunk with default configs; 1 shard with one replica. As few other people have reported there seems to be some kind of leak somewhere that causes the number of open files to grow over time when doing indexing. One thing that correlates with the open file count that the jvm reports is the count of deleted files that solr still keeps open (not sure if the problem is this or something else). The deleted but not closed files are all ending with nrm.cfs, for example solr_home/solr/data/index/_jwk_nrm.cfs (deleted) Any ideas about what could be the cause for this? I don't even know where to start looking... I think the problem is not just the (deleted) files that counts towards the max open files the user is allowed to have. There's also the mmapped files that are kept open and count towards the limit configured in /proc/sys/vm/max_map_count and eventually lead to exceptions like java.io.IOException: Map failed there's plenty of those too visible to lsof (all kinds of lucene index files), for example: java32624 sam DELREG8,36425004 /home/sam/example/solr/data/index/_4.fdx java32624 sam DELREG8,36424981 /home/sam/example/solr/data/index/_4.fdt java32624 sam DELREG8,36425019 /home/sam/example/solr/data/index/_4_nrm.cfs java32624 sam DELREG8,36425016 /home/sam/example/solr/data/index/_4_0.tim java32624 sam DELREG8,36425015 /home/sam/example/solr/data/index/_4_0.prx java32624 sam DELREG8,36425014 /home/sam/example/solr/data/index/_4_0.frq Why are all these files, regular and mmapped kept open even if merging has occurred in the background and the files are deleted? - Sami Siren
# open files with SolrCloud
I have a simple solrcloud setup from trunk with default configs; 1 shard with one replica. As few other people have reported there seems to be some kind of leak somewhere that causes the number of open files to grow over time when doing indexing. One thing that correlates with the open file count that the jvm reports is the count of deleted files that solr still keeps open (not sure if the problem is this or something else). The deleted but not closed files are all ending with nrm.cfs, for example solr_home/solr/data/index/_jwk_nrm.cfs (deleted) Any ideas about what could be the cause for this? I don't even know where to start looking... -- Sami Siren
Re: [SoldCloud] leaking file descriptors
Do you have autocommit enabled? I tested this with 1m docs indexed by using the default example config and saw used file descriptors go up to 2400 (did not come down even after the final commit at the end). Then I disabled autocommit, reindexed and the descriptor count stayed pretty much flat at around 400-500. -- Sami Siren On Thu, Mar 1, 2012 at 12:36 PM, Markus Jelsma markus.jel...@openindex.io wrote: Hi, Yesterday we had an issue with too many open files, which was solved because a username was misspelled. But there is still a problem with open files. We cannot succesfully index a few millions documents from MapReduce to a 5-node Solr cloud cluster. One of the problems is that after a while ClassNotFoundErrors and other similar weirdness begin to appear. This will not solve itself if indexing is stopped. With lsof i found that Solr keeps open roughly 9k files 8 hours after indexing failed. Out of the 9k there are roughly 7.5k deleted files that still have a file descriptor open for the tomcat6 user, these are all segments files: /opt/solr/openindex_a/data/index.20120228101550/_34s.tvd java 10049 tomcat6 DEL REG 9,0 515607 /opt/solr/openindex_a/data/index.20120228101550/_34s.tvx java 10049 tomcat6 DEL REG 9,0 515504 /opt/solr/openindex_a/data/index.20120228101550/_34s.fdx java 10049 tomcat6 DEL REG 9,0 515735 /opt/solr/openindex_a/data/index.20120228101550/_34s_nrm.cfs java 10049 tomcat6 DEL REG 9,0 515595 /opt/solr/openindex_a/data/index.20120228101550/_34v_nrm.cfs java 10049 tomcat6 DEL REG 9,0 515592 /opt/solr/openindex_a/data/index.20120228101550/_34v_0.tim java 10049 tomcat6 DEL REG 9,0 515591 /opt/solr/openindex_a/data/index.20120228101550/_34v_0.prx java 10049 tomcat6 DEL REG 9,0 515590 /opt/solr/openindex_a/data/index.20120228101550/_34v_0.frq any many more Did i misconfigure anything? This is a pretty standard (no changes to IndexDefaults section) and a recent Solr trunk revision. Is there a bug somewhere? Thanks, Markus
Re: [SolrCloud] Too many open files - internal server error
On Wed, Feb 29, 2012 at 5:53 PM, Markus Jelsma markus.jel...@openindex.io wrote: Sami, As superuser: $ lsof | wc -l But, just now, i also checked the system handler and it told me: str name=ulimit(error executing: ulimit -n)/str That's odd, you should see something like this there: openFileDescriptorCount:131, maxFileDescriptorCount:4096, Which jvm do you have? This is rather strange, it seems. lsof | wc -l is not higher than 6k right now and ulimit -n is 32k. Is lsof not to be trusted in this case or... something else? I am not sure what is going on, are you sure the open file descriptor (32k) limit is active for the user running solr? -- Sami Siren
Re: Inconsistent Results with ZooKeeper Ensemble and Four SOLR Cloud Nodes
On Wed, Feb 29, 2012 at 7:03 PM, Matthew Parker mpar...@apogeeintegration.com wrote: I also took out my requestHandler and used the standard /update/extract handler. Same result. How did you install/start the system this time? The same way as earlier? What kind of queries do you run? Would it be possible for you to check out the latest version from svn. In there we have some dev scripts for linux that can be used to setup a test system easily (you need svn, jdk and ant). Essentially the steps would be: #Checkout the sources: svn co http://svn.apache.org/repos/asf/lucene/dev/trunk #build and start solrcloud (1 shard, no replicas) cd solr/cloud-dev sh ./control.sh rebuild sh ./control.sh reinstall 1 sh ./control.sh start 1 #index content java -jar ../example/exampledocs/post.jar ../example/exampledocs/*.xml #after that you can run your queries -- Sami Siren
Re: Unique key constraint and optimistic locking (versioning)
Given that you've set a uniqueKey-field and there already exists a document with that uniqueKey, it will delete the old one and insert the new one. There is really no difference between the semantics - updates do not exist. To create a UNIQUE-constraint as you know it from a database you have to check whether a document is already in the index *or* whether it is already pending (waiting for getting flushed to the index). Fortunately Solr manages a so called pending-set with all those documents waiting for beeing flushed to disk (Solr 3.5). We are using latest greates 4.0-SNAPSHOT code, because we want to take advantage of SolrCloud stuff. Can you give a code-pointer to where I can find the pending-set stuff? I am not sure if this is what you're asking but you should be able to get the latest data from Solr by using realtime get http://wiki.apache.org/solr/RealTimeGet -- Sami Siren
Re: Unique key constraint and optimistic locking (versioning)
On Fri, Feb 24, 2012 at 12:06 PM, Per Steffensen st...@designware.dk wrote: Sami Siren skrev: Given that you've set a uniqueKey-field and there already exists a document with that uniqueKey, it will delete the old one and insert the new one. There is really no difference between the semantics - updates do not exist. To create a UNIQUE-constraint as you know it from a database you have to check whether a document is already in the index *or* whether it is already pending (waiting for getting flushed to the index). Fortunately Solr manages a so called pending-set with all those documents waiting for beeing flushed to disk (Solr 3.5). We are using latest greates 4.0-SNAPSHOT code, because we want to take advantage of SolrCloud stuff. Can you give a code-pointer to where I can find the pending-set stuff? I am not sure if this is what you're asking but you should be able to get the latest data from Solr by using realtime get http://wiki.apache.org/solr/RealTimeGet Thanks a lot! I might be very usefull, if this provide 100% real time get - that is, if it gets the latest version of the document, also when neither a soft-commit nor a hard-commit has been performed since the lastest version of the document was indexed. Does it do that, or does it need a soft commit (then I believe it is only a near real time get operation)? I believe it does not require any kind of commit to happen so it should really be a real time get as the name suggests. -- Sami Siren
Re: distributed deletes working?
On Fri, Feb 17, 2012 at 5:10 PM, Jamie Johnson jej2...@gmail.com wrote: and having looked at this closer, shouldn't the down node not be marked as active when I stop that solr instance? Currently the shard state is not updated in the cloudstate when a node goes down. This behavior should probably be changed at some point. -- Sami Siren
Re: distributed deletes working?
On Fri, Feb 17, 2012 at 6:03 PM, Jamie Johnson jej2...@gmail.com wrote: Thanks Sami, so long at it's expected ;) In regards to the replication not working the way I think it should, am I missing something or is it simply not working the way I think? It should work. I also tried to reproduce your issue but was not able to. Could you try reproduce your problem with the provided scripts that are in solr/cloud-dev/ I think example2.sh might be a good start. It's not identical to your situation (it has 1 core per instance) but would be great if you could verify that you see the issue with that setup or not. -- Sami Siren
Re: SolrCloud Replication Question
Do you have unique dataDir for each instance? 13.2.2012 14.30 Jamie Johnson jej2...@gmail.com kirjoitti:
Re: Parallel indexing in Solr
On Mon, Feb 6, 2012 at 5:55 PM, Per Steffensen st...@designware.dk wrote: Sami Siren skrev: On Mon, Feb 6, 2012 at 2:53 PM, Per Steffensen st...@designware.dk wrote: Actually right now, I am trying to find our what my bottleneck is. The setup is more complex, than I would bother you with, but basically I have servers with 80-90% IO-wait and only 5-10% real CPU usage. It might not be a Solr-related problem, I am investigating different things, but just wanted to know a little more about how Jetty/Solr works in order to make a qualified guess. What kind of/how many discs do you have for your shards? ..also what kind of server are you experimenting with? Grrr, thats where I have a little fight with operations. For now they gave me one (fairly big) machine with XenServer. I create my machines as Xen VM's on top of that. One of the things I dont like about this (besides that I dont trust Xen to do its virtualization right, or at least not provide me with correct readings on IO) is that disk space is assigned from an iSCSI connected SAN that they all share (including the line out there). But for now actually it doesnt look like disk IO problems. It looks like networks-bottlenecks (but to some extend they all also shard network) among all the components in our setup - our client plus Lily stack (HDFS, HBase, ZK, Lily Server, Solr etc). Well it is complex, but anyways ... You could try to isolate the bottleneck by testing the indexing speed from the local machine hosting Solr. Also tools like iostat or sar might give you more details about the disk side. -- Sami Siren
Re: Parallel indexing in Solr
On Mon, Feb 6, 2012 at 2:53 PM, Per Steffensen st...@designware.dk wrote: Actually right now, I am trying to find our what my bottleneck is. The setup is more complex, than I would bother you with, but basically I have servers with 80-90% IO-wait and only 5-10% real CPU usage. It might not be a Solr-related problem, I am investigating different things, but just wanted to know a little more about how Jetty/Solr works in order to make a qualified guess. What kind of/how many discs do you have for your shards? ..also what kind of server are you experimenting with? -- Sami Siren
Re: Commit and sessions
On Fri, Jan 27, 2012 at 3:25 PM, Jan Høydahl jan@cominvent.com wrote: Hi, Yep, anything added between two commits must be regarded as lost in case of crash. You can of course minimize this interval by using a low commitWithin. But after a crash you should always investigate whether the last minutes of adds made it. In addition to what Jan said I think you also need to watch out for out of memory exceptions and filled disk space because I think you loose your docs (since last commit) in those cases too. -- Sami Siren
Re: Difference between #indexed documents and #results in *:* query
Does all your 913 documents contain a unique key? The uniqueKey field is id by default. -- Sami Siren On Wed, Jan 25, 2012 at 3:16 PM, m0rt0n rau...@gmail.com wrote: Thanks a lot for your answer; really appreciated. Unfortunately, I am still getting the same number of results: - I tried by refreshing the browser cache. - I tried another search by the ID:* - And went to the http://localhost:8983/solr/browse?q= ... and got the same number of results. (383 results found in 13 ms Page 1 of 1) I don't understand why it says that it is indexing 913 (see below) and it just finds 383, that makes no sense to me and I am starting to go crazy :-) Any further help appreciated. Thanks! -- View this message in context: http://lucene.472066.n3.nabble.com/Difference-between-indexed-documents-and-results-in-query-tp3687217p3687646.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Stopword filter - refreshing stop word list periodically
On Fri, Oct 14, 2011 at 10:06 PM, Jithin jithin1...@gmail.com wrote: What will be the name of this hard coded core? I was re arranging my directory structure adding a separate directory for code. And it does work with a single core. In trunk the single core setup core is called collection1. So to reload that you'd call url: http://localhost:8983/solr/admin/cores?action=RELOADcore=collection1 -- Sami Siren
Re: basic solr cloud questions
2011/9/29 Yury Kats yuryk...@yahoo.com: True, but there is a big gap between goals and current state. Right now, there is distributed search, but not distributed indexing or auto-sharding, or auto-replication. So if you want to use the SolrCloud now (as many of us do), you need do a number of things yourself, even if they might be done by SolrCloud automatically in the future. There is a patch in Jira: https://issues.apache.org/jira/browse/SOLR-2355 that adds a update processor suitable for doing simple distributed indexing with current version of Solr. -- Sami Siren
Re: [POLL] Where do you get Lucene/Solr from? Maven? ASF Mirrors?
Where do you get your Lucene/Solr downloads from? [] ASF Mirrors (linked in our release announcements or via the Lucene website) [X] Maven repository (whether you use Maven, Ant+Ivy, Buildr, etc.) [X] I/we build them from source via an SVN/Git checkout. [] Other (someone in your company mirrors them internally or via a downstream project) -- Sami Siren
Re: unable to run the solr in tomcat 5.0
post.jar also supports specifying alternative url, see output from java -jar post.jar -help for more info. -- Sami Siren uday kumar maddigatla wrote: hi you mis understood my question. When i try to use the command java -post.jar *.*. It is trying to Post files in Solr which is there in 8983 port. If we use Jety then the default port number is 8983. But what about the thing that if we use tomcat which uses 8080 as port. If we use Jetty we can access Solr with this address http://localhost:8983/solr. If we use Tomcat we can access Solr with this address http://localhost:8080/solr. So if we use above command(java -post.jar) it clearly shows this kind of message in command promt C:\TestDocumetsjava -jar post.jar *.* SimplePostTool: version 1.2 SimplePostTool: WARNING: Make sure your XML documents are encoded in UTF-8, other encodings are not currently supported SimplePostTool: POSTing files to http://localhost:8983/solr/update.. SimplePostTool: POSTing file OIO_INV_579814008_14118.xml SimplePostTool: FATAL: Connection error (is Solr running at http://localhost:8983/solr/update ?): java.net.ConnectException: Connection refused: connect Means it is trying to post the files in Solr running at http://localhost:8983/solr/update . But in my case Solr is running in 8080 port. Only b'use of Solr i can't change my tomcat port number. Is there any other possibility in Solr to index the documents rather than command utility. Michael Ludwig-4 wrote: uday kumar maddigatla schrieb: My intention is to use 8080 as port. Is there any other way taht Solr will post the files in 8080 port Solr doesn't post, it listens. Use the curl utility as indicated in the documentation. http://wiki.apache.org/solr/UpdateXmlMessages Michael Ludwig
Re: [VOTE] Community Logo Preferences
https://issues.apache.org/jira/secure/attachment/12394314/apache_soir_001.jpg https://issues.apache.org/jira/secure/attachment/12394366/solr3_maho.png https://issues.apache.org/jira/secure/attachment/12394353/solr.s5.jpg https://issues.apache.org/jira/secure/attachment/12394376/solr_sp.png https://issues.apache.org/jira/secure/attachment/12394266/apache_solr_b_red.jpg -- Sami Siren Ryan McKinley wrote: Please submit your preferences for the solr logo. For full voting details, see: http://wiki.apache.org/solr/LogoContest#Voting The eligible logos are: http://people.apache.org/~ryan/solr-logo-options.html Any and all members of the Solr community are encouraged to reply to this thread and list (up to) 5 ranked choices by listing the Jira attachment URLs. Votes will be assigned a point value based on rank. For each vote, 1st choice has a point value of 5, 5th place has a point value of 1, and all others follow a similar pattern. https://issues.apache.org/jira/secure/attachment/12345/yourfrstchoice.jpg https://issues.apache.org/jira/secure/attachment/34567/yoursecondchoice.jpg ... This poll will be open until Wednesday November 26th, 2008 @ 11:59PM GMT When the poll is complete, the solr committers will tally the community preferences and take a final vote on the logo. A big thanks to everyone would submitted possible logos -- its great to see so many good options.
Re: solr + maven?
Ryan McKinley wrote: Is anyone managing solr projects with maven? I see: https://issues.apache.org/jira/browse/SOLR-19 but that is 1 year old If someone has a current pom.xml, can you post it on SOLR-19? I just started messing with maven, so I don't really know what I am doing yet. In my m2 projects I have messed up only with the java client so far (a very old version). The solr .war I use in integration tests (launched from the build) is just deployed in our artifactory. It would be cool if you'd end up with something similar to what Lucene has recently experienced (fully usable maven artifacts for releases and nightly builds). If it has any value I could at least put some poms together. -- Sami Siren
Re: Tomcat6 env-entry
I don't recall any differences in tomcat 6 configuration compared to tomcat 5, did you try to follow the information on wiki for tomcat 5 on your installation? -- Sami Siren Matthew Runo wrote: Ok, I updated it. I hope it makes sense =\ I'm not really familiar enough with the Context changes to add those. If someone else would be so kind as to add the other way, it'd be much appreciated. http://wiki.apache.org/solr/SolrTomcat --Matthew
Re: Solr 1.2 HTTP Client for Java
Ryan McKinley wrote: Any idea if you are going to make it distributable via the central Maven repo? It will be included in the next official solr release. I don't use maven, but assume all official apache projects are included in their repo. If they do nightly snapshots, it will be there Each project must actively push [1] released artifacts to maven repository, there is no other way of getting them there. [1] http://www.apache.org/dev/release-publishing.html#maven-repo -- Sami Siren
Re: Index search questions; special cases
Erik Hatcher wrote: Yeah, the Nutch code is highly intertwined with its unique configuration infrastructure and makes it hard to pull pieces of it out like this. This is a critique that has been heard a lot (mainly because its true :) It would be really cool if different camps of lucene could build these nice utilities to be usable between projects. Not exactly sure how this could be accomplished but anyway something to consider. -- Sami Siren
