Re: SolrCloud: Very strange behavior when doing atomic updates or documents reindexation.
I'm having a smiliar problem. Did you by any chance try the suggestion here: https://issues.apache.org/jira/browse/SOLR-4080?focusedCommentId=13498055page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13498055 ? Rakudten wrote More info: - I´m trying to update the document re-indexing the whole document again. I first retrieve the document querying by it´s id, then delete it by it´s id, and re-index including the new changes. - At the same time there are other index writing operations. *RESULT*: in most cases the document wasn´t updated. Bad news... it smells like a critical bug. Regards, - Luis Cappa. 2012/11/22 Luis Cappa Banda lt; luiscappa@ gt; For more details, my indexation App is: 1. Multithreaded. 2. NRT indexation. 3. It´s a Web App with a REST API. It receives asynchronous requests that produces those atomic updates / document reindexations I told before. I´m pretty sure that the wrong behavior is related with CloudSolrServer and with the fact that maybe you are trying to modify the index while an index update is in course. Regards, - Luis Cappa. 2012/11/22 Luis Cappa Banda lt; luiscappa@ gt; Hello! I´m using a simple test configuration with nShards=1 without any replica. SolrCloudServer is suposed to forward properly those index/update operations, isn´t it? I test with a complete document reindexation, not atomic updates, using the official LBHttpSolrServer, not my custom BinaryLBHttpSolrServer, and it dosn´t work. I think is not just a bug related with atomic updates via CloudSolrServer but a general bug when an index changes with reindexations/updates frequently. Regards, - Luis Cappa. 2012/11/22 Sami Siren lt; ssiren@ gt; It might even depend on the cluster layout! Let's say you have 2 shards (no replicas) if the doc belongs to the node you send it to so that it does not get forwarded to another node then the update should work and in case where the doc gets forwarded to another node the problem occurs. With replicas it could appear even more strange: the leader might have the doc right and the replica not. I only briefly looked at the bits that deal with this so perhaps there's something more involved. On Thu, Nov 22, 2012 at 8:29 PM, Luis Cappa Banda lt; luiscappa@ gt; wrote: Hi, Sami! But isn´t strange that some documents were updated (atomic updates) correctly and other ones not? Can´t it be a more serious problem like some kind of index writer lock, or whatever? Regards, - Luis Cappa. 2012/11/22 Sami Siren lt; ssiren@ gt; I think the problem is that even though you were able to work around the bug in the client solr still uses the xml format internally so the atomic update (with multivalued field) fails later down the stack. The bug you filed needs to be fixed to get the problem solved. On Thu, Nov 22, 2012 at 8:19 PM, Luis Cappa Banda luiscappa@ wrote: Hello everyone. I´ve starting to seriously worry about with SolrCloud due an strange behavior that I have detected. The situation is this the following: *1.* SolrCloud with one shard and two Solr instances. *2.* Indexation via SolrJ with CloudServer and a custom BinaryLBHttpSolrServer that uses BinaryRequestWriter to execute correctly atomic updates. Check JIRA-4080 https://issues.apache.org/jira/browse/SOLR-4080?focusedCommentId=13498055page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13498055 *3.* An asynchronous proccess updates partially some document fields. After that operation I automatically execute a commit, so the index must be reloaded. What I have checked is that both using atomic updates or complete document reindexations* aleatory documents are not updated* *even if I saw debugging how the add() and commit() operations were executed correctly* *and without errors*. Has anyone experienced a similar behavior? Is it posible that if an index update operation didn´t finish and CloudSolrServer receives a new one this second update operation doesn´t complete? Thank you in advance. Regards, -- - Luis Cappa -- - Luis Cappa -- - Luis Cappa -- - Luis Cappa -- - Luis Cappa -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-Very-strange-behavior-when-doing-atomic-updates-or-documents-reindexation-tp4021899p4022250.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrCloud: Very strange behavior when doing atomic updates or documents reindexation.
Yes! I opened that issue, :-P Next week I'll test with the latest trunk artifacts and check if the problem still happens. Regards, - Luis Cappa. El 25/11/2012 13:35, joe.cohe...@gmail.com joe.cohe...@gmail.com escribió: I'm having a smiliar problem. Did you by any chance try the suggestion here: https://issues.apache.org/jira/browse/SOLR-4080?focusedCommentId=13498055page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13498055 ? Rakudten wrote More info: - I´m trying to update the document re-indexing the whole document again. I first retrieve the document querying by it´s id, then delete it by it´s id, and re-index including the new changes. - At the same time there are other index writing operations. *RESULT*: in most cases the document wasn´t updated. Bad news... it smells like a critical bug. Regards, - Luis Cappa. 2012/11/22 Luis Cappa Banda lt; luiscappa@ gt; For more details, my indexation App is: 1. Multithreaded. 2. NRT indexation. 3. It´s a Web App with a REST API. It receives asynchronous requests that produces those atomic updates / document reindexations I told before. I´m pretty sure that the wrong behavior is related with CloudSolrServer and with the fact that maybe you are trying to modify the index while an index update is in course. Regards, - Luis Cappa. 2012/11/22 Luis Cappa Banda lt; luiscappa@ gt; Hello! I´m using a simple test configuration with nShards=1 without any replica. SolrCloudServer is suposed to forward properly those index/update operations, isn´t it? I test with a complete document reindexation, not atomic updates, using the official LBHttpSolrServer, not my custom BinaryLBHttpSolrServer, and it dosn´t work. I think is not just a bug related with atomic updates via CloudSolrServer but a general bug when an index changes with reindexations/updates frequently. Regards, - Luis Cappa. 2012/11/22 Sami Siren lt; ssiren@ gt; It might even depend on the cluster layout! Let's say you have 2 shards (no replicas) if the doc belongs to the node you send it to so that it does not get forwarded to another node then the update should work and in case where the doc gets forwarded to another node the problem occurs. With replicas it could appear even more strange: the leader might have the doc right and the replica not. I only briefly looked at the bits that deal with this so perhaps there's something more involved. On Thu, Nov 22, 2012 at 8:29 PM, Luis Cappa Banda lt; luiscappa@ gt; wrote: Hi, Sami! But isn´t strange that some documents were updated (atomic updates) correctly and other ones not? Can´t it be a more serious problem like some kind of index writer lock, or whatever? Regards, - Luis Cappa. 2012/11/22 Sami Siren lt; ssiren@ gt; I think the problem is that even though you were able to work around the bug in the client solr still uses the xml format internally so the atomic update (with multivalued field) fails later down the stack. The bug you filed needs to be fixed to get the problem solved. On Thu, Nov 22, 2012 at 8:19 PM, Luis Cappa Banda luiscappa@ wrote: Hello everyone. I´ve starting to seriously worry about with SolrCloud due an strange behavior that I have detected. The situation is this the following: *1.* SolrCloud with one shard and two Solr instances. *2.* Indexation via SolrJ with CloudServer and a custom BinaryLBHttpSolrServer that uses BinaryRequestWriter to execute correctly atomic updates. Check JIRA-4080 https://issues.apache.org/jira/browse/SOLR-4080?focusedCommentId=13498055page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13498055 *3.* An asynchronous proccess updates partially some document fields. After that operation I automatically execute a commit, so the index must be reloaded. What I have checked is that both using atomic updates or complete document reindexations* aleatory documents are not updated* *even if I saw debugging how the add() and commit() operations were executed correctly* *and without errors*. Has anyone experienced a similar behavior? Is it posible that if an index update operation didn´t finish and CloudSolrServer receives a new one this second update operation doesn´t complete? Thank you in advance. Regards, -- - Luis Cappa -- - Luis Cappa -- - Luis Cappa -- - Luis Cappa -- - Luis Cappa -- View this message in context:
Re: SolrCloud: Very strange behavior when doing atomic updates or documents reindexation.
I think the problem is that even though you were able to work around the bug in the client solr still uses the xml format internally so the atomic update (with multivalued field) fails later down the stack. The bug you filed needs to be fixed to get the problem solved. On Thu, Nov 22, 2012 at 8:19 PM, Luis Cappa Banda luisca...@gmail.comwrote: Hello everyone. I´ve starting to seriously worry about with SolrCloud due an strange behavior that I have detected. The situation is this the following: *1.* SolrCloud with one shard and two Solr instances. *2.* Indexation via SolrJ with CloudServer and a custom BinaryLBHttpSolrServer that uses BinaryRequestWriter to execute correctly atomic updates. Check JIRA-4080 https://issues.apache.org/jira/browse/SOLR-4080?focusedCommentId=13498055page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13498055 *3.* An asynchronous proccess updates partially some document fields. After that operation I automatically execute a commit, so the index must be reloaded. What I have checked is that both using atomic updates or complete document reindexations* aleatory documents are not updated* *even if I saw debugging how the add() and commit() operations were executed correctly* *and without errors*. Has anyone experienced a similar behavior? Is it posible that if an index update operation didn´t finish and CloudSolrServer receives a new one this second update operation doesn´t complete? Thank you in advance. Regards, -- - Luis Cappa
Re: SolrCloud: Very strange behavior when doing atomic updates or documents reindexation.
Hi, Sami! But isn´t strange that some documents were updated (atomic updates) correctly and other ones not? Can´t it be a more serious problem like some kind of index writer lock, or whatever? Regards, - Luis Cappa. 2012/11/22 Sami Siren ssi...@gmail.com I think the problem is that even though you were able to work around the bug in the client solr still uses the xml format internally so the atomic update (with multivalued field) fails later down the stack. The bug you filed needs to be fixed to get the problem solved. On Thu, Nov 22, 2012 at 8:19 PM, Luis Cappa Banda luisca...@gmail.com wrote: Hello everyone. I´ve starting to seriously worry about with SolrCloud due an strange behavior that I have detected. The situation is this the following: *1.* SolrCloud with one shard and two Solr instances. *2.* Indexation via SolrJ with CloudServer and a custom BinaryLBHttpSolrServer that uses BinaryRequestWriter to execute correctly atomic updates. Check JIRA-4080 https://issues.apache.org/jira/browse/SOLR-4080?focusedCommentId=13498055page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13498055 *3.* An asynchronous proccess updates partially some document fields. After that operation I automatically execute a commit, so the index must be reloaded. What I have checked is that both using atomic updates or complete document reindexations* aleatory documents are not updated* *even if I saw debugging how the add() and commit() operations were executed correctly* *and without errors*. Has anyone experienced a similar behavior? Is it posible that if an index update operation didn´t finish and CloudSolrServer receives a new one this second update operation doesn´t complete? Thank you in advance. Regards, -- - Luis Cappa -- - Luis Cappa
Re: SolrCloud: Very strange behavior when doing atomic updates or documents reindexation.
It might even depend on the cluster layout! Let's say you have 2 shards (no replicas) if the doc belongs to the node you send it to so that it does not get forwarded to another node then the update should work and in case where the doc gets forwarded to another node the problem occurs. With replicas it could appear even more strange: the leader might have the doc right and the replica not. I only briefly looked at the bits that deal with this so perhaps there's something more involved. On Thu, Nov 22, 2012 at 8:29 PM, Luis Cappa Banda luisca...@gmail.comwrote: Hi, Sami! But isn´t strange that some documents were updated (atomic updates) correctly and other ones not? Can´t it be a more serious problem like some kind of index writer lock, or whatever? Regards, - Luis Cappa. 2012/11/22 Sami Siren ssi...@gmail.com I think the problem is that even though you were able to work around the bug in the client solr still uses the xml format internally so the atomic update (with multivalued field) fails later down the stack. The bug you filed needs to be fixed to get the problem solved. On Thu, Nov 22, 2012 at 8:19 PM, Luis Cappa Banda luisca...@gmail.com wrote: Hello everyone. I´ve starting to seriously worry about with SolrCloud due an strange behavior that I have detected. The situation is this the following: *1.* SolrCloud with one shard and two Solr instances. *2.* Indexation via SolrJ with CloudServer and a custom BinaryLBHttpSolrServer that uses BinaryRequestWriter to execute correctly atomic updates. Check JIRA-4080 https://issues.apache.org/jira/browse/SOLR-4080?focusedCommentId=13498055page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13498055 *3.* An asynchronous proccess updates partially some document fields. After that operation I automatically execute a commit, so the index must be reloaded. What I have checked is that both using atomic updates or complete document reindexations* aleatory documents are not updated* *even if I saw debugging how the add() and commit() operations were executed correctly* *and without errors*. Has anyone experienced a similar behavior? Is it posible that if an index update operation didn´t finish and CloudSolrServer receives a new one this second update operation doesn´t complete? Thank you in advance. Regards, -- - Luis Cappa -- - Luis Cappa
Re: SolrCloud: Very strange behavior when doing atomic updates or documents reindexation.
Hello! I´m using a simple test configuration with nShards=1 without any replica. SolrCloudServer is suposed to forward properly those index/update operations, isn´t it? I test with a complete document reindexation, not atomic updates, using the official LBHttpSolrServer, not my custom BinaryLBHttpSolrServer, and it dosn´t work. I think is not just a bug related with atomic updates via CloudSolrServer but a general bug when an index changes with reindexations/updates frequently. Regards, - Luis Cappa. 2012/11/22 Sami Siren ssi...@gmail.com It might even depend on the cluster layout! Let's say you have 2 shards (no replicas) if the doc belongs to the node you send it to so that it does not get forwarded to another node then the update should work and in case where the doc gets forwarded to another node the problem occurs. With replicas it could appear even more strange: the leader might have the doc right and the replica not. I only briefly looked at the bits that deal with this so perhaps there's something more involved. On Thu, Nov 22, 2012 at 8:29 PM, Luis Cappa Banda luisca...@gmail.com wrote: Hi, Sami! But isn´t strange that some documents were updated (atomic updates) correctly and other ones not? Can´t it be a more serious problem like some kind of index writer lock, or whatever? Regards, - Luis Cappa. 2012/11/22 Sami Siren ssi...@gmail.com I think the problem is that even though you were able to work around the bug in the client solr still uses the xml format internally so the atomic update (with multivalued field) fails later down the stack. The bug you filed needs to be fixed to get the problem solved. On Thu, Nov 22, 2012 at 8:19 PM, Luis Cappa Banda luisca...@gmail.com wrote: Hello everyone. I´ve starting to seriously worry about with SolrCloud due an strange behavior that I have detected. The situation is this the following: *1.* SolrCloud with one shard and two Solr instances. *2.* Indexation via SolrJ with CloudServer and a custom BinaryLBHttpSolrServer that uses BinaryRequestWriter to execute correctly atomic updates. Check JIRA-4080 https://issues.apache.org/jira/browse/SOLR-4080?focusedCommentId=13498055page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13498055 *3.* An asynchronous proccess updates partially some document fields. After that operation I automatically execute a commit, so the index must be reloaded. What I have checked is that both using atomic updates or complete document reindexations* aleatory documents are not updated* *even if I saw debugging how the add() and commit() operations were executed correctly* *and without errors*. Has anyone experienced a similar behavior? Is it posible that if an index update operation didn´t finish and CloudSolrServer receives a new one this second update operation doesn´t complete? Thank you in advance. Regards, -- - Luis Cappa -- - Luis Cappa -- - Luis Cappa
Re: SolrCloud: Very strange behavior when doing atomic updates or documents reindexation.
For more details, my indexation App is: 1. Multithreaded. 2. NRT indexation. 3. It´s a Web App with a REST API. It receives asynchronous requests that produces those atomic updates / document reindexations I told before. I´m pretty sure that the wrong behavior is related with CloudSolrServer and with the fact that maybe you are trying to modify the index while an index update is in course. Regards, - Luis Cappa. 2012/11/22 Luis Cappa Banda luisca...@gmail.com Hello! I´m using a simple test configuration with nShards=1 without any replica. SolrCloudServer is suposed to forward properly those index/update operations, isn´t it? I test with a complete document reindexation, not atomic updates, using the official LBHttpSolrServer, not my custom BinaryLBHttpSolrServer, and it dosn´t work. I think is not just a bug related with atomic updates via CloudSolrServer but a general bug when an index changes with reindexations/updates frequently. Regards, - Luis Cappa. 2012/11/22 Sami Siren ssi...@gmail.com It might even depend on the cluster layout! Let's say you have 2 shards (no replicas) if the doc belongs to the node you send it to so that it does not get forwarded to another node then the update should work and in case where the doc gets forwarded to another node the problem occurs. With replicas it could appear even more strange: the leader might have the doc right and the replica not. I only briefly looked at the bits that deal with this so perhaps there's something more involved. On Thu, Nov 22, 2012 at 8:29 PM, Luis Cappa Banda luisca...@gmail.com wrote: Hi, Sami! But isn´t strange that some documents were updated (atomic updates) correctly and other ones not? Can´t it be a more serious problem like some kind of index writer lock, or whatever? Regards, - Luis Cappa. 2012/11/22 Sami Siren ssi...@gmail.com I think the problem is that even though you were able to work around the bug in the client solr still uses the xml format internally so the atomic update (with multivalued field) fails later down the stack. The bug you filed needs to be fixed to get the problem solved. On Thu, Nov 22, 2012 at 8:19 PM, Luis Cappa Banda luisca...@gmail.com wrote: Hello everyone. I´ve starting to seriously worry about with SolrCloud due an strange behavior that I have detected. The situation is this the following: *1.* SolrCloud with one shard and two Solr instances. *2.* Indexation via SolrJ with CloudServer and a custom BinaryLBHttpSolrServer that uses BinaryRequestWriter to execute correctly atomic updates. Check JIRA-4080 https://issues.apache.org/jira/browse/SOLR-4080?focusedCommentId=13498055page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13498055 *3.* An asynchronous proccess updates partially some document fields. After that operation I automatically execute a commit, so the index must be reloaded. What I have checked is that both using atomic updates or complete document reindexations* aleatory documents are not updated* *even if I saw debugging how the add() and commit() operations were executed correctly* *and without errors*. Has anyone experienced a similar behavior? Is it posible that if an index update operation didn´t finish and CloudSolrServer receives a new one this second update operation doesn´t complete? Thank you in advance. Regards, -- - Luis Cappa -- - Luis Cappa -- - Luis Cappa -- - Luis Cappa
Re: SolrCloud: Very strange behavior when doing atomic updates or documents reindexation.
More info: - I´m trying to update the document re-indexing the whole document again. I first retrieve the document querying by it´s id, then delete it by it´s id, and re-index including the new changes. - At the same time there are other index writing operations. *RESULT*: in most cases the document wasn´t updated. Bad news... it smells like a critical bug. Regards, - Luis Cappa. 2012/11/22 Luis Cappa Banda luisca...@gmail.com For more details, my indexation App is: 1. Multithreaded. 2. NRT indexation. 3. It´s a Web App with a REST API. It receives asynchronous requests that produces those atomic updates / document reindexations I told before. I´m pretty sure that the wrong behavior is related with CloudSolrServer and with the fact that maybe you are trying to modify the index while an index update is in course. Regards, - Luis Cappa. 2012/11/22 Luis Cappa Banda luisca...@gmail.com Hello! I´m using a simple test configuration with nShards=1 without any replica. SolrCloudServer is suposed to forward properly those index/update operations, isn´t it? I test with a complete document reindexation, not atomic updates, using the official LBHttpSolrServer, not my custom BinaryLBHttpSolrServer, and it dosn´t work. I think is not just a bug related with atomic updates via CloudSolrServer but a general bug when an index changes with reindexations/updates frequently. Regards, - Luis Cappa. 2012/11/22 Sami Siren ssi...@gmail.com It might even depend on the cluster layout! Let's say you have 2 shards (no replicas) if the doc belongs to the node you send it to so that it does not get forwarded to another node then the update should work and in case where the doc gets forwarded to another node the problem occurs. With replicas it could appear even more strange: the leader might have the doc right and the replica not. I only briefly looked at the bits that deal with this so perhaps there's something more involved. On Thu, Nov 22, 2012 at 8:29 PM, Luis Cappa Banda luisca...@gmail.com wrote: Hi, Sami! But isn´t strange that some documents were updated (atomic updates) correctly and other ones not? Can´t it be a more serious problem like some kind of index writer lock, or whatever? Regards, - Luis Cappa. 2012/11/22 Sami Siren ssi...@gmail.com I think the problem is that even though you were able to work around the bug in the client solr still uses the xml format internally so the atomic update (with multivalued field) fails later down the stack. The bug you filed needs to be fixed to get the problem solved. On Thu, Nov 22, 2012 at 8:19 PM, Luis Cappa Banda luisca...@gmail.com wrote: Hello everyone. I´ve starting to seriously worry about with SolrCloud due an strange behavior that I have detected. The situation is this the following: *1.* SolrCloud with one shard and two Solr instances. *2.* Indexation via SolrJ with CloudServer and a custom BinaryLBHttpSolrServer that uses BinaryRequestWriter to execute correctly atomic updates. Check JIRA-4080 https://issues.apache.org/jira/browse/SOLR-4080?focusedCommentId=13498055page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13498055 *3.* An asynchronous proccess updates partially some document fields. After that operation I automatically execute a commit, so the index must be reloaded. What I have checked is that both using atomic updates or complete document reindexations* aleatory documents are not updated* *even if I saw debugging how the add() and commit() operations were executed correctly* *and without errors*. Has anyone experienced a similar behavior? Is it posible that if an index update operation didn´t finish and CloudSolrServer receives a new one this second update operation doesn´t complete? Thank you in advance. Regards, -- - Luis Cappa -- - Luis Cappa -- - Luis Cappa -- - Luis Cappa -- - Luis Cappa
