Re: MOVING Partitions and Rebalancing
Hi. It's ok to have MOVING partitions in the described scenario, because the second node is started after the first and starts a rebalancing on the next topology version. But it shouldn't break query consistency, because reading from the MOVING partition is not allowed. Why does it happen for text queries? Probably we are mapping to MOVING partitions, which is incorrect. вт, 6 июл. 2021 г. в 22:07, Atri Sharma : > Gentle ping. Please help with context here. > > On Mon, 5 Jul 2021, 13:03 Atri Sharma, wrote: > > > Hi All, > > > > As part of the text queries overhaul effort, I am looking into the > > following ticket: > > > > https://issues.apache.org/jira/browse/IGNITE-12401 > > > > As I understand it, the problem lies in the fact that a partition can > > move, thus causing duplicate data between two Lucene indices (on > > different nodes). > > > > I wish to discuss the solution to this problem: > > > > 1. Fix the issue with MOVING partitions (need help here, since I am > > not aware of the internals of rebalancing). > > > > 2. Circumvent the problem for text queries (maybe assign/use a > > globally unique ID per entry, and use that to remove duplicates during > > the merge phase?) > > > > Please share thoughts and inputs. > > > > Atri > > > -- Best regards, Alexei Scherbakov
Re: MOVING Partitions and Rebalancing
Gentle ping. Please help with context here. On Mon, 5 Jul 2021, 13:03 Atri Sharma, wrote: > Hi All, > > As part of the text queries overhaul effort, I am looking into the > following ticket: > > https://issues.apache.org/jira/browse/IGNITE-12401 > > As I understand it, the problem lies in the fact that a partition can > move, thus causing duplicate data between two Lucene indices (on > different nodes). > > I wish to discuss the solution to this problem: > > 1. Fix the issue with MOVING partitions (need help here, since I am > not aware of the internals of rebalancing). > > 2. Circumvent the problem for text queries (maybe assign/use a > globally unique ID per entry, and use that to remove duplicates during > the merge phase?) > > Please share thoughts and inputs. > > Atri >
MOVING Partitions and Rebalancing
Hi All, As part of the text queries overhaul effort, I am looking into the following ticket: https://issues.apache.org/jira/browse/IGNITE-12401 As I understand it, the problem lies in the fact that a partition can move, thus causing duplicate data between two Lucene indices (on different nodes). I wish to discuss the solution to this problem: 1. Fix the issue with MOVING partitions (need help here, since I am not aware of the internals of rebalancing). 2. Circumvent the problem for text queries (maybe assign/use a globally unique ID per entry, and use that to remove duplicates during the merge phase?) Please share thoughts and inputs. Atri
