[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-05-18 Thread Ludovic Boutros (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480534#comment-16480534
 ] 

Ludovic Boutros commented on CASSANDRA-14055:
-

Thank you [~jasobrown].

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Jordan West
>Priority: Major
>  Labels: patch, sasi
> Fix For: 4.0, 3.11.3
>
> Attachments: 14055-jrwest-3.11.patch, 14055-jrwest-trunk.patch, 
> CASSANDRA-14055-jrwest.patch, CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-04-12 Thread Ludovic Boutros (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435231#comment-16435231
 ] 

Ludovic Boutros commented on CASSANDRA-14055:
-

Any news on this ? Thx.

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Jordan West
>Priority: Major
>  Labels: patch, sasi
> Fix For: 3.11.x, 4.x
>
> Attachments: 14055-jrwest-3.11.patch, 14055-jrwest-trunk.patch, 
> CASSANDRA-14055-jrwest.patch, CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-02-22 Thread Jordan West (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373192#comment-16373192
 ] 

Jordan West commented on CASSANDRA-14055:
-

[~lboutros] its in [~jasobrown]'s queue to give it one more review but I hope 
next week.

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.x, 4.x
>
> Attachments: 14055-jrwest-3.11.patch, 14055-jrwest-trunk.patch, 
> CASSANDRA-14055-jrwest.patch, CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-02-22 Thread Ludovic Boutros (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373102#comment-16373102
 ] 

Ludovic Boutros commented on CASSANDRA-14055:
-

[~jrwest] do you have an idea when this will be committed ?

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.x, 4.x
>
> Attachments: 14055-jrwest-3.11.patch, 14055-jrwest-trunk.patch, 
> CASSANDRA-14055-jrwest.patch, CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-02-21 Thread Jordan West (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371687#comment-16371687
 ] 

Jordan West commented on CASSANDRA-14055:
-

[~lboutros] Great! Thanks for taking a look. I've created 
https://issues.apache.org/jira/browse/CASSANDRA-14248. 

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.x, 4.x
>
> Attachments: 14055-jrwest-3.11.patch, 14055-jrwest-trunk.patch, 
> CASSANDRA-14055-jrwest.patch, CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-02-20 Thread Ludovic Boutros (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369818#comment-16369818
 ] 

Ludovic Boutros commented on CASSANDRA-14055:
-

[~jrwest],

I understand your changings and your explanations on the reference counting 
issue.
It looks good to me.

Thank you.

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.x, 4.x
>
> Attachments: 14055-jrwest-3.11.patch, 14055-jrwest-trunk.patch, 
> CASSANDRA-14055-jrwest.patch, CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-02-19 Thread Jordan West (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369601#comment-16369601
 ] 

Jordan West commented on CASSANDRA-14055:
-

[~lboutros]/[~jasobrown], some updates:

 
 I have attached two new patches. One for trunk and one of 3.11. Unfortunately, 
the test changes in trunk don't work well on 3.11 so we can't have one patch. 
The primary changes in this patch are to change the order we iterate over the 
indexes to ensure we retain the newer instance of {{SSTableIndex}} and thus the 
newer {{SSTableReader}}. I also changed the code to clone the {{oldSSTables}} 
collection since its visible outside the {{View}} constructor. 
||3.11||Trunk||
|[branch|https://github.com/jrwest/cassandra/tree/14055-jrwest-3.11]|[branch|https://github.com/jrwest/cassandra/tree/14055-jrwest-trunk]|
|[utests|https://circleci.com/gh/jrwest/cassandra/24]|[utests|https://circleci.com/gh/jrwest/cassandra/26]|

NOTE: same utests are failing on 
[trunk|https://circleci.com/gh/jrwest/cassandra/25] and I'm still working on 
getting dtests running with my CircleCI setup. 

 

Also, I spoke with some colleagues including [~beobal] and [~krummas] about the 
use of {{sstableRef.globalCount()}} to determine when to delete the SASI index 
file. I've come to the conclusion that its use at all is wrong because it 
represents the number of references to the instance, not globally. Given index 
summary redistribution, this isn't a safe assumption. Looking back at the 
original SASI patches, I am not sure why it got merged this way. The 
[patches|https://github.com/xedin/sasi/blob/master/src/java/org/apache/cassandra/db/index/sasi/SSTableIndex.java#L120]
 used {{sstable.isMarkedCompacted()}} but the [merged 
code|https://github.com/apache/cassandra/commit/72790dc8e34826b39ac696b03025ae6b7b6beb2b#diff-4873bb6fcef158ff18d221571ef2ec7cR124]
 used {{sstableRef.globalCount()}}. Fixing this is a larger undertaking, so I 
propose we split that work into a separate ticket and focus this one on SASI's 
failure to account for index redistribution in the {{View}}. The work covered 
by the other ticket would entail either a) deleting the SASI index files as 
part of {{SSTableTidier}} or by moving {{SSTableIndex}} to use {{Ref}} and 
implementing a tidier specific to it.

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.x, 4.x
>
> Attachments: CASSANDRA-14055-jrwest.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch, CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-02-14 Thread Jordan West (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364690#comment-16364690
 ] 

Jordan West commented on CASSANDRA-14055:
-

[~lboutros], the order we release them shouldn't matter for file deletion 
because as long as one index is open, there is one sstable open, and therefore 
the global reference count for the table is > 0. But if we keep the older 
reference we are leaking the old reference (and using the old metadata) until 
the {{SSTableIndex}} is released, which is wrong.

 

Either way, sounds like we agree on the fix :). I will post a new patch once 
approved internally. 

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.x, 4.x
>
> Attachments: CASSANDRA-14055-jrwest.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch, CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-02-14 Thread Ludovic Boutros (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364486#comment-16364486
 ] 

Ludovic Boutros commented on CASSANDRA-14055:
-

[~jrwest], 

??As I am writing this, however, I am realizing we want to keep a reference to 
the newer index, which references the newer SSTable instance and my patch does 
the opposite — keeping the old instance. I will post an updated patch along 
with my trunk patch for internal review, but the gist is to change the order we 
iterate over the old view and new indexes to favor new index instances.??

That was the point of my initial patch. But I aggree, if we can increment the 
global ref count with the new index before releasing the old one and therefore 
prevent the index file deletion, that would be better.

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.x, 4.x
>
> Attachments: CASSANDRA-14055-jrwest.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch, CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-02-14 Thread Jordan West (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364440#comment-16364440
 ] 

Jordan West commented on CASSANDRA-14055:
-

[~jasobrown], sorry for the trunk issues. The way {{TableMetadata}} is 
accessed/stored was changed and the test will need to be modified as a result. 
Will post a separate patch for trunk. 

[~lboutros], In my testing, the primary issue I saw was that files were removed 
from the SASI {{View}} that shouldn't be. The test writes 5 sstables (with 
sequence numbers 1-4 & 6) and during the test a compaction typically happens 
(that generates a sstable with generation 5 from sstables 1-4). The final SASI 
{{View}} when the queries are performed should contain either (1-4, 6) or (5, 
6)*. The test fails by returning 8 keys instead of 10 when the SASI {{View}} 
ends up containing only sstable 5 or by returning 0 keys instead of 10 when the 
SASI {{View}} ends up empty. 

The issue occurs when index redistribution completes. Depending on the 
interleaving* of events (the memtable flush, compaction, and redistribution), 
redistribution re-opens sstable 6, and sometimes re-opens sstable 5. This 
results in an {{SSTableListChangedNotification}}, which in turn results in the 
creation of a new {{View}},  where {{added=6}} (or {{added=[5,6]}}) and 
{{removed=6}} (or {{removed=[5,6]}}). The SASI {{View}} was written assuming 
these two sets were disjoint, which is why any reader in {{oldSSTables}} caused 
the index to be closed. This is incorrect in both cases because sstables 5 and 
6 are indeed the active data files (5 contains keys 0-8, and 6 contains keys 9 
& 10). 

Regarding the ref counting, we want to maintain one reference to sstables 5 & 6 
via their SSTableIndex instance but we’ve created a second reference and one 
needs to be closed. This is ensured by the 
{{newView.containsKey(sstable.descriptor)}} part of the conditional (so we are 
still indeed calling {{#release()}} on one instance). As I am writing this, 
however, I am realizing we want to keep a reference to the newer index, which 
references the newer SSTable instance and my patch does the opposite — keeping 
the old instance. I will post an updated patch along with my trunk patch for 
internal review, but the gist is to change the order we iterate over the old 
view and new indexes to favor new index instances.

NOTE: I've ignored https://issues.apache.org/jira/browse/CASSANDRA-14207 above

*I've found a few other interleavings by using another machine, but the general 
issue is the same.

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.x, 4.x
>
> Attachments: CASSANDRA-14055-jrwest.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch, CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-02-13 Thread Ludovic Boutros (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362742#comment-16362742
 ] 

Ludovic Boutros commented on CASSANDRA-14055:
-

Hi [~jrwest],

thx for your patch. The main difference between the two patches is that you do 
not release old {{SSTableIndex}} objects at all.

This way you do not remove the index file. On the other hand, don't you think 
you will have an issue in the reference count ? I'm not sure but in my 
understanding, the global count will never go down to zero. But perhaps am I 
wrong.




 

 

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.x, 4.x
>
> Attachments: CASSANDRA-14055-jrwest.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch, CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-02-13 Thread Jason Brown (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362284#comment-16362284
 ] 

Jason Brown commented on CASSANDRA-14055:
-

[~jrwest] patch does not compile on trunk. can you take a look?

{noformat}
build-test:
[javac] Compiling 587 source files to 
/orig/opt/dev/cassandra/build/test/classes
[javac] 
/orig/opt/dev/cassandra/test/unit/org/apache/cassandra/index/sasi/SASIIndexTest.java:946:
 error: cannot find symbol
[javac] int minIndexInterval = 
store.metadata.params.minIndexInterval;
[javac]  ^
[javac]   symbol:   variable params
[javac]   location: variable metadata of type TableMetadataRef
[javac] 
/orig/opt/dev/cassandra/test/unit/org/apache/cassandra/index/sasi/SASIIndexTest.java:955:
 error: cannot find symbol
[javac] store.metadata.minIndexInterval(minIndexInterval);
[javac]   ^
[javac]   symbol:   method minIndexInterval(int)
[javac]   location: variable metadata of type TableMetadataRef
[javac] 
/orig/opt/dev/cassandra/test/unit/org/apache/cassandra/index/sasi/SASIIndexTest.java:961:
 error: cannot find symbol
[javac] store.metadata.minIndexInterval(minIndexInterval);
[javac]   ^
[javac]   symbol:   method minIndexInterval(int)
[javac]   location: variable metadata of type TableMetadataRef
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.
[javac] 3 errors
{noformat}

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.x
>
> Attachments: CASSANDRA-14055-jrwest.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch, CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-02-13 Thread Jason Brown (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362246#comment-16362246
 ] 

Jason Brown commented on CASSANDRA-14055:
-

For sanity sake, I'm running the utests on [~jrwest]'s patch:

||3.11||trunk||
|[branch|https://github.com/jasobrown/cassandra/tree/14055-3.11]|[branch|https://github.com/jasobrown/cassandra/tree/14055-trunk]|
|[utests  
dtests|https://circleci.com/gh/jasobrown/workflows/cassandra/tree/14055-3.11]|[utests
  
dtests|https://circleci.com/gh/jasobrown/workflows/cassandra/tree/14055-trunk]|
||


> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.x
>
> Attachments: CASSANDRA-14055-jrwest.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch, CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-02-12 Thread Jordan West (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361610#comment-16361610
 ] 

Jordan West commented on CASSANDRA-14055:
-

Hi [~lboutros],

Attached my patch for your review and testing. If you could verify this does 
the right thing in your environments that would be especially helpful since I 
have been unable to replicate the deleted file issue – I only see the sstables 
removed from the SASI View.

The gist of the patch is to ensure the intersection of {{oldSStables}} and 
{{newIndexes}} is always empty. Your patch was doing the same by not checking 
{{oldSSTables}} in the second for-loop, but this approach doesn't require the 
changes to {{SSTableIndex#release}}.

I re-used your test but removed the version that runs with the data entirely 
in-memory since that won't be affected by index redistribution.

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.x
>
> Attachments: CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-02-12 Thread Ludovic Boutros (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360432#comment-16360432
 ] 

Ludovic Boutros commented on CASSANDRA-14055:
-

Hi [~jrwest],

no problem. I would be glad to review your patch. It will be a good way to 
learn :).

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.2
>
> Attachments: CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-02-09 Thread Jordan West (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358877#comment-16358877
 ] 

Jordan West commented on CASSANDRA-14055:
-

Hi [~lboutros],

My apologies for the delay. I am waiting on internal review of my version of 
the patch so you can take a look. I believe this patch accomplishes the same 
thing but with less changes and doesn't affect the reference counting in 
SSTableIndex. I hope to have it posted for your review and testing early next 
week.

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.2
>
> Attachments: CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-02-09 Thread Ludovic Boutros (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358491#comment-16358491
 ] 

Ludovic Boutros commented on CASSANDRA-14055:
-

Hi [~jrwest],

by any chance, did you have some time to work on this ? :)

Thx in advance.

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.2
>
> Attachments: CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-02-01 Thread Ludovic Boutros (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348404#comment-16348404
 ] 

Ludovic Boutros commented on CASSANDRA-14055:
-

Hi @Jordan,

first, thank you for reviewing this patch.
I will try to give answers to your questions.

Your global analysis is correct. The idea of this patch was to change as few 
things as possible.
I do not see any other failure scenarios currently.
We are using this patch in production with success since the end of november.

Regarding the {{keepFile}} change, with my last patch, I can reproduce the file 
deletion with the {{forceFlush}} boolean set to {{true}}.

You can just add a conditional breakpoint with {{keepFile && (obsolete.get() || 
sstableRef.globalCount() == 0)}} in the {{release}} function.
It will stop on each attempt of index redistribution with {{forceFlush}} active 
(second part of the test).

With my limited knowledge of the global code, I did not see any issue in the 
reference counting process with my patch.
But again, I'm quite new with this code :).

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.2
>
> Attachments: CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-01-31 Thread Jordan West (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347781#comment-16347781
 ] 

Jordan West commented on CASSANDRA-14055:
-

Hi [~lboutros],

One of the original authors of SASI here. I've been taking a look at this issue 
and your patch. Using the provided test against the {{cassandra-3.11}} branch 
(fc3357a00e2b6e56d399f07c5b81a82780c1e143), I see three different failure cases 
– two related directly to this issue and one tangentially related. More details 
on those below. With respect to this issue in particular, the three scenarios 
cause the test to fail because {{IndexSummaryManager}} ends up creating a new 
{{View}} where {{oldSSTables}} and {{newIndexes}} have overlapping values. This 
occurs because the {{IndexSummaryManager}} may "update" (re-open) an 
{{SSTableReader}} for an index already in the view. I believe this is unique to 
{{IndexSummaryManager}} and I am able to make your tests pass* without your 
patch by ensuring that there is no overlap between {{oldSStables}} and 
{{newIndexes}} (favoring {{newIndexes}}). Your patch looks to do this as well, 
though the approach is a bit different.

One thing I am curious about in your patch is the {{keepFile}} changes to 
{{SSTableIndex#release}}. Generally, this concerns me because it seems to be 
working around improper reference counting rather than correcting the reference 
counting itself. Also, while using the provided test, I am unable to hit a case 
where the condition {{obsolete.get() || sstableRef.globalCount() == 0}} is 
true. I see the file missing in the {{View}} but not on disk itself. Could you 
elaborate a bit more on the need for this change and your use of the 
{{keepFile}} flag?

The three failure scenarios I see using the provided test are:
h5. 8 keys returned - sequential case

In this scenario, at the time when the query that fails runs, the {{View}} is 
missing the most recently flushed sstable. As mentioned previously, this is 
because the intersection of {{oldSSTables}} and {{newIndexes}} is non-empty. 
This can be fixed* by ensuring nothing in {{newIndexes}} is in {{oldSSTables}}. 
I call this the sequential case because the compaction that occurs during the 
test completes before the index summary redistribution begins to create a new 
{{View}}. This is also addressed by your patch.
h5. 8 keys returned - race case

This scenario is similar to the previous one but has the additional issue of 
triggering improper {{SSTableIndex}} reference counting. From the perspective 
of the provided test, the failure scenario is the same and the fix* is as well. 
The issue occurs because of a race between compaction and index 
redistribution's creation of new {{View}} instances. This causes redistribution 
to create two {{View}} instances, the first of which is thrown away due to a 
failed compare and swap. The problem is the side-effects (calling 
{{SSTableIndex#release}}) have occurred already inside the creation of the 
garbage {{View}}, causing the reference count for the index to drop below 0. I 
see this issue as a separate one from this ticket and will file a separate 
JIRA. It is not fixed by the previously mentioned change and while I haven't 
checked in detail, I don't think the provided patch addresses this either.
h5. 0 keys returned

This scenario is similar to the first but there are three threads involved in 
the race: the compaction, the flushing of the last memtable, and the index 
redistribution. In this case, the end result is an empty {{View}}, which leads 
to no keys being returned since the system thinks there are no indexes to 
search. This is fixed* by what I mentioned previously and occurs because index 
redistribution re-opens both sstables in the original {{View}} instead of just 
one. It is also addressed by your patch. 

 

I am curious if you see any other failure scenarios besides these three and, in 
particular, if you can elaborate on and provide examples of the issues you see 
regarding the files being missing on disk and the need for the {{keepFile}} 
change.
 
\* While this fix makes the provided test pass I am still verifying its correct 
from the reference counting perspective.

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.2
>
> Attachments: CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-01-31 Thread Ludovic Boutros (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346524#comment-16346524
 ] 

Ludovic Boutros commented on CASSANDRA-14055:
-

[~ifesdjeen] do you think you will have some time to review this patch soon ? I 
really would like to see it included in 3.11.2 :D

Thx.

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>Priority: Major
>  Labels: patch
> Fix For: 3.11.2
>
> Attachments: CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2017-12-07 Thread Alex Petrov (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16282493#comment-16282493
 ] 

Alex Petrov commented on CASSANDRA-14055:
-

Sorry for not getting to it. I'm currently a bit overloaded with other things, 
I hope to get to it next week. If anyone has capacity for it - just assign 
yourself as a reviewer!

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>  Labels: patch
> Fix For: 3.11.2
>
> Attachments: CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2017-12-07 Thread Ludovic Boutros (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16282160#comment-16282160
 ] 

Ludovic Boutros commented on CASSANDRA-14055:
-

Any news on this ? We are currently using the patch in production, but that 
would be better to use an official version of C* :). 

[~ifesdjeen], do you need some help ?


> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>  Labels: patch
> Fix For: 3.11.x
>
> Attachments: CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2017-11-30 Thread Jeff Jirsa (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273438#comment-16273438
 ] 

Jeff Jirsa commented on CASSANDRA-14055:


[~ifesdjeen] are you still reviewing SASI patches or do we need to find someone 
else?

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Ludovic Boutros
>  Labels: patch
> Fix For: 3.11.x
>
> Attachments: CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2017-11-23 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264396#comment-16264396
 ] 

ASF GitHub Bot commented on CASSANDRA-14055:


Github user ludovic-boutros commented on the issue:

https://github.com/apache/cassandra/pull/174
  
I have added a flag in order to keep index files in the index release 
function.
This way I can prevent deletions.

Any advise on the patch ?


> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>  Labels: patch
> Fix For: 3.11.x
>
> Attachments: CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2017-11-23 Thread Ludovic Boutros (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264226#comment-16264226
 ] 

Ludovic Boutros commented on CASSANDRA-14055:
-

I've updated the PR with a more complex unit test and a better fix.

I will add another patch here for reference.

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>  Labels: patch
> Fix For: 3.11.x
>
> Attachments: CASSANDRA-14055.patch, CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2017-11-22 Thread Ludovic Boutros (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16262735#comment-16262735
 ] 

Ludovic Boutros commented on CASSANDRA-14055:
-

It seems to be a bit more complex.
There is a case that I'm not able to reproduce in a unit test which breaks as 
well SASI index, but I'm able to reproduce it in debug mode (and in production 
of course :( ).

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>  Labels: patch
> Fix For: 3.11.x
>
> Attachments: CASSANDRA-14055.patch, CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2017-11-20 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16259303#comment-16259303
 ] 

ASF GitHub Bot commented on CASSANDRA-14055:


GitHub user ludovic-boutros opened a pull request:

https://github.com/apache/cassandra/pull/174

CASSANDRA-14055: Index redistribution breaks SASI index

During index redistribution process, a new view is created.
During this creation, old indexes should be released.

But, new indexes are "attached" to the same SSTable as the old indexes.

This leads to the deletion of the last SASI index file and breaks the index.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ludovic-boutros/cassandra fix/CASSANDRA-14055

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cassandra/pull/174.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #174


commit 532ed86090c27e51c745c57678cd19ff4b606a0c
Author: lbout...@flatironsjouve.com 
Date:   2017-11-20T14:39:41Z

CASSANDRA-14055: Index redistribution breaks SASI index




> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>  Labels: patch
> Fix For: 3.11.x
>
> Attachments: CASSANDRA-14055.patch, CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org