date:20181112

[jira] [Updated] (CASSANDRA-14870) The order of application of nodetool garbagecollect is broken

2018-11-12 Thread ZhaoYang (JIRA)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-14870:
-
Status: Ready to Commit  (was: Patch Available)

> The order of application of nodetool garbagecollect is broken
> -
>
> Key: CASSANDRA-14870
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14870
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Branimir Lambov
>Assignee: Branimir Lambov
>Priority: Major
>
> {{nodetool garbagecollect}} was intended to work from oldest sstable to 
> newest, so that the collection in newer tables can purge tombstones over data 
> that has been deleted.
> However, {{SSTableReader.maxTimestampComparator}} currently sorts in the 
> opposite order (the order changed in CASSANDRA-13776 and then back in 
> CASSANDRA-14010), which makes the garbage collection unable to purge any 
> tombstones.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14870) The order of application of nodetool garbagecollect is broken

2018-11-12 Thread ZhaoYang (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684817#comment-16684817
 ] 

ZhaoYang commented on CASSANDRA-14870:
--

Sorry for delay.. Patch LGTM!

> The order of application of nodetool garbagecollect is broken
> -
>
> Key: CASSANDRA-14870
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14870
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Branimir Lambov
>Assignee: Branimir Lambov
>Priority: Major
>
> {{nodetool garbagecollect}} was intended to work from oldest sstable to 
> newest, so that the collection in newer tables can purge tombstones over data 
> that has been deleted.
> However, {{SSTableReader.maxTimestampComparator}} currently sorts in the 
> opposite order (the order changed in CASSANDRA-13776 and then back in 
> CASSANDRA-14010), which makes the garbage collection unable to purge any 
> tombstones.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-9387) Add snitch supporting Windows Azure

2018-11-12 Thread Ben Lackey (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684768#comment-16684768
 ] 

Ben Lackey commented on CASSANDRA-9387:
---

There's a whole bunch of history here, some of which I might be able to help 
with...  [~stinkymatt] hired me at DataStax back when the partnership with 
Azure was just starting.  At the time Azure had availability sets with the 
FD/UD model.  We tried to come up with a way for things to work with that along 
with the copy index in ARM.  The result of all that is the ARM templates here 
for DSE.  Those could be refactored to work with C*: 
[https://github.com/dspn/azure-resource-manager-dse]

Our decision at the time was not to invest in a snitch and handle things with 
the metadata service and a gossipping property file snitch, thinking that was 
more flexible.  Some of that thinking is captured here: 
[https://github.com/DSPN/azure-deployment-guide/blob/master/bestpractices.md]

Somewhere in there, a bunch of things happened including the open source schism 
and the release of VMSS and AZs on Azure.  I also left DataStax and Collin 
Poczatek took over the work I'd be doing.  He's since left DataStax as well.

I think any solution here would need to take into account:

VMSS

Availability Sets - FD/UD

Availability Zones

My $0.02 — take the default behavior in the VMSS and have the snitch do 
something sensible for that.  Last I heard a VMSS had 5 FDs, each with a single 
UD.  So you could treat it as a five rack thing.  That said, I can't remember 
if C* does something sensible if you have more racks than replication.  Guy 
Bowerman over at Azure could probably help with all that.

Hope this is useful!

 

 

> Add snitch supporting Windows Azure
> ---
>
> Key: CASSANDRA-9387
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9387
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Configuration
>Reporter: Jonathan Ellis
>Assignee: Yoshua Wakeham
>Priority: Major
> Fix For: 4.x
>
>
> Looks like regions / fault domains are a pretty close analogue to C* 
> DCs/racks.
> http://blogs.technet.com/b/yungchou/archive/2011/05/16/window-azure-fault-domain-and-update-domain-explained-for-it-pros.aspx



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14554) LifecycleTransaction encounters ConcurrentModificationException when used in multi-threaded context

2018-11-12 Thread Stefania (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-14554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684618#comment-16684618
]

Stefania edited comment on CASSANDRA-14554 at 11/13/18 2:05 AM:

{quote}The only reason would be simplifying analysis of the code's behaviour.
For instance, it's not clear to me how we either would (or should) behave in
the stream writers actively working (and creating sstable files) but for whom
the transaction has already been cancelled. Does such a scenario even arise? Is
it possible it would leave partially written sstables?
{quote}
I'm not sure if this scenario may arise when a streaming transaction is
aborted, it depends on streaming details which I've forgotten, but let's step
through it:
- The new sstables are recorded as new records before the files are created.
If the recording fails, because the transaction was aborted, the streamer will
abort with an exception. Fine.
- So long as the sstables are recorded, the transaction tidier will delete the
files on disk and so the contents will be removed from disk as soon as the
streamer finishes writing. Also fine.
- We may however have a race is if the streamer has added a new record to a
txn that is about to be aborted, and the streamer hasn't created sstable files
when the transaction tidier is running. This could leave files on disk. It's an
extremely small window, but it's not impossible.

We keep a reference to the txn only for obsoleted readers of existing files, we
should also keep a reference to the txn until all new files are at least
created and the directory has been synced. Child transactions would solve this
without the need for this extra reference, but we would need to enforce them
for all multi-threaded code (the presence of synchronized methods may lure
people on sharing transactions). The alternative to child transactions is to
force writers to reference the txn.
{quote}we could even do it with a delegating SynchronizedLifecycleTransaction,
which would seem to be equivalent to your patch
{quote}
This was exactly the starting point of my patch. I did not implement a fully
synchronized transaction because the API is quite large. I thought it may need
some cleanup in order to extract the methods related to the transaction
behavior. I did not have the time to look into this, and also cleaning up the
API is not an option on our released branches, due to the risk of introducing
problems, so I extracted the three methods that are used by the writers and
implemented the easiest and safest approach.

was (Author: stefania):
{quote}The only reason would be simplifying analysis of the code's behaviour.
For instance, it's not clear to me how we either would (or should) behave in
the stream writers actively working (and creating sstable files) but for whom
the transaction has already been cancelled. Does such a scenario even arise? Is
it possible it would leave partially written sstables?
{quote}
I'm not sure if this scenario may arise when a streaming transaction is
aborted, it depends on streaming details which I've forgotten, but let's step
through it:
- The new sstables are recorded as new records before the files are created.
If the recording fails, because the transaction was aborted, the streamer will
abort with an exception. Fine.
- So long as the sstables are recorded, the transaction tidier will delete the
files on disk and so the contents will be removed from disk as soon as the
streamer finishes writing. Also fine.
- We may however have a race is if the streamer has added a new record to a
txn that is about to be aborted, and the streamer hasn't created sstable files
when the transaction tidier is running. This could leave files on disk. It's an
extremely small window, but it's not impossible.

We keep a reference to the txn only for obsoleted readers of existing files, we
should also keep a reference to the txn until all new files are at least
created and the directory has been synced. Child transactions would solve this
without the need for this extra reference, but we would need to enforce them
for all multi-threaded code (the presence of synchronized methods may lure
people on sharing transactions). The alternative to child transactions is to
force writers to reference the txn.
{quote}we could even do it with a delegating SynchronizedLifecycleTransaction,
which would seem to be equivalent to your patch
{quote}
This was exactly the starting point of my patch. I did not implement a fully
synchronized transaction because the API is quite large. I thought it may need
some cleanup in order to extract the methods related to the transaction
behavior. I did not have the time to look into this, and also cleaning up the
API is not an option on out released branches, due to the risk of introducing
problems, so I just extracted the three methods that are used by

58 matches

Mail list logo