[jira] [Commented] (CASSANDRA-15269) Cassandra fails to process OperationExecutionException which causes ClassCastException
[ https://issues.apache.org/jira/browse/CASSANDRA-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911757#comment-16911757 ] Andy Tolbert commented on CASSANDRA-15269: -- I made a small comment that this should be part of the protocol v5 spec instead of v4, but other than that I think this looks good. I went ahead and created a java driver ticket for awareness: [JAVA-2397|https://datastax-oss.atlassian.net/browse/JAVA-2397]. The native protocol change will be great for the 4.0 java driver. We should also consider making the change in the 3.0 driver since that intends on supporting C* 4.0 as well. > Cassandra fails to process OperationExecutionException which causes > ClassCastException > -- > > Key: CASSANDRA-15269 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15269 > Project: Cassandra > Issue Type: Bug > Components: CQL/Interpreter >Reporter: Liudmila Kornilova >Assignee: Liudmila Kornilova >Priority: Normal > > While working on CASSANDRA-15232 I noticed that OperationExecutionException > is not processed correctly. > How to reproduce the issue: > 1. {{create table d (numerator decimal primary key, denominator decimal);}} > 2. {{insert into d (numerator, denominator) values > (123456789112345678921234567893123456, 2);}} > 3. {{select numerator % denominator from d;}} > What happens: > 1. remainder operation throws ArithmeticException (BigDecimal:1854) > 2. The exception is wrapped in OperationExecutionException > 3. ClassCastException appears (OperationExecutionException cannot be cast to > FunctionExecutionException at ErrorMessage.java:280) > What should happen: > OperationExecutionException with message "the operation 'decimal % decimal' > failed: Division impossible" should be delivered to user > Note that after fixing CASSANDRA-15232 {{select numerator % denominator from > d;}} will produce correct result of remainder operation. > Currently I am not aware of other cases when OperationExecutionException may > be treated as FunctionExecutionException -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15285) Rename StatsMetadata estimatedColumnCount to estimatedCellPerPartitionCount
[ https://issues.apache.org/jira/browse/CASSANDRA-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15285: - Fix Version/s: (was: 4.x) 4.0 Source Control Link: [db2ad0f7c59d9deb1f8755858cd630d640c5baa9|https://github.com/apache/cassandra/commit/db2ad0f7c59d9deb1f8755858cd630d640c5baa9] Resolution: Fixed Status: Resolved (was: Ready to Commit) > Rename StatsMetadata estimatedColumnCount to estimatedCellPerPartitionCount > --- > > Key: CASSANDRA-15285 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15285 > Project: Cassandra > Issue Type: Task > Components: Local/SSTable >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.0 > > > This renaming was done previously in MetadataCollector but not in > StatsMetadata. The proposed name is more descriptive of what the field now > represents. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (d280124 -> db2ad0f)
This is an automated email from the ASF dual-hosted git repository. benedict pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. omit d280124 rename related methods omit 7b9b61c Rename StatsMetadata estimatedColumnCount to estimatedCellPerPartitionCount new db2ad0f Rename StatsMetadata estimatedColumnCount to estimatedCellPerPartitionCount This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (d280124) \ N -- N -- N refs/heads/trunk (db2ad0f) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Rename StatsMetadata estimatedColumnCount to estimatedCellPerPartitionCount
This is an automated email from the ASF dual-hosted git repository. benedict pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit db2ad0f7c59d9deb1f8755858cd630d640c5baa9 Author: Jordan West AuthorDate: Mon Aug 19 10:10:37 2019 -0700 Rename StatsMetadata estimatedColumnCount to estimatedCellPerPartitionCount patch by Jordan West; reviewed by Benedict for CASSANDRA-15285 --- .../org/apache/cassandra/db/ColumnFamilyStore.java | 8 .../db/columniterator/SSTableReversedIterator.java | 2 +- .../db/compaction/AbstractCompactionStrategy.java| 6 ++ .../org/apache/cassandra/db/filter/DataLimits.java | 2 +- .../cassandra/index/SecondaryIndexManager.java | 4 +--- .../cassandra/io/sstable/format/SSTableReader.java | 4 ++-- .../cassandra/io/sstable/metadata/StatsMetadata.java | 20 ++-- .../org/apache/cassandra/metrics/TableMetrics.java | 2 +- .../cassandra/tools/SSTableMetadataViewer.java | 2 +- .../index/internal/CustomCassandraIndex.java | 3 +-- 10 files changed, 24 insertions(+), 29 deletions(-) diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index 2ef5a76..075ebac 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -2380,14 +2380,14 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean // End JMX get/set. -public int getMeanColumns() +public int getMeanEstimatedCellPerPartitionCount() { long sum = 0; long count = 0; for (SSTableReader sstable : getSSTables(SSTableSet.CANONICAL)) { -long n = sstable.getEstimatedColumnCount().count(); -sum += sstable.getEstimatedColumnCount().mean() * n; +long n = sstable.getEstimatedCellPerPartitionCount().count(); +sum += sstable.getEstimatedCellPerPartitionCount().mean() * n; count += n; } return count > 0 ? (int) (sum / count) : 0; @@ -2559,7 +2559,7 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean for (SSTableReader sstable : getSSTables(SSTableSet.LIVE)) { allDroppable += sstable.getDroppableTombstonesBefore(localTime - metadata().params.gcGraceSeconds); -allColumns += sstable.getEstimatedColumnCount().mean() * sstable.getEstimatedColumnCount().count(); +allColumns += sstable.getEstimatedCellPerPartitionCount().mean() * sstable.getEstimatedCellPerPartitionCount().count(); } return allColumns > 0 ? allDroppable / allColumns : 0; } diff --git a/src/java/org/apache/cassandra/db/columniterator/SSTableReversedIterator.java b/src/java/org/apache/cassandra/db/columniterator/SSTableReversedIterator.java index 60a6f70..1e1030c 100644 --- a/src/java/org/apache/cassandra/db/columniterator/SSTableReversedIterator.java +++ b/src/java/org/apache/cassandra/db/columniterator/SSTableReversedIterator.java @@ -108,7 +108,7 @@ public class SSTableReversedIterator extends AbstractSSTableIterator // FIXME: so far we only keep stats on cells, so to get a rough estimate on the number of rows, // we divide by the number of regular columns the table has. We should fix once we collect the // stats on rows -int estimatedRowsPerPartition = (int)(sstable.getEstimatedColumnCount().percentile(0.75) / columnCount); +int estimatedRowsPerPartition = (int)(sstable.getEstimatedCellPerPartitionCount().percentile(0.75) / columnCount); estimatedRowCount = Math.max(estimatedRowsPerPartition / blocksCount, 1); } catch (IllegalStateException e) diff --git a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java index 74c154d..ad494b1 100644 --- a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java +++ b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java @@ -21,8 +21,6 @@ import java.util.*; import com.google.common.annotations.VisibleForTesting; import com.google.common.collect.ImmutableMap; -import com.google.common.base.Predicate; -import com.google.common.collect.Iterables; import org.apache.cassandra.db.Directories; import org.apache.cassandra.db.SerializationHeader; @@ -412,8 +410,8 @@ public abstract class AbstractCompactionStrategy ranges.add(new Range<>(overlap.first.getToken(), overlap.last.getToken())); long remainingKeys = keys - sstable.estimatedKeysForRanges(ranges); // next, calculate what percentage of columns we have within those keys -long columns =
[jira] [Updated] (CASSANDRA-15285) Rename StatsMetadata estimatedColumnCount to estimatedCellPerPartitionCount
[ https://issues.apache.org/jira/browse/CASSANDRA-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15285: - Status: Ready to Commit (was: Review In Progress) > Rename StatsMetadata estimatedColumnCount to estimatedCellPerPartitionCount > --- > > Key: CASSANDRA-15285 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15285 > Project: Cassandra > Issue Type: Task > Components: Local/SSTable >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.x > > > This renaming was done previously in MetadataCollector but not in > StatsMetadata. The proposed name is more descriptive of what the field now > represents. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/02: Rename StatsMetadata estimatedColumnCount to estimatedCellPerPartitionCount
This is an automated email from the ASF dual-hosted git repository. benedict pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 7b9b61cb5bccd064b0a3980c6884d0cba3dd09db Author: Jordan West AuthorDate: Mon Aug 19 10:10:37 2019 -0700 Rename StatsMetadata estimatedColumnCount to estimatedCellPerPartitionCount patch by Jordan West; reviewed by Benedict for CASSANDRA-15285 --- .../cassandra/io/sstable/format/SSTableReader.java | 2 +- .../cassandra/io/sstable/metadata/StatsMetadata.java | 20 ++-- .../cassandra/tools/SSTableMetadataViewer.java | 2 +- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java b/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java index aa4ca55..550429f 100644 --- a/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java +++ b/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java @@ -1945,7 +1945,7 @@ public abstract class SSTableReader extends SSTable implements SelfRefCounted> commitLogPositionSetSerializer = IntervalSet.serializer(CommitLogPosition.serializer); public final EstimatedHistogram estimatedPartitionSize; -public final EstimatedHistogram estimatedColumnCount; +public final EstimatedHistogram estimatedCellPerPartitionCount; public final IntervalSet commitLogIntervals; public final long minTimestamp; public final long maxTimestamp; @@ -70,7 +70,7 @@ public class StatsMetadata extends MetadataComponent public final EncodingStats encodingStats; public StatsMetadata(EstimatedHistogram estimatedPartitionSize, - EstimatedHistogram estimatedColumnCount, + EstimatedHistogram estimatedCellPerPartitionCount, IntervalSet commitLogIntervals, long minTimestamp, long maxTimestamp, @@ -91,7 +91,7 @@ public class StatsMetadata extends MetadataComponent boolean isTransient) { this.estimatedPartitionSize = estimatedPartitionSize; -this.estimatedColumnCount = estimatedColumnCount; +this.estimatedCellPerPartitionCount = estimatedCellPerPartitionCount; this.commitLogIntervals = commitLogIntervals; this.minTimestamp = minTimestamp; this.maxTimestamp = maxTimestamp; @@ -124,7 +124,7 @@ public class StatsMetadata extends MetadataComponent */ public double getEstimatedDroppableTombstoneRatio(int gcBefore) { -long estimatedColumnCount = this.estimatedColumnCount.mean() * this.estimatedColumnCount.count(); +long estimatedColumnCount = this.estimatedCellPerPartitionCount.mean() * this.estimatedCellPerPartitionCount.count(); if (estimatedColumnCount > 0) { double droppable = getDroppableTombstonesBefore(gcBefore); @@ -145,7 +145,7 @@ public class StatsMetadata extends MetadataComponent public StatsMetadata mutateLevel(int newLevel) { return new StatsMetadata(estimatedPartitionSize, - estimatedColumnCount, + estimatedCellPerPartitionCount, commitLogIntervals, minTimestamp, maxTimestamp, @@ -169,7 +169,7 @@ public class StatsMetadata extends MetadataComponent public StatsMetadata mutateRepairedMetadata(long newRepairedAt, UUID newPendingRepair, boolean newIsTransient) { return new StatsMetadata(estimatedPartitionSize, - estimatedColumnCount, + estimatedCellPerPartitionCount, commitLogIntervals, minTimestamp, maxTimestamp, @@ -199,7 +199,7 @@ public class StatsMetadata extends MetadataComponent StatsMetadata that = (StatsMetadata) o; return new EqualsBuilder() .append(estimatedPartitionSize, that.estimatedPartitionSize) - .append(estimatedColumnCount, that.estimatedColumnCount) + .append(estimatedCellPerPartitionCount, that.estimatedCellPerPartitionCount) .append(commitLogIntervals, that.commitLogIntervals) .append(minTimestamp, that.minTimestamp) .append(maxTimestamp, that.maxTimestamp) @@ -225,7 +225,7 @@ public class StatsMetadata extends MetadataComponent { return new HashCodeBuilder() .append(estimatedPartitionSize) - .append(estimatedColumnCount) + .append(estimatedCellPerPartitionCount) .append(commitLogIntervals)
[cassandra] 02/02: rename related methods
This is an automated email from the ASF dual-hosted git repository. benedict pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit d280124bb14425ee28217dd74e29fd68b1f1dc95 Author: Benedict Elliott Smith AuthorDate: Tue Aug 20 16:14:11 2019 +0100 rename related methods --- src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 8 .../cassandra/db/columniterator/SSTableReversedIterator.java | 2 +- .../cassandra/db/compaction/AbstractCompactionStrategy.java | 6 ++ src/java/org/apache/cassandra/db/filter/DataLimits.java | 2 +- src/java/org/apache/cassandra/index/SecondaryIndexManager.java| 4 +--- .../org/apache/cassandra/io/sstable/format/SSTableReader.java | 2 +- src/java/org/apache/cassandra/metrics/TableMetrics.java | 2 +- .../org/apache/cassandra/index/internal/CustomCassandraIndex.java | 3 +-- 8 files changed, 12 insertions(+), 17 deletions(-) diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index 2ef5a76..075ebac 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -2380,14 +2380,14 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean // End JMX get/set. -public int getMeanColumns() +public int getMeanEstimatedCellPerPartitionCount() { long sum = 0; long count = 0; for (SSTableReader sstable : getSSTables(SSTableSet.CANONICAL)) { -long n = sstable.getEstimatedColumnCount().count(); -sum += sstable.getEstimatedColumnCount().mean() * n; +long n = sstable.getEstimatedCellPerPartitionCount().count(); +sum += sstable.getEstimatedCellPerPartitionCount().mean() * n; count += n; } return count > 0 ? (int) (sum / count) : 0; @@ -2559,7 +2559,7 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean for (SSTableReader sstable : getSSTables(SSTableSet.LIVE)) { allDroppable += sstable.getDroppableTombstonesBefore(localTime - metadata().params.gcGraceSeconds); -allColumns += sstable.getEstimatedColumnCount().mean() * sstable.getEstimatedColumnCount().count(); +allColumns += sstable.getEstimatedCellPerPartitionCount().mean() * sstable.getEstimatedCellPerPartitionCount().count(); } return allColumns > 0 ? allDroppable / allColumns : 0; } diff --git a/src/java/org/apache/cassandra/db/columniterator/SSTableReversedIterator.java b/src/java/org/apache/cassandra/db/columniterator/SSTableReversedIterator.java index 60a6f70..1e1030c 100644 --- a/src/java/org/apache/cassandra/db/columniterator/SSTableReversedIterator.java +++ b/src/java/org/apache/cassandra/db/columniterator/SSTableReversedIterator.java @@ -108,7 +108,7 @@ public class SSTableReversedIterator extends AbstractSSTableIterator // FIXME: so far we only keep stats on cells, so to get a rough estimate on the number of rows, // we divide by the number of regular columns the table has. We should fix once we collect the // stats on rows -int estimatedRowsPerPartition = (int)(sstable.getEstimatedColumnCount().percentile(0.75) / columnCount); +int estimatedRowsPerPartition = (int)(sstable.getEstimatedCellPerPartitionCount().percentile(0.75) / columnCount); estimatedRowCount = Math.max(estimatedRowsPerPartition / blocksCount, 1); } catch (IllegalStateException e) diff --git a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java index 74c154d..ad494b1 100644 --- a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java +++ b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java @@ -21,8 +21,6 @@ import java.util.*; import com.google.common.annotations.VisibleForTesting; import com.google.common.collect.ImmutableMap; -import com.google.common.base.Predicate; -import com.google.common.collect.Iterables; import org.apache.cassandra.db.Directories; import org.apache.cassandra.db.SerializationHeader; @@ -412,8 +410,8 @@ public abstract class AbstractCompactionStrategy ranges.add(new Range<>(overlap.first.getToken(), overlap.last.getToken())); long remainingKeys = keys - sstable.estimatedKeysForRanges(ranges); // next, calculate what percentage of columns we have within those keys -long columns = sstable.getEstimatedColumnCount().mean() * remainingKeys; -double remainingColumnsRatio = ((double) columns) / (sstable.getEstimatedColumnCount().count() *
[cassandra] branch trunk updated (7dc5b70 -> d280124)
This is an automated email from the ASF dual-hosted git repository. benedict pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 7dc5b70 Merge branch 'cassandra-3.11' into trunk new 7b9b61c Rename StatsMetadata estimatedColumnCount to estimatedCellPerPartitionCount new d280124 rename related methods The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../org/apache/cassandra/db/ColumnFamilyStore.java | 8 .../db/columniterator/SSTableReversedIterator.java | 2 +- .../db/compaction/AbstractCompactionStrategy.java| 6 ++ .../org/apache/cassandra/db/filter/DataLimits.java | 2 +- .../cassandra/index/SecondaryIndexManager.java | 4 +--- .../cassandra/io/sstable/format/SSTableReader.java | 4 ++-- .../cassandra/io/sstable/metadata/StatsMetadata.java | 20 ++-- .../org/apache/cassandra/metrics/TableMetrics.java | 2 +- .../cassandra/tools/SSTableMetadataViewer.java | 2 +- .../index/internal/CustomCassandraIndex.java | 3 +-- 10 files changed, 24 insertions(+), 29 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15285) Rename StatsMetadata estimatedColumnCount to estimatedCellPerPartitionCount
[ https://issues.apache.org/jira/browse/CASSANDRA-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911464#comment-16911464 ] Jordan West commented on CASSANDRA-15285: - +1 and agreed re: JMX compat. > Rename StatsMetadata estimatedColumnCount to estimatedCellPerPartitionCount > --- > > Key: CASSANDRA-15285 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15285 > Project: Cassandra > Issue Type: Task > Components: Local/SSTable >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.x > > > This renaming was done previously in MetadataCollector but not in > StatsMetadata. The proposed name is more descriptive of what the field now > represents. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15285) Rename StatsMetadata estimatedColumnCount to estimatedCellPerPartitionCount
[ https://issues.apache.org/jira/browse/CASSANDRA-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911450#comment-16911450 ] Benedict commented on CASSANDRA-15285: -- Thanks. I've pushed [here|https://github.com/belliottsmith/cassandra/tree/15285] a branch with the mentioned method, and also {{ColumnFamilyStore.getMeanColumns}} renamed. There's also a reasonable question around if we should rename {{TableMetrics.estimatedColumnCountHistogram}}, but given this has compatibility implications (with its JMX name) I think it's OK to leave it. If you want to +1 the follow-up, I'll commit to trunk. > Rename StatsMetadata estimatedColumnCount to estimatedCellPerPartitionCount > --- > > Key: CASSANDRA-15285 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15285 > Project: Cassandra > Issue Type: Task > Components: Local/SSTable >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.x > > > This renaming was done previously in MetadataCollector but not in > StatsMetadata. The proposed name is more descriptive of what the field now > represents. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (d0be441 -> 7dc5b70)
This is an automated email from the ASF dual-hosted git repository. benedict pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. omit d0be441 Merge branch 'cassandra-3.11' into trunk new 7dc5b70 Merge branch 'cassandra-3.11' into trunk This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (d0be441) \ N -- N -- N refs/heads/trunk (7dc5b70) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../apache/cassandra/streaming/async/StreamingInboundHandler.java| 5 - 1 file changed, 4 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (b3ed8da -> d0be441)
This is an automated email from the ASF dual-hosted git repository. benedict pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. omit b3ed8da Merge branch 'cassandra-3.11' into trunk new d0be441 Merge branch 'cassandra-3.11' into trunk This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (b3ed8da) \ N -- N -- N refs/heads/trunk (d0be441) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15170) Reduce the time needed to release in-JVM dtest cluster resources after close
[ https://issues.apache.org/jira/browse/CASSANDRA-15170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15170: - Fix Version/s: 4.0 Source Control Link: [8dcaa12baa97ce870f23ff9045f968f2fa28b2cc|https://github.com/apache/cassandra/commit/8dcaa12baa97ce870f23ff9045f968f2fa28b2cc] Resolution: Fixed Status: Resolved (was: Ready to Commit) > Reduce the time needed to release in-JVM dtest cluster resources after close > > > Key: CASSANDRA-15170 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15170 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > Fix For: 4.0 > > > There are a few issues that slow the in-JVM dtests from reclaiming metaspace > once the cluster is closed. > IsolatedExecutor issues the shutdown on a SingleExecutorThreadPool, sometimes > this thread was still running 10s after the dtest cluster was closed. > Instead, switch to a ThreadPoolExecutor with a core pool size of 0 so that > the thread executing the class loader close executes sooner. > If an OutboundTcpConnection is waiting to connect() and the endpoint is not > answering, it has to wait for a timeout before it exits. Instead it should > check the isShutdown flag and terminate early if shutdown has been requested. > In 3.0 and above, HintsCatalog.load uses java.nio.Files.list outside of a > try-with-resources construct and leaks a file handle for the directory. This > doesn't matter for normal usage, it leaks a file handle for each dtest > Instance created. > On trunk, Netty global event executor threads are still running and delay GC > for the instance class loader. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15170) Reduce the time needed to release in-JVM dtest cluster resources after close
[ https://issues.apache.org/jira/browse/CASSANDRA-15170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15170: - Status: Patch Available (was: In Progress) > Reduce the time needed to release in-JVM dtest cluster resources after close > > > Key: CASSANDRA-15170 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15170 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > > There are a few issues that slow the in-JVM dtests from reclaiming metaspace > once the cluster is closed. > IsolatedExecutor issues the shutdown on a SingleExecutorThreadPool, sometimes > this thread was still running 10s after the dtest cluster was closed. > Instead, switch to a ThreadPoolExecutor with a core pool size of 0 so that > the thread executing the class loader close executes sooner. > If an OutboundTcpConnection is waiting to connect() and the endpoint is not > answering, it has to wait for a timeout before it exits. Instead it should > check the isShutdown flag and terminate early if shutdown has been requested. > In 3.0 and above, HintsCatalog.load uses java.nio.Files.list outside of a > try-with-resources construct and leaks a file handle for the directory. This > doesn't matter for normal usage, it leaks a file handle for each dtest > Instance created. > On trunk, Netty global event executor threads are still running and delay GC > for the instance class loader. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15170) Reduce the time needed to release in-JVM dtest cluster resources after close
[ https://issues.apache.org/jira/browse/CASSANDRA-15170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15170: - Reviewers: Benedict, Benedict (was: Benedict) Benedict, Benedict (was: Benedict) Status: Review In Progress (was: Patch Available) > Reduce the time needed to release in-JVM dtest cluster resources after close > > > Key: CASSANDRA-15170 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15170 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > > There are a few issues that slow the in-JVM dtests from reclaiming metaspace > once the cluster is closed. > IsolatedExecutor issues the shutdown on a SingleExecutorThreadPool, sometimes > this thread was still running 10s after the dtest cluster was closed. > Instead, switch to a ThreadPoolExecutor with a core pool size of 0 so that > the thread executing the class loader close executes sooner. > If an OutboundTcpConnection is waiting to connect() and the endpoint is not > answering, it has to wait for a timeout before it exits. Instead it should > check the isShutdown flag and terminate early if shutdown has been requested. > In 3.0 and above, HintsCatalog.load uses java.nio.Files.list outside of a > try-with-resources construct and leaks a file handle for the directory. This > doesn't matter for normal usage, it leaks a file handle for each dtest > Instance created. > On trunk, Netty global event executor threads are still running and delay GC > for the instance class loader. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15170) Reduce the time needed to release in-JVM dtest cluster resources after close
[ https://issues.apache.org/jira/browse/CASSANDRA-15170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15170: - Status: Ready to Commit (was: Review In Progress) > Reduce the time needed to release in-JVM dtest cluster resources after close > > > Key: CASSANDRA-15170 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15170 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > > There are a few issues that slow the in-JVM dtests from reclaiming metaspace > once the cluster is closed. > IsolatedExecutor issues the shutdown on a SingleExecutorThreadPool, sometimes > this thread was still running 10s after the dtest cluster was closed. > Instead, switch to a ThreadPoolExecutor with a core pool size of 0 so that > the thread executing the class loader close executes sooner. > If an OutboundTcpConnection is waiting to connect() and the endpoint is not > answering, it has to wait for a timeout before it exits. Instead it should > check the isShutdown flag and terminate early if shutdown has been requested. > In 3.0 and above, HintsCatalog.load uses java.nio.Files.list outside of a > try-with-resources construct and leaks a file handle for the directory. This > doesn't matter for normal usage, it leaks a file handle for each dtest > Instance created. > On trunk, Netty global event executor threads are still running and delay GC > for the instance class loader. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (d60e798 -> b3ed8da)
This is an automated email from the ASF dual-hosted git repository. benedict pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from d60e798 Avoid result truncate in decimal operations new 8dcaa12 Allow instance class loaders to be garbage collected for inJVM dtest new 51c0f6b Merge branch 'cassandra-2.2' into cassandra-3.0 new 3df63ed Merge branch 'cassandra-3.0' into cassandra-3.11 new b3ed8da Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 2 + build.xml | 18 +- .../apache/cassandra/batchlog/BatchlogManager.java | 7 +- .../cassandra/concurrent/ScheduledExecutors.java | 6 +- .../cassandra/concurrent/SharedExecutorPool.java | 4 +- .../apache/cassandra/concurrent/StageManager.java | 6 +- .../org/apache/cassandra/db/ColumnFamilyStore.java | 16 +- .../apache/cassandra/db/commitlog/CommitLog.java | 1 + src/java/org/apache/cassandra/gms/Gossiper.java| 10 + .../org/apache/cassandra/hints/HintsCatalog.java | 5 +- .../cassandra/io/sstable/IndexSummaryManager.java | 11 + .../cassandra/io/sstable/format/SSTableReader.java | 9 +- .../org/apache/cassandra/net/InboundSockets.java | 17 +- .../org/apache/cassandra/net/MessagingService.java | 15 +- .../org/apache/cassandra/net/SocketFactory.java| 3 +- .../org/apache/cassandra/security/SSLFactory.java | 31 ++- .../service/PendingRangeCalculatorService.java | 5 +- .../apache/cassandra/service/StorageService.java | 23 +- .../cassandra/streaming/StreamCoordinator.java | 8 + .../cassandra/streaming/StreamReceiveTask.java | 13 + .../cassandra/streaming/StreamTransferTask.java| 10 + .../streaming/async/StreamingInboundHandler.java | 36 ++- .../org/apache/cassandra/utils/ExecutorUtils.java | 68 +- .../org/apache/cassandra/utils/concurrent/Ref.java | 5 +- .../cassandra/utils/memory/MemtablePool.java | 8 +- test/conf/logback-dtest.xml| 16 +- .../org/apache/cassandra/distributed/Cluster.java | 1 - .../cassandra/distributed/UpgradeableCluster.java | 2 - .../apache/cassandra/distributed/api/Feature.java | 5 +- .../cassandra/distributed/api/IInstance.java | 2 + .../cassandra/distributed/api/IInstanceConfig.java | 2 +- .../cassandra/distributed/api/IMessageFilters.java | 3 - .../distributed/impl/AbstractCluster.java | 30 ++- .../cassandra/distributed/impl/Instance.java | 77 -- .../distributed/impl/InstanceClassLoader.java | 1 + .../cassandra/distributed/impl/InstanceConfig.java | 18 +- .../distributed/impl/IsolatedExecutor.java | 47 +++- .../test/DistributedReadWritePathTest.java | 2 +- .../distributed/test/DistributedTestBase.java | 20 ++ .../distributed/test/GossipSettlesTest.java| 4 +- .../cassandra/distributed/test/RepairTest.java | 269 +++-- .../distributed/test/ResourceLeakTest.java | 202 .../cassandra/distributed/test/StreamingTest.java | 2 +- .../cassandra/concurrent/SEPExecutorTest.java | 3 +- 44 files changed, 777 insertions(+), 266 deletions(-) copy src/java/org/apache/cassandra/hints/SerializableHintMessage.java => test/distributed/org/apache/cassandra/distributed/api/Feature.java (90%) create mode 100644 test/distributed/org/apache/cassandra/distributed/test/ResourceLeakTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (bb126c0 -> 3df63ed)
This is an automated email from the ASF dual-hosted git repository. benedict pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from bb126c0 Merge branch 'cassandra-3.0' into cassandra-3.11 new 8dcaa12 Allow instance class loaders to be garbage collected for inJVM dtest new 51c0f6b Merge branch 'cassandra-2.2' into cassandra-3.0 new 3df63ed Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + build.xml | 18 +- .../apache/cassandra/batchlog/BatchlogManager.java | 7 +- .../cassandra/concurrent/InfiniteLoopExecutor.java | 2 +- .../cassandra/concurrent/ScheduledExecutors.java | 11 +- .../cassandra/concurrent/SharedExecutorPool.java | 4 +- .../apache/cassandra/concurrent/StageManager.java | 8 +- .../org/apache/cassandra/db/ColumnFamilyStore.java | 18 +- .../apache/cassandra/db/commitlog/CommitLog.java | 1 + src/java/org/apache/cassandra/gms/Gossiper.java| 9 + .../org/apache/cassandra/hints/HintsCatalog.java | 5 +- .../cassandra/index/SecondaryIndexManager.java | 11 +- .../cassandra/io/sstable/IndexSummaryManager.java | 11 ++ .../cassandra/io/sstable/format/SSTableReader.java | 6 +- .../org/apache/cassandra/net/MessagingService.java | 2 + .../cassandra/net/OutboundTcpConnection.java | 2 +- .../service/PendingRangeCalculatorService.java | 11 +- .../apache/cassandra/service/StorageService.java | 24 ++- .../cassandra/streaming/StreamCoordinator.java | 13 ++ .../apache/cassandra/streaming/StreamSession.java | 9 + .../org/apache/cassandra/utils/ExecutorUtils.java | 151 +++ .../org/apache/cassandra/utils/concurrent/Ref.java | 14 +- .../apache/cassandra/utils/memory/BufferPool.java | 10 +- .../cassandra/utils/memory/MemtablePool.java | 8 +- test/conf/logback-dtest.xml| 18 +- .../org/apache/cassandra/distributed/Cluster.java | 30 ++- .../cassandra/distributed/UpgradeableCluster.java | 30 +-- .../cassandra/distributed/api/IInstance.java | 3 +- .../cassandra/distributed/api/IInstanceConfig.java | 1 + .../cassandra/distributed/api/IMessageFilters.java | 6 +- .../distributed/impl/AbstractCluster.java | 154 +--- .../impl/DelegatingInvokableInstance.java | 5 +- .../cassandra/distributed/impl/Instance.java | 146 +++ .../distributed/impl/InstanceClassLoader.java | 9 +- .../cassandra/distributed/impl/InstanceConfig.java | 28 ++- .../distributed/impl/IsolatedExecutor.java | 47 - .../cassandra/distributed/impl/MessageFilters.java | 31 +--- .../distributed/test/DistributedTestBase.java | 26 ++- .../cassandra/distributed/test/GossipTest.java | 14 +- .../distributed/test/ResourceLeakTest.java | 202 + .../cassandra/concurrent/SEPExecutorTest.java | 3 +- 41 files changed, 848 insertions(+), 261 deletions(-) create mode 100644 src/java/org/apache/cassandra/utils/ExecutorUtils.java create mode 100644 test/distributed/org/apache/cassandra/distributed/test/ResourceLeakTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: Allow instance class loaders to be garbage collected for inJVM dtest
This is an automated email from the ASF dual-hosted git repository. benedict pushed a commit to branch cassandra-2.2 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-2.2 by this push: new 8dcaa12 Allow instance class loaders to be garbage collected for inJVM dtest 8dcaa12 is described below commit 8dcaa12baa97ce870f23ff9045f968f2fa28b2cc Author: Jon Meredith AuthorDate: Thu Aug 15 10:12:06 2019 -0600 Allow instance class loaders to be garbage collected for inJVM dtest Backport support optional network/gossip test features, instance generations and subnet support from trunk. Backport MessageFilter and cluster builder API changes to match trunk (except for the change in namespace for Verbs). Add a test for repeatedly creating/tearing down in-JVM dtest clusters to help find resource leaks. Change IsolatedExecutor to clean up on an executor with zero core threads so that it exits sooner enabling GC on the InstanceClassLoader. Moved classloader close after shutdown to prevent improve logging when the isolated executor is shutdown. Update the logback config for dtests to make it obvious which log threads are for instances vs the main logger. Disable native library loading until it can be reinstated (tracked in CASSANDRA-15170). Shutdown various executors and threads that were preventing the instance classloader from being unloaded. Add test-jvm-dtest-some ant target Patch by Jon Meredith; Reviewed by Alex Petrov and Benedict Elliott Smith for CASSANDRA-15170 --- CHANGES.txt| 1 + build.xml | 22 ++- .../cassandra/concurrent/InfiniteLoopExecutor.java | 2 +- .../cassandra/concurrent/ScheduledExecutors.java | 11 +- .../cassandra/concurrent/SharedExecutorPool.java | 4 +- .../apache/cassandra/concurrent/StageManager.java | 8 +- .../org/apache/cassandra/db/BatchlogManager.java | 11 +- .../org/apache/cassandra/db/ColumnFamilyStore.java | 14 +- .../apache/cassandra/db/HintedHandOffManager.java | 9 + .../apache/cassandra/db/commitlog/CommitLog.java | 1 + src/java/org/apache/cassandra/gms/Gossiper.java| 9 + .../cassandra/io/sstable/IndexSummaryManager.java | 11 ++ .../cassandra/io/sstable/format/SSTableReader.java | 5 +- .../org/apache/cassandra/net/MessagingService.java | 7 +- .../cassandra/net/OutboundTcpConnection.java | 2 +- .../apache/cassandra/service/CassandraDaemon.java | 3 +- .../service/PendingRangeCalculatorService.java | 11 +- .../apache/cassandra/service/StorageService.java | 29 ++- .../cassandra/streaming/StreamCoordinator.java | 13 ++ .../cassandra/utils/BackgroundActivityMonitor.java | 12 +- src/java/org/apache/cassandra/utils/CLibrary.java | 9 +- .../org/apache/cassandra/utils/ExecutorUtils.java | 151 .../utils/NanoTimeToCurrentTimeMillis.java | 14 +- .../org/apache/cassandra/utils/concurrent/Ref.java | 8 +- .../cassandra/utils/memory/MemtablePool.java | 8 +- test/conf/logback-dtest.xml| 20 +- .../org/apache/cassandra/distributed/Cluster.java | 20 +- .../cassandra/distributed/UpgradeableCluster.java | 22 +-- .../api/{IMessageFilters.java => Feature.java} | 28 +-- .../cassandra/distributed/api/IInstance.java | 2 + .../cassandra/distributed/api/IInstanceConfig.java | 1 + .../cassandra/distributed/api/IMessageFilters.java | 6 +- .../distributed/impl/AbstractCluster.java | 138 +- .../cassandra/distributed/impl/Instance.java | 165 ++--- .../distributed/impl/InstanceClassLoader.java | 9 +- .../cassandra/distributed/impl/InstanceConfig.java | 28 ++- .../distributed/impl/IsolatedExecutor.java | 47 - .../cassandra/distributed/impl/MessageFilters.java | 31 +--- .../distributed/test/DistributedTestBase.java | 28 +++ .../distributed/test/ResourceLeakTest.java | 201 + .../cassandra/concurrent/SEPExecutorTest.java | 3 +- 41 files changed, 893 insertions(+), 231 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index eabdcaa..caea0f4 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -8,6 +8,7 @@ * Fixing invalid CQL in security documentation (CASSANDRA-15020) * Make tools/bin/token-generator py2/3 compatible (CASSANDRA-15012) * Multi-version in-JVM dtests (CASSANDRA-14937) + * Allow instance class loaders to be garbage collected for inJVM dtest (CASSANDRA-15170) 2.2.14 diff --git a/build.xml b/build.xml index d522b59..b42c3ed 100644 --- a/build.xml +++ b/build.xml @@ -1818,6 +1818,10 @@ + + + + @@ -1851,10 +1855,24 @@ - + - + +
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. benedict pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 3df63ed054ebaaf058fb6f93d149fca38eb687e1 Merge: bb126c0 51c0f6b Author: Jon Meredith AuthorDate: Thu Aug 15 14:21:23 2019 -0600 Merge branch 'cassandra-3.0' into cassandra-3.11 CHANGES.txt| 1 + build.xml | 18 +- .../apache/cassandra/batchlog/BatchlogManager.java | 7 +- .../cassandra/concurrent/InfiniteLoopExecutor.java | 2 +- .../cassandra/concurrent/ScheduledExecutors.java | 11 +- .../cassandra/concurrent/SharedExecutorPool.java | 4 +- .../apache/cassandra/concurrent/StageManager.java | 8 +- .../org/apache/cassandra/db/ColumnFamilyStore.java | 18 +- .../apache/cassandra/db/commitlog/CommitLog.java | 1 + src/java/org/apache/cassandra/gms/Gossiper.java| 9 + .../org/apache/cassandra/hints/HintsCatalog.java | 5 +- .../cassandra/index/SecondaryIndexManager.java | 11 +- .../cassandra/io/sstable/IndexSummaryManager.java | 11 ++ .../cassandra/io/sstable/format/SSTableReader.java | 6 +- .../org/apache/cassandra/net/MessagingService.java | 2 + .../cassandra/net/OutboundTcpConnection.java | 2 +- .../service/PendingRangeCalculatorService.java | 11 +- .../apache/cassandra/service/StorageService.java | 24 ++- .../cassandra/streaming/StreamCoordinator.java | 13 ++ .../apache/cassandra/streaming/StreamSession.java | 9 + .../org/apache/cassandra/utils/ExecutorUtils.java | 151 +++ .../org/apache/cassandra/utils/concurrent/Ref.java | 14 +- .../apache/cassandra/utils/memory/BufferPool.java | 10 +- .../cassandra/utils/memory/MemtablePool.java | 8 +- test/conf/logback-dtest.xml| 18 +- .../org/apache/cassandra/distributed/Cluster.java | 30 ++- .../cassandra/distributed/UpgradeableCluster.java | 30 +-- .../cassandra/distributed/api/IInstance.java | 3 +- .../cassandra/distributed/api/IInstanceConfig.java | 1 + .../cassandra/distributed/api/IMessageFilters.java | 6 +- .../distributed/impl/AbstractCluster.java | 154 +--- .../impl/DelegatingInvokableInstance.java | 5 +- .../cassandra/distributed/impl/Instance.java | 146 +++ .../distributed/impl/InstanceClassLoader.java | 9 +- .../cassandra/distributed/impl/InstanceConfig.java | 28 ++- .../distributed/impl/IsolatedExecutor.java | 47 - .../cassandra/distributed/impl/MessageFilters.java | 31 +--- .../distributed/test/DistributedTestBase.java | 26 ++- .../cassandra/distributed/test/GossipTest.java | 14 +- .../distributed/test/ResourceLeakTest.java | 202 + .../cassandra/concurrent/SEPExecutorTest.java | 3 +- 41 files changed, 848 insertions(+), 261 deletions(-) diff --cc CHANGES.txt index 617bcc4,e956796..772446d --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -37,12 -33,10 +37,13 @@@ Merged from 2.2 * Refactor Circle CI configuration (CASSANDRA-14806) * Fixing invalid CQL in security documentation (CASSANDRA-15020) * Multi-version in-JVM dtests (CASSANDRA-14937) + * Allow instance class loaders to be garbage collected for inJVM dtest (CASSANDRA-15170) -3.0.18 +3.11.4 + * Make stop-server.bat wait for Cassandra to terminate (CASSANDRA-14829) + * Correct sstable sorting for garbagecollect and levelled compaction (CASSANDRA-14870) +Merged from 3.0: * Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser * Add a script to make running the cqlsh tests in cassandra repo easier (CASSANDRA-14951) * If SizeEstimatesRecorder misses a 'onDropTable' notification, the size_estimates table will never be cleared for that table. (CASSANDRA-14905) diff --cc src/java/org/apache/cassandra/concurrent/ScheduledExecutors.java index 5e3e5cf,13d27a8..3da4569 --- a/src/java/org/apache/cassandra/concurrent/ScheduledExecutors.java +++ b/src/java/org/apache/cassandra/concurrent/ScheduledExecutors.java @@@ -48,12 -46,8 +51,8 @@@ public class ScheduledExecutor public static final DebuggableScheduledThreadPoolExecutor optionalTasks = new DebuggableScheduledThreadPoolExecutor("OptionalTasks"); @VisibleForTesting - public static void shutdownAndWait() throws InterruptedException + public static void shutdownAndWait(long timeout, TimeUnit unit) throws InterruptedException, TimeoutException { - ExecutorService[] executors = new ExecutorService[] { scheduledFastTasks, scheduledTasks, nonPeriodicTasks, optionalTasks }; - for (ExecutorService executor : executors) - executor.shutdownNow(); - for (ExecutorService executor : executors) - executor.awaitTermination(60, TimeUnit.SECONDS); -ExecutorUtils.shutdownNowAndWait(timeout, unit,
[cassandra] branch cassandra-3.0 updated (54aeb50 -> 51c0f6b)
This is an automated email from the ASF dual-hosted git repository. benedict pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 54aeb50 ninja fix CHANGES.txt for #14952 new 8dcaa12 Allow instance class loaders to be garbage collected for inJVM dtest new 51c0f6b Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + build.xml | 18 +- .../apache/cassandra/batchlog/BatchlogManager.java | 7 +- .../cassandra/concurrent/InfiniteLoopExecutor.java | 2 +- .../cassandra/concurrent/ScheduledExecutors.java | 11 +- .../cassandra/concurrent/SharedExecutorPool.java | 4 +- .../apache/cassandra/concurrent/StageManager.java | 8 +- .../org/apache/cassandra/db/ColumnFamilyStore.java | 14 +- .../apache/cassandra/db/commitlog/CommitLog.java | 1 + src/java/org/apache/cassandra/gms/Gossiper.java| 9 + .../org/apache/cassandra/hints/HintsCatalog.java | 5 +- .../cassandra/index/SecondaryIndexManager.java | 12 +- .../cassandra/io/sstable/IndexSummaryManager.java | 11 ++ .../cassandra/io/sstable/format/SSTableReader.java | 5 +- .../org/apache/cassandra/net/MessagingService.java | 2 + .../cassandra/net/OutboundTcpConnection.java | 2 +- .../apache/cassandra/service/CassandraDaemon.java | 3 +- .../service/PendingRangeCalculatorService.java | 11 +- .../apache/cassandra/service/StorageService.java | 32 +++- .../cassandra/streaming/StreamCoordinator.java | 13 ++ .../cassandra/utils/BackgroundActivityMonitor.java | 12 +- .../org/apache/cassandra/utils/ExecutorUtils.java | 151 .../utils/NanoTimeToCurrentTimeMillis.java | 14 +- .../org/apache/cassandra/utils/concurrent/Ref.java | 14 +- .../apache/cassandra/utils/memory/BufferPool.java | 10 +- .../cassandra/utils/memory/MemtablePool.java | 7 +- test/conf/logback-dtest.xml| 18 +- .../org/apache/cassandra/distributed/Cluster.java | 30 ++- .../cassandra/distributed/UpgradeableCluster.java | 30 +-- .../cassandra/distributed/api/IInstance.java | 3 +- .../cassandra/distributed/api/IInstanceConfig.java | 1 + .../cassandra/distributed/api/IMessageFilters.java | 6 +- .../distributed/impl/AbstractCluster.java | 154 +--- .../impl/DelegatingInvokableInstance.java | 5 +- .../cassandra/distributed/impl/Instance.java | 159 .../distributed/impl/InstanceClassLoader.java | 9 +- .../cassandra/distributed/impl/InstanceConfig.java | 28 ++- .../distributed/impl/IsolatedExecutor.java | 47 - .../cassandra/distributed/impl/MessageFilters.java | 31 +--- .../distributed/test/DistributedTestBase.java | 26 ++- .../cassandra/distributed/test/GossipTest.java | 14 +- .../distributed/test/ResourceLeakTest.java | 201 + .../cassandra/concurrent/SEPExecutorTest.java | 3 +- 43 files changed, 871 insertions(+), 273 deletions(-) create mode 100644 src/java/org/apache/cassandra/utils/ExecutorUtils.java create mode 100644 test/distributed/org/apache/cassandra/distributed/test/ResourceLeakTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14740) BlockingReadRepair does not maintain monotonicity during range movements
[ https://issues.apache.org/jira/browse/CASSANDRA-14740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-14740: - Status: Open (was: Patch Available) > BlockingReadRepair does not maintain monotonicity during range movements > > > Key: CASSANDRA-14740 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14740 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Coordination >Reporter: Benedict >Assignee: Benedict >Priority: Urgent > Labels: correctness > Fix For: 4.0 > > > The BlockingReadRepair code introduced by CASSANDRA-10726 requires that each > of the queried nodes are written to, but pending nodes are not considered. > If there is a pending range movement, one of these writes may be ‘lost’ when > the range movement completes. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15269) Cassandra fails to process OperationExecutionException which causes ClassCastException
[ https://issues.apache.org/jira/browse/CASSANDRA-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liudmila Kornilova updated CASSANDRA-15269: --- Test and Documentation Plan: Documented, not tested (should not affect any tests) Status: Patch Available (was: Open) > Cassandra fails to process OperationExecutionException which causes > ClassCastException > -- > > Key: CASSANDRA-15269 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15269 > Project: Cassandra > Issue Type: Bug > Components: CQL/Interpreter >Reporter: Liudmila Kornilova >Assignee: Liudmila Kornilova >Priority: Normal > > While working on CASSANDRA-15232 I noticed that OperationExecutionException > is not processed correctly. > How to reproduce the issue: > 1. {{create table d (numerator decimal primary key, denominator decimal);}} > 2. {{insert into d (numerator, denominator) values > (123456789112345678921234567893123456, 2);}} > 3. {{select numerator % denominator from d;}} > What happens: > 1. remainder operation throws ArithmeticException (BigDecimal:1854) > 2. The exception is wrapped in OperationExecutionException > 3. ClassCastException appears (OperationExecutionException cannot be cast to > FunctionExecutionException at ErrorMessage.java:280) > What should happen: > OperationExecutionException with message "the operation 'decimal % decimal' > failed: Division impossible" should be delivered to user > Note that after fixing CASSANDRA-15232 {{select numerator % denominator from > d;}} will produce correct result of remainder operation. > Currently I am not aware of other cases when OperationExecutionException may > be treated as FunctionExecutionException -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15269) Cassandra fails to process OperationExecutionException which causes ClassCastException
[ https://issues.apache.org/jira/browse/CASSANDRA-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911178#comment-16911178 ] Liudmila Kornilova commented on CASSANDRA-15269: Each {{ExceptionCode}} correspond to specific exception class. {{OperationExecutionException}} did not have it's own code, instead it used {{FUNCTION_FAILURE}} code. So the exception was treated as {{FunctionExecutionException}} I added {{OPERATION_FAILURE}} code How to test: revert locally d60e7988736ed4358595e9c781b110a5bbb5f812 commit (void result truncate in decimal operations) Or throw {{ArithmeticException}} in {{DecimalType.mod}} {noformat} try (CqlSession session = CqlSession.builder().build()) { session.execute("drop table if exists d"); session.execute("create table d (n decimal primary key, d decimal)"); session.execute("insert into d (n, d) values (123456789112345678921234567893123456, 2)"); session.execute("select n % d from my.d"); } {noformat} What used to be: {noformat} Exception in thread "main" com.datastax.oss.driver.api.core.servererrors.ServerError: java.lang.ClassCastException: org.apache.cassandra.exceptions.OperationExecutionException cannot be cast to org.apache.cassandra.exceptions.FunctionExecutionException at com.datastax.oss.driver.api.core.servererrors.ServerError.copy(ServerError.java:54) {noformat} What is now: {noformat} Exception in thread "main" com.datastax.oss.driver.api.core.DriverExecutionException at com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:150) ... Caused by: java.lang.IllegalArgumentException: Unsupported error code: 5888 {noformat} New error code should be added to Datastax Cassandra Native Protocol in [ProtocolConstants.java|https://github.com/datastax/native-protocol/blob/1.x/src/main/java/com/datastax/oss/protocol/internal/ProtocolConstants.java#L70] If this commit is merged, I'll write a patch to the protocol > Cassandra fails to process OperationExecutionException which causes > ClassCastException > -- > > Key: CASSANDRA-15269 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15269 > Project: Cassandra > Issue Type: Bug > Components: CQL/Interpreter >Reporter: Liudmila Kornilova >Assignee: Liudmila Kornilova >Priority: Normal > > While working on CASSANDRA-15232 I noticed that OperationExecutionException > is not processed correctly. > How to reproduce the issue: > 1. {{create table d (numerator decimal primary key, denominator decimal);}} > 2. {{insert into d (numerator, denominator) values > (123456789112345678921234567893123456, 2);}} > 3. {{select numerator % denominator from d;}} > What happens: > 1. remainder operation throws ArithmeticException (BigDecimal:1854) > 2. The exception is wrapped in OperationExecutionException > 3. ClassCastException appears (OperationExecutionException cannot be cast to > FunctionExecutionException at ErrorMessage.java:280) > What should happen: > OperationExecutionException with message "the operation 'decimal % decimal' > failed: Division impossible" should be delivered to user > Note that after fixing CASSANDRA-15232 {{select numerator % denominator from > d;}} will produce correct result of remainder operation. > Currently I am not aware of other cases when OperationExecutionException may > be treated as FunctionExecutionException -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15269) Cassandra fails to process OperationExecutionException which causes ClassCastException
[ https://issues.apache.org/jira/browse/CASSANDRA-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liudmila Kornilova updated CASSANDRA-15269: --- Bug Category: Parent values: Correctness(12982)Level 1 values: API / Semantic Implementation(12988) Complexity: Low Hanging Fruit Discovered By: Code Inspection Severity: Low Status: Open (was: Triage Needed) > Cassandra fails to process OperationExecutionException which causes > ClassCastException > -- > > Key: CASSANDRA-15269 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15269 > Project: Cassandra > Issue Type: Bug > Components: CQL/Interpreter >Reporter: Liudmila Kornilova >Assignee: Liudmila Kornilova >Priority: Normal > > While working on CASSANDRA-15232 I noticed that OperationExecutionException > is not processed correctly. > How to reproduce the issue: > 1. {{create table d (numerator decimal primary key, denominator decimal);}} > 2. {{insert into d (numerator, denominator) values > (123456789112345678921234567893123456, 2);}} > 3. {{select numerator % denominator from d;}} > What happens: > 1. remainder operation throws ArithmeticException (BigDecimal:1854) > 2. The exception is wrapped in OperationExecutionException > 3. ClassCastException appears (OperationExecutionException cannot be cast to > FunctionExecutionException at ErrorMessage.java:280) > What should happen: > OperationExecutionException with message "the operation 'decimal % decimal' > failed: Division impossible" should be delivered to user > Note that after fixing CASSANDRA-15232 {{select numerator % denominator from > d;}} will produce correct result of remainder operation. > Currently I am not aware of other cases when OperationExecutionException may > be treated as FunctionExecutionException -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15274) Multiple Corrupt datafiles across entire environment
[ https://issues.apache.org/jira/browse/CASSANDRA-15274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911135#comment-16911135 ] feroz shaik commented on CASSANDRA-15274: - [~benedict]- can you please share that SSTableExport.Java (with CRC disabled) which is 2.2 compatible if you have it handy. I was trying to add that line as you suggested but had some issues. > Multiple Corrupt datafiles across entire environment > - > > Key: CASSANDRA-15274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15274 > Project: Cassandra > Issue Type: Bug > Components: Local/Compaction >Reporter: Phil O Conduin >Priority: Normal > > Cassandra Version: 2.2.13 > PRE-PROD environment. > * 2 datacenters. > * 9 physical servers in each datacenter - (_Cisco UCS C220 M4 SFF_) > * 4 Cassandra instances on each server (cass_a, cass_b, cass_c, cass_d) > * 72 Cassandra instances across the 2 data centres, 36 in site A, 36 in site > B. > We also have 2 Reaper Nodes we use for repair. One reaper node in each > datacenter each running with its own Cassandra back end in a cluster together. > OS Details [Red Hat Linux] > cass_a@x 0 10:53:01 ~ $ uname -a > Linux x 3.10.0-957.5.1.el7.x86_64 #1 SMP Wed Dec 19 10:46:58 EST 2018 x86_64 > x86_64 x86_64 GNU/Linux > cass_a@x 0 10:57:31 ~ $ cat /etc/*release > NAME="Red Hat Enterprise Linux Server" > VERSION="7.6 (Maipo)" > ID="rhel" > Storage Layout > cass_a@xx 0 10:46:28 ~ $ df -h > Filesystem Size Used Avail Use% Mounted on > /dev/mapper/vg01-lv_root 20G 2.2G 18G 11% / > devtmpfs 63G 0 63G 0% /dev > tmpfs 63G 0 63G 0% /dev/shm > tmpfs 63G 4.1G 59G 7% /run > tmpfs 63G 0 63G 0% /sys/fs/cgroup > >> 4 cassandra instances > /dev/sdd 1.5T 802G 688G 54% /data/ssd4 > /dev/sda 1.5T 798G 692G 54% /data/ssd1 > /dev/sdb 1.5T 681G 810G 46% /data/ssd2 > /dev/sdc 1.5T 558G 932G 38% /data/ssd3 > Cassandra load is about 200GB and the rest of the space is snapshots > CPU > cass_a@x 127 10:58:47 ~ $ lscpu | grep -E '^Thread|^Core|^Socket|^CPU\(' > CPU(s): 64 > Thread(s) per core: 2 > Core(s) per socket: 16 > Socket(s): 2 > *Description of problem:* > During repair of the cluster, we are seeing multiple corruptions in the log > files on a lot of instances. There seems to be no pattern to the corruption. > It seems that the repair job is finding all the corrupted files for us. The > repair will hang on the node where the corrupted file is found. To fix this > we remove/rename the datafile and bounce the Cassandra instance. Our > hardware/OS team have stated there is no problem on their side. I do not > believe it the repair causing the corruption. > > So let me give you an example of a corrupted file and maybe someone might be > able to work through it with me? > When this corrupted file was reported in the log it looks like it was the > repair that found it. > $ journalctl -u cassmeta-cass_b.service --since "2019-08-07 22:25:00" --until > "2019-08-07 22:45:00" > Aug 07 22:30:33 cassandra[34611]: INFO 21:30:33 Writing > Memtable-compactions_in_progress@830377457(0.008KiB serialized bytes, 1 ops, > 0%/0% of on/off-heap limit) > Aug 07 22:30:33 cassandra[34611]: ERROR 21:30:33 Failed creating a merkle > tree for [repair #9587a200-b95a-11e9-8920-9f72868b8375 on KeyspaceMetadata/x, > (-1476350953672479093,-1474461 > Aug 07 22:30:33 cassandra[34611]: ERROR 21:30:33 Exception in thread > Thread[ValidationExecutor:825,1,main] > Aug 07 22:30:33 cassandra[34611]: org.apache.cassandra.io.FSReadError: > org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: > /x/ssd2/data/KeyspaceMetadata/x-1e453cb0 > Aug 07 22:30:33 cassandra[34611]: at > org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:365) > ~[apache-cassandra-2.2.13.jar:2.2.13] > Aug 07 22:30:33 cassandra[34611]: at > org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:361) > ~[apache-cassandra-2.2.13.jar:2.2.13] > Aug 07 22:30:33 cassandra[34611]: at > org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:340) > ~[apache-cassandra-2.2.13.jar:2.2.13] > Aug 07 22:30:33 cassandra[34611]: at > org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382) > ~[apache-cassandra-2.2.13.jar:2.2.13] > Aug 07 22:30:33 cassandra[34611]: at > org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:366) >
[jira] [Commented] (CASSANDRA-15263) LegacyLayout RangeTombstoneList throws java.lang.NullPointerException: null
[ https://issues.apache.org/jira/browse/CASSANDRA-15263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911131#comment-16911131 ] feroz shaik commented on CASSANDRA-15263: - Cheers [~benedict] . we have now engaged our engineering to modify the table as to eliminate that dependency of null-column2. We are also looking at temporary options to shutdown the application that reads/writes to this table during the upgrade window. I will keep the chain posted with new developments from ours side. Thank you very much once again to help identify the issue. > LegacyLayout RangeTombstoneList throws java.lang.NullPointerException: null > --- > > Key: CASSANDRA-15263 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15263 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: feroz shaik >Assignee: Benedict >Priority: Normal > Labels: 2.1.16, 3.11.4 > Attachments: sample.system.log, schema.txt, > sstabledump_sal_purge_d03.json, sstablemetadata_sal_purge_d03, > stack_trace.txt, system.log, system.log, system.log, system.log, > system_latest.log > > > We have hit a problem today while upgrading from 2.1.16 to 3.11.4. > we encountered this as soon as the first node started up with 3.11.4 > The full error stack is attached - [^stack_trace.txt] > > The below errors continued in the log file as long as the process was up. > ERROR [Native-Transport-Requests-12] 2019-08-06 03:00:47,135 > ErrorMessage.java:384 - Unexpected exception during request > java.lang.NullPointerException: null > ERROR [Native-Transport-Requests-8] 2019-08-06 03:00:48,778 > ErrorMessage.java:384 - Unexpected exception during request > java.lang.NullPointerException: null > ERROR [Native-Transport-Requests-13] 2019-08-06 03:00:57,454 > > The nodetool version says 3.11.4 and the no of connections on native por t- > 9042 was similar to other nodes. The exceptions were scary that we had to > call off the change. Any help and insights to this problem from the community > is appreciated. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15269) Cassandra fails to process OperationExecutionException which causes ClassCastException
[ https://issues.apache.org/jira/browse/CASSANDRA-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liudmila Kornilova reassigned CASSANDRA-15269: -- Assignee: Liudmila Kornilova > Cassandra fails to process OperationExecutionException which causes > ClassCastException > -- > > Key: CASSANDRA-15269 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15269 > Project: Cassandra > Issue Type: Bug > Components: CQL/Interpreter >Reporter: Liudmila Kornilova >Assignee: Liudmila Kornilova >Priority: Normal > > While working on CASSANDRA-15232 I noticed that OperationExecutionException > is not processed correctly. > How to reproduce the issue: > 1. {{create table d (numerator decimal primary key, denominator decimal);}} > 2. {{insert into d (numerator, denominator) values > (123456789112345678921234567893123456, 2);}} > 3. {{select numerator % denominator from d;}} > What happens: > 1. remainder operation throws ArithmeticException (BigDecimal:1854) > 2. The exception is wrapped in OperationExecutionException > 3. ClassCastException appears (OperationExecutionException cannot be cast to > FunctionExecutionException at ErrorMessage.java:280) > What should happen: > OperationExecutionException with message "the operation 'decimal % decimal' > failed: Division impossible" should be delivered to user > Note that after fixing CASSANDRA-15232 {{select numerator % denominator from > d;}} will produce correct result of remainder operation. > Currently I am not aware of other cases when OperationExecutionException may > be treated as FunctionExecutionException -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org