[jira] [Updated] (CASSANDRA-14808) Support ORDER BY with 2ndary Indexes
[
https://issues.apache.org/jira/browse/CASSANDRA-14808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yan Yao updated CASSANDRA-14808:
Description:
Suppose we have a generic table:
{code:java}
CREATE TABLE base_table(
partition1 uuid,
...
partitionN uuid,
static_column text static,
clustering1 uuid,
...
clusteringM uuid,
regular text,
list_text list,
set_text set,
map_int_text map,
PRIMARY KEY((partition1, ..., partitionN), clustering1, ... , clusteringN)
);
{code}
And create an index on _regular text_ column, the schema of the index table
will be:
{code:java}
CREATE TABLE regular_idx(
regular text,
partitionColumns blob,
clustering1 uuid,
...
clusteringM uuid,
PRIMARY KEY((regular), partitionColumns, clustering1, ..., clusteringM)
);
{code}
Then it's possible to execute queries like:
{code:java}
SELECT * FROM base_table WHERE regular = AND partition1 =
AND ... AND partitionN = ORDER BY
clustering1;{code}
However, CQL3 would check if the _secondary index_ is used WITH _order by_
during prepare a select statement, and throw an exception at once for queries
like above.
Could we support ORDER BY with 2ndary Indexes at least for above query pattern?
was:
Suppose we have a generic table:
{code:java}
CREATE TABLE base_table(
partition1 uuid,
...
partitionN uuid,
static_column text static,
clustering1 uuid,
...
clusteringM uuid,
regular text,
list_text list,
set_text set,
map_int_text map,
PRIMARY KEY((partition1, ..., partitionN), clustering1, ... , clusteringN)
);
{code}
And create an index on _regular text_ column, the schema of the index table
will be:
{code:java}
CREATE TABLE regular_idx(
regular text,
partitionColumns blob,
clustering1 uuid,
...
clusteringM uuid,
PRIMARY KEY((regular), partitionColumns, clustering1, ..., clusteringM)
);
{code}
Then it's possible to execute queries like:
{code:java}
SELECT * FROM base_table WHERE regular = AND partition1 =
AND ... AND partitionN = ORDER BY
clustering1;{code}
However, CQL3 would check if the _secondary index_ is used WITH _order by_
during prepare a select statement, and throw an exception at once for queries
like above.
Could we support ORDER BY with 2ndary Indexes?
> Support ORDER BY with 2ndary Indexes
>
>
> Key: CASSANDRA-14808
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14808
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
> Environment: Apache Cassandra 3.11.3
>Reporter: Yan Yao
>Priority: Major
> Fix For: 3.11.4, 4.0
>
>
> Suppose we have a generic table:
> {code:java}
> CREATE TABLE base_table(
> partition1 uuid,
> ...
> partitionN uuid,
> static_column text static,
> clustering1 uuid,
> ...
> clusteringM uuid,
> regular text,
> list_text list,
> set_text set,
> map_int_text map,
> PRIMARY KEY((partition1, ..., partitionN), clustering1, ... , clusteringN)
> );
> {code}
>
> And create an index on _regular text_ column, the schema of the index table
> will be:
> {code:java}
> CREATE TABLE regular_idx(
> regular text,
> partitionColumns blob,
> clustering1 uuid,
> ...
> clusteringM uuid,
> PRIMARY KEY((regular), partitionColumns, clustering1, ..., clusteringM)
> );
> {code}
>
> Then it's possible to execute queries like:
> {code:java}
> SELECT * FROM base_table WHERE regular = AND partition1 =
> AND ... AND partitionN = ORDER BY
> clustering1;{code}
>
> However, CQL3 would check if the _secondary index_ is used WITH _order by_
> during prepare a select statement, and throw an exception at once for queries
> like above.
> Could we support ORDER BY with 2ndary Indexes at least for above query
> pattern?
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14808) Support ORDER BY with 2ndary Indexes
[
https://issues.apache.org/jira/browse/CASSANDRA-14808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yan Yao updated CASSANDRA-14808:
Description:
Suppose we have a generic table:
{code:java}
CREATE TABLE base_table(
partition1 uuid,
...
partitionN uuid,
static_column text static,
clustering1 uuid,
...
clusteringM uuid,
regular text,
list_text list,
set_text set,
map_int_text map,
PRIMARY KEY((partition1, ..., partitionN), clustering1, ... , clusteringN)
);
{code}
And create an index on _regular text_ column, the schema of the index table
will be:
{code:java}
CREATE TABLE regular_idx(
regular text,
partitionColumns blob,
clustering1 uuid,
...
clusteringM uuid,
PRIMARY KEY((regular), partitionColumns, clustering1, ..., clusteringM)
);
{code}
Then it's possible to execute queries like:
{code:java}
SELECT * FROM base_table WHERE regular = AND partition1 =
AND ... AND partitionN = ORDER BY
clustering1;{code}
However, CQL3 would check if the _secondary index_ is used WITH _order by_
during prepare a select statement, and throw an exception at once for queries
like above.
Could we support ORDER BY with 2ndary Indexes?
was:
Suppose we have a generic table:
{code:java}
CREATE TABLE base_table(
partition1 uuid,
...
partitionN uuid,
static_column text static,
clustering1 uuid,
...
clusteringM uuid,
regular text,
list_text list,
set_text set,
map_int_text map,
PRIMARY KEY((partition1, ..., partitionN), clustering1, ... , clusteringN)
);
{code}
And create an index on _regular text_ column, the schema of the index table
will be:
{code:java}
CREATE TABLE regular_idx(
regular text,
partitionColumns blob,
clustering1 uuid,
...
clusteringM uuid,
PRIMARY KEY((regular), partitionColumns, clustering1, ..., clusteringM)
);
{code}
Then it's possible to execute queries like:
{code:java}
SELECT * FROM base_table WHERE regular = AND partition1 =
AND ... AND partitionN = ORDER BY
clustering1;{code}
However, CQL3 would check if the _secondary index_ is used WITH _order by_
during prepare a select statement, and throw an exception at once for queries
like above.
Could we support ORDER BY with 2ndary Indexes?
> Support ORDER BY with 2ndary Indexes
>
>
> Key: CASSANDRA-14808
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14808
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
> Environment: Apache Cassandra 3.11.3
>Reporter: Yan Yao
>Priority: Major
> Fix For: 3.11.4, 4.0
>
>
> Suppose we have a generic table:
> {code:java}
> CREATE TABLE base_table(
> partition1 uuid,
> ...
> partitionN uuid,
> static_column text static,
> clustering1 uuid,
> ...
> clusteringM uuid,
> regular text,
> list_text list,
> set_text set,
> map_int_text map,
> PRIMARY KEY((partition1, ..., partitionN), clustering1, ... , clusteringN)
> );
> {code}
>
> And create an index on _regular text_ column, the schema of the index table
> will be:
> {code:java}
> CREATE TABLE regular_idx(
> regular text,
> partitionColumns blob,
> clustering1 uuid,
> ...
> clusteringM uuid,
> PRIMARY KEY((regular), partitionColumns, clustering1, ..., clusteringM)
> );
> {code}
>
> Then it's possible to execute queries like:
> {code:java}
> SELECT * FROM base_table WHERE regular = AND partition1 =
> AND ... AND partitionN = ORDER BY
> clustering1;{code}
>
> However, CQL3 would check if the _secondary index_ is used WITH _order by_
> during prepare a select statement, and throw an exception at once for queries
> like above.
> Could we support ORDER BY with 2ndary Indexes?
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14808) Support ORDER BY with 2ndary Indexes
Yan Yao created CASSANDRA-14808:
---
Summary: Support ORDER BY with 2ndary Indexes
Key: CASSANDRA-14808
URL: https://issues.apache.org/jira/browse/CASSANDRA-14808
Project: Cassandra
Issue Type: Bug
Components: CQL
Environment: Apache Cassandra 3.11.3
Reporter: Yan Yao
Fix For: 3.11.4, 4.0
Suppose we have a generic table:
{code:java}
CREATE TABLE base_table(
partition1 uuid,
...
partitionN uuid,
static_column text static,
clustering1 uuid,
...
clusteringM uuid,
regular text,
list_text list,
set_text set,
map_int_text map,
PRIMARY KEY((partition1, ..., partitionN), clustering1, ... , clusteringN)
);
{code}
And create an index on _regular text_ column, the schema of the index table
will be:
{code:java}
CREATE TABLE regular_idx(
regular text,
partitionColumns blob,
clustering1 uuid,
...
clusteringM uuid,
PRIMARY KEY((regular), partitionColumns, clustering1, ..., clusteringM)
);
{code}
Then it's possible to execute queries like:
{code:java}
SELECT * FROM base_table WHERE regular = AND partition1 =
AND ... AND partitionN = ORDER BY
clustering1;{code}
However, CQL3 would check if the _secondary index_ is used WITH _order by_
during prepare a select statement, and throw an exception at once for queries
like above.
Could we support ORDER BY with 2ndary Indexes?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14654) Reduce heap pressure during compactions
[ https://issues.apache.org/jira/browse/CASSANDRA-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644154#comment-16644154 ] Chris Lohfink commented on CASSANDRA-14654: --- Pushed change to just skip the invalidation and allow preemptiveopen to be hot property changed. Also made disabling keycache stop the generating of keycachekey load. In some workloads changing that can 2x the compaction throughput while cutting the allocation pressure on the heap in half. This is more for special cases so defaults is no change in behavior. > Reduce heap pressure during compactions > --- > > Key: CASSANDRA-14654 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14654 > Project: Cassandra > Issue Type: Improvement > Components: Compaction >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: Performance, pull-request-available > Fix For: 4.x > > Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png, > screenshot-4.png > > Time Spent: 10m > Remaining Estimate: 0h > > Small partition compactions are painfully slow with a lot of overhead per > partition. There also tends to be an excess of objects created (ie > 200-700mb/s) per compaction thread. > The EncoderStats walks through all the partitions and with mergeWith it will > create a new one per partition as it walks the potentially millions of > partitions. In a test scenario of about 600byte partitions and a couple 100mb > of data this consumed ~16% of the heap pressure. Changing this to instead > mutably track the min values and create one in a EncodingStats.Collector > brought this down considerably (but not 100% since the > UnfilteredRowIterator.stats() still creates 1 per partition). > The KeyCacheKey makes a full copy of the underlying byte array in > ByteBufferUtil.getArray in its constructor. This is the dominating heap > pressure as there are more sstables. By changing this to just keeping the > original it completely eliminates the current dominator of the compactions > and also improves read performance. > Minor tweak included for this as well for operators when compactions are > behind on low read clusters is to make the preemptive opening setting a > hotprop. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14717) Write tests for RepairJob
[
https://issues.apache.org/jira/browse/CASSANDRA-14717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-14717:
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Write tests for RepairJob
> --
>
> Key: CASSANDRA-14717
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14717
> Project: Cassandra
> Issue Type: Test
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Major
>
> As a part of [CASSANDRA-14693], it was suggested to write additional unit
> tests for RepairJob.
> Specifically, the unit-test candidates are {{standardSyncing}} and
> {{optimisedSynching}}. The rest of the class is slightly less important to
> test as it's mostly gathering ranges and collecting trees.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14717) Write tests for RepairJob
[
https://issues.apache.org/jira/browse/CASSANDRA-14717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644128#comment-16644128
]
Alex Petrov commented on CASSANDRA-14717:
-
Thank you for the review! Committed to trunk with
[a05785d82c621c9cd04d8a064c38fd2012ef981c|https://github.com/apache/cassandra/commit/a05785d82c621c9cd04d8a064c38fd2012ef981c]
> Write tests for RepairJob
> --
>
> Key: CASSANDRA-14717
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14717
> Project: Cassandra
> Issue Type: Test
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Major
>
> As a part of [CASSANDRA-14693], it was suggested to write additional unit
> tests for RepairJob.
> Specifically, the unit-test candidates are {{standardSyncing}} and
> {{optimisedSynching}}. The rest of the class is slightly less important to
> test as it's mostly gathering ranges and collecting trees.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
cassandra git commit: Add tests for RepairJob
Repository: cassandra
Updated Branches:
refs/heads/trunk 73ebd200c -> a05785d82
Add tests for RepairJob
Patch by Alex Petrov; reviewed by Marcus Eriksson for CASSANDRA-14717
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a05785d8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a05785d8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a05785d8
Branch: refs/heads/trunk
Commit: a05785d82c621c9cd04d8a064c38fd2012ef981c
Parents: 73ebd20
Author: Alex Petrov
Authored: Mon Oct 1 15:30:58 2018 +0200
Committer: Alex Petrov
Committed: Tue Oct 9 22:33:31 2018 +0100
--
.../repair/AsymmetricRemoteSyncTask.java| 19 +-
.../apache/cassandra/repair/LocalSyncTask.java | 39 +-
.../org/apache/cassandra/repair/RepairJob.java | 146 +++--
.../apache/cassandra/repair/RepairSession.java | 8 +-
.../repair/SymmetricRemoteSyncTask.java | 20 +-
.../org/apache/cassandra/repair/SyncTask.java | 4 +
.../cassandra/repair/LocalSyncTaskTest.java | 26 +-
.../apache/cassandra/repair/RepairJobTest.java | 625 +++
8 files changed, 793 insertions(+), 94 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a05785d8/src/java/org/apache/cassandra/repair/AsymmetricRemoteSyncTask.java
--
diff --git a/src/java/org/apache/cassandra/repair/AsymmetricRemoteSyncTask.java
b/src/java/org/apache/cassandra/repair/AsymmetricRemoteSyncTask.java
index 9ba33dd..d2a6aeb 100644
--- a/src/java/org/apache/cassandra/repair/AsymmetricRemoteSyncTask.java
+++ b/src/java/org/apache/cassandra/repair/AsymmetricRemoteSyncTask.java
@@ -41,15 +41,9 @@ import org.apache.cassandra.utils.MerkleTrees;
*/
public class AsymmetricRemoteSyncTask extends SyncTask implements
CompletableRemoteSyncTask
{
-public AsymmetricRemoteSyncTask(RepairJobDesc desc, InetAddressAndPort
fetchNode, InetAddressAndPort fetchFrom,
-List> rangesToFetch,
PreviewKind previewKind)
+public AsymmetricRemoteSyncTask(RepairJobDesc desc, InetAddressAndPort to,
InetAddressAndPort from, List> differences, PreviewKind
previewKind)
{
-super(desc, fetchNode, fetchFrom, rangesToFetch, previewKind);
-}
-
-public AsymmetricRemoteSyncTask(RepairJobDesc desc, TreeResponse to,
TreeResponse from, PreviewKind previewKind)
-{
-this(desc, to.endpoint, from.endpoint,
MerkleTrees.difference(to.trees, from.trees), previewKind);
+super(desc, to, from, differences, previewKind);
}
public void startSync()
@@ -72,4 +66,13 @@ public class AsymmetricRemoteSyncTask extends SyncTask
implements CompletableRem
setException(new RepairException(desc, previewKind,
String.format("Sync failed between %s and %s", nodePair.coordinator,
nodePair.peer)));
}
}
+
+@Override
+public String toString()
+{
+return "AsymmetricRemoteSyncTask{" +
+ "rangesToSync=" + rangesToSync +
+ ", nodePair=" + nodePair +
+ '}';
+}
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a05785d8/src/java/org/apache/cassandra/repair/LocalSyncTask.java
--
diff --git a/src/java/org/apache/cassandra/repair/LocalSyncTask.java
b/src/java/org/apache/cassandra/repair/LocalSyncTask.java
index 1923fbe..f9b5765 100644
--- a/src/java/org/apache/cassandra/repair/LocalSyncTask.java
+++ b/src/java/org/apache/cassandra/repair/LocalSyncTask.java
@@ -52,15 +52,11 @@ public class LocalSyncTask extends SyncTask implements
StreamEventHandler
private static final Logger logger =
LoggerFactory.getLogger(LocalSyncTask.class);
private final UUID pendingRepair;
-private final boolean requestRanges;
-private final boolean transferRanges;
-public LocalSyncTask(RepairJobDesc desc, TreeResponse local, TreeResponse
remote, UUID pendingRepair,
- boolean requestRanges, boolean transferRanges,
PreviewKind previewKind)
-{
-this(desc, local.endpoint, remote.endpoint,
MerkleTrees.difference(local.trees, remote.trees),
- pendingRepair, requestRanges, transferRanges, previewKind);
-}
+@VisibleForTesting
+final boolean requestRanges;
+@VisibleForTesting
+final boolean transferRanges;
public LocalSyncTask(RepairJobDesc desc, InetAddressAndPort local,
InetAddressAndPort remote,
List> diff, UUID pendingRepair,
@@ -76,8 +72,10 @@ public class LocalSyncTask extends SyncTask implements
StreamEventHandler
}
@VisibleForTesting
-StreamPlan createStreamPlan(InetAddressAndPort remote, List>
differences)
+StreamPla
[jira] [Comment Edited] (CASSANDRA-14561) Make token-generator Python3 compatible
[
https://issues.apache.org/jira/browse/CASSANDRA-14561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644100#comment-16644100
]
Thomas Pouget-Abadie edited comment on CASSANDRA-14561 at 10/9/18 9:14 PM:
---
Hello [~spo...@gmail.com],
I was looking at tools/bin and I can't seem to find the python script you're
referring too on any of the branches (trunk, 3.**,* 2.**, 1.**). Is this ticket
still valid ?
was (Author: pougetat):
Hello [~spo...@gmail.com],
I was looking at tools/bin and I can't seem to find the python script you're
referring too on any of the branches (trunk, 3.**,* 2.*, 1.*). Is this ticket
still valid ?
> Make token-generator Python3 compatible
> ---
>
> Key: CASSANDRA-14561
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14561
> Project: Cassandra
> Issue Type: Improvement
> Components: Testing, Tools
>Reporter: Stefan Podkowinski
>Priority: Major
> Labels: lhf
>
> Among all scripts in {{tools/bin/}}, {{token-generator}} is bit of an oddball
> there. While all other scripts are simple shell script wrappers around Java
> tools, the generator is a standalone Python script. Although it seems to be
> still working just fine, it's not Python3 compatible yet, which causes
> permanent test failures for {{token_generator_test.py}}, after we migrated to
> Python3.
> It would be great if we could make the script work with both Python2+3.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14561) Make token-generator Python3 compatible
[
https://issues.apache.org/jira/browse/CASSANDRA-14561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644100#comment-16644100
]
Thomas Pouget-Abadie edited comment on CASSANDRA-14561 at 10/9/18 9:14 PM:
---
Hello [~spo...@gmail.com],
I was looking at tools/bin and I can't seem to find the python script you're
referring too on any of the branches (trunk, 3.**,* 2.**,* 1.*). Is this ticket
still valid ?
was (Author: pougetat):
Hello [~spo...@gmail.com],
I was looking at tools/bin and I can't seem to find the python script you're
referring too on any of the branches (trunk, 3.**,* 2.**, 1.**). Is this ticket
still valid ?
> Make token-generator Python3 compatible
> ---
>
> Key: CASSANDRA-14561
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14561
> Project: Cassandra
> Issue Type: Improvement
> Components: Testing, Tools
>Reporter: Stefan Podkowinski
>Priority: Major
> Labels: lhf
>
> Among all scripts in {{tools/bin/}}, {{token-generator}} is bit of an oddball
> there. While all other scripts are simple shell script wrappers around Java
> tools, the generator is a standalone Python script. Although it seems to be
> still working just fine, it's not Python3 compatible yet, which causes
> permanent test failures for {{token_generator_test.py}}, after we migrated to
> Python3.
> It would be great if we could make the script work with both Python2+3.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14561) Make token-generator Python3 compatible
[
https://issues.apache.org/jira/browse/CASSANDRA-14561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644100#comment-16644100
]
Thomas Pouget-Abadie edited comment on CASSANDRA-14561 at 10/9/18 9:14 PM:
---
Hello [~spo...@gmail.com],
I was looking at tools/bin and I can't seem to find the python script you're
referring too on any of the branches (trunk, 3.**,* 2.*, 1.*). Is this ticket
still valid ?
was (Author: pougetat):
Hello [~spo...@gmail.com],
I was looking at tools/bin and I can't seem to find the python script you're
referring too on any of the branches (trunk, 3.*, 2.*, 1.*). Is this ticket
still valid ?
> Make token-generator Python3 compatible
> ---
>
> Key: CASSANDRA-14561
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14561
> Project: Cassandra
> Issue Type: Improvement
> Components: Testing, Tools
>Reporter: Stefan Podkowinski
>Priority: Major
> Labels: lhf
>
> Among all scripts in {{tools/bin/}}, {{token-generator}} is bit of an oddball
> there. While all other scripts are simple shell script wrappers around Java
> tools, the generator is a standalone Python script. Although it seems to be
> still working just fine, it's not Python3 compatible yet, which causes
> permanent test failures for {{token_generator_test.py}}, after we migrated to
> Python3.
> It would be great if we could make the script work with both Python2+3.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14561) Make token-generator Python3 compatible
[
https://issues.apache.org/jira/browse/CASSANDRA-14561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644100#comment-16644100
]
Thomas Pouget-Abadie commented on CASSANDRA-14561:
--
Hello [~spo...@gmail.com],
I was looking at tools/bin and I can't seem to find the python script you're
referring too on any of the branches (trunk, 3.*, 2.*, 1.*). Is this ticket
still valid ?
> Make token-generator Python3 compatible
> ---
>
> Key: CASSANDRA-14561
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14561
> Project: Cassandra
> Issue Type: Improvement
> Components: Testing, Tools
>Reporter: Stefan Podkowinski
>Priority: Major
> Labels: lhf
>
> Among all scripts in {{tools/bin/}}, {{token-generator}} is bit of an oddball
> there. While all other scripts are simple shell script wrappers around Java
> tools, the generator is a standalone Python script. Although it seems to be
> still working just fine, it's not Python3 compatible yet, which causes
> permanent test failures for {{token_generator_test.py}}, after we migrated to
> Python3.
> It would be great if we could make the script work with both Python2+3.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14459) DynamicEndpointSnitch should never prefer latent nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643832#comment-16643832
]
Joseph Lynch commented on CASSANDRA-14459:
--
Ok, I'll get the patch rebased and a PR opened for review today. Please reach
out to me if anything is unclear :-)
> DynamicEndpointSnitch should never prefer latent nodes
> --
>
> Key: CASSANDRA-14459
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14459
> Project: Cassandra
> Issue Type: Improvement
> Components: Coordination
>Reporter: Joseph Lynch
>Assignee: Joseph Lynch
>Priority: Minor
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
>
> The DynamicEndpointSnitch has two unfortunate behaviors that allow it to
> provide latent hosts as replicas:
> # Loses all latency information when Cassandra restarts
> # Clears latency information entirely every ten minutes (by default),
> allowing global queries to be routed to _other datacenters_ (and local
> queries cross racks/azs)
> This means that the first few queries after restart/reset could be quite slow
> compared to average latencies. I propose we solve this by resetting to the
> minimum observed latency instead of completely clearing the samples and
> extending the {{isLatencyForSnitch}} idea to a three state variable instead
> of two, in particular {{YES}}, {{NO}}, {{MAYBE}}. This extension allows
> {{EchoMessages}} and {{PingMessages}} to send {{MAYBE}} indicating that the
> DS should use those measurements if it only has one or fewer samples for a
> host. This fixes both problems because on process restart we send out
> {{PingMessages}} / {{EchoMessages}} as part of startup, and we would reset to
> effectively the RTT of the hosts (also at that point normal gossip
> {{EchoMessages}} have an opportunity to add an additional latency
> measurement).
> This strategy also nicely deals with the "a host got slow but now it's fine"
> problem that the DS resets were (afaik) designed to stop because the
> {{EchoMessage}} ping latency will count only after the reset for that host.
> Ping latency is a more reasonable lower bound on host latency (as opposed to
> status quo of zero).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14459) DynamicEndpointSnitch should never prefer latent nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643790#comment-16643790
]
Ariel Weisberg commented on CASSANDRA-14459:
A PR would be great. I'm still digesting context.
> DynamicEndpointSnitch should never prefer latent nodes
> --
>
> Key: CASSANDRA-14459
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14459
> Project: Cassandra
> Issue Type: Improvement
> Components: Coordination
>Reporter: Joseph Lynch
>Assignee: Joseph Lynch
>Priority: Minor
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
>
> The DynamicEndpointSnitch has two unfortunate behaviors that allow it to
> provide latent hosts as replicas:
> # Loses all latency information when Cassandra restarts
> # Clears latency information entirely every ten minutes (by default),
> allowing global queries to be routed to _other datacenters_ (and local
> queries cross racks/azs)
> This means that the first few queries after restart/reset could be quite slow
> compared to average latencies. I propose we solve this by resetting to the
> minimum observed latency instead of completely clearing the samples and
> extending the {{isLatencyForSnitch}} idea to a three state variable instead
> of two, in particular {{YES}}, {{NO}}, {{MAYBE}}. This extension allows
> {{EchoMessages}} and {{PingMessages}} to send {{MAYBE}} indicating that the
> DS should use those measurements if it only has one or fewer samples for a
> host. This fixes both problems because on process restart we send out
> {{PingMessages}} / {{EchoMessages}} as part of startup, and we would reset to
> effectively the RTT of the hosts (also at that point normal gossip
> {{EchoMessages}} have an opportunity to add an additional latency
> measurement).
> This strategy also nicely deals with the "a host got slow but now it's fine"
> problem that the DS resets were (afaik) designed to stop because the
> {{EchoMessage}} ping latency will count only after the reset for that host.
> Ping latency is a more reasonable lower bound on host latency (as opposed to
> status quo of zero).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14459) DynamicEndpointSnitch should never prefer latent nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643776#comment-16643776
]
Joseph Lynch commented on CASSANDRA-14459:
--
[~aweisberg] if you have cycles to take a look at this let me know and I can do
a quick rebase to fix the merge conflicts and if you want me to open a PR for
reviewing purposes I can do that too. Let me know what is easiest for you.
> DynamicEndpointSnitch should never prefer latent nodes
> --
>
> Key: CASSANDRA-14459
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14459
> Project: Cassandra
> Issue Type: Improvement
> Components: Coordination
>Reporter: Joseph Lynch
>Assignee: Joseph Lynch
>Priority: Minor
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
>
> The DynamicEndpointSnitch has two unfortunate behaviors that allow it to
> provide latent hosts as replicas:
> # Loses all latency information when Cassandra restarts
> # Clears latency information entirely every ten minutes (by default),
> allowing global queries to be routed to _other datacenters_ (and local
> queries cross racks/azs)
> This means that the first few queries after restart/reset could be quite slow
> compared to average latencies. I propose we solve this by resetting to the
> minimum observed latency instead of completely clearing the samples and
> extending the {{isLatencyForSnitch}} idea to a three state variable instead
> of two, in particular {{YES}}, {{NO}}, {{MAYBE}}. This extension allows
> {{EchoMessages}} and {{PingMessages}} to send {{MAYBE}} indicating that the
> DS should use those measurements if it only has one or fewer samples for a
> host. This fixes both problems because on process restart we send out
> {{PingMessages}} / {{EchoMessages}} as part of startup, and we would reset to
> effectively the RTT of the hosts (also at that point normal gossip
> {{EchoMessages}} have an opportunity to add an additional latency
> measurement).
> This strategy also nicely deals with the "a host got slow but now it's fine"
> problem that the DS resets were (afaik) designed to stop because the
> {{EchoMessage}} ping latency will count only after the reset for that host.
> Ping latency is a more reasonable lower bound on host latency (as opposed to
> status quo of zero).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14459) DynamicEndpointSnitch should never prefer latent nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-14459:
---
Reviewers: Ariel Weisberg
Reviewer: Ariel Weisberg
> DynamicEndpointSnitch should never prefer latent nodes
> --
>
> Key: CASSANDRA-14459
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14459
> Project: Cassandra
> Issue Type: Improvement
> Components: Coordination
>Reporter: Joseph Lynch
>Assignee: Joseph Lynch
>Priority: Minor
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
>
> The DynamicEndpointSnitch has two unfortunate behaviors that allow it to
> provide latent hosts as replicas:
> # Loses all latency information when Cassandra restarts
> # Clears latency information entirely every ten minutes (by default),
> allowing global queries to be routed to _other datacenters_ (and local
> queries cross racks/azs)
> This means that the first few queries after restart/reset could be quite slow
> compared to average latencies. I propose we solve this by resetting to the
> minimum observed latency instead of completely clearing the samples and
> extending the {{isLatencyForSnitch}} idea to a three state variable instead
> of two, in particular {{YES}}, {{NO}}, {{MAYBE}}. This extension allows
> {{EchoMessages}} and {{PingMessages}} to send {{MAYBE}} indicating that the
> DS should use those measurements if it only has one or fewer samples for a
> host. This fixes both problems because on process restart we send out
> {{PingMessages}} / {{EchoMessages}} as part of startup, and we would reset to
> effectively the RTT of the hosts (also at that point normal gossip
> {{EchoMessages}} have an opportunity to add an additional latency
> measurement).
> This strategy also nicely deals with the "a host got slow but now it's fine"
> problem that the DS resets were (afaik) designed to stop because the
> {{EchoMessage}} ping latency will count only after the reset for that host.
> Ping latency is a more reasonable lower bound on host latency (as opposed to
> status quo of zero).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14747) Evaluate 200 node, compression=none, encryption=none, coalescing=off
[ https://issues.apache.org/jira/browse/CASSANDRA-14747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643429#comment-16643429 ] C. Scott Andreas commented on CASSANDRA-14747: -- Echoing that - great find and a real nice looking before-and-after; thanks Joey! > Evaluate 200 node, compression=none, encryption=none, coalescing=off > - > > Key: CASSANDRA-14747 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14747 > Project: Cassandra > Issue Type: Sub-task >Reporter: Joseph Lynch >Assignee: Joseph Lynch >Priority: Major > Attachments: 3.0.17-QPS.png, 4.0.1-QPS.png, > 4.0.11-after-jolynch-tweaks.svg, 4.0.12-after-unconditional-flush.svg, > 4.0.15-after-sndbuf-fix.svg, 4.0.7-before-my-changes.svg, > 4.0_errors_showing_heap_pressure.txt, > 4.0_heap_histogram_showing_many_MessageOuts.txt, > i-0ed2acd2dfacab7c1-after-looping-fixes.svg, > trunk_vs_3.0.17_latency_under_load.png, > ttop_NettyOutbound-Thread_spinning.txt, > useast1c-i-0e1ddfe8b2f769060-mutation-flame.svg, > useast1e-i-08635fa1631601538_flamegraph_96node.svg, > useast1e-i-08635fa1631601538_ttop_netty_outbound_threads_96nodes, > useast1e-i-08635fa1631601538_uninlinedcpuflamegraph.0_96node_60sec_profile.svg > > > Tracks evaluating a 200 node cluster with all internode settings off (no > compression, no encryption, no coalescing). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14373) Allow using custom script for chronicle queue BinLog archival
[
https://issues.apache.org/jira/browse/CASSANDRA-14373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643394#comment-16643394
]
Marcus Eriksson edited comment on CASSANDRA-14373 at 10/9/18 1:05 PM:
--
bq. are they sequentially numbered?
yeah, the file names should have date/time:
https://github.com/OpenHFT/Chronicle-Queue#topics-and-queue-files
pushed a change to move back to POSIX {{sh}} instead of {{bash}}, [~beobal]
could you have a quick lock at that?
https://github.com/krummas/cassandra/commits/marcuse/14373
was (Author: krummas):
bq. are they sequentially numbered?
yeah, the file names should have date/time:
https://github.com/OpenHFT/Chronicle-Queue#topics-and-queue-files
pushed a change to move back to POSIX {{sh}} instead of {{bash}}, [~beobal]
could you have a quick lock at that?
> Allow using custom script for chronicle queue BinLog archival
> -
>
> Key: CASSANDRA-14373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14373
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Stefan Podkowinski
>Assignee: Pramod K Sivaraju
>Priority: Major
> Labels: lhf, pull-request-available
> Fix For: 4.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It would be nice to allow the user to configure an archival script that will
> be executed in {{BinLog.onReleased(cycle, file)}} for every deleted bin log,
> just as we do in {{CommitLogArchiver}}. The script should be able to copy the
> released file to an external location or do whatever the author hand in mind.
> Deleting the log file should be delegated to the script as well.
> See CASSANDRA-13983, CASSANDRA-12151 for use cases.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14373) Allow using custom script for chronicle queue BinLog archival
[
https://issues.apache.org/jira/browse/CASSANDRA-14373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643394#comment-16643394
]
Marcus Eriksson commented on CASSANDRA-14373:
-
bq. are they sequentially numbered?
yeah, the file names should have date/time:
https://github.com/OpenHFT/Chronicle-Queue#topics-and-queue-files
pushed a change to move back to POSIX {{sh}} instead of {{bash}}, [~beobal]
could you have a quick lock at that?
> Allow using custom script for chronicle queue BinLog archival
> -
>
> Key: CASSANDRA-14373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14373
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Stefan Podkowinski
>Assignee: Pramod K Sivaraju
>Priority: Major
> Labels: lhf, pull-request-available
> Fix For: 4.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It would be nice to allow the user to configure an archival script that will
> be executed in {{BinLog.onReleased(cycle, file)}} for every deleted bin log,
> just as we do in {{CommitLogArchiver}}. The script should be able to copy the
> released file to an external location or do whatever the author hand in mind.
> Deleting the log file should be delegated to the script as well.
> See CASSANDRA-13983, CASSANDRA-12151 for use cases.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-14805) Fails on running Cassandra server
[ https://issues.apache.org/jira/browse/CASSANDRA-14805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie resolved CASSANDRA-14805. - Resolution: Invalid > Fails on running Cassandra server > -- > > Key: CASSANDRA-14805 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14805 > Project: Cassandra > Issue Type: Bug >Reporter: Ravi Gangwar >Priority: Critical > > Full Product Version : > Fails on java version. It's Java "1.8.0_181" > Os Version : Ubuntu 16.04 LTS > EXTRA RELEVANT SYSTEM CONFIGURATION : > Just installed Cassandra 2.2.11 > A DESCRIPTION OF THE PROBLEM : > # > # A fatal error has been detected by the Java Runtime Environment: > # > # SIGBUS (0x7) at pc=0x7f42fc492e70, pid=12128, tid=0x7f42fc3c9700 > # > # JRE version: Java(TM) SE Runtime Environment (8.0_181-b13) (build > 1.8.0_181-b13) > # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.181-b13 mixed mode > linux-amd64 compressed oops) > # Problematic frame: > # C [liblz4-java8098863625230398555.so+0x5e70] LZ4_decompress_fast+0xd0 > # > # Failed to write core dump. Core dumps have been disabled. To enable core > dumping, try "ulimit -c unlimited" before starting Java again > # > # If you would like to submit a bug report, please visit: > # http://bugreport.java.com/bugreport/crash.jsp > # The crash happened outside the Java Virtual Machine in native code. > # See problematic frame for where to report the bug. > # > --- T H R E A D --- > Current thread (0x006d0800): JavaThread "CompactionExecutor:7" daemon > [_thread_in_native, id=5278, stack(0x7f42fc389000,0x7f42fc3ca000)] > siginfo: si_signo: 7 (SIGBUS), si_code: 2 (BUS_ADRERR), si_addr: > 0x7f2026b8b000 > Registers: > RAX=0x7f432200750a, RBX=0x7f2026b8affc, RCX=0x7f4322005856, > RDX=0x7f432200750a > RSP=0x7f42fc3c7e50, RBP=0x7f2026b8940f, RSI=0x7f4322011d88, > RDI=0x0007 > R8 =0x7f4322011d84, R9 =0x7f4322011d90, R10=0x0003, > R11=0x > R12=0x, R13=0x7f1ffe33d40f, R14=0x7f4322011d87, > R15=0x7f4322011d8b > RIP=0x7f42fc492e70, EFLAGS=0x00010287, CSGSFS=0x002b0033, > ERR=0x0004 > TRAPNO=0x000e > Top of Stack: (sp=0x7f42fc3c7e50) > 0x7f42fc3c7e50: 4ce9 > 0x7f42fc3c7e60: 0004 0001 > 0x7f42fc3c7e70: 0002 0001 > 0x7f42fc3c7e80: 0004 0004 > 0x7f42fc3c7e90: 0004 0004 > 0x7f42fc3c7ea0: > 0x7f42fc3c7eb0: > 0x7f42fc3c7ec0: 0001 > 0x7f42fc3c7ed0: 0002 0003 > 0x7f42fc3c7ee0: 7f42fc3c7fa8 006d09f8 > 0x7f42fc3c7ef0: > 0x7f42fc3c7f00: 7f1ffe33d40f 7f4322001d90 > 0x7f42fc3c7f10: 2884c000 7f42fc48f59d > 0x7f42fc3c7f20: 000733b5fa78 > 0x7f42fc3c7f30: 7f42fc3c7fc0 > 0x7f42fc3c7f40: 2884bfff 7f42fc3c7fa8 > 0x7f42fc3c7f50: 006d0800 7f43119e8f1d > 0x7f42fc3c7f60: 7f42fc3c7f98 > 0x7f42fc3c7f70: 0001 > 0x7f42fc3c7f80: 00070764f4d8 > 0x7f42fc3c7f90: 00075b5937f0 > 0x7f42fc3c7fa0: 7f42fc3c7ff0 0006f91d2e38 > 0x7f42fc3c7fb0: 0006d7e3f210 7f42fc3c8040 > 0x7f42fc3c7fc0: 00070764f4d8 7f4311f9eca8 > 0x7f42fc3c7fd0: 51070001 00072dc9abb0 > 0x7f42fc3c7fe0: 28851103 0006dd2d > 0x7f42fc3c7ff0: 00022883f40b eb6b26fe2884bffc > 0x7f42fc3c8000: 00070764f4d8 00012884c000 > 0x7f42fc3c8010: 00075b5937f0 00075b5937f0 > 0x7f42fc3c8020: 00060008 0006d7f67598 > 0x7f42fc3c8030: 0006d7f67348 7f43120080d4 > 0x7f42fc3c8040: 0001 0006d7f676d8 > > > SYSTEM CONFIGURATION : > > # CPU - Intel Core i7 - 4771 CPU @ 3.50 GHz * 8 > 2. RAM - 16 GB > 3. STORAGE - 967.6 GB > 4. OS - Ubuntu 16.04 LTS > 5. Apache Cassandra - 2.2.11 > 6. CPP Driver - 2.2.1-1 > 7. libuv - 1.4.2-1 > 8. Java version - "1.8.0_181" > Java (TM) SE Runtime Environment (build 1.8.0_181-b13) > Java Hotspot (TM) 64-Bit Server VM (build 25.181-b13, mixed mode) > > Cassandra was working normally before generating this bug. Now when i am > trying to restart my server getting this bug. > Please help me asap. > > Regards, > Ravi > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsub
[jira] [Commented] (CASSANDRA-14805) Fails on running Cassandra server
[
https://issues.apache.org/jira/browse/CASSANDRA-14805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643269#comment-16643269
]
Joshua McKenzie commented on CASSANDRA-14805:
-
{quote}This will not able to help me. Anyother community on Jira or you help me
in this.
Its quite urgent. Please do the needful
{quote}
The community emailing list or IRC are precisely the places you need to go to
if you want to get questions about operationally running the database as
quickly as possible by experts.
> Fails on running Cassandra server
> --
>
> Key: CASSANDRA-14805
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14805
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ravi Gangwar
>Priority: Critical
>
> Full Product Version :
> Fails on java version. It's Java "1.8.0_181"
> Os Version : Ubuntu 16.04 LTS
> EXTRA RELEVANT SYSTEM CONFIGURATION :
> Just installed Cassandra 2.2.11
> A DESCRIPTION OF THE PROBLEM :
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGBUS (0x7) at pc=0x7f42fc492e70, pid=12128, tid=0x7f42fc3c9700
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_181-b13) (build
> 1.8.0_181-b13)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.181-b13 mixed mode
> linux-amd64 compressed oops)
> # Problematic frame:
> # C [liblz4-java8098863625230398555.so+0x5e70] LZ4_decompress_fast+0xd0
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core
> dumping, try "ulimit -c unlimited" before starting Java again
> #
> # If you would like to submit a bug report, please visit:
> # http://bugreport.java.com/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #
> --- T H R E A D ---
> Current thread (0x006d0800): JavaThread "CompactionExecutor:7" daemon
> [_thread_in_native, id=5278, stack(0x7f42fc389000,0x7f42fc3ca000)]
> siginfo: si_signo: 7 (SIGBUS), si_code: 2 (BUS_ADRERR), si_addr:
> 0x7f2026b8b000
> Registers:
> RAX=0x7f432200750a, RBX=0x7f2026b8affc, RCX=0x7f4322005856,
> RDX=0x7f432200750a
> RSP=0x7f42fc3c7e50, RBP=0x7f2026b8940f, RSI=0x7f4322011d88,
> RDI=0x0007
> R8 =0x7f4322011d84, R9 =0x7f4322011d90, R10=0x0003,
> R11=0x
> R12=0x, R13=0x7f1ffe33d40f, R14=0x7f4322011d87,
> R15=0x7f4322011d8b
> RIP=0x7f42fc492e70, EFLAGS=0x00010287, CSGSFS=0x002b0033,
> ERR=0x0004
> TRAPNO=0x000e
> Top of Stack: (sp=0x7f42fc3c7e50)
> 0x7f42fc3c7e50: 4ce9
> 0x7f42fc3c7e60: 0004 0001
> 0x7f42fc3c7e70: 0002 0001
> 0x7f42fc3c7e80: 0004 0004
> 0x7f42fc3c7e90: 0004 0004
> 0x7f42fc3c7ea0:
> 0x7f42fc3c7eb0:
> 0x7f42fc3c7ec0: 0001
> 0x7f42fc3c7ed0: 0002 0003
> 0x7f42fc3c7ee0: 7f42fc3c7fa8 006d09f8
> 0x7f42fc3c7ef0:
> 0x7f42fc3c7f00: 7f1ffe33d40f 7f4322001d90
> 0x7f42fc3c7f10: 2884c000 7f42fc48f59d
> 0x7f42fc3c7f20: 000733b5fa78
> 0x7f42fc3c7f30: 7f42fc3c7fc0
> 0x7f42fc3c7f40: 2884bfff 7f42fc3c7fa8
> 0x7f42fc3c7f50: 006d0800 7f43119e8f1d
> 0x7f42fc3c7f60: 7f42fc3c7f98
> 0x7f42fc3c7f70: 0001
> 0x7f42fc3c7f80: 00070764f4d8
> 0x7f42fc3c7f90: 00075b5937f0
> 0x7f42fc3c7fa0: 7f42fc3c7ff0 0006f91d2e38
> 0x7f42fc3c7fb0: 0006d7e3f210 7f42fc3c8040
> 0x7f42fc3c7fc0: 00070764f4d8 7f4311f9eca8
> 0x7f42fc3c7fd0: 51070001 00072dc9abb0
> 0x7f42fc3c7fe0: 28851103 0006dd2d
> 0x7f42fc3c7ff0: 00022883f40b eb6b26fe2884bffc
> 0x7f42fc3c8000: 00070764f4d8 00012884c000
> 0x7f42fc3c8010: 00075b5937f0 00075b5937f0
> 0x7f42fc3c8020: 00060008 0006d7f67598
> 0x7f42fc3c8030: 0006d7f67348 7f43120080d4
> 0x7f42fc3c8040: 0001 0006d7f676d8
>
>
> SYSTEM CONFIGURATION :
>
> # CPU - Intel Core i7 - 4771 CPU @ 3.50 GHz * 8
> 2. RAM - 16 GB
> 3. STORAGE - 967.6 GB
> 4. OS - Ubuntu 16.04 LTS
> 5. Apache Cassandra - 2.2.11
> 6. CPP Driver - 2.2.1-1
> 7. libuv - 1.4.2-1
> 8. Java version - "1.8.0_181"
> Java (TM) SE Runtime Environment (build 1.8.0_181-b13)
> Java Hotspot (TM) 64-Bit Se
cassandra-builds git commit: Add docker image for running dtests
Repository: cassandra-builds Updated Branches: refs/heads/master 4f49929aa -> 82dbe9d4f Add docker image for running dtests Closes #7 Project: http://git-wip-us.apache.org/repos/asf/cassandra-builds/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra-builds/commit/82dbe9d4 Tree: http://git-wip-us.apache.org/repos/asf/cassandra-builds/tree/82dbe9d4 Diff: http://git-wip-us.apache.org/repos/asf/cassandra-builds/diff/82dbe9d4 Branch: refs/heads/master Commit: 82dbe9d4f22f3ba649d0eaee8f048f667d8bfa3d Parents: 4f49929 Author: Stefan Podkowinski Authored: Tue Oct 9 13:17:46 2018 +0200 Committer: Stefan Podkowinski Committed: Tue Oct 9 13:17:46 2018 +0200 -- docker/testing/ubuntu18_j11.docker | 107 1 file changed, 107 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/82dbe9d4/docker/testing/ubuntu18_j11.docker -- diff --git a/docker/testing/ubuntu18_j11.docker b/docker/testing/ubuntu18_j11.docker new file mode 100644 index 000..30ecdf1 --- /dev/null +++ b/docker/testing/ubuntu18_j11.docker @@ -0,0 +1,107 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# base things off the latest LTS Ubuntu Release (18.04) +FROM ubuntu:18.04 +MAINTAINER Stefan Podkowinski + +# install our python dependencies and some other stuff we need +# libev4 libev-dev are for the python driver + +RUN export DEBIAN_FRONTEND=noninteractive && \ +apt-get update && \ +apt-get install -y --no-install-recommends software-properties-common apt-utils + +RUN export DEBIAN_FRONTEND=noninteractive && \ +apt-get update && \ +apt-get install -y git-core python2.7 python3.6 python3.6-venv python3.6-dev python3-pip net-tools libev4 libev-dev wget gcc + +# solves warning: "jemalloc shared library could not be preloaded to speed up memory allocations" +RUN export DEBIAN_FRONTEND=noninteractive && \ +apt-get update && \ +apt-get install -y --no-install-recommends libjemalloc1 + +# install dumb-init as minimal init system +RUN export DEBIAN_FRONTEND=noninteractive && \ +apt-get update && \ +apt-get install -y dumb-init + +# generate locales for the standard en_US.UTF8 value we use for testing +RUN export DEBIAN_FRONTEND=noninteractive && \ +apt-get update && \ +apt-get install -y locales && \ +locale-gen en_US.UTF-8 + +# as we only need the requirements.txt file from the dtest repo, let's just get it from GitHub as a raw asset +# so we can avoid needing to clone the entire repo just to get this file +ADD https://raw.githubusercontent.com/apache/cassandra-dtest/master/requirements.txt /opt +RUN chmod 0644 /opt/requirements.txt + +# now setup python via virtualenv with all of the python dependencies we need according to requirements.txt +RUN pip3 install virtualenv +RUN pip3 install --upgrade wheel + +# openjdk + ant +RUN export DEBIAN_FRONTEND=noninteractive && \ +apt-get update && \ +apt-get install -y --no-install-recommends openjdk-8-jdk openjdk-11-jdk ant ant-optional + +# make Java 8 the default executable (we use to run all tests against Java 8) +RUN update-java-alternatives -s java-1.8.0-openjdk-amd64 + +# setup our user -- if we don't do this docker will default to root and Cassandra will fail to start +# as we appear to have a check that the user isn't starting Cassandra as root +RUN export DEBIAN_FRONTEND=noninteractive && \ +apt-get install sudo && \ +adduser --disabled-password --gecos "" cassandra && \ +echo "cassandra ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/cassandra && \ +chmod 0440 /etc/sudoers.d/cassandra + +# fix up permissions on the cassandra home dir +RUN chown -R cassandra:cassandra /home/cassandra + +# switch to the cassandra user... we are all done running things as root +USER cassandra +ENV HOME /home/cassandra +WORKDIR /home/cassandra + +# Add environment variables for Ant and Java and add them to the PATH +RUN echo 'export ANT_HOME=/usr/share/ant' >> /home/cassandra/.bashrc && \ +echo 'export JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-amd64' >> /home/cassandra/.bashrc && \ +echo 'export JAVA11_HOME=/usr/lib/jvm/java-11-openjdk-amd64' >> /home/cassandra/.bashrc && \ +echo 'export JAVA_HOME=$JAVA8_HOME' >> /home/cassandra/.bashrc + +# run pip commands and s
[jira] [Updated] (CASSANDRA-14713) Update docker image used for testing
[
https://issues.apache.org/jira/browse/CASSANDRA-14713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-14713:
Status: Ready to Commit (was: Patch Available)
> Update docker image used for testing
>
>
> Key: CASSANDRA-14713
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14713
> Project: Cassandra
> Issue Type: New Feature
> Components: Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
> Attachments: Dockerfile
>
>
> Tests executed on builds.apache.org ({{docker/jenkins/jenkinscommand.sh}})
> and circleCI ({{.circleci/config.yml}}) will currently use the same
> [cassandra-test|https://hub.docker.com/r/kjellman/cassandra-test/] docker
> image ([github|https://github.com/mkjellman/cassandra-test-docker]) by
> [~mkjellman].
> We should manage this image on our own as part of cassandra-builds, to keep
> it updated. There's also a [Apache
> user|https://hub.docker.com/u/apache/?page=1] on docker hub for publishing
> images.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14713) Update docker image used for testing
[
https://issues.apache.org/jira/browse/CASSANDRA-14713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643094#comment-16643094
]
Marcus Eriksson commented on CASSANDRA-14713:
-
lgtm, +1 (tiny comment on the dtest changes, feel free to fix that on commit as
well)
> Update docker image used for testing
>
>
> Key: CASSANDRA-14713
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14713
> Project: Cassandra
> Issue Type: New Feature
> Components: Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
> Attachments: Dockerfile
>
>
> Tests executed on builds.apache.org ({{docker/jenkins/jenkinscommand.sh}})
> and circleCI ({{.circleci/config.yml}}) will currently use the same
> [cassandra-test|https://hub.docker.com/r/kjellman/cassandra-test/] docker
> image ([github|https://github.com/mkjellman/cassandra-test-docker]) by
> [~mkjellman].
> We should manage this image on our own as part of cassandra-builds, to keep
> it updated. There's also a [Apache
> user|https://hub.docker.com/u/apache/?page=1] on docker hub for publishing
> images.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14805) Fails on running Cassandra server
[ https://issues.apache.org/jira/browse/CASSANDRA-14805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642915#comment-16642915 ] Ravi Gangwar commented on CASSANDRA-14805: -- Will it work after updating Cassandra 2.2.13 ? Because Cassandra was working fine for 2 -3 days intially when i started with same configuration but after loading around 300 GB of data, it is throwing error like this and not able to start again the Cassandra server. Please check ones > Fails on running Cassandra server > -- > > Key: CASSANDRA-14805 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14805 > Project: Cassandra > Issue Type: Bug >Reporter: Ravi Gangwar >Priority: Critical > > Full Product Version : > Fails on java version. It's Java "1.8.0_181" > Os Version : Ubuntu 16.04 LTS > EXTRA RELEVANT SYSTEM CONFIGURATION : > Just installed Cassandra 2.2.11 > A DESCRIPTION OF THE PROBLEM : > # > # A fatal error has been detected by the Java Runtime Environment: > # > # SIGBUS (0x7) at pc=0x7f42fc492e70, pid=12128, tid=0x7f42fc3c9700 > # > # JRE version: Java(TM) SE Runtime Environment (8.0_181-b13) (build > 1.8.0_181-b13) > # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.181-b13 mixed mode > linux-amd64 compressed oops) > # Problematic frame: > # C [liblz4-java8098863625230398555.so+0x5e70] LZ4_decompress_fast+0xd0 > # > # Failed to write core dump. Core dumps have been disabled. To enable core > dumping, try "ulimit -c unlimited" before starting Java again > # > # If you would like to submit a bug report, please visit: > # http://bugreport.java.com/bugreport/crash.jsp > # The crash happened outside the Java Virtual Machine in native code. > # See problematic frame for where to report the bug. > # > --- T H R E A D --- > Current thread (0x006d0800): JavaThread "CompactionExecutor:7" daemon > [_thread_in_native, id=5278, stack(0x7f42fc389000,0x7f42fc3ca000)] > siginfo: si_signo: 7 (SIGBUS), si_code: 2 (BUS_ADRERR), si_addr: > 0x7f2026b8b000 > Registers: > RAX=0x7f432200750a, RBX=0x7f2026b8affc, RCX=0x7f4322005856, > RDX=0x7f432200750a > RSP=0x7f42fc3c7e50, RBP=0x7f2026b8940f, RSI=0x7f4322011d88, > RDI=0x0007 > R8 =0x7f4322011d84, R9 =0x7f4322011d90, R10=0x0003, > R11=0x > R12=0x, R13=0x7f1ffe33d40f, R14=0x7f4322011d87, > R15=0x7f4322011d8b > RIP=0x7f42fc492e70, EFLAGS=0x00010287, CSGSFS=0x002b0033, > ERR=0x0004 > TRAPNO=0x000e > Top of Stack: (sp=0x7f42fc3c7e50) > 0x7f42fc3c7e50: 4ce9 > 0x7f42fc3c7e60: 0004 0001 > 0x7f42fc3c7e70: 0002 0001 > 0x7f42fc3c7e80: 0004 0004 > 0x7f42fc3c7e90: 0004 0004 > 0x7f42fc3c7ea0: > 0x7f42fc3c7eb0: > 0x7f42fc3c7ec0: 0001 > 0x7f42fc3c7ed0: 0002 0003 > 0x7f42fc3c7ee0: 7f42fc3c7fa8 006d09f8 > 0x7f42fc3c7ef0: > 0x7f42fc3c7f00: 7f1ffe33d40f 7f4322001d90 > 0x7f42fc3c7f10: 2884c000 7f42fc48f59d > 0x7f42fc3c7f20: 000733b5fa78 > 0x7f42fc3c7f30: 7f42fc3c7fc0 > 0x7f42fc3c7f40: 2884bfff 7f42fc3c7fa8 > 0x7f42fc3c7f50: 006d0800 7f43119e8f1d > 0x7f42fc3c7f60: 7f42fc3c7f98 > 0x7f42fc3c7f70: 0001 > 0x7f42fc3c7f80: 00070764f4d8 > 0x7f42fc3c7f90: 00075b5937f0 > 0x7f42fc3c7fa0: 7f42fc3c7ff0 0006f91d2e38 > 0x7f42fc3c7fb0: 0006d7e3f210 7f42fc3c8040 > 0x7f42fc3c7fc0: 00070764f4d8 7f4311f9eca8 > 0x7f42fc3c7fd0: 51070001 00072dc9abb0 > 0x7f42fc3c7fe0: 28851103 0006dd2d > 0x7f42fc3c7ff0: 00022883f40b eb6b26fe2884bffc > 0x7f42fc3c8000: 00070764f4d8 00012884c000 > 0x7f42fc3c8010: 00075b5937f0 00075b5937f0 > 0x7f42fc3c8020: 00060008 0006d7f67598 > 0x7f42fc3c8030: 0006d7f67348 7f43120080d4 > 0x7f42fc3c8040: 0001 0006d7f676d8 > > > SYSTEM CONFIGURATION : > > # CPU - Intel Core i7 - 4771 CPU @ 3.50 GHz * 8 > 2. RAM - 16 GB > 3. STORAGE - 967.6 GB > 4. OS - Ubuntu 16.04 LTS > 5. Apache Cassandra - 2.2.11 > 6. CPP Driver - 2.2.1-1 > 7. libuv - 1.4.2-1 > 8. Java version - "1.8.0_181" > Java (TM) SE Runtime Environment (build 1.8.0_181-b13) > Java Hotspot (TM) 64-Bit Server VM (build 25.181-b13, mixed mode) > >
[jira] [Commented] (CASSANDRA-14805) Fails on running Cassandra server
[ https://issues.apache.org/jira/browse/CASSANDRA-14805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642912#comment-16642912 ] Stefan Podkowinski commented on CASSANDRA-14805: This should have been fixed in CASSANDRA-14284, please update to Cassandra 2.2.13. > Fails on running Cassandra server > -- > > Key: CASSANDRA-14805 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14805 > Project: Cassandra > Issue Type: Bug >Reporter: Ravi Gangwar >Priority: Critical > > Full Product Version : > Fails on java version. It's Java "1.8.0_181" > Os Version : Ubuntu 16.04 LTS > EXTRA RELEVANT SYSTEM CONFIGURATION : > Just installed Cassandra 2.2.11 > A DESCRIPTION OF THE PROBLEM : > # > # A fatal error has been detected by the Java Runtime Environment: > # > # SIGBUS (0x7) at pc=0x7f42fc492e70, pid=12128, tid=0x7f42fc3c9700 > # > # JRE version: Java(TM) SE Runtime Environment (8.0_181-b13) (build > 1.8.0_181-b13) > # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.181-b13 mixed mode > linux-amd64 compressed oops) > # Problematic frame: > # C [liblz4-java8098863625230398555.so+0x5e70] LZ4_decompress_fast+0xd0 > # > # Failed to write core dump. Core dumps have been disabled. To enable core > dumping, try "ulimit -c unlimited" before starting Java again > # > # If you would like to submit a bug report, please visit: > # http://bugreport.java.com/bugreport/crash.jsp > # The crash happened outside the Java Virtual Machine in native code. > # See problematic frame for where to report the bug. > # > --- T H R E A D --- > Current thread (0x006d0800): JavaThread "CompactionExecutor:7" daemon > [_thread_in_native, id=5278, stack(0x7f42fc389000,0x7f42fc3ca000)] > siginfo: si_signo: 7 (SIGBUS), si_code: 2 (BUS_ADRERR), si_addr: > 0x7f2026b8b000 > Registers: > RAX=0x7f432200750a, RBX=0x7f2026b8affc, RCX=0x7f4322005856, > RDX=0x7f432200750a > RSP=0x7f42fc3c7e50, RBP=0x7f2026b8940f, RSI=0x7f4322011d88, > RDI=0x0007 > R8 =0x7f4322011d84, R9 =0x7f4322011d90, R10=0x0003, > R11=0x > R12=0x, R13=0x7f1ffe33d40f, R14=0x7f4322011d87, > R15=0x7f4322011d8b > RIP=0x7f42fc492e70, EFLAGS=0x00010287, CSGSFS=0x002b0033, > ERR=0x0004 > TRAPNO=0x000e > Top of Stack: (sp=0x7f42fc3c7e50) > 0x7f42fc3c7e50: 4ce9 > 0x7f42fc3c7e60: 0004 0001 > 0x7f42fc3c7e70: 0002 0001 > 0x7f42fc3c7e80: 0004 0004 > 0x7f42fc3c7e90: 0004 0004 > 0x7f42fc3c7ea0: > 0x7f42fc3c7eb0: > 0x7f42fc3c7ec0: 0001 > 0x7f42fc3c7ed0: 0002 0003 > 0x7f42fc3c7ee0: 7f42fc3c7fa8 006d09f8 > 0x7f42fc3c7ef0: > 0x7f42fc3c7f00: 7f1ffe33d40f 7f4322001d90 > 0x7f42fc3c7f10: 2884c000 7f42fc48f59d > 0x7f42fc3c7f20: 000733b5fa78 > 0x7f42fc3c7f30: 7f42fc3c7fc0 > 0x7f42fc3c7f40: 2884bfff 7f42fc3c7fa8 > 0x7f42fc3c7f50: 006d0800 7f43119e8f1d > 0x7f42fc3c7f60: 7f42fc3c7f98 > 0x7f42fc3c7f70: 0001 > 0x7f42fc3c7f80: 00070764f4d8 > 0x7f42fc3c7f90: 00075b5937f0 > 0x7f42fc3c7fa0: 7f42fc3c7ff0 0006f91d2e38 > 0x7f42fc3c7fb0: 0006d7e3f210 7f42fc3c8040 > 0x7f42fc3c7fc0: 00070764f4d8 7f4311f9eca8 > 0x7f42fc3c7fd0: 51070001 00072dc9abb0 > 0x7f42fc3c7fe0: 28851103 0006dd2d > 0x7f42fc3c7ff0: 00022883f40b eb6b26fe2884bffc > 0x7f42fc3c8000: 00070764f4d8 00012884c000 > 0x7f42fc3c8010: 00075b5937f0 00075b5937f0 > 0x7f42fc3c8020: 00060008 0006d7f67598 > 0x7f42fc3c8030: 0006d7f67348 7f43120080d4 > 0x7f42fc3c8040: 0001 0006d7f676d8 > > > SYSTEM CONFIGURATION : > > # CPU - Intel Core i7 - 4771 CPU @ 3.50 GHz * 8 > 2. RAM - 16 GB > 3. STORAGE - 967.6 GB > 4. OS - Ubuntu 16.04 LTS > 5. Apache Cassandra - 2.2.11 > 6. CPP Driver - 2.2.1-1 > 7. libuv - 1.4.2-1 > 8. Java version - "1.8.0_181" > Java (TM) SE Runtime Environment (build 1.8.0_181-b13) > Java Hotspot (TM) 64-Bit Server VM (build 25.181-b13, mixed mode) > > Cassandra was working normally before generating this bug. Now when i am > trying to restart my server getting this bug. > Please help me asap. > > Regards, > Ravi > -- This message
