[jira] [Created] (CASSANDRA-14941) Expired secondary index sstables are not promptly discarded under TWCS
Samuel Klock created CASSANDRA-14941:
Summary: Expired secondary index sstables are not promptly
discarded under TWCS
Key: CASSANDRA-14941
URL: https://issues.apache.org/jira/browse/CASSANDRA-14941
Project: Cassandra
Issue Type: Bug
Components: Secondary Indexes
Reporter: Samuel Klock
We have a table in a cluster running 3.0.17 storing roughly time-series data
using TWCS with a secondary index. We've noticed that while expired sstables
for the table are discarded mostly when we expect them to be, the expired
sstables for the secondary index would linger for weeks longer than expected –
essentially indefinitely. Eventually the sstables would fill disks, which would
require manual steps (deleting ancient index sstables) to address. We verified
with {{sstableexpiredblockers}} that there wasn't anything on disk blocking the
expired sstables from being dropped, so this looks like a bug.
Through some debugging, we traced the problem to the index's memtables, which
were consistently (except _just_ after node restarts) reporting a minimum
timestamp from September 2015 – much older than any of our live data – which
causes {{CompactionController.getFullyExpiredSSTables()}} to consistently
return an empty set. The reason that the index sstables report this minimum
timestamp is because of how index updates are created, using
{{PartitionUpdate.singleRowUpdate()}}:
{code:java}
public static PartitionUpdate singleRowUpdate(CFMetaData metadata,
DecoratedKey key, Row row, Row staticRow)
{
MutableDeletionInfo deletionInfo = MutableDeletionInfo.live();
Holder holder = new Holder(
new PartitionColumns(
staticRow == null ? Columns.NONE :
Columns.from(staticRow.columns()),
row == null ? Columns.NONE : Columns.from(row.columns())
),
row == null ? BTree.empty() : BTree.singleton(row),
deletionInfo,
staticRow == null ? Rows.EMPTY_STATIC_ROW : staticRow,
EncodingStats.NO_STATS
);
return new PartitionUpdate(metadata, key, holder, deletionInfo, false);
}
{code}
The use of {{EncodingStats.NO_STATS}} makes it appear as though the earliest
timestamp in the resulting {{PartitionUpdate}} is from September 2015. That
timestamp becomes the minimum for the memtable.
Modifying this version of {{PartitionUpdate.singleRowUpdate()}} to:
{code:java}
public static PartitionUpdate singleRowUpdate(CFMetaData metadata,
DecoratedKey key, Row row, Row staticRow)
{
MutableDeletionInfo deletionInfo = MutableDeletionInfo.live();
staticRow = (staticRow == null ? Rows.EMPTY_STATIC_ROW : staticRow);
EncodingStats stats = EncodingStats.Collector.collect(staticRow,
(row == null ?
Collections.emptyIterator() :
Iterators.singletonIterator(row)),
deletionInfo);
Holder holder = new Holder(
new PartitionColumns(
staticRow == Rows.EMPTY_STATIC_ROW ? Columns.NONE :
Columns.from(staticRow.columns()),
row == null ? Columns.NONE : Columns.from(row.columns())
),
row == null ? BTree.empty() : BTree.singleton(row),
deletionInfo,
staticRow,
stats
);
return new PartitionUpdate(metadata, key, holder, deletionInfo, false);
}
{code}
(i.e., computing an {{EncodingStats}} from the contents of the update) seems to
fix the problem. However, we're not certain whether A) there's a functional
reason the method was using {{EncodingStats.NO_STATS}} previously or B) whether
the {{EncodingStats}} the revised version creates is correct (in particular,
the use of {{deletionInfo}} feels a little suspect). We're also not sure
whether there's a more appropriate fix (e.g., changing how the memtables
compute the minimum timestamp, particularly in the {{NO_STATS}} case).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14155) [TRUNK] Gossiper somewhat frequently hitting an NPE on node startup with dtests at org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
[
https://issues.apache.org/jira/browse/CASSANDRA-14155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726138#comment-16726138
]
Ariel Weisberg commented on CASSANDRA-14155:
This is failing in the rolling upgrade case. [There is a 60 second sleep in the
test between every node upgrade to the newer
version.|https://github.com/apache/cassandra-dtest/blob/master/upgrade_tests/upgrade_through_versions_test.py#L344]
It seems unlikely A or B would not have gossiped with C before C is upgraded.
I got the entire endpoint state map.
{noformat}
ERROR [main] 2018-12-20 18:34:12,868 CassandraDaemon.java:692 - Exception
encountered during startup
java.lang.AssertionError: previous (Gossip ApplicationState.HOST_ID for this
node) is null: {127.0.0.2:7000=EndpointState: HeartBeatState = HeartBeat:
generation = 1545330606, version = 2147483647, AppStateMap = {}}
at org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:833)
at
org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:551)
at
org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:830)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:688)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:639)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:369)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:581)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:670)
{noformat}
I am going to add more logging.
> [TRUNK] Gossiper somewhat frequently hitting an NPE on node startup with
> dtests at
> org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
>
>
> Key: CASSANDRA-14155
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14155
> Project: Cassandra
> Issue Type: Bug
> Components: Lifecycle, Testing
>Reporter: Michael Kjellman
>Assignee: Jason Brown
>Priority: Major
> Labels: dtest
>
> Gossiper is somewhat frequently hitting an NPE on node startup with dtests at
> org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
> {code}
> test teardown failure
> Unexpected error found in node logs (see stdout for full details). Errors:
> [ERROR [main] 2018-01-08 21:41:01,832 CassandraDaemon.java:675 - Exception
> encountered during startup
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:511)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:761)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:621)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:568)
> ~[main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:360)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:569)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:658)
> [main/:na], ERROR [main] 2018-01-08 21:41:01,832 CassandraDaemon.java:675 -
> Exception encountered during startup
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:511)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:761)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:621)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:568)
> ~[main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:360)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:569)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:658)
> [main/:na]]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14155) [TRUNK] Gossiper somewhat frequently hitting an NPE on node startup with dtests at org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
[
https://issues.apache.org/jira/browse/CASSANDRA-14155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726102#comment-16726102
]
Ariel Weisberg commented on CASSANDRA-14155:
I had it log the endpoint state. I should have done the entire thing.
{noformat}
17:58:15,275 conftest ERROR Unexpected error in node2 log, error:
ERROR [main] 2018-12-20 17:54:19,939 CassandraDaemon.java:692 - Exception
encountered during startup
java.lang.AssertionError: previous (Gossip ApplicationState.HOST_ID for this
node) is null: EndpointState: HeartBeatState = HeartBeat: generation =
1545328265, version = 2147483647, AppStateMap = {}
at org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:833)
at
org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:551)
at
org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:830)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:688)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:639)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:369)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:581)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:670)
{noformat}
> [TRUNK] Gossiper somewhat frequently hitting an NPE on node startup with
> dtests at
> org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
>
>
> Key: CASSANDRA-14155
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14155
> Project: Cassandra
> Issue Type: Bug
> Components: Lifecycle, Testing
>Reporter: Michael Kjellman
>Assignee: Jason Brown
>Priority: Major
> Labels: dtest
>
> Gossiper is somewhat frequently hitting an NPE on node startup with dtests at
> org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
> {code}
> test teardown failure
> Unexpected error found in node logs (see stdout for full details). Errors:
> [ERROR [main] 2018-01-08 21:41:01,832 CassandraDaemon.java:675 - Exception
> encountered during startup
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:511)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:761)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:621)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:568)
> ~[main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:360)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:569)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:658)
> [main/:na], ERROR [main] 2018-01-08 21:41:01,832 CassandraDaemon.java:675 -
> Exception encountered during startup
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:511)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:761)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:621)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:568)
> ~[main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:360)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:569)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:658)
> [main/:na]]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14155) [TRUNK] Gossiper somewhat frequently hitting an NPE on node startup with dtests at org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
[
https://issues.apache.org/jira/browse/CASSANDRA-14155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726072#comment-16726072
]
Ariel Weisberg commented on CASSANDRA-14155:
I am debugging this right now. I can reproduce it fairly reliably with the
upgrade tests. https://circleci.com/gh/aweisberg/cassandra/2301
Containers 5, 31, 35 all failed.
> [TRUNK] Gossiper somewhat frequently hitting an NPE on node startup with
> dtests at
> org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
>
>
> Key: CASSANDRA-14155
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14155
> Project: Cassandra
> Issue Type: Bug
> Components: Lifecycle, Testing
>Reporter: Michael Kjellman
>Assignee: Jason Brown
>Priority: Major
> Labels: dtest
>
> Gossiper is somewhat frequently hitting an NPE on node startup with dtests at
> org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
> {code}
> test teardown failure
> Unexpected error found in node logs (see stdout for full details). Errors:
> [ERROR [main] 2018-01-08 21:41:01,832 CassandraDaemon.java:675 - Exception
> encountered during startup
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:511)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:761)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:621)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:568)
> ~[main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:360)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:569)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:658)
> [main/:na], ERROR [main] 2018-01-08 21:41:01,832 CassandraDaemon.java:675 -
> Exception encountered during startup
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:511)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:761)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:621)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:568)
> ~[main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:360)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:569)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:658)
> [main/:na]]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14935) PendingAntiCompaction should be more judicious in the compactions it cancels
[ https://issues.apache.org/jira/browse/CASSANDRA-14935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson reassigned CASSANDRA-14935: --- Assignee: Marcus Eriksson > PendingAntiCompaction should be more judicious in the compactions it cancels > > > Key: CASSANDRA-14935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14935 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Major > Fix For: 4.0 > > > PendingAntiCompaction currently cancels all ongoing compactions to be able to > grab the sstables it requires - it should; > * avoid stopping ongoing anticompactions - for example, if we have an sstable > [0, 100] and are anticompacting it on [0, 50] - now if we start a new > incremental repair on [51, 100] we will (try to) stop the first one. Instead > we should fail the new incremental repair request. > * avoid stopping unrelated regular compactions - we should only stop regular > compactions for the sstables the new anticompaction needs > This requires us to keep track of which sstables are being compacted by a > given compaction id. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14938) Use a stub IndexRegistry when initialised in non-daemon mode
[
https://issues.apache.org/jira/browse/CASSANDRA-14938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-14938:
Status: Ready to Commit (was: Patch Available)
> Use a stub IndexRegistry when initialised in non-daemon mode
>
>
> Key: CASSANDRA-14938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14938
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL, Testing
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Fix For: 4.0
>
>
> Offline processing of FQL logs for workload analysis or replay testing would
> benefit from being able to use C* as a library for parsing and preparation of
> the recorded CQL queries. One thing which makes this difficult is that
> preparing CQL statements involves validating any index restrictions via the
> {{IndexRegistry}} which means instantiating a {{ColumnFamilyStore}}, which
> has several side effects that are troublesome in an offline environment.
> In the FQL processing use case this validation is unnecessary and I imagine
> this is true for most, if not all, offline tools. In these cases, we could
> use a stub {{IndexRegistry}} which enables validation to succeed, but which
> doesn't provide any real functionality.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14938) Use a stub IndexRegistry when initialised in non-daemon mode
[
https://issues.apache.org/jira/browse/CASSANDRA-14938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725823#comment-16725823
]
Marcus Eriksson commented on CASSANDRA-14938:
-
+1
> Use a stub IndexRegistry when initialised in non-daemon mode
>
>
> Key: CASSANDRA-14938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14938
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL, Testing
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Fix For: 4.0
>
>
> Offline processing of FQL logs for workload analysis or replay testing would
> benefit from being able to use C* as a library for parsing and preparation of
> the recorded CQL queries. One thing which makes this difficult is that
> preparing CQL statements involves validating any index restrictions via the
> {{IndexRegistry}} which means instantiating a {{ColumnFamilyStore}}, which
> has several side effects that are troublesome in an offline environment.
> In the FQL processing use case this validation is unnecessary and I imagine
> this is true for most, if not all, offline tools. In these cases, we could
> use a stub {{IndexRegistry}} which enables validation to succeed, but which
> doesn't provide any real functionality.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14915) Handle ant-optional dependency
[
https://issues.apache.org/jira/browse/CASSANDRA-14915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725814#comment-16725814
]
Marcus Eriksson commented on CASSANDRA-14915:
-
and committed as {{23d722ee3a7b91bfe5ee6fe64d090ff247e80438}}, thanks!
of course I messed up the commit message and said ant-junit instead of
ant-optional, the doc change should be correct though
> Handle ant-optional dependency
> --
>
> Key: CASSANDRA-14915
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14915
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 3.0.18, 3.11.4, 4.0
>
>
> CASSANDRA-13117 added a JUnit task which dumps threads on unit test timeout,
> and it depends on a class in {{org.apache.tools.ant.taskdefs.optional}} which
> seems to not always be present depending on how {{ant}} was installed. It can
> cause this error when building;
> {code:java}
> Throws: cassandra-trunk/build.xml:1134: taskdef A class needed by class
> org.krummas.junit.JStackJUnitTask cannot be found:
> org/apache/tools/ant/taskdefs/optional/junit/JUnitTask using the classloader
> AntClassLoader[/.../cassandra-trunk/lib/jstackjunit-0.0.1.jar]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14915) Handle ant-optional dependency
[
https://issues.apache.org/jira/browse/CASSANDRA-14915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-14915:
Resolution: Fixed
Fix Version/s: (was: 3.11.x)
(was: 4.x)
(was: 3.0.x)
4.0
3.11.4
3.0.18
Status: Resolved (was: Patch Available)
> Handle ant-optional dependency
> --
>
> Key: CASSANDRA-14915
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14915
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 3.0.18, 3.11.4, 4.0
>
>
> CASSANDRA-13117 added a JUnit task which dumps threads on unit test timeout,
> and it depends on a class in {{org.apache.tools.ant.taskdefs.optional}} which
> seems to not always be present depending on how {{ant}} was installed. It can
> cause this error when building;
> {code:java}
> Throws: cassandra-trunk/build.xml:1134: taskdef A class needed by class
> org.krummas.junit.JStackJUnitTask cannot be found:
> org/apache/tools/ant/taskdefs/optional/junit/JUnitTask using the classloader
> AntClassLoader[/.../cassandra-trunk/lib/jstackjunit-0.0.1.jar]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/11384c32 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/11384c32 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/11384c32 Branch: refs/heads/trunk Commit: 11384c3279a66e6c0fb7861e2b188b25e963580f Parents: 0a79f9f 08363af Author: Marcus Eriksson Authored: Thu Dec 20 13:44:53 2018 +0100 Committer: Marcus Eriksson Committed: Thu Dec 20 13:44:53 2018 +0100 -- build.xml | 9 - doc/source/development/testing.rst | 8 2 files changed, 12 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/11384c32/build.xml -- diff --cc build.xml index f24647e,8fefd17..3973689 --- a/build.xml +++ b/build.xml @@@ -1235,10 -1193,7 +1235,7 @@@ - - - - + http://git-wip-us.apache.org/repos/asf/cassandra/blob/11384c32/doc/source/development/testing.rst -- diff --cc doc/source/development/testing.rst index eb79932,b8eea6b..7f38fe5 --- a/doc/source/development/testing.rst +++ b/doc/source/development/testing.rst @@@ -52,6 -51,6 +52,14 @@@ To run only the ``testStaticCompactTabl ant testsome -Dtest.name=org.apache.cassandra.cql3.SimpleQueryTest -Dtest.methods=testStaticCompactTables ++If you see an error like this:: ++ ++Throws: cassandra-trunk/build.xml:1134: taskdef A class needed by class org.krummas.junit.JStackJUnitTask cannot be found: ++org/apache/tools/ant/taskdefs/optional/junit/JUnitTask using the classloader ++AntClassLoader[/.../cassandra-trunk/lib/jstackjunit-0.0.1.jar] ++ ++You will need to install the ant-optional package since it contains the ``JUnitTask`` class. ++ Long running tests -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[3/6] cassandra git commit: move task definition inside testmacro to not fail build if ant-junit is not installed
move task definition inside testmacro to not fail build if ant-junit is not installed Patch by marcuse; reviewed by Jon Meredith for CASSANDRA-14915 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/23d722ee Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/23d722ee Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/23d722ee Branch: refs/heads/trunk Commit: 23d722ee3a7b91bfe5ee6fe64d090ff247e80438 Parents: 1816520 Author: Marcus Eriksson Authored: Thu Nov 29 08:53:19 2018 +0100 Committer: Marcus Eriksson Committed: Thu Dec 20 13:38:28 2018 +0100 -- build.xml | 8 1 file changed, 4 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/23d722ee/build.xml -- diff --git a/build.xml b/build.xml index d7e6c4b..7e85991 100644 --- a/build.xml +++ b/build.xml @@ -1130,8 +1130,6 @@ - - + + - + @@ -1215,7 +1215,7 @@ - + - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/08363afa Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/08363afa Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/08363afa Branch: refs/heads/trunk Commit: 08363afa5354c00a7ecd62fe273c392a678db28a Parents: 27c53b5 23d722e Author: Marcus Eriksson Authored: Thu Dec 20 13:41:48 2018 +0100 Committer: Marcus Eriksson Committed: Thu Dec 20 13:41:48 2018 +0100 -- build.xml | 9 - 1 file changed, 4 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/08363afa/build.xml -- diff --cc build.xml index c9565f8,7e85991..8fefd17 --- a/build.xml +++ b/build.xml @@@ -1174,31 -1130,8 +1174,28 @@@ + + + + + + + + + + + + + + + + + + + - - - - + - - - + + + + + + + + Failed to set File Separator. This shouldn't happen. + + + + + + - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/08363afa Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/08363afa Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/08363afa Branch: refs/heads/cassandra-3.11 Commit: 08363afa5354c00a7ecd62fe273c392a678db28a Parents: 27c53b5 23d722e Author: Marcus Eriksson Authored: Thu Dec 20 13:41:48 2018 +0100 Committer: Marcus Eriksson Committed: Thu Dec 20 13:41:48 2018 +0100 -- build.xml | 9 - 1 file changed, 4 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/08363afa/build.xml -- diff --cc build.xml index c9565f8,7e85991..8fefd17 --- a/build.xml +++ b/build.xml @@@ -1174,31 -1130,8 +1174,28 @@@ + + + + + + + + + + + + + + + + + + + - - - - + - - - + + + + + + + + Failed to set File Separator. This shouldn't happen. + + + + + + - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[2/6] cassandra git commit: move task definition inside testmacro to not fail build if ant-junit is not installed
move task definition inside testmacro to not fail build if ant-junit is not installed Patch by marcuse; reviewed by Jon Meredith for CASSANDRA-14915 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/23d722ee Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/23d722ee Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/23d722ee Branch: refs/heads/cassandra-3.11 Commit: 23d722ee3a7b91bfe5ee6fe64d090ff247e80438 Parents: 1816520 Author: Marcus Eriksson Authored: Thu Nov 29 08:53:19 2018 +0100 Committer: Marcus Eriksson Committed: Thu Dec 20 13:38:28 2018 +0100 -- build.xml | 8 1 file changed, 4 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/23d722ee/build.xml -- diff --git a/build.xml b/build.xml index d7e6c4b..7e85991 100644 --- a/build.xml +++ b/build.xml @@ -1130,8 +1130,6 @@ - - + + - + @@ -1215,7 +1215,7 @@ - + - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[1/6] cassandra git commit: move task definition inside testmacro to not fail build if ant-junit is not installed
Repository: cassandra Updated Branches: refs/heads/cassandra-3.0 1816520d6 -> 23d722ee3 refs/heads/cassandra-3.11 27c53b526 -> 08363afa5 refs/heads/trunk 0a79f9f5c -> 11384c327 move task definition inside testmacro to not fail build if ant-junit is not installed Patch by marcuse; reviewed by Jon Meredith for CASSANDRA-14915 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/23d722ee Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/23d722ee Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/23d722ee Branch: refs/heads/cassandra-3.0 Commit: 23d722ee3a7b91bfe5ee6fe64d090ff247e80438 Parents: 1816520 Author: Marcus Eriksson Authored: Thu Nov 29 08:53:19 2018 +0100 Committer: Marcus Eriksson Committed: Thu Dec 20 13:38:28 2018 +0100 -- build.xml | 8 1 file changed, 4 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/23d722ee/build.xml -- diff --git a/build.xml b/build.xml index d7e6c4b..7e85991 100644 --- a/build.xml +++ b/build.xml @@ -1130,8 +1130,6 @@ - - + + - + @@ -1215,7 +1215,7 @@ - + - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14421) Reenable upgrade tests
[
https://issues.apache.org/jira/browse/CASSANDRA-14421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725740#comment-16725740
]
Dinesh Joshi commented on CASSANDRA-14421:
--
[~aweisberg] I managed to get the compact storage tests working with minor
tweaks here -
https://github.com/aweisberg/cassandra-dtest/commit/c081a8cffd5a8a9e4a48284c2e13c8acd3a043eb
> Reenable upgrade tests
> --
>
> Key: CASSANDRA-14421
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14421
> Project: Cassandra
> Issue Type: Task
> Components: Testing
>Reporter: Sam Tunnicliffe
>Assignee: Ariel Weisberg
>Priority: Major
> Fix For: 4.0
>
>
> Since dtests were switched to pytest & python3 in CASSANDRA-13134, the
> upgrade tests have been non-functional and are deselected by default (though
> even if you ran with the {{--execute-upgrade-tests}} they wouldn't work).
> They're further broken by CASSANDRA-14420, as {{upgrade_manifest}} relies on
> {{CASSANDRA_VERSION_FROM_BUILD}}. We need to get them, or something
> equivalent, up and running.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14421) Reenable upgrade tests
[
https://issues.apache.org/jira/browse/CASSANDRA-14421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725736#comment-16725736
]
Dinesh Joshi commented on CASSANDRA-14421:
--
Hi [~philipthompson] do you mind taking a look at
[https://github.com/riptano/ccm/pull/689] as well?
> Reenable upgrade tests
> --
>
> Key: CASSANDRA-14421
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14421
> Project: Cassandra
> Issue Type: Task
> Components: Testing
>Reporter: Sam Tunnicliffe
>Assignee: Ariel Weisberg
>Priority: Major
> Fix For: 4.0
>
>
> Since dtests were switched to pytest & python3 in CASSANDRA-13134, the
> upgrade tests have been non-functional and are deselected by default (though
> even if you ran with the {{--execute-upgrade-tests}} they wouldn't work).
> They're further broken by CASSANDRA-14420, as {{upgrade_manifest}} relies on
> {{CASSANDRA_VERSION_FROM_BUILD}}. We need to get them, or something
> equivalent, up and running.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725733#comment-16725733
]
Stefan Podkowinski commented on CASSANDRA-14806:
LGTM, let's get this merged after holidays.
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
