[jira] [Updated] (CASSANDRA-15462) Purgable tombstones can cause false positives in repaired data tracking
[
https://issues.apache.org/jira/browse/CASSANDRA-15462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-15462:
Status: Ready to Commit (was: Review In Progress)
+1, nits;
* the comment in the {{RepairedDataPurger}} constructor should be removed
({{isForThrift}} is gone in 4.0)
* {{WithTracking}} - keep the {{protected DecoratedKey
applyToPartitionKey(DecoratedKey key)}} method and call
{{repairedDataInfo.onNewPartition(iterator);}} there?
> Purgable tombstones can cause false positives in repaired data tracking
> ---
>
> Key: CASSANDRA-15462
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15462
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Calculating the repaired data digest on the read path (for the purposes of
> detecting mismatches in the repaired data across replicas) is done before
> purging any tombstones due to gcgs or last repaired time. This causes false
> positives when repaired sstables include GC-able tombstones on some replicas
> but not others.
> Validation compactions do purge tombstones so it's perfectly possible for
> sstables to mismatch in this way without causing any streaming during repair.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15462) Purgable tombstones can cause false positives in repaired data tracking
[ https://issues.apache.org/jira/browse/CASSANDRA-15462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15462: Reviewers: Marcus Eriksson, Marcus Eriksson (was: Marcus Eriksson) Marcus Eriksson, Marcus Eriksson (was: Marcus Eriksson) Status: Review In Progress (was: Patch Available) > Purgable tombstones can cause false positives in repaired data tracking > --- > > Key: CASSANDRA-15462 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15462 > Project: Cassandra > Issue Type: Bug > Components: Observability/Metrics >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: 4.0-beta > > > Calculating the repaired data digest on the read path (for the purposes of > detecting mismatches in the repaired data across replicas) is done before > purging any tombstones due to gcgs or last repaired time. This causes false > positives when repaired sstables include GC-able tombstones on some replicas > but not others. > Validation compactions do purge tombstones so it's perfectly possible for > sstables to mismatch in this way without causing any streaming during repair. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15469) IndexInfo/ColumnIndex Refactors & Cleanup
[
https://issues.apache.org/jira/browse/CASSANDRA-15469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020711#comment-17020711
]
Dinesh Joshi commented on CASSANDRA-15469:
--
[~jrwest], thanks for the patch. I have skimmed over the patch and I have some
concerns about making {{column_index_cache_size_in_kb}} a hotprop. I am mainly
concerned that this patch might cause races. For example,
- {{BigTableWriter::append}} method calls {{RowIndexEntry::create}} with
{{ColumnIndex::indexSamples}}. The hotprop change might race here and you may
end up with in memory index samples when you shouldn't.
- In {{ColumnIndex::addIndexBlock}} once we switch to byte-buffer mode,
increasing {{column_index_cache_size_in_kb}} will not switch back to the
{{ArrayList}} mode.
- {{ColumnIndex::reuseOrAllocateBuffer}} will not resize its buffer when
{{column_index_cache_size_in_kb}} is reset.
I haven't yet verified whether the hotprop might cause issues in the above.
Perhaps consider adding a test that will test the effect of changing the value
of {{column_index_cache_size_in_kb}} in some of these scenarios?
I will go through the patch more thoroughly and add more feedback if needed.
> IndexInfo/ColumnIndex Refactors & Cleanup
> -
>
> Key: CASSANDRA-15469
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15469
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/SSTable
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Changes determined by audit of IndexInfo and ColumnIndex changes for
> Cassandra 4.0
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15470) Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB & Bytes
[
https://issues.apache.org/jira/browse/CASSANDRA-15470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020623#comment-17020623
]
Dinesh Joshi edited comment on CASSANDRA-15470 at 1/21/20 11:55 PM:
Thanks [~jrwest] for the review and [~mallika] for the patch. I went over it
and had some feedback. This
[branch|https://github.com/dineshjoshi/cassandra/commits/CASSANDRA-15470-review]
summarizes my proposed changes on top of your patch. If you think it is ok,
I'll go ahead and commit it.
was (Author: djoshi3):
Thanks [~jrwest] for the review and [~mallika] for the patch. I went over it
and had some feedback. This
[commit|https://github.com/apache/cassandra/commit/5232d829dbb3c086ad9eed569c6ea19829cd77ac]
summarizes my proposed changes on top of your patch. If you think it is ok,
I'll go ahead and commit it.
> Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB
> & Bytes
> -
>
> Key: CASSANDRA-15470
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15470
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Jordan West
>Assignee: Mallika Kulkarni
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-rc
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{DatabaseDescriptor}} has several functions that convert between user
> supplied sizes in KB/MB and bytes. These are implemented without much
> consistency and, while unlikely, several have the potential to overflow since
> validation on the input is missing. Meanwhile, some widen the number to a
> long correctly. Options include: widening in all places or simply doing
> better validation on start up — currently only the lower bound of the valid
> range is checked for many of these fields.
> List of Affected {{DatabaseDescriptor}} Methods:
> * {{getColumnIndexSize}}
> * {{getColumnIndexCacheSize}}
> * {{getBatchSizeWarnThreshold}}
> * {{getNativeTransportFrameBlockSize}}
> * {{getRepairSessionSpaceInMegabytes}}
> * {{getNativeTransportMaxFrameSize}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15505) Add message interceptors to in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15505: -- Reviewers: David Capwell, Yifan Cai (was: Yifan Cai) > Add message interceptors to in-jvm dtests > - > > Key: CASSANDRA-15505 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15505 > Project: Cassandra > Issue Type: New Feature > Components: Test/dtest >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > Currently we only have means to filter messages in in-jvm tests. We need a > facility to intercept and modify the messages between nodes for testing > purposes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15505) Add message interceptors to in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-15505: -- Reviewers: Yifan Cai, Yifan Cai (was: Yifan Cai) Yifan Cai, Yifan Cai Status: Review In Progress (was: Patch Available) > Add message interceptors to in-jvm dtests > - > > Key: CASSANDRA-15505 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15505 > Project: Cassandra > Issue Type: New Feature > Components: Test/dtest >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > Currently we only have means to filter messages in in-jvm tests. We need a > facility to intercept and modify the messages between nodes for testing > purposes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15505) Add message interceptors to in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-15505: -- Test and Documentation Plan: unit test and CI Status: Patch Available (was: Open) > Add message interceptors to in-jvm dtests > - > > Key: CASSANDRA-15505 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15505 > Project: Cassandra > Issue Type: New Feature > Components: Test/dtest >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > Currently we only have means to filter messages in in-jvm tests. We need a > facility to intercept and modify the messages between nodes for testing > purposes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15470) Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB & Bytes
[
https://issues.apache.org/jira/browse/CASSANDRA-15470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Joshi updated CASSANDRA-15470:
-
Reviewers: Dinesh Joshi, Jordan West, Dinesh Joshi (was: Dinesh Joshi,
Jordan West)
Dinesh Joshi, Jordan West, Dinesh Joshi (was: Jordan West)
Status: Review In Progress (was: Patch Available)
> Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB
> & Bytes
> -
>
> Key: CASSANDRA-15470
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15470
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Jordan West
>Assignee: Mallika Kulkarni
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-rc
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{DatabaseDescriptor}} has several functions that convert between user
> supplied sizes in KB/MB and bytes. These are implemented without much
> consistency and, while unlikely, several have the potential to overflow since
> validation on the input is missing. Meanwhile, some widen the number to a
> long correctly. Options include: widening in all places or simply doing
> better validation on start up — currently only the lower bound of the valid
> range is checked for many of these fields.
> List of Affected {{DatabaseDescriptor}} Methods:
> * {{getColumnIndexSize}}
> * {{getColumnIndexCacheSize}}
> * {{getBatchSizeWarnThreshold}}
> * {{getNativeTransportFrameBlockSize}}
> * {{getRepairSessionSpaceInMegabytes}}
> * {{getNativeTransportMaxFrameSize}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15470) Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB & Bytes
[
https://issues.apache.org/jira/browse/CASSANDRA-15470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Joshi updated CASSANDRA-15470:
-
Status: Changes Suggested (was: Review In Progress)
Thanks [~jrwest] for the review and [~mallika] for the patch. I went over it
and had some feedback. This
[commit|https://github.com/apache/cassandra/commit/5232d829dbb3c086ad9eed569c6ea19829cd77ac]
summarizes my proposed changes on top of your patch. If you think it is ok,
I'll go ahead and commit it.
> Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB
> & Bytes
> -
>
> Key: CASSANDRA-15470
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15470
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Jordan West
>Assignee: Mallika Kulkarni
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-rc
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{DatabaseDescriptor}} has several functions that convert between user
> supplied sizes in KB/MB and bytes. These are implemented without much
> consistency and, while unlikely, several have the potential to overflow since
> validation on the input is missing. Meanwhile, some widen the number to a
> long correctly. Options include: widening in all places or simply doing
> better validation on start up — currently only the lower bound of the valid
> range is checked for many of these fields.
> List of Affected {{DatabaseDescriptor}} Methods:
> * {{getColumnIndexSize}}
> * {{getColumnIndexCacheSize}}
> * {{getBatchSizeWarnThreshold}}
> * {{getNativeTransportFrameBlockSize}}
> * {{getRepairSessionSpaceInMegabytes}}
> * {{getNativeTransportMaxFrameSize}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15305) Fix multi DC nodetool status output
[
https://issues.apache.org/jira/browse/CASSANDRA-15305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie updated CASSANDRA-15305:
--
Reviewers: Brandon Williams
> Fix multi DC nodetool status output
> ---
>
> Key: CASSANDRA-15305
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15305
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Fix For: 4.0
>
>
> The DC headers are printed before the entire table output
> {code}
> Datacenter: eu-west-1
> =
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> Datacenter: us-east-1
> =
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- Address LoadTokens Owns (effective) Host ID
> Rack
> UN 100.67.118.115 76.85 KiB 1 16.7%
> e1bb616a-a22d-4452-a2eb-f14faa6c7900 eu-west-1c
> UN 100.67.97.192 110.42 KiB 1 16.7%
> fe552b3e-33bf-42c2-9c71-c4e49a034b70 eu-west-1a
> UN 100.67.107.135 76.85 KiB 1 16.7%
> a9079fbe-8d7b-4f07-8d5f-b9e70a63d799 eu-west-1b
> UN 100.67.97.213 94.59 KiB 1 16.7%
> 948e123a-7a92-4b35-974f-c7b380998e37 eu-west-1a
> UN 100.67.118.203 110.42 KiB 1 16.7%
> 6a7d1845-e966-4aba-9245-ed3f5c0805d6 eu-west-1c
> UN 100.67.107.110 110.42 KiB 1 16.7%
> 6e23da83-e9c8-4c43-a9d1-fb0d6e775cdc eu-west-1b
> -- Address LoadTokens Owns (effective) Host ID
> Rack
> UN 100.66.217.160 110.42 KiB 1 16.7%
> 7abef190-f4e7-46e6-9e8d-d4441e6b2e33 us-east-1c
> UN 100.66.220.37 110.42 KiB 1 16.7%
> 6712d2bf-c912-460f-abeb-278770ad7cb4 us-east-1e
> UN 100.66.220.90 110.43 KiB 1 16.7%
> dcb25670-4011-457c-8018-04b22042dde4 us-east-1e
> UN 100.66.216.184 110.42 KiB 1 16.7%
> b620b09a-3de8-41c3-a330-6cee808b4a84 us-east-1c
> UN 100.66.219.60 110.42 KiB 1 16.7%
> 34769195-c582-413f-a663-82f8544d9327 us-east-1d
> UN 100.66.219.93 110.42 KiB 1 16.7%
> 9ae3372c-1a04-457b-bcce-3562a328565e us-east-1d
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15505) Add message interceptors to in-jvm dtests
[
https://issues.apache.org/jira/browse/CASSANDRA-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020531#comment-17020531
]
Yifan Cai commented on CASSANDRA-15505:
---
Thanks [~ifesdjeen] for the patch. The message interceptor could be very useful
to test Cassandra internals by intercepting the internode messages.
My comments:
# Regarding message filtering semantics, should it allow having multiple
filters that matches the same {{from}}, {{to}} and {{verb}} condition? In
{{MessageFilters#process}}, the interceptors is allow to continue matching,
meanwhile the filters are not by returning {{false}} immediately. So the
semantics of {{interceptors}} and {{filters}} are not the same. For instance,
one can have both {{interceptorA(1: from, 2: to, Mutation: verb,
waitForCountdownLatch_1: runnable}} and {{interceptorA(1: from, 2: to,
Mutation: verb, waitForCountdownLatch_2: runnable}}. The usage is confusing and
the behavior is confusing and unpredictable. However, in the current
implementation, such usage is allowed. Personally, I feel the filtering
mechanism should only *allow 1 filter per matching condition*.
# Nice to have a test that shows a filter can be disabled by calling
{{Filter#off()}}
# Nice to have a test that shows having interceptors with blocking runnable
does not block each other. Assuming, we do not want to clog the outing message
sink.
# NIT: {{MessageFilters#DO_NOTHING}} can be {{final}}
# NIT: passing the reference of global filters when constructing a filter seems
little strange. But it is required for supporting adding a filter to the global
filters. Maybe making the constructor private, so a filter can only be
constructed with the more informed builder.
> Add message interceptors to in-jvm dtests
> -
>
> Key: CASSANDRA-15505
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15505
> Project: Cassandra
> Issue Type: New Feature
> Components: Test/dtest
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
>
> Currently we only have means to filter messages in in-jvm tests. We need a
> facility to intercept and modify the messages between nodes for testing
> purposes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch master updated: Remove timeout and concurrent throttle from Jenkins pipeline builds. (Not supported in older Jenkins)
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/master by this push:
new ab09a2d Remove timeout and concurrent throttle from Jenkins pipeline
builds. (Not supported in older Jenkins)
ab09a2d is described below
commit ab09a2d4889244d76332167b2f4d9ca5e8d4fae6
Author: mck
AuthorDate: Tue Jan 21 19:42:36 2020 +0100
Remove timeout and concurrent throttle from Jenkins pipeline builds. (Not
supported in older Jenkins)
---
jenkins-dsl/cassandra_job_dsl_seed.groovy | 8
1 file changed, 8 deletions(-)
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index 4078302..c3b4d86 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -649,14 +649,6 @@ pipelineJob('Cassandra-devbranch') {
numToKeep(50)
artifactNumToKeep(1)
}
-wrappers {
-timeout {
-noActivity(1200)
-}
-}
-throttleConcurrentBuilds {
-categories(['Cassandra'])
-}
parameters {
stringParam('REPO', 'apache', 'The github user/org to clone cassandra
repo from')
stringParam('BRANCH', 'trunk', 'The branch of cassandra to checkout')
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch master updated: Remove timeout and concurrent throttle from Jenkins pipeline builds. (Not supported in older Jenkins)
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/master by this push:
new 5928b1f Remove timeout and concurrent throttle from Jenkins pipeline
builds. (Not supported in older Jenkins)
5928b1f is described below
commit 5928b1f22a2645991db927756fd7ec33d2d0c035
Author: mck
AuthorDate: Tue Jan 21 19:36:19 2020 +0100
Remove timeout and concurrent throttle from Jenkins pipeline builds. (Not
supported in older Jenkins)
---
jenkins-dsl/cassandra_job_dsl_seed.groovy | 8
1 file changed, 8 deletions(-)
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index 28c4eb4..4078302 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -424,14 +424,6 @@ cassandraBranches.each {
numToKeep(50)
artifactNumToKeep(1)
}
-wrappers {
-timeout {
-noActivity(1200)
-}
-}
-throttleConcurrentBuilds {
-categories(['Cassandra'])
-}
definition {
cpsScm {
scm {
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch master updated: In Jenkins, add pipeline builds for each of the supported branch, delegating to the in-tree Jenkinsfile (CASSANDRA-15496)
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/master by this push:
new 5f1871c In Jenkins, add pipeline builds for each of the supported
branch, delegating to the in-tree Jenkinsfile (CASSANDRA-15496)
5f1871c is described below
commit 5f1871c016a8b1036702bf26c3f7bdf63846a8db
Author: mck
AuthorDate: Fri Jan 17 10:35:10 2020 +0100
In Jenkins, add pipeline builds for each of the supported branch,
delegating to the in-tree Jenkinsfile (CASSANDRA-15496)
---
jenkins-dsl/cassandra_job_dsl_seed.groovy | 78 ++--
jenkins-dsl/cassandra_pipeline.groovy | 195 --
2 files changed, 68 insertions(+), 205 deletions(-)
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index 2911b1f..28c4eb4 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -4,7 +4,7 @@
//
-def jobDescription = 'Apache Cassandra DSL-generated job - DSL git repo: https://gitbox.apache.org/repos/asf?p=cassandra-builds.git;>cassandra-builds'
+def jobDescription = 'http://cassandra.apache.org/img/cassandra_logo.png; />Apache
Cassandra DSL-generated job - DSL git repo: https://gitbox.apache.org/repos/asf?p=cassandra-builds.git;>cassandra-builds'
def jdkLabel = 'JDK 1.8 (latest)'
if(binding.hasVariable("CASSANDRA_JDK_LABEL")) {
jdkLabel = "${CASSANDRA_JDK_LABEL}"
@@ -169,7 +169,7 @@ job('Cassandra-template-test') {
}
publishers {
archiveArtifacts {
-pattern('build/test/**/TEST-*.xml')
+
pattern('build/test/**/TEST-*.xml,build/**/eclipse_compiler_checks.txt')
allowEmpty()
fingerprint()
}
@@ -414,6 +414,44 @@ cassandraBranches.each {
}
}
}
+
+/**
+ * Branch Pipelines
+ */
+pipelineJob("${jobNamePrefix}") {
+description(jobDescription)
+logRotator {
+numToKeep(50)
+artifactNumToKeep(1)
+}
+wrappers {
+timeout {
+noActivity(1200)
+}
+}
+throttleConcurrentBuilds {
+categories(['Cassandra'])
+}
+definition {
+cpsScm {
+scm {
+git {
+remote {
+url(mainRepo)
+}
+branch("${branchName}")
+extensions {
+cleanAfterCheckout()
+}
+}
+}
+scriptPath('.jenkins/Jenkinsfile')
+}
+}
+triggers {
+scm('H/30 * * * *')
+}
+}
}
@@ -466,11 +504,11 @@ testTargets.each {
shell("./cassandra-builds/build-scripts/cassandra-unittest.sh
${targetName}")
}
publishers {
-archiveArtifacts {
-pattern('build/test/**/TEST-*.xml')
-allowEmpty()
-fingerprint()
-}
+archiveArtifacts {
+
pattern('build/test/**/TEST-*.xml,build/**/eclipse_compiler_checks.txt')
+allowEmpty()
+fingerprint()
+}
archiveJunit('build/test/**/TEST-*.xml') {
testDataPublishers {
publishTestStabilityData()
@@ -619,16 +657,36 @@ pipelineJob('Cassandra-devbranch') {
numToKeep(50)
artifactNumToKeep(1)
}
+wrappers {
+timeout {
+noActivity(1200)
+}
+}
+throttleConcurrentBuilds {
+categories(['Cassandra'])
+}
parameters {
stringParam('REPO', 'apache', 'The github user/org to clone cassandra
repo from')
stringParam('BRANCH', 'trunk', 'The branch of cassandra to checkout')
stringParam('DTEST_REPO', "${dtestRepo}", 'The cassandra-dtest repo
URL')
stringParam('DTEST_BRANCH', 'master', 'The branch of cassandra-dtest
to checkout')
+stringParam('DOCKER_IMAGE', "${dtestDockerImage}", 'Docker image for
running dtests')
}
definition {
-cps {
-script(readFileFromWorkspace('Cassandra-Job-DSL',
'jenkins-dsl/cassandra_pipeline.groovy'))
-sandbox()
+cpsScm {
+scm {
+git {
+remote {
+url('https://github.com/${REPO}/cassandra.git')
+}
+branch('${BRANCH}')
+extensions {
+cleanAfterCheckout()
+}
+}
+}
+
[jira] [Updated] (CASSANDRA-15509) In-jvm upgrade dtest version parsing does not support 4.0 alpha/beta/rc builds
[ https://issues.apache.org/jira/browse/CASSANDRA-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15509: -- Status: Ready to Commit (was: Review In Progress) circle ci is passing and manual testing of changes works for expected versions with "preRelease" tags. > In-jvm upgrade dtest version parsing does not support 4.0 alpha/beta/rc builds > -- > > Key: CASSANDRA-15509 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15509 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > > for example: > https://circleci.com/gh/krummas/cassandra/2686 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15509) In-jvm upgrade dtest version parsing does not support 4.0 alpha/beta/rc builds
[
https://issues.apache.org/jira/browse/CASSANDRA-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020452#comment-17020452
]
David Capwell edited comment on CASSANDRA-15509 at 1/21/20 6:18 PM:
LGTM +1
Testing done outside of circle:
{code}
$ jshell
Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true
| Welcome to JShell -- Version 11.0.2
| For an introduction type: /help intro
jshell> import java.util.regex.Pattern
jshell> Pattern pattern =
Pattern.compile("dtest-(?(\\d+)\\.(\\d+)(\\.\\d+)?(\\.\\d+)?)([~\\-]\\w[.\\w]*(?:\\-\\w[.\\w]*)*)?(\\+[.\\w]+)?\\.jar");
pattern ==> dtest-(?(\d+)\.(\d+)(\.\d+)?(\.\d+)? ...
[.\w]*)*)?(\+[.\w]+)?\.jar
jshell> var m = pattern.matcher("dtest-4.0-alpha3.jar")
m ==> java.util.regex.Matcher[pattern=dtest-(? m.matches()
$9 ==> true
jshell> m.group("fullversion")
$10 ==> "4.0"
jshell> var m = pattern.matcher("dtest-4.0.0.1-alpha3.jar")
m ==> java.util.regex.Matcher[pattern=dtest-(? m.matches()
$12 ==> true
jshell> m.group("fullversion")
$13 ==> "4.0.0.1"
jshell> var m = pattern.matcher("dtest-4.0-rc-4.jar")
m ==> java.util.regex.Matcher[pattern=dtest-(? m.matches()
$15 ==> true
jshell> m.group("fullversion")
$16 ==> "4.0"
{code}
was (Author: dcapwell):
LGTM +1
Testing done outside of circle:
{code}
$ jshell
Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true
| Welcome to JShell -- Version 11.0.2
| For an introduction type: /help intro
jshell> import java.util.regex.Pattern
jshell> Pattern pattern =
Pattern.compile("dtest-(?(\\d+)\\.(\\d+)(\\.\\d+)?(\\.\\d+)?)([~\\-]\\w[.\\w]*(?:\\-\\w[.\\w]*)*)?(\\+[.\\w]+)?\\.jar");
pattern ==> dtest-(?(\d+)\.(\d+)(\.\d+)?(\.\d+)? ...
[.\w]*)*)?(\+[.\w]+)?\.jar
jshell> var m = pattern.matcher("dtest-4.0-alpha3.jar")
m ==> java.util.regex.Matcher[pattern=dtest-(? m.matches()
$9 ==> true
jshell> m.group("fullversion")
$10 ==> "4.0"
jshell> var m = pattern.matcher("dtest-4.0.0.1-alpha3.jar")
m ==> java.util.regex.Matcher[pattern=dtest-(? m.matches()
$12 ==> true
jshell> m.group("fullversion")
$13 ==> "4.0.0.1"
{code}
> In-jvm upgrade dtest version parsing does not support 4.0 alpha/beta/rc builds
> --
>
> Key: CASSANDRA-15509
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15509
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
>
> for example:
> https://circleci.com/gh/krummas/cassandra/2686
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15496:
---
Fix Version/s: (was: 3.11.x)
(was: 4.x)
(was: 3.0.x)
(was: 2.2.x)
4.0-alpha
3.11.6
3.0.20
2.2.16
Source Control Link:
https://github.com/apache/cassandra/commit/d69f7696e4fa2a48795e2e7f08e9235455eeb15a
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed as d69f7696e4fa2a48795e2e7f08e9235455eeb15a
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.16, 3.0.20, 3.11.6, 4.0-alpha
>
> Attachments: Screenshot 2020-01-17 at 21.14.52.png, Screenshot
> 2020-01-17 at 21.16.05.png, Screenshot 2020-01-17 at 21.16.47.png, Screenshot
> 2020-01-17 at 21.18.49.png
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15509) In-jvm upgrade dtest version parsing does not support 4.0 alpha/beta/rc builds
[
https://issues.apache.org/jira/browse/CASSANDRA-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020452#comment-17020452
]
David Capwell commented on CASSANDRA-15509:
---
LGTM +1
Testing done outside of circle:
{code}
$ jshell
Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true
| Welcome to JShell -- Version 11.0.2
| For an introduction type: /help intro
jshell> import java.util.regex.Pattern
jshell> Pattern pattern =
Pattern.compile("dtest-(?(\\d+)\\.(\\d+)(\\.\\d+)?(\\.\\d+)?)([~\\-]\\w[.\\w]*(?:\\-\\w[.\\w]*)*)?(\\+[.\\w]+)?\\.jar");
pattern ==> dtest-(?(\d+)\.(\d+)(\.\d+)?(\.\d+)? ...
[.\w]*)*)?(\+[.\w]+)?\.jar
jshell> var m = pattern.matcher("dtest-4.0-alpha3.jar")
m ==> java.util.regex.Matcher[pattern=dtest-(? m.matches()
$9 ==> true
jshell> m.group("fullversion")
$10 ==> "4.0"
jshell> var m = pattern.matcher("dtest-4.0.0.1-alpha3.jar")
m ==> java.util.regex.Matcher[pattern=dtest-(? m.matches()
$12 ==> true
jshell> m.group("fullversion")
$13 ==> "4.0.0.1"
{code}
> In-jvm upgrade dtest version parsing does not support 4.0 alpha/beta/rc builds
> --
>
> Key: CASSANDRA-15509
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15509
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
>
> for example:
> https://circleci.com/gh/krummas/cassandra/2686
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: Create Jenkins pipeline definition, and split out Jenkins test-all builds to individual builds for each of the test targets
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-2.2 by this push:
new d69f769 Create Jenkins pipeline definition, and split out Jenkins
test-all builds to individual builds for each of the test targets
d69f769 is described below
commit d69f7696e4fa2a48795e2e7f08e9235455eeb15a
Author: Mick Semb Wever
AuthorDate: Fri Jan 10 10:34:27 2020 +0100
Create Jenkins pipeline definition, and split out Jenkins test-all builds
to individual builds for each of the test targets
patch by Mick Semb Wever; reviewed by David Capwell for CASSANDRA-15496
---
.circleci/config.yml | 2 +-
.circleci/config.yml.HIGHRES | 2 +-
.circleci/config.yml.LOWRES | 2 +-
.jenkins/Jenkinsfile | 273 +++
build.xml| 6 +-
ide/idea/workspace.xml | 1 -
6 files changed, 277 insertions(+), 9 deletions(-)
diff --git a/.circleci/config.yml b/.circleci/config.yml
index fee6f2c..b1ed0ef 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -1085,4 +1085,4 @@ workflows:
# destination: dtest_<>
# - store_artifacts:
# path: ~/cassandra-dtest/logs
-# destination: dtest_<>_logs
\ No newline at end of file
+# destination: dtest_<>_logs
diff --git a/.circleci/config.yml.HIGHRES b/.circleci/config.yml.HIGHRES
index 7340d99..4bbef27 100644
--- a/.circleci/config.yml.HIGHRES
+++ b/.circleci/config.yml.HIGHRES
@@ -1085,4 +1085,4 @@ workflows:
# destination: dtest_<>
# - store_artifacts:
# path: ~/cassandra-dtest/logs
-# destination: dtest_<>_logs
\ No newline at end of file
+# destination: dtest_<>_logs
diff --git a/.circleci/config.yml.LOWRES b/.circleci/config.yml.LOWRES
index fee6f2c..b1ed0ef 100644
--- a/.circleci/config.yml.LOWRES
+++ b/.circleci/config.yml.LOWRES
@@ -1085,4 +1085,4 @@ workflows:
# destination: dtest_<>
# - store_artifacts:
# path: ~/cassandra-dtest/logs
-# destination: dtest_<>_logs
\ No newline at end of file
+# destination: dtest_<>_logs
diff --git a/.jenkins/Jenkinsfile b/.jenkins/Jenkinsfile
new file mode 100644
index 000..adb2a8d
--- /dev/null
+++ b/.jenkins/Jenkinsfile
@@ -0,0 +1,273 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+//https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+//
+// Jenkins declaration of how to build and test the current codebase.
+// Jenkins infrastructure related settings should be kept in
+//
https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy
+//
+
+pipeline {
+ agent any
+ stages {
+ stage('Init') {
+steps {
+cleanWs()
+}
+ }
+ stage('Build') {
+steps {
+build job: "${env.JOB_NAME}-artifacts"
+}
+ }
+ stage('Test') {
+ parallel {
+stage('JVM DTests') {
+ steps {
+ warnError('Tests unstable') {
+build job: "${env.JOB_NAME}-test-jvm-dtest-forking"
+ }
+ }
+ post {
+success {
+warnError('missing test xml files') {
+script {
+copyTestResults('test-jvm-dtest-forking')
+}
+}
+}
+unstable {
+warnError('missing test xml files') {
+script {
+copyTestResults('test-jvm-dtest-forking')
+}
+}
+}
+ }
+}
+stage('units') {
+steps {
+ warnError('Tests unstable') {
+build job: "${env.JOB_NAME}-test"
+ }
+}
+ post {
+success {
+warnError('missing test xml files') {
+script {
+
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 8f7f6e903b0dea5f3cc1f5666308216ce0bd3c7d
Merge: 1fbd329 6a8e211
Author: Mick Semb Wever
AuthorDate: Tue Jan 21 19:11:21 2020 +0100
Merge branch 'cassandra-3.11' into trunk
.jenkins/Jenkinsfile | 366 +
build.xml | 6 +-
ide/idea/workspace.xml | 1 -
3 files changed, 367 insertions(+), 6 deletions(-)
diff --cc .jenkins/Jenkinsfile
index 000,353739f..ab0280d
mode 00,100644..100644
--- a/.jenkins/Jenkinsfile
+++ b/.jenkins/Jenkinsfile
@@@ -1,0 -1,342 +1,366 @@@
+ // Licensed to the Apache Software Foundation (ASF) under one
+ // or more contributor license agreements. See the NOTICE file
+ // distributed with this work for additional information
+ // regarding copyright ownership. The ASF licenses this file
+ // to you under the Apache License, Version 2.0 (the
+ // "License"); you may not use this file except in compliance
+ // with the License. You may obtain a copy of the License at
+ //
+ //https://www.apache.org/licenses/LICENSE-2.0
+ //
+ // Unless required by applicable law or agreed to in writing,
+ // software distributed under the License is distributed on an
+ // "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ // KIND, either express or implied. See the License for the
+ // specific language governing permissions and limitations
+ // under the License.
+ //
+ //
+ // Jenkins declaration of how to build and test the current codebase.
+ // Jenkins infrastructure related settings should be kept in
+ //
https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy
+ //
+
+ pipeline {
+ agent any
+ stages {
- stage('Init') {
-steps {
-cleanWs()
-}
++stage('Init') {
++ steps {
++ cleanWs()
+ }
- stage('Build') {
-steps {
-build job: "${env.JOB_NAME}-artifacts"
-}
++}
++stage('Build') {
++ steps {
++ build job: "${env.JOB_NAME}-artifacts"
+ }
- stage('Test') {
- parallel {
-stage('stress') {
- steps {
- warnError('Tests unstable') {
- build job: "${env.JOB_NAME}-stress-test"
++}
++stage('Test') {
++parallel {
++ stage('stress') {
++steps {
++warnError('Tests unstable') {
++build job: "${env.JOB_NAME}-stress-test"
++}
++}
++post {
++ success {
++ warnError('missing test xml files') {
++ script {
++ copyTestResults('stress-test')
++ }
+ }
+ }
- post {
-success {
-warnError('missing test xml files') {
-script {
-copyTestResults('stress-test')
-}
-}
-}
-unstable {
-warnError('missing test xml files') {
-script {
-copyTestResults('stress-test')
-}
-}
-}
++ unstable {
++ warnError('missing test xml files') {
++ script {
++ copyTestResults('stress-test')
++ }
++ }
+ }
+ }
-stage('JVM DTests') {
- steps {
- warnError('Tests unstable') {
-build job: "${env.JOB_NAME}-test-jvm-dtest-forking"
++ }
++ stage('fqltool') {
++steps {
++warnError('Tests unstable') {
++build job: "${env.JOB_NAME}-fqltool-test"
++}
++}
++post {
++ success {
++ warnError('missing test xml files') {
++ script {
++ copyTestResults('fqltool-test')
++ }
+ }
+ }
- post {
-success {
-warnError('missing test xml files') {
-script {
-copyTestResults('test-jvm-dtest-forking')
-}
-}
-}
-unstable {
-warnError('missing test xml files') {
-script {
-copyTestResults('test-jvm-dtest-forking')
-}
-}
-}
++ unstable {
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 14aa561c3229585a065a2b308e5e2b236b8f56ba
Merge: 5f57212 d69f769
Author: Mick Semb Wever
AuthorDate: Tue Jan 21 19:05:54 2020 +0100
Merge branch 'cassandra-2.2' into cassandra-3.0
.jenkins/Jenkinsfile | 273 +
build.xml | 6 +-
ide/idea/workspace.xml | 1 -
3 files changed, 274 insertions(+), 6 deletions(-)
diff --cc .jenkins/Jenkinsfile
index 000,adb2a8d..572e8de
mode 00,100644..100644
--- a/.jenkins/Jenkinsfile
+++ b/.jenkins/Jenkinsfile
@@@ -1,0 -1,273 +1,273 @@@
+ // Licensed to the Apache Software Foundation (ASF) under one
+ // or more contributor license agreements. See the NOTICE file
+ // distributed with this work for additional information
+ // regarding copyright ownership. The ASF licenses this file
+ // to you under the Apache License, Version 2.0 (the
+ // "License"); you may not use this file except in compliance
+ // with the License. You may obtain a copy of the License at
+ //
+ //https://www.apache.org/licenses/LICENSE-2.0
+ //
+ // Unless required by applicable law or agreed to in writing,
+ // software distributed under the License is distributed on an
+ // "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ // KIND, either express or implied. See the License for the
+ // specific language governing permissions and limitations
+ // under the License.
+ //
+ //
+ // Jenkins declaration of how to build and test the current codebase.
+ // Jenkins infrastructure related settings should be kept in
+ //
https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy
+ //
+
+ pipeline {
+ agent any
+ stages {
+ stage('Init') {
+ steps {
+ cleanWs()
+ }
+ }
+ stage('Build') {
+ steps {
+ build job: "${env.JOB_NAME}-artifacts"
+ }
+ }
+ stage('Test') {
+ parallel {
+ stage('JVM DTests') {
+ steps {
+ warnError('Tests unstable') {
+ build job: "${env.JOB_NAME}-test-jvm-dtest-forking"
+ }
+ }
+ post {
+ success {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('test-jvm-dtest-forking')
+ }
+ }
+ }
+ unstable {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('test-jvm-dtest-forking')
+ }
+ }
+ }
+ }
+ }
+ stage('units') {
+ steps {
+ warnError('Tests unstable') {
+ build job: "${env.JOB_NAME}-test"
+ }
+ }
+ post {
+ success {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('test')
+ }
+ }
+ }
+ unstable {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('test')
+ }
+ }
+ }
+ }
+ }
+ stage('long units') {
+ steps {
+ warnError('Tests unstable') {
+ build job: "${env.JOB_NAME}-long-test"
+ }
+ }
+ post {
+ success {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('long-test')
+ }
+ }
+ }
+ unstable {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('long-test')
+ }
+ }
+ }
+ }
+ }
+ stage('burn') {
+ steps {
+ warnError('Tests unstable') {
+ build job: "${env.JOB_NAME}-test-burn"
+ }
+ }
+ post {
+ success {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('test-burn')
+ }
+ }
+ }
+ unstable {
+
[cassandra] branch trunk updated (1fbd329 -> 8f7f6e9)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 1fbd329 Release session from cache when closing QueryReplayer new d69f769 Create Jenkins pipeline definition, and split out Jenkins test-all builds to individual builds for each of the test targets new 14aa561 Merge branch 'cassandra-2.2' into cassandra-3.0 new 6a8e211 Merge branch 'cassandra-3.0' into cassandra-3.11 new 8f7f6e9 Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .jenkins/Jenkinsfile | 366 + build.xml | 6 +- ide/idea/workspace.xml | 1 - 3 files changed, 367 insertions(+), 6 deletions(-) create mode 100644 .jenkins/Jenkinsfile - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 6a8e211fbac3eba2f1877a119c3059a2b89f3354
Merge: 3bb388b 14aa561
Author: Mick Semb Wever
AuthorDate: Tue Jan 21 19:09:27 2020 +0100
Merge branch 'cassandra-3.0' into cassandra-3.11
.jenkins/Jenkinsfile | 342 +
build.xml | 6 +-
ide/idea/workspace.xml | 1 -
3 files changed, 343 insertions(+), 6 deletions(-)
diff --cc .jenkins/Jenkinsfile
index 000,572e8de..353739f
mode 00,100644..100644
--- a/.jenkins/Jenkinsfile
+++ b/.jenkins/Jenkinsfile
@@@ -1,0 -1,273 +1,342 @@@
+ // Licensed to the Apache Software Foundation (ASF) under one
+ // or more contributor license agreements. See the NOTICE file
+ // distributed with this work for additional information
+ // regarding copyright ownership. The ASF licenses this file
+ // to you under the Apache License, Version 2.0 (the
+ // "License"); you may not use this file except in compliance
+ // with the License. You may obtain a copy of the License at
+ //
+ //https://www.apache.org/licenses/LICENSE-2.0
+ //
+ // Unless required by applicable law or agreed to in writing,
+ // software distributed under the License is distributed on an
+ // "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ // KIND, either express or implied. See the License for the
+ // specific language governing permissions and limitations
+ // under the License.
+ //
+ //
+ // Jenkins declaration of how to build and test the current codebase.
+ // Jenkins infrastructure related settings should be kept in
+ //
https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy
+ //
+
+ pipeline {
+ agent any
+ stages {
+ stage('Init') {
+ steps {
+ cleanWs()
+ }
+ }
+ stage('Build') {
+ steps {
+ build job: "${env.JOB_NAME}-artifacts"
+ }
+ }
+ stage('Test') {
+ parallel {
++stage('stress') {
++ steps {
++ warnError('Tests unstable') {
++ build job: "${env.JOB_NAME}-stress-test"
++ }
++ }
++ post {
++success {
++warnError('missing test xml files') {
++script {
++copyTestResults('stress-test')
++}
++}
++}
++unstable {
++warnError('missing test xml files') {
++script {
++copyTestResults('stress-test')
++}
++}
++}
++ }
++}
+ stage('JVM DTests') {
+ steps {
+ warnError('Tests unstable') {
+ build job: "${env.JOB_NAME}-test-jvm-dtest-forking"
+ }
+ }
+ post {
+ success {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('test-jvm-dtest-forking')
+ }
+ }
+ }
+ unstable {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('test-jvm-dtest-forking')
+ }
+ }
+ }
+ }
+ }
+ stage('units') {
+ steps {
+ warnError('Tests unstable') {
+ build job: "${env.JOB_NAME}-test"
+ }
+ }
+ post {
+ success {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('test')
+ }
+ }
+ }
+ unstable {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('test')
+ }
+ }
+ }
+ }
+ }
+ stage('long units') {
+ steps {
+ warnError('Tests unstable') {
+ build job: "${env.JOB_NAME}-long-test"
+ }
+ }
+ post {
+ success {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('long-test')
+ }
+ }
+ }
+
[cassandra] branch cassandra-3.11 updated (3bb388b -> 6a8e211)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 3bb388b Merge branch 'cassandra-3.0' into cassandra-3.11 new d69f769 Create Jenkins pipeline definition, and split out Jenkins test-all builds to individual builds for each of the test targets new 14aa561 Merge branch 'cassandra-2.2' into cassandra-3.0 new 6a8e211 Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .jenkins/Jenkinsfile | 342 + build.xml | 6 +- ide/idea/workspace.xml | 1 - 3 files changed, 343 insertions(+), 6 deletions(-) create mode 100644 .jenkins/Jenkinsfile - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (5f57212 -> 14aa561)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 5f57212 Run in-jvm upgrade dtests in circle ci new d69f769 Create Jenkins pipeline definition, and split out Jenkins test-all builds to individual builds for each of the test targets new 14aa561 Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .jenkins/Jenkinsfile | 273 + build.xml | 6 +- ide/idea/workspace.xml | 1 - 3 files changed, 274 insertions(+), 6 deletions(-) create mode 100644 .jenkins/Jenkinsfile - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15496:
---
Status: Ready to Commit (was: Review In Progress)
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: Screenshot 2020-01-17 at 21.14.52.png, Screenshot
> 2020-01-17 at 21.16.05.png, Screenshot 2020-01-17 at 21.16.47.png, Screenshot
> 2020-01-17 at 21.18.49.png
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020434#comment-17020434
]
Michael Semb Wever commented on CASSANDRA-15496:
bq. Nit (aka you can ignore)
https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_15496#diff-9b83915ce4a53a5b3cc01b9f39d6a984R85.
Would be good to just call this "JVM Dtest" rather than "JVM forking" (stage
name only), that way the implementation detail doesn't cause a history naming
conflict, and more clear what is forking.
Fixed, thanks: good catch.
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: Screenshot 2020-01-17 at 21.14.52.png, Screenshot
> 2020-01-17 at 21.16.05.png, Screenshot 2020-01-17 at 21.16.47.png, Screenshot
> 2020-01-17 at 21.18.49.png
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020404#comment-17020404
]
David Capwell commented on CASSANDRA-15496:
---
Nit (aka you can ignore)
[https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_15496#diff-9b83915ce4a53a5b3cc01b9f39d6a984R85].
Would be good to just call this "JVM Dtest" rather than "JVM forking" (stage
name only), that way the implementation detail doesn't cause a history naming
conflict, and more clear what is forking.
Looked at latest changes after simplifying ant. LGTM +1
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: Screenshot 2020-01-17 at 21.14.52.png, Screenshot
> 2020-01-17 at 21.16.05.png, Screenshot 2020-01-17 at 21.16.47.png, Screenshot
> 2020-01-17 at 21.18.49.png
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15505) Add message interceptors to in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020261#comment-17020261 ] Alex Petrov commented on CASSANDRA-15505: - [~dcapwell] I do not fully understand what you mean. Both filters and interceptors are built on top of message sink, which is I'd argue the only low-level feature there is, really. I do, in fact, implement interceptors on top of the existing filters. If you elaborate a bit more what you mean by a syntactic sugar on top of a lower level feature, I can take a look at it. But given we're already using filters as a base/core for the implementation, I'm a bit lost. > Add message interceptors to in-jvm dtests > - > > Key: CASSANDRA-15505 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15505 > Project: Cassandra > Issue Type: New Feature > Components: Test/dtest >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > Currently we only have means to filter messages in in-jvm tests. We need a > facility to intercept and modify the messages between nodes for testing > purposes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14740) BlockingReadRepair does not maintain monotonicity during range movements
[
https://issues.apache.org/jira/browse/CASSANDRA-14740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-14740:
Status: Changes Suggested (was: Review In Progress)
This is nice. As well as addressing the bug, using a {{ReplicaPlan}} for the
writes really clarifies {{BlockingPartitionRepair}} & {{BlockingReadRepair}}.
There's a copy/paste bug in {{RowIteratorMergeListener::applyToPartition}} in
the {{buildFullDiff}} branch, where it should be setting
{{repairs[repairs.length - 1]}}. This is causing the new
{{movingTokenReadRepairTest}} failure, but locally at least, it doesn't affect
the other tests in that fixture so I'm not sure what's going on in CircleCI.
The patch also needs a rebase, but it doesn't look too onerous.
Nits:
* {{BlockingReadRepair}} #54: extraneous comment
* {{ReadRepairTest}} #199: formatting
* {{RowIteratorMergeListener}}: unused import
* I find the style of having the conditions and statements for an {{if/else}}
on the same line ({{RowIteratorMergeListener}} #373) makes them harder to
parse. This is clearly rather subjective though and maybe just because it's not
followed more universally in the project - feel free to ignore.
> BlockingReadRepair does not maintain monotonicity during range movements
>
>
> Key: CASSANDRA-14740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14740
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Coordination
>Reporter: Benedict Elliott Smith
>Assignee: Benedict Elliott Smith
>Priority: Urgent
> Labels: correctness
> Fix For: 4.0, 4.0-beta
>
>
> The BlockingReadRepair code introduced by CASSANDRA-10726 requires that each
> of the queried nodes are written to, but pending nodes are not considered.
> If there is a pending range movement, one of these writes may be ‘lost’ when
> the range movement completes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14740) BlockingReadRepair does not maintain monotonicity during range movements
[ https://issues.apache.org/jira/browse/CASSANDRA-14740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-14740: Reviewers: Sam Tunnicliffe, Sam Tunnicliffe (was: Sam Tunnicliffe) Sam Tunnicliffe, Sam Tunnicliffe (was: Sam Tunnicliffe) Status: Review In Progress (was: Patch Available) > BlockingReadRepair does not maintain monotonicity during range movements > > > Key: CASSANDRA-14740 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14740 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Coordination >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Urgent > Labels: correctness > Fix For: 4.0, 4.0-beta > > > The BlockingReadRepair code introduced by CASSANDRA-10726 requires that each > of the queried nodes are written to, but pending nodes are not considered. > If there is a pending range movement, one of these writes may be ‘lost’ when > the range movement completes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
