[jira] [Updated] (CASSANDRA-15921) 4.0 quality testing: Materialized View
[ https://issues.apache.org/jira/browse/CASSANDRA-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhaoYang updated CASSANDRA-15921: - Change Category: Quality Assurance Complexity: Normal Status: Open (was: Triage Needed) > 4.0 quality testing: Materialized View > -- > > Key: CASSANDRA-15921 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15921 > Project: Cassandra > Issue Type: Task > Components: Feature/Materialized Views >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Normal > Fix For: 4.x > > > The main purpose of this ticket to get a better understanding about 4.0 MV > status as a guideline for future improvements. I don't think it should block > 4.0 release since it's already marked as experimental. > Main areas to test: > * Write perf: We expect to see [10% write throughput drop per MV > added|https://www.datastax.com/blog/2016/05/materialized-view-performance-cassandra-3x]. > * Read perf: identical to normal table > * Bootstrap/Decommision: no write-path required since CASSANDRA-13065 > * Repair: write path required > * Chaos monkey: take down coordinator/base-replica/view-replica during > read/write/token-movement and verify data consistency (may need a tool) > * Hint Replay: able to throttle if table has MV - CASSANDRA-13810 > * Schema race: create/drop - CASSANDRA-15845/CASSANDRA-15918 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15921) 4.0 quality testing: Materialized View
[ https://issues.apache.org/jira/browse/CASSANDRA-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhaoYang updated CASSANDRA-15921: - Issue Type: Task (was: Improvement) > 4.0 quality testing: Materialized View > -- > > Key: CASSANDRA-15921 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15921 > Project: Cassandra > Issue Type: Task > Components: Feature/Materialized Views >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Normal > Fix For: 4.x > > > The main purpose of this ticket to get a better understanding about 4.0 MV > status as a guideline for future improvements. I don't think it should block > 4.0 release since it's already marked as experimental. > Main areas to test: > * Write perf: We expect to see [10% write throughput drop per MV > added|https://www.datastax.com/blog/2016/05/materialized-view-performance-cassandra-3x]. > * Read perf: identical to normal table > * Bootstrap/Decommision: no write-path required since CASSANDRA-13065 > * Repair: write path required > * Chaos monkey: take down coordinator/base-replica/view-replica during > read/write/token-movement and verify data consistency (may need a tool) > * Hint Replay: able to throttle if table has MV - CASSANDRA-13810 > * Schema race: create/drop - CASSANDRA-15845/CASSANDRA-15918 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15900) Close channel and reduce buffer allocation during entire sstable streaming with SSL
[ https://issues.apache.org/jira/browse/CASSANDRA-15900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151008#comment-17151008 ] ZhaoYang commented on CASSANDRA-15900: -- both SimpleReadWriteTest and ImportTest passed locally with JDK11, I don't think they use streaming. > Close channel and reduce buffer allocation during entire sstable streaming > with SSL > --- > > Key: CASSANDRA-15900 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15900 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Streaming and Messaging >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Normal > Fix For: 4.0-beta > > > CASSANDRA-15740 added the ability to stream entire sstable by loading on-disk > file into user-space off-heap buffer when SSL is enabled, because netty > doesn't support zero-copy with SSL. > But there are two issues: > # file channel is not closed. > # 1mb batch size is used. 1mb exceeds buffer pool's max allocation size, > thus it's all allocated outside the pool and will cause large amount of > allocations. > [Patch|https://github.com/apache/cassandra/pull/651]: > # close file channel when the last batch is loaded into off-heap bytebuffer. > I don't think we need to wait until buffer is flushed by netty. > # reduce the batch to 64kb which is more buffer pool friendly when streaming > entire sstable with SSL. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15214) OOMs caught and not rethrown
[ https://issues.apache.org/jira/browse/CASSANDRA-15214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17150869#comment-17150869 ] Robert Stupp commented on CASSANDRA-15214: -- Just read this ticket and the approach looks absolutely reasonable to me. One thing though is that the the (off-heap) row-cache isn't covered here - let me know whether it's reasonable to add some support regarding this ticket. IMHO, people shouldn't use the row-cache, but I'm not sure whether there are reasonable use cases out there in the wild. Don't want to start a discussion about the row-cache in this, just a heads-up. > OOMs caught and not rethrown > > > Key: CASSANDRA-15214 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15214 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client, Messaging/Internode >Reporter: Benedict Elliott Smith >Priority: Normal > Fix For: 4.0, 4.0-rc > > Attachments: oom-experiments.zip > > > Netty (at least, and perhaps elsewhere in Executors) catches all exceptions, > so presently there is no way to ensure that an OOM reaches the JVM handler to > trigger a crash/heapdump. > It may be that the simplest most consistent way to do this would be to have a > single thread spawned at startup that waits for any exceptions we must > propagate to the Runtime. > We could probably submit a patch upstream to Netty, but for a guaranteed > future proof approach, it may be worth paying the cost of a single thread. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15901) Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15901:
-
Reviewers: Mick Semb Wever, Robert Stupp, Robert Stupp (was: Mick Semb
Wever, Robert Stupp)
Mick Semb Wever, Robert Stupp, Robert Stupp (was: Mick Semb
Wever, Robert Stupp)
Status: Review In Progress (was: Patch Available)
> Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15921) 4.0 quality testing: Materialized View
[ https://issues.apache.org/jira/browse/CASSANDRA-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhaoYang updated CASSANDRA-15921: - Fix Version/s: 4.x > 4.0 quality testing: Materialized View > -- > > Key: CASSANDRA-15921 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15921 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Materialized Views >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Normal > Fix For: 4.x > > > The main purpose of this ticket to get a better understanding about 4.0 MV > status as a guideline for future improvements. I don't think it should block > 4.0 release since it's already marked as experimental. > Main areas to test: > * Write perf: We expect to see [10% write throughput drop per MV > added|https://www.datastax.com/blog/2016/05/materialized-view-performance-cassandra-3x]. > * Read perf: identical to normal table > * Bootstrap/Decommision: no write-path required since CASSANDRA-13065 > * Repair: write path required > * Chaos monkey: take down coordinator/base-replica/view-replica during > read/write/token-movement and verify data consistency (may need a tool) > * Hint Replay: able to throttle if table has MV - CASSANDRA-13810 > * Schema race: create/drop - CASSANDRA-15845/CASSANDRA-15918 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15921) 4.0 quality testing: Materialized View
ZhaoYang created CASSANDRA-15921: Summary: 4.0 quality testing: Materialized View Key: CASSANDRA-15921 URL: https://issues.apache.org/jira/browse/CASSANDRA-15921 Project: Cassandra Issue Type: Improvement Components: Feature/Materialized Views Reporter: ZhaoYang Assignee: ZhaoYang The main purpose of this ticket to get a better understanding about 4.0 MV status as a guideline for future improvements. I don't think it should block 4.0 release since it's already marked as experimental. Main areas to test: * Write perf: We expect to see [10% write throughput drop per MV added|https://www.datastax.com/blog/2016/05/materialized-view-performance-cassandra-3x]. * Read perf: identical to normal table * Bootstrap/Decommision: no write-path required since CASSANDRA-13065 * Repair: write path required * Chaos monkey: take down coordinator/base-replica/view-replica during read/write/token-movement and verify data consistency (may need a tool) * Hint Replay: able to throttle if table has MV - CASSANDRA-13810 * Schema race: create/drop - CASSANDRA-15845/CASSANDRA-15918 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17150909#comment-17150909
]
Benedict Elliott Smith commented on CASSANDRA-15234:
If we agree we will be changing the config parameters next version to this
alternative approach, then I am fairly opposed to shipping this in its current
form and creating more churn for operators when we change it again in the near
future. The point of this exercise is to clean things up, not make them
messier, and this includes the perspective of the end user and their ability to
keep up with our nomenclature.
I also think it would be a real shame not to include this work at all, given
how awful and confusing our parameter naming situation is today.
I don't think this is an absolute necessity for pre-beta, however, given that
it is backwards compatible. I don't see any issue with landing this in a later
beta.
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17150909#comment-17150909
]
Benedict Elliott Smith edited comment on CASSANDRA-15234 at 7/3/20, 10:00 AM:
--
If we agree we will be changing the config parameters next version to this
alternative approach, then I am fairly opposed to shipping this in its current
form and creating more churn for operators when we change it again in the near
future. The point of this exercise is to clean things up, not make them
messier, and this includes the perspective of the end user and their ability to
keep up with our nomenclature.
I also think it would be a real shame not to include this work at all, given
how awful and confusing our parameter naming situation is today.
I don't think this is an absolute necessity for pre-beta, however, given that
we have agreed for it to be backwards compatible. I don't see any issue with
landing this in a later beta.
was (Author: benedict):
If we agree we will be changing the config parameters next version to this
alternative approach, then I am fairly opposed to shipping this in its current
form and creating more churn for operators when we change it again in the near
future. The point of this exercise is to clean things up, not make them
messier, and this includes the perspective of the end user and their ability to
keep up with our nomenclature.
I also think it would be a real shame not to include this work at all, given
how awful and confusing our parameter naming situation is today.
I don't think this is an absolute necessity for pre-beta, however, given that
it is backwards compatible. I don't see any issue with landing this in a later
beta.
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch master updated: Send devbranch notifications to the #cassandra-builds-patches slack channel
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/master by this push:
new e094338 Send devbranch notifications to the #cassandra-builds-patches
slack channel
e094338 is described below
commit e09433881ed8f9f5c0f940b7795de61b679c82a6
Author: Mick Semb Wever
AuthorDate: Fri Jul 3 11:03:55 2020 +0200
Send devbranch notifications to the #cassandra-builds-patches slack channel
---
jenkins-dsl/cassandra_job_dsl_seed.groovy | 1 -
jenkins-dsl/cassandra_pipeline.groovy | 20 +---
2 files changed, 5 insertions(+), 16 deletions(-)
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index 9c99d2d..9536e77 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -790,7 +790,6 @@ pipelineJob('Cassandra-devbranch') {
cps {
// Cassandra-devbranch still needs custom Jenkinsfile because of
the parameters passed into the build jobs.
script(readFileFromWorkspace('Cassandra-Job-DSL',
'jenkins-dsl/cassandra_pipeline.groovy'))
-sandbox()
}
}
}
\ No newline at end of file
diff --git a/jenkins-dsl/cassandra_pipeline.groovy
b/jenkins-dsl/cassandra_pipeline.groovy
index e902eaa..903c0c9 100644
--- a/jenkins-dsl/cassandra_pipeline.groovy
+++ b/jenkins-dsl/cassandra_pipeline.groovy
@@ -239,10 +239,12 @@ pipeline {
sh "./cassandra-builds/build-scripts/cassandra-test-report.sh"
junit '**/build/test/**/TEST*.xml,**/cqlshlib.xml,**/nosetests.xml'
script {
- // though this is not used in the devbranch pipeline, it exists
here for testing the in-tree Jenkinsfile
- changes = formatChanges(currentBuild.changeSets)
- echo "changes: ${changes}"
+ // env.GIT_COMMIT or changeLogSets is not defined by
parameterised manual builds
+ commit_head_sha = sh(returnStdout: true, script:"(git -C
cassandra log -1 --no-merges --pretty=format:'%h')").trim()
+ commit_head_msg = sh(returnStdout: true, script:"(git -C
cassandra log -1 --no-merges --pretty=format:'%an %ad %s')").trim()
+ echo "sha: ${commit_head_sha}; msg: ${commit_head_msg}"
}
+slackSend channel: '#cassandra-builds-patches', message: ":apache:
<${env.BUILD_URL}|${currentBuild.fullDisplayName}> completed:
${currentBuild.result}.
\n${commit_head_msg}"
}
post {
always {
@@ -261,15 +263,3 @@ def copyTestResults(target) {
selector: [$class: 'StatusBuildSelector', stable: false],
target: target]);
}
-
-def formatChanges(changeLogSets) {
-def result = ''
-for (int i = 0; i < changeLogSets.size(); i++) {
-def entries = changeLogSets[i].items
-for (int j = 0; j < entries.length; j++) {
-def entry = entries[j]
-result = result + "${entry.commitId} by ${entry.author} on ${new
Date(entry.timestamp)}: ${entry.msg}\n"
-}
-}
-return result
-}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
