[jira] [Updated] (CASSANDRA-16000) Updates needed after 4.0-beta1 release
[ https://issues.apache.org/jira/browse/CASSANDRA-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16000: Description: We need to update for beta the Documentation/Installing Cassandra page (also the curl link provided) /CC [~lor...@datastax.com] was: We need to update for beta the Documentation/Installing Cassandra (also the curl link provided) /CC [~lor...@datastax.com] > Updates needed after 4.0-beta1 release > -- > > Key: CASSANDRA-16000 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16000 > Project: Cassandra > Issue Type: Bug > Components: Documentation/Website >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We need to update for beta the Documentation/Installing Cassandra page (also > the curl link provided) > > /CC [~lor...@datastax.com] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16000) Updates needed after 4.0-beta1 release
[ https://issues.apache.org/jira/browse/CASSANDRA-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16000: Impacts: Docs (was: None) Description: We need to update for beta the Documentation/Installing Cassandra (also the curl link provided) /CC [~lor...@datastax.com] was:We need to update for beta the Documentation/Installing Cassandra (also the curl link provided) > Updates needed after 4.0-beta1 release > -- > > Key: CASSANDRA-16000 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16000 > Project: Cassandra > Issue Type: Bug > Components: Documentation/Website >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We need to update for beta the Documentation/Installing Cassandra (also the > curl link provided) > > /CC [~lor...@datastax.com] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15925) Jenkins pipeline can copy wrong test report artefacts from stage builds
[
https://issues.apache.org/jira/browse/CASSANDRA-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169149#comment-17169149
]
Michael Semb Wever commented on CASSANDRA-15925:
bq. So… the problem here is no longer about which artefacts are getting copied,
but how they get aggregated together (which is a custom process in the
`Summary` stage).
Correction. This is related to CASSANDRA-9463 and CASSANDRA-9528.
Investigating updating {{CassandraXMLJUnitResultFormatter}} so the
"cassandra.testtag" value is suffixed to the testsuite, so testsuite elements
are separated cleanly.
> Jenkins pipeline can copy wrong test report artefacts from stage builds
> ---
>
> Key: CASSANDRA-15925
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15925
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Spotted in
> https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/console
> Looks like copyArtifact will need to be specific to a build.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15475) Full Query Logging - New Feature
[ https://issues.apache.org/jira/browse/CASSANDRA-15475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lorina Poland updated CASSANDRA-15475: -- Description: Added a page on Full Query logging, a new feature. [https://github.com/apache/cassandra/pull/404] Related ticket: https://issues.apache.org/jira/browse/CASSANDRA-15971 was: Added a page on Full Query logging, a new feature. https://github.com/apache/cassandra/pull/404 > Full Query Logging - New Feature > > > Key: CASSANDRA-15475 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15475 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > Fix For: 4.0, 4.0-alpha4 > > > Added a page on Full Query logging, a new feature. > [https://github.com/apache/cassandra/pull/404] > > Related ticket: https://issues.apache.org/jira/browse/CASSANDRA-15971 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15971) full query log needs improvement
[
https://issues.apache.org/jira/browse/CASSANDRA-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lorina Poland updated CASSANDRA-15971:
--
Labels: pull-request-available (was: )
> full query log needs improvement
>
>
> Key: CASSANDRA-15971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15971
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation/Website, Tool/fql
>Reporter: Jonathan Shook
>Assignee: Lorina Poland
>Priority: Normal
> Labels: pull-request-available
> Attachments: st1.txt
>
>
> When trying out full query logging as a possible integration for nosqlbench
> usage, I ran across many issues which would make it painful for users. Since
> there were several, they will be added to a single issue for now. This issue
> can be broken up if needed.
>
> FQL doesn't work on my system, even though it says it is logging queries.
> With the following configuration in cassandra.yaml:
>
> {noformat}
> full_query_logging_options:
> log_dir: /REDACTED/fullquerylogs
> roll_cycle: HOURLY
> block: true
> max_queue_weight: 268435456 # 256 MiB
> max_log_size: 17179869184 # 16 GiB
> ## archive command is "/path/to/script.sh %path" where %path is replaced
> with the file being rolled:
> # archive_command:
> # max_archive_retries: 10
> {noformat}
> which appears to be the minimal configuration needed to enable fql, only a
> single file `directory-listing.cq4t` is created, which is a 64K sized file of
> zeroes.
>
>
> Calling bin/nodetool enablefullquerylog throws an error initially.
> [jshook@cass4 bin]$ ./nodetool enablefullquerylog
>
> {noformat}
> error: sun.nio.ch.FileChannelImpl.map0(int,long,long)
> -- StackTrace --
> java.lang.NoSuchMethodException:
> sun.nio.ch.FileChannelImpl.map0(int,long,long)
> at java.base/java.lang.Class.getDeclaredMethod(Class.java:2553)
> at net.openhft.chronicle.core.OS.lambda$static$0(OS.java:51)
> at
> net.openhft.chronicle.core.ClassLocal.computeValue(ClassLocal.java:53){noformat}
> (full stack trace attached to this ticket)
>
> Subsequent calls produce normal output:
>
> {noformat}
> [jshook@cass4 c4b1]$ bin/nodetool enablefullquerylog
> nodetool: Already logging to /home/jshook/c4b1/data/fullquerylogs
> See 'nodetool help' or 'nodetool help '.{noformat}
>
>
> nodetool missing getfullquerylog makes it difficult to verify current
> fullquerylog state without changing it. The conventions for nodetool commands
> should be followed to avoid confusing users.
>
> (maybe)
> {noformat}
> tools/bin/fqltool help{noformat}
> should print out help for all fqltool commands rather than simply repeating
> the default The most commonly used fqltool commands are..
>
> [https://cassandra.apache.org/doc/latest/new/fqllogging.html] is
> malformatted, mixing the appearance of configuration with comments, which is
> confusing at best.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16003) ToolRunner added in CASSANDRA-15942 stdErr checks improvements
[
https://issues.apache.org/jira/browse/CASSANDRA-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-16003:
--
Reviewers: David Capwell, David Capwell (was: David Capwell)
David Capwell, David Capwell
Status: Review In Progress (was: Patch Available)
left comment in PR
> ToolRunner added in CASSANDRA-15942 stdErr checks improvements
> --
>
> Key: CASSANDRA-16003
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16003
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The JVM will output to stderr on some environments to show what flags that
> were picked up, when this happens all tests which validate stderr start to
> fail. This was found in the org.apache.cassandra.tools.ClearSnapshotTest as
> it switched to use the ToolRunner; below is a sample failure on my laptop (I
> had to modify the asserts since they don’t include the input)
> {code}
> java.lang.AssertionError:
> Expecting empty but was:<"Picked up _JAVA_OPTIONS:
> -Djava.net.preferIPv4Stack=true
> ">
> at
> org.apache.cassandra.tools.ToolRunner.assertEmptyStdErr(ToolRunner.java:339)
> at
> org.apache.cassandra.tools.ToolRunner.waitAndAssertOnCleanExit(ToolRunner.java:334)
> at
> org.apache.cassandra.tools.ClearSnapshotTest.testClearSnapshot_RemoveMultiple(ClearSnapshotTest.java:91)
> {code}
> Here _JAVA_OPTIONS is used globally on my system so fails the test; there is
> also JAVA_TOOL_RUNNER which is used the same way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15158) Wait for schema agreement rather than in flight schema requests when bootstrapping
[ https://issues.apache.org/jira/browse/CASSANDRA-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169101#comment-17169101 ] Stefan Miklosovic commented on CASSANDRA-15158: --- hi [~bdeggleston], have you had a chance to reflect the above issues? > Wait for schema agreement rather than in flight schema requests when > bootstrapping > -- > > Key: CASSANDRA-15158 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15158 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip, Cluster/Schema >Reporter: Vincent White >Assignee: Ben Bromhead >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > Currently when a node is bootstrapping we use a set of latches > (org.apache.cassandra.service.MigrationTask#inflightTasks) to keep track of > in-flight schema pull requests, and we don't proceed with > bootstrapping/stream until all the latches are released (or we timeout > waiting for each one). One issue with this is that if we have a large schema, > or the retrieval of the schema from the other nodes was unexpectedly slow > then we have no explicit check in place to ensure we have actually received a > schema before we proceed. > While it's possible to increase "migration_task_wait_in_seconds" to force the > node to wait on each latche longer, there are cases where this doesn't help > because the callbacks for the schema pull requests have expired off the > messaging service's callback map > (org.apache.cassandra.net.MessagingService#callbacks) after > request_timeout_in_ms (default 10 seconds) before the other nodes were able > to respond to the new node. > This patch checks for schema agreement between the bootstrapping node and the > rest of the live nodes before proceeding with bootstrapping. It also adds a > check to prevent the new node from flooding existing nodes with simultaneous > schema pull requests as can happen in large clusters. > Removing the latch system should also prevent new nodes in large clusters > getting stuck for extended amounts of time as they wait > `migration_task_wait_in_seconds` on each of the latches left orphaned by the > timed out callbacks. > > ||3.11|| > |[PoC|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:check_for_schema]| > |[dtest|https://github.com/apache/cassandra-dtest/compare/master...vincewhite:wait_for_schema_agreement]| > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Issue Comment Deleted] (CASSANDRA-15959) org.apache.cassandra.cql3.validation.operations.TTLTest testCapWarnExpirationOverflowPolicy
[
https://issues.apache.org/jira/browse/CASSANDRA-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15959:
-
Comment: was deleted
(was: I tried that, however:
{quote}[junit-timeout] Testcase:
channelRead_Normal(org.apache.cassandra.streaming.async.StreamingInboundHandlerTest):
FAILED
[junit-timeout] expected:<8> but was:<0>
[junit-timeout] junit.framework.AssertionFailedError: expected:<8> but was:<0>
[junit-timeout] at
org.apache.cassandra.streaming.async.StreamingInboundHandlerTest.channelRead_Normal(StreamingInboundHandlerTest.java:98)
{quote}
it failed. :()
> org.apache.cassandra.cql3.validation.operations.TTLTest
> testCapWarnExpirationOverflowPolicy
> ---
>
> Key: CASSANDRA-15959
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15959
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: 0001-computeMaxTTL-directly-before-fetching-TTL.patch
>
>
> Build:
> https://ci-cassandra.apache.org/job/Cassandra-trunk-test/194/testReport/junit/org.apache.cassandra.cql3.validation.operations/TTLTest/testCapWarnExpirationOverflowPolicy/
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.cql3.validation.operations.TTLTest.checkTTLIsCapped(TTLTest.java:321)
> at
> org.apache.cassandra.cql3.validation.operations.TTLTest.verifyData(TTLTest.java:303)
> at
> org.apache.cassandra.cql3.validation.operations.TTLTest.testCapExpirationDateOverflowPolicy(TTLTest.java:248)
> at
> org.apache.cassandra.cql3.validation.operations.TTLTest.testCapExpirationDateOverflowPolicy(TTLTest.java:199)
> at
> org.apache.cassandra.cql3.validation.operations.TTLTest.testCapWarnExpirationOverflowPolicy(TTLTest.java:140)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15959) org.apache.cassandra.cql3.validation.operations.TTLTest testCapWarnExpirationOverflowPolicy
[
https://issues.apache.org/jira/browse/CASSANDRA-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169084#comment-17169084
]
Brandon Williams commented on CASSANDRA-15959:
--
I tried that, however:
{quote}[junit-timeout] Testcase:
channelRead_Normal(org.apache.cassandra.streaming.async.StreamingInboundHandlerTest):
FAILED
[junit-timeout] expected:<8> but was:<0>
[junit-timeout] junit.framework.AssertionFailedError: expected:<8> but was:<0>
[junit-timeout] at
org.apache.cassandra.streaming.async.StreamingInboundHandlerTest.channelRead_Normal(StreamingInboundHandlerTest.java:98)
{quote}
it failed. :(
> org.apache.cassandra.cql3.validation.operations.TTLTest
> testCapWarnExpirationOverflowPolicy
> ---
>
> Key: CASSANDRA-15959
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15959
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: 0001-computeMaxTTL-directly-before-fetching-TTL.patch
>
>
> Build:
> https://ci-cassandra.apache.org/job/Cassandra-trunk-test/194/testReport/junit/org.apache.cassandra.cql3.validation.operations/TTLTest/testCapWarnExpirationOverflowPolicy/
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.cql3.validation.operations.TTLTest.checkTTLIsCapped(TTLTest.java:321)
> at
> org.apache.cassandra.cql3.validation.operations.TTLTest.verifyData(TTLTest.java:303)
> at
> org.apache.cassandra.cql3.validation.operations.TTLTest.testCapExpirationDateOverflowPolicy(TTLTest.java:248)
> at
> org.apache.cassandra.cql3.validation.operations.TTLTest.testCapExpirationDateOverflowPolicy(TTLTest.java:199)
> at
> org.apache.cassandra.cql3.validation.operations.TTLTest.testCapWarnExpirationOverflowPolicy(TTLTest.java:140)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15971) full query log needs improvement
[
https://issues.apache.org/jira/browse/CASSANDRA-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15971:
Reviewers: Ekaterina Dimitrova, Ekaterina Dimitrova (was: Ekaterina
Dimitrova)
Ekaterina Dimitrova, Ekaterina Dimitrova (was: Ekaterina
Dimitrova)
Status: Review In Progress (was: Patch Available)
> full query log needs improvement
>
>
> Key: CASSANDRA-15971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15971
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation/Website, Tool/fql
>Reporter: Jonathan Shook
>Assignee: Lorina Poland
>Priority: Normal
> Attachments: st1.txt
>
>
> When trying out full query logging as a possible integration for nosqlbench
> usage, I ran across many issues which would make it painful for users. Since
> there were several, they will be added to a single issue for now. This issue
> can be broken up if needed.
>
> FQL doesn't work on my system, even though it says it is logging queries.
> With the following configuration in cassandra.yaml:
>
> {noformat}
> full_query_logging_options:
> log_dir: /REDACTED/fullquerylogs
> roll_cycle: HOURLY
> block: true
> max_queue_weight: 268435456 # 256 MiB
> max_log_size: 17179869184 # 16 GiB
> ## archive command is "/path/to/script.sh %path" where %path is replaced
> with the file being rolled:
> # archive_command:
> # max_archive_retries: 10
> {noformat}
> which appears to be the minimal configuration needed to enable fql, only a
> single file `directory-listing.cq4t` is created, which is a 64K sized file of
> zeroes.
>
>
> Calling bin/nodetool enablefullquerylog throws an error initially.
> [jshook@cass4 bin]$ ./nodetool enablefullquerylog
>
> {noformat}
> error: sun.nio.ch.FileChannelImpl.map0(int,long,long)
> -- StackTrace --
> java.lang.NoSuchMethodException:
> sun.nio.ch.FileChannelImpl.map0(int,long,long)
> at java.base/java.lang.Class.getDeclaredMethod(Class.java:2553)
> at net.openhft.chronicle.core.OS.lambda$static$0(OS.java:51)
> at
> net.openhft.chronicle.core.ClassLocal.computeValue(ClassLocal.java:53){noformat}
> (full stack trace attached to this ticket)
>
> Subsequent calls produce normal output:
>
> {noformat}
> [jshook@cass4 c4b1]$ bin/nodetool enablefullquerylog
> nodetool: Already logging to /home/jshook/c4b1/data/fullquerylogs
> See 'nodetool help' or 'nodetool help '.{noformat}
>
>
> nodetool missing getfullquerylog makes it difficult to verify current
> fullquerylog state without changing it. The conventions for nodetool commands
> should be followed to avoid confusing users.
>
> (maybe)
> {noformat}
> tools/bin/fqltool help{noformat}
> should print out help for all fqltool commands rather than simply repeating
> the default The most commonly used fqltool commands are..
>
> [https://cassandra.apache.org/doc/latest/new/fqllogging.html] is
> malformatted, mixing the appearance of configuration with comments, which is
> confusing at best.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15971) full query log needs improvement
[
https://issues.apache.org/jira/browse/CASSANDRA-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15971:
Test and Documentation Plan: [https://github.com/apache/cassandra/pull/705]
Status: Patch Available (was: In Progress)
> full query log needs improvement
>
>
> Key: CASSANDRA-15971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15971
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation/Website, Tool/fql
>Reporter: Jonathan Shook
>Assignee: Lorina Poland
>Priority: Normal
> Attachments: st1.txt
>
>
> When trying out full query logging as a possible integration for nosqlbench
> usage, I ran across many issues which would make it painful for users. Since
> there were several, they will be added to a single issue for now. This issue
> can be broken up if needed.
>
> FQL doesn't work on my system, even though it says it is logging queries.
> With the following configuration in cassandra.yaml:
>
> {noformat}
> full_query_logging_options:
> log_dir: /REDACTED/fullquerylogs
> roll_cycle: HOURLY
> block: true
> max_queue_weight: 268435456 # 256 MiB
> max_log_size: 17179869184 # 16 GiB
> ## archive command is "/path/to/script.sh %path" where %path is replaced
> with the file being rolled:
> # archive_command:
> # max_archive_retries: 10
> {noformat}
> which appears to be the minimal configuration needed to enable fql, only a
> single file `directory-listing.cq4t` is created, which is a 64K sized file of
> zeroes.
>
>
> Calling bin/nodetool enablefullquerylog throws an error initially.
> [jshook@cass4 bin]$ ./nodetool enablefullquerylog
>
> {noformat}
> error: sun.nio.ch.FileChannelImpl.map0(int,long,long)
> -- StackTrace --
> java.lang.NoSuchMethodException:
> sun.nio.ch.FileChannelImpl.map0(int,long,long)
> at java.base/java.lang.Class.getDeclaredMethod(Class.java:2553)
> at net.openhft.chronicle.core.OS.lambda$static$0(OS.java:51)
> at
> net.openhft.chronicle.core.ClassLocal.computeValue(ClassLocal.java:53){noformat}
> (full stack trace attached to this ticket)
>
> Subsequent calls produce normal output:
>
> {noformat}
> [jshook@cass4 c4b1]$ bin/nodetool enablefullquerylog
> nodetool: Already logging to /home/jshook/c4b1/data/fullquerylogs
> See 'nodetool help' or 'nodetool help '.{noformat}
>
>
> nodetool missing getfullquerylog makes it difficult to verify current
> fullquerylog state without changing it. The conventions for nodetool commands
> should be followed to avoid confusing users.
>
> (maybe)
> {noformat}
> tools/bin/fqltool help{noformat}
> should print out help for all fqltool commands rather than simply repeating
> the default The most commonly used fqltool commands are..
>
> [https://cassandra.apache.org/doc/latest/new/fqllogging.html] is
> malformatted, mixing the appearance of configuration with comments, which is
> confusing at best.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15971) full query log needs improvement
[
https://issues.apache.org/jira/browse/CASSANDRA-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15971:
Reviewers: Ekaterina Dimitrova
> full query log needs improvement
>
>
> Key: CASSANDRA-15971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15971
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation/Website, Tool/fql
>Reporter: Jonathan Shook
>Assignee: Lorina Poland
>Priority: Normal
> Attachments: st1.txt
>
>
> When trying out full query logging as a possible integration for nosqlbench
> usage, I ran across many issues which would make it painful for users. Since
> there were several, they will be added to a single issue for now. This issue
> can be broken up if needed.
>
> FQL doesn't work on my system, even though it says it is logging queries.
> With the following configuration in cassandra.yaml:
>
> {noformat}
> full_query_logging_options:
> log_dir: /REDACTED/fullquerylogs
> roll_cycle: HOURLY
> block: true
> max_queue_weight: 268435456 # 256 MiB
> max_log_size: 17179869184 # 16 GiB
> ## archive command is "/path/to/script.sh %path" where %path is replaced
> with the file being rolled:
> # archive_command:
> # max_archive_retries: 10
> {noformat}
> which appears to be the minimal configuration needed to enable fql, only a
> single file `directory-listing.cq4t` is created, which is a 64K sized file of
> zeroes.
>
>
> Calling bin/nodetool enablefullquerylog throws an error initially.
> [jshook@cass4 bin]$ ./nodetool enablefullquerylog
>
> {noformat}
> error: sun.nio.ch.FileChannelImpl.map0(int,long,long)
> -- StackTrace --
> java.lang.NoSuchMethodException:
> sun.nio.ch.FileChannelImpl.map0(int,long,long)
> at java.base/java.lang.Class.getDeclaredMethod(Class.java:2553)
> at net.openhft.chronicle.core.OS.lambda$static$0(OS.java:51)
> at
> net.openhft.chronicle.core.ClassLocal.computeValue(ClassLocal.java:53){noformat}
> (full stack trace attached to this ticket)
>
> Subsequent calls produce normal output:
>
> {noformat}
> [jshook@cass4 c4b1]$ bin/nodetool enablefullquerylog
> nodetool: Already logging to /home/jshook/c4b1/data/fullquerylogs
> See 'nodetool help' or 'nodetool help '.{noformat}
>
>
> nodetool missing getfullquerylog makes it difficult to verify current
> fullquerylog state without changing it. The conventions for nodetool commands
> should be followed to avoid confusing users.
>
> (maybe)
> {noformat}
> tools/bin/fqltool help{noformat}
> should print out help for all fqltool commands rather than simply repeating
> the default The most commonly used fqltool commands are..
>
> [https://cassandra.apache.org/doc/latest/new/fqllogging.html] is
> malformatted, mixing the appearance of configuration with comments, which is
> confusing at best.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15971) full query log needs improvement
[
https://issues.apache.org/jira/browse/CASSANDRA-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169051#comment-17169051
]
Ekaterina Dimitrova commented on CASSANDRA-15971:
-
Thanks [~polandll], the patch LGTM, +1. And thanks for the fast actions:
* restructure
* rewording
* improved clarity
[~dcapwell], will you have time to look at it too and possibly commit?
> full query log needs improvement
>
>
> Key: CASSANDRA-15971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15971
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation/Website, Tool/fql
>Reporter: Jonathan Shook
>Assignee: Lorina Poland
>Priority: Normal
> Attachments: st1.txt
>
>
> When trying out full query logging as a possible integration for nosqlbench
> usage, I ran across many issues which would make it painful for users. Since
> there were several, they will be added to a single issue for now. This issue
> can be broken up if needed.
>
> FQL doesn't work on my system, even though it says it is logging queries.
> With the following configuration in cassandra.yaml:
>
> {noformat}
> full_query_logging_options:
> log_dir: /REDACTED/fullquerylogs
> roll_cycle: HOURLY
> block: true
> max_queue_weight: 268435456 # 256 MiB
> max_log_size: 17179869184 # 16 GiB
> ## archive command is "/path/to/script.sh %path" where %path is replaced
> with the file being rolled:
> # archive_command:
> # max_archive_retries: 10
> {noformat}
> which appears to be the minimal configuration needed to enable fql, only a
> single file `directory-listing.cq4t` is created, which is a 64K sized file of
> zeroes.
>
>
> Calling bin/nodetool enablefullquerylog throws an error initially.
> [jshook@cass4 bin]$ ./nodetool enablefullquerylog
>
> {noformat}
> error: sun.nio.ch.FileChannelImpl.map0(int,long,long)
> -- StackTrace --
> java.lang.NoSuchMethodException:
> sun.nio.ch.FileChannelImpl.map0(int,long,long)
> at java.base/java.lang.Class.getDeclaredMethod(Class.java:2553)
> at net.openhft.chronicle.core.OS.lambda$static$0(OS.java:51)
> at
> net.openhft.chronicle.core.ClassLocal.computeValue(ClassLocal.java:53){noformat}
> (full stack trace attached to this ticket)
>
> Subsequent calls produce normal output:
>
> {noformat}
> [jshook@cass4 c4b1]$ bin/nodetool enablefullquerylog
> nodetool: Already logging to /home/jshook/c4b1/data/fullquerylogs
> See 'nodetool help' or 'nodetool help '.{noformat}
>
>
> nodetool missing getfullquerylog makes it difficult to verify current
> fullquerylog state without changing it. The conventions for nodetool commands
> should be followed to avoid confusing users.
>
> (maybe)
> {noformat}
> tools/bin/fqltool help{noformat}
> should print out help for all fqltool commands rather than simply repeating
> the default The most commonly used fqltool commands are..
>
> [https://cassandra.apache.org/doc/latest/new/fqllogging.html] is
> malformatted, mixing the appearance of configuration with comments, which is
> confusing at best.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15980) Improve log messages for socket connection/disconnection
[
https://issues.apache.org/jira/browse/CASSANDRA-15980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17168991#comment-17168991
]
Jon Meredith commented on CASSANDRA-15980:
--
While testing the fix-of-the-fix, I noticed that Listen message is misleading.
It only output disabled if the encryption options were null, however encryption
could still be disabled by configuration (I noticed when I started the server
up with enabled: false). For the ServerEncryptionOptions it's also possible to
be disabled depending on the internode_encryption but inbound doesn't care
about that).
encryption off.
{code}
INFO [main] 2020-07-31 10:06:20,234 InboundConnectionInitiator.java:129 -
Listening on address: (127.0.0.1:7000), nic: lo0, encryption: disabled
{code}
encryption enabled, NOT optional
{code}
INFO [main] 2020-07-31 10:12:26,417 InboundConnectionInitiator.java:129 -
Listening on address: (127.0.0.1:7001), nic: lo0, encryption: enabled
(factory=openssl)
{code}
encryption enabled, optional, enable legacy (both listens shown here)
{code}
INFO [main] 2020-07-31 10:14:49,407 InboundConnectionInitiator.java:129 -
Listening on address: (127.0.0.1:7001), nic: lo0, encryption: enabled
(factory=openssl)
INFO [main] 2020-07-31 10:14:49,495 InboundConnectionInitiator.java:129 -
Listening on address: (127.0.0.1:7000), nic: lo0, encryption: optional
(factory=openssl)
{code}
encryption enabled. optional, disable legacy
{code}
INFO [main] 2020-07-31 10:17:37,736 InboundConnectionInitiator.java:129 -
Listening on address: (127.0.0.1:7000), nic: lo0, encryption: optional
(factory=openssl)
{code}
with an optionally encrypted inbound connection logged as
{code}
INFO [Messaging-EventLoop-3-9] 2020-07-31 10:17:44,067
InboundConnectionInitiator.java:457 -
127.0.0.3:7000(127.0.0.1:60610)->127.0.0.1:7000-SMALL_MESSAGES-75d10ff2
messaging connection established, version = 12, framing = CRC, encryption =
optional (factory=openssl;protocol=TLSv1.2;cipher=TLS_RSA_WITH_AES_256_CBC_SHA)
{code}
and an plaintext inbound connection logged as
{code}
INFO [Messaging-EventLoop-3-1] 2020-07-31 10:26:00,047
InboundConnectionInitiator.java:457 -
127.0.0.3:7000(127.0.0.1:60754)->127.0.0.1:7000-URGENT_MESSAGES-f3c595f8
messaging connection established, version = 12, framing = CRC, encryption =
disabled
{code}
> Improve log messages for socket connection/disconnection
>
>
> Key: CASSANDRA-15980
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15980
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Logging for inbound SSL connections can take place before protocol
> negotiation has taken place and logs a misleading cipher that could cause
> problems for security auditing.
>
>
> {code:java}
> INFO 2020-07-03T13:57:58,380 [Messaging-EventLoop-3-1]
> org.apache.cassandra.net.InboundConnectionInitiator:242 - connection from
> peer /1.1.1.1:57899 to /2.2.2.2:7000, protocol = TLSv1.2, cipher suite =
> SSL_NULL_WITH_NULL_NULL
> {code}
>
> Instead Cassandra should log the connection & protocol, then once the cipher
> has been negotiated log the agreed upon cipher.
>
>
> If the inbound SSL connection does not present a client certificate,
> Cassandra logs this error, even if the client wasn't required to.
> {code:java}
> ERROR 2020-07-14T11:58:45,925 [Native-Transport-Requests-1]
> org.apache.cassandra.transport.ServerConnection:140 - Failed to get peer
> certificates for peer /4.3.2.1:59263
> {code}
>
> Logging the absense of verified certificates should be a concern of the
> SaslNegotiator if it requires it, and not something worth alerting the
> operator for generally. Downgrade to debug message to make investigation
> possible if needed.
>
>
> Finally, to help with logging issues related to disconnection, add a log
> statement when an instance decides it no longer needs to keep a gossip
> connection open when cleaning up connections in
> org.apache.cassandra.net.OutboundConnections.UnusedConnectionMonitor#closeUnusedSinceLastRun
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16006) Parallelise Jenkins dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-16006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16006: --- Change Category: Performance Complexity: Normal Fix Version/s: 4.0-rc Assignee: Michael Semb Wever Status: Open (was: Triage Needed) WIP [here|https://github.com/apache/cassandra-builds/compare/master...thelastpickle:mck/jenkins-dtest-parallel] > Parallelise Jenkins dtests > -- > > Key: CASSANDRA-16006 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16006 > Project: Cassandra > Issue Type: Improvement > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-rc > > > Currently dtests in Jenkins take ~10 hours. > Using the Jenkins Matrix plugin these jobs can be split into smaller lists of > dtests and run in parallel. This is the approach CircleCI takes. > This approach was > [trialed|https://github.com/apache/cassandra-builds/pull/29] with the > dtest-upgrade jobs (which are not yet part of the branch pipelines, and > haven't previously worked at all due to their duration). > In addition to the Matrix plugin, the Priority-Sorter and Matrix Reloaded > plugins also needed to be added. > The splits will occupy all executors, and multiple builds will lead to a long > build queue. More important builds (artifacts and unit tests) need a way to > be prioritised in such saturated situations. > Splits can fail for silly reasons (false-positive), like full /tmp disks, or > connectivity issues between the donated agent servers. The Matrix Reloaded > plugin makes it easy to rebuilt just those failed splits. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16006) Parallelise Jenkins dtests
Michael Semb Wever created CASSANDRA-16006: -- Summary: Parallelise Jenkins dtests Key: CASSANDRA-16006 URL: https://issues.apache.org/jira/browse/CASSANDRA-16006 Project: Cassandra Issue Type: Improvement Components: CI Reporter: Michael Semb Wever Currently dtests in Jenkins take ~10 hours. Using the Jenkins Matrix plugin these jobs can be split into smaller lists of dtests and run in parallel. This is the approach CircleCI takes. This approach was [trialed|https://github.com/apache/cassandra-builds/pull/29] with the dtest-upgrade jobs (which are not yet part of the branch pipelines, and haven't previously worked at all due to their duration). In addition to the Matrix plugin, the Priority-Sorter and Matrix Reloaded plugins also needed to be added. The splits will occupy all executors, and multiple builds will lead to a long build queue. More important builds (artifacts and unit tests) need a way to be prioritised in such saturated situations. Splits can fail for silly reasons (false-positive), like full /tmp disks, or connectivity issues between the donated agent servers. The Matrix Reloaded plugin makes it easy to rebuilt just those failed splits. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15979) CircleCI UnitTests error - java.lang.OutOfMemoryError: Java heap space
[
https://issues.apache.org/jira/browse/CASSANDRA-15979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17168902#comment-17168902
]
Michael Semb Wever edited comment on CASSANDRA-15979 at 7/31/20, 2:45 PM:
--
bq. Has anyone seen this outside of CircleCI? I've never encountered it
locally and I don't recall seeing it in Jenkins either.
[~brandon.williams], yes I've seen this and had to work around it in Jenkins,
when aggregating all the build stage test xml reports into one file and then
transforming it. The aggregated xml containing ~18k tests is 1.5GB+ and there's
no way any of those agents can Xalan xslt transform them. (Saxon was needed.)
To solve the problem there, I had to do the aggregation manually with ant, and
then the xslt transformation with saxon outside of a JVM.
-
https://github.com/apache/cassandra-builds/blob/master/build-scripts/cassandra-test-report.sh
-
https://github.com/apache/cassandra-builds/blob/master/docker/jenkins/generate_plaintext_test_report.docker#L23
This is also why the Test results in the Jenkins branch pipelines are built
upon all the individual test xml files, instead of this aggregated massive xml
file.
- https://github.com/apache/cassandra/blob/trunk/.jenkins/Jenkinsfile#L347
was (Author: michaelsembwever):
bq. Has anyone seen this outside of CircleCI? I've never encountered it
locally and I don't recall seeing it in Jenkins either.
[~brandon.williams], yes I've seen this and had to work around it in Jenkins,
when aggregating all the build stage test xml reports into one file and then
transforming it. The aggregated xml containing ~18k tests is 1.5GB+ and there's
no way any of those agents can Xalan xslt transform them. (Saxon was needed.)
To solve the problem there, I had to do the aggregation manually with ant, and
then the xslt transformation with saxon outside of a JVM.
-
https://github.com/apache/cassandra-builds/blob/master/build-scripts/cassandra-test-report.sh
-
https://github.com/apache/cassandra-builds/blob/master/docker/jenkins/generate_plaintext_test_report.docker#L23
> CircleCI UnitTests error - java.lang.OutOfMemoryError: Java heap space
> --
>
> Key: CASSANDRA-15979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15979
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
>
> We persistently see on the latest CircleCI trunk the following error:
> (MIDRES)
>
> {code:java}
> BUILD FAILED
> /tmp/cassandra/build.xml:1982: The following error occurred while executing
> this line:
> /tmp/cassandra/build.xml:1866: java.lang.OutOfMemoryError: Java heap space
> at java.lang.AbstractStringBuilder.(AbstractStringBuilder.java:68)
> at java.lang.StringBuffer.(StringBuffer.java:128)
> at
> com.sun.org.apache.xml.internal.utils.FastStringBuffer.getString(FastStringBuffer.java:872)
> at
> com.sun.org.apache.xml.internal.dtm.ref.sax2dtm.SAX2DTM2.getStringValueX(SAX2DTM2.java:2937)
> at
> com.sun.org.apache.xalan.internal.xsltc.dom.DOMAdapter.getStringValueX(DOMAdapter.java:284)
> at junit_frames.template$dot$5()
> at junit_frames.applyTemplates5()
> at junit_frames.package()
> at junit_frames.template$dot$0()
> at junit_frames.applyTemplates()
> at junit_frames.applyTemplates()
> at junit_frames.transform()
> at
> com.sun.org.apache.xalan.internal.xsltc.runtime.AbstractTranslet.transform(AbstractTranslet.java:620)
> at
> com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:730)
> at
> com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:343)
> at
> org.apache.tools.ant.taskdefs.optional.TraXLiaison.transform(TraXLiaison.java:201)
> at org.apache.tools.ant.taskdefs.XSLTProcess.process(XSLTProcess.java:870)
> at org.apache.tools.ant.taskdefs.XSLTProcess.execute(XSLTProcess.java:408)
> at
> org.apache.tools.ant.taskdefs.optional.junit.AggregateTransformer.transform(AggregateTransformer.java:281)
> at
> org.apache.tools.ant.taskdefs.optional.junit.XMLResultAggregator.execute(XMLResultAggregator.java:157)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99)
> at org.apache.tools.ant.Task.perform(Task.java:350)
> at
> org.apache.tools.ant.taskdefs.Sequential$$Lambda$149/1543351283.accept(Unknown
> Source)
> at java.util.Vector.forEach(Vector.java:1275)
[jira] [Comment Edited] (CASSANDRA-15979) CircleCI UnitTests error - java.lang.OutOfMemoryError: Java heap space
[
https://issues.apache.org/jira/browse/CASSANDRA-15979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17168902#comment-17168902
]
Michael Semb Wever edited comment on CASSANDRA-15979 at 7/31/20, 2:41 PM:
--
bq. Has anyone seen this outside of CircleCI? I've never encountered it
locally and I don't recall seeing it in Jenkins either.
[~brandon.williams], yes I've seen this and had to work around it in Jenkins,
when aggregating all the build stage test xml reports into one file and then
transforming it. The aggregated xml containing ~18k tests is 1.5GB+ and there's
no way any of those agents can Xalan xslt transform them. (Saxon was needed.)
To solve the problem there, I had to do the aggregation manually with ant, and
then the xslt transformation with saxon outside of a JVM.
-
https://github.com/apache/cassandra-builds/blob/master/build-scripts/cassandra-test-report.sh
-
https://github.com/apache/cassandra-builds/blob/master/docker/jenkins/generate_plaintext_test_report.docker#L23
was (Author: michaelsembwever):
bq. Has anyone seen this outside of CircleCI? I've never encountered it
locally and I don't recall seeing it in Jenkins either.
[~brandon.williams], yes I've seen this and had to work around it in Jenkins,
when aggregating all the build stage test xml reports into one file and then
transforming it. The aggregated xml containing ~18k tests is 1.5GB+ and there's
no way any of those agents can Xalan xslt transform them. (Saxon was needed.)
To solve the problem there, I had to do the aggregation and xslt transformation
outside of a JVM.
-
https://github.com/apache/cassandra-builds/blob/master/build-scripts/cassandra-test-report.sh
-
https://github.com/apache/cassandra-builds/blob/master/docker/jenkins/generate_plaintext_test_report.docker#L23
> CircleCI UnitTests error - java.lang.OutOfMemoryError: Java heap space
> --
>
> Key: CASSANDRA-15979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15979
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
>
> We persistently see on the latest CircleCI trunk the following error:
> (MIDRES)
>
> {code:java}
> BUILD FAILED
> /tmp/cassandra/build.xml:1982: The following error occurred while executing
> this line:
> /tmp/cassandra/build.xml:1866: java.lang.OutOfMemoryError: Java heap space
> at java.lang.AbstractStringBuilder.(AbstractStringBuilder.java:68)
> at java.lang.StringBuffer.(StringBuffer.java:128)
> at
> com.sun.org.apache.xml.internal.utils.FastStringBuffer.getString(FastStringBuffer.java:872)
> at
> com.sun.org.apache.xml.internal.dtm.ref.sax2dtm.SAX2DTM2.getStringValueX(SAX2DTM2.java:2937)
> at
> com.sun.org.apache.xalan.internal.xsltc.dom.DOMAdapter.getStringValueX(DOMAdapter.java:284)
> at junit_frames.template$dot$5()
> at junit_frames.applyTemplates5()
> at junit_frames.package()
> at junit_frames.template$dot$0()
> at junit_frames.applyTemplates()
> at junit_frames.applyTemplates()
> at junit_frames.transform()
> at
> com.sun.org.apache.xalan.internal.xsltc.runtime.AbstractTranslet.transform(AbstractTranslet.java:620)
> at
> com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:730)
> at
> com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:343)
> at
> org.apache.tools.ant.taskdefs.optional.TraXLiaison.transform(TraXLiaison.java:201)
> at org.apache.tools.ant.taskdefs.XSLTProcess.process(XSLTProcess.java:870)
> at org.apache.tools.ant.taskdefs.XSLTProcess.execute(XSLTProcess.java:408)
> at
> org.apache.tools.ant.taskdefs.optional.junit.AggregateTransformer.transform(AggregateTransformer.java:281)
> at
> org.apache.tools.ant.taskdefs.optional.junit.XMLResultAggregator.execute(XMLResultAggregator.java:157)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99)
> at org.apache.tools.ant.Task.perform(Task.java:350)
> at
> org.apache.tools.ant.taskdefs.Sequential$$Lambda$149/1543351283.accept(Unknown
> Source)
> at java.util.Vector.forEach(Vector.java:1275)
> at org.apache.tools.ant.taskdefs.Sequential.execute(Sequential.java:67)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(Delega
[jira] [Commented] (CASSANDRA-15979) CircleCI UnitTests error - java.lang.OutOfMemoryError: Java heap space
[
https://issues.apache.org/jira/browse/CASSANDRA-15979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17168902#comment-17168902
]
Michael Semb Wever commented on CASSANDRA-15979:
bq. Has anyone seen this outside of CircleCI? I've never encountered it
locally and I don't recall seeing it in Jenkins either.
[~brandon.williams], yes I've seen this and had to work around it in Jenkins,
when aggregating all the build stage test xml reports into one file and then
transforming it. The aggregated xml containing ~18k tests is 1.5GB+ and there's
no way any of those agents can Xalan xslt transform them. (Saxon was needed.)
To solve the problem there, I had to do the aggregation and xslt transformation
outside of a JVM.
-
https://github.com/apache/cassandra-builds/blob/master/build-scripts/cassandra-test-report.sh
-
https://github.com/apache/cassandra-builds/blob/master/docker/jenkins/generate_plaintext_test_report.docker#L23
> CircleCI UnitTests error - java.lang.OutOfMemoryError: Java heap space
> --
>
> Key: CASSANDRA-15979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15979
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
>
> We persistently see on the latest CircleCI trunk the following error:
> (MIDRES)
>
> {code:java}
> BUILD FAILED
> /tmp/cassandra/build.xml:1982: The following error occurred while executing
> this line:
> /tmp/cassandra/build.xml:1866: java.lang.OutOfMemoryError: Java heap space
> at java.lang.AbstractStringBuilder.(AbstractStringBuilder.java:68)
> at java.lang.StringBuffer.(StringBuffer.java:128)
> at
> com.sun.org.apache.xml.internal.utils.FastStringBuffer.getString(FastStringBuffer.java:872)
> at
> com.sun.org.apache.xml.internal.dtm.ref.sax2dtm.SAX2DTM2.getStringValueX(SAX2DTM2.java:2937)
> at
> com.sun.org.apache.xalan.internal.xsltc.dom.DOMAdapter.getStringValueX(DOMAdapter.java:284)
> at junit_frames.template$dot$5()
> at junit_frames.applyTemplates5()
> at junit_frames.package()
> at junit_frames.template$dot$0()
> at junit_frames.applyTemplates()
> at junit_frames.applyTemplates()
> at junit_frames.transform()
> at
> com.sun.org.apache.xalan.internal.xsltc.runtime.AbstractTranslet.transform(AbstractTranslet.java:620)
> at
> com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:730)
> at
> com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:343)
> at
> org.apache.tools.ant.taskdefs.optional.TraXLiaison.transform(TraXLiaison.java:201)
> at org.apache.tools.ant.taskdefs.XSLTProcess.process(XSLTProcess.java:870)
> at org.apache.tools.ant.taskdefs.XSLTProcess.execute(XSLTProcess.java:408)
> at
> org.apache.tools.ant.taskdefs.optional.junit.AggregateTransformer.transform(AggregateTransformer.java:281)
> at
> org.apache.tools.ant.taskdefs.optional.junit.XMLResultAggregator.execute(XMLResultAggregator.java:157)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99)
> at org.apache.tools.ant.Task.perform(Task.java:350)
> at
> org.apache.tools.ant.taskdefs.Sequential$$Lambda$149/1543351283.accept(Unknown
> Source)
> at java.util.Vector.forEach(Vector.java:1275)
> at org.apache.tools.ant.taskdefs.Sequential.execute(Sequential.java:67)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Total time: 2 minutes 52 seconds
> Exited with code exit status 1
> CircleCI received exit code 1
> {code}
>
> *Example:*
> [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/285/workflows/f2722016-2353-4c38-9fd2-8614f6609f55/jobs/1648/parallel-runs/16?filterBy=FAILED]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch master updated: In Jenkins, parameterise and shorten the docker prune filter time
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/master by this push:
new b59b6bf In Jenkins, parameterise and shorten the docker prune filter
time
b59b6bf is described below
commit b59b6bf696a1fc7e474c8fe7c29bd1f1a5d3fdaf
Author: mck
AuthorDate: Fri Jul 31 08:46:47 2020 +0200
In Jenkins, parameterise and shorten the docker prune filter time
---
jenkins-dsl/cassandra_job_dsl_seed.groovy | 78 +--
1 file changed, 42 insertions(+), 36 deletions(-)
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index 98f8ec8..619ddd5 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -62,6 +62,12 @@ if(binding.hasVariable("CASSANDRA_DOCKER_IMAGE")) {
dtestDockerImage = "${CASSANDRA_DOCKER_IMAGE}"
}
+// expected longest job runtime
+def maxJobHours = '18'
+if(binding.hasVariable("MAX_JOB_HOURS")) {
+maxJobHours = "${MAX_JOB_HOURS}"
+}
+
//
// Job Templates
@@ -136,14 +142,14 @@ matrixJob('Cassandra-template-artifacts') {
}
}
postBuildTask {
-task('.', '''
+task('.', """
echo "Cleaning project…"; git clean -xdff ;
-echo "Pruning docker…" ; docker system prune -f --filter
"until=48h" ;
-echo "Reporting disk usage…"; df -h ; du -hs ../* ;
+echo "Pruning docker…" ; docker system prune -f --filter
"until=${maxJobHours}h" ;
+echo "Reporting disk usage…"; df -h ; du -hs ../* ; du -hs
../../* ;
echo "Cleaning tmp…";
find . -type d -name tmp -delete 2>/dev/null ;
find /tmp -type f -atime +2 -user jenkins -and -not -exec
fuser -s {} ';' -and -delete 2>/dev/null
-''')
+""")
}
}
}
@@ -202,15 +208,15 @@ job('Cassandra-template-test') {
}
}
postBuildTask {
-task('.', '''
+task('.', """
echo "Finding job process orphans…"; if pgrep -af
"${JOB_BASE_NAME}"; then pkill -9 -f "${JOB_BASE_NAME}"; fi;
echo "Cleaning project…"; git clean -xdff ;
-echo "Pruning docker…" ; docker system prune -f --filter
"until=48h" ;
-echo "Reporting disk usage…"; df -h ; du -hs ../* ;
+echo "Pruning docker…" ; docker system prune -f --filter
"until=${maxJobHours}h" ;
+echo "Reporting disk usage…"; df -h ; du -hs ../* ; du -hs
../../* ;
echo "Cleaning tmp…";
find . -type d -name tmp -delete 2>/dev/null ;
find /tmp -type f -atime +2 -user jenkins -and -not -exec
fuser -s {} ';' -and -delete 2>/dev/null
-''')
+""")
}
}
}
@@ -266,14 +272,14 @@ job('Cassandra-template-dtest') {
}
}
postBuildTask {
-task('.', '''
+task('.', """
echo "Cleaning project…"; git clean -xdff ;
-echo "Pruning docker…" ; if pgrep -af jenkinscommand.sh; then
docker system prune -f --filter 'until=48h'; else docker system prune -f
--volumes ; fi;
-echo "Reporting disk usage…"; df -h ; du -hs ../* ;
+echo "Pruning docker…" ; if pgrep -af jenkinscommand.sh; then
docker system prune -f --filter 'until=${maxJobHours}h'; else docker system
prune -f --volumes ; fi;
+echo "Reporting disk usage…"; df -h ; du -hs ../* ; du -hs
../../* ;
echo "Cleaning tmp…";
find . -type d -name tmp -delete 2>/dev/null ;
find /tmp -type f -atime +2 -user jenkins -and -not -exec
fuser -s {} ';' -and -delete 2>/dev/null
-''')
+""")
}
}
}
@@ -325,14 +331,14 @@ matrixJob('Cassandra-template-dtest-matrix') {
}
}
postBuildTask {
-task('.', '''
+task('.', """
echo "Cleaning project…"; git clean -xdff ;
-echo "Pruning docker…" ; if pgrep -af jenkinscommand.sh; then
docker system prune -f --filter 'until=48h'; else docker system prune -f
--volumes ; fi;
-echo "Reporting disk usage…"; df -h ; du -hs ../* ;
+echo "Pruning docker…" ; if pgrep -af jenkinscommand.sh; then
docker system prune -f --filter 'until=${maxJobHours}h'; else docker system
prune -f --volumes ; fi;
+echo "Reporting disk usage…"; df -h ; du -hs ../* ; du -hs
../../* ;
echo "Cleaning tmp…";
find . -type d -name tmp -delete 2>/dev/null ;
find /tmp -type f -atime +2 -user jenk
[jira] [Comment Edited] (CASSANDRA-16003) ToolRunner added in CASSANDRA-15942 stdErr checks improvements
[
https://issues.apache.org/jira/browse/CASSANDRA-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17168491#comment-17168491
]
Berenguer Blasi edited comment on CASSANDRA-16003 at 7/31/20, 7:58 AM:
---
[~dcapwell] Thanks for raising this one, good catch!. I don't think the problem
lies in ToolRunner not being portable, but the actual stdErr we are doing now
and we didn't do previously. So this is just the natural consequence of us
improving in tooling testing imo :-)
I have added a super short PR, fortunately, which seems to work when I repro'ed
locally. Would you be so kind to review please? Thx in advance.
Edit: Latest code on CASSANDRA-15583 will be blocked on this as well, so the
sooner we can merge this one the better #collaborating
was (Author: bereng):
[~dcapwell] Thanks for raising this one, good catch!. I don't think the problem
lies in ToolRunner not being portable, but the actual stdErr we are doing now
and we didn't do previously. So this is just the natural consequence of us
improving in tooling testing imo :-)
I have added a super short PR, fortunately, which seems to work when I repro'ed
locally. Would you be so kind to review please? Thx in advance.
> ToolRunner added in CASSANDRA-15942 stdErr checks improvements
> --
>
> Key: CASSANDRA-16003
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16003
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The JVM will output to stderr on some environments to show what flags that
> were picked up, when this happens all tests which validate stderr start to
> fail. This was found in the org.apache.cassandra.tools.ClearSnapshotTest as
> it switched to use the ToolRunner; below is a sample failure on my laptop (I
> had to modify the asserts since they don’t include the input)
> {code}
> java.lang.AssertionError:
> Expecting empty but was:<"Picked up _JAVA_OPTIONS:
> -Djava.net.preferIPv4Stack=true
> ">
> at
> org.apache.cassandra.tools.ToolRunner.assertEmptyStdErr(ToolRunner.java:339)
> at
> org.apache.cassandra.tools.ToolRunner.waitAndAssertOnCleanExit(ToolRunner.java:334)
> at
> org.apache.cassandra.tools.ClearSnapshotTest.testClearSnapshot_RemoveMultiple(ClearSnapshotTest.java:91)
> {code}
> Here _JAVA_OPTIONS is used globally on my system so fails the test; there is
> also JAVA_TOOL_RUNNER which is used the same way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16003) ToolRunner added in CASSANDRA-15942 stdErr checks improvements
[
https://issues.apache.org/jira/browse/CASSANDRA-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-16003:
Test and Documentation Plan: See PR
Status: Patch Available (was: In Progress)
> ToolRunner added in CASSANDRA-15942 stdErr checks improvements
> --
>
> Key: CASSANDRA-16003
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16003
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The JVM will output to stderr on some environments to show what flags that
> were picked up, when this happens all tests which validate stderr start to
> fail. This was found in the org.apache.cassandra.tools.ClearSnapshotTest as
> it switched to use the ToolRunner; below is a sample failure on my laptop (I
> had to modify the asserts since they don’t include the input)
> {code}
> java.lang.AssertionError:
> Expecting empty but was:<"Picked up _JAVA_OPTIONS:
> -Djava.net.preferIPv4Stack=true
> ">
> at
> org.apache.cassandra.tools.ToolRunner.assertEmptyStdErr(ToolRunner.java:339)
> at
> org.apache.cassandra.tools.ToolRunner.waitAndAssertOnCleanExit(ToolRunner.java:334)
> at
> org.apache.cassandra.tools.ClearSnapshotTest.testClearSnapshot_RemoveMultiple(ClearSnapshotTest.java:91)
> {code}
> Here _JAVA_OPTIONS is used globally on my system so fails the test; there is
> also JAVA_TOOL_RUNNER which is used the same way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16003) ToolRunner added in CASSANDRA-15942 isn't portable
[
https://issues.apache.org/jira/browse/CASSANDRA-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17168491#comment-17168491
]
Berenguer Blasi commented on CASSANDRA-16003:
-
[~dcapwell] Thanks for raising this one, good catch!. I don't think the problem
lies in ToolRunner not being portable, but the actual stdErr we are doing now
and we didn't do previously. So this is just the natural consequence of us
improving in tooling testing imo :-)
I have added a super short PR, fortunately, which seems to work when I repro'ed
locally. Would you be so kind to review please? Thx in advance.
> ToolRunner added in CASSANDRA-15942 isn't portable
> --
>
> Key: CASSANDRA-16003
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16003
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The JVM will output to stderr on some environments to show what flags that
> were picked up, when this happens all tests which validate stderr start to
> fail. This was found in the org.apache.cassandra.tools.ClearSnapshotTest as
> it switched to use the ToolRunner; below is a sample failure on my laptop (I
> had to modify the asserts since they don’t include the input)
> {code}
> java.lang.AssertionError:
> Expecting empty but was:<"Picked up _JAVA_OPTIONS:
> -Djava.net.preferIPv4Stack=true
> ">
> at
> org.apache.cassandra.tools.ToolRunner.assertEmptyStdErr(ToolRunner.java:339)
> at
> org.apache.cassandra.tools.ToolRunner.waitAndAssertOnCleanExit(ToolRunner.java:334)
> at
> org.apache.cassandra.tools.ClearSnapshotTest.testClearSnapshot_RemoveMultiple(ClearSnapshotTest.java:91)
> {code}
> Here _JAVA_OPTIONS is used globally on my system so fails the test; there is
> also JAVA_TOOL_RUNNER which is used the same way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16003) ToolRunner added in CASSANDRA-15942 stdErr checks improvements
[
https://issues.apache.org/jira/browse/CASSANDRA-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-16003:
Summary: ToolRunner added in CASSANDRA-15942 stdErr checks improvements
(was: ToolRunner added in CASSANDRA-15942 isn't portable)
> ToolRunner added in CASSANDRA-15942 stdErr checks improvements
> --
>
> Key: CASSANDRA-16003
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16003
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The JVM will output to stderr on some environments to show what flags that
> were picked up, when this happens all tests which validate stderr start to
> fail. This was found in the org.apache.cassandra.tools.ClearSnapshotTest as
> it switched to use the ToolRunner; below is a sample failure on my laptop (I
> had to modify the asserts since they don’t include the input)
> {code}
> java.lang.AssertionError:
> Expecting empty but was:<"Picked up _JAVA_OPTIONS:
> -Djava.net.preferIPv4Stack=true
> ">
> at
> org.apache.cassandra.tools.ToolRunner.assertEmptyStdErr(ToolRunner.java:339)
> at
> org.apache.cassandra.tools.ToolRunner.waitAndAssertOnCleanExit(ToolRunner.java:334)
> at
> org.apache.cassandra.tools.ClearSnapshotTest.testClearSnapshot_RemoveMultiple(ClearSnapshotTest.java:91)
> {code}
> Here _JAVA_OPTIONS is used globally on my system so fails the test; there is
> also JAVA_TOOL_RUNNER which is used the same way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
