[jira] [Commented] (CASSANDRA-13935) Indexes and UDTs creation should have IF NOT EXISTS on its String representation
[
https://issues.apache.org/jira/browse/CASSANDRA-13935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191668#comment-17191668
]
Robert Stupp commented on CASSANDRA-13935:
--
Sorry, I'll probably not have time to review.
> Indexes and UDTs creation should have IF NOT EXISTS on its String
> representation
>
>
> Key: CASSANDRA-13935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13935
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index, Legacy/CQL
> Environment: Ubuntu 16.04.2 LTS
> java version "1.8.0_144"
> Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
>Reporter: Javier Canillas
>Assignee: Stefan Miklosovic
>Priority: Low
> Fix For: 4.0-beta
>
> Attachments: 13935-3.0.txt, 13935-3.11.txt, 13935-trunk.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I came across something that bothers me a lot. I'm using snapshots to backup
> data from my Cassandra cluster in case something really bad happens (like
> dropping a table or a keyspace).
> Exercising the recovery actions from those backups, I discover that the
> schema put on the file "schema.cql" as a result of the snapshot has the
> "CREATE IF NOT EXISTS" for the table, but not for the indexes.
> When restoring from snapshots, and relying on the execution of these schemas
> to build up the table structure, everything seems fine for tables without
> secondary indexes, but for the ones that make use of them, the execution of
> these statements fail miserably.
> Here I paste a generated schema.cql content for a table with indexes:
> CREATE TABLE IF NOT EXISTS keyspace1.table1 (
> id text PRIMARY KEY,
> content text,
> last_update_date date,
> last_update_date_time timestamp)
> WITH ID = f1045fc0-2f59-11e7-95ec-295c3c064920
> AND bloom_filter_fp_chance = 0.01
> AND dclocal_read_repair_chance = 0.1
> AND crc_check_chance = 1.0
> AND default_time_to_live = 864
> AND gc_grace_seconds = 864000
> AND min_index_interval = 128
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE'
> AND caching = { 'keys': 'NONE', 'rows_per_partition': 'NONE' }
> AND compaction = { 'max_threshold': '32', 'min_threshold': '4',
> 'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy' }
> AND compression = { 'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor' }
> AND cdc = false
> AND extensions = { };
> CREATE INDEX table1_last_update_date_idx ON keyspace1.table1
> (last_update_date);
> I think the last part should be:
> CREATE INDEX IF NOT EXISTS table1_last_update_date_idx ON keyspace1.table1
> (last_update_date);
> // edit by Stefan Miklosovic
> PR: https://github.com/apache/cassandra/pull/731
> I have added UDTs as part of this patch as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12372) Remove deprecated memtable_cleanup_threshold for 4.0
[
https://issues.apache.org/jira/browse/CASSANDRA-12372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189038#comment-17189038
]
Robert Stupp commented on CASSANDRA-12372:
--
Recently came across situations where it's counter-productive to force the
default of {{mct = (1 + (1 / flush_writers))}}. Therefore I'm also against
removing {{memtable_clearnup_threshold}} and remove the deprecation warning.
> Remove deprecated memtable_cleanup_threshold for 4.0
>
>
> Key: CASSANDRA-12372
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12372
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 4.x
>
>
> This is going to be deprecated in 3.10 since it doesn't make sense to specify
> a value. It only makes sense to calculate it based on memtable_flush_writers.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16051) sstablemetadata line 44: : command not found
[
https://issues.apache.org/jira/browse/CASSANDRA-16051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17178926#comment-17178926
]
Robert Stupp commented on CASSANDRA-16051:
--
The chance is correct. The code went into {{cassandra.in.sh}}.
> sstablemetadata line 44: : command not found
>
>
> Key: CASSANDRA-16051
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16051
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Arvinder Singh
>Priority: Normal
> Fix For: 4.0-beta2
>
>
> sstablemetadata shipped with
> [4.0-beta|https://issues.apache.org/jira/issues/?jql=project+%3D+CASSANDRA+AND+fixVersion+%3D+4.0-beta]
> is missing the code "#Use JAVA_HOME if set, otherwise look for java in PATH"
> and report error:
> sstablemetadata: line 44: : command not found
>
> The fix is to merge below code from 3.x:
> ^# Use JAVA_HOME if set, otherwise look for java in PATH^
> ^if [ -x "$JAVA_HOME/bin/java" ]; then^
> ^JAVA="$JAVA_HOME/bin/java"^
> ^else^
> ^JAVA="`which java`"^
> ^fi^
> ^if [ "x$JAVA" = "x" ]; then^
> ^echo "Java executable not found (hint: set JAVA_HOME)" >&2^
> ^exit 1^
> ^fi^
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15810) Default StringTableSize parameter causes GC slowdown
[
https://issues.apache.org/jira/browse/CASSANDRA-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176876#comment-17176876
]
Robert Stupp edited comment on CASSANDRA-15810 at 8/13/20, 9:36 AM:
Yea - I think, the biggest "consumer" is probably the list of tokens, which is
also quite unique per node. So no savings for that one.
Potential saving might happen for "release-version", "dc", "rack" and current
"schema-version" - not particularly long strings so I doubt that it's actually
worth the {{String.intern()}} as the total savings are rather low. The other
values of {{VersionedValue}} are unique per peer and put unnecessary pressure
on the string-intern-map.
Other usages of {{String.intern()}} in the whole production code (i.e.
including JDK and libraries) are (intentionally) not on a hot-path.
This change feels IMO safe for 4.0 and also safe to be backported to 3.11 + 3.0.
was (Author: snazy):
Yea - I think, the biggest "consumer" is probably the list of tokens, which is
also quite unique per node. So no savings for that one.
Potential saving might happen for "release-version", "dc", "rack" and current
"schema-version" - not particularly long strings so I doubt that it's actually
not worth the {{String.intern()}} as the total savings are rather low. The
other values of {{VersionedValue}} are unique per peer and put unnecessary
pressure on the string-intern-map.
Other usages of {{String.intern()}} in the whole production code (i.e.
including JDK and libraries) are (intentionally) not on a hot-path.
This change feels IMO safe for 4.0 and also safe to be backported to 3.11 + 3.0.
> Default StringTableSize parameter causes GC slowdown
>
>
> Key: CASSANDRA-15810
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15810
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Tom van der Woerdt
>Priority: Normal
> Labels: gc, performance
>
> While looking at tail latency on a Cassandra cluster, it came up that the
> default StringTableSize in Cassandra is set to a million:
> {code:java}
> # Larger interned string table, for gossip's benefit (CASSANDRA-6410)
> -XX:StringTableSize=103{code}
> This was done for CASSANDRA-6410 by [~jbellis] in '13, to optimize heap usage
> on a test case, running with 500 nodes and num_tokens=512.
> Until Java 13, this string table is implemented as native code, and has to be
> traversed entirely during the GC initial marking phase, which is a STW event.
> Some testing on my end shows that the pause time of a GC cycle can be reduced
> by approximately 10 milliseconds if we lower the string table size back to
> the Java 8 default of 60013 entries.
> Thus, I would recommend this patch (3.11 branch, similar patch for 4.0):
> {code:java}
> diff --git a/conf/jvm.options b/conf/jvm.options
> index 01bb1685b3..c184d18c5d 100644
> --- a/conf/jvm.options
> +++ b/conf/jvm.options
> @@ -107,9 +107,6 @@
> # Per-thread stack size.
> -Xss256k
> -# Larger interned string table, for gossip's benefit (CASSANDRA-6410)
> --XX:StringTableSize=103
> -
> # Make sure all memory is faulted and zeroed on startup.
> # This helps prevent soft faults in containers and makes
> # transparent hugepage allocation more effective.
> {code}
> It does need some testing on more extreme clusters than I have access to, but
> I ran some Cassandra nodes with {{-XX:+PrintStringTableStatistics}} which
> suggested that the Java default will suffice here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15810) Default StringTableSize parameter causes GC slowdown
[
https://issues.apache.org/jira/browse/CASSANDRA-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176876#comment-17176876
]
Robert Stupp commented on CASSANDRA-15810:
--
Yea - I think, the biggest "consumer" is probably the list of tokens, which is
also quite unique per node. So no savings for that one.
Potential saving might happen for "release-version", "dc", "rack" and current
"schema-version" - not particularly long strings so I doubt that it's actually
not worth the {{String.intern()}} as the total savings are rather low. The
other values of {{VersionedValue}} are unique per peer and put unnecessary
pressure on the string-intern-map.
Other usages of {{String.intern()}} in the whole production code (i.e.
including JDK and libraries) are (intentionally) not on a hot-path.
This change feels IMO safe for 4.0 and also safe to be backported to 3.11 + 3.0.
> Default StringTableSize parameter causes GC slowdown
>
>
> Key: CASSANDRA-15810
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15810
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Tom van der Woerdt
>Priority: Normal
> Labels: gc, performance
>
> While looking at tail latency on a Cassandra cluster, it came up that the
> default StringTableSize in Cassandra is set to a million:
> {code:java}
> # Larger interned string table, for gossip's benefit (CASSANDRA-6410)
> -XX:StringTableSize=103{code}
> This was done for CASSANDRA-6410 by [~jbellis] in '13, to optimize heap usage
> on a test case, running with 500 nodes and num_tokens=512.
> Until Java 13, this string table is implemented as native code, and has to be
> traversed entirely during the GC initial marking phase, which is a STW event.
> Some testing on my end shows that the pause time of a GC cycle can be reduced
> by approximately 10 milliseconds if we lower the string table size back to
> the Java 8 default of 60013 entries.
> Thus, I would recommend this patch (3.11 branch, similar patch for 4.0):
> {code:java}
> diff --git a/conf/jvm.options b/conf/jvm.options
> index 01bb1685b3..c184d18c5d 100644
> --- a/conf/jvm.options
> +++ b/conf/jvm.options
> @@ -107,9 +107,6 @@
> # Per-thread stack size.
> -Xss256k
> -# Larger interned string table, for gossip's benefit (CASSANDRA-6410)
> --XX:StringTableSize=103
> -
> # Make sure all memory is faulted and zeroed on startup.
> # This helps prevent soft faults in containers and makes
> # transparent hugepage allocation more effective.
> {code}
> It does need some testing on more extreme clusters than I have access to, but
> I ran some Cassandra nodes with {{-XX:+PrintStringTableStatistics}} which
> suggested that the Java default will suffice here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15810) Default StringTableSize parameter causes GC slowdown
[
https://issues.apache.org/jira/browse/CASSANDRA-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176835#comment-17176835
]
Robert Stupp commented on CASSANDRA-15810:
--
ISTM that it's better to remove the only usage of {{String.intern()}} in
{{org.apache.cassandra.gms.VersionedValue#VersionedValue(java.lang.String)}}
and the {{-XX:StringTableSize}} JVM argument entirely. [~benedict] WDYT?
There's [evidence|https://shipilev.net/jvm/anatomy-quarks/10-string-intern/]
that String-interning can cause GC and performance issues.
> Default StringTableSize parameter causes GC slowdown
>
>
> Key: CASSANDRA-15810
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15810
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Tom van der Woerdt
>Priority: Normal
> Labels: gc, performance
>
> While looking at tail latency on a Cassandra cluster, it came up that the
> default StringTableSize in Cassandra is set to a million:
> {code:java}
> # Larger interned string table, for gossip's benefit (CASSANDRA-6410)
> -XX:StringTableSize=103{code}
> This was done for CASSANDRA-6410 by [~jbellis] in '13, to optimize heap usage
> on a test case, running with 500 nodes and num_tokens=512.
> Until Java 13, this string table is implemented as native code, and has to be
> traversed entirely during the GC initial marking phase, which is a STW event.
> Some testing on my end shows that the pause time of a GC cycle can be reduced
> by approximately 10 milliseconds if we lower the string table size back to
> the Java 8 default of 60013 entries.
> Thus, I would recommend this patch (3.11 branch, similar patch for 4.0):
> {code:java}
> diff --git a/conf/jvm.options b/conf/jvm.options
> index 01bb1685b3..c184d18c5d 100644
> --- a/conf/jvm.options
> +++ b/conf/jvm.options
> @@ -107,9 +107,6 @@
> # Per-thread stack size.
> -Xss256k
> -# Larger interned string table, for gossip's benefit (CASSANDRA-6410)
> --XX:StringTableSize=103
> -
> # Make sure all memory is faulted and zeroed on startup.
> # This helps prevent soft faults in containers and makes
> # transparent hugepage allocation more effective.
> {code}
> It does need some testing on more extreme clusters than I have access to, but
> I ran some Cassandra nodes with {{-XX:+PrintStringTableStatistics}} which
> suggested that the Java default will suffice here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15393) Add byte array backed cells
[
https://issues.apache.org/jira/browse/CASSANDRA-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175291#comment-17175291
]
Robert Stupp commented on CASSANDRA-15393:
--
This one compiles without errors just by omitting the generic type information:
{code}
ValueAccessor acc = ByteArrayAccessor.instance;
CounterContext.headerLength(ByteBuffer.allocate(10), acc);
{code}
^ This is a verbose example e.g. of the method signatures
{{o.a.c.db.rows.RangeTombstoneMarker#openBound}} + {{closeBound}} /
{{o.a.c.db.Clusterable#clustering}} - and actual production code like the above
(e.g. in {{o.a.c.db.MutableDeletionInfo.Builder#add}}). It seems that the
existing production code base including the unchanged parts with all call sites
need a full re-review to validate that this change is safe.
Another source of errors is that positions & offsets are manually handled now.
For example
[here|https://github.com/apache/cassandra/pull/712/files#diff-a9c2759ec4c4663dd4a24d5e0f89358fR225]
or
[here|https://github.com/apache/cassandra/pull/712/files#diff-1e045cbe2fa513c1d2e7ac63c1b764ccR58].
Just think about merging a change from the 3.x line into trunk but not adding
the necessary position/offsets arithmetics.
As I said, there's definitive value in reducing heap pressure in critical hot
paths. But my concern that the approach is error prone due to the issues
mentioned here still stands.
It might have been less of an issue, if the whole code base would have proper
unit testing for all production code paths with all potential inputs or even
fuzzy testing, but that's unfortunately not the case.
The risks are:
* Mis-computing offsets+positions can lead to potentially unnoticed data
corruption, overflows, etc that initially go unnoticed
* Spurious new runtime errors in rare situations (barely tested ones) leading
to ClassCastExceptions
I'm sorry, if my objections sound harsh. But my point is that it's better to
fix the whole heap-pressure-nightmare with (de)serialization
(reads/writes/re-serializations/compaction/etc) in the next major release.
> Add byte array backed cells
> ---
>
> Key: CASSANDRA-15393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15393
> Project: Cassandra
> Issue Type: Sub-task
> Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We currently materialize all values as on heap byte buffers. Byte buffers
> have a fairly high overhead given how frequently they’re used, and on the
> compaction and local read path we don’t do anything that needs them. Use of
> byte buffer methods only happens on the coordinator. Using cells that are
> backed by byte arrays instead in these situations reduces compaction and read
> garbage up to 22% in many cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15393) Add byte array backed cells
[
https://issues.apache.org/jira/browse/CASSANDRA-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173009#comment-17173009
]
Robert Stupp commented on CASSANDRA-15393:
--
Thanks for tackling this! Heap pressure induced by {{ByteBuffer}} is definitely
a pita. I'm not sure though whether this patch is suitable to be included in
4.0 this late (beta stage) as it looks quite intrusive and is pretty big.
Haven't thoroughly reviewed it, but here are a few notes:
* The new *Accessor classes have constants for boolean (true/false)
representations, which are modifiable. IIRC there's been an issue in the past
when that (or a similar) constant was accidentally changed.
* The split of {{ByteBuffer}} into a pair of {{accessor + container}} is error
prone - e.g. accidentally passing a {{byte[]}} with the {{ByteBufferAccessor}}
or vice versa leads to "interesting" results.
The underlying issue IMO is not {{ByteBuffer}} itself but the vast amount of
those intermediate and often tiny BB instances. But introducing an API that's
error prone is dangerous.
It feels both more efficient and much safer to have an API & implementation
that actually _prevents_ (the vast majority of) these new objects. I.e. instead
of creating a new {{ByteBuffer}} (or "just" a new {{byte[]}} as in the PR) to
write some value (e.g. an {{int}}) to a target buffer, write that value
_directly_ into the target buffer. _How_ that's implemented is a different
topic, but I'd prefer to consider all the new developments in Java and make it
safe & transparent to use.
> Add byte array backed cells
> ---
>
> Key: CASSANDRA-15393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15393
> Project: Cassandra
> Issue Type: Sub-task
> Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We currently materialize all values as on heap byte buffers. Byte buffers
> have a fairly high overhead given how frequently they’re used, and on the
> compaction and local read path we don’t do anything that needs them. Use of
> byte buffer methods only happens on the coordinator. Using cells that are
> backed by byte arrays instead in these situations reduces compaction and read
> garbage up to 22% in many cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15857) Frozen RawTuple is not annotated with frozen in the toString method
[
https://issues.apache.org/jira/browse/CASSANDRA-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15857:
-
Fix Version/s: (was: 3.11.x)
(was: 4.0)
4.0-beta2
3.11.8
Since Version: 3.11.0
Source Control Link:
https://github.com/apache/cassandra/commit/d51c18f807c5c88ef44114341592214747487645
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Thanks for the patch!
Committed as
[d51c18f807c5c88ef44114341592214747487645|https://github.com/apache/cassandra/commit/d51c18f807c5c88ef44114341592214747487645]
to [cassandra-3.11|https://github.com/apache/cassandra/tree/cassandra-3.11],
[merged|https://github.com/apache/cassandra/commit/a76d288c32e9a062dfafc8e38a33b8aed064913e]
to [trunk|https://github.com/apache/cassandra/tree/trunk].
> Frozen RawTuple is not annotated with frozen in the toString method
> ---
>
> Key: CASSANDRA-15857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15857
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 3.11.8, 4.0-beta2
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> All raw types (e.g. RawCollection, RawUT) that supports freezing wraps the
> type name with 'frozen<>' in the toString method, except RawTuple.
> Therefore, the RawTuple::toString output misses the frozen wrapper.
> Tuple is always frozen. However since CASSANDRA-15035, it throws when the
> inner tuple is not explicitly wrapped with frozen within a collection.
> The method, CQL3Type.Raw::toString, is referenced at multiple places in the
> source. For example, referenced in CreateTypeStatement.Raw and involved in
> CQLSSTableWriter. Another example is that it is called to produce the
> SchemaChange at several AlterSchemaStatement implementations.
> A test can prove that missing the frozen wrapper causes exception when
> building CQLSSTableWriter for user types defined like below. Note that the
> inner tuple is wrapped with frozen in the initial CQL statement.
> {code:java}
> CREATE TYPE ks.fooType ( f list>> )
> {code}
> {code:java}
> org.apache.cassandra.exceptions.InvalidRequestException: Non-frozen tuples
> are not allowed inside collections: list>
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.throwNestedNonFrozenError(CQL3Type.java:710)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepare(CQL3Type.java:669)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepareInternal(CQL3Type.java:661)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.lambda$prepare$1(Types.java:341)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.prepare(Types.java:342)
> at org.apache.cassandra.schema.Types$RawBuilder.build(Types.java:291)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.createTypes(CQLSSTableWriter.java:551)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.build(CQLSSTableWriter.java:527)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15857) Frozen RawTuple is not annotated with frozen in the toString method
[
https://issues.apache.org/jira/browse/CASSANDRA-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15857:
-
Status: Ready to Commit (was: Review In Progress)
+1
> Frozen RawTuple is not annotated with frozen in the toString method
> ---
>
> Key: CASSANDRA-15857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15857
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0, 3.11.x
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> All raw types (e.g. RawCollection, RawUT) that supports freezing wraps the
> type name with 'frozen<>' in the toString method, except RawTuple.
> Therefore, the RawTuple::toString output misses the frozen wrapper.
> Tuple is always frozen. However since CASSANDRA-15035, it throws when the
> inner tuple is not explicitly wrapped with frozen within a collection.
> The method, CQL3Type.Raw::toString, is referenced at multiple places in the
> source. For example, referenced in CreateTypeStatement.Raw and involved in
> CQLSSTableWriter. Another example is that it is called to produce the
> SchemaChange at several AlterSchemaStatement implementations.
> A test can prove that missing the frozen wrapper causes exception when
> building CQLSSTableWriter for user types defined like below. Note that the
> inner tuple is wrapped with frozen in the initial CQL statement.
> {code:java}
> CREATE TYPE ks.fooType ( f list>> )
> {code}
> {code:java}
> org.apache.cassandra.exceptions.InvalidRequestException: Non-frozen tuples
> are not allowed inside collections: list>
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.throwNestedNonFrozenError(CQL3Type.java:710)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepare(CQL3Type.java:669)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepareInternal(CQL3Type.java:661)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.lambda$prepare$1(Types.java:341)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.prepare(Types.java:342)
> at org.apache.cassandra.schema.Types$RawBuilder.build(Types.java:291)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.createTypes(CQLSSTableWriter.java:551)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.build(CQLSSTableWriter.java:527)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15857) Frozen RawTuple is not annotated with frozen in the toString method
[
https://issues.apache.org/jira/browse/CASSANDRA-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15857:
-
Reviewers: Robert Stupp, Robert Stupp (was: Robert Stupp)
Robert Stupp, Robert Stupp
Status: Review In Progress (was: Patch Available)
> Frozen RawTuple is not annotated with frozen in the toString method
> ---
>
> Key: CASSANDRA-15857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15857
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0, 3.11.x
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> All raw types (e.g. RawCollection, RawUT) that supports freezing wraps the
> type name with 'frozen<>' in the toString method, except RawTuple.
> Therefore, the RawTuple::toString output misses the frozen wrapper.
> Tuple is always frozen. However since CASSANDRA-15035, it throws when the
> inner tuple is not explicitly wrapped with frozen within a collection.
> The method, CQL3Type.Raw::toString, is referenced at multiple places in the
> source. For example, referenced in CreateTypeStatement.Raw and involved in
> CQLSSTableWriter. Another example is that it is called to produce the
> SchemaChange at several AlterSchemaStatement implementations.
> A test can prove that missing the frozen wrapper causes exception when
> building CQLSSTableWriter for user types defined like below. Note that the
> inner tuple is wrapped with frozen in the initial CQL statement.
> {code:java}
> CREATE TYPE ks.fooType ( f list>> )
> {code}
> {code:java}
> org.apache.cassandra.exceptions.InvalidRequestException: Non-frozen tuples
> are not allowed inside collections: list>
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.throwNestedNonFrozenError(CQL3Type.java:710)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepare(CQL3Type.java:669)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepareInternal(CQL3Type.java:661)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.lambda$prepare$1(Types.java:341)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.prepare(Types.java:342)
> at org.apache.cassandra.schema.Types$RawBuilder.build(Types.java:291)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.createTypes(CQLSSTableWriter.java:551)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.build(CQLSSTableWriter.java:527)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15857) Frozen RawTuple is not annotated with frozen in the toString method
[
https://issues.apache.org/jira/browse/CASSANDRA-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17161999#comment-17161999
]
Robert Stupp edited comment on CASSANDRA-15857 at 7/21/20, 12:25 PM:
-
Hm, not really a fan of sneaking unrelated changes in a 3.11-patch, although
it's probably not an issue. Let's stick with the "old" (non-{{@Rule}}) approach
in 3.11 for {{CQLSSTableWriterTest}}.
Otherwise the PR looks good.
was (Author: snazy):
Hm, not really a fan of sneaking unrelated changes in a 3.11-patch, although
it's probably not an issue. Let's stick with the "old" (non-{{@Rule}}) approach
in 3.11 for {{CQLSSTableWriterTest}}.
> Frozen RawTuple is not annotated with frozen in the toString method
> ---
>
> Key: CASSANDRA-15857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15857
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0, 3.11.x
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> All raw types (e.g. RawCollection, RawUT) that supports freezing wraps the
> type name with 'frozen<>' in the toString method, except RawTuple.
> Therefore, the RawTuple::toString output misses the frozen wrapper.
> Tuple is always frozen. However since CASSANDRA-15035, it throws when the
> inner tuple is not explicitly wrapped with frozen within a collection.
> The method, CQL3Type.Raw::toString, is referenced at multiple places in the
> source. For example, referenced in CreateTypeStatement.Raw and involved in
> CQLSSTableWriter. Another example is that it is called to produce the
> SchemaChange at several AlterSchemaStatement implementations.
> A test can prove that missing the frozen wrapper causes exception when
> building CQLSSTableWriter for user types defined like below. Note that the
> inner tuple is wrapped with frozen in the initial CQL statement.
> {code:java}
> CREATE TYPE ks.fooType ( f list>> )
> {code}
> {code:java}
> org.apache.cassandra.exceptions.InvalidRequestException: Non-frozen tuples
> are not allowed inside collections: list>
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.throwNestedNonFrozenError(CQL3Type.java:710)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepare(CQL3Type.java:669)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepareInternal(CQL3Type.java:661)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.lambda$prepare$1(Types.java:341)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.prepare(Types.java:342)
> at org.apache.cassandra.schema.Types$RawBuilder.build(Types.java:291)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.createTypes(CQLSSTableWriter.java:551)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.build(CQLSSTableWriter.java:527)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15857) Frozen RawTuple is not annotated with frozen in the toString method
[
https://issues.apache.org/jira/browse/CASSANDRA-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17161999#comment-17161999
]
Robert Stupp commented on CASSANDRA-15857:
--
Hm, not really a fan of sneaking unrelated changes in a 3.11-patch, although
it's probably not an issue. Let's stick with the "old" (non-{{@Rule}}) approach
in 3.11 for {{CQLSSTableWriterTest}}.
> Frozen RawTuple is not annotated with frozen in the toString method
> ---
>
> Key: CASSANDRA-15857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15857
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0, 3.11.x
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> All raw types (e.g. RawCollection, RawUT) that supports freezing wraps the
> type name with 'frozen<>' in the toString method, except RawTuple.
> Therefore, the RawTuple::toString output misses the frozen wrapper.
> Tuple is always frozen. However since CASSANDRA-15035, it throws when the
> inner tuple is not explicitly wrapped with frozen within a collection.
> The method, CQL3Type.Raw::toString, is referenced at multiple places in the
> source. For example, referenced in CreateTypeStatement.Raw and involved in
> CQLSSTableWriter. Another example is that it is called to produce the
> SchemaChange at several AlterSchemaStatement implementations.
> A test can prove that missing the frozen wrapper causes exception when
> building CQLSSTableWriter for user types defined like below. Note that the
> inner tuple is wrapped with frozen in the initial CQL statement.
> {code:java}
> CREATE TYPE ks.fooType ( f list>> )
> {code}
> {code:java}
> org.apache.cassandra.exceptions.InvalidRequestException: Non-frozen tuples
> are not allowed inside collections: list>
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.throwNestedNonFrozenError(CQL3Type.java:710)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepare(CQL3Type.java:669)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepareInternal(CQL3Type.java:661)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.lambda$prepare$1(Types.java:341)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.prepare(Types.java:342)
> at org.apache.cassandra.schema.Types$RawBuilder.build(Types.java:291)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.createTypes(CQLSSTableWriter.java:551)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.build(CQLSSTableWriter.java:527)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15857) Frozen RawTuple is not annotated with frozen in the toString method
[
https://issues.apache.org/jira/browse/CASSANDRA-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17161020#comment-17161020
]
Robert Stupp commented on CASSANDRA-15857:
--
LGTM, can you provide a backport to 3.11?
> Frozen RawTuple is not annotated with frozen in the toString method
> ---
>
> Key: CASSANDRA-15857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15857
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0, 3.11.x
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> All raw types (e.g. RawCollection, RawUT) that supports freezing wraps the
> type name with 'frozen<>' in the toString method, except RawTuple.
> Therefore, the RawTuple::toString output misses the frozen wrapper.
> Tuple is always frozen. However since CASSANDRA-15035, it throws when the
> inner tuple is not explicitly wrapped with frozen within a collection.
> The method, CQL3Type.Raw::toString, is referenced at multiple places in the
> source. For example, referenced in CreateTypeStatement.Raw and involved in
> CQLSSTableWriter. Another example is that it is called to produce the
> SchemaChange at several AlterSchemaStatement implementations.
> A test can prove that missing the frozen wrapper causes exception when
> building CQLSSTableWriter for user types defined like below. Note that the
> inner tuple is wrapped with frozen in the initial CQL statement.
> {code:java}
> CREATE TYPE ks.fooType ( f list>> )
> {code}
> {code:java}
> org.apache.cassandra.exceptions.InvalidRequestException: Non-frozen tuples
> are not allowed inside collections: list>
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.throwNestedNonFrozenError(CQL3Type.java:710)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepare(CQL3Type.java:669)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepareInternal(CQL3Type.java:661)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.lambda$prepare$1(Types.java:341)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.prepare(Types.java:342)
> at org.apache.cassandra.schema.Types$RawBuilder.build(Types.java:291)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.createTypes(CQLSSTableWriter.java:551)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.build(CQLSSTableWriter.java:527)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15375) Remove BackPressureStrategy
[
https://issues.apache.org/jira/browse/CASSANDRA-15375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159876#comment-17159876
]
Robert Stupp commented on CASSANDRA-15375:
--
+1
super nit: maybe [change the wording
here|https://github.com/belliottsmith/cassandra/commit/2ba8b4d162c20142c3d4c7a225432337b7bdbc36#diff-4805e34bd9553ede03778be66ddc06c7R262]
to "removed" instead of "deprecated"?
> Remove BackPressureStrategy
> ---
>
> Key: CASSANDRA-15375
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15375
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client, Observability/Logging
>Reporter: Jon Haddad
>Assignee: Jon Haddad
>Priority: Low
> Fix For: 4.0, 4.0-beta
>
>
> This is odd:
> {{INFO [main] 2019-10-25 10:33:07,985 DatabaseDescriptor.java:803 -
> Back-pressure is disabled with strategy
> org.apache.cassandra.net.RateBasedBackPressure\{high_ratio=0.9, factor=5,
> flow=FAST}.}}
> When I saw that, I wasn't sure if back pressure was actually disabled, or if
> I was really using {{RateBasedBackPressure.}}
> This should change to output either:
> {{Back-pressure is disabled}}
> {{or}}
> {{Back-pressure is enabled with strategy
> org.apache.cassandra.net.RateBasedBackPressure\{high_ratio=0.9, factor=5,
> flow=FAST}.}}{{}}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15955) java.nio.file.AccessDeniedException
[
https://issues.apache.org/jira/browse/CASSANDRA-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15955:
-
Resolution: Not A Problem
Status: Resolved (was: Triage Needed)
This is obviously related to your file-system permissions. Please check on the
C* user-mailing-list or ASF Slack {{#cassandra}}.
> java.nio.file.AccessDeniedException
> ---
>
> Key: CASSANDRA-15955
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15955
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Startup and Shutdown
>Reporter: rafi saeputra
>Priority: Normal
>
> when i installed cassandra with binnary tarball i got an error log like this
> ERROR [COMMIT-LOG-ALLOCATOR] 2020-07-17 11:47:33,712
> JVMStabilityInspector.java:113 - Exiting due to error while processing commit
> log during initialization.
> org.apache.cassandra.io.FSWriteError: java.nio.file.AccessDeniedException:
> /var/lib/cassandra/commitlog/CommitLog-7-1594961253708.log
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.(CommitLogSegment.java:180)
> at
> org.apache.cassandra.db.commitlog.MemoryMappedSegment.(MemoryMappedSegment.java:45)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.createSegment(CommitLogSegment.java:137)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.createSegment(CommitLogSegmentManagerStandard.java:66)
> at
> org.apache.cassandra.db.commitlog.AbstractCommitLogSegmentManager$1.runMayThrow(AbstractCommitLogSegmentManager.java:114)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.nio.file.AccessDeniedException:
> /var/lib/cassandra/commitlog/CommitLog-7-1594961253708.log
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
> at
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
> at java.nio.channels.FileChannel.open(FileChannel.java:287)
> at java.nio.channels.FileChannel.open(FileChannel.java:335)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.(CommitLogSegment.java:175)
> ... 7 common frames omitted
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15942) Tooling testing foundation
[ https://issues.apache.org/jira/browse/CASSANDRA-15942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-15942: - Reviewers: Robert Stupp, Robert Stupp (was: Robert Stupp) Robert Stupp, Robert Stupp Status: Review In Progress (was: Patch Available) Left a comment on the PR. Otherwise the patch looks good (and familiar ;) ). Ran a few of the changed tests and those passed. +1 w/ the PR comment (assume CI won't complain) > Tooling testing foundation > -- > > Key: CASSANDRA-15942 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15942 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > Tooling testing is not as thorough as we'd like it to be. The purpose of this > ticket, under the umbrella of CASSANDRA-15583, is to provide the foundations > to make it easy for devs, migrate the few existing tests as a POC and put > that forward as a proposal to build on top of. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15942) Tooling testing foundation
[ https://issues.apache.org/jira/browse/CASSANDRA-15942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-15942: - Status: Ready to Commit (was: Review In Progress) > Tooling testing foundation > -- > > Key: CASSANDRA-15942 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15942 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > Tooling testing is not as thorough as we'd like it to be. The purpose of this > ticket, under the umbrella of CASSANDRA-15583, is to provide the foundations > to make it easy for devs, migrate the few existing tests as a POC and put > that forward as a proposal to build on top of. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15857) Frozen RawTuple is not annotated with frozen in the toString method
[
https://issues.apache.org/jira/browse/CASSANDRA-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17158000#comment-17158000
]
Robert Stupp commented on CASSANDRA-15857:
--
Ugh - actually, the recorded type in system_schema.dropped_columns for a
dropped column with an unfrozen UDT is already broken (unrelated to this ticket
though):
{code}
[cqlsh 5.0.1 | Cassandra 3.11.7-SNAPSHOT | CQL spec 3.4.4 | Native protocol v4]
cqlsh> CREATE TYPE ks.some_type ( i int, t text );
cqlsh> CREATE TABLE ks.t ( pk int primary key, t some_type);
cqlsh> ALTER TABLE ks.t DROP t;
cqlsh> SELECT * FROM system_schema.dropped_columns ;
keyspace_name | table_name | column_name | dropped_time|
kind| type
---++-+-+-+
ks | t | t | 2020-07-15 08:35:10.494000+ |
regular | frozen>
{code}
> Frozen RawTuple is not annotated with frozen in the toString method
> ---
>
> Key: CASSANDRA-15857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15857
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0, 3.11.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> All raw types (e.g. RawCollection, RawUT) that supports freezing wraps the
> type name with 'frozen<>' in the toString method, except RawTuple.
> Therefore, the RawTuple::toString output misses the frozen wrapper.
> Tuple is always frozen. However since CASSANDRA-15035, it throws when the
> inner tuple is not explicitly wrapped with frozen within a collection.
> The method, CQL3Type.Raw::toString, is referenced at multiple places in the
> source. For example, referenced in CreateTypeStatement.Raw and involved in
> CQLSSTableWriter. Another example is that it is called to produce the
> SchemaChange at several AlterSchemaStatement implementations.
> A test can prove that missing the frozen wrapper causes exception when
> building CQLSSTableWriter for user types defined like below. Note that the
> inner tuple is wrapped with frozen in the initial CQL statement.
> {code:java}
> CREATE TYPE ks.fooType ( f list>> )
> {code}
> {code:java}
> org.apache.cassandra.exceptions.InvalidRequestException: Non-frozen tuples
> are not allowed inside collections: list>
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.throwNestedNonFrozenError(CQL3Type.java:710)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepare(CQL3Type.java:669)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepareInternal(CQL3Type.java:661)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.lambda$prepare$1(Types.java:341)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.prepare(Types.java:342)
> at org.apache.cassandra.schema.Types$RawBuilder.build(Types.java:291)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.createTypes(CQLSSTableWriter.java:551)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.build(CQLSSTableWriter.java:527)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15857) Frozen RawTuple is not annotated with frozen in the toString method
[
https://issues.apache.org/jira/browse/CASSANDRA-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157977#comment-17157977
]
Robert Stupp commented on CASSANDRA-15857:
--
Ah, sorry, my bad. We cannot "blindly" freeze all tuples, because we do have a
notion of non-frozen tuples for the very special use-case of dropped columns w/
non-frozen UDTs (UDTs are "converted" into a tuple when a column's dropped).
> Frozen RawTuple is not annotated with frozen in the toString method
> ---
>
> Key: CASSANDRA-15857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15857
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0, 3.11.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> All raw types (e.g. RawCollection, RawUT) that supports freezing wraps the
> type name with 'frozen<>' in the toString method, except RawTuple.
> Therefore, the RawTuple::toString output misses the frozen wrapper.
> Tuple is always frozen. However since CASSANDRA-15035, it throws when the
> inner tuple is not explicitly wrapped with frozen within a collection.
> The method, CQL3Type.Raw::toString, is referenced at multiple places in the
> source. For example, referenced in CreateTypeStatement.Raw and involved in
> CQLSSTableWriter. Another example is that it is called to produce the
> SchemaChange at several AlterSchemaStatement implementations.
> A test can prove that missing the frozen wrapper causes exception when
> building CQLSSTableWriter for user types defined like below. Note that the
> inner tuple is wrapped with frozen in the initial CQL statement.
> {code:java}
> CREATE TYPE ks.fooType ( f list>> )
> {code}
> {code:java}
> org.apache.cassandra.exceptions.InvalidRequestException: Non-frozen tuples
> are not allowed inside collections: list>
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.throwNestedNonFrozenError(CQL3Type.java:710)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepare(CQL3Type.java:669)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepareInternal(CQL3Type.java:661)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.lambda$prepare$1(Types.java:341)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.prepare(Types.java:342)
> at org.apache.cassandra.schema.Types$RawBuilder.build(Types.java:291)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.createTypes(CQLSSTableWriter.java:551)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.build(CQLSSTableWriter.java:527)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15857) Frozen RawTuple is not annotated with frozen in the toString method
[
https://issues.apache.org/jira/browse/CASSANDRA-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17156549#comment-17156549
]
Robert Stupp commented on CASSANDRA-15857:
--
Not really a typo, but there's no point in freezing a tuple, as there is no
unfrozen tuple. I.e. a {{frozen<>}} around a tuple doesn't add any value. So
dealing with an unfrozen tuple is meaningless IMO.
> Frozen RawTuple is not annotated with frozen in the toString method
> ---
>
> Key: CASSANDRA-15857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15857
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0, 3.11.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> All raw types (e.g. RawCollection, RawUT) that supports freezing wraps the
> type name with 'frozen<>' in the toString method, except RawTuple.
> Therefore, the RawTuple::toString output misses the frozen wrapper.
> Tuple is always frozen. However since CASSANDRA-15035, it throws when the
> inner tuple is not explicitly wrapped with frozen within a collection.
> The method, CQL3Type.Raw::toString, is referenced at multiple places in the
> source. For example, referenced in CreateTypeStatement.Raw and involved in
> CQLSSTableWriter. Another example is that it is called to produce the
> SchemaChange at several AlterSchemaStatement implementations.
> A test can prove that missing the frozen wrapper causes exception when
> building CQLSSTableWriter for user types defined like below. Note that the
> inner tuple is wrapped with frozen in the initial CQL statement.
> {code:java}
> CREATE TYPE ks.fooType ( f list>> )
> {code}
> {code:java}
> org.apache.cassandra.exceptions.InvalidRequestException: Non-frozen tuples
> are not allowed inside collections: list>
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.throwNestedNonFrozenError(CQL3Type.java:710)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepare(CQL3Type.java:669)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepareInternal(CQL3Type.java:661)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.lambda$prepare$1(Types.java:341)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.prepare(Types.java:342)
> at org.apache.cassandra.schema.Types$RawBuilder.build(Types.java:291)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.createTypes(CQLSSTableWriter.java:551)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.build(CQLSSTableWriter.java:527)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15857) Frozen RawTuple is not annotated with frozen in the toString method
[
https://issues.apache.org/jira/browse/CASSANDRA-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155304#comment-17155304
]
Robert Stupp commented on CASSANDRA-15857:
--
As tuples cannot be frozen at all, it seems more convenient to completely omit
the {{frozen<>}} bracket. I.e. removing the {{frozen}} param from
{{RawTuple.}}, removing the {{frozen}} handling in
{{CQL3Type.Tuple.toString()}} (remove {{frozen<>}} entirely from the string)
and adopting the frozen-validation-logic in
{{CQL3Type.Raw.RawCollection.prepare()}}.
Honestly, the handling of frozen isn't actually great in the code base, but
cleaning that up is quite a bit of work (too much for a patch release and even
for 4.0 now). E.g. it should be ensured that all subtypes of a type are always
frozen no matter what, the whole partition key must always be frozen,
clustering key elements must be frozen, a bunch of code constructs like "if
(some-check) freeze()" should be simplified and generalized. Things become a
bit complicated when freezable types are nested, for example a non-frozen-UDT
in a non-frozen-collection in a UDT, which is legal.
> Frozen RawTuple is not annotated with frozen in the toString method
> ---
>
> Key: CASSANDRA-15857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15857
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0, 3.11.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> All raw types (e.g. RawCollection, RawUT) that supports freezing wraps the
> type name with 'frozen<>' in the toString method, except RawTuple.
> Therefore, the RawTuple::toString output misses the frozen wrapper.
> Tuple is always frozen. However since CASSANDRA-15035, it throws when the
> inner tuple is not explicitly wrapped with frozen within a collection.
> The method, CQL3Type.Raw::toString, is referenced at multiple places in the
> source. For example, referenced in CreateTypeStatement.Raw and involved in
> CQLSSTableWriter. Another example is that it is called to produce the
> SchemaChange at several AlterSchemaStatement implementations.
> A test can prove that missing the frozen wrapper causes exception when
> building CQLSSTableWriter for user types defined like below. Note that the
> inner tuple is wrapped with frozen in the initial CQL statement.
> {code:java}
> CREATE TYPE ks.fooType ( f list>> )
> {code}
> {code:java}
> org.apache.cassandra.exceptions.InvalidRequestException: Non-frozen tuples
> are not allowed inside collections: list>
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.throwNestedNonFrozenError(CQL3Type.java:710)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepare(CQL3Type.java:669)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepareInternal(CQL3Type.java:661)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.lambda$prepare$1(Types.java:341)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.prepare(Types.java:342)
> at org.apache.cassandra.schema.Types$RawBuilder.build(Types.java:291)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.createTypes(CQLSSTableWriter.java:551)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.build(CQLSSTableWriter.java:527)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15859) Avoid per-host hinted-handoff throttle being rounded to 0 in large cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-15859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15859:
-
Reviewers: Robert Stupp, Robert Stupp (was: Robert Stupp)
Robert Stupp, Robert Stupp
Status: Review In Progress (was: Patch Available)
+1
Mind backporting it to 3.11 and 3.0?
> Avoid per-host hinted-handoff throttle being rounded to 0 in large cluster
> --
>
> Key: CASSANDRA-15859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15859
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> When "hinted_handoff_throttle_in_kb" is sufficiently small or num of nodes in
> the cluster is sufficiently large, the per-host throttle will be rounded to
> 0, aka. unthrottled.
>
> {code:java|title=HintsDispatchExecutor.java}
> int throttleInKB = DatabaseDescriptor.getHintedHandoffThrottleInKB() /
> nodesCount;
> this.rateLimiter = RateLimiter.create(throttleInKB == 0 ? Double.MAX_VALUE :
> throttleInKB * 1024);
> {code}
> [trunk-patch|https://github.com/apache/cassandra/pull/616]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-9739) Migrate counter-cache to be fully off-heap
[ https://issues.apache.org/jira/browse/CASSANDRA-9739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-9739: Resolution: Won't Fix Status: Resolved (was: Open) > Migrate counter-cache to be fully off-heap > -- > > Key: CASSANDRA-9739 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9739 > Project: Cassandra > Issue Type: Sub-task > Components: Legacy/Core >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Normal > Fix For: 4.x > > > Counter cache still uses a concurrent map on-heap. This could go to off-heap > and feels doable now after CASSANDRA-8099. > Evaluation should be done in advance based on a POC to prove that pure > off-heap counter cache buys a performance and/or gc-pressure improvement. > In theory, elimination of on-heap management of the map should buy us some > benefit. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-9555) Don't let offline tools run while cassandra is running
[ https://issues.apache.org/jira/browse/CASSANDRA-9555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp reassigned CASSANDRA-9555: --- Assignee: (was: Robert Stupp) > Don't let offline tools run while cassandra is running > -- > > Key: CASSANDRA-9555 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9555 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > We should not let offline tools that modify sstables run while Cassandra is > running. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13838) Ensure all threads are FastThreadLocal.removeAll() is called for all threads
[ https://issues.apache.org/jira/browse/CASSANDRA-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-13838: - Status: Open (was: Patch Available) > Ensure all threads are FastThreadLocal.removeAll() is called for all threads > > > Key: CASSANDRA-13838 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13838 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Normal > > There are a couple of places, there it's not guaranteed that > FastThreadLocal.removeAll() is called. Most misses are actually not that > critical, but the miss for the thread created via in > org.apache.cassandra.streaming.ConnectionHandler.MessageHandler#start(java.net.Socket, > int, boolean) could be critical, because these threads are created for every > stream-session. > (Follow-up from CASSANDRA-13754) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-9739) Migrate counter-cache to be fully off-heap
[ https://issues.apache.org/jira/browse/CASSANDRA-9739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152061#comment-17152061 ] Robert Stupp commented on CASSANDRA-9739: - [~aleksey] do you mind if i close this one? > Migrate counter-cache to be fully off-heap > -- > > Key: CASSANDRA-9739 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9739 > Project: Cassandra > Issue Type: Sub-task > Components: Legacy/Core >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Normal > Fix For: 4.x > > > Counter cache still uses a concurrent map on-heap. This could go to off-heap > and feels doable now after CASSANDRA-8099. > Evaluation should be done in advance based on a POC to prove that pure > off-heap counter cache buys a performance and/or gc-pressure improvement. > In theory, elimination of on-heap management of the map should buy us some > benefit. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-9454) Log WARN on Multi Partition IN clause Queries
[
https://issues.apache.org/jira/browse/CASSANDRA-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-9454:
Reviewers: (was: Robert Stupp)
> Log WARN on Multi Partition IN clause Queries
> -
>
> Key: CASSANDRA-9454
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9454
> Project: Cassandra
> Issue Type: New Feature
> Components: Legacy/CQL
>Reporter: Sebastian Estevez
>Assignee: T Jake Luciani
>Priority: Low
> Fix For: 2.2.x
>
>
> Similar to CASSANDRA-6487 but for multi-partition queries.
> Show warning (ideally at the client CASSANDRA-8930) when users try to use IN
> clauses when clustering columns span multiple partitions. The right way to go
> is async requests per partition.
> **Update**: Unless the query is CL.ONE and all the partition ranges are on
> the node! In which case multi partition IN is okay.
> This can cause an OOM
> {code}
> ERROR [Thread-388] 2015-05-18 12:11:10,147 CassandraDaemon.java (line 199)
> Exception in thread Thread[Thread-388,5,main]
> java.lang.OutOfMemoryError: Java heap space
> ERROR [ReadStage:321] 2015-05-18 12:11:10,147 CassandraDaemon.java (line 199)
> Exception in thread Thread[ReadStage:321,5,main]
> java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
> at
> org.apache.cassandra.io.util.MappedFileDataInput.readBytes(MappedFileDataInput.java:146)
> at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
> at
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:371)
> at
> org.apache.cassandra.io.sstable.IndexHelper$IndexInfo.deserialize(IndexHelper.java:187)
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:122)
> at
> org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:970)
> at
> org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:871)
> at
> org.apache.cassandra.db.columniterator.SSTableSliceIterator.(SSTableSliceIterator.java:41)
> at
> org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:167)
> at
> org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62)
> at
> org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:250)
> at
> org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1547)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1376)
> at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:327)
> at
> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
> at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> {code}
> By flooding heap with:
> {code}org.apache.cassandra.io.sstable.IndexHelper$IndexInfo{code}
> taken from:
> http://stackoverflow.com/questions/30366729/out-of-memory-error-in-cassandra-when-querying-big-rows-containing-a-collection
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15922) High CAS failures in NativeAllocator.Region.allocate(..)
[
https://issues.apache.org/jira/browse/CASSANDRA-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15922:
-
Reviewers: Benedict Elliott Smith, Robert Stupp, Robert Stupp (was:
Benedict Elliott Smith, Robert Stupp)
Benedict Elliott Smith, Robert Stupp, Robert Stupp (was:
Benedict Elliott Smith, Robert Stupp)
Status: Review In Progress (was: Patch Available)
> High CAS failures in NativeAllocator.Region.allocate(..)
> -
>
> Key: CASSANDRA-15922
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15922
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Memtable
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0, 3.0.x, 3.11.x
>
> Attachments: NativeAllocatorRegion2Test.java,
> NativeAllocatorRegionTest.java, Screen Shot 2020-07-05 at 13.16.10.png,
> Screen Shot 2020-07-05 at 13.26.17.png, Screen Shot 2020-07-05 at
> 13.35.55.png, Screen Shot 2020-07-05 at 13.37.01.png, Screen Shot 2020-07-05
> at 13.48.16.png, Screen Shot 2020-07-06 at 11.35.35.png, Screen Shot
> 2020-07-06 at 11.36.44.png, Screen Shot 2020-07-06 at 13.26.10.png,
> profile_pbdpc23zafsrh_20200702.svg
>
>
> h4. Problem
> The method {{NativeAllocator.Region.allocate(..)}} uses an {{AtomicInteger}}
> for the current offset in the region. Allocations depends on a
> {{.compareAndSet(..)}} call.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node.
> h4. Example
> It has been witnessed up to 33% of CPU time stuck in the
> {{NativeAllocator.Region.allocate(..)}} loop (due to the CAS failures) during
> a heavy spark analytics write load.
> These nodes: 40 CPU cores and 256GB ram; have relevant settings
> - {{memtable_allocation_type: offheap_objects}}
> - {{memtable_offheap_space_in_mb: 5120}}
> - {{concurrent_writes: 160}}
> Numerous flamegraphs demonstrate the problem. See attached
> [^profile_pbdpc23zafsrh_20200702.svg].
> h4. Suggestion: ThreadLocal Regions
> One possible solution is to have separate Regions per thread.
> Code wise this is relatively easy to do, for example replacing
> NativeAllocator:59
> {code}private final AtomicReference currentRegion = new
> AtomicReference<>();{code}
> with
> {code}private final ThreadLocal> currentRegion = new
> ThreadLocal<>() {...};{code}
> But this approach substantially changes the allocation behaviour, with more
> than concurrent_writes number of Regions in use at any one time. For example
> with {{concurrent_writes: 160}} that's 160+ regions, each of 1MB.
> h4. Suggestion: Simple Contention Management Algorithm (Constant Backoff)
> Another possible solution is to introduce a contention management algorithm
> that a) reduces CAS failures in high contention environments, b) doesn't
> impact normal environments, and c) keeps the allocation strategy of using one
> region at a time.
> The research paper [arXiv:1305.5800|https://arxiv.org/abs/1305.5800]
> describes this contention CAS problem and demonstrates a number of algorithms
> to apply. The simplest of these algorithms is the Constant Backoff CAS
> Algorithm.
> Applying the Constant Backoff CAS Algorithm involves adding one line of code
> to {{NativeAllocator.Region.allocate(..)}} to sleep for one (or some constant
> number) nanoseconds after a CAS failure occurs.
> That is...
> {code}
> // we raced and lost alloc, try again
> LockSupport.parkNanos(1);
> {code}
> h4. Constant Backoff CAS Algorithm Experiments
> Using the code attached in NativeAllocatorRegionTest.java the concurrency and
> CAS failures of {{NativeAllocator.Region.allocate(..)}} can be demonstrated.
> In the attached [^NativeAllocatorRegionTest.java] class, which can be run
> standalone, the {{Region}} class: copied from {{NativeAllocator.Region}}; has
> also the {{casFailures}} field added. The following two screenshots are from
> data collected from this class on a 6 CPU (12 core) MBP, running the
> {{NativeAllocatorRegionTest.testRegionCAS}} method.
> This attached screenshot shows the number of CAS failures during the life of
> a Region (over ~215 millions allocations), using different threads and park
> times. This illustrates the improvement (reduction) of CAS failures from zero
> park time, through orders of magnitude, up to 1000ns (10ms). The biggest
> improvement is from no algorithm to a park time of 1ns where CAS failures are
> ~two orders of magnitude lower. From a park time 10μs and higher there is a
> significant drop also at low contention rates.
> !Screen Shot 2020-07-05 at 13.16.10.png|width=500px!
> This attached screenshot shows the time it takes to fill a Region (~215
> million allocations),
[jira] [Updated] (CASSANDRA-15922) High CAS failures in NativeAllocator.Region.allocate(..)
[
https://issues.apache.org/jira/browse/CASSANDRA-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15922:
-
Reviewers: Benedict Elliott Smith, Robert Stupp
> High CAS failures in NativeAllocator.Region.allocate(..)
> -
>
> Key: CASSANDRA-15922
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15922
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Memtable
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0, 3.0.x, 3.11.x
>
> Attachments: NativeAllocatorRegion2Test.java,
> NativeAllocatorRegionTest.java, Screen Shot 2020-07-05 at 13.16.10.png,
> Screen Shot 2020-07-05 at 13.26.17.png, Screen Shot 2020-07-05 at
> 13.35.55.png, Screen Shot 2020-07-05 at 13.37.01.png, Screen Shot 2020-07-05
> at 13.48.16.png, Screen Shot 2020-07-06 at 11.35.35.png, Screen Shot
> 2020-07-06 at 11.36.44.png, Screen Shot 2020-07-06 at 13.26.10.png,
> profile_pbdpc23zafsrh_20200702.svg
>
>
> h4. Problem
> The method {{NativeAllocator.Region.allocate(..)}} uses an {{AtomicInteger}}
> for the current offset in the region. Allocations depends on a
> {{.compareAndSet(..)}} call.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node.
> h4. Example
> It has been witnessed up to 33% of CPU time stuck in the
> {{NativeAllocator.Region.allocate(..)}} loop (due to the CAS failures) during
> a heavy spark analytics write load.
> These nodes: 40 CPU cores and 256GB ram; have relevant settings
> - {{memtable_allocation_type: offheap_objects}}
> - {{memtable_offheap_space_in_mb: 5120}}
> - {{concurrent_writes: 160}}
> Numerous flamegraphs demonstrate the problem. See attached
> [^profile_pbdpc23zafsrh_20200702.svg].
> h4. Suggestion: ThreadLocal Regions
> One possible solution is to have separate Regions per thread.
> Code wise this is relatively easy to do, for example replacing
> NativeAllocator:59
> {code}private final AtomicReference currentRegion = new
> AtomicReference<>();{code}
> with
> {code}private final ThreadLocal> currentRegion = new
> ThreadLocal<>() {...};{code}
> But this approach substantially changes the allocation behaviour, with more
> than concurrent_writes number of Regions in use at any one time. For example
> with {{concurrent_writes: 160}} that's 160+ regions, each of 1MB.
> h4. Suggestion: Simple Contention Management Algorithm (Constant Backoff)
> Another possible solution is to introduce a contention management algorithm
> that a) reduces CAS failures in high contention environments, b) doesn't
> impact normal environments, and c) keeps the allocation strategy of using one
> region at a time.
> The research paper [arXiv:1305.5800|https://arxiv.org/abs/1305.5800]
> describes this contention CAS problem and demonstrates a number of algorithms
> to apply. The simplest of these algorithms is the Constant Backoff CAS
> Algorithm.
> Applying the Constant Backoff CAS Algorithm involves adding one line of code
> to {{NativeAllocator.Region.allocate(..)}} to sleep for one (or some constant
> number) nanoseconds after a CAS failure occurs.
> That is...
> {code}
> // we raced and lost alloc, try again
> LockSupport.parkNanos(1);
> {code}
> h4. Constant Backoff CAS Algorithm Experiments
> Using the code attached in NativeAllocatorRegionTest.java the concurrency and
> CAS failures of {{NativeAllocator.Region.allocate(..)}} can be demonstrated.
> In the attached [^NativeAllocatorRegionTest.java] class, which can be run
> standalone, the {{Region}} class: copied from {{NativeAllocator.Region}}; has
> also the {{casFailures}} field added. The following two screenshots are from
> data collected from this class on a 6 CPU (12 core) MBP, running the
> {{NativeAllocatorRegionTest.testRegionCAS}} method.
> This attached screenshot shows the number of CAS failures during the life of
> a Region (over ~215 millions allocations), using different threads and park
> times. This illustrates the improvement (reduction) of CAS failures from zero
> park time, through orders of magnitude, up to 1000ns (10ms). The biggest
> improvement is from no algorithm to a park time of 1ns where CAS failures are
> ~two orders of magnitude lower. From a park time 10μs and higher there is a
> significant drop also at low contention rates.
> !Screen Shot 2020-07-05 at 13.16.10.png|width=500px!
> This attached screenshot shows the time it takes to fill a Region (~215
> million allocations), using different threads and park times. The biggest
> improvement is from no algorithm to a park time of 1ns where performance is
> one order of magnitude faster. From a park time of 100μs and higher there is
> a even further
[jira] [Commented] (CASSANDRA-15922) High CAS failures in NativeAllocator.Region.allocate(..)
[
https://issues.apache.org/jira/browse/CASSANDRA-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152006#comment-17152006
]
Robert Stupp commented on CASSANDRA-15922:
--
+1 (assuming CI looks good and 3.11+3.0 back-ports are clean)
> High CAS failures in NativeAllocator.Region.allocate(..)
> -
>
> Key: CASSANDRA-15922
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15922
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Memtable
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0, 3.0.x, 3.11.x
>
> Attachments: NativeAllocatorRegion2Test.java,
> NativeAllocatorRegionTest.java, Screen Shot 2020-07-05 at 13.16.10.png,
> Screen Shot 2020-07-05 at 13.26.17.png, Screen Shot 2020-07-05 at
> 13.35.55.png, Screen Shot 2020-07-05 at 13.37.01.png, Screen Shot 2020-07-05
> at 13.48.16.png, Screen Shot 2020-07-06 at 11.35.35.png, Screen Shot
> 2020-07-06 at 11.36.44.png, Screen Shot 2020-07-06 at 13.26.10.png,
> profile_pbdpc23zafsrh_20200702.svg
>
>
> h4. Problem
> The method {{NativeAllocator.Region.allocate(..)}} uses an {{AtomicInteger}}
> for the current offset in the region. Allocations depends on a
> {{.compareAndSet(..)}} call.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node.
> h4. Example
> It has been witnessed up to 33% of CPU time stuck in the
> {{NativeAllocator.Region.allocate(..)}} loop (due to the CAS failures) during
> a heavy spark analytics write load.
> These nodes: 40 CPU cores and 256GB ram; have relevant settings
> - {{memtable_allocation_type: offheap_objects}}
> - {{memtable_offheap_space_in_mb: 5120}}
> - {{concurrent_writes: 160}}
> Numerous flamegraphs demonstrate the problem. See attached
> [^profile_pbdpc23zafsrh_20200702.svg].
> h4. Suggestion: ThreadLocal Regions
> One possible solution is to have separate Regions per thread.
> Code wise this is relatively easy to do, for example replacing
> NativeAllocator:59
> {code}private final AtomicReference currentRegion = new
> AtomicReference<>();{code}
> with
> {code}private final ThreadLocal> currentRegion = new
> ThreadLocal<>() {...};{code}
> But this approach substantially changes the allocation behaviour, with more
> than concurrent_writes number of Regions in use at any one time. For example
> with {{concurrent_writes: 160}} that's 160+ regions, each of 1MB.
> h4. Suggestion: Simple Contention Management Algorithm (Constant Backoff)
> Another possible solution is to introduce a contention management algorithm
> that a) reduces CAS failures in high contention environments, b) doesn't
> impact normal environments, and c) keeps the allocation strategy of using one
> region at a time.
> The research paper [arXiv:1305.5800|https://arxiv.org/abs/1305.5800]
> describes this contention CAS problem and demonstrates a number of algorithms
> to apply. The simplest of these algorithms is the Constant Backoff CAS
> Algorithm.
> Applying the Constant Backoff CAS Algorithm involves adding one line of code
> to {{NativeAllocator.Region.allocate(..)}} to sleep for one (or some constant
> number) nanoseconds after a CAS failure occurs.
> That is...
> {code}
> // we raced and lost alloc, try again
> LockSupport.parkNanos(1);
> {code}
> h4. Constant Backoff CAS Algorithm Experiments
> Using the code attached in NativeAllocatorRegionTest.java the concurrency and
> CAS failures of {{NativeAllocator.Region.allocate(..)}} can be demonstrated.
> In the attached [^NativeAllocatorRegionTest.java] class, which can be run
> standalone, the {{Region}} class: copied from {{NativeAllocator.Region}}; has
> also the {{casFailures}} field added. The following two screenshots are from
> data collected from this class on a 6 CPU (12 core) MBP, running the
> {{NativeAllocatorRegionTest.testRegionCAS}} method.
> This attached screenshot shows the number of CAS failures during the life of
> a Region (over ~215 millions allocations), using different threads and park
> times. This illustrates the improvement (reduction) of CAS failures from zero
> park time, through orders of magnitude, up to 1000ns (10ms). The biggest
> improvement is from no algorithm to a park time of 1ns where CAS failures are
> ~two orders of magnitude lower. From a park time 10μs and higher there is a
> significant drop also at low contention rates.
> !Screen Shot 2020-07-05 at 13.16.10.png|width=500px!
> This attached screenshot shows the time it takes to fill a Region (~215
> million allocations), using different threads and park times. The biggest
> improvement is from no algorithm to a park time of 1ns where performance is
> one order of magnitude faster. From a park
[jira] [Commented] (CASSANDRA-15922) High CAS failures in NativeAllocator.Region.allocate(..)
[
https://issues.apache.org/jira/browse/CASSANDRA-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151969#comment-17151969
]
Robert Stupp commented on CASSANDRA-15922:
--
+1 on {{addAndGet}} (or {{getAndAdd}}, whichever works best).
And I agree, the allocation-model that we currently have is not great, but as
you said, it's a ton of work to get it right (less (ideally no) fragmentation,
no unnecessary tiny allocations, no unnecessary copying, etc etc).
> High CAS failures in NativeAllocator.Region.allocate(..)
> -
>
> Key: CASSANDRA-15922
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15922
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Memtable
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0, 3.0.x, 3.11.x
>
> Attachments: NativeAllocatorRegion2Test.java,
> NativeAllocatorRegionTest.java, Screen Shot 2020-07-05 at 13.16.10.png,
> Screen Shot 2020-07-05 at 13.26.17.png, Screen Shot 2020-07-05 at
> 13.35.55.png, Screen Shot 2020-07-05 at 13.37.01.png, Screen Shot 2020-07-05
> at 13.48.16.png, Screen Shot 2020-07-06 at 11.35.35.png, Screen Shot
> 2020-07-06 at 11.36.44.png, Screen Shot 2020-07-06 at 13.26.10.png,
> profile_pbdpc23zafsrh_20200702.svg
>
>
> h4. Problem
> The method {{NativeAllocator.Region.allocate(..)}} uses an {{AtomicInteger}}
> for the current offset in the region. Allocations depends on a
> {{.compareAndSet(..)}} call.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node.
> h4. Example
> It has been witnessed up to 33% of CPU time stuck in the
> {{NativeAllocator.Region.allocate(..)}} loop (due to the CAS failures) during
> a heavy spark analytics write load.
> These nodes: 40 CPU cores and 256GB ram; have relevant settings
> - {{memtable_allocation_type: offheap_objects}}
> - {{memtable_offheap_space_in_mb: 5120}}
> - {{concurrent_writes: 160}}
> Numerous flamegraphs demonstrate the problem. See attached
> [^profile_pbdpc23zafsrh_20200702.svg].
> h4. Suggestion: ThreadLocal Regions
> One possible solution is to have separate Regions per thread.
> Code wise this is relatively easy to do, for example replacing
> NativeAllocator:59
> {code}private final AtomicReference currentRegion = new
> AtomicReference<>();{code}
> with
> {code}private final ThreadLocal> currentRegion = new
> ThreadLocal<>() {...};{code}
> But this approach substantially changes the allocation behaviour, with more
> than concurrent_writes number of Regions in use at any one time. For example
> with {{concurrent_writes: 160}} that's 160+ regions, each of 1MB.
> h4. Suggestion: Simple Contention Management Algorithm (Constant Backoff)
> Another possible solution is to introduce a contention management algorithm
> that a) reduces CAS failures in high contention environments, b) doesn't
> impact normal environments, and c) keeps the allocation strategy of using one
> region at a time.
> The research paper [arXiv:1305.5800|https://arxiv.org/abs/1305.5800]
> describes this contention CAS problem and demonstrates a number of algorithms
> to apply. The simplest of these algorithms is the Constant Backoff CAS
> Algorithm.
> Applying the Constant Backoff CAS Algorithm involves adding one line of code
> to {{NativeAllocator.Region.allocate(..)}} to sleep for one (or some constant
> number) nanoseconds after a CAS failure occurs.
> That is...
> {code}
> // we raced and lost alloc, try again
> LockSupport.parkNanos(1);
> {code}
> h4. Constant Backoff CAS Algorithm Experiments
> Using the code attached in NativeAllocatorRegionTest.java the concurrency and
> CAS failures of {{NativeAllocator.Region.allocate(..)}} can be demonstrated.
> In the attached [^NativeAllocatorRegionTest.java] class, which can be run
> standalone, the {{Region}} class: copied from {{NativeAllocator.Region}}; has
> also the {{casFailures}} field added. The following two screenshots are from
> data collected from this class on a 6 CPU (12 core) MBP, running the
> {{NativeAllocatorRegionTest.testRegionCAS}} method.
> This attached screenshot shows the number of CAS failures during the life of
> a Region (over ~215 millions allocations), using different threads and park
> times. This illustrates the improvement (reduction) of CAS failures from zero
> park time, through orders of magnitude, up to 1000ns (10ms). The biggest
> improvement is from no algorithm to a park time of 1ns where CAS failures are
> ~two orders of magnitude lower. From a park time 10μs and higher there is a
> significant drop also at low contention rates.
> !Screen Shot 2020-07-05 at 13.16.10.png|width=500px!
> This attached screenshot shows the time it takes
[jira] [Updated] (CASSANDRA-9954) Improve Java-UDF timeout detection
[
https://issues.apache.org/jira/browse/CASSANDRA-9954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-9954:
Reviewers: (was: Tom Hobbs)
> Improve Java-UDF timeout detection
> --
>
> Key: CASSANDRA-9954
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9954
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 3.11.x
>
>
> CASSANDRA-9402 introduced a sandbox using a thread-pool to enforce security
> constraints and to detect "amok UDFs" - i.e. UDFs that essentially never
> return (e.g. {{while (true)}}.
> Currently the safest way to react on such an "amok UDF" is to _fail-fast_ -
> to stop the C* daemon since stopping a thread (in Java) is just no solution.
> CASSANDRA-9890 introduced further protection by inspecting the byte-code. The
> same mechanism can also be used to manipulate the Java-UDF byte-code.
> By manipulating the byte-code I mean to add regular "is-amok-UDF" checks in
> the compiled code.
> EDIT: These "is-amok-UDF" checks would also work for _UNFENCED_ Java-UDFs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-11870) Consider allocating direct buffers bypassing ByteBuffer.allocateDirect
[
https://issues.apache.org/jira/browse/CASSANDRA-11870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp reassigned CASSANDRA-11870:
Assignee: (was: Robert Stupp)
> Consider allocating direct buffers bypassing ByteBuffer.allocateDirect
> --
>
> Key: CASSANDRA-11870
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11870
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Robert Stupp
>Priority: Low
> Fix For: 3.11.x
>
>
> As outlined in CASSANDRA-11818, {{ByteBuffer.allocateDirect}} uses
> {{Bits.reserveMemory}}, which is there to respect the JVM setting
> {{-XX:MaxDirectMemorySize=...}}.
> {{Bits.reserveMemory}} first tries an "optimistic" {{tryReserveMemory}} and
> exits immediately on success. However, if that somehow doesn't succeed, it
> triggers a {{System.gc()}}, which is bad IMO (however, kind of how direct
> buffers work in Java). After that GC it sleeps and tries to reserve the
> memory up to 9 times - up to 511 ms - and then throws
> {{OutOfMemoryError("Direct buffer memory")}}.
> This is unnecessary for us since we always immediately "free" direct buffers
> as soon as we no longer need them.
> Proposal: Manage direct-memory reservations in our own code and skip
> {{Bits.reserveMemory}} that way.
> (However, Netty direct buffers are not under our control.)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-11052) Cannot use Java 8 lambda expression inside UDF code body
[
https://issues.apache.org/jira/browse/CASSANDRA-11052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp reassigned CASSANDRA-11052:
Assignee: (was: Robert Stupp)
> Cannot use Java 8 lambda expression inside UDF code body
>
>
> Key: CASSANDRA-11052
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11052
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: DuyHai Doan
>Priority: Normal
> Fix For: 4.x
>
> Attachments: 11052-2.patch, 11052.patch
>
>
> When creating the following **UDF** using Java 8 lambda syntax
> {code:sql}
> CREATE FUNCTION IF NOT EXISTS music.udf(state map, styles
> list)
> RETURNS NULL ON NULL INPUT
> RETURNS map
> LANGUAGE java
> AS $$
>styles.forEach((Object o) -> {
>String style = (String)o;
>if(state.containsKey(style)) {
> state.put(style, (Long)state.get(style)+1);
>} else {
> state.put(style, 1L);
>}
>});
>
>return state;
> $$;
> {code}
> I got the following exception:
> {code:java}
> Caused by: com.datastax.driver.core.exceptions.InvalidQueryException: Could
> not compile function 'music.udf' from Java source:
> org.apache.cassandra.exceptions.InvalidRequestException: Java source
> compilation failed:
> Line 2: The type java.util.function.Consumer cannot be resolved. It is
> indirectly referenced from required .class files
> Line 2: The method forEach(Consumer) from the type Iterable refers to the
> missing type Consumer
> Line 2: The target type of this expression must be a functional interface
> at
> com.datastax.driver.core.Responses$Error.asException(Responses.java:136)
> at
> com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:179)
> at
> com.datastax.driver.core.RequestHandler.setFinalResult(RequestHandler.java:184)
> at
> com.datastax.driver.core.RequestHandler.access$2500(RequestHandler.java:43)
> at
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.setFinalResult(RequestHandler.java:798)
> at
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:617)
> at
> com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1005)
> at
> com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:928)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:318)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:304)
> at
> io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:266)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:318)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:304)
> at
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:318)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:304)
> at
> io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:276)
> at
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:263)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:318)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:304)
> at
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846)
> at
> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
> ... 1 more
> {code}
> It looks like the compiler requires importing java.util.Consumer but I have
> checked the source code and compiler options already support Java 8 source
> code so I'm pretty puzzled here ...
> /cc [~snazy]
--
This
[jira] [Updated] (CASSANDRA-15816) Transports are stopped in the wrong order
[ https://issues.apache.org/jira/browse/CASSANDRA-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-15816: - Reviewers: Robert Stupp, Robert Stupp (was: Robert Stupp) Robert Stupp, Robert Stupp Status: Review In Progress (was: Patch Available) Sounds reasonable to me. +1 (on both the approach and PR) > Transports are stopped in the wrong order > - > > Key: CASSANDRA-15816 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15816 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > Stopping gossip while native is running is almost always wrong, change the > order of shutdown and log a warning when done manually -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15816) Transports are stopped in the wrong order
[ https://issues.apache.org/jira/browse/CASSANDRA-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-15816: - Status: Ready to Commit (was: Review In Progress) > Transports are stopped in the wrong order > - > > Key: CASSANDRA-15816 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15816 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > Stopping gossip while native is running is almost always wrong, change the > order of shutdown and log a warning when done manually -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15922) High CAS failures in NativeAllocator.Region.allocate(..)
[
https://issues.apache.org/jira/browse/CASSANDRA-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151832#comment-17151832
]
Robert Stupp commented on CASSANDRA-15922:
--
Talked to Mick a bit about this offline. The demonstrated effects (in the
attached charts and SVG) are IMO "good enough" to justify the change.
However, there's a slight issue in the attached
{{NativeAllocatorRegionTest.java}} {{Region.allocate()}} method that adds
another CAS ({{casFailures}}) to every failed CAS against {{nextFreeOffset}}.
It's probably better to count the number of failed CAS's in a local variable
and add it to {{this.casFailures}} when the test's {{Region.allocate()}}
returns.
I think, the proposed solution here to add a constant
{{LockSupport.sleepNanos(1)}} is fine and "non-intrusive enough". Although the
same change probably needs to be applied to
{{org.apache.cassandra.utils.memory.SlabAllocator.Region#allocate}} as well.
> High CAS failures in NativeAllocator.Region.allocate(..)
> -
>
> Key: CASSANDRA-15922
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15922
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Memtable
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0, 3.0.x, 3.11.x
>
> Attachments: NativeAllocatorRegionTest.java, Screen Shot 2020-07-05
> at 13.16.10.png, Screen Shot 2020-07-05 at 13.26.17.png, Screen Shot
> 2020-07-05 at 13.35.55.png, Screen Shot 2020-07-05 at 13.37.01.png, Screen
> Shot 2020-07-05 at 13.48.16.png, profile_pbdpc23zafsrh_20200702.svg
>
>
> h4. Problem
> The method {{NativeAllocator.Region.allocate(..)}} uses an {{AtomicInteger}}
> for the current offset in the region. Allocations depends on a
> {{.compareAndSet(..)}} call.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node.
> h4. Example
> It has been witnessed up to 33% of CPU time stuck in the
> {{NativeAllocator.Region.allocate(..)}} loop (due to the CAS failures) during
> a heavy spark analytics write load.
> These nodes: 40 CPU cores and 256GB ram; have relevant settings
> - {{memtable_allocation_type: offheap_objects}}
> - {{memtable_offheap_space_in_mb: 5120}}
> - {{concurrent_writes: 160}}
> Numerous flamegraphs demonstrate the problem. See attached
> [^profile_pbdpc23zafsrh_20200702.svg].
> h4. Suggestion: ThreadLocal Regions
> One possible solution is to have separate Regions per thread.
> Code wise this is relatively easy to do, for example replacing
> NativeAllocator:59
> {code}private final AtomicReference currentRegion = new
> AtomicReference<>();{code}
> with
> {code}private final ThreadLocal> currentRegion = new
> ThreadLocal<>() {...};{code}
> But this approach substantially changes the allocation behaviour, with more
> than concurrent_writes number of Regions in use at any one time. For example
> with {{concurrent_writes: 160}} that's 160+ regions, each of 1MB.
> h4. Suggestion: Simple Contention Management Algorithm (Constant Backoff)
> Another possible solution is to introduce a contention management algorithm
> that a) reduces CAS failures in high contention environments, b) doesn't
> impact normal environments, and c) keeps the allocation strategy of using one
> region at a time.
> The research paper [arXiv:1305.5800|https://arxiv.org/abs/1305.5800]
> describes this contention CAS problem and demonstrates a number of algorithms
> to apply. The simplest of these algorithms is the Constant Backoff CAS
> Algorithm.
> Applying the Constant Backoff CAS Algorithm involves adding one line of code
> to {{NativeAllocator.Region.allocate(..)}} to sleep for one (or some constant
> number) nanoseconds after a CAS failure occurs.
> That is...
> {code}
> // we raced and lost alloc, try again
> LockSupport.parkNanos(1);
> {code}
> h4. Constant Backoff CAS Algorithm Experiments
> Using the code attached in NativeAllocatorRegionTest.java the concurrency and
> CAS failures of {{NativeAllocator.Region.allocate(..)}} can be demonstrated.
> In the attached [^NativeAllocatorRegionTest.java] class, which can be run
> standalone, the {{Region}} class: copied from {{NativeAllocator.Region}}; has
> also the {{casFailures}} field added. The following two screenshots are from
> data collected from this class on a 6 CPU (12 core) MBP, running the
> {{NativeAllocatorRegionTest.testRegionCAS}} method.
> This attached screenshot shows the number of CAS failures during the life of
> a Region (over ~215 millions allocations), using different threads and park
> times. This illustrates the improvement (reduction) of CAS failures from zero
> park time, through orders of magnitude, up to 1000ns (10ms). The biggest
[jira] [Commented] (CASSANDRA-15214) OOMs caught and not rethrown
[ https://issues.apache.org/jira/browse/CASSANDRA-15214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17150869#comment-17150869 ] Robert Stupp commented on CASSANDRA-15214: -- Just read this ticket and the approach looks absolutely reasonable to me. One thing though is that the the (off-heap) row-cache isn't covered here - let me know whether it's reasonable to add some support regarding this ticket. IMHO, people shouldn't use the row-cache, but I'm not sure whether there are reasonable use cases out there in the wild. Don't want to start a discussion about the row-cache in this, just a heads-up. > OOMs caught and not rethrown > > > Key: CASSANDRA-15214 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15214 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client, Messaging/Internode >Reporter: Benedict Elliott Smith >Priority: Normal > Fix For: 4.0, 4.0-rc > > Attachments: oom-experiments.zip > > > Netty (at least, and perhaps elsewhere in Executors) catches all exceptions, > so presently there is no way to ensure that an OOM reaches the JVM handler to > trigger a crash/heapdump. > It may be that the simplest most consistent way to do this would be to have a > single thread spawned at startup that waits for any exceptions we must > propagate to the Runtime. > We could probably submit a patch upstream to Netty, but for a guaranteed > future proof approach, it may be worth paying the cost of a single thread. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15901) Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15901:
-
Reviewers: Mick Semb Wever, Robert Stupp, Robert Stupp (was: Mick Semb
Wever, Robert Stupp)
Mick Semb Wever, Robert Stupp, Robert Stupp (was: Mick Semb
Wever, Robert Stupp)
Status: Review In Progress (was: Patch Available)
> Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15901) Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15901:
-
Reviewers: Mick Semb Wever, Robert Stupp
> Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15901) Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17150084#comment-17150084
]
Robert Stupp commented on CASSANDRA-15901:
--
Well, the most unintrusive way would be to add the hostname to the 127.0.0.1
(or even better the host's public IP) entry in /etc/hosts.
Mosts tests in question (e.g. the {{ReplicaCollectionTest}} or
{{SEPExecutorTest}}) do not need a c-yaml at all (and didn't need it before),
because those are "pure" unit tests (i.e. don't need a daemon or any
configuration). Other tests (like the {{DatabaseDescriptorRefTest}}) are
"allergic" to touching classes before the actual test. The next guy who comes
along these tests, likely thinks that the DD call is unnecessary and removes
it, because the test passes locally and e.g. in CircleCI.
The change to {{DatabaseDescriptorRefTest}} actually causes the test to not
fully exercise the DD-init code being tested (and "silently pass" although it
didn't test the whole init code or, if new changes get in, fail because some
expectations aren't met). TL;DR it breaks that test.
The implemented default for {{broadcast_address}} (defaulting to
{{listen_address}}, which in turn defaults to {{InetAddress.getLocalHost()}}),
is correct (the yaml says: _"{{Leaving it blank leaves it up to
InetAddress.getLocalHost(). This will always do the Right Thing *if the node is
properly configured (hostname, name resolution, etc)*, and the Right Thing is
to use the address associated with the hostname (it might not be).}}"_). I
admit, it might be wrong to add a fallback to
{{InetAddress.getLoopbackAddress()}}.
TL;DR I think it's much safer to fix the test environment (and not work around
test environment issues by changing any existing code).
Side note (feel free to ignore this): I do think that using
{{InetAddress.getLocalHost()}} is a mistake in general. It seems that (in
practice) there is no guarantee that it {{IA.getLocalHost()}} deterministically
return the same address - think: DHCP changes, interfaces going up or down,
hostnames that resolve to multiple IPs, etc etc.
> Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15901) Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149303#comment-17149303
]
Robert Stupp commented on CASSANDRA-15901:
--
Hm - maybe it's easier to change
{{org.apache.cassandra.utils.FBUtilities#getJustLocalAddress}} and add a
try-catch(UnknownHostException) around the {{InetAddress.getLocalHost()}}
(_"Returns the address of the local host. This is achieved by retrieving the
name of the host from the system, then resolving that name into an
{{InetAddress}}."_) and use {{InetAddress.getLoopbackAddress()}} when that's
thrown (to avoid a change of the current behavior). Guess, this affects all
branches (and all other uses of {{InetAddress.getLocalHost()}}).
> Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15901) Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149205#comment-17149205
]
Robert Stupp commented on CASSANDRA-15901:
--
I'm not sure whether adding {{DatabaseDescriptor.daemonInitialization()}} to
the tests is the right way here. Those tests do not need the daemon at all and
adding these DD.daemonInit() makes those tests slower. The change to
{{DatabaseDescriptorRefTest}} can actually break that test itself. The
{{SSTable*Test.java}} tests test offline tools, so DD.daemonInit() feels wrong
here.
> Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15849) Remove generated files from apache-cassandra-*-src.tar.gz artifacts
[ https://issues.apache.org/jira/browse/CASSANDRA-15849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17128171#comment-17128171 ] Robert Stupp commented on CASSANDRA-15849: -- +1 > Remove generated files from apache-cassandra-*-src.tar.gz artifacts > --- > > Key: CASSANDRA-15849 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15849 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-beta > > > The apache-cassandra-*-src.tar.gz source artifacts contain a lot of generated > files. > These need to be excluded when putting together the source tarball. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15849) Remove generated files from apache-cassandra-*-src.tar.gz artifacts
[ https://issues.apache.org/jira/browse/CASSANDRA-15849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-15849: - Reviewers: Robert Stupp, Robert Stupp (was: Robert Stupp) Robert Stupp, Robert Stupp Status: Review In Progress (was: Patch Available) > Remove generated files from apache-cassandra-*-src.tar.gz artifacts > --- > > Key: CASSANDRA-15849 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15849 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-beta > > > The apache-cassandra-*-src.tar.gz source artifacts contain a lot of generated > files. > These need to be excluded when putting together the source tarball. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15849) Remove generated files from apache-cassandra-*-src.tar.gz artifacts
[ https://issues.apache.org/jira/browse/CASSANDRA-15849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-15849: - Status: Ready to Commit (was: Review In Progress) > Remove generated files from apache-cassandra-*-src.tar.gz artifacts > --- > > Key: CASSANDRA-15849 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15849 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-beta > > > The apache-cassandra-*-src.tar.gz source artifacts contain a lot of generated > files. > These need to be excluded when putting together the source tarball. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15835) Upgrade-dtests on trunk not working in CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15835:
-
Since Version: 4.0-alpha1
Source Control Link:
https://github.com/apache/cassandra/commit/d6552ce3a42d3394277e8156959c9409bb302d81
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed as
[d6552ce3a42d3394277e8156959c9409bb302d81|https://github.com/apache/cassandra/commit/d6552ce3a42d3394277e8156959c9409bb302d81]
to [trunk|https://github.com/apache/cassandra/tree/trunk].
Merged PRs:
https://github.com/riptano/ccm/pull/713
https://github.com/riptano/ccm/pull/714
https://github.com/apache/cassandra-dtest/pull/71
https://github.com/apache/cassandra-builds/pull/22
> Upgrade-dtests on trunk not working in CircleCI
> ---
>
> Key: CASSANDRA-15835
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15835
> Project: Cassandra
> Issue Type: Bug
> Components: CI, Test/dtest
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> ~3600 Upgrade-dtests are failing in CircleCI for trunk due to the missing
> {{JAVA8_HOME}} environment variable.
> Patching the Docker image is rather simple by creating a new image:
> {code}
> FROM nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200406
> ENV JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-amd64
> {code}
> Pushed the above to Docker hub as
> [snazy/cassandra-testing-ubuntu1910-java11-w-dependencies:202005261540|https://hub.docker.com/layers/snazy/cassandra-testing-ubuntu1910-java11-w-dependencies/202005261540/images/sha256-ac8a713be58694f095c491921e006c2d1a7823a3c23299e477198e2c93a6bbd7?context=explore]
> The size of the whole Docker image is a little concerning though (1.85G
> compressed), but that's out of the scope of this ticket.
> I'll prepare a patch soon-ish.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15835) Upgrade-dtests on trunk not working in CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15835:
-
Status: Ready to Commit (was: Review In Progress)
+1'd on the linked PRs
> Upgrade-dtests on trunk not working in CircleCI
> ---
>
> Key: CASSANDRA-15835
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15835
> Project: Cassandra
> Issue Type: Bug
> Components: CI, Test/dtest
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> ~3600 Upgrade-dtests are failing in CircleCI for trunk due to the missing
> {{JAVA8_HOME}} environment variable.
> Patching the Docker image is rather simple by creating a new image:
> {code}
> FROM nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200406
> ENV JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-amd64
> {code}
> Pushed the above to Docker hub as
> [snazy/cassandra-testing-ubuntu1910-java11-w-dependencies:202005261540|https://hub.docker.com/layers/snazy/cassandra-testing-ubuntu1910-java11-w-dependencies/202005261540/images/sha256-ac8a713be58694f095c491921e006c2d1a7823a3c23299e477198e2c93a6bbd7?context=explore]
> The size of the whole Docker image is a little concerning though (1.85G
> compressed), but that's out of the scope of this ticket.
> I'll prepare a patch soon-ish.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15853) @since missing/wrong for upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33
[ https://issues.apache.org/jira/browse/CASSANDRA-15853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-15853: - Fix Version/s: 4.0-beta 3.11.7 3.0.21 Since Version: 3.0.0 Source Control Link: https://github.com/apache/cassandra-dtest/commit/ca009392d1697ef5a42e87a62930fe9274a3c2fe Resolution: Fixed Status: Resolved (was: Ready to Commit) > @since missing/wrong for > upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33 > - > > Key: CASSANDRA-15853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15853 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Normal > Fix For: 3.0.21, 3.11.7, 4.0-beta > > > @since missing/wrong for > upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15853) @since missing/wrong for upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33
[ https://issues.apache.org/jira/browse/CASSANDRA-15853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-15853: - Reviewers: Eduard Tudenhoefner, Robert Stupp (was: Eduard Tudenhoefner) Status: Review In Progress (was: Patch Available) +1'd on the PR > @since missing/wrong for > upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33 > - > > Key: CASSANDRA-15853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15853 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Normal > > @since missing/wrong for > upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15853) @since missing/wrong for upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33
[ https://issues.apache.org/jira/browse/CASSANDRA-15853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-15853: - Test and Documentation Plan: - Status: Patch Available (was: In Progress) PR: https://github.com/apache/cassandra-dtest/pull/73 > @since missing/wrong for > upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33 > - > > Key: CASSANDRA-15853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15853 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Normal > > @since missing/wrong for > upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15853) @since missing/wrong for upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33
[ https://issues.apache.org/jira/browse/CASSANDRA-15853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-15853: - Status: Ready to Commit (was: Review In Progress) > @since missing/wrong for > upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33 > - > > Key: CASSANDRA-15853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15853 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Normal > > @since missing/wrong for > upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15853) @since missing/wrong for upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33
[ https://issues.apache.org/jira/browse/CASSANDRA-15853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-15853: - Bug Category: Parent values: Code(13163)Level 1 values: Bug - Unclear Impact(13164) Complexity: Low Hanging Fruit Discovered By: Unit Test Reviewers: Eduard Tudenhoefner Severity: Low Status: Open (was: Triage Needed) > @since missing/wrong for > upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33 > - > > Key: CASSANDRA-15853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15853 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Normal > > @since missing/wrong for > upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15853) @since missing/wrong for upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33
Robert Stupp created CASSANDRA-15853: Summary: @since missing/wrong for upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33 Key: CASSANDRA-15853 URL: https://issues.apache.org/jira/browse/CASSANDRA-15853 Project: Cassandra Issue Type: Bug Components: Test/dtest Reporter: Robert Stupp Assignee: Robert Stupp @since missing/wrong for upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15835) Upgrade-dtests on trunk not working in CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17123593#comment-17123593
]
Robert Stupp edited comment on CASSANDRA-15835 at 6/2/20, 1:27 PM:
---
PRs:
* ccm/cassandra-test: https://github.com/riptano/ccm/pull/713
* ccm/master: https://github.com/riptano/ccm/pull/714
* dtest/master: https://github.com/apache/cassandra-dtest/pull/71
* cassandra/trunk: https://github.com/apache/cassandra/pull/611
* cassandra-builds/master: https://github.com/apache/cassandra-builds/pull/22
The above PRs fix a couple of issues w/ dtests:
* Fix/re-enable upgrade-dtests broken by CASSANDRA-14420 + CASSANDRA-14421
* Fix building C* using Java 11 during upgrade-dtests
* Use the correct Java version when starting C* and related processes
* Don't let check_logs_for_errors() fail, if there's no log file
* Always create a node*-startup-stdout/stderr.log (which helps a lot to
investigate why nodes don't start)
* Add `--exists-option w` for dtests requiements.txt installation
[~eduard.tudenhoefner] can you review?
was (Author: snazy):
PRs:
* ccm/cassandra-test: https://github.com/riptano/ccm/pull/713
* ccm/master: https://github.com/riptano/ccm/pull/714
* dtest/master: https://github.com/apache/cassandra-dtest/pull/71
* cassandra/trunk: https://github.com/apache/cassandra/pull/611
* cassandra-builds/master: https://github.com/apache/cassandra-builds/pull/22
The above PRs fix a couple of issues w/ dtests:
* Fix/re-enable upgrade-dtests broken by CASSANDRA-14420 + CASSANDRA-14421
* Fix building C* using Java 11 during upgrade-dtests
* Use the correct Java version when starting C* and related processes
* Don't let check_logs_for_errors() fail, if there's no log file
* Always create a node*-startup-stdout/stderr.log (which helps a lot to
investigate why nodes don't start)
[~eduard.tudenhoefner] can you review?
> Upgrade-dtests on trunk not working in CircleCI
> ---
>
> Key: CASSANDRA-15835
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15835
> Project: Cassandra
> Issue Type: Bug
> Components: CI, Test/dtest
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> ~3600 Upgrade-dtests are failing in CircleCI for trunk due to the missing
> {{JAVA8_HOME}} environment variable.
> Patching the Docker image is rather simple by creating a new image:
> {code}
> FROM nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200406
> ENV JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-amd64
> {code}
> Pushed the above to Docker hub as
> [snazy/cassandra-testing-ubuntu1910-java11-w-dependencies:202005261540|https://hub.docker.com/layers/snazy/cassandra-testing-ubuntu1910-java11-w-dependencies/202005261540/images/sha256-ac8a713be58694f095c491921e006c2d1a7823a3c23299e477198e2c93a6bbd7?context=explore]
> The size of the whole Docker image is a little concerning though (1.85G
> compressed), but that's out of the scope of this ticket.
> I'll prepare a patch soon-ish.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15835) Upgrade-dtests on trunk not working in CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15835:
-
Reviewers: Eduard Tudenhoefner
> Upgrade-dtests on trunk not working in CircleCI
> ---
>
> Key: CASSANDRA-15835
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15835
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> ~3600 Upgrade-dtests are failing in CircleCI for trunk due to the missing
> {{JAVA8_HOME}} environment variable.
> Patching the Docker image is rather simple by creating a new image:
> {code}
> FROM nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200406
> ENV JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-amd64
> {code}
> Pushed the above to Docker hub as
> [snazy/cassandra-testing-ubuntu1910-java11-w-dependencies:202005261540|https://hub.docker.com/layers/snazy/cassandra-testing-ubuntu1910-java11-w-dependencies/202005261540/images/sha256-ac8a713be58694f095c491921e006c2d1a7823a3c23299e477198e2c93a6bbd7?context=explore]
> The size of the whole Docker image is a little concerning though (1.85G
> compressed), but that's out of the scope of this ticket.
> I'll prepare a patch soon-ish.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15835) Upgrade-dtests on trunk not working in CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15835:
-
Test and Documentation Plan: Internal Circle-CI runs
Status: Patch Available (was: In Progress)
> Upgrade-dtests on trunk not working in CircleCI
> ---
>
> Key: CASSANDRA-15835
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15835
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> ~3600 Upgrade-dtests are failing in CircleCI for trunk due to the missing
> {{JAVA8_HOME}} environment variable.
> Patching the Docker image is rather simple by creating a new image:
> {code}
> FROM nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200406
> ENV JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-amd64
> {code}
> Pushed the above to Docker hub as
> [snazy/cassandra-testing-ubuntu1910-java11-w-dependencies:202005261540|https://hub.docker.com/layers/snazy/cassandra-testing-ubuntu1910-java11-w-dependencies/202005261540/images/sha256-ac8a713be58694f095c491921e006c2d1a7823a3c23299e477198e2c93a6bbd7?context=explore]
> The size of the whole Docker image is a little concerning though (1.85G
> compressed), but that's out of the scope of this ticket.
> I'll prepare a patch soon-ish.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15835) Upgrade-dtests on trunk not working in CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17123593#comment-17123593
]
Robert Stupp edited comment on CASSANDRA-15835 at 6/2/20, 10:44 AM:
PRs:
* ccm/cassandra-test: https://github.com/riptano/ccm/pull/713
* ccm/master: https://github.com/riptano/ccm/pull/714
* dtest/master: https://github.com/apache/cassandra-dtest/pull/71
* cassandra/trunk: https://github.com/apache/cassandra/pull/611
* cassandra-builds/master: https://github.com/apache/cassandra-builds/pull/22
The above PRs fix a couple of issues w/ dtests:
* Fix/re-enable upgrade-dtests broken by CASSANDRA-14420 + CASSANDRA-14421
* Fix building C* using Java 11 during upgrade-dtests
* Use the correct Java version when starting C* and related processes
* Don't let check_logs_for_errors() fail, if there's no log file
* Always create a node*-startup-stdout/stderr.log (which helps a lot to
investigate why nodes don't start)
[~eduard.tudenhoefner] can you review?
was (Author: snazy):
PRs:
* ccm/cassandra-test: https://github.com/riptano/ccm/pull/713
* ccm/master: https://github.com/riptano/ccm/pull/714
* dtest/master: https://github.com/apache/cassandra-dtest/pull/71
* cassandra/trunk: https://github.com/apache/cassandra/pull/611
The above PRs fix a couple of issues w/ dtests:
* Fix/re-enable upgrade-dtests broken by CASSANDRA-14420 + CASSANDRA-14421
* Fix building C* using Java 11 during upgrade-dtests
* Use the correct Java version when starting C* and related processes
* Don't let check_logs_for_errors() fail, if there's no log file
* Always create a node*-startup-stdout/stderr.log (which helps a lot to
investigate why nodes don't start)
[~eduard.tudenhoefner] can you review?
> Upgrade-dtests on trunk not working in CircleCI
> ---
>
> Key: CASSANDRA-15835
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15835
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> ~3600 Upgrade-dtests are failing in CircleCI for trunk due to the missing
> {{JAVA8_HOME}} environment variable.
> Patching the Docker image is rather simple by creating a new image:
> {code}
> FROM nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200406
> ENV JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-amd64
> {code}
> Pushed the above to Docker hub as
> [snazy/cassandra-testing-ubuntu1910-java11-w-dependencies:202005261540|https://hub.docker.com/layers/snazy/cassandra-testing-ubuntu1910-java11-w-dependencies/202005261540/images/sha256-ac8a713be58694f095c491921e006c2d1a7823a3c23299e477198e2c93a6bbd7?context=explore]
> The size of the whole Docker image is a little concerning though (1.85G
> compressed), but that's out of the scope of this ticket.
> I'll prepare a patch soon-ish.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15835) Upgrade-dtests on trunk not working in CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17123612#comment-17123612
]
Robert Stupp commented on CASSANDRA-15835:
--
[~jmckenzie] not worth to block any release on this one. The CI runs with the
above changes look almost good.
> Upgrade-dtests on trunk not working in CircleCI
> ---
>
> Key: CASSANDRA-15835
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15835
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> ~3600 Upgrade-dtests are failing in CircleCI for trunk due to the missing
> {{JAVA8_HOME}} environment variable.
> Patching the Docker image is rather simple by creating a new image:
> {code}
> FROM nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200406
> ENV JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-amd64
> {code}
> Pushed the above to Docker hub as
> [snazy/cassandra-testing-ubuntu1910-java11-w-dependencies:202005261540|https://hub.docker.com/layers/snazy/cassandra-testing-ubuntu1910-java11-w-dependencies/202005261540/images/sha256-ac8a713be58694f095c491921e006c2d1a7823a3c23299e477198e2c93a6bbd7?context=explore]
> The size of the whole Docker image is a little concerning though (1.85G
> compressed), but that's out of the scope of this ticket.
> I'll prepare a patch soon-ish.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15835) Upgrade-dtests on trunk not working in CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17123593#comment-17123593
]
Robert Stupp edited comment on CASSANDRA-15835 at 6/2/20, 10:33 AM:
PRs:
* ccm/cassandra-test: https://github.com/riptano/ccm/pull/713
* ccm/master: https://github.com/riptano/ccm/pull/714
* dtest/master: https://github.com/apache/cassandra-dtest/pull/71
* cassandra/trunk: https://github.com/apache/cassandra/pull/611
The above PRs fix a couple of issues w/ dtests:
* Fix/re-enable upgrade-dtests broken by CASSANDRA-14420 + CASSANDRA-14421
* Fix building C* using Java 11 during upgrade-dtests
* Use the correct Java version when starting C* and related processes
* Don't let check_logs_for_errors() fail, if there's no log file
* Always create a node*-startup-stdout/stderr.log (which helps a lot to
investigate why nodes don't start)
[~eduard.tudenhoefner] can you review?
was (Author: snazy):
PRs:
* ccm/cassandra-test: https://github.com/riptano/ccm/pull/713
* ccm/master: https://github.com/riptano/ccm/pull/714
* dtest/master: https://github.com/apache/cassandra-dtest/pull/71
The above PRs fix a couple of issues w/ dtests:
* Fix/re-enable upgrade-dtests broken by CASSANDRA-14420 + CASSANDRA-14421
* Fix building C* using Java 11 during upgrade-dtests
* Use the correct Java version when starting C* and related processes
* Don't let check_logs_for_errors() fail, if there's no log file
* Always create a node*-startup-stdout/stderr.log (which helps a lot to
investigate why nodes don't start)
[~eduard.tudenhoefner] can you review?
> Upgrade-dtests on trunk not working in CircleCI
> ---
>
> Key: CASSANDRA-15835
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15835
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> ~3600 Upgrade-dtests are failing in CircleCI for trunk due to the missing
> {{JAVA8_HOME}} environment variable.
> Patching the Docker image is rather simple by creating a new image:
> {code}
> FROM nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200406
> ENV JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-amd64
> {code}
> Pushed the above to Docker hub as
> [snazy/cassandra-testing-ubuntu1910-java11-w-dependencies:202005261540|https://hub.docker.com/layers/snazy/cassandra-testing-ubuntu1910-java11-w-dependencies/202005261540/images/sha256-ac8a713be58694f095c491921e006c2d1a7823a3c23299e477198e2c93a6bbd7?context=explore]
> The size of the whole Docker image is a little concerning though (1.85G
> compressed), but that's out of the scope of this ticket.
> I'll prepare a patch soon-ish.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15835) Upgrade-dtests on trunk not working in CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17123593#comment-17123593
]
Robert Stupp commented on CASSANDRA-15835:
--
PRs:
* ccm/cassandra-test: https://github.com/riptano/ccm/pull/713
* ccm/master: https://github.com/riptano/ccm/pull/714
* dtest/master: https://github.com/apache/cassandra-dtest/pull/71
The above PRs fix a couple of issues w/ dtests:
* Fix/re-enable upgrade-dtests broken by CASSANDRA-14420 + CASSANDRA-14421
* Fix building C* using Java 11 during upgrade-dtests
* Use the correct Java version when starting C* and related processes
* Don't let check_logs_for_errors() fail, if there's no log file
* Always create a node*-startup-stdout/stderr.log (which helps a lot to
investigate why nodes don't start)
[~eduard.tudenhoefner] can you review?
> Upgrade-dtests on trunk not working in CircleCI
> ---
>
> Key: CASSANDRA-15835
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15835
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> ~3600 Upgrade-dtests are failing in CircleCI for trunk due to the missing
> {{JAVA8_HOME}} environment variable.
> Patching the Docker image is rather simple by creating a new image:
> {code}
> FROM nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200406
> ENV JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-amd64
> {code}
> Pushed the above to Docker hub as
> [snazy/cassandra-testing-ubuntu1910-java11-w-dependencies:202005261540|https://hub.docker.com/layers/snazy/cassandra-testing-ubuntu1910-java11-w-dependencies/202005261540/images/sha256-ac8a713be58694f095c491921e006c2d1a7823a3c23299e477198e2c93a6bbd7?context=explore]
> The size of the whole Docker image is a little concerning though (1.85G
> compressed), but that's out of the scope of this ticket.
> I'll prepare a patch soon-ish.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15835) Upgrade-dtests on trunk not working in CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15835:
-
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
Complexity: Low Hanging Fruit
Discovered By: User Report
Severity: Normal
Status: Open (was: Triage Needed)
> Upgrade-dtests on trunk not working in CircleCI
> ---
>
> Key: CASSANDRA-15835
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15835
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> ~3600 Upgrade-dtests are failing in CircleCI for trunk due to the missing
> {{JAVA8_HOME}} environment variable.
> Patching the Docker image is rather simple by creating a new image:
> {code}
> FROM nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200406
> ENV JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-amd64
> {code}
> Pushed the above to Docker hub as
> [snazy/cassandra-testing-ubuntu1910-java11-w-dependencies:202005261540|https://hub.docker.com/layers/snazy/cassandra-testing-ubuntu1910-java11-w-dependencies/202005261540/images/sha256-ac8a713be58694f095c491921e006c2d1a7823a3c23299e477198e2c93a6bbd7?context=explore]
> The size of the whole Docker image is a little concerning though (1.85G
> compressed), but that's out of the scope of this ticket.
> I'll prepare a patch soon-ish.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15835) Upgrade-dtests on trunk not working in CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15835:
-
Fix Version/s: 4.0-alpha
> Upgrade-dtests on trunk not working in CircleCI
> ---
>
> Key: CASSANDRA-15835
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15835
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> ~3600 Upgrade-dtests are failing in CircleCI for trunk due to the missing
> {{JAVA8_HOME}} environment variable.
> Patching the Docker image is rather simple by creating a new image:
> {code}
> FROM nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200406
> ENV JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-amd64
> {code}
> Pushed the above to Docker hub as
> [snazy/cassandra-testing-ubuntu1910-java11-w-dependencies:202005261540|https://hub.docker.com/layers/snazy/cassandra-testing-ubuntu1910-java11-w-dependencies/202005261540/images/sha256-ac8a713be58694f095c491921e006c2d1a7823a3c23299e477198e2c93a6bbd7?context=explore]
> The size of the whole Docker image is a little concerning though (1.85G
> compressed), but that's out of the scope of this ticket.
> I'll prepare a patch soon-ish.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15835) Upgrade-dtests on trunk not working in CircleCI
Robert Stupp created CASSANDRA-15835:
Summary: Upgrade-dtests on trunk not working in CircleCI
Key: CASSANDRA-15835
URL: https://issues.apache.org/jira/browse/CASSANDRA-15835
Project: Cassandra
Issue Type: Bug
Components: Test/dtest
Reporter: Robert Stupp
Assignee: Robert Stupp
~3600 Upgrade-dtests are failing in CircleCI for trunk due to the missing
{{JAVA8_HOME}} environment variable.
Patching the Docker image is rather simple by creating a new image:
{code}
FROM nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200406
ENV JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-amd64
{code}
Pushed the above to Docker hub as
[snazy/cassandra-testing-ubuntu1910-java11-w-dependencies:202005261540|https://hub.docker.com/layers/snazy/cassandra-testing-ubuntu1910-java11-w-dependencies/202005261540/images/sha256-ac8a713be58694f095c491921e006c2d1a7823a3c23299e477198e2c93a6bbd7?context=explore]
The size of the whole Docker image is a little concerning though (1.85G
compressed), but that's out of the scope of this ticket.
I'll prepare a patch soon-ish.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15676) flaky test testWriteUnknownResult- org.apache.cassandra.distributed.test.CasWriteTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15676:
-
Source Control Link:
https://github.com/apache/cassandra/commit/fdcd0dff216d9e1ad242be1a7d5be3ef67044ac3
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
+1 from me as well
Thanks for the patch!
Committed as
[fdcd0dff216d9e1ad242be1a7d5be3ef67044ac3|https://github.com/apache/cassandra/commit/fdcd0dff216d9e1ad242be1a7d5be3ef67044ac3]
to [trunk|https://github.com/apache/cassandra/tree/trunk].
> flaky test testWriteUnknownResult-
> org.apache.cassandra.distributed.test.CasWriteTest
> -
>
> Key: CASSANDRA-15676
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15676
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest
>Reporter: Kevin Gallardo
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: Screen Shot 2020-05-07 at 7.25.19 PM.png
>
>
> Failure observed in:
> https://app.circleci.com/pipelines/github/newkek/cassandra/33/workflows/54007cf7-4424-4ec1-9655-665f6044e6d1/jobs/187/tests
> {noformat}
> testWriteUnknownResult - org.apache.cassandra.distributed.test.CasWriteTest
> junit.framework.AssertionFailedError: Expecting cause to be
> CasWriteUncertainException
> at
> org.apache.cassandra.distributed.test.CasWriteTest.testWriteUnknownResult(CasWriteTest.java:257)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15676) flaky test testWriteUnknownResult- org.apache.cassandra.distributed.test.CasWriteTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15676:
-
Reviewers: Ekaterina Dimitrova, Yifan Cai, Robert Stupp (was: Ekaterina
Dimitrova, Robert Stupp, Yifan Cai)
Ekaterina Dimitrova, Yifan Cai, Robert Stupp (was: Ekaterina
Dimitrova, Yifan Cai)
Status: Review In Progress (was: Patch Available)
> flaky test testWriteUnknownResult-
> org.apache.cassandra.distributed.test.CasWriteTest
> -
>
> Key: CASSANDRA-15676
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15676
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest
>Reporter: Kevin Gallardo
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: Screen Shot 2020-05-07 at 7.25.19 PM.png
>
>
> Failure observed in:
> https://app.circleci.com/pipelines/github/newkek/cassandra/33/workflows/54007cf7-4424-4ec1-9655-665f6044e6d1/jobs/187/tests
> {noformat}
> testWriteUnknownResult - org.apache.cassandra.distributed.test.CasWriteTest
> junit.framework.AssertionFailedError: Expecting cause to be
> CasWriteUncertainException
> at
> org.apache.cassandra.distributed.test.CasWriteTest.testWriteUnknownResult(CasWriteTest.java:257)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15676) flaky test testWriteUnknownResult- org.apache.cassandra.distributed.test.CasWriteTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15676:
-
Status: Ready to Commit (was: Review In Progress)
> flaky test testWriteUnknownResult-
> org.apache.cassandra.distributed.test.CasWriteTest
> -
>
> Key: CASSANDRA-15676
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15676
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest
>Reporter: Kevin Gallardo
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: Screen Shot 2020-05-07 at 7.25.19 PM.png
>
>
> Failure observed in:
> https://app.circleci.com/pipelines/github/newkek/cassandra/33/workflows/54007cf7-4424-4ec1-9655-665f6044e6d1/jobs/187/tests
> {noformat}
> testWriteUnknownResult - org.apache.cassandra.distributed.test.CasWriteTest
> junit.framework.AssertionFailedError: Expecting cause to be
> CasWriteUncertainException
> at
> org.apache.cassandra.distributed.test.CasWriteTest.testWriteUnknownResult(CasWriteTest.java:257)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15806) C* 4.0 is missing a way to identify transient/non-transient SSTables on disk
[ https://issues.apache.org/jira/browse/CASSANDRA-15806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-15806: - Since Version: 4.0-alpha Source Control Link: https://github.com/apache/cassandra/commit/3f689e93768ea670f7a8351ec30128dd4b410c9c Resolution: Fixed Status: Resolved (was: Ready to Commit) Thanks! Committed as [3f689e93768ea670f7a8351ec30128dd4b410c9c|https://github.com/apache/cassandra/commit/3f689e93768ea670f7a8351ec30128dd4b410c9c] to [trunk|https://github.com/apache/cassandra/tree/trunk]. > C* 4.0 is missing a way to identify transient/non-transient SSTables on disk > > > Key: CASSANDRA-15806 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15806 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: sequoyha pelletier >Assignee: sequoyha pelletier >Priority: Normal > Fix For: 4.0-alpha > > Attachments: 15806-4.0.txt > > > Currently, there is no way to identify SSTables that were created as > transient replicated data. Even thought the feature is experimental we should > open that up for those that want to experiment. This seems to be an > oversight. I have added the missing line of code to the SStableMetadataViewer > and will attach a patch shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15820) tools/bin/fqltool doesn't work on all distributions
[
https://issues.apache.org/jira/browse/CASSANDRA-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15820:
-
Since Version: 4.0-alpha
Source Control Link:
https://github.com/apache/cassandra/commit/ec07cd7e76c93bf713618f381480f500f6c4e62f
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Thanks!
Committed as
[ec07cd7e76c93bf713618f381480f500f6c4e62f|https://github.com/apache/cassandra/commit/ec07cd7e76c93bf713618f381480f500f6c4e62f]
to [trunk|https://github.com/apache/cassandra/tree/trunk].
> tools/bin/fqltool doesn't work on all distributions
> ---
>
> Key: CASSANDRA-15820
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15820
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> The line
> {code}
> if [ ! $1 ]; then break; fi
> {code}
> doesn't work on all OSes/Linux distributions (e.g. a bare Ubuntu 18.04) with
> {{#!/bin/sh}} causing {{fqltool}} to fail. The fix is quite simple
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15820) tools/bin/fqltool doesn't work on all distributions
[
https://issues.apache.org/jira/browse/CASSANDRA-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15820:
-
Test and Documentation Plan: Local dtest run
Status: Patch Available (was: Open)
> tools/bin/fqltool doesn't work on all distributions
> ---
>
> Key: CASSANDRA-15820
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15820
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> The line
> {code}
> if [ ! $1 ]; then break; fi
> {code}
> doesn't work on all OSes/Linux distributions (e.g. a bare Ubuntu 18.04) with
> {{#!/bin/sh}} causing {{fqltool}} to fail. The fix is quite simple
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15820) tools/bin/fqltool doesn't work on all distributions
[
https://issues.apache.org/jira/browse/CASSANDRA-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15820:
-
Bug Category: Parent values: Code(13163)Level 1 values: Bug - Unclear
Impact(13164)
Complexity: Low Hanging Fruit
Discovered By: Adhoc Test
Severity: Normal
Status: Open (was: Triage Needed)
> tools/bin/fqltool doesn't work on all distributions
> ---
>
> Key: CASSANDRA-15820
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15820
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> The line
> {code}
> if [ ! $1 ]; then break; fi
> {code}
> doesn't work on all OSes/Linux distributions (e.g. a bare Ubuntu 18.04) with
> {{#!/bin/sh}} causing {{fqltool}} to fail. The fix is quite simple
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15820) tools/bin/fqltool doesn't work on all distributions
[
https://issues.apache.org/jira/browse/CASSANDRA-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110855#comment-17110855
]
Robert Stupp commented on CASSANDRA-15820:
--
PR: https://github.com/apache/cassandra/pull/595
> tools/bin/fqltool doesn't work on all distributions
> ---
>
> Key: CASSANDRA-15820
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15820
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> The line
> {code}
> if [ ! $1 ]; then break; fi
> {code}
> doesn't work on all OSes/Linux distributions (e.g. a bare Ubuntu 18.04) with
> {{#!/bin/sh}} causing {{fqltool}} to fail. The fix is quite simple
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15820) tools/bin/fqltool doesn't work on all distributions
[
https://issues.apache.org/jira/browse/CASSANDRA-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15820:
-
Fix Version/s: 4.0-alpha
> tools/bin/fqltool doesn't work on all distributions
> ---
>
> Key: CASSANDRA-15820
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15820
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> The line
> {code}
> if [ ! $1 ]; then break; fi
> {code}
> doesn't work on all OSes/Linux distributions (e.g. a bare Ubuntu 18.04) with
> {{#!/bin/sh}} causing {{fqltool}} to fail. The fix is quite simple
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15820) tools/bin/fqltool doesn't work on all distributions
Robert Stupp created CASSANDRA-15820:
Summary: tools/bin/fqltool doesn't work on all distributions
Key: CASSANDRA-15820
URL: https://issues.apache.org/jira/browse/CASSANDRA-15820
Project: Cassandra
Issue Type: Bug
Components: Tool/fql
Reporter: Robert Stupp
Assignee: Robert Stupp
The line
{code}
if [ ! $1 ]; then break; fi
{code}
doesn't work on all OSes/Linux distributions (e.g. a bare Ubuntu 18.04) with
{{#!/bin/sh}} causing {{fqltool}} to fail. The fix is quite simple
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15786) ChecksummingTransformerTest#corruptionCausesFailure fails for seed 43595190254702
[
https://issues.apache.org/jira/browse/CASSANDRA-15786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15786:
-
Resolution: Duplicate
Status: Resolved (was: Triage Needed)
> ChecksummingTransformerTest#corruptionCausesFailure fails for seed
> 43595190254702
> -
>
> Key: CASSANDRA-15786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15786
> Project: Cassandra
> Issue Type: Bug
>Reporter: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> Reproducer:
> * latest trunk (e.g. df8e736700ae2a06675ff50381788d708bc22b96)
> * change 1st line in {{corruptionCausesFailure()}} to
> {{qt().withFixedSeed(43595190254702L)}}
> * run test
> Fails with:
> {code:java}
> java.lang.AssertionError: Property falsified after 617 example(s)
> Smallest found falsifying value(s) :-
>
[jira] [Updated] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15780:
-
Source Control Link:
https://github.com/apache/cassandra/commit/e394dc0bb32f612a476269010930c617dd1ed3cb
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Thanks!
Committed as
[e394dc0bb32f612a476269010930c617dd1ed3cb|https://github.com/apache/cassandra/commit/e394dc0bb32f612a476269010930c617dd1ed3cb]
to [trunk|https://github.com/apache/cassandra/tree/trunk].
> Less test stderr spam when running tests w/ java 11
> ---
>
> Key: CASSANDRA-15780
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> There are a bunch of tests that, when run with Java 11, emit the boilerplate
> {{WARNING: An illegal reflective access operation has occurred}} messages.
> The proposed change just adds more invocations to
> {{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to
> get rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15780:
-
Reviewers: Eduard Tudenhoefner, Robert Stupp (was: Eduard Tudenhoefner)
Status: Review In Progress (was: Patch Available)
> Less test stderr spam when running tests w/ java 11
> ---
>
> Key: CASSANDRA-15780
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> There are a bunch of tests that, when run with Java 11, emit the boilerplate
> {{WARNING: An illegal reflective access operation has occurred}} messages.
> The proposed change just adds more invocations to
> {{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to
> get rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15780:
-
Status: Ready to Commit (was: Review In Progress)
> Less test stderr spam when running tests w/ java 11
> ---
>
> Key: CASSANDRA-15780
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> There are a bunch of tests that, when run with Java 11, emit the boilerplate
> {{WARNING: An illegal reflective access operation has occurred}} messages.
> The proposed change just adds more invocations to
> {{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to
> get rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099251#comment-17099251
]
Robert Stupp commented on CASSANDRA-15780:
--
[https://app.circleci.com/pipelines/github/snazy/cassandra/10/workflows/fa15ded3-6428-426c-be0a-ce0fb87b9d5e]
[https://app.circleci.com/pipelines/github/snazy/cassandra/10/workflows/60eb74ea-d7da-4567-b33d-62da21c78feb]
Those look "as good as trunk".
> Less test stderr spam when running tests w/ java 11
> ---
>
> Key: CASSANDRA-15780
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> There are a bunch of tests that, when run with Java 11, emit the boilerplate
> {{WARNING: An illegal reflective access operation has occurred}} messages.
> The proposed change just adds more invocations to
> {{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to
> get rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15786) ChecksummingTransformerTest#corruptionCausesFailure fails for seed 43595190254702
[
https://issues.apache.org/jira/browse/CASSANDRA-15786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15786:
-
Fix Version/s: 4.0
> ChecksummingTransformerTest#corruptionCausesFailure fails for seed
> 43595190254702
> -
>
> Key: CASSANDRA-15786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15786
> Project: Cassandra
> Issue Type: Bug
>Reporter: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> Reproducer:
> * latest trunk (e.g. df8e736700ae2a06675ff50381788d708bc22b96)
> * change 1st line in {{corruptionCausesFailure()}} to {{
> qt().withFixedSeed(43595190254702L)}}
> * run test
> Fails with:
> {code}
> java.lang.AssertionError: Property falsified after 617 example(s)
> Smallest found falsifying value(s) :-
>
[jira] [Created] (CASSANDRA-15786) ChecksummingTransformerTest#corruptionCausesFailure fails for seed 43595190254702
Robert Stupp created CASSANDRA-15786:
Summary: ChecksummingTransformerTest#corruptionCausesFailure fails
for seed 43595190254702
Key: CASSANDRA-15786
URL: https://issues.apache.org/jira/browse/CASSANDRA-15786
Project: Cassandra
Issue Type: Bug
Reporter: Robert Stupp
Reproducer:
* latest trunk (e.g. df8e736700ae2a06675ff50381788d708bc22b96)
* change 1st line in {{corruptionCausesFailure()}} to {{
qt().withFixedSeed(43595190254702L)}}
* run test
Fails with:
{code}
java.lang.AssertionError: Property falsified after 617 example(s)
Smallest found falsifying value(s) :-
[jira] [Updated] (CASSANDRA-15781) Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and apply the same "fix" to other tests
[
https://issues.apache.org/jira/browse/CASSANDRA-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15781:
-
Status: Ready to Commit (was: Review In Progress)
Thanks!
Committed as
[df8e736700ae2a06675ff50381788d708bc22b96|https://github.com/apache/cassandra/commit/df8e736700ae2a06675ff50381788d708bc22b96]
to [trunk|https://github.com/apache/cassandra/tree/trunk].
> Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and
> apply the same "fix" to other tests
> -
>
> Key: CASSANDRA-15781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15781
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The unit test {{DecayingEstimatedHistogramReservoirTest#testStriping}} is
> flaky, especially when many tests are running concurrently, because the used
> thread-pool can still be running when the assertions are tested (i.e. the
> {{ExecutorService.awaitTermination()}} returned false).
> The "fix" is rather a band aid: extend the wait-period to 1 minute, check the
> return value of {{awaitTermination}} and apply the same change to some other
> tests that _might_ be affected as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15781) Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and apply the same "fix" to other tests
[
https://issues.apache.org/jira/browse/CASSANDRA-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15781:
-
Source Control Link:
https://github.com/apache/cassandra/commit/df8e736700ae2a06675ff50381788d708bc22b96
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and
> apply the same "fix" to other tests
> -
>
> Key: CASSANDRA-15781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15781
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The unit test {{DecayingEstimatedHistogramReservoirTest#testStriping}} is
> flaky, especially when many tests are running concurrently, because the used
> thread-pool can still be running when the assertions are tested (i.e. the
> {{ExecutorService.awaitTermination()}} returned false).
> The "fix" is rather a band aid: extend the wait-period to 1 minute, check the
> return value of {{awaitTermination}} and apply the same change to some other
> tests that _might_ be affected as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15781) Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and apply the same "fix" to other tests
[
https://issues.apache.org/jira/browse/CASSANDRA-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15781:
-
Reviewers: Eduard Tudenhoefner, Robert Stupp (was: Eduard Tudenhoefner)
Status: Review In Progress (was: Patch Available)
> Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and
> apply the same "fix" to other tests
> -
>
> Key: CASSANDRA-15781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15781
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The unit test {{DecayingEstimatedHistogramReservoirTest#testStriping}} is
> flaky, especially when many tests are running concurrently, because the used
> thread-pool can still be running when the assertions are tested (i.e. the
> {{ExecutorService.awaitTermination()}} returned false).
> The "fix" is rather a band aid: extend the wait-period to 1 minute, check the
> return value of {{awaitTermination}} and apply the same change to some other
> tests that _might_ be affected as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15502) In Tree Tooling with Java 11
[ https://issues.apache.org/jira/browse/CASSANDRA-15502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17098774#comment-17098774 ] Robert Stupp commented on CASSANDRA-15502: -- I've looked into this and all the tools are exercised in the unit-tests and/or dtests. Looks like we're good here? > In Tree Tooling with Java 11 > > > Key: CASSANDRA-15502 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15502 > Project: Cassandra > Issue Type: Task > Components: Test/dtest, Test/fuzz, Tool/bulk load, Tool/cqlsh, > Tool/diff, Tool/fql, Tool/nodetool, Tool/sstable, Tool/stress >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > This is to cover testing the various tools running on java 11. > The scope of this testing is manual testing and not automated, different > JIRAs should cover automation testing these tool. > The tools in question are: nodetool, sstableloader, sstablescrub, > sstableupgrade, sstableutil, sstableverify, fqltool, stress, auditlogviewer, > compaction-stress, sstabledump, sstableexpiredblockers, sstablemetadata, > sstableofflinerelevel, sstablesplit, and sstablerepairedset (many of these > may be tested already in dtest) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15781) Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and apply the same "fix" to other tests
[
https://issues.apache.org/jira/browse/CASSANDRA-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15781:
-
Test and Documentation Plan: -
Status: Patch Available (was: Open)
> Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and
> apply the same "fix" to other tests
> -
>
> Key: CASSANDRA-15781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15781
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The unit test {{DecayingEstimatedHistogramReservoirTest#testStriping}} is
> flaky, especially when many tests are running concurrently, because the used
> thread-pool can still be running when the assertions are tested (i.e. the
> {{ExecutorService.awaitTermination()}} returned false).
> The "fix" is rather a band aid: extend the wait-period to 1 minute, check the
> return value of {{awaitTermination}} and apply the same change to some other
> tests that _might_ be affected as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15781) Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and apply the same "fix" to other tests
[
https://issues.apache.org/jira/browse/CASSANDRA-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15781:
-
Change Category: Quality Assurance
Complexity: Low Hanging Fruit
Fix Version/s: 4.0
Reviewers: Eduard Tudenhoefner
Status: Open (was: Triage Needed)
[Github PR here|https://github.com/apache/cassandra/pull/581]
> Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and
> apply the same "fix" to other tests
> -
>
> Key: CASSANDRA-15781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15781
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The unit test {{DecayingEstimatedHistogramReservoirTest#testStriping}} is
> flaky, especially when many tests are running concurrently, because the used
> thread-pool can still be running when the assertions are tested (i.e. the
> {{ExecutorService.awaitTermination()}} returned false).
> The "fix" is rather a band aid: extend the wait-period to 1 minute, check the
> return value of {{awaitTermination}} and apply the same change to some other
> tests that _might_ be affected as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15781) Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and apply the same "fix" to other tests
Robert Stupp created CASSANDRA-15781:
Summary: Fix flakiness of
DecayingEstimatedHistogramReservoirTest#testStriping and apply the same "fix"
to other tests
Key: CASSANDRA-15781
URL: https://issues.apache.org/jira/browse/CASSANDRA-15781
Project: Cassandra
Issue Type: Improvement
Components: Test/unit
Reporter: Robert Stupp
Assignee: Robert Stupp
The unit test {{DecayingEstimatedHistogramReservoirTest#testStriping}} is
flaky, especially when many tests are running concurrently, because the used
thread-pool can still be running when the assertions are tested (i.e. the
{{ExecutorService.awaitTermination()}} returned false).
The "fix" is rather a band aid: extend the wait-period to 1 minute, check the
return value of {{awaitTermination}} and apply the same change to some other
tests that _might_ be affected as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15780:
-
Reviewers: Eduard Tudenhoefner
[Github PR here|https://github.com/apache/cassandra/pull/580]
> Less test stderr spam when running tests w/ java 11
> ---
>
> Key: CASSANDRA-15780
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> There are a bunch of tests that, when run with Java 11, emit the boilerplate
> {{WARNING: An illegal reflective access operation has occurred}} messages.
> The proposed change just adds more invocations to
> {{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to
> get rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15780:
-
Test and Documentation Plan: -
Status: Patch Available (was: Open)
> Less test stderr spam when running tests w/ java 11
> ---
>
> Key: CASSANDRA-15780
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> There are a bunch of tests that, when run with Java 11, emit the boilerplate
> {{WARNING: An illegal reflective access operation has occurred}} messages.
> The proposed change just adds more invocations to
> {{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to
> get rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15780:
-
Change Category: Quality Assurance
Complexity: Low Hanging Fruit
Fix Version/s: 4.0
Status: Open (was: Triage Needed)
> Less test stderr spam when running tests w/ java 11
> ---
>
> Key: CASSANDRA-15780
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> There are a bunch of tests that, when run with Java 11, emit the boilerplate
> {{WARNING: An illegal reflective access operation has occurred}} messages.
> The proposed change just adds more invocations to
> {{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to
> get rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
Robert Stupp created CASSANDRA-15780:
Summary: Less test stderr spam when running tests w/ java 11
Key: CASSANDRA-15780
URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
Project: Cassandra
Issue Type: Improvement
Components: Test/unit
Reporter: Robert Stupp
Assignee: Robert Stupp
There are a bunch of tests that, when run with Java 11, emit the boilerplate
{{WARNING: An illegal reflective access operation has occurred}} messages.
The proposed change just adds more invocations to
{{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to get
rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15751) CustomNowInSecondsTest.testSelectQuery() is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15751:
-
Source Control Link:
https://github.com/apache/cassandra/commit/7f0df2e5e55ccad7ae23ed74e3caabba53d63d5e
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Thanks!
Committed as
[7f0df2e5e55ccad7ae23ed74e3caabba53d63d5e|https://github.com/apache/cassandra/commit/7f0df2e5e55ccad7ae23ed74e3caabba53d63d5e]
to [trunk|https://github.com/apache/cassandra/tree/trunk].
> CustomNowInSecondsTest.testSelectQuery() is flaky
> -
>
> Key: CASSANDRA-15751
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15751
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The test {{CustomNowInSecondsTest.testSelectQuery()}} is flaky, because the
> {{now}} [calculated
> here|https://github.com/snazy/cassandra/blob/5242f7d4306f034f32e26834df06ff7fc85a3c16/test/unit/org/apache/cassandra/cql3/CustomNowInSecondsTest.java#L57]
> but [used
> here|https://github.com/snazy/cassandra/blob/5242f7d4306f034f32e26834df06ff7fc85a3c16/test/unit/org/apache/cassandra/cql3/CustomNowInSecondsTest.java#L70]
> can be wrong, if the thread running the test method gets stalled.
> The flakiness can be simulated by adding a {{Thread.sleep(1000)}} before the
> {{executeModify()}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15751) CustomNowInSecondsTest.testSelectQuery() is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15751:
-
PR: https://github.com/apache/cassandra/pull/556
> CustomNowInSecondsTest.testSelectQuery() is flaky
> -
>
> Key: CASSANDRA-15751
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15751
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The test {{CustomNowInSecondsTest.testSelectQuery()}} is flaky, because the
> {{now}} [calculated
> here|https://github.com/snazy/cassandra/blob/5242f7d4306f034f32e26834df06ff7fc85a3c16/test/unit/org/apache/cassandra/cql3/CustomNowInSecondsTest.java#L57]
> but [used
> here|https://github.com/snazy/cassandra/blob/5242f7d4306f034f32e26834df06ff7fc85a3c16/test/unit/org/apache/cassandra/cql3/CustomNowInSecondsTest.java#L70]
> can be wrong, if the thread running the test method gets stalled.
> The flakiness can be simulated by adding a {{Thread.sleep(1000)}} before the
> {{executeModify()}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15751) CustomNowInSecondsTest.testSelectQuery() is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15751:
-
Test and Documentation Plan: -
Status: Patch Available (was: In Progress)
> CustomNowInSecondsTest.testSelectQuery() is flaky
> -
>
> Key: CASSANDRA-15751
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15751
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The test {{CustomNowInSecondsTest.testSelectQuery()}} is flaky, because the
> {{now}} [calculated
> here|https://github.com/snazy/cassandra/blob/5242f7d4306f034f32e26834df06ff7fc85a3c16/test/unit/org/apache/cassandra/cql3/CustomNowInSecondsTest.java#L57]
> but [used
> here|https://github.com/snazy/cassandra/blob/5242f7d4306f034f32e26834df06ff7fc85a3c16/test/unit/org/apache/cassandra/cql3/CustomNowInSecondsTest.java#L70]
> can be wrong, if the thread running the test method gets stalled.
> The flakiness can be simulated by adding a {{Thread.sleep(1000)}} before the
> {{executeModify()}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15751) CustomNowInSecondsTest.testSelectQuery() is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15751:
-
Bug Category: Parent values: Code(13163)
Complexity: Low Hanging Fruit
Component/s: Test/unit
Discovered By: Unit Test
Severity: Normal
Status: Open (was: Triage Needed)
> CustomNowInSecondsTest.testSelectQuery() is flaky
> -
>
> Key: CASSANDRA-15751
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15751
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The test {{CustomNowInSecondsTest.testSelectQuery()}} is flaky, because the
> {{now}} [calculated
> here|https://github.com/snazy/cassandra/blob/5242f7d4306f034f32e26834df06ff7fc85a3c16/test/unit/org/apache/cassandra/cql3/CustomNowInSecondsTest.java#L57]
> but [used
> here|https://github.com/snazy/cassandra/blob/5242f7d4306f034f32e26834df06ff7fc85a3c16/test/unit/org/apache/cassandra/cql3/CustomNowInSecondsTest.java#L70]
> can be wrong, if the thread running the test method gets stalled.
> The flakiness can be simulated by adding a {{Thread.sleep(1000)}} before the
> {{executeModify()}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15751) CustomNowInSecondsTest.testSelectQuery() is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15751:
-
Since Version: 4.0-alpha
> CustomNowInSecondsTest.testSelectQuery() is flaky
> -
>
> Key: CASSANDRA-15751
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15751
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The test {{CustomNowInSecondsTest.testSelectQuery()}} is flaky, because the
> {{now}} [calculated
> here|https://github.com/snazy/cassandra/blob/5242f7d4306f034f32e26834df06ff7fc85a3c16/test/unit/org/apache/cassandra/cql3/CustomNowInSecondsTest.java#L57]
> but [used
> here|https://github.com/snazy/cassandra/blob/5242f7d4306f034f32e26834df06ff7fc85a3c16/test/unit/org/apache/cassandra/cql3/CustomNowInSecondsTest.java#L70]
> can be wrong, if the thread running the test method gets stalled.
> The flakiness can be simulated by adding a {{Thread.sleep(1000)}} before the
> {{executeModify()}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15751) CustomNowInSecondsTest.testSelectQuery() is flaky
Robert Stupp created CASSANDRA-15751:
Summary: CustomNowInSecondsTest.testSelectQuery() is flaky
Key: CASSANDRA-15751
URL: https://issues.apache.org/jira/browse/CASSANDRA-15751
Project: Cassandra
Issue Type: Bug
Reporter: Robert Stupp
Assignee: Robert Stupp
The test {{CustomNowInSecondsTest.testSelectQuery()}} is flaky, because the
{{now}} [calculated
here|https://github.com/snazy/cassandra/blob/5242f7d4306f034f32e26834df06ff7fc85a3c16/test/unit/org/apache/cassandra/cql3/CustomNowInSecondsTest.java#L57]
but [used
here|https://github.com/snazy/cassandra/blob/5242f7d4306f034f32e26834df06ff7fc85a3c16/test/unit/org/apache/cassandra/cql3/CustomNowInSecondsTest.java#L70]
can be wrong, if the thread running the test method gets stalled.
The flakiness can be simulated by adding a {{Thread.sleep(1000)}} before the
{{executeModify()}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15751) CustomNowInSecondsTest.testSelectQuery() is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15751:
-
Fix Version/s: 4.0
> CustomNowInSecondsTest.testSelectQuery() is flaky
> -
>
> Key: CASSANDRA-15751
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15751
> Project: Cassandra
> Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The test {{CustomNowInSecondsTest.testSelectQuery()}} is flaky, because the
> {{now}} [calculated
> here|https://github.com/snazy/cassandra/blob/5242f7d4306f034f32e26834df06ff7fc85a3c16/test/unit/org/apache/cassandra/cql3/CustomNowInSecondsTest.java#L57]
> but [used
> here|https://github.com/snazy/cassandra/blob/5242f7d4306f034f32e26834df06ff7fc85a3c16/test/unit/org/apache/cassandra/cql3/CustomNowInSecondsTest.java#L70]
> can be wrong, if the thread running the test method gets stalled.
> The flakiness can be simulated by adding a {{Thread.sleep(1000)}} before the
> {{executeModify()}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
