[jira] [Updated] (CASSANDRA-13047) Point cqlsh help to the new doc
[ https://issues.apache.org/jira/browse/CASSANDRA-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Bannister updated CASSANDRA-13047: -- Attachment: (was: 13047-3.11.txt) > Point cqlsh help to the new doc > --- > > Key: CASSANDRA-13047 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13047 > Project: Cassandra > Issue Type: Bug > Components: Legacy/CQL >Reporter: Sylvain Lebresne >Assignee: Patrick Bannister >Priority: Major > Fix For: 4.0 > > > Cqlsh still points to the "old" textile CQL doc, but that's not really > maintain anymore now that we have the new doc (which include everything the > old doc had and more). We should update cqlsh to point to the new doc so we > can remove the old one completely. > Any takers? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13047) Point cqlsh help to the new doc
[ https://issues.apache.org/jira/browse/CASSANDRA-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Bannister updated CASSANDRA-13047: -- Status: Open (was: Patch Available) Looking at this in light of CASSANDRA-13047. Currently we have two versions of the docs: 3.11 for 3.11 and older, latest for 4.0. I anticipate the versions will be a moving target going forward. I'm thinking the answer to this is to have the versioned docs website redirect requests for all version numbers to the appropriate versioned docs for that version number. > Point cqlsh help to the new doc > --- > > Key: CASSANDRA-13047 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13047 > Project: Cassandra > Issue Type: Bug > Components: Legacy/CQL >Reporter: Sylvain Lebresne >Assignee: Patrick Bannister >Priority: Major > Fix For: 4.0 > > Attachments: 13047-3.11.txt > > > Cqlsh still points to the "old" textile CQL doc, but that's not really > maintain anymore now that we have the new doc (which include everything the > old doc had and more). We should update cqlsh to point to the new doc so we > can remove the old one completely. > Any takers? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15002) Avoid leaking threads when anticompaction fails
[ https://issues.apache.org/jira/browse/CASSANDRA-15002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-15002: Status: Ready to Commit (was: Patch Available) > Avoid leaking threads when anticompaction fails > --- > > Key: CASSANDRA-15002 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15002 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Major > Fix For: 4.0 > > > If anticompaction fails on a node, a message is sent to all repair > participants that this session is failed. If the other participants > successfully finish their anticompactions we will not shut down the executor > in `LocalSessions` since we can't change the state from "FAILED" to > "PREPARED" and throw exception before calling shutdown. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15002) Avoid leaking threads when anticompaction fails
[ https://issues.apache.org/jira/browse/CASSANDRA-15002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756718#comment-16756718 ] Blake Eggleston commented on CASSANDRA-15002: - +1 > Avoid leaking threads when anticompaction fails > --- > > Key: CASSANDRA-15002 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15002 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Major > Fix For: 4.0 > > > If anticompaction fails on a node, a message is sent to all repair > participants that this session is failed. If the other participants > successfully finish their anticompactions we will not shut down the executor > in `LocalSessions` since we can't change the state from "FAILED" to > "PREPARED" and throw exception before calling shutdown. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14871) Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser
[
https://issues.apache.org/jira/browse/CASSANDRA-14871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756703#comment-16756703
]
Blake Eggleston commented on CASSANDRA-14871:
-
[~snazy] go for it
> Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser
> ---
>
> Key: CASSANDRA-14871
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14871
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Critical
> Fix For: 4.0, 3.0.x, 3.11.x
>
>
> There are a couple of places in the code base that do not respect that
> j.u.HashMap + related classes are not thread safe and some parts rely on
> internals of the implementation of HM, which can change.
> We have observed failures like {{NullPointerException}} and
> {{ConcurrentModificationException}} as well as wrong behavior.
> Affected areas in the code base:
> * {{SizeTieredCompactionStrategy}}
> * {{DateTieredCompactionStrategy}}
> * {{TimeWindowCompactionStrategy}}
> * {{TokenMetadata.Topology}}
> * {{TypeParser}}
> * streaming / concurrent access to {{LifecycleTransaction}} (handled in
> CASSANDRA-14554)
> While the patches for the compaction strategies + {{TypeParser}} are pretty
> straight forward, the patch for {{TokenMetadata.Topology}} requires it to be
> made immutable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14344) Support filtering using IN restrictions
[
https://issues.apache.org/jira/browse/CASSANDRA-14344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756386#comment-16756386
]
Venkata Harikrishna Nukala commented on CASSANDRA-14344:
[~blerer] Updated the patch ( [^14344-trunk-3.patch] ) and uploaded to this
ticket.
> Support filtering using IN restrictions
> ---
>
> Key: CASSANDRA-14344
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14344
> Project: Cassandra
> Issue Type: New Feature
> Components: Legacy/CQL
>Reporter: Dikang Gu
>Assignee: Venkata Harikrishna Nukala
>Priority: Major
> Fix For: 4.x
>
> Attachments: 14344-trunk-2.txt, 14344-trunk-3.patch,
> 14344-trunk-inexpression-approach-2.txt,
> 14344-trunk-inexpression-approach.txt, 14344-trunk.txt
>
>
> Support IN filter query like this:
>
> CREATE TABLE ks1.t1 (
> key int,
> col1 int,
> col2 int,
> value int,
> PRIMARY KEY (key, col1, col2)
> ) WITH CLUSTERING ORDER BY (col1 ASC, col2 ASC)
>
> cqlsh:ks1> select * from t1 where key = 1 and col2 in (1) allow filtering;
>
> key | col1 | col2 | value
> -+--+--+---
> 1 | 1 | 1 | 1
> 1 | 2 | 1 | 3
>
> (2 rows)
> cqlsh:ks1> select * from t1 where key = 1 and col2 in (1, 2) allow filtering;
> *{color:#ff}InvalidRequest: Error from server: code=2200 [Invalid query]
> message="IN restrictions are not supported on indexed columns"{color}*
> cqlsh:ks1>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14344) Support filtering using IN restrictions
[
https://issues.apache.org/jira/browse/CASSANDRA-14344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Harikrishna Nukala updated CASSANDRA-14344:
---
Attachment: 14344-trunk-3.patch
> Support filtering using IN restrictions
> ---
>
> Key: CASSANDRA-14344
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14344
> Project: Cassandra
> Issue Type: New Feature
> Components: Legacy/CQL
>Reporter: Dikang Gu
>Assignee: Venkata Harikrishna Nukala
>Priority: Major
> Fix For: 4.x
>
> Attachments: 14344-trunk-2.txt, 14344-trunk-3.patch,
> 14344-trunk-inexpression-approach-2.txt,
> 14344-trunk-inexpression-approach.txt, 14344-trunk.txt
>
>
> Support IN filter query like this:
>
> CREATE TABLE ks1.t1 (
> key int,
> col1 int,
> col2 int,
> value int,
> PRIMARY KEY (key, col1, col2)
> ) WITH CLUSTERING ORDER BY (col1 ASC, col2 ASC)
>
> cqlsh:ks1> select * from t1 where key = 1 and col2 in (1) allow filtering;
>
> key | col1 | col2 | value
> -+--+--+---
> 1 | 1 | 1 | 1
> 1 | 2 | 1 | 3
>
> (2 rows)
> cqlsh:ks1> select * from t1 where key = 1 and col2 in (1, 2) allow filtering;
> *{color:#ff}InvalidRequest: Error from server: code=2200 [Invalid query]
> message="IN restrictions are not supported on indexed columns"{color}*
> cqlsh:ks1>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15002) Avoid leaking threads when anticompaction fails
[ https://issues.apache.org/jira/browse/CASSANDRA-15002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15002: Status: Patch Available (was: Open) Patch: https://github.com/krummas/cassandra/commits/marcuse/15002 Tests: https://circleci.com/gh/krummas/workflows/cassandra/tree/marcuse%2F15002 The branch contains 2 commits, first one fixes the thread leak and retries acquiring sstables for 1 minute, the second commit adds rate limiting to anticompaction and makes anticompactions stoppable again (should have been done in CASSANDRA-14935). The reasoning behind adding retries when trying to acquire sstables is that if there is an sstable [0, 100] and the user submits a repair on [0, 50], we will anticompact the sstable in to two new sstables [0, 50], [51, 100] - if a user submits a repair on [51, 100] while the first anticompaction is executing, we would fail immediately, since the sstable the new repair requires is busy anticompacting. By retrying to acquire sstables we give the first anticompaction a chance to finish and the new repair would be able to grab the new [51, 100] sstable - it does not need to wait for the whole first repair to finish. > Avoid leaking threads when anticompaction fails > --- > > Key: CASSANDRA-15002 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15002 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Major > Fix For: 4.0 > > > If anticompaction fails on a node, a message is sent to all repair > participants that this session is failed. If the other participants > successfully finish their anticompactions we will not shut down the executor > in `LocalSessions` since we can't change the state from "FAILED" to > "PREPARED" and throw exception before calling shutdown. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15002) Avoid leaking threads when anticompaction fails
Marcus Eriksson created CASSANDRA-15002: --- Summary: Avoid leaking threads when anticompaction fails Key: CASSANDRA-15002 URL: https://issues.apache.org/jira/browse/CASSANDRA-15002 Project: Cassandra Issue Type: Bug Components: Consistency/Repair Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 4.0 If anticompaction fails on a node, a message is sent to all repair participants that this session is failed. If the other participants successfully finish their anticompactions we will not shut down the executor in `LocalSessions` since we can't change the state from "FAILED" to "PREPARED" and throw exception before calling shutdown. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14922) In JVM dtests need to clean up after instance shutdown
[
https://issues.apache.org/jira/browse/CASSANDRA-14922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-14922:
Attachment: Screen Shot 2019-01-30 at 15.46.35.png
> In JVM dtests need to clean up after instance shutdown
> --
>
> Key: CASSANDRA-14922
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14922
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Joseph Lynch
>Assignee: Joseph Lynch
>Priority: Minor
> Fix For: 4.0
>
> Attachments: AllThreadsStopped.png, ClassLoadersRetaining.png,
> LeakedNativeMemory.png, Leaking_Metrics_On_Shutdown.png,
> MainClassRetaining.png, MemoryReclaimedFix.png,
> Metaspace_Actually_Collected.png, OnlyThreeRootsLeft.png, Screen Shot
> 2019-01-30 at 15.46.35.png, Screen Shot 2019-01-30 at 15.47.13.png,
> no_more_references.png
>
>
> Currently the unit tests are failing on circleci ([example
> one|https://circleci.com/gh/jolynch/cassandra/300#tests/containers/1],
> [example
> two|https://circleci.com/gh/rustyrazorblade/cassandra/44#tests/containers/1])
> because we use a small container (medium) for unit tests by default and the
> in JVM dtests are leaking a few hundred megabytes of memory per test right
> now. This is not a big deal because the dtest runs with the larger containers
> continue to function fine as well as local testing as the number of in JVM
> dtests is not yet high enough to cause a problem with more than 2GB of
> available heap. However we should fix the memory leak so that going forwards
> we can add more in JVM dtests without worry.
> I've been working with [~ifesdjeen] to debug, and the issue appears to be
> unreleased Table/Keyspace metrics (screenshot showing the leak attached). I
> believe that we have a few potential issues that are leading to the leaks:
> 1. The
> [{{Instance::shutdown}}|https://github.com/apache/cassandra/blob/f22fec927de7ac29120c2f34de5b8cc1c695/test/distributed/org/apache/cassandra/distributed/Instance.java#L328-L354]
> method is not successfully cleaning up all the metrics created by the
> {{CassandraMetricsRegistry}}
> 2. The
> [{{TestCluster::close}}|https://github.com/apache/cassandra/blob/f22fec927de7ac29120c2f34de5b8cc1c695/test/distributed/org/apache/cassandra/distributed/TestCluster.java#L283]
> method is not waiting for all the instances to finish shutting down and
> cleaning up before continuing on
> 3. I'm not sure if this is an issue assuming we clear all metrics, but
> [{{TableMetrics::release}}|https://github.com/apache/cassandra/blob/4ae229f5cd270c2b43475b3f752a7b228de260ea/src/java/org/apache/cassandra/metrics/TableMetrics.java#L951]
> does not release all the metric references (which could leak them)
> I am working on a patch which shuts down everything and assures that we do
> not leak memory.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14922) In JVM dtests need to clean up after instance shutdown
[
https://issues.apache.org/jira/browse/CASSANDRA-14922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-14922:
Attachment: Screen Shot 2019-01-30 at 15.47.13.png
> In JVM dtests need to clean up after instance shutdown
> --
>
> Key: CASSANDRA-14922
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14922
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Joseph Lynch
>Assignee: Joseph Lynch
>Priority: Minor
> Fix For: 4.0
>
> Attachments: AllThreadsStopped.png, ClassLoadersRetaining.png,
> LeakedNativeMemory.png, Leaking_Metrics_On_Shutdown.png,
> MainClassRetaining.png, MemoryReclaimedFix.png,
> Metaspace_Actually_Collected.png, OnlyThreeRootsLeft.png, Screen Shot
> 2019-01-30 at 15.47.13.png, no_more_references.png
>
>
> Currently the unit tests are failing on circleci ([example
> one|https://circleci.com/gh/jolynch/cassandra/300#tests/containers/1],
> [example
> two|https://circleci.com/gh/rustyrazorblade/cassandra/44#tests/containers/1])
> because we use a small container (medium) for unit tests by default and the
> in JVM dtests are leaking a few hundred megabytes of memory per test right
> now. This is not a big deal because the dtest runs with the larger containers
> continue to function fine as well as local testing as the number of in JVM
> dtests is not yet high enough to cause a problem with more than 2GB of
> available heap. However we should fix the memory leak so that going forwards
> we can add more in JVM dtests without worry.
> I've been working with [~ifesdjeen] to debug, and the issue appears to be
> unreleased Table/Keyspace metrics (screenshot showing the leak attached). I
> believe that we have a few potential issues that are leading to the leaks:
> 1. The
> [{{Instance::shutdown}}|https://github.com/apache/cassandra/blob/f22fec927de7ac29120c2f34de5b8cc1c695/test/distributed/org/apache/cassandra/distributed/Instance.java#L328-L354]
> method is not successfully cleaning up all the metrics created by the
> {{CassandraMetricsRegistry}}
> 2. The
> [{{TestCluster::close}}|https://github.com/apache/cassandra/blob/f22fec927de7ac29120c2f34de5b8cc1c695/test/distributed/org/apache/cassandra/distributed/TestCluster.java#L283]
> method is not waiting for all the instances to finish shutting down and
> cleaning up before continuing on
> 3. I'm not sure if this is an issue assuming we clear all metrics, but
> [{{TableMetrics::release}}|https://github.com/apache/cassandra/blob/4ae229f5cd270c2b43475b3f752a7b228de260ea/src/java/org/apache/cassandra/metrics/TableMetrics.java#L951]
> does not release all the metric references (which could leak them)
> I am working on a patch which shuts down everything and assures that we do
> not leak memory.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13651) Large amount of CPU used by epoll_wait(.., .., .., 0)
[
https://issues.apache.org/jira/browse/CASSANDRA-13651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CASSANDRA-13651:
---
Labels: pull-request-available (was: )
> Large amount of CPU used by epoll_wait(.., .., .., 0)
> -
>
> Key: CASSANDRA-13651
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13651
> Project: Cassandra
> Issue Type: Bug
>Reporter: Corentin Chary
>Assignee: Michael Burman
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0
>
> Attachments: cpu-usage.png
>
>
> I was trying to profile Cassandra under my workload and I kept seeing this
> backtrace:
> {code}
> epollEventLoopGroup-2-3 State: RUNNABLE CPU usage on sample: 240ms
> io.netty.channel.epoll.Native.epollWait0(int, long, int, int) Native.java
> (native)
> io.netty.channel.epoll.Native.epollWait(int, EpollEventArray, int)
> Native.java:111
> io.netty.channel.epoll.EpollEventLoop.epollWait(boolean)
> EpollEventLoop.java:230
> io.netty.channel.epoll.EpollEventLoop.run() EpollEventLoop.java:254
> io.netty.util.concurrent.SingleThreadEventExecutor$5.run()
> SingleThreadEventExecutor.java:858
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run()
> DefaultThreadFactory.java:138
> java.lang.Thread.run() Thread.java:745
> {code}
> At fist I though that the profiler might not be able to profile native code
> properly, but I wen't further and I realized that most of the CPU was used by
> {{epoll_wait()}} calls with a timeout of zero.
> Here is the output of perf on this system, which confirms that most of the
> overhead was with timeout == 0.
> {code}
> Samples: 11M of event 'syscalls:sys_enter_epoll_wait', Event count (approx.):
> 11594448
> Overhead Trace output
>
> ◆
> 90.06% epfd: 0x0047, events: 0x7f5588c0c000, maxevents: 0x2000,
> timeout: 0x
> ▒
>5.77% epfd: 0x00b5, events: 0x7fca419ef000, maxevents: 0x1000,
> timeout: 0x
> ▒
>1.98% epfd: 0x00b5, events: 0x7fca419ef000, maxevents: 0x1000,
> timeout: 0x03e8
> ▒
>0.04% epfd: 0x0003, events: 0x2f6af77b9c00, maxevents: 0x0020,
> timeout: 0x
> ▒
>0.04% epfd: 0x002b, events: 0x121ebf63ac00, maxevents: 0x0040,
> timeout: 0x
> ▒
>0.03% epfd: 0x0026, events: 0x7f51f80019c0, maxevents: 0x0020,
> timeout: 0x
> ▒
>0.02% epfd: 0x0003, events: 0x7fe4d80019d0, maxevents: 0x0020,
> timeout: 0x
> {code}
> Running this time with perf record -ag for call traces:
> {code}
> # Children Self sys usr Trace output
>
> #
>
> #
> 8.61% 8.61% 0.00% 8.61% epfd: 0x00a7, events:
> 0x7fca452d6000, maxevents: 0x1000, timeout: 0x
> |
> ---0x1000200af313
>|
> --8.61%--0x7fca6117bdac
> 0x7fca60459804
> epoll_wait
> 2.98% 2.98% 0.00% 2.98% epfd: 0x00a7, events:
> 0x7fca452d6000, maxevents: 0x1000, timeout: 0x03e8
> |
> ---0x1000200af313
>0x7fca6117b830
>0x7fca60459804
>epoll_wait
> {code}
> That looks like a lot of CPU used to wait for nothing. I'm not sure if pref
> reports a per-CPU percentage or a per-system percentage, but that would be
> still be 10% of the total CPU usage of Cassandra at the minimum.
> I went further and found the code of all that: We schedule a lot of
> {{Message::Flusher}} with a deadline of 10 usec (5 per messages I think) but
> netty+epoll only support timeouts above the milliseconds and will convert
>
