[jira] [Commented] (CASSANDRA-16114) Fix tests CQLTester.assertLastSchemaChange causes ClassCastException

2020-10-26 Thread Berenguer Blasi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221158#comment-17221158
 ] 

Berenguer Blasi commented on CASSANDRA-16114:
-

[~dcapwell] there is a new proposal in the PR from [~blerer] in case you missed 
the GH update #collborating

> Fix tests CQLTester.assertLastSchemaChange causes ClassCastException
> 
>
> Key: CASSANDRA-16114
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16114
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: David Capwell
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> Build: 
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/494/workflows/b3765545-7b09-48dd-85ff-830c4f348329/jobs/2681
> {code}
> java.lang.ClassCastException: 
> org.apache.cassandra.transport.messages.ResultMessage$Void cannot be cast to 
> org.apache.cassandra.transport.messages.ResultMessage$SchemaChange
>   at 
> org.apache.cassandra.cql3.CQLTester.assertLastSchemaChange(CQLTester.java:916)
>   at 
> org.apache.cassandra.cql3.validation.entities.UFTest.testSchemaChange(UFTest.java:94)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16227) Nodetool Netstats and tablestats tests

2020-10-26 Thread Berenguer Blasi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221154#comment-17221154
 ] 

Berenguer Blasi commented on CASSANDRA-16227:
-

Adding some notes for the reviewer:
- TableStats: Under `test/unit/org/apache/cassandra/tools/nodetool/stats` the 
tool internals are tested extensively already
- NetStats: It's a simple app and there isn't much to test besides making sure 
it behaves and responds to the user interface as expected.

> Nodetool Netstats and tablestats tests
> --
>
> Key: CASSANDRA-16227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16227
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> As part of the CASSANDRA-15583 effort we're adding unit tests for netstats 
> and tablestats



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16171) Remove Windows scripts

2020-10-26 Thread Berenguer Blasi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221150#comment-17221150
 ] 

Berenguer Blasi commented on CASSANDRA-16171:
-

Hi [~yukim] do you think you'll have time to move this forward or do you mind 
if I take over to take it across the finish line?

> Remove Windows scripts
> --
>
> Key: CASSANDRA-16171
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16171
> Project: Cassandra
>  Issue Type: Task
>  Components: Packaging
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Normal
> Fix For: 4.0-rc
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> As per the email thread in cassandra-dev mailing list[1], remove windows 
> scripts from Cassandra 4.0 onwards, due to the lack of maintenance and tests.
> 1: https://www.mail-archive.com/dev@cassandra.apache.org/msg15583.html 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16220) python dtest pending_range_test.py::TestPendingRangeMovements::test_pending_range (@pytest.mark.resource_intensive) fails on trunk

2020-10-26 Thread Berenguer Blasi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221149#comment-17221149
 ] 

Berenguer Blasi commented on CASSANDRA-16220:
-

SGTM if you prefer to do in another ticket. Can you get a committer to run that 
dtest branch for you so we have CI?

> python dtest 
> pending_range_test.py::TestPendingRangeMovements::test_pending_range 
> (@pytest.mark.resource_intensive) fails on trunk
> --
>
> Key: CASSANDRA-16220
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16220
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: David Capwell
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> The test has the following section
> {code}
> if cluster.version() >= '2.2':
>   node2.watch_log_for('127.0.0.1 state moving', timeout=10, 
> filename='debug.log’)
> {code}
> The issue is that in trunk we have the port attached to the log, so the log 
> is now
> {code}
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,104 StorageService.java:2452 - Node 
> /127.0.0.1:7000 state MOVING, tokens [-9223372036854775808]
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,105 StorageService.java:2670 - Node 
> /127.0.0.1:7000 state moving, new token -634023222112864484
> {code}
> Since the log now contains the port, this test always times out on trunk when 
> it hits this
> {code}
> self = 
>  @pytest.mark.resource_intensive
> def test_pending_range(self):
> """
> @jira_ticket CASSANDRA-10887
> """
> cluster = self.cluster
> # If we are on 2.1, we need to set the log level to debug or higher, 
> as debug.log does not exist.
> if cluster.version() < '2.2':
> cluster.set_log_level('DEBUG')
>
> # Create 5 node cluster
> cluster.populate(5).start()
> node1, node2 = cluster.nodelist()[0:2]
>
> # Set up RF=3 keyspace
> session = self.patient_cql_connection(node1)
> create_ks(session, 'ks', 3)
>
> session.execute("CREATE TABLE users (login text PRIMARY KEY, email 
> text, name text, login_count int)")
>
> # We use the partition key 'jdoe3' because it belongs to node1.
> # The key MUST belong to node1 to repro the bug.
> session.execute("INSERT INTO users (login, email, name, login_count) 
> VALUES ('jdoe3', 'j...@abc.com', 'Jane Doe', 1) IF NOT EXISTS;")
>
> lwt_query = SimpleStatement("UPDATE users SET email = 
> 'jane...@abc.com' WHERE login = 'jdoe3' IF email = 'j...@abc.com'")
>
> # Show we can execute LWT no problem
> for i in range(1000):
> session.execute(lwt_query)
>
> token = '-634023222112864484'
>
> mark = node1.mark_log()
>
> # Move a node
> node1.nodetool('move {}'.format(token))
>
> # Watch the log so we know when the node is moving
> node1.watch_log_for('Moving .* to {}'.format(token), timeout=10, 
> from_mark=mark)
> node1.watch_log_for('Sleeping 3 ms before start 
> streaming/fetching ranges', timeout=10, from_mark=mark)
>
> if cluster.version() >= '2.2':
> >   node2.watch_log_for('127.0.0.1 state moving', timeout=10, 
> > filename='debug.log')
>  pending_range_test.py:57: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
>  self = 
> exprs = '127.0.0.1 state moving', from_mark = None, timeout = 10, process = 
> None
> verbose = False, filename = 'debug.log'
>  def watch_log_for(self, exprs, from_mark=None, timeout=600, 
> process=None, verbose=False, filename='system.log'):
> """
> Watch the log until one or more (regular) expressions are found 
> or timeouts (a
> TimeoutError is then raised). On successful completion, a list of 
> pair (line matched,
> match object) is returned.
> """
> start = time.time()
> tofind = [exprs] if isinstance(exprs, string_types) else exprs
> tofind = [re.compile(e) for e in tofind]
> matchings = []
> reads = ""
> if len(tofind) == 0:
> return None
>
> log_file = os.path.join(self.log_directory(), filename)
> output_read = False
> while not os.path.exists(log_file):
> time.sleep(.5)
> if start + timeout < time.time():
> raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S", 
> time.gmtime()) + " [" + self.name + "] Timed out waiting for {} to be 
> created.".format(log_file))
> if process and not output_read:
> 

[jira] [Commented] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info

2020-10-26 Thread Kornel Pal (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221100#comment-17221100
 ] 

Kornel Pal commented on CASSANDRA-16226:


If I understand correctly, only upgraded tables are affected, so new tables (or 
even INSERTs to upgraded tables?) are not affected. Could the migration code be 
fixed so that row timestamps are properly generated? While the proposed 
solution improves performance for migrated tables, also seems to take away the 
performance benefits of max timestamps from COMPACT STORAGE tables. Fixing the 
issue for newly upgraded and rebuilt tables and recommending {{nodetool 
upgradesstables -a}} for fixing the performance issue on already migrated 
tables might result in even better performance improvements.

> COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by 
> timestamp due to missing primary key liveness info
> ---
>
> Key: CASSANDRA-16226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16226
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Local Write-Read Paths
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
>  Labels: perfomance, upgrade
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This was discovered while tracking down a spike in the number of  SSTables 
> per read for a COMPACT STORAGE table after a 2.1 -> 3.0 upgrade. Before 3.0, 
> there is no direct analog of 3.0's primary key liveness info. When we upgrade 
> 2.1 COMPACT STORAGE SSTables to the mf format, we simply don't write row 
> timestamps, even if the original mutations were INSERTs. On read, when we 
> look at SSTables in order from newest to oldest max timestamp, we expect to 
> have this primary key liveness information to determine whether we can skip 
> older SSTables after finding completely populated rows.
> ex. I have three SSTables in a COMPACT STORAGE table with max timestamps 
> 1000, 2000, and 3000. There are many rows in a particular partition, making 
> filtering on the min and max clustering effectively a no-op. All data is 
> inserted, and there are no partial updates. A fully specified row with 
> timestamp 2500 exists in the SSTable with a max timestamp of 3000. With a 
> proper row timestamp in hand, we can easily ignore the SSTables w/ max 
> timestamps of 1000 and 2000. Without it, we read 3 SSTables instead of 1, 
> which likely means a significant performance regression. 
> The following test illustrates this difference in behavior between 2.1 and 
> 3.0:
> https://github.com/maedhroz/cassandra/commit/84ce9242bedd735ca79d4f06007d127de6a82800
> A solution here might be as simple as having 
> {{SinglePartitionReadCommand#canRemoveRow()}} only inspect primary key 
> liveness information for non-compact/CQL tables. Tombstones seem to be 
> handled at a level above that anyway. (One potential problem with that is 
> whether or not the distinction will continue to exist in 4.0, and dropping 
> compact storage from a table doesn't magically make pk liveness information 
> appear.)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-16115) New Cassandra website design, content and layout to work with Antora

2020-10-26 Thread Anthony Grasso (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221037#comment-17221037
 ] 

Anthony Grasso edited comment on CASSANDRA-16115 at 10/26/20, 11:47 PM:


Hi [~mklogan] ,
 Unless I've missed something, the proposal sounds very similar to what we are 
doing with Antora. As part of CASSANDRA-16066, I have started developing a 
prototype which does the static content generation using Antora. Currently, it 
does the following transformations:
 * *Cassandra Code -> AsciiDoc* (generate AsciiDoc files using code in 
Cassandra repository)
 * *AsciiDoc -> HTML* (AsciiDoc files sourced from Cassandra and Cassandra 
Website repositories)

In the above case AsciiDoc files are generated from the Cassandra repository. 
To generate the HTML for the website Antora uses the AsciiDoc files from both 
the Cassandra and the Cassandra Website repositories. The AsciiDoc files are in 
a plain-text format similar to markdown.
 The HTML content generated by Antora is styled using predefined HTML and 
JavaScript templates. This allows for the separation of concerns highlighted in 
your above proposal. To explicitly define the separation, the prototype I am 
developing for the Cassandra Website repository uses the following two top 
level directories:
 * */cassandra-website/site-content*
 * */cassandra-website/site-ui*

*site-content* directory has the AssciDoc files containing the content to be 
published on the site. Any changes to wording on the website would be made to 
the files in here. The content would be unrelated to a particular Cassandra 
release version. All content related to a Cassandra release version e.g. 
manuals, technical docs, etc would be generated from the Cassandra repository. 
In this case, any changes to the technical documentation for a particular 
Cassandra version would made in the Cassandra repository.

*site-ui* directory has the HTML templates and JavaScript that defines the 
presentation of the content. Any website styling or behaviour changes would be 
made to the files in here.
 Antora's real power is being able to generate content for multiple versions of 
a document. In our case we have a version of the Cassandra documentation for 
each release. Hence, Antora will allow us to generate the documents for each of 
these versions. Given we need Antora to create the versioned documentation, we 
may as well use it for the non-versioned content as well. The reason is reduce 
the the number of tools we need to perform the HTML generation. As illustrated 
above we can easily split the content from the styling for the non-versioned 
documentation.

Let me know if what I've written makes sense or if I've missed something 
crucial in your proposal. Let's catch up off-ticket when you have time to go 
through the details and if it can be made simpler?


was (Author: anthony grasso):
Hi Melissa,
Unless I've missed something, the proposal sounds very similar to what we are 
doing with Antora. As part of CASSANDRA-16066, I have started developing a 
prototype which does the static content generation using Antora. Currently, it 
does the following transformations:

* *Cassandra Code -> AsciiDoc* (generate AsciiDoc files using code in Cassandra 
repository)
* *AsciiDoc -> HTML* (AsciiDoc files sourced from Cassandra and Cassandra 
Website repositories)

In the above case AsciiDoc files are generated from the Cassandra repository. 
To generate the HTML for the website Antora uses the AsciiDoc files from both 
the Cassandra and the Cassandra Website repositories. The AsciiDoc files are in 
a plain-text format similar to markdown.
The HTML content generated by Antora is styled using predefined HTML and 
JavaScript templates. This allows for the separation of concerns highlighted in 
your above proposal. To explicitly define the separation, the prototype I am 
developing for the Cassandra Website repository uses the following two top 
level directories:

* */cassandra-website/site-content*
* */cassandra-website/site-ui*

*site-content* directory has the AssciDoc files containing the content to be 
published on the site. Any changes to wording on the website would be made to 
the files in here. The content would be unrelated to a particular Cassandra 
release version. All content related to a Cassandra release version e.g. 
manuals, technical docs, etc would be generated from the Cassandra repository. 
In this case, any changes to the technical documentation for a particular 
Cassandra version would made in the Cassandra repository.

*site-ui* directory has the HTML templates and JavaScript that defines the 
presentation of the content. Any website styling or behaviour changes would be 
made to the files in here.
Antora's real power is being able to generate content for multiple versions of 
a document. In our case we have a version of the Cassandra documentation for 

[jira] [Commented] (CASSANDRA-16115) New Cassandra website design, content and layout to work with Antora

2020-10-26 Thread Anthony Grasso (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221037#comment-17221037
 ] 

Anthony Grasso commented on CASSANDRA-16115:


Hi Melissa,
Unless I've missed something, the proposal sounds very similar to what we are 
doing with Antora. As part of CASSANDRA-16066, I have started developing a 
prototype which does the static content generation using Antora. Currently, it 
does the following transformations:

* *Cassandra Code -> AsciiDoc* (generate AsciiDoc files using code in Cassandra 
repository)
* *AsciiDoc -> HTML* (AsciiDoc files sourced from Cassandra and Cassandra 
Website repositories)

In the above case AsciiDoc files are generated from the Cassandra repository. 
To generate the HTML for the website Antora uses the AsciiDoc files from both 
the Cassandra and the Cassandra Website repositories. The AsciiDoc files are in 
a plain-text format similar to markdown.
The HTML content generated by Antora is styled using predefined HTML and 
JavaScript templates. This allows for the separation of concerns highlighted in 
your above proposal. To explicitly define the separation, the prototype I am 
developing for the Cassandra Website repository uses the following two top 
level directories:

* */cassandra-website/site-content*
* */cassandra-website/site-ui*

*site-content* directory has the AssciDoc files containing the content to be 
published on the site. Any changes to wording on the website would be made to 
the files in here. The content would be unrelated to a particular Cassandra 
release version. All content related to a Cassandra release version e.g. 
manuals, technical docs, etc would be generated from the Cassandra repository. 
In this case, any changes to the technical documentation for a particular 
Cassandra version would made in the Cassandra repository.

*site-ui* directory has the HTML templates and JavaScript that defines the 
presentation of the content. Any website styling or behaviour changes would be 
made to the files in here.
Antora's real power is being able to generate content for multiple versions of 
a document. In our case we have a version of the Cassandra documentation for 
each release. Hence, Antora will allow us to generate the documents for each of 
these versions. Given we need Antora to create the versioned documentation, we 
may as well use it for the non-versioned content as well. The reason is reduce 
the the number of tools we need to perform the HTML generation. As illustrated 
above we can easily split the content from the styling for the non-versioned 
documentation.

Let me know if what I've written makes sense or if I've missed something 
crucial in your proposal. Let's catch up off-ticket when you have time to go 
through the details and if it can be made simpler?

> New Cassandra website design, content and layout to work with Antora
> 
>
> Key: CASSANDRA-16115
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16115
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Melissa Logan
>Assignee: Melissa Logan
>Priority: Normal
> Fix For: 4.0.x
>
> Attachments: Screen Shot 2020-09-03 at 09.48.53.png
>
>
> This task is related to CASSANDRA-16066 (Update and rework the 
> cassandra-website material to work with Antora). The goal is to update the 
> front-end of the C* website (design, IA and content) to work with Antora to 
> help modernize the website as discussed on the [mailing 
> list|https://www.mail-archive.com/dev@cassandra.apache.org/msg15537.html].
> *Design Concepts:* A minimum of two homepage design concepts will be created 
> and shared for input, which will help standardize a brand palette for C* and 
> a design language for the site. This may include custom iconography and 
> graphics. The chosen design language will be used to develop the remaining 
> templates. 
> *Template Design*: It's estimated that 7 template designs will be needed 
> including the creation of several new pages: 
>  * Homepage template
>  * Toplevel template - e.g. Community.
>  * General template - Mostly textual with some images, e.g. Intro, Quickstart 
>  * “Library” template - A library of assets (links, downloads, logos etc) 
> that are sortable by metadata, e.g Resources, or Kafka's Powered By page).
>  * Blog landing template 
>  * Blog single template
>  * Docs template 
> *Website Content:* Along with new design will be a need for new or updated 
> content to fit the new page layouts. The intention is to use as much as 
> possible from existing content, and augment with new content where needed.
> *Template Development:* This includes the frontend development, such as any 
> HTML markup to achieve designs. HTML would be 

[jira] [Commented] (CASSANDRA-16103) Invalid serialized size for responses

2020-10-26 Thread Yifan Cai (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221013#comment-17221013
 ] 

Yifan Cai commented on CASSANDRA-16103:
---

Fixed the test and addressed the comments in the GH PR. 

CI results looks good. (Unit test, jvm dtest, dtest and upgrade test)

https://app.circleci.com/pipelines/github/yifan-c/cassandra/136/workflows/b267771a-dd3f-40c2-9ec7-611bf81f5b20

> Invalid serialized size for responses
> -
>
> Key: CASSANDRA-16103
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16103
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: David Capwell
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Was running a benchmark at LOCAL_ONE and eventually saw the below exception; 
> this is related to CASSANDRA-16097 as it was found during the same test.
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized 
> size; expected 14, actual size at least 13, for verb HINT_RSP
>   at 
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
>   at 
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
>   at 
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
>   at 
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
>   at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
>   at 
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>   at 
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>   at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized 
> size; expected 14, actual size at least 13, for verb MUTATION_RSP
>   at 
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
>   at 
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
>   at 
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
>   at 
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
>   at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
>   at 
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>   at 
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>   at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta

2020-10-26 Thread Sam Tunnicliffe (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220921#comment-17220921
 ] 

Sam Tunnicliffe commented on CASSANDRA-15299:
-

Here are some numbers from running tlp-stress against 4.0-beta2 vs the 15299 
branch {{70923bae4}} with protocol v4 & v5.
The dataset was sized to be mostly in-memory and reading/writing at {{CL.ONE}}. 
Each workload ran for 10 minutes, with a 1m warmup, on a 3 node cluster. 

Both latency and throughput are pretty much on a par, with v5 showing a bit of 
an improvement where the workload is amenable to compression. 
Memory usage and GC were pretty much the same too. If anything, during the v5 
runs the servers spent less time in GC, but this was so close as not be 
significant.

There are definitely some places we can improve the performance of v5, and I 
don't think anything here indicates a serious regression in either v4 or v5 
performance. Given the additional integrity checks in v5, I think not 
regressing meets the perf bar here, at least in the first instance.


h4. 100% read workload, reads a single row at a time. 

{code}
Count  Latency (p99)  1min (req/s) 
4.0-beta2 V4135449878  72.43 224222.86
15299 V4138424112   68.3 229765.45 
15299 V5137618437  70.35 231602.36 
4.0-beta2 V4 LZ4103348953  66.68 173437.38
15299 V4 LZ4105114560  68.83 176192.36
15299 V5 LZ4131833462  70.19 222092.99

{code}

h4. Mixed r/w workload (50/50). K/V with blobs upto 65k

{code}
   Count  Latency (p99)  1min (req/s) |  Count  
Latency (p99)  1min (req/s) 
4.0-beta2 V434455009  76.59  56557.78 |   34464217  
74.85  56546.40
15299 V433368171  74.90  54859.94 |   33361940  
67.77  54848.57 
15299 V532991815  76.00  54780.74 |   33001153  
76.72  54856.99
4.0-beta2 V4 LZ432152220  83.41  53306.03 |   32147206  
83.22  53334.00
15299 V4 LZ431158895  71.01  51106.60 |   31153453  
72.75  51087.01
15299 V5 LZ432634296  75.73  54370.71 |   32644765  
76.70  54396.15

{code}

h4. Mixed r/w/d workload (60/30/10). Wide rows with values up to 200b and 
slicing on both the inserts and deletions.

{code}
   Count  Latency (p99)  1min (req/s) |   Count  
Latency (p99)  1min (req/s) |   Count  Latency (p99)  1min (req/s)
4.0-beta2 V418725688 197.47  25377.16 | 9357971 
193.86  12687.94 | 3117394 178.44   4221.90
15299 V417975636 185.88  24160.78 | 8986125 
184.97  12087.13 | 2995562 179.99   4023.88
15299 V518429252 192.91  25349.35 | 9223277 
188.15  12678.46 | 3073312 184.24   4232.23
4.0-beta2 V4 LZ418407719 179.94  25160.16 | 9197664 
178.25  12575.03 | 3068134 180.90   4195.38
15299 V4 LZ417678994 171.39  24073.09 | 8842952 
196.06  12064.18 | 2947344 170.53   4026.37 
15299 V5 LZ418274085 208.57  25127.02 | 9138491 
163.23  12558.28 | 3045264 203.54   4188.88 

{code}


> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol 
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Sam Tunnicliffe
>Priority: Normal
>  Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it 
> introduced checksumming/CRC32 to request and response bodies. It’s an 
> important step forward, but it doesn’t cover the entire stream. In 
> particular, the message header is not covered by a checksum or a crc, which 
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of 
> computing the CRC32 on the bytes written on the wire - losing the properties 
> of the CRC32. In some cases, due to this sequencing, attempting to decompress 
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order, 
> also losing some of 

[jira] [Updated] (CASSANDRA-16166) Rename master branch to trunk in cassandra-dtest

2020-10-26 Thread Michael Semb Wever (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-16166:
---
Reviewers: Brandon Williams  (was: Brandon Williams, Michael Semb Wever)

> Rename master branch to trunk in cassandra-dtest
> 
>
> Key: CASSANDRA-16166
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16166
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Build
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16166) Rename master branch to trunk in cassandra-dtest

2020-10-26 Thread Michael Semb Wever (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-16166:
---
Reviewers: Brandon Williams, Michael Semb Wever
   Status: Review In Progress  (was: Patch Available)

> Rename master branch to trunk in cassandra-dtest
> 
>
> Key: CASSANDRA-16166
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16166
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Build
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16166) Rename master branch to trunk in cassandra-dtest

2020-10-26 Thread Michael Semb Wever (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220893#comment-17220893
 ] 

Michael Semb Wever commented on CASSANDRA-16166:


Jenkins: 
https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/135/
CircleCI: 
https://app.circleci.com/pipelines/github/michaelsembwever/cassandra/3/workflows/b8029024-1ace-47f7-ae72-4cc3892c1a08/jobs/79/tests

> Rename master branch to trunk in cassandra-dtest
> 
>
> Key: CASSANDRA-16166
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16166
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Build
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16213) Cannot replace_address /X because it doesn't exist in gossip

2020-10-26 Thread David Capwell (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220881#comment-17220881
 ] 

David Capwell commented on CASSANDRA-16213:
---

Thanks for the review [~paulo]

bq.  I would just limit propagating state about the downed host only during 
shadow gossip response

I will test this out

bq. I'm not very familiar with in-jvm dtest infrastructure so it would be nice 
to get a pair of eyes on that to make sure the framework changes look good.

Agree.  Jvm-dtest forked a lot of CassandraDaemon so took a while to make sure 
the logic matched, this was a good chunk of this patch =(.

> Cannot replace_address /X because it doesn't exist in gossip
> 
>
> Key: CASSANDRA-16213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16213
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Gossip, Cluster/Membership
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> We see this exception around nodes crashing and trying to do a host 
> replacement; this error appears to be correlated around multiple node 
> failures.
> A simplified case to trigger this is the following
> *) Have a N node cluster
> *) Shutdown all N nodes
> *) Bring up N-1 nodes (at least 1 seed, else replace seed)
> *) Host replace the N-1th node -> this will fail with the above
> The reason this happens is that the N-1th node isn’t gossiping anymore, and 
> the existing nodes do not have its details in gossip (but have the details in 
> the peers table), so the host replacement fails as the node isn’t known in 
> gossip.
> This affects all versions (tested 3.0 and trunk, assume 2.2 as well)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16221) Jenkins doesn't run high resource python dtests with novonode so missing some tests

2020-10-26 Thread David Capwell (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220876#comment-17220876
 ] 

David Capwell commented on CASSANDRA-16221:
---

backlogged for the next 2 weeks, so won't likely get to the python dtest 
changes for at least 2 weeks; if anyone wants to take up go for it, else ill 
come back to this later.

> Jenkins doesn't run high resource python dtests with novonode so missing some 
> tests
> ---
>
> Key: CASSANDRA-16221
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16221
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Build, CI
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Low
> Fix For: NA
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In CASSANDRA-16220 it was found that a test is failing on trunk, and its 
> because of the address to address with port changes; looking closer into it 
> it was found that Jenkins ignores the novnode case which causes this test to 
> get skipped.
> We should enable novnode as well to match the rest of the pipeline, but also 
> to handle some of the missing tests not currently run.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15241) Virtual table to expose current running queries

2020-10-26 Thread Chris Lohfink (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lohfink updated CASSANDRA-15241:
--
Fix Version/s: (was: 4.0)

> Virtual table to expose current running queries
> ---
>
> Key: CASSANDRA-15241
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15241
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Feature/Virtual Tables
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
>
> Expose current running queries and their duration.
> {code}cqlsh> select * from system_views.queries;
>  thread_id| duration_micros | task
> --+-+-
>  Native-Transport-Requests-17 |6325 |  QUERY 
> select * from system_views.queries; [pageSize = 100]
>   Native-Transport-Requests-4 |   14681 | EXECUTE 
> f4115f91190d4acf09e452637f1f2444 with 0 values at consistency LOCAL_ONE
>   Native-Transport-Requests-6 |   14678 | EXECUTE 
> f4115f91190d4acf09e452637f1f2444 with 0 values at consistency LOCAL_ONE
>  ReadStage-10 |   16535 | 
>SELECT * FROM basic.wide1 LIMIT 5000
>  ReadStage-13 |   16535 | 
>SELECT * FROM basic.wide1 LIMIT 5000
>  ReadStage-14 |   16535 | 
>SELECT * FROM basic.wide1 LIMIT 5000
>  ReadStage-19 |   11861 | 
>SELECT * FROM basic.wide1 LIMIT 5000
>  ReadStage-20 |   11861 | 
>SELECT * FROM basic.wide1 LIMIT 5000
>  ReadStage-22 |7279 | 
>SELECT * FROM basic.wide1 LIMIT 5000
>  ReadStage-23 |4716 | 
>SELECT * FROM basic.wide1 LIMIT 5000
>   ReadStage-5 |   16535 | 
>SELECT * FROM basic.wide1 LIMIT 5000
>   ReadStage-7 |   16535 | 
>SELECT * FROM basic.wide1 LIMIT 5000
>   ReadStage-8 |   16535 | 
>SELECT * FROM basic.wide1 LIMIT 5000{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info

2020-10-26 Thread Caleb Rackliffe (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-16226:

Test and Documentation Plan: 
The patch includes a new in-JVM upgrade test that exercises the 2.2 -> 3.0 
upgrade path were an existing table uses COMPACT STORAGE and we hit the 
particular scenario outlined in the description. Outside that, the interesting 
parts of the regression suite are the tests around row deletions and cell 
deletions where only a primary key remains.

Performance should also be improved significantly in this scenario, so a smoke 
test to verify that would be helpful. (There should be zero impact on 
non-compact tables.)
 Status: Patch Available  (was: In Progress)

[3.0 patch|https://github.com/apache/cassandra/pull/789], 
[CircleCI|https://app.circleci.com/pipelines/github/maedhroz/cassandra/137/workflows/40098e98-f383-4bcd-ba01-be9a11a90190]

> COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by 
> timestamp due to missing primary key liveness info
> ---
>
> Key: CASSANDRA-16226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16226
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Local Write-Read Paths
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
>  Labels: perfomance, upgrade
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This was discovered while tracking down a spike in the number of  SSTables 
> per read for a COMPACT STORAGE table after a 2.1 -> 3.0 upgrade. Before 3.0, 
> there is no direct analog of 3.0's primary key liveness info. When we upgrade 
> 2.1 COMPACT STORAGE SSTables to the mf format, we simply don't write row 
> timestamps, even if the original mutations were INSERTs. On read, when we 
> look at SSTables in order from newest to oldest max timestamp, we expect to 
> have this primary key liveness information to determine whether we can skip 
> older SSTables after finding completely populated rows.
> ex. I have three SSTables in a COMPACT STORAGE table with max timestamps 
> 1000, 2000, and 3000. There are many rows in a particular partition, making 
> filtering on the min and max clustering effectively a no-op. All data is 
> inserted, and there are no partial updates. A fully specified row with 
> timestamp 2500 exists in the SSTable with a max timestamp of 3000. With a 
> proper row timestamp in hand, we can easily ignore the SSTables w/ max 
> timestamps of 1000 and 2000. Without it, we read 3 SSTables instead of 1, 
> which likely means a significant performance regression. 
> The following test illustrates this difference in behavior between 2.1 and 
> 3.0:
> https://github.com/maedhroz/cassandra/commit/84ce9242bedd735ca79d4f06007d127de6a82800
> A solution here might be as simple as having 
> {{SinglePartitionReadCommand#canRemoveRow()}} only inspect primary key 
> liveness information for non-compact/CQL tables. Tombstones seem to be 
> handled at a level above that anyway. (One potential problem with that is 
> whether or not the distinction will continue to exist in 4.0, and dropping 
> compact storage from a table doesn't magically make pk liveness information 
> appear.)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable

2020-10-26 Thread Ekaterina Dimitrova (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220778#comment-17220778
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15897:
-

[~marcuse], [~aleksey], [~ifesdjeen], I was wondering whether you had the time 
to look at this one? 

> Dropping compact storage with 2.1-sstables on disk make them unreadable
> ---
>
> Key: CASSANDRA-15897
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15897
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Local Write-Read Paths
>Reporter: Marcus Eriksson
>Assignee: Sylvain Lebresne
>Priority: Normal
> Fix For: 3.0.x, 4.0-beta3
>
>
> Test reproducing: 
> https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info

2020-10-26 Thread Caleb Rackliffe (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-16226:

Labels: perfomance upgrade  (was: )

> COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by 
> timestamp due to missing primary key liveness info
> ---
>
> Key: CASSANDRA-16226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16226
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Local Write-Read Paths
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
>  Labels: perfomance, upgrade
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This was discovered while tracking down a spike in the number of  SSTables 
> per read for a COMPACT STORAGE table after a 2.1 -> 3.0 upgrade. Before 3.0, 
> there is no direct analog of 3.0's primary key liveness info. When we upgrade 
> 2.1 COMPACT STORAGE SSTables to the mf format, we simply don't write row 
> timestamps, even if the original mutations were INSERTs. On read, when we 
> look at SSTables in order from newest to oldest max timestamp, we expect to 
> have this primary key liveness information to determine whether we can skip 
> older SSTables after finding completely populated rows.
> ex. I have three SSTables in a COMPACT STORAGE table with max timestamps 
> 1000, 2000, and 3000. There are many rows in a particular partition, making 
> filtering on the min and max clustering effectively a no-op. All data is 
> inserted, and there are no partial updates. A fully specified row with 
> timestamp 2500 exists in the SSTable with a max timestamp of 3000. With a 
> proper row timestamp in hand, we can easily ignore the SSTables w/ max 
> timestamps of 1000 and 2000. Without it, we read 3 SSTables instead of 1, 
> which likely means a significant performance regression. 
> The following test illustrates this difference in behavior between 2.1 and 
> 3.0:
> https://github.com/maedhroz/cassandra/commit/84ce9242bedd735ca79d4f06007d127de6a82800
> A solution here might be as simple as having 
> {{SinglePartitionReadCommand#canRemoveRow()}} only inspect primary key 
> liveness information for non-compact/CQL tables. Tombstones seem to be 
> handled at a level above that anyway. (One potential problem with that is 
> whether or not the distinction will continue to exist in 4.0, and dropping 
> compact storage from a table doesn't magically make pk liveness information 
> appear.)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16220) python dtest pending_range_test.py::TestPendingRangeMovements::test_pending_range (@pytest.mark.resource_intensive) fails on trunk

2020-10-26 Thread Ekaterina Dimitrova (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220751#comment-17220751
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16220:
-

As we talked in Slack, updating the mapping is not applicable to re.search(). 
On a further read actually it is good to update  assert_regexp_matches() to 
assertRegex().

I suggest this to be done in a follow up ticket and to leave this one with the 
current patch. (in this case it doesn't matter whether we use 
assert_regexp_matches() or re.search() I think?)

 

 

 

> python dtest 
> pending_range_test.py::TestPendingRangeMovements::test_pending_range 
> (@pytest.mark.resource_intensive) fails on trunk
> --
>
> Key: CASSANDRA-16220
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16220
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: David Capwell
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> The test has the following section
> {code}
> if cluster.version() >= '2.2':
>   node2.watch_log_for('127.0.0.1 state moving', timeout=10, 
> filename='debug.log’)
> {code}
> The issue is that in trunk we have the port attached to the log, so the log 
> is now
> {code}
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,104 StorageService.java:2452 - Node 
> /127.0.0.1:7000 state MOVING, tokens [-9223372036854775808]
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,105 StorageService.java:2670 - Node 
> /127.0.0.1:7000 state moving, new token -634023222112864484
> {code}
> Since the log now contains the port, this test always times out on trunk when 
> it hits this
> {code}
> self = 
>  @pytest.mark.resource_intensive
> def test_pending_range(self):
> """
> @jira_ticket CASSANDRA-10887
> """
> cluster = self.cluster
> # If we are on 2.1, we need to set the log level to debug or higher, 
> as debug.log does not exist.
> if cluster.version() < '2.2':
> cluster.set_log_level('DEBUG')
>
> # Create 5 node cluster
> cluster.populate(5).start()
> node1, node2 = cluster.nodelist()[0:2]
>
> # Set up RF=3 keyspace
> session = self.patient_cql_connection(node1)
> create_ks(session, 'ks', 3)
>
> session.execute("CREATE TABLE users (login text PRIMARY KEY, email 
> text, name text, login_count int)")
>
> # We use the partition key 'jdoe3' because it belongs to node1.
> # The key MUST belong to node1 to repro the bug.
> session.execute("INSERT INTO users (login, email, name, login_count) 
> VALUES ('jdoe3', 'j...@abc.com', 'Jane Doe', 1) IF NOT EXISTS;")
>
> lwt_query = SimpleStatement("UPDATE users SET email = 
> 'jane...@abc.com' WHERE login = 'jdoe3' IF email = 'j...@abc.com'")
>
> # Show we can execute LWT no problem
> for i in range(1000):
> session.execute(lwt_query)
>
> token = '-634023222112864484'
>
> mark = node1.mark_log()
>
> # Move a node
> node1.nodetool('move {}'.format(token))
>
> # Watch the log so we know when the node is moving
> node1.watch_log_for('Moving .* to {}'.format(token), timeout=10, 
> from_mark=mark)
> node1.watch_log_for('Sleeping 3 ms before start 
> streaming/fetching ranges', timeout=10, from_mark=mark)
>
> if cluster.version() >= '2.2':
> >   node2.watch_log_for('127.0.0.1 state moving', timeout=10, 
> > filename='debug.log')
>  pending_range_test.py:57: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
>  self = 
> exprs = '127.0.0.1 state moving', from_mark = None, timeout = 10, process = 
> None
> verbose = False, filename = 'debug.log'
>  def watch_log_for(self, exprs, from_mark=None, timeout=600, 
> process=None, verbose=False, filename='system.log'):
> """
> Watch the log until one or more (regular) expressions are found 
> or timeouts (a
> TimeoutError is then raised). On successful completion, a list of 
> pair (line matched,
> match object) is returned.
> """
> start = time.time()
> tofind = [exprs] if isinstance(exprs, string_types) else exprs
> tofind = [re.compile(e) for e in tofind]
> matchings = []
> reads = ""
> if len(tofind) == 0:
> return None
>
> log_file = os.path.join(self.log_directory(), filename)
> output_read = False
> while not os.path.exists(log_file):
> time.sleep(.5)
> if start + 

[jira] [Assigned] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info

2020-10-26 Thread Caleb Rackliffe (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe reassigned CASSANDRA-16226:
---

Assignee: Caleb Rackliffe

> COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by 
> timestamp due to missing primary key liveness info
> ---
>
> Key: CASSANDRA-16226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16226
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Local Write-Read Paths
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> This was discovered while tracking down a spike in the number of  SSTables 
> per read for a COMPACT STORAGE table after a 2.1 -> 3.0 upgrade. Before 3.0, 
> there is no direct analog of 3.0's primary key liveness info. When we upgrade 
> 2.1 COMPACT STORAGE SSTables to the mf format, we simply don't write row 
> timestamps, even if the original mutations were INSERTs. On read, when we 
> look at SSTables in order from newest to oldest max timestamp, we expect to 
> have this primary key liveness information to determine whether we can skip 
> older SSTables after finding completely populated rows.
> ex. I have three SSTables in a COMPACT STORAGE table with max timestamps 
> 1000, 2000, and 3000. There are many rows in a particular partition, making 
> filtering on the min and max clustering effectively a no-op. All data is 
> inserted, and there are no partial updates. A fully specified row with 
> timestamp 2500 exists in the SSTable with a max timestamp of 3000. With a 
> proper row timestamp in hand, we can easily ignore the SSTables w/ max 
> timestamps of 1000 and 2000. Without it, we read 3 SSTables instead of 1, 
> which likely means a significant performance regression. 
> The following test illustrates this difference in behavior between 2.1 and 
> 3.0:
> https://github.com/maedhroz/cassandra/commit/84ce9242bedd735ca79d4f06007d127de6a82800
> A solution here might be as simple as having 
> {{SinglePartitionReadCommand#canRemoveRow()}} only inspect primary key 
> liveness information for non-compact/CQL tables. Tombstones seem to be 
> handled at a level above that anyway. (One potential problem with that is 
> whether or not the distinction will continue to exist in 4.0, and dropping 
> compact storage from a table doesn't magically make pk liveness information 
> appear.)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16227) Nodetool Netstats and tablestats tests

2020-10-26 Thread Brandon Williams (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16227:
-
Summary: Nodetool Netstats and tablestats tests  (was: Nodetool Netstatas 
and tablestats tests)

> Nodetool Netstats and tablestats tests
> --
>
> Key: CASSANDRA-16227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16227
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> As part of the CASSANDRA-15583 effort we're adding unit tests for netstats 
> and tablestats



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16166) Rename master branch to trunk in cassandra-dtest

2020-10-26 Thread Brandon Williams (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220717#comment-17220717
 ] 

Brandon Williams commented on CASSANDRA-16166:
--

+1 on the patches.

> Rename master branch to trunk in cassandra-dtest
> 
>
> Key: CASSANDRA-16166
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16166
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Build
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-12126) CAS Reads Inconsistencies

2020-10-26 Thread Benjamin Lerer (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-12126:
---
Status: Ready to Commit  (was: Review In Progress)

> CAS Reads Inconsistencies 
> --
>
> Key: CASSANDRA-12126
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12126
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Lightweight Transactions, Legacy/Coordination
>Reporter: Sankalp Kohli
>Assignee: Sylvain Lebresne
>Priority: Normal
>  Labels: LWT, pull-request-available
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> While looking at the CAS code in Cassandra, I found a potential issue with 
> CAS Reads. Here is how it can happen with RF=3
> 1) You issue a CAS Write and it fails in the propose phase. A machine replies 
> true to a propose and saves the commit in accepted filed. The other two 
> machines B and C does not get to the accept phase. 
> Current state is that machine A has this commit in paxos table as accepted 
> but not committed and B and C does not. 
> 2) Issue a CAS Read and it goes to only B and C. You wont be able to read the 
> value written in step 1. This step is as if nothing is inflight. 
> 3) Issue another CAS Read and it goes to A and B. Now we will discover that 
> there is something inflight from A and will propose and commit it with the 
> current ballot. Now we can read the value written in step 1 as part of this 
> CAS read.
> If we skip step 3 and instead run step 4, we will never learn about value 
> written in step 1. 
> 4. Issue a CAS Write and it involves only B and C. This will succeed and 
> commit a different value than step 1. Step 1 value will never be seen again 
> and was never seen before. 
> If you read the Lamport “paxos made simple” paper and read section 2.3. It 
> talks about this issue which is how learners can find out if majority of the 
> acceptors have accepted the proposal. 
> In step 3, it is correct that we propose the value again since we dont know 
> if it was accepted by majority of acceptors. When we ask majority of 
> acceptors, and more than one acceptors but not majority has something in 
> flight, we have no way of knowing if it is accepted by majority of acceptors. 
> So this behavior is correct. 
> However we need to fix step 2, since it caused reads to not be linearizable 
> with respect to writes and other reads. In this case, we know that majority 
> of acceptors have no inflight commit which means we have majority that 
> nothing was accepted by majority. I think we should run a propose step here 
> with empty commit and that will cause write written in step 1 to not be 
> visible ever after. 
> With this fix, we will either see data written in step 1 on next serial read 
> or will never see it which is what we want. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-12126) CAS Reads Inconsistencies

2020-10-26 Thread Benjamin Lerer (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-12126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220716#comment-17220716
 ] 

Benjamin Lerer commented on CASSANDRA-12126:


Sorry, for the delay. The patches looks good.

> CAS Reads Inconsistencies 
> --
>
> Key: CASSANDRA-12126
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12126
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Lightweight Transactions, Legacy/Coordination
>Reporter: Sankalp Kohli
>Assignee: Sylvain Lebresne
>Priority: Normal
>  Labels: LWT, pull-request-available
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> While looking at the CAS code in Cassandra, I found a potential issue with 
> CAS Reads. Here is how it can happen with RF=3
> 1) You issue a CAS Write and it fails in the propose phase. A machine replies 
> true to a propose and saves the commit in accepted filed. The other two 
> machines B and C does not get to the accept phase. 
> Current state is that machine A has this commit in paxos table as accepted 
> but not committed and B and C does not. 
> 2) Issue a CAS Read and it goes to only B and C. You wont be able to read the 
> value written in step 1. This step is as if nothing is inflight. 
> 3) Issue another CAS Read and it goes to A and B. Now we will discover that 
> there is something inflight from A and will propose and commit it with the 
> current ballot. Now we can read the value written in step 1 as part of this 
> CAS read.
> If we skip step 3 and instead run step 4, we will never learn about value 
> written in step 1. 
> 4. Issue a CAS Write and it involves only B and C. This will succeed and 
> commit a different value than step 1. Step 1 value will never be seen again 
> and was never seen before. 
> If you read the Lamport “paxos made simple” paper and read section 2.3. It 
> talks about this issue which is how learners can find out if majority of the 
> acceptors have accepted the proposal. 
> In step 3, it is correct that we propose the value again since we dont know 
> if it was accepted by majority of acceptors. When we ask majority of 
> acceptors, and more than one acceptors but not majority has something in 
> flight, we have no way of knowing if it is accepted by majority of acceptors. 
> So this behavior is correct. 
> However we need to fix step 2, since it caused reads to not be linearizable 
> with respect to writes and other reads. In this case, we know that majority 
> of acceptors have no inflight commit which means we have majority that 
> nothing was accepted by majority. I think we should run a propose step here 
> with empty commit and that will cause write written in step 1 to not be 
> visible ever after. 
> With this fix, we will either see data written in step 1 on next serial read 
> or will never see it which is what we want. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-13767) update a row which was inserted with 'IF NOT EXISTS' key word will fail siently

2020-10-26 Thread Benjamin Lerer (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13767:
---
Resolution: Not A Problem
Status: Resolved  (was: Open)

> update a row which was inserted with 'IF NOT EXISTS' key word will fail 
> siently
> ---
>
> Key: CASSANDRA-13767
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13767
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Lightweight Transactions, Legacy/CQL
> Environment: [cqlsh 5.0.1 | Cassandra 3.11.0 | CQL spec 3.4.4 | 
> Native protocol v4]
> cassandra python driver = 3.5.0
> Run in dcoker with the following:
> docker run --name cassandra -v /data/cassandra:/var/lib/cassandra
> -p9042:9042 -p9160:9160 -p7000:7000 -p7001:7001  cassandra
>Reporter: mmh
>Priority: Low
> Fix For: 3.11.x
>
>
> First, create keyspace and a table using the following
> {code:java}
> CREATE KEYSPACE scheduler WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': '1'}  AND durable_writes = true;
> CREATE TABLE scheduler.job_info (
> id timeuuid PRIMARY KEY,
> create_time int,
> cur_retry int,
> cur_run_times int,
> expire_time int,
> max_retry int,
> max_run_times int,
> payload text,
> period int,
> retry_interval int,
> status tinyint,
> topic text,
> type text,
> update_time int
> ) with caching = {'keys':'ALL', 'rows_per_partition':'NONE'};
> {code}
> then, execute the following cql:
> {code:java}
> insert into job_info (id, create_time) values 
> (5be224c6-8231-11e7-9619-9801b2a97471, 0) IF NOT EXISTS;
> insert into job_info (id, create_time) values 
> (5be224c6-8231-11e7-9619-9801b2a97471, 1);
> select * from job_info;
> {code}
> You will find that create_time is still 0, it is not updated.
> but, if you remove the IF NOT EXISTS keyword in the first cql, the update 
> will success.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7822) Confusing timeout on CAS contention

2020-10-26 Thread Benjamin Lerer (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-7822:
--
Resolution: Duplicate
Status: Resolved  (was: Open)

> Confusing timeout on CAS contention
> ---
>
> Key: CASSANDRA-7822
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7822
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Feature/Lightweight Transactions, Legacy/Coordination
>Reporter: Sankalp Kohli
>Priority: Low
>  Labels: LWT
>
> If we have contention in CAS and we hit the cas contention timeout, we throw 
> an exception. In this timeout exception, we pass that 0 replicas responded. 
> This is very confusing to someone looking at the client logs. I think we 
> might need to throw a separate exception for contention or may be add a flag 
> in the timeout exception. 
> We have seen many people confused by this so I think we should fix it. 
> This is how we throw it on contention. 
> throw new WriteTimeoutException(WriteType.CAS, consistencyForPaxos, 0, 
> consistencyForPaxos.blockFor(Keyspace.open(keyspaceName)));



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-9330) CAS timeout errors should use a different exception than WriteTimeoutException as WTE can happen when nodes fail to respond.

2020-10-26 Thread Benjamin Lerer (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-9330:
--
Resolution: Duplicate
Status: Resolved  (was: Open)

> CAS timeout errors should use a different exception than 
> WriteTimeoutException as WTE can happen when nodes fail to respond.
> 
>
> Key: CASSANDRA-9330
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9330
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Feature/Lightweight Transactions, Legacy/Coordination
>Reporter: Aaron Whiteside
>Priority: Normal
>  Labels: LWT
>
> Perhaps a CASContentionTimeoutException?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16223) Reading dense table yields invalid results in case of row scan queries

2020-10-26 Thread Jacek Lewandowski (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220659#comment-17220659
 ] 

Jacek Lewandowski commented on CASSANDRA-16223:
---

Thanks!

> Reading dense table yields invalid results in case of row scan queries
> --
>
> Key: CASSANDRA-16223
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16223
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 3.11.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {{ThriftIntegrationTest}} is broken in the way that it does not actually test 
> reads before and after flushing, because it does not do flush at all (see 
> https://github.com/apache/cassandra/blob/cassandra-3.11/test/unit/org/apache/cassandra/cql3/validation/ThriftIntegrationTest.java#L939).
>  After fixing that method so that it really flushes memtables to disk, we can 
> see inconsistency in reads from dense table - the results returned from 
> memtable differs from the results returned from sstable (the later are wrong, 
> cell values are skipped unexpectedly).
> {noformat}
> java.lang.AssertionError: Invalid value for row 0 column 0 (value of type 
> ascii), expected  but got <>
> {noformat}
> In principle this problems is about skipping column values when doing row 
> scan queries with explicitly selected columns (not wildcard), when the 
> columns belong to a super column. This happens only when reading from 
> sstables, it does not happen when reading from memtables.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16223) Reading dense table yields invalid results in case of row scan queries

2020-10-26 Thread Sylvain Lebresne (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220657#comment-17220657
 ] 

Sylvain Lebresne commented on CASSANDRA-16223:
--

bq. It does not impact 3.0.x because AFAIU there is no value skipping 
optimization in 3.0.x.

It's true that the bug here does not manifest on 3.0. That said, the code is 
still kind of wrong in 3.0 as well (range queries just don't get the proper 
column filter for super columns), and while this may not manifest in a bug in 
practice, it still feel a bit dodgy to leave as is in practice. Plus, fixing 
{{ThriftIntegrationTest}} in 3.0 doesn't hurt either.

Anyway, I've pulled the patch against 3.0 as well (it applies without any 
changes whatsoever) and started CI on both branches. Planning to commit both 
when I get clean result from CI (unless someone object quickly on the 3.0 part).

|| patch || CI run ||
| [3.0|https://github.com/pcmanus/cassandra/commits/C-16223-3.0] | 
[#133|https://ci-cassandra.apache.org/job/Cassandra-devbranch/133/] |
| [3.11|https://github.com/pcmanus/cassandra/commits/C-16223-3.11] | 
[#134|https://ci-cassandra.apache.org/job/Cassandra-devbranch/134/] |


> Reading dense table yields invalid results in case of row scan queries
> --
>
> Key: CASSANDRA-16223
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16223
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 3.11.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {{ThriftIntegrationTest}} is broken in the way that it does not actually test 
> reads before and after flushing, because it does not do flush at all (see 
> https://github.com/apache/cassandra/blob/cassandra-3.11/test/unit/org/apache/cassandra/cql3/validation/ThriftIntegrationTest.java#L939).
>  After fixing that method so that it really flushes memtables to disk, we can 
> see inconsistency in reads from dense table - the results returned from 
> memtable differs from the results returned from sstable (the later are wrong, 
> cell values are skipped unexpectedly).
> {noformat}
> java.lang.AssertionError: Invalid value for row 0 column 0 (value of type 
> ascii), expected  but got <>
> {noformat}
> In principle this problems is about skipping column values when doing row 
> scan queries with explicitly selected columns (not wildcard), when the 
> columns belong to a super column. This happens only when reading from 
> sstables, it does not happen when reading from memtables.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16223) Reading dense table yields invalid results in case of row scan queries

2020-10-26 Thread Sylvain Lebresne (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-16223:
-
Reviewers: Sylvain Lebresne, Sylvain Lebresne  (was: Sylvain Lebresne)
   Sylvain Lebresne, Sylvain Lebresne  (was: Sylvain Lebresne)
   Status: Review In Progress  (was: Patch Available)

> Reading dense table yields invalid results in case of row scan queries
> --
>
> Key: CASSANDRA-16223
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16223
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 3.11.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {{ThriftIntegrationTest}} is broken in the way that it does not actually test 
> reads before and after flushing, because it does not do flush at all (see 
> https://github.com/apache/cassandra/blob/cassandra-3.11/test/unit/org/apache/cassandra/cql3/validation/ThriftIntegrationTest.java#L939).
>  After fixing that method so that it really flushes memtables to disk, we can 
> see inconsistency in reads from dense table - the results returned from 
> memtable differs from the results returned from sstable (the later are wrong, 
> cell values are skipped unexpectedly).
> {noformat}
> java.lang.AssertionError: Invalid value for row 0 column 0 (value of type 
> ascii), expected  but got <>
> {noformat}
> In principle this problems is about skipping column values when doing row 
> scan queries with explicitly selected columns (not wildcard), when the 
> columns belong to a super column. This happens only when reading from 
> sstables, it does not happen when reading from memtables.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15663) DESCRIBE KEYSPACE does not properly quote table names

2020-10-26 Thread Jira


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andres de la Peña updated CASSANDRA-15663:
--
Status: Changes Suggested  (was: Review In Progress)

> DESCRIBE KEYSPACE does not properly quote table names
> -
>
> Key: CASSANDRA-15663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15663
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL/Syntax
>Reporter: Oskar Liljeblad
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 3.11.x
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> How to reproduce (3.11.6) - cqlsh:
> {code}
> CREATE KEYSPACE test1 WITH replication = \{'class': 'SimpleStrategy', 
> 'replication_factor': '1'} AND durable_writes = true;
> CREATE TABLE test1."default" (id text PRIMARY KEY, data text, etag text);
> DESCRIBE KEYSPACE test1;
> {code}
> Output will be:
> {code}
> CREATE TABLE test1.default (
>  id text PRIMARY KEY,
>  data text,
>  etag text
> ) WITH [..]
> {code}
> Output should be:
> {code}
> CREATE TABLE test1."default" (
>  id text PRIMARY KEY,
>  data text,
>  etag text
> ) WITH [..]
> {code}
>  If you try to run {{CREATE TABLE test1.default [..]}} you will get an error 
> SyntaxException: line 1:19 no viable alternative at input 'default' (CREATE 
> TABLE test1.[default]...)
> Oskar Liljeblad
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15663) DESCRIBE KEYSPACE does not properly quote table names

2020-10-26 Thread Jira


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andres de la Peña updated CASSANDRA-15663:
--
Reviewers: Andres de la Peña, Andres de la Peña  (was: Andres de la Peña)
   Andres de la Peña, Andres de la Peña  (was: Andres de la Peña)
   Status: Review In Progress  (was: Patch Available)

> DESCRIBE KEYSPACE does not properly quote table names
> -
>
> Key: CASSANDRA-15663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15663
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL/Syntax
>Reporter: Oskar Liljeblad
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 3.11.x
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> How to reproduce (3.11.6) - cqlsh:
> {code}
> CREATE KEYSPACE test1 WITH replication = \{'class': 'SimpleStrategy', 
> 'replication_factor': '1'} AND durable_writes = true;
> CREATE TABLE test1."default" (id text PRIMARY KEY, data text, etag text);
> DESCRIBE KEYSPACE test1;
> {code}
> Output will be:
> {code}
> CREATE TABLE test1.default (
>  id text PRIMARY KEY,
>  data text,
>  etag text
> ) WITH [..]
> {code}
> Output should be:
> {code}
> CREATE TABLE test1."default" (
>  id text PRIMARY KEY,
>  data text,
>  etag text
> ) WITH [..]
> {code}
>  If you try to run {{CREATE TABLE test1.default [..]}} you will get an error 
> SyntaxException: line 1:19 no viable alternative at input 'default' (CREATE 
> TABLE test1.[default]...)
> Oskar Liljeblad
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16223) Reading dense table yields invalid results in case of row scan queries

2020-10-26 Thread Sylvain Lebresne (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220652#comment-17220652
 ] 

Sylvain Lebresne commented on CASSANDRA-16223:
--

Good catch, and the fix lgtm, +1.

> Reading dense table yields invalid results in case of row scan queries
> --
>
> Key: CASSANDRA-16223
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16223
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 3.11.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {{ThriftIntegrationTest}} is broken in the way that it does not actually test 
> reads before and after flushing, because it does not do flush at all (see 
> https://github.com/apache/cassandra/blob/cassandra-3.11/test/unit/org/apache/cassandra/cql3/validation/ThriftIntegrationTest.java#L939).
>  After fixing that method so that it really flushes memtables to disk, we can 
> see inconsistency in reads from dense table - the results returned from 
> memtable differs from the results returned from sstable (the later are wrong, 
> cell values are skipped unexpectedly).
> {noformat}
> java.lang.AssertionError: Invalid value for row 0 column 0 (value of type 
> ascii), expected  but got <>
> {noformat}
> In principle this problems is about skipping column values when doing row 
> scan queries with explicitly selected columns (not wildcard), when the 
> columns belong to a super column. This happens only when reading from 
> sstables, it does not happen when reading from memtables.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16227) Nodetool Netstatas and tablestats tests

2020-10-26 Thread Jira


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andres de la Peña updated CASSANDRA-16227:
--
Reviewers: Andres de la Peña

> Nodetool Netstatas and tablestats tests
> ---
>
> Key: CASSANDRA-16227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16227
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> As part of the CASSANDRA-15583 effort we're adding unit tests for netstats 
> and tablestats



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16223) Reading dense table yields invalid results in case of row scan queries

2020-10-26 Thread Jacek Lewandowski (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220636#comment-17220636
 ] 

Jacek Lewandowski commented on CASSANDRA-16223:
---

It does not impact 3.0.x because AFAIU there is no value skipping optimization 
in 3.0.x.

> Reading dense table yields invalid results in case of row scan queries
> --
>
> Key: CASSANDRA-16223
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16223
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 3.11.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {{ThriftIntegrationTest}} is broken in the way that it does not actually test 
> reads before and after flushing, because it does not do flush at all (see 
> https://github.com/apache/cassandra/blob/cassandra-3.11/test/unit/org/apache/cassandra/cql3/validation/ThriftIntegrationTest.java#L939).
>  After fixing that method so that it really flushes memtables to disk, we can 
> see inconsistency in reads from dense table - the results returned from 
> memtable differs from the results returned from sstable (the later are wrong, 
> cell values are skipped unexpectedly).
> {noformat}
> java.lang.AssertionError: Invalid value for row 0 column 0 (value of type 
> ascii), expected  but got <>
> {noformat}
> In principle this problems is about skipping column values when doing row 
> scan queries with explicitly selected columns (not wildcard), when the 
> columns belong to a super column. This happens only when reading from 
> sstables, it does not happen when reading from memtables.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-15583) 4.0 quality testing: Tooling, Bundled and First Party

2020-10-26 Thread Berenguer Blasi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17210016#comment-17210016
 ] 

Berenguer Blasi edited comment on CASSANDRA-15583 at 10/26/20, 11:39 AM:
-

So this is trickier than expected as some commands need a node to run properly. 
So I managed to make {{ToolRunner}} work against jvm-dtests nodetool commands. 
I also looked at the sstable utils you mentioned and it's going to need some 
hacking as well. You'll need sstables to upgrade from that the CQLTester 
instance is aware of etc..

For starters I got nodetool ring test up so far. I will update here as I make 
progress:
 * nodetool
 ** ring CASSANDRA-16200 (up for review)
 ** tablestats CASSANDRA-16227
 ** tpstats
 ** netstats CASSANDRA-16227
 ** disablebinary
 ** enablebinary
 ** gossipinfo
 * sstableupgrade
 * sstablescrub


was (Author: bereng):
So this is trickier than expected as some commands need a node to run properly. 
So I managed to make {{ToolRunner}} work against jvm-dtests nodetool commands. 
I also looked at the sstable utils you mentioned and it's going to need some 
hacking as well. You'll need sstables to upgrade from that the CQLTester 
instance is aware of etc..

For starters I got nodetool ring test up so far. I will update here as I make 
progress:
 * nodetool
 ** ring CASSANDRA-16200 (up for review)
 ** tablestats
 ** tpstats
 ** netstats
 ** disablebinary
 ** enablebinary
 ** gossipinfo
 * sstableupgrade
 * sstablescrub

> 4.0 quality testing: Tooling, Bundled and First Party
> -
>
> Key: CASSANDRA-15583
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15583
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/python, Test/unit
>Reporter: Josh McKenzie
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Reference [doc from 
> NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
>  for context.
> *Shepherd: Sam Tunnicliffe*
> Test plans should cover bundled first-party tooling and CLIs such as 
> nodetool, cqlsh, and new tools supporting full query and audit logging 
> (CASSANDRA-13983, CASSANDRA-12151).
> *Progress as of Aug 2020*
> {{ToolRunner}} has been added enabling us to test tools in java unit tests. 
> This includes capturing their stdout/err and stdin i.e. Most tools have a 
> starting unit test testing their cmd line args happy path. Tickets have been 
> created to improve coverage of those  and flagged LHF. Also for those tools 
> big enough they can't be addressed in a simple ticket such as nodetool, a 
> placeholder ticket for future improvements has been created as well. Tickets 
> and status are:
> ||Tool||UX test||UT coverage||dtest coverage||Comments||
> |Nodetool|(x)|(x) CASSANDRA-16026|(!)|Not all the sub commands are tested. 
> Dtest also test nodetool as a side effect|
> |Cqlsh|(x)|(x) CASSANDRA-16025|(!)| |
> |Cassandra-stress|(x)|(x) CASSANDRA-16024|(x)| |
> |debug-cql|(x)|(x) CASSANDRA-16023|(x)| |
> |fqltool|(x)|(/) CASSANDRA-16022|(!)| |
> |auditlogviewer|(/) CASSANDRA-15991|(!) CASSANDRA-16021|(!)| |
> |*Sstable utilities*| | | | |
> |sstabledump|(/) CASSANDRA-15991|(/) CASSANDRA-16020|(!)| |
> |sstableexpiredblockers|(/) CASSANDRA-15991|(x) CASSANDRA-16019|(!)| |
> |sstablelevelreset|(/) CASSANDRA-15991|(x) CASSANDRA-16018|(!)| |
> |sstableloader|(x)|(x) CASSANDRA-16017|(!)| |
> |sstablemetadata|(/) CASSANDRA-15991|(x) CASSANDRA-16016|(x)|Ran in dtests, 
> no dedicated test|
> |sstableofflinerelevel|(/) CASSANDRA-15991|(x) CASSANDRA-16015|(!)| |
> |sstablerepairedset|(/) CASSANDRA-15991|(x) CASSANDRA-16014|(x)|Ran in 
> dtests, no dedicated test|
> |sstablescrub|(/) CASSANDRA-15991|(x) CASSANDRA-16013|(!)| |
> |sstablesplit|(/) CASSANDRA-15991|(x) CASSANDRA-16012|(!)| |
> |sstableupgrade|(/) CASSANDRA-15991|(x) CASSANDRA-16011|(!)| |
> |sstableutil|(/) CASSANDRA-15991|(x) CASSANDRA-16010|(!)| |
> |sstableverify|(/) CASSANDRA-15991|(x) CASSANDRA-16009|(!)| |



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16227) Nodetool Netstatas and tablestats tests

2020-10-26 Thread Berenguer Blasi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220634#comment-17220634
 ] 

Berenguer Blasi commented on CASSANDRA-16227:
-

CI results in PR

> Nodetool Netstatas and tablestats tests
> ---
>
> Key: CASSANDRA-16227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16227
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> As part of the CASSANDRA-15583 effort we're adding unit tests for netstats 
> and tablestats



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16227) Nodetool Netstatas and tablestats tests

2020-10-26 Thread Berenguer Blasi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Berenguer Blasi updated CASSANDRA-16227:

Test and Documentation Plan: See PR
 Status: Patch Available  (was: In Progress)

> Nodetool Netstatas and tablestats tests
> ---
>
> Key: CASSANDRA-16227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16227
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> As part of the CASSANDRA-15583 effort we're adding unit tests for netstats 
> and tablestats



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16227) Nodetool Netstatas and tablestats tests

2020-10-26 Thread Berenguer Blasi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Berenguer Blasi updated CASSANDRA-16227:

Change Category: Quality Assurance
 Complexity: Normal
 Status: Open  (was: Triage Needed)

> Nodetool Netstatas and tablestats tests
> ---
>
> Key: CASSANDRA-16227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16227
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0
>
>
> As part of the CASSANDRA-15583 effort we're adding unit tests for netstats 
> and tablestats



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Assigned] (CASSANDRA-16227) Nodetool Netstatas and tablestats tests

2020-10-26 Thread Berenguer Blasi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Berenguer Blasi reassigned CASSANDRA-16227:
---

Assignee: Berenguer Blasi

> Nodetool Netstatas and tablestats tests
> ---
>
> Key: CASSANDRA-16227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16227
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
>
> As part of the CASSANDRA-15583 effort we're adding unit tests for netstats 
> and tablestats



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-16227) Nodetool Netstatas and tablestats tests

2020-10-26 Thread Berenguer Blasi (Jira)
Berenguer Blasi created CASSANDRA-16227:
---

 Summary: Nodetool Netstatas and tablestats tests
 Key: CASSANDRA-16227
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16227
 Project: Cassandra
  Issue Type: Improvement
  Components: Test/unit
Reporter: Berenguer Blasi


As part of the CASSANDRA-15583 effort we're adding unit tests for netstats and 
tablestats



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16227) Nodetool Netstatas and tablestats tests

2020-10-26 Thread Berenguer Blasi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Berenguer Blasi updated CASSANDRA-16227:

Fix Version/s: 4.0

> Nodetool Netstatas and tablestats tests
> ---
>
> Key: CASSANDRA-16227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16227
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0
>
>
> As part of the CASSANDRA-15583 effort we're adding unit tests for netstats 
> and tablestats



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16166) Rename master branch to trunk in cassandra-dtest

2020-10-26 Thread Michael Semb Wever (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220613#comment-17220613
 ] 

Michael Semb Wever commented on CASSANDRA-16166:


bq. new docker images deployed (see cassandra-builds patch)

[~eduard.tudenhoefner] has done this, and I tested it with a dtest build on 
ci-cassandra.a.o

> Rename master branch to trunk in cassandra-dtest
> 
>
> Key: CASSANDRA-16166
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16166
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Build
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra-builds] branch trunk updated: ninja-fix: jenkins labels in devbranch dtests jobs

2020-10-26 Thread mck
This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git


The following commit(s) were added to refs/heads/trunk by this push:
 new d1e0ca7  ninja-fix: jenkins labels in devbranch dtests jobs
d1e0ca7 is described below

commit d1e0ca7bff71d13b0174745e810ceb2669232246
Author: Mick Semb Wever 
AuthorDate: Mon Oct 26 11:32:34 2020 +0100

ninja-fix: jenkins labels in devbranch dtests jobs
---
 jenkins-dsl/cassandra_job_dsl_seed.groovy | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy 
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index ddd1284..e55d339 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -803,9 +803,9 @@ dtestTargets.each {
 (1..splits).each { values << it.toString() }
 text('split', values)
 if (targetName == 'dtest-large') {
-label(largeSlaveLabel)
+label('label', largeSlaveLabel)
 } else {
-label(slaveLabel)
+label('label', slaveLabel)
 }
  }
 properties {


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-13931) Cassandra JVM stop itself randomly

2020-10-26 Thread Alex Petrov (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-13931:

Description: 
Before I set  -XX:MaxDirectMemorySize  I receive  OOM on OS level like;

# # grep "Out of" /var/log/messages-20170918
Sep 16 06:54:07 p00skimnosql04 kernel: Out of memory: Kill process 26619 (java) 
score 287 or sacrifice child
Sep 16 06:54:07 p00skimnosql04 kernel: Out of memory: Kill process 26640 (java) 
score 289 or sacrifice child

If set  -XX:MaxDirectMemorySize=5G limitation then periodicaly begin receive:
HeapUtils.java:136 - Dumping heap to 
/egov/dumps/cassandra-1506868110-pid11155.hprof

It seems like  JVM kill itself when off-heap memory leaks occur.
Typical errors in  system.log before JVM begin dumping:

ERROR [MessagingService-Incoming-/172.20.4.143] 2017-10-01 19:00:36,336 
CassandraDaemon.java:228 - Exception in thread 
Thread[MessagingService-Incoming-/172.20.4.143,5,main]
ERROR [Native-Transport-Requests-139] 2017-10-01 19:04:02,675 Message.java:625 
- Unexpected exception during request; channel = [id: 0x3c0c1c26, 
L:/172.20.4.142:9042 - R:/172.20.4.139:44874]

Full stack traces:

{code}
ERROR [Native-Transport-Requests-139] 2017-10-01 19:04:02,675 Message.java:625 
- Unexpected exception during request; channel = [id: 0x3c0c1c26, 
L:/172.20.4.142:9042 -
R:/172.20.4.139:44874]
java.lang.AssertionError: null
at 
org.apache.cassandra.transport.ServerConnection.applyStateTransition(ServerConnection.java:97)
 ~[apache-cassandra-3.11.0.jar:3.11.0]
at 
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:521)
 [apache-cassandra-3.11.0.jar:3.11.0]
at 
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:410)
 [apache-cassandra-3.11.0.jar:3.11.0]
at 
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
 [netty-all-4.0.44.Final.jar:4.0.44.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
 [netty-all-4.0.44.Final.jar:4.0.44.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:35)
 [netty-all-4.0.44.Final.jar:4.0.44.Final]
at 
io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:348)
 [netty-all-4.0.44.Final.jar:4.0.44.Final]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[na:1.8.0_121]
at 
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
 [apache-cassandra-3.11.0.jar:3.1
1.0]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) 
[apache-cassandra-3.11.0.jar:3.11.0]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
{code}

{code}
INFO  [MutationStage-127] 2017-10-01 19:08:24,255 HeapUtils.java:136 - Dumping 
heap to /egov/dumps/cassandra-1506868110-pid11155.hprof ...
Heap dump file created
{code}

{code}
ERROR [MessagingService-Incoming-/172.20.4.143] 2017-10-01 19:08:33,493 
CassandraDaemon.java:228 - Exception in thread 
Thread[MessagingService-Incoming-/172.20.4.143,5,main]
java.io.IOError: java.io.EOFException: Stream ended prematurely
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNext(UnfilteredRowIteratorSerializer.java:227)
 ~[apache-cassandra-3.11.0.jar:3.11.0]
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNext(UnfilteredRowIteratorSerializer.java:215)
 ~[apache-cassandra-3.11.0.jar:3.11.0]
at 
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
~[apache-cassandra-3.11.0.jar:3.11.0]
at 
org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:839)
 ~[apache-cassandra-3.11.0.jar:3.11.0]
at 
org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:800)
 ~[apache-cassandra-3.11.0.jar:3.11.0]
at 
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:415)
 ~[apache-cassandra-3.11.0.jar:3.11.0]
at 
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:434)
 ~[apache-cassandra-3.11.0.jar:3.11.0]
at 
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:371)
 ~[apache-cassandra-3.11.0.jar:3.11.0]
at org.apache.cassandra.net.MessageIn.read(MessageIn.java:123) 
~[apache-cassandra-3.11.0.jar:3.11.0]
at 
org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:192)
 ~[apache-cassandra-3.11.0.jar:3.11.0]
at 
org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:180)
 ~[apache-cassandra-3.11.0.jar:3.11.0]
at