[
https://issues.apache.org/jira/browse/CASSANDRA-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071884#comment-17071884
]
Stefan Miklosovic commented on CASSANDRA-15406:
-----------------------------------------------
I have added "global percentage" functionality into netstats both for sending
and receiving. It looks like this:
{code:java}
Mode: NORMAL
Rebuild 2c0b43f0-735d-11ea-9346-fb0ffe238736
/127.0.0.1
Receiving 133 files, 27664559 bytes total. Already received 21 files
(15.79%), 918834 bytes total (3.32%)
/tmp/dtests15682026295742741219/node2/data/distributed_test_keyspace/cf-196b3...
....{code}
On sending:
{code:java}
Mode: NORMAL
Rebuild 2c0b43f0-735d-11ea-9346-fb0ffe238736
/127.0.0.2
Sending 133 files, 27664559 bytes total. Already sent 42 files
(31.58%), 1841015 bytes total (6.65%)
/tmp/dtests15682026295742741219/node2/data/distributed_test_keyspace/cf-196b3...
....{code}
There is a bug in the current code as if we are streaming entire SSTables via
CassandraEntireSSTableStreamWriter and CassandraOutgoingFile respectively,
there is not any update on particular components of a SSTable as it counts only
"db" file to be there. That introduces this bug:
{code:java}
Mode: NORMAL
Rebuild 2c0b43f0-735d-11ea-9346-fb0ffe238736
/127.0.0.2 Sending 19 files, 27664559 bytes total. Already sent 133 files,
27664559 bytes total
/tmp/dtests15682026295742741219/node2/data/distributed_test_keyspace/cf-196b3...
....
{code}
Basically, number of files to be sent is lower than files sent.
Maybe something for [~djoshi] as food for thought.
I have fixed it here
[https://github.com/apache/cassandra/compare/trunk...smiklosovic:CASSANDRA-15406]
(forget the test, will be finished).
[~jasonstack], this fix would be way better if we manage to merge yours
CASSANDRA-15657 because mine is checking if SSTable is meant to be streamed
entirely and this is not performance-friendly. With your patch it would be just
one call to already cached value because you have introduced a field for that.
Could somebody please give me some feedback here if this is what we want?
Thanks
@
> Add command to show the progress of data streaming and index build
> -------------------------------------------------------------------
>
> Key: CASSANDRA-15406
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15406
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Streaming, Legacy/Streaming and Messaging,
> Tool/nodetool
> Reporter: maxwellguo
> Assignee: Stefan Miklosovic
> Priority: Normal
> Fix For: 4.0, 4.x
>
>
> I found that we should supply a command to show the progress of streaming
> when we do the operation of bootstrap/move/decommission/removenode. For when
> do data streaming , noboday knows which steps there program are in , so I
> think a command to show the joing/leaving node's is needed .
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
