[jira] [Commented] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892271#comment-13892271 ] Nick Bailey commented on CASSANDRA-4757: Yeah that would probably be fine. Maybe an option to either print out the plan id or to print progress information. For the jmx case, a bulkLoadAsync call would probably be fine and is consistent with other jmx methods we have. My only other question would be if the building indexes step of bulk loading would be included in that progress as well. Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Tyler Hobbs Priority: Minor Labels: lhf Fix For: 2.0.6 Attachments: CASSANDRA-4757.txt, CASSANDRA-4757v2.txt The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892350#comment-13892350 ] Tyler Hobbs commented on CASSANDRA-4757: bq. My only other question would be if the building indexes step of bulk loading would be included in that progress as well. In 2.0 index building is the last step before marking a streaming task as complete, so it's included. Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Tyler Hobbs Priority: Minor Labels: lhf Fix For: 2.0.6 Attachments: CASSANDRA-4757.txt, CASSANDRA-4757v2.txt The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892474#comment-13892474 ] Nick Bailey commented on CASSANDRA-4757: Patch LGTM. Haven't tested it out though. Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Tyler Hobbs Priority: Minor Labels: lhf Fix For: 2.0.6 Attachments: 4757-2.0.patch, CASSANDRA-4757.txt, CASSANDRA-4757v2.txt The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891571#comment-13891571 ] Tyler Hobbs commented on CASSANDRA-4757: [~nickmbailey] should sstableloader just print out the planID for the streaming session at the start of the loading process? I'm not sure what to do for the bulkLoad case because that call blocks for the streaming session to complete (at which point the planID is useless). Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Tyler Hobbs Priority: Minor Labels: lhf Fix For: 2.0.6 Attachments: CASSANDRA-4757.txt, CASSANDRA-4757v2.txt The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13784180#comment-13784180 ] Jonathan Ellis commented on CASSANDRA-4757: --- [~gdeangel], are you still working on this? Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Greg DeAngelis Priority: Minor Labels: lhf Fix For: 2.0.0 Attachments: CASSANDRA-4757.txt, CASSANDRA-4757v2.txt The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13757006#comment-13757006 ] Nick Bailey commented on CASSANDRA-4757: I'm confused what this patch accomplished that wasn't exposed by ProgressInfoCompositeData already. Isn't the file bytes/percent transferred information already there? Also we still don't have a way to associate a call to sstableloader/bulkload with a specific streaming session right? Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Greg DeAngelis Priority: Minor Labels: lhf Fix For: 2.0 Attachments: CASSANDRA-4757.txt, CASSANDRA-4757v2.txt The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754864#comment-13754864 ] Greg DeAngelis commented on CASSANDRA-4757: --- Sorry that was an oversight by me. I will fix that today. Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Greg DeAngelis Priority: Minor Labels: lhf Fix For: 2.0 Attachments: CASSANDRA-4757.txt, CASSANDRA-4757v2.txt The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754344#comment-13754344 ] Dave Brosius commented on CASSANDRA-4757: - +double rxPercentage = (totalRxBytes == 0 ? 100L : currentRxBytes * 100L / totalRxBytes); +double txPercentage = (totalTxBytes == 0 ? 100L : currentTxBytes * 100L / totalTxBytes); generates integer based percentages and stores them as doubles. Is this what was desired? Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Greg DeAngelis Priority: Minor Labels: lhf Fix For: 2.0 Attachments: CASSANDRA-4757.txt, CASSANDRA-4757v2.txt The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13750206#comment-13750206 ] Yuki Morishita commented on CASSANDRA-4757: --- Greg, Thanks for the patch. Your approach seems good, except SessionInfo already has methods like getTotalSizeReceived that returns current received bytes and so on, so you can use those. Also, you should use spaces instead of tabs: see http://wiki.apache.org/cassandra/CodeStyle. Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Priority: Minor Labels: lhf Attachments: CASSANDRA-4757.txt The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13750261#comment-13750261 ] Greg DeAngelis commented on CASSANDRA-4757: --- Yuki, Thanks for the comments. The v2 patch should address your suggestions. Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Greg DeAngelis Priority: Minor Labels: lhf Attachments: CASSANDRA-4757.txt, CASSANDRA-4757v2.txt The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13749702#comment-13749702 ] Greg DeAngelis commented on CASSANDRA-4757: --- It looks like each stream has a set of sessions that maintain the current state of all files being sent and received. Would it be sufficient to roll that session data up and expose it on the jmx interface for each stream? It would be pretty easy to expose the current and total rx and tx values as well as a percentage. Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Priority: Minor Labels: lhf The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13749728#comment-13749728 ] Nick Bailey commented on CASSANDRA-4757: Yeah, if the sstabeloader or jmx bulkload call returned a stream session id, and that could map to the streaming information already returned over jmx, that would get us 99% of the way. There are still some post streaming actions like building indexes that wouldn't be included with that information, but that might not be a huge deal. Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Priority: Minor Labels: lhf The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4757) Expose bulk loading progress/status over jmx
[ https://issues.apache.org/jira/browse/CASSANDRA-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13749748#comment-13749748 ] Greg DeAngelis commented on CASSANDRA-4757: --- Ok so given that, does the patch I just submitted bring anything to the table? Can the stream plan id be used as a correlation id? Expose bulk loading progress/status over jmx Key: CASSANDRA-4757 URL: https://issues.apache.org/jira/browse/CASSANDRA-4757 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Priority: Minor Labels: lhf Attachments: CASSANDRA-4757.txt The bulk loading interface should be exposing some progress or status information over jmx. This shouldn't be too difficult and should be exposed in a way that the information is available whether you are using the separate sstableloader utility or calling the bulkload jmx call. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
