[GitHub] helix pull request: [HELIX-524] Add a getProgress to the Task inte...
Github user kanakb commented on the pull request: https://github.com/apache/helix/pull/6#issuecomment-57588016 I like this in general, but I have a couple questions/comments: * This is an interface change. Is there a way we can provide this functionality without requiring that all existing task implementations be rewritten? * What is the persistence story (if any)? Who calls getProgress? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HELIX-524) add getProgress() to Task interface
[ https://issues.apache.org/jira/browse/HELIX-524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14156118#comment-14156118 ] ASF GitHub Bot commented on HELIX-524: -- Github user kanakb commented on the pull request: https://github.com/apache/helix/pull/6#issuecomment-57588016 I like this in general, but I have a couple questions/comments: * This is an interface change. Is there a way we can provide this functionality without requiring that all existing task implementations be rewritten? * What is the persistence story (if any)? Who calls getProgress? add getProgress() to Task interface --- Key: HELIX-524 URL: https://issues.apache.org/jira/browse/HELIX-524 Project: Apache Helix Issue Type: Improvement Components: helix-core Affects Versions: 0.6.4 Reporter: Hongbo Zeng Fix For: 0.6.5 Add a getProgress to the Task interface, this is very helpful for long running tasks, from which we know the status of a task and see if it's blocked. The return value is a double, ranging from 0 to 1.0, 1.0 indicates a task is finished -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] helix pull request: [HELIX-524] Add a getProgress to the Task inte...
Github user brandtg commented on the pull request:
https://github.com/apache/helix/pull/6#issuecomment-57596146
For persistence story, what about putting a new health report:
`/STAND_ALONE/INSTANCES/{instanceName}/HEALTHREPORT/taskStatus`?
Another thing to consider is that adding `#getProgress` to the task
interface requires users to be able to intelligently report task status. I
suspect that in practice someone might be annoyed at this extra responsibility,
and provide dummy numbers (sounds stupid, but saw it before).
Maybe a better approach would be to try to monitor on things we know a
priori about task (e.g. lifetime) and provide tools to inspect ones that seem
stuck (e.g. task/partition-addressable stack trace)?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (HELIX-524) add getProgress() to Task interface
[
https://issues.apache.org/jira/browse/HELIX-524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14156204#comment-14156204
]
ASF GitHub Bot commented on HELIX-524:
--
Github user brandtg commented on the pull request:
https://github.com/apache/helix/pull/6#issuecomment-57596146
For persistence story, what about putting a new health report:
`/STAND_ALONE/INSTANCES/{instanceName}/HEALTHREPORT/taskStatus`?
Another thing to consider is that adding `#getProgress` to the task
interface requires users to be able to intelligently report task status. I
suspect that in practice someone might be annoyed at this extra responsibility,
and provide dummy numbers (sounds stupid, but saw it before).
Maybe a better approach would be to try to monitor on things we know a
priori about task (e.g. lifetime) and provide tools to inspect ones that seem
stuck (e.g. task/partition-addressable stack trace)?
add getProgress() to Task interface
---
Key: HELIX-524
URL: https://issues.apache.org/jira/browse/HELIX-524
Project: Apache Helix
Issue Type: Improvement
Components: helix-core
Affects Versions: 0.6.4
Reporter: Hongbo Zeng
Fix For: 0.6.5
Add a getProgress to the Task interface, this is very helpful for long
running tasks, from which we know the status of a task and see if it's
blocked. The return value is a double, ranging from 0 to 1.0, 1.0 indicates a
task is finished
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HELIX-524) add getProgress() to Task interface
[ https://issues.apache.org/jira/browse/HELIX-524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14156218#comment-14156218 ] ASF GitHub Bot commented on HELIX-524: -- Github user brandtg commented on the pull request: https://github.com/apache/helix/pull/6#issuecomment-57597306 Wait, that stuff's gone now? Ok never mind. Is persistence of the state of a running task completely necessary? (Like would JMX suffice?) add getProgress() to Task interface --- Key: HELIX-524 URL: https://issues.apache.org/jira/browse/HELIX-524 Project: Apache Helix Issue Type: Improvement Components: helix-core Affects Versions: 0.6.4 Reporter: Hongbo Zeng Fix For: 0.6.5 Add a getProgress to the Task interface, this is very helpful for long running tasks, from which we know the status of a task and see if it's blocked. The return value is a double, ranging from 0 to 1.0, 1.0 indicates a task is finished -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Review Request 26272: [HELIX-518] Add integration tests to ensure helix tasks work as expected during master failover
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26272/ --- Review request for helix, Kanak Biscuitwala and Shi Lu. Bugs: 518 Repository: helix-git Description --- [HELIX-518] Add integration tests to ensure helix tasks work as expected during master failover Diffs - helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java 546e3bb helix-core/src/test/java/org/apache/helix/integration/task/DummyTask.java e69de29 helix-core/src/test/java/org/apache/helix/integration/task/TestTaskRebalancerFailover.java e69de29 Diff: https://reviews.apache.org/r/26272/diff/ Testing --- mvn test Thanks, Zhen Zhang
Re: Review Request 26272: [HELIX-518] Add integration tests to ensure helix tasks work as expected during master failover
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26272/#review55248 --- Ship it! Ship It! - Shi Lu On Oct. 2, 2014, 5:30 p.m., Zhen Zhang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26272/ --- (Updated Oct. 2, 2014, 5:30 p.m.) Review request for helix, Kanak Biscuitwala and Shi Lu. Bugs: 518 Repository: helix-git Description --- [HELIX-518] Add integration tests to ensure helix tasks work as expected during master failover Diffs - helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java 546e3bb helix-core/src/test/java/org/apache/helix/integration/task/DummyTask.java e69de29 helix-core/src/test/java/org/apache/helix/integration/task/TestTaskRebalancerFailover.java e69de29 Diff: https://reviews.apache.org/r/26272/diff/ Testing --- mvn test Thanks, Zhen Zhang
[jira] [Commented] (HELIX-524) add getProgress() to Task interface
[ https://issues.apache.org/jira/browse/HELIX-524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14157419#comment-14157419 ] ASF GitHub Bot commented on HELIX-524: -- Github user dayzzz commented on the pull request: https://github.com/apache/helix/pull/6#issuecomment-57728028 * About the interface change, IMHO, this is a nature part of the task interface. If the existing customers want to update, adding the support is encouraged. We can have another interface for monitoring the progress, but doesn't seem to be a nice design. * The task framework should call the progress interface and expose the status somehow, discussed below. * Persistency story, ZK is a good place if we only want to record the final result. If we want to expose the progress as a task runs, putting these periodical status updates in ZK is not a choice due to the large traffic, generally ZK is not a good place for reporting and monitoring service status. I also discussed this with Jason, we thought about inGraph (which is not an option for open source), Kafka or Riemann. (Greg, JMX sounds a good idea.) Without a conclusion of where to put these status stats, I agree that the progress interface is not of much value. For the first step, it would be good enough if we can monitor the progress. What do you guys think? * For the bogus progress number, it's the customer themselves who need to track the progress, if they want to see the bogus value, I'm fine with that :). The controller should be set not to act on the bogus value by the customers. add getProgress() to Task interface --- Key: HELIX-524 URL: https://issues.apache.org/jira/browse/HELIX-524 Project: Apache Helix Issue Type: Improvement Components: helix-core Affects Versions: 0.6.4 Reporter: Hongbo Zeng Fix For: 0.6.5 Add a getProgress to the Task interface, this is very helpful for long running tasks, from which we know the status of a task and see if it's blocked. The return value is a double, ranging from 0 to 1.0, 1.0 indicates a task is finished -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] helix pull request: [HELIX-524] Add a getProgress to the Task inte...
Github user kanakb commented on the pull request: https://github.com/apache/helix/pull/6#issuecomment-57730714 Why not have an abstract ProgressReportingTask that implements task and includes all the JMX persistence, and then have tasks extends from that and implement getProgress()? That avoids the interface change, but also allows smarter tasks to let their progress be known. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HELIX-524) add getProgress() to Task interface
[ https://issues.apache.org/jira/browse/HELIX-524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14157483#comment-14157483 ] ASF GitHub Bot commented on HELIX-524: -- Github user kanakb commented on the pull request: https://github.com/apache/helix/pull/6#issuecomment-57730714 Why not have an abstract ProgressReportingTask that implements task and includes all the JMX persistence, and then have tasks extends from that and implement getProgress()? That avoids the interface change, but also allows smarter tasks to let their progress be known. add getProgress() to Task interface --- Key: HELIX-524 URL: https://issues.apache.org/jira/browse/HELIX-524 Project: Apache Helix Issue Type: Improvement Components: helix-core Affects Versions: 0.6.4 Reporter: Hongbo Zeng Fix For: 0.6.5 Add a getProgress to the Task interface, this is very helpful for long running tasks, from which we know the status of a task and see if it's blocked. The return value is a double, ranging from 0 to 1.0, 1.0 indicates a task is finished -- This message was sent by Atlassian JIRA (v6.3.4#6332)
