[GitHub] kramasamy commented on a change in pull request #2892: fixing eco parser test

kramasamy commented on a change in pull request #2892: fixing eco parser test URL: https://github.com/apache/incubator-heron/pull/2892#discussion_r186253432 ## File path: eco/tests/java/org/apache/heron/eco/parser/EcoParserTest.java ## @@ -33,17 +140,17 @@ + "#

[GitHub] nwangtw commented on a change in pull request #2892: fixing eco parser test

nwangtw commented on a change in pull request #2892: fixing eco parser test URL: https://github.com/apache/incubator-heron/pull/2892#discussion_r186253357 ## File path: eco/tests/java/org/apache/heron/eco/parser/EcoParserTest.java ## @@ -33,17 +140,17 @@ + "# spout

[GitHub] nwangtw commented on a change in pull request #2892: fixing eco parser test

nwangtw commented on a change in pull request #2892: fixing eco parser test URL: https://github.com/apache/incubator-heron/pull/2892#discussion_r186253366 ## File path: eco/tests/java/org/apache/heron/eco/parser/EcoParserTest.java ## @@ -16,7 +16,114 @@ * specific

Re: Stateful updating and deterministic routing

Yeah, state recovery is a bit more difficult with Heron's architecture. In Storm, the task IDs are not just values used for routing they actually equate to a task instance within the executor. An executor which currently processes the keys 4-8 actually contains 5 task instances of the same

Re: Stateful updating and deterministic routing

+1 for this idea. As long as the predefined key space is large enough, it should work for most of the cases. Based on my experience with topologies, I never saw one component has more than 1000 instances in a topology. For recovering states from an update, there will be some problems though.

[GitHub] huijunw commented on a change in pull request #2886: [wip] submit topology file by uri

huijunw commented on a change in pull request #2886: [wip] submit topology file by uri URL: https://github.com/apache/incubator-heron/pull/2886#discussion_r186236468 ## File path: heron/tools/cli/src/python/submit.py ## @@ -366,6 +368,16 @@ def submit_cpp(cl_args,

[GitHub] nlu90 commented on issue #2884: move downloader registry map to yaml

nlu90 commented on issue #2884: move downloader registry map to yaml URL: https://github.com/apache/incubator-heron/pull/2884#issuecomment-386751809 @kramasamy This change may break your downloader usage, please update your code accordingly using the new `downloader.yaml` file.

Re: Stateful updating and deterministic routing

Interesting. Thanks for sharing~ On Fri, May 4, 2018 at 2:31 PM, Thomas Cooper wrote: > Hi all, > > A while ago I emailed about the issue of how fields (key) grouped routing > in Heron was not consistent across an update and how this makes preserving > state across an

[GitHub] nwangtw commented on a change in pull request #2891: [WIP] Refactor StatefulStorage

nwangtw commented on a change in pull request #2891: [WIP] Refactor StatefulStorage URL: https://github.com/apache/incubator-heron/pull/2891#discussion_r186226835 ## File path: heron/spi/src/java/org/apache/heron/spi/statefulstorage/IStatefulStorage.java ## @@ -16,37

Stateful updating and deterministic routing

Hi all, A while ago I emailed about the issue of how fields (key) grouped routing in Heron was not consistent across an update and how this makes preserving state across an update very difficult and also makes it difficult/impossible to analyse or predict tuple flows through a current/proposed

[GitHub] nlu90 commented on a change in pull request #2891: [WIP] Refactor StatefulStorage

nlu90 commented on a change in pull request #2891: [WIP] Refactor StatefulStorage URL: https://github.com/apache/incubator-heron/pull/2891#discussion_r186222743 ## File path: heron/proto/ckptmgr.proto ## @@ -171,7 +175,25 @@ message CleanStatefulCheckpointResponse {

[GitHub] nlu90 commented on a change in pull request #2891: [WIP] Refactor StatefulStorage

nlu90 commented on a change in pull request #2891: [WIP] Refactor StatefulStorage URL: https://github.com/apache/incubator-heron/pull/2891#discussion_r186222743 ## File path: heron/proto/ckptmgr.proto ## @@ -171,7 +175,25 @@ message CleanStatefulCheckpointResponse {

[GitHub] nwangtw commented on a change in pull request #2891: [WIP] Refactor StatefulStorage

nwangtw commented on a change in pull request #2891: [WIP] Refactor StatefulStorage URL: https://github.com/apache/incubator-heron/pull/2891#discussion_r186219947 ## File path: heron/proto/ckptmgr.proto ## @@ -171,7 +175,25 @@ message CleanStatefulCheckpointResponse {

[GitHub] nlu90 closed pull request #2883: check checkpoint file exists before deleting it

nlu90 closed pull request #2883: check checkpoint file exists before deleting it URL: https://github.com/apache/incubator-heron/pull/2883 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is

[GitHub] nlu90 commented on a change in pull request #2891: [WIP] Refactor StatefulStorage

nlu90 commented on a change in pull request #2891: [WIP] Refactor StatefulStorage URL: https://github.com/apache/incubator-heron/pull/2891#discussion_r186203313 ## File path: heron/spi/src/java/org/apache/heron/spi/statefulstorage/CheckpointMetadata.java ## @@ -0,0 +1,39

[GitHub] nwangtw opened a new pull request #2891: [WIP] Refactor StatefulStorage

nwangtw opened a new pull request #2891: [WIP] Refactor StatefulStorage URL: https://github.com/apache/incubator-heron/pull/2891 - Refactor Stateful Storage to be more independent of topology - Refactor data structures.

[GitHub] comes5 opened a new issue #2890: Error building with Bazel

comes5 opened a new issue #2890: Error building with Bazel URL: https://github.com/apache/incubator-heron/issues/2890 After cloning the repository: `git clone --branch 0.17.8 https://github.com/apache/incubator-heron.git` I started building with Bazel, but there was an error: `ERROR: