[DISCUSS] A design proposal for incorporating machine learning algorithms into heron

Hello Dev community, I have created the initial API design documentation around building storm topologies around a set of machine learning streaming algorithms here: https://docs.google.com/document/d/1LrO7XRcMxJoMM83wjRd-Ov74VAaomA_mXOAhCStgGng/edit?usp=sharing, this is very much a work in

[GitHub] joshfischer1108 commented on issue #2894: fixing test

joshfischer1108 commented on issue #2894: fixing test URL: https://github.com/apache/incubator-heron/pull/2894#issuecomment-387156035 Just kicked it off again  This is an automated message from the Apache Git Service. To

[GitHub] jerrypeng commented on issue #2884: move downloader registry map to yaml

jerrypeng commented on issue #2884: move downloader registry map to yaml URL: https://github.com/apache/incubator-heron/pull/2884#issuecomment-387153148 I feel the changes have made the downloader a lot more complicated. Now we have to potentially pass in a cluster mode, heron home path,

[GitHub] jerrypeng commented on issue #2884: move downloader registry map to yaml

jerrypeng commented on issue #2884: move downloader registry map to yaml URL: https://github.com/apache/incubator-heron/pull/2884#issuecomment-387153148 I feel the changes have made the downloader a lot more complicated. Now we have to pass in a cluster mode or config path because we have

[GitHub] jerrypeng commented on issue #2884: move downloader registry map to yaml

jerrypeng commented on issue #2884: move downloader registry map to yaml URL: https://github.com/apache/incubator-heron/pull/2884#issuecomment-387153148 I feel the changes have made the downloader a lot more complicated. Now we have to pass in a cluster mode or config path because we have

[GitHub] jerrypeng commented on issue #2884: move downloader registry map to yaml

jerrypeng commented on issue #2884: move downloader registry map to yaml URL: https://github.com/apache/incubator-heron/pull/2884#issuecomment-387148313 So the arguments are changed for the downloader right? Won't this break existing usages of the downloader. Heron standalone/nomad calls

Re: Stateful updating and deterministic routing

I see. Then the doc I was reading might not be it either. I will ask maosong then. Thanks for the info! On Mon, May 7, 2018 at 9:23 AM, Bill Graham wrote: > Yeah, that's not it. The stateful scaling part of that doc got lengthy > enough that we broke it into a doc of

[GitHub] nwangtw commented on issue #2894: fixing test

nwangtw commented on issue #2894: fixing test URL: https://github.com/apache/incubator-heron/pull/2894#issuecomment-387137205 "(15:41:38) ERROR: build interrupted". :p This is an automated message from the Apache Git

Re: Stateful updating and deterministic routing

Yeah, that's not it. The stateful scaling part of that doc got lengthy enough that we broke it into a doc of it's own, per Sanjeev's suggestion IIRC. The fact that I can't locate it makes me think it was a twitter doc of mine (although it was not Twitter-specific), which I'm sure was shared with

[GitHub] joshfischer1108 closed pull request #2892: fixing eco parser test

joshfischer1108 closed pull request #2892: fixing eco parser test URL: https://github.com/apache/incubator-heron/pull/2892 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull

[GitHub] joshfischer1108 opened a new pull request #2894: fixing test

joshfischer1108 opened a new pull request #2894: fixing test URL: https://github.com/apache/incubator-heron/pull/2894 @nwangtw Lets try this one. This is an automated message from the Apache Git Service. To respond to the

[GitHub] joshfischer1108 commented on issue #2892: fixing eco parser test

joshfischer1108 commented on issue #2892: fixing eco parser test URL: https://github.com/apache/incubator-heron/pull/2892#issuecomment-387101604 @nwangtw Can you look at this CI error please? I'm not sure where this is coming from.

[GitHub] joshfischer1108 commented on issue #2892: fixing eco parser test

joshfischer1108 commented on issue #2892: fixing eco parser test URL: https://github.com/apache/incubator-heron/pull/2892#issuecomment-387085922 @nwangtw Didn't realize I had the permissions to restart build. I don't think I did when Heron was in Twitter repo. Just restarted it. Thanks

[GitHub] nwangtw commented on issue #2892: fixing eco parser test

nwangtw commented on issue #2892: fixing eco parser test URL: https://github.com/apache/incubator-heron/pull/2892#issuecomment-387083443 Looks like integration test failed. Need to retrigger travis. This is an automated

[GitHub] nwangtw commented on issue #2890: YAML cpp noboost deprecated

nwangtw commented on issue #2890: YAML cpp noboost deprecated URL: https://github.com/apache/incubator-heron/issues/2890#issuecomment-386988477 Related PR: https://github.com/apache/incubator-heron/pull/2893/files This is

[GitHub] nwangtw commented on a change in pull request #2891: [WIP] Refactor StatefulStorage

nwangtw commented on a change in pull request #2891: [WIP] Refactor StatefulStorage URL: https://github.com/apache/incubator-heron/pull/2891#discussion_r186349909 ## File path: heron/proto/ckptmgr.proto ## @@ -171,7 +175,25 @@ message CleanStatefulCheckpointResponse {

Re: Stateful updating and deterministic routing

Thanks Karthik. The doc is not exactly the same but close enough. It seems my doc is an internal one so let's use your doc as reference. I will see if there is any major differences and comment. On Sun, May 6, 2018 at 1:22 PM, Karthik Ramasamy wrote: > Here it is > >