Re: Propose to combine module gearpump-daemon into gearpump-core

[Jiang Weihua]

+1 as I can’t find another better solution.

在 2016/10/11 下午4:29，“Karol Brejna” 写入:

+1
I think moving EmbeddedCluster capabilities to gearpump-core makes
sense and simplifies things.

On Tue, Oct 11, 2016 at 7:04 AM, Manu Zhang  wrote:
> +1.  Integration tests for Beam Gearpump runner heavily depends on
> EmbeddedCluster. It's quite weird to add `gearpump-daemon` dependency to
> run gearpump examples.
>
> Thanks,
> Manu
>
> On Tue, Oct 11, 2016 at 10:17 AM Vincent Wang  wrote:
>
>> Hi all,
>>
>>   Currently we have an EmbeddedCluster in gearpump-daemon module which
>> allows user to run their gearpump application in a local JVM.
>>   Now suppose user want to implement an streaming application using
>> Gearpump and use the EmbeddedCluster to test it locally or even in an 
IDE.
>> So the application needs to have extra dependency on the gearpump-daemon
>> module to access EmbeddedCluster,  and user have to write code like
>> * if (debugMode) {*
>> *  localCluster = new EmbeddedCluster(akkaConf);*
>> *  localCluster.start();*
>> *}*
>> *ClientContext masterClient = null;*
>> *if (localCluster != null) {*
>> *  masterClient = localCluster.newClientContext();*
>> *} else {*
>> *  masterClient = new ClientContext(akkaConf);*
>> *}*
>> *masterClient.submit(app);*
>>
>>   Which is obviously tedious and inconvenient.
>>
>>   So we'd like to provide user with a implementation just like what Flink
>> did.  Basically we want to integrate EmbeddedCluster into ClientContext 
so
>> that user can switch the application running mode easily by some
>> configuration or decision made by ClientConext automatically.
>>   However, currently the ClientContext is in module gearpump-core and
>> EmbeddedCluster
>> has dependencies on Worker and Master daemon actors, which means these
>> classes need to be moved along with EmbeddedCluster and finally almost 
the
>> whole gearpump-daemon module need to be integrated into gearpump-core
>> module.
>>That's the side effect of achieving a more powerful ClientContext
>> integrated with a EmbeddedCluster and I'd like to hear your opinions 
about
>> on this proposal.
>>
>> Thanks,
>> Huafeng
>>

Re: Propose to combine module gearpump-daemon into gearpump-core

[Karol Brejna]

+1
I think moving EmbeddedCluster capabilities to gearpump-core makes
sense and simplifies things.

On Tue, Oct 11, 2016 at 7:04 AM, Manu Zhang  wrote:
> +1.  Integration tests for Beam Gearpump runner heavily depends on
> EmbeddedCluster. It's quite weird to add `gearpump-daemon` dependency to
> run gearpump examples.
>
> Thanks,
> Manu
>
> On Tue, Oct 11, 2016 at 10:17 AM Vincent Wang  wrote:
>
>> Hi all,
>>
>>   Currently we have an EmbeddedCluster in gearpump-daemon module which
>> allows user to run their gearpump application in a local JVM.
>>   Now suppose user want to implement an streaming application using
>> Gearpump and use the EmbeddedCluster to test it locally or even in an IDE.
>> So the application needs to have extra dependency on the gearpump-daemon
>> module to access EmbeddedCluster,  and user have to write code like
>> * if (debugMode) {*
>> *  localCluster = new EmbeddedCluster(akkaConf);*
>> *  localCluster.start();*
>> *}*
>> *ClientContext masterClient = null;*
>> *if (localCluster != null) {*
>> *  masterClient = localCluster.newClientContext();*
>> *} else {*
>> *  masterClient = new ClientContext(akkaConf);*
>> *}*
>> *masterClient.submit(app);*
>>
>>   Which is obviously tedious and inconvenient.
>>
>>   So we'd like to provide user with a implementation just like what Flink
>> did.  Basically we want to integrate EmbeddedCluster into ClientContext so
>> that user can switch the application running mode easily by some
>> configuration or decision made by ClientConext automatically.
>>   However, currently the ClientContext is in module gearpump-core and
>> EmbeddedCluster
>> has dependencies on Worker and Master daemon actors, which means these
>> classes need to be moved along with EmbeddedCluster and finally almost the
>> whole gearpump-daemon module need to be integrated into gearpump-core
>> module.
>>That's the side effect of achieving a more powerful ClientContext
>> integrated with a EmbeddedCluster and I'd like to hear your opinions about
>> on this proposal.
>>
>> Thanks,
>> Huafeng
>>

Re: Propose to combine module gearpump-daemon into gearpump-core

[Manu Zhang]

+1.  Integration tests for Beam Gearpump runner heavily depends on
EmbeddedCluster. It's quite weird to add `gearpump-daemon` dependency to
run gearpump examples.

Thanks,
Manu

On Tue, Oct 11, 2016 at 10:17 AM Vincent Wang  wrote:

> Hi all,
>
>   Currently we have an EmbeddedCluster in gearpump-daemon module which
> allows user to run their gearpump application in a local JVM.
>   Now suppose user want to implement an streaming application using
> Gearpump and use the EmbeddedCluster to test it locally or even in an IDE.
> So the application needs to have extra dependency on the gearpump-daemon
> module to access EmbeddedCluster,  and user have to write code like
> * if (debugMode) {*
> *  localCluster = new EmbeddedCluster(akkaConf);*
> *  localCluster.start();*
> *}*
> *ClientContext masterClient = null;*
> *if (localCluster != null) {*
> *  masterClient = localCluster.newClientContext();*
> *} else {*
> *  masterClient = new ClientContext(akkaConf);*
> *}*
> *masterClient.submit(app);*
>
>   Which is obviously tedious and inconvenient.
>
>   So we'd like to provide user with a implementation just like what Flink
> did.  Basically we want to integrate EmbeddedCluster into ClientContext so
> that user can switch the application running mode easily by some
> configuration or decision made by ClientConext automatically.
>   However, currently the ClientContext is in module gearpump-core and
> EmbeddedCluster
> has dependencies on Worker and Master daemon actors, which means these
> classes need to be moved along with EmbeddedCluster and finally almost the
> whole gearpump-daemon module need to be integrated into gearpump-core
> module.
>That's the side effect of achieving a more powerful ClientContext
> integrated with a EmbeddedCluster and I'd like to hear your opinions about
> on this proposal.
>
> Thanks,
> Huafeng
>