Re: Propose to combine module gearpump-daemon into gearpump-core

[Jiang Weihua]

+1 as I can’t find another better solution.

在 2016/10/11 下午4:29，“Karol Brejna” 写入:

+1
I think moving EmbeddedCluster capabilities to gearpump-core makes
sense and simplifies things.

On Tue, Oct 11, 2016 at 7:04 AM, Manu Zhang  wrote:
> +1.  Integration tests for Beam Gearpump runner heavily depends on
> EmbeddedCluster. It's quite weird to add `gearpump-daemon` dependency to
> run gearpump examples.
>
> Thanks,
> Manu
>
> On Tue, Oct 11, 2016 at 10:17 AM Vincent Wang  wrote:
>
>> Hi all,
>>
>>   Currently we have an EmbeddedCluster in gearpump-daemon module which
>> allows user to run their gearpump application in a local JVM.
>>   Now suppose user want to implement an streaming application using
>> Gearpump and use the EmbeddedCluster to test it locally or even in an 
IDE.
>> So the application needs to have extra dependency on the gearpump-daemon
>> module to access EmbeddedCluster,  and user have to write code like
>> * if (debugMode) {*
>> *  localCluster = new EmbeddedCluster(akkaConf);*
>> *  localCluster.start();*
>> *}*
>> *ClientContext masterClient = null;*
>> *if (localCluster != null) {*
>> *  masterClient = localCluster.newClientContext();*
>> *} else {*
>> *  masterClient = new ClientContext(akkaConf);*
>> *}*
>> *masterClient.submit(app);*
>>
>>   Which is obviously tedious and inconvenient.
>>
>>   So we'd like to provide user with a implementation just like what Flink
>> did.  Basically we want to integrate EmbeddedCluster into ClientContext 
so
>> that user can switch the application running mode easily by some
>> configuration or decision made by ClientConext automatically.
>>   However, currently the ClientContext is in module gearpump-core and
>> EmbeddedCluster
>> has dependencies on Worker and Master daemon actors, which means these
>> classes need to be moved along with EmbeddedCluster and finally almost 
the
>> whole gearpump-daemon module need to be integrated into gearpump-core
>> module.
>>That's the side effect of achieving a more powerful ClientContext
>> integrated with a EmbeddedCluster and I'd like to hear your opinions 
about
>> on this proposal.
>>
>> Thanks,
>> Huafeng
>>

Re: Propose to combine module gearpump-daemon into gearpump-core

[Karol Brejna]

+1
I think moving EmbeddedCluster capabilities to gearpump-core makes
sense and simplifies things.

On Tue, Oct 11, 2016 at 7:04 AM, Manu Zhang  wrote:
> +1.  Integration tests for Beam Gearpump runner heavily depends on
> EmbeddedCluster. It's quite weird to add `gearpump-daemon` dependency to
> run gearpump examples.
>
> Thanks,
> Manu
>
> On Tue, Oct 11, 2016 at 10:17 AM Vincent Wang  wrote:
>
>> Hi all,
>>
>>   Currently we have an EmbeddedCluster in gearpump-daemon module which
>> allows user to run their gearpump application in a local JVM.
>>   Now suppose user want to implement an streaming application using
>> Gearpump and use the EmbeddedCluster to test it locally or even in an IDE.
>> So the application needs to have extra dependency on the gearpump-daemon
>> module to access EmbeddedCluster,  and user have to write code like
>> * if (debugMode) {*
>> *  localCluster = new EmbeddedCluster(akkaConf);*
>> *  localCluster.start();*
>> *}*
>> *ClientContext masterClient = null;*
>> *if (localCluster != null) {*
>> *  masterClient = localCluster.newClientContext();*
>> *} else {*
>> *  masterClient = new ClientContext(akkaConf);*
>> *}*
>> *masterClient.submit(app);*
>>
>>   Which is obviously tedious and inconvenient.
>>
>>   So we'd like to provide user with a implementation just like what Flink
>> did.  Basically we want to integrate EmbeddedCluster into ClientContext so
>> that user can switch the application running mode easily by some
>> configuration or decision made by ClientConext automatically.
>>   However, currently the ClientContext is in module gearpump-core and
>> EmbeddedCluster
>> has dependencies on Worker and Master daemon actors, which means these
>> classes need to be moved along with EmbeddedCluster and finally almost the
>> whole gearpump-daemon module need to be integrated into gearpump-core
>> module.
>>That's the side effect of achieving a more powerful ClientContext
>> integrated with a EmbeddedCluster and I'd like to hear your opinions about
>> on this proposal.
>>
>> Thanks,
>> Huafeng
>>

Re: Propose to combine module gearpump-daemon into gearpump-core

[Manu Zhang]

+1.  Integration tests for Beam Gearpump runner heavily depends on
EmbeddedCluster. It's quite weird to add `gearpump-daemon` dependency to
run gearpump examples.

Thanks,
Manu

On Tue, Oct 11, 2016 at 10:17 AM Vincent Wang  wrote:

> Hi all,
>
>   Currently we have an EmbeddedCluster in gearpump-daemon module which
> allows user to run their gearpump application in a local JVM.
>   Now suppose user want to implement an streaming application using
> Gearpump and use the EmbeddedCluster to test it locally or even in an IDE.
> So the application needs to have extra dependency on the gearpump-daemon
> module to access EmbeddedCluster,  and user have to write code like
> * if (debugMode) {*
> *  localCluster = new EmbeddedCluster(akkaConf);*
> *  localCluster.start();*
> *}*
> *ClientContext masterClient = null;*
> *if (localCluster != null) {*
> *  masterClient = localCluster.newClientContext();*
> *} else {*
> *  masterClient = new ClientContext(akkaConf);*
> *}*
> *masterClient.submit(app);*
>
>   Which is obviously tedious and inconvenient.
>
>   So we'd like to provide user with a implementation just like what Flink
> did.  Basically we want to integrate EmbeddedCluster into ClientContext so
> that user can switch the application running mode easily by some
> configuration or decision made by ClientConext automatically.
>   However, currently the ClientContext is in module gearpump-core and
> EmbeddedCluster
> has dependencies on Worker and Master daemon actors, which means these
> classes need to be moved along with EmbeddedCluster and finally almost the
> whole gearpump-daemon module need to be integrated into gearpump-core
> module.
>That's the side effect of achieving a more powerful ClientContext
> integrated with a EmbeddedCluster and I'd like to hear your opinions about
> on this proposal.
>
> Thanks,
> Huafeng
>

Propose to combine module gearpump-daemon into gearpump-core

[Vincent Wang]

Hi all,

  Currently we have an EmbeddedCluster in gearpump-daemon module which
allows user to run their gearpump application in a local JVM.
  Now suppose user want to implement an streaming application using
Gearpump and use the EmbeddedCluster to test it locally or even in an IDE.
So the application needs to have extra dependency on the gearpump-daemon
module to access EmbeddedCluster,  and user have to write code like
* if (debugMode) {*
*  localCluster = new EmbeddedCluster(akkaConf);*
*  localCluster.start();*
*}*
*ClientContext masterClient = null;*
*if (localCluster != null) {*
*  masterClient = localCluster.newClientContext();*
*} else {*
*  masterClient = new ClientContext(akkaConf);*
*}*
*masterClient.submit(app);*

  Which is obviously tedious and inconvenient.

  So we'd like to provide user with a implementation just like what Flink
did.  Basically we want to integrate EmbeddedCluster into ClientContext so
that user can switch the application running mode easily by some
configuration or decision made by ClientConext automatically.
  However, currently the ClientContext is in module gearpump-core and
EmbeddedCluster
has dependencies on Worker and Master daemon actors, which means these
classes need to be moved along with EmbeddedCluster and finally almost the
whole gearpump-daemon module need to be integrated into gearpump-core
module.
   That's the side effect of achieving a more powerful ClientContext
integrated with a EmbeddedCluster and I'd like to hear your opinions about
on this proposal.

Thanks,
Huafeng