Re: Proposal for apex/malhar extensions

[Chinmay Kolhatkar]

Thanks everyone for the response.
I've created a Jira for this:
https://issues.apache.org/jira/browse/APEXCORE-576

Feel free to contribute in any way possible to increasing contributions and
usage of Apache Apex.

Thanks,
Chinmay.


On Thu, Nov 17, 2016 at 2:56 AM, Pramod Immaneni 
wrote:

> Yes, I think it could be useful for core as well.
>
> On Wed, Nov 16, 2016 at 11:19 AM, Chinmay Kolhatkar 
> wrote:
>
> > @sanjay, yes we can define the process around this.
> >
> > @pramod, Well, I said apex-malhar because the extensions can be operators
> > and and other plugins to apex engine.
> > Do you see the use of this for apex-core as well?
> >
> > -Chinmay.
> >
> >
> > On Wed, Nov 16, 2016 at 7:24 PM, Pramod Immaneni  >
> > wrote:
> >
> > > So it would be like a yellow pages for the apex ecosystem. Sounds like
> a
> > > good idea. Why limit it to malhar?
> > >
> > > On Wed, Nov 16, 2016 at 3:17 AM, Chinmay Kolhatkar  >
> > > wrote:
> > >
> > > > Dear Community,
> > > >
> > > > This is in relation to malhar cleanup work that is ongoing.
> > > >
> > > > In one of the talks during Apache BigData Europe, I got to know about
> > > > Spark-Packages (https://spark-packages.org/) (I believe lot of you
> > must
> > > be
> > > > aware of it).
> > > > Spark package is basically functionality over and above and using
> Spark
> > > > core functionality. The spark packages can initially present in
> > someone's
> > > > public repository and one could register that with
> > > > https://spark-packages.org/ and later on as it matures and finds
> more
> > > use,
> > > > it gets consumed in mainstream Spark repository and releases.
> > > >
> > > > I found this idea quite interesting to keep our apex-malhar releases
> > > > cleaner.
> > > >
> > > > One could have extension to apex-malhar in their own repository and
> > just
> > > > register itself with Apache Apex. As it matures and find more and
> more
> > > use
> > > > we can consume that in mainstream releases.
> > > > Advantages to this are multiple:
> > > > 1. The entry point for registering extensions with Apache Apex can be
> > > > minimal. This way we get more indirect contributions.
> > > > 2. Faster way to add more feature in the project.
> > > > 3. We keep our releases cleaner.
> > > > 4. One could progress on feature-set faster balancing both Apache Way
> > as
> > > > well as their own Enterprise Interests.
> > > >
> > > > Please share your thoughts on this.
> > > >
> > > > Thanks,
> > > > Chinmay.
> > > >
> > >
> >
>

Re: Proposal for apex/malhar extensions

[Pramod Immaneni]

Yes, I think it could be useful for core as well.

On Wed, Nov 16, 2016 at 11:19 AM, Chinmay Kolhatkar 
wrote:

> @sanjay, yes we can define the process around this.
>
> @pramod, Well, I said apex-malhar because the extensions can be operators
> and and other plugins to apex engine.
> Do you see the use of this for apex-core as well?
>
> -Chinmay.
>
>
> On Wed, Nov 16, 2016 at 7:24 PM, Pramod Immaneni 
> wrote:
>
> > So it would be like a yellow pages for the apex ecosystem. Sounds like a
> > good idea. Why limit it to malhar?
> >
> > On Wed, Nov 16, 2016 at 3:17 AM, Chinmay Kolhatkar 
> > wrote:
> >
> > > Dear Community,
> > >
> > > This is in relation to malhar cleanup work that is ongoing.
> > >
> > > In one of the talks during Apache BigData Europe, I got to know about
> > > Spark-Packages (https://spark-packages.org/) (I believe lot of you
> must
> > be
> > > aware of it).
> > > Spark package is basically functionality over and above and using Spark
> > > core functionality. The spark packages can initially present in
> someone's
> > > public repository and one could register that with
> > > https://spark-packages.org/ and later on as it matures and finds more
> > use,
> > > it gets consumed in mainstream Spark repository and releases.
> > >
> > > I found this idea quite interesting to keep our apex-malhar releases
> > > cleaner.
> > >
> > > One could have extension to apex-malhar in their own repository and
> just
> > > register itself with Apache Apex. As it matures and find more and more
> > use
> > > we can consume that in mainstream releases.
> > > Advantages to this are multiple:
> > > 1. The entry point for registering extensions with Apache Apex can be
> > > minimal. This way we get more indirect contributions.
> > > 2. Faster way to add more feature in the project.
> > > 3. We keep our releases cleaner.
> > > 4. One could progress on feature-set faster balancing both Apache Way
> as
> > > well as their own Enterprise Interests.
> > >
> > > Please share your thoughts on this.
> > >
> > > Thanks,
> > > Chinmay.
> > >
> >
>

Re: Proposal for apex/malhar extensions

[Sandesh Hegde]

Do we have any projects today that can benefit from this setup?
Earlier in this mail thread, we discussed "contrib (low bar) & graduation"
in Malhar, that is not sufficient?

On Wed, Nov 16, 2016 at 11:19 AM Chinmay Kolhatkar 
wrote:

> @sanjay, yes we can define the process around this.
>
> @pramod, Well, I said apex-malhar because the extensions can be operators
> and and other plugins to apex engine.
> Do you see the use of this for apex-core as well?
>
> -Chinmay.
>
>
> On Wed, Nov 16, 2016 at 7:24 PM, Pramod Immaneni 
> wrote:
>
> > So it would be like a yellow pages for the apex ecosystem. Sounds like a
> > good idea. Why limit it to malhar?
> >
> > On Wed, Nov 16, 2016 at 3:17 AM, Chinmay Kolhatkar 
> > wrote:
> >
> > > Dear Community,
> > >
> > > This is in relation to malhar cleanup work that is ongoing.
> > >
> > > In one of the talks during Apache BigData Europe, I got to know about
> > > Spark-Packages (https://spark-packages.org/) (I believe lot of you
> must
> > be
> > > aware of it).
> > > Spark package is basically functionality over and above and using Spark
> > > core functionality. The spark packages can initially present in
> someone's
> > > public repository and one could register that with
> > > https://spark-packages.org/ and later on as it matures and finds more
> > use,
> > > it gets consumed in mainstream Spark repository and releases.
> > >
> > > I found this idea quite interesting to keep our apex-malhar releases
> > > cleaner.
> > >
> > > One could have extension to apex-malhar in their own repository and
> just
> > > register itself with Apache Apex. As it matures and find more and more
> > use
> > > we can consume that in mainstream releases.
> > > Advantages to this are multiple:
> > > 1. The entry point for registering extensions with Apache Apex can be
> > > minimal. This way we get more indirect contributions.
> > > 2. Faster way to add more feature in the project.
> > > 3. We keep our releases cleaner.
> > > 4. One could progress on feature-set faster balancing both Apache Way
> as
> > > well as their own Enterprise Interests.
> > >
> > > Please share your thoughts on this.
> > >
> > > Thanks,
> > > Chinmay.
> > >
> >
>

Re: Proposal for apex/malhar extensions

[Chinmay Kolhatkar]

@sanjay, yes we can define the process around this.

@pramod, Well, I said apex-malhar because the extensions can be operators
and and other plugins to apex engine.
Do you see the use of this for apex-core as well?

-Chinmay.


On Wed, Nov 16, 2016 at 7:24 PM, Pramod Immaneni 
wrote:

> So it would be like a yellow pages for the apex ecosystem. Sounds like a
> good idea. Why limit it to malhar?
>
> On Wed, Nov 16, 2016 at 3:17 AM, Chinmay Kolhatkar 
> wrote:
>
> > Dear Community,
> >
> > This is in relation to malhar cleanup work that is ongoing.
> >
> > In one of the talks during Apache BigData Europe, I got to know about
> > Spark-Packages (https://spark-packages.org/) (I believe lot of you must
> be
> > aware of it).
> > Spark package is basically functionality over and above and using Spark
> > core functionality. The spark packages can initially present in someone's
> > public repository and one could register that with
> > https://spark-packages.org/ and later on as it matures and finds more
> use,
> > it gets consumed in mainstream Spark repository and releases.
> >
> > I found this idea quite interesting to keep our apex-malhar releases
> > cleaner.
> >
> > One could have extension to apex-malhar in their own repository and just
> > register itself with Apache Apex. As it matures and find more and more
> use
> > we can consume that in mainstream releases.
> > Advantages to this are multiple:
> > 1. The entry point for registering extensions with Apache Apex can be
> > minimal. This way we get more indirect contributions.
> > 2. Faster way to add more feature in the project.
> > 3. We keep our releases cleaner.
> > 4. One could progress on feature-set faster balancing both Apache Way as
> > well as their own Enterprise Interests.
> >
> > Please share your thoughts on this.
> >
> > Thanks,
> > Chinmay.
> >
>

Re: Proposal for apex/malhar extensions

[Pramod Immaneni]

So it would be like a yellow pages for the apex ecosystem. Sounds like a
good idea. Why limit it to malhar?

On Wed, Nov 16, 2016 at 3:17 AM, Chinmay Kolhatkar 
wrote:

> Dear Community,
>
> This is in relation to malhar cleanup work that is ongoing.
>
> In one of the talks during Apache BigData Europe, I got to know about
> Spark-Packages (https://spark-packages.org/) (I believe lot of you must be
> aware of it).
> Spark package is basically functionality over and above and using Spark
> core functionality. The spark packages can initially present in someone's
> public repository and one could register that with
> https://spark-packages.org/ and later on as it matures and finds more use,
> it gets consumed in mainstream Spark repository and releases.
>
> I found this idea quite interesting to keep our apex-malhar releases
> cleaner.
>
> One could have extension to apex-malhar in their own repository and just
> register itself with Apache Apex. As it matures and find more and more use
> we can consume that in mainstream releases.
> Advantages to this are multiple:
> 1. The entry point for registering extensions with Apache Apex can be
> minimal. This way we get more indirect contributions.
> 2. Faster way to add more feature in the project.
> 3. We keep our releases cleaner.
> 4. One could progress on feature-set faster balancing both Apache Way as
> well as their own Enterprise Interests.
>
> Please share your thoughts on this.
>
> Thanks,
> Chinmay.
>

Re: Proposal for apex/malhar extensions

[Sanjay Pujare]

+1 for the idea. Will be good to describe the registration mechanism to be used.

On 11/16/16, 3:17 AM, "Chinmay Kolhatkar"  wrote:

Dear Community,

This is in relation to malhar cleanup work that is ongoing.

In one of the talks during Apache BigData Europe, I got to know about
Spark-Packages (https://spark-packages.org/) (I believe lot of you must be
aware of it).
Spark package is basically functionality over and above and using Spark
core functionality. The spark packages can initially present in someone's
public repository and one could register that with
https://spark-packages.org/ and later on as it matures and finds more use,
it gets consumed in mainstream Spark repository and releases.

I found this idea quite interesting to keep our apex-malhar releases
cleaner.

One could have extension to apex-malhar in their own repository and just
register itself with Apache Apex. As it matures and find more and more use
we can consume that in mainstream releases.
Advantages to this are multiple:
1. The entry point for registering extensions with Apache Apex can be
minimal. This way we get more indirect contributions.
2. Faster way to add more feature in the project.
3. We keep our releases cleaner.
4. One could progress on feature-set faster balancing both Apache Way as
well as their own Enterprise Interests.

Please share your thoughts on this.

Thanks,
Chinmay.

Re: Proposal for apex/malhar extensions

[Amol Kekre]

+1

Thks
Amol

On Wed, Nov 16, 2016 at 5:37 AM, AJAY GUPTA  wrote:

> +1
> This is a good idea.
>
> Ajay
>
> On Wed, Nov 16, 2016 at 4:47 PM, Chinmay Kolhatkar 
> wrote:
>
> > Dear Community,
> >
> > This is in relation to malhar cleanup work that is ongoing.
> >
> > In one of the talks during Apache BigData Europe, I got to know about
> > Spark-Packages (https://spark-packages.org/) (I believe lot of you must
> be
> > aware of it).
> > Spark package is basically functionality over and above and using Spark
> > core functionality. The spark packages can initially present in someone's
> > public repository and one could register that with
> > https://spark-packages.org/ and later on as it matures and finds more
> use,
> > it gets consumed in mainstream Spark repository and releases.
> >
> > I found this idea quite interesting to keep our apex-malhar releases
> > cleaner.
> >
> > One could have extension to apex-malhar in their own repository and just
> > register itself with Apache Apex. As it matures and find more and more
> use
> > we can consume that in mainstream releases.
> > Advantages to this are multiple:
> > 1. The entry point for registering extensions with Apache Apex can be
> > minimal. This way we get more indirect contributions.
> > 2. Faster way to add more feature in the project.
> > 3. We keep our releases cleaner.
> > 4. One could progress on feature-set faster balancing both Apache Way as
> > well as their own Enterprise Interests.
> >
> > Please share your thoughts on this.
> >
> > Thanks,
> > Chinmay.
> >
>

Re: Proposal for apex/malhar extensions

[AJAY GUPTA]

+1
This is a good idea.

Ajay

On Wed, Nov 16, 2016 at 4:47 PM, Chinmay Kolhatkar 
wrote:

> Dear Community,
>
> This is in relation to malhar cleanup work that is ongoing.
>
> In one of the talks during Apache BigData Europe, I got to know about
> Spark-Packages (https://spark-packages.org/) (I believe lot of you must be
> aware of it).
> Spark package is basically functionality over and above and using Spark
> core functionality. The spark packages can initially present in someone's
> public repository and one could register that with
> https://spark-packages.org/ and later on as it matures and finds more use,
> it gets consumed in mainstream Spark repository and releases.
>
> I found this idea quite interesting to keep our apex-malhar releases
> cleaner.
>
> One could have extension to apex-malhar in their own repository and just
> register itself with Apache Apex. As it matures and find more and more use
> we can consume that in mainstream releases.
> Advantages to this are multiple:
> 1. The entry point for registering extensions with Apache Apex can be
> minimal. This way we get more indirect contributions.
> 2. Faster way to add more feature in the project.
> 3. We keep our releases cleaner.
> 4. One could progress on feature-set faster balancing both Apache Way as
> well as their own Enterprise Interests.
>
> Please share your thoughts on this.
>
> Thanks,
> Chinmay.
>

Proposal for apex/malhar extensions

[Chinmay Kolhatkar]

Dear Community,

This is in relation to malhar cleanup work that is ongoing.

In one of the talks during Apache BigData Europe, I got to know about
Spark-Packages (https://spark-packages.org/) (I believe lot of you must be
aware of it).
Spark package is basically functionality over and above and using Spark
core functionality. The spark packages can initially present in someone's
public repository and one could register that with
https://spark-packages.org/ and later on as it matures and finds more use,
it gets consumed in mainstream Spark repository and releases.

I found this idea quite interesting to keep our apex-malhar releases
cleaner.

One could have extension to apex-malhar in their own repository and just
register itself with Apache Apex. As it matures and find more and more use
we can consume that in mainstream releases.
Advantages to this are multiple:
1. The entry point for registering extensions with Apache Apex can be
minimal. This way we get more indirect contributions.
2. Faster way to add more feature in the project.
3. We keep our releases cleaner.
4. One could progress on feature-set faster balancing both Apache Way as
well as their own Enterprise Interests.

Please share your thoughts on this.

Thanks,
Chinmay.