On 1 June 2016 at 06:43, Jean-Baptiste Onofré <[email protected]> wrote:

> Hi Luciano,
>
> it looks good to me. The challenge could be when accepting extensions for
> different projects (Spark, Flink, Beam for instance), not necessary sharing
> the same dependencies/technologies. We will have to provide kind of BoM for
> the dependencies.
>

ooh, have we got that in scope. nice. I'll be at berlin buzzwords next week
and talk to the flink team, though I think their stage of project is likely
to have this plugin problem.


Something else to consider: use of package names from other projects in
plugins. I'm thinking generally of spark here, where you end up having to
put things into org.apache.spark or packages below in order to get at
private[spark] stuff. Simiarly, spark itself puts some stuff into
org.apache.hive

While this is currently necessary, it's dangerous as it creates brittle
code —and when it does break, you can't really expect the upstream project
to care that much. Really we should be advocating making stuff (spark:
management, history) public in some form, even if there is an explicit
compatibility statement around plugins saying "you are expected to rebuild
this even for point releases". That's something to discuss with the
projects, with the arguments in favour being: things are less likely to
break, and when they do break, the stack traces won't have your package
names at the top.

The cost: more obligation to expose code with stability guarantees.

FWIW, I've started a discussion on yarn-dev about the semantics of @Stable
classes there, as SLIDER-1130 showed that it didn't meet my expectations.
We may end up with some attribute to indicate "stable to use, evolving to
extend", and one for @Experimental —may go away without any warning.


>
> Regards
> JB
>
>
> On 06/01/2016 01:40 AM, Luciano Resende wrote:
>
>> I have drafted a very basic guideline for contributing new extensions for
>> Apache Bahir. My main goal is to use that as the start point to a
>> community
>> discussion that can flush out some details and make the process for
>> proposing, accepting and retiring a extension very transparent.
>>
>> Please review the initial draft at :
>> http://bahir.apache.org/contributing-extensions/
>>
>> And let's use this thread for any question, comments, feedback, etc.
>>
>>
> --
> Jean-Baptiste Onofré
> [email protected]
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Reply via email to