Re: Should Flume integration be behind a profile?

2017-10-02 Thread Luciano Resende
On Mon, Oct 2, 2017 at 12:34 AM, Nick Pentreath wrote: > I'd agree with #1 or #2. Deprecation now seems fine. > > Perhaps this should be raised on the user list also? > > And perhaps it makes sense to look at moving the Flume support into Apache > Bahir if there is

Re: Should Flume integration be behind a profile?

2017-10-02 Thread Sean Owen
CCing user@ Yeah good point about perhaps moving the examples into the module itself. Actually removing it would be a long way off, no matter what. On Mon, Oct 2, 2017 at 8:35 AM Nick Pentreath wrote: > I'd agree with #1 or #2. Deprecation now seems fine. > > Perhaps

Re: Should Flume integration be behind a profile?

2017-10-02 Thread Nick Pentreath
I'd agree with #1 or #2. Deprecation now seems fine. Perhaps this should be raised on the user list also? And perhaps it makes sense to look at moving the Flume support into Apache Bahir if there is interest (I've cc'ed Bahir dev list here)? That way the current state of the connector could keep

Re: Should Flume integration be behind a profile?

2017-10-01 Thread Mridul Muralidharan
I agree, proposal 1 sounds better among the options. Regards, Mridul On Sun, Oct 1, 2017 at 3:50 PM, Reynold Xin wrote: > Probably should do 1, and then it is an easier transition in 3.0. > > On Sun, Oct 1, 2017 at 1:28 AM Sean Owen wrote: >> >> I

Re: Should Flume integration be behind a profile?

2017-10-01 Thread Reynold Xin
Probably should do 1, and then it is an easier transition in 3.0. On Sun, Oct 1, 2017 at 1:28 AM Sean Owen wrote: > I tried and failed to do this in > https://issues.apache.org/jira/browse/SPARK-22142 because it became clear > that the Flume examples would have to be removed

Re: Should Flume integration be behind a profile?

2017-10-01 Thread Sean Owen
I tried and failed to do this in https://issues.apache.org/jira/browse/SPARK-22142 because it became clear that the Flume examples would have to be removed to make this work, too. (Well, you can imagine other solutions with extra source dirs or modules for flume examples enabled by a profile, but

Re: Should Flume integration be behind a profile?

2017-09-26 Thread Mridul Muralidharan
Sounds good to me. +1 Regards, Mridul On Tue, Sep 26, 2017 at 2:36 AM, Sean Owen wrote: > Not a big deal, but I'm wondering whether Flume integration should at least > be opt-in and behind a profile? it still sees some use (at least on our end) > but not applicable to the

Re: Should Flume integration be behind a profile?

2017-09-26 Thread Ryan Blue
+1 for a Flume profile. On Tue, Sep 26, 2017 at 2:36 AM, Sean Owen wrote: > Not a big deal, but I'm wondering whether Flume integration should at > least be opt-in and behind a profile? it still sees some use (at least on > our end) but not applicable to the majority of

Should Flume integration be behind a profile?

2017-09-26 Thread Sean Owen
Not a big deal, but I'm wondering whether Flume integration should at least be opt-in and behind a profile? it still sees some use (at least on our end) but not applicable to the majority of users. Most other third-party framework integrations are behind a profile, like YARN, Mesos, Kinesis, Kafka