Thanks everybody for your comments.

I opened FLINK-4676 [1] for merging the connector modules.

[1] https://issues.apache.org/jira/browse/FLINK-4676

2016-09-26 13:17 GMT+02:00 Robert Metzger <rmetz...@apache.org>:

> +1 good suggestion.
>
> On Mon, Sep 26, 2016 at 1:03 PM, Stephan Ewen <se...@apache.org> wrote:
>
> > The module would have both dependencies, but both are provided anyways,
> so
> > that would not be much of an issue, I think.
> >
> > On Mon, Sep 26, 2016 at 12:25 PM, Till Rohrmann <trohrm...@apache.org>
> > wrote:
> >
> > > I think this only holds true for modules which depend on the batch or
> > > streaming counter part, respectively. We could refactor these modules
> by
> > > pulling out common types which are independent of streaming/batch and
> are
> > > used by the batch and streaming module.
> > >
> > > Cheers,
> > > Till
> > >
> > > On Fri, Sep 23, 2016 at 11:15 AM, Aljoscha Krettek <
> aljos...@apache.org>
> > > wrote:
> > >
> > > > I don't think it's that easy. The streaming connectors have
> > > flink-streaming
> > > > as dependency while the batch connectors have the batch dependencies.
> > > >
> > > > Combining them would mean that users always have all dependencies,
> > right?
> > > >
> > > > On Thu, 22 Sep 2016 at 15:41 Stephan Ewen <se...@apache.org> wrote:
> > > >
> > > > > +1 for Fabian's suggestion
> > > > >
> > > > > On Thu, Sep 22, 2016 at 3:25 PM, Swapnil Chougule <
> > > > the.swapni...@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > +1
> > > > > > It will be good to have one module flink-connectors (union of
> > > streaming
> > > > > and
> > > > > > batch connectors).
> > > > > >
> > > > > > Regards,
> > > > > > Swapnil
> > > > > >
> > > > > > On Thu, Sep 22, 2016 at 6:35 PM, Fabian Hueske <
> fhue...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > Hi everybody,
> > > > > > >
> > > > > > > right now, we have two separate Maven modules for batch and
> > > streaming
> > > > > > > connectors (flink-batch-connectors and
> > flink-streaming-connectors)
> > > > that
> > > > > > > contain modules for the individual external systems and storage
> > > > formats
> > > > > > > such as HBase, Cassandra, Avro, Elasticsearch, etc.
> > > > > > >
> > > > > > > Some of these systems can be used in streaming as well as batch
> > > jobs
> > > > as
> > > > > > for
> > > > > > > instance HBase, Cassandra, and Elasticsearch. However, due to
> the
> > > > > > separate
> > > > > > > main modules for streaming and batch connectors, we currently
> > need
> > > to
> > > > > > > decide where to put a connector. For example, the
> > > > > > flink-connector-cassandra
> > > > > > > module is located in flink-streaming-connectors but includes a
> > > > > > > CassandraInputFormat and CassandraOutputFormat (i.e., a batch
> > > source
> > > > > and
> > > > > > > sink).
> > > > > > >
> > > > > > > In my opinion, it would be better to just merge
> > > > flink-batch-connectors
> > > > > > and
> > > > > > > flink-streaming-connectors into a joint flink-connectors
> module.
> > > > > > >
> > > > > > > This would be only an internal restructuring of code and not be
> > > > visible
> > > > > > to
> > > > > > > users (unless we change the module names of the individual
> > > connectors
> > > > > > which
> > > > > > > is not necessary, IMO).
> > > > > > >
> > > > > > > What do others think?
> > > > > > >
> > > > > > > Best, Fabian
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Reply via email to