Thanks a lot for your summary Chesnay.
I agree with you that we have no consensus in the community for splitting
up the repository immediately, and I agree with you that we should have a
separate discussion about reducing the build time (which is already making
good progress).
Also, I will keep th
Let's recap a bit:
Several people have raised the argument that build times can be kept in
check via other means (mostly differential builds via some means, be it
custom scripts or switching to gradle). I will start a separate
discussion thread on this topic, since it is a useful discussion in
-1 for rushing into conclusions that we need to split the repo before
saturating our efforts in improving current build/CI mechanism. Besides all
the build system issues mentioned above (no incremental builds, no
flexibility to build only docs or subsets of components), it's hard to keep
configurat
I split small and medium-sized repositories in several projects for various
reasons. In general, the more mature a project, the fewer pain after the
split. If interfaces are somewhat stable, it's naturally easier to work in
a distributed manner.
However, projects should be split for the right reas
Thanks a lot for starting the discussion Chesnay!
I would like to throw in another aspect into the discussion: What if we
consider this repo split as a first step towards making connectors, machine
learning, gelly, table/SQL? independent projects within the ASF, with their
own mailing lists, comm
Just in case we decide to pursue the repo split in the end, some thoughts
on Chesnay's questions:
(1) Git History
We can also use "git filter-branch" to rewrite the history to only contain
the connectors.
It changes commit hashes, but not sure that this is a problem. The commit
hashes are still v
Thank you all for the good discussion.
I was one of the folks that thinking about such a repository split together
with Chesnay, but due to lack of prior experience, happy to hear all the
points that
Let's investigate a bit what would be alternatives to this that solve the
two problems.
(1) B
Apart from a technical explanation, the initial suggestion does not propose how
the repository should be split up. The only meaningful split I see is for the
connectors.
This discussion dates back a few years:
https://lists.apache.org/thread.html/4ee502667a5801d23d76a01406e747e1a934417dc67ef7d2
Hi folks,
Thanks for bringing this discussion Chesnay.
+1 for the motivation. It's really a bad experience of waiting Travis
building for a long time.
WRT the solution, personally I agree with Dawid/David.
IMO the biggest benefit of splitting repository is reducing build time. I
think it could
Hi,
Re Jark’s:
> Ad. 2 I can't see how unstable connectors tests can be fixed more quickly
> after moved to a separate repositories.
It’s more about probability of intermittent failures across all of the modules
adding up, causing whole build to fail almost all the time. With separate
reposit
Hi,
First of all, I agree with Dawid and David's point.
I will share some experience on the repository split. We have been through
it for Alibaba Blink, which is the most worthwhile project to learn from I
think.
We split Blink project into "blink-connectors" and "blink", but we didn't
get much b
I pretty much agree with your points Dav/wid. Some problems which we want
to solve with a respository split are clearly caused by the existing build
system (no incremental builds, not enough flexibility to only build a
subset of modules). Given that a repository split would be a major
endeavour wit
Hey,
I retract my +1 (at least temporarily, until we discuss about alternative
solutions).
>> I would like to also raise an additional issue: currently quite some bugs
>> (like release blockers [1]) are being discovered by ITCases of the
>> connectors. It means that at least initially, the ma
+1 for the motivation, -1 for the solution as all of the problems mention
above can be addressed with the mono-repo as well.
Multiple repositories:
1) This creates a big pain in case of change that targets code base in
multiple repositories. Change needs to be split in multiple PRs, that need
to b
First of all I don't have much(if not at all) experience with working
with a multi repository project of Flink's size. I would like to mention
a few thoughts of mine, though. In general I am slightly against
splitting the repository. I fear that what we actually want to do is to
introduce double st
> I would like to also raise an additional issue: currently quite some
bugs (like release blockers [1]) are being discovered by ITCases of the
connectors. It means that at least initially, the main repository will
lose some test coverage.
True, but I think this is more a symptom of us not pro
Hi,
Thanks for proposing and writing this down Chesney.
Generally speaking +1 from my side for the idea. It will create additional pain
for cross repository development, like some new feature in connectors that need
some change in the main repository. I’ve worked in such setup before and the
t
Hello everyone,
The Flink project sees an ever-increasing amount of dev activity, both
in terms of reworked and new features.
This is of course an excellent situation to be in, but we are getting to
a point where the associate downsides are becoming increasingly troublesome.
The ever increa
18 matches
Mail list logo
