1) That's a fair point. So far everything that came out of the connector
e2e framework was unfortunately somehow coupled to connectors
conceptually. I wouldn't mind changing that, because as you said this
could also be useful for other things, but it needs a more holistic view
over the entire story.
That said I'm not sure how usable these things are at this time outside
of the connector e2e framework, hence why I'd also consider
connector-test-utils as a valid place _for now_.
2) As far as I'm concerned flink-end-to-end-tests-common is a deprecated
module containing legacy code. It primarily contains the java based e2e
framework I wrote years back which never got any traction, and in all
likelihood will not gain traction in the future. Unfortunately the
connector framework was written completely independent from what is
already there.
3) Generally I agree. That the environment has a dependency on
connector-test-utils was one of the reasons why I proposed to move it there.
In general we need to re-think how we organize e2e tests/utils, and
whether we even want to differentiate between ITCases and e2e tests,
because as is the tests are all over the place (some in the modules,
some in flink-e2e-tests).
On 25/05/2022 17:06, Alexander Fedulov wrote:
I see. A couple of considerations:
1. I would assume FlinkContainerTestEnvironment could be used for more
things than solely connectors, hence it should ideally not be placed into
flink-connector-test-utils
2. If we put it and its dependencies into flink-test-utils, what would be
the purpose of the existence of the flink-end-to-end-tests-common module,
if we then will also have E2E utilities in flink-test-utils? (other than
for segregating stuff that isn't really in a state where we want to share
it )
3. If (1.) is true, it seems that ClusterControllable, TestEnvironment, and
TestEnvironmentSettings are currently misplaced in
flink-connector-test-utils and should be moved together
with FlinkContainerTestEnvironment.
Let me know what you think.
Thanks,
Alexander Fedulov
On Wed, May 25, 2022 at 3:56 PM Chesnay Schepler <ches...@apache.org> wrote:
I'd be more inclined to move the FlinkContainerTestEnvironment to
flink-(connector-)test-utils for the time being, because the vast
majority of stuff in flink-end-to-end-tests-common isn't really in a
state where we want to share it.
On 25/05/2022 11:55, Alexander Fedulov wrote:
Hi everyone,
As part of the efforts to externalize the connectors, it became evident
that some test utilities that reside in Flink will be required in the
external connector repositories. For instance,
FlinkContainerTestEnvironment
and its dependencies reside in the flink-end-to-end-tests-common module
that
is currently not published to Maven (there is only an older version from
Cloudera available) [1].
I would like to propose to also publish this dependency to Maven. What do
you think?
[1] https://mvnrepository.com/search?q=flink-end-to-end-tests-common
Best,
Alexander Fedulov
