[
https://issues.apache.org/jira/browse/FLINK-34508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17837486#comment-17837486
]
Muhammet Orazov edited comment on FLINK-34508 at 4/16/24 2:49 AM:
------------------------------------------------------------------
Hey all,
I have add changes to remove the `IT_CASE_S3_*` variables. There is one issue I
want to discuss.
There is an integration test case in `flink-yarn` module, it is
`[YarnFileStateTestS3ITCase.java|https://github.com/apache/flink/blob/master/flink-yarn/src/test/java/org/apache/flink/yarn/YarnFileStageTestS3ITCase.java]`.
This file also depends on the S3 variables. It also uses the S3 Native (s3n)
in one of the tests, when run this test fails.
However, it is not enabled in the CI. The `flink-yarn` module is not used in
the tests (or I could not find).
The NativeS3Filesystem does not work with Minio, since it directly goes to the
default `.amazonaws.` endpoint. The S3 native library is deprecated in the
3.3.4+ version of `hadoop-aws` library, but not used for yarn module for now.
I would suggest to remove the `s3n` test from the YarnFileStageTestS3ITCase
test. What do you think?
was (Author: JIRAUSER303922):
Hey all,
I have add changes to remove the `IT_CASE_S3_*` variables. There is one issue I
want to discuss.
There is an integration test case in `flink-yarn` module, it is
`YarnFileStateTestS3ITCase.java`. This file also depends on the S3 variables.
It also uses the S3 Native (s3n) in one of the tests, when run this test fails.
However, it is not enabled in the CI. The `flink-yarn` module is not used in
the tests (or I could not find).
The NativeS3Filesystem does not work with Minio, since it directly goes to the
default `.amazonaws.` endpoint. The S3 native library is deprecated in the
3.3.4+ version of `hadoop-aws` library, but not used for yarn module for now.
I would suggest to remove the `s3n` test from the YarnFileStageTestS3ITCase
test. What do you think?
> Migrate S3-related ITCases and e2e tests to Minio
> --------------------------------------------------
>
> Key: FLINK-34508
> URL: https://issues.apache.org/jira/browse/FLINK-34508
> Project: Flink
> Issue Type: Sub-task
> Components: Build System / CI
> Affects Versions: 1.19.0, 1.18.1, 1.20.0
> Reporter: Matthias Pohl
> Assignee: Muhammet Orazov
> Priority: Major
> Labels: github-actions, pull-request-available
>
> Anything that uses {{org.apache.flink.testutils.s3.S3TestCredentials}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
