MartijnVisser commented on code in PR #28560:
URL: https://github.com/apache/flink/pull/28560#discussion_r4019787632
##########
flink-end-to-end-tests/test-scripts/common_s3_seaweedfs.sh:
##########
@@ -47,19 +47,20 @@ function s3_start {
export SEAWEEDFS_CONTAINER_ID=$(docker run -d \
-P \
-e "AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID" -e
"AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY" \
- chrislusf/seaweedfs:4.34 \
- server \
- -s3 \
+ chrislusf/seaweedfs:4.38 \
+ mini \
-s3.port=8333 \
- -dir=/data)
+ -dir=/data \
+ -bucket="$IT_CASE_S3_BUCKET")
while [[ "$(docker inspect -f {{.State.Running}} "$SEAWEEDFS_CONTAINER_ID")"
-ne "true" ]]; do
sleep 0.1
done
export S3_ENDPOINT="http://$(docker port "$SEAWEEDFS_CONTAINER_ID" 8333 |
grep -F '0.0.0.0' | sed s'/0\.0\.0\.0/localhost/')"
echo "Started seaweedfs @ $S3_ENDPOINT"
on_exit s3_stop
- while [[ "$(curl -s -o /dev/null -w '%{http_code}' "$S3_ENDPOINT/healthz")"
!= "200" ]]; do
+ # mini pre-creates the bucket and reports readiness only once all components
are up.
+ while ! docker logs "$SEAWEEDFS_CONTAINER_ID" 2>&1 | grep -q "All enabled
components are running and ready to use"; do
Review Comment:
`grep -q` exits early, so `docker logs` gets SIGPIPE and under the
`pipefail` from `common.sh` the loop sees 141 on a match. 182 of 200 iterations
here. Dropping `-q` fixes it.
##########
flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/SeaweedFsTestContainer.java:
##########
@@ -71,21 +69,16 @@ public SeaweedFsTestContainer(String defaultBucketName) {
withEnv(AWS_ACCESS_KEY_ID, this.accessKey);
withEnv(AWS_SECRET_ACCESS_KEY, this.secretKey);
withCommand(
- "server", "-s3", "-s3.port=" + DEFAULT_PORT, "-dir=" +
DEFAULT_STORAGE_DIRECTORY);
+ "mini",
Review Comment:
What does `mini` buy over `server -s3`? It also starts WebDAV, the admin UI
and a maintenance worker, and idles at 87 MiB against 49 MiB on the same image.
##########
flink-test-utils-parent/flink-test-utils-junit/src/main/java/org/apache/flink/util/DockerImageVersions.java:
##########
@@ -36,7 +36,7 @@ public class DockerImageVersions {
public static final String LOCALSTACK = "localstack/localstack:0.13.3";
- public static final String SEAWEEDFS = "chrislusf/seaweedfs:4.34";
+ public static final String SEAWEEDFS = "chrislusf/seaweedfs:4.38";
Review Comment:
I compared 4.34 and 4.38, the directory probes change as you describe.
Nothing in the suite restores from a savepoint though, so I think that fix
needs a case.
##########
flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/SeaweedFsTestContainer.java:
##########
@@ -71,21 +69,16 @@ public SeaweedFsTestContainer(String defaultBucketName) {
withEnv(AWS_ACCESS_KEY_ID, this.accessKey);
withEnv(AWS_SECRET_ACCESS_KEY, this.secretKey);
withCommand(
- "server", "-s3", "-s3.port=" + DEFAULT_PORT, "-dir=" +
DEFAULT_STORAGE_DIRECTORY);
+ "mini",
+ "-s3.port=" + DEFAULT_PORT,
+ "-dir=" + DEFAULT_STORAGE_DIRECTORY,
+ "-bucket=" + defaultBucketName);
+ // mini pre-creates the bucket and only reports readiness once every
component,
Review Comment:
mini warns and carries on when bucket creation fails, so the banner prints
with no bucket, as `-bucket=Test_Bucket` shows. Please keep a check after start.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
