Tom Ho created FLINK-38949:
------------------------------
Summary: unpack_build_artifact.sh script speed improvement
Key: FLINK-38949
URL: https://issues.apache.org/jira/browse/FLINK-38949
Project: Flink
Issue Type: Improvement
Components: Build System / CI
Affects Versions: 2.1.1, 2.2.0, 2.0.1, 2.1.0
Reporter: Tom Ho
Current unpack_build_artifact.sh script use hardcoded sleep time, find+chmod
approach is slow because it will spin up one thread for chmod for every file.
Spinning up thread will take very long if the number of file found is large. To
address this, we can use code like this, which will join all file as a list and
pass those file list into one thread, which is tested in flink repo, it will
save 4 min of time with the same result.
{code:java}
find . -type f -name '*.proto' -exec touch -d "$PROTO_TIME" {} + {code}
https://github.com/apache/flink/blob/9e2d2b8ed0d90d1c6277d5eb630b47f9a4d5fc54/tools/azure-pipelines/unpack_build_artifact.sh#L38-L48
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
