[
https://issues.apache.org/jira/browse/FLINK-38949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tom Ho updated FLINK-38949:
---------------------------
Summary: unpack_build_artifact.sh script logic and speed improvement (was:
unpack_build_artifact.sh script speed improvement)
> unpack_build_artifact.sh script logic and speed improvement
> -----------------------------------------------------------
>
> Key: FLINK-38949
> URL: https://issues.apache.org/jira/browse/FLINK-38949
> Project: Flink
> Issue Type: Improvement
> Components: Build System / CI
> Affects Versions: 2.1.0, 2.0.1, 2.2.0, 2.1.1
> Reporter: Tom Ho
> Priority: Minor
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Current unpack_build_artifact.sh script use hardcoded sleep time, find+chmod
> approach is slow because it will spin up one thread for chmod for every file.
> Spinning up thread will take very long if the number of file found is large.
> To address this, we can use code like this, which will join all file as a
> list and pass those file list into one thread, which is tested in flink repo,
> it will save 4 min of time with the same result.
> {code:java}
> find . -type f -name '*.proto' -exec touch -d "$PROTO_TIME" {} + {code}
> https://github.com/apache/flink/blob/9e2d2b8ed0d90d1c6277d5eb630b47f9a4d5fc54/tools/azure-pipelines/unpack_build_artifact.sh#L38-L48
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
