This is an automated email from the ASF dual-hosted git repository.
kou pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow.git
The following commit(s) were added to refs/heads/main by this push:
new 397b3d00230 GH-51090: [CI][C++] Fix shellcheck errors in
cpp/build-support/fuzzing/generate_corpuses.sh (#51091)
397b3d00230 is described below
commit 397b3d0023030f6c6bc69d214ea7b27a687256f8
Author: Hiroyuki Sato <[email protected]>
AuthorDate: Tue Sep 8 15:55:15 2026 +0900
GH-51090: [CI][C++] Fix shellcheck errors in
cpp/build-support/fuzzing/generate_corpuses.sh (#51091)
### Rationale for this change
This is the sub issue #44748.
* SC2046: Quote this to prevent word splitting.
* SC2128: Expanding an array without an index only gives the element in the
index 0.
* SC2086: Double quote to prevent globbing and word splitting.
```
In generate_corpuses.sh line 32:
ARROW_ROOT=$(cd $(dirname "$BASH_SOURCE")/../../..; pwd)
^-----------------------^ SC2046 (warning): Quote this to
prevent word splitting.
^----------^ SC2128 (warning): Expanding an
array without an index only gives the first element.
In generate_corpuses.sh line 43:
${OUT}/arrow-ipc-generate-fuzz-corpus -stream ${CORPUS_DIR}
^----^ SC2086 (info): Double quote to prevent globbing and word splitting.
Did you mean:
"${OUT}"/arrow-ipc-generate-fuzz-corpus -stream ${CORPUS_DIR}
In generate_corpuses.sh line 45:
IPC_INTEGRATION_FILES=$(find
${ARROW_ROOT}/testing/data/arrow-ipc-stream/integration -name "*.stream")
^-----------^ SC2086 (info): Double quote to
prevent globbing and word splitting.
Did you mean:
IPC_INTEGRATION_FILES=$(find
"${ARROW_ROOT}"/testing/data/arrow-ipc-stream/integration -name "*.stream")
In generate_corpuses.sh line 49:
cp --backup=numbered ${IPC_INTEGRATION_FILES} ${CORPUS_DIR}
^----------------------^ SC2086 (info): Double quote
to prevent globbing and word splitting.
Did you mean:
cp --backup=numbered "${IPC_INTEGRATION_FILES}" ${CORPUS_DIR}
In generate_corpuses.sh line 50:
${ARROW_CPP}/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
${OUT}/arrow-ipc-stream-fuzz_seed_corpus.zip
^----------^ SC2086 (info): Double quote to prevent globbing and word
splitting.
^----^
SC2086 (info): Double quote to prevent globbing and word splitting.
Did you mean:
"${ARROW_CPP}"/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
"${OUT}"/arrow-ipc-stream-fuzz_seed_corpus.zip
In generate_corpuses.sh line 53:
${OUT}/arrow-ipc-generate-fuzz-corpus -file ${CORPUS_DIR}
^----^ SC2086 (info): Double quote to prevent globbing and word splitting.
Did you mean:
"${OUT}"/arrow-ipc-generate-fuzz-corpus -file ${CORPUS_DIR}
In generate_corpuses.sh line 54:
IPC_INTEGRATION_FILES=$(find
${ARROW_ROOT}/testing/data/arrow-ipc-stream/integration -name "*.arrow_file")
^-----------^ SC2086 (info): Double quote to
prevent globbing and word splitting.
Did you mean:
IPC_INTEGRATION_FILES=$(find
"${ARROW_ROOT}"/testing/data/arrow-ipc-stream/integration -name "*.arrow_file")
In generate_corpuses.sh line 56:
cp --backup=numbered ${IPC_INTEGRATION_FILES} ${CORPUS_DIR}
^----------------------^ SC2086 (info): Double quote
to prevent globbing and word splitting.
Did you mean:
cp --backup=numbered "${IPC_INTEGRATION_FILES}" ${CORPUS_DIR}
In generate_corpuses.sh line 57:
${ARROW_CPP}/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
${OUT}/arrow-ipc-file-fuzz_seed_corpus.zip
^----------^ SC2086 (info): Double quote to prevent globbing and word
splitting.
^----^
SC2086 (info): Double quote to prevent globbing and word splitting.
Did you mean:
"${ARROW_CPP}"/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
"${OUT}"/arrow-ipc-file-fuzz_seed_corpus.zip
In generate_corpuses.sh line 60:
${OUT}/arrow-ipc-generate-tensor-fuzz-corpus -stream ${CORPUS_DIR}
^----^ SC2086 (info): Double quote to prevent globbing and word splitting.
Did you mean:
"${OUT}"/arrow-ipc-generate-tensor-fuzz-corpus -stream ${CORPUS_DIR}
In generate_corpuses.sh line 61:
${ARROW_CPP}/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
${OUT}/arrow-ipc-tensor-stream-fuzz_seed_corpus.zip
^----------^ SC2086 (info): Double quote to prevent globbing and word
splitting.
^----^
SC2086 (info): Double quote to prevent globbing and word splitting.
Did you mean:
"${ARROW_CPP}"/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
"${OUT}"/arrow-ipc-tensor-stream-fuzz_seed_corpus.zip
In generate_corpuses.sh line 66:
${OUT}/parquet-arrow-generate-fuzz-corpus ${CORPUS_DIR}
^----^ SC2086 (info): Double quote to prevent globbing and word splitting.
Did you mean:
"${OUT}"/parquet-arrow-generate-fuzz-corpus ${CORPUS_DIR}
In generate_corpuses.sh line 68:
cp ${ARROW_CPP}/submodules/parquet-testing/data/*.parquet ${CORPUS_DIR}
^----------^ SC2086 (info): Double quote to prevent globbing and word
splitting.
Did you mean:
cp "${ARROW_CPP}"/submodules/parquet-testing/data/*.parquet ${CORPUS_DIR}
In generate_corpuses.sh line 69:
cp ${ARROW_CPP}/submodules/parquet-testing/bad_data/*.parquet ${CORPUS_DIR}
^----------^ SC2086 (info): Double quote to prevent globbing and word
splitting.
Did you mean:
cp "${ARROW_CPP}"/submodules/parquet-testing/bad_data/*.parquet
${CORPUS_DIR}
In generate_corpuses.sh line 70:
${ARROW_CPP}/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
${OUT}/parquet-arrow-fuzz_seed_corpus.zip
^----------^ SC2086 (info): Double quote to prevent globbing and word
splitting.
^----^
SC2086 (info): Double quote to prevent globbing and word splitting.
Did you mean:
"${ARROW_CPP}"/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
"${OUT}"/parquet-arrow-fuzz_seed_corpus.zip
In generate_corpuses.sh line 75:
${OUT}/parquet-generate-encoding-fuzz-corpus ${CORPUS_DIR}
^----^ SC2086 (info): Double quote to prevent globbing and word splitting.
Did you mean:
"${OUT}"/parquet-generate-encoding-fuzz-corpus ${CORPUS_DIR}
In generate_corpuses.sh line 76:
${ARROW_CPP}/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
${OUT}/parquet-encoding-fuzz_seed_corpus.zip
^----------^ SC2086 (info): Double quote to prevent globbing and word
splitting.
^----^
SC2086 (info): Double quote to prevent globbing and word splitting.
Did you mean:
"${ARROW_CPP}"/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
"${OUT}"/parquet-encoding-fuzz_seed_corpus.zip
In generate_corpuses.sh line 84:
${OUT}/arrow-csv-generate-fuzz-corpus ${CORPUS_DIR}
^----^ SC2086 (info): Double quote to prevent globbing and word splitting.
Did you mean:
"${OUT}"/arrow-csv-generate-fuzz-corpus ${CORPUS_DIR}
In generate_corpuses.sh line 86:
cp ${ARROW_ROOT}/testing/data/csv/*.csv ${CORPUS_DIR}
^-----------^ SC2086 (info): Double quote to prevent globbing and word
splitting.
Did you mean:
cp "${ARROW_ROOT}"/testing/data/csv/*.csv ${CORPUS_DIR}
In generate_corpuses.sh line 89:
${ARROW_CPP}/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
${OUT}/arrow-csv-fuzz_seed_corpus.zip
^----------^ SC2086 (info): Double quote to prevent globbing and word
splitting.
^----^
SC2086 (info): Double quote to prevent globbing and word splitting.
Did you mean:
"${ARROW_CPP}"/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
"${OUT}"/arrow-csv-fuzz_seed_corpus.zip
For more information:
https://www.shellcheck.net/wiki/SC2046 -- Quote this to prevent word
splitt...
https://www.shellcheck.net/wiki/SC2128 -- Expanding an array without an
ind...
https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent
globbing ...
```
### What changes are included in this PR?
* SC2046: Quote command substitutions to prevent word splitting.
* SC2128: Specify an array index explicitly.
* SC2086: Quote variables to prevent globbing and word splitting.
### Are these changes tested?
Yes.
### Are there any user-facing changes?
No.
* GitHub Issue: #51090
Authored-by: Hiroyuki Sato <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
---
.pre-commit-config.yaml | 8 +---
cpp/build-support/fuzzing/generate_corpuses.sh | 63 ++++++++++++++------------
2 files changed, 34 insertions(+), 37 deletions(-)
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index c4f736b39be..b47ce4ac02c 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -284,13 +284,7 @@ repos:
(
?^c_glib/.*\.sh$|
?^ci/.*\.sh$|
- ?^cpp/build-support/build-lz4-lib\.sh$|
- ?^cpp/build-support/build-zstd-lib\.sh$|
- ?^cpp/build-support/get-upstream-commit\.sh$|
- ?^cpp/build-support/run-test\.sh$|
- ?^cpp/build-support/update-flatbuffers\.sh$|
- ?^cpp/build-support/update-thrift\.sh$|
- ?^cpp/build-support/vendor-flatbuffers\.sh$|
+ ?^cpp/build-support/.*\.sh$|
?^cpp/examples/minimal_build/run\.sh$|
?^cpp/examples/tutorial_examples/run\.sh$|
?^cpp/src/arrow/flight/sql/odbc/install/mac/postinstall$|
diff --git a/cpp/build-support/fuzzing/generate_corpuses.sh
b/cpp/build-support/fuzzing/generate_corpuses.sh
index 07afa793dc6..2d95e110b51 100755
--- a/cpp/build-support/fuzzing/generate_corpuses.sh
+++ b/cpp/build-support/fuzzing/generate_corpuses.sh
@@ -29,7 +29,7 @@ set -ex
CORPUS_DIR=/tmp/corpus
PANDAS_DIR=/tmp/pandas
-ARROW_ROOT=$(cd $(dirname "$BASH_SOURCE")/../../..; pwd)
+ARROW_ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)
ARROW_CPP=$ARROW_ROOT/cpp
OUT=$1
@@ -39,51 +39,54 @@ OUT=$1
# Arrow IPC
-rm -rf ${CORPUS_DIR}
-${OUT}/arrow-ipc-generate-fuzz-corpus -stream ${CORPUS_DIR}
+rm -rf "${CORPUS_DIR}"
+"${OUT}/arrow-ipc-generate-fuzz-corpus" -stream "${CORPUS_DIR}"
# Add "golden" IPC integration files
-IPC_INTEGRATION_FILES=$(find
${ARROW_ROOT}/testing/data/arrow-ipc-stream/integration -name "*.stream")
-[ -z "${IPC_INTEGRATION_FILES}" ] && exit 1
# Several IPC integration files can have the same name, make sure
# they all appear in the corpus by numbering the duplicates.
-cp --backup=numbered ${IPC_INTEGRATION_FILES} ${CORPUS_DIR}
-${ARROW_CPP}/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
${OUT}/arrow-ipc-stream-fuzz_seed_corpus.zip
+find "${ARROW_ROOT}/testing/data/arrow-ipc-stream/integration" \
+ -name "*.stream" \
+ -exec cp --backup=numbered '{}' "${CORPUS_DIR}" \;
-rm -rf ${CORPUS_DIR}
-${OUT}/arrow-ipc-generate-fuzz-corpus -file ${CORPUS_DIR}
-IPC_INTEGRATION_FILES=$(find
${ARROW_ROOT}/testing/data/arrow-ipc-stream/integration -name "*.arrow_file")
-[ -z "${IPC_INTEGRATION_FILES}" ] && exit 1
-cp --backup=numbered ${IPC_INTEGRATION_FILES} ${CORPUS_DIR}
-${ARROW_CPP}/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
${OUT}/arrow-ipc-file-fuzz_seed_corpus.zip
+"${ARROW_CPP}/build-support/fuzzing/pack_corpus.py" "${CORPUS_DIR}"
"${OUT}/arrow-ipc-stream-fuzz_seed_corpus.zip"
-rm -rf ${CORPUS_DIR}
-${OUT}/arrow-ipc-generate-tensor-fuzz-corpus -stream ${CORPUS_DIR}
-${ARROW_CPP}/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
${OUT}/arrow-ipc-tensor-stream-fuzz_seed_corpus.zip
+rm -rf "${CORPUS_DIR}"
+"${OUT}/arrow-ipc-generate-fuzz-corpus" -file "${CORPUS_DIR}"
+
+find "${ARROW_ROOT}/testing/data/arrow-ipc-stream/integration" \
+ -name "*.arrow_file" \
+ -exec cp --backup=numbered '{}' "${CORPUS_DIR}" \;
+
+"${ARROW_CPP}/build-support/fuzzing/pack_corpus.py" "${CORPUS_DIR}"
"${OUT}/arrow-ipc-file-fuzz_seed_corpus.zip"
+
+rm -rf "${CORPUS_DIR}"
+"${OUT}/arrow-ipc-generate-tensor-fuzz-corpus" -stream "${CORPUS_DIR}"
+"${ARROW_CPP}/build-support/fuzzing/pack_corpus.py" "${CORPUS_DIR}"
"${OUT}/arrow-ipc-tensor-stream-fuzz_seed_corpus.zip"
# Parquet file-level fuzzer
-rm -rf ${CORPUS_DIR}
-${OUT}/parquet-arrow-generate-fuzz-corpus ${CORPUS_DIR}
+rm -rf "${CORPUS_DIR}"
+"${OUT}/parquet-arrow-generate-fuzz-corpus" "${CORPUS_DIR}"
# Add Parquet testing examples
-cp ${ARROW_CPP}/submodules/parquet-testing/data/*.parquet ${CORPUS_DIR}
-cp ${ARROW_CPP}/submodules/parquet-testing/bad_data/*.parquet ${CORPUS_DIR}
-${ARROW_CPP}/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
${OUT}/parquet-arrow-fuzz_seed_corpus.zip
+cp "${ARROW_CPP}"/submodules/parquet-testing/data/*.parquet "${CORPUS_DIR}"
+cp "${ARROW_CPP}"/submodules/parquet-testing/bad_data/*.parquet "${CORPUS_DIR}"
+"${ARROW_CPP}/build-support/fuzzing/pack_corpus.py" "${CORPUS_DIR}"
"${OUT}/parquet-arrow-fuzz_seed_corpus.zip"
# Parquet encoding fuzzer
rm -rf ${CORPUS_DIR}
-${OUT}/parquet-generate-encoding-fuzz-corpus ${CORPUS_DIR}
-${ARROW_CPP}/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
${OUT}/parquet-encoding-fuzz_seed_corpus.zip
+"${OUT}/parquet-generate-encoding-fuzz-corpus" "${CORPUS_DIR}"
+"${ARROW_CPP}/build-support/fuzzing/pack_corpus.py" "${CORPUS_DIR}"
"${OUT}/parquet-encoding-fuzz_seed_corpus.zip"
# CSV
-rm -rf ${PANDAS_DIR}
-git clone --depth=1 https://github.com/pandas-dev/pandas ${PANDAS_DIR}
+rm -rf "${PANDAS_DIR}"
+git clone --depth=1 https://github.com/pandas-dev/pandas "${PANDAS_DIR}"
-rm -rf ${CORPUS_DIR}
-${OUT}/arrow-csv-generate-fuzz-corpus ${CORPUS_DIR}
+rm -rf "${CORPUS_DIR}"
+"${OUT}/arrow-csv-generate-fuzz-corpus" "${CORPUS_DIR}"
# Add examples from arrow-testing repo
-cp ${ARROW_ROOT}/testing/data/csv/*.csv ${CORPUS_DIR}
+cp "${ARROW_ROOT}"/testing/data/csv/*.csv "${CORPUS_DIR}"
# Add examples from Pandas test suite
-find ${PANDAS_DIR}/ -name "*.csv" -exec cp --backup=numbered '{}'
${CORPUS_DIR} \;
-${ARROW_CPP}/build-support/fuzzing/pack_corpus.py ${CORPUS_DIR}
${OUT}/arrow-csv-fuzz_seed_corpus.zip
+find "${PANDAS_DIR}/" -name "*.csv" -exec cp --backup=numbered '{}'
"${CORPUS_DIR}" \;
+"${ARROW_CPP}/build-support/fuzzing/pack_corpus.py" "${CORPUS_DIR}"
"${OUT}/arrow-csv-fuzz_seed_corpus.zip"
