Repository: yetus Updated Branches: refs/heads/master ae566eef8 -> 66352f53e
YETUS-300. CHANGED_FILES, CHANGED_MODULES, etc should really be arrays Signed-off-by: Sean Busbey <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/yetus/repo Commit: http://git-wip-us.apache.org/repos/asf/yetus/commit/66352f53 Tree: http://git-wip-us.apache.org/repos/asf/yetus/tree/66352f53 Diff: http://git-wip-us.apache.org/repos/asf/yetus/diff/66352f53 Branch: refs/heads/master Commit: 66352f53e0f1b17521618a9a99e6d29496897fbe Parents: ae566ee Author: Allen Wittenauer <[email protected]> Authored: Mon Mar 21 21:57:30 2016 -0700 Committer: Allen Wittenauer <[email protected]> Committed: Tue Mar 22 15:52:55 2016 -0700 ---------------------------------------------------------------------- .../in-progress/precommit-advanced.md | 4 +- .../in-progress/precommit-buildtools.md | 4 +- precommit/core.d/00-yetuslib.sh | 16 +++ precommit/personality/accumulo.sh | 5 +- precommit/personality/hadoop.sh | 23 +++-- precommit/personality/hbase.sh | 4 +- precommit/personality/kafka.sh | 4 +- precommit/test-patch.d/ant.sh | 5 +- precommit/test-patch.d/author.sh | 13 ++- precommit/test-patch.d/checkstyle.sh | 2 +- precommit/test-patch.d/cmake.sh | 2 +- precommit/test-patch.d/gradle.sh | 5 +- precommit/test-patch.d/make.sh | 5 +- precommit/test-patch.d/maven.sh | 39 ++++---- precommit/test-patch.d/nobuild.sh | 5 +- precommit/test-patch.d/perlcritic.sh | 4 +- precommit/test-patch.d/pylint.sh | 4 +- precommit/test-patch.d/rubocop.sh | 4 +- precommit/test-patch.d/ruby-lint.sh | 4 +- precommit/test-patch.d/test4tests.sh | 2 +- precommit/test-patch.d/xml.sh | 2 +- precommit/test-patch.sh | 100 +++++++++---------- 22 files changed, 138 insertions(+), 118 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/asf-site-src/source/documentation/in-progress/precommit-advanced.md ---------------------------------------------------------------------- diff --git a/asf-site-src/source/documentation/in-progress/precommit-advanced.md b/asf-site-src/source/documentation/in-progress/precommit-advanced.md index 3a66799..d64392a 100644 --- a/asf-site-src/source/documentation/in-progress/precommit-advanced.md +++ b/asf-site-src/source/documentation/in-progress/precommit-advanced.md @@ -249,9 +249,9 @@ There are a handful of extremely important system variables that make life easie * BUILDTOOL specifies which tool is currently being used to drive compilation. Additionally, many build tools define xyz\_ARGS to pass on to the build tool command line. (e.g., MAVEN\_ARGS if maven is in use). Projects may set this in their personality. NOTE: today, only one build tool at a time is supported. This may change in the future. -* CHANGED\_FILES is a list of all files that appear to be added, deleted, or modified in the patch. +* CHANGED\_FILES[@] is an array of all files that appear to be added, deleted, or modified in the patch. -* CHANGED\_MODULES is a list of all modules that house all of the CHANGED\_FILES. Be aware that the root of the source tree is reported as '.'. +* CHANGED\_MODULES[@] is an array of all modules that house all of the CHANGED\_FILES[@]. Be aware that the root of the source tree is reported as '.'. * GITHUB\_REPO is to help test-patch when talking to Github. If test-patch is given just a number on the command line, it will default to using this repo to determine the pull request. http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/asf-site-src/source/documentation/in-progress/precommit-buildtools.md ---------------------------------------------------------------------- diff --git a/asf-site-src/source/documentation/in-progress/precommit-buildtools.md b/asf-site-src/source/documentation/in-progress/precommit-buildtools.md index 98d94e9..e800135 100644 --- a/asf-site-src/source/documentation/in-progress/precommit-buildtools.md +++ b/asf-site-src/source/documentation/in-progress/precommit-buildtools.md @@ -90,7 +90,7 @@ For example, the gradle build tool does not have a standard way to execute check * pluginname\_reorder\_modules - - This functions allows the plugin to (re-)order the modules (e.g. based on the output of the maven dependency plugin). When called CHANGED\_MODULES already contains all changed modules. It must be altered to have an effect. + - This functions allows the plugin to (re-)order the modules (e.g. based on the output of the maven dependency plugin). When called CHANGED\_MODULES[@] already contains all changed modules. It must be altered to have an effect. * pluginname\_(test)\_logfilter @@ -166,4 +166,4 @@ By default, test-patch will pass -Ptest-patch to Maven. This will allow you to c Maven will test eclipse and site if maven is being used as the build tool and appropriate files trigger them. -Maven will trigger add a maven install test when the `maven_add_install` function has been used and the related tests are requierd. Plug-ins that need to run maven before MUST call it as part of their respective initialize functions, otherwise maven may fail unexpectedly. All Yetus provided plug-ins that require maven will trigger the maven install functionality. \ No newline at end of file +Maven will trigger add a maven install test when the `maven_add_install` function has been used and the related tests are requierd. Plug-ins that need to run maven before MUST call it as part of their respective initialize functions, otherwise maven may fail unexpectedly. All Yetus provided plug-ins that require maven will trigger the maven install functionality. http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/core.d/00-yetuslib.sh ---------------------------------------------------------------------- diff --git a/precommit/core.d/00-yetuslib.sh b/precommit/core.d/00-yetuslib.sh index 1410b33..1bd63b7 100755 --- a/precommit/core.d/00-yetuslib.sh +++ b/precommit/core.d/00-yetuslib.sh @@ -224,3 +224,19 @@ function yetus_generic_columnprinter ((i=i+1)) done } + +## @description Convert a comma-delimited string to an array +## @audience public +## @stability evolving +## @replaceable no +## @param arrayname +## @param string +function yetus_comma_to_array +{ + declare var=$1 + declare string=$2 + + oldifs="${IFS}" + IFS=',' read -r -a "${var}" <<< "${string}" + IFS="${oldifs}" +} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/personality/accumulo.sh ---------------------------------------------------------------------- diff --git a/precommit/personality/accumulo.sh b/precommit/personality/accumulo.sh index 84a9c8c..de1b890 100755 --- a/precommit/personality/accumulo.sh +++ b/precommit/personality/accumulo.sh @@ -50,8 +50,7 @@ function personality_modules fi # Make sure we re-add the changed modules if we didn't short-circuit out - for module in ${CHANGED_MODULES}; do - # shellcheck disable=SC2086 - personality_enqueue_module ${module} + for module in "${CHANGED_MODULES[@]}"; do + personality_enqueue_module "${module}" done } http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/personality/hadoop.sh ---------------------------------------------------------------------- diff --git a/precommit/personality/hadoop.sh b/precommit/personality/hadoop.sh index 1084cd5..02df4b1 100755 --- a/precommit/personality/hadoop.sh +++ b/precommit/personality/hadoop.sh @@ -38,9 +38,9 @@ function hadoop_order declare hadoopm if [[ ${ordering} = normal ]]; then - hadoopm=${CHANGED_MODULES} + hadoopm="${CHANGED_MODULES[*]}" elif [[ ${ordering} = union ]]; then - hadoopm=${CHANGED_UNION_MODULES} + hadoopm="${CHANGED_UNION_MODULES}" else hadoopm="${ordering}" fi @@ -144,6 +144,7 @@ function personality_modules declare extra="" declare ordering="normal" declare needflags=false + declare foundbats=false declare flags declare fn declare i @@ -170,7 +171,7 @@ function personality_modules needflags=true # if something in common changed, we build the whole world - if [[ ${CHANGED_MODULES} =~ hadoop-common ]]; then + if [[ "${CHANGED_MODULES[*]}" =~ hadoop-common ]]; then yetus_debug "hadoop personality: javac + hadoop-common = ordering set to . " ordering="." fi @@ -180,7 +181,7 @@ function personality_modules extra="-DskipTests" ;; javadoc) - if [[ "${CHANGED_MODULES}" =~ \. ]]; then + if [[ "${CHANGED_MODULES[*]}" =~ \. ]]; then ordering=. fi @@ -199,7 +200,7 @@ function personality_modules extra="-Pdocs -DskipTests" ;; mvneclipse) - if [[ "${CHANGED_MODULES}" =~ \. ]]; then + if [[ "${CHANGED_MODULES[*]}" =~ \. ]]; then ordering=. fi ;; @@ -210,12 +211,12 @@ function personality_modules fi ;; mvnsite) - if [[ "${CHANGED_MODULES}" =~ \. ]]; then + if [[ "${CHANGED_MODULES[*]}" =~ \. ]]; then ordering=. fi ;; unit) - if [[ "${CHANGED_MODULES}" =~ \. ]]; then + if [[ "${CHANGED_MODULES[*]}" =~ \. ]]; then ordering=. fi @@ -237,7 +238,13 @@ function personality_modules fi fi - if ! verify_needed_test shellcheck && [[ ! ${CHANGED_FILES} =~ \.bats ]]; then + for i in "${CHANGED_FILES[@]}"; do + if [[ "${i}" =~ \.bats ]]; then + foundbats=true + fi + done + + if ! verify_needed_test shellcheck && [[ ${foundbats} = false ]]; then yetus_debug "hadoop: NO shell code change detected; disabling shelltest profile" extra="${extra} -P!shelltest" else http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/personality/hbase.sh ---------------------------------------------------------------------- diff --git a/precommit/personality/hbase.sh b/precommit/personality/hbase.sh index 4705903..7863bb2 100755 --- a/precommit/personality/hbase.sh +++ b/precommit/personality/hbase.sh @@ -51,7 +51,7 @@ function personality_modules fi if [[ ${testtype} = findbugs ]]; then - for module in ${CHANGED_MODULES}; do + for module in "${CHANGED_MODULES[@]}"; do # skip findbugs on hbase-shell if [[ ${module} == hbase-shell ]]; then continue @@ -63,7 +63,7 @@ function personality_modules return fi - for module in ${CHANGED_MODULES}; do + for module in "${CHANGED_MODULES[@]}"; do # shellcheck disable=SC2086 personality_enqueue_module ${module} ${extra} done http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/personality/kafka.sh ---------------------------------------------------------------------- diff --git a/precommit/personality/kafka.sh b/precommit/personality/kafka.sh index 3c2f2c4..8d866c9 100755 --- a/precommit/personality/kafka.sh +++ b/precommit/personality/kafka.sh @@ -56,8 +56,8 @@ function personality_modules ;; esac - for module in ${CHANGED_MODULES}; do + for module in "${CHANGED_MODULES[@]}"; do # shellcheck disable=SC2086 - personality_enqueue_module ${module} ${extra} + personality_enqueue_module "${module}" ${extra} done } http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/test-patch.d/ant.sh ---------------------------------------------------------------------- diff --git a/precommit/test-patch.d/ant.sh b/precommit/test-patch.d/ant.sh index 51356e7..64c4067 100755 --- a/precommit/test-patch.d/ant.sh +++ b/precommit/test-patch.d/ant.sh @@ -156,9 +156,8 @@ function ant_builtin_personality_modules clear_personality_queue - for module in ${CHANGED_MODULES}; do - # shellcheck disable=SC2086 - personality_enqueue_module ${module} + for module in "${CHANGED_MODULES[@]}"; do + personality_enqueue_module "${module}" done } http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/test-patch.d/author.sh ---------------------------------------------------------------------- diff --git a/precommit/test-patch.d/author.sh b/precommit/test-patch.d/author.sh index 19b4dee..4497d03 100755 --- a/precommit/test-patch.d/author.sh +++ b/precommit/test-patch.d/author.sh @@ -28,16 +28,19 @@ function author_patchfile declare authorTags # shellcheck disable=SC2155 declare -r appname=$(basename "${BASH_SOURCE-$0}") + declare i big_console_header "Checking there are no @author tags in the patch." start_clock - if [[ ${CHANGED_FILES} =~ ${appname} ]]; then - echo "Skipping @author checks as ${appname} has been patched." - add_vote_table 0 @author "Skipping @author checks as ${appname} has been patched." - return 0 - fi + for i in "${CHANGED_FILES[@]}"; do + if [[ ${i} =~ ${appname} ]]; then + echo "Skipping @author checks as ${appname} has been patched." + add_vote_table 0 @author "Skipping @author checks as ${appname} has been patched." + return 0 + fi + done ${GREP} -i -n '^[^-].*@author' "${patchfile}" >> "${PATCH_DIR}/author-tags.txt" # shellcheck disable=SC2016 http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/test-patch.d/checkstyle.sh ---------------------------------------------------------------------- diff --git a/precommit/test-patch.d/checkstyle.sh b/precommit/test-patch.d/checkstyle.sh index 1c0e078..de1935f 100755 --- a/precommit/test-patch.d/checkstyle.sh +++ b/precommit/test-patch.d/checkstyle.sh @@ -215,7 +215,7 @@ function checkstyle_runner # this will grealy cut down how much work we # have to do later - for j in ${CHANGED_FILES}; do + for j in "${CHANGED_FILES[@]}"; do ${GREP} "${j}" "${tmp}" >> "${tmp}.1" done http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/test-patch.d/cmake.sh ---------------------------------------------------------------------- diff --git a/precommit/test-patch.d/cmake.sh b/precommit/test-patch.d/cmake.sh index 2eabd1e..9b5f265 100755 --- a/precommit/test-patch.d/cmake.sh +++ b/precommit/test-patch.d/cmake.sh @@ -77,7 +77,7 @@ function cmake_reorder_modules #shellcheck disable=SC2034 BUILDTOOLCWD="@@@BASEDIR@@@/${CMAKE_BUILD_DIR}" #shellcheck disable=SC2034 - CHANGED_MODULES="." + CHANGED_MODULES=(".") #shellcheck disable=SC2034 CHANGED_UNION_MODULES="." else http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/test-patch.d/gradle.sh ---------------------------------------------------------------------- diff --git a/precommit/test-patch.d/gradle.sh b/precommit/test-patch.d/gradle.sh index f6724ad..57e023d 100755 --- a/precommit/test-patch.d/gradle.sh +++ b/precommit/test-patch.d/gradle.sh @@ -217,9 +217,8 @@ function gradle_builtin_personality_modules clear_personality_queue - for module in ${CHANGED_MODULES}; do - # shellcheck disable=SC2086 - personality_enqueue_module ${module} + for module in "${CHANGED_MODULES[@]}"; do + personality_enqueue_module "${module}" done } http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/test-patch.d/make.sh ---------------------------------------------------------------------- diff --git a/precommit/test-patch.d/make.sh b/precommit/test-patch.d/make.sh index 5e80132..92ba8e6 100755 --- a/precommit/test-patch.d/make.sh +++ b/precommit/test-patch.d/make.sh @@ -131,9 +131,8 @@ function make_builtin_personality_modules clear_personality_queue - for module in ${CHANGED_MODULES}; do - # shellcheck disable=SC2086 - personality_enqueue_module ${module} + for module in "${CHANGED_MODULES[@]}"; do + personality_enqueue_module "${module}" done } http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/test-patch.d/maven.sh ---------------------------------------------------------------------- diff --git a/precommit/test-patch.d/maven.sh b/precommit/test-patch.d/maven.sh index 3e98957..6cc64f5 100755 --- a/precommit/test-patch.d/maven.sh +++ b/precommit/test-patch.d/maven.sh @@ -356,10 +356,10 @@ function maven_javadoc_calcdiffs function maven_builtin_personality_modules { - local repostatus=$1 - local testtype=$2 + declare repostatus=$1 + declare testtype=$2 - local module + declare module yetus_debug "Using builtin personality_modules" yetus_debug "Personality: ${repostatus} ${testtype}" @@ -369,14 +369,13 @@ function maven_builtin_personality_modules # this always makes sure the local repo has a fresh # copy of everything per pom rules. if [[ ${repostatus} == branch - && ${testtype} == mvninstall ]];then - personality_enqueue_module "${CHANGED_UNION_MODULES}" - return - fi + && ${testtype} == mvninstall ]];then + personality_enqueue_module "${CHANGED_UNION_MODULES}" + return + fi - for module in ${CHANGED_MODULES}; do - # shellcheck disable=SC2086 - personality_enqueue_module ${module} + for module in "${CHANGED_MODULES[@]}"; do + personality_enqueue_module "${module}" done } @@ -581,9 +580,11 @@ function maven_reorder_module_process declare needroot=false declare found - if [[ ${CHANGED_MODULES} =~ \. ]]; then - needroot=true - fi + for module in "${CHANGED_MODULES[@]}"; do + if [[ "${module}" =~ \. ]]; then + needroot=true + fi + done fn=$(module_file_fragment "${CHANGED_UNION_MODULES}") pushd "${BASEDIR}/${CHANGED_UNION_MODULES}" >/dev/null @@ -600,7 +601,7 @@ function maven_reorder_module_process else continue fi - for indexm in ${CHANGED_MODULES}; do + for indexm in "${CHANGED_MODULES[@]}"; do # modules could be foo/bar, where bar is the artifactid # so get the basename and compare that too basemod=${indexm##*/} @@ -619,12 +620,12 @@ function maven_reorder_module_process newlist=("${newlist[@]}" " . ") fi - indexm=$(echo "${CHANGED_MODULES}" | wc -w) + indexm="${#CHANGED_MODULES[@]}" indexn="${#newlist[@]}" if [[ ${indexm} -ne ${indexn} ]]; then yetus_debug "mrm: Missed a module" - for indexm in ${CHANGED_MODULES}; do + for indexm in "${CHANGED_MODULES[@]}"; do found=false for indexn in ${newlist[*]}; do if [[ "${indexn}" = "${indexm}" ]]; then @@ -638,7 +639,7 @@ function maven_reorder_module_process done fi - CHANGED_MODULES="${newlist[*]}" + CHANGED_MODULES=("${newlist[@]}") } ## @description take a stab at reordering modules based upon @@ -658,7 +659,7 @@ function maven_reorder_modules fi # don't bother if there is only one - index=$(echo "${CHANGED_MODULES}" | wc -w) + index="${#CHANGED_MODULES[@]}" if [[ ${index} -eq 1 ]]; then return fi @@ -670,7 +671,7 @@ function maven_reorder_modules maven_reorder_module_process "${repostatus}" yetus_debug "Maven: finish re-ordering modules" - yetus_debug "Finished list: ${CHANGED_MODULES}" + yetus_debug "Finished list: ${CHANGED_MODULES[*]}" add_vote_table 0 mvndep "Maven dependency ordering for ${repostatus}" } http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/test-patch.d/nobuild.sh ---------------------------------------------------------------------- diff --git a/precommit/test-patch.d/nobuild.sh b/precommit/test-patch.d/nobuild.sh old mode 100644 new mode 100755 index 74d35aa..cf5a2ad --- a/precommit/test-patch.d/nobuild.sh +++ b/precommit/test-patch.d/nobuild.sh @@ -40,9 +40,8 @@ function nobuild_builtin_personality_modules yetus_debug "built-in personality for no build system: ${status} ${testtype}" clear_personality_queue - for module in ${CHANGED_MODULES}; do - # shellcheck disable=SC2086 - personality_enqueue_module ${module} + for module in "${CHANGED_MODULES[@]}"; do + personality_enqueue_module "${module}" done } http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/test-patch.d/perlcritic.sh ---------------------------------------------------------------------- diff --git a/precommit/test-patch.d/perlcritic.sh b/precommit/test-patch.d/perlcritic.sh index c346e3d..8297f46 100755 --- a/precommit/test-patch.d/perlcritic.sh +++ b/precommit/test-patch.d/perlcritic.sh @@ -70,7 +70,7 @@ function perlcritic_preapply echo "Running perlcritic against modified perl scripts/modules." pushd "${BASEDIR}" >/dev/null - for i in ${CHANGED_FILES}; do + for i in "${CHANGED_FILES[@]}"; do if [[ ${i} =~ \.p[lm]$ && -f ${i} ]]; then ${PERLCRITIC} -1 --verbose 1 "${i}" 2>/dev/null >> "${PATCH_DIR}/branch-perlcritic-result.txt" fi @@ -117,7 +117,7 @@ function perlcritic_postapply echo "Running perlcritic against modified perl scripts/modules." # we re-check this in case one has been added pushd "${BASEDIR}" >/dev/null - for i in ${CHANGED_FILES}; do + for i in "${CHANGED_FILES[@]}"; do if [[ ${i} =~ \.p[lm]$ && -f ${i} ]]; then ${PERLCRITIC} -1 --verbose 1 "${i}" 2>/dev/null >> "${PATCH_DIR}/patch-perlcritic-result.txt" fi http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/test-patch.d/pylint.sh ---------------------------------------------------------------------- diff --git a/precommit/test-patch.d/pylint.sh b/precommit/test-patch.d/pylint.sh index b73d8a0..2a9e1f8 100755 --- a/precommit/test-patch.d/pylint.sh +++ b/precommit/test-patch.d/pylint.sh @@ -77,7 +77,7 @@ function pylint_preapply echo "Running pylint against modified python scripts." pushd "${BASEDIR}" >/dev/null - for i in ${CHANGED_FILES}; do + for i in "${CHANGED_FILES[@]}"; do if [[ ${i} =~ \.py$ && -f ${i} ]]; then # shellcheck disable=SC2086 eval "${PYLINT} ${PYLINT_OPTIONS} --msg-template='{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}' --reports=n ${i}" \ @@ -124,7 +124,7 @@ function pylint_postapply echo "Running pylint against modified python scripts." # we re-check this in case one has been added pushd "${BASEDIR}" >/dev/null - for i in ${CHANGED_FILES}; do + for i in "${CHANGED_FILES[@]}"; do if [[ ${i} =~ \.py$ && -f ${i} ]]; then # shellcheck disable=SC2086 eval "${PYLINT} ${PYLINT_OPTIONS} --msg-template='{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}' --reports=n ${i}" \ http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/test-patch.d/rubocop.sh ---------------------------------------------------------------------- diff --git a/precommit/test-patch.d/rubocop.sh b/precommit/test-patch.d/rubocop.sh index 070eb55..8e1d5b8 100755 --- a/precommit/test-patch.d/rubocop.sh +++ b/precommit/test-patch.d/rubocop.sh @@ -70,7 +70,7 @@ function rubocop_preapply echo "Running rubocop against modified ruby scripts." pushd "${BASEDIR}" >/dev/null - for i in ${CHANGED_FILES}; do + for i in "${CHANGED_FILES[@]}"; do if [[ ${i} =~ \.rb$ && -f ${i} ]]; then ${RUBOCOP} -f e "${i}" | ${AWK} '!/[0-9]* files? inspected/' >> "${PATCH_DIR}/branch-rubocop-result.txt" fi @@ -117,7 +117,7 @@ function rubocop_postapply echo "Running rubocop against modified ruby scripts." # we re-check this in case one has been added pushd "${BASEDIR}" >/dev/null - for i in ${CHANGED_FILES}; do + for i in "${CHANGED_FILES[@]}"; do if [[ ${i} =~ \.rb$ && -f ${i} ]]; then ${RUBOCOP} -f e "${i}" | ${AWK} '!/[0-9]* files? inspected/' >> "${PATCH_DIR}/patch-rubocop-result.txt" fi http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/test-patch.d/ruby-lint.sh ---------------------------------------------------------------------- diff --git a/precommit/test-patch.d/ruby-lint.sh b/precommit/test-patch.d/ruby-lint.sh index 350733e..94d4f53 100755 --- a/precommit/test-patch.d/ruby-lint.sh +++ b/precommit/test-patch.d/ruby-lint.sh @@ -69,7 +69,7 @@ function ruby_lint_preapply echo "Running ruby-lint against modified ruby scripts." pushd "${BASEDIR}" >/dev/null - for i in ${CHANGED_FILES}; do + for i in "${CHANGED_FILES[@]}"; do if [[ ${i} =~ \.rb$ && -f ${i} ]]; then ${RUBY_LINT} -p syntastic "${i}" | sort -t : -k 1,1 -k 3,3n -k 4,4n >> "${PATCH_DIR}/branch-ruby-lint-result.txt" fi @@ -147,7 +147,7 @@ function ruby_lint_postapply echo "Running ruby-lint against modified ruby scripts." # we re-check this in case one has been added pushd "${BASEDIR}" >/dev/null - for i in ${CHANGED_FILES}; do + for i in "${CHANGED_FILES[@]}"; do if [[ ${i} =~ \.rb$ && -f ${i} ]]; then ${RUBY_LINT} -p syntastic "${i}" | sort -t : -k 1,1 -k 3,3n -k 4,4n >> "${PATCH_DIR}/patch-ruby-lint-result.txt" fi http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/test-patch.d/test4tests.sh ---------------------------------------------------------------------- diff --git a/precommit/test-patch.d/test4tests.sh b/precommit/test-patch.d/test4tests.sh index f4ad1c7..644fbf9 100755 --- a/precommit/test-patch.d/test4tests.sh +++ b/precommit/test-patch.d/test4tests.sh @@ -36,7 +36,7 @@ function test4tests_patchfile start_clock - for i in ${CHANGED_FILES}; do + for i in "${CHANGED_FILES[@]}"; do if [[ ${i} =~ (^|/)test/ ]]; then ((testReferences=testReferences + 1)) fi http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/test-patch.d/xml.sh ---------------------------------------------------------------------- diff --git a/precommit/test-patch.d/xml.sh b/precommit/test-patch.d/xml.sh index dd8f206..08e0a61 100755 --- a/precommit/test-patch.d/xml.sh +++ b/precommit/test-patch.d/xml.sh @@ -55,7 +55,7 @@ function xml_postcompile start_clock pushd "${BASEDIR}" >/dev/null - for i in ${CHANGED_FILES}; do + for i in "${CHANGED_FILES[@]}"; do if [[ ${i} =~ \.xml$ && -f ${i} ]]; then ${js} -e "XMLDocument(arguments[0])" "${i}" >> "${PATCH_DIR}/xml.txt" 2>&1 if [[ $? != 0 ]]; then http://git-wip-us.apache.org/repos/asf/yetus/blob/66352f53/precommit/test-patch.sh ---------------------------------------------------------------------- diff --git a/precommit/test-patch.sh b/precommit/test-patch.sh index 3f42cd2..7e93b35 100755 --- a/precommit/test-patch.sh +++ b/precommit/test-patch.sh @@ -32,6 +32,8 @@ GLOBALTIMER=$(date +"%s") QATESTMODE=false # global arrays +declare -a CHANGED_FILES +declare -a CHANGED_MODULES declare -a TP_HEADER declare -a TP_VOTE_TABLE declare -a TP_TEST_TABLE @@ -42,6 +44,7 @@ declare -a MODULE_STATUS_MSG declare -a MODULE_STATUS_LOG declare -a MODULE_COMPILE_LOG declare -a MODULE +declare -a USER_MODULE_LIST TP_HEADER_COUNTER=0 TP_VOTE_COUNTER=0 @@ -80,12 +83,7 @@ function setup_defaults BUILDTOOLCWD=module # shellcheck disable=SC2034 - CHANGED_MODULES="" - - # shellcheck disable=SC2034 CHANGED_UNION_MODULES="" - USER_MODULE_LIST="" - CHANGED_FILES="" REEXECED=false RESETREPO=false ISSUE="" @@ -318,7 +316,7 @@ function prepopulate_footer ## @replaceable no function finish_footer_table { - add_footer_table "modules" "C: ${CHANGED_MODULES} U: ${CHANGED_UNION_MODULES}" + add_footer_table "modules" "C: ${CHANGED_MODULES[*]} U: ${CHANGED_UNION_MODULES}" } ## @description Put the final elapsed time at the bottom of the table. @@ -570,16 +568,16 @@ function compute_unidiff # finally rewriting the line so that it is in # './filename:diff line:content' format - for fn in ${CHANGED_FILES}; do + for fn in "${CHANGED_FILES[@]}"; do filen=${fn##./} if [[ -f "${filen}" ]]; then - ${GIT} diff ${filen} \ + ${GIT} diff "${filen}" \ | tail -n +6 \ | ${GREP} -n '^+' \ | ${GREP} -vE '^[0-9]*:\+\+\+' \ | ${SED} -e 's,^\([0-9]*:\)\+,\1,g' \ - -e s,^,./${filen}:,g \ + -e "s,^,./${filen}:,g" \ >> "${tmpfile}" fi done @@ -810,9 +808,8 @@ function parse_args BUGLINECOMMENTS=${BUGLINECOMMENTS//,/ } ;; --modulelist=*) - USER_MODULE_LIST=${i#*=} - USER_MODULE_LIST=${USER_MODULE_LIST//,/ } - yetus_debug "Manually forcing modules ${USER_MODULE_LIST}" + yetus_comma_to_array USER_MODULE_LIST "${i#*=}" + yetus_debug "Manually forcing modules ${USER_MODULE_LIST[*]}" ;; --multijdkdirs=*) JDK_DIR_LIST=${i#*=} @@ -1007,14 +1004,19 @@ function find_buildfile_dir ## @audience private ## @stability stable ## @replaceable no -## @return None; sets ${CHANGED_FILES} +## @return None; sets ${CHANGED_FILES[@]} function find_changed_files { + declare line + # get a list of all of the files that have been changed, # except for /dev/null (which would be present for new files). # Additionally, remove any a/ b/ patterns at the front of the patch filenames. # shellcheck disable=SC2016 - CHANGED_FILES=$(${AWK} 'function p(s){sub("^[ab]/","",s); if(s!~"^/dev/null"){print s}} + while read -r line; do + CHANGED_FILES=("${CHANGED_FILES[@]}" "${line}") + done < <( + ${AWK} 'function p(s){sub("^[ab]/","",s); if(s!~"^/dev/null"){print s}} /^diff --git / { p($3); p($4) } /^(\+\+\+|---) / { p($2) }' "${PATCH_DIR}/patch" | sort -u) } @@ -1060,19 +1062,19 @@ function module_skipdir ## @stability stable ## @replaceable no ## @param repostatus -## @return None; sets ${CHANGED_MODULES} +## @return None; sets ${CHANGED_MODULES[@]} function find_changed_modules { - local repostatus=$1 - local i - local changed_dirs - local builddirs - local builddir - local module - local prev_builddir - local i=1 - local dir - local buildfile + declare repostatus=$1 + declare i + declare builddir + declare module + declare prev_builddir + declare i=1 + declare dir + declare dirt + declare buildfile + declare -a tmpmods buildfile=$("${BUILDTOOL}_buildfile") @@ -1082,55 +1084,51 @@ function find_changed_modules cleanup_and_exit 1 fi - changed_dirs=$(for i in ${CHANGED_FILES}; do dirname "${i}"; done | sort -u) - # Empty string indicates the build system wants to disable module detection if [[ -z ${buildfile} ]]; then - builddirs="." + tmpmods=(".") else # Now find all the modules that were changed - for i in ${changed_dirs}; do + for i in "${CHANGED_FILES[@]}"; do + dirt=$(dirname "${i}") - module_skipdir "${i}" + module_skipdir "${dirt}" if [[ $? != 0 ]]; then continue fi - builddir=$(find_buildfile_dir "${buildfile}" "${i}") + builddir=$(find_buildfile_dir "${buildfile}" "${dirt}") if [[ -z ${builddir} ]]; then yetus_error "ERROR: ${buildfile} is not found. Make sure the target is a ${BUILDTOOL}-based project." bugsystem_finalreport 1 cleanup_and_exit 1 fi - builddirs="${builddirs} ${builddir}" + tmpmods=("${tmpmods[@]}" "${builddir}") done fi - #shellcheck disable=SC2086,SC2034 - CHANGED_MODULES=$(echo ${builddirs} ${USER_MODULE_LIST} | tr ' ' '\n' | sort -u) - #shellcheck disable=SC2086,SC2034 - CHANGED_MODULES=$(echo ${CHANGED_MODULES} ${USER_MODULE_LIST} | tr ' ' '\n' | sort -u) + tmpmods=("${tmpmods[@]}" "${USER_MODULE_LIST[@]}") - # turn it back into a list so that anyone printing doesn't - # generate multiline output - #shellcheck disable=SC2086,SC2116 - CHANGED_MODULES=$(echo ${CHANGED_MODULES}) + CHANGED_MODULES=($(printf "%s\n" "${tmpmods[@]}" | sort -u)) - yetus_debug "Locate the union of ${CHANGED_MODULES}" - # shellcheck disable=SC2086 - count=$(echo ${CHANGED_MODULES} | wc -w) + yetus_debug "Locate the union of ${CHANGED_MODULES[*]}" + count=${#CHANGED_MODULES[@]} if [[ ${count} -lt 2 ]]; then - yetus_debug "Only one entry, so keeping it ${CHANGED_MODULES}" + yetus_debug "Only one entry, so keeping it ${CHANGED_MODULES[0]}" # shellcheck disable=SC2034 - CHANGED_UNION_MODULES=${CHANGED_MODULES} - + CHANGED_UNION_MODULES="${CHANGED_MODULES[0]}" else - i=1 while [[ ${i} -lt 100 ]] do - module=$(echo "${CHANGED_MODULES}" | tr ' ' '\n' | cut -f1-${i} -d/ | uniq) - count=$(echo "${module}" | wc -w) + tmpmods=() + for j in "${CHANGED_MODULES[@]}"; do + tmpmods=("${tmpmods[@]}" $(echo "${j}" | cut -f1-${i} -d/)) + done + tmpmods=($(printf "%s\n" "${tmpmods[@]}" | sort -u)) + + module=${tmpmods[0]} + count=${#tmpmods[@]} if [[ ${count} -eq 1 && -f ${module}/${buildfile} ]]; then prev_builddir=${module} @@ -1389,7 +1387,7 @@ function determine_needed_tests local i local plugin - for i in ${CHANGED_FILES}; do + for i in "${CHANGED_FILES[@]}"; do yetus_debug "Determining needed tests for ${i}" personality_file_tests "${i}" @@ -1575,7 +1573,7 @@ function check_reexec "${DOCKERFILE}"; do tpdir=$(relative_dir "${testdir}") if [[ $? == 0 - && ${CHANGED_FILES} =~ ${tpdir} ]]; then + && "${CHANGED_FILES[*]}" =~ ${tpdir} ]]; then copy=true fi done
