[incubator-celeborn] branch main updated: [CELEBORN-978] Improve dependency.sh replacement mode

[zhouky]

This is an automated email from the ASF dual-hosted git repository.

zhouky pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git


The following commit(s) were added to refs/heads/main by this push:
     new 045c682c8 [CELEBORN-978] Improve dependency.sh replacement mode
045c682c8 is described below

commit 045c682c8901314d2b1171fbd481857334e885b8
Author: sychen <syc...@ctrip.com>
AuthorDate: Sat Sep 16 09:35:13 2023 +0800

    [CELEBORN-978] Improve dependency.sh replacement mode
    
    ### What changes were proposed in this pull request?
    
    ### Why are the changes needed?
    When executing the update script locally, it may generate such a Log, which 
causes awk to exit with an error.
    ```
    Downloading from nexus: httpxxxx
    ```
    
    ```bash
    ./dev/dependencies.sh --replace
    ```
    
    ```
    awk: trying to access out of range field -1
     input record number 1, file
     source line number 2
    ```
    
    ### Does this PR introduce _any_ user-facing change?
    
    ### How was this patch tested?
    
    Closes #1914 from cxzl25/CELEBORN-978.
    
    Authored-by: sychen <syc...@ctrip.com>
    Signed-off-by: zky.zhoukeyong <zky.zhoukey...@alibaba-inc.com>
---
 .github/workflows/deps.yml | 1 +
 dev/dependencies.sh        | 8 +++++---
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/.github/workflows/deps.yml b/.github/workflows/deps.yml
index 1fa8c0dc9..1567ab401 100644
--- a/.github/workflows/deps.yml
+++ b/.github/workflows/deps.yml
@@ -26,6 +26,7 @@ on:
       - '**/pom.xml'
       - 'project/**'
       - '.github/workflows/deps.yml'
+      - 'dev/dependencies.sh'
 
 concurrency:
   group: deps-${{ github.head_ref || github.run_id }}
diff --git a/dev/dependencies.sh b/dev/dependencies.sh
index 1d84dd5c7..6dc486a45 100755
--- a/dev/dependencies.sh
+++ b/dev/dependencies.sh
@@ -39,9 +39,10 @@ DEP=""
 function mvn_build_classpath() {
   $MVN -P$MODULE clean install -DskipTests -am -pl $MVN_MODULES
   $MVN -P$MODULE dependency:build-classpath -am -pl $MVN_MODULES | \
-    grep -v "INFO\|WARN" | \
-    # This will skip the first two lines 
-    tail -n +3 | \
+    grep -A1 "Dependencies classpath:" | \
+    grep -v "^--$" | \
+    grep -v "Dependencies classpath:" | \
+    grep -v '^$' | \
     tr ":" "\n" | \
     awk -F '/' '{
       artifact_id=$(NF-2);
@@ -186,6 +187,7 @@ case "$MODULE" in
     SBT_PROJECT="celeborn-client-flink-1_17"
     ;;
   *)
+    MODULE="server"
     MVN_MODULES="worker,master"
     ;;
 esac