kou commented on code in PR #13453:
URL: https://github.com/apache/arrow/pull/13453#discussion_r910490933
##########
.github/workflows/r_nightly.yml:
##########
@@ -77,10 +86,47 @@ jobs:
echo "No files found. Stopping upload."
exit 1
fi
+ - name: Cache Repo
+ uses: actions/cache@v3
+ with:
+ path: repo
+ key: r-nightly-${{ github.run_id }}
+ restore-keys: r-nightly-
+ - name: Sync from Remote
+ env:
+ RSYNC_PATH: repo
+ SSH_KEY: "${{ secrets.NIGHTLIES_RSYNC_KEY }}"
+ PORT: "${{ secrets.NIGHTLIES_RSYNC_PORT }}"
+ USER: "${{ secrets.NIGHTLIES_RSYNC_USER }}"
+ HOST: "${{ secrets.NIGHTLIES_RSYNC_HOST }}"
+ HOST_KEY: "${{ secrets.NIGHTLIES_RSYNC_HOST_KEY }}"
+ REMOTE_PATH: "${{ secrets.NIGHTLIES_RSYNC_PATH }}/arrow/r"
+ run: |
+ # Make SSH key available and add remote to known hosts
+ eval "$(ssh-agent)" > /dev/null
+ echo "$SSH_KEY" | tr -d '\r' | DISPLAY=1 ssh-add - >/dev/null
+ mkdir -p ~/.ssh
+ echo "$HOST_KEY" >> ~/.ssh/known_hosts
+
+ # strict errors
+ set -eu
+
+ # avzrh - archive mode, verbose, with compression, recurse into
dirs, human readable numbers
+ # transfer only new files, delete files not in source, show
progress info
+ RSYNC_SWITCHES="-avzrh --update --delete --progress"
+ # We have to use a custom remote shell to supply the port
+ RSH="ssh -p $PORT"
+ LOCAL_PATH="$GITHUB_WORKSPACE/$RSYNC_PATH"
+ DSN="$USER@$HOST"
+
+ # Deploy. It is important to append '/*' to the source path
otherwise
+ # the entire source dir will be created as a sub dir in the
destination
+ sh -c "rsync $RSYNC_SWITCHES -e '$RSH' $DSN:$REMOTE_PATH/*
$LOCAL_PATH"
+ - run: tree repo
Review Comment:
Could you show an output of this?
I don't know why we don't use the same directory structure for `repo/` and
https://nightlies.apache.org/arrow/r/ . If we use the same directory structure,
we don't need to append `/*` for `rsync` source.
##########
.github/workflows/r_nightly.yml:
##########
@@ -77,10 +86,47 @@ jobs:
echo "No files found. Stopping upload."
exit 1
fi
+ - name: Cache Repo
+ uses: actions/cache@v3
+ with:
+ path: repo
+ key: r-nightly-${{ github.run_id }}
+ restore-keys: r-nightly-
+ - name: Sync from Remote
+ env:
+ RSYNC_PATH: repo
+ SSH_KEY: "${{ secrets.NIGHTLIES_RSYNC_KEY }}"
+ PORT: "${{ secrets.NIGHTLIES_RSYNC_PORT }}"
+ USER: "${{ secrets.NIGHTLIES_RSYNC_USER }}"
+ HOST: "${{ secrets.NIGHTLIES_RSYNC_HOST }}"
+ HOST_KEY: "${{ secrets.NIGHTLIES_RSYNC_HOST_KEY }}"
+ REMOTE_PATH: "${{ secrets.NIGHTLIES_RSYNC_PATH }}/arrow/r"
+ run: |
+ # Make SSH key available and add remote to known hosts
+ eval "$(ssh-agent)" > /dev/null
+ echo "$SSH_KEY" | tr -d '\r' | DISPLAY=1 ssh-add - >/dev/null
+ mkdir -p ~/.ssh
+ echo "$HOST_KEY" >> ~/.ssh/known_hosts
+
+ # strict errors
+ set -eu
+
+ # avzrh - archive mode, verbose, with compression, recurse into
dirs, human readable numbers
+ # transfer only new files, delete files not in source, show
progress info
+ RSYNC_SWITCHES="-avzrh --update --delete --progress"
+ # We have to use a custom remote shell to supply the port
+ RSH="ssh -p $PORT"
+ LOCAL_PATH="$GITHUB_WORKSPACE/$RSYNC_PATH"
+ DSN="$USER@$HOST"
+
+ # Deploy. It is important to append '/*' to the source path
otherwise
+ # the entire source dir will be created as a sub dir in the
destination
+ sh -c "rsync $RSYNC_SWITCHES -e '$RSH' $DSN:$REMOTE_PATH/*
$LOCAL_PATH"
Review Comment:
Why do we need `sh -c`?
##########
.github/workflows/r_nightly.yml:
##########
@@ -77,10 +86,47 @@ jobs:
echo "No files found. Stopping upload."
exit 1
fi
+ - name: Cache Repo
+ uses: actions/cache@v3
+ with:
+ path: repo
+ key: r-nightly-${{ github.run_id }}
+ restore-keys: r-nightly-
+ - name: Sync from Remote
+ env:
+ RSYNC_PATH: repo
+ SSH_KEY: "${{ secrets.NIGHTLIES_RSYNC_KEY }}"
+ PORT: "${{ secrets.NIGHTLIES_RSYNC_PORT }}"
+ USER: "${{ secrets.NIGHTLIES_RSYNC_USER }}"
+ HOST: "${{ secrets.NIGHTLIES_RSYNC_HOST }}"
+ HOST_KEY: "${{ secrets.NIGHTLIES_RSYNC_HOST_KEY }}"
+ REMOTE_PATH: "${{ secrets.NIGHTLIES_RSYNC_PATH }}/arrow/r"
+ run: |
+ # Make SSH key available and add remote to known hosts
+ eval "$(ssh-agent)" > /dev/null
+ echo "$SSH_KEY" | tr -d '\r' | DISPLAY=1 ssh-add - >/dev/null
+ mkdir -p ~/.ssh
+ echo "$HOST_KEY" >> ~/.ssh/known_hosts
+
+ # strict errors
+ set -eu
+
+ # avzrh - archive mode, verbose, with compression, recurse into
dirs, human readable numbers
+ # transfer only new files, delete files not in source, show
progress info
+ RSYNC_SWITCHES="-avzrh --update --delete --progress"
Review Comment:
`-r` is redundant because `a` includes `-r`:
```bash
$ rsync --help
....
--archive, -a archive mode is -rlptgoD (no -A,-X,-U,-N,-H)
...
```
##########
.github/workflows/r_nightly.yml:
##########
@@ -77,10 +86,47 @@ jobs:
echo "No files found. Stopping upload."
exit 1
fi
+ - name: Cache Repo
+ uses: actions/cache@v3
+ with:
+ path: repo
+ key: r-nightly-${{ github.run_id }}
+ restore-keys: r-nightly-
+ - name: Sync from Remote
+ env:
+ RSYNC_PATH: repo
+ SSH_KEY: "${{ secrets.NIGHTLIES_RSYNC_KEY }}"
+ PORT: "${{ secrets.NIGHTLIES_RSYNC_PORT }}"
+ USER: "${{ secrets.NIGHTLIES_RSYNC_USER }}"
+ HOST: "${{ secrets.NIGHTLIES_RSYNC_HOST }}"
+ HOST_KEY: "${{ secrets.NIGHTLIES_RSYNC_HOST_KEY }}"
+ REMOTE_PATH: "${{ secrets.NIGHTLIES_RSYNC_PATH }}/arrow/r"
+ run: |
+ # Make SSH key available and add remote to known hosts
+ eval "$(ssh-agent)" > /dev/null
+ echo "$SSH_KEY" | tr -d '\r' | DISPLAY=1 ssh-add - >/dev/null
+ mkdir -p ~/.ssh
Review Comment:
Do we need `chmod go-rwx ~/.ssh`?
##########
.github/workflows/r_nightly.yml:
##########
@@ -77,10 +86,47 @@ jobs:
echo "No files found. Stopping upload."
exit 1
fi
+ - name: Cache Repo
+ uses: actions/cache@v3
+ with:
+ path: repo
+ key: r-nightly-${{ github.run_id }}
+ restore-keys: r-nightly-
+ - name: Sync from Remote
+ env:
+ RSYNC_PATH: repo
+ SSH_KEY: "${{ secrets.NIGHTLIES_RSYNC_KEY }}"
+ PORT: "${{ secrets.NIGHTLIES_RSYNC_PORT }}"
+ USER: "${{ secrets.NIGHTLIES_RSYNC_USER }}"
+ HOST: "${{ secrets.NIGHTLIES_RSYNC_HOST }}"
+ HOST_KEY: "${{ secrets.NIGHTLIES_RSYNC_HOST_KEY }}"
+ REMOTE_PATH: "${{ secrets.NIGHTLIES_RSYNC_PATH }}/arrow/r"
+ run: |
+ # Make SSH key available and add remote to known hosts
+ eval "$(ssh-agent)" > /dev/null
+ echo "$SSH_KEY" | tr -d '\r' | DISPLAY=1 ssh-add - >/dev/null
Review Comment:
Why do we need `DISPLAY=1`?
I think that there is no X on GitHub Actions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
