assignUser commented on code in PR #13453:
URL: https://github.com/apache/arrow/pull/13453#discussion_r910865912
##########
.github/workflows/r_nightly.yml:
##########
@@ -77,10 +86,47 @@ jobs:
echo "No files found. Stopping upload."
exit 1
fi
+ - name: Cache Repo
+ uses: actions/cache@v3
+ with:
+ path: repo
+ key: r-nightly-${{ github.run_id }}
+ restore-keys: r-nightly-
+ - name: Sync from Remote
+ env:
+ RSYNC_PATH: repo
+ SSH_KEY: "${{ secrets.NIGHTLIES_RSYNC_KEY }}"
+ PORT: "${{ secrets.NIGHTLIES_RSYNC_PORT }}"
+ USER: "${{ secrets.NIGHTLIES_RSYNC_USER }}"
+ HOST: "${{ secrets.NIGHTLIES_RSYNC_HOST }}"
+ HOST_KEY: "${{ secrets.NIGHTLIES_RSYNC_HOST_KEY }}"
+ REMOTE_PATH: "${{ secrets.NIGHTLIES_RSYNC_PATH }}/arrow/r"
+ run: |
+ # Make SSH key available and add remote to known hosts
+ eval "$(ssh-agent)" > /dev/null
+ echo "$SSH_KEY" | tr -d '\r' | DISPLAY=1 ssh-add - >/dev/null
+ mkdir -p ~/.ssh
+ echo "$HOST_KEY" >> ~/.ssh/known_hosts
+
+ # strict errors
+ set -eu
+
+ # avzrh - archive mode, verbose, with compression, recurse into
dirs, human readable numbers
+ # transfer only new files, delete files not in source, show
progress info
+ RSYNC_SWITCHES="-avzrh --update --delete --progress"
+ # We have to use a custom remote shell to supply the port
+ RSH="ssh -p $PORT"
+ LOCAL_PATH="$GITHUB_WORKSPACE/$RSYNC_PATH"
+ DSN="$USER@$HOST"
+
+ # Deploy. It is important to append '/*' to the source path
otherwise
+ # the entire source dir will be created as a sub dir in the
destination
+ sh -c "rsync $RSYNC_SWITCHES -e '$RSH' $DSN:$REMOTE_PATH/*
$LOCAL_PATH"
+ - run: tree repo
Review Comment:
A small example:
```
$ tree src
src
└── arrow
└── r
└── pkg.file
2 directories, 1 file
$ tree dest
dest
0 directories, 0 files
$ rsync -avzh --update --delete src dest
sending incremental file list
src/
src/arrow/
src/arrow/r/
src/arrow/r/pkg.file
sent 180 bytes received 51 bytes 462.00 bytes/sec
total size is 0 speedup is 0.00
$ tree dest
dest
└── src
└── arrow
└── r
└── pkg.file
3 directories, 1 file
```
vs
```
$ rsync -avzh --update --delete src/* dest
sending incremental file list
arrow/
arrow/r/
arrow/r/pkg.file
sent 158 bytes received 47 bytes 410.00 bytes/sec
total size is 0 speedup is 0.00
$ tree dest
dest
└── arrow
└── r
└── pkg.file
2 directories, 1 file
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
