dmsuehir commented on PR #9969: URL: https://github.com/apache/incubator-gluten/pull/9969#issuecomment-2986141172
@PHILO-HE Thanks, I've tried that out and ran into a couple of issues: * The install directory for `install_spark_resources.sh` is hardcoded to `/opt`. When trying to run this without sudo, I get a permission denied error * After running `install_spark_resources.sh`, the directory has the Spark test resource files, but it does not have the Spark binaries. This works for running unit tests, but without binaries the doc generation script fails. You can see from this [PR](https://github.com/apache/incubator-gluten/pull/9499/files#diff-cc9b23a1706d7aac3ffaa94c96692d477c2cfb375b994d23289190d0f581e3c4R125), which is trying to add a GHA weekly run for the doc generation script, that it's needing to download the Spark binaries and then combine that with the test resource files from the `install_spark_resources.sh` directory. I can mitigate these issues with a couple of changes to the `install_spark_resource.sh` script: * Adding an optional second positional arg to the script for the spark dir (if it's not provided, it can still default to `/opt`) * Extracting the bin/python dir from the spark .tgz file (right now the script only grabs `jars`) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
