[
https://issues.apache.org/jira/browse/SPARK-22940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16308491#comment-16308491
]
Sean Owen commented on SPARK-22940:
---
Agreed. Many other scripts require wget, though at least one will use either.
The easiest option is just to document that you need wget, which is simple
enough. I forget whether I have it via XCode tools or brew, but, brew install
wget should always be sufficient.
Using a library is just fine too and probably more desirable within the test
code.
Feel free to open a PR for any of the options, I'd say.
> Test suite HiveExternalCatalogVersionsSuite fails on platforms that don't
> have wget installed
> -
>
> Key: SPARK-22940
> URL: https://issues.apache.org/jira/browse/SPARK-22940
> Project: Spark
> Issue Type: Bug
> Components: Tests
>Affects Versions: 2.2.1
> Environment: MacOS Sierra 10.12.6
>Reporter: Bruce Robbins
>Priority: Minor
>
> On platforms that don't have wget installed (e.g., Mac OS X), test suite
> org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite throws an
> exception and aborts:
> java.io.IOException: Cannot run program "wget": error=2, No such file or
> directory
> HiveExternalCatalogVersionsSuite uses wget to download older versions of
> Spark for compatibility testing. First it uses wget to find a suitable
> mirror, and then it uses wget to download a tar file from the mirror.
> There are several ways to fix this (in reverse order of difficulty of
> implementation)
> 1. Require Mac OS X users to install wget if they wish to run unit tests (or
> at the very least if they wish to run HiveExternalCatalogVersionsSuite).
> Also, update documentation to make this requirement explicit.
> 2. Fall back on curl when wget is not available.
> 3. Use an HTTP library to query for a suitable mirror and download the tar
> file.
> Number 2 is easy to implement, and I did so to get the unit test to run. But
> it relies on another external program if wget is not installed.
> Number 3 is probably slightly more complex to implement and requires more
> corner-case checking (e.g, redirects, etc.).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
