dongjoon-hyun commented on a change in pull request #29040:
URL: https://github.com/apache/spark/pull/29040#discussion_r451600551
##########
File path: appveyor.yml
##########
@@ -49,7 +49,7 @@ build_script:
# '-Djna.nosys=true' is required to avoid kernel32.dll load failure.
# See SPARK-28759.
# Ideally we should check the tests related to Hive in SparkR as well
(SPARK-31745).
- - cmd: mvn -DskipTests -Psparkr -Djna.nosys=true package
+ - cmd: mvn -DskipTests -Phadoop-2.7 -Psparkr -Djna.nosys=true package
Review comment:
Can we upgrade to @steveloughran 's Hadoop 3.0 instead of using Hadoop
2.7.1 here?
-
https://github.com/apache/spark/blob/master/dev/appveyor-install-dependencies.ps1#L98
```
# ========================== Hadoop bin package
# This must match the version at
https://github.com/steveloughran/winutils/tree/master/hadoop-2.7.1
$hadoopVer = "2.7.1"
```
He provides Hadoop 3 also.
- https://github.com/steveloughran/winutils/tree/master/hadoop-3.0.0/bin
##########
File path: appveyor.yml
##########
@@ -49,7 +49,7 @@ build_script:
# '-Djna.nosys=true' is required to avoid kernel32.dll load failure.
# See SPARK-28759.
# Ideally we should check the tests related to Hive in SparkR as well
(SPARK-31745).
- - cmd: mvn -DskipTests -Psparkr -Djna.nosys=true package
+ - cmd: mvn -DskipTests -Phadoop-2.7 -Psparkr -Djna.nosys=true package
Review comment:
Can we upgrade to @steveloughran 's Hadoop 3.0 instead of using Hadoop
2.7.1 here?
-
https://github.com/apache/spark/blob/master/dev/appveyor-install-dependencies.ps1#L98
```
# ========================== Hadoop bin package
# This must match the version at
https://github.com/steveloughran/winutils/tree/master/hadoop-2.7.1
$hadoopVer = "2.7.1"
```
He already provides Hadoop 3 also.
- https://github.com/steveloughran/winutils/tree/master/hadoop-3.0.0/bin
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
