steveloughran commented on a change in pull request #25559: [WIP][DO-NOT-MERGE]
Test updating Kinesis deps and current state of Kinesis Python tests
URL: https://github.com/apache/spark/pull/25559#discussion_r318557499
##########
File path: hadoop-cloud/pom.xml
##########
@@ -100,6 +100,10 @@
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
</exclusion>
+ <exclusion>
Review comment:
+1 to excluding the AWS dependency. It is not actually something you can
bundle into ASF releases anyway
[https://issues.apache.org/jira/browse/HADOOP-13794](https://issues.apache.org/jira/browse/HADOOP-13794).
But: it'd be good for a spark-hadoop-cloud artifact to be published with that
dependency for downstream users, or at least the things you have to add
documented somewhere.
FWIW, I do build and test the spark kinesis module as part of my AWS SDK
update process -one that actually went pretty smoothly for a change last time.
No regressions, no new error messages in logs, shaded JARs really are shaded,
etc. This is progress and means that backporting is something we should be doing
see
https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md#-qualifying-an-aws-sdk-update
for the runbook there
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
