[
https://issues.apache.org/jira/browse/BEAM-8876?focusedWorklogId=512341&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-512341
]
ASF GitHub Bot logged work on BEAM-8876:
----------------------------------------
Author: ASF GitHub Bot
Created on: 16/Nov/20 13:06
Start Date: 16/Nov/20 13:06
Worklog Time Spent: 10m
Work Description: piotr-szuberski commented on a change in pull request
#13287:
URL: https://github.com/apache/beam/pull/13287#discussion_r524253396
##########
File path: sdks/java/io/hadoop-common/build.gradle
##########
@@ -16,17 +16,21 @@
* limitations under the License.
*/
+import java.util.stream.Collectors
+
plugins { id 'org.apache.beam.module' }
applyJavaNature(automaticModuleName: 'org.apache.beam.sdk.io.hadoop.common')
description = "Apache Beam :: SDKs :: Java :: IO :: Hadoop Common"
ext.summary = "Library to add shared Hadoop classes among Beam IOs."
-configurations {
- hadoopVersion285
-}
+def hadoopVersions = [
+ "285": "2.8.5",
+ "292": "2.9.2",
+ "2101": "2.10.1",
Review comment:
I have the same issue - I tried to add some global variable to
BeamModulePlugin but I don't get the Gradle's keikaku how to do such thing.
About 3.2.1 - there are Jira tasks for it so I wanted to do it in another PR
(as this one already is about 2 things at once - IT test and version testing).
I'm not sure whether every module passes the tests. I'll do that right after
this is merged.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 512341)
Time Spent: 5h 40m (was: 5.5h)
> Tests to confirm Hadoop version compatibility
> ---------------------------------------------
>
> Key: BEAM-8876
> URL: https://issues.apache.org/jira/browse/BEAM-8876
> Project: Beam
> Issue Type: Test
> Components: build-system
> Reporter: Tomo Suzuki
> Assignee: Piotr Szuberski
> Priority: P3
> Time Spent: 5h 40m
> Remaining Estimate: 0h
>
> Add test(s) to confirm Hadoop version compatibility for hadoop_client library
> version against different users' environment.
> https://github.com/apache/beam/pull/10222#issuecomment-559158067
> [~iemejia] had an idea:
> {quote}After a quick review most of the major users and systems still support
> 2.7 and ‘even worse’ 2.6. So I was wondering how can we ensure that new
> patches on Hadoop modules don’t break backwards compatibility.
> One idea could be to add an extra set of tests that run the build for the
> provided previous versions of the dependencies. But I don’t have a clue how
> to do this in gradle. Do you by any chance? (Note that a fix for this will
> solve other compatibility issues we have with other systems and provided
> dependencies e.g. Spark and Kafka). {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
