det101 commented on issue #17244: URL: https://github.com/apache/dolphinscheduler/issues/17244#issuecomment-2955000165
> Thanks for bringing this. Before PR there are still some issues to be sorted out. > > 1. There are a lot of third-party cloud dependencies on data source types, task types and storage types, such as AWS, Tencent Cloud, Alibaba Cloud, GCP and so on. These closed-source tools can't be realized by open-source automated testing. How to ensure that these will work normally after the upgrade is completed? > 2. Hadoop 2 does not support JDK17, and there are still a large number of users using Hadoop 2. How to achieve Hadoop 2 compatibility under JDK 17? > 3. Many third-party open source dependencies need to be upgraded. How to ensure the compatibility after upgrading? The purpose of the upgrade is to enable DS to run on JDK17. The problem you mentioned can be solved by the following methods: 1. Regarding third-party cloud dependencies, including compatibility issues of data source types, task types, and storage types, I would first read the developer documentation of all third-party clouds to understand the support of the current reference version of DS for JDK17. If it is clearly not supported, check whether the new version of the dependency is forward compatible with lower-version services (I have seen S3 2.X versions support JDK17). If there is a situation where a higher-version dependency cannot support a lower-version cloud service, then users can only choose to run on JDK8/11. 2. There should be no problem submitting through the command line (such as hive -e, spark-sql, hdfs commands). Components use their own dependent JDKs, and DS uses JDK17 (multiple JDK environments are also common now). Similar to hadoop dependencies, low-version JDK dependencies may indeed have problems under JDK17 (to be verified). This issue requires determining which version hadoop DS supports by default. Or if it is 2.x, DS at least provides support for JDK17, and users only need to update the hadoop3-related dependency version. Regarding (hadoop3+DS), I can fully verify it in JDK17. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
