amanraj2520 commented on PR #213: URL: https://github.com/apache/tez/pull/213#issuecomment-1322109580
Hi @ayushtkn Thanks a lot for looking into this I checked the PR you were involved about upgrading Hive to 3.3.4 so I am referring this PR : https://github.com/apache/hive/pull/3578 I have answered your questions below: The reason why we wanted to use Hadoop 3.3.4 was because of some component vulnerabilities issues which were fixed by version upgrades from 3.3.1. Also 3.3.4 is the latest stable release version of Hadoop so it would be better to upgrade to the latest in Hive as well. Having said that, I see in the PR of Hive I have referred, testing had started for Hadoop 3.3.4 with Tez 0.10.2 and the possible solution to them was to upgrade Hive and Tez both to 3.3.4 simultaneously. Therefore, we want to start fresh and include the latest Hadoop 3.3.4 and Tez 0.10.3 (if approved will use 3.3.4), so that we ensure both hive-3.2.0 release and oss master start to use the same version. I can take up the responsibility of driving this 0.10.3 Tez release if approved. I don't think major changes will go for this Tez release. We can just upgrade the Hadoop to 3.3.4 on top of 0.10.2. 1. I completely agree with you on this. I think the PR was raised for upgrade in master was successful. So, I think we just need to upgrade the tez version on top of this PR and I can test it in my clusters before signing it off. 2. I see that Hive with 3.3.4 Hadoop and 0.10.2 passed all the unit test cases right? By basic tests you mean unit tests right? It was only while testing on the cluster we found some issues since Tez classpath had 3.3.1 jars instead of 3.3.4. So I think if we just upgrade to Hadoop 3.3.4 in Tez and use that Tez version in Hive, we should be good. Please let me know if you have any concerns, Ayush. Looking forward to your reply. @abstractdog Please suggest anything you find concerning in this release. Happy to help. Thanks, Aman. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
