[
https://issues.apache.org/jira/browse/HUDI-9118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-9118:
--------------------------------------
Description:
Fix and re-enable TestHiveClientUtils.reuseHiveClientFromSparkSession
as part of investigating a flaky test issue, we had to disable 2 to 3 tests to
unblock master.
https://issues.apache.org/jira/browse/HUDI-9086
After some investigation, we narrowed it down to TestHiveClientUtils.setUp is
instantiating a spark session which is never shutdown. We could not really find
a fix for it. So, for now, we are disabling the only test in this class
(TestHiveClientUtils.reuseHiveClientFromSparkSession) so that we can re-enable
other two tests.
This test being disabled just validated the double checked locking paradigm
which is very common. So, its ok to have it disabled.
the two tests of interest that we are unblocking here are
[TestDataSourceUtils.java|https://github.com/apache/hudi/pull/12909/files#diff-3d53e5db41eda9c06f63851cfc4afc6a7bf8b9787fec08644e5ed1612c8b5edc]
. testDeduplicationAgainstRecordsAlreadyInTable
and
[TestHoodieDataSourceHelper.scala|https://github.com/apache/hudi/pull/12909/files#diff-f0a2a68911e46a43dc076937003a36390e416bed6b2c74b91682c927d2456911]
. testExtractPredicatesWithinOutputSet
was:
Fix and re-enable TestHiveClientUtils.reuseHiveClientFromSparkSession
as part of investigating a flaky test issue, we had to disable 2 to 3 tests to
unblock master.
https://issues.apache.org/jira/browse/HUDI-9086
After some investigation, we narrowed it down to TestHiveClientUtils.setUp is
instantiating a spark session which is never shutdown. We could not really find
a fix for it. So, for now, we are disabling the only test in this class
(TestHiveClientUtils.reuseHiveClientFromSparkSession) so that we can re-enable
other two tests.
This test being disabled just validated the double checked locking paradigm
which is very common. So, its ok to have it disabled.
> Fix and re-enable TestHiveClientUtils.reuseHiveClientFromSparkSession
> ---------------------------------------------------------------------
>
> Key: HUDI-9118
> URL: https://issues.apache.org/jira/browse/HUDI-9118
> Project: Apache Hudi
> Issue Type: Test
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Major
> Fix For: 1.1.0
>
>
> Fix and re-enable TestHiveClientUtils.reuseHiveClientFromSparkSession
>
> as part of investigating a flaky test issue, we had to disable 2 to 3 tests
> to unblock master.
> https://issues.apache.org/jira/browse/HUDI-9086
>
> After some investigation, we narrowed it down to TestHiveClientUtils.setUp is
> instantiating a spark session which is never shutdown. We could not really
> find a fix for it. So, for now, we are disabling the only test in this class
> (TestHiveClientUtils.reuseHiveClientFromSparkSession) so that we can
> re-enable other two tests.
>
> This test being disabled just validated the double checked locking paradigm
> which is very common. So, its ok to have it disabled.
>
>
> the two tests of interest that we are unblocking here are
> [TestDataSourceUtils.java|https://github.com/apache/hudi/pull/12909/files#diff-3d53e5db41eda9c06f63851cfc4afc6a7bf8b9787fec08644e5ed1612c8b5edc]
> . testDeduplicationAgainstRecordsAlreadyInTable
> and
> [TestHoodieDataSourceHelper.scala|https://github.com/apache/hudi/pull/12909/files#diff-f0a2a68911e46a43dc076937003a36390e416bed6b2c74b91682c927d2456911]
> . testExtractPredicatesWithinOutputSet
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
