[
https://issues.apache.org/jira/browse/HIVE-28603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitriy Fingerman resolved HIVE-28603.
--------------------------------------
Fix Version/s: 4.1.0
Resolution: Fixed
Thanks for your contribution, [~nikunjagarwal321] !
> Multiple tests in serde module are flaky
> ----------------------------------------
>
> Key: HIVE-28603
> URL: https://issues.apache.org/jira/browse/HIVE-28603
> Project: Hive
> Issue Type: Improvement
> Security Level: Public(Viewable by anyone)
> Components: Serializers/Deserializers, Tests
> Affects Versions: 4.0.1
> Reporter: Nikunj Agarwal
> Assignee: Nikunj Agarwal
> Priority: Minor
> Labels: flaky-test, pull-request-available, serde, test
> Fix For: 4.1.0
>
> Attachments: Screenshot 2024-11-02 at 2.48.35 PM.png, Screenshot
> 2024-11-02 at 2.54.31 PM.png
>
>
> There are multiple tests in serde module which are flaky.
> h2. Flakiness in the tests -
> A tool called [nondex|https://github.com/TestingResearchIllinois/NonDex] was
> used to identify the flaky tests. NonDex is a tool for detecting and
> debugging wrong assumptions on under-determined Java APIs.
> h2. Steps to reproduce flakiness using nondex -
>
> {code:java}
> mvn install -pl serde -DskipTests
> mvn -pl serde test
> -Dtest=org.apache.hadoop.hive.serde2.objectinspector.TestProtocolBuffersObjectInspectors#testProtocolBuffersObjectInspectors
> mvn -pl serde edu.illinois:nondex-maven-plugin:2.1.7:nondex
> -Dtest=org.apache.hadoop.hive.serde2.objectinspector.TestProtocolBuffersObjectInspectors#testProtocolBuffersObjectInspectors
> {code}
> h3. ERROR logs:
>
> {code:java}
> [ERROR] Failures:
> [ERROR]
> TestProtocolBuffersObjectInspectors.testProtocolBuffersObjectInspectors:71
> expected:<test> but was:<[one, two]>{code}
>
>
> Similar Errors were found across different test cases across different
> classes. The list of test cases is as follows:
> |org.apache.hadoop.hive.serde2.objectinspector.TestProtocolBuffersObjectInspectors.testProtocolBuffersObjectInspectors|
> |org.apache.hadoop.hive.serde2.objectinspector.TestThriftObjectInspectors.testThriftObjectInspectors|
> |org.apache.hadoop.hive.serde2.objectinspector.TestReflectionObjectInspectors.testReflectionObjectInspectors|
> |org.apache.hadoop.hive.serde2.columnar.TestLazyBinaryColumnarSerDe.testLazyBinaryColumnarSerDeWithEmptyBinary|
> |org.apache.hadoop.hive.serde2.columnar.TestLazyBinaryColumnarSerDe.testSerDeInnerNulls|
> |org.apache.hadoop.hive.serde2.lazybinary.TestLazyBinarySerDe.testLazyBinarySerDe|
> |org.apache.hadoop.hive.serde2.TestStatsSerde.testLazyBinarySerDe|
> |org.apache.hadoop.hive.serde2.lazy.TestLazySimpleSerDe#testSerDeParameters|
> |org.apache.hadoop.hive.serde2.binarysortable.TestBinarySortableSerDe#testBinarySortableSerDe|
>
> h2. Reason for failure :
> The method *_getDeclaredNonStaticFields(Class<?> c)_* in class
> _*ObjectInspectorUtils.java*_ ** uses class.getDeclaredFields() method which
> is nondeterministic as per [java
> docs|https://docs.oracle.com/javase/8/docs/api/java/lang/Class.html#getDeclaredFields--].
>
> {quote} Field[] f = c.getDeclaredFields();
> {quote}
> As per the test cases, it is assumed that the return order of the fields will
> always be the same, which may not be the case.
>
> h2. Proposed Fix :
> Ideally we should make the getDeclaredNonStaticFields() function
> deterministic, which is not the case as of now.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
