Nikunj Agarwal created HIVE-28603:
-------------------------------------
Summary: Multiple tests in serde module are flaky
Key: HIVE-28603
URL: https://issues.apache.org/jira/browse/HIVE-28603
Project: Hive
Issue Type: Improvement
Security Level: Public (Viewable by anyone)
Components: Serializers/Deserializers, Tests
Affects Versions: 4.0.1
Reporter: Nikunj Agarwal
Attachments: Screenshot 2024-11-02 at 2.48.35 PM.png, Screenshot
2024-11-02 at 2.54.31 PM.png
There are multiple tests in serde module which are flaky.
h2. Flakiness in the tests -
A tool called [nondex|https://github.com/TestingResearchIllinois/NonDex] was
used to identify the flaky tests. NonDex is a tool for detecting and debugging
wrong assumptions on under-determined Java APIs.
h2. Steps to reproduce flakiness using nondex -
{code:java}
mvn install -pl serde -DskipTests
mvn -pl serde test
-Dtest=org.apache.hadoop.hive.serde2.objectinspector.TestProtocolBuffersObjectInspectors#testProtocolBuffersObjectInspectors
mvn -pl serde edu.illinois:nondex-maven-plugin:2.1.7:nondex
-Dtest=org.apache.hadoop.hive.serde2.objectinspector.TestProtocolBuffersObjectInspectors#testProtocolBuffersObjectInspectors
{code}
h3. ERROR logs:
{code:java}
[ERROR] Failures:
[ERROR]
TestProtocolBuffersObjectInspectors.testProtocolBuffersObjectInspectors:71
expected:<test> but was:<[one, two]>{code}
Similar Errors were found across different test cases across different classes.
The list of test cases is as follows:
|org.apache.hadoop.hive.serde2.objectinspector.TestProtocolBuffersObjectInspectors.testProtocolBuffersObjectInspectors|
|org.apache.hadoop.hive.serde2.objectinspector.TestThriftObjectInspectors.testThriftObjectInspectors|
|org.apache.hadoop.hive.serde2.objectinspector.TestReflectionObjectInspectors.testReflectionObjectInspectors|
|org.apache.hadoop.hive.serde2.columnar.TestLazyBinaryColumnarSerDe.testLazyBinaryColumnarSerDeWithEmptyBinary|
|org.apache.hadoop.hive.serde2.columnar.TestLazyBinaryColumnarSerDe.testSerDeInnerNulls|
|org.apache.hadoop.hive.serde2.lazybinary.TestLazyBinarySerDe.testLazyBinarySerDe|
|org.apache.hadoop.hive.serde2.TestStatsSerde.testLazyBinarySerDe|
|org.apache.hadoop.hive.serde2.lazy.TestLazySimpleSerDe#testSerDeParameters|
|org.apache.hadoop.hive.serde2.binarysortable.TestBinarySortableSerDe#testBinarySortableSerDe|
h2. Reason for failure :
The method *_getDeclaredNonStaticFields(Class<?> c)_* in class
_*ObjectInspectorUtils.java*_ ** uses class.getDeclaredFields() method which is
nondeterministic as per [java
docs|https://docs.oracle.com/javase/8/docs/api/java/lang/Class.html#getDeclaredFields--].
{quote} Field[] f = c.getDeclaredFields();
{quote}
As per the test cases, it is assumed that the return order of the fields will
always be the same, which may not be the case.
h2. Proposed Fix :
Ideally we should make the getDeclaredNonStaticFields() function deterministic,
which is not the case as of now.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
