LucasEby opened a new issue, #17376: URL: https://github.com/apache/pinot/issues/17376
The below tests fail due to a combination of problems: - Currently, the implementation of [SegmentDictionaryCreator.java](https://github.com/apache/pinot/blob/master/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentDictionaryCreator.java) has a bug in it where, if a key is not matched, row 0's automatically pollutes the value in the row with the unmatched key. This is because the class utilizes [Double2IntOpenHashMap](https://javadoc.io/doc/it.unimi.dsi/fastutil/latest/it/unimi/dsi/fastutil/doubles/Double2IntFunction.html#defaultReturnValue()), [Float2IntOpenHashMap](https://javadoc.io/doc/it.unimi.dsi/fastutil/latest/it/unimi/dsi/fastutil/floats/Float2IntMap.html#defaultReturnValue()), [Int2IntOpenHashMap](https://javadoc.io/doc/it.unimi.dsi/fastutil/latest/it/unimi/dsi/fastutil/ints/Int2IntFunction.html#defaultReturnValue()), [Long2IntOpenHashMap](https://javadoc.io/doc/it.unimi.dsi/fastutil/latest/it/unimi/dsi/fastutil/longs/Long2IntFunction.html#defaultReturnValue()), and [Object2IntOpen HashMap](https://javadoc.io/doc/it.unimi.dsi/fastutil/latest/it/unimi/dsi/fastutil/objects/Object2IntFunction.html#defaultReturnValue()) to compare the dictionary keys. According to the documentation for each of their `get`/`getInt` methods, the default return value is `set to 0/false at creation` if a key is not matched. Therefore, row 0 can sometimes pollute other rows, thus causing data to be incorrectly duplicated across rows. - Avro Records actually do not claim to enforce ordering for the key-value pairings in [Avro Records](https://avro.apache.org/docs/1.11.1/specification/). Avro's [GenericDatumReader](https://github.com/LucasEby/avro/blob/0ddfb1e08d1ebc067fc2d370a383b23a8093ee19/lang/java/avro/src/main/java/org/apache/avro/generic/GenericDatumReader.java) actually utilizes a HashMap internally so we cannot rely on it's mapped ordering. [HashMaps](https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html), make no guarantees to the order of it's map. Therefore, each time an Avro Record was read in the below test failures, the data would be returned in an unpredictable order (data is read from the file twice for each of these tests), thus sometimes causing the order of the JSON key-value pairs to be different, causing the key matching to fail. As mentioned, with the current implementation, row 0 pollutes these rows by default. - [SegmentDictionaryCreator.java](https://github.com/apache/pinot/blob/master/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentDictionaryCreator.java) makes an incorrect assumption in regards to the ordering of key-value pairs of stringified JSON objects. However, JSON objects actually do not enforce order. According to their documentation, `an object is an unordered set of name/value pairs`. Therefore, an allowance needs to be made when comparing JSON key-value pairs inside of JSON objects if a string comparison fails in `SegmentDictionaryCreator`. - Additionally, multiple of the below tests make incorrect assumptions on the ordering of the JSON key-value pairs when performing assertions. As previously mentioned, the key-value pair ordering can actually be returned in different orders. The ordering for each of the above mentioned order-related problems can change due to different environments producing the contents in different orders despite the logical contents being the same. Since the raw strings/trees were compared "as-is", harmless re-ordering could flip the test from pass to fail despite the data being semantically the same. ### Tool Utilized I discovered these problem with the [NonDex](https://github.com/TestingResearchIllinois/NonDex) tool. NonDex systematically detects incorrect tests that rely on non-deterministic behaviors in Java APIs—like assuming order of the contents of HashMaps — by exploring all specification-allowed outcomes. It does this by instrumenting undetermined APIs and randomizing the returned order/behavior within what the spec allows. Failures it exposes reliably indicate flawed assumptions in order that were never promised and should be fixed. ### Reproducible Error Outputs Test: JsonIngestionFromAvroQueriesTest#testComplexSelectOnJsonColumn Reproducible command: `mvn -pl pinot-core -Dtest=org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest#testComplexSelectOnJsonColumn -DreuseForks=false -DnondexRuns=1 -DnondexRunsWithoutShuffling=0 -DnondexSeed=1098954 edu.illinois:nondex-maven-plugin:2.2.1:nondex` Error output: [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- WARNING: package jdk.internal.util.random not in java.base [INFO] Running org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest [ERROR] Tests run: 6, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 4.185 s <<< FAILURE! -- in org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest [ERROR] org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest.testComplexSelectOnJsonColumn -- Time elapsed: 0.357 s <<< FAILURE! java.lang.AssertionError: expected [[[{"data":{"a":"1","b":"2"},"timestamp":1719390721}]]] but found [[[{"data":{"a":"2","b":"4"},"timestamp":1719390722}]]] at org.testng.Assert.fail(Assert.java:111) at org.testng.Assert.failNotEquals(Assert.java:1590) at org.testng.Assert.assertEqualsImpl(Assert.java:150) at org.testng.Assert.assertEquals(Assert.java:132) at org.testng.Assert.assertEquals(Assert.java:656) at org.testng.Assert.assertEquals(Assert.java:666) at org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest.testComplexSelectOnJsonColumn(JsonIngestionFromAvroQueriesTest.java:373) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:141) at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:687) at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:230) at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:63) at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:995) at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:203) at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:154) at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:134) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) at org.testng.TestRunner.privateRun(TestRunner.java:741) at org.testng.TestRunner.run(TestRunner.java:616) at org.testng.SuiteRunner.runTest(SuiteRunner.java:421) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:413) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:373) at org.testng.SuiteRunner.run(SuiteRunner.java:312) at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95) at org.testng.TestNG.runSuitesSequentially(TestNG.java:1274) at org.testng.TestNG.runSuitesLocally(TestNG.java:1208) at org.testng.TestNG.runSuites(TestNG.java:1112) at org.testng.TestNG.run(TestNG.java:1079) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:155) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:102) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:91) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:137) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) [ERROR] org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest.testJsonPathSelectOnJsonColumn -- Time elapsed: 0.116 s [ERROR] org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest.testSimpleSelectOnBytesJsonColumn -- Time elapsed: 0.004 s [ERROR] org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest.testSimpleSelectOnFixedJsonColumn -- Time elapsed: 0.001 s [ERROR] org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest.testSimpleSelectOnJsonColumn -- Time elapsed: 0.004 s <<< FAILURE! java.lang.AssertionError: expected [[3, donald duck, {"a":"1","b":"2"}, "UP"]] but found [[3, donald duck, "test", "UP"]] at org.testng.Assert.fail(Assert.java:111) at org.testng.Assert.failNotEquals(Assert.java:1590) at org.testng.Assert.assertEqualsImpl(Assert.java:150) at org.testng.Assert.assertEquals(Assert.java:132) at org.testng.Assert.assertEquals(Assert.java:656) at org.testng.Assert.assertEquals(Assert.java:666) at org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest.testSimpleSelectOnJsonColumn(JsonIngestionFromAvroQueriesTest.java:295) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:141) at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:687) at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:230) at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:63) at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:995) at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:203) at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:154) at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:134) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) at org.testng.TestRunner.privateRun(TestRunner.java:741) at org.testng.TestRunner.run(TestRunner.java:616) at org.testng.SuiteRunner.runTest(SuiteRunner.java:421) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:413) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:373) at org.testng.SuiteRunner.run(SuiteRunner.java:312) at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95) at org.testng.TestNG.runSuitesSequentially(TestNG.java:1274) at org.testng.TestNG.runSuitesLocally(TestNG.java:1208) at org.testng.TestNG.runSuites(TestNG.java:1112) at org.testng.TestNG.run(TestNG.java:1079) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:155) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:102) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:91) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:137) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) [ERROR] org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest.testStringValueSelectOnJsonColumn -- Time elapsed: 0.019 s [INFO] [INFO] Results: [INFO] [ERROR] Failures: [ERROR] JsonIngestionFromAvroQueriesTest.testComplexSelectOnJsonColumn:373 expected [[[{"data":{"a":"1","b":"2"},"timestamp":1719390721}]]] but found [[[{"data":{"a":"2","b":"4"},"timestamp":1719390722}]]] [ERROR] JsonIngestionFromAvroQueriesTest.testSimpleSelectOnJsonColumn:295 expected [[3, donald duck, {"a":"1","b":"2"}, "UP"]] but found [[3, donald duck, "test", "UP"]] [INFO] [ERROR] Tests run: 6, Failures: 2, Errors: 0, Skipped: 0 Test: JsonIngestionFromAvroQueriesTest#testSimpleSelectOnJsonColumn Reproducible command: `mvn -pl pinot-core -Dtest=org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest#testSimpleSelectOnJsonColumn -DreuseForks=false -DnondexRuns=1 -DnondexRunsWithoutShuffling=0 -DnondexSeed=974622 edu.illinois:nondex-maven-plugin:2.2.1:nondex` Error output: [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- WARNING: package jdk.internal.util.random not in java.base [INFO] Running org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.167 s <<< FAILURE! -- in org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest [ERROR] org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest.testSimpleSelectOnJsonColumn -- Time elapsed: 0.369 s <<< FAILURE! java.lang.AssertionError: expected [[2, mickey mouse, {"a":"1","b":"2"}, "DOWN"]] but found [[2, mickey mouse, "test", "DOWN"]] at org.testng.Assert.fail(Assert.java:111) at org.testng.Assert.failNotEquals(Assert.java:1590) at org.testng.Assert.assertEqualsImpl(Assert.java:150) at org.testng.Assert.assertEquals(Assert.java:132) at org.testng.Assert.assertEquals(Assert.java:656) at org.testng.Assert.assertEquals(Assert.java:666) at org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest.testSimpleSelectOnJsonColumn(JsonIngestionFromAvroQueriesTest.java:295) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:141) at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:687) at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:230) at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:63) at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:995) at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:203) at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:154) at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:134) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) at org.testng.TestRunner.privateRun(TestRunner.java:741) at org.testng.TestRunner.run(TestRunner.java:616) at org.testng.SuiteRunner.runTest(SuiteRunner.java:421) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:413) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:373) at org.testng.SuiteRunner.run(SuiteRunner.java:312) at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95) at org.testng.TestNG.runSuitesSequentially(TestNG.java:1274) at org.testng.TestNG.runSuitesLocally(TestNG.java:1208) at org.testng.TestNG.runSuites(TestNG.java:1112) at org.testng.TestNG.run(TestNG.java:1079) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:155) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:102) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:91) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:137) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) [INFO] [INFO] Results: [INFO] [ERROR] Failures: [ERROR] JsonIngestionFromAvroQueriesTest.testSimpleSelectOnJsonColumn:295 expected [[2, mickey mouse, {"a":"1","b":"2"}, "DOWN"]] but found [[2, mickey mouse, "test", "DOWN"]] [INFO] [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0 Test: JsonIngestionFromAvroQueriesTest#testJsonPathSelectOnJsonColumn Reproducible command: `mvn -pl pinot-core -Dtest=org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest#testJsonPathSelectOnJsonColumn -DreuseForks=false -DnondexRuns=1 -DnondexRunsWithoutShuffling=0 -DnondexSeed=974622 edu.illinois:nondex-maven-plugin:2.2.1:nondex` Error output: [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- WARNING: package jdk.internal.util.random not in java.base [INFO] Running org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.138 s <<< FAILURE! -- in org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest [ERROR] org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest.testJsonPathSelectOnJsonColumn -- Time elapsed: 0.464 s <<< FAILURE! java.lang.AssertionError: expected [[5, minney]] but found [[5, null]] at org.testng.Assert.fail(Assert.java:111) at org.testng.Assert.failNotEquals(Assert.java:1590) at org.testng.Assert.assertEqualsImpl(Assert.java:150) at org.testng.Assert.assertEquals(Assert.java:132) at org.testng.Assert.assertEquals(Assert.java:656) at org.testng.Assert.assertEquals(Assert.java:666) at org.apache.pinot.queries.JsonIngestionFromAvroQueriesTest.testJsonPathSelectOnJsonColumn(JsonIngestionFromAvroQueriesTest.java:316) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:141) at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:687) at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:230) at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:63) at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:995) at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:203) at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:154) at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:134) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) at org.testng.TestRunner.privateRun(TestRunner.java:741) at org.testng.TestRunner.run(TestRunner.java:616) at org.testng.SuiteRunner.runTest(SuiteRunner.java:421) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:413) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:373) at org.testng.SuiteRunner.run(SuiteRunner.java:312) at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95) at org.testng.TestNG.runSuitesSequentially(TestNG.java:1274) at org.testng.TestNG.runSuitesLocally(TestNG.java:1208) at org.testng.TestNG.runSuites(TestNG.java:1112) at org.testng.TestNG.run(TestNG.java:1079) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:155) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:102) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:91) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:137) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) [INFO] [INFO] Results: [INFO] [ERROR] Failures: [ERROR] JsonIngestionFromAvroQueriesTest.testJsonPathSelectOnJsonColumn:316 expected [[5, minney]] but found [[5, null]] [INFO] [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
