ZhanwangZhou opened a new pull request, #296:
URL: https://github.com/apache/gora/pull/296
### Description
I encountered non-deterministic behavior while running the following tests
using NonDex:
`org.apache.gora.ignite.store.TestIgniteStore`
In gora-ignite/src/main/java/org/apache/gora/ignite/utils/IgniteSQLBuilder,
method `createInsertQuery` and `fillInsertQuery` convert the entrySet of Map
object `data` into List object `list`:
```java
List<Entry<Column, Object>> list = new ArrayList<>(data.entrySet());
```
These two methods then iterate the list and pass its elements as other
methods' parameters. The conversion from entrySet of Map to List causes the
random order of List's element. This randomness further causes the
non-deterministic behavior of `put` method in
gora-ignite/src/main/java/org/apache/gora/ignite/store/IgniteStore, which calls
these two methods. All tests calling `put` method of class `IgniteStore` are
thus flaky.
Similarly, in
gora-ignite/src/main/java/org/apache/gora/ignite/store/IgnoreStoreMetadataAnalyzer,
method `getTableNames` iterates an unsorted ResultSet object `executeQuery`
and appends its elements into List of table name Strings. The order of the
table names in the list is random, but the list is then returned and compared
with a list with fixed order in line 42 of
gora-ignite/src/test/java/org/apache/gora/ignite/store/TestIgniteStore
```java
Assert.assertTrue("Ignite Store Metadata Table Names",
createAnalyzer.getTablesNames().equals(Lists.newArrayList("WEBPAGE",
"EMPLOYEE")));
```
The randomness of the list returned by `getTableNames` thus may cause
unexpected failure of TestIgniteStore.
### Steps to Reproduce
I used tool NonDex to detect the flaky tests.
NonDex: https://github.com/TestingResearchIllinois/NonDex
Run the tests with NonDex:
```
mvn -pl gora-ignite edu.illinois:nondex-maven-plugin:2.1.7:nondex
-Dtest=org.apache.gora.ignite.store.TestIgniteStore
```
The error message shows:
<details>
<summary>Click to view</summary>
```
[ERROR] Tests run: 45, Failures: 12, Errors: 17, Skipped: 0, Time elapsed:
12.609 s <<< FAILURE! - in org.apache.gora.ignite.store.TestIgniteStore
[ERROR]
igniteStoreMetadataAnalyzerTest(org.apache.gora.ignite.store.TestIgniteStore)
Time elapsed: 1.6 s <<< FAILURE!
java.lang.AssertionError: Ignite Store Metadata Table Names
at
org.apache.gora.ignite.store.TestIgniteStore.igniteStoreMetadataAnalyzerTest(TestIgniteStore.java:42)
[ERROR] testQuery(org.apache.gora.ignite.store.TestIgniteStore) Time
elapsed: 0.339 s <<< FAILURE!
java.lang.AssertionError: expected:<10> but was:<0>
[ERROR] testGet3UnionField(org.apache.gora.ignite.store.TestIgniteStore)
Time elapsed: 0.402 s <<< ERROR!
org.apache.gora.util.GoraException: org.apache.gora.util.GoraException:
java.sql.SQLException: Value conversion failed [column=SALARY,
from=java.lang.String, to=java.lang.Integer]
Caused by: org.apache.gora.util.GoraException: java.sql.SQLException: Value
conversion failed [column=SALARY, from=java.lang.String, to=java.lang.Integer]
Caused by: java.sql.SQLException: Value conversion failed [column=SALARY,
from=java.lang.String, to=java.lang.Integer]
[ERROR] testGetPartitions(org.apache.gora.ignite.store.TestIgniteStore)
Time elapsed: 0.291 s <<< FAILURE!
java.lang.AssertionError
[ERROR] testAutoCreateSchema(org.apache.gora.ignite.store.TestIgniteStore)
Time elapsed: 0.39 s <<< ERROR!
org.apache.gora.util.GoraException: org.apache.gora.util.GoraException:
java.sql.SQLException: Value conversion failed [column=SALARY,
from=java.lang.Long, to=java.lang.Integer]
Caused by: org.apache.gora.util.GoraException: java.sql.SQLException: Value
conversion failed [column=SALARY, from=java.lang.Long, to=java.lang.Integer]
Caused by: java.sql.SQLException: Value conversion failed [column=SALARY,
from=java.lang.Long, to=java.lang.Integer]
[ERROR] testBenchmarkExists(org.apache.gora.ignite.store.TestIgniteStore)
Time elapsed: 0.371 s <<< ERROR!
org.apache.gora.util.GoraException: org.apache.gora.util.GoraException:
java.sql.SQLException: Value conversion failed [column=SALARY,
from=java.lang.String, to=java.lang.Integer]
Caused by: org.apache.gora.util.GoraException: java.sql.SQLException: Value
conversion failed [column=SALARY, from=java.lang.String, to=java.lang.Integer]
Caused by: java.sql.SQLException: Value conversion failed [column=SALARY,
from=java.lang.String, to=java.lang.Integer]
[ERROR] testPutArray(org.apache.gora.ignite.store.TestIgniteStore) Time
elapsed: 0.271 s <<< ERROR!
org.apache.gora.util.GoraException: org.apache.gora.util.GoraException:
java.sql.SQLException: Value conversion failed [column=BYTEDATA,
from=java.lang.String, to=[B]
Caused by: org.apache.gora.util.GoraException: java.sql.SQLException: Value
conversion failed [column=BYTEDATA, from=java.lang.String, to=[B]
Caused by: java.sql.SQLException: Value conversion failed [column=BYTEDATA,
from=java.lang.String, to=[B]
[ERROR] testGetNested(org.apache.gora.ignite.store.TestIgniteStore) Time
elapsed: 0.485 s <<< ERROR!
org.apache.gora.util.GoraException: org.apache.gora.util.GoraException:
java.sql.SQLException: Value conversion failed [column=WEBPAGE,
from=java.lang.String, to=[B]
Caused by: org.apache.gora.util.GoraException: java.sql.SQLException: Value
conversion failed [column=WEBPAGE, from=java.lang.String, to=[B]
Caused by: java.sql.SQLException: Value conversion failed [column=WEBPAGE,
from=java.lang.String, to=[B]
[ERROR]
testQueryWebPageSingleKey(org.apache.gora.ignite.store.TestIgniteStore) Time
elapsed: 0.186 s <<< FAILURE!
java.lang.AssertionError
[ERROR]
testGetWebPageDefaultFields(org.apache.gora.ignite.store.TestIgniteStore) Time
elapsed: 0.266 s <<< ERROR!
org.apache.gora.util.GoraException: java.io.EOFException
Caused by: java.io.EOFException
[ERROR] testPutNested(org.apache.gora.ignite.store.TestIgniteStore) Time
elapsed: 0.19 s <<< ERROR!
org.apache.gora.util.GoraException: org.apache.gora.util.GoraException:
java.sql.SQLException: Value conversion failed [column=PARSEDCONTENT,
from=java.lang.String, to=[B]
Caused by: org.apache.gora.util.GoraException: java.sql.SQLException: Value
conversion failed [column=PARSEDCONTENT, from=java.lang.String, to=[B]
Caused by: java.sql.SQLException: Value conversion failed
[column=PARSEDCONTENT, from=java.lang.String, to=[B]
[ERROR] testUpdate(org.apache.gora.ignite.store.TestIgniteStore) Time
elapsed: 0.14 s <<< ERROR!
org.apache.gora.util.GoraException: org.apache.gora.util.GoraException:
java.sql.SQLException: Value conversion failed [column=DATEOFBIRTH,
from=java.lang.String, to=java.lang.Long]
Caused by: org.apache.gora.util.GoraException: java.sql.SQLException: Value
conversion failed [column=DATEOFBIRTH, from=java.lang.String, to=java.lang.Long]
Caused by: java.sql.SQLException: Value conversion failed
[column=DATEOFBIRTH, from=java.lang.String, to=java.lang.Long]
[ERROR]
testDeleteByQueryFields(org.apache.gora.ignite.store.TestIgniteStore) Time
elapsed: 0.136 s <<< FAILURE!
java.lang.AssertionError: expected:<10> but was:<0>
[ERROR] testQueryEndKey(org.apache.gora.ignite.store.TestIgniteStore) Time
elapsed: 0.296 s <<< FAILURE!
java.lang.AssertionError: expected:<1> but was:<0>
[ERROR] testGetDoubleRecursive(org.apache.gora.ignite.store.TestIgniteStore)
Time elapsed: 0.303 s <<< ERROR!
org.apache.gora.util.GoraException: org.apache.gora.util.GoraException:
java.sql.SQLException: Value conversion failed [column=SALARY,
from=java.lang.String, to=java.lang.Integer]
Caused by: org.apache.gora.util.GoraException: java.sql.SQLException: Value
conversion failed [column=SALARY, from=java.lang.String, to=java.lang.Integer]
Caused by: java.sql.SQLException: Value conversion failed [column=SALARY,
from=java.lang.String, to=java.lang.Integer]
[ERROR] testQueryKeyRange(org.apache.gora.ignite.store.TestIgniteStore)
Time elapsed: 0.121 s <<< FAILURE!
java.lang.AssertionError: expected:<1> but was:<0>
[ERROR] testExists(org.apache.gora.ignite.store.TestIgniteStore) Time
elapsed: 0.147 s <<< ERROR!
org.apache.gora.util.GoraException: org.apache.gora.util.GoraException:
java.sql.SQLException: Value conversion failed [column=SALARY,
from=java.lang.String, to=java.lang.Integer]
Caused by: org.apache.gora.util.GoraException: java.sql.SQLException: Value
conversion failed [column=SALARY, from=java.lang.String, to=java.lang.Integer]
Caused by: java.sql.SQLException: Value conversion failed [column=SALARY,
from=java.lang.String, to=java.lang.Integer]
[ERROR]
testQueryWebPageSingleKeyDefaultFields(org.apache.gora.ignite.store.TestIgniteStore)
Time elapsed: 0.111 s <<< FAILURE!
java.lang.AssertionError
[ERROR] testGet(org.apache.gora.ignite.store.TestIgniteStore) Time elapsed:
0.107 s <<< ERROR!
org.apache.gora.util.GoraException: org.apache.gora.util.GoraException:
java.sql.SQLException: Value conversion failed [column=SALARY,
from=java.lang.String, to=java.lang.Integer]
Caused by: org.apache.gora.util.GoraException: java.sql.SQLException: Value
conversion failed [column=SALARY, from=java.lang.String, to=java.lang.Integer]
Caused by: java.sql.SQLException: Value conversion failed [column=SALARY,
from=java.lang.String, to=java.lang.Integer]
[ERROR] testGetRecursive(org.apache.gora.ignite.store.TestIgniteStore) Time
elapsed: 0.556 s <<< ERROR!
org.apache.gora.util.GoraException: org.apache.gora.util.GoraException:
java.sql.SQLException: Value conversion failed [column=SALARY,
from=java.lang.String, to=java.lang.Integer]
Caused by: org.apache.gora.util.GoraException: java.sql.SQLException: Value
conversion failed [column=SALARY, from=java.lang.String, to=java.lang.Integer]
Caused by: java.sql.SQLException: Value conversion failed [column=SALARY,
from=java.lang.String, to=java.lang.Integer]
[ERROR] testGetWithFields(org.apache.gora.ignite.store.TestIgniteStore)
Time elapsed: 0.149 s <<< ERROR!
org.apache.gora.util.GoraException: org.apache.gora.util.GoraException:
java.sql.SQLException: Value conversion failed [column=WEBPAGE,
from=java.lang.String, to=[B]
Caused by: org.apache.gora.util.GoraException: java.sql.SQLException: Value
conversion failed [column=WEBPAGE, from=java.lang.String, to=[B]
Caused by: java.sql.SQLException: Value conversion failed [column=WEBPAGE,
from=java.lang.String, to=[B]
[ERROR] testObjectFieldValue(org.apache.gora.ignite.store.TestIgniteStore)
Time elapsed: 0.109 s <<< ERROR!
org.apache.gora.util.GoraException: org.apache.gora.util.GoraException:
java.sql.SQLException: Value conversion failed [column=DATEOFBIRTH,
from=java.lang.String, to=java.lang.Long]
Caused by: org.apache.gora.util.GoraException: java.sql.SQLException: Value
conversion failed [column=DATEOFBIRTH, from=java.lang.String, to=java.lang.Long]
Caused by: java.sql.SQLException: Value conversion failed
[column=DATEOFBIRTH, from=java.lang.String, to=java.lang.Long]
[ERROR] testPutMixedMaps(org.apache.gora.ignite.store.TestIgniteStore) Time
elapsed: 0.118 s <<< ERROR!
org.apache.gora.util.GoraException: org.apache.gora.util.GoraException:
java.sql.SQLException: Value conversion failed [column=OUTLINKS,
from=java.lang.String, to=[B]
Caused by: org.apache.gora.util.GoraException: java.sql.SQLException: Value
conversion failed [column=OUTLINKS, from=java.lang.String, to=[B]
Caused by: java.sql.SQLException: Value conversion failed [column=OUTLINKS,
from=java.lang.String, to=[B]
[ERROR] testGetWebPage(org.apache.gora.ignite.store.TestIgniteStore) Time
elapsed: 0.117 s <<< FAILURE!
java.lang.AssertionError
[ERROR] testPutMap(org.apache.gora.ignite.store.TestIgniteStore) Time
elapsed: 0.12 s <<< ERROR!
org.apache.gora.util.GoraException: org.apache.gora.util.GoraException:
java.sql.SQLException: Value conversion failed [column=PARSEDCONTENT,
from=java.lang.String, to=[B]
Caused by: org.apache.gora.util.GoraException: java.sql.SQLException: Value
conversion failed [column=PARSEDCONTENT, from=java.lang.String, to=[B]
Caused by: java.sql.SQLException: Value conversion failed
[column=PARSEDCONTENT, from=java.lang.String, to=[B]
[ERROR] testDelete(org.apache.gora.ignite.store.TestIgniteStore) Time
elapsed: 0.331 s <<< FAILURE!
java.lang.AssertionError: expected:<9> but was:<0>
[ERROR] testDeleteByQuery(org.apache.gora.ignite.store.TestIgniteStore)
Time elapsed: 0.269 s <<< FAILURE!
java.lang.AssertionError: expected:<10> but was:<0>
[ERROR] testQueryStartKey(org.apache.gora.ignite.store.TestIgniteStore)
Time elapsed: 0.156 s <<< FAILURE!
java.lang.AssertionError: expected:<10> but was:<0>
[ERROR] testPutBytes(org.apache.gora.ignite.store.TestIgniteStore) Time
elapsed: 0.13 s <<< ERROR!
org.apache.gora.util.GoraException: org.apache.gora.util.GoraException:
java.sql.SQLException: Value conversion failed [column=PARSEDCONTENT,
from=java.lang.String, to=[B]
Caused by: org.apache.gora.util.GoraException: java.sql.SQLException: Value
conversion failed [column=PARSEDCONTENT, from=java.lang.String, to=[B]
Caused by: java.sql.SQLException: Value conversion failed
[column=PARSEDCONTENT, from=java.lang.String, to=[B]
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestIgniteStore.igniteStoreMetadataAnalyzerTest:42 Ignite Store
Metadata Table Names
[ERROR] TestIgniteStore>DataStoreTestBase.testDelete:353 expected:<9> but
was:<0>
[ERROR] TestIgniteStore>DataStoreTestBase.testDeleteByQuery:359
expected:<10> but was:<0>
[ERROR] TestIgniteStore>DataStoreTestBase.testDeleteByQueryFields:365
expected:<10> but was:<0>
[ERROR] TestIgniteStore>DataStoreTestBase.testGetPartitions:371
[ERROR] TestIgniteStore>DataStoreTestBase.testGetWebPage:293
[ERROR] TestIgniteStore>DataStoreTestBase.testQuery:311 expected:<10> but
was:<0>
[ERROR] TestIgniteStore>DataStoreTestBase.testQueryEndKey:323 expected:<1>
but was:<0>
[ERROR] TestIgniteStore>DataStoreTestBase.testQueryKeyRange:329
expected:<1> but was:<0>
[ERROR] TestIgniteStore>DataStoreTestBase.testQueryStartKey:317
expected:<10> but was:<0>
[ERROR] TestIgniteStore>DataStoreTestBase.testQueryWebPageSingleKey:335
[ERROR]
TestIgniteStore>DataStoreTestBase.testQueryWebPageSingleKeyDefaultFields:341
[ERROR] Errors:
[ERROR] TestIgniteStore>DataStoreTestBase.testAutoCreateSchema:123 » Gora
org.apache.g...
[ERROR] TestIgniteStore>DataStoreTestBase.testBenchmarkExists:226 » Gora
org.apache.go...
[ERROR] TestIgniteStore>DataStoreTestBase.testExists:220 » Gora
org.apache.gora.util.G...
[ERROR] TestIgniteStore>DataStoreTestBase.testGet:232 » Gora
org.apache.gora.util.Gora...
[ERROR] TestIgniteStore>DataStoreTestBase.testGet3UnionField:281 » Gora
org.apache.gor...
[ERROR] TestIgniteStore>DataStoreTestBase.testGetDoubleRecursive:256 »
Gora org.apache...
[ERROR] TestIgniteStore>DataStoreTestBase.testGetNested:269 » Gora
org.apache.gora.uti...
[ERROR] TestIgniteStore>DataStoreTestBase.testGetRecursive:244 » Gora
org.apache.gora....
[ERROR] TestIgniteStore>DataStoreTestBase.testGetWebPageDefaultFields:299
» Gora java....
[ERROR] TestIgniteStore>DataStoreTestBase.testGetWithFields:287 » Gora
org.apache.gora...
[ERROR] TestIgniteStore>DataStoreTestBase.testObjectFieldValue:426 » Gora
org.apache.g...
[ERROR] TestIgniteStore>DataStoreTestBase.testPutArray:169 » Gora
org.apache.gora.util...
[ERROR] TestIgniteStore>DataStoreTestBase.testPutBytes:179 » Gora
org.apache.gora.util...
[ERROR] TestIgniteStore>DataStoreTestBase.testPutMap:189 » Gora
org.apache.gora.util.G...
[ERROR] TestIgniteStore>DataStoreTestBase.testPutMixedMaps:199 » Gora
org.apache.gora....
[ERROR] TestIgniteStore>DataStoreTestBase.testPutNested:163 » Gora
org.apache.gora.uti...
[ERROR] TestIgniteStore>DataStoreTestBase.testUpdate:205 » Gora
org.apache.gora.util.G...
[INFO]
[ERROR] Tests run: 45, Failures: 12, Errors: 17, Skipped: 0
```
</details>
### Potential Solution
To fix the non-deterministic behavior of method `createInsertQuery` and
`fillInsertQuery` of class `IgniteSQLBuilder`, we can pass a fix-ordered List
instead of a random-ordered Map as parameters into these two functions:
```java
public static String createInsertQuery(IgniteMapping mapping, List<Column>
dataKeyList)
public static void fillInsertQuery(PreparedStatement statement, List<Object>
insertData)
```
Meanwhile, in `put` method of class `IgniteStore`, we maintain two lists
storing Map object `data`'s keys and values:
```java
List<Column> dataKeyList = new ArrayList<>();
List<Object> dataValueList = new ArrayList<>();
```
Whenever we put a key-value pair into Map object `data`, we update these two
lists by adding the key and value. Therefore, the order of elements in these
two lists will be deterministic, exactly matching the order in which we put
key-value pairs into Map object `data`. We can then pass these two lists into
method `createInsertQuery` and `fillInsertQuery` and fix the non-deterministic
behavior.
To fix the non-deterministic behavior of method `getTableNames` in class
IgnoreStoreMetadataAnalyzer, we can sort the List `tab` before returning it:
```java
Collections.sort(tabs);
```
Then, the order of elements in List `tab` will be deterministic, following
alphabetical order.
Correspondingly, we need to change the list for comparison in line 42 of
`TestIgniteStore` to alphabetical order:
```java
Assert.assertTrue("Ignite Store Metadata Table Names",
createAnalyzer.getTablesNames().equals(Lists.newArrayList("EMPLOYEE",
"WEBPAGE")));
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
