Repository: samza
Updated Branches:
refs/heads/master fd9f0802f -> 62904334b
SAMZA-1466: Flaky test: TestRocksDbKeyValueStore suite
TestRocksDbKeyValueStore tests are failing intermittently with errors like:
```
testFlush FAILED
org.apache.samza.SamzaException: Error opening RocksDB store dbStore at
location /tmp, received the following exception from RocksDB
org.rocksdb.RocksDBException: "
at
org.apache.samza.storage.kv.RocksDbKeyValueStore$.openDB(RocksDbKeyValueStore.scala:99)
at
org.apache.samza.storage.kv.TestRocksDbKeyValueStore.testFlush(TestRocksDbKeyValueStore.scala:73)
```
These happen intermittently because:
1. RocksDB throws an exception if open is called on a store that's already open.
2. A new unit test was added in PR #327 that didn't close the store.
3. Other tests in this class use the same store directory
(System.getProperty("java.io.tmpdir")) and run concurrently. Any test that runs
after the one in 2 above fails.
Even when we log the RocksDBException (fixed in pr #332), the exception message
is malformed due to a bug in RocksDB:
https://github.com/facebook/rocksdb/issues/1688. This is fixed in the latest
RocksDB version (verified with 5.8.0), so the messages should be more
meaningful after an upgrade. Fixed stack trace will say something like:
```
Caused by: org.rocksdb.RocksDBException: While lock file:
/var/folders/1b/4nqqvf4s27sby0frjr0q5t_h0004hp/T/LOCK: No locks available
at org.rocksdb.RocksDB.open(Native Method)
at org.rocksdb.RocksDB.open(RocksDB.java:231)
at
org.apache.samza.storage.kv.RocksDbKeyValueStore$.openDB(RocksDbKeyValueStore.scala:70)
```
Fixing just the test in the mean time.
Author: Prateek Maheshwari <[email protected]>
Reviewers: Jacob Maes <[email protected]>
Closes #333 from prateekm/store-test-fix
Project: http://git-wip-us.apache.org/repos/asf/samza/repo
Commit: http://git-wip-us.apache.org/repos/asf/samza/commit/62904334
Tree: http://git-wip-us.apache.org/repos/asf/samza/tree/62904334
Diff: http://git-wip-us.apache.org/repos/asf/samza/diff/62904334
Branch: refs/heads/master
Commit: 62904334b6269f9862b4139987b45bdb3f8b35dd
Parents: fd9f080
Author: Prateek Maheshwari <[email protected]>
Authored: Thu Oct 19 11:01:56 2017 -0700
Committer: Prateek Maheshwari <[email protected]>
Committed: Thu Oct 19 11:01:56 2017 -0700
----------------------------------------------------------------------
.../org/apache/samza/storage/kv/TestRocksDbKeyValueStore.scala | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/samza/blob/62904334/samza-kv-rocksdb/src/test/scala/org/apache/samza/storage/kv/TestRocksDbKeyValueStore.scala
----------------------------------------------------------------------
diff --git
a/samza-kv-rocksdb/src/test/scala/org/apache/samza/storage/kv/TestRocksDbKeyValueStore.scala
b/samza-kv-rocksdb/src/test/scala/org/apache/samza/storage/kv/TestRocksDbKeyValueStore.scala
index 6f129be..994387f 100644
---
a/samza-kv-rocksdb/src/test/scala/org/apache/samza/storage/kv/TestRocksDbKeyValueStore.scala
+++
b/samza-kv-rocksdb/src/test/scala/org/apache/samza/storage/kv/TestRocksDbKeyValueStore.scala
@@ -33,7 +33,7 @@ class TestRocksDbKeyValueStore
{
@Test
def testTTL() {
- val map = new java.util.HashMap[String, String]();
+ val map = new java.util.HashMap[String, String]()
map.put("rocksdb.ttl.ms", "1000")
val config = new MapConfig(map)
val options = new Options()
@@ -173,5 +173,7 @@ class TestRocksDbKeyValueStore
val estimateLiveDataSizeMetric =
metricsGroup.get("dbstore-rocksdb.estimate-live-data-size")
assert(estimateLiveDataSizeMetric.isInstanceOf[Gauge[String]])
+
+ rocksDB.close()
}
}
