Chris Pettitt created SAMZA-984:
-----------------------------------
Summary: Upgrade to RocksDB to 4.8.0
Key: SAMZA-984
URL: https://issues.apache.org/jira/browse/SAMZA-984
Project: Samza
Issue Type: Improvement
Reporter: Chris Pettitt
We're currently on a ~9 month old build of RocksDB.
I was experimenting with setting limits on the amount of logging done from
RocksDB with the keys:
- rocksdb.max.log.file.size.bytes
- rocksdb.keep.log.file.num
With these keys enabled I consistently get the java process to SIGABRT within
about a minute with the following backtrace:
{code}
Thread 3 Crashed:
0 libsystem_kernel.dylib 0x00007fff849cf002 __pthread_kill + 10
1 libsystem_pthread.dylib 0x00007fff96c365c5 pthread_kill + 90
2 libsystem_c.dylib 0x00007fff862446e7 abort + 129
3 libc++abi.dylib 0x00007fff911dff81 abort_message + 257
4 libc++abi.dylib 0x00007fff91205a2f
default_terminate_handler() + 243
5 libobjc.A.dylib 0x00007fff957d36c3 _objc_terminate() +
124
6 libc++abi.dylib 0x00007fff9120319e
std::__terminate(void (*)()) + 8
7 libc++abi.dylib 0x00007fff91202c12 __cxa_throw + 121
8 libc++.1.dylib 0x00007fff94e79781
std::__1::__vector_base_common<true>::__throw_out_of_range() const + 71
9 librocksdbjni213472405106490933..jnilib 0x000000012c51503f
rocksdb::DBImpl::PurgeObsoleteFiles(rocksdb::JobContext const&) + 3535
10 librocksdbjni213472405106490933..jnilib 0x000000012c516403
rocksdb::DBImpl::DeleteObsoleteFiles() + 531
{code}
Upgrading to a more recent version of RocksDB (i.e. 4.8.0) resolves this issue.
The upgrade worked with a drop in replacement.
To properly do upgrade we'd need:
- Integration testing (do we have something for this?)
- Perf testing
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
