[
https://issues.apache.org/jira/browse/SAMZA-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Pan updated SAMZA-2456:
----------------------------
Description:
We are currently on RocksDB 5.7.3 version which is over 2 and 1/2 years old
build. As such we should upgrade to the latest RocksDB version to include any
fixes or new features. Also there is a segment fault when trying to run RocksDB
5.7.3 with Tensorflow described below:
RocksDB version in Samza is
5.7.3<[https://jarvis.corp.linkedin.com/codesearch/result/?name=dependency-versions.gradle&path=samza-li%2Fsamza%2Fgradle&reponame=samza%2Fsamza-li]>.
However, this would cause a segment fault if the dependency order is not
properly provisioned. For example, if RockDB library is loaded before
Tensorflow library, segment fault is triggered. In contrast, if Tensorflow
library is loaded first, then there is no run-time error. Please see the
following experiments for more details.
===========experiments==============
public class RocksDBTest{
public static void main(String[] args) throws InterruptedException
{ //System.load("/home/yangzhan/libs/librocksdbjni7259590262328036607.so"); //
RocksDB v=5.0.1 => Failed //
System.load("/home/yangzhan/libs/librocksdbjni575105474288895339.so"); //
RocksDB v=5.7.3 => Failed
System.load("/home/yangzhan/libs/librocksdbjni6484631531885492614.so"); //
RockdsDB v=5.15.10 => Passed
System.load("/home/yangzhan/libs/libtensorflow_framework.so"); }
}
=========================
This issue was reported in a Tensorflow open-source
ticket<[https://github.com/tensorflow/tensorflow/issues/22307]> created by
@Shanthoosh
Venkataraman<[[email protected]|mailto:[email protected]]>.
was:We are currently on RocksDB 5.7.3 version which is over 2 and 1/2 years
old build. As such we should upgrade to the latest RocksDB version to include
any fixes or new features.
> Upgrade RocksDB to 6.5.3
> ------------------------
>
> Key: SAMZA-2456
> URL: https://issues.apache.org/jira/browse/SAMZA-2456
> Project: Samza
> Issue Type: Improvement
> Affects Versions: 1.3
> Reporter: Eric Pan
> Priority: Major
>
> We are currently on RocksDB 5.7.3 version which is over 2 and 1/2 years old
> build. As such we should upgrade to the latest RocksDB version to include any
> fixes or new features. Also there is a segment fault when trying to run
> RocksDB 5.7.3 with Tensorflow described below:
> RocksDB version in Samza is
> 5.7.3<[https://jarvis.corp.linkedin.com/codesearch/result/?name=dependency-versions.gradle&path=samza-li%2Fsamza%2Fgradle&reponame=samza%2Fsamza-li]>.
> However, this would cause a segment fault if the dependency order is not
> properly provisioned. For example, if RockDB library is loaded before
> Tensorflow library, segment fault is triggered. In contrast, if Tensorflow
> library is loaded first, then there is no run-time error. Please see the
> following experiments for more details.
> ===========experiments==============
> public class RocksDBTest{
> public static void main(String[] args) throws InterruptedException
> { //System.load("/home/yangzhan/libs/librocksdbjni7259590262328036607.so");
> // RocksDB v=5.0.1 => Failed //
> System.load("/home/yangzhan/libs/librocksdbjni575105474288895339.so"); //
> RocksDB v=5.7.3 => Failed
> System.load("/home/yangzhan/libs/librocksdbjni6484631531885492614.so"); //
> RockdsDB v=5.15.10 => Passed
> System.load("/home/yangzhan/libs/libtensorflow_framework.so"); }
> }
> =========================
> This issue was reported in a Tensorflow open-source
> ticket<[https://github.com/tensorflow/tensorflow/issues/22307]> created by
> @Shanthoosh
> Venkataraman<[[email protected]|mailto:[email protected]]>.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
