[
https://issues.apache.org/jira/browse/BEAM-11626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomo Suzuki updated BEAM-11626:
-------------------------------
Description:
As per BEAM-8911, Cassandra-all requires Guava 25.1. But other parts of Beam
dependencies (such as gcsio) may want to use newer version of Guava. Excerpt
from Kiley's finding using Linkage Checker today:
{noformat}
(com.google.guava:guava:25.1-jre)
com.google.common.util.concurrent.Uninterruptibles's method
sleepUninterruptibly(java.time.Duration) is not found;
referenced by 3 class files
com.google.cloud.hadoop.gcsio.cooplock.CoopLockOperationDao
(com.google.cloud.bigdataoss:gcsio:2.1.6)
com.google.cloud.hadoop.gcsio.cooplock.CoopLockRecordsDao
(com.google.cloud.bigdataoss:gcsio:2.1.6)
com.google.cloud.hadoop.gcsio.testing.InMemoryObjectEntry
(com.google.cloud.bigdataoss:gcsio:2.1.6)
{noformat}
Note that Guava's
[Uninterruptibles.sleepUninterruptibly(Duration)|https://guava.dev/releases/30.1-jre/api/docs/com/google/common/util/concurrent/Uninterruptibles.html#sleepUninterruptibly(java.time.Duration)]
has been introduced since 28.0.
This ticket is attempts to declare the newer version of Guava while keeping the
Guava version used by Beam's Cassandra tests.
Failures in my local tests:
org.apache.beam.sdk.io.hdfs.HadoopFileSystemRegistrarTest and other tests in
the package:
{noformat}
java.lang.NoSuchMethodError:
com.google.common.util.concurrent.Futures.addCallback(Lcom/google/common/util/concurrent/ListenableFuture;Lcom/google/common/util/concurrent/FutureCallback;)V
at
org.apache.hadoop.hdfs.server.datanode.checker.ThrottledAsyncChecker.addResultCachingCallback(ThrottledAsyncChecker.java:167)
at
org.apache.hadoop.hdfs.server.datanode.checker.ThrottledAsyncChecker.schedule(ThrottledAsyncChecker.java:156)
at
org.apache.hadoop.hdfs.server.datanode.checker.StorageLocationChecker.check(StorageLocationChecker.java:165)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2703)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2613)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1683)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:905)
at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:517)
at
org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:476)
at
org.apache.beam.sdk.io.hdfs.HadoopFileSystemRegistrarTest.setUp(HadoopFileSystemRegistrarTest.java:58)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{noformat}
The deprecated
[Futures.addCallback|https://guava.dev/releases/25.1-jre/api/docs/com/google/common/util/concurrent/Futures.html]
has been removed since 26.0.
The build (master) fails in my machine:
{noformat}
suztomo@suztomo:~/beam$ ./gradlew :sdks:java:io:cassandra:test
Configuration on demand is an incubating feature.
> Task :sdks:java:io:cassandra:test
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007fbdbc875a3f, pid=3453267, tid=0x000000000034b177
#
# JRE version: OpenJDK Runtime Environment (8.0_181) (build
1.8.0_181-google-v7-341329124-341329124)
# Java VM: OpenJDK 64-Bit Server VM (25.181-b01 mixed mode linux-amd64
compressed oops)
# Problematic frame:
# C [libc.so.6+0xbaa3f] __gettimeofday+0x7f
#
# Failed to write core dump. Core dumps have been disabled. To enable core
dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
#
/usr/local/google/home/suztomo/beam/sdks/java/io/cassandra/hs_err_pid3453267.log
#
# If you would like to submit a bug report, please visit:
# http://b.corp.google.com/createIssue?component=32554
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
{noformat}
was:
As per BEAM-8911, Cassandra-all requires Guava 25.1. But other parts of Beam
dependencies (such as gcsio) may want to use newer version of Guava. Excerpt
from Kiley's finding using Linkage Checker today:
{noformat}
(com.google.guava:guava:25.1-jre)
com.google.common.util.concurrent.Uninterruptibles's method
sleepUninterruptibly(java.time.Duration) is not found;
referenced by 3 class files
com.google.cloud.hadoop.gcsio.cooplock.CoopLockOperationDao
(com.google.cloud.bigdataoss:gcsio:2.1.6)
com.google.cloud.hadoop.gcsio.cooplock.CoopLockRecordsDao
(com.google.cloud.bigdataoss:gcsio:2.1.6)
com.google.cloud.hadoop.gcsio.testing.InMemoryObjectEntry
(com.google.cloud.bigdataoss:gcsio:2.1.6)
{noformat}
Note that Guava's
[Uninterruptibles.sleepUninterruptibly(Duration)|https://guava.dev/releases/30.1-jre/api/docs/com/google/common/util/concurrent/Uninterruptibles.html#sleepUninterruptibly(java.time.Duration)]
has been introduced since 28.0.
This ticket is attempts to declare the newer version of Guava while keeping the
Guava version used by Beam's Cassandra tests.
Failures in my local tests:
org.apache.beam.sdk.io.hdfs.HadoopFileSystemRegistrarTest and other tests in
the package:
{noformat}
java.lang.NoSuchMethodError:
com.google.common.util.concurrent.Futures.addCallback(Lcom/google/common/util/concurrent/ListenableFuture;Lcom/google/common/util/concurrent/FutureCallback;)V
at
org.apache.hadoop.hdfs.server.datanode.checker.ThrottledAsyncChecker.addResultCachingCallback(ThrottledAsyncChecker.java:167)
at
org.apache.hadoop.hdfs.server.datanode.checker.ThrottledAsyncChecker.schedule(ThrottledAsyncChecker.java:156)
at
org.apache.hadoop.hdfs.server.datanode.checker.StorageLocationChecker.check(StorageLocationChecker.java:165)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2703)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2613)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1683)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:905)
at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:517)
at
org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:476)
at
org.apache.beam.sdk.io.hdfs.HadoopFileSystemRegistrarTest.setUp(HadoopFileSystemRegistrarTest.java:58)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{noformat}
The deprecated
[Futures.addCallback|https://guava.dev/releases/25.1-jre/api/docs/com/google/common/util/concurrent/Futures.html]
has been removed since 26.0.
> Split Guava version to Cassandra and others
> -------------------------------------------
>
> Key: BEAM-11626
> URL: https://issues.apache.org/jira/browse/BEAM-11626
> Project: Beam
> Issue Type: Improvement
> Components: build-system
> Reporter: Tomo Suzuki
> Assignee: Tomo Suzuki
> Priority: P2
>
> As per BEAM-8911, Cassandra-all requires Guava 25.1. But other parts of Beam
> dependencies (such as gcsio) may want to use newer version of Guava. Excerpt
> from Kiley's finding using Linkage Checker today:
> {noformat}
> (com.google.guava:guava:25.1-jre)
> com.google.common.util.concurrent.Uninterruptibles's method
> sleepUninterruptibly(java.time.Duration) is not found;
> referenced by 3 class files
> com.google.cloud.hadoop.gcsio.cooplock.CoopLockOperationDao
> (com.google.cloud.bigdataoss:gcsio:2.1.6)
> com.google.cloud.hadoop.gcsio.cooplock.CoopLockRecordsDao
> (com.google.cloud.bigdataoss:gcsio:2.1.6)
> com.google.cloud.hadoop.gcsio.testing.InMemoryObjectEntry
> (com.google.cloud.bigdataoss:gcsio:2.1.6)
> {noformat}
> Note that Guava's
> [Uninterruptibles.sleepUninterruptibly(Duration)|https://guava.dev/releases/30.1-jre/api/docs/com/google/common/util/concurrent/Uninterruptibles.html#sleepUninterruptibly(java.time.Duration)]
> has been introduced since 28.0.
> This ticket is attempts to declare the newer version of Guava while keeping
> the Guava version used by Beam's Cassandra tests.
> Failures in my local tests:
> org.apache.beam.sdk.io.hdfs.HadoopFileSystemRegistrarTest and other tests in
> the package:
> {noformat}
> java.lang.NoSuchMethodError:
> com.google.common.util.concurrent.Futures.addCallback(Lcom/google/common/util/concurrent/ListenableFuture;Lcom/google/common/util/concurrent/FutureCallback;)V
> at
> org.apache.hadoop.hdfs.server.datanode.checker.ThrottledAsyncChecker.addResultCachingCallback(ThrottledAsyncChecker.java:167)
> at
> org.apache.hadoop.hdfs.server.datanode.checker.ThrottledAsyncChecker.schedule(ThrottledAsyncChecker.java:156)
> at
> org.apache.hadoop.hdfs.server.datanode.checker.StorageLocationChecker.check(StorageLocationChecker.java:165)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2703)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2613)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1683)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:905)
> at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:517)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:476)
> at
> org.apache.beam.sdk.io.hdfs.HadoopFileSystemRegistrarTest.setUp(HadoopFileSystemRegistrarTest.java:58)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> {noformat}
> The deprecated
> [Futures.addCallback|https://guava.dev/releases/25.1-jre/api/docs/com/google/common/util/concurrent/Futures.html]
> has been removed since 26.0.
> The build (master) fails in my machine:
> {noformat}
> suztomo@suztomo:~/beam$ ./gradlew :sdks:java:io:cassandra:test
> Configuration on demand is an incubating feature.
> > Task :sdks:java:io:cassandra:test
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x00007fbdbc875a3f, pid=3453267, tid=0x000000000034b177
> #
> # JRE version: OpenJDK Runtime Environment (8.0_181) (build
> 1.8.0_181-google-v7-341329124-341329124)
> # Java VM: OpenJDK 64-Bit Server VM (25.181-b01 mixed mode linux-amd64
> compressed oops)
> # Problematic frame:
> # C [libc.so.6+0xbaa3f] __gettimeofday+0x7f
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core
> dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> #
> /usr/local/google/home/suztomo/beam/sdks/java/io/cassandra/hs_err_pid3453267.log
> #
> # If you would like to submit a bug report, please visit:
> # http://b.corp.google.com/createIssue?component=32554
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
