[
https://issues.apache.org/jira/browse/FLINK-3328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15976649#comment-15976649
]
Luke Hutchison commented on FLINK-3328:
---------------------------------------
What's the advantage to Flink of including all these shaded deps, rather than
just relying on something like Maven or Gradle to pull in deps? There are
numerous reasons why it is not usually a good idea to incorporate copies of
external dependencies.
> Incorrectly shaded dependencies in flink-runtime
> ------------------------------------------------
>
> Key: FLINK-3328
> URL: https://issues.apache.org/jira/browse/FLINK-3328
> Project: Flink
> Issue Type: Bug
> Components: Build System
> Affects Versions: 1.0.0
> Reporter: Stephan Ewen
> Assignee: Robert Metzger
> Priority: Blocker
> Fix For: 1.0.0
>
>
> There are apparently some dependencies shaded into {{flink-runtime}} fat jar
> that are not relocated. (the flink-runtime jar is now 70 MB)
> From the output of the shading in flink-dist, it looks as if this concerns at
> least
> - Zookeeper
> - slf4j
> - jline
> - netty (3.x)
> Possible more.
> {code}
> [WARNING] zookeeper-3.4.6.jar, flink-runtime_2.10-1.0-SNAPSHOT.jar define 440
> overlapping classes:
> [WARNING] - org.apache.zookeeper.server.NettyServerCnxnFactory
> [WARNING] - org.apache.jute.compiler.JFile
> [WARNING] - org.apache.zookeeper.server.SessionTracker$Session
> [WARNING] - org.apache.zookeeper.server.quorum.AuthFastLeaderElection$1
> [WARNING] - org.apache.jute.compiler.JLong
> [WARNING] - org.apache.zookeeper.client.ZooKeeperSaslClient$SaslState
> [WARNING] - org.apache.zookeeper.server.auth.KerberosName$Rule
> [WARNING] - org.apache.jute.CsvOutputArchive
> [WARNING] - org.apache.zookeeper.server.quorum.QuorumPeer
> [WARNING] - org.apache.zookeeper.ZooKeeper$DataWatchRegistration
> [WARNING] - 430 more...
> [WARNING] slf4j-api-1.7.7.jar, flink-runtime_2.10-1.0-SNAPSHOT.jar define 24
> overlapping classes:
> [WARNING] - org.slf4j.spi.MarkerFactoryBinder
> [WARNING] - org.slf4j.helpers.SubstituteLogger
> [WARNING] - org.slf4j.helpers.BasicMarker
> [WARNING] - org.slf4j.helpers.Util
> [WARNING] - org.slf4j.LoggerFactory
> [WARNING] - org.slf4j.Marker
> [WARNING] - org.slf4j.helpers.NamedLoggerBase
> [WARNING] - org.slf4j.Logger
> [WARNING] - org.slf4j.spi.LocationAwareLogger
> [WARNING] - org.slf4j.ILoggerFactory
> [WARNING] - 14 more...
> [WARNING] jansi-1.4.jar, jline-2.10.4.jar define 23 overlapping classes:
> [WARNING] - org.fusesource.jansi.Ansi$Erase
> [WARNING] - org.fusesource.jansi.Ansi
> [WARNING] - org.fusesource.jansi.AnsiOutputStream
> [WARNING] - org.fusesource.jansi.internal.CLibrary
> [WARNING] - org.fusesource.jansi.Ansi$2
> [WARNING] - org.fusesource.jansi.WindowsAnsiOutputStream
> [WARNING] - org.fusesource.jansi.AnsiRenderer$Code
> [WARNING] - org.fusesource.jansi.AnsiConsole
> [WARNING] - org.fusesource.jansi.Ansi$Attribute
> [WARNING] - org.fusesource.jansi.internal.Kernel32
> [WARNING] - 13 more...
> [WARNING] commons-beanutils-core-1.8.0.jar, commons-collections-3.2.2.jar,
> commons-beanutils-1.7.0.jar define 10 overlapping classes:
> [WARNING] - org.apache.commons.collections.FastHashMap$EntrySet
> [WARNING] - org.apache.commons.collections.ArrayStack
> [WARNING] - org.apache.commons.collections.FastHashMap$1
> [WARNING] - org.apache.commons.collections.FastHashMap$KeySet
> [WARNING] - org.apache.commons.collections.FastHashMap$CollectionView
> [WARNING] - org.apache.commons.collections.BufferUnderflowException
> [WARNING] - org.apache.commons.collections.Buffer
> [WARNING] -
> org.apache.commons.collections.FastHashMap$CollectionView$CollectionViewIterator
> [WARNING] - org.apache.commons.collections.FastHashMap$Values
> [WARNING] - org.apache.commons.collections.FastHashMap
> [WARNING] flink-streaming-scala_2.10-1.0-SNAPSHOT.jar,
> flink-core-1.0-SNAPSHOT.jar, flink-runtime_2.10-1.0-SNAPSHOT.jar,
> flink-java-1.0-SNAPSHOT.jar, flink-streaming-java_2.10-1.0-SNAPSHOT.jar,
> flink-scala_2.10-1.0-SNAPSHOT.jar, flink-clients_2.10-1.0-SNAPSHOT.jar,
> flink-optimizer_2.10-1.0-SNAPSHOT.jar,
> flink-runtime-web_2.10-1.0-SNAPSHOT.jar define 1690 overlapping classes:
> [WARNING] -
> org.apache.flink.shaded.com.google.common.collect.LinkedListMultimap
> [WARNING] -
> org.apache.flink.shaded.com.google.common.io.ByteSource$AsCharSource
> [WARNING] - org.apache.flink.shaded.com.google.common.escape.Platform
> [WARNING] -
> org.apache.flink.shaded.com.google.common.util.concurrent.Futures$ImmediateFailedCheckedFuture
> [WARNING] -
> org.apache.flink.shaded.com.google.common.primitives.SignedBytes$LexicographicalComparator
> [WARNING] -
> org.apache.flink.shaded.com.google.common.cache.LocalCache$WriteQueue$2
> [WARNING] - org.apache.flink.shaded.com.google.common.escape.Escaper$1
> [WARNING] -
> org.apache.flink.shaded.com.google.common.collect.MultimapBuilder$SetMultimapBuilder
> [WARNING] -
> org.apache.flink.shaded.com.google.common.collect.Ordering$ArbitraryOrdering
> [WARNING] -
> org.apache.flink.shaded.com.google.common.collect.Synchronized$SynchronizedAsMapEntries$1
> [WARNING] - 1680 more...
> [WARNING] flink-scala_2.10-1.0-SNAPSHOT.jar, flink-java-1.0-SNAPSHOT.jar,
> flink-streaming-scala_2.10-1.0-SNAPSHOT.jar,
> flink-runtime_2.10-1.0-SNAPSHOT.jar define 25 overlapping classes:
> [WARNING] - org.apache.flink.shaded.org.objectweb.asm.Context
> [WARNING] - org.apache.flink.shaded.org.objectweb.asm.FieldWriter
> [WARNING] - org.apache.flink.shaded.org.objectweb.asm.TypePath
> [WARNING] - org.apache.flink.shaded.org.objectweb.asm.Handler
> [WARNING] - org.apache.flink.shaded.org.objectweb.asm.TypeReference
> [WARNING] -
> org.apache.flink.shaded.org.objectweb.asm.signature.SignatureVisitor
> [WARNING] - org.apache.flink.shaded.org.objectweb.asm.Frame
> [WARNING] - org.apache.flink.shaded.org.objectweb.asm.FieldVisitor
> [WARNING] - org.apache.flink.shaded.org.objectweb.asm.ByteVector
> [WARNING] - org.apache.flink.shaded.org.objectweb.asm.ClassVisitor
> [WARNING] - 15 more...
> [WARNING] jline-0.9.94.jar, flink-runtime_2.10-1.0-SNAPSHOT.jar define 35
> overlapping classes:
> [WARNING] - jline.ArgumentCompletor$ArgumentList
> [WARNING] - jline.UnsupportedTerminal
> [WARNING] - jline.Terminal
> [WARNING] - jline.WindowsTerminal$ReplayPrefixOneCharInputStream
> [WARNING] - jline.History
> [WARNING] - jline.WindowsTerminal$1
> [WARNING] - jline.ConsoleReader
> [WARNING] - jline.ClassNameCompletor
> [WARNING] - jline.SimpleCompletor$SimpleCompletorFilter
> [WARNING] - jline.CandidateCycleCompletionHandler
> [WARNING] - 25 more...
> [WARNING] netty-3.8.0.Final.jar, flink-runtime_2.10-1.0-SNAPSHOT.jar define
> 810 overlapping classes:
> [WARNING] -
> org.jboss.netty.handler.codec.http.websocketx.WebSocketClientHandshaker
> [WARNING] - org.jboss.netty.handler.codec.base64.Base64Decoder
> [WARNING] - org.jboss.netty.channel.socket.nio.NioDatagramPipelineSink$1
> [WARNING] - org.jboss.netty.util.VirtualExecutorService
> [WARNING] - org.jboss.netty.util.DefaultObjectSizeEstimator
> [WARNING] -
> org.jboss.netty.util.internal.ConcurrentIdentityHashMap$HashEntry
> [WARNING] - org.jboss.netty.channel.socket.oio.OioDatagramChannel
> [WARNING] - org.jboss.netty.logging.InternalLoggerFactory
> [WARNING] - org.jboss.netty.handler.codec.spdy.DefaultSpdyDataFrame
> [WARNING] - org.jboss.netty.channel.LifeCycleAwareChannelHandler
> [WARNING] - 800 more...
> [WARNING] flink-java-1.0-SNAPSHOT.jar, flink-runtime_2.10-1.0-SNAPSHOT.jar
> define 141 overlapping classes:
> [WARNING] - org.apache.flink.shaded.org.objectweb.asm.tree.ClassNode
> [WARNING] -
> org.apache.flink.shaded.org.objectweb.asm.commons.JSRInlinerAdapter$Instantiation
> [WARNING] -
> org.apache.flink.shaded.org.objectweb.asm.tree.analysis.BasicValue
> [WARNING] -
> org.apache.flink.shaded.org.objectweb.asm.xml.Processor$SingleDocElement
> [WARNING] -
> org.apache.flink.shaded.org.objectweb.asm.tree.TypeAnnotationNode
> [WARNING] -
> org.apache.flink.shaded.org.objectweb.asm.commons.CodeSizeEvaluator
> [WARNING] -
> org.apache.flink.shaded.org.objectweb.asm.xml.Processor$SAXWriter
> [WARNING] - org.apache.flink.shaded.org.objectweb.asm.util.TraceFieldVisitor
> [WARNING] -
> org.apache.flink.shaded.org.objectweb.asm.xml.Processor$ContentHandlerFactory
> [WARNING] -
> org.apache.flink.shaded.org.objectweb.asm.xml.ASMContentHandler$FrameTypeRule
> [WARNING] - 131 more...
> [WARNING] commons-beanutils-core-1.8.0.jar, commons-beanutils-1.7.0.jar
> define 82 overlapping classes:
> [WARNING] - org.apache.commons.beanutils.Converter
> [WARNING] - org.apache.commons.beanutils.WrapDynaBean
> [WARNING] - org.apache.commons.beanutils.converters.IntegerConverter
> [WARNING] - org.apache.commons.beanutils.locale.LocaleBeanUtilsBean
> [WARNING] -
> org.apache.commons.beanutils.locale.converters.DoubleLocaleConverter
> [WARNING] -
> org.apache.commons.beanutils.locale.converters.DecimalLocaleConverter
> [WARNING] - org.apache.commons.beanutils.converters.ShortConverter
> [WARNING] - org.apache.commons.beanutils.converters.StringArrayConverter
> [WARNING] - org.apache.commons.beanutils.locale.LocaleConvertUtilsBean
> [WARNING] - org.apache.commons.beanutils.LazyDynaClass
> [WARNING] - 72 more...
> [WARNING] commons-lang3-3.3.2.jar, flink-runtime_2.10-1.0-SNAPSHOT.jar define
> 217 overlapping classes:
> [WARNING] - org.apache.commons.lang3.builder.DiffResult
> [WARNING] - org.apache.commons.lang3.CharRange
> [WARNING] -
> org.apache.commons.lang3.builder.ToStringStyle$ShortPrefixToStringStyle
> [WARNING] - org.apache.commons.lang3.concurrent.ConcurrentException
> [WARNING] - org.apache.commons.lang3.builder.DiffBuilder$1
> [WARNING] - org.apache.commons.lang3.builder.DiffBuilder
> [WARNING] - org.apache.commons.lang3.builder.Diff
> [WARNING] - org.apache.commons.lang3.time.FastDatePrinter$TwoDigitYearField
> [WARNING] - org.apache.commons.lang3.ObjectUtils$Null
> [WARNING] - org.apache.commons.lang3.reflect.MemberUtils
> [WARNING] - 207 more...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
