Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/1726#discussion_r16626631
--- Diff: external/flume-sink/pom.xml ---
@@ -65,12 +66,15 @@
</exclusions>
</dependency>
<dependency>
- <groupId>org.scala-lang</groupId>
- <artifactId>scala-library</artifactId>
--- End diff --
Isn't that the problem? You can see the dependency on streaming test code,
but not streaming non-test code. But non-test streaming code is required at
runtime. Who are we to argue with the compiler?
On Sat, Aug 23, 2014 at 12:47 AM, Hari Shreedharan <[email protected]
> wrote:
> In external/flume-sink/pom.xml:
>
> > @@ -65,12 +66,15 @@
> > </exclusions>
> > </dependency>
> > <dependency>
> > - <groupId>org.scala-lang</groupId>
> > - <artifactId>scala-library</artifactId>
>
> The sink does not depend on streaming either. The streaming jar should not
> even be dependency - even for test only the streaming test jar is
required:
>
> [INFO] org.apache.spark:spark-streaming-flume-sink_2.10:jar:1.1.0-SNAPSHOT
> [INFO] +- org.apache.flume:flume-ng-sdk:jar:1.4.0:compile
> [INFO] | +- org.apache.avro:avro:jar:1.7.6:compile
> [INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
> [INFO] | | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile
> [INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
> [INFO] | | +- org.xerial.snappy:snappy-java:jar:1.1.1.3:compile
> [INFO] | | - org.apache.commons:commons-compress:jar:1.4.1:compile
> [INFO] | | - org.tukaani:xz:jar:1.0:compile
> [INFO] | - org.apache.avro:avro-ipc:jar:1.7.6:compile
> [INFO] +- org.apache.flume:flume-ng-core:jar:1.4.0:compile
> [INFO] | +- org.apache.flume:flume-ng-configuration:jar:1.4.0:compile
> [INFO] | +- org.slf4j:slf4j-api:jar:1.7.5:compile
> [INFO] | +- com.google.guava:guava:jar:14.0.1:provided
> [INFO] | +- commons-io:commons-io:jar:2.1:compile
> [INFO] | +- commons-codec:commons-codec:jar:1.5:compile
> [INFO] | +- log4j:log4j:jar:1.2.17:compile
> [INFO] | +- org.slf4j:slf4j-log4j12:jar:1.7.5:compile
> [INFO] | +- commons-cli:commons-cli:jar:1.2:compile
> [INFO] | +- commons-lang:commons-lang:jar:2.5:compile
> [INFO] | +- joda-time:joda-time:jar:2.1:compile
> [INFO] | +- org.mortbay.jetty:servlet-api:jar:2.5-20110124:compile
> [INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
> [INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile
> [INFO] | +- com.google.code.gson:gson:jar:2.2.2:compile
> [INFO] | - org.apache.mina:mina-core:jar:2.0.4:compile
> [INFO] +- org.scala-lang:scala-library:jar:2.10.4:compile
> [INFO] +- org.scalatest:scalatest_2.10:jar:2.1.5:test
> [INFO] | - org.scala-lang:scala-reflect:jar:2.10.4:test
> [INFO] -
> org.apache.spark:spark-streaming_2.10:test-jar:tests:1.1.0-SNAPSHOT:test
> [INFO] +- org.apache.spark:spark-core_2.10:jar:1.1.0-SNAPSHOT:test
> [INFO] | +- org.apache.hadoop:hadoop-client:jar:1.0.4:test
> [INFO] | | - org.apache.hadoop:hadoop-core:jar:1.0.4:test
> [INFO] | | +- xmlenc:xmlenc:jar:0.52:test
> [INFO] | | +- org.apache.commons:commons-math:jar:2.1:test
> [INFO] | | +- commons-configuration:commons-configuration:jar:1.6:test
> [INFO] | | | +- commons-collections:commons-collections:jar:3.2.1:test
> [INFO] | | | +- commons-digester:commons-digester:jar:1.8:test
> [INFO] | | | | - commons-beanutils:commons-beanutils:jar:1.7.0:test
> [INFO] | | | - commons-beanutils:commons-beanutils-core:jar:1.8.0:test
> [INFO] | | +- commons-el:commons-el:jar:1.0:test
> [INFO] | | +- hsqldb:hsqldb:jar:1.8.0.10:test
> [INFO] | | - oro:oro:jar:2.0.8:test
> [INFO] | +- net.java.dev.jets3t:jets3t:jar:0.7.1:test
> [INFO] | | - commons-httpclient:commons-httpclient:jar:3.1:test
> [INFO] | +- org.apache.curator:curator-recipes:jar:2.4.0:test
> [INFO] | | +- org.apache.curator:curator-framework:jar:2.4.0:test
> [INFO] | | | - org.apache.curator:curator-client:jar:2.4.0:test
> [INFO] | | - org.apache.zookeeper:zookeeper:jar:3.4.5:test
> [INFO] | | - jline:jline:jar:0.9.94:test
> [INFO] | +- org.eclipse.jetty:jetty-plus:jar:8.1.14.v20131031:test
> [INFO] | | +-
> org.eclipse.jetty.orbit:javax.transaction:jar:1.1.1.v201105210645:test
> [INFO] | | +- org.eclipse.jetty:jetty-webapp:jar:8.1.14.v20131031:test
> [INFO] | | | +- org.eclipse.jetty:jetty-xml:jar:8.1.14.v20131031:test
> [INFO] | | | - org.eclipse.jetty:jetty-servlet:jar:8.1.14.v20131031:test
> [INFO] | | - org.eclipse.jetty:jetty-jndi:jar:8.1.14.v20131031:test
> [INFO] | | -
> org.eclipse.jetty.orbit:javax.mail.glassfish:jar:1.4.1.v201005082020:test
> [INFO] | | -
> org.eclipse.jetty.orbit:javax.activation:jar:1.1.0.v201105071233:test
> [INFO] | +- org.eclipse.jetty:jetty-security:jar:8.1.14.v20131031:test
> [INFO] | +- org.eclipse.jetty:jetty-util:jar:8.1.14.v20131031:test
> [INFO] | +- org.apache.commons:commons-lang3:jar:3.3.2:test
> [INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:test
> [INFO] | +- org.slf4j:jul-to-slf4j:jar:1.7.5:test
> [INFO] | +- org.slf4j:jcl-over-slf4j:jar:1.7.5:test
> [INFO] | +- com.ning:compress-lzf:jar:1.0.0:test
> [INFO] | +- net.jpountz.lz4:lz4:jar:1.2.0:test
> [INFO] | +- com.twitter:chill_2.10:jar:0.3.6:test
> [INFO] | | - com.esotericsoftware.kryo:kryo:jar:2.21:test
> [INFO] | | +-
> com.esotericsoftware.reflectasm:reflectasm:jar:shaded:1.07:test
> [INFO] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:test
> [INFO] | | - org.objenesis:objenesis:jar:1.2:test
> [INFO] | +- com.twitter:chill-java:jar:0.3.6:test
> [INFO] | +- commons-net:commons-net:jar:2.2:test
> [INFO] | +-
> org.spark-project.akka:akka-remote_2.10:jar:2.2.3-shaded-protobuf:test
> [INFO] | | +-
> org.spark-project.akka:akka-actor_2.10:jar:2.2.3-shaded-protobuf:test
> [INFO] | | | - com.typesafe:config:jar:1.0.2:test
> [INFO] | | +- io.netty:netty:jar:3.6.6.Final:test
> [INFO] | | +-
> org.spark-project.protobuf:protobuf-java:jar:2.4.1-shaded:test
> [INFO] | | - org.uncommons.maths:uncommons-maths:jar:1.2.2a:test
> [INFO] | +-
> org.spark-project.akka:akka-slf4j_2.10:jar:2.2.3-shaded-protobuf:test
> [INFO] | +- org.json4s:json4s-jackson_2.10:jar:3.2.10:test
> [INFO] | | +- org.json4s:json4s-core_2.10:jar:3.2.10:test
> [INFO] | | | +- org.json4s:json4s-ast_2.10:jar:3.2.10:test
> [INFO] | | | - org.scala-lang:scalap:jar:2.10.4:test
> [INFO] | | | - org.scala-lang:scala-compiler:jar:2.10.4:test
> [INFO] | | - com.fasterxml.jackson.core:jackson-databind:jar:2.3.1:test
> [INFO] | | +-
com.fasterxml.jackson.core:jackson-annotations:jar:2.3.0:test
> [INFO] | | - com.fasterxml.jackson.core:jackson-core:jar:2.3.1:test
> [INFO] | +- colt:colt:jar:1.2.0:test
> [INFO] | | - concurrent:concurrent:jar:1.3.4:test
> [INFO] | +- org.apache.mesos:mesos:jar:shaded-protobuf:0.18.1:test
> [INFO] | +- io.netty:netty-all:jar:4.0.23.Final:test
> [INFO] | +- com.clearspring.analytics:stream:jar:2.7.0:test
> [INFO] | +- com.codahale.metrics:metrics-core:jar:3.0.0:test
> [INFO] | +- com.codahale.metrics:metrics-jvm:jar:3.0.0:test
> [INFO] | +- com.codahale.metrics:metrics-json:jar:3.0.0:test
> [INFO] | +- com.codahale.metrics:metrics-graphite:jar:3.0.0:test
> [INFO] | +- org.tachyonproject:tachyon-client:jar:0.5.0:test
> [INFO] | | - org.tachyonproject:tachyon:jar:0.5.0:test
> [INFO] | +- org.spark-project:pyrolite:jar:2.0.1:test
> [INFO] | - net.sf.py4j:py4j:jar:0.8.2.1:test
> [INFO] - org.eclipse.jetty:jetty-server:jar:8.1.14.v20131031:test
> [INFO] +-
> org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016:test
> [INFO] +- org.eclipse.jetty:jetty-continuation:jar:8.1.14.v20131031:test
> [INFO] - org.eclipse.jetty:jetty-http:jar:8.1.14.v20131031:test
> [INFO] - org.eclipse.jetty:jetty-io:jar:8.1.14.v20131031:test
>
> â
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/1726/files#r16626588>.
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
