Thanks for quick reply.
If I put all this code in comment the project will compile in 7 seconds.

build.log file attached to mail contains debug of  “mvn clean package 
-Pbuild-jar” command. 





On 19/02/2018, 12:29, "Stephan Ewen" <se...@apache.org> wrote:

    Hi!
    
    It looks like something is not correct with your project setup. The build
    puts countless dependencies of core flink into your application jar (like
    akka, scala, netty).
    
    The important thing is to set the core Flink dependencies to "provided"
    (they will not be in the Jar file), but keep the connectors and libraries
    in the proper scope.
    
    If you use one of the newer Flink quickstart projects, this should
    automatically happen.
    
    Best,
    Stephan
    
    
    On Sun, Feb 18, 2018 at 3:38 PM, Melekh, Gregory <gm5...@intl.att.com>
    wrote:
    
    > Hi all.
    > I have streaming job that reads from Kafka 0.10 manipulates data and write
    > to Cassandra (Tuple18)
    > Also this job has window and CustomReducer class involved to reduce data.
    > If groupedBy_windowed_stream DataStream defined with 9 fields (Tuple9)
    > compilation takes 5 seconds.
    > In current (Tuple11) setup it takes more then 16 minutes.
    >
    > The artifact is Ok, it runs and does the job…
    > The problem is only to long compilation time.
    >
    > Has someone been met with this issue?
    >
    >
    > DataStream<Tuple11<String, String, String, Integer, Integer, Integer,
    > Integer, String,Integer, Integer,Integer>> forAggregations
    >         = streamParsedWithTimestamps  // This stream is Tuple18
    >         .map(x -> new Tuple11<>(x.f6, x.f7, x.f8, x.f0, x.f1, x.f2, x.f3,
    > x.f11,x.f16,x.f17,new Integer(1)));
    >
    >
    > WindowedStream<Tuple11<String, String, String, Integer, Integer, Integer,
    > Integer, String, Integer,Integer,Integer>, Tuple, TimeWindow>
    > windowed_stream = forAggregations
    >         .keyBy(7)
    >         .window(TumblingProcessingTimeWindows.of(Time.hours(1)));
    >
    >
    >
    > DataStream<Tuple11<String, String, String, Integer, Integer, Integer,
    > Integer, String, Integer,Integer,Integer>> groupedBy_windowed_stream =
    > windowed_stream
    >      .reduce(new CustomReducer());
    >
    > …….
    >
    >
    > private static class CustomReducer
    >         implements ReduceFunction<Tuple11<String, String, String, Integer,
    > Integer, Integer, Integer, String, Integer,Integer,Integer>> {
    >     @Override
    >     public  Tuple11<String, String, String, Integer, Integer, Integer,
    > Integer, String, Integer,Integer,Integer> reduce(
    >             Tuple11<String, String, String, Integer, Integer, Integer,
    > Integer, String, Integer,Integer,Integer> v1,
    >             Tuple11<String, String, String, Integer, Integer, Integer,
    > Integer, String, Integer,Integer,Integer> v2) throws Exception {
    >
    >         v1.f8 += v2.f8;
    >         v1.f9 += v2.f9;
    >         v1.f10 += v2.f10;
    >
    >         return new Tuple11<>(v1.f0,v1.f1,v1.f2,
    > v1.f3,v1.f4,v1.f5,v1.f6,v1.f7,v1.f8,v1.f9,v1.f10);
    >     }
    >
    > }
    >
    >
    >
    >  mvn clean package -Pbuild-jar
    >
    > [INFO] Scanning for projects...
    >
    > [INFO]
    >
    > [INFO] ------------------------------------------------------------
    > ------------
    >
    > [INFO] Building analytics 1.0-SNAPSHOT
    >
    > [INFO] ------------------------------------------------------------
    > ------------
    >
    > [INFO]
    >
    > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ analytics ---
    >
    > [INFO] Deleting /Users/gm5806/activity-monitoring/analytics/target
    >
    > [INFO]
    >
    > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
    > analytics ---
    >
    > [INFO] Using 'UTF-8' encoding to copy filtered resources.
    >
    > [INFO] Copying 1 resource
    >
    > [INFO]
    >
    > [INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @
    > analytics ---
    >
    > [INFO] Changes detected - recompiling the module!
    >
    > [INFO] Compiling 12 source files to /Users/gm5806/activity-
    > monitoring/analytics/target/classes
    >
    > [INFO]
    >
    > [INFO] --- maven-resources-plugin:2.6:testResources
    > (default-testResources) @ analytics ---
    >
    > [INFO] Using 'UTF-8' encoding to copy filtered resources.
    >
    > [INFO] skip non existing resourceDirectory /Users/gm5806/activity-
    > monitoring/analytics/src/test/resources
    >
    > [INFO]
    >
    > [INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile)
    > @ analytics ---
    >
    > [INFO] No sources to compile
    >
    > [INFO]
    >
    > [INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ analytics
    > ---
    >
    > [INFO] No tests to run.
    >
    > [INFO]
    >
    > [INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ analytics ---
    >
    > [INFO] Building jar: /Users/gm5806/activity-monitoring/analytics/target/
    > analytics-1.0-SNAPSHOT.jar
    >
    > [INFO]
    >
    > [INFO] --- maven-shade-plugin:2.4.1:shade (default) @ analytics ---
    >
    > [INFO] Including org.slf4j:slf4j-api:jar:1.7.7 in the shaded jar.
    >
    > [INFO] Including org.xerial.snappy:snappy-java:jar:1.0.5 in the shaded
    > jar.
    >
    > [INFO] Including org.apache.flink:force-shading:jar:1.3.2 in the shaded
    > jar.
    >
    > [INFO] Including org.apache.flink:flink-table_2.11:jar:1.3.2 in the
    > shaded jar.
    >
    > [INFO] Including org.scala-lang:scala-library:jar:2.11.7 in the shaded
    > jar.
    >
    > [INFO] Including org.scala-lang.modules:scala-
    > parser-combinators_2.11:jar:1.0.4 in the shaded jar.
    >
    > [INFO] Including org.apache.flink:flink-cep_2.11:jar:1.3.2 in the shaded
    > jar.
    >
    > [INFO] Including 
org.apache.flink:flink-connector-kafka-0.10_2.11:jar:1.3.2
    > in the shaded jar.
    >
    > [INFO] Including org.apache.flink:flink-connector-kafka-0.9_2.11:jar:1.3.2
    > in the shaded jar.
    >
    > [INFO] Including 
org.apache.flink:flink-connector-kafka-base_2.11:jar:1.3.2
    > in the shaded jar.
    >
    > [INFO] Including org.apache.kafka:kafka-clients:jar:0.10.2.1 in the
    > shaded jar.
    >
    > [INFO] Including net.jpountz.lz4:lz4:jar:1.3.0 in the shaded jar.
    >
    > [INFO] Including org.apache.flink:flink-connector-cassandra_2.11:jar:1.3.2
    > in the shaded jar.
    >
    > [INFO] Including io.netty:netty-handler:jar:4.0.33.Final in the shaded
    > jar.
    >
    > [INFO] Including io.netty:netty-buffer:jar:4.0.33.Final in the shaded jar.
    >
    > [INFO] Including io.netty:netty-common:jar:4.0.33.Final in the shaded jar.
    >
    > [INFO] Including io.netty:netty-transport:jar:4.0.33.Final in the shaded
    > jar.
    >
    > [INFO] Including io.netty:netty-codec:jar:4.0.33.Final in the shaded jar.
    >
    > [INFO] Including io.dropwizard.metrics:metrics-core:jar:3.1.2 in the
    > shaded jar.
    >
    > [INFO] Including net.sourceforge.javacsv:javacsv:jar:2.0 in the shaded
    > jar.
    >
    > [INFO] Including org.apache.httpcomponents:httpclient:jar:4.2.6 in the
    > shaded jar.
    >
    > [INFO] Including org.apache.httpcomponents:httpcore:jar:4.2.5 in the
    > shaded jar.
    >
    > [INFO] Including commons-logging:commons-logging:jar:1.1.1 in the shaded
    > jar.
    >
    > [INFO] Including commons-codec:commons-codec:jar:1.6 in the shaded jar.
    >
    > [INFO] Including com.google.code.gson:gson:jar:2.2.2 in the shaded jar.
    >
    > [INFO] Including com.typesafe.akka:akka-actor_2.11:jar:2.5.0 in the
    > shaded jar.
    >
    > [INFO] Including com.typesafe:config:jar:1.3.1 in the shaded jar.
    >
    > [INFO] Including org.scala-lang.modules:scala-java8-compat_2.11:jar:0.7.0
    > in the shaded jar.
    >
    > [INFO] Including com.typesafe.akka:akka-http_2.11:jar:10.1.0-RC1 in the
    > shaded jar.
    >
    > [INFO] Including com.typesafe.akka:akka-http-core_2.11:jar:10.0.1 in the
    > shaded jar.
    >
    > [INFO] Including com.typesafe.akka:akka-parsing_2.11:jar:10.0.1 in the
    > shaded jar.
    >
    > [INFO] Including com.typesafe.akka:akka-stream_2.11:jar:2.4.16 in the
    > shaded jar.
    >
    > [INFO] Including org.reactivestreams:reactive-streams:jar:1.0.0 in the
    > shaded jar.
    >
    > [INFO] Including com.typesafe:ssl-config-core_2.11:jar:0.2.1 in the
    > shaded jar.
    >
    > [INFO] Replacing original artifact with shaded artifact.
    >
    > [INFO] Replacing 
/Users/gm5806/activity-monitoring/analytics/target/analytics-1.0-SNAPSHOT.jar
    > with /Users/gm5806/activity-monitoring/analytics/target/
    > analytics-1.0-SNAPSHOT-shaded.jar
    >
    > [INFO] ------------------------------------------------------------
    > ------------
    >
    > [INFO] BUILD SUCCESS
    >
    > [INFO] ------------------------------------------------------------
    > ------------
    >
    > [INFO] Total time: 16:33 min
    >
    > [INFO] Finished at: 2018-02-18T16:34:27+02:00
    >
    > [INFO] Final Memory: 48M/503M
    >
    > [INFO] ------------------------------------------------------------
    > ------------
    >
    >
    

Reply via email to