Hi Folks,
I tried beam with spark 2 , although it compiles and runs without
errors, but no data gets passed through the pipeline. Below is a trace of a
pipeline run with spark 2.0, same code works with spark 1.6.3, any pointers
will be helpful. Thanks
*Trace:*
/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/bin/java
-Didea.launcher.port=7550 "-Didea.launcher.bin.path=/Applications/IntelliJ
IDEA 15 CE.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath
"/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/lib/tools.jar:/Users/jsundi/git/processing-beam-poc/target/classes:/Users/jsundi/.m2/repository/org/apache/beam/beam-sdks-java-core/2.1.0-SNAPSHOT/beam-sdks-java-core-2.1.0-SNAPSHOT.jar:/Users/jsundi/.m2/repository/com/google/code/findbugs/jsr305/3.0.1/jsr305-3.0.1.jar:/Users/jsundi/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.8.9/jackson-core-2.8.9.jar:/Users/jsundi/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.8.9/jackson-annotations-2.8.9.jar:/Users/jsundi/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.8.9/jackson-databind-2.8.9.jar:/Users/jsundi/.m2/repository/org/apache/avro/avro/1.8.2/avro-1.8.2.jar:/Users/jsundi/.m2/repository/com/thoughtworks/paranamer/paranamer/2.7/paranamer-2.7.jar:/Users/jsundi/.m2/repository/org/tukaani/xz/1.5/xz-1.5.jar:/Users/jsundi/.m2/repository/org/xerial/snappy/snappy-java/1.1.4-M3/snappy-java-1.1.4-M3.jar:/Users/jsundi/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.0.0/beam-sdks-java-io-google-cloud-platform-2.0.0.jar:/Users/jsundi/.m2/repository/org/apache/beam/beam-sdks-java-extensions-google-cloud-platform-core/2.0.0/beam-sdks-java-extensions-google-cloud-platform-core-2.0.0.jar:/Users/jsundi/.m2/repository/com/google/cloud/bigdataoss/gcsio/1.4.5/gcsio-1.4.5.jar:/Users/jsundi/.m2/repository/com/google/apis/google-api-services-cloudresourcemanager/v1-rev6-1.22.0/google-api-services-cloudresourcemanager-v1-rev6-1.22.0.jar:/Users/jsundi/.m2/repository/com/google/apis/google-api-services-storage/v1-rev71-1.22.0/google-api-services-storage-v1-rev71-1.22.0.jar:/Users/jsundi/.m2/repository/org/apache/beam/beam-sdks-java-extensions-protobuf/2.0.0/beam-sdks-java-extensions-protobuf-2.0.0.jar:/Users/jsundi/.m2/repository/com/google/api/grpc/grpc-google-pubsub-v1/0.1.0/grpc-google-pubsub-v1-0.1.0.jar:/Users/jsundi/.m2/repository/com/google/api/grpc/grpc-google-iam-v1/0.1.0/grpc-google-iam-v1-0.1.0.jar:/Users/jsundi/.m2/repository/com/google/cloud/bigdataoss/util/1.4.5/util-1.4.5.jar:/Users/jsundi/.m2/repository/com/google/api-client/google-api-client-java6/1.20.0/google-api-client-java6-1.20.0.jar:/Users/jsundi/.m2/repository/com/google/api-client/google-api-client-jackson2/1.20.0/google-api-client-jackson2-1.20.0.jar:/Users/jsundi/.m2/repository/com/google/oauth-client/google-oauth-client-java6/1.20.0/google-oauth-client-java6-1.20.0.jar:/Users/jsundi/.m2/repository/com/google/cloud/datastore/datastore-v1-proto-client/1.4.0/datastore-v1-proto-client-1.4.0.jar:/Users/jsundi/.m2/repository/com/google/http-client/google-http-client-protobuf/1.20.0/google-http-client-protobuf-1.20.0.jar:/Users/jsundi/.m2/repository/com/google/http-client/google-http-client-jackson/1.20.0/google-http-client-jackson-1.20.0.jar:/Users/jsundi/.m2/repository/com/google/cloud/datastore/datastore-v1-protos/1.3.0/datastore-v1-protos-1.3.0.jar:/Users/jsundi/.m2/repository/io/grpc/grpc-auth/1.2.0/grpc-auth-1.2.0.jar:/Users/jsundi/.m2/repository/io/grpc/grpc-core/1.2.0/grpc-core-1.2.0.jar:/Users/jsundi/.m2/repository/com/google/errorprone/error_prone_annotations/2.0.11/error_prone_annotations-2.0.11.jar:/Users/jsundi/.m2/repository/io/grpc/grpc-context/1.2.0/grpc-context-1.2.0.jar:/Users/jsundi/.m2/repository/com/google/instrumentation/instrumentation-api/0.3.0/instrumentation-api-0.3.0.jar:/Users/jsundi/.m2/repository/io/grpc/grpc-netty/1.2.0/grpc-netty-1.2.0.jar:/Users/jsundi/.m2/repository/io/netty/netty-codec-http2/4.1.8.Final/netty-codec-http2-4.1.8.Final.jar:/Users/jsundi/.m2/repository/io/netty/netty-codec-http/4.1.8.Final/netty-codec-http-4.1.8.Final.jar:/Users/jsundi/.m2/repository/io/netty/netty-handler-proxy/4.1.8.Final/netty-handler-proxy-4.1.8.Final.jar:/Users/jsundi/.m2/repository/io/netty/netty-codec-socks/4.1.8.Final/netty-codec-socks-4.1.8.Final.jar:/Users/jsundi/.m2/repository/io/netty/netty-handler/4.1.8.Final/netty-handler-4.1.8.Final.jar:/Users/jsundi/.m2/repository/io/netty/netty-buffer/4.1.8.Final/netty-buffer-4.1.8.Final.jar:/Users/jsundi/.m2/repository/io/netty/netty-common/4.1.8.Final/netty-common-4.1.8.Final.jar:/Users/jsundi/.m2/repository/io/netty/netty-transport/4.1.8.Final/netty-transport-4.1.8.Final.jar:/Users/jsundi/.m2/repository/io/netty/netty-resolver/4.1.8.Final/netty-resolver-4.1.8.Final.jar:/Users/jsundi/.m2/repository/io/netty/netty-codec/4.1.8.Final/netty-codec-4.1.8.Final.jar:/Users/jsundi/.m2/repository/io/grpc/grpc-stub/1.2.0/grpc-stub-1.2.0.jar:/Users/jsundi/.m2/repository/io/grpc/grpc-all/1.2.0/grpc-all-1.2.0.jar:/Users/jsundi/.m2/repository/io/grpc/grpc-okhttp/1.2.0/grpc-okhttp-1.2.0.jar:/Users/jsundi/.m2/repository/com/squareup/okhttp/okhttp/2.5.0/okhttp-2.5.0.jar:/Users/jsundi/.m2/repository/com/squareup/okio/okio/1.6.0/okio-1.6.0.jar:/Users/jsundi/.m2/repository/io/grpc/grpc-protobuf/1.2.0/grpc-protobuf-1.2.0.jar:/Users/jsundi/.m2/repository/io/grpc/grpc-protobuf-lite/1.2.0/grpc-protobuf-lite-1.2.0.jar:/Users/jsundi/.m2/repository/io/grpc/grpc-protobuf-nano/1.2.0/grpc-protobuf-nano-1.2.0.jar:/Users/jsundi/.m2/repository/com/google/protobuf/nano/protobuf-javanano/3.0.0-alpha-5/protobuf-javanano-3.0.0-alpha-5.jar:/Users/jsundi/.m2/repository/com/google/cloud/bigtable/bigtable-protos/
0.9.6.2/bigtable-protos-0.9.6.2.jar:/Users/jsundi/.m2/repository/com/google/cloud/bigtable/bigtable-client-core/0.9.6.2/bigtable-client-core-0.9.6.2.jar:/Users/jsundi/.m2/repository/com/google/auth/google-auth-library-appengine/0.6.0/google-auth-library-appengine-0.6.0.jar:/Users/jsundi/.m2/repository/com/google/appengine/appengine-api-1.0-sdk/1.9.34/appengine-api-1.0-sdk-1.9.34.jar:/Users/jsundi/.m2/repository/com/google/http-client/google-http-client-jackson2/1.22.0/google-http-client-jackson2-1.22.0.jar:/Users/jsundi/.m2/repository/com/google/auth/google-auth-library-credentials/0.6.1/google-auth-library-credentials-0.6.1.jar:/Users/jsundi/.m2/repository/com/google/auth/google-auth-library-oauth2-http/0.6.1/google-auth-library-oauth2-http-0.6.1.jar:/Users/jsundi/.m2/repository/com/google/api/grpc/grpc-google-common-protos/0.1.0/grpc-google-common-protos-0.1.0.jar:/Users/jsundi/.m2/repository/com/google/protobuf/protobuf-java/3.2.0/protobuf-java-3.2.0.jar:/Users/jsundi/.m2/repository/io/netty/netty-tcnative-boringssl-static/1.1.33.Fork26/netty-tcnative-boringssl-static-1.1.33.Fork26.jar:/Users/jsundi/.m2/repository/com/google/api-client/google-api-client/1.22.0/google-api-client-1.22.0.jar:/Users/jsundi/.m2/repository/com/google/oauth-client/google-oauth-client/1.22.0/google-oauth-client-1.22.0.jar:/Users/jsundi/.m2/repository/com/google/apis/google-api-services-bigquery/v2-rev295-1.22.0/google-api-services-bigquery-v2-rev295-1.22.0.jar:/Users/jsundi/.m2/repository/com/google/http-client/google-http-client/1.22.0/google-http-client-1.22.0.jar:/Users/jsundi/.m2/repository/com/google/apis/google-api-services-pubsub/v1-rev10-1.22.0/google-api-services-pubsub-v1-rev10-1.22.0.jar:/Users/jsundi/.m2/repository/joda-time/joda-time/2.4/joda-time-2.4.jar:/Users/jsundi/.m2/repository/com/google/guava/guava/20.0/guava-20.0.jar:/Users/jsundi/.m2/repository/org/slf4j/slf4j-api/1.7.14/slf4j-api-1.7.14.jar:/Users/jsundi/.m2/repository/org/slf4j/slf4j-jdk14/1.7.14/slf4j-jdk14-1.7.14.jar:/Users/jsundi/.m2/repository/org/hamcrest/hamcrest-all/1.3/hamcrest-all-1.3.jar:/Users/jsundi/.m2/repository/junit/junit/4.12/junit-4.12.jar:/Users/jsundi/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/Users/jsundi/.m2/repository/org/apache/beam/beam-sdks-java-io-hadoop-file-system/2.0.0/beam-sdks-java-io-hadoop-file-system-2.0.0.jar:/Users/jsundi/.m2/repository/org/apache/hadoop/hadoop-common/2.7.0/hadoop-common-2.7.0.jar:/Users/jsundi/.m2/repository/org/apache/hadoop/hadoop-annotations/2.7.0/hadoop-annotations-2.7.0.jar:/Users/jsundi/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/Users/jsundi/.m2/repository/org/apache/commons/commons-math3/3.1.1/commons-math3-3.1.1.jar:/Users/jsundi/.m2/repository/xmlenc/xmlenc/0.52/xmlenc-0.52.jar:/Users/jsundi/.m2/repository/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar:/Users/jsundi/.m2/repository/commons-codec/commons-codec/1.4/commons-codec-1.4.jar:/Users/jsundi/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar:/Users/jsundi/.m2/repository/commons-net/commons-net/3.1/commons-net-3.1.jar:/Users/jsundi/.m2/repository/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar:/Users/jsundi/.m2/repository/javax/servlet/servlet-api/2.5/servlet-api-2.5.jar:/Users/jsundi/.m2/repository/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.jar:/Users/jsundi/.m2/repository/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar:/Users/jsundi/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar:/Users/jsundi/.m2/repository/com/sun/jersey/jersey-core/1.9/jersey-core-1.9.jar:/Users/jsundi/.m2/repository/com/sun/jersey/jersey-json/1.9/jersey-json-1.9.jar:/Users/jsundi/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/Users/jsundi/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar:/Users/jsundi/.m2/repository/javax/xml/bind/jaxb-api/2.2.2/jaxb-api-2.2.2.jar:/Users/jsundi/.m2/repository/javax/xml/stream/stax-api/1.0-2/stax-api-1.0-2.jar:/Users/jsundi/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar:/Users/jsundi/.m2/repository/org/codehaus/jackson/jackson-jaxrs/1.8.3/jackson-jaxrs-1.8.3.jar:/Users/jsundi/.m2/repository/org/codehaus/jackson/jackson-xc/1.8.3/jackson-xc-1.8.3.jar:/Users/jsundi/.m2/repository/com/sun/jersey/jersey-server/1.9/jersey-server-1.9.jar:/Users/jsundi/.m2/repository/asm/asm/3.1/asm-3.1.jar:/Users/jsundi/.m2/repository/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.jar:/Users/jsundi/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/Users/jsundi/.m2/repository/net/java/dev/jets3t/jets3t/0.9.0/jets3t-0.9.0.jar:/Users/jsundi/.m2/repository/com/jamesmurty/utils/java-xmlbuilder/0.4/java-xmlbuilder-0.4.jar:/Users/jsundi/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar:/Users/jsundi/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/Users/jsundi/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/Users/jsundi/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar:/Users/jsundi/.m2/repository/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar:/Users/jsundi/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar:/Users/jsundi/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar:/Users/jsundi/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar:/Users/jsundi/.m2/repository/com/google/code/gson/gson/2.2.4/gson-2.2.4.jar:/Users/jsundi/.m2/repository/org/apache/hadoop/hadoop-auth/2.7.0/hadoop-auth-2.7.0.jar:/Users/jsundi/.m2/repository/org/apache/directory/server/apacheds-kerberos-codec/2.0.0-M15/apacheds-kerberos-codec-2.0.0-M15.jar:/Users/jsundi/.m2/repository/org/apache/directory/server/apacheds-i18n/2.0.0-M15/apacheds-i18n-2.0.0-M15.jar:/Users/jsundi/.m2/repository/org/apache/directory/api/api-asn1-api/1.0.0-M20/api-asn1-api-1.0.0-M20.jar:/Users/jsundi/.m2/repository/org/apache/directory/api/api-util/1.0.0-M20/api-util-1.0.0-M20.jar:/Users/jsundi/.m2/repository/org/apache/curator/curator-framework/2.7.1/curator-framework-2.7.1.jar:/Users/jsundi/.m2/repository/com/jcraft/jsch/0.1.42/jsch-0.1.42.jar:/Users/jsundi/.m2/repository/org/apache/curator/curator-client/2.7.1/curator-client-2.7.1.jar:/Users/jsundi/.m2/repository/org/apache/curator/curator-recipes/2.7.1/curator-recipes-2.7.1.jar:/Users/jsundi/.m2/repository/org/apache/htrace/htrace-core/3.1.0-incubating/htrace-core-3.1.0-incubating.jar:/Users/jsundi/.m2/repository/org/apache/zookeeper/zookeeper/3.4.6/zookeeper-3.4.6.jar:/Users/jsundi/.m2/repository/io/netty/netty/3.7.0.Final/netty-3.7.0.Final.jar:/Users/jsundi/.m2/repository/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk/1.11.146/aws-java-sdk-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-greengrass/1.11.146/aws-java-sdk-greengrass-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-athena/1.11.146/aws-java-sdk-athena-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-marketplaceentitlement/1.11.146/aws-java-sdk-marketplaceentitlement-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-codestar/1.11.146/aws-java-sdk-codestar-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-lexmodelbuilding/1.11.146/aws-java-sdk-lexmodelbuilding-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-resourcegroupstaggingapi/1.11.146/aws-java-sdk-resourcegroupstaggingapi-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-pinpoint/1.11.146/aws-java-sdk-pinpoint-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-xray/1.11.146/aws-java-sdk-xray-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-opsworkscm/1.11.146/aws-java-sdk-opsworkscm-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-support/1.11.146/aws-java-sdk-support-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-simpledb/1.11.146/aws-java-sdk-simpledb-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-servicecatalog/1.11.146/aws-java-sdk-servicecatalog-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-servermigration/1.11.146/aws-java-sdk-servermigration-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-simpleworkflow/1.11.146/aws-java-sdk-simpleworkflow-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-storagegateway/1.11.146/aws-java-sdk-storagegateway-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-route53/1.11.146/aws-java-sdk-route53-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-importexport/1.11.146/aws-java-sdk-importexport-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-sts/1.11.146/aws-java-sdk-sts-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-sqs/1.11.146/aws-java-sdk-sqs-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-rds/1.11.146/aws-java-sdk-rds-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-redshift/1.11.146/aws-java-sdk-redshift-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-elasticbeanstalk/1.11.146/aws-java-sdk-elasticbeanstalk-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-glacier/1.11.146/aws-java-sdk-glacier-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-iam/1.11.146/aws-java-sdk-iam-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-datapipeline/1.11.146/aws-java-sdk-datapipeline-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-elasticloadbalancing/1.11.146/aws-java-sdk-elasticloadbalancing-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-elasticloadbalancingv2/1.11.146/aws-java-sdk-elasticloadbalancingv2-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-emr/1.11.146/aws-java-sdk-emr-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-elasticache/1.11.146/aws-java-sdk-elasticache-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-elastictranscoder/1.11.146/aws-java-sdk-elastictranscoder-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-ec2/1.11.146/aws-java-sdk-ec2-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-dynamodb/1.11.146/aws-java-sdk-dynamodb-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-sns/1.11.146/aws-java-sdk-sns-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-budgets/1.11.146/aws-java-sdk-budgets-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-cloudtrail/1.11.146/aws-java-sdk-cloudtrail-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-cloudwatch/1.11.146/aws-java-sdk-cloudwatch-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-logs/1.11.146/aws-java-sdk-logs-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-events/1.11.146/aws-java-sdk-events-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-cognitoidentity/1.11.146/aws-java-sdk-cognitoidentity-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-cognitosync/1.11.146/aws-java-sdk-cognitosync-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-directconnect/1.11.146/aws-java-sdk-directconnect-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-cloudformation/1.11.146/aws-java-sdk-cloudformation-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-cloudfront/1.11.146/aws-java-sdk-cloudfront-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-clouddirectory/1.11.146/aws-java-sdk-clouddirectory-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-kinesis/1.11.146/aws-java-sdk-kinesis-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-opsworks/1.11.146/aws-java-sdk-opsworks-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-ses/1.11.146/aws-java-sdk-ses-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-autoscaling/1.11.146/aws-java-sdk-autoscaling-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-cloudsearch/1.11.146/aws-java-sdk-cloudsearch-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-cloudwatchmetrics/1.11.146/aws-java-sdk-cloudwatchmetrics-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-codedeploy/1.11.146/aws-java-sdk-codedeploy-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-codepipeline/1.11.146/aws-java-sdk-codepipeline-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-kms/1.11.146/aws-java-sdk-kms-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-config/1.11.146/aws-java-sdk-config-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-lambda/1.11.146/aws-java-sdk-lambda-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-ecs/1.11.146/aws-java-sdk-ecs-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-ecr/1.11.146/aws-java-sdk-ecr-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-cloudhsm/1.11.146/aws-java-sdk-cloudhsm-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-ssm/1.11.146/aws-java-sdk-ssm-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-workspaces/1.11.146/aws-java-sdk-workspaces-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-machinelearning/1.11.146/aws-java-sdk-machinelearning-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-directory/1.11.146/aws-java-sdk-directory-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-efs/1.11.146/aws-java-sdk-efs-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-codecommit/1.11.146/aws-java-sdk-codecommit-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-devicefarm/1.11.146/aws-java-sdk-devicefarm-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-elasticsearch/1.11.146/aws-java-sdk-elasticsearch-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-waf/1.11.146/aws-java-sdk-waf-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-marketplacecommerceanalytics/1.11.146/aws-java-sdk-marketplacecommerceanalytics-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-inspector/1.11.146/aws-java-sdk-inspector-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-iot/1.11.146/aws-java-sdk-iot-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-api-gateway/1.11.146/aws-java-sdk-api-gateway-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-acm/1.11.146/aws-java-sdk-acm-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-gamelift/1.11.146/aws-java-sdk-gamelift-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-dms/1.11.146/aws-java-sdk-dms-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-marketplacemeteringservice/1.11.146/aws-java-sdk-marketplacemeteringservice-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-cognitoidp/1.11.146/aws-java-sdk-cognitoidp-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-discovery/1.11.146/aws-java-sdk-discovery-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-applicationautoscaling/1.11.146/aws-java-sdk-applicationautoscaling-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-snowball/1.11.146/aws-java-sdk-snowball-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-rekognition/1.11.146/aws-java-sdk-rekognition-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-polly/1.11.146/aws-java-sdk-polly-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-lightsail/1.11.146/aws-java-sdk-lightsail-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-stepfunctions/1.11.146/aws-java-sdk-stepfunctions-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-health/1.11.146/aws-java-sdk-health-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-costandusagereport/1.11.146/aws-java-sdk-costandusagereport-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-codebuild/1.11.146/aws-java-sdk-codebuild-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-appstream/1.11.146/aws-java-sdk-appstream-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-shield/1.11.146/aws-java-sdk-shield-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-batch/1.11.146/aws-java-sdk-batch-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-lex/1.11.146/aws-java-sdk-lex-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-mechanicalturkrequester/1.11.146/aws-java-sdk-mechanicalturkrequester-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-organizations/1.11.146/aws-java-sdk-organizations-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-workdocs/1.11.146/aws-java-sdk-workdocs-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-models/1.11.146/aws-java-sdk-models-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-swf-libraries/1.11.22/aws-java-sdk-swf-libraries-1.11.22.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-s3/1.11.146/aws-java-sdk-s3-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/jmespath-java/1.11.146/jmespath-java-1.11.146.jar:/Users/jsundi/.m2/repository/com/amazonaws/aws-java-sdk-core/1.11.146/aws-java-sdk-core-1.11.146.jar:/Users/jsundi/.m2/repository/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.jar:/Users/jsundi/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.6/jackson-dataformat-cbor-2.6.6.jar:/Users/jsundi/.m2/repository/org/apache/httpcomponents/httpclient/4.5.2/httpclient-4.5.2.jar:/Users/jsundi/.m2/repository/org/apache/httpcomponents/httpcore/4.4.4/httpcore-4.4.4.jar:/Users/jsundi/.m2/repository/org/apache/beam/beam-runners-spark/2.1.0-SNAPSHOT/beam-runners-spark-2.1.0-SNAPSHOT.jar:/Users/jsundi/.m2/repository/org/apache/commons/commons-text/1.1/commons-text-1.1.jar:/Users/jsundi/.m2/repository/org/apache/commons/commons-lang3/3.5/commons-lang3-3.5.jar:/Users/jsundi/.m2/repository/org/apache/beam/beam-sdks-common-runner-api/2.1.0-SNAPSHOT/beam-sdks-common-runner-api-2.1.0-20170705.070224-61.jar:/Users/jsundi/.m2/repository/org/apache/beam/beam-runners-core-construction-java/2.1.0-SNAPSHOT/beam-runners-core-construction-java-2.1.0-20170705.070553-61.jar:/Users/jsundi/.m2/repository/com/google/protobuf/protobuf-java-util/3.2.0/protobuf-java-util-3.2.0.jar:/Users/jsundi/.m2/repository/org/apache/beam/beam-runners-core-java/2.1.0-SNAPSHOT/beam-runners-core-java-2.1.0-20170705.070645-61.jar:/Users/jsundi/.m2/repository/io/dropwizard/metrics/metrics-core/3.1.2/metrics-core-3.1.2.jar:/Users/jsundi/.m2/repository/org/apache/spark/spark-streaming_2.10/2.0.1/spark-streaming_2.10-2.0.1.jar:/Users/jsundi/.m2/repository/org/apache/spark/spark-core_2.10/2.0.1/spark-core_2.10-2.0.1.jar:/Users/jsundi/.m2/repository/org/apache/avro/avro-mapred/1.7.7/avro-mapred-1.7.7-hadoop2.jar:/Users/jsundi/.m2/repository/org/apache/avro/avro-ipc/1.7.7/avro-ipc-1.7.7.jar:/Users/jsundi/.m2/repository/org/apache/avro/avro-ipc/1.7.7/avro-ipc-1.7.7-tests.jar:/Users/jsundi/.m2/repository/com/twitter/chill_2.10/0.8.0/chill_2.10-0.8.0.jar:/Users/jsundi/.m2/repository/com/esotericsoftware/kryo-shaded/3.0.3/kryo-shaded-3.0.3.jar:/Users/jsundi/.m2/repository/com/esotericsoftware/minlog/1.3.0/minlog-1.3.0.jar:/Users/jsundi/.m2/repository/org/objenesis/objenesis/2.1/objenesis-2.1.jar:/Users/jsundi/.m2/repository/com/twitter/chill-java/0.8.0/chill-java-0.8.0.jar:/Users/jsundi/.m2/repository/org/apache/xbean/xbean-asm5-shaded/4.4/xbean-asm5-shaded-4.4.jar:/Users/jsundi/.m2/repository/org/apache/hadoop/hadoop-client/2.2.0/hadoop-client-2.2.0.jar:/Users/jsundi/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.2.0/hadoop-hdfs-2.2.0.jar:/Users/jsundi/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-app/2.2.0/hadoop-mapreduce-client-app-2.2.0.jar:/Users/jsundi/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-common/2.2.0/hadoop-mapreduce-client-common-2.2.0.jar:/Users/jsundi/.m2/repository/org/apache/hadoop/hadoop-yarn-client/2.2.0/hadoop-yarn-client-2.2.0.jar:/Users/jsundi/.m2/repository/com/google/inject/guice/3.0/guice-3.0.jar:/Users/jsundi/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar:/Users/jsundi/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar:/Users/jsundi/.m2/repository/org/apache/hadoop/hadoop-yarn-server-common/2.2.0/hadoop-yarn-server-common-2.2.0.jar:/Users/jsundi/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-shuffle/2.2.0/hadoop-mapreduce-client-shuffle-2.2.0.jar:/Users/jsundi/.m2/repository/org/apache/hadoop/hadoop-yarn-api/2.2.0/hadoop-yarn-api-2.2.0.jar:/Users/jsundi/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/2.2.0/hadoop-mapreduce-client-core-2.2.0.jar:/Users/jsundi/.m2/repository/org/apache/hadoop/hadoop-yarn-common/2.2.0/hadoop-yarn-common-2.2.0.jar:/Users/jsundi/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.2.0/hadoop-mapreduce-client-jobclient-2.2.0.jar:/Users/jsundi/.m2/repository/org/apache/spark/spark-launcher_2.10/2.0.1/spark-launcher_2.10-2.0.1.jar:/Users/jsundi/.m2/repository/org/apache/spark/spark-network-common_2.10/2.0.1/spark-network-common_2.10-2.0.1.jar:/Users/jsundi/.m2/repository/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar:/Users/jsundi/.m2/repository/org/apache/spark/spark-network-shuffle_2.10/2.0.1/spark-network-shuffle_2.10-2.0.1.jar:/Users/jsundi/.m2/repository/org/apache/spark/spark-unsafe_2.10/2.0.1/spark-unsafe_2.10-2.0.1.jar:/Users/jsundi/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar:/Users/jsundi/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.16/jcl-over-slf4j-1.7.16.jar:/Users/jsundi/.m2/repository/com/ning/compress-lzf/1.0.3/compress-lzf-1.0.3.jar:/Users/jsundi/.m2/repository/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.jar:/Users/jsundi/.m2/repository/org/roaringbitmap/RoaringBitmap/0.5.11/RoaringBitmap-0.5.11.jar:/Users/jsundi/.m2/repository/org/json4s/json4s-jackson_2.10/3.2.11/json4s-jackson_2.10-3.2.11.jar:/Users/jsundi/.m2/repository/org/json4s/json4s-core_2.10/3.2.11/json4s-core_2.10-3.2.11.jar:/Users/jsundi/.m2/repository/org/json4s/json4s-ast_2.10/3.2.11/json4s-ast_2.10-3.2.11.jar:/Users/jsundi/.m2/repository/org/scala-lang/scalap/2.10.0/scalap-2.10.0.jar:/Users/jsundi/.m2/repository/org/scala-lang/scala-compiler/2.10.0/scala-compiler-2.10.0.jar:/Users/jsundi/.m2/repository/org/glassfish/jersey/core/jersey-client/2.22.2/jersey-client-2.22.2.jar:/Users/jsundi/.m2/repository/javax/ws/rs/javax.ws.rs-api/2.0.1/javax.ws.rs-api-2.0.1.jar:/Users/jsundi/.m2/repository/org/glassfish/hk2/hk2-api/2.4.0-b34/hk2-api-2.4.0-b34.jar:/Users/jsundi/.m2/repository/org/glassfish/hk2/hk2-utils/2.4.0-b34/hk2-utils-2.4.0-b34.jar:/Users/jsundi/.m2/repository/org/glassfish/hk2/external/aopalliance-repackaged/2.4.0-b34/aopalliance-repackaged-2.4.0-b34.jar:/Users/jsundi/.m2/repository/org/glassfish/hk2/external/javax.inject/2.4.0-b34/javax.inject-2.4.0-b34.jar:/Users/jsundi/.m2/repository/org/glassfish/hk2/hk2-locator/2.4.0-b34/hk2-locator-2.4.0-b34.jar:/Users/jsundi/.m2/repository/org/javassist/javassist/3.18.1-GA/javassist-3.18.1-GA.jar:/Users/jsundi/.m2/repository/org/glassfish/jersey/core/jersey-common/2.22.2/jersey-common-2.22.2.jar:/Users/jsundi/.m2/repository/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.jar:/Users/jsundi/.m2/repository/org/glassfish/jersey/bundles/repackaged/jersey-guava/2.22.2/jersey-guava-2.22.2.jar:/Users/jsundi/.m2/repository/org/glassfish/hk2/osgi-resource-locator/1.0.1/osgi-resource-locator-1.0.1.jar:/Users/jsundi/.m2/repository/org/glassfish/jersey/core/jersey-server/2.22.2/jersey-server-2.22.2.jar:/Users/jsundi/.m2/repository/org/glassfish/jersey/media/jersey-media-jaxb/2.22.2/jersey-media-jaxb-2.22.2.jar:/Users/jsundi/.m2/repository/javax/validation/validation-api/1.1.0.Final/validation-api-1.1.0.Final.jar:/Users/jsundi/.m2/repository/org/glassfish/jersey/containers/jersey-container-servlet/2.22.2/jersey-container-servlet-2.22.2.jar:/Users/jsundi/.m2/repository/org/glassfish/jersey/containers/jersey-container-servlet-core/2.22.2/jersey-container-servlet-core-2.22.2.jar:/Users/jsundi/.m2/repository/org/apache/mesos/mesos/0.21.1/mesos-0.21.1-shaded-protobuf.jar:/Users/jsundi/.m2/repository/io/netty/netty-all/4.0.29.Final/netty-all-4.0.29.Final.jar:/Users/jsundi/.m2/repository/com/clearspring/analytics/stream/2.7.0/stream-2.7.0.jar:/Users/jsundi/.m2/repository/io/dropwizard/metrics/metrics-jvm/3.1.2/metrics-jvm-3.1.2.jar:/Users/jsundi/.m2/repository/io/dropwizard/metrics/metrics-json/3.1.2/metrics-json-3.1.2.jar:/Users/jsundi/.m2/repository/io/dropwizard/metrics/metrics-graphite/3.1.2/metrics-graphite-3.1.2.jar:/Users/jsundi/.m2/repository/org/apache/ivy/ivy/2.4.0/ivy-2.4.0.jar:/Users/jsundi/.m2/repository/oro/oro/2.0.8/oro-2.0.8.jar:/Users/jsundi/.m2/repository/net/razorvine/pyrolite/4.9/pyrolite-4.9.jar:/Users/jsundi/.m2/repository/net/sf/py4j/py4j/0.10.3/py4j-0.10.3.jar:/Users/jsundi/.m2/repository/org/apache/spark/spark-tags_2.10/2.0.1/spark-tags_2.10-2.0.1.jar:/Users/jsundi/.m2/repository/org/scalatest/scalatest_2.10/2.2.6/scalatest_2.10-2.2.6.jar:/Users/jsundi/.m2/repository/org/scala-lang/scala-library/2.10.6/scala-library-2.10.6.jar:/Users/jsundi/.m2/repository/org/spark-project/spark/unused/1.0.0/unused-1.0.0.jar:/Users/jsundi/.m2/repository/com/fasterxml/jackson/module/jackson-module-scala_2.10/2.8.8/jackson-module-scala_2.10-2.8.8.jar:/Users/jsundi/.m2/repository/org/scala-lang/scala-reflect/2.10.6/scala-reflect-2.10.6.jar:/Users/jsundi/.m2/repository/com/fasterxml/jackson/module/jackson-module-paranamer/2.8.8/jackson-module-paranamer-2.8.8.jar:/Applications/IntelliJ
IDEA 15 CE.app/Contents/lib/idea_rt.jar"
com.intellij.rt.execution.application.AppMain
com.intuit.pml.sessions.ReadWriteS3 --input=/tmp/session
--output=/Users/jsundi/Downloads/beam --runner=SparkRunner
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/Users/jsundi/.m2/repository/org/slf4j/slf4j-jdk14/1.7.14/slf4j-jdk14-1.7.14.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/Users/jsundi/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
filesystems available [file, gs]
Jul 12, 2017 3:01:51 AM org.apache.beam.runners.spark.SparkRunner run
INFO: Executing pipeline using the SparkRunner.
Jul 12, 2017 3:01:51 AM
org.apache.beam.runners.spark.translation.SparkContextFactory
createSparkContext
INFO: Creating a brand new Spark Context.
Jul 12, 2017 3:01:52 AM org.apache.spark.internal.Logging$class logInfo
INFO: Running Spark version 2.0.1
Jul 12, 2017 3:01:52 AM org.apache.hadoop.util.NativeCodeLoader <clinit>
WARNING: Unable to load native-hadoop library for your platform... using
builtin-java classes where applicable
Jul 12, 2017 3:01:53 AM org.apache.spark.internal.Logging$class logInfo
INFO: Changing view acls to: jsundi
Jul 12, 2017 3:01:53 AM org.apache.spark.internal.Logging$class logInfo
INFO: Changing modify acls to: jsundi
Jul 12, 2017 3:01:53 AM org.apache.spark.internal.Logging$class logInfo
INFO: Changing view acls groups to:
Jul 12, 2017 3:01:53 AM org.apache.spark.internal.Logging$class logInfo
INFO: Changing modify acls groups to:
Jul 12, 2017 3:01:53 AM org.apache.spark.internal.Logging$class logInfo
INFO: SecurityManager: authentication disabled; ui acls disabled; users
with view permissions: Set(jsundi); groups with view permissions: Set();
users with modify permissions: Set(jsundi); groups with modify
permissions: Set()
Jul 12, 2017 3:01:54 AM org.apache.spark.internal.Logging$class logInfo
INFO: Successfully started service 'sparkDriver' on port 55620.
Jul 12, 2017 3:01:54 AM org.apache.spark.internal.Logging$class logInfo
INFO: Registering MapOutputTracker
Jul 12, 2017 3:01:54 AM org.apache.spark.internal.Logging$class logInfo
INFO: Registering BlockManagerMaster
Jul 12, 2017 3:01:54 AM org.apache.spark.internal.Logging$class logInfo
INFO: Created local directory at
/private/var/folders/yj/dqkflr_x11d9dn6b4tkzpqy9t5py9t/T/blockmgr-d0968d0b-f8b1-44de-b4d5-b198da9672fd
Jul 12, 2017 3:01:54 AM org.apache.spark.internal.Logging$class logInfo
INFO: MemoryStore started with capacity 2004.6 MB
Jul 12, 2017 3:01:55 AM org.apache.spark.internal.Logging$class logInfo
INFO: Registering OutputCommitCoordinator
Jul 12, 2017 3:01:55 AM org.spark_project.jetty.util.log.Log initialized
INFO: Logging initialized @10115ms
Jul 12, 2017 3:01:55 AM org.spark_project.jetty.server.Server doStart
INFO: jetty-9.2.z-SNAPSHOT
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@30669dac{/jobs,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@629adce
{/jobs/json,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@6a282fdd
{/jobs/job,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@743c6ce4
{/jobs/job/json,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@70331432{/stages,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@3bbf9027
{/stages/json,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@10c2064a
{/stages/stage,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@70e13fa
{/stages/stage/json,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@6ff415ad
{/stages/pool,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@280d9edc
{/stages/pool/json,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@28fd3dc1
{/storage,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@5f9b6ae7
{/storage/json,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@108d55c4
{/storage/rdd,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@5432c277
{/storage/rdd/json,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@15e0fe05
{/environment,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@1128620c
{/environment/json,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@6bf13698
{/executors,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@299270eb
{/executors/json,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@3b90a30a
{/executors/threadDump,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@69fa8e76
{/executors/threadDump/json,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@bdd2027{/static,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@31f20c9f{/,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@f446158{/api,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@32f0c7f8
{/stages/stage/kill,null,AVAILABLE}
Jul 12, 2017 3:01:55 AM org.spark_project.jetty.server.AbstractConnector
doStart
INFO: Started ServerConnector@3a5c2626{HTTP/1.1}{0.0.0.0:4040}
Jul 12, 2017 3:01:55 AM org.spark_project.jetty.server.Server doStart
INFO: Started @10594ms
Jul 12, 2017 3:01:55 AM org.apache.spark.internal.Logging$class logInfo
INFO: Successfully started service 'SparkUI' on port 4040.
Jul 12, 2017 3:01:55 AM org.apache.spark.internal.Logging$class logInfo
INFO: Bound SparkUI to 0.0.0.0, and started at http://10.0.0.139:4040
Jul 12, 2017 3:01:56 AM org.apache.spark.internal.Logging$class logInfo
INFO: Starting executor ID driver on host localhost
Jul 12, 2017 3:01:56 AM org.apache.spark.internal.Logging$class logInfo
INFO: Successfully started service
'org.apache.spark.network.netty.NettyBlockTransferService' on port 55621.
Jul 12, 2017 3:01:56 AM org.apache.spark.internal.Logging$class logInfo
INFO: Server created on 10.0.0.139:55621
Jul 12, 2017 3:01:56 AM org.apache.spark.internal.Logging$class logInfo
INFO: Registering BlockManager BlockManagerId(driver, 10.0.0.139, 55621)
Jul 12, 2017 3:01:56 AM org.apache.spark.internal.Logging$class logInfo
INFO: Registering block manager 10.0.0.139:55621 with 2004.6 MB RAM,
BlockManagerId(driver, 10.0.0.139, 55621)
Jul 12, 2017 3:01:56 AM org.apache.spark.internal.Logging$class logInfo
INFO: Registered BlockManager BlockManagerId(driver, 10.0.0.139, 55621)
Jul 12, 2017 3:01:56 AM
org.spark_project.jetty.server.handler.ContextHandler doStart
INFO: Started o.s.j.s.ServletContextHandler@7942a854
{/metrics/json,null,AVAILABLE}
Jul 12, 2017 3:01:57 AM org.apache.beam.runners.spark.SparkRunner$Evaluator
enterCompositeTransform
INFO: Entering directly-translatable composite transform:
'TextIO.Write/WriteFiles/Create.Values'
Jul 12, 2017 3:01:57 AM
org.apache.beam.runners.spark.metrics.MetricsAccumulator init
INFO: Instantiated metrics accumulator:
org.apache.beam.runners.core.metrics.MetricsContainerStepMap@54e02f6a
Jul 12, 2017 3:01:57 AM
org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator init
INFO: Instantiated aggregators accumulator:
Jul 12, 2017 3:01:57 AM org.apache.beam.runners.spark.SparkRunner$Evaluator
doVisitTransform
INFO: Evaluating Read(CompressedSource)
Jul 12, 2017 3:01:57 AM org.apache.beam.runners.spark.SparkRunner$Evaluator
doVisitTransform
INFO: Evaluating Window.Assign
Jul 12, 2017 3:01:57 AM org.apache.beam.runners.spark.SparkRunner$Evaluator
doVisitTransform
INFO: Evaluating ParMultiDo(WriteUnwindowedBundles)
Jul 12, 2017 3:01:57 AM org.spark_project.jetty.server.AbstractConnector
doStop
INFO: Stopped ServerConnector@3a5c2626{HTTP/1.1}{0.0.0.0:4040}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@32f0c7f8
{/stages/stage/kill,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@f446158{/api,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@31f20c9f{/,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@bdd2027
{/static,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@69fa8e76
{/executors/threadDump/json,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@3b90a30a
{/executors/threadDump,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@299270eb
{/executors/json,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@6bf13698
{/executors,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@1128620c
{/environment/json,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@15e0fe05
{/environment,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@5432c277
{/storage/rdd/json,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@108d55c4
{/storage/rdd,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@5f9b6ae7
{/storage/json,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@28fd3dc1
{/storage,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@280d9edc
{/stages/pool/json,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@6ff415ad
{/stages/pool,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@70e13fa
{/stages/stage/json,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@10c2064a
{/stages/stage,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@3bbf9027
{/stages/json,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@70331432
{/stages,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@743c6ce4
{/jobs/job/json,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@6a282fdd
{/jobs/job,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@629adce
{/jobs/json,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM
org.spark_project.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.s.j.s.ServletContextHandler@30669dac{/jobs,null,UNAVAILABLE}
Jul 12, 2017 3:01:57 AM org.apache.spark.internal.Logging$class logInfo
INFO: Stopped Spark web UI at http://10.0.0.139:4040
Jul 12, 2017 3:01:58 AM org.apache.spark.internal.Logging$class logInfo
INFO: MapOutputTrackerMasterEndpoint stopped!
Jul 12, 2017 3:01:58 AM org.apache.spark.internal.Logging$class logInfo
INFO: MemoryStore cleared
Jul 12, 2017 3:01:58 AM org.apache.spark.internal.Logging$class logInfo
INFO: BlockManager stopped
Jul 12, 2017 3:01:58 AM org.apache.spark.internal.Logging$class logInfo
INFO: BlockManagerMaster stopped
Jul 12, 2017 3:01:58 AM org.apache.spark.internal.Logging$class logInfo
INFO: OutputCommitCoordinator stopped!
Jul 12, 2017 3:01:58 AM org.apache.spark.internal.Logging$class logInfo
INFO: Successfully stopped SparkContext
