If the tuples processed by the output operator are not of type String, then the recovery code may fail because it attempts to interpret the messages that were already stored as String. That's a bug in the operator. The workaround is to convert the object to String in the upstream operator and then pass the String to the Kafka output operator.
Thanks, Thomas On Thu, Oct 13, 2016 at 10:46 AM, Bandaru, Srinivas < srinivas.band...@optum.com> wrote: > Hi, > > Need some help. While running DT application with “*Kafka”* running into > issues with application. When monitoring the application, We are observing > that the *operator getting inactive* and restarts continuously. Could > you please refer the below log let us know if any configuration need to > be changed? > > > > > > 2016-10-122016-10-12 16:40:16,835 INFO > org.apache.kafka.common.utils.AppInfoParser: > Kafka version : 0.9.0.1 > > 2016-10-12 16:40:16,835 INFO org.apache.kafka.common.utils.AppInfoParser: > Kafka commitId : 23c69d62a0cabf06 > > 16:40:17,153 INFO org.apache.apex.malhar.kafka. > KafkaSinglePortExactlyOnceOutputOperator: Rebuild the partial window > after 6340695403456888950 > > 2016-10-12 16:40:18,827 ERROR com.datatorrent.stram.engine.StreamingContainer: > Operator set > [OperatorDeployInfo[id=4,name=kafkaOut1,type=GENERIC,checkpoint={57feace00000003b, > 0, 0},inputs=[OperatorDeployInfo.InputDeployInfo[portName= > inputPort,streamId=updateTopic1,sourceNodeId=3,sourcePortName=outTopic1, > locality=<null>,partitionMask=0,partitionKeys=<null>]],outputs=[]]] > stopped running due to an exception. > > java.lang.RuntimeException: Violates Exactly once. Not all the tuples > received after operator reset. > > at org.apache.apex.malhar.kafka.KafkaSinglePortExactlyOnceOutp > utOperator.endWindow(KafkaSinglePortExactlyOnceOutputOperator.java:174) > > at com.datatorrent.stram.engine.GenericNode.processEndWindow( > GenericNode.java:146) > > at com.datatorrent.stram.engine.GenericNode.run(GenericNode. > java:357) > > at com.datatorrent.stram.engine.StreamingContainer$2.run( > StreamingContainer.java:1407) > > 2016-10-12 16:40:18,838 INFO org.apache.kafka.clients.producer.KafkaProducer: > Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms. > > 2016-10-12 16:40:18,862 INFO com.datatorrent.stram.engine.StreamingContainer: > Undeploy request: [4] > > 2016-10-12 16:40:18,863 INFO com.datatorrent.stram.engine.StreamingContainer: > Undeploy complete. > > __stderr__0__stdout__0,_*container_e31_1476212051326_ > 0045_01_000087¨Oø__dt.log__204352016-10-12 16:51:42,333 INFO > com.datatorrent.stram.engine.StreamingContainer: Child starting with > classpath: ./commons-beanutils-1.8.3.jar:./apex-api-3.4.0.jar:./apex- > bufferserver-3.4.0.jar:./commons-lang3-3.1.jar:./ > httpcore-4.3.2.jar:./snappy-java-1.1.1.7.jar:./zkclient-0. > 7.jar:./jctools-core-1.1.jar:./jopt-simple-3.2.jar:./apex- > shaded-ning19-1.0.0.jar:./malhar-library-3.5.0.jar:./ > Kafka2Datatorrent-1.0-SNAPSHOT.jar:./bval-jsr303-0. > 5.jar:./httpclient-4.3.5.jar:./jackson-mapper-asl-1.9.2.jar: > ./kafka_2.10-0.9.0.1.jar:./bval-core-0.5.jar:./minlog-1. > 2.jar:./jersey-apache-client4-1.9.jar:./malhar-contrib-3.4. > 0.jar:./metrics-core-2.2.0.jar:./jackson-core-asl-1.9.2. > jar:./validation-api-1.1.0.Final.jar:./gson-2.0.jar:./ > kryo-2.24.0.jar:./netlet-1.2.1.jar:./lz4-1.2.0.jar:./ > mbassador-1.1.9.jar:./slf4j-api-1.7.5.jar:./kafka-clients- > 0.9.0.1.jar:./scala-library-2.10.5.jar:./kafka-unit-0.4.jar: > ./apex-common-3.4.0.jar:./xbean-asm5-shaded-4.3.jar:./ > jersey-client-1.9.jar:./apex-engine-4.jar:./malhar-kafka-3. > 5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop:/opt/ > mapr/hadoop/hadoop-2.7.0/share/hadoop/common/hadoop- > nfs-2.7.0-mapr-1602.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/hadoop-common-2.7.0-mapr-1602.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/hadoop- > common-2.7.0-mapr-1602-tests.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprfs- > diagnostic-tools-5.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > commons-compress-1.4.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > curator-client-2.7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/lib/maprdb-5.1.0-mapr-tests.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > jets3t-0.9.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/common/lib/jsp-api-2.1.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/common/lib/curator-framework-2.7.1.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr- > hbase-5.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/lib/hamcrest-core-1.3.jar:/opt/ > mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > htrace-core-3.1.0-incubating.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/common/lib/httpclient-4.2.5.jar:/opt/ > mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/json- > 20080701.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > common/lib/slf4j-log4j12-1.7.12.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/common/lib/zookeeper-3.4.5- > mapr-1503.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/common/lib/protobuf-java-2.5.0.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/common/lib/netty-3.6.2. > Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons- > configuration-1.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/lib/maprdb-mapreduce-5.1.0-mapr. > jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > activation-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/common/lib/jetty-util-6.1.26.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/common/lib/jackson-databind-2. > 7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > common/lib/jackson-core-2.7.1.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/common/lib/commons-io-2.4.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/common/lib/servlet-api- > 2.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > common/lib/mapr-tools-5.1.0-mapr-tests.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/common/lib/curator- > recipes-2.7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/common/lib/mapr-tools-5.1.0-mapr.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-cli- > 1.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > common/lib/jackson-mapper-asl-1.9.13.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/common/lib/hadoop-auth-2.7.0- > mapr-1602.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/common/lib/hadoop-azure-2.7.0-mapr-1602.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ojai- > 1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > common/lib/avro-1.7.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/lib/log4j-1.2.17.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/common/lib/maprdb-5.1.0-mapr. > jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > central-logging-5.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/common/lib/hadoop-aws-2.7.0-mapr-1602. > jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > jsch-0.1.42.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/common/lib/commons-math3-3.1. > 1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > snappy-java-1.0.4.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/lib/ojai-mapreduce-1.0.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/common/lib/jersey-json- > 1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > common/lib/commons-beanutils-1.7.0.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/common/lib/azure-storage-2.0. > 0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > jettison-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/common/lib/paranamer-2.3.jar:/opt/mapr/hadoop/hadoop- > 2.7.0/share/hadoop/common/lib/mapr-hbase-5.1.0-mapr-tests. > jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > commons-beanutils-core-1.8.0.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/common/lib/asm-3.2.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/common/lib/xz-1.0.jar:/opt/ > mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > hadoop-annotations-2.7.0-mapr-1602.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/common/lib/antlr4-runtime-4.5. > jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > aws-java-sdk-1.7.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/lib/httpcore-4.2.5.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons- > collections-3.2.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/lib/jaxb-api-2.2.2.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb- > shell-5.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/lib/jersey-server-1.9.jar:/opt/ > mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > jackson-xc-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/lib/jetty-6.1.26.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/common/lib/jackson-annotations-2.7.1.jar:/opt/ > mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > maprfs-5.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/lib/commons-httpclient-3.1.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr- > java-utils-5.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/common/lib/junit-4.11.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons- > codec-1.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/common/lib/commons-logging-1.1.3.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons- > digester-1.8.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/common/lib/api-util-1.0.0-M20.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/common/lib/jackson-jaxrs-1.9. > 13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > common/lib/maprdb-mapreduce-5.1.0-mapr-tests.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/common/lib/java- > xmlbuilder-0.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/common/lib/gson-2.2.4.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0- > M15.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > common/lib/jersey-core-1.9.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/opt/ > mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/guava- > 14.0.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > common/lib/xmlenc-0.52.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/opt/ > mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > jsr305-3.0.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/common/lib/commons-net-3.1.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/common/lib/slf4j-api-1.7.12. > jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > mysql-connector-java-5.1.25.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/common/lib/commons-lang-2.6.jar:/opt/ > mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/hadoop-hdfs- > nfs-2.7.0-mapr-1602.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/hdfs/hadoop-hdfs-2.7.0-mapr-1602.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/hdfs/hadoop-hdfs-2.7.0- > mapr-1602-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/xercesImpl-2. > 9.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > hdfs/lib/htrace-core-3.1.0-incubating.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/protobuf-java- > 2.5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > hdfs/lib/netty-3.6.2.Final.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/opt/ > mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons- > io-2.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > hdfs/lib/servlet-api-2.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/hdfs/lib/commons-cli-1.2.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1. > 9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > hdfs/lib/leveldbjni-all-1.8.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/xml-apis-1.3. > 04.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > hdfs/lib/asm-3.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/hdfs/lib/jersey-server-1.9.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jetty-6.1.26. > jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/ > netty-all-4.0.23.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/hdfs/lib/commons-codec-1.4.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons- > logging-1.1.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/hdfs/lib/jersey-core-1.9.jar:/opt/mapr/hadoop/hadoop- > 2.7.0/share/hadoop/hdfs/lib/guava-14.0.1.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons- > daemon-1.0.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/hdfs/lib/xmlenc-0.52.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/opt/ > mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn- > server-tests-2.7.0-mapr-1602.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.7. > 0-mapr-1602.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/yarn/hadoop-yarn-server-common-2.7.0-mapr-1602. > jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop- > yarn-applications-distributedshell-2.7.0-mapr-1602.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server- > sharedcachemanager-2.7.0-mapr-1602.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-applications- > unmanaged-am-launcher-2.7.0-mapr-1602.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn- > registry-2.7.0-mapr-1602.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/yarn/hadoop-yarn-client-2.7.0-mapr-1602.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn- > server-nodemanager-2.7.0-mapr-1602.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-web- > proxy-2.7.0-mapr-1602.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/yarn/hadoop-yarn-api-2.7.0-mapr-1602.jar:/opt/ > mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn- > common-2.7.0-mapr-1602.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.7.0- > mapr-1602.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/yarn/lib/stax-api-1.0-2.jar:/opt/mapr/hadoop/hadoop- > 2.7.0/share/hadoop/yarn/lib/javax.inject-1.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons- > compress-1.4.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/zookeeper-3.4. > 5-mapr-1503.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/yarn/lib/protobuf-java-2.5.0.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/yarn/lib/netty-3.6.2.Final. > jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/ > zookeeper-3.4.5-mapr-1503-tests.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/yarn/lib/activation-1.1.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guice-3. > 0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/ > jetty-util-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/yarn/lib/commons-io-2.4.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons- > cli-1.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > yarn/lib/jackson-mapper-asl-1.9.13.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/yarn/lib/leveldbjni-all-1.8. > jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/ > log4j-1.2.17.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey- > json-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > yarn/lib/jettison-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/yarn/lib/asm-3.2.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/yarn/lib/xz-1.0.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/yarn/lib/guice-servlet-3.0. > jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/ > jersey-guice-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/yarn/lib/eval-0.5.jar:/opt/mapr/hadoop/hadoop- > 2.7.0/share/hadoop/yarn/lib/commons-collections-3.2.2.jar: > /opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jaxb- > api-2.2.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/yarn/lib/jersey-server-1.9.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/yarn/lib/jackson-xc-1.9.13. > jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/ > jetty-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/yarn/lib/commons-codec-1.4.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/yarn/lib/commons-logging-1.1. > 3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/ > jackson-jaxrs-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/yarn/lib/jersey-core-1.9.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/yarn/lib/guava-14.0.1.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey- > client-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/yarn/lib/commons-lang-2.6.jar:. > > 2016-10-12 16:51:43,750 INFO com.datatorrent.bufferserver.storage.DiskStorage: > using /opt/mapr/tmp/hadoop-mapr/nm-local-dir/usercache/mapr/ > appcache/application_1476212051326_0045/container_ > e31_1476212051326_0045_01_000087/tmp as the basepath for spooling. > > 2016-10-12 16:51:43,755 INFO com.datatorrent.bufferserver.server.Server: > Server started listening at /0:0:0:0:0:0:0:0:55309 > > 2016-10-12 16:51:44,796 INFO com.datatorrent.stram.engine.StreamingContainer: > Deploy request: > [OperatorDeployInfo[id=4,name=kafkaOut1,type=GENERIC,checkpoint={57feace00000003b, > 0, 0},inputs=[OperatorDeployInfo.InputDeployInfo[portName= > inputPort,streamId=updateTopic1,sourceNodeId=3,sourcePortName=outTopic1, > locality=<null>,partitionMask=0,partitionKeys=<null>]],outputs=[]]] > > 2016-10-12 16:51:44,800 INFO org.apache.hadoop.conf.Configuration.deprecation: > io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum > > 2016-10-12 16:51:44,910 INFO org.apache.kafka.clients.producer.ProducerConfig: > ProducerConfig values: > > compression.type = none > > metric.reporters = [] > > metadata.max.age.ms = 300000 > > metadata.fetch.timeout.ms = 60000 > > reconnect.backoff.ms = 50 > > sasl.kerberos.ticket.renew.window.factor = 0.8 > > bootstrap.servers = [apsrd7102.uhc.com:9092] > > retry.backoff.ms = 100 > > sasl.kerberos.kinit.cmd = /usr/bin/kinit > > buffer.memory = 33554432 > > timeout.ms = 30000 > > key.serializer = class org.apache.kafka.common. > serialization.StringSerializer > > sasl.kerberos.service.name = null > > sasl.kerberos.ticket.renew.jitter = 0.05 > > ssl.keystore.type = JKS > > ssl.trustmanager.algorithm = PKIX > > block.on.buffer.full = false > > ssl.key.password = null > > max.block.ms = 60000 > > sasl.kerberos.min.time.before.relogin = 60000 > > connections.max.idle.ms = 540000 > > ssl.truststore.password = null > > max.in.flight.requests.per.connection = 5 > > metrics.num.samples = 2 > > client.id = > > ssl.endpoint.identification.algorithm = null > > ssl.protocol = TLS > > request.timeout.ms = 30000 > > ssl.provider = null > > ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] > > acks = all > > batch.size = 16384 > > ssl.keystore.location = null > > receive.buffer.bytes = 32768 > > ssl.cipher.suites = null > > ssl.truststore.type = JKS > > security.protocol = PLAINTEXT > > retries = 2 > > max.request.size = 1048576 > > value.serializer = class com.example.datatorrent. > Application$TenantSerializer > > ssl.truststore.location = null > > ssl.keystore.password = null > > ssl.keymanager.algorithm = SunX509 > > metrics.sample.window.ms = 30000 > > partitioner.class = class org.apache.kafka.clients. > producer.internals.DefaultPartitioner > > send.buffer.bytes = 131072 > > linger.ms = 1 > > > > > > > > 2016-10-122016-10-12 16:40:16,835 INFO > org.apache.kafka.common.utils.AppInfoParser: > Kafka version : 0.9.0.1 > > 2016-10-12 16:40:16,835 INFO org.apache.kafka.common.utils.AppInfoParser: > Kafka commitId : 23c69d62a0cabf06 > > 16:40:17,153 INFO org.apache.apex.malhar.kafka. > KafkaSinglePortExactlyOnceOutputOperator: Rebuild the partial window > after 6340695403456888950 > > 2016-10-12 16:40:18,827 ERROR com.datatorrent.stram.engine.StreamingContainer: > Operator set > [OperatorDeployInfo[id=4,name=kafkaOut1,type=GENERIC,checkpoint={57feace00000003b, > 0, 0},inputs=[OperatorDeployInfo.InputDeployInfo[portName= > inputPort,streamId=updateTopic1,sourceNodeId=3,sourcePortName=outTopic1, > locality=<null>,partitionMask=0,partitionKeys=<null>]],outputs=[]]] > stopped running due to an exception. > > java.lang.RuntimeException: Violates Exactly once. Not all the tuples > received after operator reset. > > at org.apache.apex.malhar.kafka.KafkaSinglePortExactlyOnceOutp > utOperator.endWindow(KafkaSinglePortExactlyOnceOutputOperator.java:174) > > at com.datatorrent.stram.engine.GenericNode.processEndWindow( > GenericNode.java:146) > > at com.datatorrent.stram.engine.GenericNode.run(GenericNode. > java:357) > > at com.datatorrent.stram.engine.StreamingContainer$2.run( > StreamingContainer.java:1407) > > 2016-10-12 16:40:18,838 INFO org.apache.kafka.clients.producer.KafkaProducer: > Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms. > > 2016-10-12 16:40:18,862 INFO com.datatorrent.stram.engine.StreamingContainer: > Undeploy request: [4] > > 2016-10-12 16:40:18,863 INFO com.datatorrent.stram.engine.StreamingContainer: > Undeploy complete. > > __stderr__0__stdout__0,_*container_e31_1476212051326_ > 0045_01_000087¨Oø__dt.log__204352016-10-12 16:51:42,333 INFO > com.datatorrent.stram.engine.StreamingContainer: Child starting with > classpath: ./commons-beanutils-1.8.3.jar:./apex-api-3.4.0.jar:./apex- > bufferserver-3.4.0.jar:./commons-lang3-3.1.jar:./ > httpcore-4.3.2.jar:./snappy-java-1.1.1.7.jar:./zkclient-0. > 7.jar:./jctools-core-1.1.jar:./jopt-simple-3.2.jar:./apex- > shaded-ning19-1.0.0.jar:./malhar-library-3.5.0.jar:./ > Kafka2Datatorrent-1.0-SNAPSHOT.jar:./bval-jsr303-0. > 5.jar:./httpclient-4.3.5.jar:./jackson-mapper-asl-1.9.2.jar: > ./kafka_2.10-0.9.0.1.jar:./bval-core-0.5.jar:./minlog-1. > 2.jar:./jersey-apache-client4-1.9.jar:./malhar-contrib-3.4. > 0.jar:./metrics-core-2.2.0.jar:./jackson-core-asl-1.9.2. > jar:./validation-api-1.1.0.Final.jar:./gson-2.0.jar:./ > kryo-2.24.0.jar:./netlet-1.2.1.jar:./lz4-1.2.0.jar:./ > mbassador-1.1.9.jar:./slf4j-api-1.7.5.jar:./kafka-clients- > 0.9.0.1.jar:./scala-library-2.10.5.jar:./kafka-unit-0.4.jar: > ./apex-common-3.4.0.jar:./xbean-asm5-shaded-4.3.jar:./ > jersey-client-1.9.jar:./apex-engine-4.jar:./malhar-kafka-3. > 5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop:/opt/ > mapr/hadoop/hadoop-2.7.0/share/hadoop/common/hadoop- > nfs-2.7.0-mapr-1602.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/hadoop-common-2.7.0-mapr-1602.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/hadoop- > common-2.7.0-mapr-1602-tests.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprfs- > diagnostic-tools-5.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > commons-compress-1.4.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > curator-client-2.7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/lib/maprdb-5.1.0-mapr-tests.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > jets3t-0.9.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/common/lib/jsp-api-2.1.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/common/lib/curator-framework-2.7.1.jar:/ > opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr- > hbase-5.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/lib/hamcrest-core-1.3.jar:/opt/ > mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > htrace-core-3.1.0-incubating.jar:/opt/mapr/hadoop/hadoop-2. > 7.0/share/hadoop/common/lib/httpclient-4.2.5.jar:/opt/ > mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/json- > 20080701.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/ > common/lib/slf4j-log4j12-1.7.12.jar:/opt/mapr/hadoop/ > hadoop-2.7.0/share/hadoop/common/lib/zookeeper-3.4.5- > mapr-1503.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/common/lib/protobuf-java-2.5.0.jar:/opt/mapr/ > hadoop/hadoop-2.7.0/share/hadoop/common/lib/netty-3.6.2. > Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons- > configuration-1.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/ > share/hadoop/common/lib/maprdb-mapreduce-5.1.0-mapr. > jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ > activation-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/ > hadoop/common/lib/jetty-util-6.1.26.jar:/opt/m > > > > Thanks, > > Srinivas > > > This e-mail, including attachments, may include confidential and/or > proprietary information, and may be used only by the person or entity > to which it is addressed. If the reader of this e-mail is not the intended > recipient or his or her authorized agent, the reader is hereby notified > that any dissemination, distribution or copying of this e-mail is > prohibited. If you have received this e-mail in error, please notify the > sender by replying to this message and delete this e-mail immediately. >