[
https://issues.apache.org/jira/browse/ATLAS-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426197#comment-15426197
]
Ayub Khan commented on ATLAS-675:
---------------------------------
Verified the storm topology submission, everything is fine and UI also looks
good. No changes needed on UI.
{noformat}
[root@atlas-resilience-testing-1 ~]# storm jar
storm-samples-1.0-jar-with-dependencies.jar
com.dsinpractice.storm.samples.WordCountTopology --cluster true --name
erie_demo_topology --path /user/storm/storm-hdfs-test --topic my-topic
Running: /usr/lib/jvm/java-1.7.0-openjdk.x86_64/bin/java -server -Ddaemon.name=
-Dstorm.options= -Dstorm.home=/grid/0/hdp/2.5.0.0-1206/storm
-Dstorm.log.dir=/grid/0/log/storm
-Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file=
-cp
/grid/0/hdp/2.5.0.0-1206/storm/lib/log4j-api-2.1.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/zookeeper.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/disruptor-3.3.2.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/clojure-1.7.0.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/storm-core-1.0.1.2.5.0.0-1206.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/log4j-over-slf4j-1.6.6.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/asm-5.0.3.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/objenesis-2.1.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/storm-rename-hack-1.0.1.2.5.0.0-1206.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/minlog-1.3.0.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/log4j-slf4j-impl-2.1.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/servlet-api-2.5.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/kryo-3.0.3.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/reflectasm-1.10.1.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/log4j-core-2.1.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/ambari-metrics-storm-sink.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/slf4j-api-1.7.7.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/ring-cors-0.1.5.jar:/grid/0/hdp/2.5.0.0-1206/storm/extlib/atlas-plugin-classloader-0.7.0.2.5.0.0-1206.jar:/grid/0/hdp/2.5.0.0-1206/storm/extlib/storm-bridge-shim-0.7.0.2.5.0.0-1206.jar
org.apache.storm.daemon.ClientJarTransformerRunner
org.apache.storm.hack.StormShadeTransformer
storm-samples-1.0-jar-with-dependencies.jar
/tmp/c35a6fe4652211e68963fa163ecea884.jar
Running: /usr/lib/jvm/java-1.7.0-openjdk.x86_64/bin/java -client -Ddaemon.name=
-Dstorm.options= -Dstorm.home=/grid/0/hdp/2.5.0.0-1206/storm
-Dstorm.log.dir=/grid/0/log/storm
-Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
-Dstorm.conf.file= -cp
/grid/0/hdp/2.5.0.0-1206/storm/lib/log4j-api-2.1.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/zookeeper.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/disruptor-3.3.2.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/clojure-1.7.0.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/storm-core-1.0.1.2.5.0.0-1206.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/log4j-over-slf4j-1.6.6.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/asm-5.0.3.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/objenesis-2.1.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/storm-rename-hack-1.0.1.2.5.0.0-1206.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/minlog-1.3.0.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/log4j-slf4j-impl-2.1.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/servlet-api-2.5.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/kryo-3.0.3.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/reflectasm-1.10.1.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/log4j-core-2.1.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/ambari-metrics-storm-sink.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/slf4j-api-1.7.7.jar:/grid/0/hdp/2.5.0.0-1206/storm/lib/ring-cors-0.1.5.jar:/grid/0/hdp/2.5.0.0-1206/storm/extlib/atlas-plugin-classloader-0.7.0.2.5.0.0-1206.jar:/grid/0/hdp/2.5.0.0-1206/storm/extlib/storm-bridge-shim-0.7.0.2.5.0.0-1206.jar:/tmp/c35a6fe4652211e68963fa163ecea884.jar:/usr/hdp/current/storm-supervisor/conf:/grid/0/hdp/2.5.0.0-1206/storm/bin
-Dstorm.jar=/tmp/c35a6fe4652211e68963fa163ecea884.jar
com.dsinpractice.storm.samples.WordCountTopology --cluster true --name
erie_demo_topology --path /user/storm/storm-hdfs-test --topic my-topic
1539 [main] INFO o.a.s.u.TupleUtils - Enabling tick tuple with interval [15]
1756 [main] INFO o.a.s.StormSubmitter - Generated ZooKeeper secret payload for
MD5-digest: -8642044523715052640:-8214783921473533571
1871 [main] INFO o.a.s.s.a.AuthUtils - Got AutoCreds []
2033 [main] INFO o.a.s.m.n.Login - successfully logged in.
30005 [main] INFO o.a.s.m.n.Login - successfully logged in.
30054 [main] INFO o.a.s.m.n.Login - successfully logged in.
30107 [main] INFO o.a.s.m.n.Login - successfully logged in.
30375 [main] INFO o.a.s.StormSubmitter - Uploading topology jar
/tmp/c35a6fe4652211e68963fa163ecea884.jar to assigned location:
/hadoop/storm/nimbus/inbox/stormjar-6463b40a-e1b4-418a-bc6b-ba3dbf306a0a.jar
31268 [main] INFO o.a.s.StormSubmitter - Successfully uploaded topology jar to
assigned location:
/hadoop/storm/nimbus/inbox/stormjar-6463b40a-e1b4-418a-bc6b-ba3dbf306a0a.jar
31268 [main] INFO o.a.s.StormSubmitter - Submitting topology
erie_demo_topology in distributed mode with conf
{"storm.zookeeper.topology.auth.scheme":"digest","storm.zookeeper.topology.auth.payload":"-8642044523715052640:-8214783921473533571","topology.workers":3,"topology.debug":true}
31282 [main] INFO o.a.s.m.n.Login - successfully logged in.
31328 [main] INFO o.a.s.m.n.Login - successfully logged in.
32186 [main] INFO o.a.s.StormSubmitter - Finished submitting topology:
erie_demo_topology
32186 [main] INFO o.a.s.StormSubmitter - Initializing the registered
ISubmitterHook [org.apache.atlas.storm.hook.StormAtlasHook]
32786 [main] INFO o.a.a.ApplicationProperties - Looking for
atlas-application.properties in classpath
32786 [main] INFO o.a.a.ApplicationProperties - Loading
atlas-application.properties from
file:/etc/storm/2.5.0.0-1206/0/atlas-application.properties
log4j:WARN No appenders could be found for logger
(org.apache.atlas.ApplicationProperties).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
34277 [main] INFO o.a.a.h.AtlasHook - Created Atlas Hook
34306 [main] INFO o.a.s.m.n.Login - successfully logged in.
34364 [main] INFO o.a.s.m.n.Login - successfully logged in.
34560 [main] INFO o.a.s.StormSubmitter - Invoking the registered
ISubmitterHook [org.apache.atlas.storm.hook.StormAtlasHook]
34561 [main] INFO o.a.a.s.h.StormAtlasHook - Collecting metadata for a new
storm topology: erie_demo_topology
35160 [main] INFO o.a.a.h.AtlasHook - Adding entity for type: kafka_topic
36470 [main] INFO o.a.a.h.AtlasHook - Adding entity for type: hdfs_path
36477 [main] INFO o.a.a.h.AtlasHook - Adding entity for type: storm_topology
36648 [main] INFO o.a.k.c.p.ProducerConfig - ProducerConfig values:
interceptor.classes = null
request.timeout.ms = 30000
ssl.truststore.password = null
retry.backoff.ms = 100
buffer.memory = 33554432
batch.size = 16384
ssl.keymanager.algorithm = SunX509
receive.buffer.bytes = 32768
ssl.key.password = null
ssl.cipher.suites = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.service.name = kafka
ssl.provider = null
max.in.flight.requests.per.connection = 5
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
bootstrap.servers = [atlas-resilience-testing-1.openstacklocal:6667,
atlas-resilience-testing-2.openstacklocal:6667]
client.id =
max.request.size = 1048576
acks = 1
linger.ms = 0
sasl.kerberos.kinit.cmd = /usr/bin/kinit
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
metadata.fetch.timeout.ms = 60000
ssl.endpoint.identification.algorithm = null
ssl.keystore.location = null
value.serializer = class
org.apache.kafka.common.serialization.StringSerializer
ssl.truststore.location = null
ssl.keystore.password = null
block.on.buffer.full = false
key.serializer = class
org.apache.kafka.common.serialization.StringSerializer
metrics.sample.window.ms = 30000
security.protocol = PLAINTEXTSASL
metadata.max.age.ms = 300000
ssl.protocol = TLS
sasl.kerberos.min.time.before.relogin = 60000
timeout.ms = 30000
connections.max.idle.ms = 540000
ssl.trustmanager.algorithm = PKIX
metric.reporters = []
ssl.truststore.type = JKS
compression.type = none
retries = 0
max.block.ms = 60000
partitioner.class = class
org.apache.kafka.clients.producer.internals.DefaultPartitioner
send.buffer.bytes = 131072
reconnect.backoff.ms = 50
metrics.num.samples = 2
ssl.keystore.type = JKS
36698 [main] INFO o.a.k.c.s.a.AbstractLogin - Successfully logged in.
36701 [kafka-kerberos-refresh-thread] INFO o.a.k.c.s.k.KerberosLogin - TGT
refresh thread started.
36703 [kafka-kerberos-refresh-thread] INFO o.a.k.c.s.k.KerberosLogin - TGT
valid starting at: 2016-08-18T09:05:37.000+0000
36704 [kafka-kerberos-refresh-thread] INFO o.a.k.c.s.k.KerberosLogin - TGT
expires: 2016-08-18T19:05:37.000+0000
36704 [kafka-kerberos-refresh-thread] INFO o.a.k.c.s.k.KerberosLogin - TGT
refresh sleeping until: 2016-08-18T17:19:01.968+0000
36727 [main] INFO o.a.k.c.p.ProducerConfig - ProducerConfig values:
interceptor.classes = null
request.timeout.ms = 30000
ssl.truststore.password = null
retry.backoff.ms = 100
buffer.memory = 33554432
batch.size = 16384
ssl.keymanager.algorithm = SunX509
receive.buffer.bytes = 32768
ssl.key.password = null
ssl.cipher.suites = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.service.name = kafka
ssl.provider = null
max.in.flight.requests.per.connection = 5
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
bootstrap.servers = [atlas-resilience-testing-1.openstacklocal:6667,
atlas-resilience-testing-2.openstacklocal:6667]
client.id = producer-1
max.request.size = 1048576
acks = 1
linger.ms = 0
sasl.kerberos.kinit.cmd = /usr/bin/kinit
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
metadata.fetch.timeout.ms = 60000
ssl.endpoint.identification.algorithm = null
ssl.keystore.location = null
value.serializer = class
org.apache.kafka.common.serialization.StringSerializer
ssl.truststore.location = null
ssl.keystore.password = null
block.on.buffer.full = false
key.serializer = class
org.apache.kafka.common.serialization.StringSerializer
metrics.sample.window.ms = 30000
security.protocol = PLAINTEXTSASL
metadata.max.age.ms = 300000
ssl.protocol = TLS
sasl.kerberos.min.time.before.relogin = 60000
timeout.ms = 30000
connections.max.idle.ms = 540000
ssl.trustmanager.algorithm = PKIX
metric.reporters = []
ssl.truststore.type = JKS
compression.type = none
retries = 0
max.block.ms = 60000
partitioner.class = class
org.apache.kafka.clients.producer.internals.DefaultPartitioner
send.buffer.bytes = 131072
reconnect.backoff.ms = 50
metrics.num.samples = 2
ssl.keystore.type = JKS
36731 [main] WARN o.a.k.c.p.ProducerConfig - The configuration
zookeeper.sync.time.ms = 20 was supplied but isn't a known config.
36731 [main] WARN o.a.k.c.p.ProducerConfig - The configuration hook.group.id =
atlas was supplied but isn't a known config.
36732 [main] WARN o.a.k.c.p.ProducerConfig - The configuration
zookeeper.connection.timeout.ms = 200 was supplied but isn't a known config.
36732 [main] WARN o.a.k.c.p.ProducerConfig - The configuration
zookeeper.session.timeout.ms = 400 was supplied but isn't a known config.
36732 [main] WARN o.a.k.c.p.ProducerConfig - The configuration
zookeeper.connect =
atlas-resilience-testing-2.openstacklocal:2181,atlas-resilience-testing-1.openstacklocal:2181
was supplied but isn't a known config.
36732 [main] WARN o.a.k.c.p.ProducerConfig - The configuration
key.deserializer = org.apache.kafka.common.serialization.StringDeserializer was
supplied but isn't a known config.
36733 [main] WARN o.a.k.c.p.ProducerConfig - The configuration
auto.offset.reset = smallest was supplied but isn't a known config.
36733 [main] WARN o.a.k.c.p.ProducerConfig - The configuration
value.deserializer = org.apache.kafka.common.serialization.StringDeserializer
was supplied but isn't a known config.
36733 [main] WARN o.a.k.c.p.ProducerConfig - The configuration
partition.assignment.strategy = roundrobin was supplied but isn't a known
config.
36736 [main] INFO o.a.k.c.u.AppInfoParser - Kafka version : 0.10.0.2.5.0.0-1206
36736 [main] INFO o.a.k.c.u.AppInfoParser - Kafka commitId : 022ed507ec080025
37169 [kafka-producer-network-thread | producer-1] WARN o.a.k.c.NetworkClient
- Error while fetching metadata with correlation id 0 :
{ATLAS_HOOK=LEADER_NOT_AVAILABLE}
[root@atlas-resilience-testing-1 ~]#
{noformat}
Attaching UI snapshot as well.
> Storm Hook should use timetsamps as Date type instead of Long
> -------------------------------------------------------------
>
> Key: ATLAS-675
> URL: https://issues.apache.org/jira/browse/ATLAS-675
> Project: Atlas
> Issue Type: Bug
> Affects Versions: trunk, 0.7-incubating
> Reporter: Suma Shivaprasad
> Assignee: Ayub Khan
> Fix For: trunk
>
> Attachments: ATLAS-675-1.patch, ATLAS-675-2.patch, ATLAS-675-3.patch,
> ATLAS-675.patch, Atlas 2016-08-18 15-24-55.png
>
>
> Storm topology startTime and endTime should be of type Date
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
