Hi, all:
I was trying to build a Storm topology with Hadoop and Hbase
dependencies. And I want to run this topology with Storm-on-Yarn. The
version of Storm in it is 0.9.0-wip21. I created the Jar file with Maven,
and the pom.xml file is attached.
I submitted the topology (with-dependencies), and there was no
exceptions. But it did't run at all. I checked the supervisor logs (see
bottom), which shows that the workers failed to start.
To trace the reason of this issue, I add a test topology
(WordCountTopology) in the jar file. The problems remains when trying to
submit the WordCountTopology with dependencies, but it works well when
trying to submit the topology without dependencies.
To make it clear that where is this issue from, I ran the topology
without Storm-on-Yarn. I tried Storm-0.8.1 and Storm-0.9.0-wip21. The
results show that:
1) Storm-0.8.1 is okay for the jar topology, no matter with or without
dependencies.
2) Storm-0.9.0-wip21 is okay for the jar topology without dependencies,
but the same issue (workers failed to start) appears for the
"with-dependencies" jar file.
I am a newer for Storm, and I don't know the differences between these
two Storm version. What will result in the worker failure when trying to
submit the "with-dependencies" topology: Is the "with-dependencies" jar
file too big? or something conflicting?
Any idea?
Will Storm-0.8.1 work with Storm-on-Yarn?
Thanks~
--
------------------
Best Regards!
Zheng Xue
*http://netlab.sysu.edu.cn/~zhxue <http://netlab.sysu.edu.cn/~zhxue>*
Computer Science and Technology college,Sun Yat-sen University
E-mail: [email protected], [email protected]
