Hi Matthias, I am using Maven as a build tool. I am not using any IDE. I have a Storm cluster deployed on Ubuntu VMs,and i am suing sshfs to mount my Storm folder from the remote machine, to my local machine. Then I am just using Sublime Text on my local machine to edit the necessary files. Storm is working fine in bothlocal mode and as a cluster. It is just the KafkaBolt that is the problem. I am brand new to Storm/Maven/Kafka so i am hoping it is a small issue that i am missing. Here is the contents of my pom.xml <project xmlns="http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> <modelVersion>4.0.0</modelVersion> <groupId>storm.starter</groupId> <artifactId>storm-starter</artifactId> <version>0.0.1-SNAPSHOT</version> <packaging>jar</packaging> <name>storm-starter</name> <url>https://github.com/nathanmarz/storm-starter</url> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> </properties> <repositories> <repository> <id>github-releases</id> <url>http://oss.sonatype.org/content/repositories/github-releases/</url> </repository> <repository> <id>clojars.org</id> <url>http://clojars.org/repo</url> </repository> </repositories> <dependencies> <dependency> <groupId>org.apache.kafka</groupId> <artifactId>kafka_2.9.2</artifactId> <version>0.8.1.1</version> <exclusions> <exclusion> <groupId>org.apache.zookeeper</groupId> <artifactId>zookeeper</artifactId> </exclusion> <exclusion> <groupId>log4j</groupId> <artifactId>log4j</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <version>5.1.38</version> </dependency> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>3.8.1</version> <scope>test</scope> </dependency> <dependency> <groupId>org.testng</groupId> <artifactId>testng</artifactId> <version>6.8.5</version> <scope>test</scope> </dependency> <dependency> <groupId>org.mockito</groupId> <artifactId>mockito-all</artifactId> <version>1.9.0</version> <scope>test</scope> </dependency> <dependency> <groupId>org.easytesting</groupId> <artifactId>fest-assert-core</artifactId> <version>2.0M8</version> <scope>test</scope> </dependency> <dependency> <groupId>org.jmock</groupId> <artifactId>jmock</artifactId> <version>2.6.0</version> <scope>test</scope> </dependency> <dependency> <groupId>storm</groupId> <artifactId>storm</artifactId> <version>0.9.0.1</version> <!-- keep storm out of the jar-with-dependencies --> <scope>provided</scope> </dependency> <dependency> <groupId>commons-collections</groupId> <artifactId>commons-collections</artifactId> <version>3.2.1</version> </dependency> <dependency> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> <version>15.0</version> </dependency> </dependencies> <build> <sourceDirectory>src/jvm</sourceDirectory> <testSourceDirectory>test/jvm</testSourceDirectory> <resources> <resource> <directory>${basedir}/multilang</directory> </resource> </resources> <plugins> <!-- Bind the maven-assembly-plugin to the package phase this will create a jar file without the storm dependencies suitable for deployment to a cluster. --> <plugin> <artifactId>maven-assembly-plugin</artifactId> <configuration> <descriptorRefs> <descriptorRef>jar-with-dependencies</descriptorRef> </descriptorRefs> <archive> <manifest> <mainClass></mainClass> </manifest> </archive> </configuration> <executions> <execution> <id>make-assembly</id> <phase>package</phase> <goals> <goal>single</goal> </goals> </execution> </executions> </plugin> <plugin> <groupId>com.theoryinpractise</groupId> <artifactId>clojure-maven-plugin</artifactId> <version>1.3.12</version> <extensions>true</extensions> <configuration> <sourceDirectories> <sourceDirectory>src/clj</sourceDirectory> </sourceDirectories> </configuration> <executions> <execution> <id>compile</id> <phase>compile</phase> <goals> <goal>compile</goal> </goals> </execution> <execution> <id>test</id> <phase>test</phase> <goals> <goal>test</goal> </goals> </execution> </executions> </plugin> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>exec-maven-plugin</artifactId> <version>1.2.1</version> <executions> <execution> <goals> <goal>exec</goal> </goals> </execution> </executions> <configuration> <executable>java</executable> <includeProjectDependencies>true</includeProjectDependencies> <includePluginDependencies>false</includePluginDependencies> <classpathScope>compile</classpathScope> <mainClass>${storm.topology}</mainClass> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <configuration> <source>1.7</source> <target>1.7</target> </configuration> </plugin> </plugins> </build></project> Any help at all would be really appreciated. I have been stuck on this for 2 days now and i can't really move forward with the project until i am sure i can get the output from Storm into Kafka, and in turn my application. I do have Eclipse on my local machine so i can use that if it will helpfigure out the issue. Thank you very much for your reply. Kind RegardsDavid
> Subject: Re: Storm Kafka Bolt Integration > To: [email protected] > From: [email protected] > Date: Sun, 21 Feb 2016 14:44:00 +0100 > > Hi David, > > yes you can just use the provide KafkaBolt. > > If the package is not found, I guess that you did not set up your > development environment correctly. What do you use? An IDE? Some build > tool like maven? > > -Matthias > > On 02/20/2016 02:13 PM, david kavanagh wrote: > > Hello, > > > > I am using Storm as part of my fourth year project. I have a Storm > > cluster running on some Ubuntu VM's > > in an OpenStack environment. Kafka is setup and working fine on its own > > VM. I am currently trying > > to integrate the two so that a Storm Topology sends its output to a > > Kafka Topic. I have been trying to understand > > KafkaBolt but i am having difficulty implementing it. > > > > Reading the documentation > > at http://storm.apache.org/documentation/storm-kafka.html , it seems to > > me that > > i can use a KafkaBolt class that has already been packaged. It is stated > > in the documentation: > > > > "/You can create an instance of storm.kafka.bolt.KafkaBolt and attach it > > as a component to your topology" / > > > > I have been trying to implement this but have had no success. No import > > statement i have tried has worked. > > i.e. > > / > > / > > /import storm.kafka.bolt.KafkaBolt;/ > > > > The error being thrown up is: /error: package storm.kafka.bolt > > does not exist/ > > > > I am new to Storm and Kafka and am aware that i may be getting the wrong > > impression from the documentation. > > Can i use import a KafkaBolt class or do i need to write my own > > KafkaBolt class? My only reason for using Kafka is to > > get the Storm Topology output to a Node.js application. I have searched > > exhaustively online to find any information > > on connecting Storms output to a node app and couldn't find anything > > useful so Kafka seemed the only way to go. > > If you could give me some information on sending a storm topologies > > output to Kafka, or straight to a nodejs app > > without Kafka, i would really appreciate it. Thank you in advance. > > > > Kind Regards > > David Kavanagh >
