I can just guess here... the error message is a little weird, as it does not say which symbol it does not find... What is line 76?
On 02/21/2016 04:32 PM, david kavanagh wrote:
> That has solved the issue of importing the KafkaBolt class. Thank you!
> I see now that i will need to include any dependencies into the pom file
> before i can use any classes, etc.. from them.
>
> I am getting a different error now, but after 2 days of no progress, new
> errors are good :-)
>
> I am trying to use the example from the storm-kafka git account which is:
>
> Note: I am using the ExclamationTopology to get all the components working
> before i begin to write my own topologies. It worked fine before i
> started adding
> in the KafkaBolt.
>
> public static void main(String[] args) throws Exception {
> TopologyBuilder builder = new TopologyBuilder();
>
> builder.setSpout("word", new MySQLSpout(), 10);
> builder.setBolt("exclaim1", new ExclamationBolt(),
> 3).shuffleGrouping("word");
> builder.setBolt("exclaim2", new ExclamationBolt(),
> 2).shuffleGrouping("exclaim1");
>
> Config conf = new Config();
> conf.setDebug(true);
>
> //set producer properties.
> Properties props = new Properties();
> props.put("bootstrap.servers", "192.168.1.23:9092");
> props.put("acks", "1");
> props.put("key.serializer",
> "org.apache.kafka.common.serialization.StringSerializer");
> props.put("value.serializer",
> "org.apache.kafka.common.serialization.StringSerializer");
>
> KafkaBolt bolt = new KafkaBolt();
>
> bolt.withTopicSelector(new DefaultTopicSelector("test"));
> bolt.withTupleToKafkaMapper(new FieldNameBasedTupleToKafkaMapper());
> bolt.withProducerProperties(props);
>
> builder.setBolt("forwardToKafka", bolt, 8).shuffleGrouping("exclaim2");
>
>
> The error i am getting is regard to
> the bolt.withProducerProperties(props); line.
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile
> (default-compile) on project storm-starter: Compilation failure
> [ERROR]
> /root/storm-starter/src/jvm/storm/starter/ExclamationTopology.java:[76,8]
> error:
> cannot find symbol
> [ERROR] -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
> execute goal
> org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile
> (default-compile) on project storm-starter: Compilation failure
> /root/storm-starter/src/jvm/storm/starter/ExclamationTopology.java:[76,8]
> error:
> cannot find symbol
>
>
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.CompilationFailureException:
> Compilation failure
> /root/storm-starter/src/jvm/storm/starter/ExclamationTopology.java:[76,8]
> error:
> cannot find symbol
>
>
> at
> org.apache.maven.plugin.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:516)
> at org.apache.maven.plugin.CompilerMojo.execute(CompilerMojo.java:114)
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
> ... 19 more
> [ERROR]
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
>
> I have checked and that is definitely the correct method name from the
> KafkaBolt class.
> I have also tried using a different Properties object as the parameter
> just in case that was
> the problem, but no luck. Again, any help will be greatly appreciated.
>
> Thanks
> David
>
>
>> Subject: Re: Storm Kafka Bolt Integration
>> To: [email protected]
>> From: [email protected]
>> Date: Sun, 21 Feb 2016 15:19:46 +0100
>>
>> Eclipse is not required. Just need to know what your setup is. :)
>>
>> Try to include storm-kafka as dependency:
>>
>> > <dependency>
>> > <groupId>org.apache.storm</groupId>
>> > <artifactId>storm-kafka</artifactId>
>> > <version>0.9.0.1</version>
>> > </dependency>
>>
>> Btw: I guess that your Storm dependency is not correct. I guess it
>> should be:
>>
>> > <dependency>
>> > <groupId>org.apache.storm</groupId>
>> > <artifactId>storm-core</artifactId>
>> > <version>0.9.0.1</version>
>> > <!-- keep storm out of the jar-with-dependencies -->
>> > <scope>provided</scope>
>> > </dependency>
>>
>>
>> -Matthias
>>
>>
>> On 02/21/2016 03:09 PM, david kavanagh wrote:
>> > Hi Matthias,
>> >
>> > I am using Maven as a build tool. I am not using any IDE. I have a Storm
>> > cluster deployed on Ubuntu VMs,
>> > and i am suing sshfs to mount my Storm folder from the remote machine,
>> > to my local machine. Then
>> > I am just using Sublime Text on my local machine to edit the necessary
>> > files. Storm is working fine in both
>> > local mode and as a cluster. It is just the KafkaBolt that is the
> problem.
>> >
>> > I am brand new to Storm/Maven/Kafka so i am hoping it is a small issue
>> > that i am missing.
>> > Here is the contents of my pom.xml
>> >
>> > /<project xmlns="http://maven.apache.org/POM/4.0.0";
>> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/
>> > / xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
>> > http://maven.apache.org/xsd/maven-4.0.0.xsd";>/
>> > / <modelVersion>4.0.0</modelVersion>/
>> > /
>> > /
>> > / <groupId>storm.starter</groupId>/
>> > / <artifactId>storm-starter</artifactId>/
>> > / <version>0.0.1-SNAPSHOT</version>/
>> > / <packaging>jar</packaging>/
>> > /
>> > /
>> > / <name>storm-starter</name>/
>> > / <url>https://github.com/nathanmarz/storm-starter</url>/
>> > /
>> > /
>> > / <properties>/
>> > / <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>/
>> > / </properties>/
>> > /
>> > /
>> > / <repositories>/
>> > / <repository>/
>> > / <id>github-releases</id>/
>> > /
>> >
> <url>http://oss.sonatype.org/content/repositories/github-releases/</url>/
>> > / </repository>/
>> > / <repository>/
>> > / <id>clojars.org</id>/
>> > / <url>http://clojars.org/repo</url>/
>> > / </repository>/
>> > / </repositories>/
>> > /
>> > /
>> > / <dependencies>/
>> > / <dependency>/
>> > / <groupId>org.apache.kafka</groupId>/
>> > / <artifactId>kafka_2.9.2</artifactId>/
>> > / <version>0.8.1.1</version>/
>> > / <exclusions>/
>> > / <exclusion>/
>> > / <groupId>org.apache.zookeeper</groupId>/
>> > / <artifactId>zookeeper</artifactId>/
>> > / </exclusion>/
>> > / <exclusion>/
>> > / <groupId>log4j</groupId>/
>> > / <artifactId>log4j</artifactId>/
>> > / </exclusion>/
>> > / </exclusions>/
>> > / </dependency>/
>> > / <dependency>/
>> > / <groupId>mysql</groupId>/
>> > / <artifactId>mysql-connector-java</artifactId>/
>> > / <version>5.1.38</version>/
>> > / </dependency>/
>> > / <dependency>/
>> > / <groupId>junit</groupId>/
>> > / <artifactId>junit</artifactId>/
>> > / <version>3.8.1</version>/
>> > / <scope>test</scope>/
>> > / </dependency>/
>> > / <dependency>/
>> > / <groupId>org.testng</groupId>/
>> > / <artifactId>testng</artifactId>/
>> > / <version>6.8.5</version>/
>> > / <scope>test</scope>/
>> > / </dependency>/
>> > / <dependency>/
>> > / <groupId>org.mockito</groupId>/
>> > / <artifactId>mockito-all</artifactId>/
>> > / <version>1.9.0</version>/
>> > / <scope>test</scope>/
>> > / </dependency>/
>> > / <dependency>/
>> > / <groupId>org.easytesting</groupId>/
>> > / <artifactId>fest-assert-core</artifactId>/
>> > / <version>2.0M8</version>/
>> > / <scope>test</scope>/
>> > / </dependency>/
>> > / <dependency>/
>> > / <groupId>org.jmock</groupId>/
>> > / <artifactId>jmock</artifactId>/
>> > / <version>2.6.0</version>/
>> > / <scope>test</scope>/
>> > / </dependency>/
>> > / <dependency>/
>> > / <groupId>storm</groupId>/
>> > / <artifactId>storm</artifactId>/
>> > / <version>0.9.0.1</version>/
>> > / <!-- keep storm out of the jar-with-dependencies -->/
>> > / <scope>provided</scope>/
>> > / </dependency>/
>> > / <dependency>/
>> > / <groupId>commons-collections</groupId>/
>> > / <artifactId>commons-collections</artifactId>/
>> > / <version>3.2.1</version>/
>> > / </dependency>/
>> > / <dependency>/
>> > / <groupId>com.google.guava</groupId>/
>> > / <artifactId>guava</artifactId>/
>> > / <version>15.0</version>/
>> > / </dependency>/
>> > / </dependencies>/
>> > /
>> > /
>> > / <build>/
>> > / <sourceDirectory>src/jvm</sourceDirectory>/
>> > / <testSourceDirectory>test/jvm</testSourceDirectory>/
>> > / <resources>/
>> > / <resource>/
>> > / <directory>${basedir}/multilang</directory>/
>> > / </resource>/
>> > / </resources>/
>> > /
>> > /
>> > / <plugins>/
>> > / <!--/
>> > / Bind the maven-assembly-plugin to the package phase/
>> > / this will create a jar file without the storm dependencies/
>> > / suitable for deployment to a cluster./
>> > / -->/
>> > / <plugin>/
>> > / <artifactId>maven-assembly-plugin</artifactId>/
>> > / <configuration>/
>> > / <descriptorRefs>/
>> > / <descriptorRef>jar-with-dependencies</descriptorRef>/
>> > / </descriptorRefs>/
>> > / <archive>/
>> > / <manifest>/
>> > / <mainClass></mainClass>/
>> > / </manifest>/
>> > / </archive>/
>> > / </configuration>/
>> > / <executions>/
>> > / <execution>/
>> > / <id>make-assembly</id>/
>> > / <phase>package</phase>/
>> > / <goals>/
>> > / <goal>single</goal>/
>> > / </goals>/
>> > / </execution>/
>> > / </executions>/
>> > / </plugin>/
>> > / /
>> > / <plugin>/
>> > / <groupId>com.theoryinpractise</groupId>/
>> > / <artifactId>clojure-maven-plugin</artifactId>/
>> > / <version>1.3.12</version>/
>> > / <extensions>true</extensions>/
>> > / <configuration>/
>> > / <sourceDirectories>/
>> > / <sourceDirectory>src/clj</sourceDirectory>/
>> > / </sourceDirectories>/
>> > / </configuration>/
>> > / <executions>/
>> > / <execution>/
>> > / <id>compile</id>/
>> > / <phase>compile</phase>/
>> > / <goals>/
>> > / <goal>compile</goal>/
>> > / </goals>/
>> > / </execution>/
>> > / <execution>/
>> > / <id>test</id>/
>> > / <phase>test</phase>/
>> > / <goals>/
>> > / <goal>test</goal>/
>> > / </goals>/
>> > / </execution>/
>> > / </executions>/
>> > / </plugin>/
>> > /
>> > /
>> > / <plugin>/
>> > / <groupId>org.codehaus.mojo</groupId>/
>> > / <artifactId>exec-maven-plugin</artifactId>/
>> > / <version>1.2.1</version>/
>> > / <executions>/
>> > / <execution>/
>> > / <goals>/
>> > / <goal>exec</goal>/
>> > / </goals>/
>> > / </execution>/
>> > / </executions>/
>> > / <configuration>/
>> > / <executable>java</executable>/
>> > / <includeProjectDependencies>true</includeProjectDependencies>/
>> > / <includePluginDependencies>false</includePluginDependencies>/
>> > / <classpathScope>compile</classpathScope>/
>> > / <mainClass>${storm.topology}</mainClass>/
>> > / </configuration>/
>> > / </plugin>/
>> > /
>> > /
>> > / <plugin>/
>> > / <groupId>org.apache.maven.plugins</groupId>/
>> > / <artifactId>maven-compiler-plugin</artifactId>/
>> > / <configuration>/
>> > / <source>1.7</source>/
>> > / <target>1.7</target>/
>> > / </configuration>/
>> > / </plugin>/
>> > /
>> > /
>> > / </plugins>/
>> > / </build>/
>> > /</project>/
>> >
>> > Any help at all would be really appreciated. I have been stuck on this
>> > for 2 days now and i can't
>> > really move forward with the project until i am sure i can get the
>> > output from Storm into Kafka,
>> > and in turn my application. I do have Eclipse on my local machine so i
>> > can use that if it will help
>> > figure out the issue. Thank you very much for your reply.
>> >
>> > Kind Regards
>> > David
>> >
>> >
>> >
>> >
>> >
>> >
>> >> Subject: Re: Storm Kafka Bolt Integration
>> >> To: [email protected]
>> >> From: [email protected]
>> >> Date: Sun, 21 Feb 2016 14:44:00 +0100
>> >>
>> >> Hi David,
>> >>
>> >> yes you can just use the provide KafkaBolt.
>> >>
>> >> If the package is not found, I guess that you did not set up your
>> >> development environment correctly. What do you use? An IDE? Some build
>> >> tool like maven?
>> >>
>> >> -Matthias
>> >>
>> >> On 02/20/2016 02:13 PM, david kavanagh wrote:
>> >> > Hello,
>> >> >
>> >> > I am using Storm as part of my fourth year project. I have a Storm
>> >> > cluster running on some Ubuntu VM's
>> >> > in an OpenStack environment. Kafka is setup and working fine on
> its own
>> >> > VM. I am currently trying
>> >> > to integrate the two so that a Storm Topology sends its output to a
>> >> > Kafka Topic. I have been trying to understand
>> >> > KafkaBolt but i am having difficulty implementing it.
>> >> >
>> >> > Reading the documentation
>> >> > at http://storm.apache.org/documentation/storm-kafka.html , it
> seems to
>> >> > me that
>> >> > i can use a KafkaBolt class that has already been packaged. It is
> stated
>> >> > in the documentation:
>> >> >
>> >> > "/You can create an instance of storm.kafka.bolt.KafkaBolt and
> attach it
>> >> > as a component to your topology" /
>> >> >
>> >> > I have been trying to implement this but have had no success. No
> import
>> >> > statement i have tried has worked.
>> >> > i.e.
>> >> > /
>> >> > /
>> >> > /import storm.kafka.bolt.KafkaBolt;/
>> >> >
>> >> > The error being thrown up is: /error: package storm.kafka.bolt
>> >> > does not exist/
>> >> >
>> >> > I am new to Storm and Kafka and am aware that i may be getting
> the wrong
>> >> > impression from the documentation.
>> >> > Can i use import a KafkaBolt class or do i need to write my own
>> >> > KafkaBolt class? My only reason for using Kafka is to
>> >> > get the Storm Topology output to a Node.js application. I have
> searched
>> >> > exhaustively online to find any information
>> >> > on connecting Storms output to a node app and couldn't find anything
>> >> > useful so Kafka seemed the only way to go.
>> >> > If you could give me some information on sending a storm topologies
>> >> > output to Kafka, or straight to a nodejs app
>> >> > without Kafka, i would really appreciate it. Thank you in advance.
>> >> >
>> >> > Kind Regards
>> >> > David Kavanagh
>> >>
>>
signature.asc
Description: OpenPGP digital signature
