ExclamationTopology.java:[76,8] error: cannot find symbol
Line 76 is
bolt.withProducerProperties(props);
and column 8 is the "." for calling the method.
There seems to be some problem with the withProducerProperties(props) method. I
checked the KafkaBolt class from the storm-kafka project on github and
themethod name is definitely correct. Like you said, the error message is a bit
vague so it is hard to see what is wrong. The error is saying "cannot find
symbol",which is usually a syntax error, but i can't see how it can be that.
Thank you again, i appreciate the help.
David
> Subject: Re: Storm Kafka Bolt Integration
> To: [email protected]
> From: [email protected]
> Date: Sun, 21 Feb 2016 18:29:59 +0100
>
> I can just guess here... the error message is a little weird, as it does
> not say which symbol it does not find... What is line 76?
>
>
>
>
> On 02/21/2016 04:32 PM, david kavanagh wrote:
> > That has solved the issue of importing the KafkaBolt class. Thank you!
> > I see now that i will need to include any dependencies into the pom file
> > before i can use any classes, etc.. from them.
> >
> > I am getting a different error now, but after 2 days of no progress, new
> > errors are good :-)
> >
> > I am trying to use the example from the storm-kafka git account which is:
> >
> > Note: I am using the ExclamationTopology to get all the components working
> > before i begin to write my own topologies. It worked fine before i
> > started adding
> > in the KafkaBolt.
> >
> > public static void main(String[] args) throws Exception {
> > TopologyBuilder builder = new TopologyBuilder();
> >
> > builder.setSpout("word", new MySQLSpout(), 10);
> > builder.setBolt("exclaim1", new ExclamationBolt(),
> > 3).shuffleGrouping("word");
> > builder.setBolt("exclaim2", new ExclamationBolt(),
> > 2).shuffleGrouping("exclaim1");
> >
> > Config conf = new Config();
> > conf.setDebug(true);
> >
> > //set producer properties.
> > Properties props = new Properties();
> > props.put("bootstrap.servers", "192.168.1.23:9092");
> > props.put("acks", "1");
> > props.put("key.serializer",
> > "org.apache.kafka.common.serialization.StringSerializer");
> > props.put("value.serializer",
> > "org.apache.kafka.common.serialization.StringSerializer");
> >
> > KafkaBolt bolt = new KafkaBolt();
> >
> > bolt.withTopicSelector(new DefaultTopicSelector("test"));
> > bolt.withTupleToKafkaMapper(new FieldNameBasedTupleToKafkaMapper());
> > bolt.withProducerProperties(props);
> >
> > builder.setBolt("forwardToKafka", bolt, 8).shuffleGrouping("exclaim2");
> >
> >
> > The error i am getting is regard to
> > the bolt.withProducerProperties(props); line.
> >
> > [ERROR] Failed to execute goal
> > org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile
> > (default-compile) on project storm-starter: Compilation failure
> > [ERROR]
> > /root/storm-starter/src/jvm/storm/starter/ExclamationTopology.java:[76,8]
> > error:
> > cannot find symbol
> > [ERROR] -> [Help 1]
> > org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
> > execute goal
> > org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile
> > (default-compile) on project storm-starter: Compilation failure
> > /root/storm-starter/src/jvm/storm/starter/ExclamationTopology.java:[76,8]
> > error:
> > cannot find symbol
> >
> >
> > at
> > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
> > at
> > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> > at
> > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> > at
> > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
> > at
> > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
> > at
> > org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
> > at
> > org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
> > at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
> > at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
> > at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
> > at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
> > at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> > org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> > at
> > org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> > at
> > org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> > at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> > Caused by: org.apache.maven.plugin.CompilationFailureException:
> > Compilation failure
> > /root/storm-starter/src/jvm/storm/starter/ExclamationTopology.java:[76,8]
> > error:
> > cannot find symbol
> >
> >
> > at
> > org.apache.maven.plugin.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:516)
> > at org.apache.maven.plugin.CompilerMojo.execute(CompilerMojo.java:114)
> > at
> > org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
> > at
> > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
> > ... 19 more
> > [ERROR]
> > [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> > [ERROR]
> > [ERROR] For more information about the errors and possible solutions,
> > please read the following articles:
> > [ERROR] [Help 1]
> > http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> >
> > I have checked and that is definitely the correct method name from the
> > KafkaBolt class.
> > I have also tried using a different Properties object as the parameter
> > just in case that was
> > the problem, but no luck. Again, any help will be greatly appreciated.
> >
> > Thanks
> > David
> >
> >
> >> Subject: Re: Storm Kafka Bolt Integration
> >> To: [email protected]
> >> From: [email protected]
> >> Date: Sun, 21 Feb 2016 15:19:46 +0100
> >>
> >> Eclipse is not required. Just need to know what your setup is. :)
> >>
> >> Try to include storm-kafka as dependency:
> >>
> >> > <dependency>
> >> > <groupId>org.apache.storm</groupId>
> >> > <artifactId>storm-kafka</artifactId>
> >> > <version>0.9.0.1</version>
> >> > </dependency>
> >>
> >> Btw: I guess that your Storm dependency is not correct. I guess it
> >> should be:
> >>
> >> > <dependency>
> >> > <groupId>org.apache.storm</groupId>
> >> > <artifactId>storm-core</artifactId>
> >> > <version>0.9.0.1</version>
> >> > <!-- keep storm out of the jar-with-dependencies -->
> >> > <scope>provided</scope>
> >> > </dependency>
> >>
> >>
> >> -Matthias
> >>
> >>
> >> On 02/21/2016 03:09 PM, david kavanagh wrote:
> >> > Hi Matthias,
> >> >
> >> > I am using Maven as a build tool. I am not using any IDE. I have a Storm
> >> > cluster deployed on Ubuntu VMs,
> >> > and i am suing sshfs to mount my Storm folder from the remote machine,
> >> > to my local machine. Then
> >> > I am just using Sublime Text on my local machine to edit the necessary
> >> > files. Storm is working fine in both
> >> > local mode and as a cluster. It is just the KafkaBolt that is the
> > problem.
> >> >
> >> > I am brand new to Storm/Maven/Kafka so i am hoping it is a small issue
> >> > that i am missing.
> >> > Here is the contents of my pom.xml
> >> >
> >> > /<project xmlns="http://maven.apache.org/POM/4.0.0";
> >> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/
> >> > / xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> >> > http://maven.apache.org/xsd/maven-4.0.0.xsd";>/
> >> > / <modelVersion>4.0.0</modelVersion>/
> >> > /
> >> > /
> >> > / <groupId>storm.starter</groupId>/
> >> > / <artifactId>storm-starter</artifactId>/
> >> > / <version>0.0.1-SNAPSHOT</version>/
> >> > / <packaging>jar</packaging>/
> >> > /
> >> > /
> >> > / <name>storm-starter</name>/
> >> > / <url>https://github.com/nathanmarz/storm-starter</url>/
> >> > /
> >> > /
> >> > / <properties>/
> >> > / <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>/
> >> > / </properties>/
> >> > /
> >> > /
> >> > / <repositories>/
> >> > / <repository>/
> >> > / <id>github-releases</id>/
> >> > /
> >> >
> > <url>http://oss.sonatype.org/content/repositories/github-releases/</url>/
> >> > / </repository>/
> >> > / <repository>/
> >> > / <id>clojars.org</id>/
> >> > / <url>http://clojars.org/repo</url>/
> >> > / </repository>/
> >> > / </repositories>/
> >> > /
> >> > /
> >> > / <dependencies>/
> >> > / <dependency>/
> >> > / <groupId>org.apache.kafka</groupId>/
> >> > / <artifactId>kafka_2.9.2</artifactId>/
> >> > / <version>0.8.1.1</version>/
> >> > / <exclusions>/
> >> > / <exclusion>/
> >> > / <groupId>org.apache.zookeeper</groupId>/
> >> > / <artifactId>zookeeper</artifactId>/
> >> > / </exclusion>/
> >> > / <exclusion>/
> >> > / <groupId>log4j</groupId>/
> >> > / <artifactId>log4j</artifactId>/
> >> > / </exclusion>/
> >> > / </exclusions>/
> >> > / </dependency>/
> >> > / <dependency>/
> >> > / <groupId>mysql</groupId>/
> >> > / <artifactId>mysql-connector-java</artifactId>/
> >> > / <version>5.1.38</version>/
> >> > / </dependency>/
> >> > / <dependency>/
> >> > / <groupId>junit</groupId>/
> >> > / <artifactId>junit</artifactId>/
> >> > / <version>3.8.1</version>/
> >> > / <scope>test</scope>/
> >> > / </dependency>/
> >> > / <dependency>/
> >> > / <groupId>org.testng</groupId>/
> >> > / <artifactId>testng</artifactId>/
> >> > / <version>6.8.5</version>/
> >> > / <scope>test</scope>/
> >> > / </dependency>/
> >> > / <dependency>/
> >> > / <groupId>org.mockito</groupId>/
> >> > / <artifactId>mockito-all</artifactId>/
> >> > / <version>1.9.0</version>/
> >> > / <scope>test</scope>/
> >> > / </dependency>/
> >> > / <dependency>/
> >> > / <groupId>org.easytesting</groupId>/
> >> > / <artifactId>fest-assert-core</artifactId>/
> >> > / <version>2.0M8</version>/
> >> > / <scope>test</scope>/
> >> > / </dependency>/
> >> > / <dependency>/
> >> > / <groupId>org.jmock</groupId>/
> >> > / <artifactId>jmock</artifactId>/
> >> > / <version>2.6.0</version>/
> >> > / <scope>test</scope>/
> >> > / </dependency>/
> >> > / <dependency>/
> >> > / <groupId>storm</groupId>/
> >> > / <artifactId>storm</artifactId>/
> >> > / <version>0.9.0.1</version>/
> >> > / <!-- keep storm out of the jar-with-dependencies -->/
> >> > / <scope>provided</scope>/
> >> > / </dependency>/
> >> > / <dependency>/
> >> > / <groupId>commons-collections</groupId>/
> >> > / <artifactId>commons-collections</artifactId>/
> >> > / <version>3.2.1</version>/
> >> > / </dependency>/
> >> > / <dependency>/
> >> > / <groupId>com.google.guava</groupId>/
> >> > / <artifactId>guava</artifactId>/
> >> > / <version>15.0</version>/
> >> > / </dependency>/
> >> > / </dependencies>/
> >> > /
> >> > /
> >> > / <build>/
> >> > / <sourceDirectory>src/jvm</sourceDirectory>/
> >> > / <testSourceDirectory>test/jvm</testSourceDirectory>/
> >> > / <resources>/
> >> > / <resource>/
> >> > / <directory>${basedir}/multilang</directory>/
> >> > / </resource>/
> >> > / </resources>/
> >> > /
> >> > /
> >> > / <plugins>/
> >> > / <!--/
> >> > / Bind the maven-assembly-plugin to the package phase/
> >> > / this will create a jar file without the storm dependencies/
> >> > / suitable for deployment to a cluster./
> >> > / -->/
> >> > / <plugin>/
> >> > / <artifactId>maven-assembly-plugin</artifactId>/
> >> > / <configuration>/
> >> > / <descriptorRefs>/
> >> > / <descriptorRef>jar-with-dependencies</descriptorRef>/
> >> > / </descriptorRefs>/
> >> > / <archive>/
> >> > / <manifest>/
> >> > / <mainClass></mainClass>/
> >> > / </manifest>/
> >> > / </archive>/
> >> > / </configuration>/
> >> > / <executions>/
> >> > / <execution>/
> >> > / <id>make-assembly</id>/
> >> > / <phase>package</phase>/
> >> > / <goals>/
> >> > / <goal>single</goal>/
> >> > / </goals>/
> >> > / </execution>/
> >> > / </executions>/
> >> > / </plugin>/
> >> > / /
> >> > / <plugin>/
> >> > / <groupId>com.theoryinpractise</groupId>/
> >> > / <artifactId>clojure-maven-plugin</artifactId>/
> >> > / <version>1.3.12</version>/
> >> > / <extensions>true</extensions>/
> >> > / <configuration>/
> >> > / <sourceDirectories>/
> >> > / <sourceDirectory>src/clj</sourceDirectory>/
> >> > / </sourceDirectories>/
> >> > / </configuration>/
> >> > / <executions>/
> >> > / <execution>/
> >> > / <id>compile</id>/
> >> > / <phase>compile</phase>/
> >> > / <goals>/
> >> > / <goal>compile</goal>/
> >> > / </goals>/
> >> > / </execution>/
> >> > / <execution>/
> >> > / <id>test</id>/
> >> > / <phase>test</phase>/
> >> > / <goals>/
> >> > / <goal>test</goal>/
> >> > / </goals>/
> >> > / </execution>/
> >> > / </executions>/
> >> > / </plugin>/
> >> > /
> >> > /
> >> > / <plugin>/
> >> > / <groupId>org.codehaus.mojo</groupId>/
> >> > / <artifactId>exec-maven-plugin</artifactId>/
> >> > / <version>1.2.1</version>/
> >> > / <executions>/
> >> > / <execution>/
> >> > / <goals>/
> >> > / <goal>exec</goal>/
> >> > / </goals>/
> >> > / </execution>/
> >> > / </executions>/
> >> > / <configuration>/
> >> > / <executable>java</executable>/
> >> > / <includeProjectDependencies>true</includeProjectDependencies>/
> >> > / <includePluginDependencies>false</includePluginDependencies>/
> >> > / <classpathScope>compile</classpathScope>/
> >> > / <mainClass>${storm.topology}</mainClass>/
> >> > / </configuration>/
> >> > / </plugin>/
> >> > /
> >> > /
> >> > / <plugin>/
> >> > / <groupId>org.apache.maven.plugins</groupId>/
> >> > / <artifactId>maven-compiler-plugin</artifactId>/
> >> > / <configuration>/
> >> > / <source>1.7</source>/
> >> > / <target>1.7</target>/
> >> > / </configuration>/
> >> > / </plugin>/
> >> > /
> >> > /
> >> > / </plugins>/
> >> > / </build>/
> >> > /</project>/
> >> >
> >> > Any help at all would be really appreciated. I have been stuck on this
> >> > for 2 days now and i can't
> >> > really move forward with the project until i am sure i can get the
> >> > output from Storm into Kafka,
> >> > and in turn my application. I do have Eclipse on my local machine so i
> >> > can use that if it will help
> >> > figure out the issue. Thank you very much for your reply.
> >> >
> >> > Kind Regards
> >> > David
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >> Subject: Re: Storm Kafka Bolt Integration
> >> >> To: [email protected]
> >> >> From: [email protected]
> >> >> Date: Sun, 21 Feb 2016 14:44:00 +0100
> >> >>
> >> >> Hi David,
> >> >>
> >> >> yes you can just use the provide KafkaBolt.
> >> >>
> >> >> If the package is not found, I guess that you did not set up your
> >> >> development environment correctly. What do you use? An IDE? Some build
> >> >> tool like maven?
> >> >>
> >> >> -Matthias
> >> >>
> >> >> On 02/20/2016 02:13 PM, david kavanagh wrote:
> >> >> > Hello,
> >> >> >
> >> >> > I am using Storm as part of my fourth year project. I have a Storm
> >> >> > cluster running on some Ubuntu VM's
> >> >> > in an OpenStack environment. Kafka is setup and working fine on
> > its own
> >> >> > VM. I am currently trying
> >> >> > to integrate the two so that a Storm Topology sends its output to a
> >> >> > Kafka Topic. I have been trying to understand
> >> >> > KafkaBolt but i am having difficulty implementing it.
> >> >> >
> >> >> > Reading the documentation
> >> >> > at http://storm.apache.org/documentation/storm-kafka.html , it
> > seems to
> >> >> > me that
> >> >> > i can use a KafkaBolt class that has already been packaged. It is
> > stated
> >> >> > in the documentation:
> >> >> >
> >> >> > "/You can create an instance of storm.kafka.bolt.KafkaBolt and
> > attach it
> >> >> > as a component to your topology" /
> >> >> >
> >> >> > I have been trying to implement this but have had no success. No
> > import
> >> >> > statement i have tried has worked.
> >> >> > i.e.
> >> >> > /
> >> >> > /
> >> >> > /import storm.kafka.bolt.KafkaBolt;/
> >> >> >
> >> >> > The error being thrown up is: /error: package storm.kafka.bolt
> >> >> > does not exist/
> >> >> >
> >> >> > I am new to Storm and Kafka and am aware that i may be getting
> > the wrong
> >> >> > impression from the documentation.
> >> >> > Can i use import a KafkaBolt class or do i need to write my own
> >> >> > KafkaBolt class? My only reason for using Kafka is to
> >> >> > get the Storm Topology output to a Node.js application. I have
> > searched
> >> >> > exhaustively online to find any information
> >> >> > on connecting Storms output to a node app and couldn't find anything
> >> >> > useful so Kafka seemed the only way to go.
> >> >> > If you could give me some information on sending a storm topologies
> >> >> > output to Kafka, or straight to a nodejs app
> >> >> > without Kafka, i would really appreciate it. Thank you in advance.
> >> >> >
> >> >> > Kind Regards
> >> >> > David Kavanagh
> >> >>
> >>
>
