Thanks for the explanation! The hint with PYHTONPATH got me to the solution:
The SplitSentence bolt must be configured correctly.
I just change the code from
> builder.setBolt("split", new SplitSentence(), 8).shuffleGrouping("spout");
to
> SplitSentence pythonSplit = new SplitSentence();
> Map env = new HashMap();
> env.put("PYTHONPATH",
> "/home/mjsax/workspace_storm/storm/storm-multilang/python/src/main/resources/resources/");
> pythonSplit.setEnv(env);
> builder.setBolt("split",pythonSplit, 8).shuffleGrouping("spout");
-Matthias
On 08/19/2015 04:29 AM, 임정택 wrote:
> Hi Matthias, sorry to respond lately.
>
> Unfortunately, AFAIK you can't run multilang feature with LocalCluster
> without having packaged file.
>
> ShellProcess relies on codeDir of TopologyContext, which is used by
> supervisor. Workers are serialized to stormcode.ser, but multilang files
> should extracted to outside of serialized file so that python/ruby/node/etc
> can load it.
>
> Accomplishing this with distribute mode is easy because there's always user
> submitted jar, and supervisor can know it is what user submitted.
>
> But accomplishing this with local mode is not easy cause supervisor cannot
> know user submitted jar, and users can run topology to local mode without
> packaging.
>
> So, Supervisor in local mode finds resource directory ("resources") from
> each jars (which ends with "jar") in classpath, and copy first occurrence
> to codeDir.
>
> storm jar places user topology jar to the first of classpath, so it can be
> run without issue.
>
> So normally, it's natural for ShellProcess to not find "splitsentence.py".
> Maybe your working directory or PYTHONPATH do the trick.
>
> Hope this helps.
>
> Best,
> Jungtaek Lim (HeartSaVioR)
> ps.
> I also respond to your SO question with same content.
> http://stackoverflow.com/a/32085316/3480122
>
> 2015-08-10 6:49 GMT+09:00 Matthias J. Sax <[email protected]>:
>
>> Hi,
>>
>> I work with Storm for a while already, but want to get started with
>> development. As suggested, I am using Intellij (up to now, I was using
>> Eclipse).
>>
>> I was also looking at
>>
>> https://github.com/apache/storm/tree/master/examples/storm-starter#intellij-idea
>>
>> This documentation is not complete. I was not able to run anything in
>> Intellij first. I could figure out, that I need to remove the scope of
>> storm-core dependency (in storm-starter pom.xml). (found here:
>>
>> https://stackoverflow.com/questions/30724424/storm-starter-with-intellij-idea-maven-project-could-not-find-class
>> )
>>
>> After that I wass able to build the project. I can also run
>> ExclamationTopology with no problems within Intellij. However,
>> WordCountTopology fails.
>>
>> First I got the following error:
>>
>>> java.lang.RuntimeException: backtype.storm.multilang.NoOutputException:
>> Pipe to subprocess seems to be broken! No output read.
>>> Serializer Exception:
>>> Traceback (most recent call last):
>>> File "splitsentence.py", line 16, in <module>
>>> import storm
>>> ImportError: No module named storm
>>
>> I was able to resolve it via: apt-get install python-storm
>> (from StackOverflow)
>>
>> However, I don't speak Python and was wondering what the problem is and
>> why I could resolve it like this. Just want to get deeper into it. Maybe
>> someone can explain.
>>
>> Unfortunately, I am getting a different error now:
>>
>>> java.lang.RuntimeException: backtype.storm.multilang.NoOutputException:
>> Pipe to subprocess seems to be broken! No output read.
>>> Serializer Exception:
>>> Traceback (most recent call last):
>>> File "splitsentence.py", line 16, in <module>
>>> import storm
>>> ImportError: No module named storm
>>
>> I did not find any solution on the Internet. And as I am not familiar
>> with Python and never used Storm differently as low-level Java API I am
>> stuck now. Because ExclamationTopology runs, I guess my basic setup is
>> correct.
>>
>> What do I do wrong?
>>
>> -Matthias
>>
>>
>>
>>
>
>
signature.asc
Description: OpenPGP digital signature
