Re: Decline in apache storm usage

[Stig Rohde Døssing]

Regarding cloud providers offering hosted Storm clusters, that's really up
to the cloud providers to figure out. I know at least Azure used to provide
hosted Storm clusters under the HDInsight label. They may still do.

Den tor. 29. aug. 2019 kl. 20.28 skrev Ethan Li :

> Sorry I don’t have knowledge about that.
>
> On Aug 29, 2019, at 11:32 AM, Gunjan Dave 
> wrote:
>
> Thanks ethan for getting back.
>
> Would you be aware if there are any plans of cloud providers such as
> google, aws or azure providing  apache storm cloud services similar to how
> GCP has done it for apache spark/gcp-proc and apache-beam/gcp-dataflow?
>
>
> On Thu, Aug 29, 2019, 20:54 Ethan Li  wrote:
>
>> Hi Gunjan,
>>
>> I can’t speak for general usages. But the company I work for is using
>> Apache Storm very heavily.
>>
>>
>> On Aug 28, 2019, at 9:44 AM, Gunjan Dave 
>> wrote:
>>
>>
>> Hello devs,
>>> Seeing that apache storms usage is declining. Do you attribute this to
>>> anything?
>>> Even though storm do umentation shows quite number of companies using
>>> storm, seems like that page is outdated.
>>>
>>> Is storm community taking any steps to increase the usage and adoption
>>> again?
>>>
>>> Also seems like a lot of cloud providers are also not giving storm based
>>> service as opposed to other streaming solutions. Is storm community doing
>>> anything to have more cloud adoption for storm
>>>
>>
>>
>

streamparse

[Zainal Arifin (BLOOMBERG/ 731 LEX)]

Hi, anyone is using streamparse to run python code on apache storm?

need help: storm worker keep crashing

[Zainal Arifin (BLOOMBERG/ 731 LEX)]

Hi,
I wonder if anyone who ever used streamparse (for storm topology in python 
code), ever run into this issue I am facing right now.
Unfortunately, I don't see any error from streamparse log, but I see exception 
below in the storm worker log.
The serializer exception section seems the culprit, but it doesn't have 
sufficient information for me to figure it out.
Perhaps any of you have idea/pointer for me? Thanks!


2019-08-28 15:05:32.947 o.a.s.s.ShellSpout Thread-11-event_spout-executor[10 
10] [INFO] Launched subprocess with pid 10054
2019-08-28 15:05:32.951 o.a.s.d.executor Thread-11-event_spout-executor[10 10] 
[INFO] Opened spout event_spout:(10)
2019-08-28 15:05:32.953 o.a.s.d.executor Thread-11-event_spout-executor[10 10] 
[INFO] Activating spout event_spout:(10)
2019-08-28 15:05:32.953 o.a.s.s.ShellSpout Thread-11-event_spout-executor[10 
10] [INFO] Start checking heartbeat...
2019-08-28 15:05:32.961 o.a.s.util Thread-11-event_spout-executor[10 10] 
[ERROR] Async loop died!
java.lang.RuntimeException: pid:10054, name:event_spout exitCode:-1, 
errorString:
at 
org.apache.storm.spout.ShellSpout.querySubprocess(ShellSpout.java:218) 
~[storm-core-1.1.1.jar:1.1.1]
at 
org.apache.storm.spout.ShellSpout.sendSyncCommand(ShellSpout.java:145) 
~[storm-core-1.1.1.jar:1.1.1]
at org.apache.storm.spout.ShellSpout.activate(ShellSpout.java:266) 
~[storm-core-1.1.1.jar:1.1.1]
at 
org.apache.storm.daemon.executor$fn__4962$fn__4977$fn__5008.invoke(executor.clj:641)
 ~[storm-core-1.1.1.jar:1.1.1]
at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484) 
[storm-core-1.1.1.jar:1.1.1]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
Caused by: java.lang.RuntimeException: 
org.apache.storm.multilang.NoOutputException: Pipe to subprocess seems to be 
broken! No output read.
Serializer Exception:


at 
org.apache.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:127) 
~[storm-core-1.1.1.jar:1.1.1]
at 
org.apache.storm.spout.ShellSpout.querySubprocess(ShellSpout.java:183) 
~[storm-core-1.1.1.jar:1.1.1]
... 6 more
2019-08-28 15:05:32.968 o.a.s.d.executor Thread-11-event_spout-executor[10 10] 
[ERROR]
java.lang.RuntimeException: pid:10054, name:event_spout exitCode:-1, 
errorString:
at 
org.apache.storm.spout.ShellSpout.querySubprocess(ShellSpout.java:218) 
~[storm-core-1.1.1.jar:1.1.1]
at 
org.apache.storm.spout.ShellSpout.sendSyncCommand(ShellSpout.java:145) 
~[storm-core-1.1.1.jar:1.1.1]
at org.apache.storm.spout.ShellSpout.activate(ShellSpout.java:266) 
~[storm-core-1.1.1.jar:1.1.1]
at 
org.apache.storm.daemon.executor$fn__4962$fn__4977$fn__5008.invoke(executor.clj:641)
 ~[storm-core-1.1.1.jar:1.1.1]
at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484) 
[storm-core-1.1.1.jar:1.1.1]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
Caused by: java.lang.RuntimeException: 
org.apache.storm.multilang.NoOutputException: Pipe to subprocess seems to be 
broken! No output read.
Serializer Exception:


at 
org.apache.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:127) 
~[storm-core-1.1.1.jar:1.1.1]
at 
org.apache.storm.spout.ShellSpout.querySubprocess(ShellSpout.java:183) 
~[storm-core-1.1.1.jar:1.1.1]
... 6 more
2019-08-28 15:05:33.009 o.a.s.util Thread-11-event_spout-executor[10 10] 
[ERROR] Halting process: ("Worker died")
java.lang.RuntimeException: ("Worker died")
at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341) 
[storm-core-1.1.1.jar:1.1.1]
at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.7.0.jar:?]
at 
org.apache.storm.daemon.worker$fn__5632$fn__5633.invoke(worker.clj:763) 
[storm-core-1.1.1.jar:1.1.1]
at 
org.apache.storm.daemon.executor$mk_executor_data$fn__4848$fn__4849.invoke(executor.clj:276)
 [storm-core-1.1.1.jar:1.1.1]
at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:494) 
[storm-core-1.1.1.jar:1.1.1]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
2019-08-28 15:05:33.018 o.a.s.d.worker Thread-16 [INFO] Shutting down worker 
tmon-4-1567019114 ba5b3695-b390-4c3e-9d92-af0771f17b86 6700

Re: Decline in apache storm usage

[Ethan Li]

Sorry I don’t have knowledge about that. 

> On Aug 29, 2019, at 11:32 AM, Gunjan Dave  wrote:
> 
> Thanks ethan for getting back.
> 
> Would you be aware if there are any plans of cloud providers such as google, 
> aws or azure providing  apache storm cloud services similar to how GCP has 
> done it for apache spark/gcp-proc and apache-beam/gcp-dataflow? 
> 
> 
> On Thu, Aug 29, 2019, 20:54 Ethan Li  > wrote:
> Hi Gunjan,
> 
> I can’t speak for general usages. But the company I work for is using Apache 
> Storm very heavily. 
> 
> 
>> On Aug 28, 2019, at 9:44 AM, Gunjan Dave > > wrote:
>> 
>> 
>> Hello devs,
>> Seeing that apache storms usage is declining. Do you attribute this to 
>> anything?
>> Even though storm do umentation shows quite number of companies using storm, 
>> seems like that page is outdated. 
>> 
>> Is storm community taking any steps to increase the usage and adoption again?
>> 
>> Also seems like a lot of cloud providers are also not giving storm based 
>> service as opposed to other streaming solutions. Is storm community doing 
>> anything to have more cloud adoption for storm 
> 

Re: Decline in apache storm usage

[Gunjan Dave]

Thanks ethan for getting back.

Would you be aware if there are any plans of cloud providers such as
google, aws or azure providing  apache storm cloud services similar to how
GCP has done it for apache spark/gcp-proc and apache-beam/gcp-dataflow?


On Thu, Aug 29, 2019, 20:54 Ethan Li  wrote:

> Hi Gunjan,
>
> I can’t speak for general usages. But the company I work for is using
> Apache Storm very heavily.
>
>
> On Aug 28, 2019, at 9:44 AM, Gunjan Dave 
> wrote:
>
>
> Hello devs,
>> Seeing that apache storms usage is declining. Do you attribute this to
>> anything?
>> Even though storm do umentation shows quite number of companies using
>> storm, seems like that page is outdated.
>>
>> Is storm community taking any steps to increase the usage and adoption
>> again?
>>
>> Also seems like a lot of cloud providers are also not giving storm based
>> service as opposed to other streaming solutions. Is storm community doing
>> anything to have more cloud adoption for storm
>>
>
>

Re: Decline in apache storm usage

[Ethan Li]

Hi Gunjan,

I can’t speak for general usages. But the company I work for is using Apache 
Storm very heavily. 


> On Aug 28, 2019, at 9:44 AM, Gunjan Dave  wrote:
> 
> 
> Hello devs,
> Seeing that apache storms usage is declining. Do you attribute this to 
> anything?
> Even though storm do umentation shows quite number of companies using storm, 
> seems like that page is outdated. 
> 
> Is storm community taking any steps to increase the usage and adoption again?
> 
> Also seems like a lot of cloud providers are also not giving storm based 
> service as opposed to other streaming solutions. Is storm community doing 
> anything to have more cloud adoption for storm