Re: Unsubscribe

[Antoine Tran]

I can see multiple attempt of unsubscribe.

To those who wants to unsubscribe from this mailing list, please follow 
the correct procedure: https://storm.apache.org/getting-help.html


For those who can't read, please send to 
user-unsubscr...@storm.apache.org to unsubscribe, not the normal mailing 
list! Thanks!


On 13/11/2020 12:15, jerome moliere wrote:

Unsubscribe

Fwd: stom nimbus HA in kubernetes

[Antoine Tran]

Hi,

I would like to deploy Storm Nimbus HA in Kubernetes. I have an 
interrogation. We have a service nimbus named "storm-nimbus", that has 
nimbus endpoints (eg nimbus1, nimbus2, nimbus3). Should we set in 
nimbus.seeds an array of one element that is the service name, and let 
kubernetes handle the health of nimbus:


nimbus.seeds: ["storm-nimbus"]

or put as the doc says, the full list of machines and let nimbus handle 
the nimbus failure?


nimbus.seeds: ["nimbus1", "nimbus2", "nimbus3"]?

Thank you.

Re: storm versions with jdk versions

[Antoine Tran]

Right now, we use openjdk version "1.8.0_121" (not the Oracle one, and 
not the JDK, only the *JRE*). It works without issue.



On 10/07/2017 16:11, Bobby Evans wrote:
NOTE that the numbers are JDK 1.X+  so that means storm 0.9.6 should 
work fine with at least version 1.6 of the JDK (that includes 1.7 and 
1.8).  Java has been very good about not breaking backwards 
compatibility so far.  This indicates the minimum version of the JDK 
needed to run storm.  NOTE java 9 supposedly breaks this, so until 
someone has thoroughly tested it with java 9 please be careful.


If you use a version that is too old you may get exceptions about the 
class files not being for the supported version, or you could get 
errors about methods missing at run time.



- Bobby



On Monday, July 10, 2017, 8:57:01 AM CDT, sam mohel 
 wrote:



Thanks for replying. But what are the problems that may happen if I use
Incompatible versions  ?

On Monday, July 10, 2017, Xin Wang > wrote:

> Hi sam,
> Storm 0.9.x -> JDK1.6+
> Storm 0.10.x -> JDK1.7+
> Storm 1.x -> JDK1.7+
> Storm 2.x -> JDK1.8+
>
> IMHO, I recommend you to use the latest 1.1.x release.
>
> Thanks,
> Xin
> 2017-07-10 21:27 GMT+08:00 sam mohel >:

>>
>> Is there any help , please?
>>
>> On Monday, July 10, 2017, sam mohel > wrote:
>> > I'm sorry for my question if it is silly , i've apache-storm 
version -0.9.6 . is that working with jdk1.8 ?? or it's not depending 
on JDK versions

>


--
My THALES email is antoine.t...@thales-services.fr.
+33 (0)5 62 88 84 40
Thales Services, Toulouse, France

Re: open storm ui very slow(Internet mail)

[Antoine Tran]

When you have this issue, can you restart storm-manager or storm-nimbus 
to see if Storm can load normally? We did one of these each time and it 
works as a workaround.



On 26/06/2017 12:12, Rui Abreu wrote:
Don't have the  logs anymore, I'm afraid. Not much information that 
could help debugging, other than it appear to be spending some time 
trying to redistribute the existent topologies among the 600 workers 
(300 *2).

I'll get back to you if I'm able to reproduce the problem.

On 25 June 2017 at 20:42, Erik Weathers > wrote:


300 is not that many supervisors from my perspective.  Have any of
you experiencing this issue dug in to see what's slowing it down?

- Erik

On Fri, Jun 23, 2017 at 1:48 AM Rui Abreu > wrote:


On 22 June 2017 at 19:35, Erik Weathers > wrote:

Sounds like you've hit some scaling bottleneck with
Storm.  I've never tried running nearly that number of
topologies.
It might be an inefficiency with the number of API calls,
or with the interactions between the Nimbus and ZooKeeper.



A similar situation occurs on Storm 1.1.0 when the number of
supervisors approaches 300 (2 workers per supervisor). Storm
UI fails to load because /api/vi/{supervisor,topology, nimbus}
calls timeout.




--
My THALES email is antoine.t...@thales-services.fr.
+33 (0)5 62 88 84 40
Thales Services, Toulouse, France

Re: DELIVERY FAILURE: Error transferring to GAPAR017/SRV/SOCGEN mail.box; Maximum hop count exceeded. Message probably in a routing loop.

[Antoine Tran]

Anyone delete this user?


On 05/05/2017 10:50, postmas...@sgcib.com wrote:

Your message

   Subject: Re: Unit Testing Spouts + Bolts

was not delivered to:

   pierre.lanvin-...@sgcib.com

because:

   Error transferring to GAPAR017/SRV/SOCGEN mail.box; Maximum hop count 
exceeded.  Message probably in a routing loop.



Reporting-MTA: dns;gapar028.fr.world.socgen

Final-Recipient: rfc822;pierre.lanvin-...@sgcib.com
Action: failed
Status: 5.0.0
Remote-MTA: x-Notes;CN=GAPAR017/OU=SRV/O=SOCGEN
Diagnostic-Code: X-Notes; Error transferring to GAPAR017/SRV/SOCGEN ma
  il.box; Maximum hop count exceeded.  Message probably in a routing loo
  p.


*
This message and any attachments (the "message") are confidential, 
intended solely for the addresse(s), and may contain legally 
privileged information.
Any unauthorized use or dissemination is prohibited. E-mails are 
susceptible to alteration.
Neither SOCIETE GENERALE nor any of its subsidiaries or affiliates 
shall be liable for the message if altered, changed or

falsified.
Please visit http://swapdisclosure.sgcib.com for important information 
with respect to derivative products.

  
Ce message et toutes les pieces jointes (ci-apres le "message") sont 
confidentiels et susceptibles de contenir des informations couvertes

par le secret professionnel.
Ce message est etabli a l'intention exclusive de ses destinataires. 
Toute utilisation ou diffusion non autorisee est interdite.

Tout message electronique est susceptible d'alteration.
La SOCIETE GENERALE et ses filiales declinent toute responsabilite au 
titre de ce message s'il a ete altere, deforme ou falsifie.
Veuillez consulter le site http://swapdisclosure.sgcib.com afin de 
recueillir d'importantes informations sur les produits derives.

*



--
My THALES email is antoine.t...@thales-services.fr.
+33 (0)5 62 88 84 40
Thales Services, Toulouse, France

Re: Storm files and folders permissions on Linux

[Antoine Tran]

Have you modified in Storm.yaml settings related to ACL? The defaults 
one disabled ACL for blob



On 03/04/2017 18:27, I PVP wrote:

Thanks for everyone that is answering and helping to look into this issue.



I tried from /tmp. Same error.
I tried submitting the jar from the ec2-user $HOME directory. Same error
I created a new directory to which storm:storm is the owner and tried 
submitting the jar  from there. Same error.



The error log is not really clear on what is going on.

Where/what is setup-path that is mentioned on the log? is it the 
/opt/storm where  storm was “installed”  ?


What is the exact "No such file or directory” that  is missing or 
inaccessible ?



First it says that :
—
2017-04-03 13:13:00.148 pool-15-thread-30 o.a.s.d.nimbus [INFO] 
Finished uploading file from client: 
/opt/storm/storm-local/nimbus/inbox/stormjar-f90ff743-e9ec-4e59-b945-878a47fd9057.j

ar
2017-04-03 13:13:00.173 pool-15-thread-32 o.a.s.d.nimbus [INFO] 
Received topology submission for Topology with conf 
{"topology.max.task.parallelism" nil, "topology
.submitter.principal" "", "topology.acker.executors" nil, 
"topology.eventlogger.executors" 0, "topology.workers" 1, 
"topology.debug" false, "storm.zookeeper.superACL" nil, "topology.us
ers" (), "topology.submitter.user" "storm", "topology.kryo.register" 
nil, "topology.kryo.decorators" (), "storm.id" 
"Topology-3-1491235980", "topology.name" "Emplo

yeePasswordResetTopology"}
2017-04-03 13:13:00.189 pool-15-thread-32 o.a.s.d.nimbus [INFO] 
uploadedJar 
/opt/storm/storm-local/nimbus/inbox/stormjar-f90ff743-e9ec-4e59-b945-878a47fd9057.jar
2017-04-03 13:13:00.189 pool-15-thread-32 
o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting

..



Before trowing the exception it says :
——
2017-04-03 13:13:00.207 pool-15-thread-32 o.a.s.cluster [INFO] 
setup-path/blobstore/Topology-3-1491235980-stormjar.jar/ip-172-31-5-153.sa-east-1.compute.internal:6

627-1
2017-04-03 13:13:00.207 pool-15-thread-32-EventThread 
o.a.s.s.o.a.z.ClientCnxn [INFO] EventThread shut down
2017-04-03 13:13:00.217 pool-15-thread-32 o.a.s.d.nimbus [WARN] 
Topology submission exception. (topology name='Topology') 
#error {

 :cause No such file or directory
 :via
 [{:type java.lang.RuntimeException
   :message java.io.IOException: No such file or directory
   :at [org.apache.storm.blobstore.LocalFsBlobStore createBlob 
LocalFsBlobStore.java 117]}

...
-



best,
IPVP

On April 3, 2017 at 12:12:02 PM, Santosh Pingale 
(pingalesant...@gmail.com ) wrote:


Can you try to submit the jar from a directory which is writable by 
storm user.  Try from /tmp maybe.


On Mon, Apr 3, 2017 at 3:56 PM, I PVP > wrote:


that is the error:


2017-04-03 10:31:44.535 pool-15-thread-56 o.a.s.d.nimbus [INFO]
Uploading file from client to

/opt/storm/storm-local/nimbus/inbox/stormjar-fceb1c76-b88e-415d-803d-ab73c0d2fafe.jar
2017-04-03 10:31:44.820 pool-15-thread-17 o.a.s.d.nimbus [INFO]
Finished uploading file from client:

/opt/storm/storm-local/nimbus/inbox/stormjar-fceb1c76-b88e-415d-803d-ab73c0d2fafe.jar
2017-04-03 10:31:44.842 pool-15-thread-20 o.a.s.d.nimbus [INFO]
Received topology submission for Topology with conf
{"topology.max.task.parallelism" nil,
"topology.submitter.principal" "", "topology.acker.executors"
nil, "topology.eventlogger.executors" 0, "topology.workers" 1,
"topology.debug" false, "storm.zookeeper.superACL" nil,
"topology.users" (), "topology.submitter.user" "storm",
"topology.kryo.register" nil, "topology.kryo.decorators" (),
"storm.id "
"Topology-2-1491226304", "topology.name
" "Topology"}
2017-04-03 10:31:44.846 pool-15-thread-20 o.a.s.d.nimbus [INFO]
uploadedJar

/opt/storm/storm-local/nimbus/inbox/stormjar-fceb1c76-b88e-415d-803d-ab73c0d2fafe.jar
2017-04-03 10:31:44.847 pool-15-thread-20
o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2017-04-03 10:31:44.848 pool-15-thread-20 o.a.s.s.o.a.z.ZooKeeper
[INFO] Initiating client connection,
connectString=172.31.5.153:2181
,172.31.3.29:2181
,172.31.0.218:2181/storm
 sessionTimeout=2
watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@13bf52ea
2017-04-03 10:31:44.851
pool-15-thread-20-SendThread(172.31.0.218:2181
) o.a.s.s.o.a.z.ClientCnxn [INFO]
Opening socket connection to server
172.31.0.218/172.31.0.218:2181
. Will not attempt to
authenticate using SASL (unknown error)
2017-04-03 10:31:44.852
pool-15-thread-20-SendThread(172.31.0.218:2181
) o.a.s.s.o.a.z.ClientCnxn [INFO]
Socket connection established to 172.31.0.218/172.31.0.218:2181

Re: Centralized logging for storm

[Antoine Tran]

How about modifying the worker.xml configuration, so that we add a 
appender to logstash/elasticsearch ? No need to add a filebeat if this 
is handled by Storm itself.



On 31/03/2017 16:44, Cody Lee wrote:


Ditto, filebeat + ELK works very well. You can even tokenize these 
logs appropriately to have a richer search/filtering.


Cody

*From: *Harsh Choudhary 
*Reply-To: *"user@storm.apache.org" 
*Date: *Friday, March 31, 2017 at 4:38 AM
*To: *"user@storm.apache.org" 
*Subject: *Re: Centralized logging for storm

Hi Shashank

What we do is, we have filebeats installed on our Storm clusters and 
they send the log files data to our central log server, Graylog. This 
tool is great and you can see your logs like they are one stream of 
messages, sorted by timestamp. One thing that really helps is that you 
can also lookup all the other logs near a timestamp.



/Cheers!/

*Harsh Choudhary*

On Fri, Mar 31, 2017 at 1:16 PM, Shashank Prasad > wrote:


Hi folks,

Storm is a great tool but the logs are all over the place. As you
increase your workers, your log files will increase as well and
there is no single file it logs to.

This makes it very hard to troubleshoot since you have to tail
multiple logs.

Ideally, i would like to ship all the logs for a topology to a
centralized log server where i could use something like Kibana and
filter the logs on what i am searching for.

Anyone has any suggestion on how to achieve this or has a use case
of how you currently doing it.

Thanks a lot for your time!

-shashank



--
My THALES email is antoine.t...@thales-services.fr.
+33 (0)5 62 88 84 40
Thales Services, Toulouse, France

Re: nimbus.thrift.max_buffer_size

[Antoine Tran]

https://issues.apache.org/jira/browse/STORM-1469


On 31/03/2017 05:37, 马庆祥 wrote:

Hello all,

I have a question about the "nimbus.thrift.max_buffer_size" in storm.yaml.
nimbus.thrift.max_buffer_size: 1048576
I want to get the cluster state and topology state by new 
NimbusClient, but i got a error:


内嵌图片 1

Thanks & Regards
Qingxiang Ma


--
My THALES email is antoine.t...@thales-services.fr.
+33 (0)5 62 88 84 40
Thales Services, Toulouse, France

Re: DELIVERY FAILURE: Error transferring to QCMBSJ601.HERMES.SI.SOCGEN; Maximum hop count exceeded. Message probably in a routing loop.

[antoine . tran]

Can someone remove this user? It probably does not exist anymore.


On 03/03/2017 10:48, postmas...@sgcib.com wrote:

Your message

   Subject: Re: Enabling internal source code logging for Storm

was not delivered to:

   pierre.lanvin-...@sgcib.com

because:

   Error transferring to QCMBSJ601.HERMES.SI.SOCGEN; Maximum hop count 
exceeded.  Message probably in a routing loop.



Reporting-MTA: dns;gapar018.fr.world.socgen

Final-Recipient: rfc822;pierre.lanvin-...@sgcib.com
Action: failed
Status: 5.0.0
Remote-MTA: smtp;QCMBSJ601.HERMES.SI.SOCGEN
Diagnostic-Code: X-Notes; Error transferring to QCMBSJ601.HERMES.SI.SO
  CGEN; Maximum hop count exceeded.  Message probably in a routing loop.


*
This message and any attachments (the "message") are confidential, 
intended solely for the addresse(s), and may contain legally 
privileged information.
Any unauthorized use or dissemination is prohibited. E-mails are 
susceptible to alteration.
Neither SOCIETE GENERALE nor any of its subsidiaries or affiliates 
shall be liable for the message if altered, changed or

falsified.
Please visit http://swapdisclosure.sgcib.com for important information 
with respect to derivative products.

  
Ce message et toutes les pieces jointes (ci-apres le "message") sont 
confidentiels et susceptibles de contenir des informations couvertes

par le secret professionnel.
Ce message est etabli a l'intention exclusive de ses destinataires. 
Toute utilisation ou diffusion non autorisee est interdite.

Tout message electronique est susceptible d'alteration.
La SOCIETE GENERALE et ses filiales declinent toute responsabilite au 
titre de ce message s'il a ete altere, deforme ou falsifie.
Veuillez consulter le site http://swapdisclosure.sgcib.com afin de 
recueillir d'importantes informations sur les produits derives.

*

Re: Enabling internal source code logging for Storm

[antoine . tran]

You can set log level in Storm UI, but you have to know the package. At 
least, you can set a root package's level like "org", "com", etc.


What we did in our project is to replace INFO to DEBUG in worker.xml.


On 03/03/2017 09:42, Sree V wrote:

DId you tried, setting LOG level on Storm UI ?

Thanking you.
With Regards
Sree


On Friday, March 3, 2017 12:10 AM, anshu shukla 
 wrote:



Hello,

I just want to know how to turn on logging for debug mode in 
storm-core. I tried updating cluster.xml and worker.xml files and set 
them to ALL  level even then there are no logs for statements like -


LOG.info("Checkpoint interval is {} millis", interval);
*also I have done - *conf.setDebug(true);
Please help !

--
Thanks & Regards,
Anshu Shukla