Re: Sending custom config values from nimbus to components

2017-01-25 Thread Joaquin Menchaca
ontainer to a nimbus in the cluster. The container encapsulates all the config, and library, vm versions required. - Joaquin Menchaca On Jan 9, 2017 10:10 AM, "Matt Lowe" <matthewneill...@gmail.com> wrote: Hello! When you deploy a topology from nimbus, you can see the topology c

Anyone use JavaScript for bolts?

2017-01-25 Thread Joaquin Menchaca
Just curious if anyone else did this. - Joaquin

Re: How could one debug the root cause of this error?

2017-01-24 Thread Joaquin Menchaca
, i.e. rm -rf. I wish that the root key would be unique or have a schema version in it, so that storm 1.x would use a diff root. Then this could be avoided. On Jan 14, 2017 12:20 PM, "Erik Weathers" <eweath...@groupon.com> wrote: > On Fri, Jan 13, 2017 at 8:56 PM, Joaqu

Re: How to correctly setup Nimbus UI to have public instead of private FQDN for debug

2017-01-24 Thread Joaquin Menchaca
You have to configure a few things: - determine set of names to configure public cname, hosts file entries or local dns cache entries, and virtualhost confis on a proxy. ex: supervisor01.mycompany.com. - reverse proxy front end, like nginx with routes back to appropriate backends based on

Re: Is there a Storm hosted/cloud solution somewhere available?

2017-01-24 Thread Joaquin Menchaca
cloudera? On Jan 23, 2017 2:44 AM, "André Martin" wrote: > Dear Storm users, > I was wondering if there is a cloud hosted storm solution somewhere > available similar to Amazon EMR for hadoop where you do not have to setup a > cluster manually yourself - you only use

Re: How to set number of Nimbus child processes

2017-01-13 Thread Joaquin Menchaca
Which java processes are they? If you do not run any services, calling storm will bring up a few java services to run a topology. If you bring up the daemons and refer to it in a storm.yaml, then you bring up the services you manually bring up. A storm supervisor will bring up 4 workers. You

Re: How could one debug the root cause of this error?

2017-01-13 Thread Joaquin Menchaca
le, can > you do so, and then restart storm and deploy your storm topology again. > > On Dec 22, 2016, at 3:22 PM, Joaquin Menchaca <jmench...@gobalto.com> > wrote: > > Found nimbuses [] none of which is elected as leader, please try again after > some time > > >

Re: storm 1.0.2 nimbus HA design doc doesn't match with code

2017-01-13 Thread Joaquin Menchaca
That doesn't surprise me, there are numerous doc bugs. If I knew how to do a PR against a tag, I would correct some of them, but my git gymnastics is not strong. On Dec 30, 2016 12:52 AM, "Renjie Liu" wrote: Hi, all: I'm reading the doc on storm 1.0.2 nimbus HA design.

How could one debug the root cause of this error?

2016-12-22 Thread Joaquin Menchaca
org.apache.storm.utils.NimbusLeaderNotFoundException: Found nimbuses [] none of which is elected as leader, please try again after some time. at org.apache.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:85) at

Re: NimbusLeaderNotFoundException problem?

2016-12-12 Thread Joaquin Menchaca
Alright, found a comment in a post, and put these in my zoo.cfg, they did the trick: autopurge.snapRetainCount = 5 autopurge.purgeInterval = 24 On Mon, Dec 12, 2016 at 4:41 PM, Joaquin Menchaca <jmench...@gobalto.com> wrote: > Diskspace. Newbiew to zk. There's 59 GB of these

Re: NimbusLeaderNotFoundException problem?

2016-12-12 Thread Joaquin Menchaca
Diskspace. Newbiew to zk. There's 59 GB of these snapshots in /var/lib/zookeeper/version-2 On Mon, Dec 12, 2016 at 4:35 PM, Mostafa Gomaa <mgo...@trendak.com> wrote: > Can you us your zookeeper logs and config? > > On Dec 13, 2016 2:32 AM, "Joaquin Menchaca" <

Re: NimbusLeaderNotFoundException problem?

2016-12-12 Thread Joaquin Menchaca
between the two machines. > > On Dec 13, 2016 1:49 AM, "Joaquin Menchaca" <jmench...@gobalto.com> wrote: > >> Storm.yaml (Storm 1.0.2): >> >> storm.zookeeper.servers: >> - "zookeeper-stormcluster.gobalto.com" >>

Re: NimbusLeaderNotFoundException problem?

2016-12-12 Thread Joaquin Menchaca
eck > out that zookeeper is running and that there aren't any networking issues > between the two machines. > > On Dec 13, 2016 1:49 AM, "Joaquin Menchaca" <jmench...@gobalto.com> wrote: > >> Storm.yaml (Storm 1.0.2): >> >> storm.zookeeper.servers: >&g

Re: NimbusLeaderNotFoundException problem?

2016-12-12 Thread Joaquin Menchaca
rg.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) at org.apache.storm.shade.org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) On Mon, Dec 12, 2016 at 3:45 PM, Mostafa Gomaa <mgo...@trendak.com> wrote: > Can you show us your config files as well as the nimbus log? > > On D

Re: NimbusLeaderNotFoundException problem?

2016-12-12 Thread Joaquin Menchaca
dClientAs(NimbusClient.java:90) at org.apache.storm.command.list$_main.invoke(list.clj:22) at clojure.lang.AFn.applyToHelper(AFn.java:152) at clojure.lang.AFn.applyTo(AFn.java:144) at org.apache.storm.command.list.main(Unknown Source) On Mon, Dec 12, 2016 at 3:39 PM, Joaquin Menchaca <jmench..

NimbusLeaderNotFoundException problem?

2016-12-12 Thread Joaquin Menchaca
*Ideas?* org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find leader nimbus from seed hosts ["nimbus-stormcluster.gobalto.com" "nimbus2-stormcluster.gobalto.com"]. Did you specify a valid list of nimbus hosts for config nimbus.seeds? at

Re: Cant connect to remote cluster

2016-11-21 Thread Joaquin Menchaca
What version of Storm are you running? What is the header information from the storm list, not just the exception. On Fri, Nov 18, 2016 at 1:27 PM, Al Hennessey wrote: > Hi, i am having an issue connecting to my remote clustered storm server, > when i try running

Zero Worker Bug in Topology on 1.0.2

2016-11-21 Thread Joaquin Menchaca
I am not sure what is causing this, but it seems that after 70 days of running the cluster, it will not be able to run a topology with any workers or tasks. I tried a sample topology, which used to work, but now it is broken? storm jar

Re: Could not find leader nimbus from seed hosts

2016-11-04 Thread Joaquin Menchaca
. In my case, I went into ZK, stop the services, rm -rf where the data store was, started ZK again, bounced the nimbus services (and supervisors), and it all worked again. - Joaquin Menchaca > On Nov 4, 2016, at 3:44 AM, Назар Кушпір <nazarkush...@gmail.com> wrote: > > I'm us

Re: Storm 1.0.2 - HELP - ignores storm.yaml uses localhost:6627

2016-11-03 Thread Joaquin Menchaca
Ignore this, this is my environment, picked up bad base image, storm 0.10.0, not storm 1.0.2. On Thu, Nov 3, 2016 at 1:38 PM, Joaquin Menchaca <jmench...@gobalto.com> wrote: > Why does storm tool seem to be stuck on only using localhost? (or is it > using localhost)? > > # s

Storm 1.0.2 - HELP - ignores storm.yaml uses localhost:6627

2016-11-03 Thread Joaquin Menchaca
Why does storm tool seem to be stuck on only using localhost? (or is it using localhost)? # storm list --config /templates/storm.yaml Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/share/storm -Dstorm.log.dir=/usr/share/storm/logs

Re: Issue with Storm 1.0.2 on docker

2016-10-13 Thread Joaquin Menchaca
I am a little fuzzy about what you are trying to to. The you expose the correct ports (Dockerfile) and map them to the host (docker command option)? 6127, 8000, 8080? Are you trying to run storm jar and storm list inside the container, or outside the container. Are you trying to do this on

Storm Cluster End-Points vs. Direct Access

2016-10-06 Thread Joaquin Menchaca
Two things I wish were possible. 1) storm client I wish that a storm client did not have to talk to every node in a cluster directly, just interfaced with one end point, and that service fetched everything as necessary. I am not sure if all the systems that one talks to is fully documented. 2)

Re: Storm Bug? Cannot connect to cluster

2016-10-06 Thread Joaquin Menchaca
Alright, ignore everything I said... I did not know storm list will need to communicate to other systems within the cluster. I was thinking that if the nimbus were documented, then that was all that would be needed. I did not know that I would need to communicate to other systems. Thus, I had

Re: Storm Bug? Cannot connect to cluster

2016-10-06 Thread Joaquin Menchaca
a bug on your part. I tested it and found that if only > nimbus.seed is set, the 'storm list' command will attempt to connect with > localhost. > > -Taylor > > On Oct 6, 2016, at 6:16 PM, Joaquin Menchaca <jmench...@gobalto.com> > wrote: > > It might be a bug on m

Re: Storm Bug? Cannot connect to cluster

2016-10-06 Thread Joaquin Menchaca
I tested it and found that if only > nimbus.seed is set, the 'storm list' command will attempt to connect with > localhost. > > -Taylor > > On Oct 6, 2016, at 6:16 PM, Joaquin Menchaca <jmench...@gobalto.com> > wrote: > > It might be a bug on my part, trying a test, of

Re: Storm Bug? Cannot connect to cluster

2016-10-06 Thread Joaquin Menchaca
storm.yaml file with ONE of your nimbus > hostnames. > > Let me know if that works, and I’ll file a JIRA to get this fixed. > > -Taylor > > > On Oct 6, 2016, at 3:31 PM, Joaquin Menchaca <jmench...@gobalto.com> > wrote: > > I don't get it, it seems to pick up t

Re: Storm Bug? Cannot connect to cluster

2016-10-06 Thread Joaquin Menchaca
he contents of the storm.yaml you are using in your commands? > > -Taylor > > > On Oct 6, 2016, at 2:45 PM, Joaquin Menchaca <jmench...@gobalto.com> > wrote: > > I have only the nimbus.seeds configured, do I need anything else > configured to get his to work? > > $

Re: Submitting Topologies to Nimbus - port 6627

2016-10-06 Thread Joaquin Menchaca
Why does storm list use localhost? On Thu, Oct 6, 2016 at 10:27 AM, Joaquin Menchaca <jmench...@gobalto.com> wrote: > Just making sure it is port 6627, correct? > > Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name= > -Dstorm.options= -Dstorm.hom

Need more than nimbus config to submit topology?

2016-10-06 Thread Joaquin Menchaca
I am running a build-config container environment to deploy topologies. I can connect to nimbus servers to 6627, but when I do simple storm list, it stack traces. I have only the *nimbus.seeds *configured. Do zookeepers need to be configured in *storm.yaml*? What is the minimum necessary

Storm Bug? Cannot connect to cluster

2016-10-06 Thread Joaquin Menchaca
I have only the nimbus.seeds configured, do I need anything else configured to get his to work? $ storm --config /templates/storm.yaml localconfvalue nimbus.seeds nimbus.seeds: [ip-10-110-20-7.us-west-2.compute.internal ip-10-110-20-146.us-west-2.compute.internal] $ nc -vz

Submitting Topologies to Nimbus - port 6627

2016-10-06 Thread Joaquin Menchaca
Just making sure it is port 6627, correct? Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/share/storm -Dstorm.log.dir=/usr/share/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp

Re: Submitting Topology Questions (2 small questions)

2016-09-22 Thread Joaquin Menchaca
remote* On Wed, Sep 21, 2016 at 11:44 PM, Ravi Sharma <ping2r...@gmail.com> wrote: > From this documentation : http://storm.apache.org/releases/1.0.2/flux.html > > storm jar mytopology.jar org.apache.storm.flux.Flux --local config.yaml > > > > On Thu, Sep 22, 2016 at 2:50 A

Submitting Topology Questions (2 small questions)

2016-09-21 Thread Joaquin Menchaca
What is the minimal storm.yaml configuration do I need for `storm jar ... remote`? Is there a command line option or way to specify locally crafted storm.yaml, e.g. /tmp/storm.yaml? -- 是故勝兵先勝而後求戰,敗兵先戰而後求勝。

Metrics Instrumentation

2016-09-20 Thread Joaquin Menchaca
-internal-actions.html) - Joaquin Menchaca -- 是故勝兵先勝而後求戰,敗兵先戰而後求勝。

Re: Cannot submit topology in local mode on Storm 1.0.1

2016-09-20 Thread Joaquin Menchaca
What happens if you run it in single / local mode on a supervisor? On Tue, Sep 20, 2016 at 1:37 AM, Jungtaek Lim wrote: > Hi Chen, > > Could you try running your topology with Storm 1.0.2 in local mode? Since > it's in local mode you can easily try it out. > > Thanks, >

Storm 1.0.2 Monitoring Docs

2016-09-20 Thread Joaquin Menchaca
http://storm.apache.org/releases/1.0.2/storm-metrics-profiling-internal-actions.html Not Found The requested URL /releases/1.0.2/storm-metrics-profiling-internal-actions.html was not found on this server. -- 是故勝兵先勝而後求戰,敗兵先戰而後求勝。

Re: [SURVEY] Which external modules (connector) do you use actively?

2016-09-16 Thread Joaquin Menchaca
I did not know about Storm SQL. We use NodeJS for spouts that pull from PostgreSQL. We pull from Rails ActiveRecord database and transform it through Storm to a Star Schema created by Sequelize. On Tue, Sep 13, 2016 at 1:57 AM, Jungtaek Lim wrote: > Hi users, > > We're

Re: Storm Integration Tests

2016-09-16 Thread Joaquin Menchaca
Wow, Noice. On Thu, Sep 15, 2016 at 1:13 AM, Ravi Sharma wrote: > Hi Guys, > Recently i have written a small framework for integration tests(including > flux yaml file), thought of sharing with you all. May be it can help > someone. > >

What environment do you use to run your cluster?

2016-09-11 Thread Joaquin Menchaca
are not using containers yet, but will look to using Docker/Ansible, and later Kubernetes. I am curious, as I write automation scripts to help the community, I want to cover use cases that may be useful for installing, configuring, and documenting how to install and manage an Apache Storm cluster. - Joaquin

Vagrant Scripts Updated for Storm 1.0.2

2016-09-11 Thread Joaquin Menchaca
Hi all. So with the command *vagrant up*, you can bring up 1.0.2 cluster now. Also, 0.10.1 and 0.9.7 are supported. Repo at: https://github.com/darkn3rd/storm-vagrant Or other options, # 1 nimbus, 2 supervisor, 1 zookeeper STORM_VERSION="0.9.7" *vagrant up* STORM_VERSION="0.10.1" *vagrant up*

Re: Fwd: [meetup-group-aYOqisgB] [SURVEY] What version of Storm are you using?

2016-09-11 Thread Joaquin Menchaca
I migrated my company to 1.0.2 from 0.10.0. We only use 1.0.2 now. On Sat, Sep 10, 2016 at 10:52 AM, James Tyack wrote: > Andrew, are you also using Kafka and if so, which version please? Thanks > > On Sat, Sep 10, 2016 at 10:35 AM, Andrew Montalenti

Re: Storm 1.0.2 errors being reported coming from Nimbus instead of worker hosts

2016-09-09 Thread Joaquin Menchaca
It sounds like you are running the topology in local mode, rather than remote mode. On Fri, Sep 9, 2016 at 6:20 AM, Stephen Powis wrote: > Hey! > > We've recently been piloting storm 1.0.2 in a pre-production environment > and have noticed something odd. Whenever an

Re: Nimbus Leader Issue - 1 nimbus ok?

2016-09-09 Thread Joaquin Menchaca
out, worked). On Fri, Sep 9, 2016 at 4:30 AM, Alberto São Marcos <alberto@gmail.com> wrote: > Can you run a cluster with one nimbus? Yes you can. > Check your network. > > On Thu, Sep 8, 2016 at 8:29 PM, Joaquin Menchaca <jmench...@gobalto.com> > wrote: > >

Painful Experience From Storm 0.10 to Storm 1.0.2

2016-09-08 Thread Joaquin Menchaca
Summary, a lot of the issue I was having were due to Storm 0.10.0 zookeeper key/values existing, when trying to bring up Storm 1.0.2 using the same zookeeper and same server hostnames (essentially and in place upgrade). Solution was to stop zookeeper, blast out all the data, then start zookeeper

Storm 1.0.2 Exceptions - nimbus initialization issues - no nimbus leader participant

2016-09-08 Thread Joaquin Menchaca
I'm not sure how to get past this. I have 2 nimbus, 1 zookeeper, 3 supervisors. Both nimbus say something like: 2016-09-08 23:39:36.778 o.a.s.d.nimbus [INFO] not a leader, skipping assignments 2016-09-08 23:39:36.778 o.a.s.d.nimbus [INFO] not a leader, skipping cleanup I noticed I had

Nimbus Leader Issue - 1 nimbus ok?

2016-09-08 Thread Joaquin Menchaca
Can you run a cluster with one nimbus? I am stuck at the moment, maybe doing something dumb. I am getting: org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find leader nimbus from seed hosts ["nimbus-stormcluster.gobalto.com"]. Did you specify a valid list of nimbus hosts for

Storm 0.10.0 to 1.0.2: storm.local.dir

2016-09-08 Thread Joaquin Menchaca
Should this directly be cleaned out before moving the cluster from 0.10.0 to 1.0.2? -- 是故勝兵先勝而後求戰,敗兵先戰而後求勝。

Netty Configurations in 1.0.2 still valid?

2016-09-08 Thread Joaquin Menchaca
I forgot where I got this from, but in Storm 0.10.0, I used to do this: # netty transport storm.messaging.transport: "backtype.storm.messaging.netty.Context" storm.messaging.netty.buffer_size: 16384 storm.messaging.netty.max_retries: 10 storm.messaging.netty.min_wait_ms: 1000

Re: Storm 1.0.2 crashes with storm-starter

2016-09-08 Thread Joaquin Menchaca
Ultimately, this was just a memory issue, move the VM up to 1.5gb, fixed issue. On Wed, Sep 7, 2016 at 5:07 PM, Joaquin Menchaca <jmench...@gobalto.com> wrote: > I just realized, that the location of the storm starter was changed. But > still getting the similar error: >

Re: Storm 1.0.2 crashes with storm-starter

2016-09-07 Thread Joaquin Menchaca
ib/apache/storm/1.0.2/bin/storm.py", line 766, in main (COMMANDS.get(COMMAND, unknown_command))(*ARGS) File "/usr/lib/apache/storm/1.0.2/bin/storm.py", line 248, in jar os.remove(tmpjar) OSError: [Errno 2] No such file or directory: '/tmp/e080074c755711e6b19b0800278493db.jar'

Storm 1.0.2 crashes with storm-starter

2016-09-07 Thread Joaquin Menchaca
I was able to run storm 0.10.0, but now with 1.0.2 it crashes (below). It it because of memory a real issue, as did not have this problem with storm 0.10.0? It seems to be looking for a org.apache.storm.utils, but this directory does not exist in the JAR file. There's only: ./org/apache/storm

Re: Storm UI generated logviewer links problematic

2016-09-07 Thread Joaquin Menchaca
lor > > > On Sep 6, 2016, at 2:47 PM, Joaquin Menchaca <jmench...@gobalto.com> > wrote: > > This is a big issue to how *storm ui* generates the URLs to access *storm > logviewer*. I need to have some control to how storm ui generates the > links, such as using ro

doc confusion - nimbus.seed config

2016-09-07 Thread Joaquin Menchaca
The docs say: nimbus.seeds: ["111.222.333.44"] Does this mean we can: nimbus.seeds: ["192.168.51.5","192.168.51.6"] Where each seed refers to a nimbus server? -- 是故勝兵先勝而後求戰,敗兵先戰而後求勝。

Re: [SURVEY] What version of Storm are you using?

2016-09-06 Thread Joaquin Menchaca
this issue, "blocked by us dealing with mysterious Storm >>> Worker process heartbeat failures", do you have any topo's @ Groupon using >>> ShellSpout (multi-lang)? >>> >>> If so, the patch/resolution in STORM-1928 (released in 1.0.2, see JIRA >

Enhancement Request

2016-09-06 Thread Joaquin Menchaca
How can I make an enhancement request. I wish there was some way to have some configuration ability on storm ui, such that I can make a map between the storm logviewer URLs to the hostnames. For example, I could take a bunch of AWS internal names, and map them to /supervisor1 and /supervsior2

Storm UI generated logviewer links problematic

2016-09-06 Thread Joaquin Menchaca
This is a big issue to how *storm ui* generates the URLs to access *storm logviewer*. I need to have some control to how storm ui generates the links, such as using routes or custom name to represent the supervisors. In my *use case*: 1) secure VPC, only accessible through 22 from admin VPC, 2)

State Management - Config Files in Jar, Outside Jar? Suggestions

2016-08-17 Thread Joaquin Menchaca
Hi. I was looking for ideas for state management of configuration for data sources and tenants. We pull for customer databases, may have dev/prod/etc environments, and JARs represents version of data source (15.4 vs. 16.3 schemas). Currently, they are storing data sources in a config file in

Re: [SURVEY] What version of Storm are you using?

2016-08-17 Thread Joaquin Menchaca
0.10.0

How can I debug storm.py

2016-08-02 Thread Joaquin Menchaca
I wanted to learn learn how storm.py bootstraps the environment to understand Apache Storm better and trace down a bug. Anyhow know of good docs, tutorials on pydevd? I was trying to use PyCharm on my Mac, storm on nimbus runs on virtualbox (192.168.54.4), but unsure how to setup this session,

Re: Is there a reason storm has a default of 4 ports?

2016-07-22 Thread Joaquin Menchaca
ul 20, 2016 at 7:22 AM, Navin Ipe < >> navin@searchlighthealth.com> wrote: >> >> From what I know, you can figure out how much memory your application >> needs and allocate the memory as shown below. The below code allocates 2GiB >> of memory for each worker of the t

Re: Is there a reason storm has a default of 4 ports?

2016-07-19 Thread Joaquin Menchaca
How can one calculate how much memory is needed? On Tue, Jul 19, 2016 at 1:05 AM, Navin Ipe wrote: > I've figured out the answer to this. A slot is used by a worker. A worker > is a JVM. So each JVM would require a clump of heap memory of its own. So a > default

Apache Storm Doc Bugs

2016-07-19 Thread Joaquin Menchaca
How can I write documentation bugs from the website. -- 是故勝兵先勝而後求戰,敗兵先戰而後求勝。

What is storm_env.ini?

2016-07-18 Thread Joaquin Menchaca
Using Storm 0.10.0. I did not see docs on this in http://storm.apache.org/releases/0.10.0/Setting-up-a-Storm-cluster.html, but I see it in the python bootstrap code. I was wondering what this is? - Joaquin -- 是故勝兵先勝而後求戰,敗兵先戰而後求勝。

Storm 0.10.0 "storm nimbus" ignores storm.yaml configuration?

2016-07-18 Thread Joaquin Menchaca
This may be obvious, but just cannot see it now. I launch my nimbus process using */usr/local/bin/storm nimbus* This is a symlink to */usr/lib/apache/storm/0.10.0/bin/storm* My *storm.yaml* is in */usr/lib/apache/storm/0.10.0/conf* I see that it list it has one parameter as *-Dstorm.conf.file

Storm 0.10.0 - Zookeeper Connection ignoring configuration?

2016-07-18 Thread Joaquin Menchaca
I'm trying to understand the Zk interaction, is it trying to really run on 127.0.0.1? 3959 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x155fffd30df0004 closed 3959 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs [1000] the maxSleepTimeMs [3] the maxRetries

Need help with Unknown Error, not sure

2016-07-18 Thread Joaquin Menchaca
Hi. Anyone have any suggestions how to debug this and find out what is happening? Any troubleshooting tools I can use to test the functionality of the storm cluster? 3899 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 3908 [main] INFO

Re: Set Number of supervisors

2016-07-11 Thread Joaquin Menchaca
I'm guessing what you are doing is you have the standard ports for supervisor running. When you launch supervisor again, it will not be able to bind on those same ports, so it'll just quit. The UI will only report one, as the second supervisor cannot bind to the ports already in use. To really

Memory Alloc Requirement Suggestions?

2016-07-11 Thread Joaquin Menchaca
Is there a best practices for this, not a Java buff. I was wondering how can I analyze what exactly I need for my topology. Right now I am getting this, for nimbus with "Xmx768m": when I run my topology, I get Java HotSpot(TM) 64-Bit Server VM warning: INFO:

Re: Setup Apache Storm for single machine

2016-06-22 Thread Joaquin Menchaca
I made a sample storm repo on 0.10.0: https://github.com/darkn3rd/storm-vagrant There's some docs in there that might be useful. For writing code, don't know. On Wed, Jun 22, 2016 at 3:05 AM, Walid Aljoby wrote: > Hi all, > > I have installed single node Apache storm

Re: Un excellent outil pour votre smartphone android

2016-05-03 Thread Joaquin Menchaca
C'est spam.

Re: Configuring Python environment

2016-05-02 Thread Joaquin Menchaca
virtualenvwrappers - http://virtualenvwrapper.readthedocs.io/en/latest/ - https://pypi.python.org/pypi/virtualenvwrapper/ Pyenv - https://amaral.northwestern.edu/resources/guides/pyenv-tutorial - https://github.com/yyuu/pyenv Docker You can have a Python in another directory, then put that

Storm-Vagrant for 0.10.0

2016-05-02 Thread Joaquin Menchaca
I was able to get a successful setup for Apache Storm 0.10.0 using Vagrant. Thanks for the help. I used some of the same scripts to build out one in AWS as well. - https://github.com/darkn3rd/storm-vagrant -- 是故勝兵先勝而後求戰,敗兵先戰而後求勝。

Re: UI showing no topologies

2016-05-02 Thread Joaquin Menchaca
Ignore the last part. There was a permission problem on local.dir, no idea how ownership changed on that... weird. On Mon, May 2, 2016 at 2:33 AM, Joaquin Menchaca <jmench...@gobalto.com> wrote: > I was doing originally: > > $ storm jar storm-sta

Re: UI showing no topologies

2016-05-02 Thread Joaquin Menchaca
ubmitJarAs(StormSubmitter.java:370) ... 5 more On Mon, May 2, 2016 at 2:25 AM, Matthew Lowe <githubpoi...@gmail.com> wrote: > In your topology code, what call are you using to deploy to storm? > > Best Regards > Matthew Lowe > >> On 02 May 2016, at 10:43, Joaquin M

Any Docs on Topologies (more than Readme)

2016-05-02 Thread Joaquin Menchaca
The readme mentions options, for things like parameters of "production-topology remote" for RollingTopWords. But the information is sparse, there's little to no information on these topologies. Are there any docs anywhere on them, some of their options, etc.? Other than going through the code?

Re: Cannot launch Supervisor, missing unknown file

2016-05-02 Thread Joaquin Menchaca
you know what precise package version of Python you were running before > the upgrade? Would be nice to be able to look up the exception backtrack > lines of code. > > - Erik > > > On Sunday, May 1, 2016, Joaquin Menchaca <jmench...@gobalto.com> wrote: >

Re: Cannot launch Supervisor, missing unknown file

2016-05-01 Thread Joaquin Menchaca
I have no idea why this was happening. It was on an Ubuntu 14.04 image, but after I did an apt-get upgrade, everything worked. I was curious if anyone came across this, but it not, leave it as some bizarre anomaly. On Fri, Apr 29, 2016 at 4:59 PM, Joaquin Menchaca <jmench...@gobalto.com>

Re: Storm Cluster Docs: nimbus.seeds vs. nimbus.host

2016-05-01 Thread Joaquin Menchaca
c...@gmail.com> wrote: > >> Joaquin, >> I can understand. It would be great if you can file JIRAs for any such >> issues in the documentation so that they can be tracked and fixed. >> like below for example >> https://issues.apache.org/jira/browse/STORM-1752

Good Topology for high utilization?

2016-04-30 Thread Joaquin Menchaca
Is there a good topology that will stress a multi-node cluster to run through workers across 3 supervisors? -- 是故勝兵先勝而後求戰,敗兵先戰而後求勝。

Not seeing supervisor process listening on TCP ports - only UDP port?

2016-04-30 Thread Joaquin Menchaca
I was wondering what ports I should open up with the AWS security group for Apache Storm 0.10.0. I opened up the typical ports: TCP 6700, 6701, 6702, 6703 But when when I check on the supervisor process, I only see it listening on UDP 57944. $ sudo netstat -tuap | grep $(ps ax | grep

Apache Storm Docs - 0.10.0 says JDK6

2016-04-29 Thread Joaquin Menchaca
I noticed that the Apache Storm Docs online, and also in the READMEs still says JDK6, when it should be JDK7 according to prior email thread. - https://github.com/apache/storm/tree/v0.10.0/examples/storm-starter - http://storm.apache.org/releases/0.10.0/Setting-up-a-Storm-cluster.html --

storm 0.10.0 ui showing anything

2016-04-29 Thread Joaquin Menchaca
I get nothing with *curl 127.0.0.1:8080 * How could I troubleshoot further? HTTP/1.1 302 Found Date: Sat, 30 Apr 2016 03:10:17 GMT Location: /index.html Content-Length: 0 Server: Jetty(7.x.y-SNAPSHOT) -- 是故勝兵先勝而後求戰,敗兵先戰而後求勝。

Cannot launch Supervisor, missing unknown file

2016-04-29 Thread Joaquin Menchaca
And it says missing file. This is from the 0.10.0 tarball. I tried a *bin/storm supervisor &* $ Traceback (most recent call last): File "bin/storm.py", line 568, in main() File "bin/storm.py", line 565, in main (COMMANDS.get(COMMAND, unknown_command))(*ARGS) File "bin/storm.py",

Re: docs on storm starter frustrating, but success

2016-04-29 Thread Joaquin Menchaca
on a macbook, and deployable to a red hat machine where > on aws or azure. > > Please advise. > > Henry > On Apr 29, 2016 1:01 PM, "Joaquin Menchaca" <jmench...@gobalto.com> wrote: > > I looked through the all of the docs, and not one tells explicitly how to &

Re: docs on storm starter frustrating, but success

2016-04-29 Thread Joaquin Menchaca
bhishc...@gmail.com> wrote: > Documentation for storm 0.10 is available here - > http://storm.apache.org/releases/0.10.0/index.html > > On Wed, Apr 27, 2016 at 4:11 AM, Joaquin Menchaca <jmench...@gobalto.com> > wrote: > >> Ignore the docs. I looked at the final

Re: Any trick to getting example storm starter to work?

2016-04-27 Thread Joaquin Menchaca
ng the code ? what is storm version >> you installed? >> >> >> >> *Best regards,* >> >> *K.Sai Dilip Reddy.* >> >> On Wed, Apr 27, 2016 at 3:00 AM, Joaquin Menchaca <jmench...@gobalto.com> >> wrote: >> >>> I was able

docs on storm starter frustrating, but success

2016-04-26 Thread Joaquin Menchaca
Ignore the docs. I looked at the final packaged jar from mvn clean install, noticed the path was different. I was able to at least get ExlamationTopology to work. tar -xvf apache-storm-0.10.0.tar.gz cd apache-storm-0.10.0/examples/storm-starter mvn clean install -DskipTests=true storm jar

Any trick to getting example storm starter to work?

2016-04-26 Thread Joaquin Menchaca
I was able to successfully compile (mvn clean install) storm starter examples from the 0.10.0 (i.e. http://www.apache.org/dyn/closer.lua/storm/apache-storm-0.10.0/apache-storm-0.10.0.zip ) I was wondering if there's something else I can do to run these, something else I need to compile? I

Building Starter Code Broken w/ 0.10.1-SNAPSHOT

2016-04-25 Thread Joaquin Menchaca
How could I build this? I am using "*0.10.x-branch*": https://github.com/apache/storm/tree/0.10.x-branch. *cd storm-starter*, do an *mvn clean install*, and get errors. It looks like 0.10.1-SNAPSHOT from https://clojars.org/repo/ and http://repository.apache.org/snapshots, but the snapshot

Re: Newbie Questions for Cluster Dependencies

2016-04-25 Thread Joaquin Menchaca
ker. > https://github.com/wurstmeister/storm-docker > > http://blog.baqend.com/post/142795871760/tutorial-deploying-apache-storm-on-docker-swarm > > Hope this helps. > > Thanks, > Jungtaek Lim (HeartSaVioR) > > 2016년 4월 23일 (토) 오후 1:22, Joaquin Menchaca <jmench...@gobalto.c

Newbie Questions for Cluster Dependencies

2016-04-22 Thread Joaquin Menchaca
port using Environment variables for items configured in storm.yaml? I had hope to maintain state of systems outside of the container, so that I can dynamically manage the cluster as a set. Thank you, Joaquin Menchaca -- 是故勝兵先勝而後求戰,敗兵先戰而後求勝。