Sorry, that mail kind of slipped through. Where you able to figure this out in the meantime?
The idea is basically: * Mesos keeps track of a range of ports on slaves. Unused ports from this range are offered to frameworks. See http://mesos.apache.org/documentation/latest/attributes-resources/ for details * When starting a task, a framework such as Aurora can pick any port from the range offered by Mesos. * If you want to get a port assigned to your task via the mechanism described above, you will have to write a specification similar to the following one: https://github.com/apache/aurora/blob/master/docs/features/services.md#ports. And yes, the port will be unique for each instance of your job. Regards, Stephan ________________________________ From: Krish <[email protected]> Sent: Wednesday, April 6, 2016 15:48 To: Erb, Stephan Cc: [email protected] Subject: Re: Aurora DSL & Hooks Hi Stephen, Can you please elaborate on this? I didn't quite understand, and googling hasn't helped much. Is this a random port number that aurora assigns to this job? Will it be a random number again when addInstances is called? Can I also cache this port number outside the Process() definition? Does mesos guarantee that this will be an unused port number from the slave on which this task is launched? I was thinking that if all the above holds true, then I can simply pass this random port number to the process, and also as an env to docker container, and get stuff to work. -- κρισhναν On Fri, Apr 1, 2016 at 2:55 PM, Erb, Stephan <[email protected]<mailto:[email protected]>> wrote: Have you considered using Mesos to generate a unique, random port number for you? Then you don't have to resort to any collision-prevention as it is already done for you automatically. You can find details and an example in our documentation: https://github.com/apache/aurora/blob/master/docs/features/services.md#ports The auto-assigned port is then available via different ways: a) it is registered in zookeeper and external tools (like loadbalancers) can do the lookup (see https://github.com/apache/aurora/blob/master/docs/features/service-discovery.md and https://github.com/apache/aurora/blob/master/docs/additional-resources/tools.md for details) b) you can ask the scheduler API for the endpoint of your service (so IP and port) Hope this gets you started. Best Regards, Stephan ________________________________ From: Krish <[email protected]<mailto:[email protected]>> Sent: Friday, April 1, 2016 10:40 To: [email protected]<mailto:[email protected]> Subject: Aurora DSL & Hooks Hi, Currently, I have a small web server to which I pump traffic, and once this crosses a threshold, I call the addInstances API for this job. Now, the job will fail if the second instance of the job is scheduled on the same machine in the cluster, as the port number is already in use. What I was thinking to do is generate a random number (say, between 5000-6000) to bind to, which can avoid this clash. I currently pass the port number as a static parameter using the --env docker flag. A more complex use case is to add this dynamic port number to the EC2 load balancer that I use for testing stuff when the job runs, and delete the port when the job stops. I was reading about Hooks (http://aurora.apache.org/documentation/latest/hooks/), and this seems to be the way to go, but can't get my head around it. One thing is because the other documentation (http://aurora.apache.org/documentation/latest/configuration-reference/) has no reference to it. I have added a enable_hooks=True in my .aurora and in the process of defining my custom class with requisite hooks. Questions: Is it ok to write small pieces of code like random number generation, etc in the DSL? Does the hook code run in the slave or in aurora? Are the imports restricted to the hooks? Like import logging, and import aws.boto fails? Where does the hooks = [] line go in the DSL - in the jobs=[] section? Anything else that I need to know to get this done. Thanks. -- κρισhναν
