Thanks – the solution I’m leaning towards is to deploy a load balancer with a list of the nodes in the cluster, once Apex spins up, the load balancer should be able to establish connections to the deployed operators and route data appropriately.
- Ilya Ganelin On 3/2/17, 8:34 AM, "Amol Kekre" <[email protected]> wrote: Ilya, As Thomas says, attaching a JVM to an operator is do-able, but is against the norm in a distributed cluster. A distributed OS cannot guarantee a node. It could be down or not have resources, .... ZK way or any other way to discover post deployment is the way to go. I think a webservice call through Stram to get the specifics will work too. Thks Amol E:[email protected] | M: 510-449-2606 | Twitter: @*amolhkekre* www.datatorrent.com | apex.apache.org *Join us at Apex Big Data World-San Jose <http://www.apexbigdata.com/san-jose.html>, April 4, 2017!* [image: http://www.apexbigdata.com/san-jose-register.html] <http://www.apexbigdata.com/san-jose-register.html> On Wed, Mar 1, 2017 at 8:16 PM, Thomas Weise <[email protected]> wrote: > If I understand it correctly you want to run a server in an operator, > discover its endpoint and push data to it? The preferred way of doing that > would be to announce the endpoint through a discovery mechanism (such as > ZooKeeper or a shared file) that the upstream entity can use to find the > endpoint. > > If you are looking for a way to force deploy on a specific node, then have > a look at the OperatorContext.LOCALITY_HOST attribute (and also > AffinityRulesTest). AFAIK you can use a specific host name and the > scheduler will make best effort to get a container on that host, but there > isn't a guarantee. Generally, services running on the cluster shouldn't > make assumptions about hosts and ports and use discovery instead. > > HTH, > Thomas > > > On Wed, Mar 1, 2017 at 7:53 PM, Ganelin, Ilya <[email protected] > > > wrote: > > > Hello, all – is there any way to deploy a given operator to a specific > > Node? E.g. if I’m trying to create a listener for a TCP socket that can > > then pipe data to a DAG, is there any way for the location of that > listener > > to be deterministic so an upstream entity knows what to connect to? > > > > > > > > - Ilya Ganelin > > > > [image: id:[email protected]] > > > > ------------------------------ > > > > The information contained in this e-mail is confidential and/or > > proprietary to Capital One and/or its affiliates and may only be used > > solely in performance of work or services for Capital One. The > information > > transmitted herewith is intended only for use by the individual or entity > > to which it is addressed. If the reader of this message is not the > intended > > recipient, you are hereby notified that any review, retransmission, > > dissemination, distribution, copying or other use of, or taking of any > > action in reliance upon this information is strictly prohibited. If you > > have received this communication in error, please contact the sender and > > delete the material from your computer. > > > ________________________________________________________ The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
