Would it make sense to deprecate stream locality in favor of this? On Tue, Feb 23, 2016 at 6:05 PM, Isha Arkatkar <[email protected]> wrote:
> Hi Pramod, > > We can have a list or operators or regex to specify operators list for > affinity rule. Though, internal implementation will translate both these > into list of pairs. Since, having pair of operators helps in validation > phase. > > And yes, validation phase will catch conflicting locality or affinity > rules and throw validation exception. > > Thanks for input regarding unifiers. It is probably alright to not include > it in the affinity settings. Will leave the current implementation as is. > > Thanks! > Isha > > > On Tue, Feb 23, 2016 at 5:56 PM, Pramod Immaneni <[email protected]> > wrote: > > > Isha, > > > > If the implementation would support (eventually) affinity between more > than > > 2 logical operators, instead of relying on specifying it as multiple > rules > > why not make provision in the API to specify more than 2 logical > operators? > > If there is a stream locality rule that conflicts with the affinity rule > > will the validation catch this? With unifiers the common case of MxN > > locality is already chosen based on downstream operator. In other cases > > such as single downstream or cascade case I am not sure it is all the > > important to have it. > > > > Thanks > > > > On Tue, Feb 23, 2016 at 5:37 PM, Isha Arkatkar <[email protected]> > > wrote: > > > > > Hi all, > > > > > > I have opened a review only pull request for handling affinity > rules > > in > > > Apex: > > > https://github.com/apache/incubator-apex-core/pull/234 > > > > > > Just wanted to confirm with everyone, if the APIs for specifying > > affinity > > > rules look alright. Also, wanted to get views on how unifiers should be > > > handled in case of operator affinity rules. More details follow: > > > > > > I have added a list of affinity rules as an attribute in Dag context. > > For > > > now, have added AffinityRule to be specified for operator pairs. We can > > add > > > regex support in next iteration. > > > > > > Here is sample usage: > > > > > > List<AffinityRule> rules = new ArrayList<>(); > > > // To add node locality between 2 not connected operators rand and > > > console: > > > rules.add(new AffinityRule(Type.AFFINITY, new OperatorPair( "rand", > > > "console"), Locality.NODE_LOCAL, false)); > > > > > > // To add anti-affinity between all partitions of rand operator with > > all > > > partitions of passThru operator: > > > rules.add(new AffinityRule(Type.ANTI_AFFINITY, new OperatorPair( > > "rand", > > > "passThru"), Locality.NODE_LOCAL, false)); > > > > > > // To add anti-affinity between partitions of passThru operator, > give > > > same operator name in pair: > > > rules.add(new AffinityRule(Type.ANTI_AFFINITY, new OperatorPair( > > > "passThru", "passThru"), Locality.NODE_LOCAL, false)); > > > > > > // Set the rules in dag context > > > dag.setAttribute(DAGContext.AFFINITY_RULES, rules); > > > > > > > > > Please find a sample application using these affinity rules here: > > > https://github.com/ishark/Apex-Samples/tree/master/affinity-example > > > > > > > > > The actual implementation of affinity rules heavily depends node > specific > > > requests, that was already being handled in StreamingAppMasterService. > To > > > handle node requests for cloudera, I have added an override to > blacklist > > > all other nodes except the ones where container request is to be > issued. > > > > > > There is one open question I wanted to bring up: In case of partitioned > > > operators, should the unifiers also follow affinity or anti-affinity > > rules? > > > Or should they be independent. For now, I have kept them independent > and > > > only the actual operators follow affinity rules. Please suggest what > > would > > > make more sense from user's specification perspective. > > > > > > Thanks, > > > Isha > > > > > > > > > > > > > > > On Mon, Feb 1, 2016 at 2:12 PM, Isha Arkatkar <[email protected]> > > > wrote: > > > > > > > Hi folks, > > > > > > > > Summarizing the proposal for affinity/anti-affinity rules in apex > as > > > > per discussions in this mail thread. Please suggest if I missed > > > something. > > > > > > > > * For configuration:* > > > > > > > > - We will have application level affinity/anti-affinity rules. For > > the > > > > first iteration, we can support for (anti-)affinity among operators > > > within > > > > single application. We can revisit it in the next iteration to > support > > > > across applications. > > > > > > > > - Each rule will consist of 4 components: > > > > <List of operators>, AFFINITY/ANTI-AFFINITY, STRICT/RELAXED > > > policy, > > > > CONTAINER/NODE/RACK > > > > I have checked that Apex supports container locality between 2 > > operators > > > > that are not connected by Steam. Though did not find an API for > setting > > > it > > > > through application. So, affinity rules can cover container affinity > as > > > > well. > > > > We will not support THREAD locality in app level affinity or > > > > anti-affinity, since it does not apply to unconnected operators. > > > > > > > > The list of operators can also be specified by regex > > > > > > > > *For Implementation: * > > > > > > > > - I have verified that Node specific requests are honored by > > > HortonWorks > > > > for both Fair and Capacity scheduler. So we can go with the approach > > of > > > > node specific container requests to implement affinity rules. > > > > > > > > - However, we may need to fall back on blacklist approach for CDH > > > distro, > > > > as node specific requests may not work well. As: > > > > -> With Fair scheduler, node specific requests are not > honored > > > in > > > > CDH 4. Application master keeps sending requests to Yarn, but no > > > containers > > > > are allocated. > > > > -> I could not yet verify the same with Fair and Capacity > > > > Scheduler on CDH 5. Ran into some configuration issues when setting > up > > > CDH > > > > 5.5.1. Will update the thread with findings. > > > > > > > > I will try to build a prototype for affinity rules with these > things > > in > > > > mind. Please let me know if anyone would like to collaborate on this > > > > effort! :) > > > > > > > > Thanks! > > > > Isha > > > > > > > > On Mon, Jan 25, 2016 at 10:23 AM, Vlad Rozov < > [email protected]> > > > > wrote: > > > > > > > >> Yes, with ability to refer not only to a host name, but to operators > > > >> (possibly both in the same and other applications). In general this > > > should > > > >> allow to specify both affinity and anti affinity using expressions. > > For > > > >> example LOCALITY_CONTAINER = "{app1.op1 or app1.op2} and > {!app2.op1}". > > > >> > > > >> > > > >> On 1/25/16 09:39, Thomas Weise wrote: > > > >> > > > >>> Agreed. Originally I thought you wanted both operators on the same > > > host. > > > >>> To > > > >>> pin them to the same container the same idea can be applied, > through > > a > > > >>> "LOCALITY_CONTAINER" attribute. > > > >>> > > > >>> > > > >>> On Mon, Jan 25, 2016 at 9:18 AM, Vlad Rozov < > [email protected] > > > > > > >>> wrote: > > > >>> > > > >>> Works as a workaround and requires two extra ports that operators > > > >>>> designer > > > >>>> may not necessarily provide out of the box. We may apply the same > > hack > > > >>>> to > > > >>>> anti-affinity and request that any two operators with > anti-affinity > > > rule > > > >>>> are connected by non functional stream, but it does not look like > a > > > good > > > >>>> option :). > > > >>>> > > > >>>> > > > >>>> > > > >>>> On 1/25/16 08:52, Thomas Weise wrote: > > > >>>> > > > >>>> Sandeep suggested to connect them with a stream. > > > >>>>> > > > >>>>> On Mon, Jan 25, 2016 at 8:49 AM, Vlad Rozov < > > [email protected] > > > > > > > >>>>> wrote: > > > >>>>> > > > >>>>> Sorry, possibly I miss something. X and Z are not directly > > connected > > > >>>>> by a > > > >>>>> > > > >>>>>> stream, how can I apply THREAD_LOCAL? > > > >>>>>> > > > >>>>>> > > > >>>>>> On 1/25/16 08:44, Thomas Weise wrote: > > > >>>>>> > > > >>>>>> Why does THREAD_LOCAL not work? > > > >>>>>> > > > >>>>>>> On Mon, Jan 25, 2016 at 7:43 AM, Vlad Rozov < > > > [email protected] > > > >>>>>>> > > > > >>>>>>> wrote: > > > >>>>>>> > > > >>>>>>> Neither LOCALITY_HOST or THREAD_LOCAL provides required > > > >>>>>>> functionality. > > > >>>>>>> The > > > >>>>>>> > > > >>>>>>> goal is to deploy X and Z to the same container/JVM. X may be > > > >>>>>>>> deployed > > > >>>>>>>> on > > > >>>>>>>> any node. Z needs to be deployed on the same container as X, > so > > X > > > >>>>>>>> does > > > >>>>>>>> not > > > >>>>>>>> have any host affinity, and Z has affinity to X, not to any > > > specific > > > >>>>>>>> host. > > > >>>>>>>> If database client caches writes, reading inside the same JVM > > will > > > >>>>>>>> be > > > >>>>>>>> in > > > >>>>>>>> process lookup, while deploying Z to any other host/JVM leads > to > > > >>>>>>>> inter > > > >>>>>>>> process or inter host lookup. > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> On 1/24/16 21:03, Thomas Weise wrote: > > > >>>>>>>> > > > >>>>>>>> There are the attributes LOCALITY_HOST and LOCALITY_RACK for > an > > > >>>>>>>> operator > > > >>>>>>>> > > > >>>>>>>> to > > > >>>>>>>>> achieve precisely what you are describing. The value would be > > an > > > >>>>>>>>> alias > > > >>>>>>>>> that > > > >>>>>>>>> can be mapped to physical hosts for allocation. The same > could > > be > > > >>>>>>>>> used > > > >>>>>>>>> of > > > >>>>>>>>> anti-affinity, by assigning different values. > > > >>>>>>>>> > > > >>>>>>>>> Stream locality is a special case of expressing affinity for > 2 > > > >>>>>>>>> operators. > > > >>>>>>>>> Note that we also need a way to extend this to partitions of > > the > > > >>>>>>>>> same > > > >>>>>>>>> operator. > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> On Sun, Jan 24, 2016 at 8:52 PM, Vlad Rozov < > > > >>>>>>>>> [email protected]> > > > >>>>>>>>> wrote: > > > >>>>>>>>> > > > >>>>>>>>> It may be quite extreme use case, but suppose that operator X > > > >>>>>>>>> writes > > > >>>>>>>>> to > > > >>>>>>>>> a > > > >>>>>>>>> > > > >>>>>>>>> distributed in-memory database and operator Z reads from it. > Z > > is > > > >>>>>>>>> not > > > >>>>>>>>> > > > >>>>>>>>>> directly connected to X. In such cases it may be necessary > to > > > let > > > >>>>>>>>>> an > > > >>>>>>>>>> application request container affinity (deploying to the > same > > > JVM) > > > >>>>>>>>>> for > > > >>>>>>>>>> operators X and Z as writes may be cached and reads from the > > > same > > > >>>>>>>>>> JVM > > > >>>>>>>>>> will > > > >>>>>>>>>> be potentially faster. > > > >>>>>>>>>> > > > >>>>>>>>>> Technically the same may be applied to thread local on NUMA > > > >>>>>>>>>> boxes, as > > > >>>>>>>>>> NUMA > > > >>>>>>>>>> aware deployment of containers is not supported. > > > >>>>>>>>>> > > > >>>>>>>>>> Vlad > > > >>>>>>>>>> > > > >>>>>>>>>> On 1/23/16 18:01, Yogi Devendra wrote: > > > >>>>>>>>>> > > > >>>>>>>>>> @Isha > > > >>>>>>>>>> > > > >>>>>>>>>> In my opinion: > > > >>>>>>>>>> > > > >>>>>>>>>>> THREAD_LOCAL, CONTAINER_LOCAL on stream is a special case > of > > > >>>>>>>>>>> generic > > > >>>>>>>>>>> rules > > > >>>>>>>>>>> for Operator X and Operator Y. > > > >>>>>>>>>>> > > > >>>>>>>>>>> We can say that, THREAD_LOCAL, CONTAINER_LOCAL would be > > > >>>>>>>>>>> applicable > > > >>>>>>>>>>> only > > > >>>>>>>>>>> if > > > >>>>>>>>>>> operator X and Y are connected by stream. But, way to > express > > > >>>>>>>>>>> this > > > >>>>>>>>>>> should > > > >>>>>>>>>>> be similar to other rules for affinity. > > > >>>>>>>>>>> > > > >>>>>>>>>>> ~ Yogi > > > >>>>>>>>>>> > > > >>>>>>>>>>> On 24 January 2016 at 03:49, Isha Arkatkar < > > > [email protected] > > > >>>>>>>>>>> > > > > >>>>>>>>>>> wrote: > > > >>>>>>>>>>> > > > >>>>>>>>>>> Hey Chinmay, > > > >>>>>>>>>>> > > > >>>>>>>>>>> I certainly agree on common set of rules for > > > configuring > > > >>>>>>>>>>> > > > >>>>>>>>>>> affinity! > > > >>>>>>>>>>>> Well > > > >>>>>>>>>>>> put by a concrete example. :) > > > >>>>>>>>>>>> Only thing I would like to point is: affinity of > > > >>>>>>>>>>>> operators > > > >>>>>>>>>>>> should > > > >>>>>>>>>>>> not > > > >>>>>>>>>>>> cover thread and container locality. Since this is only > > > >>>>>>>>>>>> applicable > > > >>>>>>>>>>>> when > > > >>>>>>>>>>>> operators are connected by stream. So, it makes sense to > > have > > > >>>>>>>>>>>> it on > > > >>>>>>>>>>>> Stream > > > >>>>>>>>>>>> rather than in common configuration. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> And yes, DAG.validate should only check for > REQUIRED > > or > > > >>>>>>>>>>>> STRICT > > > >>>>>>>>>>>> policy. We > > > >>>>>>>>>>>> can agree on one of the terminologies STRICT/RELAXED or > > > >>>>>>>>>>>> REQUIRED/PREFERRED. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Thanks! > > > >>>>>>>>>>>> Isha > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> On Fri, Jan 22, 2016 at 8:38 PM, Chinmay Kolhatkar < > > > >>>>>>>>>>>> [email protected]> > > > >>>>>>>>>>>> wrote: > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Hi Isha, Bhupesh, > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> When I suggested singe affinity rule, I was mainly talking > > > about > > > >>>>>>>>>>>> "how > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> to" > > > >>>>>>>>>>>>> of configuration and not of implementation. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> I see locality is in a way suggesting an affinity of > > > operators. > > > >>>>>>>>>>>>> They're > > > >>>>>>>>>>>>> close terminologies. > > > >>>>>>>>>>>>> By configuring a locality on stream, we're also, in a > way, > > > >>>>>>>>>>>>> defining > > > >>>>>>>>>>>>> affinity of operators. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> Until now, only locality was there and hence was straight > > > >>>>>>>>>>>>> forward > > > >>>>>>>>>>>>> in > > > >>>>>>>>>>>>> configuration for user. > > > >>>>>>>>>>>>> Tomorrow, when anti-affinity configuration comes up, one > > > might > > > >>>>>>>>>>>>> get > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> confused > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> on how to best use both locality & anti-affinity. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>> Hence suggested to make both (locality/affinity & > > > >>>>>>>>>>>> anti-affinity) as > > > >>>>>>>>>>>> > > > >>>>>>>>>>>>> a > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> part > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> of single configuration. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>> Suggestion is to have a more commonly adopted > configuration > > > >>>>>>>>>>>> which > > > >>>>>>>>>>>> > > > >>>>>>>>>>>>> admins > > > >>>>>>>>>>>>> and developer's are familiar with. > > > >>>>>>>>>>>>> Again referring to vShere Hypervisor's affinity rules. I > > > think > > > >>>>>>>>>>>>> they > > > >>>>>>>>>>>>> have > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> a > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> single configuration which does both. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>> Having said that, here is a quick suggestion on how both > can > > > be > > > >>>>>>>>>>>> > > > >>>>>>>>>>>>> achieved > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> in > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> a single configuration: > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>> CATEGORY TYPE POLICY ENTITIES > > > >>>>>>>>>>>> > > > >>>>>>>>>>>>> Affinity THREAD REQUIRED O1, O2 > > > >>>>>>>>>>>>> //Meaning > > > >>>>>>>>>>>>> Operator1 & Operator2 should be thread local > > > >>>>>>>>>>>>> Affinity NODE PREFERRED O3, O4 > > > >>>>>>>>>>>>> //Meaning > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> O3 & > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> O4 are preferred to be in node local > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>> AntiAffinity NODE REQUIRED O1, O4 > > > >>>>>>>>>>>> > > > >>>>>>>>>>>>> //Meaning > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> O1 & > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> O4 should not be on same node. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>> AntiAffinity RACK PREFERRED O2, O4 > > > >>>>>>>>>>>> > > > >>>>>>>>>>>>> //Meaning > > > >>>>>>>>>>>>> O2 > > > >>>>>>>>>>>>> & > > > >>>>>>>>>>>>> O4 are preferred not to be on same rack. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> Linux setting affinity of CPUs for threads is another way > > of > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> configuration > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> we can take a look at. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>> Learning from these commonly adopted configuration > pattern, > > we > > > >>>>>>>>>>>> > > > >>>>>>>>>>>>> should > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> come > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> up with best configuration suitable for distributed > > > >>>>>>>>>>>>> environment. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>> Idea here is to not have our own configuration and give > > > >>>>>>>>>>>> something > > > >>>>>>>>>>>> > > > >>>>>>>>>>>>> new > > > >>>>>>>>>>>>> to > > > >>>>>>>>>>>>> the users. Otherwise such an important concept might > > quickly > > > >>>>>>>>>>>>> get > > > >>>>>>>>>>>>> lost. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> Regarding the DAG.validate I think we would need to add > > some > > > >>>>>>>>>>>>> new > > > >>>>>>>>>>>>> stuff > > > >>>>>>>>>>>>> to > > > >>>>>>>>>>>>> take care of anti-affinity. > > > >>>>>>>>>>>>> Plus, anti-affinity/affinity should be validated at > > > >>>>>>>>>>>>> DAG.validate > > > >>>>>>>>>>>>> only > > > >>>>>>>>>>>>> for > > > >>>>>>>>>>>>> the ones which are required. > > > >>>>>>>>>>>>> For preferred policies, validation in logical plan might > > be a > > > >>>>>>>>>>>>> early > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> check. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> Thanks, > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>> Chinmay. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> On Sat, Jan 23, 2016 at 3:15 AM, Isha Arkatkar < > > > >>>>>>>>>>>>> [email protected]> > > > >>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> Hi, > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> Thanks for inputs! I like the idea of having > single > > > set > > > >>>>>>>>>>>>> of > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> (anti?)affinity rules at dag context level. > > > >>>>>>>>>>>>>> There could still be conflicts based on Locality > > for > > > >>>>>>>>>>>>>> Streams > > > >>>>>>>>>>>>>> or > > > >>>>>>>>>>>>>> Node > > > >>>>>>>>>>>>>> locality attribute set for operators. But as Sandeep > > > >>>>>>>>>>>>>> suggested, I > > > >>>>>>>>>>>>>> also > > > >>>>>>>>>>>>>> think Dag.validate should fail in case of contradicting > > > >>>>>>>>>>>>>> constraints. > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> We currently do not have 'affinity' among non > stream > > > >>>>>>>>>>>>>> operators, as > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> Bhupesh > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> pointed out. It is somewhat achievable by requesting > node > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>> locality > > > >>>>>>>>>>>>> for > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> operators (Assuming node requests work as expected). But > > > >>>>>>>>>>>>> should we > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>>> consider > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> adding affinity specifications support as well along > with > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>> anti-affinity? > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>>> Regarding specification of attributes from > > > dt-site.xml. > > > >>>>>>>>>>>>>> We > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>> can > > > >>>>>>>>>>>>> go > > > >>>>>>>>>>>>> with > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> Json like string or even xml representation for complex > > > >>>>>>>>>>>>> objects. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> What > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> is > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> our current behavior for setting Java object properties > > > >>>>>>>>>>>>>> through > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>> XML? > > > >>>>>>>>>>>>> We > > > >>>>>>>>>>>>> can > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> follow the same for this as well. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> As for precedence or ability to satisfy > constraints: > > > >>>>>>>>>>>>> Right > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>>> now in > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> normal > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> scenario, if resources are not available for allocating > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>> containers, > > > >>>>>>>>>>>>> we > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> keep > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>>> sending to request till all are obtained. Likewise, in > > case > > > of > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>> strict > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> anti-affinity policy, we should keep the application in > > > >>>>>>>>>>>>> ACCEPTED > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>>> state > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> till > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> the anti-affinity constraint is satisfied. For relaxed > > > >>>>>>>>>>>>>> policy, we > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>> can > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> decide timeout for relaxing the anti-affinity rule. > Please > > > note > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>>> this > > > >>>>>>>>>>>>>> applies only when we have non-contradicting rules. > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> Thanks! > > > >>>>>>>>>>>>>> Isha > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 5:30 AM, Bhupesh Chawda < > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> [email protected] > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>> I agree on having a single set of rules for the affinity > as > > > >>>>>>>>>>>>> well > > > >>>>>>>>>>>>> as > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> anti-affinity of operators / partitions on containers. > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> However, I noted the following points: > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> 1. AFAIK, we do not support affinity (locality) > > in > > > a > > > >>>>>>>>>>>>>>> general > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> sense. > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> The > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> affinity is only for a stream, not for *any* two > > > >>>>>>>>>>>>> operators. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> So, > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> we > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> should also look at the general case and see > how > > it > > > >>>>>>>>>>>>>>> can > > > >>>>>>>>>>>>>>> be > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> supported, > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> if > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> there are valid use cases. > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> 2. Coming to anti-affinity, we cannot type cast > it > > > as > > > >>>>>>>>>>>>>> a > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> type of > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> affinity > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> rule. Saying "two operators must be on the same > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> container" is > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> very > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> different from saying "these two operators must > > not > > > >>>>>>>>>>>>>>> be on > > > >>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>> same > > > >>>>>>>>>>>>>>> container". In this case, the second one is a > > much > > > >>>>>>>>>>>>>>> more > > > >>>>>>>>>>>>>>> relaxed > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> rule > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> as > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> compared to the first one. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> 3. Once we have this generic set of rules, there > > must > > > >>>>>>>>>>>>>> be a > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> "satisfiability" test run before requesting > YARN > > > for > > > >>>>>>>>>>>>>>> containers. > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> If > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> request is not satisfiable, then there is no > point > > > >>>>>>>>>>>>> asking > > > >>>>>>>>>>>>> YARN > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> to > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> allocate > > > >>>>>>>>>>>>>>> containers in this manner. In case it is not > > > >>>>>>>>>>>>>>> satisfiable, we > > > >>>>>>>>>>>>>>> must > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> also > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> have > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> a default order in which the rules can be > > "relaxed" > > > >>>>>>>>>>>>>> and > > > >>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> request > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> be > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> made > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> satisfiable. For example, some very strict rules > > may > > > be > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>>> ignored, > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> or > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> made > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> less constraining ( for example "on the same > > > >>>>>>>>>>>>> container" => > > > >>>>>>>>>>>>> "on > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> the > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> same > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> node"). > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> Thanks. > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> -Bhupesh > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 2:54 PM, Aniruddha Thombare < > > > >>>>>>>>>>>>>>> [email protected]> wrote: > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> +1 On Chinmay's suggestion about having single set of > > > >>>>>>>>>>>>>>> affinity > > > >>>>>>>>>>>>>>> rules. > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> Thanks, > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> Aniruddha > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 1:57 PM, Sandeep Deshmukh < > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> [email protected] > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> Between 2 operators, if one configures thread/container > > > local > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> and > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> anti-affinity as well, which one will take affect? > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> The DAG validation step should error out in this > case. > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> +1 on suggestion by Chinmay to name it "Affinity > Rules" > > > >>>>>>>>>>>>>>> than > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> anti-affinity. We are just extending our container > > > allocation > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> scheme > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> support containers not to be allocated together. > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> Regards, > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> Sandeep > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 1:43 PM, Chinmay Kolhatkar < > > > >>>>>>>>>>>>>>>>> [email protected]> > > > >>>>>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> Hi Isha, > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> Couple of points: > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> 1. About the interface to configuring anti-affinity, > as > > > per > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> suggestion > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> above there are 2 different way to configure > locality > > > and > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> anti-affinity: > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> i.e. dag.setAttribute - for anti-affinity & > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> dag.addStream(...).setLocality for locality. > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> Between 2 operators, if one configures > thread/container > > > >>>>>>>>>>>>>>>>>> local > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> and > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> anti-affinity as well, which one will take affect? > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> 2. Consider there could be such confusion as above, > > would > > > >>>>>>>>>>>>>>>> it > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> make > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> sense > > > >>>>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> have a single API which takes care of both > anti-affinity > > > and > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> locality. > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> This > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> way, one is configurable. > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> 3. This point is coming from how VM affinity is > > configured > > > >>>>>>>>>>>>>>>>> in > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> vSphere. > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> The VMs are configured affinity are called as > > "affinity > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> rules" > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> and > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> not > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> "anti-affinity rules". Ultimately idea is to allocate > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> processing > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> nodes. > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> Via "VM-VM affinity rules", anti-affinity is also > > > configured. > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> But > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> there > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> is > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> a single set of rule definition for both affinity > > (similar > > > to > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> locality > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> in > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> our case) and anti-affinity. > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> Would it be a better approach for configuring locality > > > rules > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> and > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> anti-affinity rules in a single rule and call it > > > "affinity > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> rule". > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> Thanks, > > > >>>>>>>>>>>>>>> Chinmay. > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 12:24 PM, Yogi Devendra < > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> [email protected] > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> @Isha > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> I understand that anti-affinity across application is > > not > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> straight-forward. > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> It would be OK even if we do not have it in > iteration > > > 1. > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> But, for attributes syntax; I still think that Java > > > object > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> should > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> be > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> avoided as they will be hard to configure from > > > >>>>>>>>>>>>>>>>> dt-site.xml or > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> other > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> config > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> files. > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> Other suggestion for this could be JSON representation > > of > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> String > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> array: > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> ["O2", "O3"]. (If operator names has some special > > > >>>>>>>>>>>>>>>>> characters > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> like > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> " > > > >>>>>>>>>>>>>>>>> or [ > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> or , those will be escaped in the JSON > representation.) > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> Not sure if others agree on this; but attribute syntax > > > >>>>>>>>>>>>>>>>> should > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> be > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> finalized > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> in iteration 1 to avoid backward compatibility > issues > > > >>>>>>>>>>>>>>>>> later. > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> ~ Yogi > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> On 22 January 2016 at 00:43, Thomas Weise < > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> [email protected]> > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> Node based requests is the best approach - if it > works > > > :-) > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> Blacklisting will require to allocate the containers > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> sequentially. > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> It > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> will > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> work, but slow down application startup, especially > for > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> larger > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> topologies. > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> On Thu, Jan 21, 2016 at 10:42 AM, Isha Arkatkar < > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> [email protected]> > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> Hi, > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> Should we consider the node based requests if > > it > > > >>>>>>>>>>>>>>>>>>> works > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> with > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> Capacity > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> Scheduler or avoid 2b approach altogether? I > checked > > > >>>>>>>>>>>>>>>>>>> that > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> node > > > >>>>>>>>>>>>>>>>> requests > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> do > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> not work with fair scheduler on CDH cluster. Yarn > does > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> not > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> return > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> any > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> container if hostname is given in the container > > request. > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> I > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> am > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> trying > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> setup a small virtual hortonworks cluster to check the > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> this > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> behavior > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> on > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> that. > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> YARN-2027 < > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/YARN-2027 > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> mentioned > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> that > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> container requests are not honored in capacity > > scheduler > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> too. > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> But I > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> am > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> not > > > >>>>>>>>>>>>>>>>> sure if it is because of distro dependent issue. > Please > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> share > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> insights. > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> @Vlad, Adding support for regular expression > sounds > > > >>>>>>>>>>>>>>>>>>> good. > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> We > > > >>>>>>>>>>>>>>>>> could > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> translate to list of operator names internally based > > on > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> regex. > > > >>>>>>>>>>>>>>>>> @Yogi, I went with a list of strings for attribute > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> because > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> "O2, > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> O3" > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> could > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> be a valid single operator name too :) > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> I am not sure of ways to implement anti-affinity > across > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> application. > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> Though > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> something to consider for later iteration. > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> Thanks, > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> Isha > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> On Wed, Jan 20, 2016 at 8:59 PM, Thomas Weise < > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> [email protected]> > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/SLIDER-82 > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> On Wed, Jan 20, 2016 at 8:56 PM, Thomas Weise < > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> [email protected]> > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> The point was that containers are taken away from > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> other > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> apps > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> that > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> may > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> have > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> to discard work etc. It's not good style to claim > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> resources > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> and > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> not > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> use > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> them eventually :-) > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> For this feature it is necessary to look at the > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> scheduler > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> capabilities/semantics and limitations. For > > > example, > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> don't > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> bet > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> exclusively > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> on node requests if the goal is for it to work with > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> FairScheduler. > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> Also look at Slider, which just recently added > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> support > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> for > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> anti-affinity > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> (using node requests). When you run it on the CDH > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> cluster, > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> it > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> probably > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> won't work... > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> On Wed, Jan 20, 2016 at 3:19 PM, Pramod Immaneni < > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> [email protected] > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> Once released won't the containers be available > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> again > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> in > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> pool. > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> This > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> would only be optional and not mandatory. > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> Thanks > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> On Tue, Jan 19, 2016 at 2:02 PM, Thomas Weise < > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> [email protected] > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> How about also supporting a minor variation of > it > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> as > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> an > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> option > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> where it greedily gets the total number of > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> containers > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> and > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> discards > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> ones > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> it > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> can't use and repeats the process for the > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> remaining > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> till > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> everything > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> has > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> been allocated. > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> This is problematic as with resource preemption > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> these > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> containers > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> will > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> be > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> potentially taken away from other applications and > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> then > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> thrown > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> away. > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> Also does it make sense to support anti-cluster > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> affinity? > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> Thanks > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jan 19, 2016 at 1:21 PM, Isha > Arkatkar > > < > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> [email protected]> > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> Hi all, > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> We want add support for Anti-affinity > in > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> Apex > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> allow > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> applications > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> launch specific physical operators on > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> different > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> nodes(APEXCORE-10 > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> < > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> > > https://issues.apache.org/jira/browse/APEXCORE-10 > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> ). > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> Want > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> request > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> your > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> suggestions/ideas for the same! > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> The reasons for using anti-affinity in > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> operators > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> could > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> be: > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> ensure > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> reliability, for performance reasons (such as > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> application > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> may > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> not > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> want > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> 2 > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> i/o intensive operators to land on the same > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> node > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> improve > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> performance) > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> or > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> for some application specific constraints(for > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> example, > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> 2 > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> partitions > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> cannot > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> be run on the same node since they use same > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> port > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> number). > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> This > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> is > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> general rationale for adding Anti-affinity > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> support. > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> Since, Yarn does not support anti-affinity yet > > > >>>>>>>>>>>>>>>>>>>>>>>>> (YARN-1042 > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> < > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/YARN-1042 > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> ), > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> we > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> need > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> implement > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> the logic in AM. Wanted to get your views on > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> following > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> aspects > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> for > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> this > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> implementation: > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> *1. How to specify anti-affinity for physical > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> operators/partitions > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> in > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> application:* > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> One way for this is to have an > > attribute > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> for > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> setting > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> anti-affinity > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> at > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> the logical operator context. And an operator > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> can > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> set > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> this > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> attribute > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> with > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> list of operator names which should not be > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> collocated. > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> Consider dag with 3 operators: > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> TestOperator o1 = > > > dag.addOperator("O1", > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> new > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> TestOperator()); > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> TestOperator o2 = dag.addOperator("O2", > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> new > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> TestOperator()); > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> TestOperator o3 = dag.addOperator("O3", > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> new > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> TestOperator()); > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> To set anti-affinity for O1 operator: > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> dag.setAttribute(o1, > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> OperatorContext.ANTI_AFFINITY, > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> new > > > >>>>>>>>>>>>>>>>>>>>>>>>> ArrayList<String>(Arrays.asList("O2", > "O3"))); > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> This would mean O1 should not be > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> allocated > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> on > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> nodes > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> containing > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> operators O2 and O3. This applies to all > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> allocated > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> partitions > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> of > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> O1, > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> O2, > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> O3. > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> Also, if same operator name is part of > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> anti-affinity > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> list, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >> > > > > > > > > > >
