How about this use case: a performance test program that measures the number of CCBs per second that IMM can execute?
/ Anders Widell On 09/05/2016 03:17 PM, [email protected] wrote: > Hi, > > Neither of those examples (1) A shell script calling immcfg command from a > loop; (2) A buggy application; are real use cases. > For (2) it is obvious. > > For (1), it is at best a very sloppy implementation of some use case. > The intention of that shell script is to accomplish what amounts to one > configuration change, but it is doing so using thousands of separate ccbs! > Say that the script itself crashes after it has completed some immcfg > commands but not all the intended ones. > What does the user do then ? > So (1) would be either just a very sloppy (and error prone) implementation or > possibly a test case of some kind, but a test that is not a valid > functional/reliability test. Possibly a performance test of a metric where > the imm is not intended to have high performance (high transaction > throughput). > > /AndersBj > > > > > >> ----Ursprungligt meddelande---- >> Från : [email protected] >> Datum : 2016-09-05 - 14:00 (CEST) >> Till : [email protected], [email protected], >> [email protected], [email protected], >> [email protected] >> Ämne : Re: [devel] Immnd: maximum Ccbs limit 10000 has been reached very >> quickly >> >> I already mentioned one use case: a shell script calling the immcfg >> command from a loop. There is also another important use case: a buggy >> application. Imagine that an application by mistake creates a huge >> amount of CCBs. Since this new limit is global, the buggy application >> will now affect all other applications in the whole cluster. A variant >> of this use-case is an attacker carrying out a denial-of-service attack >> on the system by somehow (e.g. via NBI) creating many CCBs. >> >> regards, >> >> Anders Widell >> >> >> On 09/05/2016 12:56 PM, Zoran Milinkovic wrote: >>> Hi Hung, >>> >>> Agree with you and Anders Widell for the first item. >>> >>> For the second item, as I mentioned earlier, IMM is not developed to >>> handle a huge amount of CCBs. If we need to support that, then we need >>> to rewrite CCB part, and replace std::vector with some other >>> structure, possibly with std::map. Then the performance degradation >>> will be minimal. >>> >>> And when this is changed, probably CCB limitation would me more >>> related to memory usage instead of IMM performance. Then we can >>> consider all CCB. >>> >>> To repeat myself again, I don’t see a valid case where I’m not able to >>> open/use a CCB for minutes because some other application intensively >>> used CCBs and managed to close 10000 handles within a minute. >>> >>> BR, >>> >>> Zoran >>> >>> *From:*Hung Nguyen [mailto:[email protected]] >>> *Sent:* den 5 september 2016 12:29 >>> *To:* Anders Widell; Zoran Milinkovic; A V Mahesh; Neelakanta Reddy; >>> [email protected] >>> *Subject:* Re: [devel] Immnd: maximum Ccbs limit 10000 has been >>> reached very quickly >>> >>> Hi, >>> >>> I agree with Anders that we should let users enable the limit instead of >>> enabling it automatically. >>> Maybe we can set default value of those attributes to 0 and IMM will >>> interpret 0 as unlimited. >>> It's going to be a defect ticket for #195. >>> >>> About the second problem (i.e. the meaning of maxCcbs limit), I think the >>> limit should be applied on all CCBs as we store them in the same list. >>> This limit is for pure performance purpose. >>> >>> It should be well documented though. >>> The easiest way for users to check how much resource left is to use 'immadm >>> -O displayverbose'. >>> >>> BR, >>> Hung Nguyen - DEK Technologies >>> >>> -------------------------------------------------------------------------------- >>> From: Anders [email protected] >>> <mailto:[email protected]> >>> Sent: Friday, September 02, 2016 9:53PM >>> To: Zoran Milinkovic, Mahesh Valla, Neelakanta Reddy, Opensaf-devel >>> [email protected] >>> <mailto:[email protected]>,[email protected] >>> <mailto:[email protected]>,[email protected] >>> <mailto:[email protected]>,[email protected] >>> <mailto:[email protected]> >>> Cc: >>> >>> Subject: Re: [devel] Immnd: maximum Ccbs limit 10000 has been reached very >>> quickly >>> >>> >>> We have to consider the backwards compatibility aspect here. The general >>> principle is that if an application works fine with one version of >>> OpenSAF, then it should work fine also with a newer version of OpenSAF. >>> Mahesh has evidently an example application (a test program) that starts >>> to fail after upgrading from OpenSAF 5.0 to OpenSAF 5.1.FC. If it is a >>> pathological example program then you can argue that it is not relevant, >>> but I don't think it is the case here. You could imagine a real-life >>> case where someone has written a shell script that runs the immcfg >>> command in a loop 10000 times. >>> >>> The safest approach to introduce a new limit where we previously didn't >>> have any limit, is to set the default value of the new limit to infinite >>> (unlimited), and let the user take an active decision to configure a >>> lower value of the limit. This way, applications will not get unpleasant >>> surprises after upgrading OpenSAF. >>> >>> regards, >>> >>> Anders Widell >>> >>> On 09/02/2016 10:56 AM, Zoran Milinkovic wrote: >>> >>> Hi Mahesh, >>> >>> >>> >>> Ticket #195 introduced this limitation. >>> >>> >>> >>> After all, I'm not sure who is right... Neelakanta or me :) >>> >>> Earlier, we didn't have any limitation, and limitation is introduced >>> to have some reasonable amount of CCB handles in IMM. >>> >>> IMM keeps non-used and active CCB handles in the same place. If the >>> amount of CCB handles grows a lot, then we will see a degradation of IMM >>> performance. >>> >>> >>> >>> >From earlier discussion with Anders, it's not meant from the >>> beginning that CCB will be used so often. This might be very specific case. >>> That's also the reason why CCB handles are handled in a way that is not >>> good for huge amount of CCB handles. >>> >>> >From this point of view, Neelakanta is correct. >>> >>> >From my point of view, we should only consider CCBs that can be used. >>> I don't see acceptable that CCB operations cannot be done for minutes if we >>> have a huge amount of closed CCBs. >>> >>> >>> >>> We can improve CCB handling in the next OpenSAF release. Now it's too >>> late. >>> >>> >>> >>> For this particular problem, I would go with checking only CCBs that >>> can be used (CCB that have SA_AIS_OK on mVeto variable, or calling ->isOk() >>> method on CcbInfo struct). This solution may have impact on IMM performance. >>> >>> Or we can revert the ticket and implement it in the next release with >>> better performance. >>> >>> >>> >>> I would also like to hear Neelakanta's and Hung's proposals. >>> >>> >>> >>> Thanks, >>> >>> Zoran >>> >>> >>> >>> -----Original Message----- >>> >>> From: A V Mahesh [mailto:[email protected]] >>> >>> Sent: den 2 september 2016 05:56 >>> >>> To: Zoran Milinkovic; Neelakanta >>> Reddy;[email protected] >>> <mailto:[email protected]> >>> >>> Subject: Re: [devel] Immnd: maximum Ccbs limit 10000 has been reached >>> very quickly >>> >>> >>> >>> Hi Zoran, >>> >>> >>> >>> Thanks for the update , >>> >>> >>> >>> you are right ERR_NO_RESOURCES only justifiable if 10000 IMM >>> application concurrently holding 10000 CCB`s ( active CCB), in current >>> case agent is already finalized. >>> >>> >>> >>> By the way by roll backing which change set I can continue my testing ? >>> >>> >>> >>> -AVM >>> >>> >>> >>> On 9/1/2016 5:52 PM, Zoran Milinkovic wrote: >>> >>> Hi Neelakanta, >>> >>> >>> >>> This is definitely a bug. >>> >>> When CCB is finalized, it's still shown as an active CCB. >>> >>> If you check number of CCBs with resource display functionality, >>> you will see that number of CCBs is not decreasing. >>> >>> >>> >>> BR, >>> >>> Zoran >>> >>> >>> >>> -----Original Message----- >>> >>> From: Neelakanta Reddy [mailto:[email protected]] >>> >>> Sent: den 1 september 2016 11:09 >>> >>> To:[email protected] >>> <mailto:[email protected]> >>> >>> Subject: Re: [devel] Immnd: maximum Ccbs limit 10000 has been >>> reached >>> >>> very quickly >>> >>> >>> >>> Hi, >>> >>> >>> >>> I think this is not yet documented and will be documented. >>> >>> Try to increase the ccb limits and check. >>> >>> >>> >>> /Neel. >>> >>> >>> >>> On 2016/09/01 02:01 PM, Chani Srivastava wrote: >>> >>> Hi, >>> >>> >>> >>> With current 5.1 OpenSAF changeset 7997 the issue is >>> reproducible >>> >>> >>> >>> ============================================== >>> >>> Sep 1 12:16:52 OSAF-SC1 osafimmnd[27298]: NO Ccb 10002 >>> COMMITTED >>> >>> (chaniTestClass) >>> >>> Sep 1 12:16:52 OSAF-SC1 osafimmnd[27298]: NO Ccb 10003 >>> COMMITTED >>> >>> (chaniTestClass) >>> >>> Sep 1 12:16:52 OSAF-SC1 osafimmnd[27298]: NO Ccb 10004 >>> COMMITTED >>> >>> (chaniTestClass) >>> >>> Sep 1 12:16:52 OSAF-SC1 osafimmnd[27298]: NO Ccb 10005 >>> COMMITTED >>> >>> (chaniTestClass) >>> >>> Sep 1 12:16:52 OSAF-SC1 osafimmnd[27298]: NO ERR_NO_RESOURCES: >>> >>> maximum Ccbs limit 10000 has been reached for the cluster >>> >>> =============================================== >>> >>> >>> >>> This looks like a candidate for ticket. Let me know i'll raise >>> it in >>> >>> community. >>> >>> >>> >>> -Chani >>> >>> >>> >>> On 9/1/2016 1:49 PM, Neelakanta Reddy wrote: >>> >>> Hi, >>> >>> >>> >>> can you verify the same with 5.1. >>> >>> >>> >>> /Neel. >>> >>> >>> >>> >>> >>> On 2016/09/01 12:58 PM, Chani Srivastava wrote: >>> >>> Hi All, >>> >>> >>> >>> I verified this is a break in functionality. In >>> OpenSAF 5.0 I tried >>> >>> creating 15000 objects per CCB and it worked fine. >>> >>> >>> >>> ===================================== >>> >>> Nov 26 15:21:17 SCALE_SLOT-91 osafimmnd[24451]: NO Ccb >>> 237 >>> >>> COMMITTED >>> >>> (chaniTestClass) >>> >>> Nov 26 15:21:17 SCALE_SLOT-91 osafimmnd[24451]: NO Ccb >>> 238 >>> >>> COMMITTED >>> >>> (chaniTestClass) >>> >>> Nov 26 15:21:17 SCALE_SLOT-91 osafimmnd[24451]: NO Ccb >>> 239 >>> >>> COMMITTED >>> >>> (chaniTestClass) >>> >>> Nov 26 15:21:17 SCALE_SLOT-91 osafimmnd[24451]: NO Ccb >>> 240 >>> >>> COMMITTED >>> >>> (chaniTestClass) >>> >>> . >>> >>> . >>> >>> . >>> >>> Nov 26 15:23:05 SCALE_SLOT-91 osafimmnd[24451]: NO Ccb >>> 15230 >>> >>> COMMITTED >>> >>> (chaniTestClass) >>> >>> Nov 26 15:23:05 SCALE_SLOT-91 osafimmnd[24451]: NO Ccb >>> 15231 >>> >>> COMMITTED >>> >>> (chaniTestClass) >>> >>> Nov 26 15:23:05 SCALE_SLOT-91 osafimmnd[24451]: NO Ccb >>> 15232 >>> >>> COMMITTED >>> >>> (chaniTestClass) >>> >>> Nov 26 15:23:05 SCALE_SLOT-91 osafimmnd[24451]: NO Ccb >>> 15233 >>> >>> COMMITTED >>> >>> (chaniTestClass) >>> >>> Nov 26 15:23:05 SCALE_SLOT-91 osafimmnd[24451]: NO Ccb >>> 15234 >>> >>> COMMITTED >>> >>> (chaniTestClass) >>> >>> Nov 26 15:23:05 SCALE_SLOT-91 osafimmnd[24451]: NO Ccb >>> 15235 >>> >>> COMMITTED >>> >>> (chaniTestClass) >>> >>> Nov 26 15:23:05 SCALE_SLOT-91 osafimmnd[24451]: NO Ccb >>> 15236 >>> >>> COMMITTED >>> >>> (chaniTestClass) >>> >>> ====================================== >>> >>> >>> >>> The ticket #195 only makes the MAX parameters >>> configurable. >>> >>> If accepted, I'll raise the ticket in sourceforge. >>> >>> >>> >>> -Chani >>> >>> >>> >>> On 9/1/2016 10:59 AM, A V Mahesh wrote: >>> >>> Hi All, >>> >>> >>> >>> I was running `immcfg` in a loop to create some >>> object , once it >>> >>> reaches 10001objects creation Immnd is returning >>> `ERR_NO_RESOURCES: >>> >>> maximum Ccbs limit 10000 has been reached for the >>> cluster` error >>> >>> which is unexpected , once `immcfg` reruns , it >>> is expected that >>> >>> the `maximum Ccbs` will decremented ( this was >>> previous behavior) >>> >>> , >>> >>> >>> >>> can you please check. >>> >>> >>> >>> >>> ================================================================== >>> >>> = ====================================== >>> >>> >>> >>> >>> >>> for (( i = 1 ; i <=300000; i++)) >>> >>> immcfg -c PinvId -a >>> pinvPhoneNumber=+46768 pinvRdn=$i >>> >>> >>> >>> Sep 1 10:43:33 SC-1 osafimmnd[4466]: NO Ccb 9996 >>> COMMITTED >>> >>> (immcfg_SC-1_2334) >>> >>> Sep 1 10:43:33 SC-1 osafimmnd[4466]: NO Ccb 9997 >>> COMMITTED >>> >>> (immcfg_SC-1_2337) >>> >>> Sep 1 10:43:33 SC-1 osafimmnd[4466]: NO Ccb 9998 >>> COMMITTED >>> >>> (immcfg_SC-1_2340) >>> >>> Sep 1 10:43:33 SC-1 osafimmnd[4466]: NO Ccb 9999 >>> COMMITTED >>> >>> (immcfg_SC-1_2343) >>> >>> Sep 1 10:43:33 SC-1 osafimmnd[4466]: NO Ccb 10000 >>> COMMITTED >>> >>> (immcfg_SC-1_2346) >>> >>> Sep 1 10:43:33 SC-1 osafimmnd[4466]: NO Ccb 10001 >>> COMMITTED >>> >>> (immcfg_SC-1_2349) >>> >>> Sep 1 10:43:33 SC-1 osafimmnd[4466]: NO >>> ERR_NO_RESOURCES: maximum >>> >>> Ccbs limit 10000 has been reached for the cluster >>> Sep 1 10:43:33 >>> >>> SC-1 osafimmnd[4466]: NO ERR_NO_RESOURCES: maximum >>> Ccbs limit >>> >>> 10000 has been reached for the cluster Sep 1 >>> 10:43:33 SC-1 >>> >>> osafimmnd[4466]: NO ERR_NO_RESOURCES: maximum Ccbs >>> limit 10000 has >>> >>> been reached for the cluster >>> >>> >>> >>> >>> >>> >>> ================================================================== >>> >>> = ====================================== >>> >>> >>> >>> >>> >>> -AVM >>> >>> >>> >>> >>> >>> >>> ------------------------------------------------------------------ >>> >>> - >>> >>> ----------- >>> _______________________________________________ >>> >>> Opensaf-devel mailing list >>> >>> [email protected] >>> <mailto:[email protected]> >>> >>> >>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel >>> >>> >>> ------------------------------------------------------------------- >>> >>> - >>> >>> ---------- >>> _______________________________________________ >>> >>> Opensaf-devel mailing list >>> >>> [email protected] >>> <mailto:[email protected]> >>> >>> >>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel >>> >>> >>> -------------------------------------------------------------------- >>> >>> - >>> >>> --------- _______________________________________________ >>> >>> Opensaf-devel mailing list >>> >>> [email protected] >>> <mailto:[email protected]> >>> >>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel >>> >>> >>> --------------------------------------------------------------------- >>> >>> - >>> >>> -------- _______________________________________________ >>> >>> Opensaf-devel mailing list >>> >>> [email protected] >>> <mailto:[email protected]> >>> >>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel >>> >>> >>> ---------------------------------------------------------------------- >>> >>> -------- _______________________________________________ >>> >>> Opensaf-devel mailing list >>> >>> [email protected] >>> <mailto:[email protected]> >>> >>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel >>> >>> >>> >>> >>> ---------------------------------------------------------------------- >>> >>> -------- _______________________________________________ >>> >>> Opensaf-devel mailing list >>> >>> [email protected] >>> <mailto:[email protected]> >>> >>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel >>> >>> >>> ------------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> >>> Opensaf-devel mailing list >>> >>> [email protected] >>> <mailto:[email protected]> >>> >>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel >>> >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> _______________________________________________ >>> Opensaf-devel mailing list >>> [email protected] >>> <mailto:[email protected]> >>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel >>> >> ------------------------------------------------------------------------------ >> _______________________________________________ >> Opensaf-devel mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/opensaf-devel >> ------------------------------------------------------------------------------ _______________________________________________ Opensaf-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/opensaf-devel
