Rick, everything that generates NextID's must come through an ARSystem server so I don't know what you mean when you say 'If you have multiple outside entities creating large number of records at once'
All requests go through the AR Server. I would theorize that you may have other factors in play once you remove the NextID bottleneck by using chunks of 100. Something to bear in mind is that by chunking NextID's, you cause the scalability of the system to go up (i.e. more records per second getting created) - but the per record response time under high load is likely to go down (due to the system being under heavy load, other downstream constraints such as CPU/Disk Speed start to appear, where they would not appear if the NextID bottleneck was in place) It effectively acts as a throttle. So your results could be easily misinterpreted. I would think a way to confirm this is to generate parallel load from a high number of worker threads, and time how long it took to reach the same number of records created with 1 or 100 for the NextID chunk size. Then compare the times. Anyway, we are off on a tangent from the original topic Thomas, I am actually curious as to why you want to know a nextid. What Abhijeet said explains why it doesn't make sense to ask for it, but often in workflow scenarios, where you want to generate something unique and refer/tie back to something else. you are better off using Application-Generate-GUID [" *GUIDPrefix*" ] On Thu, Oct 12, 2017 at 5:27 PM, Rick Cook <[email protected]> wrote: > ** > Abhijeet, my results were repeated and conclusive. Here is the only > logical explanation for my findings that makes sense: > > There is a time cost to the DB call for Entry IDs. The cost increases > incrementally the more that are requested at once. The thought is that 100 > individual requests will take substantially more time than retrieving 100 > IDs in one request. That thought has been proven to be correct - it's why > the feature was added. However, the difference in time was tested against a > system that had multiple sources (NMS) attempting to grab multiple Entry > IDs at the same time, not a normal system without those multiple outside > requests, because it was the first environment that was reporting the > problem with performance. > > On a system where the volume of IDs is high enough to cause a system > bottleneck at the DB ID request, *which requires multiple simultaneous > requests from different sources*, that bottleneck costs more additional > time than is lost by the increased time it takes to request multiple IDs at > one time. However, on a system that is not bound by the Next ID calls to > the DB, or has only a single source (AR System) requesting IDs, there is no > advantage gained by the multiple requests, because the time "gained" by > having a cached ID is negligible - too small to even notice. And, as my > testing proved (to my surprise), there was an increasing net cost to system > performance as the number of IDs requested in a single call grew. This > must be because the time it takes to gather, say, 100 records in one call > is actually *higher* than it is to do 100 individual calls - IF AR System > is the only source of those calls. The simple correlation is that if AR > System is the only thing generating new Request ID requests to the DB, you > don't need, and will not benefit from, a number larger than 1. If you have > multiple outside entities creating large number of records at once, you > very well may benefit from it. > > My tests that showed decreasing performance as the Next Id block grew were > creating about 1.5 million records, but, and this is important - all were > from the same source - AR System workflow. There were no other > simultaneous demands on the table, and few on the system. Therefore, the > system of generating one ID at a time had a cumulatively lower transaction > time than it did when it had to wait for multiples to be retrieved and then > allocated individually. The BMC Engineer, who I know to be very smart and > experienced with Remedy, had no explanation for my results. I believe this > to be that BMC didn't test against a control (normal) system, and therefore > had no data on its effect on them. > > Why then, they chose to recommend that setting to all customers, is a > mystery to me. > > Rick Cook > > On Thu, Oct 12, 2017 at 4:56 PM, Gadgil, Abhijeet <[email protected] > > wrote: > >> ** >> >> Rick, I do not think that is accurate. >> >> Logically, if the block size is 100 then the server will access the DB >> once to retrieve the next block id every 100 records. If it is 1, then the >> server goes to the DB for every record. >> >> Later cannot be faster. >> >> >> >> Further, to answer the original question, returning the next id in an API >> call would mean preventing creation of ids by anyone other than the caller >> until the id has actually been used -- otherwise the information might be >> obsolete before the client receives it. That is the reason for not exposing >> it via an API >> >> >> >> Regards, >> >> Abhijeet >> >> >> >> >> >> *From:* Action Request System discussion list(ARSList) [mailto: >> [email protected]] *On Behalf Of *Rick Cook >> *Sent:* 13 October 2017 02:12 >> *To:* [email protected] >> *Subject:* Re: What's the NextId? >> >> >> >> ** >> >> Here's the skinny on that. I got this from the engineer who built that >> feature, btw. >> >> >> >> The problem was that system performance was being constricted around the >> action of getting the NextId for a record when multiple sources (say, >> Netcool and HPOV) were throwing tons of requests at the (Incident) form at >> the same time. The process of getting each Next Entry ID in individual DB >> calls was bottlenecking the process of creating the records. So that's why >> BMC came up with a way to pre-allocate those in bulk, so that only every N >> times (whatever the number is set to) would an actual call to the DB to get >> Next IDs be necessary. The transaction time to retrieve 1 or 100 wasn't >> much different, and those customers with multiple programs requiring many >> simultaneous record creations saw a marked performance increase. It was, >> and is, a good feature add. >> >> >> >> So (then a miracle occurs) and BMC announces that this particular corner >> case had a solution that everyone should benefit from, and announced that >> the number should be 100 for *all* customers. I tested this back in 7.6.04 >> against a RH server, with settings at 1, 10, 100, and 1000, and found that >> performance was actually NEGATIVELY affected the higher the number was >> set. It wasn't a huge difference (10%~), but it was a clear, repeatable >> one for which BMC's engineer had no explanation. It is why I have always >> advocated that unless a customer has the specific set of circumstances that >> caused the feature to be beneficial, there is no real benefit to setting >> the number larger than 1. And there are minor drawbacks to doing so, the >> current subject being one of them. >> >> >> >> Every time I ask someone from BMC to justify a larger number to the >> average customer, they repeat the party line, unaware of the history behind >> the feature. I will continue to tilt at this windmill until someone at BMC >> shows me some performance testing numbers that justify this setting for the >> entire customer base. >> >> Rick >> >> >> >> On Oct 12, 2017 13:30, "Thomas Miskiewicz" <[email protected]> wrote: >> >> ** >> >> i.e. there is no hack to find out the nextID before it actually gets >> submitted? >> >> >> >> Apart from that, I really don’t understand why BMC makes such a fuss >> around the nextID. Why can’t they just provide a special command GET-NEXTID? >> >> >> >> >> >> Thomas >> >> >> >> >> >> On Oct 12, 2017, at 10:26 PM, LJ LongWing <[email protected]> wrote: >> >> >> >> ** >> >> There are no interfaces that I'm aware of to ask a specific server what >> the next id it will hand out for a specific form is >> >> >> >> On Thu, Oct 12, 2017 at 2:14 PM, Thomas Miskiewicz <[email protected]> >> wrote: >> >> Hello List, >> >> with NextID Block size being set to 100 —> is it possible the find out >> using the API which will be the next Request-ID that the server will assign >> to a request? Or what other options do I have to find out? >> >> >> --Thomas >> >> ____________________________________________________________ >> ___________________ >> UNSUBSCRIBE or access ARSlist Archives at www.arslist.org >> "Where the Answers Are, and have been for 20 years" >> >> >> >> _ARSlist: "Where the Answers Are" and have been for 20 years_ >> >> >> >> _ARSlist: "Where the Answers Are" and have been for 20 years_ >> >> _ARSlist: "Where the Answers Are" and have been for 20 years_ >> _ARSlist: "Where the Answers Are" and have been for 20 years_ >> > > _ARSlist: "Where the Answers Are" and have been for 20 years_ > _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org "Where the Answers Are, and have been for 20 years"
