Re: What's the NextId?

[Thomas Miskiewicz]

;> demands on the table, and few on the system. Therefore, the system of 
>> generating one ID at a time had a cumulatively lower transaction time than 
>> it did when it had to wait for multiples to be retrieved and then allocated 
>> individually. The BMC Engineer, who I know to be very smart and experienced 
>> with Remedy, had no explanation for my results. I believe this to be that 
>> BMC didn't test against a control (normal) system, and therefore had no data 
>> on its effect on them.
>> Why then, they chose to recommend that setting to all customers, is a 
>> mystery to me.
>> Rick Cook
>> On Thu, Oct 12, 2017 at 4:56 PM, Gadgil, Abhijeet <abhijeet_gad...@bmc.com 
>> <mailto:abhijeet_gad...@bmc.com>> wrote:
>> **
>> Rick, I do not think that is accurate.
>> 
>> Logically, if the block size is 100 then the server will access the DB once 
>> to retrieve the next block id every 100 records. If it is 1, then the server 
>> goes to the DB for every record.
>> 
>> Later cannot be faster.
>> 
>> Further, to answer the original question, returning the next id in an API 
>> call would mean preventing creation of ids by anyone other than the caller 
>> until the id has actually been used -- otherwise the information might be 
>> obsolete before the client receives it. That is the reason for not exposing 
>> it via an API
>> 
>> Regards,
>> 
>> Abhijeet
>> 
>> From: Action Request System discussion list(ARSList) 
>> [mailto:arslist@ARSLIST.ORG <mailto:arslist@ARSLIST.ORG>] On Behalf Of Rick 
>> Cook
>> Sent: 13 October 2017 02:12
>> To: arslist@ARSLIST.ORG <mailto:arslist@ARSLIST.ORG>
>> Subject: Re: What's the NextId?
>> 
>> **
>> 
>> Here's the skinny on that. I got this from the engineer who built that 
>> feature, btw.
>> 
>> The problem was that system performance was being constricted around the 
>> action of getting the NextId for a record when multiple sources (say, 
>> Netcool and HPOV) were throwing tons of requests at the (Incident) form at 
>> the same time. The process of getting each Next Entry ID in individual DB 
>> calls was bottlenecking the process of creating the records. So that's why 
>> BMC came up with a way to pre-allocate those in bulk, so that only every N 
>> times (whatever the number is set to) would an actual call to the DB to get 
>> Next IDs be necessary. The transaction time to retrieve 1 or 100 wasn't much 
>> different, and those customers with multiple programs requiring many 
>> simultaneous record creations saw a marked performance increase. It was, and 
>> is, a good feature add.
>> 
>> So (then a miracle occurs) and BMC announces that this particular corner 
>> case had a solution that everyone should benefit from, and announced that 
>> the number should be 100 for *all* customers. I tested this back in 7.6.04 
>> against a RH server, with settings at 1, 10, 100, and 1000, and found that 
>> performance was actually NEGATIVELY affected the higher the number was set. 
>> It wasn't a huge difference (10%~), but it was a clear, repeatable one for 
>> which BMC's engineer had no explanation. It is why I have always advocated 
>> that unless a customer has the specific set of circumstances that caused the 
>> feature to be beneficial, there is no real benefit to setting the number 
>> larger than 1. And there are minor drawbacks to doing so, the current 
>> subject being one of them.
>> 
>> Every time I ask someone from BMC to justify a larger number to the average 
>> customer, they repeat the party line, unaware of the history behind the 
>> feature. I will continue to tilt at this windmill until someone at BMC shows 
>> me some performance testing numbers that justify this setting for the entire 
>> customer base.
>> 
>> Rick
>> 
>> On Oct 12, 2017 13:30, "Thomas Miskiewicz" <tmisk...@gmail.com 
>> <mailto:tmisk...@gmail.com>> wrote:
>> 
>> **
>> 
>> i.e. there is no hack to find out the nextID before it actually gets 
>> submitted?
>> 
>> Apart from that, I really don’t understand why BMC makes such a fuss around 
>> the nextID. Why can’t they just provide a special command GET-NEXTID?
>> 
>> Thomas
>> 
>> On Oct 12, 2017, at 10:26 PM, LJ LongWing <lj.longw...@gmail.com 
>> <mailto:lj.longw...@gmail.com>> wrote:
>> 
>> **
>> 
>> There are no interfaces that I'm aware of to ask a specific server what the 
>> next id it will hand out for a specific form is

Re: What's the NextId?

[LJ LongWing]

In general, when building a client/server application (Remedy server is the
client, DB is the server in this example), it's best to limit the
communication between the client and the server to the absolute minimum
necessary to get the work done.  In this case, forcing every submit to a
table to do 1 extra call to the db to get the next id, vs knowing what the
next id is already in memory reduces the number of calls to the db, thus
'should' increase performanceRick has stated numerous times over the
years that the caching of id's slowed performanceI've never personally
tested the different scenarios so I can't speak to themI agree that the
'everyone needs this' approach that BMC took at the time seemed a bit heavy
handed, but at the same time, in general it doesn't seem to 'hurt' things
by having them run this way...so...I typically don't bother changing the
defaults unless I have reason in a specific situation.

On Fri, Oct 13, 2017 at 7:49 AM, Grooms, Frederick W <
frederick.w.gro...@xo.com> wrote:

> **
>
> Where is there a difference in the time to create the next ID
>
>UPDATE ARSCHEMA SET NEXTID = NEXTID + 1 WHERE SCHEMAID = x
>
>UPDATE ARSCHEMA SET NEXTID = NEXTID + 100 WHERE SCHEMAID = x
>
> It is not like the database goes 1+1+1+1+1+1…   with a save to the file
> each time.
>
>
>
> My db bottleneck was pure ARS workflow.  Legal required an audit trail on
> some 15 fields of the main form.  One user doing an update on the main
> form could then generate 15 new child records in an audit table.  Multiply
> that by 200 users and you get a possibility of 3000 new records at a given
> time for a single table.  Actual results in the SQL.log showed that in a
> 5 minute timeframe the Set NEXTID was called for the audit table 23 times
> when the size was set to 20 (460 new records in the audit table in 5
> minutes from 1 ARS server).  Bumping the size to 100 eliminated the
> Oracle db_lock contention errors.
>
>
>
> *From:* Action Request System discussion list(ARSList) [
> mailto:arslist@ARSLIST.ORG <arslist@ARSLIST.ORG>] *On Behalf Of *Rick Cook
> *Sent:* Thursday, October 12, 2017 7:27 PM
>
> *To:* arslist@ARSLIST.ORG
> *Subject:* Re: What's the NextId?
>
>
>
> **
>
> Abhijeet, my results were repeated and conclusive.  Here is the only
> logical explanation for my findings that makes sense:
>
>
>
> There is a time cost to the DB call for Entry IDs.  The cost increases
> incrementally the more that are requested at once.  The thought is that 100
> individual requests will take substantially more time than retrieving 100
> IDs in one request.  That thought has been proven to be correct - it's why
> the feature was added. However, the difference in time was tested against a
> system that had multiple sources (NMS) attempting to grab multiple Entry
> IDs at the same time, not a normal system without those multiple outside
> requests, because it was the first environment that was reporting the
> problem with performance.
>
>
>
> On a system where the volume of IDs is high enough to cause a system
> bottleneck at the DB ID request, *which requires multiple simultaneous
> requests from different sources*, that bottleneck costs more additional
> time than is lost by the increased time it takes to request multiple IDs at
> one time.  However, on a system that is not bound by the Next ID calls to
> the DB, or has only a single source (AR System) requesting IDs, there is no
> advantage gained by the multiple requests, because the time "gained" by
> having a cached ID is negligible - too small to even notice.  And, as my
> testing proved (to my surprise), there was an increasing net cost to system
> performance as the number of IDs requested in a single call grew.  This
> must be because the time it takes to gather, say, 100 records in one call
> is actually *higher* than it is to do 100 individual calls - IF AR System
> is the only source of those calls.  The simple correlation is that if AR
> System is the only thing generating new Request ID requests to the DB, you
> don't need, and will not benefit from, a number larger than 1.  If you have
> multiple outside entities creating large number of records at once, you
> very well may benefit from it.
>
>
>
> My tests that showed decreasing performance as the Next Id block grew were
> creating about 1.5 million records, but, and this is important - all were
> from the same source - AR System workflow.  There were no other
> simultaneous demands on the table, and few on the system.  Therefore, the
> system of generating one ID at a time had a cumulatively lower transaction
> time than it did when it had to wait for multiples to be retrieved and then
> allocated individually.  The BMC E

Re: What's the NextId?

[Grooms, Frederick W]

Where is there a difference in the time to create the next ID
   UPDATE ARSCHEMA SET NEXTID = NEXTID + 1 WHERE SCHEMAID = x
   UPDATE ARSCHEMA SET NEXTID = NEXTID + 100 WHERE SCHEMAID = x
It is not like the database goes 1+1+1+1+1+1…   with a save to the file each 
time.

My db bottleneck was pure ARS workflow.  Legal required an audit trail on some 
15 fields of the main form.  One user doing an update on the main form could 
then generate 15 new child records in an audit table.  Multiply that by 200 
users and you get a possibility of 3000 new records at a given time for a 
single table.  Actual results in the SQL.log showed that in a 5 minute 
timeframe the Set NEXTID was called for the audit table 23 times when the size 
was set to 20 (460 new records in the audit table in 5 minutes from 1 ARS 
server).  Bumping the size to 100 eliminated the Oracle db_lock contention 
errors.

From: Action Request System discussion list(ARSList) 
[mailto:arslist@ARSLIST.ORG] On Behalf Of Rick Cook
Sent: Thursday, October 12, 2017 7:27 PM
To: arslist@ARSLIST.ORG<mailto:arslist@ARSLIST.ORG>
Subject: Re: What's the NextId?

**
Abhijeet, my results were repeated and conclusive.  Here is the only logical 
explanation for my findings that makes sense:

There is a time cost to the DB call for Entry IDs.  The cost increases 
incrementally the more that are requested at once.  The thought is that 100 
individual requests will take substantially more time than retrieving 100 IDs 
in one request.  That thought has been proven to be correct - it's why the 
feature was added. However, the difference in time was tested against a system 
that had multiple sources (NMS) attempting to grab multiple Entry IDs at the 
same time, not a normal system without those multiple outside requests, because 
it was the first environment that was reporting the problem with performance.

On a system where the volume of IDs is high enough to cause a system bottleneck 
at the DB ID request, which requires multiple simultaneous requests from 
different sources, that bottleneck costs more additional time than is lost by 
the increased time it takes to request multiple IDs at one time.  However, on a 
system that is not bound by the Next ID calls to the DB, or has only a single 
source (AR System) requesting IDs, there is no advantage gained by the multiple 
requests, because the time "gained" by having a cached ID is negligible - too 
small to even notice.  And, as my testing proved (to my surprise), there was an 
increasing net cost to system performance as the number of IDs requested in a 
single call grew.  This must be because the time it takes to gather, say, 100 
records in one call is actually *higher* than it is to do 100 individual calls 
- IF AR System is the only source of those calls.  The simple correlation is 
that if AR System is the only thing generating new Request ID requests to the 
DB, you don't need, and will not benefit from, a number larger than 1.  If you 
have multiple outside entities creating large number of records at once, you 
very well may benefit from it.

My tests that showed decreasing performance as the Next Id block grew were 
creating about 1.5 million records, but, and this is important - all were from 
the same source - AR System workflow.  There were no other simultaneous demands 
on the table, and few on the system.  Therefore, the system of generating one 
ID at a time had a cumulatively lower transaction time than it did when it had 
to wait for multiples to be retrieved and then allocated individually.  The BMC 
Engineer, who I know to be very smart and experienced with Remedy, had no 
explanation for my results.  I believe this to be that BMC didn't test against 
a control (normal) system, and therefore had no data on its effect on them.

Why then, they chose to recommend that setting to all customers, is a mystery 
to me.

Rick Cook

On Thu, Oct 12, 2017 at 4:56 PM, Gadgil, Abhijeet 
<abhijeet_gad...@bmc.com<mailto:abhijeet_gad...@bmc.com>> wrote:
**
Rick, I do not think that is accurate.
Logically, if the block size is 100 then the server will access the DB once to 
retrieve the next block id every 100 records.  If it is 1, then the server goes 
to the DB for every record.
Later cannot be faster.

Further, to answer the original question, returning the next id in an API call 
would mean preventing creation of ids by anyone other than the caller until the 
id has actually been used -- otherwise the information might be obsolete before 
the client receives it. That is the reason for not exposing it via an API

Regards,
Abhijeet


From: Action Request System discussion list(ARSList) 
[mailto:arslist@ARSLIST.ORG<mailto:arslist@ARSLIST.ORG>] On Behalf Of Rick Cook
Sent: 13 October 2017 02:12
To: arslist@ARSLIST.ORG<mailto:arslist@ARSLIST.ORG>
Subject: Re: What's the NextId?

**
Here's the skinny on that.  I got this from the engineer who built that 
feature, btw.

Re: What's the NextId?

[LJ LongWing]

s.  The simple correlation is that if AR
>> System is the only thing generating new Request ID requests to the DB, you
>> don't need, and will not benefit from, a number larger than 1.  If you have
>> multiple outside entities creating large number of records at once, you
>> very well may benefit from it.
>>
>> My tests that showed decreasing performance as the Next Id block grew
>> were creating about 1.5 million records, but, and this is important - all
>> were from the same source - AR System workflow.  There were no other
>> simultaneous demands on the table, and few on the system.  Therefore, the
>> system of generating one ID at a time had a cumulatively lower transaction
>> time than it did when it had to wait for multiples to be retrieved and then
>> allocated individually.  The BMC Engineer, who I know to be very smart and
>> experienced with Remedy, had no explanation for my results.  I believe this
>> to be that BMC didn't test against a control (normal) system, and therefore
>> had no data on its effect on them.
>>
>> Why then, they chose to recommend that setting to all customers, is a
>> mystery to me.
>>
>> Rick Cook
>>
>> On Thu, Oct 12, 2017 at 4:56 PM, Gadgil, Abhijeet <
>> abhijeet_gad...@bmc.com> wrote:
>>
>>> **
>>>
>>> Rick, I do not think that is accurate.
>>>
>>> Logically, if the block size is 100 then the server will access the DB
>>> once to retrieve the next block id every 100 records.  If it is 1, then the
>>> server goes to the DB for every record.
>>>
>>> Later cannot be faster.
>>>
>>>
>>>
>>> Further, to answer the original question, returning the next id in an
>>> API call would mean preventing creation of ids by anyone other than the
>>> caller until the id has actually been used -- otherwise the information
>>> might be obsolete before the client receives it. That is the reason for not
>>> exposing it via an API
>>>
>>>
>>>
>>> Regards,
>>>
>>> Abhijeet
>>>
>>>
>>>
>>>
>>>
>>> *From:* Action Request System discussion list(ARSList) [mailto:
>>> arslist@ARSLIST.ORG] *On Behalf Of *Rick Cook
>>> *Sent:* 13 October 2017 02:12
>>> *To:* arslist@ARSLIST.ORG
>>> *Subject:* Re: What's the NextId?
>>>
>>>
>>>
>>> **
>>>
>>> Here's the skinny on that.  I got this from the engineer who built that
>>> feature, btw.
>>>
>>>
>>>
>>> The problem was that system performance was being constricted around the
>>> action of getting the NextId for a record when multiple sources (say,
>>> Netcool and HPOV) were throwing tons of requests at the (Incident) form at
>>> the same time.  The process of getting each Next Entry ID in individual DB
>>> calls was bottlenecking the process of creating the records.  So that's why
>>> BMC came up with a way to pre-allocate those in bulk, so that only every N
>>> times (whatever the number is set to) would an actual call to the DB to get
>>> Next IDs be necessary.  The transaction time to retrieve 1 or 100 wasn't
>>> much different, and those customers with multiple programs requiring many
>>> simultaneous record creations saw a marked performance increase.  It was,
>>> and is, a good feature add.
>>>
>>>
>>>
>>> So (then a miracle occurs) and BMC announces that this particular corner
>>> case had a solution that everyone should benefit from, and announced that
>>> the number should be 100 for *all* customers.  I tested this back in 7.6.04
>>> against a RH server, with settings at 1, 10, 100, and 1000, and found that
>>> performance was actually NEGATIVELY affected the higher the number was
>>> set.  It wasn't a huge difference (10%~), but it was a clear, repeatable
>>> one for which BMC's engineer had no explanation.  It is why I have always
>>> advocated that unless a customer has the specific set of circumstances that
>>> caused the feature to be beneficial, there is no real benefit to setting
>>> the number larger than 1.  And there are minor drawbacks to doing so, the
>>> current subject being one of them.
>>>
>>>
>>>
>>> Every time I ask someone from BMC to justify a larger number to the
>>> average customer, they repeat the party line, unaware of the history behind
>>> the

Re: What's the NextId?

[Misi Mladoniczky]

high load is likely to go down (due to the system being under heavy load, other 
downstream constraints such as CPU/Disk Speed start to appear, where they would 
not appear if the NextID bottleneck was in place) It effectively acts as a 
throttle. So your results could be easily misinterpreted. 
I would think a way to confirm this is to generate parallel load from a high 
number of worker threads, and time how long it took to reach the same number of 
records created with 1 or 100 for the NextID chunk size. Then compare the 
times. Anyway, we are off on a tangent from the original topic 
Thomas, I am actually curious as to why you want to know a nextid. What 
Abhijeet said explains why it doesn't make sense to ask for it, but often in 
workflow scenarios, where you want to generate something unique and refer/tie 
back to something else. you are better off using Application-Generate-GUID [" 
GUIDPrefix" ]  
On Thu, Oct 12, 2017 at 5:27 PM, Rick Cook  wrote:**Abhijeet, my results were 
repeated and conclusive. Here is the only logical explanation for my findings 
that makes sense:
There is a time cost to the DB call for Entry IDs. The cost increases 
incrementally the more that are requested at once. The thought is that 100 
individual requests will take substantially more time than retrieving 100 IDs 
in one request. That thought has been proven to be correct - it's why the 
feature was added. However, the difference in time was tested against a system 
that had multiple sources (NMS) attempting to grab multiple Entry IDs at the 
same time, not a normal system without those multiple outside requests, because 
it was the first environment that was reporting the problem with performance. 
On a system where the volume of IDs is high enough to cause a system bottleneck 
at the DB ID request, which requires multiple simultaneous requests from 
different sources, that bottleneck costs more additional time than is lost by 
the increased time it takes to request multiple IDs at one time. However, on a 
system that is not bound by the Next ID calls to the DB, or has only a single 
source (AR System) requesting IDs, there is no advantage gained by the multiple 
requests, because the time "gained" by having a cached ID is negligible - too 
small to even notice. And, as my testing proved (to my surprise), there was an 
increasing net cost to system performance as the number of IDs requested in a 
single call grew. This must be because the time it takes to gather, say, 100 
records in one call is actually *higher* than it is to do 100 individual calls 
- IF AR System is the only source of those calls. The simple correlation is 
that if AR System is the only thing generating new Request ID requests to the 
DB, you don't need, and will not benefit from, a number larger than 1. If you 
have multiple outside entities creating large number of records at once, you 
very well may benefit from it. 
My tests that showed decreasing performance as the Next Id block grew were 
creating about 1.5 million records, but, and this is important - all were from 
the same source - AR System workflow. There were no other simultaneous demands 
on the table, and few on the system. Therefore, the system of generating one ID 
at a time had a cumulatively lower transaction time than it did when it had to 
wait for multiples to be retrieved and then allocated individually. The BMC 
Engineer, who I know to be very smart and experienced with Remedy, had no 
explanation for my results. I believe this to be that BMC didn't test against a 
control (normal) system, and therefore had no data on its effect on them. 
Why then, they chose to recommend that setting to all customers, is a mystery 
to me.
Rick Cook 
On Thu, Oct 12, 2017 at 4:56 PM, Gadgil, Abhijeet  wrote:**
Rick, I do not think that is accurate. 

Logically, if the block size is 100 then the server will access the DB 
once to retrieve the next block id every 100 records. If it is 1, then the 
server goes to the DB for every record. 

Later cannot be faster. 

Further, to answer the original question, returning the next id in an 
API call would mean preventing creation of ids by anyone other than the caller 
until the id has actually been used -- otherwise the information might be 
obsolete before the client receives it. That is the reason for not exposing it 
via an API 

Regards, 

Abhijeet 

From: Action Request System discussion list(ARSList) 
[mailto:arslist@ARSLIST.ORG (mailto:arslist@ARSLIST.ORG)] On Behalf Of Rick Cook
Sent: 13 October 2017 02:12
To: arslist@ARSLIST.ORG (mailto:arslist@ARSLIST.ORG)
Subject: Re: What's the NextId? 

** 
Here's the skinny on that. I got this from the engineer who built that 
feature, btw. 
The problem was that system performance was being constricted around 
the action of getting the NextId for a record when multiple sources (say, 
Netcool 

Re: What's the NextId?

[Rick Cook]

at 100
>>> individual requests will take substantially more time than retrieving 100
>>> IDs in one request.  That thought has been proven to be correct - it's why
>>> the feature was added. However, the difference in time was tested against a
>>> system that had multiple sources (NMS) attempting to grab multiple Entry
>>> IDs at the same time, not a normal system without those multiple outside
>>> requests, because it was the first environment that was reporting the
>>> problem with performance.
>>>
>>> On a system where the volume of IDs is high enough to cause a system
>>> bottleneck at the DB ID request, *which requires multiple simultaneous
>>> requests from different sources*, that bottleneck costs more additional
>>> time than is lost by the increased time it takes to request multiple IDs at
>>> one time.  However, on a system that is not bound by the Next ID calls to
>>> the DB, or has only a single source (AR System) requesting IDs, there is no
>>> advantage gained by the multiple requests, because the time "gained" by
>>> having a cached ID is negligible - too small to even notice.  And, as my
>>> testing proved (to my surprise), there was an increasing net cost to system
>>> performance as the number of IDs requested in a single call grew.  This
>>> must be because the time it takes to gather, say, 100 records in one call
>>> is actually *higher* than it is to do 100 individual calls - IF AR System
>>> is the only source of those calls.  The simple correlation is that if AR
>>> System is the only thing generating new Request ID requests to the DB, you
>>> don't need, and will not benefit from, a number larger than 1.  If you have
>>> multiple outside entities creating large number of records at once, you
>>> very well may benefit from it.
>>>
>>> My tests that showed decreasing performance as the Next Id block grew
>>> were creating about 1.5 million records, but, and this is important - all
>>> were from the same source - AR System workflow.  There were no other
>>> simultaneous demands on the table, and few on the system.  Therefore, the
>>> system of generating one ID at a time had a cumulatively lower transaction
>>> time than it did when it had to wait for multiples to be retrieved and then
>>> allocated individually.  The BMC Engineer, who I know to be very smart and
>>> experienced with Remedy, had no explanation for my results.  I believe this
>>> to be that BMC didn't test against a control (normal) system, and therefore
>>> had no data on its effect on them.
>>>
>>> Why then, they chose to recommend that setting to all customers, is a
>>> mystery to me.
>>>
>>> Rick Cook
>>>
>>> On Thu, Oct 12, 2017 at 4:56 PM, Gadgil, Abhijeet <
>>> abhijeet_gad...@bmc.com> wrote:
>>>
>>>> **
>>>>
>>>> Rick, I do not think that is accurate.
>>>>
>>>> Logically, if the block size is 100 then the server will access the DB
>>>> once to retrieve the next block id every 100 records.  If it is 1, then the
>>>> server goes to the DB for every record.
>>>>
>>>> Later cannot be faster.
>>>>
>>>>
>>>>
>>>> Further, to answer the original question, returning the next id in an
>>>> API call would mean preventing creation of ids by anyone other than the
>>>> caller until the id has actually been used -- otherwise the information
>>>> might be obsolete before the client receives it. That is the reason for not
>>>> exposing it via an API
>>>>
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Abhijeet
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *From:* Action Request System discussion list(ARSList) [mailto:
>>>> arslist@ARSLIST.ORG] *On Behalf Of *Rick Cook
>>>> *Sent:* 13 October 2017 02:12
>>>> *To:* arslist@ARSLIST.ORG
>>>> *Subject:* Re: What's the NextId?
>>>>
>>>>
>>>>
>>>> **
>>>>
>>>> Here's the skinny on that.  I got this from the engineer who built that
>>>> feature, btw.
>>>>
>>>>
>>>>
>>>> The problem was that system performance was being constricted around
>>>> the action of getting the NextId for a record when multiple sources (say,
>>>> Netcool and HPOV) were throwing tons of reques

Re: What's the NextId?

[Randeep Atwal]

IDs at
>> one time.  However, on a system that is not bound by the Next ID calls to
>> the DB, or has only a single source (AR System) requesting IDs, there is no
>> advantage gained by the multiple requests, because the time "gained" by
>> having a cached ID is negligible - too small to even notice.  And, as my
>> testing proved (to my surprise), there was an increasing net cost to system
>> performance as the number of IDs requested in a single call grew.  This
>> must be because the time it takes to gather, say, 100 records in one call
>> is actually *higher* than it is to do 100 individual calls - IF AR System
>> is the only source of those calls.  The simple correlation is that if AR
>> System is the only thing generating new Request ID requests to the DB, you
>> don't need, and will not benefit from, a number larger than 1.  If you have
>> multiple outside entities creating large number of records at once, you
>> very well may benefit from it.
>>
>> My tests that showed decreasing performance as the Next Id block grew
>> were creating about 1.5 million records, but, and this is important - all
>> were from the same source - AR System workflow.  There were no other
>> simultaneous demands on the table, and few on the system.  Therefore, the
>> system of generating one ID at a time had a cumulatively lower transaction
>> time than it did when it had to wait for multiples to be retrieved and then
>> allocated individually.  The BMC Engineer, who I know to be very smart and
>> experienced with Remedy, had no explanation for my results.  I believe this
>> to be that BMC didn't test against a control (normal) system, and therefore
>> had no data on its effect on them.
>>
>> Why then, they chose to recommend that setting to all customers, is a
>> mystery to me.
>>
>> Rick Cook
>>
>> On Thu, Oct 12, 2017 at 4:56 PM, Gadgil, Abhijeet <
>> abhijeet_gad...@bmc.com> wrote:
>>
>>> **
>>>
>>> Rick, I do not think that is accurate.
>>>
>>> Logically, if the block size is 100 then the server will access the DB
>>> once to retrieve the next block id every 100 records.  If it is 1, then the
>>> server goes to the DB for every record.
>>>
>>> Later cannot be faster.
>>>
>>>
>>>
>>> Further, to answer the original question, returning the next id in an
>>> API call would mean preventing creation of ids by anyone other than the
>>> caller until the id has actually been used -- otherwise the information
>>> might be obsolete before the client receives it. That is the reason for not
>>> exposing it via an API
>>>
>>>
>>>
>>> Regards,
>>>
>>> Abhijeet
>>>
>>>
>>>
>>>
>>>
>>> *From:* Action Request System discussion list(ARSList) [mailto:
>>> arslist@ARSLIST.ORG] *On Behalf Of *Rick Cook
>>> *Sent:* 13 October 2017 02:12
>>> *To:* arslist@ARSLIST.ORG
>>> *Subject:* Re: What's the NextId?
>>>
>>>
>>>
>>> **
>>>
>>> Here's the skinny on that.  I got this from the engineer who built that
>>> feature, btw.
>>>
>>>
>>>
>>> The problem was that system performance was being constricted around the
>>> action of getting the NextId for a record when multiple sources (say,
>>> Netcool and HPOV) were throwing tons of requests at the (Incident) form at
>>> the same time.  The process of getting each Next Entry ID in individual DB
>>> calls was bottlenecking the process of creating the records.  So that's why
>>> BMC came up with a way to pre-allocate those in bulk, so that only every N
>>> times (whatever the number is set to) would an actual call to the DB to get
>>> Next IDs be necessary.  The transaction time to retrieve 1 or 100 wasn't
>>> much different, and those customers with multiple programs requiring many
>>> simultaneous record creations saw a marked performance increase.  It was,
>>> and is, a good feature add.
>>>
>>>
>>>
>>> So (then a miracle occurs) and BMC announces that this particular corner
>>> case had a solution that everyone should benefit from, and announced that
>>> the number should be 100 for *all* customers.  I tested this back in 7.6.04
>>> against a RH server, with settings at 1, 10, 100, and 1000, and found that
>>> performance was actually NEGATIVELY affected the higher the number was
>>> set.  It wasn't a huge difference (10%~), but it

Re: What's the NextId?

[Thomas Miskiewicz]

re 
>> creating about 1.5 million records, but, and this is important - all were 
>> from the same source - AR System workflow.  There were no other simultaneous 
>> demands on the table, and few on the system.  Therefore, the system of 
>> generating one ID at a time had a cumulatively lower transaction time than 
>> it did when it had to wait for multiples to be retrieved and then allocated 
>> individually.  The BMC Engineer, who I know to be very smart and experienced 
>> with Remedy, had no explanation for my results.  I believe this to be that 
>> BMC didn't test against a control (normal) system, and therefore had no data 
>> on its effect on them.
>> 
>> Why then, they chose to recommend that setting to all customers, is a 
>> mystery to me.
>> 
>> Rick Cook
>> 
>>> On Thu, Oct 12, 2017 at 4:56 PM, Gadgil, Abhijeet <abhijeet_gad...@bmc.com> 
>>> wrote:
>>> **
>>> Rick, I do not think that is accurate.
>>> 
>>> Logically, if the block size is 100 then the server will access the DB once 
>>> to retrieve the next block id every 100 records.  If it is 1, then the 
>>> server goes to the DB for every record.
>>> 
>>> Later cannot be faster.
>>> 
>>>  
>>> 
>>> Further, to answer the original question, returning the next id in an API 
>>> call would mean preventing creation of ids by anyone other than the caller 
>>> until the id has actually been used -- otherwise the information might be 
>>> obsolete before the client receives it. That is the reason for not exposing 
>>> it via an API
>>> 
>>>  
>>> 
>>> Regards,
>>> 
>>> Abhijeet
>>> 
>>>  
>>> 
>>>  
>>> 
>>> From: Action Request System discussion list(ARSList) 
>>> [mailto:arslist@ARSLIST.ORG] On Behalf Of Rick Cook
>>> Sent: 13 October 2017 02:12
>>> To: arslist@ARSLIST.ORG
>>> Subject: Re: What's the NextId?
>>> 
>>>  
>>> 
>>> **
>>> 
>>> Here's the skinny on that.  I got this from the engineer who built that 
>>> feature, btw.
>>> 
>>>  
>>> 
>>> The problem was that system performance was being constricted around the 
>>> action of getting the NextId for a record when multiple sources (say, 
>>> Netcool and HPOV) were throwing tons of requests at the (Incident) form at 
>>> the same time.  The process of getting each Next Entry ID in individual DB 
>>> calls was bottlenecking the process of creating the records.  So that's why 
>>> BMC came up with a way to pre-allocate those in bulk, so that only every N 
>>> times (whatever the number is set to) would an actual call to the DB to get 
>>> Next IDs be necessary.  The transaction time to retrieve 1 or 100 wasn't 
>>> much different, and those customers with multiple programs requiring many 
>>> simultaneous record creations saw a marked performance increase.  It was, 
>>> and is, a good feature add.
>>> 
>>>  
>>> 
>>> So (then a miracle occurs) and BMC announces that this particular corner 
>>> case had a solution that everyone should benefit from, and announced that 
>>> the number should be 100 for *all* customers.  I tested this back in 7.6.04 
>>> against a RH server, with settings at 1, 10, 100, and 1000, and found that 
>>> performance was actually NEGATIVELY affected the higher the number was set. 
>>>  It wasn't a huge difference (10%~), but it was a clear, repeatable one for 
>>> which BMC's engineer had no explanation.  It is why I have always advocated 
>>> that unless a customer has the specific set of circumstances that caused 
>>> the feature to be beneficial, there is no real benefit to setting the 
>>> number larger than 1.  And there are minor drawbacks to doing so, the 
>>> current subject being one of them.
>>> 
>>>  
>>> 
>>> Every time I ask someone from BMC to justify a larger number to the average 
>>> customer, they repeat the party line, unaware of the history behind the 
>>> feature.  I will continue to tilt at this windmill until someone at BMC 
>>> shows me some performance testing numbers that justify this setting for the 
>>> entire customer base.
>>> 
>>> Rick
>>> 
>>>  
>>> 
>>> On Oct 12, 2017 13:30, "Thomas Miskiewicz" <tmisk...@gmail.com> wrote:
>>> 
>>> **
>>> 
>

Re: What's the NextId?

[Randeep Atwal]

me source - AR System workflow.  There were no other
>>> simultaneous demands on the table, and few on the system.  Therefore, the
>>> system of generating one ID at a time had a cumulatively lower transaction
>>> time than it did when it had to wait for multiples to be retrieved and then
>>> allocated individually.  The BMC Engineer, who I know to be very smart and
>>> experienced with Remedy, had no explanation for my results.  I believe this
>>> to be that BMC didn't test against a control (normal) system, and therefore
>>> had no data on its effect on them.
>>>
>>> Why then, they chose to recommend that setting to all customers, is a
>>> mystery to me.
>>>
>>> Rick Cook
>>>
>>> On Thu, Oct 12, 2017 at 4:56 PM, Gadgil, Abhijeet <
>>> abhijeet_gad...@bmc.com> wrote:
>>>
>>>> **
>>>>
>>>> Rick, I do not think that is accurate.
>>>>
>>>> Logically, if the block size is 100 then the server will access the DB
>>>> once to retrieve the next block id every 100 records.  If it is 1, then the
>>>> server goes to the DB for every record.
>>>>
>>>> Later cannot be faster.
>>>>
>>>>
>>>>
>>>> Further, to answer the original question, returning the next id in an
>>>> API call would mean preventing creation of ids by anyone other than the
>>>> caller until the id has actually been used -- otherwise the information
>>>> might be obsolete before the client receives it. That is the reason for not
>>>> exposing it via an API
>>>>
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Abhijeet
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *From:* Action Request System discussion list(ARSList) [mailto:
>>>> arslist@ARSLIST.ORG] *On Behalf Of *Rick Cook
>>>> *Sent:* 13 October 2017 02:12
>>>> *To:* arslist@ARSLIST.ORG
>>>> *Subject:* Re: What's the NextId?
>>>>
>>>>
>>>>
>>>> **
>>>>
>>>> Here's the skinny on that.  I got this from the engineer who built that
>>>> feature, btw.
>>>>
>>>>
>>>>
>>>> The problem was that system performance was being constricted around
>>>> the action of getting the NextId for a record when multiple sources (say,
>>>> Netcool and HPOV) were throwing tons of requests at the (Incident) form at
>>>> the same time.  The process of getting each Next Entry ID in individual DB
>>>> calls was bottlenecking the process of creating the records.  So that's why
>>>> BMC came up with a way to pre-allocate those in bulk, so that only every N
>>>> times (whatever the number is set to) would an actual call to the DB to get
>>>> Next IDs be necessary.  The transaction time to retrieve 1 or 100 wasn't
>>>> much different, and those customers with multiple programs requiring many
>>>> simultaneous record creations saw a marked performance increase.  It was,
>>>> and is, a good feature add.
>>>>
>>>>
>>>>
>>>> So (then a miracle occurs) and BMC announces that this particular
>>>> corner case had a solution that everyone should benefit from, and announced
>>>> that the number should be 100 for *all* customers.  I tested this back in
>>>> 7.6.04 against a RH server, with settings at 1, 10, 100, and 1000, and
>>>> found that performance was actually NEGATIVELY affected the higher the
>>>> number was set.  It wasn't a huge difference (10%~), but it was a clear,
>>>> repeatable one for which BMC's engineer had no explanation.  It is why I
>>>> have always advocated that unless a customer has the specific set of
>>>> circumstances that caused the feature to be beneficial, there is no real
>>>> benefit to setting the number larger than 1.  And there are minor drawbacks
>>>> to doing so, the current subject being one of them.
>>>>
>>>>
>>>>
>>>> Every time I ask someone from BMC to justify a larger number to the
>>>> average customer, they repeat the party line, unaware of the history behind
>>>> the feature.  I will continue to tilt at this windmill until someone at BMC
>>>> shows me some performance testing numbers that justify this setting for the
>>>> entire customer base.
>>>>
>>>> Ri

Re: What's the NextId?

[Dave Shellman]

 that BMC didn't test against a control (normal) system, and therefore
>> had no data on its effect on them.
>>
>> Why then, they chose to recommend that setting to all customers, is a
>> mystery to me.
>>
>> Rick Cook
>>
>> On Thu, Oct 12, 2017 at 4:56 PM, Gadgil, Abhijeet <
>> abhijeet_gad...@bmc.com> wrote:
>>
>>> **
>>>
>>> Rick, I do not think that is accurate.
>>>
>>> Logically, if the block size is 100 then the server will access the DB
>>> once to retrieve the next block id every 100 records.  If it is 1, then the
>>> server goes to the DB for every record.
>>>
>>> Later cannot be faster.
>>>
>>>
>>>
>>> Further, to answer the original question, returning the next id in an
>>> API call would mean preventing creation of ids by anyone other than the
>>> caller until the id has actually been used -- otherwise the information
>>> might be obsolete before the client receives it. That is the reason for not
>>> exposing it via an API
>>>
>>>
>>>
>>> Regards,
>>>
>>> Abhijeet
>>>
>>>
>>>
>>>
>>>
>>> *From:* Action Request System discussion list(ARSList) [mailto:
>>> arslist@ARSLIST.ORG] *On Behalf Of *Rick Cook
>>> *Sent:* 13 October 2017 02:12
>>> *To:* arslist@ARSLIST.ORG
>>> *Subject:* Re: What's the NextId?
>>>
>>>
>>>
>>> **
>>>
>>> Here's the skinny on that.  I got this from the engineer who built that
>>> feature, btw.
>>>
>>>
>>>
>>> The problem was that system performance was being constricted around the
>>> action of getting the NextId for a record when multiple sources (say,
>>> Netcool and HPOV) were throwing tons of requests at the (Incident) form at
>>> the same time.  The process of getting each Next Entry ID in individual DB
>>> calls was bottlenecking the process of creating the records.  So that's why
>>> BMC came up with a way to pre-allocate those in bulk, so that only every N
>>> times (whatever the number is set to) would an actual call to the DB to get
>>> Next IDs be necessary.  The transaction time to retrieve 1 or 100 wasn't
>>> much different, and those customers with multiple programs requiring many
>>> simultaneous record creations saw a marked performance increase.  It was,
>>> and is, a good feature add.
>>>
>>>
>>>
>>> So (then a miracle occurs) and BMC announces that this particular corner
>>> case had a solution that everyone should benefit from, and announced that
>>> the number should be 100 for *all* customers.  I tested this back in 7.6.04
>>> against a RH server, with settings at 1, 10, 100, and 1000, and found that
>>> performance was actually NEGATIVELY affected the higher the number was
>>> set.  It wasn't a huge difference (10%~), but it was a clear, repeatable
>>> one for which BMC's engineer had no explanation.  It is why I have always
>>> advocated that unless a customer has the specific set of circumstances that
>>> caused the feature to be beneficial, there is no real benefit to setting
>>> the number larger than 1.  And there are minor drawbacks to doing so, the
>>> current subject being one of them.
>>>
>>>
>>>
>>> Every time I ask someone from BMC to justify a larger number to the
>>> average customer, they repeat the party line, unaware of the history behind
>>> the feature.  I will continue to tilt at this windmill until someone at BMC
>>> shows me some performance testing numbers that justify this setting for the
>>> entire customer base.
>>>
>>> Rick
>>>
>>>
>>>
>>> On Oct 12, 2017 13:30, "Thomas Miskiewicz" <tmisk...@gmail.com> wrote:
>>>
>>> **
>>>
>>> i.e. there is no hack to find out the nextID before it actually gets
>>> submitted?
>>>
>>>
>>>
>>> Apart from that, I really don’t understand why BMC makes such a fuss
>>> around the nextID. Why can’t they just provide a special command GET-NEXTID?
>>>
>>>
>>>
>>>
>>>
>>> Thomas
>>>
>>>
>>>
>>>
>>>
>>> On Oct 12, 2017, at 10:26 PM, LJ LongWing <lj.longw...@gmail.com> wrote:
>>>
>>>
>>>
>>> **
>>>
>>> There are no interfaces that I'm aware of to ask a specific server what
>>> the next id it will hand out for a specific form is
>>>
>>>
>>>
>>> On Thu, Oct 12, 2017 at 2:14 PM, Thomas Miskiewicz <tmisk...@gmail.com>
>>> wrote:
>>>
>>> Hello List,
>>>
>>> with NextID Block size being set to 100 —>  is it possible the find out
>>> using the API which will be the next Request-ID that the server will assign
>>> to a request? Or what other options do I have to find out?
>>>
>>>
>>> --Thomas
>>>
>>>
>>> ___
>>> UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
>>> "Where the Answers Are, and have been for 20 years"
>>>
>>>
>>>
>>> _ARSlist: "Where the Answers Are" and have been for 20 years_
>>>
>>>
>>>
>>> _ARSlist: "Where the Answers Are" and have been for 20 years_
>>>
>>> _ARSlist: "Where the Answers Are" and have been for 20 years_
>>> _ARSlist: "Where the Answers Are" and have been for 20 years_
>>>
>>
>> _ARSlist: "Where the Answers Are" and have been for 20 years_
>>
>
> _ARSlist: "Where the Answers Are" and have been for 20 years_

___
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
"Where the Answers Are, and have been for 20 years"

Re: What's the NextId?

[Randeep Atwal]

.
>>
>>
>>
>> Further, to answer the original question, returning the next id in an API
>> call would mean preventing creation of ids by anyone other than the caller
>> until the id has actually been used -- otherwise the information might be
>> obsolete before the client receives it. That is the reason for not exposing
>> it via an API
>>
>>
>>
>> Regards,
>>
>> Abhijeet
>>
>>
>>
>>
>>
>> *From:* Action Request System discussion list(ARSList) [mailto:
>> arslist@ARSLIST.ORG] *On Behalf Of *Rick Cook
>> *Sent:* 13 October 2017 02:12
>> *To:* arslist@ARSLIST.ORG
>> *Subject:* Re: What's the NextId?
>>
>>
>>
>> **
>>
>> Here's the skinny on that.  I got this from the engineer who built that
>> feature, btw.
>>
>>
>>
>> The problem was that system performance was being constricted around the
>> action of getting the NextId for a record when multiple sources (say,
>> Netcool and HPOV) were throwing tons of requests at the (Incident) form at
>> the same time.  The process of getting each Next Entry ID in individual DB
>> calls was bottlenecking the process of creating the records.  So that's why
>> BMC came up with a way to pre-allocate those in bulk, so that only every N
>> times (whatever the number is set to) would an actual call to the DB to get
>> Next IDs be necessary.  The transaction time to retrieve 1 or 100 wasn't
>> much different, and those customers with multiple programs requiring many
>> simultaneous record creations saw a marked performance increase.  It was,
>> and is, a good feature add.
>>
>>
>>
>> So (then a miracle occurs) and BMC announces that this particular corner
>> case had a solution that everyone should benefit from, and announced that
>> the number should be 100 for *all* customers.  I tested this back in 7.6.04
>> against a RH server, with settings at 1, 10, 100, and 1000, and found that
>> performance was actually NEGATIVELY affected the higher the number was
>> set.  It wasn't a huge difference (10%~), but it was a clear, repeatable
>> one for which BMC's engineer had no explanation.  It is why I have always
>> advocated that unless a customer has the specific set of circumstances that
>> caused the feature to be beneficial, there is no real benefit to setting
>> the number larger than 1.  And there are minor drawbacks to doing so, the
>> current subject being one of them.
>>
>>
>>
>> Every time I ask someone from BMC to justify a larger number to the
>> average customer, they repeat the party line, unaware of the history behind
>> the feature.  I will continue to tilt at this windmill until someone at BMC
>> shows me some performance testing numbers that justify this setting for the
>> entire customer base.
>>
>> Rick
>>
>>
>>
>> On Oct 12, 2017 13:30, "Thomas Miskiewicz" <tmisk...@gmail.com> wrote:
>>
>> **
>>
>> i.e. there is no hack to find out the nextID before it actually gets
>> submitted?
>>
>>
>>
>> Apart from that, I really don’t understand why BMC makes such a fuss
>> around the nextID. Why can’t they just provide a special command GET-NEXTID?
>>
>>
>>
>>
>>
>> Thomas
>>
>>
>>
>>
>>
>> On Oct 12, 2017, at 10:26 PM, LJ LongWing <lj.longw...@gmail.com> wrote:
>>
>>
>>
>> **
>>
>> There are no interfaces that I'm aware of to ask a specific server what
>> the next id it will hand out for a specific form is
>>
>>
>>
>> On Thu, Oct 12, 2017 at 2:14 PM, Thomas Miskiewicz <tmisk...@gmail.com>
>> wrote:
>>
>> Hello List,
>>
>> with NextID Block size being set to 100 —>  is it possible the find out
>> using the API which will be the next Request-ID that the server will assign
>> to a request? Or what other options do I have to find out?
>>
>>
>> --Thomas
>>
>> 
>> ___
>> UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
>> "Where the Answers Are, and have been for 20 years"
>>
>>
>>
>> _ARSlist: "Where the Answers Are" and have been for 20 years_
>>
>>
>>
>> _ARSlist: "Where the Answers Are" and have been for 20 years_
>>
>> _ARSlist: "Where the Answers Are" and have been for 20 years_
>> _ARSlist: "Where the Answers Are" and have been for 20 years_
>>
>
> _ARSlist: "Where the Answers Are" and have been for 20 years_
>

___
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
"Where the Answers Are, and have been for 20 years"

Re: What's the NextId?

[Rick Cook]

Abhijeet, my results were repeated and conclusive.  Here is the only
logical explanation for my findings that makes sense:

There is a time cost to the DB call for Entry IDs.  The cost increases
incrementally the more that are requested at once.  The thought is that 100
individual requests will take substantially more time than retrieving 100
IDs in one request.  That thought has been proven to be correct - it's why
the feature was added. However, the difference in time was tested against a
system that had multiple sources (NMS) attempting to grab multiple Entry
IDs at the same time, not a normal system without those multiple outside
requests, because it was the first environment that was reporting the
problem with performance.

On a system where the volume of IDs is high enough to cause a system
bottleneck at the DB ID request, *which requires multiple simultaneous
requests from different sources*, that bottleneck costs more additional
time than is lost by the increased time it takes to request multiple IDs at
one time.  However, on a system that is not bound by the Next ID calls to
the DB, or has only a single source (AR System) requesting IDs, there is no
advantage gained by the multiple requests, because the time "gained" by
having a cached ID is negligible - too small to even notice.  And, as my
testing proved (to my surprise), there was an increasing net cost to system
performance as the number of IDs requested in a single call grew.  This
must be because the time it takes to gather, say, 100 records in one call
is actually *higher* than it is to do 100 individual calls - IF AR System
is the only source of those calls.  The simple correlation is that if AR
System is the only thing generating new Request ID requests to the DB, you
don't need, and will not benefit from, a number larger than 1.  If you have
multiple outside entities creating large number of records at once, you
very well may benefit from it.

My tests that showed decreasing performance as the Next Id block grew were
creating about 1.5 million records, but, and this is important - all were
from the same source - AR System workflow.  There were no other
simultaneous demands on the table, and few on the system.  Therefore, the
system of generating one ID at a time had a cumulatively lower transaction
time than it did when it had to wait for multiples to be retrieved and then
allocated individually.  The BMC Engineer, who I know to be very smart and
experienced with Remedy, had no explanation for my results.  I believe this
to be that BMC didn't test against a control (normal) system, and therefore
had no data on its effect on them.

Why then, they chose to recommend that setting to all customers, is a
mystery to me.

Rick Cook

On Thu, Oct 12, 2017 at 4:56 PM, Gadgil, Abhijeet <abhijeet_gad...@bmc.com>
wrote:

> **
>
> Rick, I do not think that is accurate.
>
> Logically, if the block size is 100 then the server will access the DB
> once to retrieve the next block id every 100 records.  If it is 1, then the
> server goes to the DB for every record.
>
> Later cannot be faster.
>
>
>
> Further, to answer the original question, returning the next id in an API
> call would mean preventing creation of ids by anyone other than the caller
> until the id has actually been used -- otherwise the information might be
> obsolete before the client receives it. That is the reason for not exposing
> it via an API
>
>
>
> Regards,
>
> Abhijeet
>
>
>
>
>
> *From:* Action Request System discussion list(ARSList) [mailto:
> arslist@ARSLIST.ORG] *On Behalf Of *Rick Cook
> *Sent:* 13 October 2017 02:12
> *To:* arslist@ARSLIST.ORG
> *Subject:* Re: What's the NextId?
>
>
>
> **
>
> Here's the skinny on that.  I got this from the engineer who built that
> feature, btw.
>
>
>
> The problem was that system performance was being constricted around the
> action of getting the NextId for a record when multiple sources (say,
> Netcool and HPOV) were throwing tons of requests at the (Incident) form at
> the same time.  The process of getting each Next Entry ID in individual DB
> calls was bottlenecking the process of creating the records.  So that's why
> BMC came up with a way to pre-allocate those in bulk, so that only every N
> times (whatever the number is set to) would an actual call to the DB to get
> Next IDs be necessary.  The transaction time to retrieve 1 or 100 wasn't
> much different, and those customers with multiple programs requiring many
> simultaneous record creations saw a marked performance increase.  It was,
> and is, a good feature add.
>
>
>
> So (then a miracle occurs) and BMC announces that this particular corner
> case had a solution that everyone should benefit from, and announced that
> the number should be 100 for *all* customers.  I tested this back

Re: What's the NextId?

[Gadgil, Abhijeet]

Rick, I do not think that is accurate.
Logically, if the block size is 100 then the server will access the DB once to 
retrieve the next block id every 100 records.  If it is 1, then the server goes 
to the DB for every record.
Later cannot be faster.

Further, to answer the original question, returning the next id in an API call 
would mean preventing creation of ids by anyone other than the caller until the 
id has actually been used -- otherwise the information might be obsolete before 
the client receives it. That is the reason for not exposing it via an API

Regards,
Abhijeet


From: Action Request System discussion list(ARSList) 
[mailto:arslist@ARSLIST.ORG] On Behalf Of Rick Cook
Sent: 13 October 2017 02:12
To: arslist@ARSLIST.ORG
Subject: Re: What's the NextId?

**
Here's the skinny on that.  I got this from the engineer who built that 
feature, btw.

The problem was that system performance was being constricted around the action 
of getting the NextId for a record when multiple sources (say, Netcool and 
HPOV) were throwing tons of requests at the (Incident) form at the same time.  
The process of getting each Next Entry ID in individual DB calls was 
bottlenecking the process of creating the records.  So that's why BMC came up 
with a way to pre-allocate those in bulk, so that only every N times (whatever 
the number is set to) would an actual call to the DB to get Next IDs be 
necessary.  The transaction time to retrieve 1 or 100 wasn't much different, 
and those customers with multiple programs requiring many simultaneous record 
creations saw a marked performance increase.  It was, and is, a good feature 
add.

So (then a miracle occurs) and BMC announces that this particular corner case 
had a solution that everyone should benefit from, and announced that the number 
should be 100 for *all* customers.  I tested this back in 7.6.04 against a RH 
server, with settings at 1, 10, 100, and 1000, and found that performance was 
actually NEGATIVELY affected the higher the number was set.  It wasn't a huge 
difference (10%~), but it was a clear, repeatable one for which BMC's engineer 
had no explanation.  It is why I have always advocated that unless a customer 
has the specific set of circumstances that caused the feature to be beneficial, 
there is no real benefit to setting the number larger than 1.  And there are 
minor drawbacks to doing so, the current subject being one of them.

Every time I ask someone from BMC to justify a larger number to the average 
customer, they repeat the party line, unaware of the history behind the 
feature.  I will continue to tilt at this windmill until someone at BMC shows 
me some performance testing numbers that justify this setting for the entire 
customer base.
Rick

On Oct 12, 2017 13:30, "Thomas Miskiewicz" 
<tmisk...@gmail.com<mailto:tmisk...@gmail.com>> wrote:
**
i.e. there is no hack to find out the nextID before it actually gets submitted?

Apart from that, I really don’t understand why BMC makes such a fuss around the 
nextID. Why can’t they just provide a special command GET-NEXTID?


Thomas


On Oct 12, 2017, at 10:26 PM, LJ LongWing 
<lj.longw...@gmail.com<mailto:lj.longw...@gmail.com>> wrote:

**
There are no interfaces that I'm aware of to ask a specific server what the 
next id it will hand out for a specific form is

On Thu, Oct 12, 2017 at 2:14 PM, Thomas Miskiewicz 
<tmisk...@gmail.com<mailto:tmisk...@gmail.com>> wrote:
Hello List,

with NextID Block size being set to 100 —>  is it possible the find out using 
the API which will be the next Request-ID that the server will assign to a 
request? Or what other options do I have to find out?


--Thomas

___
UNSUBSCRIBE or access ARSlist Archives at 
www.arslist.org<http://www.arslist.org/>
"Where the Answers Are, and have been for 20 years"

_ARSlist: "Where the Answers Are" and have been for 20 years_

_ARSlist: "Where the Answers Are" and have been for 20 years_
_ARSlist: "Where the Answers Are" and have been for 20 years_

___
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
"Where the Answers Are, and have been for 20 years"

Re: What's the NextId?

[Grooms, Frederick W]

Actually I had a case where we needed Next ID > 1 due to DB performance.   It 
wasn’t on the primary form, it was on a child form (something like the Worklog 
in ITSM where every action to a ticket is recorded).  The database was being 
blocked on the update of the Arschema record for that form.  On the primary 
(i.e. HelpDesk) form the size is set to 1.

Fred

From: Action Request System discussion list(ARSList) 
[mailto:arslist@ARSLIST.ORG] On Behalf Of Rick Cook
Sent: Thursday, October 12, 2017 3:42 PM
To: arslist@ARSLIST.ORG
Subject: Re: What's the NextId?

**
Here's the skinny on that.  I got this from the engineer who built that 
feature, btw.

The problem was that system performance was being constricted around the action 
of getting the NextId for a record when multiple sources (say, Netcool and 
HPOV) were throwing tons of requests at the (Incident) form at the same time.  
The process of getting each Next Entry ID in individual DB calls was 
bottlenecking the process of creating the records.  So that's why BMC came up 
with a way to pre-allocate those in bulk, so that only every N times (whatever 
the number is set to) would an actual call to the DB to get Next IDs be 
necessary.  The transaction time to retrieve 1 or 100 wasn't much different, 
and those customers with multiple programs requiring many simultaneous record 
creations saw a marked performance increase.  It was, and is, a good feature 
add.

So (then a miracle occurs) and BMC announces that this particular corner case 
had a solution that everyone should benefit from, and announced that the number 
should be 100 for *all* customers.  I tested this back in 7.6.04 against a RH 
server, with settings at 1, 10, 100, and 1000, and found that performance was 
actually NEGATIVELY affected the higher the number was set.  It wasn't a huge 
difference (10%~), but it was a clear, repeatable one for which BMC's engineer 
had no explanation.  It is why I have always advocated that unless a customer 
has the specific set of circumstances that caused the feature to be beneficial, 
there is no real benefit to setting the number larger than 1.  And there are 
minor drawbacks to doing so, the current subject being one of them.

Every time I ask someone from BMC to justify a larger number to the average 
customer, they repeat the party line, unaware of the history behind the 
feature.  I will continue to tilt at this windmill until someone at BMC shows 
me some performance testing numbers that justify this setting for the entire 
customer base.
Rick

On Oct 12, 2017 13:30, "Thomas Miskiewicz" 
<tmisk...@gmail.com<mailto:tmisk...@gmail.com>> wrote:
**
i.e. there is no hack to find out the nextID before it actually gets submitted?

Apart from that, I really don’t understand why BMC makes such a fuss around the 
nextID. Why can’t they just provide a special command GET-NEXTID?


Thomas


On Oct 12, 2017, at 10:26 PM, LJ LongWing 
<lj.longw...@gmail.com<mailto:lj.longw...@gmail.com>> wrote:

**
There are no interfaces that I'm aware of to ask a specific server what the 
next id it will hand out for a specific form is

On Thu, Oct 12, 2017 at 2:14 PM, Thomas Miskiewicz 
<tmisk...@gmail.com<mailto:tmisk...@gmail.com>> wrote:
Hello List,

with NextID Block size being set to 100 —>  is it possible the find out using 
the API which will be the next Request-ID that the server will assign to a 
request? Or what other options do I have to find out?


--Thomas

___
UNSUBSCRIBE or access ARSlist Archives at 
www.arslist.org<http://www.arslist.org/>
"Where the Answers Are, and have been for 20 years"

_ARSlist: "Where the Answers Are" and have been for 20 years_

_ARSlist: "Where the Answers Are" and have been for 20 years_
_ARSlist: "Where the Answers Are" and have been for 20 years_

___
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
"Where the Answers Are, and have been for 20 years"

Re: What's the NextId?

[Rick Cook]

Tim, I think he is looking not for the next one that the DB has, but the
one that's already been allocated to the AR server and would be the next
Entry ID for the next record actually created.  In the current scenario,
anywhere between 1 and 99 Entry IDs would have already been fetched from
the DB and cached to EACH AR server you have, so the call would have to
come from the actual AR server, not from the DB.

Rick Cook

On Thu, Oct 12, 2017 at 1:47 PM, Hulmes, Timothy W CTR MDA/ICTO <
timothy.hulmes@mda.mil> wrote:

> Thomas
>
> You can get this data from the "AR System Metadata: arschema" form.
>
> The fields of importance on this form:
> Name = the Form Name
> nextId = The next available ID for that form
>
>
> Tim
>
>
> -Original Message-
> From: Action Request System discussion list(ARSList) [mailto:
> arslist@ARSLIST.ORG] On Behalf Of Thomas Miskiewicz
> Sent: Thursday, October 12, 2017 2:14 PM
> To: arslist@ARSLIST.ORG
> Subject: What's the NextId?
>
> Hello List,
>
> with NextID Block size being set to 100 —>  is it possible the find out
> using the API which will be the next Request-ID that the server will assign
> to a request? Or what other options do I have to find out?
>
>
> --Thomas
>
> 
> ___
> UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
> "Where the Answers Are, and have been for 20 years"
>
> 
> ___
> UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
> "Where the Answers Are, and have been for 20 years"
>

___
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
"Where the Answers Are, and have been for 20 years"

Re: What's the NextId?

[Hulmes, Timothy W CTR MDA/ICTO]

Thomas

You can get this data from the "AR System Metadata: arschema" form.

The fields of importance on this form:
Name = the Form Name
nextId = The next available ID for that form


Tim


-Original Message-
From: Action Request System discussion list(ARSList) 
[mailto:arslist@ARSLIST.ORG] On Behalf Of Thomas Miskiewicz
Sent: Thursday, October 12, 2017 2:14 PM
To: arslist@ARSLIST.ORG
Subject: What's the NextId?

Hello List,

with NextID Block size being set to 100 —>  is it possible the find out using 
the API which will be the next Request-ID that the server will assign to a 
request? Or what other options do I have to find out?


--Thomas

___
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
"Where the Answers Are, and have been for 20 years"

___
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
"Where the Answers Are, and have been for 20 years"

Re: What's the NextId?

[Thomas Miskiewicz]

Thanks Rick! I actually don’t care whether they reserve the block or not as 
long as they allow us to access the info what the next Id is. It could be a 
special workflow action or at least an API call. I really don’t get why is that 
so difficult to provide? 

> On 12. Oct 2017, at 22:42, Rick Cook  wrote:
> 
> **
> Here's the skinny on that.  I got this from the engineer who built that 
> feature, btw.
> 
> The problem was that system performance was being constricted around the 
> action of getting the NextId for a record when multiple sources (say, Netcool 
> and HPOV) were throwing tons of requests at the (Incident) form at the same 
> time.  The process of getting each Next Entry ID in individual DB calls was 
> bottlenecking the process of creating the records.  So that's why BMC came up 
> with a way to pre-allocate those in bulk, so that only every N times 
> (whatever the number is set to) would an actual call to the DB to get Next 
> IDs be necessary.  The transaction time to retrieve 1 or 100 wasn't much 
> different, and those customers with multiple programs requiring many 
> simultaneous record creations saw a marked performance increase.  It was, and 
> is, a good feature add.
> 
> So (then a miracle occurs) and BMC announces that this particular corner case 
> had a solution that everyone should benefit from, and announced that the 
> number should be 100 for *all* customers.  I tested this back in 7.6.04 
> against a RH server, with settings at 1, 10, 100, and 1000, and found that 
> performance was actually NEGATIVELY affected the higher the number was set.  
> It wasn't a huge difference (10%~), but it was a clear, repeatable one for 
> which BMC's engineer had no explanation.  It is why I have always advocated 
> that unless a customer has the specific set of circumstances that caused the 
> feature to be beneficial, there is no real benefit to setting the number 
> larger than 1.  And there are minor drawbacks to doing so, the current 
> subject being one of them.
> 
> Every time I ask someone from BMC to justify a larger number to the average 
> customer, they repeat the party line, unaware of the history behind the 
> feature.  I will continue to tilt at this windmill until someone at BMC shows 
> me some performance testing numbers that justify this setting for the entire 
> customer base.
> 
> Rick
> 
>> On Oct 12, 2017 13:30, "Thomas Miskiewicz"  wrote:
>> **
>> i.e. there is no hack to find out the nextID before it actually gets 
>> submitted?
>> 
>> Apart from that, I really don’t understand why BMC makes such a fuss around 
>> the nextID. Why can’t they just provide a special command GET-NEXTID?
>> 
>> 
>> Thomas 
>> 
>> 
>>> On Oct 12, 2017, at 10:26 PM, LJ LongWing  wrote:
>>> 
>>> **
>>> There are no interfaces that I'm aware of to ask a specific server what the 
>>> next id it will hand out for a specific form is
>>> 
 On Thu, Oct 12, 2017 at 2:14 PM, Thomas Miskiewicz  
 wrote:
 Hello List,
 
 with NextID Block size being set to 100 —>  is it possible the find out 
 using the API which will be the next Request-ID that the server will 
 assign to a request? Or what other options do I have to find out?
 
 
 --Thomas
 
 ___
 UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
 "Where the Answers Are, and have been for 20 years"
>>> 
>>> _ARSlist: "Where the Answers Are" and have been for 20 years_
>> 
>> _ARSlist: "Where the Answers Are" and have been for 20 years_
> _ARSlist: "Where the Answers Are" and have been for 20 years_

___
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
"Where the Answers Are, and have been for 20 years"

Re: What's the NextId?

[Rick Cook]

Here's the skinny on that.  I got this from the engineer who built that
feature, btw.

The problem was that system performance was being constricted around the
action of getting the NextId for a record when multiple sources (say,
Netcool and HPOV) were throwing tons of requests at the (Incident) form at
the same time.  The process of getting each Next Entry ID in individual DB
calls was bottlenecking the process of creating the records.  So that's why
BMC came up with a way to pre-allocate those in bulk, so that only every N
times (whatever the number is set to) would an actual call to the DB to get
Next IDs be necessary.  The transaction time to retrieve 1 or 100 wasn't
much different, and those customers with multiple programs requiring many
simultaneous record creations saw a marked performance increase.  It was,
and is, a good feature add.

So (then a miracle occurs) and BMC announces that this particular corner
case had a solution that everyone should benefit from, and announced that
the number should be 100 for *all* customers.  I tested this back in 7.6.04
against a RH server, with settings at 1, 10, 100, and 1000, and found that
performance was actually NEGATIVELY affected the higher the number was
set.  It wasn't a huge difference (10%~), but it was a clear, repeatable
one for which BMC's engineer had no explanation.  It is why I have always
advocated that unless a customer has the specific set of circumstances that
caused the feature to be beneficial, there is no real benefit to setting
the number larger than 1.  And there are minor drawbacks to doing so, the
current subject being one of them.

Every time I ask someone from BMC to justify a larger number to the average
customer, they repeat the party line, unaware of the history behind the
feature.  I will continue to tilt at this windmill until someone at BMC
shows me some performance testing numbers that justify this setting for the
entire customer base.

Rick

On Oct 12, 2017 13:30, "Thomas Miskiewicz"  wrote:

> **
> i.e. there is no hack to find out the nextID before it actually gets
> submitted?
>
> Apart from that, I really don’t understand why BMC makes such a fuss
> around the nextID. Why can’t they just provide a special command GET-NEXTID?
>
>
> Thomas
>
>
> On Oct 12, 2017, at 10:26 PM, LJ LongWing  wrote:
>
> **
> There are no interfaces that I'm aware of to ask a specific server what
> the next id it will hand out for a specific form is
>
> On Thu, Oct 12, 2017 at 2:14 PM, Thomas Miskiewicz 
> wrote:
>
>> Hello List,
>>
>> with NextID Block size being set to 100 —>  is it possible the find out
>> using the API which will be the next Request-ID that the server will assign
>> to a request? Or what other options do I have to find out?
>>
>>
>> --Thomas
>>
>> 
>> ___
>> UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
>> "Where the Answers Are, and have been for 20 years"
>>
>
> _ARSlist: "Where the Answers Are" and have been for 20 years_
>
>
> _ARSlist: "Where the Answers Are" and have been for 20 years_

___
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
"Where the Answers Are, and have been for 20 years"

Re: What's the NextId?

[Thomas Miskiewicz]

i.e. there is no hack to find out the nextID before it actually gets submitted?

Apart from that, I really don’t understand why BMC makes such a fuss around the 
nextID. Why can’t they just provide a special command GET-NEXTID?


Thomas 


> On Oct 12, 2017, at 10:26 PM, LJ LongWing  wrote:
> 
> **
> There are no interfaces that I'm aware of to ask a specific server what the 
> next id it will hand out for a specific form is
> 
> On Thu, Oct 12, 2017 at 2:14 PM, Thomas Miskiewicz  > wrote:
> Hello List,
> 
> with NextID Block size being set to 100 —>  is it possible the find out using 
> the API which will be the next Request-ID that the server will assign to a 
> request? Or what other options do I have to find out?
> 
> 
> --Thomas
> 
> ___
> UNSUBSCRIBE or access ARSlist Archives at www.arslist.org 
> 
> "Where the Answers Are, and have been for 20 years"
> 
> _ARSlist: "Where the Answers Are" and have been for 20 years_


___
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
"Where the Answers Are, and have been for 20 years"

Re: What's the NextId?

[LJ LongWing]

There are no interfaces that I'm aware of to ask a specific server what the
next id it will hand out for a specific form is

On Thu, Oct 12, 2017 at 2:14 PM, Thomas Miskiewicz 
wrote:

> Hello List,
>
> with NextID Block size being set to 100 —>  is it possible the find out
> using the API which will be the next Request-ID that the server will assign
> to a request? Or what other options do I have to find out?
>
>
> --Thomas
>
> 
> ___
> UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
> "Where the Answers Are, and have been for 20 years"
>

___
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
"Where the Answers Are, and have been for 20 years"

What's the NextId?

[Thomas Miskiewicz]

Hello List,

with NextID Block size being set to 100 —>  is it possible the find out using 
the API which will be the next Request-ID that the server will assign to a 
request? Or what other options do I have to find out?


--Thomas

___
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
"Where the Answers Are, and have been for 20 years"