Hi,
I do not think that we disagree here.
I checked some logs, and apparently my statement about "list" in the name
does not guarantee that the call is routed to list-thread. But the
ARGetListEntry() and ARGetListEntryWithFields() is definitely calls that
is routed to the List-threads, and the ARCreateEntry() and ARSetEntry() is
going to the Fast-threads.
My opinion is that the fast/list-threads has lost a little of their function.
This is definitely true if you look at the ARCreateEntry() and
ARSetEntry() calls, as they quite commonly performs a lot of stuff on the
server side, and is not fast at all.
Why would it be an advantage if you have queuing going on on either the
fast- or list-threads and the other type (fast or list) has no work to do?
It would be better to throw everything into the same pool, and remove the
settings for list/fast threads.
The admin queue, escalation queues and private queues still serve a valid
purpose.
I checked some server logs from ITSM7, and hear are the numbers:
Thread/Calls/Time
Fast/35881/~26 minutes
List/32543/~11 minutes
The "Fast" threads have a slightly higher number of calls, but user more
than double time per call. The "Fast" calls ran at 0.044 seconds per call,
and the "List"-calls ran at 0.019 seconds per call.
Best Regards - Misi, RRR AB, http://www.rrr.se
Products from RRR Scandinavia:
* RRR|License - Not enough Remedy licenses? Save money by optimizing.
* RRR|Log - Performance issues or elusive bugs? Analyze your Remedy logs.
Find these products, and many free tools and utilities, at http://rrr.se.
> Misi, I know that you know AR System internals very well, and so normally,
> I
> would never question you. So consider this response with that in mind, as
> well as the disclaimer that this is the last thing I was taught on the
> topic
> - if the architecture of the internals has changed, I must have missed
> that
> memo.
>
> The purpose behind Fast and List threads was to separate single-API
> functions (Submits, mostly) from multiple-API functions (pretty much
> everything else). The thought, from what I can infer, was to have an
> express line of sorts so that quick DB transactions wouldn't have to wait
> in
> line behind long search/retrieve tasks. If, as you infer, Fast/List
> threads
> no longer have any relevance in terms of different function, why do they
> still exist as separately tunable parameters?
>
> Rick
>
> On Mon, Jan 18, 2010 at 5:06 AM, Misi Mladoniczky <[email protected]> wrote:
>
>> Hi,
>>
>> I do not think that updates from Web Services use the list threads. It
>> would be the same thing as the Save-button in a Modify-form.
>>
>> It performs a ARSetEntry()-call through a Fast-thread. It is performed
>> by
>> the Mid-Tier-server just as any other client.
>>
>> The Fast- and List-threads are just a blunt way to group and route
>> various
>> calls.
>>
>> If we exclude the admin-tool/developer-studio calls, I think that any
>> call
>> with "list" in it's name access the List-servers and any call without
>> "list" in its name go to a Fast-server.
>>
>> There is no technical difference between Fast and List threads. They
>> perform in the same way.
>>
>> As a Fast-call can be very slow, and List-calls very quick, I would
>> argue
>> that differentiating between these types of calls are not that useful.
>>
>> The calls to the fast an list threads typically comes in pairs anyway:
>>
>> An ACTL-Set-Fields performs two calls. One ARGetListEntry() to the
>> List-thread, and one ARGetEntry() to the Fast-thread. An
>> ACTL-Push-Fields
>> performs a ARGetListEntry() and an ARCreateEntry()/ARSetEntry() call.
>>
>> In the old days, I think that the List-threads and Fast-threads had
>> another distinction. When a user logged in, he was "assigned" to a
>> Fast-thread but accessed the List-thread in a more random way. The idea
>> was to minimize the calls to the transaction-daemon and portmapper to
>> find
>> the tcp-port of the fast-threads each time they were called. Now
>> everything goes to a single tcp-port.
>>
>> When we got the FLTR-Push-Fields calls, the ARCreateEntry() and
>> ARSetEntry() calls stopped being very fast. WebServices adds even more
>> to
>> this problem.
>>
>> In most cases, it is possible to rewrite workflow to be much faster. In
>> some cases, especially with the bigger apps such as ITSM, it is more
>> difficult. And you do not want to customize ITSM anyway...
>>
>> Best Regards - Misi, RRR AB, http://www.rrr.se
>>
>> Products from RRR Scandinavia:
>> * RRR|License - Not enough Remedy licenses? Save money by optimizing.
>> * RRR|Log - Performance issues or elusive bugs? Analyze your Remedy
>> logs.
>> Find these products, and many free tools and utilities, at
>> http://rrr.se.
>>
>> > As an aside, technically your submits are going through the Fast
>> > (single-API) threads, while your updates from Web Services would use
>> the
>> > List (multiple-API) threads. That's if your submit is allowed to
>> finish
>> > and
>> > then the ensuing record is updated by the WS. If the WS return values
>> are
>> > part of the Submit process, then you have long transactions processing
>> > through and clogging up your Fast threads, which is not what they are
>> > intended to handle. That will be the case regardless of the source of
>> the
>> > external data retrieval process or source, though the time they take
>> may
>> > be
>> > affected by using DB vs. WS.
>> >
>> > Rick
>> >
>> > On Fri, Jan 15, 2010 at 3:35 PM, Jason Miller
>> > <[email protected]>wrote:
>> >
>> >> ** Just to make sure that I understand, the requesting system is
>> waiting
>> >> 40
>> >> seconds for the quote to be returned? Does the quote return need to
>> >> done as
>> >> soon as possible or can there be a minute or two before the return?
>> >>
>> >> Maybe have them only submit the record and not trigger the processing
>> >> workflow. Then have an escalation that will process the new
>> record(s)
>> >> and
>> >> then make a call to a web service on requester's end that is designed
>> to
>> >> accept the return from a previous transaction. It would take some
>> >> redesigning on both sides but would alleviate the waiting and hanging
>> on
>> >> to
>> >> threads/connections.
>> >>
>> >> The other thing might be to offload some of the Remedy processing (if
>> >> there
>> >> is a lot) to more efficient scripts and/or direct database actions.
>> Web
>> >> services can be a wonderful thing but not always the fastest.
>> >>
>> >> Jason
>> >>
>> >>
>> >> On Fri, Jan 15, 2010 at 3:02 PM, LJ Longwing
>> >> <[email protected]>wrote:
>> >>
>> >>> **
>> >>> LOL.....consider this Rick....Consider a 'submit' to be an interface
>> >>> that
>> >>> allows ALL attributes needed to completely configure a change with a
>> >>> single
>> >>> button press. This may be a bad example...if it is I'm sorry...I
>> >>> really
>> >>> don't do ITSM....but the submit in question on our system actually
>> >>> averages
>> >>> about 40 seconds. Makes no less than 4 calls to other external
>> systems
>> >>> via
>> >>> web services and performs more calculations than I care to think
>> about.
>> >>> That being what it is....what would you do in that situation?
>> >>>
>> >>> ------------------------------
>> >>> *From:* Action Request System discussion list(ARSList) [mailto:
>> >>> [email protected]] *On Behalf Of *Rick Cook
>> >>> *Sent:* Friday, January 15, 2010 3:55 PM
>> >>>
>> >>> *To:* [email protected]
>> >>> *Subject:* Re: Fast/List Concurrent settings?
>> >>>
>> >>> ** If your submits are taking 10 seconds each, you have a
>> significant
>> >>> problem, my friend!
>> >>>
>> >>> Rick
>> >>>
>> >>> On Fri, Jan 15, 2010 at 2:52 PM, LJ Longwing
>> >>> <[email protected]>wrote:
>> >>>
>> >>>> **
>> >>>> I completely agree with everything you said....but as you
>> mentioned,
>> >>>> if
>> >>>> you have 60 submits and have 10 threads...you are only handling 10
>> of
>> >>>> those
>> >>>> 60 concurrently....so if each transaction takes an average of
>> say...10
>> >>>> seconds...the first 10 will complete in 10, the next 10 in 20, and
>> so
>> >>>> on
>> >>>> with the final set of 10 taking a full 60 seconds from time to
>> >>>> complete to
>> >>>> process. If your external application has a timeout of say....45
>> >>>> seconds
>> >>>> then you are only going to be able to handle 40 of the 60 submitted
>> >>>> concurrently and as such would have a 1/3 failure rate....in that
>> >>>> situation
>> >>>> would you then set your threads to say....15 to make it so that you
>> >>>> could
>> >>>> handle 60 in 40 seconds or would you take it to 60 to be able to
>> >>>> handle 60
>> >>>> in 10 seconds?
>> >>>>
>> >>>> ------------------------------
>> >>>> *From:* Action Request System discussion list(ARSList) [mailto:
>> >>>> [email protected]] *On Behalf Of *Rick Cook
>> >>>> *Sent:* Friday, January 15, 2010 3:39 PM
>> >>>> *To:* [email protected]
>> >>>> *Subject:* Re: Fast/List Concurrent settings?
>> >>>>
>> >>>> ** I would think that one of the larger software-related issues
>> that
>> >>>> would affect the number of concurrent Inserts would be the Index
>> >>>> structure.
>> >>>> I am sure you know this, but for the benefit of those who don't,
>> >>>> indexes are
>> >>>> meant to help us search on a form more easily. However, above a
>> >>>> certain
>> >>>> number (around 8 for a Remedy form), that cause performance
>> >>>> degradation when
>> >>>> creating a new record, because each of the indexes has to be
>> updated
>> >>>> as part
>> >>>> of the creation process. If a system is seeing slow creates,
>> that's
>> >>>> the
>> >>>> first thing I check.
>> >>>>
>> >>>>
>> >>>> One other thing to think about is that since each thread can cache
>> 5
>> >>>> processes (that's the last number I heard, anyway) in addition to
>> the
>> >>>> one
>> >>>> currently being handled, you could, with 10 Fast (Single-API)
>> threads,
>> >>>> handle 60 concurrent create processes without a transaction loss.
>> >>>> There
>> >>>> would probably be a bit of a delay for those farther back in the
>> >>>> queue, but
>> >>>> if you had a reasonably robust system, most users wouldn't notice
>> it
>> >>>> much.
>> >>>> I seriously doubt all but a very few systems have to handle
>> anything
>> >>>> resembling that kind of concurrent load, with the exception of
>> those
>> >>>> who
>> >>>> have a large number of system (i.e. NMS) generated records.
>> >>>>
>> >>>> Also look at the Entry ID Block size when doing this test. If -
>> AND
>> >>>> ONLY
>> >>>> IF - you are regularly having large numbers of concurrent inserts,
>> you
>> >>>> can
>> >>>> set the Entry ID Block size to something like 10 to cut down the
>> >>>> number of
>> >>>> requests to the DB for Entry IDs. That is alleged to help with
>> create
>> >>>> times, though I have not seen that be the case in practical use.
>> >>>>
>> >>>> Rick
>> >>>>
>> >>>> On Fri, Jan 15, 2010 at 2:27 PM, LJ Longwing
>> >>>> <[email protected]>wrote:
>> >>>>
>> >>>>> **
>> >>>>> Here is an interesting question for you thread experts out there.
>> >>>>>
>> >>>>> How many concurrent 'creates' does your system support? Creates
>> >>>>> being a
>> >>>>> generic term for any given process that your system supports. The
>> >>>>> system
>> >>>>> that I support is a home grown quote/order management system and
>> last
>> >>>>> year
>> >>>>> we stood up a web service interface for people to be able to
>> generate
>> >>>>> quotes
>> >>>>> from their systems and get pricing back. The initial interface
>> was
>> >>>>> setup to
>> >>>>> handle 3 concurrent creates...but as soon as it was a success we
>> >>>>> started
>> >>>>> getting slammed with several hundred at a time and choking our
>> >>>>> system.
>> >>>>> Through rigorous testing and tweaking over the last couple of
>> weeks I
>> >>>>> have
>> >>>>> been able to get roughly 60 concurrent going through the system
>> with
>> >>>>> reasonable performance....so at this point I have my Fast set to
>> >>>>> 30/100
>> >>>>> min/max....confirmed that I'm not maxing anything specific
>> out....but
>> >>>>> I
>> >>>>> personally have never run above 20ish threads as a high because
>> most
>> >>>>> transactions are short and a fast thread count of 20 will handle
>> >>>>> hundreds of
>> >>>>> users in 'normal' operation.....so I was just wondering how many
>> >>>>> requests
>> >>>>> you guys have your system to handle concurrently....and just for
>> >>>>> verification...I'm talking about 'all of them hit the button
>> within a
>> >>>>> second
>> >>>>> of each other' type of concurrent...not 'I hove 400 people logged
>> on
>> >>>>> concurrently'
>> >>>>> _Platinum Sponsor: [email protected] ARSlist: "Where the
>> >>>>> Answers
>> >>>>> Are"_
>> >>>>
>> >>>>
>> >>>> _Platinum Sponsor: [email protected] ARSlist: "Where the
>> >>>> Answers
>> >>>> Are"_
>> >>>> _Platinum Sponsor: [email protected] ARSlist: "Where the
>> >>>> Answers
>> >>>> Are"_
>> >>>>
>> >>>
>> >>> _Platinum Sponsor: [email protected] ARSlist: "Where the
>> Answers
>> >>> Are"_
>> >>> _Platinum Sponsor: [email protected] ARSlist: "Where the
>> Answers
>> >>> Are"_
>> >>>
>> >>
>> >> _Platinum Sponsor: [email protected] ARSlist: "Where the
>> Answers
>> >> Are"_
>> >>
>> >
>> >
>> _______________________________________________________________________________
>> > UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
>> > Platinum
>> Sponsor:[email protected]<sponsor%[email protected]>ARSlist:
>> "Where the Answers Are"
>> >
>> > --
>> > This message was scanned by ESVA and is believed to be clean.
>> >
>> >
>>
>>
>> _______________________________________________________________________________
>> UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
>> Platinum
>> Sponsor:[email protected]<sponsor%[email protected]>ARSlist:
>> "Where the Answers Are"
>>
>
> _______________________________________________________________________________
> UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
> Platinum Sponsor:[email protected] ARSlist: "Where the Answers Are"
>
> --
> This message was scanned by ESVA and is believed to be clean.
>
>
_______________________________________________________________________________
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
Platinum Sponsor:[email protected] ARSlist: "Where the Answers Are"
