Does anyone have a good method to 'throttle' the number of child processes?
We authorize a few $million per day in CC transactions doing each 'near' real-time auths as the orders arrive. This is single threaded master phantom opening / writing/ reading the socket for each transaction. Works fairly well. Very reliable but not real time enough to hold the customer on the phone till the 'approve' comes back. I've toyed with having the master phantom spawn child phantom processes but need a way to 'throttle' the thread count, and queue anything over a specified threshold. The method I tried, grabbing a 'token' from a control record when the child phantom spawns, and replacing it when the child process concludes does not work. I found that this becomes unworkable after 4 or 5 child phantoms get spawned. They conclude closely enough that the contention over the update lock for the 'tokens' record degrades any performance gain by going multi-threaded. Seems I can get an auth faster than the control record can be locked, updated, and written back. I fully agree with everything David said about logging the pid and have done this also. @nschroth - if you want some code snippets of the socket calls email me off list. Thank you. -Baker -----Original Message----- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Hona, David Sent: Monday, March 07, 2011 7:04 PM To: 'U2 Users List' Subject: Re: [U2] Multi-Threading Universe Socket traffic I'd agree with this approach...I've used it many times before... A good well thought out design well help you...design first: code second :-) I've in the past done the following: - control program to configure, start/monitor/stop phantom processes - I find writing a phantom process which logs what it is doing/done is used for debugging purposes too (saves guessing what's going) - just remember to have a toggle to turn this on/off for production (to save disk space) - it is important that each phantom logs its progress, its process ID, start, last checkpoint reached and that it has successfully terminated. This helps prevent the case of you accidently firing off 100 phantoms and consuming all your UV licenses, etc. :-) The purpose of this is to: - make the process scalable and tuneable without re-coding via parameter record (include things like operating windows, etc) - have an application to manage the phantoms and monitor what they're doing (or if they're doing anything) - have phantom processes log what they're doing and allow options for a verbose logging mode to log everything they're doing in case of problems Some approaches I've used: a) each phantom could be started with a unique saved list of record keys to processed (generated by a control program or some other process beforehand) b) each phantom can perform its own query (but sleeping for a specified period so not to continuously performing disk I/O) c) Inbound and outbound transaction phantom process - each handles only response or send requests respectively. Such an approach only works if you can reconcile the response with the original request. Very application specific and not generally optimised for throughput. Good luck. From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of nschroth Sent: 05 March 2011 16:55 To: u2-users@listserver.u2ug.org Subject: [U2] Multi-Threading Universe Socket traffic On Universe 10.1.14 over AIX 5.3, we currently communicate Credit Card transactions via sockets (ISO-8583) using the following logic (works fine): OPEN.ERR=openSocket(THIS.IP,THIS.PORT,TCP.MODE,TIMEOUT,THIS.HANDLE) INFO.ERR=getSocketInformation(THIS.HANDLE,PEER.FLAG,SOCKETINFO) WRITE.ERR=writeSocket(THIS.HANDLE,SEND.MSG,TIMEOUT,TCP.MODE,SEND.SIZE) READ.ERR=readSocket(THIS.HANDLE,RECV.MSG,RECV.LEN,TIMEOUT,TCP.MODE,RECV.SIZE ) I am looking into using some sort of multi-threading logic to allow increased volume in shorter timeframes. For example, we now send batches of up to 3,000 cards via a ftps mechanism that responds normally within 3-5 minutes. We want to benchmark doing this via the sockets instead. The Subroutine that does this Socket comm takes about 0.7 secs per trx, so a 3,000 card batch would take over 30 minutes single-threaded (unacceptable). Probably 80% of that 0.7 secs is transforming the data to send and then transforming the response back to process. This communication, its contents and any file attachments transmitted with it are intended solely for the addressee(s) and may contain confidential proprietary information. Access by any other party without the express written permission of the sender is STRICTLY PROHIBITED. If you have received this communication in error you may not copy, distribute or use the contents, attachments or information in any way. Please destroy it and contact the sender. _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
