Re: [U2] Slow ascii output
Well said, Mark. My opportunity to really learn about Pick performance was circa 1985 on an ADDS 1500, a PC XT clone with 256K RAM, a 10MB hard drive and a port of the ADDS Mentor Pick O/S. We ported some code from an ADDS 4000 to it and I was shocked to see how slowly a 3-column screen of multivalued data displayed on the 1500. The code was a for-next loop to display REC23,X , REC24,X and REC25,X. Took about one-half second per cell, or about half a minute per 20-line by 3-column screen! So the 1500 was slow enough to make the internal workings of Pick (and Unidata today) visible to the user: (1) the data was deep enough in the record to cause a frame-fault and (2) dynamic arrays find the requested data by searching forward from the first character of the record each time they're referenced, so the record was being scanned 60 times per screenful. The solution was simple: assign each attribute to its own variable before the display loop, display the variables (e.g. PRICE1,X rather than REC24,X), and voila!, the screen became nice and fast again. Hunting down (or better yet, avoiding) performance issues on modern systems is a lot easier having seen similar behavior on older, slower machines. Have fun, Tom [EMAIL PROTECTED] 10/21/06 9:43 PM Keep in mind that 16MB for 32 users is 250 times the memory available circa late 1970s' with the Microdata Royale series. There's something to be said for programming on an older system that gives you respect for todays horsepower. snip - This e-mail and any attachments may contain CONFIDENTIAL information, including PROTECTED HEALTH INFORMATION. If you are not the intended recipient, any use or disclosure of this information is STRICTLY PROHIBITED; you are requested to delete this e-mail and any attachments, notify the sender immediately, and notify the LabCorp Privacy Officer at [EMAIL PROTECTED] or call (877) 23-HIPAA / (877) 234-4722. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Slow ascii output
The c/r delay may have had something to do with it. PRINT expression: ran faster than PRINT expression and If you had lots of PRINTs Does anyone out there remember the Newt, the Visa and also (perchance) where the MV file transfer protocol based on More Tea Vicar got it's name? McD Reality - (exchangeable hard disk packs) - (gosh). I remember building the whole screen template into a single variable to PRINT it without undue delays. Doing the same with the screen variable content for a second PRINT to keep savings there as well. One night a SYSOP did a Disk Pack Set copy of C (oldest)- A (current) instead of A (current) - C (oldest) - gosh what fun.. Was it: INT-CLOCK-SS1-LOAD-RUN or INT-CLOCK-RESET-SS1-LOAD-RUN? !PCB.182;2 . = .0184 Regards JayJay --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Slow ascii output
On Fri, 20 Oct 2006, Kevin King wrote: Jeff, why WRITESEQF instead of WRITESEQ? You're completely undermining write buffering with WRITESEQF are you not? I think this may be from an example where I was debugging code and used WRITESEQF to ensure that all I/O was written before the program crashed. It may be a fluke but I'll check my code to be sure. Jeff Butera, Ph.D. Administrative Systems Hampshire College [EMAIL PROTECTED] 413-559-5556 I'm not grumpy Daddy, I'm whining. my 3-year old showing her wisdom --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] Slow ascii output
On Fri, 20 Oct 2006, Anthony W. Youngman wrote: Our system was short of RAM - 16 meg for 32 users (PI/Open on an EXL 7330). It thrashed enough under normal load, even before you started to try and build large (and I mean LARGE) strings in BASIC... We don't have this issue - we have 16Gig for about 90 users. Jeff Butera, Ph.D. Administrative Systems Hampshire College [EMAIL PROTECTED] 413-559-5556 I'm not grumpy Daddy, I'm whining. my 3-year old showing her wisdom --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] Slow ascii output
Mark Johnson wrote on 10/21/2006 09:43:13 PM: Keep in mind that 16MB for 32 users is 250 times the memory available circa late 1970s' with the Microdata Royale series. There's something to be said for programming on an older system that gives you respect for todays horsepower. I agree 100%. In the late 70s I was working at a Savings and Loan with seven branch offices. There were several teller terminals and one or two VDTs at each branch. We ran with a grand total of 256 KB (not MB - KB) of memory - and that was after the upgrade from 128 KB. [Hey, it seemed like a lot because my TRS-80 only had 4 KB ;-)] You learned to be very efficient. And we couldn't make any programming mistakes. Our online program took all night to compile. We kept a few KB of memory set aside in the program so we could patch in hex code and branch in and out of the main program. At the end of the online day we rebooted the system with different memory parameters for nightly batch processing. Ah, the good old days... Tim Snyder Consulting I/T Specialist U2 Consulting North American Lab Services IBM Software Group --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] Slow ascii output
Keep in mind that 16MB for 32 users is 250 times the memory available circa late 1970s' with the Microdata Royale series. There's something to be said for programming on an older system that gives you respect for todays horsepower. I experience this first hand every other week when visiting my Microdata client from my normal U2/D3 client base. While you can't get really technically sophisticated as with the current systems, you do become more effecient when programming within limits. Perhaps the single thing I miss the most when programming on this old system is the fact that I cannot use external subroutiness as freely as on every other platform. I have a whole bunch of handy subs that I install on all of my clients but have to convert them to INCLUDES if I want to use them there. Microdatas don't like mixing RUN and cataloged programs together. My 1 cent. Mark Johnson - Original Message - From: Anthony W. Youngman [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Sent: Saturday, October 21, 2006 6:29 PM Subject: Re: [U2] Slow ascii output In message [EMAIL PROTECTED], Claus Derlien [EMAIL PROTECTED] writes no one and I mean NO ONE uses a system with 16 MB ram today! we have 65 users on 2 gig ram, and when we do payments of unemployment salaries to our members we do everything in memory, and just write the edi file to a record a large batch takes less than two minutes and it also generates payment specifications for storage in pdf format (using cross pdf package). oh and we also do an xml conversion of the edi file on the fly aswell.. UniVerse is really a top performer when it comes to number crunching and file management how do you power a 16 meg system today ?? - with steam ? Note the use of the PAST tense. That machine is now salvage in my garage, waiting for me to restore it to personal use. I was just pointing out that MMV, and some things may work for some people and not for others. Why we were trying to run 32 users on 16 meg, even when the system was brand new (1990ish), I don't know. Penny-pinching, I guess. It's just that WRITESEQ makes a lot of sense when you're building BIG strings and are short of RAM... Med venlig hilsen Claus Derlien programmxr Edb-afdelingen Cheers, Wol Our system was short of RAM - 16 meg for 32 users (PI/Open on an EXL 7330). It thrashed enough under normal load, even before you started to try and build large (and I mean LARGE) strings in BASIC... Cheers, Wol -- Anthony W. Youngman [EMAIL PROTECTED] 'Yings, yow graley yin! Suz ae rikt dheu,' said the blue man, taking the thimble. 'What *is* he?' said Magrat. 'They're gnomes,' said Nanny. The man lowered the thimble. 'Pictsies!' Carpe Jugulum, Terry Pratchett 1998 Visit the MaVerick web-site - http://www.maverick-dbms.org Open Source Pick --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ -- Anthony W. Youngman [EMAIL PROTECTED] 'Yings, yow graley yin! Suz ae rikt dheu,' said the blue man, taking the thimble. 'What *is* he?' said Magrat. 'They're gnomes,' said Nanny. The man lowered the thimble. 'Pictsies!' Carpe Jugulum, Terry Pratchett 1998 Visit the MaVerick web-site - http://www.maverick-dbms.org Open Source Pick --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
[U2] Slow ascii output
I'm going to ask yet another dumb question - Unidata 6.1.4 on Solaris (soon to be 7.1.x). I'm selecting a bunch of records and then outputting data from them into an ascii file in _HOLD_. If I open a sequential file, write the data line-by-line (WRITESEQF) and close the file, it takes about 5 minutes. If I save the data in a @FM delimited record and then write the record out at the end, it takes about 3 seconds. I'm well aware that writing sequentially is doing a whole lot more disk I/O but I can't believe the difference in speed. Are their any subtleties other than disk I/O? I seem to recall some discussion about seq files and maintaining the pointer of where the current position in the file, but I'm foggy on these topics... -- Jeff Butera, Ph.D. Administrative Systems Hampshire College [EMAIL PROTECTED] 413-559-5556 ...our behavior matters more than the beliefs that we profess. Elizabeth Deutsch Earle --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] Slow ascii output
Hi Jeff, Couple of ideas: -1- Try WRITESEQ instead of WRITESEQF so the system will buffer your output in memory and write it to disk in more efficient chunks. According to HELP WRITESEQF, your command forces UniData to immediately write the data to the disk so you're taking an I/O hit. -2- Would this be easier? _HOLD_ is a DIR-type file, so you can just PRINT each data line you want. The data is then delimited by linefeeds (ASCII 10) at the Unix level. Unidata converts the linefeeds to attribute marks whenever you work with a DIR-type record from a Unidata command. There's an option in the SETPTR command to choose the output file name you want, but don't recall the exact syntax. Hope this helps, Tom [EMAIL PROTECTED] 10/20/06 11:04 AM I'm going to ask yet another dumb question - Unidata 6.1.4 on Solaris (soon to be 7.1.x). I'm selecting a bunch of records and then outputting data from them into an ascii file in _HOLD_. If I open a sequential file, write the data line-by-line (WRITESEQF) and close the file, it takes about 5 minutes. If I save the data in a @FM delimited record and then write the record out at the end, it takes about 3 seconds. snip - This e-mail and any attachments may contain CONFIDENTIAL information, including PROTECTED HEALTH INFORMATION. If you are not the intended recipient, any use or disclosure of this information is STRICTLY PROHIBITED; you are requested to delete this e-mail and any attachments, notify the sender immediately, and notify the LabCorp Privacy Officer at [EMAIL PROTECTED] or call (877) 23-HIPAA / (877) 234-4722. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Slow ascii output
Jeff, why WRITESEQF instead of WRITESEQ? You're completely undermining write buffering with WRITESEQF are you not? -Kevin [EMAIL PROTECTED] http://www.PrecisOnline.com ** Check out scheduled Connect! training courses at http://www.PrecisOnline.com/train.html. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] Slow ascii output
In message [EMAIL PROTECTED], Jeffrey Butera [EMAIL PROTECTED] writes I'm going to ask yet another dumb question - Unidata 6.1.4 on Solaris (soon to be 7.1.x). I'm selecting a bunch of records and then outputting data from them into an ascii file in _HOLD_. If I open a sequential file, write the data line-by-line (WRITESEQF) and close the file, it takes about 5 minutes. If I save the data in a @FM delimited record and then write the record out at the end, it takes about 3 seconds. BE WARNED. We had exactly the same thing the other way round - indeed I rewrote a lot of our routines to use WRITESEQ instead of building an array in BASIC. I'm well aware that writing sequentially is doing a whole lot more disk I/O but I can't believe the difference in speed. Are their any subtleties other than disk I/O? I seem to recall some discussion about seq files and maintaining the pointer of where the current position in the file, but I'm foggy on these topics... Our system was short of RAM - 16 meg for 32 users (PI/Open on an EXL 7330). It thrashed enough under normal load, even before you started to try and build large (and I mean LARGE) strings in BASIC... Cheers, Wol -- Anthony W. Youngman [EMAIL PROTECTED] 'Yings, yow graley yin! Suz ae rikt dheu,' said the blue man, taking the thimble. 'What *is* he?' said Magrat. 'They're gnomes,' said Nanny. The man lowered the thimble. 'Pictsies!' Carpe Jugulum, Terry Pratchett 1998 Visit the MaVerick web-site - http://www.maverick-dbms.org Open Source Pick --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Slow ascii output
I use writeseq. Yesterday I ran a process that selects a number of files, parses them into special formats, and writes them out. I created 65 files, 106 MB. Total time: 4 minutes. UD 5.1.27, Windows 2003 However, if you are writing to a dir file then you can use either a dynamic array or dimensioned (make sure you put something in each row or you will get an unassigned variable error) and simply write to the file. I find the writeseq faster. I use the best of both worlds and do: OPENSEQ 'dir type file', 'txt file' TO myfile ELSE STOP WRITESEQ line APPEND ON myfile ELSE STOP This way I don't have a hard-coded file path in my code and can simply move the file around by changing the VOC pointer. Hth Colin Alfke Calgary, Canada -Original Message- From: Jeffrey Butera I'm going to ask yet another dumb question - Unidata 6.1.4 on Solaris (soon to be 7.1.x). I'm selecting a bunch of records and then outputting data from them into an ascii file in _HOLD_. If I open a sequential file, write the data line-by-line (WRITESEQF) and close the file, it takes about 5 minutes. If I save the data in a @FM delimited record and then write the record out at the end, it takes about 3 seconds. I'm well aware that writing sequentially is doing a whole lot more disk I/O but I can't believe the difference in speed. Are their any subtleties other than disk I/O? I seem to recall some discussion about seq files and maintaining the pointer of where the current position in the file, but I'm foggy on these topics... -- Jeff Butera, Ph.D. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] Slow ascii output
no one and I mean NO ONE uses a system with 16 MB ram today! we have 65 users on 2 gig ram, and when we do payments of unemployment salaries to our members we do everything in memory, and just write the edi file to a record a large batch takes less than two minutes and it also generates payment specifications for storage in pdf format (using cross pdf package). oh and we also do an xml conversion of the edi file on the fly aswell.. UniVerse is really a top performer when it comes to number crunching and file management how do you power a 16 meg system today ?? - with steam ? Med venlig hilsen Claus Derlien programmxr Edb-afdelingen Our system was short of RAM - 16 meg for 32 users (PI/Open on an EXL 7330). It thrashed enough under normal load, even before you started to try and build large (and I mean LARGE) strings in BASIC... Cheers, Wol -- Anthony W. Youngman [EMAIL PROTECTED] 'Yings, yow graley yin! Suz ae rikt dheu,' said the blue man, taking the thimble. 'What *is* he?' said Magrat. 'They're gnomes,' said Nanny. The man lowered the thimble. 'Pictsies!' Carpe Jugulum, Terry Pratchett 1998 Visit the MaVerick web-site - http://www.maverick-dbms.org Open Source Pick --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
