There are a lot of Universe features I Would like to see in Unidata. Distributed files and all the hashing algorithms would be a nice start.
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Wjhonson Sent: Friday, July 12, 2013 12:45 PM To: [email protected] Subject: Re: [U2] UniData Dynamic File Splitting Question Ahhh if that's true is amazing. Universe has had this for over twelve years. I guess the number of customers who used it was too small for them to port it to Unidata. -----Original Message----- From: Dave Davis <[email protected]> To: U2 Users List <[email protected]> Sent: Fri, Jul 12, 2013 10:40 am Subject: Re: [U2] UniData Dynamic File Splitting Question UniData doesn't do distributed files. -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Wjhonson Sent: Friday, July 12, 2013 1:38 PM To: [email protected] Subject: Re: [U2] UniData Dynamic File Splitting Question This statement of yours is a wonder <<As for distributed files, Ellucian doesn't support them in the application so they are not an option at this point.>> Ellucian has an application that reaches in and reads and writes data in some kind of raw form? Otherwise I have no idea how the application would even *know* that the underlying file is dynamic since that detail should be completely hidden at the application level. It's as if an application would know that a file is dynamic or not dynamic, the application should both not know, and more importantly not *care*. -----Original Message----- From: Cinda Goff <[email protected]> To: U2 Users List <[email protected]> Sent: Fri, Jul 12, 2013 10:28 am Subject: Re: [U2] UniData Dynamic File Splitting Question Definitely crazy. When we rebuild some of files, they fit into one dat and one over so there was no reason for them to split in the first place. Application never purges records in most files so it is not likely a space reclamation issue. I'm used to a split grabbing a large chunk of space so basically, dat and over pretty much increase together if needed with hopefully, over lagging behind. We normally use KEYONLY which is the application default unless we have looked at the file and determined we need to go to KEYDATA. The one Rocket issue that I did not think applied was for KEYDATA files and at the time we looked to see if any of the files at that particular college had been converted to KEYDATA. We use hash type 0. I don't have a GROUP.STAT on this file but I'm working with one of the colleges today and will gather that information. Below is a FILE.STAT from another small file which we found at another college yesterday. The file does have overflow but the records are very small and there really aren't that many records in the file. :FILE.STAT UI.LOG.INFO File name(Dynamic File) = UI.LOG.INFO Number of groups in file (modulo) = 3277 Dynamic hashing, hash type = 0 Split/Merge type = KEYONLY Block size = 1024 File has 1632 groups in level one overflow. Number of records = 44217 Total number of bytes = 2781198 Average number of records per group = 13.5 Standard deviation from average = 3.9 Average number of bytes per group = 848.7 Standard deviation from average = 244.4 Average number of bytes in a record = 62.9 Average number of bytes in record ID = 18.6 Standard deviation from average = 3.3 Minimum number of bytes in a record = 49 Maximum number of bytes in a record = 76 Minimum number of fields in a record = 5 Maximum number of fields in a record = 5 Average number of fields per record = 5.0 Enter <New line> to continue... Standard deviation from average = 0.0 File has 17 over files, 22 prime files As for distributed files, Ellucian doesn't support them in the application so they are not an option at this point. -------------------------------------------------- Cinda Goff N.C. Community College System Database Administrator 919 807-7060 vRoom Link: https://sas.elluminate.com/m.jnlp?password=M.BDCC127B096D131E11EAC16A0F9473&sid=2008362 E-mail correspondence to and from this address may be subject to the North Carolina Public Records Law and shall be disclosed to third parties when required by the statutes. (NCGS.Ch.132) -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Dave Laansma Sent: Friday, July 12, 2013 1:02 PM To: 'U2 Users List' Subject: Re: [U2] UniData Dynamic File Splitting Question Frankly, this doesn't look like a very big file at all, and I'm amazed you're getting any 'performance' out of it at all with all those OVER files. Crazy! Sincerely, David Laansma Hubbard Supply Co. Direct: 810-342-7143 Office: 810-234-8681 Fax: 810-234-6142 www.hubbardsupply.com "Delivering Products, Services and Innovative Solutions" -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Susan Lynch Sent: Friday, July 12, 2013 12:53 PM To: U2 Users List Subject: Re: [U2] UniData Dynamic File Splitting Question Cinda, my apologies for mis-spelling your name in my previous response . Susan Lynch -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Cinda Goff Sent: Friday, July 12, 2013 12:48 PM To: U2 Users List Subject: Re: [U2] UniData Dynamic File Splitting Question We could with some of them but a good number of them need to remain dynamic because they are in total over the 2gb limit. -------------------------------------------------- Cinda Goff N.C. Community College System Database Administrator 919 807-7060 vRoom Link: https://sas.elluminate.com/m.jnlp?password=M.BDCC127B096D131E11EAC16A0F947 3&sid=2008362 E-mail correspondence to and from this address may be subject to the North Carolina Public Records Law and shall be disclosed to third parties when required by the statutes. (NCGS.Ch.132) -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Wjhonson Sent: Friday, July 12, 2013 12:46 PM To: [email protected] Subject: Re: [U2] UniData Dynamic File Splitting Question Instead of trying to keep recreating these dynamically every night, why not switch to a non-dynamic form ? -----Original Message----- From: Cinda Goff <[email protected]> To: u2-users <[email protected]> Sent: Fri, Jul 12, 2013 9:36 am Subject: [U2] UniData Dynamic File Splitting Question I work with the North Carolina Community College System which supports the 58 NC colleges. We have an unusual dynamic file issue at a couple of the colleges and am hoping someone could provide some insight/direction. OS - Solaris 10 UniData version 7.2.5 (have upgraded a couple of times in the past from 6.1 and 7.1) Colleges have been running Ellucian's Colleague software since 2001 The problems do not seem to be occurring in all dynamic files but where there are issues, there are more than 100 files affected. From timestamps in some of the files, we can see that this has been going on for several years but we are unaware of the issue until the files have split into so many parts that the application can no longer open enough files to read/write the files. Database tools do not show these files as undersized or in overflow2. Guide will show you the many parts if you know what to look for but it is not obvious that the file is fragmented. When we found the issue at the first college we opened a call with Ellucian and was told to resize the files. We had a call to Rocket opened and unfortunately didn't get much further. The Rocket support analyst looked at udtdiags for us and did not find anything out of the ordinary. Rocket identified a couple of potential dynamic file issues at our UniData version and suggested that we upgraded to UniData 7.3. One of the identified issues seemed to apply but it did not explain all issues. We are looking at the UniData upgrade but we likely will still need to address the current file issues. We started resizing files and the files were compressed as expected but soon began splitting again. To 'fix' the files now, we are manually recreating the files, copying data to new files, renaming everything and rebuilding any indices. Long and tedious process but the file seem to behave better, at least so far. Below is a listing of one of the main application files and all its parts. Many of the files have many more parts that the listing below. The splitting occurs with dat, over and idx with files some files having many of one part while other have many of all. If anyone has any insight into how/why the files split this way and suggestion on how to correct and prevent it in the future, I would very much like to hear your ideas. Thanks for your input and suggestions. Cinda Goff sun2:/datatel/coll18/production/apphome/DATA/DATA_P/PERSON>165-$ ls -al total 533958 drwxrwx--- 2 datatel users 1536 Jun 26 12:42 . drwxrwx--- 269 datatel users 13824 Feb 18 10:24 .. -rw-rw---- 1 datatel users 6292480 Jul 1 12:57 dat001 -rw-rw---- 1 datatel users 152743936 Jul 1 12:56 idx001 -rw-rw-rw- 1 gmoore users 4096 Aug 12 2011 idx002 -rw-rw-rw- 1 gmoore users 4096 Aug 12 2011 idx003 -rw-rw-rw- 1 ijackson users 4096 Aug 12 2011 idx004 -rw-rw-rw- 1 gmoore users 4096 Aug 12 2011 idx005 -rw-rw-rw- 1 sbarnes users 4096 Aug 12 2011 idx006 -rw-rw-rw- 1 whardwic users 4096 Nov 5 2011 idx007 -rw-rw-rw- 1 jjacobs users 4096 Nov 5 2011 idx008 -rw-rw-rw- 1 jlevister users 4096 Nov 5 2011 idx009 -rw-rw-rw- 1 rcox users 4096 Nov 5 2011 idx010 -rw-rw-rw- 1 gmoore users 4096 Nov 5 2011 idx011 -rw-rw-rw- 1 jjacobs users 4096 Nov 5 2011 idx012 -rw-rw-rw- 1 tbozeman users 4096 Nov 5 2011 idx013 -rw-rw-rw- 1 pperry users 4096 Nov 5 2011 idx014 -rw-rw-rw- 1 kconner users 4096 Nov 5 2011 idx015 -rw-rw-rw- 1 pperry users 4096 Nov 5 2011 idx016 -rw-rw-rw- 1 gmoore users 4096 Nov 5 2011 idx017 -rw-rw-rw- 1 sbarnes users 4096 Nov 5 2011 idx018 -rw-rw-rw- 1 jjacobs users 4096 Nov 5 2011 idx019 -rw-rw---- 1 datatel users 113200128 Jul 1 12:57 over001 -rw-rw---- 1 whardwic users 19456 Jun 20 08:26 over002 -rw-rw---- 1 ijackson users 18432 May 16 08:04 over003 -rw-rw---- 1 gmoore users 18432 May 14 12:09 over004 -rw-rw---- 1 sbarnes users 17408 Jun 4 19:03 over005 -rw-rw---- 1 llowry users 18432 Feb 1 13:01 over006 -rw-rw---- 1 dmiadmin users 17408 Jun 20 08:26 over007 -rw-rw---- 1 gmoore users 24576 Jun 28 07:31 over008 -rw-rw---- 1 gmoore users 18432 Mar 14 09:10 over009 -rw-rw---- 1 gmoore users 21504 May 25 06:15 over010 -rw-rw---- 1 gmoore users 18432 Jun 22 12:43 over011 -rw-rw---- 1 ijackson users 19456 Jun 4 20:05 over012 -rw-rw---- 1 sbarnes users 19456 Jun 17 14:47 over013 -rw-rw---- 1 bsampson users 19456 May 26 06:11 over014 -rw-rw---- 1 gmoore users 23552 Jun 25 06:15 over015 -rw-rw---- 1 plocklea users 21504 Jun 24 16:22 over016 -rw-rw---- 1 gmoore users 19456 Jan 31 11:31 over017 -rw-rw---- 1 gmoore users 22528 Jun 6 12:15 over018 -rw-rw---- 1 sbarnes users 19456 Jun 26 06:15 over019 -rw-rw---- 1 llowry users 21504 Jun 20 08:26 over020 -rw-rw---- 1 pperry users 21504 Jul 1 10:27 over021 -rw-rw---- 1 sbarnes users 19456 Jun 20 15:34 over022 -rw-rw---- 1 ebarton users 17408 Jun 20 08:26 over023 -rw-rw---- 1 pperry users 19456 May 26 06:09 over024 -rw-rw---- 1 mhester users 20480 Jun 12 15:07 over025 -rw-rw---- 1 pperry users 18432 May 20 12:20 over026 -rw-rw---- 1 pperry users 18432 May 6 13:24 over027 -rw-rw---- 1 jhickman users 19456 May 31 12:15 over028 -rw-rw---- 1 mhester users 18432 Jun 26 06:15 over029 -rw-rw---- 1 tbozeman users 19456 May 31 12:15 over030 -rw-rw---- 1 whardwic users 20480 Aug 8 2012 over031 -rw-rw---- 1 dbrooks users 21504 May 31 12:15 over032 -rw-rw---- 1 sbarnes users 21504 May 25 06:14 over033 -rw-rw---- 1 kconner users 20480 May 26 06:14 over034 -rw-rw---- 1 gmoore users 22528 May 26 06:14 over035 -rw-rw---- 1 whardwic users 19456 Jun 20 08:26 over036 -rw-rw---- 1 sbarnes users 23552 Jun 20 08:26 over037 -rw-rw---- 1 mchavis users 20480 Jun 12 15:06 over038 -rw-rw---- 1 sbarnes users 17408 Jun 20 16:28 over039 -rw-rw---- 1 suchavis users 20480 Jun 17 13:23 over040 -rw-rw---- 1 bj0663827 users 20480 Jun 27 06:15 over041 -rw-rw---- 1 bj0663827 users 17408 Apr 9 11:11 over042 -rw-rw---- 1 tsampson users 21504 Jun 3 15:54 over043 -rw-rw---- 1 whardwic users 19456 Jun 20 12:47 over044 -rw-rw---- 1 sbarnes users 21504 Jun 26 12:42 over045 sun2:/datatel/coll18/production/apphome/DATA/DATA_P/PERSON>166-$ -------------------------------------------------- Cinda Goff N.C. Community College System Database Administrator 919 807-7060 vRoom Link: https://sas.elluminate.com/m.jnlp?password=M.BDCC127B096D131E11EAC16A0F947 3&sid=2008362 E-mail correspondence to and from this address may be subject to the North Carolina Public Records Law and shall be disclosed to third parties when required by the statutes. (NCGS.Ch.132) _______________________________________________ U2-Users mailing list [email protected] http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list [email protected] http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list [email protected] http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list [email protected] http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list [email protected] http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list [email protected] http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list [email protected] http://listserver.u2ug.org/mailman/listinfo/u2-users ________________________________ Dave Davis Team Lead, Research & Development P: 614-875-4910 x108 F: 614-875-4088 E: [email protected] [http://www.harriscomputer.com/images/signatures/HarrisSchools.jpg] [http://www.harriscomputer.com/images/signatures/DivisionofHarris.gif]<http://www.harriscomputer.com/> 6110 Enterprise Parkway Grove City, OH 43123 www.harris-schoolsolutions.com<http://www.harris-schoolsolutions.com> This message is intended exclusively for the individual or entity to which it is addressed. This communication may contain information that is proprietary, privileged or confidential or otherwise legally exempt from disclosure. If you are not the named addressee, you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this message in error, please notify the sender immediately by e-mail and delete all copies of the message. _______________________________________________ U2-Users mailing list [email protected] http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list [email protected] http://listserver.u2ug.org/mailman/listinfo/u2-users ------------------------------------------------------------------------------ CONFIDENTIALITY NOTICE: If you have received this email in error, please immediately notify the sender by e-mail at the address shown. This email transmission may contain confidential information. This information is intended only for the use of the individual(s) or entity to whom it is intended even if addressed incorrectly. Please delete it from your files if you are not the intended recipient. Thank you for your compliance. Copyright (c) 2013 Cigna ============================================================================== _______________________________________________ U2-Users mailing list [email protected] http://listserver.u2ug.org/mailman/listinfo/u2-users
