Thanks for the reply Will, but I do understand how U2 files work. That's why I'm using UniBasic to read the record then write it out to UNIX.
If the files we search weren't so big I would just dump a copy to a dir than search it, but that isn't practical in most cases for us. Maybe I can improve it to write the records in blocks of 'x', search them. That would at least save some of the processing cost of jumping around, but it doesn't solve the Disk I/O bottleneck. -----Original Message----- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of fft2...@aol.com Sent: Tuesday, 30 March 2010 5:14 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Regex searching UD files Dan you can't do it because the "records" in Universe are not files at all, not even logical files. They are just possible frame-discontinous streams of bytes that *Universe* interprets in run-time as a connected "record". Nothing else is going to be able to do that, without re-writing the frame(block)-walker directly in Unix. Alternatively you could write egrep in Universe yourself which would probably be easier! But I wouldn't read and write after each record, I would read and write the entire file at *once*. And then egrep. I sure that would save on thrashing at least. If you're processing at least 10 percent of the file, then you might see that writing the whole file, then switching to Unix for the rest will still be faster than breaking after each write. Will Johnson In a message dated 3/28/2010 11:22:25 P.M. Pacific Daylight Time, stuart.boyd...@spotless.com.au writes: I think this should be one for Rocket to implement a RegexSearch in U2. However, if it's size non-impactive, what I would normally do is copy the file to a temporary directory then run grep across that. Stuart Boydell -----Original Message----- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Dan McGrath Sent: Monday, 29 March 2010 17:08 To: U2 Users List Subject: [U2] Regex searching UD files I've written a small utility to be able to run egrep on a UD file (not UD directory), however its implementation is not ideal. Essentially, I select the file I'm searching, writing each record one at a time to a temp UNIX file and running egrep on it as follows !egrep -q -f MyTempRegexFile MyTempRecordFile ; echo $? Where MyTempRegexFile is a file containing the desired regex pattern stored earlier and MyTempRecordFile is a file name unique to each user. My problem with it is that I have to do a READ on each record, followed by a WRITE then have egrep read it in as well. That's a lot of seemingly unneeded disk IO if I could only stream the record to egrep without have to do a WRITE after the READ. Any ideas? I feel I'm missing something quite obvious. Regards, Dan ######################################################################## #### ############### The information transmitted in this message and attachments (if any) is intended only for the person or entity to which it is addressed. The message may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer. The intended recipient of this e-mail may only use, reproduce, disclose or distribute the information contained in this e-mail and any attached files with the permission of IMB. ######################################################################## #### ############### _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________ ########################################################################################### The information transmitted in this message and attachments (if any) is intended only for the person or entity to which it is addressed. The message may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer. The intended recipient of this e-mail may only use, reproduce, disclose or distribute the information contained in this e-mail and any attached files with the permission of IMB. ########################################################################################### _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
