Yes, I see. Good point. I'll experiment with your handler,
although I guess I need a better understanding of potential unseen
errors. The raw data files are unlikely to change in format generally.
My approach has been to examine a small, random subset of records
visually and then test handlers on known records. When I'm satisfied
with the handlers, I test them on larger, random subsets. I also extract
these larger subsets by another means, and verify the results by other
methods, say, by using simple spreadsheets. It's a primitive approach,
that shows my inexperience, but it seems to have worked so far.
Greg
On 29/11/2000 10:59 AM, George Wade wrote:
>From: "George Wade" <[EMAIL PROTECTED]>
>Subject: Re: Processing a big data file
>Date: Wed, 29 Nov 2000 12:04:26 +0900
>MIME-Version: 1.0
>Content-Type: text/plain; charset="ISO-8859-1"
>Content-Transfer-Encoding: 8bit
>
>
>What I was trying to say was: a stack processing a big data file could
>produce some errors without you noticing. The data that you are processing
>appears to be of some importance. The results may be important, too.
>
>How important is it to know that the results are accurate or reliable?
>
>I made a suggestion for checking which may be useful or, more likely,
>could be improved upon.
>
>George,
>Kochi
>
>On November 29 2000, Gregory Lypny <[EMAIL PROTECTED]> wrote:
>
>>Hi George,
>>
>> I'm not sure that I understand your question. The stack is being
>>developed for data files which.....
>
>>On 28/11/2000 1:24 PM, George Wade wrote:
>>
>>>Something that strikes me with this question is:
>>>
>>>How do you check that the stack is doing what you want it to.....
>
_________________________________________
Gregory Lypny
Associate Professor of Finance
Concordia University
_________________________________________
"Take chances, make mistakes!"
- Ms Frizzle, The Magic School Bus
Archives: http://www.mail-archive.com/[email protected]/
Info: http://www.xworlds.com/metacard/mailinglist.htm
Please send bug reports to <[EMAIL PROTECTED]>, not this list.
