RE: Flat file generation integrity ideas...

[Grabowy, Chris]

Title: Message

(slapping my forehead)  Duh.

 

Nice.  I like it.  Simple but effective.  Minimal impact on performance.  And easy for the other systems to implement. 

 

You da man, Tom, I don't care what Jared and Rachel said about you...

 

Thanks.

-----Original Message-----
From: Mercadante, Thomas F [mailto:[EMAIL PROTECTED]]
Sent: Thursday, October 24, 2002 2:14 PM
To: Multiple recipients of list ORACLE-L
Subject: RE: Flat file generation integrity ideas...

Chris,

 

have you thought about summing a number column in the record and placing this sum in a trailer record?  this way, you have a header and trailer record which helps you be confident that the whole file made it to the target system.  and by comparing the sum of the imported records with the trailer record, it gives you a better level of confidence that things didn't get scrambled.

 

I used to do all sorts of these things when file transfer was not as good as it is now.  the above method is pretty simple, easy to do, and pretty accurate.

 

hope this helps.

 

Tom Mercadante
Oracle Certified Professional

-----Original Message-----
From: Grabowy, Chris [mailto:[EMAIL PROTECTED]]
Sent: Thursday, October 24, 2002 1:35 PM
To: Multiple recipients of list ORACLE-L
Subject: Flat file generation integrity ideas...

I have to create packages that will generate several flat files of data from tables that will be sent to other systems to be processed.

I am looking for ideas on how to ensure data integrity in the flat files.

For example, the expected record count is stored on the first line of the file to ensure that the correct amount of records was received.

The systems group is chartered to ensure the flat files are correctly FTPed between systems, so that's covered.

I just worry that if "somehow" a flat file is scrambled then the scrambled data is loaded into the database, therefore corrupting it.

At this phase, XML is not an option

I keep thinking that some sort of CRC should be stored with each line in the flat file.  And then before the line is loaded into the database, the CRC is compared against the generated CRC of the just read line.  Has anyone done anything like this?  Any examples out there?

Many TIA!!