Anthony Beecher takes in EDI batches, which he wants to map to XML
files; some are too large (20 - 30 Megs) for Gentran:Server NT to
handle. We all suspect the performance problems have got something to
do with the database used by Gentran. Either the SQL calls will have to
be optimized, or the database will have to be isolated on its own path,
or a RAID will have to be installed, or we can add more RAM, or we have
to upgrade to a quad-processor water-cooled Pentium 800mH, or move to a
different platform altogether, or ....
I think Laurent Szyster hit the nail on the head - that "[it's] the
software design." Laurent goes on:
I assume that Gentran [puts] a lot in a database (which
implies sorting and retrieving from a sorted collection).
The larger the dataset, the slower the processing ;-)
Obviously, at one point, Gentran use of the database
generates so much overhead that it accounts for more than
4/5 of the total processing time.
As the index for a database grows, especially when random keys are
added, each subsequent access requires more and more time. Probably
this accounts for much of the geometric growth in processing or elapsed
time for Anthony's EDI to XML translation using Gentran:Server NT.
Back in my day, in these circumstances we'd just extract information to
a flat file, much as Eliot Muir suggested. The 3M or or 20M or 100M or
6GB or whatever were slightly reformatted or rearranged, but nothing
requiring database calls or much brain-power. Then we'd sort by the key
fields and load the data sequentially into the VSAM keyed database.
Since the indices were built in key order, it went lickety-split.
But that was then, in the olden days of COBOL and VSAM and two digit
years. Young whipper-snappers today insist in going directly to the
database in any old random order across a network with an n-tier
application design, even for a batch process, because they can always
scale up the hardware.
Anthony: how big are the resulting XML files going to be? And if
they're proportional in size to the 20-30 megabyte input EDI files,
whatever are you going to do with them?
William J. Kammerer
FORESIGHT Corp.
4950 Blazer Memorial Pkwy.
Dublin, OH USA 43017-3305
(614) 791-1600
Visit FORESIGHT Corp. at http://www.foresightcorp.com/
"Commerce for a New World"
=======================================================================
To signoff the EDI-L list, mailto:[EMAIL PROTECTED]
To subscribe, mailto:[EMAIL PROTECTED]
To contact the list owner: mailto:[EMAIL PROTECTED]
Archives at http://www.mail-archive.com/edi-l%40listserv.ucop.edu/
