On 02/12/2014 11:56 AM, John Redford wrote:
From: [email protected]
I wonder if it's similar in the financial and medical industries, but
with us the fixed record format is not the thing that's standardized but
some binary format I've never seen, handling of which is the domain of a
specialized company known as an "EDI Provider." Would it be cynical to
think the standardization in these areas is driven as much by
rent-seeking as by interoperability? Cause it sounds like a sweet
deal. ...
I
didn't attend, and perhaps someone already made this point, but it
often seems to be lost in Perl and Perl-like communities
where sequential IO is considered the only kind of IO, and memory
allocation is hidden from the developer.
There are two generally significant "wins" for fixed-record formats.
1.
Static allocation. Records can be read without reallocating or wasting
memory. This is faster/better for obvious reasons. Many languages make
it possible to map a fixed structure over the memory used, so inner
elements can be accessed conveniently -- and unions make it possible to
access conditional content nicely.
that is only true if all the records are the same length (good old 80
column punchcards). the data i was parsing has fixed sizes but each
record type was a different size. it is a parsing problem, not an i/o
problem i had.
2. Dynamic record access. When
data is stored on a filesystem, mmap/CreateFileMapping can be used to
map the record-data and offsets can index to an arbitrary record
directly.
not applicable to the talk. again, not many file formats can be fixed in
size like that. padding is needed for shorter records and longer ones
can be truncated or run to multiple records (continuation columns to the
rescue!).
There are many languages that can take advantage of these factors. Perl is bad
at both.
not really. perl can do both seek and sysseek and can random access
fixed size records easily. no worse than any other lang for that. and
you mentioned mmap which it can do too.
thanx,
uri
--
Uri Guttman - The Perl Hunter
The Best Perl Jobs, The Best Perl Hackers
http://PerlHunter.com
_______________________________________________
Boston-pm mailing list
[email protected]
http://mail.pm.org/mailman/listinfo/boston-pm
