David:
What you're saying is mostly true, totally true with older machines with
.25Gb of memory, etc. Today I have an 8Gb notebook and opening a 40Mb
file should be trivial in our development environment. It should be our
developers who decide which tools to place into production, not the dbms
vendor. Secondly, these niggly little memory limitations that may (note
I wrote 'may') have been necessary in 1990, with machines having 100Mb
of memory, are ridiculous today (20Mb limit for AE, 256 variable limit
in a query line, etc, etc, etc).
What's worse, in this case, is the crashing. Do we want our software
vendors to say "don't do that" to users who complain of systems
crashing. We're getting so used to crashes with things like Thunderbird
and browsers maybe the presumption is we should be using more modern
tools, that essentially do the same thing differently, have a whole new
learning curve, and crash at least as often if not more. :-)
So, now that I think about it, maybe Jack should be grateful it's a
crash he's familiar with. :-)
Bill
------------------------------------------------------------------------
----- Original Message -----
*From:* [email protected]
*To:* U2 Users List <[email protected]>
*Date:* 6/6/2011 5:04 AM
*Subject:* Re: [U2] Universe 11.1.2 list line limit
Hi Jack
I would use the OPENSEQ and READSEQ to deal with such files. The problem with
ED and READ apart from loading the whole record into memory is that to go to
attribute 1M it goes to the start and then counts a million field marks. Then
you want to read 1,000,001 it goes back to the beginning and reads 1,000,001
field marks it is extremely slow and unworkable. READSEQ, WRITESEQ leaves a
pointer from the last attribute position. You have to process these files
sequential but it would read 40MB in nanoseconds, far more efficient.
Regards
David Jordan
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of jig
Sent: Monday, 6 June 2011 8:15 PM
To: [email protected]
Subject: Re: [U2] Universe 11.1.2 list line limit
OK sorry Kate,
I should have given a little more information.
On our system (AIX v5.3) we recently upgraded to 11.1.2 of UV from v10.3, and
we deal a lot with lists. Large lists with hundreds of thousands or even
millions of lines (elements). These are just lists of keys to records in files.
One in particular is a fixed length seven byte key (alphanumeric).
However the largest file has 3.5 million records in it. This means that our
lists can get to over 3 million lines ( this is about 40MB of a single file in
a Unix directory).
On version 10.3 when a basic program or the ED editor was used to read one of these
lists (with anything over 2 million lines of keys in it) it would just core dump and
not be able to read it. The same thing occurred whether one had the list in
the&SAVEDLISTS& file or anywhere else.
To get around this problem we wrote software to split these large lists.
However that was cumbersome and not always worked to specification.
With UV 11.1.2 (and because according to Rocket they did a lot of work to
improve memory management) we thought we'd be better off, handling some of
these large lists. However it seems we were wrong.
A Basic routine cannot read a list that is over 4 million records in it. It
just falls over and the process dies. The AE editor is a little more graceful
with some information before it also dies trying to open any large list of keys.
I know there are ways around this issue, but we expected a lot better
performance from the new version. I dont have the exact error at the moment,
because I am not at work, but I can add it here if requested. One can always
try it themselves to see the error, because it happens every time.
I am sure most companies do not work with such large lists, but even so, core
dumping instead of a gracious exit is not good enough. This is the first thing
we learned when we started out as programmers decades ago.
regards
jack
Kate Stanton wrote:
What limit? I do not understand. TIA., Kate
On 6 June 2011 19:32, jig<[email protected]> wrote:
Wondering if anyone is aware of the line limit in a saved list for
the latest UV version?
Has it improved since we went to 64-bit?
--
View this message in context:
http://old.nabble.com/Universe-11.1.2-list-line-limit-tp31781147p3178
1147.html Sent from the U2 - Users mailing list archive at
Nabble.com.
_______________________________________________
U2-Users mailing list
[email protected]
http://listserver.u2ug.org/mailman/listinfo/u2-users
--
Kate Stanton
Walstan Systems Ltd
4 Kelmarna Ave, Herne Bay, Auckland 1011, New Zealand
Phone: + 64 9 360 5310 Mobile: + 64 21 400 486
Email: [email protected]
_______________________________________________
U2-Users mailing list
[email protected]
http://listserver.u2ug.org/mailman/listinfo/u2-users
--
View this message in context:
http://old.nabble.com/Universe-11.1.2-list-line-limit-tp31781147p31782039.html
Sent from the U2 - Users mailing list archive at Nabble.com.
_______________________________________________
U2-Users mailing list
[email protected]
http://listserver.u2ug.org/mailman/listinfo/u2-users
_______________________________________________
U2-Users mailing list
[email protected]
http://listserver.u2ug.org/mailman/listinfo/u2-users
_______________________________________________
U2-Users mailing list
[email protected]
http://listserver.u2ug.org/mailman/listinfo/u2-users
