Re: [U2] UV 10.1.22; Dynamic File question

[Charles Stevenson]

Allen Egerton wrote:

The situation is that I'm potentially adding a million records to a
dynamic file.  That's not hypothetical, it's a real life scenario.  And
I don't want to wait while the file splits multiple times over the
course of the add.  I don't have that large a window.  I want to force
the file to expand the weekend before so that when I add the records the
file isn't rebuilding.

The file's already dynamic, so setting Minimum.Modulus doesn't seem to
do me much good unless I can figure out how to force it to split to that
minium.modulos.  And there's the crux of my question.  How can I force
it to split?

Allen,
Only doing CONFIGURE.FILE . . . MINIMUM.MODULUS  does not make the file 
grow.  It just tells UV that if & when the file ever grows that big, 
don't let it shrink smaller. You need to be sneakier.

Here are 2 ways reserving space on DATA.30,  & one for OVER.30.

1.  Do it this way if you can't have any downtime,  but can sneak 
writing one bogus little record.


Tinker with MERGE.LOAD & SPLIT.LOAD then repeatedly write a fake record, 
so that each write triggers a split until you get to the size you want.

Let's say you want min mod of 10,000 and your current mod is 1,000
CONFIGURE.FILE YOURFILE   MINIMUM.MODULUS 1   MERGE.LOAD 1  
SPLIT.LOAD 2

run this:
  open "YOURFILE" to F else stop
  for i  = 1000 to 1  ;* you need 9000 =10,000-1,000 splits
 write '' F, 'dummy'
 delete F, 'dummy'
  next i

Now set MERGE.LOAD & SPLIT.LOAD TO where you want them (usually 50 & 80).
Until you load your real data you may be well below MERGE percentage,  
but merges will not happen because of minimum.modulus parameter..



2. Use resize if you can get exclusive use of the file.  (I don't think 
"CONCURRENT" works with dynamic files.  but I'm wary with static too.)
RESIZE will allow you to set the minimum modulus, but maybe not using 
the syntax as documented. 
And syntax differs according to acct flavour and OS (and UV rel?)
If RESIZE does nOt recognize MINIMUM.MODULUS on the command line,  then 
specify the mod explicitly, like you would a static hashed file.  That 
will become the minimum.modulus and the file will be sized accordingly.


 RESIZE YOURFILE 30 1 4  [SEQ.NUM]  (syntax different in Pick 
flavour)


will resize the file,  type 30, setting MINIMUM.MODULUS TO 1  and 
reserve the disk space in DATA.30, setting the current modulus to 1 too.
(Of course, if the file is already larger than that, the current modulus 
will be larger than 1.)


3. Neither of the above will reserve room in OVER.30.   If you know what 
your data will be like, then you know how much overload you'll need.
If you have the luxury of being able to create bogus data, you could 
write many large records so that OVER.30 grows.  Then delete all the 
bogus large records.

Other han that,  I don't know a way to make OVER.30 grow.
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

Re: [U2] Need assistance replicating UniObjects.NET issue

[Bill Haskett]

   Symeon:
   Do you think this is pretty bogus or is it the Windows lag on releasing
   resources?
   Bill
   __

   From: Symeon Breen 
   Sent: 4/23/2009 1:34 PM
   To: u2-users@listserver.u2ug.org
   Subject: Re: [U2] Need assistance replicating UniObjects.NET issue

Glen wrote

After speaking with our administrator he said:

Technically the card and switches are set to "Auto-Duplex", which means
they are running in Full Duplex mode.

That may be the case but auto negotiate between the PC NIC and the switch
port does not always work - you should be able to tell from the switch port
if it actually has negotiated to full duplex or not. - Probably a barking up
the wrong tree there anyway.

Yes I agree a better way of capturing and dealing with these problems is the
way to go - my experience is of uniobjects.net in a web environment where we
are getting over 1 million hits per day, peaking at say 20 or more per
second and we do get a multitude of errors most days in the logs. - We
should not really ignore them but i think blips do just happen, certainly i
have found that the udt licence management is always a bit behind so if I
have all licences used and i drop a uniobjects connection and connect a new
one straight away it gives me a out of licence error - i sometimes have to
wait a few seconds before connecting again, which in web terms is a long
time. Sometimes i also getto thepoint where udt does not accept any new
uniobjects connections and the only way to fix this is to kill the unirpcd
service and restart it.
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

RE: [U2] Record Size Limit in UV {unclassified}

[HENDERSON MIKE, MR]

Adam,

I hate to contradict you, but the system *HAS* outgrown its design!
In a previous job, we had the exact same problem with serialised stock,
and eventually we had to bite the bullet and re-architect with a
more-normalised 'traditional SQL' joining table instead of embedded
multi-values.

We ended up with a table with attributes (IIRC) of
** as ID, Stock #, Serial #, Order #,
Line #. We may even have stuffed Customer # in too, as a de-normalised
performance-improver on reporting, I can't remember now, it was quite a
few years ago. And several UV indices to give access reasonably quickly.
The pain was significant, and the system became probably on average a
tiny tad slower - not enough for users to notice, though. The change did
stop the l-o-n-g pauses, which users certainly did notice, while UV grew
a record from a very large size to a little bit larger to add another
serial number to a sales record. 


YMMV as usual


Regards


Mike

 

> -Original Message-
> From: owner-u2-us...@listserver.u2ug.org On Behalf Of Adam Taylor
> Sent: Friday, 24 April 2009 10:57
> To: u2-users@listserver.u2ug.org
> Subject: RE: [U2] Record Size Limit in UV
> 
> Our scenario isn't so much outgrowing its design as expanding 
> upon its design.  Basically what happened was due to 
> accounting requirements, we had to make a change to our 
> company's inventory process to move it from a mixture of 
> serialized parts (for large-ticket items) and non-serialized 
> parts (for small-ticket items) to have everything be 
> serialized.  Combined with the fact that some of the orders 
> our customers place have line items with small-ticket items 
> where quantities range into the multiple thousands, this 
> causes very large amounts of data to be stored in the serial 
> number field of the records.
> 
> We are already seeing a performance hit in certain areas with 
> these instances.  Fortunately they are not completely common 
> occurrences, but they do occur frequently enough that we have 
> had two or three instances of connections timing out on some 
> of our users since we made the conversion three months ago.
>  
> Adam Taylor
> Program Manager
> 713.795.2352
> www.INXI.com
> 
> 
> 
[snip]
The information contained in this Internet Email message is intended
for the addressee only and may contain privileged information, but not
necessarily the official views or opinions of the New Zealand Defence Force.
If you are not the intended recipient you must not use, disclose, copy or 
distribute this message or the information in it.

If you have received this message in error, please Email or telephone
the sender immediately.
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

RE: [U2] Record Size Limit in UV

[Adam Taylor]

Our scenario isn't so much outgrowing its design as expanding upon its design.  
Basically what happened was due to accounting requirements, we had to make a 
change to our company's inventory process to move it from a mixture of 
serialized parts (for large-ticket items) and non-serialized parts (for 
small-ticket items) to have everything be serialized.  Combined with the fact 
that some of the orders our customers place have line items with small-ticket 
items where quantities range into the multiple thousands, this causes very 
large amounts of data to be stored in the serial number field of the records.

We are already seeing a performance hit in certain areas with these instances.  
Fortunately they are not completely common occurrences, but they do occur 
frequently enough that we have had two or three instances of connections timing 
out on some of our users since we made the conversion three months ago.
 
Adam Taylor
Program Manager
713.795.2352
www.INXI.com




-Original Message-
From: owner-u2-us...@listserver.u2ug.org 
[mailto:owner-u2-us...@listserver.u2ug.org] On Behalf Of Charles Stevenson
Sent: Thursday, April 23, 2009 12:12 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] Record Size Limit in UV

I've seen records at least an order of magnitude larger than that.  Unix 
& Windows. Megabytes.  I want to say much larger than that, but I can't 
verify that and when numbers get that big, my brain can't get around 
them.  ( e.g.,  $1,000,000,000,000: www.pagetutor.com/trillion/index.html  )

I don't know of absolute limits,  but performance takes a dive, of 
course: lock contention, frequent updates, inappropriate indexes, 
selections w/o indexing poorly sized files (they should be type 30, if 
they must exist.  The large records get isolated in their own 
overflow).  In general, U2 responds well to size abuse. Performance 
degradation is fairly linear & responds well to throwing more hardware 
at it.  But at some point degradation curve becomes geometric.

I'm curious about whether your scenario matches the usual: i.e., an 
application that has outgrown its original design, with a multi-valued 
association in that gets updated frequently through the lfe of the record.
The original design intended a few mv entires, until the business 
operation it represents gets closed.   At some point someone decided to 
use the association for some additional purpose.   Some records remain 
small, some grow, resulting in a very lumpy file.  Many times it's the 
users trying to respond quickly to a changing business need,  w/o 
waiting for IT to do the enhancement (they invent new status codes, 
etc.).  Sometimes it's programmers enamored by the siren call of mv nesting.

Adam Taylor wrote:
> Hey all,
>
> Can anyone tell me what (if any) the current record size limit is in UV 10.2?
> We've currently found ourselves in a situation where certain fields in a data
> file will contain 100K+ characters and we would like to be proactive about any
> potential problems if they exist.
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/
IMPORTANT/CONFIDENTIAL: This message from INX Inc. is intended only for the use 
of the addressees shown above. 
It contains information that may be privileged, confidential and/or exempt from 
disclosure under applicable law. 
If you are not the intended recipient of this message, you are hereby notified 
that the copying, use or distribution of any information or materials 
transmitted in or with this message is strictly prohibited. 
If you received this message by mistake, please immediately email or call us 
collect at (469) 549-3800 and delete/destroy the original message.
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

Re: [U2] UV 10.1.22; Dynamic File question

[Anthony W. Youngman]

In message <49f0c3cc.2050...@pobox.com>, Allen Egerton 
 writes

Martin Phillips wrote:

Hi Allen,


If I know in advance that I'm adding a large number of records
to a file, is there any way to force the file to "grow" in advance?


You could do this with the MINIMUM.MODULUS parameter but the whole point
of dynamic files is that they do their own tuning. Although a dynamic
file is unlikely to be as good as a perfectly tuned static file, it will
be a lot better than a typical static file that has been left alone for
a long while.

Another important point is that a split affects only one (or arguably
two) groups unlike resizing a static file from modulo N to modulo N+1
which would affect every group.

My real point here is that it is usually best to let a dynamic file look
after itself.


In general I'd agree with you.

The situation is that I'm potentially adding a million records to a
dynamic file.  That's not hypothetical, it's a real life scenario.  And
I don't want to wait while the file splits multiple times over the
course of the add.  I don't have that large a window.  I want to force
the file to expand the weekend before so that when I add the records the
file isn't rebuilding.


Just taken a look at the manual. Use CONFIGURE.FILE to change the 
parameters of a dynamic file, including MINIMUM.MODULUS. Note that 
RESIZE will apparently object to (or ignore) MIMIMUM.MODULUS if the file 
starts off as a dynamic file.


I'd just try changing the minimum modulus and seeing if it really does 
what it says. If it doesn't actually increase the modulus, try doing a 
RESIZE * * * and see if that adds the extra groups.


Bearing in mind, however, that resizing a file is normally pretty fast, 
why aren't you happy letting it do its own thing? The only thing to 
watch out for is if something locks the file and prevents resizing, like 
doing a BASIC SELECT. I've come across files that were 120% or so full 
...


In fact, if you felt that the impact of overflow was less than that of 
splitting (say the file is growing by less than 100%), you could simply 
do ED FILENAME * to disable splitting while you're adding all these 
records. The file will then gradually put itself right as it's updated 
later.


Cheers,
Wol
--
Anthony W. Youngman 
'Yings, yow graley yin! Suz ae rikt dheu,' said the blue man, taking the
thimble. 'What *is* he?' said Magrat. 'They're gnomes,' said Nanny. The man
lowered the thimble. 'Pictsies!' Carpe Jugulum, Terry Pratchett 1998
Visit the MaVerick web-site -  Open Source Pick
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

RE: [U2] UV 10.1.22; Dynamic File question

[Ken Wallis]

UniData has a CONFIGURE.FILE verb - I'd have thought UniVerse had it too.
This allows you to set the MINIMUM.MODULUS on an existing file to make it
bigger.

Cheers,

Ken

-Original Message-
From: Allen Egerton

Martin Phillips wrote:
> Hi Allen,
> 
>> If I know in advance that I'm adding a large number of records
>> to a file, is there any way to force the file to "grow" in advance?
> 
> You could do this with the MINIMUM.MODULUS parameter but the whole point
> of dynamic files is that they do their own tuning. Although a dynamic
> file is unlikely to be as good as a perfectly tuned static file, it will
> be a lot better than a typical static file that has been left alone for
> a long while.
> 
> Another important point is that a split affects only one (or arguably
> two) groups unlike resizing a static file from modulo N to modulo N+1
> which would affect every group.
> 
> My real point here is that it is usually best to let a dynamic file look
> after itself.

In general I'd agree with you.

The situation is that I'm potentially adding a million records to a
dynamic file.  That's not hypothetical, it's a real life scenario.  And
I don't want to wait while the file splits multiple times over the
course of the add.  I don't have that large a window.  I want to force
the file to expand the weekend before so that when I add the records the
file isn't rebuilding.

The file's already dynamic, so setting Minimum.Modulus doesn't seem to
do me much good unless I can figure out how to force it to split to that
minium.modulos.  And there's the crux of my question.  How can I force
it to split?

The file's numerically keyed, (sequential), so it hashes pretty evenly
across the groups, consequently I'm not overly worried about empty
groups due to forcing the splits beforehand.

I know this isn't how you, (or I), typically work with dynamic files.
I've got some slightly unusual circumstances and client induced
constraints that I'm working under.
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

[U2] Question on UniVerse VOC record for CREATE.FILE

[Doug Farmer]

I am mainly a UniData person, so UniVerse is a bit foreign to me.

I am running UniVerse on a Windows Platform.

Using the CREATE.FILE command, I created a file with a long name (15
characters) with my standard VOC pointer, and the VOC pointer in the
UV\NEWAC,PICK file.

Using the CREATE.FILE verb in my VOC, I am able to create the file
correctly.  Using the verb as it is on the NEWAC,PICK file, it truncates
the file name.  This, even though the LONGNAMES is on.  See the examples
at the end of the E mail.

My VOC entry looks like

>AE VOC CREATE.FILE
Top of "CREATE.FILE" in "VOC", 6 lines, 40 characters.
*--: L22
001: V
002: create.file
003: E
004: VF
005: LONGNAMES
006: PICK.FORMAT
Bottom.
*--:

The NEWACC\PICK file verb looks like this (After I copy it to my VOC and
set LONGNAMES)
>AE VOC PICK-CREATE-FILE
Top of "CREATE.FILE" in "VOC", 6 lines, 40 characters.
*--: L22
001: V
002: create.file
003: E
004: FG
005: LONGNAMES
006: PICK.FORMAT
Bottom.
*--:

The only difference is attribute 4 which has an FG instead of VF in it.
I have no idea what FG vs VF is.

You may wonder why I am asking?  I have an old application that creates
files specifically using the PICK version, no matter what the VOC
version is.

I don't know why, is there a difference?

The only difference I know of is that LONGNAMES do not seem to work
under the PICK version.

Any ideas?

Thanks in Advance

Doug


Using PICK-CREATE-FILE

>PICK-CREATE-FILE DAF1234567890123456 1,1,18 1,1,19
WARNING: An operating system file will be created with a truncated name.
Creating file "DAF123456000" as Type 19.
Creating file "D_DAF123456000" as Type 18, Modulo 1, Separation 1.

Using CREATE-FILE
>CREATE.FILE DAF123456789012345 1,1,18 1,1,19
Creating file "DAF123456789012345" as Type 19.
Creating file "D_DAF123456789012345" as Type 18, Modulo 1, Separation 1.
Added "@ID", the default record for RetrieVe, to "D_DAF123456789012345".


This e-mail is for the use of the intended recipient(s) only. If you have 
received this e-mail in error, please notify the sender immediately and then 
delete it. If you are not the intended recipient, you must not use, disclose or 
distribute this e-mail without the author's prior permission. We have taken 
precautions to minimize the risk of transmitting software viruses, but we 
advise you to carry out your own virus checks on any attachment to this 
message. We cannot accept liability for any loss or damage caused by software 
viruses. Any views and/or opinions expressed in this e-mail are of the author 
only and do not represent the views of Epicor Software Corporation or any other 
company within its group.
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

RE: [U2] Record Size Limit in UV

[Dan Fitzgerald]

Add to this the fact that, given the usual page size of 4k, you're going to
do 25 physical disk reads to retrieve each item. I'd consider a redesign.

One place I tuned was doing 11 transactions per minute ("you mean, per
second?, I asked). It turns out that they were opening batches & leaving the
open for a month; the mv field holding the transaction remote key was
growing to 1-2Mb, and they weren't using raise() & lower() to maintain the
pointer, so every transaction had to start reading at the first byte... 

Anyway, they put a forced limit of 100 transactions per batch into the code,
and then with some resizing went to 1400 tpm. Further resizing & hardware
upgrades & app tuning got 'em above 3k tpm.

-Original Message-
From: owner-u2-us...@listserver.u2ug.org
[mailto:owner-u2-us...@listserver.u2ug.org] On Behalf Of Charles Stevenson
Sent: Thursday, April 23, 2009 1:12 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] Record Size Limit in UV

I've seen records at least an order of magnitude larger than that.  Unix 
& Windows. Megabytes.  I want to say much larger than that, but I can't 
verify that and when numbers get that big, my brain can't get around 
them.  ( e.g.,  $1,000,000,000,000: www.pagetutor.com/trillion/index.html  )

I don't know of absolute limits,  but performance takes a dive, of 
course: lock contention, frequent updates, inappropriate indexes, 
selections w/o indexing poorly sized files (they should be type 30, if 
they must exist.  The large records get isolated in their own 
overflow).  In general, U2 responds well to size abuse. Performance 
degradation is fairly linear & responds well to throwing more hardware 
at it.  But at some point degradation curve becomes geometric.

I'm curious about whether your scenario matches the usual: i.e., an 
application that has outgrown its original design, with a multi-valued 
association in that gets updated frequently through the lfe of the record.
The original design intended a few mv entires, until the business 
operation it represents gets closed.   At some point someone decided to 
use the association for some additional purpose.   Some records remain 
small, some grow, resulting in a very lumpy file.  Many times it's the 
users trying to respond quickly to a changing business need,  w/o 
waiting for IT to do the enhancement (they invent new status codes, 
etc.).  Sometimes it's programmers enamored by the siren call of mv nesting.

Adam Taylor wrote:
> Hey all,
>
> Can anyone tell me what (if any) the current record size limit is in UV
10.2?
> We've currently found ourselves in a situation where certain fields in a
data
> file will contain 100K+ characters and we would like to be proactive about
any
> potential problems if they exist.
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

RE: [U2] UV 10.1.22; Dynamic File question {Unclassified}

[HENDERSON MIKE, MR]

RESIZE  DYNAMIC GROUP.SIZE 2 MINIMUM.MODULUS  

I think the manual says this doesn't work but it does, on Windows
anyway.
Usual caveats about resizing files apply!
YMMV

Regards


Mike

 

> -Original Message-
> From: owner-u2-us...@listserver.u2ug.org 
> [mailto:owner-u2-us...@listserver.u2ug.org] On Behalf Of Allen Egerton
> Sent: Friday, 24 April 2009 07:39
> To: u2-users@listserver.u2ug.org
> Subject: Re: [U2] UV 10.1.22; Dynamic File question
> 
[snip]
> The situation is that I'm potentially adding a million 
> records to a dynamic file.  That's not hypothetical, it's a 
> real life scenario.  And I don't want to wait while the file 
> splits multiple times over the course of the add.  I don't 
> have that large a window.  I want to force the file to expand 
> the weekend before so that when I add the records the file 
> isn't rebuilding.
> 
> The file's already dynamic, so setting Minimum.Modulus 
> doesn't seem to do me much good unless I can figure out how 
> to force it to split to that minium.modulos.  And there's the 
> crux of my question.  How can I force it to split?
> 
[snip]
> 
> --
> Allen Egerton
> aegerton at pobox dot com
> PGP Key ID 0x8EA57261
> ---
> u2-users mailing list
> u2-users@listserver.u2ug.org
> To unsubscribe please visit http://listserver.u2ug.org/
> 
The information contained in this Internet Email message is intended
for the addressee only and may contain privileged information, but not
necessarily the official views or opinions of the New Zealand Defence Force.
If you are not the intended recipient you must not use, disclose, copy or 
distribute this message or the information in it.

If you have received this message in error, please Email or telephone
the sender immediately.
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

RE: [U2] Need assistance replicating UniObjects.NET issue

[Symeon Breen]

Glen wrote

> After speaking with our administrator he said:
>
>Technically the card and switches are set to "Auto-Duplex", which means
>they are running in Full Duplex mode.


That may be the case but auto negotiate between the PC NIC and the switch
port does not always work - you should be able to tell from the switch port
if it actually has negotiated to full duplex or not. - Probably a barking up
the wrong tree there anyway.


Yes I agree a better way of capturing and dealing with these problems is the
way to go - my experience is of uniobjects.net in a web environment where we
are getting over 1 million hits per day, peaking at say 20 or more per
second and we do get a multitude of errors most days in the logs. - We
should not really ignore them but i think blips do just happen, certainly i
have found that the udt licence management is always a bit behind so if I
have all licences used and i drop a uniobjects connection and connect a new
one straight away it gives me a out of licence error - i sometimes have to
wait a few seconds before connecting again, which in web terms is a long
time. Sometimes i also getto thepoint where udt does not accept any new
uniobjects connections and the only way to fix this is to kill the unirpcd
service and restart it. 
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

Re: [U2] U2 Application / Solutions providers list? "PICK / U2

[Scott Richardson]

A quick notes of THANKS to ALL who have responded, both on list and off.
Excellent suggestions and feedback from each.

I am still waiting to hear if I have made it through to the "second round".
If I get to the second round, or "in the door" on this IT Manager 
opportunity, I will surely
be back in touch with many who responded and digging deeper into the 
history of what
exactly the product was and uncovering the roots of the problems 
encountered.

Thank you again.

--
Regards,
Scott Richardson
**
Sr. Systems Engineer *  IT Consultant
Marlborough, MA 01752
CheetahFTL can be contacted at comcast.net
**



Mark Eastwood wrote:
> There's a list of U2 solution providers on IBM's web site:
> http://www-01.ibm.com/software/data/u2/solutions/find.html
>
>
> Mark
> 
> From: Scott Richardson 
> To: "u2-users@listserver.u2ug.org" 
> Sent: Tuesday, April 21, 2009 8:08:23 AM
> Subject: [U2] U2 Application / Solutions providers list?  "PICK / U2
> Hits"?   Any "ERP Light" applications out there?
>
> Hello folks,
>
> A good number of years ago, there used to be a published list / document
>
> / book  of various PICK / U2  "Solutions Providers" or Value Added 
> Resellers and ISV (aka Independent Software Vendors) listing the VAR or 
> ISV, their vertical markets served, and descriptions of the applications
>
> they provided, (I think it was called "PICK HITS" or something like
> that?).
>
> Is there anything similar produced/published these days? It would appear
>
> that such a document would be an excellent vehicle for the continued 
> proliferation of the "multi-valued" market. (IBM U2 Marketing - where 
> are you?)
> ---
> u2-users mailing list
> u2-users@listserver.u2ug.org
> To unsubscribe please visit http://listserver.u2ug.org/
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

RE: [U2] UV 10.1.22; Dynamic File question

[Rick Nuckolls]

There isn't that much that is secret about the algorithm:

Mod:next_group

1:1, 2:1, 3:1 4:2, 5:1 6:2 7:3 8:4, 9:1 ...16:8, 17:1 ... 32:16, 33:1

Theoretically, it could be a variation on that, but that is the basic idea of 
linear hashing -- Whenever the modulus reaches a new power of two, the next 
group to split will be the first group, then #2, 3, 4 etc.

Rick Nuckolls
Lynden Inc.



-Original Message-
From: owner-u2-us...@listserver.u2ug.org 
[mailto:owner-u2-us...@listserver.u2ug.org] On Behalf Of Scott Ballinger
Sent: Thursday, April 23, 2009 12:14 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] UV 10.1.22; Dynamic File question

My understanding is the when a dynamic file's *total* size exceeds the
SPLIT.LOAD threshold, a group (almost certainly not the group you are
writing to or just wrote to) will be split. The group(s) chosen to be split
are controlled by some secret algorithm, but I don't believe that the
largest groups are the ones first chosen to be split. This can be verified
by doing a GROUP.STAT on a dynamic file (with poorly hashing keys) and
noting that there are many empty groups as well as many overflowing groups.
Dynamic files perform well on the total file (*average* group size) level,
but still suffer from the same problems as static hashed files regarding
poorly hashing keys when you get down to individual groups within the file.
If this is not how the splitting process in dynamic files actually works, I
would appreciate being enlightened.

/Scott Ballinger
Pareto Corporation
Edmonds WA USA
206 713 6006

On Thu, Apr 23, 2009 at 11:47 AM, Martin Phillips <
martinphill...@ladybridge.com> wrote:

> Hi Allen,
>
>  If I know in advance that I'm adding a large number of records
>> to a file, is there any way to force the file to "grow" in advance?
>>
>
> You could do this with the MINIMUM.MODULUS parameter but the whole point of
> dynamic files is that they do their own tuning. Although a dynamic file is
> unlikely to be as good as a perfectly tuned static file, it will be a lot
> better than a typical static file that has been left alone for a long while.
>
> Another important point is that a split affects only one (or arguably two)
> groups unlike resizing a static file from modulo N to modulo N+1 which would
> affect every group.
>
> My real point here is that it is usually best to let a dynamic file look
> after itself.
>
>
> Martin Phillips
> Ladybridge Systems Ltd
> 17b Coldstream Lane, Hardingstone, Northampton, NN4 6DB
> +44-(0)1604-709200 ---
>
> u2-users mailing list
> u2-users@listserver.u2ug.org
> To unsubscribe please visit http://listserver.u2ug.org/
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

Re: [U2] UV 10.1.22; Dynamic File question

[Allen Egerton]

Martin Phillips wrote:
> Hi Allen,
> 
>> If I know in advance that I'm adding a large number of records
>> to a file, is there any way to force the file to "grow" in advance?
> 
> You could do this with the MINIMUM.MODULUS parameter but the whole point
> of dynamic files is that they do their own tuning. Although a dynamic
> file is unlikely to be as good as a perfectly tuned static file, it will
> be a lot better than a typical static file that has been left alone for
> a long while.
> 
> Another important point is that a split affects only one (or arguably
> two) groups unlike resizing a static file from modulo N to modulo N+1
> which would affect every group.
> 
> My real point here is that it is usually best to let a dynamic file look
> after itself.

In general I'd agree with you.

The situation is that I'm potentially adding a million records to a
dynamic file.  That's not hypothetical, it's a real life scenario.  And
I don't want to wait while the file splits multiple times over the
course of the add.  I don't have that large a window.  I want to force
the file to expand the weekend before so that when I add the records the
file isn't rebuilding.

The file's already dynamic, so setting Minimum.Modulus doesn't seem to
do me much good unless I can figure out how to force it to split to that
minium.modulos.  And there's the crux of my question.  How can I force
it to split?

The file's numerically keyed, (sequential), so it hashes pretty evenly
across the groups, consequently I'm not overly worried about empty
groups due to forcing the splits beforehand.

I know this isn't how you, (or I), typically work with dynamic files.
I've got some slightly unusual circumstances and client induced
constraints that I'm working under.

Thank you.

-- 
Allen Egerton
aegerton at pobox dot com
PGP Key ID 0x8EA57261
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

Re: [U2] UV 10.1.22; Dynamic File question

[Charles Stevenson]

While CREATE.FILE . . . MINIMUM.MODULUS does indeed reserver room in the 
primary "DATA30" file,  it does NOT reserve any room in "OVER30" where 
large records (& group overflow) lives.


But you can leverage that comment about purges:  At some non-primetime 
convenient to you, create a bunch of dummy records that are larger than 
the LARGE.RECORD parameter (default  1600+ bytes, iirc).  Then delete 
them. that will create a large OVER.30 with lots of reserved empty space.


On the other hand, if you do want to shrink OVER.30 after a purge, RESIZE.

If you have the luxury of having all large records the same size, 
consider setting LARGE.RECORD  to just below that number of bytes.


While we're at it, here's another trick for fine tuning a dynamic file's 
size on the fly:


1.Use CONFIGURE.FILE to set MInIMUM.MODULUS where you want it.
2,Temporarilly set MERGE.LOAD & SPLIT.LOAD only 1% apart.
3.Since the file will now be outside acceptable parameters, every 
write/delete will cause either split/merge, depending on which way 
you're going.  So do a loop that repeatedly writes & deletes  the same 
dummy record until you get from current mod to desired mod.

4.Put MERGE.LOAD / SPLIT.LOAD back to where it belongs.



You can set a min modulo on a dynamic file when you build it (or rebuild it).  
In theory, this will reduce the splitting until you hit that min modulo.  This 
does of course assume your files hash fairly evenly, etc, etc.

However, if you purge the file, it will still take up all that space based on 
the min mod.


-Original Message-
...Dynamic files ...
If I know in advance that I'm adding a large number of records to a
file, is there any way to force the file to "grow" in advance?

---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

Re: [U2] UV 10.1.22; Dynamic File question

[Scott Ballinger]

My understanding is the when a dynamic file's *total* size exceeds the
SPLIT.LOAD threshold, a group (almost certainly not the group you are
writing to or just wrote to) will be split. The group(s) chosen to be split
are controlled by some secret algorithm, but I don't believe that the
largest groups are the ones first chosen to be split. This can be verified
by doing a GROUP.STAT on a dynamic file (with poorly hashing keys) and
noting that there are many empty groups as well as many overflowing groups.
Dynamic files perform well on the total file (*average* group size) level,
but still suffer from the same problems as static hashed files regarding
poorly hashing keys when you get down to individual groups within the file.
If this is not how the splitting process in dynamic files actually works, I
would appreciate being enlightened.

/Scott Ballinger
Pareto Corporation
Edmonds WA USA
206 713 6006

On Thu, Apr 23, 2009 at 11:47 AM, Martin Phillips <
martinphill...@ladybridge.com> wrote:

> Hi Allen,
>
>  If I know in advance that I'm adding a large number of records
>> to a file, is there any way to force the file to "grow" in advance?
>>
>
> You could do this with the MINIMUM.MODULUS parameter but the whole point of
> dynamic files is that they do their own tuning. Although a dynamic file is
> unlikely to be as good as a perfectly tuned static file, it will be a lot
> better than a typical static file that has been left alone for a long while.
>
> Another important point is that a split affects only one (or arguably two)
> groups unlike resizing a static file from modulo N to modulo N+1 which would
> affect every group.
>
> My real point here is that it is usually best to let a dynamic file look
> after itself.
>
>
> Martin Phillips
> Ladybridge Systems Ltd
> 17b Coldstream Lane, Hardingstone, Northampton, NN4 6DB
> +44-(0)1604-709200 ---
>
> u2-users mailing list
> u2-users@listserver.u2ug.org
> To unsubscribe please visit http://listserver.u2ug.org/
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

RE: [U2] Need assistance replicating UniObjects.NET issue

[Glenn Batson]

After speaking with our administrator he said:

Technically the card and switches are set to "Auto-Duplex", which means
they are running in Full Duplex mode.

One weird things is these transport errors only occur between 1 AM and 6
AM.  We receive multiple in that time span but not all at the same time
and the same client does not have more than one.  For example:

2:01 AM Client A has the issue
2:32 AM Client C has the issue
2:46 AM Client B has the issue
2:59 AM Client D has the issue

I should mention that our backups complete before 2AM.  The night before
they were staggered like this but it all started at 3:00 AM.  Here is
the interesting thing.  We have 5 minute trapping of the listuser on our
UniData server.  We trap how many connections each client is using
(count of udcs by authentication credentials).  Every time this
transport occurs it occurs when a new udcs connection is being spawned.
For example client A may have not connections because the worker process
on IIS has shutdown for inactivity.  When activity occurs and the worker
process spawns the min connection I believe that connection is throwing
the transport error.  It also occurs when a client has had a min
connection of one and then it spawns a new connection and that new
connection has the transport error.  I believe these new connections are
the ones getting the transport error and I don't see the count go up
because a new one is spawned as a result of the transport error
happening.  This is because the logs on the server are only every 5
minutes.

Weird.  We are bumping up network logging but again we are pretty much
done with trying to figure out why and just writing better code to
handle the error when it occurs.

-Original Message-
From: owner-u2-us...@listserver.u2ug.org
[mailto:owner-u2-us...@listserver.u2ug.org] On Behalf Of Symeon Breen
Sent: Wednesday, April 22, 2009 1:19 AM
To: u2-users@listserver.u2ug.org
Subject: RE: [U2] Need assistance replicating UniObjects.NET issue

I presume this is an intermittent error you are getting ?

Are you using the latest uodotnet.dll that goes with udt 7.1 ?

I see you are on windows, i am not sure if it has the same problems - i
know
on *nix you have to get the LANG env variable correct.

This may happen i suppose if your backend process crashes out, maybe
becuae
of a corrupt file or the machine is having some sort of problem - it may
be
worth seeing what else is happening when this happens. If you are
capturing
the exception in your .net app you can add some further info regarding
what
processes are running, time date etc into your error log.

Is your network card set to full duplex and the switch port it is
connected
to as well, i have had problems when comms have been in half duplex, and
the
connection is busy.



-Original Message-
From: owner-u2-us...@listserver.u2ug.org
[mailto:owner-u2-us...@listserver.u2ug.org] On Behalf Of Glenn Batson
Sent: 22 April 2009 02:17
To: u2-users@listserver.u2ug.org
Subject: [U2] Need assistance replicating UniObjects.NET issue

Does anyone have any insight into how to create the following error.

Error Message: IBMU2.UODOTNET.UniXMLException: Unable to read data from
th
 transport connection: An existing connection was forcibly closed by the
re
 host.[IBM U2][UODOTNET - UNIRPC][ErrorCode=81009] The RPC failed[IBM

 U2][UODOTNET - UNICLIENT][ErrorCode=91]  Please verify UniVerse or
UniData
 Version. This feature may not be supported in older UniVerse or UniData

 version.

 UNIDATA 7.1.0

 Windows 2003


I've tried killing the udapislave.exe process associated with the
connection and then accessing the connection.  I get a similar error but
not this one.  I've tried to sleep to long but noticed I we don't set
the timeout property on the UniObject and the default is no timeout.  So
it just sits there waiting.  I haven't tried unplugging the network
cable yet or mimicking some network interruption.

Any ideas would be greatly appreciated.

Thanks,

Glenn
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

RE: [U2] UV 10.1.22; Dynamic File question

[Henry Unger]

You can create the dynamic file with the desired size.

>From HELP CREATE.FILE:

MINIMUM.MODULUS n
This value is also the initial value of the modulo (sic) of the dynamic
file.

Best regards,

Henry

Henry P. Unger
Hitech Systems, Inc.
http://www.hitech.com

-Original Message-
From: owner-u2-us...@listserver.u2ug.org
[mailto:owner-u2-us...@listserver.u2ug.org] On Behalf Of Allen Egerton
Sent: Thursday, April 23, 2009 11:21 AM
To: .U2 List
Subject: [U2] UV 10.1.22; Dynamic File question

As I understand it, Dynamic files split single groups as you add records
to the specific group.  So, if you add a large number of records to a
file that's properly sized, it will (potentially) split multiple times
as you go along.

If I know in advance that I'm adding a large number of records to a
file, is there any way to force the file to "grow" in advance?

If it were a static file, I'd simply resize it to a new guesstimated
modulo.  Since it's dynamic, I don't see any way to do this other than
to convert it to back to a static file, then resize it to a dynamic file
with a minium modulus.  It seems as though there must be a better way...

Thanks.

-- 
Allen Egerton
aegerton at pobox dot com
PGP Key ID 0x8EA57261
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

Re: [U2] UV 10.1.22; Dynamic File question

[Martin Phillips]

Hi Allen,


If I know in advance that I'm adding a large number of records
to a file, is there any way to force the file to "grow" in advance?


You could do this with the MINIMUM.MODULUS parameter but the whole point of 
dynamic files is that they do their own tuning. Although a dynamic file is 
unlikely to be as good as a perfectly tuned static file, it will be a lot 
better than a typical static file that has been left alone for a long while.


Another important point is that a split affects only one (or arguably two) 
groups unlike resizing a static file from modulo N to modulo N+1 which would 
affect every group.


My real point here is that it is usually best to let a dynamic file look 
after itself.



Martin Phillips
Ladybridge Systems Ltd
17b Coldstream Lane, Hardingstone, Northampton, NN4 6DB
+44-(0)1604-709200 
---

u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

RE: [U2] UV 10.1.22; Dynamic File question

[Israel, John R.]

You can set a min modulo on a dynamic file when you build it (or rebuild it).  
In theory, this will reduce the splitting until you hit that min modulo.  This 
does of course assume your files hash fairly evenly, etc, etc.

However, if you purge the file, it will still take up all that space based on 
the min mod.

John Israel
Sr. Programmer/Analyst
Dayton Superior Corporation
721 Richard St.
Dayton, OH  45342
937-866-0711 x44380

-Original Message-
From: owner-u2-us...@listserver.u2ug.org 
[mailto:owner-u2-us...@listserver.u2ug.org] On Behalf Of Allen Egerton
Sent: Thursday, April 23, 2009 2:21 PM
To: .U2 List
Subject: [U2] UV 10.1.22; Dynamic File question

As I understand it, Dynamic files split single groups as you add records
to the specific group.  So, if you add a large number of records to a
file that's properly sized, it will (potentially) split multiple times
as you go along.

If I know in advance that I'm adding a large number of records to a
file, is there any way to force the file to "grow" in advance?

If it were a static file, I'd simply resize it to a new guesstimated
modulo.  Since it's dynamic, I don't see any way to do this other than
to convert it to back to a static file, then resize it to a dynamic file
with a minium modulus.  It seems as though there must be a better way...

Thanks.

-- 
Allen Egerton
aegerton at pobox dot com
PGP Key ID 0x8EA57261
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

[U2] UV 10.1.22; Dynamic File question

[Allen Egerton]

As I understand it, Dynamic files split single groups as you add records
to the specific group.  So, if you add a large number of records to a
file that's properly sized, it will (potentially) split multiple times
as you go along.

If I know in advance that I'm adding a large number of records to a
file, is there any way to force the file to "grow" in advance?

If it were a static file, I'd simply resize it to a new guesstimated
modulo.  Since it's dynamic, I don't see any way to do this other than
to convert it to back to a static file, then resize it to a dynamic file
with a minium modulus.  It seems as though there must be a better way...

Thanks.

-- 
Allen Egerton
aegerton at pobox dot com
PGP Key ID 0x8EA57261
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

[U2] unsubscribe *

[Noah Hart]

CONFIDENTIALITY NOTICE: 
This message may contain confidential and/or privileged information. If you are 
not the addressee or authorized to receive this for the addressee, you must not 
use, copy, disclose, or take any action based on this message or any 
information herein. If you have received this message in error, please advise 
the sender immediately by reply e-mail and delete this message. Thank you for 
your cooperation.
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

Re: [U2] Record Size Limit in UV

[Charles Stevenson]

I've seen records at least an order of magnitude larger than that.  Unix 
& Windows. Megabytes.  I want to say much larger than that, but I can't 
verify that and when numbers get that big, my brain can't get around 
them.  ( e.g.,  $1,000,000,000,000: www.pagetutor.com/trillion/index.html  )


I don't know of absolute limits,  but performance takes a dive, of 
course: lock contention, frequent updates, inappropriate indexes, 
selections w/o indexing poorly sized files (they should be type 30, if 
they must exist.  The large records get isolated in their own 
overflow).  In general, U2 responds well to size abuse. Performance 
degradation is fairly linear & responds well to throwing more hardware 
at it.  But at some point degradation curve becomes geometric.


I'm curious about whether your scenario matches the usual: i.e., an 
application that has outgrown its original design, with a multi-valued 
association in that gets updated frequently through the lfe of the record.
The original design intended a few mv entires, until the business 
operation it represents gets closed.   At some point someone decided to 
use the association for some additional purpose.   Some records remain 
small, some grow, resulting in a very lumpy file.  Many times it's the 
users trying to respond quickly to a changing business need,  w/o 
waiting for IT to do the enhancement (they invent new status codes, 
etc.).  Sometimes it's programmers enamored by the siren call of mv nesting.


Adam Taylor wrote:

Hey all,

Can anyone tell me what (if any) the current record size limit is in UV 10.2?
We've currently found ourselves in a situation where certain fields in a data
file will contain 100K+ characters and we would like to be proactive about any
potential problems if they exist.

---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

Re: [U2] Employment disaster

[Mecki Foerthmann]

That may be right, but smart companies don't hire someone, who got stuck 
somewhere and only knows that little niche, even if he knows it inside out.
Smart employers hire people who bring new ideas, new skills and show the 
ability to adapt to a new environment quickly.
Smart companies train their employees and let them constantly learn new 
skills (and pay them decent salaries as well).
Dinosaurs became extinct, because they couldn't adapt to a rapidly 
changing world.
That's why you call some people 'dinosaurs', not because they are over 
55 like myself.


If you work for an end user company for more than 5 years, you most 
likely already lost touch with what is going on in rest of the world.
After 20 years with the same outfit (probably also still running the 
same software as 20 years ago) you become unemployable.

And in today's climate, no job is save.


jpb-u2ug wrote:

I wouldn't say that too loudly there's a lot of potential employers on this
list. Most smart companies like to have someone that is willing to stick
around long enough to learn their business logic.

Jerry Banker

-Original Message-
From: owner-u2-us...@listserver.u2ug.org
[mailto:owner-u2-us...@listserver.u2ug.org] On Behalf Of Mecki Foerthmann
Sent: Wednesday, April 22, 2009 5:54 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] UV to SAP migration disaster

jpb-u2ug wrote:
  


Oh and one of these days you are going to be sitting in the same seat
talking about the good old days too, and someone is going to be telling


you
  

that you're a dinosaur. It's not easy trying to keep up with the


technology
  

and if your company isn't using it you tend to gravitate toward what they


do
  

use.

Jerry Banker
  


Not me Jerry, I usually don't stay long enough for that to happen.
If there's nothing new to learn I move on.
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/

[U2] Employment disaster

[jpb-u2ug]

I wouldn't say that too loudly there's a lot of potential employers on this
list. Most smart companies like to have someone that is willing to stick
around long enough to learn their business logic.

Jerry Banker

-Original Message-
From: owner-u2-us...@listserver.u2ug.org
[mailto:owner-u2-us...@listserver.u2ug.org] On Behalf Of Mecki Foerthmann
Sent: Wednesday, April 22, 2009 5:54 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] UV to SAP migration disaster

jpb-u2ug wrote:
> 
> Oh and one of these days you are going to be sitting in the same seat
> talking about the good old days too, and someone is going to be telling
you
> that you're a dinosaur. It's not easy trying to keep up with the
technology
> and if your company isn't using it you tend to gravitate toward what they
do
> use.
>
> Jerry Banker
>   
Not me Jerry, I usually don't stay long enough for that to happen.
If there's nothing new to learn I move on.
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/