Re: getNextSeqId and multiple sequence on entity

Nicolas Malin Thu, 01 Apr 2010 05:48:45 -0700

Hello brett,

I forgot to warn you on this. We create an addon to manage delegatornextSeqId improvement.

http://addons.neogia.org/addons/delegator-getNextSeqId-ignore-entity/0.1.5/

If you want try it you can install by addon manager or if you prefer apatch, I can do it ;)


Nicolas

Brett Palmer wrote:

Nicolas,

Your proposal for a new method like, "SequenceUtil.getNextSequence(String
key, long staggerMax)" sounds like what I am proposing.  Would the
"staggerMax" parameter set the next range of sequence IDs?  For example:

Before calling "getNextSequence" method"

SeqName:  MySequence
SeqId:  1000

Now call your method:  SequenceUtil.getNextSequence("MySequence", 1500)

After the call the SequenceValueItem record has the following:

SeqName: MySequence
SeqId:  2501

We ran into some performance problems for some of our entities (OrderItem,
InventoryItem, etc) when the application servers were generating sequenceIds
for these entities and the default sequence-bank-size was only 10.  The
process of getting a new range of sequence IDs from the database is a
locking process and we had multiple servers trying to perform the same
function.  This was creating table locks during high loads.

In the entity definition you can specify the "sequence-bank-size" like so:

<entity entity-name=*"ProductSurveyResource" *package-name=*
"org.ofbiz.product.survey" *sequence-bank-size=*"1000"*>
but we couldn't find a way to specify the bank size for non-entity related
sequence IDs.  We also use the "sequence-id" simple method a lot but this
only works for entity related sequences and doesn't have a parameter to set
the "staggerMax".

My proposal was simply to add a new column to the SequenceValueItem table
called "sequenceBankSize" (or something like that).  Using my example above
the SequenceValueItem table might look like the following:

column names: SeqName, SeqId, SeqBankSize
record values:  MySequence,  1000,  1500

Then when the you used the SequenceUtil.getNextSequence("MySequence") or the
simple method "sequence-id" for the first time the sequencer would update
the SequenceValueItem table as follows:

MySequence, 2501, 1500

Subsequent "getNextSequence" calls would continue to return unique IDs
without hitting the database until the bank of 1500 IDs was used up and then
the above process would continue again.

For us the key points we would like to see for the sequencer are as follows:

1. Ability to use non-entity related sequences by just providing a sequence
name.

2. Ability to configure the sequence bank size so we can adjust the
frequency for database updates.

3. Any APIs for the sequence util should be available via simple
methods.  We could provide a patch for the simple method support for the new
utils if needed.

Thanks,



Brett


On Thu, Dec 24, 2009 at 8:52 AM, Nicolas Malin <
[email protected]> wrote:

After some reflection on this subject, I arrive to :
getNextSeqId is use normaly to get an single id that identify an
entity : Invoice.invoiceId, Order.orderId, Party.partyId etc ...
It's easier to remove the problematic instruction but bypasses
getNextSeqId purpose.

For multiple sequence on Entity I suggest to create new function on
delegator : getNextSequence (String entityName, String diff, long
staggerMax) that manage sequence as getNextSeqId but analyse just the
validity of entityName and generate sequenceValueId : entityName.diff.

As this, Delegator continue to work only on entity and not use for other
not in relation with entitymodel.

For sequence not in relation with entity, I can extend sequenceUtil to
have a function SequenceUtil.getNextSequence(String key, long
staggerMax).

Brett can you give me an example for your proposal ? I don't really
understand your improvement.

If you wait to read, Have a nice end year ;)
Nicolas


Le mercredi 23 décembre 2009 à 13:49 -0700, Brett Palmer a écrit :
 > I confirmed that if you use the delegator.getNextSeqId() you will get
an

exception every time it is used with a complaint that an entity doesn't
exist for the sequence you requested.  It does give still generate an ID

but

the exception is a little concerning when you are running in a production
environment.  This exception probably isn't intended but is a consequence

of

the call to look for the entity name.

I would prefer we just outputted a warning log and not threw an

exception.

Another request for the sequence generator is the ability to specify the

gap

between the next sequence ID update.  This can be configured in the

entity

engine but only for those IDs that have a corresponding entity.  If you

try

to use a generic sequencer with no attached entity the default is 10

which

can be low for a production environment with multiple servers.  Could we

add

a column to the SequenceValueItem to include the next increment value?


Brett

Here is the stack trace:


---- exception report
----------------------------------------------------------
Error getting entity definition from model
Exception: org.ofbiz.entity.GenericModelException
Message: Could not find definition for entity name DummySequence
---- stack trace
---------------------------------------------------------------
org.ofbiz.entity.GenericModelException: Could not find definition for

entity

name DummySequence
org.ofbiz.entity.model.ModelReader.getModelEntity(ModelReader.java:451)
org.ofbiz.entity.DelegatorImpl.getModelEntity(DelegatorImpl.java:1544)
org.ofbiz.entity.DelegatorImpl.getNextSeqIdLong(DelegatorImpl.java:1693)
org.ofbiz.entity.DelegatorImpl.getNextSeqId(DelegatorImpl.java:1655)
org.ofbiz.entity.DelegatorImpl.getNextSeqId(DelegatorImpl.java:1651)

com.automationgroups.pe.util.InventoryContainer.start(InventoryContainer.java:27)

org.ofbiz.base.container.ContainerLoader.start(ContainerLoader.java:101)
org.ofbiz.base.start.Start.startStartLoaders(Start.java:272)
org.ofbiz.base.start.Start.startServer(Start.java:322)
org.ofbiz.base.start.Start.start(Start.java:326)
org.ofbiz.base.start.Start.main(Start.java:409)

--------------------------------------------------------------------------------

2009-12-23 13:29:07,218 (main) [       SequenceUtil.java:236:WARN ]
[SequenceUtil.SequenceBank.fillBank] first select failed: will try to add
new row, result set was empty for sequence [DummySequence]
Used SQL: SELECT SEQ_ID FROM SEQUENCE_VALUE_ITEM WHERE
SEQ_NAME='DummySequence'
 Thread Name is: main:Thread[main,5,main]
2009-12-23 13:29:07,312 (main) [       SequenceUtil.java:341:INFO ] Got

bank

of sequenced IDs for [DummySequence]; curSeqId=10000, maxSeqId=10010,
bankSize=10
2009-12-23 13:29:07,312 (main) [ InventoryContainer.java:28 :INFO ] Here

is

the sample sequence Id: 10000
2009-12-23 13:29:07,343 (main) [  GenericDispatcher.java:61 :INFO ]

Creating

new dispatcher [RMIDispatcher] (main)



On Wed, Dec 23, 2009 at 12:54 PM, Scott Gray <[email protected]
wrote:

It looks to me as though an exception is being logged rather than

thrown,

is the id still being generated?

Regards
Scott

HotWax Media
http://www.hotwaxmedia.com


On 24/12/2009, at 3:31 AM, Nicolas Malin wrote:

 Hi all,

It's possible that this subject has been already discussed here but I
don't find any trace in the archives.

A long time ago, I used delegator.getNextSeqId to use multiple

sequences

in an entity like invoice (SALES, PURCHASE, ...).
Example : in France we use a sequence for all invoiceType and I call

the

delegator this way :
invoiceId = delegator.getNextSeqId("Invoice." + invoiceTypeId);
To have :
        for sales : VE10000, VE10001,  VE10002, ...
        for purchases : AC10000, AC10001,  AC10002, ...

Now when using this, GenericDelegator throws an exception : "Could not
find definition for entity name Invoice.PURCHASE_INVOICE"

I checked the code (GenericDelegotor.java:3049 on trunk) and found  :
        // might be null, but will usually match the entity name
         ModelEntity seqModelEntity = this.getModelEntity(seqName);

I suppose that there are some reasons to check the seqName as an
entityName but I don't understand why.

To manage many sequences for one entity, do you use directly the
sequenceUtil ?

Is it possible to add an escape case before checking entity (as add

test

if we find a point) ? Or extend getNextSeqId to manage prefix/suffix
parameters ?

Nicolas

--
Nicolas MALIN
Consultant
Tél : 06.17.66.40.06
Site projet : http://www.neogia.org/
-------
Société LibrenBerry
Tél : 02.48.02.56.12
Site : http://www.librenberry.net/

--
Nicolas MALIN
Consultant
Tél : 06.17.66.40.06
Site projet : http://www.neogia.org/
-------
Société LibrenBerry
Tél : 02.48.02.56.12
Site : http://www.librenberry.net/



--
Nicolas MALIN
Consultant
Tél : 06.17.66.40.06
Site projet : http://www.neogia.org/
-------
Société LibrenBerry
Tél : 02.48.02.56.12
Site : http://www.librenberry.net/

Re: getNextSeqId and multiple sequence on entity

Reply via email to