[
https://issues.apache.org/jira/browse/TRAFODION-2952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16592625#comment-16592625
]
ASF GitHub Bot commented on TRAFODION-2952:
-------------------------------------------
GitHub user traflm opened a pull request:
https://github.com/apache/trafodion/pull/1704
[TRAFODION-2952] large amount of data will cause error in sequence ge…
…nerating
Currently, the sequence next range value is NOT protected by DTM, but a
retry logic.
If multiple processes try to use the same sequence, when the cache is used,
all of them try to get the next range and go into race. So one may get a range
that another get. The logic is any one get the next range will check the
updated timestamp, if it is itself, then it is good for him. Otherwise, the
range is got by someone else. So the code wait and retry.
For the use case described in JIRA TRAFODION-2952, there are 48 partitioned
table and want to do a load of 4 billion rows. It is high concurrency, and many
times conflict.
The that large load, it is very frequent to fail, since the retry time is
hardcoded to 10.
This patch introduce a CQD to control the retry number. It is set to a
higher number by default, so in most cases, users don't need to modify this.
But if still run into issue, one can change the CQD.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/traflm/trafodion TRAFODION-2952
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/trafodion/pull/1704.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1704
----
commit 8958ce6aa28a96a15ed067dc49f556e0945165a6
Author: Liu Ming <ovis_poly@...>
Date: 2018-08-25T15:20:32Z
[TRAFODION-2952] large amount of data will cause error in sequence
generating
----
> large amount of data will cause error in sequence generating
> ------------------------------------------------------------
>
> Key: TRAFODION-2952
> URL: https://issues.apache.org/jira/browse/TRAFODION-2952
> Project: Apache Trafodion
> Issue Type: Bug
> Reporter: liu ming
> Assignee: liu ming
> Priority: Major
>
> insert error
> ** ERROR[1583] Sequence metadata could not be updated.
> create table test1
> (id LARGEINT not null
> ) primary key(id)
> SALT USING 48 PARTITIONS
> ATTRIBUTES ALIGNED FORMAT
> HBASE_OPTIONS
> (
> DATA_BLOCK_ENCODING = 'FAST_DIFF',
> COMPRESSION = 'SNAPPY',
> MEMSTORE_FLUSH_SIZE = '1073741824'
> );
> create table test2
> (id LARGEINT not null
> ) primary key(id)
> SALT USING 48 PARTITIONS
> ATTRIBUTES ALIGNED FORMAT
> HBASE_OPTIONS
> (
> DATA_BLOCK_ENCODING = 'FAST_DIFF',
> COMPRESSION = 'SNAPPY',
> MEMSTORE_FLUSH_SIZE = '1073741824'
> );
> create sequence seq ;
> alter sequence seq cache 20000;
> upsert into test2 select seqnum(seq, next) from test1;
> test1 table has about 4 billion rows. When error raised , next value of seq
> is 5300001
> CREATE SEQUENCE TRAFODION.SEABASE.SEQ
> START WITH 1 /* NEXT AVAILABLE VALUE 5300001 */
> INCREMENT BY 1
> MAXVALUE 9223372036854775806
> MINVALUE 1
> CACHE 20000
> NO CYCLE
> LARGEINT
> ;
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
