[
https://issues.apache.org/jira/browse/PHOENIX-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Fernando updated PHOENIX-1954:
----------------------------------
Attachment: PHOENIX-1954-wip4.patch
Attaching patch that has all the changes I think we need.
Per my last comment there was indeed a race condition in the code I added. I
added several multi-threaded tests to verify this no longer an issue. The issue
was that we can't rely on the state value.currentValue in Sequence.java when
executing rs.next() to get the value to return from a NEXT <n> VALUES FOR <seq>
expression. We need to store the Bulk Allocation start values in Sequence.java
the same way we store the actual values. We store them by requested slots
allocated so that clients that requested <n> slots are guaranteed to get a
start value for range values that reserved that number of slots. See the tests
and code changes for more details.
I also made the tests run in both regular and multi-tenant mode to verify there
were not MT issues.
Tomorrow I'll do a bit more manual verification including some backwards
compatibility checks locally.
[~jamestaylor] [~tdsilva] [~elilevine]
> Reserve chunks of numbers for a sequence
> ----------------------------------------
>
> Key: PHOENIX-1954
> URL: https://issues.apache.org/jira/browse/PHOENIX-1954
> Project: Phoenix
> Issue Type: New Feature
> Reporter: Lars Hofhansl
> Assignee: Jan Fernando
> Attachments: PHOENIX-1954-wip.patch, PHOENIX-1954-wip2.patch.txt,
> PHOENIX-1954-wip3.patch, PHOENIX-1954-wip4.patch
>
>
> In order to be able to generate many ids in bulk (for example in map reduce
> jobs) we need a way to generate or reserve large sets of ids. We also need to
> mix ids reserved with incrementally generated ids from other clients.
> For this we need to atomically increment the sequence and return the value it
> had when the increment happened.
> If we're OK to throw the current cached set of values away we can do
> {{NEXT VALUE FOR <seq>(,<N>)}}, that needs to increment value and return the
> value it incremented from (i.e. it has to throw the current cache away, and
> return the next value it found at the server).
> Or we can invent a new syntax {{RESERVE VALUES FOR <seq>, <N>}} that does the
> same, but does not invalidate the cache.
> Note that in either case we won't retrieve the reserved set of values via
> {{NEXT VALUE FOR}} because we'd need to be idempotent in our case, all we
> need to guarantee is that after a call to {{RESERVE VALUES FOR <seq>, <N>}},
> which returns a value <M> is that the range [M, M+N) won't be used by any
> other user of the sequence. My might need reserve 1bn ids this way ahead of a
> map reduce run.
> Any better ideas?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
