[
https://issues.apache.org/jira/browse/PHOENIX-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382782#comment-16382782
]
James Taylor commented on PHOENIX-3860:
---------------------------------------
bq. Could you please refer me to this code? We also have problem with the
shadow cells in this case.
The server-side code that does local index population is in
UngroupedAggregateRegionObserver here:
{code}
} else if (buildLocalIndex) {
for (IndexMaintainer maintainer : indexMaintainers)
{
if (!results.isEmpty()) {
result.getKey(ptr);
ValueGetter valueGetter =
maintainer.createGetterFromKeyValues(
ImmutableBytesPtr.copyBytesIfNecessary(ptr),
results);
Put put =
maintainer.buildUpdateMutation(kvBuilder,
valueGetter, ptr,
results.get(0).getTimestamp(),
env.getRegion().getRegionInfo().getStartKey(),
env.getRegion().getRegionInfo().getEndKey());
indexMutations.add(put);
}
}
result.setKeyValues(results);
{code}
bq. Do you create an index using a transaction and commit when you finish
populate all the data table info to the index? What happened if the commit
fails? You drop the index table, take a new fence and start the process again?
No, we don't use a transaction during the initial index population. This
population of an index is done synchronously by default, but you can tack on
the ASYNC keyword to the CREATE INDEX call and then a MR job will
asynchronously build it. Either way, the index is only marked active (and hence
used) if the population is completed successfully. If the table is
transactional, then we do start and end a transaction solely for the purpose of
getting a transaction ID to use for the initial population. If the population
fails, then it's up to the user to issue a DROP INDEX and then call CREATE
INDEX again (though it'd be nice if we did this cleanup ourselves). One more
detail: as the index is being populated, we also perform the index maintenance
as writes are done to the table. These mutations may lay down delete markers
(our own tx delete marker) for rows that have not yet been populated. This all
works out because the mutations will have a later timestamp than the timestamp
of the rows created during the initial population.
For the issue with the garbage collector, can we determine in the coprocessor
that the index is in a "building" state and make sure that no garbage
collection is done? Or perhaps the commit table can be updated right before the
index is moved to an active state?
> Implement TAL functionality for Omid
> ------------------------------------
>
> Key: PHOENIX-3860
> URL: https://issues.apache.org/jira/browse/PHOENIX-3860
> Project: Phoenix
> Issue Type: Sub-task
> Reporter: Ohad Shacham
> Assignee: Ohad Shacham
> Priority: Major
>
> Implement TAL functionality for Omid in order to be able to use Omid as
> Phoenix's transaction processing engine.
> To support the integration jira [OMID-82] was opened that encapsulates all
> Omid related development for Phoenix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
