[
https://issues.apache.org/jira/browse/PHOENIX-5494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16978294#comment-16978294
]
Hadoop QA commented on PHOENIX-5494:
------------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12986310/PHOENIX-5494_v6-4.x-HBase-1.4.patch
against 4.x-HBase-1.4 branch at commit
8f4cbf028585d3e017143e579d07369be384ff7d.
ATTACHMENT ID: 12986310
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 14 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the master's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ Collection<Pair<Mutation, byte[]>> updates =
delegate.getIndexUpdate(m, indexMetaData, cachedLocalTable);
+ Collection<Pair<Mutation, byte[]>> updates = delegate.getIndexUpdate(m,
indexMetaData, cachedLocalTable);
+ public Collection<Pair<Mutation, byte[]>> getIndexUpdate(Mutation mutation,
IndexMetaData context, LocalHBaseState localHBaseState) throws IOException;
+ this.addUpdateCells(this.table.getCurrentRowState(update,
toCover, ignoreNewerMutations), false);
+ public Collection<Pair<Mutation, byte[]>> getIndexUpdate(Mutation
mutation, IndexMetaData indexMetaData, LocalHBaseState localHBaseState) throws
IOException {
+ * because of previous {@link
IndexManagementUtil#flattenMutationsByTimestamp}(which is called
+ * in {@link IndexRegionObserver#groupMutations} or {@link
Indexer#preBatchMutateWithExceptions}),
+ public List<Cell> getCurrentRowState(Mutation m, Collection<? extends
ColumnReference> toCover, boolean ignoreNewerMutations)
+ List<Mutation> flattenedMutations =
Lists.newArrayListWithExpectedSize(mutations.size() * 10);
+ * Pre-scan all the required rows before we building the indexes for the
dataTableMutationsWithSameRowKeyAndTimestamp
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/3148//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/3148//artifact/patchprocess/patchReleaseAuditWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/3148//console
This message is automatically generated.
> Batched, mutable Index updates are unnecessarily run one-by-one
> ---------------------------------------------------------------
>
> Key: PHOENIX-5494
> URL: https://issues.apache.org/jira/browse/PHOENIX-5494
> Project: Phoenix
> Issue Type: Improvement
> Reporter: Lars Hofhansl
> Assignee: Kadir OZDEMIR
> Priority: Major
> Labels: performance
> Attachments: 5494-4.x-HBase-1.5.txt,
> PHOENIX-5494-4.x-HBase-1.4.patch, PHOENIX-5494.master.001.patch,
> PHOENIX-5494.master.002.patch, PHOENIX-5494.master.003.patch,
> PHOENIX-5494_v3-4.x-HBase-1.4.patch, PHOENIX-5494_v6-4.x-HBase-1.4.patch,
> Screenshot_20191110_160243.png, Screenshot_20191110_160351.png,
> Screenshot_20191110_161453.png
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> I just noticed that index updates on mutable tables retrieve their deletes
> (to invalidate the old index entry) one-by-one.
> For batches, this can be *the* major time spent during an index update. The
> cost is mostly incured by the repeated setup (and seeking) of the new region
> scanner (for each row).
> We can instead do a skip scan and get all updates in a single scan per region.
> (Logically that is simple, but it will require some refactoring)
> I won't be getting to this, but recording it here in case someone feels
> inclined.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
