[
https://issues.apache.org/jira/browse/PHOENIX-5435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17246849#comment-17246849
]
ASF GitHub Bot commented on PHOENIX-5435:
-----------------------------------------
gokceni commented on a change in pull request #913:
URL: https://github.com/apache/phoenix/pull/913#discussion_r539564620
##########
File path:
phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ViewIndexIT.java
##########
@@ -72,6 +72,7 @@
import org.apache.phoenix.util.PropertiesUtil;
import org.apache.phoenix.util.QueryUtil;
import org.apache.phoenix.util.SchemaUtil;
+import org.apache.phoenix.util.TestDDLUtil;
Review comment:
ditto
##########
File path:
phoenix-core/src/it/java/org/apache/phoenix/end2end/ImmutableTableIT.java
##########
@@ -17,6 +17,7 @@
*/
package org.apache.phoenix.end2end;
+import org.apache.phoenix.util.TestDDLUtil;
Review comment:
nit: is the import necessary? Seems like the only change on this file is
this import
##########
File path:
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/GroupedAggregateRegionObserver.java
##########
@@ -400,7 +400,6 @@ protected boolean isRegionObserverFor(Scan scan) {
* Used for an aggregate query in which the key order does not necessarily
match the group by
* key order. In this case, we must collect all distinct groups within a
region into a map,
* aggregating as we go.
- * @param limit TODO
Review comment:
TODO seems to be dropped but not implemented?
##########
File path:
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/UngroupedAggregateRegionObserver.java
##########
@@ -73,6 +73,7 @@
import org.apache.hadoop.io.WritableUtils;
import org.apache.phoenix.coprocessor.generated.PTableProtos;
import org.apache.phoenix.exception.SQLExceptionCode;
+import org.apache.phoenix.execute.MutationState;
Review comment:
nit: lonely import
##########
File path:
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/UngroupedAggregateRegionScanner.java
##########
@@ -609,6 +613,9 @@ public boolean next(List<Cell> resultsToReturn) throws
IOException {
} while (hasMore &&
(EnvironmentEdgeManager.currentTimeMillis() - startTime) < pageSizeInMs);
if (!mutations.isEmpty()) {
+ if (isDelete || isUpsert) {
+ annotateDataMutations(mutations, scan);
+ }
Review comment:
It would be nice to refactor this perhaps to a new method? Lines 596 to
602 is same as 616 to 622
##########
File path:
phoenix-core/src/main/java/org/apache/phoenix/execute/MutationState.java
##########
@@ -682,6 +687,49 @@ private void generateMutations(final TableRef tableRef,
final long mutationTimes
values.putAll(modifiedValues);
}
+ private void annotateMutationsWithMetadata(PTable table, List<Mutation>
rowMutations) {
+ //only annotate if the change detection flag is on the table and HBase
supports
+ // preWALAppend coprocs server-side
+ if (table == null || !table.isChangeDetectionEnabled()
+ || !HbaseCompatCapabilities.hasPreWALAppend()) {
+ return;
+ }
+ //annotate each mutation with enough metadata so that anyone
interested can
+ // deterministically figure out exactly what Phoenix schema object
created the mutation
+ // Server-side we can annotate the HBase WAL with these.
+ for (Mutation mutation : rowMutations) {
+ annotateMutationWithMetadata(table, mutation);
+ }
+
+ }
+
+ public static void annotateMutationWithMetadata(PTable table, Mutation
mutation) {
Review comment:
Actually why not use the this method in the Ungroupped..Scanner in that
for loop that goes over mutations? Also mutation durability was not checked
there I think
##########
File path:
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/generated/ServerCachingProtos.java
##########
@@ -2192,6 +2192,21 @@ public Builder setColumnNameBytes(
*/
com.google.protobuf.ByteString
getParentTableTypeBytes();
+
+ // optional string logicalIndexName = 25;
+ /**
+ * <code>optional string logicalIndexName = 25;</code>
+ */
+ boolean hasLogicalIndexName();
+ /**
+ * <code>optional string logicalIndexName = 25;</code>
+ */
+ java.lang.String getLogicalIndexName();
Review comment:
Would this be populated for both view indexes and regular indexes and
local indexes?
##########
File path:
phoenix-core/src/main/java/org/apache/phoenix/execute/MutationState.java
##########
@@ -674,6 +678,7 @@ private void generateMutations(final TableRef tableRef,
final long mutationTimes
mutation.setAttribute(PhoenixIndexBuilder.ATOMIC_OP_ATTRIB, onDupKeyBytes);
}
}
+ annotateMutationsWithMetadata(table, rowMutations);
Review comment:
perhaps move this to line 684? (out of if else?)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Annotate HBase WALs with Phoenix Metadata
> -----------------------------------------
>
> Key: PHOENIX-5435
> URL: https://issues.apache.org/jira/browse/PHOENIX-5435
> Project: Phoenix
> Issue Type: Sub-task
> Reporter: Geoffrey Jacoby
> Assignee: Geoffrey Jacoby
> Priority: Major
> Attachments: PHOENIX-5435-4.x.patch
>
>
> HBase write-ahead-logs (WALs) drive not only failure recovery, but HBase
> replication and some HBase backup frameworks. The WALs contain HBase-level
> metadata such as table and region, but lack Phoenix-level metadata. That
> means that it's quite difficult to build correct logic that needs to know
> about Phoenix-level constructs such as multi-tenancy, views, or indexes.
> HBASE-22622 and HBASE-22623 add the capacity for coprocessors to annotate
> extra key/value pairs of metadata into the HBase WAL. We should have the
> option to annotate the tuple <tenant_id, table-or-view-name, timestamp>, or
> some hashed way to reconstruct that tuple into the WAL. It should have a
> feature toggle so operators who don't need it don't bear the slight extra
> storage cost.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
