amogh-jahagirdar commented on code in PR #6861:
URL: https://github.com/apache/iceberg/pull/6861#discussion_r1113713141
##########
api/src/main/java/org/apache/iceberg/view/ViewVersion.java:
##########
@@ -43,8 +47,7 @@ public interface ViewVersion {
long timestampMillis();
/**
- * Return the version summary such as the name of the operation that created
that version of the
- * view
+ * Return a string map of summary data for the operation that produced this
view version.
Review Comment:
In the data model itself, I think we should follow what Snapshot does where
summary returns all the fields other than the operation itself. We have a
separate operation method below for the operation. When we serialize the
version, the operation will get put in the summary of the version. Let me know
if this makes sense @nastra @jackye1995 @rdblue
##########
api/src/main/java/org/apache/iceberg/view/ViewVersion.java:
##########
@@ -58,4 +61,25 @@ public interface ViewVersion {
* @return the list of view representations
*/
List<ViewRepresentation> representations();
+
+ /**
+ * Return the operation which produced the view version
+ *
+ * @return Return the operation which produced the view version
+ */
+ Operation operation();
+
+ enum Operation {
+ CREATE,
+ REPLACE;
+
+ public static Operation fromName(String operation) {
+ Preconditions.checkArgument(operation != null, "Invalid operation:
null");
+ try {
+ return Operation.valueOf(operation.toUpperCase(Locale.ENGLISH));
+ } catch (IllegalArgumentException e) {
+ throw new IllegalArgumentException(String.format("Invalid operation:
%s", operation), e);
+ }
Review Comment:
@rdblue let me know if you have any concerns on doing a strict validation on
the operation type. This is in the spec, and I think it makes sense as a format
for views for Iceberg to rigorously enforce what are the acceptable operations
to produce a version.
Was digging through the history for why we don't validate the operations
when parsing a snapshot, and found this
https://github.com/apache/iceberg/pull/74/files#r257350134 . Let me know if you
have any concerns on forward compatibility with this approach?
##########
core/src/main/java/org/apache/iceberg/view/ViewVersionParser.java:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iceberg.view;
+
+import com.fasterxml.jackson.core.JsonGenerator;
+import com.fasterxml.jackson.databind.JsonNode;
+import java.io.IOException;
+import java.util.Locale;
+import java.util.Map;
+import org.apache.iceberg.relocated.com.google.common.base.Preconditions;
+import org.apache.iceberg.relocated.com.google.common.collect.ImmutableList;
+import org.apache.iceberg.relocated.com.google.common.collect.Maps;
+import org.apache.iceberg.util.JsonUtil;
+
+class ViewVersionParser {
+
+ private static final String VERSION_ID = "version-id";
+ private static final String TIMESTAMP_MS = "timestamp-ms";
+ private static final String SUMMARY = "summary";
+ private static final String OPERATION = "operation";
+
+ private static final String REPRESENTATIONS = "representations";
+
+ private ViewVersionParser() {}
+
+ static void toJson(ViewVersion version, JsonGenerator generator) throws
IOException {
+ Preconditions.checkArgument(version != null, "Invalid view version: null");
+ generator.writeStartObject();
+ generator.writeNumberField(VERSION_ID, version.versionId());
+ generator.writeNumberField(TIMESTAMP_MS, version.timestampMillis());
+ generator.writeObjectFieldStart(SUMMARY);
+ generator.writeStringField(OPERATION,
version.operation().name().toLowerCase(Locale.ENGLISH));
+ for (Map.Entry<String, String> summaryEntry :
version.summary().entrySet()) {
+ if (!summaryEntry.getKey().equals(OPERATION)) {
+ generator.writeStringField(summaryEntry.getKey(),
summaryEntry.getValue());
+ }
+ }
+
+ generator.writeEndObject();
+
+ generator.writeArrayFieldStart(REPRESENTATIONS);
+ for (ViewRepresentation representation : version.representations()) {
+ ViewRepresentationParser.toJson(representation, generator);
+ }
+ generator.writeEndArray();
+
+ generator.writeEndObject();
+ }
+
+ static String toJson(ViewVersion version) {
+ return JsonUtil.generate(gen -> toJson(version, gen), false);
+ }
+
+ static ViewVersion fromJson(String json) {
+ return JsonUtil.parse(json, ViewVersionParser::fromJson);
+ }
+
+ static ViewVersion fromJson(JsonNode node) {
+ Preconditions.checkArgument(node != null, "Cannot parse view version from
null object");
+
+ Preconditions.checkArgument(
+ node.isObject(), "Cannot parse view version from a non-object: %s",
node);
+
+ int versionId = JsonUtil.getInt(VERSION_ID, node);
+ long timestamp = JsonUtil.getLong(TIMESTAMP_MS, node);
+ Map<String, String> summary = Maps.newHashMap();
+ Map<String, String> serializedSummary = JsonUtil.getStringMap(SUMMARY,
node);
+ ViewVersion.Operation operation = null;
+
+ for (Map.Entry<String, String> summaryEntry :
serializedSummary.entrySet()) {
+ if (summaryEntry.getKey().equals(OPERATION)) {
+ operation = ViewVersion.Operation.fromName(summaryEntry.getValue());
+ } else {
+ summary.put(summaryEntry.getKey(), summaryEntry.getValue());
+ }
+ }
Review Comment:
When deserializing the operation we take it out of the summary map, that way
the summary map has everything else, and we set the operation on the version
itself. Same as `SnapshotParser` and operation
##########
core/src/main/java/org/apache/iceberg/view/ViewVersionParser.java:
##########
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iceberg.view;
+
+import com.fasterxml.jackson.core.JsonGenerator;
+import com.fasterxml.jackson.databind.JsonNode;
+import java.io.IOException;
+import java.util.Locale;
+import java.util.Map;
+import org.apache.iceberg.relocated.com.google.common.base.Preconditions;
+import org.apache.iceberg.relocated.com.google.common.collect.ImmutableList;
+import org.apache.iceberg.util.JsonUtil;
+
+class ViewVersionParser {
+
+ private static final String VERSION_ID = "version-id";
+ private static final String TIMESTAMP_MS = "timestamp-ms";
+ private static final String SUMMARY = "summary";
+ private static final String OPERATION = "operation";
+
+ private static final String REPRESENTATIONS = "representations";
+
+ private ViewVersionParser() {}
+
+ static void toJson(ViewVersion version, JsonGenerator generator) throws
IOException {
+ Preconditions.checkArgument(version != null, "Invalid view version: null");
+ generator.writeStartObject();
+ generator.writeNumberField(VERSION_ID, version.versionId());
+ generator.writeNumberField(TIMESTAMP_MS, version.timestampMillis());
+ JsonUtil.writeStringMap(SUMMARY, version.summary(), generator);
+
+ generator.writeArrayFieldStart(REPRESENTATIONS);
+ for (ViewRepresentation representation : version.representations()) {
+ ViewRepresentationParser.toJson(representation, generator);
+ }
+ generator.writeEndArray();
+
+ generator.writeEndObject();
+ }
+
+ static String toJson(ViewVersion version) {
+ return JsonUtil.generate(gen -> toJson(version, gen), false);
+ }
+
+ static ViewVersion fromJson(String json) {
+ return JsonUtil.parse(json, ViewVersionParser::fromJson);
+ }
+
+ static ViewVersion fromJson(JsonNode node) {
+ Preconditions.checkArgument(node != null, "Cannot parse view version from
null object");
+
+ Preconditions.checkArgument(
+ node.isObject(), "Cannot parse table version from a non-object: %s",
node);
Review Comment:
ooh good catch, I fixed it
##########
core/src/main/java/org/apache/iceberg/view/ViewVersionParser.java:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iceberg.view;
+
+import com.fasterxml.jackson.core.JsonGenerator;
+import com.fasterxml.jackson.databind.JsonNode;
+import java.io.IOException;
+import java.util.Locale;
+import java.util.Map;
+import org.apache.iceberg.relocated.com.google.common.base.Preconditions;
+import org.apache.iceberg.relocated.com.google.common.collect.ImmutableList;
+import org.apache.iceberg.relocated.com.google.common.collect.Maps;
+import org.apache.iceberg.util.JsonUtil;
+
+class ViewVersionParser {
+
+ private static final String VERSION_ID = "version-id";
+ private static final String TIMESTAMP_MS = "timestamp-ms";
+ private static final String SUMMARY = "summary";
+ private static final String OPERATION = "operation";
+
+ private static final String REPRESENTATIONS = "representations";
+
+ private ViewVersionParser() {}
+
+ static void toJson(ViewVersion version, JsonGenerator generator) throws
IOException {
+ Preconditions.checkArgument(version != null, "Invalid view version: null");
+ generator.writeStartObject();
+ generator.writeNumberField(VERSION_ID, version.versionId());
+ generator.writeNumberField(TIMESTAMP_MS, version.timestampMillis());
+ generator.writeObjectFieldStart(SUMMARY);
+ generator.writeStringField(OPERATION,
version.operation().name().toLowerCase(Locale.ENGLISH));
+ for (Map.Entry<String, String> summaryEntry :
version.summary().entrySet()) {
+ if (!summaryEntry.getKey().equals(OPERATION)) {
+ generator.writeStringField(summaryEntry.getKey(),
summaryEntry.getValue());
+ }
+ }
Review Comment:
We serialize the operation in the summary. We always use the operation on
the version itself as a source of truth, if something randomly sets operation
in the summary map it'll be ignored. Same as what happens in SnapshotParser
with operations
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
