rdblue commented on code in PR #16936: URL: https://github.com/apache/iceberg/pull/16936#discussion_r3546162596
########## core/src/main/java/org/apache/iceberg/ContentEntryAdapters.java: ########## @@ -0,0 +1,400 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.iceberg; + +import java.nio.ByteBuffer; +import java.util.Collections; +import java.util.Locale; +import java.util.Map; +import java.util.WeakHashMap; +import org.apache.iceberg.relocated.com.google.common.base.Preconditions; +import org.apache.iceberg.relocated.com.google.common.collect.Maps; +import org.apache.iceberg.types.Type; +import org.apache.iceberg.types.Types; +import org.apache.iceberg.util.ContentFileUtil; + +/** + * Builds {@link TrackedFile} instances for v4 content_entry rows from legacy {@link ManifestEntry} + * and {@link ManifestFile} inputs. + */ +class ContentEntryAdapters { + /** + * writer_format_version for content_entry rows produced by a v4 writer. Matches the table format + * version (4). + */ + static final int V4_WRITER_FORMAT_VERSION = 4; + + /** + * writer_format_version for content_entry rows that reference a leaf manifest written by a pre-v4 + * writer (v1, v2, or v3). Used at the root manifest level when a v4 root carries over legacy leaf + * manifests during a v3-to-v4 upgrade. Matches a pre-v4 table format version (0 is the sentinel; + * v1/v2/v3 leaves are not re-encoded as content_entry, so the root just tags the reference as + * legacy). + */ + static final int LEGACY_WRITER_FORMAT_VERSION = 0; + + // Cache of primitive-field types keyed by table schema. The originalTypes map a Metrics + // instance carries is read-only inside MetricsUtil.fromMetrics, so a single per-schema map + // can be shared across every adapter call for a writer's lifetime instead of being rebuilt + // per row. WeakHashMap keys release when callers stop referencing the schema. + private static final Map<Schema, Map<Integer, Type>> PRIMITIVE_TYPES_BY_SCHEMA = + Collections.synchronizedMap(new WeakHashMap<>()); + + private ContentEntryAdapters() {} + + static TrackedFile fromDataFile( Review Comment: I agree with keeping these separate for data and delete file cases. What I think we need to separate is `ManifestEntry` and `DataFile`. We accept `DataFile` instances through the API and create entires in operations like `AppendFiles`. The best use case is when accepting data files through the public API. There's a secondary use case for wrapping/adapting `ManifestEntry` for internal uses like rewriting v3 files to v4, but these are more narrow. For now, I think we need the `DataFile` to `TrackedFile` adapter, not `ManifestEntry`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
