rok commented on code in PR #51248: URL: https://github.com/apache/arrow/pull/51248#discussion_r4390852603
########## docs/source/format/CanonicalExtensions.rst: ########## @@ -546,6 +546,86 @@ Primitive Type Mappings | UUID extension type | UUID | +----------------------+------------------------+ +.. _parquet_file_extension: + +Parquet File +============ + +File represents a reference to a range of bytes stored inline or in an external +file. It is intended for use cases such as file inventories, manifests, and +references to unstructured data such as images or audio files in object +storage. + +Field semantics, byte resolution rules, and value validation are defined by the +`Parquet FILE logical type specification <https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#file>`__. +These requirements apply to this extension type. + +* Extension name: ``arrow.parquet.file``. + +* The storage type of this extension is a ``Struct`` containing at least one of + the following nullable fields: + + .. list-table:: + :header-rows: 1 + + * - Field + - Type + * - ``uri`` + - ``String``, ``LargeString``, or ``StringView`` + * - ``offset`` Review Comment: For FileType in Parquet `offset` will point to a region of a `uri` referenced file. However Parquet allows for the `uri` to be relative. How should Arrow users interpret that? How about we specify that `offset` with relative `uri` is - unlinke in Parquet - meaningless. We can stop there or also recommend that Parquet reader can attempt and record absolute `uri`, by checking for the file relative to the source parquet file's location. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
