adarshsanjeev commented on code in PR #15953: URL: https://github.com/apache/druid/pull/15953#discussion_r1545961839
########## docs/multi-stage-query/reference.md: ########## @@ -99,6 +99,17 @@ For more information, see [Read external data with EXTERN](concepts.md#read-exte This variation of EXTERN requires one argument, the details of the destination as specified below. This variation additionally requires an `AS` clause to specify the format of the exported rows. +While exporting data, some metadata files will also be created at the destination in addition to the data. These files will be created in a directory `_symlink_format_manifest`. +- `_symlink_format_manifest/manifest`: Lists the files which were created as part of the export. The file is in the symlink manifest format, and consists of a list of absolute paths to the files created. Review Comment: The format itself does not seem to be well documented. It's not for Druid's use, other data stores have the capability to read the format, like [delta.compatibility.symlinkFormatManifest.enabled](https://docs.databricks.com/en/delta/table-properties.html) Athena, Trino etc support reading them. Given this, I think that we can skip documenting the format itself, but mention that it follows the symlink manifest format, wdyt? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
