wgtmac commented on code in PR #801:
URL: https://github.com/apache/iceberg-cpp/pull/801#discussion_r3675626905
##########
src/iceberg/inspect/metadata_table.h:
##########
@@ -23,14 +23,28 @@
/// \brief Define base APIs for metadata tables.
#include <memory>
+#include <optional>
+#include <string>
+#include "iceberg/arrow_c_data.h"
#include "iceberg/iceberg_export.h"
#include "iceberg/result.h"
#include "iceberg/table_identifier.h"
#include "iceberg/type_fwd.h"
+#include "iceberg/util/timepoint.h"
namespace iceberg {
+/// \brief Parameters for snapshot selection (time travel).
+struct SnapshotSelection {
+ /// \brief The snapshot ID to read.
+ std::optional<int64_t> snapshot_id;
+ /// \brief Read the snapshot that was current at this timestamp.
+ std::optional<TimePointMs> as_of_timestamp;
Review Comment:
Should it be `std::variant<std::monostate, int64_t, TimePointMs>`? Otherwise
we need to define the priority if both `snapshot_id` and `as_of_timestamp` are
set.
##########
src/iceberg/inspect/metadata_table.h:
##########
@@ -46,6 +60,27 @@ class ICEBERG_EXPORT MetadataTable {
virtual Kind kind() const noexcept = 0;
+ /// \brief Whether this metadata table supports time-travel queries.
+ ///
+ /// The currently supported snapshots and history metadata tables do not
+ /// support time travel.
+ bool supports_time_travel() const noexcept;
+
+ /// \brief Scan the metadata table using the current snapshot.
+ ///
+ /// Convenience overload — delegates to Scan(std::nullopt).
+ Result<ArrowArray> Scan() { return Scan(std::nullopt); }
+
+ /// \brief Scan the metadata table and return all rows as an Arrow struct
array.
+ ///
+ /// The returned ArrowArray is a struct array where each element is one row.
+ /// The caller takes ownership and must call ArrowArrayRelease when done.
+ ///
+ /// The default implementation returns NotSupported. Subclasses override this
+ /// to materialize their data.
+ virtual Result<ArrowArray> Scan(
+ const std::optional<SnapshotSelection>& snapshot_selection);
Review Comment:
It doesn't seem necessary to add a std::optional on top of SnapshotSelection
if all its values are optional.
##########
src/iceberg/inspect/metadata_table.h:
##########
@@ -23,14 +23,28 @@
/// \brief Define base APIs for metadata tables.
#include <memory>
+#include <optional>
+#include <string>
+#include "iceberg/arrow_c_data.h"
#include "iceberg/iceberg_export.h"
#include "iceberg/result.h"
#include "iceberg/table_identifier.h"
#include "iceberg/type_fwd.h"
+#include "iceberg/util/timepoint.h"
namespace iceberg {
+/// \brief Parameters for snapshot selection (time travel).
+struct SnapshotSelection {
+ /// \brief The snapshot ID to read.
+ std::optional<int64_t> snapshot_id;
+ /// \brief Read the snapshot that was current at this timestamp.
+ std::optional<TimePointMs> as_of_timestamp;
+ /// \brief Read the snapshot referenced by this named ref (branch or tag).
+ std::optional<std::string> ref_name;
Review Comment:
Is `std::string` enough? An empty one is unset.
##########
src/iceberg/inspect/metadata_table.h:
##########
@@ -46,6 +60,27 @@ class ICEBERG_EXPORT MetadataTable {
virtual Kind kind() const noexcept = 0;
+ /// \brief Whether this metadata table supports time-travel queries.
+ ///
+ /// The currently supported snapshots and history metadata tables do not
+ /// support time travel.
+ bool supports_time_travel() const noexcept;
+
+ /// \brief Scan the metadata table using the current snapshot.
+ ///
+ /// Convenience overload — delegates to Scan(std::nullopt).
+ Result<ArrowArray> Scan() { return Scan(std::nullopt); }
+
+ /// \brief Scan the metadata table and return all rows as an Arrow struct
array.
+ ///
+ /// The returned ArrowArray is a struct array where each element is one row.
+ /// The caller takes ownership and must call ArrowArrayRelease when done.
+ ///
+ /// The default implementation returns NotSupported. Subclasses override this
+ /// to materialize their data.
+ virtual Result<ArrowArray> Scan(
Review Comment:
Should we return `ArrowArrayStream` instead? My concern is that a single
ArrowArray may lead to OOM if the result is too large. `ArrowArrayStream` is an
iterator of `ArrowArray`s which is more scalable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
