WZhuo commented on code in PR #1144:
URL: https://github.com/apache/iceberg-cpp/pull/1144#discussion_r4639458515


##########
README.md:
##########
@@ -71,6 +71,19 @@ To run a specific test suite:
 ctest --test-dir build -R schema_test --output-on-failure
 ```
 
+`verification_test` runs the conformance fixtures from
+[apache/iceberg-verification](https://github.com/apache/iceberg-verification),
+pinned as a git submodule at `testing/iceberg-verification`. It is built only
+with `-DICEBERG_BUILD_VERIFICATION_TESTS=ON`, and CMake fails at configure time
+when the submodule is not checked out. To fetch it:
+
+```bash
+git submodule update --init testing/iceberg-verification
+```
+
+Pass `-DICEBERG_VERIFICATION_DIR=/path/to/iceberg-verification` to run against

Review Comment:
   Thanks, noted. Phase 1 only reads `cases.json`, so paths don't come into 
play yet. Since the reference tables hard-code `/tmp/iceberg-verification/...`, 
`ICEBERG_VERIFICATION_DIR` won't be enough for them; phase 3 will need the 
symlink to point at the same checkout, and will skip Windows until that route 
is verified.



##########
src/iceberg/test/verification/verification_util.cc:
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#include "iceberg/test/verification/verification_util.h"
+
+#include <algorithm>
+#include <cctype>
+#include <exception>
+#include <filesystem>
+#include <format>
+#include <fstream>
+#include <utility>
+
+#include "iceberg/test/test_config.h"
+
+namespace iceberg::verification {
+
+namespace {
+
+// Fixture fields read only by the loader and the outcome report.
+constexpr std::string_view kCases = "cases";
+constexpr std::string_view kNormativeLevel = "normative_level";
+
+std::filesystem::path TableSpecDir() {
+  return std::filesystem::path(ICEBERG_VERIFICATION_DIR) / "table-spec";
+}
+
+std::string JoinDiffs(const std::vector<std::string>& diffs) {
+  std::string out;
+  for (const auto& diff : diffs) {
+    out += "\n  ";
+    out += diff;
+  }
+  return out;
+}
+
+}  // namespace
+
+bool FixturesAvailable() { return 
std::filesystem::is_directory(TableSpecDir()); }
+
+std::vector<VerificationCase> LoadCases(std::string_view surface,
+                                        std::span<const std::string_view> 
subdirs) {
+  std::vector<VerificationCase> cases;
+  if (!FixturesAvailable()) {

Review Comment:
   I'd prefer to keep the guard clauses here. Each check bails out for one 
reason, and the happy path stays at one indentation level instead of nesting 
several deep. I did move opening, parsing and checking the file into a 
`ParseFixture` helper in c9c7a843, so `LoadCases` now has a single early exit 
per subdirectory.



##########
src/iceberg/test/verification/verification_util.cc:
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#include "iceberg/test/verification/verification_util.h"
+
+#include <algorithm>
+#include <cctype>
+#include <exception>
+#include <filesystem>
+#include <format>
+#include <fstream>
+#include <utility>
+
+#include "iceberg/test/test_config.h"
+
+namespace iceberg::verification {
+
+namespace {
+
+// Fixture fields read only by the loader and the outcome report.
+constexpr std::string_view kCases = "cases";
+constexpr std::string_view kNormativeLevel = "normative_level";
+
+std::filesystem::path TableSpecDir() {
+  return std::filesystem::path(ICEBERG_VERIFICATION_DIR) / "table-spec";
+}
+
+std::string JoinDiffs(const std::vector<std::string>& diffs) {
+  std::string out;
+  for (const auto& diff : diffs) {
+    out += "\n  ";
+    out += diff;
+  }
+  return out;
+}
+
+}  // namespace
+
+bool FixturesAvailable() { return 
std::filesystem::is_directory(TableSpecDir()); }
+
+std::vector<VerificationCase> LoadCases(std::string_view surface,
+                                        std::span<const std::string_view> 
subdirs) {
+  std::vector<VerificationCase> cases;
+  if (!FixturesAvailable()) {
+    return cases;
+  }
+  for (auto subdir : subdirs) {
+    auto path = TableSpecDir() / surface / subdir / "cases.json";
+    auto load_error = [&](std::string message) {
+      cases.push_back({.subdir = std::string(subdir),
+                       .id = "load",
+                       .load_error = std::format("{}: {}", path.string(), 
message)});
+    };
+    std::ifstream file(path);
+    if (!file) {
+      load_error("cannot open file");
+      continue;
+    }
+    nlohmann::json fixture;
+    try {
+      fixture = nlohmann::json::parse(file);
+    } catch (const std::exception& e) {
+      load_error(e.what());
+      continue;
+    }
+    if (!fixture.contains(kCases) || !fixture[kCases].is_array()) {

Review Comment:
   An empty `cases` array is a subdirectory with no cases yet, so it loads as 
no cases and passes. Only a missing or non-array `cases` is a load error.



##########
src/iceberg/test/verification/verification_util.cc:
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#include "iceberg/test/verification/verification_util.h"
+
+#include <algorithm>
+#include <cctype>
+#include <exception>
+#include <filesystem>
+#include <format>
+#include <fstream>
+#include <utility>
+
+#include "iceberg/test/test_config.h"
+
+namespace iceberg::verification {
+
+namespace {
+
+// Fixture fields read only by the loader and the outcome report.
+constexpr std::string_view kCases = "cases";
+constexpr std::string_view kNormativeLevel = "normative_level";
+
+std::filesystem::path TableSpecDir() {
+  return std::filesystem::path(ICEBERG_VERIFICATION_DIR) / "table-spec";
+}
+
+std::string JoinDiffs(const std::vector<std::string>& diffs) {
+  std::string out;
+  for (const auto& diff : diffs) {
+    out += "\n  ";
+    out += diff;
+  }
+  return out;
+}
+
+}  // namespace
+
+bool FixturesAvailable() { return 
std::filesystem::is_directory(TableSpecDir()); }
+
+std::vector<VerificationCase> LoadCases(std::string_view surface,
+                                        std::span<const std::string_view> 
subdirs) {
+  std::vector<VerificationCase> cases;
+  if (!FixturesAvailable()) {
+    return cases;
+  }
+  for (auto subdir : subdirs) {
+    auto path = TableSpecDir() / surface / subdir / "cases.json";
+    auto load_error = [&](std::string message) {
+      cases.push_back({.subdir = std::string(subdir),
+                       .id = "load",
+                       .load_error = std::format("{}: {}", path.string(), 
message)});
+    };
+    std::ifstream file(path);
+    if (!file) {
+      load_error("cannot open file");
+      continue;
+    }
+    nlohmann::json fixture;
+    try {
+      fixture = nlohmann::json::parse(file);
+    } catch (const std::exception& e) {
+      load_error(e.what());
+      continue;
+    }
+    if (!fixture.contains(kCases) || !fixture[kCases].is_array()) {
+      load_error("no `cases` array");
+      continue;
+    }
+    for (auto& json : fixture[kCases]) {

Review Comment:
   Good catch, fixed in c9c7a843. An entry that isn't an object with a string 
`id` is now reported as a load error labeled `<subdir>/cases-<index>` with the 
fixture path, instead of throwing during test registration.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to