kosiew commented on code in PR #115:
URL: https://github.com/apache/datafusion-spark/pull/115#discussion_r4627218322


##########
dev/update_function_docs.sh:
##########
@@ -0,0 +1,40 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: See the NOTICE file distributed with this work for 
additional information regarding copyright ownership
+# SPDX-FileContributor: Licensed to the Apache Software Foundation (ASF) under 
one or more contributor license agreements
+
+# Regenerates docs/functions.md from the #[user_doc] attributes on each 
function.
+#
+# Usage: dev/update_function_docs.sh
+
+set -euo pipefail
+
+ROOT_DIR="$(git rev-parse --show-toplevel)"
+cd "${ROOT_DIR}"
+
+TARGET_FILE="docs/functions.md"
+mkdir -p "$(dirname "${TARGET_FILE}")"
+
+{
+  cat <<'HEADER'
+<!--
+SPDX-License-Identifier: Apache-2.0
+SPDX-FileCopyrightText: See the NOTICE file distributed with this work for 
additional information regarding copyright ownership
+SPDX-FileContributor: Licensed to the Apache Software Foundation (ASF) under 
one or more contributor license agreements
+-->
+
+<!---
+This file was generated by the dev/update_function_docs.sh script.
+Do not edit it manually as changes will be overwritten.
+Instead, edit the #[user_doc] attributes on the functions and run the script.
+-->
+
+# datafusion-spark Function Reference
+
+Functions in this crate emulate Apache Spark's behavior. Each entry describes 
how the function
+differs from the DataFusion built-in of the same name, if there is one.
+HEADER
+  cargo run --quiet -p datafusion-spark-docgen --bin print_functions_docs
+} > "${TARGET_FILE}"

Review Comment:
   A failed `cargo run` leaves the reference truncated to its header. Could we 
generate into a temporary file, replace the target only on success, and clean 
up with a trap?



##########
crates/spark/README.md:
##########
@@ -24,7 +24,8 @@ errors, ...). This crate is useful when you are:
 - running SQL migrated from Spark on DataFusion.
 
 Not every Spark function is implemented yet. Progress is tracked in the
-[`datafusion-spark` epic]; see the [API documentation] for the functions 
available today.
+[`datafusion-spark` epic]. See the [function reference] for the functions 
available today, with

Review Comment:
   The reference excludes registered functions without `documentation()`. Could 
we say "documented functions" here and in `README.md:52` and 
`crates/spark/src/lib.rs:49` to avoid overstating coverage?



##########
.github/workflows/lint.yml:
##########
@@ -51,3 +51,18 @@ jobs:
         run: cargo check --no-default-features -p datafusion-spark
       - name: Check datafusion-spark (core)
         run: cargo check --no-default-features --features core -p 
datafusion-spark
+
+  function-docs:
+    name: function docs up to date
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - run: rustup toolchain install stable --profile minimal 
--no-self-update && rustup default stable
+      - uses: Swatinem/rust-cache@v2
+      - run: dev/update_function_docs.sh
+      - name: Check docs/functions.md is up to date
+        run: |
+          if ! git diff --exit-code -- docs/functions.md; then

Review Comment:
   Could we use `git status --porcelain -- docs/functions.md` instead? `git 
diff --exit-code` misses untracked files, so CI passes if the reference is 
deleted and regenerated.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to