This is an automated email from the ASF dual-hosted git repository.
MaxGekk pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 7f3cd8bc68ae [SPARK-57807][R][DOC] Document that SparkR does not
support nanosecond-precision timestamp types
7f3cd8bc68ae is described below
commit 7f3cd8bc68aef0ef3e791b677cbebabd81cd6cbf
Author: AgenticSpark <[email protected]>
AuthorDate: Thu Jul 2 11:49:45 2026 +0200
[SPARK-57807][R][DOC] Document that SparkR does not support
nanosecond-precision timestamp types
### What changes were proposed in this pull request?
Add a note to the SparkR programming guide (`docs/sparkr.md`), right after
the R↔Spark data type mapping table, stating that SparkR does not support the
nanosecond-precision timestamp types `TIMESTAMP_NTZ(p)` / `TIMESTAMP_LTZ(p)`
(precision `p` in `[7, 9]`).
This is a documentation-only sub-task of the nanosecond-timestamp umbrella
(SPARK-56822). Per the umbrella decision, SparkR support is not being added, so
the limitation is documented rather than implemented. SparkR's
`PRIMITIVE_TYPES` (`R/pkg/R/types.R`) maps only the microsecond `timestamp`
type to `POSIXct` and has no awareness of `TimestampNTZNanosType` /
`TimestampLTZNanosType`; `structField`/`structType` reject unknown types via
`checkType()` in `R/pkg/R/schema.R`, so such colum [...]
### Why are the changes needed?
Without this note, SparkR users have no indication that
nanosecond-precision timestamp types are unsupported, and would only find out
via a runtime "Unsupported type" error. It closes SPARK-57807.
### Does this PR introduce _any_ user-facing change?
No. Documentation-only update.
### How was this patch tested?
Docs-only change. Verified the claims against `R/pkg/R/types.R`
(`PRIMITIVE_TYPES`) and `R/pkg/R/schema.R` (`checkType` / `structField`), and
that the cross-linked `sql-ref-datatypes.html` page documents these types.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: GitHub Copilot CLI (Claude Opus 4.8)
Closes #56955 from AgenticSpark/agenticspark/SPARK-57807-sparkr-nanos-docs.
Authored-by: AgenticSpark <[email protected]>
Signed-off-by: Max Gekk <[email protected]>
(cherry picked from commit c3732771442ce187e6312785e801e80beafc13e2)
Signed-off-by: Max Gekk <[email protected]>
---
docs/sparkr.md | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/docs/sparkr.md b/docs/sparkr.md
index 45f506c9dc51..8e55ccc1ee79 100644
--- a/docs/sparkr.md
+++ b/docs/sparkr.md
@@ -658,6 +658,13 @@ The following example shows how to save/load a MLlib model
by SparkR.
</tr>
</table>
+Note that SparkR maps only the microsecond `timestamp` type to R's `POSIXct`.
The
+nanosecond-precision timestamp types `TIMESTAMP_NTZ(p)` and `TIMESTAMP_LTZ(p)`
(with
+precision `p` in `[7, 9]`, see [Data Types](sql-ref-datatypes.html)) are **not
+supported** in SparkR. Referencing one of these types when building a schema
with
+`structField`/`structType` raises an "Unsupported type" error, and columns of
these
+types cannot be collected into R.
+
# Structured Streaming
SparkR supports the Structured Streaming API. Structured Streaming is a
scalable and fault-tolerant stream processing engine built on the Spark SQL
engine. For more information see the R API on the [Structured Streaming
Programming Guide](./streaming/index.html).
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
