This is an automated email from the ASF dual-hosted git repository.

HyukjinKwon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark-connect-rust.git


The following commit(s) were added to refs/heads/master by this push:
     new daae9e4  [SPARK-60915][CONNECT][PYTHON] Send the origins of DataFrame 
API calls, so that errors carry the DataFrame query contexts of PySpark
daae9e4 is described below

commit daae9e47bfb23e3a75294f4e7b857b60a97a5bb7
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Sat Nov 7 03:48:08 2026 +0900

    [SPARK-60915][CONNECT][PYTHON] Send the origins of DataFrame API calls, so 
that errors carry the DataFrame query contexts of PySpark
    
    ### What changes were proposed in this pull request?
    
    [SPARK-60915](https://issues.apache.org/jira/browse/SPARK-60915): the 
Python client now sends the origins of DataFrame API calls, as PySpark Connect 
does, so that errors carry the DataFrame query contexts of PySpark.
    
    As PySpark's `_with_origin` and `with_origin_to_class`, `col()` and the 
`Column` methods give the expressions that they build an origin. It has the 
name of the function, such as `col` or `__add__`, and its call site in user 
code. The client sends it as the `python_origin` of the expressions' common 
fields. So does a column name that a DataFrame method takes, as PySpark's 
`F._to_col` makes it with `col()`. 
`spark.python.sql.dataFrameDebugging.enabled`, which PySpark reads once, turns 
t [...]
    
    For an error in such an expression, the server reports a 
`DataFrameQueryContext` whose `fragment()` is the function and whose 
`callSite()` is the `file:line` of the user code.
    
    `repartitionById`, `sampleBy` and `unpivot` make their columns of column 
names with `col()`, as PySpark's `F._to_col` does. A column name of a UDF or 
UDTF call is a `ColumnReference` of it without an origin, as PySpark's 
`to_expr` makes it, rather than the column of `col()`.
    
    The client reads the configuration and the call site with 
`pyspark.errors.utils.is_debugging_enabled` and `_capture_call_site`, as 
PySpark's `_with_origin` does. The core expressions take the origin of the 
thread when they are made, as PySpark's `Expression.__init__` does.
    
    ### Why are the changes needed?
    
    Errors carried no DataFrame query context. In PySpark 4.3.0, 
`spark.range(1).select("no_such_col").collect()` and `select(F.col("nope") + 
1)` raise errors whose `getQueryContext()` is a `DataFrameQueryContext` of 
`col` at the call site. An ANSI `F.lit(1) / F.lit(0)` reports `__truediv__`. 
The client returned none.
    
    ### Does this PR introduce _any_ user-facing change?
    
    Yes. Errors carry the DataFrame query contexts of PySpark 4.3.0.
    
    ### How was this patch tested?
    
    - A new live test checks the `DataFrameQueryContext` of each of these 
errors, with the call site in the test file, of `repartitionById`, `sampleBy` 
and `unpivot` by a name, and none for a UDF called with a name. It also passes 
with official PySpark 4.3.0, and fails without this change.
    - The offline tests read `spark.python.sql.dataFrameDebugging.enabled` 
once, without a session, before they run, so that a test whose session has no 
server does not wait to read it.
    - A new offline test checks, in new interpreters against a mock Connect 
server, the origins of the expressions of `F.col("a") + 1`, and none when 
DataFrame debugging is disabled. It fails without this change. The mock now 
serves Config `GetWithDefault` requests.
    - A new lib test checks the origins of core expressions.
    - The lib tests and every integration test, e2e included, against a local 
Spark 4.3.0 server (673 passed, and 129 of the transport crate), the build of 
all tests with `wasm-udf`, the offline Python suites (1788 passed, 8 skipped), 
and the live drop-in and Arrow suites (96 passed) and live cases (10 passed). 
Clippy reports no diagnostics in the changed lines.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Opus 5.5
    
    This pull request and its description were written by Isaac.
    
    Closes #225 from HyukjinKwon/SPARK-60915-origins.
    
    Authored-by: Hyukjin Kwon <[email protected]>
    Signed-off-by: Hyukjin Kwon <[email protected]>
---
 crates/pyspark-rs/src/column.rs        | 64 +++++++++++++++++++++++++++
 crates/pyspark-rs/src/dataframe.rs     |  6 +++
 crates/pyspark-rs/src/functions.rs     |  7 +++
 crates/pyspark-rs/src/group.rs         |  2 +
 crates/pyspark-rs/src/lib.rs           |  1 +
 crates/pyspark-rs/src/origin.rs        | 31 +++++++++++++
 crates/pyspark-rs/src/stat.rs          |  2 +
 crates/spark-connect/src/expression.rs | 41 +++++++++++++++---
 crates/spark-connect/src/lib.rs        |  1 +
 crates/spark-connect/src/origin.rs     | 79 ++++++++++++++++++++++++++++++++++
 python/pyspark/sql/udf.py              | 14 +++---
 python/pyspark/sql/udtf.py             | 10 ++---
 python/tests/test_dropin_e2e.py        | 45 +++++++++++++++++++
 python/tests/test_dropin_offline.py    | 68 +++++++++++++++++++++++++++++
 14 files changed, 352 insertions(+), 19 deletions(-)

diff --git a/crates/pyspark-rs/src/column.rs b/crates/pyspark-rs/src/column.rs
index e78334a..00bf4e2 100644
--- a/crates/pyspark-rs/src/column.rs
+++ b/crates/pyspark-rs/src/column.rs
@@ -74,6 +74,7 @@ impl PyColumn {
         alias: Vec<String>,
         kwargs: Option<&Bound<'_, PyDict>>,
     ) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("alias");
         let mut metadata = None;
         if let Some(kwargs) = kwargs {
             let rest = kwargs.copy()?;
@@ -127,12 +128,14 @@ impl PyColumn {
         alias: Vec<String>,
         kwargs: Option<&Bound<'_, PyDict>>,
     ) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("name");
         self.alias(py, alias, kwargs)
     }
 
     /// Apply a transformation function to this column. Mirrors 
`Column.transform(f)`,
     /// which is simply `f(self)`.
     fn transform<'py>(slf: Bound<'py, Self>, f: Bound<'py, PyAny>) -> 
PyResult<Bound<'py, PyAny>> {
+        let _origin = crate::origin::with_origin("transform");
         f.call1((slf,))
     }
 
@@ -140,12 +143,14 @@ impl PyColumn {
     /// joins). Mirrors `Column.outer()`, which returns the same expression — 
outer
     /// resolution is performed server-side via the plan id.
     fn outer(&self) -> PyColumn {
+        let _origin = crate::origin::with_origin("outer");
         PyColumn::new(self.column.clone())
     }
 
     /// Cast to a different type. Accepts a `DataType` or a DDL type string,
     /// matching `pyspark.sql.Column.cast`.
     fn cast(&self, dataType: &Bound<'_, PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("cast");
         // A DDL string keeps the unparsed type_str form (reference 
cast("int"));
         // any DataType object is converted to a typed cast.
         if let Ok(s) = dataType.extract::<String>() {
@@ -158,6 +163,7 @@ impl PyColumn {
 
     /// Alias for `cast` (pyspark `Column.astype`).
     fn astype(&self, dataType: &Bound<'_, PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("astype");
         self.cast(dataType)
     }
 
@@ -165,6 +171,7 @@ impl PyColumn {
     /// Mirrors `pyspark.sql.Column.try_cast`.
     #[pyo3(name = "try_cast")]
     fn try_cast(&self, dataType: &Bound<'_, PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("try_cast");
         if let Ok(s) = dataType.extract::<String>() {
             Ok(PyColumn::new(self.column.clone().try_cast_str(&s)))
         } else {
@@ -176,6 +183,7 @@ impl PyColumn {
     /// Get an item from a list/map by key (pyspark `Column.getItem`).
     #[pyo3(name = "getItem")]
     fn get_item(&self, key: &Bound<'_, PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("getItem");
         let key_col = py_obj_to_column(key)?;
         Ok(PyColumn::new(self.column.clone().get_item(key_col)))
     }
@@ -183,17 +191,20 @@ impl PyColumn {
     /// Check if NULL.
     #[pyo3(name = "isNull")]
     fn is_null(&self) -> PyColumn {
+        let _origin = crate::origin::with_origin("isNull");
         PyColumn::new(self.column.clone().is_null())
     }
 
     /// Check if NOT NULL.
     #[pyo3(name = "isNotNull")]
     fn is_not_null(&self) -> PyColumn {
+        let _origin = crate::origin::with_origin("isNotNull");
         PyColumn::new(self.column.clone().is_not_null())
     }
 
     /// Substring extraction (start and length as columns).
     fn substr(&self, startPos: &Bound<'_, PyAny>, length: &Bound<'_, PyAny>) 
-> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("substr");
         // As PySpark: the arguments, as values of an Enum, must be of one 
type, a Column or
         // an int.
         let py = startPos.py();
@@ -239,65 +250,77 @@ impl PyColumn {
 
     /// Pattern matching with LIKE.
     fn like(&self, other: &str) -> PyColumn {
+        let _origin = crate::origin::with_origin("like");
         PyColumn::new(self.column.clone().like(other))
     }
 
     /// Pattern matching with RLIKE (regex).
     fn rlike(&self, other: &str) -> PyColumn {
+        let _origin = crate::origin::with_origin("rlike");
         PyColumn::new(self.column.clone().rlike(other))
     }
 
     /// Check if contains another column value.
     fn contains(&self, other: &Bound<'_, PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("contains");
         let other_col = py_obj_to_column(other)?;
         Ok(PyColumn::new(self.column.clone().contains(other_col)))
     }
 
     /// Check if starts with another column value.
     fn startswith(&self, other: &Bound<'_, PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("startswith");
         let other_col = py_obj_to_column(other)?;
         Ok(PyColumn::new(self.column.clone().startswith(other_col)))
     }
 
     /// Check if ends with another column value.
     fn endswith(&self, other: &Bound<'_, PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("endswith");
         let other_col = py_obj_to_column(other)?;
         Ok(PyColumn::new(self.column.clone().endswith(other_col)))
     }
 
     /// Ascending sort.
     fn asc(&self) -> PyColumn {
+        let _origin = crate::origin::with_origin("asc");
         PyColumn::new(self.column.clone().asc())
     }
 
     /// Ascending sort with nulls first.
     fn asc_nulls_first(&self) -> PyColumn {
+        let _origin = crate::origin::with_origin("asc_nulls_first");
         PyColumn::new(self.column.clone().asc_nulls_first())
     }
 
     /// Ascending sort with nulls last.
     fn asc_nulls_last(&self) -> PyColumn {
+        let _origin = crate::origin::with_origin("asc_nulls_last");
         PyColumn::new(self.column.clone().asc_nulls_last())
     }
 
     /// Descending sort.
     fn desc(&self) -> PyColumn {
+        let _origin = crate::origin::with_origin("desc");
         PyColumn::new(self.column.clone().desc())
     }
 
     /// Descending sort with nulls first.
     fn desc_nulls_first(&self) -> PyColumn {
+        let _origin = crate::origin::with_origin("desc_nulls_first");
         PyColumn::new(self.column.clone().desc_nulls_first())
     }
 
     /// Descending sort with nulls last.
     fn desc_nulls_last(&self) -> PyColumn {
+        let _origin = crate::origin::with_origin("desc_nulls_last");
         PyColumn::new(self.column.clone().desc_nulls_last())
     }
 
     /// A branch more of a CASE WHEN. Mirrors `Column.when`: the condition is 
a Column, and
     /// the column a `when()` without `otherwise()` (`INVALID_WHEN_USAGE`).
     fn when(&self, condition: &Bound<'_, PyAny>, value: &Bound<'_, PyAny>) -> 
PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("when");
         let py = condition.py();
         let Ok(condition) = condition.extract::<PyRef<'_, PyColumn>>() else {
             let arg_type = condition.get_type().name()?.to_string();
@@ -332,6 +355,7 @@ impl PyColumn {
     /// The value of a CASE WHEN when no branch holds. Mirrors 
`Column.otherwise`, which
     /// raises PySparkTypeError for a column that is not a `when()`, or one 
that has one.
     fn otherwise(&self, value: &Bound<'_, PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("otherwise");
         let message = match self.column.expression().node() {
             ExpressionNode::CaseWhen(case_when) if 
case_when.else_expr.is_none() => None,
             ExpressionNode::CaseWhen(_) => Some(
@@ -354,6 +378,7 @@ impl PyColumn {
     /// Get item by key/index, or a slice as `substr(start, stop)`. Mirrors
     /// `Column.__getitem__`.
     fn __getitem__(&self, key: &Bound<'_, PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__getitem__");
         if key.is_instance_of::<pyo3::types::PySlice>() {
             if !key.getattr("step")?.is_none() {
                 return Err(raise_pyspark(
@@ -373,6 +398,7 @@ impl PyColumn {
     /// `col["name"]`. Mirrors `Column.__getattr__`, which raises for `_jc` 
and for a name
     /// that starts with `__`.
     fn __getattr__(&self, py: Python<'_>, item: &str) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__getattr__");
         if item == "_jc" {
             return Err(raise_pyspark(
                 py,
@@ -394,36 +420,42 @@ impl PyColumn {
 
     /// Equality (==) - returns a Column condition
     fn __eq__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__eq__");
         let other_col = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().eq(other_col)))
     }
 
     /// Inequality (!=) - returns a Column condition
     fn __ne__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__ne__");
         let other_col = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().ne(other_col)))
     }
 
     /// Greater than (>) - returns a Column condition
     fn __gt__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__gt__");
         let other_col = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().gt(other_col)))
     }
 
     /// Less than (<) - returns a Column condition
     fn __lt__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__lt__");
         let other_col = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().lt(other_col)))
     }
 
     /// Greater than or equal (>=) - returns a Column condition
     fn __ge__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__ge__");
         let other_col = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().ge(other_col)))
     }
 
     /// Less than or equal (<=) - returns a Column condition
     fn __le__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__le__");
         let other_col = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().le(other_col)))
     }
@@ -431,48 +463,56 @@ impl PyColumn {
     // Arithmetic operators
     /// Addition (+)
     fn __add__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__add__");
         let other_col = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().add(other_col)))
     }
 
     /// Right addition (for reverse operations)
     fn __radd__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> 
{
+        let _origin = crate::origin::with_origin("__radd__");
         let other_col = to_column(&other.bind(py))?;
         Ok(PyColumn::new(other_col.add(self.column.clone())))
     }
 
     /// Subtraction (-)
     fn __sub__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__sub__");
         let other_col = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().sub(other_col)))
     }
 
     /// Right subtraction (for reverse operations)
     fn __rsub__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> 
{
+        let _origin = crate::origin::with_origin("__rsub__");
         let other_col = to_column(&other.bind(py))?;
         Ok(PyColumn::new(other_col.sub(self.column.clone())))
     }
 
     /// Multiplication (*)
     fn __mul__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__mul__");
         let other_col = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().mul(other_col)))
     }
 
     /// Right multiplication (for reverse operations)
     fn __rmul__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> 
{
+        let _origin = crate::origin::with_origin("__rmul__");
         let other_col = to_column(&other.bind(py))?;
         Ok(PyColumn::new(other_col.mul(self.column.clone())))
     }
 
     /// Division (/)
     fn __truediv__(&self, py: Python<'_>, other: Py<PyAny>) -> 
PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__truediv__");
         let other_col = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().div(other_col)))
     }
 
     /// Modulo (%)
     fn __mod__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__mod__");
         let other_col = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().modulo(other_col)))
     }
@@ -480,34 +520,40 @@ impl PyColumn {
     // Bitwise operators (logical in Spark context)
     /// Bitwise AND (&) - logical AND in Spark
     fn __and__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__and__");
         let other_col = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().and(other_col)))
     }
 
     /// Bitwise OR (|) - logical OR in Spark
     fn __or__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__or__");
         let other_col = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().or(other_col)))
     }
 
     /// Bitwise NOT (~) - logical NOT in Spark
     fn __invert__(&self) -> PyColumn {
+        let _origin = crate::origin::with_origin("__invert__");
         PyColumn::new(self.column.clone().not())
     }
 
     /// Negation (-)
     fn __neg__(&self) -> PyColumn {
+        let _origin = crate::origin::with_origin("__neg__");
         PyColumn::new(self.column.clone().neg())
     }
 
     /// Get field by name (for struct/nested types).
     #[pyo3(name = "getField")]
     fn get_field(&self, name: &str) -> PyColumn {
+        let _origin = crate::origin::with_origin("getField");
         PyColumn::new(self.column.clone().get_field(name))
     }
 
     /// Apply a window function over a window specification.
     fn over(&self, window: &crate::window::PyWindowSpec) -> PyColumn {
+        let _origin = crate::origin::with_origin("over");
         PyColumn::new(self.column.clone().over(window.spec.clone()))
     }
 
@@ -518,23 +564,28 @@ impl PyColumn {
     }
 
     fn __str__(&self) -> String {
+        let _origin = crate::origin::with_origin("__str__");
         self.__repr__()
     }
 
     // --- reverse arithmetic/logical dunders (operand order swapped) ---
     fn __rtruediv__(&self, py: Python<'_>, other: Py<PyAny>) -> 
PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__rtruediv__");
         let o = to_column(&other.bind(py))?;
         Ok(PyColumn::new(o.div(self.column.clone())))
     }
     fn __rmod__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> 
{
+        let _origin = crate::origin::with_origin("__rmod__");
         let o = to_column(&other.bind(py))?;
         Ok(PyColumn::new(o.modulo(self.column.clone())))
     }
     fn __rand__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> 
{
+        let _origin = crate::origin::with_origin("__rand__");
         let o = to_column(&other.bind(py))?;
         Ok(PyColumn::new(o.and(self.column.clone())))
     }
     fn __ror__(&self, py: Python<'_>, other: Py<PyAny>) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__ror__");
         let o = to_column(&other.bind(py))?;
         Ok(PyColumn::new(o.or(self.column.clone())))
     }
@@ -542,6 +593,7 @@ impl PyColumn {
     /// `col ** other`. Mirrors `Column.__pow__` = `power(self, other)`.
     /// (`modulo` is part of Python's ternary pow protocol; Spark has no 3-arg 
pow.)
     fn __pow__(&self, py: Python<'_>, other: Py<PyAny>, _modulo: Py<PyAny>) -> 
PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__pow__");
         let o = to_column(&other.bind(py))?;
         Ok(PyColumn::new(spark_connect::functions::pow(
             self.column.clone(),
@@ -551,6 +603,7 @@ impl PyColumn {
 
     /// `other ** col`. Mirrors `Column.__rpow__` = `power(other, self)`.
     fn __rpow__(&self, py: Python<'_>, other: Py<PyAny>, _modulo: Py<PyAny>) 
-> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("__rpow__");
         let o = to_column(&other.bind(py))?;
         Ok(PyColumn::new(spark_connect::functions::pow(
             o,
@@ -563,6 +616,7 @@ impl PyColumn {
 
     /// `x in col` is not supported. Mirrors `Column.__contains__`.
     fn __contains__(&self, py: Python<'_>, _item: Py<PyAny>) -> PyResult<()> {
+        let _origin = crate::origin::with_origin("__contains__");
         Err(raise_pyspark(
             py,
             "PySparkValueError",
@@ -598,39 +652,47 @@ impl PyColumn {
         lowerBound: Py<PyAny>,
         upperBound: Py<PyAny>,
     ) -> PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("between");
         let l = to_column(&lowerBound.bind(py))?;
         let u = to_column(&upperBound.bind(py))?;
         Ok(PyColumn::new(self.column.clone().between(l, u)))
     }
     #[pyo3(name = "bitwiseAND")]
     fn bitwise_and(&self, py: Python<'_>, other: Py<PyAny>) -> 
PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("bitwiseAND");
         let o = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().bitwise_and(o)))
     }
     #[pyo3(name = "bitwiseOR")]
     fn bitwise_or(&self, py: Python<'_>, other: Py<PyAny>) -> 
PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("bitwiseOR");
         let o = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().bitwise_or(o)))
     }
     #[pyo3(name = "bitwiseXOR")]
     fn bitwise_xor(&self, py: Python<'_>, other: Py<PyAny>) -> 
PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("bitwiseXOR");
         let o = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().bitwise_xor(o)))
     }
     fn ilike(&self, other: &str) -> PyColumn {
+        let _origin = crate::origin::with_origin("ilike");
         PyColumn::new(self.column.clone().ilike(other))
     }
     #[pyo3(name = "isNaN")]
     fn is_nan(&self) -> PyColumn {
+        let _origin = crate::origin::with_origin("isNaN");
         PyColumn::new(self.column.clone().is_nan())
     }
     #[pyo3(name = "eqNullSafe")]
     fn eq_null_safe(&self, py: Python<'_>, other: Py<PyAny>) -> 
PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("eqNullSafe");
         let o = to_column(&other.bind(py))?;
         Ok(PyColumn::new(self.column.clone().eq_null_safe(o)))
     }
     #[pyo3(signature = (*cols))]
     fn isin(&self, py: Python<'_>, cols: Vec<Py<PyAny>>) -> PyResult<PyColumn> 
{
+        let _origin = crate::origin::with_origin("isin");
         // As pyspark's, a single DataFrame makes an IN subquery of it.
         if let [only] = cols.as_slice() {
             if let Ok(df) = only
@@ -666,11 +728,13 @@ impl PyColumn {
     }
     #[pyo3(name = "withField")]
     fn with_field(&self, py: Python<'_>, fieldName: &str, col: Py<PyAny>) -> 
PyResult<PyColumn> {
+        let _origin = crate::origin::with_origin("withField");
         let v = to_column(&col.bind(py))?;
         Ok(PyColumn::new(self.column.clone().with_field(fieldName, v)))
     }
     #[pyo3(name = "dropFields", signature = (*fieldNames))]
     fn drop_fields(&self, fieldNames: Vec<String>) -> PyColumn {
+        let _origin = crate::origin::with_origin("dropFields");
         let refs: Vec<&str> = fieldNames.iter().map(|s| s.as_str()).collect();
         PyColumn::new(self.column.clone().drop_fields(refs))
     }
diff --git a/crates/pyspark-rs/src/dataframe.rs 
b/crates/pyspark-rs/src/dataframe.rs
index 52e8516..a1e0517 100644
--- a/crates/pyspark-rs/src/dataframe.rs
+++ b/crates/pyspark-rs/src/dataframe.rs
@@ -183,6 +183,8 @@ pub(crate) fn to_column_list(
         } else {
             // Try as string (column name)
             if let Ok(name) = arg.extract::<String>() {
+                // As PySpark's `F._to_col`, by `col`, of its origin.
+                let _origin = crate::origin::with_origin("col");
                 cols.push(spark_connect::functions::col(&name));
             } else {
                 return Err(PyErr::new::<pyo3::exceptions::PyTypeError, _>(
@@ -510,6 +512,8 @@ impl PyDataFrame {
         let col = if let Ok(c) = partitionIdCol.extract::<Bound<'_, 
PyColumn>>() {
             c.borrow().column.clone()
         } else if let Ok(name) = partitionIdCol.extract::<String>() {
+            // As PySpark's `F._to_col`, by `col`, of its origin.
+            let _origin = crate::origin::with_origin("col");
             spark_connect::functions::col(&name)
         } else {
             return Err(PyErr::new::<pyo3::exceptions::PyTypeError, _>(
@@ -2014,6 +2018,8 @@ fn column_or_name(col: &Bound<'_, PyAny>) -> 
PyResult<spark_connect::column::Col
         return Ok(column.column.clone());
     }
     if col.is_instance_of::<pyo3::types::PyString>() {
+        // As PySpark's `F._to_col`, by `col`, of its origin.
+        let _origin = crate::origin::with_origin("col");
         return Ok(spark_connect::functions::col(&col.extract::<String>()?));
     }
     let type_name = col
diff --git a/crates/pyspark-rs/src/functions.rs 
b/crates/pyspark-rs/src/functions.rs
index 763812c..a806563 100644
--- a/crates/pyspark-rs/src/functions.rs
+++ b/crates/pyspark-rs/src/functions.rs
@@ -76,6 +76,11 @@ fn udf_arguments(
                 return Ok(table_arg.inner.to_column());
             }
         }
+        // As PySpark's `to_expr` of a UDF call: a column name is a 
ColumnReference of it,
+        // without the origin of `col`.
+        if arg.is_instance_of::<pyo3::types::PyString>() {
+            return 
Ok(Column::new(Expression::column(arg.extract::<String>()?)));
+        }
         to_column(arg)
     };
     let mut columns = Vec::with_capacity(args.len() + named_args.len());
@@ -350,6 +355,8 @@ fn scalar_literal(obj: &Bound<'_, PyAny>) -> 
PyResult<Option<LiteralExpression>>
 
 #[pyfunction]
 fn pyfunc_col(name: &str) -> PyColumn {
+    // As PySpark's `col`, of `_with_origin`.
+    let _origin = crate::origin::with_origin("col");
     PyColumn::new(spark_funcs::col(name))
 }
 
diff --git a/crates/pyspark-rs/src/group.rs b/crates/pyspark-rs/src/group.rs
index 107d84f..8b7d4ad 100644
--- a/crates/pyspark-rs/src/group.rs
+++ b/crates/pyspark-rs/src/group.rs
@@ -35,6 +35,8 @@ impl PyGroupedData {
                 for (k, v) in dict.iter() {
                     let col_name: String = k.extract()?;
                     let func_name: String = v.extract()?;
+                    // As PySpark's `agg` of a dict, by `col`, of its origin.
+                    let _origin = crate::origin::with_origin("col");
                     let c = spark_connect::functions::col(&col_name);
                     // Build func(col) as an unresolved function resolved 
server-side, so
                     // aggregate names (max/min/sum/avg/count/...) all work 
regardless of
diff --git a/crates/pyspark-rs/src/lib.rs b/crates/pyspark-rs/src/lib.rs
index 96c9f43..d9cbbad 100644
--- a/crates/pyspark-rs/src/lib.rs
+++ b/crates/pyspark-rs/src/lib.rs
@@ -36,6 +36,7 @@ mod group;
 mod infer;
 mod ml;
 mod observation;
+mod origin;
 mod pandas_input;
 mod pipelines;
 mod profiler;
diff --git a/crates/pyspark-rs/src/origin.rs b/crates/pyspark-rs/src/origin.rs
new file mode 100644
index 0000000..c73b78a
--- /dev/null
+++ b/crates/pyspark-rs/src/origin.rs
@@ -0,0 +1,31 @@
+//! As PySpark's `_with_origin`: the origin in user code of the expressions 
that an API function
+//! builds, which the server reports in the `DataFrameQueryContext` of an 
error in them.
+
+use pyo3::prelude::*;
+use pyo3::sync::PyOnceLock;
+use spark_connect::origin::{set_current, Origin, OriginGuard};
+
+static UTILS: PyOnceLock<Py<PyModule>> = PyOnceLock::new();
+
+/// While the returned guard lives, the expressions that this thread builds 
have the origin of
+/// `fragment`, the name of the API function, at its call site in user code, 
when DataFrame
+/// debugging is enabled. As PySpark's `_with_origin`, with the functions of
+/// `pyspark.errors.utils` that it calls: `is_debugging_enabled`, which reads
+/// `spark.python.sql.dataFrameDebugging.enabled` once, and 
`_capture_call_site`. An origin that
+/// cannot be read is not set, as an expression is the same without one.
+pub(crate) fn with_origin(fragment: &str) -> Option<OriginGuard> {
+    Python::attach(|py| -> PyResult<Option<OriginGuard>> {
+        let utils = UTILS
+            .get_or_try_init(py, || 
py.import("pyspark.errors.utils").map(Bound::unbind))?
+            .bind(py);
+        if !utils.call_method0("is_debugging_enabled")?.is_truthy()? {
+            return Ok(None);
+        }
+        let call_site: String = utils.call_method1("_capture_call_site", 
(1,))?.extract()?;
+        Ok(Some(set_current(Origin {
+            fragment: fragment.to_string(),
+            call_site,
+        })))
+    })
+    .unwrap_or(None)
+}
diff --git a/crates/pyspark-rs/src/stat.rs b/crates/pyspark-rs/src/stat.rs
index bb45d83..cc7ced3 100644
--- a/crates/pyspark-rs/src/stat.rs
+++ b/crates/pyspark-rs/src/stat.rs
@@ -105,6 +105,8 @@ pub(crate) fn sample_by_arguments(
     let type_name =
         |obj: &Bound<'_, PyAny>| -> PyResult<String> { 
Ok(obj.get_type().name()?.to_string()) };
     let col = if let Ok(name) = col.cast::<PyString>() {
+        // As PySpark's `F._to_col`, by `col`, of its origin.
+        let _origin = crate::origin::with_origin("col");
         spark_connect::functions::col(&name.to_cow()?)
     } else if let Ok(column) = col.extract::<PyRef<'_, PyColumn>>() {
         column.column.clone()
diff --git a/crates/spark-connect/src/expression.rs 
b/crates/spark-connect/src/expression.rs
index b80fbfc..c96787b 100644
--- a/crates/spark-connect/src/expression.rs
+++ b/crates/spark-connect/src/expression.rs
@@ -24,9 +24,13 @@ pub fn next_lambda_var_index() -> u32 {
 /// Cloning a handle shares its node rather than copying the expression 
subtree.
 /// Build common expressions with [`Expression::literal`], 
[`Expression::column`],
 /// and [`Expression::function`], or use [`Expression::new`] for other node 
types.
-/// Inspect expressions with [`Expression::node`].
+/// Inspect expressions with [`Expression::node`]. An expression keeps the 
origin in user code
+/// of its construction, if any, as PySpark's does (see [`crate::origin`]).
 #[derive(Clone)]
-pub struct Expression(Arc<ExpressionNode>);
+pub struct Expression {
+    node: Arc<ExpressionNode>,
+    origin: Option<Arc<crate::origin::Origin>>,
+}
 
 impl std::fmt::Debug for Expression {
     fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
@@ -287,9 +291,18 @@ impl NamedArgument {
 }
 
 impl Expression {
-    /// Allocate an immutable expression node.
+    /// Allocate an immutable expression node, of the current origin, as 
PySpark's
+    /// `Expression.__init__` takes it.
     pub fn new(node: ExpressionNode) -> Self {
-        Self(Arc::new(node))
+        Self {
+            node: Arc::new(node),
+            origin: crate::origin::current(),
+        }
+    }
+
+    /// The origin in user code of this expression, if any.
+    pub fn origin(&self) -> Option<&crate::origin::Origin> {
+        self.origin.as_deref()
     }
 
     /// Create a literal expression.
@@ -311,7 +324,7 @@ impl Expression {
 
     /// Borrow this expression's node without exposing mutable shared state.
     pub fn node(&self) -> &ExpressionNode {
-        self.0.as_ref()
+        self.node.as_ref()
     }
 
     /// Render this expression to a human-readable string, mirroring the
@@ -394,8 +407,24 @@ impl Expression {
         }
     }
 
-    /// Converts the expression to a Spark Connect protobuf expression.
+    /// Converts the expression to a Spark Connect protobuf expression, with 
its origin, as
+    /// PySpark's `Expression._create_proto_expression` sends it.
     pub fn to_proto(&self) -> proto::Expression {
+        let mut expr = self.node_to_proto();
+        if let Some(origin) = &self.origin {
+            expr.common = Some(proto::ExpressionCommon {
+                origin: Some(proto::Origin {
+                    function: 
Some(proto::origin::Function::PythonOrigin(proto::PythonOrigin {
+                        fragment: origin.fragment.clone(),
+                        call_site: origin.call_site.clone(),
+                    })),
+                }),
+            });
+        }
+        expr
+    }
+
+    fn node_to_proto(&self) -> proto::Expression {
         match self.node() {
             ExpressionNode::Literal(lit) => lit.to_proto(),
             ExpressionNode::ColumnReference(col_ref) => col_ref.to_proto(),
diff --git a/crates/spark-connect/src/lib.rs b/crates/spark-connect/src/lib.rs
index e12c0a4..38fb6a9 100644
--- a/crates/spark-connect/src/lib.rs
+++ b/crates/spark-connect/src/lib.rs
@@ -17,6 +17,7 @@ pub mod group;
 pub mod merge;
 pub mod ml;
 pub mod observation;
+pub mod origin;
 pub mod pipelines;
 pub mod plan;
 pub mod profiler;
diff --git a/crates/spark-connect/src/origin.rs 
b/crates/spark-connect/src/origin.rs
new file mode 100644
index 0000000..d5df917
--- /dev/null
+++ b/crates/spark-connect/src/origin.rs
@@ -0,0 +1,79 @@
+//! The origins of expressions in user code, as PySpark's 
`pyspark.errors.utils.current_origin`:
+//! the API function that builds an expression, and its call site, which the 
server reports in
+//! the `DataFrameQueryContext` of an error in the expression.
+
+use std::cell::RefCell;
+use std::sync::Arc;
+
+/// The origin of an expression: the API function that built it, and its call 
site in user
+/// code. Mirrors `pyspark.sql.connect.proto.PythonOrigin`.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct Origin {
+    /// The name of the function, such as `col` or `__add__`.
+    pub fragment: String,
+    /// Where the user code called it, such as `script.py:12`.
+    pub call_site: String,
+}
+
+thread_local! {
+    /// The origin of the expressions that this thread builds now, as 
PySpark's thread-local
+    /// `_current_origin`.
+    static CURRENT: RefCell<Option<Arc<Origin>>> = const { RefCell::new(None) 
};
+}
+
+/// The origin of the expressions that this thread builds now, if any.
+pub(crate) fn current() -> Option<Arc<Origin>> {
+    CURRENT.with(|current| current.borrow().clone())
+}
+
+/// Sets the origin of the expressions that this thread builds while the 
returned guard lives,
+/// as PySpark's `_with_origin` does for a call of an API function. As 
PySpark's, the guard then
+/// unsets the origin.
+pub fn set_current(origin: Origin) -> OriginGuard {
+    CURRENT.with(|current| *current.borrow_mut() = Some(Arc::new(origin)));
+    OriginGuard { _private: () }
+}
+
+/// Unsets the current origin when dropped. See [`set_current`].
+#[must_use]
+pub struct OriginGuard {
+    _private: (),
+}
+
+impl Drop for OriginGuard {
+    fn drop(&mut self) {
+        CURRENT.with(|current| *current.borrow_mut() = None);
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use crate::expression::Expression;
+    use spark_connect_proto as proto;
+
+    #[test]
+    fn expressions_carry_the_origin_of_their_construction() {
+        // As PySpark's Expression, which takes the current origin when it is 
made, and sends
+        // it as the python_origin of its common fields.
+        assert!(Expression::column("before").to_proto().common.is_none());
+        let built = {
+            let _origin = set_current(Origin {
+                fragment: "col".to_string(),
+                call_site: "script.py:3".to_string(),
+            });
+            Expression::column("a")
+        };
+        let after = Expression::column("after");
+        assert!(after.to_proto().common.is_none());
+        assert_eq!(built, Expression::column("a"));
+        let common = built.to_proto().common.expect("the origin");
+        let Some(proto::origin::Function::PythonOrigin(origin)) =
+            common.origin.and_then(|origin| origin.function)
+        else {
+            panic!("expected a PythonOrigin");
+        };
+        assert_eq!(origin.fragment, "col");
+        assert_eq!(origin.call_site, "script.py:3");
+    }
+}
diff --git a/python/pyspark/sql/udf.py b/python/pyspark/sql/udf.py
index 67d79e9..65d846c 100644
--- a/python/pyspark/sql/udf.py
+++ b/python/pyspark/sql/udf.py
@@ -305,14 +305,12 @@ class UserDefinedFunction:
         Returns a Column representing the UDF call.
         """
         from pyspark import _pyspark
-        from pyspark.sql.functions import col
-
-        # A UDF argument is ColumnOrName: a bare string is a COLUMN NAME (like 
PySpark),
-        # not a string literal. Coerce str -> col(str) before handing to the 
Rust layer
-        # (whose to_column would otherwise treat a str as a literal). As 
PySpark's, each
-        # keyword argument is a NamedArgumentExpression, after the positional 
ones.
-        cols = [col(a) if isinstance(a, str) else a for a in args]
-        named = [(k, col(v) if isinstance(v, str) else v) for k, v in 
kwargs.items()]
+
+        # A UDF argument is ColumnOrName: as PySpark's, a bare string is a 
column name, of
+        # which pyfunc_make_udf makes a ColumnReference, and each keyword 
argument is a
+        # NamedArgumentExpression, after the positional ones.
+        cols = list(args)
+        named = list(kwargs.items())
 
         # As PySpark's PythonUDF, the command is the pickled (func, 
returnType) for the
         # server-side Python worker, and pyfunc_make_udf builds the proto 
output type.
diff --git a/python/pyspark/sql/udtf.py b/python/pyspark/sql/udtf.py
index 7b7d165..79ea32b 100644
--- a/python/pyspark/sql/udtf.py
+++ b/python/pyspark/sql/udtf.py
@@ -117,12 +117,12 @@ class UserDefinedTableFunction:
     def __call__(self, *args: Any, **kwargs: Any) -> Any:
         from pyspark import _pyspark
         from pyspark.sql import SparkSession
-        from pyspark.sql.functions import col
 
-        # As PySpark's: an argument is ColumnOrName, so a bare string is a 
column name, and
-        # each keyword argument is a NamedArgumentExpression, after the 
positional ones.
-        cols = [col(a) if isinstance(a, str) else a for a in args]
-        named = [(k, col(v) if isinstance(v, str) else v) for k, v in 
kwargs.items()]
+        # As PySpark's: an argument is ColumnOrName, so a bare string is a 
column name, of
+        # which pyfunc_make_udtf makes a ColumnReference, and each keyword 
argument is a
+        # NamedArgumentExpression, after the positional ones.
+        cols = list(args)
+        named = list(kwargs.items())
         return _pyspark.functions.pyfunc_make_udtf(
             SparkSession.active(),
             self.name,
diff --git a/python/tests/test_dropin_e2e.py b/python/tests/test_dropin_e2e.py
index 61042fa..e11b270 100644
--- a/python/tests/test_dropin_e2e.py
+++ b/python/tests/test_dropin_e2e.py
@@ -783,6 +783,51 @@ def 
test_timedelta_is_a_day_time_interval_as_pyspark(session):
         session.createDataFrame([("x",)], "d interval day to second")
 
 
+def test_errors_carry_the_dataframe_query_contexts_of_pyspark(session):
+    # As PySpark's: an error in an expression that col() or a Column method 
built reports its
+    # DataFrameQueryContext, of the function and of its call site in this file.
+    import inspect
+
+    from pyspark.errors import ArithmeticException, QueryContextType
+
+    def contexts(df):
+        with pytest.raises((AnalysisException, ArithmeticException)) as error:
+            df.collect()
+        return [
+            (c.contextType(), c.fragment(), c.callSite().rsplit("/", 1)[-1])
+            for c in error.value.getQueryContext()
+        ]
+
+    here = f"{os.path.basename(__file__)}:{{}}"
+    line = inspect.currentframe().f_lineno
+    assert contexts(session.range(1).select("no_such_col")) == [
+        (QueryContextType.DataFrame, "col", here.format(line + 1))
+    ]
+    assert contexts(session.range(1).select(F.col("nope") + 1)) == [
+        (QueryContextType.DataFrame, "col", here.format(line + 4))
+    ]
+    session.conf.set("spark.sql.ansi.enabled", "true")
+    assert contexts(session.range(1).select(F.lit(1) / F.lit(0))) == [
+        (QueryContextType.DataFrame, "__truediv__", here.format(line + 8))
+    ]
+    session.conf.unset("spark.sql.ansi.enabled")
+    # repartitionById makes a column of a name with col(), while a UDF call 
refers to a
+    # column by its name without an origin.
+    assert contexts(session.range(1).repartitionById(2, "nope")) == [
+        (QueryContextType.DataFrame, "col", here.format(line + 14))
+    ]
+    udf = F.udf(lambda x: x, "long")
+    assert contexts(session.range(1).select(udf("nope"), udf(x="nope2"))) == []
+    # sampleBy takes a column of a name with col(), as PySpark's F._to_col.
+    assert contexts(session.range(1).sampleBy("nope", {0: 1.0})) == [
+        (QueryContextType.DataFrame, "col", here.format(line + 20))
+    ]
+    # unpivot takes its columns of names with col(), as PySpark's F._to_col.
+    assert contexts(session.range(1).unpivot("nope", ["id"], "v", "w")) == [
+        (QueryContextType.DataFrame, "col", here.format(line + 24))
+    ]
+
+
 def test_server_errors_carry_their_condition_and_details(session):
     from pyspark.errors import AnalysisException, ArithmeticException, 
ParseException
     from pyspark.errors.exceptions.connect import SparkConnectGrpcException, 
SQLQueryContext
diff --git a/python/tests/test_dropin_offline.py 
b/python/tests/test_dropin_offline.py
index 404731a..9852f80 100644
--- a/python/tests/test_dropin_offline.py
+++ b/python/tests/test_dropin_offline.py
@@ -5636,6 +5636,20 @@ def mock_connect(_shared_mock_connect):
     spark.stop()
 
 
[email protected](autouse=True, scope="session")
+def _debugging_read_without_a_session():
+    # PySpark's is_debugging_enabled reads 
spark.python.sql.dataFrameDebugging.enabled of the
+    # active session at the first expression of a Column method, and keeps it 
for the
+    # interpreter. It is read without a session, before any test, so that a 
test whose session
+    # has no server, such as one of _offline_session(), does not wait to read 
it, whatever
+    # the tests that run. The tests of origins run in their own interpreters.
+    from pyspark.errors.utils import is_debugging_enabled
+    from pyspark.sql import SparkSession
+
+    assert SparkSession.getActiveSession() is None
+    is_debugging_enabled()
+
+
 @pytest.fixture(autouse=True)
 def _no_session_left_active():
     # Each test starts without an active session, as a new interpreter does. 
What reads the
@@ -6550,6 +6564,60 @@ def test_window_and_lambda_reprs_as_pyspark():
         repr(F.transform("a", lambda x: x + 1)))
 
 
+def test_expressions_carry_their_origins_as_pyspark():
+    # As PySpark's _with_origin: col() and the Column methods give the 
expressions that they
+    # build the origin of their name and their call site in user code, as the 
python_origin of
+    # their common fields, unless spark.python.sql.dataFrameDebugging.enabled, 
which PySpark
+    # reads once, is false. In new interpreters, whose first column of a 
session reads it.
+    import subprocess
+    import sys
+    from pathlib import Path
+
+    pytest.importorskip("grpc")
+    code = """
+import sys
+sys.path.insert(0, sys.argv[1])
+from _mock_connect_server import mock_connect_session, parse_message
+from pyspark.sql import functions as F
+
+def origins(expression):
+    # Expression.common=18 -> ExpressionCommon.origin=1 -> 
Origin.python_origin=1:
+    # fragment=1, call_site=2; .unresolved_function=3, whose arguments=2.
+    expression = parse_message(expression)
+    found = []
+    if 18 in expression:
+        origin = 
parse_message(parse_message(parse_message(expression[18][0])[1][0])[1][0])
+        found.append((origin[1][0].decode(), origin[2][0].decode()))
+    if 3 in expression:
+        for argument in parse_message(expression[3][0]).get(2, []):
+            found += origins(argument)
+    return found
+
+if __name__ == "__main__":
+    with mock_connect_session() as (spark, service):
+        service.confs = dict(arg.split("=") for arg in sys.argv[2:])
+        column = F.col("a") + 1
+        spark.range(1).select(column).collect()
+        # ExecutePlanRequest.plan=3, Plan.root=1, Relation.project=3, 
Project.expressions=3.
+        plan = 
parse_message(parse_message(service.execute_requests[-1])[3][0])[1][0]
+        project = parse_message(parse_message(plan)[3][0])
+        print(origins(project[3][0]))
+"""
+    tests = str(Path(__file__).resolve().parent)
+
+    def run(*confs):
+        result = subprocess.run(
+            [sys.executable, "-c", code, tests, *confs], check=True, 
capture_output=True,
+            text=True,
+        )
+        return result.stdout.strip().splitlines()[-1]
+
+    assert run() == (
+        "[('__add__', '<string>:23'), ('col', '<string>:23'), ('__add__', 
'<string>:23')]"
+    )
+    assert run("spark.python.sql.dataFrameDebugging.enabled=false") == "[]"
+
+
 @pytest.mark.parametrize("column,expected", [
     # instr and regexp_replace are plain functions, as other functions are.
     (lambda F: F.instr("s", "b"), "instr(s, b)"),


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to