cloud-fan commented on PR #45377:
URL: https://github.com/apache/spark/pull/45377#issuecomment-2041828693
@itholic what if we don't use thread local? IIUC, PySpark calls JVM methods
to build the column instances at the end. On the JVM side, we wrap code with
`withOrigin` to capture the call site automatically.
My idea: we add new `Column` creation methods for PySpark, which takes
python call site information. The implementation should set call site before
calling `withOrigin`, as `withOrigin` respects the already captured call site
```
private[sql] def withOrigin[T](f: => T): T = {
if (CurrentOrigin.get.stackTrace.isDefined) {
f
} else {
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
