cloud-fan commented on code in PR #45377:

@@ -171,6 +171,29 @@ class Column(val expr: Expression) extends Logging {
     Column.fn(name, this, lit(other))
+  /**
+   * A version of the `fn` method specifically designed for binary operations 
in PySpark
+   * that require logging information.
+   * This method is used when the operation involves another Column.
+   *
+   * @param name                The name of the operation to be performed.
+   * @param other               The value to be used in the operation, which 
will be converted to a
+   *                            Column if not already one.
+   * @param pysparkFragment     A string representing the 'fragment' of the 
PySpark error context,
+   *                            typically indicates the name of PySpark 
+   * @param pysparkCallSite     A string representing the 'callSite' of the 
PySpark error context,
+   *                            providing the exact location within the 
PySpark code where the
+   *                            operation originated.
+   * @return A Column resulting from the operation.
+   */
+  private def fn(

Review Comment:
   @HyukjinKwon This probably can't cover all the cases, and we may need to add 
more overloads for certain functions that require non-expression parameters, 
but it shouldn't be many.
   I think it's better than using ThreadLocal which can be quite fragile to 
pass values between Python and JVM.

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail:

For queries about this service, please contact Infrastructure at:

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to