WweiL commented on code in PR #40785:
URL: https://github.com/apache/spark/pull/40785#discussion_r1166125939
##########
connector/connect/common/src/main/protobuf/spark/connect/commands.proto:
##########
@@ -298,6 +306,16 @@ message StreamingQueryCommandResult {
// Logical and physical plans as string
string result = 1;
}
+
+ message ExceptionResult {
Review Comment:
Yes I agree, currently the support for error is limited.
I tracked how they handle errors, and found that it's through here:
https://github.com/apache/spark/blob/0c2776a7863b0ec2dd7001638a65840568453a89/python/pyspark/sql/connect/client.py#L1115
And then in the `convert_exception` method:
https://github.com/apache/spark/blob/0c2776a7863b0ec2dd7001638a65840568453a89/python/pyspark/errors/exceptions/connect.py#L57
Only the message is directly passed.
I guess we could file a ticket to wait until batch side's change, and then
we could align with them?
##########
connector/connect/common/src/main/protobuf/spark/connect/commands.proto:
##########
@@ -298,6 +306,16 @@ message StreamingQueryCommandResult {
// Logical and physical plans as string
string result = 1;
}
+
+ message ExceptionResult {
Review Comment:
Yes I agree, currently the support for error is limited.
I tracked how they handle errors, and found that it's through here:
https://github.com/apache/spark/blob/0c2776a7863b0ec2dd7001638a65840568453a89/python/pyspark/sql/connect/client.py#L1115
And then in the `convert_exception` method:
https://github.com/apache/spark/blob/0c2776a7863b0ec2dd7001638a65840568453a89/python/pyspark/errors/exceptions/connect.py#L57-L58
Only the message is directly passed.
I guess we could file a ticket to wait until batch side's change, and then
we could align with them?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
