rangadi commented on code in PR #46125:
URL: https://github.com/apache/spark/pull/46125#discussion_r1575186174
##########
python/pyspark/sql/tests/connect/streaming/test_parity_foreach_batch.py:
##########
@@ -66,6 +66,27 @@ def func(df, _):
q = df.writeStream.foreachBatch(func).start()
q.processAllAvailable()
+ def test_pickling_deserialization_error(self):
+ class NoUnpickle:
+
+ def __reduce__(self):
+ # Serialize only the data attribute
+ return self.throw_exception(), ()
+
+ def throw_exception(self):
+ raise RuntimeError("Cannot unpickle instance of NoUnpickle")
+
+ no_unpickle = NoUnpickle()
+
+ def func(df, _):
+ print(no_unpickle)
+ df.count()
+
+ with self.assertRaises(Exception, msg="Cannot unpickle instance of
NoUnpickle"):
Review Comment:
For reference, could you include the exception in this PR thread before and
after the fix?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
