Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/17671#discussion_r154887421
--- Diff: python/pyspark/worker.py ---
@@ -160,6 +166,24 @@ def read_udfs(pickleSer, infile, eval_type):
def main(infile, outfile):
+ if raven:
--- End diff --
I mean does this overhead give us anything? Like do we need raven set up
before the exception occurs? For something like streaming folks have done a
bunch of work to get the worker startup time down and I'd rather not add things
in the hot path if we don't need to (even if they are small). Of course if its
needs to be there that's a different discussion.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
