WeichenXu123 edited a comment on issue #25138: [SPARK-26175][PYSPARK] Closing stdin of the worker process right after fork URL: https://github.com/apache/spark/pull/25138#issuecomment-511458893 @JoshRosen @srowen @HyukjinKwon After I test more on this, I found another issue. We cannot simply close stdin. Instead, we should create another file to occupy the file descriptor "0". Otherwise the file descriptor "0" will be possible to be allocated to other file such as here https://github.com/apache/spark/blob/8ecbb67b3b29bb4fffa5863de3a82921ddf20bb3/python/pyspark/daemon.py#L58 (i.e. in L58 the dup sock file will be allocated file descriptor "0") So I update my code. If this is OK, I will update test then. Thanks!
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
