This is extremely helpful!
So just to make sure I’m understanding correctly:
I’m also looking into fixing my cron jobs so they don’t stack up by implementing flock in the jobs and changing how teardowns of the spark cluster work as far as failed workers.
Thanks again,
—Ken
|
- Limit pyspark.daemon threads Carlile, Ken
- Re: Limit pyspark.daemon threads Ted Yu
- ?????? Limit pyspark.daemon threads Sea
- Re: Limit pyspark.daemon threads Carlile, Ken
- Re: Limit pyspark.daemon threads Carlile, Ken
- Re: Limit pyspark.daemon threads Carlile, Ken
- Re: Limit pyspark.daemon threads Sven Krasser
- Re: Limit pyspark.daemon threads Carlile, Ken
- Re: Limit pyspark.daemon thread... Sven Krasser
- Re: Limit pyspark.daemon thread... Carlile, Ken
- Re: Limit pyspark.daemon thread... Sven Krasser