Hi, I guess the title says it all, right now it seems like BEAM caches all the intermediate RDD results for my pipeline when using the Spark runner, this leads to a very inefficient usage of memory. Any way to control this?
Best regards, Augusto
Hi, I guess the title says it all, right now it seems like BEAM caches all the intermediate RDD results for my pipeline when using the Spark runner, this leads to a very inefficient usage of memory. Any way to control this?
Best regards, Augusto