Fwd: Spark CacheManager Thread-safety

2016-05-20 Thread Pietro Gentile
Hi all, I have a series of doubts about CacheManager used by SQLContext to cache DataFrame. My use case requires different threads persisting/reading dataframes cuncurrently. I realized using spark that persistence really does not work in parallel mode. I would like it if I'm persisting a data

Spark CacheManager Thread-safety

2016-05-20 Thread Pietro Gentile
Hi all, I have a series of doubts about CacheManager used by SQLContext to cache DataFrame. My use case requires different threads persisting/reading dataframes cuncurrently. I realized using spark that persistence really does not work in parallel mode. I would like it if I'm persisting a data