Re: handling of empty partitions

2017-01-11 Thread Georg Heiler
I was wondering > >> what > >> you want to ask in the previous reply. > >> > >> > >> > >> geoHeil wrote > >> > Thanks a lot, Holden. > >> > > >> > @Liang-Chi Hsieh did you try to run > >> > https://gist

Re: handling of empty partitions

2017-01-11 Thread Liang-Chi Hsieh
ibed the problem >> > pretty well. Ist it clear for you now? >> > >> > Cheers, >> > Georg >> > >> > Holden Karau [via Apache Spark Developers List] < >> >> > ml-node+s1001551n20516h45@.nabble >> >> >> schrieb

Re: handling of empty partitions

2017-01-09 Thread Georg Heiler
Developers List] < > > > ml-node+s1001551n20516h45@.nabble > > >> schrieb am Mo., 9. Jan. 2017 um > > 06:40 Uhr: > > > >> Hi Georg, > >> > >> Thanks for the question along with the code (as well as posting to stack > >&g

Re: handling of empty partitions

2017-01-09 Thread Liang-Chi Hsieh
uited for stackoverflow its >> probably better suited to the user@ list instead of the dev@ list so I've >> cc'd the user@ list for you. >> >> As far as handling empty partitions when working mapPartitions (and >> similar), the general approach is to return an empty itera

Re: handling of empty partitions

2017-01-08 Thread geoHeil
ted to the user@ list instead of the dev@ list so I've > cc'd the user@ list for you. > > As far as handling empty partitions when working mapPartitions (and > similar), the general approach is to return an empty iterator of the > correct type when you have an empty input iterator.

Re: handling of empty partitions

2017-01-08 Thread Holden Karau
Hi Georg, Thanks for the question along with the code (as well as posting to stack overflow). In general if a question is well suited for stackoverflow its probably better suited to the user@ list instead of the dev@ list so I've cc'd the user@ list for you. As far as handling empty partitions

Re: handling of empty partitions

2017-01-08 Thread Liang-Chi Hsieh
empty partitions? > http://stackoverflow.com/questions/41474175/spark-mappartitionswithindex-handling-empty-partitions > > Kind regards, > Georg - Liang-Chi Hsieh | @viirya Spark Technology Center http://www.spark.tc/ -- View this message in context: http://apache-spark-developers

handling of empty partitions

2017-01-06 Thread geoHeil
with empty partitions. How should this be implemented properly to handle such empty partitions? http://stackoverflow.com/questions/41474175/spark-mappartitionswithindex-handling-empty-partitions Kind regards, Georg -- View this message in context: http://apache-spark-developers-list.1001551.n3