Re: spark vs flink low memory available

2015-08-11 Thread Pa Rö
my first post is here and a log too: http://mail-archives.us.apache.org/mod_mbox/spark-user/201508.mbox/%3ccah2_pykqhfr4tbvpbt2tdhgm+zrkcbzfnk7uedkjpdhe472...@mail.gmail.com%3E i use cloudera live, i think i can not use spark 1.5. i will try to run it again and post the current logfile here. 201

Re: spark vs flink low memory available

2015-08-11 Thread Ted Yu
Pa: Can you try 1.5.0 SNAPSHOT ? See SPARK-7075 Project Tungsten (Spark 1.5 Phase 1) Cheers On Tue, Aug 11, 2015 at 12:49 AM, jun wrote: > your detail of log file? > > At 2015-08-10 22:02:16, "Pa Rö" wrote: > > hi community, > > i have build a spark and flink k-means application. > my test ca

spark vs flink low memory available

2015-08-11 Thread parö
: http://apache-spark-user-list.1001560.n3.nabble.com/spark-vs-flink-low-memory-available-tp24208.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org

spark vs flink low memory available

2015-08-10 Thread Pa Rö
hi community, i have build a spark and flink k-means application. my test case is a clustering on 1 million points on 3node cluster. in memory bottlenecks begins flink to outsource to disk and work slowly but works. however spark lose executers if the memory is full and starts again (infinety loo

spark vs flink low memory available

2015-08-10 Thread Pa Rö
hi community, i have build a spark and flink k-means application. my test case is a clustering on 1 million points on 3node cluster. in memory bottlenecks begins flink to outsource to disk and work slowly but works. however spark lose executers if the memory is full and starts again (infinety loo