Pa: Can you try 1.5.0 SNAPSHOT ? See SPARK-7075 Project Tungsten (Spark 1.5 Phase 1)
Cheers On Tue, Aug 11, 2015 at 12:49 AM, jun <kit...@126.com> wrote: > your detail of log file? > > At 2015-08-10 22:02:16, "Pa Rö" <paul.roewer1...@googlemail.com> wrote: > > hi community, > > i have build a spark and flink k-means application. > my test case is a clustering on 1 million points on 3node cluster. > > in memory bottlenecks begins flink to outsource to disk and work slowly > but works. > however spark lose executers if the memory is full and starts again > (infinety loop?). > > i try to customize the memory setting with the help from the mailing list > here > <http://mail-archives.us.apache.org/mod_mbox/spark-user/201508.mbox/%3ccah2_pykqhfr4tbvpbt2tdhgm+zrkcbzfnk7uedkjpdhe472...@mail.gmail.com%3E>, > thanks. > but spark not work. > > is it necessary to have any configurations to be set? i mean flink work > with low memory, spark must also be able to or not? > > best regards, > paul > > > >
