RE: Stage contains task of large size

2016-03-03 Thread Mohammed Guller
/dp/1484209656/> From: Silvio Fiorito [mailto:silvio.fior...@granturing.com] Sent: Wednesday, March 2, 2016 8:43 PM To: Bijuna; user Subject: RE: Stage contains task of large size One source of this could be more than you intended (or realized) getting serialized as part of your operations

RE: Stage contains task of large size

2016-03-02 Thread Silvio Fiorito
ilto:user@spark.apache.org> Subject: Stage contains task of large size Spark users, We are running spark application in standalone mode. We see warn messages in the logs which says Stage 46 contains a task of very large size (983 KB) . The maximum recommended task size is 100 KB.

Stage contains task of large size

2016-03-02 Thread Bijuna
Spark users, We are running spark application in standalone mode. We see warn messages in the logs which says Stage 46 contains a task of very large size (983 KB) . The maximum recommended task size is 100 KB. What is the recommended approach to fix this warning. Please let me know. Thank yo