<dependencies>
<dependency> <!-- Spark dependency -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.6.1</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.10</artifactId>
<version>1.6.1</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-kafka_2.10</artifactId>
<version>1.6.1</version>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>2.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.10</artifactId>
<version>0.8.2.2</version>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch-spark_2.10</artifactId>
<version>2.2.0</version>
</dependency>
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>2.8.0</version>
<type>jar</type>
<scope>compile</scope>
</dependency>
</dependencies>
How can I look at those tasks?
Van: Ted Yu [mailto:[email protected]]
Verzonden: donderdag 24 maart 2016 14:33
Aan: Michel Hubert <[email protected]>
CC: [email protected]
Onderwerp: Re: apache spark errors
Which release of Spark are you using ?
Have you looked the tasks whose Ids were printed to see if there was more clue ?
Thanks
On Thu, Mar 24, 2016 at 6:12 AM, Michel Hubert
<[email protected]<mailto:[email protected]>> wrote:
HI,
I constantly get these errors:
0 [Executor task launch worker-15] ERROR org.apache.spark.executor.Executor
- Managed memory leak detected; size = 6564500 bytes, TID = 38969
310002 [Executor task launch worker-12] ERROR
org.apache.spark.executor.Executor - Managed memory leak detected; size =
5523550 bytes, TID = 43270
318445 [Executor task launch worker-12] ERROR
org.apache.spark.executor.Executor - Managed memory leak detected; size =
6879566 bytes, TID = 43408
388893 [Executor task launch worker-12] ERROR
org.apache.spark.executor.Executor - Managed memory leak detected; size =
5572546 bytes, TID = 44382
418186 [Executor task launch worker-13] ERROR
org.apache.spark.executor.Executor - Managed memory leak detected; size =
5289222 bytes, TID = 44795
488421 [Executor task launch worker-4] ERROR org.apache.spark.executor.Executor
- Managed memory leak detected; size = 8738142 bytes, TID = 45769
619276 [Executor task launch worker-4] ERROR org.apache.spark.executor.Executor
- Managed memory leak detected; size = 5759312 bytes, TID = 47571
632275 [Executor task launch worker-12] ERROR
org.apache.spark.executor.Executor - Managed memory leak detected; size =
5602240 bytes, TID = 47709
644989 [Executor task launch worker-13] ERROR
org.apache.spark.executor.Executor - Managed memory leak detected; size =
5326260 bytes, TID = 47863
720701 [Executor task launch worker-12] ERROR
org.apache.spark.executor.Executor - Managed memory leak detected; size =
5399578 bytes, TID = 48959
1147961 [Executor task launch worker-16] ERROR
org.apache.spark.executor.Executor - Managed memory leak detected; size =
5251872 bytes, TID = 54922
How can I fix this?
With kind regard,
Michel
