[
https://issues.apache.org/jira/browse/FLINK-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14644351#comment-14644351
]
Andra Lungu commented on FLINK-2412:
------------------------------------
19:17:33,944 WARN org.apache.hadoop.util.NativeCodeLoader
- Unable to load native-hadoop library for your platform... using builtin-java
classes where applicable
19:17:34,034 INFO org.apache.flink.runtime.taskmanager.TaskManager
-
--------------------------------------------------------------------------------
19:17:34,034 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Starting TaskManager (Version: 0.10-SNAPSHOT, Rev:590c6d0, Date:09.07.2015
@ 10:38:34 CEST)
19:17:34,034 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Current user: andra.lungu
19:17:34,034 INFO org.apache.flink.runtime.taskmanager.TaskManager
- JVM: Java HotSpot(TM) 64-Bit Server VM - Oracle Corporation - 1.7/23.7-b01
19:17:34,034 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Maximum heap size: 14720 MiBytes
19:17:34,034 INFO org.apache.flink.runtime.taskmanager.TaskManager
- JAVA_HOME: /usr/lib/jvm/oracle-jdk-bin-1.7
19:17:34,034 INFO org.apache.flink.runtime.taskmanager.TaskManager
- JVM Options:
19:17:34,035 INFO org.apache.flink.runtime.taskmanager.TaskManager
- -Xms15360m
19:17:34,035 INFO org.apache.flink.runtime.taskmanager.TaskManager
- -Xmx15360m
19:17:34,035 INFO org.apache.flink.runtime.taskmanager.TaskManager
- -XX:MaxPermSize=256m
19:17:34,035 INFO org.apache.flink.runtime.taskmanager.TaskManager
-
-Dlog.file=/home/andra.lungu/peel/systems/flink-0.10-SNAPSHOT/bin/../log/flink-andra.lungu-taskmanager-0-wally012.log
19:17:34,035 INFO org.apache.flink.runtime.taskmanager.TaskManager
-
-Dlog4j.configuration=file:/home/andra.lungu/peel/systems/flink-0.10-SNAPSHOT/bin/../conf/log4j.properties
19:17:34,035 INFO org.apache.flink.runtime.taskmanager.TaskManager
-
-Dlogback.configurationFile=file:/home/andra.lungu/peel/systems/flink-0.10-SNAPSHOT/bin/../conf/logback.xml
19:17:34,035 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Program Arguments:
19:17:34,035 INFO org.apache.flink.runtime.taskmanager.TaskManager
- --configDir
19:17:34,035 INFO org.apache.flink.runtime.taskmanager.TaskManager
- /home/andra.lungu/peel/systems/flink-0.10-SNAPSHOT/bin/../conf
19:17:34,035 INFO org.apache.flink.runtime.taskmanager.TaskManager
- --streamingMode
19:17:34,035 INFO org.apache.flink.runtime.taskmanager.TaskManager
- batch
19:17:34,035 INFO org.apache.flink.runtime.taskmanager.TaskManager
-
--------------------------------------------------------------------------------
19:17:34,039 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Maximum number of open file descriptors is 50000
19:17:34,059 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Loading configuration from
/home/andra.lungu/peel/systems/flink-0.10-SNAPSHOT/bin/../conf
19:17:34,106 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Security is not enabled. Starting non-authenticated TaskManager.
19:17:34,106 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Trying to select the network interface and address to use by connecting to
the configured JobManager.
19:17:34,107 INFO org.apache.flink.runtime.taskmanager.TaskManager
- TaskManager will try to connect for 120000 seconds before falling back to
heuristics
19:17:34,468 INFO org.apache.flink.runtime.taskmanager.TaskManager
- TaskManager will use hostname/address 'wally012.cit.tu-berlin.de'
(130.149.249.22) for communication.
19:17:34,468 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Starting TaskManager in streaming mode BATCH_ONLY
19:17:34,469 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Starting TaskManager actor system at wally012.cit.tu-berlin.de:0
19:17:34,769 INFO akka.event.slf4j.Slf4jLogger
- Slf4jLogger started
19:17:34,801 INFO Remoting
- Starting remoting
19:17:34,918 INFO Remoting
- Remoting started; listening on addresses
:[akka.tcp://[email protected]:59193]
19:17:34,922 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Starting TaskManager actor
19:17:34,927 INFO org.apache.flink.runtime.io.network.netty.NettyConfig
- NettyConfig [server address: wally012.cit.tu-berlin.de/130.149.249.22,
server port: 50815, memory segment size (bytes): 32768, transport type: NIO,
number of server threads: 0 (use Netty's default), number of client threads: 0
(use Netty's default), server connect backlog: 0 (use Netty's default), client
connect timeout (sec): 120, send/receive buffer size (bytes): 0 (use Netty's
default)]
19:17:34,929 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Messages between TaskManager and JobManager have a max timeout of 100000
milliseconds
19:17:34,930 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Temporary file directory '/data/andra.lungu/flink_tmp': total 2685 GB,
usable 2170 GB (80.82% usable)
19:17:35,329 INFO org.apache.flink.runtime.io.network.buffer.NetworkBufferPool
- Allocated 1562 MB for network buffer pool (number of memory segments: 50000,
bytes per segment: 32768).
19:17:35,838 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Using 0.4 of the currently free heap space for Flink managed memory (5256
MB).
19:17:37,053 INFO org.apache.flink.runtime.io.disk.iomanager.IOManager
- I/O manager uses directory
/data/andra.lungu/flink_tmp/flink-io-2440a2bc-630e-462b-914d-8c2de3904ed3 for
spill files.
19:17:37,059 INFO org.apache.flink.runtime.filecache.FileCache
- User file cache uses directory
/data/andra.lungu/flink_tmp/flink-dist-cache-ccb9c658-5888-4f68-947d-0bb2d5bbaf06
19:17:37,176 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Starting TaskManager actor at akka://flink/user/taskmanager#-1424706842.
19:17:37,176 INFO org.apache.flink.runtime.taskmanager.TaskManager
- TaskManager data connection information: wally012.cit.tu-berlin.de
(dataPort=50815)
19:17:37,176 INFO org.apache.flink.runtime.taskmanager.TaskManager
- TaskManager has 8 task slot(s).
19:17:37,177 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Memory usage stats: [HEAP: 6936/14720/14720 MB, NON HEAP: 25/45/304 MB
(used/committed/max)]
19:17:37,179 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Trying to register at JobManager
akka.tcp://[email protected]:6123/user/jobmanager (attempt 1, timeout: 500
milliseconds)
19:17:37,275 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Successful registration at JobManager
(akka.tcp://[email protected]:6123/user/jobmanager), starting network stack
and library cache.
19:17:37,381 INFO org.apache.flink.runtime.io.network.netty.NettyClient
- Successful initialization (took 35 ms).
19:17:37,406 INFO org.apache.flink.runtime.io.network.netty.NettyServer
- Successful initialization (took 25 ms). Listening on SocketAddress
/130.149.249.22:50815.
19:17:37,407 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Determined BLOB server address to be /130.149.249.21:51598. Starting BLOB
cache.
19:17:37,412 INFO org.apache.flink.runtime.blob.BlobCache
- Created BLOB cache storage directory
/tmp/blobStore-c638b003-e58d-44ec-8ddc-5d7b2b13380e
19:18:41,199 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Received task CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (2/60)
19:18:41,359 INFO org.apache.flink.runtime.taskmanager.Task
- Loading JAR files for task CHAIN DataSource (at
getEdgesDataSet(Degrees.java:64) (org.apache.flink.api.java.io.CsvInputFormat))
-> Map (Map at getEdgesDataSet(Degrees.java:64)) (2/60)
19:18:41,841 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Received task CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (58/60)
19:18:41,841 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Received task CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (58/60)
19:18:41,926 INFO org.apache.flink.runtime.taskmanager.Task
- Loading JAR files for task CHAIN DataSource (at
getEdgesDataSet(Degrees.java:64) (org.apache.flink.api.java.io.CsvInputFormat))
-> Map (Map at getEdgesDataSet(Degrees.java:64)) (58/60)
19:18:42,372 INFO org.apache.flink.runtime.blob.BlobCache
- Downloading 4963176c3b88c01ef39ec57be1bf8e632a8548ab from
/130.149.249.21:51598
19:19:22,169 INFO org.apache.flink.runtime.taskmanager.Task
- Registering task at network: CHAIN DataSource (at
getEdgesDataSet(Degrees.java:64) (org.apache.flink.api.java.io.CsvInputFormat))
-> Map (Map at getEdgesDataSet(Degrees.java:64)) (2/60) [DEPLOYING]
19:19:22,169 INFO org.apache.flink.runtime.taskmanager.Task
- Registering task at network: CHAIN DataSource (at
getEdgesDataSet(Degrees.java:64) (org.apache.flink.api.java.io.CsvInputFormat))
-> Map (Map at getEdgesDataSet(Degrees.java:64)) (58/60) [DEPLOYING]
19:19:47,991 INFO org.apache.flink.runtime.taskmanager.Task
- CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (58/60) switched to RUNNING
19:19:47,991 INFO org.apache.flink.runtime.taskmanager.Task
- CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (2/60) switched to RUNNING
19:25:16,775 INFO org.apache.flink.runtime.taskmanager.Task
- CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (58/60) switched to FINISHED
19:18:41,841 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Received task CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (58/60)
19:18:41,926 INFO org.apache.flink.runtime.taskmanager.Task
- Loading JAR files for task CHAIN DataSource (at
getEdgesDataSet(Degrees.java:64) (org.apache.flink.api.java.io.CsvInputFormat))
-> Map (Map at getEdgesDataSet(Degrees.java:64)) (58/60)
19:18:42,372 INFO org.apache.flink.runtime.blob.BlobCache
- Downloading 4963176c3b88c01ef39ec57be1bf8e632a8548ab from
/130.149.249.21:51598
19:19:22,169 INFO org.apache.flink.runtime.taskmanager.Task
- Registering task at network: CHAIN DataSource (at
getEdgesDataSet(Degrees.java:64) (org.apache.flink.api.java.io.CsvInputFormat))
-> Map (Map at getEdgesDataSet(Degrees.java:64)) (2/60) [DEPLOYING]
19:19:22,169 INFO org.apache.flink.runtime.taskmanager.Task
- Registering task at network: CHAIN DataSource (at
getEdgesDataSet(Degrees.java:64) (org.apache.flink.api.java.io.CsvInputFormat))
-> Map (Map at getEdgesDataSet(Degrees.java:64)) (58/60) [DEPLOYING]
19:19:47,991 INFO org.apache.flink.runtime.taskmanager.Task
- CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (58/60) switched to RUNNING
19:19:47,991 INFO org.apache.flink.runtime.taskmanager.Task
- CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (2/60) switched to RUNNING
19:25:16,775 INFO org.apache.flink.runtime.taskmanager.Task
- CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (58/60) switched to FINISHED
19:18:41,841 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Received task CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (58/60)
19:18:41,926 INFO org.apache.flink.runtime.taskmanager.Task
- Loading JAR files for task CHAIN DataSource (at
getEdgesDataSet(Degrees.java:64) (org.apache.flink.api.java.io.CsvInputFormat))
-> Map (Map at getEdgesDataSet(Degrees.java:64)) (58/60)
19:18:42,372 INFO org.apache.flink.runtime.blob.BlobCache
- Downloading 4963176c3b88c01ef39ec57be1bf8e632a8548ab from
/130.149.249.21:51598
19:19:22,169 INFO org.apache.flink.runtime.taskmanager.Task
- Registering task at network: CHAIN DataSource (at
getEdgesDataSet(Degrees.java:64) (org.apache.flink.api.java.io.CsvInputFormat))
-> Map (Map at getEdgesDataSet(Degrees.java:64)) (2/60) [DEPLOYING]
19:19:22,169 INFO org.apache.flink.runtime.taskmanager.Task
- Registering task at network: CHAIN DataSource (at
getEdgesDataSet(Degrees.java:64) (org.apache.flink.api.java.io.CsvInputFormat))
-> Map (Map at getEdgesDataSet(Degrees.java:64)) (58/60) [DEPLOYING]
19:19:47,991 INFO org.apache.flink.runtime.taskmanager.Task
- CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (58/60) switched to RUNNING
19:19:47,991 INFO org.apache.flink.runtime.taskmanager.Task
- CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (2/60) switched to RUNNING
19:25:16,775 INFO org.apache.flink.runtime.taskmanager.Task
- CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (58/60) switched to FINISHED
19:25:16,923 INFO org.apache.flink.runtime.taskmanager.Task
- Freeing task resources for CHAIN DataSource (at
getEdgesDataSet(Degrees.java:64) (org.apache.flink.api.java.io.CsvInputFormat))
-> Map (Map at getEdgesDataSet(Degrees.java:64)) (58/60)
19:25:16,986 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Unregistering task and sending final execution state FINISHED to JobManager
for task CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (4b0325c8f21be79d7d3b059d49fb6ad9)
19:25:21,709 INFO org.apache.flink.runtime.taskmanager.Task
- CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (2/60) switched to FINISHED
19:25:21,709 INFO org.apache.flink.runtime.taskmanager.Task
- Freeing task resources for CHAIN DataSource (at
getEdgesDataSet(Degrees.java:64) (org.apache.flink.api.java.io.CsvInputFormat))
-> Map (Map at getEdgesDataSet(Degrees.java:64)) (2/60)
19:25:21,761 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Unregistering task and sending final execution state FINISHED to JobManager
for task CHAIN DataSource (at getEdgesDataSet(Degrees.java:64)
(org.apache.flink.api.java.io.CsvInputFormat)) -> Map (Map at
getEdgesDataSet(Degrees.java:64)) (bc512a37f814a788b13f1f8528127f47)
19:29:14,686 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Received task CHAIN FlatMap (FlatMap at fromDataSet(Graph.java:171)) ->
Combine(Distinct at fromDataSet(Graph.java:171)) (2/60)
19:29:14,687 INFO org.apache.flink.runtime.taskmanager.Task
- Loading JAR files for task CHAIN FlatMap (FlatMap at
fromDataSet(Graph.java:171)) -> Combine(Distinct at
fromDataSet(Graph.java:171)) (2/60)
19:29:14,688 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Received task CHAIN FlatMap (FlatMap at fromDataSet(Graph.java:171)) ->
Combine(Distinct at fromDataSet(Graph.java:171)) (58/60)
19:29:14,727 INFO org.apache.flink.runtime.taskmanager.Task
- Loading JAR files for task CHAIN FlatMap (FlatMap at
fromDataSet(Graph.java:171)) -> Combine(Distinct at
fromDataSet(Graph.java:171)) (58/60)
19:29:14,751 INFO org.apache.flink.runtime.taskmanager.Task
- Registering task at network: CHAIN FlatMap (FlatMap at
fromDataSet(Graph.java:171)) -> Combine(Distinct at
fromDataSet(Graph.java:171)) (58/60) [DEPLOYING]
19:29:14,751 INFO org.apache.flink.runtime.taskmanager.Task
- Registering task at network: CHAIN FlatMap (FlatMap at
fromDataSet(Graph.java:171)) -> Combine(Distinct at
fromDataSet(Graph.java:171)) (2/60) [DEPLOYING]
19:29:14,811 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Received task CoGroup (CoGroup at outDegrees(Graph.java:671)) (5/60)
19:29:14,811 INFO org.apache.flink.runtime.taskmanager.Task
- Loading JAR files for task CoGroup (CoGroup at outDegrees(Graph.java:671))
(5/60)
19:29:14,823 INFO org.apache.flink.runtime.taskmanager.Task
- Registering task at network: CoGroup (CoGroup at outDegrees(Graph.java:671))
(5/60) [DEPLOYING]
19:29:14,880 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Received task CoGroup (CoGroup at outDegrees(Graph.java:671)) (7/60)
19:29:14,881 INFO org.apache.flink.runtime.taskmanager.Task
- Loading JAR files for task CoGroup (CoGroup at outDegrees(Graph.java:671))
(7/60)
19:29:14,913 INFO org.apache.flink.runtime.taskmanager.Task
- Registering task at network: CoGroup (CoGroup at outDegrees(Graph.java:671))
(7/60) [DEPLOYING]
19:29:14,941 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Received task CoGroup (CoGroup at inDegrees(Graph.java:701)) (7/60)
19:29:14,941 INFO org.apache.flink.runtime.taskmanager.Task
- Loading JAR files for task CoGroup (CoGroup at inDegrees(Graph.java:701))
(7/60)
19:29:15,046 INFO org.apache.flink.runtime.taskmanager.Task
- Registering task at network: CoGroup (CoGroup at inDegrees(Graph.java:701))
(7/60) [DEPLOYING]
19:29:15,066 INFO org.apache.flink.runtime.taskmanager.Task
- CoGroup (CoGroup at outDegrees(Graph.java:671)) (5/60) switched to RUNNING
19:29:15,066 INFO org.apache.flink.runtime.taskmanager.Task
- CoGroup (CoGroup at outDegrees(Graph.java:671)) (7/60) switched to RUNNING
19:29:15,067 INFO org.apache.flink.runtime.taskmanager.Task
- CoGroup (CoGroup at inDegrees(Graph.java:701)) (7/60) switched to RUNNING
19:29:15,067 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Received task CoGroup (CoGroup at inDegrees(Graph.java:701)) (8/60)
19:29:15,068 INFO org.apache.flink.runtime.taskmanager.Task
- Loading JAR files for task CoGroup (CoGroup at inDegrees(Graph.java:701))
(8/60)
19:29:15,070 INFO org.apache.flink.runtime.taskmanager.Task
- Registering task at network: CoGroup (CoGroup at inDegrees(Graph.java:701))
(8/60) [DEPLOYING]
19:29:15,074 INFO org.apache.flink.runtime.taskmanager.Task
- CoGroup (CoGroup at inDegrees(Graph.java:701)) (8/60) switched to RUNNING
19:29:15,526 INFO org.apache.flink.runtime.taskmanager.Task
- CHAIN FlatMap (FlatMap at fromDataSet(Graph.java:171)) -> Combine(Distinct
at fromDataSet(Graph.java:171)) (58/60) switched to RUNNING
19:29:15,567 INFO org.apache.flink.runtime.taskmanager.Task
- CHAIN FlatMap (FlatMap at fromDataSet(Graph.java:171)) -> Combine(Distinct
at fromDataSet(Graph.java:171)) (2/60) switched to RUNNING
19:29:28,270 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Received task Reduce (Distinct at fromDataSet(Graph.java:171)) (4/60)
19:29:28,352 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Received task Reduce (Distinct at fromDataSet(Graph.java:171)) (2/60)
19:29:28,399 INFO org.apache.flink.runtime.taskmanager.Task
- Loading JAR files for task Reduce (Distinct at fromDataSet(Graph.java:171))
(4/60)
19:29:28,399 INFO org.apache.flink.runtime.taskmanager.Task
- Loading JAR files for task Reduce (Distinct at fromDataSet(Graph.java:171))
(2/60)
19:29:28,494 INFO org.apache.flink.runtime.taskmanager.Task
- Registering task at network: Reduce (Distinct at
fromDataSet(Graph.java:171)) (2/60) [DEPLOYING]
19:29:28,481 INFO org.apache.flink.runtime.taskmanager.Task
- Registering task at network: Reduce (Distinct at
fromDataSet(Graph.java:171)) (4/60) [DEPLOYING]
19:29:28,546 INFO org.apache.flink.runtime.taskmanager.Task
- Reduce (Distinct at fromDataSet(Graph.java:171)) (4/60) switched to RUNNING
19:29:28,547 INFO org.apache.flink.runtime.taskmanager.Task
- Reduce (Distinct at fromDataSet(Graph.java:171)) (2/60) switched to RUNNING
19:31:25,216 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Discarding the results produced by task execution
bc512a37f814a788b13f1f8528127f47
19:31:30,310 ERROR org.apache.flink.runtime.operators.RegularPactTask
- Error in task code: Reduce (Distinct at fromDataSet(Graph.java:171)) (4/60)
java.lang.Exception: The data preparation for task 'Reduce (Distinct at
fromDataSet(Graph.java:171))' , caused an error: Error obtaining the sorted
input: Thread 'SortMerger Reading Thread' terminated due to an exception: Error
at remote task manager 'wally019.cit.tu-berlin.de/130.149.249.29:56160'.
at
org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:471)
at
org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.RuntimeException: Error obtaining the sorted input: Thread
'SortMerger Reading Thread' terminated due to an exception: Error at remote
task manager 'wally019.cit.tu-berlin.de/130.149.249.29:56160'.
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger.getIterator(UnilateralSortMerger.java:607)
at
org.apache.flink.runtime.operators.RegularPactTask.getInput(RegularPactTask.java:1145)
at
org.apache.flink.runtime.operators.GroupReduceDriver.prepare(GroupReduceDriver.java:94)
at
org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:466)
... 3 more
Caused by: java.io.IOException: Thread 'SortMerger Reading Thread' terminated
due to an exception: Error at remote task manager
'wally019.cit.tu-berlin.de/130.149.249.29:56160'.
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:784)
Caused by:
org.apache.flink.runtime.io.network.netty.exception.RemoteTransportException:
Error at remote task manager 'wally019.cit.tu-berlin.de/130.149.249.29:56160'.
at
org.apache.flink.runtime.io.network.netty.PartitionRequestClientHandler.decodeMsg(PartitionRequestClientHandler.java:239)
at
org.apache.flink.runtime.io.network.netty.PartitionRequestClientHandler.channelRead(PartitionRequestClientHandler.java:162)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:722)
Caused by:
org.apache.flink.runtime.io.network.partition.ProducerFailedException:
java.lang.IndexOutOfBoundsException: Index: 2856, Size: 0
at
org.apache.flink.runtime.io.network.netty.PartitionRequestQueue.writeAndFlushNextMessageIfPossible(PartitionRequestQueue.java:164)
at
org.apache.flink.runtime.io.network.netty.PartitionRequestQueue.userEventTriggered(PartitionRequestQueue.java:96)
at
io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:308)
at
io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:294)
at
io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108)
at
io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:308)
at
io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:294)
at
io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108)
at
io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:308)
at
io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:294)
at
io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108)
at
io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:308)
at
io.netty.channel.AbstractChannelHandlerContext.access$500(AbstractChannelHandlerContext.java:32)
at
io.netty.channel.AbstractChannelHandlerContext$6.run(AbstractChannelHandlerContext.java:299)
at
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
... 2 more
Caused by: java.lang.IndexOutOfBoundsException: Index: 2856, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:604)
at java.util.ArrayList.get(ArrayList.java:382)
at
org.apache.flink.runtime.io.network.partition.SpillableSubpartitionView.getNextBuffer(SpillableSubpartitionView.java:78)
at
org.apache.flink.runtime.io.network.partition.consumer.LocalInputChannel.getNextLookAhead(LocalInputChannel.java:256)
at
org.apache.flink.runtime.io.network.partition.consumer.LocalInputChannel.getNextBuffer(LocalInputChannel.java:166)
at
org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate.getNextBufferOrEvent(SingleInputGate.java:407)
at
org.apache.flink.runtime.io.network.api.reader.AbstractRecordReader.getNextRecord(AbstractRecordReader.java:76)
at
org.apache.flink.runtime.io.network.api.reader.MutableRecordReader.next(MutableRecordReader.java:34)
at
org.apache.flink.runtime.operators.util.ReaderIterator.next(ReaderIterator.java:73)
at
org.apache.flink.runtime.operators.FlatMapDriver.run(FlatMapDriver.java:101)
at
org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:496)
at
org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
... 1 more
19:31:30,382 ERROR org.apache.flink.runtime.operators.RegularPactTask
- Error in task code: CoGroup (CoGroup at outDegrees(Graph.java:671)) (5/60)
java.lang.Exception: The data preparation for task 'CoGroup (CoGroup at
outDegrees(Graph.java:671))' , caused an error: Error obtaining the sorted
input: Thread 'SortMerger Reading Thread' terminated due to an exception: Fatal
error at remote task manager 'wally019.cit.tu-berlin.de/130.149.249.29:56160'.
at
org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:471)
at
org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.RuntimeException: Error obtaining the sorted input: Thread
'SortMerger Reading Thread' terminated due to an exception: Fatal error at
remote task manager 'wally019.cit.tu-berlin.de/130.149.249.29:56160'.
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger.getIterator(UnilateralSortMerger.java:607)
at
org.apache.flink.runtime.operators.RegularPactTask.getInput(RegularPactTask.java:1145)
at
org.apache.flink.runtime.operators.CoGroupDriver.prepare(CoGroupDriver.java:98)
at
org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:466)
... 3 more
Caused by: java.io.IOException: Thread 'SortMerger Reading Thread' terminated
due to an exception: Fatal error at remote task manager
'wally019.cit.tu-berlin.de/130.149.249.29:56160'.
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:784)
Caused by:
org.apache.flink.runtime.io.network.netty.exception.RemoteTransportException:
Fatal error at remote task manager
'wally019.cit.tu-berlin.de/130.149.249.29:56160'.
at
org.apache.flink.runtime.io.network.netty.PartitionRequestClientHandler.decodeMsg(PartitionRequestClientHandler.java:227)
at
org.apache.flink.runtime.io.network.netty.PartitionRequestClientHandler.channelRead(PartitionRequestClientHandler.java:162)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.io.IOException: Index: 166, Size: 0
at
org.apache.flink.runtime.io.network.netty.PartitionRequestQueue.writeAndFlushNextMessageIfPossible(PartitionRequestQueue.java:197)
at
org.apache.flink.runtime.io.network.netty.PartitionRequestQueue.channelWritabilityChanged(PartitionRequestQueue.java:135)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:438)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
at
io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:438)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
at
io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:438)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
at
io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:438)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
at
io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:862)
at
io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:557)
at
io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:523)
at
io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:196)
at
io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:260)
at
io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:334)
at
io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:317)
at
io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:750)
at
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.forceFlush(AbstractNioChannel.java:309)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:519)
... 5 more
Caused by: java.lang.IndexOutOfBoundsException: Index: 166, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:604)
at java.util.ArrayList.get(ArrayList.java:382)
at
org.apache.flink.runtime.io.network.partition.SpillableSubpartitionView.getNextBuffer(SpillableSubpartitionView.java:78)
at
org.apache.flink.runtime.io.network.netty.PartitionRequestQueue$SequenceNumberingSubpartitionView.getNextBuffer(PartitionRequestQueue.java:295)
at
org.apache.flink.runtime.io.network.netty.PartitionRequestQueue.writeAndFlushNextMessageIfPossible(PartitionRequestQueue.java:152)
... 27 more
19:31:31,745 INFO org.apache.flink.runtime.taskmanager.Task
- Reduce (Distinct at fromDataSet(Graph.java:171)) (4/60) switched to FAILED
with exception.
java.lang.Exception: The data preparation for task 'Reduce (Distinct at
fromDataSet(Graph.java:171))' , caused an error: Error obtaining the sorted
input: Thread 'SortMerger Reading Thread' terminated due to an exception: Error
at remote task manager 'wally019.cit.tu-berlin.de/130.149.249.29:56160'.
at
org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:471)
..... goes on like this; then cancels the remaining tasks...
> Index Out of Bounds Exception
> -----------------------------
>
> Key: FLINK-2412
> URL: https://issues.apache.org/jira/browse/FLINK-2412
> Project: Flink
> Issue Type: Bug
> Components: Local Runtime
> Affects Versions: 0.10
> Reporter: Andra Lungu
>
> When running a code as simple as:
> {noformat}
> ExecutionEnvironment env =
> ExecutionEnvironment.getExecutionEnvironment();
> DataSet<Edge<String, NullValue>> edges = getEdgesDataSet(env);
> Graph<String, NullValue, NullValue> graph =
> Graph.fromDataSet(edges, env);
> DataSet<Tuple2<String, Long>> degrees = graph.getDegrees();
> degrees.writeAsCsv(outputPath, "\n", " ");
> env.execute();
> on the Freindster data set:
> https://snap.stanford.edu/data/com-Friendster.html; on 30 Wally nodes
>
> I get the following exception:
> java.lang.Exception: The data preparation for task 'CoGroup (CoGroup at
> inDegrees(Graph.java:701))' , caused an error: Error obtaining the sorted
> input: Thread 'SortMerger Reading Thread' terminated due to an exception:
> Fatal error at remote task manager
> 'wally028.cit.tu-berlin.de/130.149.249.38:53730'.
> at
> org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:471)
> at
> org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: java.lang.RuntimeException: Error obtaining the sorted input:
> Thread 'SortMerger Reading Thread' terminated due to an exception: Fatal
> error at remote task manager 'wally028.cit.tu-berlin.de/130.149.249.38:53730'.
> at
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger.getIterator(UnilateralSortMerger.java:607)
> at
> org.apache.flink.runtime.operators.RegularPactTask.getInput(RegularPactTask.java:1145)
> at
> org.apache.flink.runtime.operators.CoGroupDriver.prepare(CoGroupDriver.java:98)
> at
> org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:466)
> ... 3 more
> Caused by: java.io.IOException: Thread 'SortMerger Reading Thread' terminated
> due to an exception: Fatal error at remote task manager
> 'wally028.cit.tu-berlin.de/130.149.249.38:53730'.
> at
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:784)
> Caused by:
> org.apache.flink.runtime.io.network.netty.exception.RemoteTransportException:
> Fatal error at remote task manager
> 'wally028.cit.tu-berlin.de/130.149.249.38:53730'.
> at
> org.apache.flink.runtime.io.network.netty.PartitionRequestClientHandler.decodeMsg(PartitionRequestClientHandler.java:227)
> at
> org.apache.flink.runtime.io.network.netty.PartitionRequestClientHandler.channelRead(PartitionRequestClientHandler.java:162)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
> at
> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: java.io.IOException: Index: 133, Size: 0
> {noformat}
> Code works fine for the twitter data set, for instance, which is bigger in
> size, but contains less vertices.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
