Hi guys, I am running into a problem when trying to upload my Beam job to a flink cluster. However, when testing the pipeline using the local cluster there is no problem.
So for extra clarification, I have no problem running: mvn exec:java -D exec.mainClass=com.asml.wfa.swinfra.dataflow.DataFlowModeling -D exec.args= "--runner=FlinkRunner --filesToStage=C:\dev\beam-experiments\target\dataflow-bundled-0.1.jar --inputDirectory=C:\Localdata\LIS\local-lis\input-modeling --outputDirectory=C:\Localdata\LIS\local-lis\output --tempDirectory=C:\Localdata\LIS\local-lis\temp" -P flink-runner However as soon as I add a flink master I run into problems: mvn exec:java -D exec.mainClass=com.asml.wfa.swinfra.dataflow.DataFlowModeling -D exec.args= "--runner=FlinkRunner --filesToStage=C:\dev\beam-experiments\target\dataflow-bundled-0.1.jar --inputDirectory=C:\Localdata\LIS\local-lis\input-modeling --outputDirectory=C:\Localdata\LIS\local-lis\output --tempDirectory=C:\Localdata\LIS\local-lis\temp --flinkMaster=localhost:8081" -P flink-runner The error I get is the following: Pipeline execution failed: Could not retrieve the execution result. (JobID : 0aa23a9193720ecf0bb7442a1ce8037a): Failed to submit JobGraph. [Internal server error., <Exception on server side: [ERROR] java.lang.StackOverflowError [ERROR] at java.io.FileInputStream.read(Unknown Source) [ERROR] at org.apache.flink.core.fs.local.LocalDataInputStream.read(LocalDataInputStream.java:73) [ERROR] at java.io.ObjectInputStream$PeekInputStream.read(Unknown Source) [ERROR] at java.io.ObjectInputStream$PeekInputStream.readFully(Unknown Source) [ERROR] at java.io.ObjectInputStream$BlockDataInputStream.readUTFBody(Unknown Source) [ERROR] at java.io.ObjectInputStream$BlockDataInputStream.readUTF(Unknown Source) [ERROR] at java.io.ObjectInputStream.readString(Unknown Source) [ERROR] at java.io.ObjectInputStream.readObject0(Unknown Source) [ERROR] at java.io.ObjectInputStream.readObject(Unknown Source) [ERROR] at java.util.HashMap.readObject(Unknown Source) [ERROR] at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source) [ERROR] at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) [ERROR] at java.lang.reflect.Method.invoke(Unknown Source) [ERROR] at java.io.ObjectStreamClass.invokeReadObject(Unknown Source) [ERROR] at java.io.ObjectInputStream.readSerialData(Unknown Source) [ERROR] at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) [ERROR] at java.io.ObjectInputStream.readObject0(Unknown Source) [ERROR] at java.io.ObjectInputStream.defaultReadFields(Unknown Source) [ERROR] at java.io.ObjectInputStream.readSerialData(Unknown Source) [ERROR] at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) [ERROR] at java.io.ObjectInputStream.readObject0(Unknown Source) [ERROR] at java.io.ObjectInputStream.defaultReadFields(Unknown Source) [ERROR] at java.io.ObjectInputStream.readSerialData(Unknown Source) [ERROR] at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) [ERROR] at java.io.ObjectInputStream.readObject0(Unknown Source) [ERROR] at java.io.ObjectInputStream.defaultReadFields(Unknown Source) [ERROR] at java.io.ObjectInputStream.readSerialData(Unknown Source) [ERROR] at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) [ERROR] at java.io.ObjectInputStream.readObject0(Unknown Source) [ERROR] at java.io.ObjectInputStream.defaultReadFields(Unknown Source) [ERROR] at java.io.ObjectInputStream.readSerialData(Unknown Source) [ERROR] at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) [ERROR] at java.io.ObjectInputStream.readObject0(Unknown Source) [ERROR] at java.io.ObjectInputStream.readObject(Unknown Source) [ERROR] at java.util.ArrayList.readObject(Unknown Source) [ERROR] at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source) (...) [ERROR] at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source) [ERROR] at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) [ERROR] at java.lang.reflect.Method.invoke(Unknown Source) [ERROR] at java.io.ObjectStreamClass.invokeReadObject(Unknown Source) [ERROR] at java.io.ObjectInputStream.readSerialData(Unknown Source) [ERROR] at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) [ERROR] at java.io.ObjectInputStream.readObject0(Unknown Source) [ERROR] at java.io.ObjectInputStream.defaultReadFields(Unknown Source) [ERROR] at java.io.ObjectInputStream.readSerialData(Unknown Source) [ERROR] at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) [ERROR] at java.io.ObjectInputStream.readObject0(Unknown Source) I looked into the logs of the JobManager for extra information. But unfortunately there is not much I can find, the startup of the JobManager is normal and the first reported error is the StackOverFlowError listed above. It is possible to run the WordCount examples both with and without the "--flinkMaster" argument. Has anybody ran into this error earlier, or does anybody have an idea where I can continue my investigation? Thanks for your help. Regards, Jorik van Kemenade -- The information contained in this communication and any attachments is confidential and may be privileged, and is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. Unless explicitly stated otherwise in the body of this communication or the attachment thereto (if any), the information is provided on an AS-IS basis without any express or implied warranties or liabilities. To the extent you are relying on this information, you are doing so at your own risk. If you are not the intended recipient, please notify the sender immediately by replying to this message and destroy all copies of this message and any attachments. Neither the sender nor the company/group of companies he or she represents shall be liable for the proper and complete transmission of the information contained in this communication, or for any delay in its receipt.
