[
https://issues.apache.org/jira/browse/FLINK-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sunjincheng updated FLINK-11972:
--------------------------------
Description:
Since the difference between 1.8.0 and 1.7.x is that 1.8.x does not put the
`hadoop-shaded` JAR integrated into the dist. It will cause an error when the
end-to-end test cannot be found with `Hadoop` Related classes, such as:
`java.lang.NoClassDefFoundError: Lorg/apache/hadoop/fs/FileSystem`. So we need
to improve the end-to-end test script, or explicitly stated in the README, i.e.
end-to-end test need to add `flink-shaded-hadoop2-uber-XXXX.jar` to the
classpath. So, we will get the exception something like:
{code:java}
[INFO] 3 instance(s) of taskexecutor are already running on
jinchengsunjcs-iMac.local.
Starting taskexecutor daemon on host jinchengsunjcs-iMac.local.
java.lang.NoClassDefFoundError: Lorg/apache/hadoop/fs/FileSystem;
at java.lang.Class.getDeclaredFields0(Native Method)
at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
at java.lang.Class.getDeclaredFields(Class.java:1916)
at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:72)
at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.clean(StreamExecutionEnvironment.java:1558)
at
org.apache.flink.streaming.api.datastream.DataStream.clean(DataStream.java:185)
at
org.apache.flink.streaming.api.datastream.DataStream.addSink(DataStream.java:1227)
at
org.apache.flink.streaming.tests.BucketingSinkTestProgram.main(BucketingSinkTestProgram.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:423)
at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:813)
at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:287)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
at
org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
at org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
at
org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.FileSystem
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 22 more
Job () is running.{code}
So, I think we can import the test script or import the README.
What do you think?
was:
When I did the end-to-end test under Mac OS, I found the following two problems:
1. The verification returned for different `minikube status` is not enough for
the robustness. The strings returned by different versions of different
platforms are different. the following misjudgment is caused:
When the `Command: start_kubernetes_if_not_ruunning failed` error occurs, the
`minikube` has actually started successfully. The core reason is that there is
a bug in the `test_kubernetes_embedded_job.sh` script. The error message as
follows:
!image-2019-03-20-14-02-29-636.png!
!image-2019-03-20-14-04-17-933.png!
{code:java}
Current check logic: echo ${status} | grep -q "minikube: Running cluster:
Running kubectl: Correctly Configured"
==== My local messae====
jinchengsunjcs-iMac:flink-1.8.0 jincheng$ minikube status
host: Running
kubelet: Running
apiserver: Running
kubectl: Correctly Configured: pointing to minikube-vm at 192.168.99.101{code}
So, I think we should improve the check logic of `minikube status`, What do you
think?
> Fix `Command: start_kubernetes_if_not_ruunning failed` error
> ------------------------------------------------------------
>
> Key: FLINK-11972
> URL: https://issues.apache.org/jira/browse/FLINK-11972
> Project: Flink
> Issue Type: Bug
> Components: Tests
> Affects Versions: 1.8.0, 1.9.0
> Reporter: sunjincheng
> Assignee: Hequn Cheng
> Priority: Major
> Attachments: image-2019-03-20-14-04-17-933.png
>
>
> Since the difference between 1.8.0 and 1.7.x is that 1.8.x does not put the
> `hadoop-shaded` JAR integrated into the dist. It will cause an error when
> the end-to-end test cannot be found with `Hadoop` Related classes, such as:
> `java.lang.NoClassDefFoundError: Lorg/apache/hadoop/fs/FileSystem`. So we
> need to improve the end-to-end test script, or explicitly stated in the
> README, i.e. end-to-end test need to add `flink-shaded-hadoop2-uber-XXXX.jar`
> to the classpath. So, we will get the exception something like:
> {code:java}
> [INFO] 3 instance(s) of taskexecutor are already running on
> jinchengsunjcs-iMac.local.
> Starting taskexecutor daemon on host jinchengsunjcs-iMac.local.
> java.lang.NoClassDefFoundError: Lorg/apache/hadoop/fs/FileSystem;
> at java.lang.Class.getDeclaredFields0(Native Method)
> at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
> at java.lang.Class.getDeclaredFields(Class.java:1916)
> at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:72)
> at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.clean(StreamExecutionEnvironment.java:1558)
> at
> org.apache.flink.streaming.api.datastream.DataStream.clean(DataStream.java:185)
> at
> org.apache.flink.streaming.api.datastream.DataStream.addSink(DataStream.java:1227)
> at
> org.apache.flink.streaming.tests.BucketingSinkTestProgram.main(BucketingSinkTestProgram.java:80)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)
> at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:423)
> at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:813)
> at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:287)
> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
> at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
> at
> org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.FileSystem
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 22 more
> Job () is running.{code}
> So, I think we can import the test script or import the README.
> What do you think?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
