[GitHub] flink issue #5528: [FLINK-6489] [FLINK-8696] [shell scripts] Remove JobManag...

2018-02-21 Thread StephanEwen
Github user StephanEwen commented on the issue:

https://github.com/apache/flink/pull/5528
  
Merged in 74c5570c9fa94d35e47899b0dcdc74a5d18750f6


---


[GitHub] flink issue #5528: [FLINK-6489] [FLINK-8696] [shell scripts] Remove JobManag...

2018-02-21 Thread StephanEwen
Github user StephanEwen commented on the issue:

https://github.com/apache/flink/pull/5528
  
Tested it locally again, works (with current flip-6 Web UI at 9065)

Adjusted the test-infra scripts to remove local mode. Tose are only 
executed on Travis, so waiting until the CI build passes...


---


[GitHub] flink issue #5528: [FLINK-6489] [FLINK-8696] [shell scripts] Remove JobManag...

2018-02-21 Thread zentol
Github user zentol commented on the issue:

https://github.com/apache/flink/pull/5528
  
I tried it again with java 8. The job submission through the client now 
also works for me, and the warning is no longer printed.


---


[GitHub] flink issue #5528: [FLINK-6489] [FLINK-8696] [shell scripts] Remove JobManag...

2018-02-21 Thread StephanEwen
Github user StephanEwen commented on the issue:

https://github.com/apache/flink/pull/5528
  
Okay, after a quick check with @zentol 
  - Some issues were caused by Java 9. Apparently Flink on Windows is 
incompatible with Java 9
  - We need to change the JobManager and TaskManager to use the flip-6 
entry points.
  - The `common.sh` in `flink-end-to-end-tests` needs to be updated.




---


[GitHub] flink issue #5528: [FLINK-6489] [FLINK-8696] [shell scripts] Remove JobManag...

2018-02-21 Thread StephanEwen
Github user StephanEwen commented on the issue:

https://github.com/apache/flink/pull/5528
  
Hmm, weird, I tried it out under Windows (7) two days ago and it worked 
quite well.

Let me check into what you found there...


---


[GitHub] flink issue #5528: [FLINK-6489] [FLINK-8696] [shell scripts] Remove JobManag...

2018-02-21 Thread zentol
Github user zentol commented on the issue:

https://github.com/apache/flink/pull/5528
  
The JM/TM can connect to each other though, and the job submission through 
the UI is also working. Metrics appear to be broken though.


---


[GitHub] flink issue #5528: [FLINK-6489] [FLINK-8696] [shell scripts] Remove JobManag...

2018-02-21 Thread zentol
Github user zentol commented on the issue:

https://github.com/apache/flink/pull/5528
  
I've tried using the start-local.bat scripts and found 2 issues:
1)
The following warning is printed in the JobManager and TaskManager windows:
```
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by 
org.apache.hadoop.security.authentication.util.KerberosUtil 
(file:/C:/Users/Zento/Documents/GitHub/flink/flink-dist/target/flink-1.5-SNAPSHOT-bin/flink-1.5-SNAPSHOT/lib/flink-shaded-hadoop2-uber-1.5-SNAPSHOT.jar)
 to method sun.security.krb5.Config.getInstance()
WARNING: Please consider reporting this to the maintainers of 
org.apache.hadoop.security.authentication.util.KerberosUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal 
reflective access operations
WARNING: All illegal access operations will be denied in a future release
```
2)
You cannot use them out-of-the-box as they start the old 
JobManager/TaskManager components, but the client works against FLIP-6 by 
default.
3)
After explicitly configuring `mode: "old"` in `flink-conf.yaml` the job 
submissions times out. In the JM/TM logs I found this funky exception:
```
2018-02-21 10:44:09,363 ERROR akka.remote.Remoting  
- [B cannot be cast to [C
java.lang.ClassCastException: [B cannot be cast to [C
at akka.remote.artery.FastHash$.ofString(LruBoundedCache.scala:18)
at 
akka.remote.serialization.ActorRefResolveCache.hash(ActorRefResolveCache.scala:61)
at 
akka.remote.serialization.ActorRefResolveCache.hash(ActorRefResolveCache.scala:55)
at 
akka.remote.artery.LruBoundedCache.getOrCompute(LruBoundedCache.scala:110)
at 
akka.remote.RemoteActorRefProvider.resolveActorRef(RemoteActorRefProvider.scala:403)
at akka.actor.SerializedActorRef.readResolve(ActorRef.scala:433)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown 
Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at java.base/java.io.ObjectStreamClass.invokeReadResolve(Unknown 
Source)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown 
Source)
at java.base/java.io.ObjectInputStream.readObject0(Unknown Source)
at java.base/java.io.ObjectInputStream.defaultReadFields(Unknown 
Source)
at java.base/java.io.ObjectInputStream.readSerialData(Unknown 
Source)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown 
Source)
at java.base/java.io.ObjectInputStream.readObject0(Unknown Source)
at java.base/java.io.ObjectInputStream.readObject(Unknown Source)
at 
akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:328)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at 
akka.serialization.JavaSerializer.fromBinary(Serializer.scala:328)
at 
akka.serialization.Serialization.akka$serialization$Serialization$$deserializeByteArray(Serialization.scala:156)
at 
akka.serialization.Serialization$$anonfun$deserialize$2.apply(Serialization.scala:142)
at scala.util.Try$.apply(Try.scala:192)
at 
akka.serialization.Serialization.deserialize(Serialization.scala:136)
at 
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:30)
at 
akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:64)
at akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:64)
at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:82)
at 
akka.remote.EndpointReader$$anonfun$akka$remote$EndpointReader$$deliverAndAck$1.apply(Endpoint.scala:1047)
at 
akka.remote.EndpointReader$$anonfun$akka$remote$EndpointReader$$deliverAndAck$1.apply(Endpoint.scala:1046)
at scala.collection.Iterator$class.foreach(Iterator.scala:891)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
at 
scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at 
akka.remote.EndpointReader.akka$remote$EndpointReader$$deliverAndAck(Endpoint.scala:1046)
at 
akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:980)
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
at akka.remote.EndpointActor.aroundReceive(Endpoint.scala:446)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at 

[GitHub] flink issue #5528: [FLINK-6489] [FLINK-8696] [shell scripts] Remove JobManag...

2018-02-21 Thread zentol
Github user zentol commented on the issue:

https://github.com/apache/flink/pull/5528
  
There's still a reference to `start-local.sh` in `common.sh`:
```
function start_cluster {
  if [[ "$CLUSTER_MODE" == "local" ]]; then
$FLINK_DIR/bin/start-local.sh
  elif [[ "$CLUSTER_MODE" == "cluster" ]]; then
$FLINK_DIR/bin/start-cluster.sh
  else
echo "Unrecognized cluster mode: $CLUSTER_MODE"
exit
  fi
```


---