date:20150224

Re: Could not build up connection to JobManager

2015-02-24 Thread Stephan Ewen

BTW: Does still work if you enter localhost for jobmanager.rpc.address
in your flink-conf.yaml ?

On Tue, Feb 24, 2015 at 7:50 PM, Stephan Ewen se...@apache.org wrote:

Hi!

I think that this is a problem in the current master (probably in there
since a few days ago). I am fixing it...

Thanks for reporting it!

Stephan

On Tue, Feb 24, 2015 at 6:52 PM, Stephan Ewen se...@apache.org wrote:

Hi Dulaj!

The log suggests that the JobManager binds itself to the IP
address 10.216.192.98 and the WebClient runs at 127.0.0.1

The 127.0.0.1 actor system cannot connect to the 10.216.192.98.

Let me verify whether this is a quirk of your particular setup, or a bug
recently introduces in the 0.9-SNAPSHOT.

Does the command line work for you? (bin/flink run jar)

taskmanager.numberOfTaskSlots: -1 is also okay, this will mean that the
default of '1' is used.

Greetings,
Stephan

On Tue, Feb 24, 2015 at 5:18 PM, Dulaj Viduranga vidura...@icloud.com
wrote:

Is taskmanager.numberOfTaskSlots: -1 normal?

On Feb 24, 2015, at 9:44 PM, Robert Metzger rmetz...@apache.org
wrote:

Hi,
I could not find the logfiles attached to your mails. I think the
mailinglists are not accepting attachments.
Can you put the logs on gist.github.com?

The configuration values are documented here:
http://flink.apache.org/docs/0.8/config.html
For the webclient's port its called webclient.port

On Tue, Feb 24, 2015 at 5:04 PM, Dulaj Viduranga vidura...@icloud.com

wrote:

I tried to kill the job manager manually in the terminal and start it
again but no luck. Also could you tell me if it’s possible to change
webclient’s port (8080) ?

On Feb 24, 2015, at 1:41 PM, Stephan Ewen se...@apache.org wrote:

Hey Dulaj!

As a contributor, I would go against the latest version, which is
0.9-SNAPSHOT.

It may be in your case that the JobManager actor is down, but the
process
still lingers. (BTW: I have a patch pending that makes sure the
process
disappears when the actor via down).

Could you have a look at the log
flink-user-jobmanager-host-.log
and
see if there are any errors logged?

Greetings,
Stephan
Am 24.02.2015 06:29 schrieb Dulaj Viduranga vidura...@icloud.com
:

The JobManager seems to run fine. I don't know. When I tried to run
start-local.sh again, It shows the PID of the running JobManager and
also
:8081 runs fine. I want to contribute to the project and I could
get a
little boost if I could see the capabilities of FLINK. :)
Will it be OK to use 0.8.1 as a developer?

On Feb 24, 2015, at 04:15 AM, Stephan Ewen se...@apache.org
wrote:

Hi Dulaj,

That error message indicates that the JobManager is not running.
Are you
sure that the JobManager runs properly? Anything in the JobManager
logs?

BTW: The 0.9 branch is under heavy development / changes. That is
why it
may behave a bit different on different days right now. I would
recommend
to use the 0.8.1 release for a stable experience.

Greetings,
Stephan

On Mon, Feb 23, 2015 at 7:39 PM, Robert Metzger
rmetz...@apache.org
wrote:

Thank you for the quick reply.

The log you've send is from the webclient. Can you also send the
log of
the

JobManager?

On Mon, Feb 23, 2015 at 7:28 PM, Dulaj Viduranga
vidura...@icloud.com

wrote:

Yes. It seams it is not a problem with the arguments. I tried two
days

but

different error occurs. It seams the web client can’t connect to
the
job

manager although it is running

Right now, I can’t even get the webclient to run.

./bin/start-webclient.sh

executes fine but I cannot connect to localhost:8080 (even with
telnet
or

curl)

Here is the log for jobManager

23:22:31,933 INFO org.apache.flink.client.web.WebInterfaceServer

- Setting up web frontend server, using web-root directory

'jar:

file:/Users/Vidura/Documents/Development/flink/flink-dist/target/flink-0.9-SNAPSHOT-bin/flink-0.9-SNAPSHOT/lib/flink-clients-0.9-SNAPSHOT.jar!/web-docs
'.

23:22:31,934 INFO org.apache.flink.client.web.WebInterfaceServer

- Web frontend server will store temporary files in

'/var/folders/3_/7gzbv7ks7q71lpm5d9hzrw2cgn/T', uploaded jobs
in

'/var/folders/3_/7gzbv7ks7q71lpm5d9hzrw2cgn/T/webclient-jobs',

plan-json-dumps in

'/var/folders/3_/7gzbv7ks7q71lpm5d9hzrw2cgn/T/webclient-plans'.

23:22:31,934 INFO org.apache.flink.client.web.WebInterfaceServer

- Web-frontend will submit jobs to nephele job-manager on

localhost,

port 6123.

23:22:32,580 INFO akka.event.slf4j.Slf4jLogger

- Slf4jLogger started

23:22:32,625 INFO Remoting

- Starting remoting

23:22:32,838 INFO Remoting

- Remoting started; listening on addresses :[akka.tcp://

JobsInfoServletActorSystem@127.0.0.1:51517]

23:23:48,119 WARN Remoting

- Tried to associate with unreachable remote address [akka.tcp://

[jira] [Created] (FLINK-1608) TaskManagers may pick wrong network interface when starting before JobManager

2015-02-24 Thread Stephan Ewen (JIRA)

Stephan Ewen created FLINK-1608:
---

 Summary: TaskManagers may pick wrong network interface when 
starting before JobManager
 Key: FLINK-1608
 URL: https://issues.apache.org/jira/browse/FLINK-1608
 Project: Flink
  Issue Type: Bug
  Components: TaskManager
Affects Versions: 0.9
Reporter: Stephan Ewen
 Fix For: 0.9


The taskmanagers use a NetUtils routine to find an interface that lets them 
talk to the Jobmanager. However, if the JobManager is not online yet, they fall 
back to localhost.

In cases where the TaskManagers start faster than the JobManager, they pick the 
wrong hostname and interface.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Could not build up connection to JobManager

2015-02-24 Thread Robert Metzger

Hi,
I could not find the logfiles attached to your mails. I think the
mailinglists are not accepting attachments.
Can you put the logs on gist.github.com?

The configuration values are documented here:
http://flink.apache.org/docs/0.8/config.html
For the webclient's port its called webclient.port

On Tue, Feb 24, 2015 at 5:04 PM, Dulaj Viduranga vidura...@icloud.com
wrote:

I tried to kill the job manager manually in the terminal and start it
again but no luck. Also could you tell me if it’s possible to change
webclient’s port (8080) ?

On Feb 24, 2015, at 1:41 PM, Stephan Ewen se...@apache.org wrote:

Hey Dulaj!

As a contributor, I would go against the latest version, which is
0.9-SNAPSHOT.

It may be in your case that the JobManager actor is down, but the process
still lingers. (BTW: I have a patch pending that makes sure the process
disappears when the actor via down).

Could you have a look at the log flink-user-jobmanager-host-.log
and
see if there are any errors logged?

Greetings,
Stephan
Am 24.02.2015 06:29 schrieb Dulaj Viduranga vidura...@icloud.com:

The JobManager seems to run fine. I don't know. When I tried to run
start-local.sh again, It shows the PID of the running JobManager and
also
:8081 runs fine. I want to contribute to the project and I could get a
little boost if I could see the capabilities of FLINK. :)
Will it be OK to use 0.8.1 as a developer?

On Feb 24, 2015, at 04:15 AM, Stephan Ewen se...@apache.org wrote:

Hi Dulaj,

That error message indicates that the JobManager is not running. Are you
sure that the JobManager runs properly? Anything in the JobManager logs?

BTW: The 0.9 branch is under heavy development / changes. That is why it
may behave a bit different on different days right now. I would
recommend
to use the 0.8.1 release for a stable experience.

Greetings,
Stephan

On Mon, Feb 23, 2015 at 7:39 PM, Robert Metzger rmetz...@apache.org
wrote:

Thank you for the quick reply.

The log you've send is from the webclient. Can you also send the log of
the

JobManager?

On Mon, Feb 23, 2015 at 7:28 PM, Dulaj Viduranga vidura...@icloud.com

wrote:

Yes. It seams it is not a problem with the arguments. I tried two days

but

different error occurs. It seams the web client can’t connect to the
job

manager although it is running

Right now, I can’t even get the webclient to run.

./bin/start-webclient.sh

executes fine but I cannot connect to localhost:8080 (even with telnet
or

curl)

Here is the log for jobManager

23:22:31,933 INFO org.apache.flink.client.web.WebInterfaceServer

- Setting up web frontend server, using web-root directory

'jar:

file:/Users/Vidura/Documents/Development/flink/flink-dist/target/flink-0.9-SNAPSHOT-bin/flink-0.9-SNAPSHOT/lib/flink-clients-0.9-SNAPSHOT.jar!/web-docs
'.

23:22:31,934 INFO org.apache.flink.client.web.WebInterfaceServer

- Web frontend server will store temporary files in

'/var/folders/3_/7gzbv7ks7q71lpm5d9hzrw2cgn/T', uploaded jobs in

'/var/folders/3_/7gzbv7ks7q71lpm5d9hzrw2cgn/T/webclient-jobs',

plan-json-dumps in

'/var/folders/3_/7gzbv7ks7q71lpm5d9hzrw2cgn/T/webclient-plans'.

23:22:31,934 INFO org.apache.flink.client.web.WebInterfaceServer

- Web-frontend will submit jobs to nephele job-manager on

localhost,

port 6123.

23:22:32,580 INFO akka.event.slf4j.Slf4jLogger

- Slf4jLogger started

23:22:32,625 INFO Remoting

- Starting remoting

23:22:32,838 INFO Remoting

- Remoting started; listening on addresses :[akka.tcp://

JobsInfoServletActorSystem@127.0.0.1:51517]

23:23:48,119 WARN Remoting

- Tried to associate with unreachable remote address [akka.tcp://

flink@10.218.98.169:6123]. Address is now gated for 5000 ms, all

messages

to this address will be delivered to dead letters. Reason: Operation

timed

out: /10.218.98.169:6123

23:23:48,124 ERROR org.apache.flink.client.WebFrontend

- Unexpected exception: Could not find job manager at specified

address akka.flink@10.218.98.169:6123/user/jobmanager'tcp://
flink@10.218.98.169:6123/user/jobmanager.

java.lang.RuntimeException: Could not find job manager at specified

address akka.flink@10.218.98.169:6123/user/jobmanager'tcp://
flink@10.218.98.169:6123/user/jobmanager.

org.apache.flink.client.web.JobsInfoServlet.init(JobsInfoServlet.java:82)

org.apache.flink.client.web.WebInterfaceServer.init(WebInterfaceServer.java:158)

at org.apache.flink.client.WebFrontend.main(WebFrontend.java:74)

On Feb 23, 2015, at 11:46 PM, Robert Metzger rmetz...@apache.org

wrote:

Hi,

you said in the other email thread that the error only occurs for

Wordcount, not for Kmeans.

Can you copy me the commands for both examples?

I can not really believe that there is a difference

Flink Master broken...

2015-02-24 Thread Matthias J. Sax

Hi,

I just pulled the latest version flink master and I got a compilation
error. I tracked the bug down. It seems that the following commit moved
a class without changing the package statement...


Cheers,
  Matthias


commit 354efec0f9da0fa03ea9b337b02a1a2a03a9ac16
Author: Robert Metzger rmetz...@apache.org 2015-01-26 17:54:25
Committer: Robert Metzger rmetz...@apache.org 2015-02-18 15:52:08
Parent: 7407076d3990752eb5fa4072cd036efd2f656cbc ([FLINK-1369] [types]
Add support for Subclasses, Interfaces, Abstract Classes.)
Branches: master, flink/master

[FLINK-1417] Automatically register types with Kryo


 diff --git 
 a/flink-java/src/test/java/org/apache/flink/api/java/typeutils/runtime/KryoWithCustomSerializersTest.java
  
 b/flink-java/src/test/java/org/apache/flink/api/java/typeutils/runtime/kryo/KryoWithCustomSerializersTest.java
 similarity index 96%
 rename from 
 flink-java/src/test/java/org/apache/flink/api/java/typeutils/runtime/KryoWithCustomSerializersTest.java
 rename to 
 flink-java/src/test/java/org/apache/flink/api/java/typeutils/runtime/kryo/KryoWithCustomSerializersTest.java
 index 7020d80..155010e 100644
 --- 
 a/flink-java/src/test/java/org/apache/flink/api/java/typeutils/runtime/KryoWithCustomSerializersTest.java
 +++ 
 b/flink-java/src/test/java/org/apache/flink/api/java/typeutils/runtime/kryo/KryoWithCustomSerializersTest.java
 @@ -50,7 +50,7 @@
   @Override
   protected T TypeSerializerT createSerializer(ClassT type) {
   ExecutionConfig conf = new ExecutionConfig();
 - conf.registerKryoSerializer(LocalDate.class, 
 LocalDateSerializer.class);
 + conf.registerTypeWithKryoSerializer(LocalDate.class, 
 LocalDateSerializer.class);
   TypeInformationT typeInfo = new GenericTypeInfoT(type);
   return typeInfo.createSerializer(conf);
   }




signature.asc
Description: OpenPGP digital signature

Re: Could not build up connection to JobManager

2015-02-24 Thread Dulaj Viduranga

Is taskmanager.numberOfTaskSlots: -1 normal?

On Feb 24, 2015, at 9:44 PM, Robert Metzger rmetz...@apache.org wrote: