Re: Connecting the channel failed: Connection refused

2015-06-25 Thread Aaron Jackson
So the JobManager was running on host1. This also explains why I didn't see the problem until I had asked for a sizeable degree of parallelism since it probably never assigned a task to host3. Thanks for your help On Thu, Jun 25, 2015 at 3:34 AM, Stephan Ewen se...@apache.org wrote: Nice!

Re: Documentation Error

2015-06-25 Thread Robert Metzger
Hey Maximilian Alber, I don't know if you are interested in contributing in Flink, but if you would like to, these small fixes to the documentation are really helpful for us! Its actually quite easy to work with the documentation locally. It is located in the docs/ directory of the Flink source.

Re: writeAsCsv on HDFS

2015-06-25 Thread Chiwan Park
It represents the folder containing the hadoop config files. :) Regards, Chiwan Park On Jun 25, 2015, at 10:07 PM, Flavio Pompermaier pomperma...@okkam.it wrote: fs.hdfs.hadoopconf represents the folder containing the hadoop config files (*-site.xml) or just one specific hadoop config

Re: writeAsCsv on HDFS

2015-06-25 Thread Flavio Pompermaier
Could you describe it better with an example please? Why Flink doesn't load automatically the properties of the hadoop conf files within the jar? On Thu, Jun 25, 2015 at 2:55 PM, Robert Metzger rmetz...@apache.org wrote: Hi, Flink is not loading the Hadoop configuration from the classloader.

Re: writeAsCsv on HDFS

2015-06-25 Thread Flavio Pompermaier
Do I have to put the hadoop conf file on each task manager or just on the job-manager? On Thu, Jun 25, 2015 at 3:12 PM, Chiwan Park chiwanp...@apache.org wrote: It represents the folder containing the hadoop config files. :) Regards, Chiwan Park On Jun 25, 2015, at 10:07 PM, Flavio

Re: Documentation Error

2015-06-25 Thread Maximilian Alber
Something different. I just read through the Spark documentation and yours. While the Spark one is quite unstructured and easy to understand, yours is structured and really detailed. It's great that you have that in depth documentation, but I would recommend you to make a boiled-down page with

Re: Documentation Error

2015-06-25 Thread Maximilian Michels
Thanks for noticing, Chiwan. I have the feeling this problem arose when the website was updated. The problem about linking documentation pages from the main website is that it is currently hard to go back to the main web site from the documentation (the nav and URL changes). However, now we are

Re: Documentation Error

2015-06-25 Thread Maximilian Michels
Thanks Max. I think the documentation has grown a lot and needs an overhaul. We should remove the unnecessary non-Flink-related stuff (e.g. configuring ssh keys in the setup guide). I like your idea of having an essential guide that just covers the basics for people already familiar with other big

Re: Documentation Error

2015-06-25 Thread Ufuk Celebi
On 25 Jun 2015, at 14:31, Maximilian Michels m...@apache.org wrote: Thanks for noticing, Chiwan. I have the feeling this problem arose when the website was updated. The problem about linking documentation pages from the main website is that it is currently hard to go back to the main web

Re: Documentation Error

2015-06-25 Thread Maximilian Alber
Another one: in the stream guide under Connecting to the outside world Sources I guess one by should be a be. http://ci.apache.org/projects/flink/flink-docs-master/apis/streaming_guide.html On Thu, Jun 25, 2015 at 2:42 PM, Maximilian Michels m...@apache.org wrote: Thanks Max. I think the

writeAsCsv on HDFS

2015-06-25 Thread Flavio Pompermaier
Hi to all, I'm experiencing some problem in writing a file as csv on HDFS with flink 0.9.0. The code I use is myDataset.writeAsCsv(new Path(hdfs:///tmp, myFile.csv).toString()); If I run the job from Eclipse everything works fine but when I deploy the job on the cluster (cloudera 5.1.3) I

Re: writeAsCsv on HDFS

2015-06-25 Thread Stephan Ewen
You could also just qualify the HDFS URL, if that is simpler (put host and port of the namenode in there): hdfs://myhost:40010/path/to/file On Thu, Jun 25, 2015 at 3:20 PM, Robert Metzger rmetz...@apache.org wrote: You have to put it into all machines On Thu, Jun 25, 2015 at 3:17 PM, Flavio

Re: Connecting the channel failed: Connection refused

2015-06-25 Thread Stephan Ewen
That makes perfect sense, thanks! Am 25.06.2015 21:39 schrieb Aaron Jackson ajack...@pobox.com: So the JobManager was running on host1. This also explains why I didn't see the problem until I had asked for a sizeable degree of parallelism since it probably never assigned a task to host3.

ArrayIndexOutOfBoundsException when running job from JAR

2015-06-25 Thread Mihail Vieru
Hi, I get an ArrayIndexOutOfBoundsException when I run my job from a JAR in the CLI. This doesn't occur in the IDE. I've build the JAR using the maven-shade-plugin and the pom.xml configuration Robert has provided here:

Re: Cannot instantiate Mysql connection

2015-06-25 Thread Stephan Ewen
Good to hear it works. Libraries, class-loading, and initialization seems to be one of the things that remains tricky once one switches to distributed processed. On Thu, Jun 25, 2015 at 10:58 AM, Flavio Pompermaier pomperma...@okkam.it wrote: Sorry for the late response but I was on vacation

Re: Documentation Error

2015-06-25 Thread Maximilian Alber
Another one: on http://ci.apache.org/projects/flink/flink-docs-master/faq.html in the What is parallelism? How do I set it? Section the links are broken. Cheers, Max On Wed, Jun 24, 2015 at 9:52 AM, Maximilian Michels m...@apache.org wrote: Hi Max, Thanks for noticing! Fixed on the master