buildbot failure in on flink-docs-release-0.9

2016-05-31 Thread buildbot 
The Buildbot has detected a new failure on builder flink-docs-release-0.9 while 
building . Full details are available at:
https://ci.apache.org/builders/flink-docs-release-0.9/builds/339

Buildbot URL: https://ci.apache.org/

Buildslave for this Build: tethys_ubuntu

Build Reason: The Nightly scheduler named 'flink-nightly-docs-release-0.9' 
triggered this build
Build Source Stamp: [branch release-0.9] HEAD
Blamelist: 

BUILD FAILED: failed compile

Sincerely,
 -The Buildbot

buildbot success in on flink-docs-release-0.10

2016-05-31 Thread buildbot 
The Buildbot has detected a restored build on builder flink-docs-release-0.10 
while building . Full details are available at:
https://ci.apache.org/builders/flink-docs-release-0.10/builds/224

Buildbot URL: https://ci.apache.org/

Buildslave for this Build: orcus_ubuntu

Build Reason: The Nightly scheduler named 'flink-nightly-docs-release-0.10' 
triggered this build
Build Source Stamp: [branch release-0.10] HEAD
Blamelist: 

Build succeeded!

Sincerely,
 -The Buildbot

flink git commit: [FLINK-3994] [ml, tests] Fix flaky KNN integration tests

2016-05-31 Thread chiwanpark 
Repository: flink
Updated Branches:
  refs/heads/master b849da14c -> aedc0fd48


[FLINK-3994] [ml, tests] Fix flaky KNN integration tests

This closes #2056.


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/aedc0fd4
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/aedc0fd4
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/aedc0fd4

Branch: refs/heads/master
Commit: aedc0fd481a756ef2b1708f896d5500475715232
Parents: b849da1
Author: Chiwan Park 
Authored: Tue May 31 17:50:05 2016 +0900
Committer: Chiwan Park 
Committed: Wed Jun 1 10:23:05 2016 +0900

--
 .../org/apache/flink/ml/nn/KNNITSuite.scala | 22 ++--
 1 file changed, 16 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/flink/blob/aedc0fd4/flink-libraries/flink-ml/src/test/scala/org/apache/flink/ml/nn/KNNITSuite.scala
--
diff --git 
a/flink-libraries/flink-ml/src/test/scala/org/apache/flink/ml/nn/KNNITSuite.scala
 
b/flink-libraries/flink-ml/src/test/scala/org/apache/flink/ml/nn/KNNITSuite.scala
index ac30c3f..389d69a 100644
--- 
a/flink-libraries/flink-ml/src/test/scala/org/apache/flink/ml/nn/KNNITSuite.scala
+++ 
b/flink-libraries/flink-ml/src/test/scala/org/apache/flink/ml/nn/KNNITSuite.scala
@@ -42,18 +42,17 @@ class KNNITSuite extends FlatSpec with Matchers with 
FlinkTestBase {
 }
   }
 
-  val env = ExecutionEnvironment.getExecutionEnvironment
-
-  // prepare data
-  val trainingSet = 
env.fromCollection(Classification.trainingData).map(_.vector)
-  val testingSet = env.fromElements(DenseVector(0.0, 0.0))
-
   // calculate answer
   val answer = Classification.trainingData.map {
 v => (v.vector, SquaredEuclideanDistanceMetric().distance(DenseVector(0.0, 
0.0), v.vector))
   }.sortBy(_._2).take(3).map(_._1).toArray
 
   it should "calculate kNN join correctly without using a Quadtree" in {
+val env = ExecutionEnvironment.getExecutionEnvironment
+
+// prepare data
+val trainingSet = 
env.fromCollection(Classification.trainingData).map(_.vector)
+val testingSet = env.fromElements(DenseVector(0.0, 0.0))
 
 val knn = KNN()
   .setK(3)
@@ -72,6 +71,11 @@ class KNNITSuite extends FlatSpec with Matchers with 
FlinkTestBase {
   }
 
   it should "calculate kNN join correctly with a Quadtree" in {
+val env = ExecutionEnvironment.getExecutionEnvironment
+
+// prepare data
+val trainingSet = 
env.fromCollection(Classification.trainingData).map(_.vector)
+val testingSet = env.fromElements(DenseVector(0.0, 0.0))
 
 val knn = KNN()
   .setK(3)
@@ -91,6 +95,12 @@ class KNNITSuite extends FlatSpec with Matchers with 
FlinkTestBase {
 
   it should "throw an exception when using a Quadtree with an incompatible 
metric" in {
 intercept[IllegalArgumentException] {
+  val env = ExecutionEnvironment.getExecutionEnvironment
+
+  // prepare data
+  val trainingSet = 
env.fromCollection(Classification.trainingData).map(_.vector)
+  val testingSet = env.fromElements(DenseVector(0.0, 0.0))
+
   val knn = KNN()
 .setK(3)
 .setBlocks(10)

buildbot failure in on flink-docs-master

2016-05-31 Thread buildbot 
The Buildbot has detected a new failure on builder flink-docs-master while 
building . Full details are available at:
https://ci.apache.org/builders/flink-docs-master/builds/347

Buildbot URL: https://ci.apache.org/

Buildslave for this Build: tethys_ubuntu

Build Reason: The Nightly scheduler named 'flink-nightly-docs-master' triggered 
this build
Build Source Stamp: [branch master] HEAD
Blamelist: 

BUILD FAILED: failed compile

Sincerely,
 -The Buildbot

[3/4] flink git commit: [FLINK-3979] [docs] Add import statements for last code sample in the quickstart docs.

2016-05-31 Thread sewen 
[FLINK-3979] [docs] Add import statements for last code sample in the 
quickstart docs.

This closes #2038


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/ad52a5f0
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/ad52a5f0
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/ad52a5f0

Branch: refs/heads/master
Commit: ad52a5f0cebe6e5d593be0ae40656b28470af106
Parents: 5b02879
Author: Zhai Jia 
Authored: Thu May 26 23:24:40 2016 +0800
Committer: Stephan Ewen 
Committed: Tue May 31 16:39:56 2016 +0200

--
 docs/quickstart/run_example_quickstart.md | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/flink/blob/ad52a5f0/docs/quickstart/run_example_quickstart.md
--
diff --git a/docs/quickstart/run_example_quickstart.md 
b/docs/quickstart/run_example_quickstart.md
index 0d5e919..0939297 100644
--- a/docs/quickstart/run_example_quickstart.md
+++ b/docs/quickstart/run_example_quickstart.md
@@ -315,6 +315,13 @@ result
 .addSink(new FlinkKafkaProducer08<>("localhost:9092", "wiki-result", new 
SimpleStringSchema()));
 {% endhighlight %}
 
+The related classes also need to be imported:
+{% highlight java %}
+import org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer08;
+import org.apache.flink.streaming.util.serialization.SimpleStringSchema;
+import org.apache.flink.api.common.functions.MapFunction;
+{% endhighlight %}
+
 Note how we first transform the Stream of `Tuple2` to a Stream 
of `String` using
 a MapFunction. We are doing this because it is easier to write plain strings 
to Kafka. Then,
 we create a Kafka sink. You might have to adapt the hostname and port to your 
setup. `"wiki-result"`
@@ -382,4 +389,4 @@ and, for example, see the number of processed elements:
 
 
 
-This concludes our little tour of Flink. If you have any questions, please 
don't hesitate to ask on our [Mailing 
Lists](http://flink.apache.org/community.html#mailing-lists).
\ No newline at end of file
+This concludes our little tour of Flink. If you have any questions, please 
don't hesitate to ask on our [Mailing 
Lists](http://flink.apache.org/community.html#mailing-lists).

[4/4] flink git commit: [FLINK-3886] [clients] Give a better error when the application Main class is not public.

2016-05-31 Thread sewen 
[FLINK-3886] [clients] Give a better error when the application Main class is 
not public.

This closes #2043


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/6db9e6ae
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/6db9e6ae
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/6db9e6ae

Branch: refs/heads/master
Commit: 6db9e6ae5c2f4968687a3fcae0520e05442bd9ab
Parents: bcf5f46
Author: Niels Basjes 
Authored: Fri May 27 13:51:04 2016 +0200
Committer: Stephan Ewen 
Committed: Tue May 31 16:39:56 2016 +0200

--
 .../apache/flink/client/program/PackagedProgram.java|  4 
 .../org/apache/flink/client/program/ClientTest.java | 12 ++--
 2 files changed, 10 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/flink/blob/6db9e6ae/flink-clients/src/main/java/org/apache/flink/client/program/PackagedProgram.java
--
diff --git 
a/flink-clients/src/main/java/org/apache/flink/client/program/PackagedProgram.java
 
b/flink-clients/src/main/java/org/apache/flink/client/program/PackagedProgram.java
index f78502a..40092d8 100644
--- 
a/flink-clients/src/main/java/org/apache/flink/client/program/PackagedProgram.java
+++ 
b/flink-clients/src/main/java/org/apache/flink/client/program/PackagedProgram.java
@@ -484,6 +484,10 @@ public class PackagedProgram {

private static void callMainMethod(Class entryClass, String[] args) 
throws ProgramInvocationException {
Method mainMethod;
+   if (!Modifier.isPublic(entryClass.getModifiers())) {
+   throw new ProgramInvocationException("The class " + 
entryClass.getName() + " must be public.");
+   }
+
try {
mainMethod = entryClass.getMethod("main", 
String[].class);
} catch (NoSuchMethodException e) {

http://git-wip-us.apache.org/repos/asf/flink/blob/6db9e6ae/flink-clients/src/test/java/org/apache/flink/client/program/ClientTest.java
--
diff --git 
a/flink-clients/src/test/java/org/apache/flink/client/program/ClientTest.java 
b/flink-clients/src/test/java/org/apache/flink/client/program/ClientTest.java
index cc32d9c..4f9b367 100644
--- 
a/flink-clients/src/test/java/org/apache/flink/client/program/ClientTest.java
+++ 
b/flink-clients/src/test/java/org/apache/flink/client/program/ClientTest.java
@@ -378,7 +378,7 @@ public class ClientTest {
}
}
 
-   private static final class TestExecuteTwice {
+   public static final class TestExecuteTwice {
 
public static void main(String args[]) throws Exception {
final ExecutionEnvironment env = 
ExecutionEnvironment.getExecutionEnvironment();
@@ -388,7 +388,7 @@ public class ClientTest {
}
}
 
-   private static final class TestEager {
+   public static final class TestEager {
 
public static void main(String args[]) throws Exception {
final ExecutionEnvironment env = 
ExecutionEnvironment.getExecutionEnvironment();
@@ -396,7 +396,7 @@ public class ClientTest {
}
}
 
-   private static final class TestGetRuntime {
+   public static final class TestGetRuntime {
 
public static void main(String args[]) throws Exception {
final ExecutionEnvironment env = 
ExecutionEnvironment.getExecutionEnvironment();
@@ -405,7 +405,7 @@ public class ClientTest {
}
}
 
-   private static final class TestGetJobID {
+   public static final class TestGetJobID {
 
public static void main(String args[]) throws Exception {
final ExecutionEnvironment env = 
ExecutionEnvironment.getExecutionEnvironment();
@@ -414,7 +414,7 @@ public class ClientTest {
}
}
 
-   private static final class TestGetAccumulator {
+   public static final class TestGetAccumulator {
 
public static void main(String args[]) throws Exception {
final ExecutionEnvironment env = 
ExecutionEnvironment.getExecutionEnvironment();
@@ -423,7 +423,7 @@ public class ClientTest {
}
}
 
-   private static final class TestGetAllAccumulator {
+   public static final class TestGetAllAccumulator {
 
public static void main(String args[]) throws Exception {
final ExecutionEnvironment env = 
ExecutionEnvironment.getExecutionEnvironment();

[2/4] flink git commit: [FLINK-3978] [core] Add hasBroadcastVariable method to RuntimeContext

2016-05-31 Thread sewen 
[FLINK-3978] [core] Add hasBroadcastVariable method to RuntimeContext

New method RuntimeContext.hasBroadcastVariable(String).


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/5b028797
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/5b028797
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/5b028797

Branch: refs/heads/master
Commit: 5b0287971fa2beda360105d96e7bfbc7a110fae7
Parents: 6db9e6a
Author: Greg Hogan 
Authored: Thu May 26 14:45:00 2016 -0400
Committer: Stephan Ewen 
Committed: Tue May 31 16:39:56 2016 +0200

--
 .../api/common/functions/RuntimeContext.java  | 10 ++
 .../common/functions/util/RuntimeUDFContext.java  |  5 +
 .../functions/util/RuntimeUDFContextTest.java |  9 +++--
 .../util/DistributedRuntimeUDFContext.java| 18 +-
 .../api/operators/StreamingRuntimeContext.java|  5 +
 5 files changed, 36 insertions(+), 11 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/flink/blob/5b028797/flink-core/src/main/java/org/apache/flink/api/common/functions/RuntimeContext.java
--
diff --git 
a/flink-core/src/main/java/org/apache/flink/api/common/functions/RuntimeContext.java
 
b/flink-core/src/main/java/org/apache/flink/api/common/functions/RuntimeContext.java
index 72e1a4d..e409c11 100644
--- 
a/flink-core/src/main/java/org/apache/flink/api/common/functions/RuntimeContext.java
+++ 
b/flink-core/src/main/java/org/apache/flink/api/common/functions/RuntimeContext.java
@@ -164,6 +164,16 @@ public interface RuntimeContext {
// 

 
/**
+* Tests for the existence of the broadcast variable identified by the
+* given {@code name}.
+*
+* @param name The name under which the broadcast variable is 
registered;
+* @return Whether a broadcast variable exists for the given name.
+*/
+   @PublicEvolving
+   boolean hasBroadcastVariable(String name);
+
+   /**
 * Returns the result bound to the broadcast variable identified by the 
 * given {@code name}.
 * 

http://git-wip-us.apache.org/repos/asf/flink/blob/5b028797/flink-core/src/main/java/org/apache/flink/api/common/functions/util/RuntimeUDFContext.java
--
diff --git 
a/flink-core/src/main/java/org/apache/flink/api/common/functions/util/RuntimeUDFContext.java
 
b/flink-core/src/main/java/org/apache/flink/api/common/functions/util/RuntimeUDFContext.java
index 6571d0d..ba3f85e 100644
--- 
a/flink-core/src/main/java/org/apache/flink/api/common/functions/util/RuntimeUDFContext.java
+++ 
b/flink-core/src/main/java/org/apache/flink/api/common/functions/util/RuntimeUDFContext.java
@@ -49,6 +49,11 @@ public class RuntimeUDFContext extends 
AbstractRuntimeUDFContext {
}
 
@Override
+   public boolean hasBroadcastVariable(String name) {
+   return this.initializedBroadcastVars.containsKey(name) || 
this.uninitializedBroadcastVars.containsKey(name);
+   }
+
+   @Override
@SuppressWarnings("unchecked")
public  List getBroadcastVariable(String name) {


http://git-wip-us.apache.org/repos/asf/flink/blob/5b028797/flink-core/src/test/java/org/apache/flink/api/common/functions/util/RuntimeUDFContextTest.java
--
diff --git 
a/flink-core/src/test/java/org/apache/flink/api/common/functions/util/RuntimeUDFContextTest.java
 
b/flink-core/src/test/java/org/apache/flink/api/common/functions/util/RuntimeUDFContextTest.java
index 8f00cd5..4cd2a64 100644
--- 
a/flink-core/src/test/java/org/apache/flink/api/common/functions/util/RuntimeUDFContextTest.java
+++ 
b/flink-core/src/test/java/org/apache/flink/api/common/functions/util/RuntimeUDFContextTest.java
@@ -48,7 +48,9 @@ public class RuntimeUDFContextTest {
new HashMap(),
new HashMap(),
new UnregisteredMetricsGroup());
-   
+
+   assertFalse(ctx.hasBroadcastVariable("some name"));
+
try {
ctx.getBroadcastVariable("some name");
fail("should throw an exception");
@@ -85,7 +87,10 @@ public class RuntimeUDFContextTest {

ctx.setBroadcastVariable("name1", Arrays.asList(1, 2, 
3, 4));
ctx.setBroadcastVariable("name2",

flink git commit: [FLINK-3607] decrease default forkCount for tests

2016-05-31 Thread mxm 
Repository: flink
Updated Branches:
  refs/heads/master 8a6bdf4c1 -> bcf5f4648


[FLINK-3607] decrease default forkCount for tests

This makes running test locally a more pleasant experience. It still
uses all exposed CPU cores (virtual or real). A custom forkCount can be
set using the flink.forkCount property, e.g. -Dflink.forkCount=4.


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/bcf5f464
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/bcf5f464
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/bcf5f464

Branch: refs/heads/master
Commit: bcf5f4648670770b6c5fcfd56fd24660fae5ad44
Parents: 8a6bdf4
Author: Maximilian Michels 
Authored: Wed Jun 1 15:11:29 2016 +0200
Committer: Maximilian Michels 
Committed: Tue May 31 15:21:14 2016 +0200

--
 pom.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/flink/blob/bcf5f464/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 6f444b3..23f6dc5 100644
--- a/pom.xml
+++ b/pom.xml
@@ -89,7 +89,7 @@ under the License.

-   1.5C
+   1C
true
log4j-test.properties
1.7.7

flink git commit: [FLINK-3887] improve dependency management for building docs

2016-05-31 Thread mxm 
Repository: flink
Updated Branches:
  refs/heads/release-0.8 33e9df69d -> 96849b5e3


[FLINK-3887] improve dependency management for building docs

The Flink documentation build process is currently quite messy. These
changes move us to a new build process with proper dependency
handling. It assures that we all use the same dependency versions for
consistent build output. Also, it eases the automated building process
on other systems (like the ASF Buildbot). The goal was to make the
documentation build process easier and self-contained.

- use Ruby's Bundler Gem to install dependencies
- update README
- adapt Dockerfile
- add additional rules to .gitignore
- change default doc output path from /target to /content
(default path of the flink-web repository)

This closes #2033


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/96849b5e
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/96849b5e
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/96849b5e

Branch: refs/heads/release-0.8
Commit: 96849b5e3689a21690d45decce7b29972b248898
Parents: 33e9df6
Author: Maximilian Michels 
Authored: Mon May 16 15:02:59 2016 +0200
Committer: Maximilian Michels 
Committed: Tue May 31 12:23:11 2016 +0200

--
 .gitignore |  7 +--
 docs/Gemfile   | 25 +
 docs/README.md | 26 ++
 docs/build_docs.sh | 42 ++
 pom.xml|  5 +++--
 5 files changed, 65 insertions(+), 40 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/flink/blob/96849b5e/.gitignore
--
diff --git a/.gitignore b/.gitignore
index 9622cc4..9e75841 100644
--- a/.gitignore
+++ b/.gitignore
@@ -16,6 +16,9 @@ tmp
 *.jar
 *.log
 .DS_Store
-_site
-docs/api
 
flink-addons/flink-avro/src/test/java/org/apache/flink/api/java/record/io/avro/generated
+/docs/api
+/docs/content
+/docs/Gemfile.lock
+/docs/.bundle
+/docs/.rubydeps

http://git-wip-us.apache.org/repos/asf/flink/blob/96849b5e/docs/Gemfile
--
diff --git a/docs/Gemfile b/docs/Gemfile
new file mode 100644
index 000..d367999
--- /dev/null
+++ b/docs/Gemfile
@@ -0,0 +1,25 @@
+
+#  Licensed to the Apache Software Foundation (ASF) under one
+#  or more contributor license agreements.  See the NOTICE file
+#  distributed with this work for additional information
+#  regarding copyright ownership.  The ASF licenses this file
+#  to you under the Apache License, Version 2.0 (the
+#  "License"); you may not use this file except in compliance
+#  with the License.  You may obtain a copy of the License at
+#
+#  http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+source 'https://rubygems.org'
+
+# Dependencies required to build the Flink docs
+gem 'jekyll', '2.5.3'
+gem 'kramdown', '1.10.0'
+gem 'pygments.rb', '0.6.3'
+gem 'therubyracer', '0.12.2'

http://git-wip-us.apache.org/repos/asf/flink/blob/96849b5e/docs/README.md
--
diff --git a/docs/README.md b/docs/README.md
index 5ab0f2f..a66c031 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -8,25 +8,27 @@ files found here.
 
 # Requirements
 
-We use Markdown to write and Jekyll to translate the documentation to static
-HTML. You can install all needed software via:
+The dependencies are declared in the Gemfile in this directory. We use Markdown
+to write and Jekyll to translate the documentation to static HTML. All required
+dependencies are installed locally when you build the documentation through the
+`build_docs.sh` script. If you want to install the software manually, use 
Ruby's
+Bundler Gem to install all dependencies:
 
-gem install jekyll
-gem install redcarpet
-sudo easy_install Pygments
+gem install bundler
+bundle install
 
-Redcarpet is needed for Markdown processing and the Python based Pygments is
-used for syntax highlighting.
+Note that in Ubuntu based systems, it may be necessary to install the 
`ruby-dev`
+via apt to build native code.
 
 # Build
 
-The `docs/build_docs.sh` script calls Jekyll and generates the documentation to
-`docs/target`. You can then point your browser to

flink git commit: [FLINK-3887] improve dependency management for building docs

2016-05-31 Thread mxm 
Repository: flink
Updated Branches:
  refs/heads/release-0.9 19abef456 -> 1a1c134ac


[FLINK-3887] improve dependency management for building docs

The Flink documentation build process is currently quite messy. These
changes move us to a new build process with proper dependency
handling. It assures that we all use the same dependency versions for
consistent build output. Also, it eases the automated building process
on other systems (like the ASF Buildbot). The goal was to make the
documentation build process easier and self-contained.

- use Ruby's Bundler Gem to install dependencies
- update README
- adapt Dockerfile
- add additional rules to .gitignore
- change default doc output path from /target to /content
(default path of the flink-web repository)

This closes #2033


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/1a1c134a
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/1a1c134a
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/1a1c134a

Branch: refs/heads/release-0.9
Commit: 1a1c134ac5e93faf9e04f9b125b330794f8372bd
Parents: 19abef4
Author: Maximilian Michels 
Authored: Mon May 16 15:02:59 2016 +0200
Committer: Maximilian Michels 
Committed: Tue May 31 12:20:28 2016 +0200

--
 .gitignore |  7 +--
 docs/Gemfile   | 25 +
 docs/README.md | 26 +++---
 docs/build_docs.sh | 42 ++
 pom.xml|  5 +++--
 5 files changed, 66 insertions(+), 39 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/flink/blob/1a1c134a/.gitignore
--
diff --git a/.gitignore b/.gitignore
index 36d98e6..6f6d889 100644
--- a/.gitignore
+++ b/.gitignore
@@ -16,8 +16,11 @@ tmp
 *.jar
 *.log
 .DS_Store
-_site
-docs/api
 build-target
 flink-staging/flink-avro/src/test/java/org/apache/flink/api/io/avro/generated/
 atlassian-ide-plugin.xml
+/docs/api
+/docs/content
+/docs/Gemfile.lock
+/docs/.bundle
+/docs/.rubydeps

http://git-wip-us.apache.org/repos/asf/flink/blob/1a1c134a/docs/Gemfile
--
diff --git a/docs/Gemfile b/docs/Gemfile
new file mode 100644
index 000..d367999
--- /dev/null
+++ b/docs/Gemfile
@@ -0,0 +1,25 @@
+
+#  Licensed to the Apache Software Foundation (ASF) under one
+#  or more contributor license agreements.  See the NOTICE file
+#  distributed with this work for additional information
+#  regarding copyright ownership.  The ASF licenses this file
+#  to you under the Apache License, Version 2.0 (the
+#  "License"); you may not use this file except in compliance
+#  with the License.  You may obtain a copy of the License at
+#
+#  http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+source 'https://rubygems.org'
+
+# Dependencies required to build the Flink docs
+gem 'jekyll', '2.5.3'
+gem 'kramdown', '1.10.0'
+gem 'pygments.rb', '0.6.3'
+gem 'therubyracer', '0.12.2'

http://git-wip-us.apache.org/repos/asf/flink/blob/1a1c134a/docs/README.md
--
diff --git a/docs/README.md b/docs/README.md
index 206b04c..a9f6e28 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -6,23 +6,27 @@ http://flink.apache.org/ is also generated from the files 
found here.
 
 # Requirements
 
-We use Markdown to write and Jekyll to translate the documentation to static 
HTML. You can install
-all needed software via:
+The dependencies are declared in the Gemfile in this directory. We use Markdown
+to write and Jekyll to translate the documentation to static HTML. All required
+dependencies are installed locally when you build the documentation through the
+`build_docs.sh` script. If you want to install the software manually, use 
Ruby's
+Bundler Gem to install all dependencies:
 
-gem install jekyll
-gem install kramdown
-sudo easy_install Pygments
+gem install bundler
+bundle install
 
-Kramdown is needed for Markdown processing and the Python based Pygments is 
used for syntax
-highlighting.
+Note that in Ubuntu based systems, it may be necessary to install the 
`ruby-dev`
+via apt to build native code.
 
 # Build
 
-The `docs/build_docs.sh` script calls Jekyll and generates the

flink git commit: [FLINK-3975] [docs] Override baseurl when serving docs locally

2016-05-31 Thread uce 
Repository: flink
Updated Branches:
  refs/heads/release-1.0 b033d2a54 -> bba82cf9e


[FLINK-3975] [docs] Override baseurl when serving docs locally

This closes #2040


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/bba82cf9
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/bba82cf9
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/bba82cf9

Branch: refs/heads/release-1.0
Commit: bba82cf9e3857a1908e4952b1c241d68790ee2af
Parents: b033d2a
Author: Dyana Rose 
Authored: Thu May 26 21:40:07 2016 +0100
Committer: Ufuk Celebi 
Committed: Tue May 31 12:08:13 2016 +0200

--
 docs/_local_preview_conf.yml | 19 ---
 docs/build_docs.sh   |  2 +-
 2 files changed, 1 insertion(+), 20 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/flink/blob/bba82cf9/docs/_local_preview_conf.yml
--
diff --git a/docs/_local_preview_conf.yml b/docs/_local_preview_conf.yml
deleted file mode 100644
index 82841aa..000
--- a/docs/_local_preview_conf.yml
+++ /dev/null
@@ -1,19 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements.  See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership.  The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License.  You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied.  See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
-# We unset the baseurl for local testing
-baseurl: ""

http://git-wip-us.apache.org/repos/asf/flink/blob/bba82cf9/docs/build_docs.sh
--
diff --git a/docs/build_docs.sh b/docs/build_docs.sh
index 412008a..adc8400 100755
--- a/docs/build_docs.sh
+++ b/docs/build_docs.sh
@@ -49,7 +49,7 @@ JEKYLL_CMD="build"
 while getopts ":p" opt; do
case $opt in
p)
-   JEKYLL_CMD="serve --config _config.yml,_local_preview_conf.yml 
--watch"
+   JEKYLL_CMD="serve --baseurl= --watch"
;;
esac
 done

flink git commit: [FLINK-3975] [docs] Override baseurl when serving docs locally

2016-05-31 Thread uce 
Repository: flink
Updated Branches:
  refs/heads/master 43ad1a2a6 -> 546b64fd4


[FLINK-3975] [docs] Override baseurl when serving docs locally

This closes #2040


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/546b64fd
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/546b64fd
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/546b64fd

Branch: refs/heads/master
Commit: 546b64fd46afe0d5ec6ce9568b6d9e3d057f5ac8
Parents: 43ad1a2
Author: Dyana Rose 
Authored: Thu May 26 21:40:07 2016 +0100
Committer: Ufuk Celebi 
Committed: Tue May 31 12:07:16 2016 +0200

--
 docs/_local_preview_conf.yml | 19 ---
 docs/build_docs.sh   |  2 +-
 2 files changed, 1 insertion(+), 20 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/flink/blob/546b64fd/docs/_local_preview_conf.yml
--
diff --git a/docs/_local_preview_conf.yml b/docs/_local_preview_conf.yml
deleted file mode 100644
index 82841aa..000
--- a/docs/_local_preview_conf.yml
+++ /dev/null
@@ -1,19 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements.  See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership.  The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License.  You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied.  See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
-# We unset the baseurl for local testing
-baseurl: ""

http://git-wip-us.apache.org/repos/asf/flink/blob/546b64fd/docs/build_docs.sh
--
diff --git a/docs/build_docs.sh b/docs/build_docs.sh
index 412008a..adc8400 100755
--- a/docs/build_docs.sh
+++ b/docs/build_docs.sh
@@ -49,7 +49,7 @@ JEKYLL_CMD="build"
 while getopts ":p" opt; do
case $opt in
p)
-   JEKYLL_CMD="serve --config _config.yml,_local_preview_conf.yml 
--watch"
+   JEKYLL_CMD="serve --baseurl= --watch"
;;
esac
 done

flink git commit: [FLINK-3887] improve dependency management for building docs

2016-05-31 Thread mxm 
Repository: flink
Updated Branches:
  refs/heads/release-1.0 9ec1fe0a8 -> b033d2a54


[FLINK-3887] improve dependency management for building docs

The Flink documentation build process is currently quite messy. These
changes move us to a new build process with proper dependency
handling. It assures that we all use the same dependency versions for
consistent build output. Also, it eases the automated building process
on other systems (like the ASF Buildbot). The goal was to make the
documentation build process easier and self-contained.

- use Ruby's Bundler Gem to install dependencies
- update README
- adapt Dockerfile
- add additional rules to .gitignore
- change default doc output path from /target to /content
(default path of the flink-web repository)

This closes #2033


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/b033d2a5
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/b033d2a5
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/b033d2a5

Branch: refs/heads/release-1.0
Commit: b033d2a54c25a8be4ff0b0787dbcb1b4c78f25c4
Parents: 9ec1fe0
Author: Maximilian Michels 
Authored: Mon May 16 15:02:59 2016 +0200
Committer: Maximilian Michels 
Committed: Tue May 31 11:38:57 2016 +0200

--
 .gitignore |  7 +--
 docs/Gemfile   | 25 +
 docs/README.md | 30 +++---
 docs/build_docs.sh | 42 ++
 docs/docker/Dockerfile |  6 +-
 pom.xml|  5 +++--
 6 files changed, 67 insertions(+), 48 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/flink/blob/b033d2a5/.gitignore
--
diff --git a/.gitignore b/.gitignore
index 629d62c..9985ffb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -16,8 +16,6 @@ tmp
 *.jar
 *.log
 .DS_Store
-_site
-docs/api
 build-target
 
flink-batch-connectors/flink-avro/src/test/java/org/apache/flink/api/io/avro/generated/
 flink-runtime-web/web-dashboard/assets/fonts/
@@ -25,3 +23,8 @@ flink-runtime-web/web-dashboard/node_modules/
 flink-runtime-web/web-dashboard/bower_components/
 atlassian-ide-plugin.xml
 out/
+/docs/api
+/docs/content
+/docs/Gemfile.lock
+/docs/.bundle
+/docs/.rubydeps

http://git-wip-us.apache.org/repos/asf/flink/blob/b033d2a5/docs/Gemfile
--
diff --git a/docs/Gemfile b/docs/Gemfile
new file mode 100644
index 000..d367999
--- /dev/null
+++ b/docs/Gemfile
@@ -0,0 +1,25 @@
+
+#  Licensed to the Apache Software Foundation (ASF) under one
+#  or more contributor license agreements.  See the NOTICE file
+#  distributed with this work for additional information
+#  regarding copyright ownership.  The ASF licenses this file
+#  to you under the Apache License, Version 2.0 (the
+#  "License"); you may not use this file except in compliance
+#  with the License.  You may obtain a copy of the License at
+#
+#  http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+source 'https://rubygems.org'
+
+# Dependencies required to build the Flink docs
+gem 'jekyll', '2.5.3'
+gem 'kramdown', '1.10.0'
+gem 'pygments.rb', '0.6.3'
+gem 'therubyracer', '0.12.2'

http://git-wip-us.apache.org/repos/asf/flink/blob/b033d2a5/docs/README.md
--
diff --git a/docs/README.md b/docs/README.md
index d37dc77..52dfad3 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -6,19 +6,17 @@ http://flink.apache.org/ is also generated from the files 
found here.
 
 # Requirements
 
-We use Markdown to write and Jekyll to translate the documentation to static 
HTML. Kramdown is
-needed for Markdown processing and the Python based Pygments is used for 
syntax highlighting. To run
-Javascript code from Ruby, you need to install a javascript runtime (e.g. 
`therubyracer`). You can
-install all needed software via the following commands:
+The dependencies are declared in the Gemfile in this directory. We use Markdown
+to write and Jekyll to translate the documentation to static HTML. All required
+dependencies are installed locally when you build the documentation through the
+`build_docs.sh` script. If you want to install the software manually, use 
Ruby's

flink git commit: [docs] Add notes about shipped log4j properties files

2016-05-31 Thread uce 
Repository: flink
Updated Branches:
  refs/heads/release-1.0 845d1e635 -> 9ec1fe0a8


[docs] Add notes about shipped log4j properties files


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/9ec1fe0a
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/9ec1fe0a
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/9ec1fe0a

Branch: refs/heads/release-1.0
Commit: 9ec1fe0a82288f24cdb2950382687b345b99c67c
Parents: 845d1e6
Author: Ufuk Celebi 
Authored: Tue May 31 11:33:47 2016 +0200
Committer: Ufuk Celebi 
Committed: Tue May 31 11:34:49 2016 +0200

--
 docs/internals/logging.md | 6 ++
 1 file changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/flink/blob/9ec1fe0a/docs/internals/logging.md
--
diff --git a/docs/internals/logging.md b/docs/internals/logging.md
index d2c0cba..f68bc73 100644
--- a/docs/internals/logging.md
+++ b/docs/internals/logging.md
@@ -33,6 +33,12 @@ The logging in Flink is implemented using the slf4j logging 
interface. As underl
 
 Log4j is controlled using property files. In Flink's case, the file is usually 
called `log4j.properties`. We pass the filename and location of this file using 
the `-Dlog4j.configuration=` parameter to the JVM.
 
+Flink ships with the following default properties files:
+
+- `log4j-cli.properties`: Used by the Flink command line client (e.g. `flink 
run`) (not code executed on the cluster)
+- `log4j-yarn-session.properties`: Used by the Flink command line client when 
starting a YARN session (`yarn-session.sh`)
+- `log4j.properties`: JobManager/Taskmanager logs (both standalone and YARN)
+
 ## Configuring logback
 
 For users and developers alike it is important to control the logging 
framework.

flink git commit: [docs] Add notes about shipped log4j properties files

2016-05-31 Thread uce 
Repository: flink
Updated Branches:
  refs/heads/master aec8b5a72 -> 43ad1a2a6


[docs] Add notes about shipped log4j properties files


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/43ad1a2a
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/43ad1a2a
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/43ad1a2a

Branch: refs/heads/master
Commit: 43ad1a2a6fc657c5e41d67deb648fccc9e2eb2d6
Parents: aec8b5a
Author: Ufuk Celebi 
Authored: Tue May 31 11:33:47 2016 +0200
Committer: Ufuk Celebi 
Committed: Tue May 31 11:34:30 2016 +0200

--
 docs/internals/logging.md | 6 ++
 1 file changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/flink/blob/43ad1a2a/docs/internals/logging.md
--
diff --git a/docs/internals/logging.md b/docs/internals/logging.md
index d2c0cba..f68bc73 100644
--- a/docs/internals/logging.md
+++ b/docs/internals/logging.md
@@ -33,6 +33,12 @@ The logging in Flink is implemented using the slf4j logging 
interface. As underl
 
 Log4j is controlled using property files. In Flink's case, the file is usually 
called `log4j.properties`. We pass the filename and location of this file using 
the `-Dlog4j.configuration=` parameter to the JVM.
 
+Flink ships with the following default properties files:
+
+- `log4j-cli.properties`: Used by the Flink command line client (e.g. `flink 
run`) (not code executed on the cluster)
+- `log4j-yarn-session.properties`: Used by the Flink command line client when 
starting a YARN session (`yarn-session.sh`)
+- `log4j.properties`: JobManager/Taskmanager logs (both standalone and YARN)
+
 ## Configuring logback
 
 For users and developers alike it is important to control the logging 
framework.

flink git commit: [FLINK-3887] improve dependency management for building docs

2016-05-31 Thread mxm 
Repository: flink
Updated Branches:
  refs/heads/master c60326f85 -> aec8b5a72


[FLINK-3887] improve dependency management for building docs

The Flink documentation build process is currently quite messy. These
changes move us to a new build process with proper dependency
handling. It assures that we all use the same dependency versions for
consistent build output. Also, it eases the automated building process
on other systems (like the ASF Buildbot). The goal was to make the
documentation build process easier and self-contained.

- use Ruby's Bundler Gem to install dependencies
- update README
- adapt Dockerfile
- add additional rules to .gitignore
- change default doc output path from /target to /content
(default path of the flink-web repository)

This closes #2033


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/aec8b5a7
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/aec8b5a7
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/aec8b5a7

Branch: refs/heads/master
Commit: aec8b5a729498a8a4c9d4498b3f05671692c3ba6
Parents: c60326f
Author: Maximilian Michels 
Authored: Mon May 16 15:02:59 2016 +0200
Committer: Maximilian Michels 
Committed: Tue May 31 11:29:22 2016 +0200

--
 .gitignore |  7 +--
 docs/Gemfile   | 25 +
 docs/README.md | 30 +++---
 docs/build_docs.sh | 42 ++
 docs/docker/Dockerfile |  6 +-
 pom.xml|  5 +++--
 6 files changed, 67 insertions(+), 48 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/flink/blob/aec8b5a7/.gitignore
--
diff --git a/.gitignore b/.gitignore
index 629d62c..9985ffb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -16,8 +16,6 @@ tmp
 *.jar
 *.log
 .DS_Store
-_site
-docs/api
 build-target
 
flink-batch-connectors/flink-avro/src/test/java/org/apache/flink/api/io/avro/generated/
 flink-runtime-web/web-dashboard/assets/fonts/
@@ -25,3 +23,8 @@ flink-runtime-web/web-dashboard/node_modules/
 flink-runtime-web/web-dashboard/bower_components/
 atlassian-ide-plugin.xml
 out/
+/docs/api
+/docs/content
+/docs/Gemfile.lock
+/docs/.bundle
+/docs/.rubydeps

http://git-wip-us.apache.org/repos/asf/flink/blob/aec8b5a7/docs/Gemfile
--
diff --git a/docs/Gemfile b/docs/Gemfile
new file mode 100644
index 000..d367999
--- /dev/null
+++ b/docs/Gemfile
@@ -0,0 +1,25 @@
+
+#  Licensed to the Apache Software Foundation (ASF) under one
+#  or more contributor license agreements.  See the NOTICE file
+#  distributed with this work for additional information
+#  regarding copyright ownership.  The ASF licenses this file
+#  to you under the Apache License, Version 2.0 (the
+#  "License"); you may not use this file except in compliance
+#  with the License.  You may obtain a copy of the License at
+#
+#  http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+source 'https://rubygems.org'
+
+# Dependencies required to build the Flink docs
+gem 'jekyll', '2.5.3'
+gem 'kramdown', '1.10.0'
+gem 'pygments.rb', '0.6.3'
+gem 'therubyracer', '0.12.2'

http://git-wip-us.apache.org/repos/asf/flink/blob/aec8b5a7/docs/README.md
--
diff --git a/docs/README.md b/docs/README.md
index d37dc77..52dfad3 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -6,19 +6,17 @@ http://flink.apache.org/ is also generated from the files 
found here.
 
 # Requirements
 
-We use Markdown to write and Jekyll to translate the documentation to static 
HTML. Kramdown is
-needed for Markdown processing and the Python based Pygments is used for 
syntax highlighting. To run
-Javascript code from Ruby, you need to install a javascript runtime (e.g. 
`therubyracer`). You can
-install all needed software via the following commands:
+The dependencies are declared in the Gemfile in this directory. We use Markdown
+to write and Jekyll to translate the documentation to static HTML. All required
+dependencies are installed locally when you build the documentation through the
+`build_docs.sh` script. If you want to install the software manually, use 
Ruby's
+Bundler Gem