[GitHub] flink pull request #3275: [FLINK-5618][docs] add queryable state (user) docu...

2017-02-06 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3275 [FLINK-5618][docs] add queryable state (user) documentation This adds initial documentation of the queryable state from a user's perspective. You can merge this pull request into a Git repository

[GitHub] flink pull request #3272: [FLINK-5721] Add FoldingState to State Documentati...

2017-02-06 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3272#discussion_r99614763 --- Diff: docs/dev/stream/state.md --- @@ -113,9 +113,19 @@ be retrieved using `Iterable get()`. added to the state. The interface is the same

[GitHub] flink pull request #3272: [FLINK-5721] Add FoldingState to State Documentati...

2017-02-06 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3272#discussion_r99615295 --- Diff: docs/dev/stream/state.md --- @@ -126,8 +136,8 @@ To get a state handle, you have to create a `StateDescriptor`. This holds the na (as we

[GitHub] flink pull request #3279: [FLINK-5618][docs] createSerializer must actually ...

2017-02-06 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3279 [FLINK-5618][docs] createSerializer must actually get a non-null ExecutionConfig providing `null` fails with a NPE You can merge this pull request into a Git repository by running: $ git pull

[GitHub] flink issue #3218: [FLINK-5642][query] fix a race condition with HeadListSta...

2017-02-03 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3218 1. Actually, RocksDB state's get() method has the idiom of returning a (deserialized) **copy** with which the user can do whatever he likes to, knowing that changes are not reflected in the state back

[GitHub] flink pull request #3172: [FLINK-5559] let KvStateRequestSerializer#deserial...

2017-01-23 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3172#discussion_r97279222 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/query/netty/message/KvStateRequestSerializer.java --- @@ -377,22 +376,24 @@ public static

[GitHub] flink pull request #3142: [FLINK-5527][query] querying a non-existing key do...

2017-01-23 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/3142 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink issue #3142: [FLINK-5527][query] querying a non-existing key does not ...

2017-01-23 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3142 Ok, let's not introduce the (now deprecated) default values in the queryable state API. I'll create a new Jira and PR for removing that part from the RocksDB back-end and consistently return `null

[GitHub] flink pull request #3194: [FLINK-5615][query] execute the QueryableStateITCa...

2017-01-23 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3194 [FLINK-5615][query] execute the QueryableStateITCase for all three state back-ends This extends the `QueryableStateITCase` so that it is able to run with any selected state backend. Some

[GitHub] flink pull request #3193: [FLINK-5527][query] querying a non-existing key is...

2017-01-23 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3193 [FLINK-5527][query] querying a non-existing key is inconsistent among state backends Querying for a non-existing key for a state that has a default value set currently results

[GitHub] flink pull request #3218: [FLINK-5642][query] fix a race condition with Head...

2017-01-26 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3218 [FLINK-5642][query] fix a race condition with HeadListState The idiom behind `AppendingState#get()` is to return a copy of the value behind or at least not to allow changes to the underlying state

[GitHub] flink pull request #3222: [FLINK-5666] add unit tests verifying that BlobSer...

2017-01-26 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3222 [FLINK-5666] add unit tests verifying that BlobServer#delete() deletes from HDFS this does not fix FLINK-5666 but adds some more unit tests verifying intended behaviour You can merge this pull

[GitHub] flink pull request #3348: [FLINK-5090] [network] Add metrics for details abo...

2017-02-21 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3348#discussion_r102207017 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/ResultPartitionMetrics.java --- @@ -0,0 +1,136

[GitHub] flink pull request #3348: [FLINK-5090] [network] Add metrics for details abo...

2017-02-21 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3348#discussion_r102206831 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/InputGateMetrics.java --- @@ -0,0 +1,167

[GitHub] flink pull request #3348: [FLINK-5090] [network] Add metrics for details abo...

2017-02-21 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3348#discussion_r102223763 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/ConfigConstants.java --- @@ -227,6 +227,14 @@ public static final String

[GitHub] flink pull request #3348: [FLINK-5090] [network] Add metrics for details abo...

2017-02-17 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3348 [FLINK-5090] [network] Add metrics for details about inbound/outbound network queues These metrics are optimised go go through the channels only once in order to gather all metrics, i.e. min, max

[GitHub] flink pull request #3308: [FLINK-5796] fix some broken links in the docs

2017-02-14 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3308 [FLINK-5796] fix some broken links in the docs this probably also applies to the release-1.2 docs You can merge this pull request into a Git repository by running: $ git pull https://github.com

[GitHub] flink issue #3293: [FLINK-5745] set an uncaught exception handler for netty ...

2017-02-14 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3293 @StephanEwen already did when #3290 got in --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] flink pull request #3298: [FLINK-5672] add special cases for a local setup i...

2017-02-14 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3298#discussion_r101008195 --- Diff: flink-dist/src/main/flink-bin/bin/stop-cluster.sh --- @@ -25,14 +25,30 @@ bin=`cd "$bin"; pwd` # Stop TaskManager instance(s)

[GitHub] flink pull request #3298: [FLINK-5672] add special cases for a local setup i...

2017-02-14 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3298#discussion_r101008339 --- Diff: flink-dist/src/main/flink-bin/bin/stop-cluster.sh --- @@ -25,14 +25,30 @@ bin=`cd "$bin"; pwd` # Stop TaskManager instance(s)

[GitHub] flink pull request #3309: [FLINK-5277] add unit tests for ResultPartition#ad...

2017-02-14 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3309 [FLINK-5277] add unit tests for ResultPartition#add() in case of failures This verifies that the given network buffer is recycled as expected and that no notifiers are called upon failures to add

[GitHub] flink pull request #3298: [FLINK-5672] add special cases for a local setup i...

2017-02-13 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3298 [FLINK-5672] add special cases for a local setup in cluster start/stop scripts With this PR, if all slaves refer to `"localhost"` we run the daemons from the script itself instead of using

[GitHub] flink pull request #3299: [FLINK-5553] keep the original throwable in Partit...

2017-02-13 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3299 [FLINK-5553] keep the original throwable in PartitionRequestClientHandler This way, when checking for a previous error in any input channel, we can throw a meaningful exception instead

[GitHub] flink pull request #3290: [FLINK-5759] [jobmanager] Set UncaughtExceptionHan...

2017-02-10 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3290#discussion_r100511182 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/jobmaster/JobManagerServices.java --- @@ -116,12 +116,17 @@ public static JobManagerServices

[GitHub] flink pull request #3293: [FLINK-5745] set an uncaught exception handler for...

2017-02-10 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3293 [FLINK-5745] set an uncaught exception handler for netty threads This adds a JVM-terminating handler that logs errors from uncaught exceptions and terminates the process so that critical

[GitHub] flink issue #3293: [FLINK-5745] set an uncaught exception handler for netty ...

2017-02-10 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3293 @uce I'll extract the inner class and use it here as well as soon as the final #3290 is merged --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] flink issue #3293: [FLINK-5745] set an uncaught exception handler for netty ...

2017-02-10 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3293 wouldn't it be `NettyServer$FatalExitExceptionHandler` vs. `ExecutorThreadFactory$FatalExitExceptionHandler`? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] flink issue #3293: [FLINK-5745] set an uncaught exception handler for netty ...

2017-02-10 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3293 would be a different LOG handler though - does it make sense to have two or is it enough to have a single one in an outer class? --- If your project is set up for it, you can reply to this email

[GitHub] flink issue #3293: [FLINK-5745] set an uncaught exception handler for netty ...

2017-02-10 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3293 I was actually looking through the code to find something like this but it seems that every class does this locally for now. Global exit codes make sense though - also for documentation --- If your

[GitHub] flink pull request #3331: [FLINK-5814] fix packaging flink-dist in unclean s...

2017-02-16 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3331 [FLINK-5814] fix packaging flink-dist in unclean source directory If `/build-target` already existed, running `mvn package` for flink-dist would create a symbolic link inside `/build-target

[GitHub] flink issue #3322: [FLINK-4813][flink-test-utils] make the hadoop-minikdc de...

2017-02-16 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3322 sure, that makes sense actually, I only had to add it to the flink-test-utils sub-project since all the others already included the bundler :) --- If your project is set up for it, you can reply

[GitHub] flink pull request #3322: [FLINK-4813][flink-test-utils] make the hadoop-min...

2017-02-15 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3322 [FLINK-4813][flink-test-utils] make the hadoop-minikdc dependency optional This removes the need to add the `maven-bundle-plugin`plugin for most projects using `flink-test-utils`. Instead

[GitHub] flink issue #3341: [FLINK-5817]Fix test concurrent execution failure by test...

2017-02-17 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3341 Thanks, this looks like a really nice addition and simplifies the code a lot. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink pull request #3172: [FLINK-5559] let KvStateRequestSerializer#deserial...

2017-01-19 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3172 [FLINK-5559] let KvStateRequestSerializer#deserializeKeyAndNamespace() throw a proper IOException This adds the hint that a deserialisation failure probably results from a `"mismatch in th

[GitHub] flink pull request #3171: [FLINK-5561] fix DataInputDeserializer#available()...

2017-01-19 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3171 [FLINK-5561] fix DataInputDeserializer#available() 1 smaller than correct This also adds a unit test for `DataInputDeserializer#available()` - the first one for `DataInputDeserializer` unfortunately

[GitHub] flink pull request #3139: [FLINK-5528][query][tests] reduce the retry delay ...

2017-01-17 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3139 [FLINK-5528][query][tests] reduce the retry delay in QueryableStateITCase Using 100ms instead of the 1s previously used does not impose too much additional query load and reduces the test suite's

[GitHub] flink pull request #3142: [FLINK-5527][query] querying a non-existing key do...

2017-01-17 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3142 [FLINK-5527][query] querying a non-existing key does not return the default value Querying for a non-existing key for a state that has a default value set currently results

[GitHub] flink pull request #3143: [FLINK-5530] fix race condition in AbstractRocksDB...

2017-01-17 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3143 [FLINK-5530] fix race condition in AbstractRocksDBState#getSerializedValue `AbstractRocksDBState#getSerializedValue()` uses the same key serialisation stream as the ordinary state access methods

[GitHub] flink pull request #3129: [FLINK-5507] remove KeyedStream#asQueryableState(n...

2017-01-16 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3129 [FLINK-5507] remove KeyedStream#asQueryableState(name, ListStateDescriptor) The queryable state "sink" using ListState stores all incoming data forever and is never cleaned. Eventually, it

[GitHub] flink pull request #3131: [FLINK-5515] remove unused kvState.getSerializedVa...

2017-01-16 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3131 [FLINK-5515] remove unused kvState.getSerializedValue call in KvStateServerHandler this seems like a simple left-over from a merge that is doing unnecessary extra work You can merge this pull

[GitHub] flink pull request #3135: [FLINK-5521] remove unused KvStateRequestSerialize...

2017-01-17 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3135 [FLINK-5521] remove unused KvStateRequestSerializer#serializeList Also make sure that the serialization via the state backends' list states matches the deserialization

[GitHub] flink issue #3142: [FLINK-5527][query] querying a non-existing key does not ...

2017-01-18 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3142 I saw that deprecation but nonetheless the default value is exposed which is why a consistent behaviour is needed. Since the state descriptor says "that is the value if nothing is se

[GitHub] flink pull request #3143: [FLINK-5530] fix race condition in AbstractRocksDB...

2017-01-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3143#discussion_r96638451 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/state/StateBackendTestBase.java --- @@ -242,6 +245,132 @@ public void testValueState() throws

[GitHub] flink pull request #3143: [FLINK-5530] fix race condition in AbstractRocksDB...

2017-01-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3143#discussion_r96638478 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/AbstractRocksDBState.java --- @@ -132,55 +132,95

[GitHub] flink pull request #3143: [FLINK-5530] fix race condition in AbstractRocksDB...

2017-01-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3143#discussion_r96638710 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/AbstractRocksDBState.java --- @@ -132,55 +132,95

[GitHub] flink pull request #3143: [FLINK-5530] fix race condition in AbstractRocksDB...

2017-01-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3143#discussion_r96638594 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/AbstractRocksDBState.java --- @@ -132,55 +132,95

[GitHub] flink pull request #3171: [FLINK-5561] fix DataInputDeserializer#available()...

2017-01-20 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3171#discussion_r97063511 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/util/DataInputDeserializerTest.java --- @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request #3143: [FLINK-5530] fix race condition in AbstractRocksDB...

2017-01-20 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3143#discussion_r97065423 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/AbstractRocksDBState.java --- @@ -132,55 +132,91

[GitHub] flink pull request #3174: [FLINK-5576] extend deserialization functions of K...

2017-01-19 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3174 [FLINK-5576] extend deserialization functions of KvStateRequestSerializer to detect unconsumed bytes `KvStateRequestSerializer#deserializeValue()` deserializes a given byte array. This is used

[GitHub] flink issue #3348: [FLINK-5090] [network] Add metrics for details about inbo...

2017-02-27 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3348 Right, that was missing indeed. I also found some bugs and useful extensions / inconsistencies that I fixed. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] flink pull request #2778: [hotfix] fix duplicate "ms" time unit

2016-11-09 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/2778 [hotfix] fix duplicate "ms" time unit as in "Restart with fixed delay (1 ms ms)." in the web interface under "Max. number of execution retries" (org.apache.flin

[GitHub] flink pull request #2764: [FLINK-5008] Update quickstart documentation

2016-11-07 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/2764 [FLINK-5008] Update quickstart documentation This PR updates the outdated quickstart guides regarding IDE setup and the first example. You can merge this pull request into a Git repository

[GitHub] flink pull request #2805: [FLINK-5059] only serialise events once in RecordW...

2016-11-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/2805#discussion_r87965585 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java --- @@ -586,7 +588,18 @@ private boolean

[GitHub] flink pull request #2890: [hotfix] properly encapsulate the original excepti...

2016-11-28 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/2890 [hotfix] properly encapsulate the original exception in JobClient In the job client, an exception was re-thrown without including the original exception. This commit adds the original exception. You

[GitHub] flink pull request #2891: [FLINK-5129] make the BlobServer use a distributed...

2016-11-28 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/2891 [FLINK-5129] make the BlobServer use a distributed file system Previously, the BlobServer held a local copy and in case high availability (HA) is set, it also copied jar files to a distributed

[GitHub] flink issue #2764: [FLINK-5008] Update quickstart documentation

2016-11-15 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/2764 I tried with several versions of Eclipse and Scala IDE, even with the one claimed to work. Unfortunately, I got none of them to work. --- If your project is set up for it, you can reply to this email

[GitHub] flink pull request #2829: Hotfix 2016 11 18

2016-11-18 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/2829 Hotfix 2016 11 18 Prevent RecordWriter#flush() to clear the serializer twice. Also add some documentation to RecordWriter, RecordSerializer and SpanningRecordSerializer. You can merge this pull

[GitHub] flink pull request #2829: Hotfix 2016 11 18

2016-11-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/2829#discussion_r88649470 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/serialization/SpanningRecordSerializer.java --- @@ -151,6 +176,15 @@ private

[GitHub] flink issue #2829: [hotfix] prevent RecordWriter#flush() to clear the serial...

2016-11-18 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/2829 I don't expect this to change any behaviour as clearing the serializer twice does actually not hurt and is only some waste of resources so FLINK-4719 should not be affected at all --- If your

[GitHub] flink pull request #2805: [FLINK-5059] only serialise events once in RecordW...

2016-11-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/2805#discussion_r87968119 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java --- @@ -586,7 +588,18 @@ private boolean

[GitHub] flink pull request #2805: Flink 5059

2016-11-14 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/2805 Flink 5059 Only serialise events once in RecordWriter#broadcastEvent. While adapting this, also clean up some related APIs which is now unused or used similar patterns. You can merge

[GitHub] flink issue #2806: [FLINK-5066] Prevent LocalInputChannel#getNextBuffer from...

2016-12-08 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/2806 done, and yes, the code is now not relevant for `LocalInputChannel` anymore but for `PartitionRequestQueue` instead --- If your project is set up for it, you can reply to this email and have your

[GitHub] flink issue #2829: [hotfix] prevent RecordWriter#flush() to clear the serial...

2016-12-08 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/2829 I put a bit more emphasis on that fact in the new docs. I'd say, that's enough and after reading the docs, the difference should be clear. --- If your project is set up for it, you can reply

[GitHub] flink pull request #2764: [FLINK-5008] Update quickstart documentation

2016-12-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/2764#discussion_r92608012 --- Diff: docs/quickstart/run_example_quickstart.md --- @@ -90,23 +92,23 @@ use it in our program. Edit the `dependencies` section so that it looks like thi

[GitHub] flink pull request #2764: [FLINK-5008] Update quickstart documentation

2016-12-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/2764#discussion_r92608383 --- Diff: docs/quickstart/java_api_quickstart.md --- @@ -46,39 +46,79 @@ Use one of the following commands to __create a project__: {% highlight bash

[GitHub] flink pull request #2764: [FLINK-5008] Update quickstart documentation

2016-12-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/2764#discussion_r92611981 --- Diff: docs/quickstart/java_api_quickstart.md --- @@ -46,39 +46,79 @@ Use one of the following commands to __create a project__: {% highlight bash

[GitHub] flink issue #2764: [FLINK-5008] Update quickstart documentation

2016-12-13 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/2764 Developing Flink programs still works with Eclipse (tested with Eclipse 4.6.1 and Scala IDE 4.4.1 for Scala 2.11). Alongside testing the quickstarts, I also updated them as promised and made a switch

[GitHub] flink issue #2764: [FLINK-5008] Update quickstart documentation

2016-12-13 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/2764 I wasn't able to test the Scala SBT path though, so this may need some additional love by someone with a working SBT environment --- If your project is set up for it, you can reply to this email

[GitHub] flink pull request #2764: [FLINK-5008] Update quickstart documentation

2016-12-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/2764#discussion_r92619301 --- Diff: README.md --- @@ -104,25 +104,11 @@ Check out our [Setting up IntelliJ](https://github.com/apache/flink/blob/master/ ### Eclipse Scala IDE

[GitHub] flink pull request #:

2016-12-15 Thread NicoK
Github user NicoK commented on the pull request: https://github.com/apache/flink/commit/79d7e3017efe7c96e449e6f339fd7184ef3d1ba2#commitcomment-20200802 In docs/Gemfile: In docs/Gemfile on line 20: was it necessary to increase this dependency? --- If your project is set up

[GitHub] flink pull request #:

2016-12-15 Thread NicoK
Github user NicoK commented on the pull request: https://github.com/apache/flink/commit/79d7e3017efe7c96e449e6f339fd7184ef3d1ba2#commitcomment-20200919 In docs/Gemfile: In docs/Gemfile on line 23: seems that `./build_docs -p` is broken, i.e. it does neither enable auto

[GitHub] flink pull request #3120: [FLINK-5482] QueryableStateClient does not recover...

2017-01-13 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3120 [FLINK-5482] QueryableStateClient does not recover from a failed lookup due to a non-running job This PR checks each cached lookup query whether it is complete and removes any failed lookup from

[GitHub] flink issue #2764: [FLINK-5008] Update quickstart documentation

2016-11-29 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/2764 I'll look into writing Flink programs with Eclipse and update the documentation if needed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] flink pull request #2911: [FLINK-5178] allow BLOB_STORAGE_DIRECTORY_KEY to p...

2016-11-30 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/2911 [FLINK-5178] allow BLOB_STORAGE_DIRECTORY_KEY to point to a distributed file system Previously, this was restricted to a local file system path but now we can allow it to be distributed, too

[GitHub] flink issue #2911: [FLINK-5178] allow BLOB_STORAGE_DIRECTORY_KEY to point to...

2016-11-30 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/2911 @uce can you have a look after processing #2891 (FLINK-5129)? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink issue #2891: [FLINK-5129] make the BlobServer use a distributed file s...

2016-11-30 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/2891 Sorry for the hassle, found a regression and added a fix plus an appropriate test for it. Should be fine now. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] flink pull request #2891: [FLINK-5129] make the BlobServer use a distributed...

2017-01-05 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/2891 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink pull request #2911: [FLINK-5178] allow BLOB_STORAGE_DIRECTORY_KEY to p...

2017-01-05 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/2911 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink issue #2891: [FLINK-5129] make the BlobServer use a distributed file s...

2017-01-05 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/2891 I need to adapt a few things and choose a different approach - I'll re-open later --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] flink issue #2911: [FLINK-5178] allow BLOB_STORAGE_DIRECTORY_KEY to point to...

2017-01-05 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/2911 I need to adapt a few things and choose a different approach - I'll re-open later --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] flink pull request #3056: [FLINK-3150] make YARN container invocation config...

2017-01-03 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3056 [FLINK-3150] make YARN container invocation configurable By using the `yarn.container-start-command-template` configuration parameter, the Flink start command can be altered/extended. By default

[GitHub] flink pull request #3056: [FLINK-3150] make YARN container invocation config...

2017-01-05 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3056#discussion_r94798020 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java --- @@ -347,43 +351,88 @@ public static String

[GitHub] flink pull request #3084: [FLINK-5129] make the BlobServer use a distributed...

2017-01-09 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3084 [FLINK-5129] make the BlobServer use a distributed file system Make the BlobCache use the BlobServer's distributed file system in HA mode: previously even in HA mode and if the cache has access

[GitHub] flink issue #3076: [FLINK-5129] make the BlobServer use a distributed file s...

2017-01-09 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3076 fixed a typo in the unit test that lead to the tests passing although there was still something wrong which is now fixed as well --- If your project is set up for it, you can reply to this email

[GitHub] flink pull request #3076: [FLINK-5129] make the BlobServer use a distributed...

2017-01-09 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/3076 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink pull request #3085: [FLINK-5178] allow BlobCache to use a distributed ...

2017-01-09 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3085 [FLINK-5178] allow BlobCache to use a distributed file system irrespective of the HA mode Allow the BlobServer and BlobCache to use a distributed file system for distributing BLOBs even if not in HA

[GitHub] flink pull request #3076: [FLINK-5129] make the BlobServer use a distributed...

2017-01-06 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3076 [FLINK-5129] make the BlobServer use a distributed file system Make the BlobCache use the BlobServer's distributed file system in HA mode: previously even in HA mode and if the cache has access

[GitHub] flink issue #2891: [FLINK-5129] make the BlobServer use a distributed file s...

2016-12-22 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/2891 despite the tests completing successfully, I do still need to check a few things: - `BlobService#getURL()` may now return a URL for a distributed file system, however: - related code, e.g

[GitHub] flink pull request #2805: [FLINK-5059] only serialise events once in RecordW...

2016-12-07 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/2805#discussion_r91306180 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/writer/RecordWriter.java --- @@ -131,35 +132,30 @@ private void sendToTarget(T

[GitHub] flink pull request #3551: [FLINK-6064][flip6] fix BlobServer connection in T...

2017-03-15 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3551 [FLINK-6064][flip6] fix BlobServer connection in TaskExecutor The hostname used for the `BlobServer` was set to the akka address which is invalid for this use. Instead, this adds the hostname

[GitHub] flink pull request #3551: [FLINK-6064][flip6] fix BlobServer connection in T...

2017-03-16 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3551#discussion_r106376150 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rpc/akka/AkkaRpcService.java --- @@ -143,9 +142,17 @@ public C checkedApply(Object obj) throws

[GitHub] flink issue #3348: [FLINK-5090] [network] Add metrics for details about inbo...

2017-04-04 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3348 ok, sorry, these slipped through... please note however, that the not-null checks in #3610 become obsolete with this PR --- If your project is set up for it, you can reply to this email

[GitHub] flink issue #3348: [FLINK-5090] [network] Add metrics for details about inbo...

2017-04-03 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3348 @zentol can you have a look again? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink pull request #3713: [FLINK-6299] make all IT cases extend from TestLog...

2017-04-12 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3713 [FLINK-6299] make all IT cases extend from TestLogger This way, currently executed tests and their failures are properly logged. You can merge this pull request into a Git repository by running

[GitHub] flink pull request #3683: [FLINK-6270] Port several network config parameter...

2017-04-06 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3683 [FLINK-6270] Port several network config parameters to ConfigOption This ports some memory and network buffers related config options to new `ConfigOption` instances. These include

[GitHub] flink pull request #3683: [FLINK-6270] Port several network config parameter...

2017-04-06 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3683#discussion_r110106044 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java --- @@ -39,10 +39,53 @@ key

[GitHub] flink pull request #3683: [FLINK-6270] Port several network config parameter...

2017-04-06 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3683#discussion_r110108276 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/minicluster/LocalFlinkMiniCluster.scala --- @@ -350,23 +350,18 @@ class LocalFlinkMiniCluster

[GitHub] flink pull request #3708: [FLINK-6292] fix transfer.sh upload by using https

2017-04-11 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3708 [FLINK-6292] fix transfer.sh upload by using https Seems the upload via http is not supported anymore. You can merge this pull request into a Git repository by running: $ git pull https

[GitHub] flink issue #3512: [FLINK-6008] collection of BlobServer improvements

2017-04-19 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3512 * I removed the exposed `BlobService` from the `LibraryCacheManager` * Also, I developed a new cleanup story that removes blobs only if there are no tasks referring to the job ID anymore

[GitHub] flink pull request #3721: [FLINK-4545] replace the network buffers parameter

2017-04-19 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3721#discussion_r112236022 --- Diff: flink-dist/src/main/flink-bin/bin/config.sh --- @@ -398,3 +428,106 @@ readSlaves() { useOffHeapMemory() { [[ "`echo ${FLINK_TM_OF

  1   2   3   4   5   6   7   8   9   10   >