[GitHub] flink pull request: [FLINK-1419] [runtime] DC properly synchronize...

2015-01-26 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/339#issuecomment-71489135 but that is exactly what is changing, both the delete and copy process are synchronized on the same object. --- If your project is set up for it, you can reply

[GitHub] flink pull request: [FLINK-1419] [runtime] DC properly synchronize...

2015-01-26 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/339#issuecomment-71490079 oh i see what you mean, maybe extend the synchronized block to include the actual delete stuff. yup that's a good idea, all i know is i tried it without the change and ran

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-03 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-72652471 hmm...you are now the second person to report that creating the tmp files does not work on OS X. i don't know why that doesn't work. the file creation is done from java

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-02 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/202#discussion_r23955578 --- Diff: docs/python_programming_guide.md --- @@ -0,0 +1,600 @@ +--- +title: Python Programming Guide +--- +!-- +Licensed to the Apache

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-02 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/202#discussion_r23958310 --- Diff: flink-addons/flink-language-binding/src/main/java/org/apache/flink/languagebinding/api/java/common/OperationInfo.java --- @@ -0,0 +1,48

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-02 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/202#discussion_r23954389 --- Diff: flink-addons/flink-language-binding/src/main/java/org/apache/flink/languagebinding/api/java/common/OperationInfo.java --- @@ -0,0 +1,48

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-02 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-72537053 hard to find error messages what do you mean by that? what did you run the job with (data size, dop) ? --- If your project is set up for it, you can reply

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-03 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-72744279 about error messages going to command-line: the only way i see for that to work is by wrapping the complete error message into an exception, since they do show up

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-03 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-72746379 Ive added a potential fix for the /tmp path issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink pull request: [FLINK-1422] Add withParameters() to documenta...

2015-02-05 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/350#discussion_r24168173 --- Diff: docs/programming_guide.md --- @@ -2398,6 +2399,61 @@ of a function, or use the `withParameters(...)` method to pass in a configuratio [Back

[GitHub] flink pull request: [FLINK-785] Chained AllReduce

2015-02-06 Thread zentol
GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/370 [FLINK-785] Chained AllReduce This a a preliminary PR to see whether I'm on the right track. I'm wondering whether this would be everything needed to add a Chained AllReduce, before i

[GitHub] flink pull request: [FLINK-785] Chained AllReduce

2015-02-08 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/370#issuecomment-73404704 there's something funky going on with the tests here. i got 2 failing tests in ObjectReuseITCase

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-03 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-72752968 @qmlmoon sweet. @rmetzger errors should show up on the console now. and in the .out file. and i suppose by extension in the .log file aswell. --- If your

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-03 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/202#discussion_r23990492 --- Diff: flink-addons/flink-language-binding/src/main/python/org/apache/flink/languagebinding/api/python/dill/__diff.py --- @@ -0,0 +1,247

[GitHub] flink pull request: [FLINK-1398] Introduce extractSingleField() in...

2015-01-18 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/308#issuecomment-70413423 @StephanEwen would that look like this? ```java reverse(dataset.map(...)).filter(...) ``` Ive been thinking about the API overload issue a bit: could

[GitHub] flink pull request: [FLINK-785] Chained AllReduce

2015-02-11 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/370#issuecomment-73923184 sorry for not making it clear that i forced pushed some changes, forgot making a separate branch. the changes you made to the driver are already in, sorry

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-11 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-73928499 too bad, I'll revert the change. i assume that with your change to /etc/hosts it can now resolve your hostname to some address(this failed earlier

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-11 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-73936205 in my case, it returns this: ```python import socket socket.gethostname() 'Linux' socket.getaddrinfo(socket.gethostname(), None, socket.AF_INET

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-11 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-73945405 @mxm I've added another potential fix. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-10 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-73727421 ah probably something went wrong when adding debugging mode, gimme a sec... --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-10 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-73731632 if you don't mind, try the example again and let it run. both processes should timeout after 5 minutes throwing exceptions, hopefully pointing to the origin of the lock

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-10 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-73740648 now that's a lot more helpful. looks like there is some problem getting the hostname. I'll keep googling a bit, but so far it appears that your OS or rather it's network

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-10 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-73729595 hmm, works for me. could be the same issue robert reported. is a python process active? --- If your project is set up for it, you can reply to this email and have your

[GitHub] flink pull request: [FLINK-785] Chained AllReduce / AllGroupReduce...

2015-02-12 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/370#discussion_r24633778 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/operators/chaining/ChainedAllGroupReduceCombineDriver.java --- @@ -0,0 +1,118

[GitHub] flink pull request: [FLINK-1521] Chained operators respect reuse

2015-02-12 Thread zentol
GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/392 [FLINK-1521] Chained operators respect reuse You can merge this pull request into a Git repository by running: $ git pull https://github.com/zentol/incubator-flink flink-1521 Alternatively you

[GitHub] flink pull request: [FLINK-785] Chained AllReduce / AllGroupReduce...

2015-02-13 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/370#issuecomment-74240805 the SynchronouesChainedCombineDriver stores multiple records as well doesn't it? except up to a fixed size instead of # of elements. could we not do the same here

[GitHub] flink pull request: [FLINK-785] Chained AllReduce / AllGroupReduce...

2015-02-19 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/370#issuecomment-75030486 okay, I will the remove the chained combiner. I'm curious though, the issue description specifically mentions the AllGroupReduce greatly benefiting from a chained

[GitHub] flink pull request: [FLINK-1521] Chained operators respect reuse

2015-02-19 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/392#issuecomment-75029399 Alright, I`m closing this PR then. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-01-27 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-71611872 Tests run on travis (they don't right now because fabian merged something that changes the CSVInputFormat constructor, which breaks stuff on my end) but see: https

[GitHub] flink pull request: [FLINK-1419] [runtime] DC properly synchronize...

2015-01-27 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/339#issuecomment-71654749 both good points. I'll address them after lunch! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink pull request: [FLINK-1419] [runtime] DC properly synchronize...

2015-01-27 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/339#issuecomment-71638812 updated to include discussed changes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request: [FLINK-1419] [runtime] DC properly synchronize...

2015-01-28 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/339#issuecomment-71815201 updated to include discussed changes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request: [FLINK-1419] [runtime] DC properly synchronize...

2015-01-28 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/339#issuecomment-71825712 well you sure know how to keep me busy :) you are right about moving it back. Updated. --- If your project is set up for it, you can reply to this email and have

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-01-28 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-71880245 i don't have a clue about all this licensing stuff. what i have seen though is that spark uses a file that has a similar license as the dill library. (https://github.com

[GitHub] flink pull request: [FLINK-1419] [runtime] DC properly synchronize...

2015-01-26 Thread zentol
GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/339 [FLINK-1419] [runtime] DC properly synchronized Addresses the issue of files not being preserved in subsequent operations. You can merge this pull request into a Git repository by running: $ git

[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons (...

2015-02-10 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/335#issuecomment-73689455 in the pom.xml in flink-gelly; change artifactIdflink-addons/artifactId to flink-staging --- If your project is set up for it, you can reply to this email and have your

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-09 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-73505531 @qmlmoon has provided TPCH Query 3 / 10 and WebLogAnalysis examples --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-03-10 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/202#discussion_r26102129 --- Diff: flink-addons/flink-language-binding/src/main/python/org/apache/flink/languagebinding/api/python/dill/__diff.py --- @@ -0,0 +1,247

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-03-04 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-77140113 and here i thought i was being clever by swapping to built-in functions. addressed both issues. --- If your project is set up for it, you can reply to this email

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-03-03 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-76935408 @aljoscha Im quite undecided on that one. What i would like the most is having the language binding module remain where it is, with 2 submodules: flink-generic and flink

[GitHub] flink pull request: [FLINK-1422] Add withParameters() to documenta...

2015-01-29 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/350#issuecomment-72105374 gotcha, I'll address the points mentioned. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-04-14 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-92765309 Now uses TCP to exchange signals. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-04-20 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-94403584 yes that is correct. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-04-20 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-94397131 All issues that I'm aware of are resolved. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-04-20 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-94402062 @aljoscha Timeout is removed. Data transfer is still done with mapped file, access to these files is synchronized using TCP. Im not sure what you mean with your last

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-04-21 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-94771487 nah I'll do it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-04-21 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-94760254 @rmetzger Done. Unless you want me to merge commits as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-04-21 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-94774209 Done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink pull request: [FLINK-1924] Minor Refactoring

2015-04-22 Thread zentol
GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/616 [FLINK-1924] Minor Refactoring This PR resolves a few minor issues, including formatting simpler python process initialization renaming of the python connection following the switch to tcp

[GitHub] flink pull request: [FLINK-1927] [py] Operator distribution rework

2015-04-29 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/638#issuecomment-97389037 I didn't check performance, it shouldn't have any noticeable effect on it. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] flink pull request: [FLINK-1927] [py] Operator distribution rework

2015-04-29 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/638#issuecomment-97417603 @aljoscha that variable must be declared somewhere within the plan file. during the plan rebuild this would be executed as well, so i don't think this is a problem

[GitHub] flink pull request: [FLINK-1927] [py] Operator distribution rework

2015-04-29 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/638#issuecomment-97418740 oh snap i just noticed a big flaw... well let's put this PR on hold for a bit. I'm simply re executing the plan file on each node, but forgot to deal

[GitHub] flink pull request: [FLINK-1927] [py] Operator distribution rework

2015-04-28 Thread zentol
GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/638 [FLINK-1927] [py] Operator distribution rework Python operators are no longer serialized and instead rebuilt on each node. This also means that the dill library is no longer necessary. You can merge

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-04-08 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-90859901 I can't say for sure whether the current timeout is enough, we don't have enough data for that. we could make it configurable, that way a user can just increase

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-04-08 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-90865196 i guess it's nice to have in case the java side dies *somehow* without calling close() on the java function. --- If your project is set up for it, you can reply

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-04-08 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-90844274 The python process is now being terminated using kill -9, instead of process.destroy(). This should prevent python processes from lingering indefinitely. --- If your

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-04-08 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-90860600 the whole timeout thing may actually be no longer required with the changes to the process termination. --- If your project is set up for it, you can reply to this email

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-04-08 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-90921879 The timeout measures how long java or python are stuck in a blocking udp operation. this generally means how long it takes for the python side to compute one chunk of data

[GitHub] flink pull request: [FLINK-2037] Provide flink-python.jar in lib/

2015-05-20 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/691#issuecomment-103813090 Since it reverts to the way the python API worked in the past I'd say this is the most appropriate quick solution. having a separate cli frontend would indeed

[GitHub] flink pull request: [FLINK-1927] [py] Operator distribution rework

2015-05-20 Thread zentol
Github user zentol closed the pull request at: https://github.com/apache/flink/pull/638 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink pull request: [FLINK-1927] [py] Operator distribution rework

2015-05-20 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/638#issuecomment-103813351 Implementing this in a clean way has become trickier that i initially expected, as such I'll postpone it and close this PR for now. --- If your project is set up

[GitHub] flink pull request: [FLINK-2448]Clear cache file list in Execution...

2015-08-17 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/1031#issuecomment-131769530 Does this work properly with collect() calls? as in, would the following plan still work? `env = .. env.registerCacheFile() ... someSet.collect

[GitHub] flink pull request: [FLINK-2462] [streaming] Major cleanup of stre...

2015-08-16 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/1017#discussion_r37143081 --- Diff: flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/tasks/TwoInputStreamTask.java --- @@ -27,114

[GitHub] flink pull request: [FLINK-2530]optimize equal() of AcknowledgeChe...

2015-08-16 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/1024#issuecomment-131512254 i don't think these statements are equivalent. Assume that this.state == null and that.state != null. In the original version we evaluate that.state

[GitHub] flink pull request: [FLINK-2530]optimize equal() of AcknowledgeChe...

2015-08-16 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/1024#issuecomment-131534953 Looking at the pure logic this would work, but you can't remove that.state != null since that could result in a NullPointerException inside equals. --- If your project

[GitHub] flink pull request: [FLINK-2458][FLINK-2449]Access distributed cac...

2015-08-16 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/970#discussion_r37144152 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala --- @@ -897,7 +897,7 @@ class TaskManager

[GitHub] flink pull request: [FLINK-2458][FLINK-2449]Access distributed cac...

2015-08-16 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/970#discussion_r37144275 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala --- @@ -897,7 +897,7 @@ class TaskManager

[GitHub] flink pull request: [FLINK-2565] Support primitive Arrays as keys

2015-08-23 Thread zentol
GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/1043 [FLINK-2565] Support primitive Arrays as keys Adds a comparator and test for every primitive array type. Modifies the CustomType2 class in GroupingTest to retain a field with an unsupported type

[GitHub] flink pull request: [FLINK-2557] TypeExtractor properly returns Mi...

2015-08-23 Thread zentol
GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/1045 [FLINK-2557] TypeExtractor properly returns MissingTypeInfo This fix is not really obvious so let me explain: getParameterTye() is called from two different places in the TypeExtractor

[GitHub] flink pull request: [FLINK-2556] Refactor/Fix pre-flight Key valid...

2015-08-23 Thread zentol
GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/1044 [FLINK-2556] Refactor/Fix pre-flight Key validation Removed redundant key validation in DistinctOperator Keys constructors now make sure the type of every key is an instance of AtomicType

[GitHub] flink pull request: [FLINK-2557] TypeExtractor properly returns Mi...

2015-08-24 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/1045#issuecomment-134203619 InvalidTypesException (from here on abbreviated as ITE) is no longer unchecked. For this to work i had to make changes in surprisingly many classes so let's

[GitHub] flink pull request: [FLINK-2556] Refactor/Fix pre-flight Key valid...

2015-08-24 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/1044#discussion_r37766287 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/operators/Keys.java --- @@ -209,6 +209,9 @@ public ExpressionKeys(int[] groupingFields

[GitHub] flink pull request: [FLINK-2556] Refactor/Fix pre-flight Key valid...

2015-08-24 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/1044#discussion_r37766532 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/operators/Keys.java --- @@ -209,6 +209,9 @@ public ExpressionKeys(int[] groupingFields

[GitHub] flink pull request: [FLINK-2565] Support primitive Arrays as keys

2015-08-24 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/1043#issuecomment-134345186 I've added a test case to make sure a primitive array is accepted as a key. is that what you had in mind @tillrohrmann ? --- If your project is set up for it, you can

[GitHub] flink pull request: [FLINK-2448]Clear cache file list in Execution...

2015-08-20 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/1031#issuecomment-132917505 using clearsinks would cause the above example with collect() to fail, since this clears sinks aswell. (the sinks are only not cleared when getExecutionPlan is called

[GitHub] flink pull request: [FLINK-2077] [core] Rework Path class and add ...

2015-08-19 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/1035#issuecomment-132481210 The changes i see are: * removed hashCode() * moved makeQualified to a new file as a static method * reordered the remaining methods in Path

[GitHub] flink pull request: [FLINK-2534][RUNTIME]Improve in CompactingHash...

2015-08-17 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/1029#discussion_r37166847 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/operators/hash/CompactingHashTable.java --- @@ -223,7 +223,7 @@ public CompactingHashTable

[GitHub] flink pull request: [FLINK-2534][RUNTIME]Improve in CompactingHash...

2015-08-17 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/1029#discussion_r37166816 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/operators/hash/CompactingHashTable.java --- @@ -406,9 +403,7 @@ public void

[GitHub] flink pull request: [FLINK-2534][RUNTIME]Improve in CompactingHash...

2015-08-17 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/1029#discussion_r37167787 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/operators/hash/CompactingHashTable.java --- @@ -406,9 +403,7 @@ public void

[GitHub] flink pull request: [FLINK-2557] TypeExtractor properly returns Mi...

2015-08-24 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/1045#issuecomment-134129643 I agree, will get right on it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request: [FLINK-2565] Support primitive Arrays as keys

2015-08-24 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/1043#discussion_r37729559 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/typeutils/base/array/BooleanPrimitiveArrayComparator.java --- @@ -0,0 +1,56

[GitHub] flink pull request: [FLINK-1681] Remove Record API from jdbc modul...

2015-08-04 Thread zentol
GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/982 [FLINK-1681] Remove Record API from jdbc module You can merge this pull request into a Git repository by running: $ git pull https://github.com/zentol/flink 1681_jdbc Alternatively you can

[GitHub] flink pull request: [FLINK-1882] Removed RemotedCollector

2015-08-04 Thread zentol
GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/985 [FLINK-1882] Removed RemotedCollector You can merge this pull request into a Git repository by running: $ git pull https://github.com/zentol/flink 1882_remoteCollector Alternatively you can

[GitHub] flink pull request: [FLINK-2483]Add default branch of switch(sched...

2015-08-04 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/984#issuecomment-127595762 this change is not necessary imo. both in ExecutionGraph and JobGraph, scheduleMode is initialized to a proper ScheduleMode. setScheduleMode requires a ScheduleMode

[GitHub] flink pull request: [FLINK-2432] Custom serializer support

2015-08-05 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/962#discussion_r36296145 --- Diff: flink-staging/flink-language-binding/flink-language-binding-generic/src/main/java/org/apache/flink/languagebinding/api/java/common/streaming/Sender.java

[GitHub] flink pull request: Serialized String comparison, Unicode support

2015-08-05 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/4#issuecomment-128009698 oh well now this has been a while, let's see... from what i can tell the comparison doesn't work on code points but compares on single bytes, so it should

[GitHub] flink pull request: [FLINK-2432] Custom serializer support

2015-07-30 Thread zentol
GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/962 [FLINK-2432] Custom serializer support Users can now use custom serializers in the Python API. The registration is done using env.register_custom_type(type class,serializer instance

[GitHub] flink pull request: [FLINK-2431] Refactor PlanBinder/OperationInfo

2015-07-30 Thread zentol
GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/961 [FLINK-2431] Refactor PlanBinder/OperationInfo PlanBinder methods were restructured to make the class more readable. Keys are now stored as strings to simplify string-key-expression support

[GitHub] flink pull request: [FLINK-1927][py] Operator distribution rework

2015-07-29 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/931#issuecomment-125983186 Thanks for the review @mxm . I've addressed the cosmetic issue you mentioned, and added a small fix for a separate issue as well (error reporting was partially

[GitHub] flink pull request: [FLINK-2432] Custom serializer support

2015-08-06 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/962#discussion_r36395811 --- Diff: flink-staging/flink-language-binding/flink-python/src/test/python/org/apache/flink/languagebinding/api/python/flink/test/test_custom.py --- @@ -0,0

[GitHub] flink pull request: [FLINK-2490][FIX]Remove the retryForever check...

2015-08-06 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/992#issuecomment-128309202 If you think it was necessary why was your first step to remove it's usage... --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] flink pull request: [FLINK-2494 ]Fix StreamGraph getJobGraph bug

2015-08-06 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/998#issuecomment-128605743 i would assume that forceCheckpoint is supposed to do exactly that, enforce checkpointing regardless of its support. this change also means that if checkPointint

[GitHub] flink pull request: [FLINK-2494 ]Fix StreamGraph getJobGraph bug

2015-08-07 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/998#issuecomment-128641012 If you read the message of the UnsupportedOperationException you'll find this bit: The user can force enable state checkpoints with the reduced guarantees by calling

[GitHub] flink pull request: [FLINK-2432] Custom serializer support

2015-08-07 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/962#issuecomment-128681688 I've addressed the mentioned issues, and added a CUSTOM type constant, so that users don't have to constantly create new instances of their classes just to pass as a type

[GitHub] flink pull request: [FLINK-2458][FLINK-2449]Access distributed cac...

2015-08-07 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/970#discussion_r36512933 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/operators/CollectionExecutor.java --- @@ -501,4 +536,22 @@ public int getSuperstepNumber

[GitHub] flink pull request: [FLINK-2458][FLINK-2449]Access distributed cac...

2015-08-07 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/970#discussion_r36512186 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/functions/util/RuntimeUDFContext.java --- @@ -37,18 +37,17 @@ private final

[GitHub] flink pull request: [FLINK-2526]Add try-catch for task when it sto...

2015-08-15 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/1020#discussion_r37134398 --- Diff: flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/tasks/OneInputStreamTask.java --- @@ -100,9

[GitHub] flink pull request: [FLINK-2490][FIX]Remove the retryForever check...

2015-08-06 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/992#issuecomment-128301542 if you remove that check, retryForever is unused and can be removed completely. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] flink pull request: [FLINK-1927][py] Operator distribution rework

2015-07-22 Thread zentol
GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/931 [FLINK-1927][py] Operator distribution rework Python operators are no longer serialized and shipped across the cluster. Instead the plan file is executed on each node, followed by usage

[GitHub] flink pull request: [FLINK-2565] Support primitive Arrays as keys

2015-08-25 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/1043#issuecomment-134618042 @StephanEwen I've reimplemented hashCode() and compare() accordingly. --- If your project is set up for it, you can reply to this email and have your reply appear

  1   2   3   4   5   6   7   8   9   10   >