Re: [DISCUSS] Move away from Apache Maven as build tool

2017-11-27 Thread Jean-Baptiste Onofré
Yeah, especially, I think it would have been great to have a vote before merging on master. Not a big deal, however, I'm really community focus ;) Regards JB On 11/28/2017 07:36 AM, Reuven Lax wrote: Agreed. I thinking having a formal vote before Luke had numbers and results would have been t

Re: [DISCUSS] Move away from Apache Maven as build tool

2017-11-27 Thread Reuven Lax
Agreed. I thinking having a formal vote before Luke had numbers and results would have been too early. However now that we have such numbers, we should think about having a vote. Also, while I disagree with Romain that Gradle is not "enterprise ready" (it's heavily used by Netflix, LinkedIn, Sieme

Re: [DISCUSS] Move away from Apache Maven as build tool

2017-11-27 Thread Jean-Baptiste Onofré
Hi Luke, just curious (and maybe I missed it): did we do a formal vote to merge the gradle build ? Gradle is now on master, we have some Jira to update the release guide with gradle. It's fine, but I remember only a discussion, not a vote. In order to embrace the community and avoid to have s

Re: [DISCUSS] Move away from Apache Maven as build tool

2017-11-27 Thread Romain Manni-Bucau
Le 27 nov. 2017 23:07, "Eugene Kirpichov" a écrit : On Mon, Nov 27, 2017 at 11:52 AM Romain Manni-Bucau wrote: > 2017-11-27 20:26 GMT+01:00 Lukasz Cwik : > > Romain, as mentioned earlier, I identified that Maven was slower because > it > > needed to finish building the entire module before depe

Re: [discuss] java profile

2017-11-27 Thread Romain Manni-Bucau
Hmm, no. Incremental build is never correctly implemented cause there is just no way to detect some dependencies statically with java code - or any dynamic language. Side note: same applies for gradle daemon usage BTW. After if the list is not maintained it is a bug at the same level than coding

Re: GitBox down

2017-11-27 Thread Jean-Baptiste Onofré
Good point. I will prepare a PR for that. Thanks ! Regards JB On 11/27/2017 08:58 PM, Reuven Lax wrote: We need to update the contribution guide as well. On Nov 27, 2017 10:30 AM, "Jean-Baptiste Onofré" wrote: No I don't think so, In your case, it's because you still refering git-wip-us (

[GitHub] kennknowles commented on issue #3538: Add a test for Avro write with RVP; fix code

2017-11-27 Thread GitBox
kennknowles commented on issue #3538: Add a test for Avro write with RVP; fix code URL: https://github.com/apache/beam/pull/3538#issuecomment-347408240 There are conflicts introduced - would you mind rebasing? That would also give more recent signal for our use of the GitHub merge button,

[GitHub] kennknowles commented on issue #4168: [BEAM-3238][SQL] Add BeamRecordSqlTypeBuilder

2017-11-27 Thread GitBox
kennknowles commented on issue #4168: [BEAM-3238][SQL] Add BeamRecordSqlTypeBuilder URL: https://github.com/apache/beam/pull/4168#issuecomment-347407859 @xumingming what do you think now? This is an automated message from the

[GitHub] kennknowles closed pull request #3677: Tez runner

2017-11-27 Thread GitBox
kennknowles closed pull request #3677: Tez runner URL: https://github.com/apache/beam/pull/3677 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the

[GitHub] kennknowles commented on issue #3677: Tez runner

2017-11-27 Thread GitBox
kennknowles commented on issue #3677: Tez runner URL: https://github.com/apache/beam/pull/3677#issuecomment-347407078 Now that we have migrated to gitbox, I can close this. I am going to do so, since it is on the actual target branch of `tez-runner`. ---

[GitHub] kennknowles closed pull request #4024: [BEAM-2304] Allow declared state to be accessed as a superclass.

2017-11-27 Thread GitBox
kennknowles closed pull request #4024: [BEAM-2304] Allow declared state to be accessed as a superclass. URL: https://github.com/apache/beam/pull/4024 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance:

[GitHub] kennknowles closed pull request #4153: [BEAM-3219] DataflowRunner: delegate @Setup and @Teardown in stateful ParDo

2017-11-27 Thread GitBox
kennknowles closed pull request #4153: [BEAM-3219] DataflowRunner: delegate @Setup and @Teardown in stateful ParDo URL: https://github.com/apache/beam/pull/4153 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of

[GitHub] pabloem commented on issue #4180: Updating dataflow API version to newer release.

2017-11-27 Thread GitBox
pabloem commented on issue #4180: Updating dataflow API version to newer release. URL: https://github.com/apache/beam/pull/4180#issuecomment-347406119 Retest this please This is an automated message from the Apache Git Servic

[GitHub] pabloem commented on issue #4180: Updating dataflow API version to newer release.

2017-11-27 Thread GitBox
pabloem commented on issue #4180: Updating dataflow API version to newer release. URL: https://github.com/apache/beam/pull/4180#issuecomment-347405675 retest this This is an automated message from the Apache Git Service. To r

[GitHub] pabloem commented on issue #4180: Updating dataflow API version to newer release.

2017-11-27 Thread GitBox
pabloem commented on issue #4180: Updating dataflow API version to newer release. URL: https://github.com/apache/beam/pull/4180#issuecomment-347405675 retest this This is an automated message from the Apache Git Service. To r

[GitHub] kennknowles commented on issue #4153: [BEAM-3219] DataflowRunner: delegate @Setup and @Teardown in stateful ParDo

2017-11-27 Thread GitBox
kennknowles commented on issue #4153: [BEAM-3219] DataflowRunner: delegate @Setup and @Teardown in stateful ParDo URL: https://github.com/apache/beam/pull/4153#issuecomment-347399933 run dataflow ValidatesRunner This is an au

[GitHub] luke-zhu commented on issue #4176: [BEAM-3143] Type Inference Compatibility with Python 3

2017-11-27 Thread GitBox
luke-zhu commented on issue #4176: [BEAM-3143] Type Inference Compatibility with Python 3 URL: https://github.com/apache/beam/pull/4176#issuecomment-347397998 After another look, I think that the base of this solution won't improve Python 3 compatibility in the long run. I'll submit a more

[GitHub] luke-zhu closed pull request #4176: [BEAM-3143] Type Inference Compatibility with Python 3

2017-11-27 Thread GitBox
luke-zhu closed pull request #4176: [BEAM-3143] Type Inference Compatibility with Python 3 URL: https://github.com/apache/beam/pull/4176 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is

[GitHub] asfgit closed pull request #4182: Add license header to SideInputTranslationTest

2017-11-27 Thread GitBox
asfgit closed pull request #4182: Add license header to SideInputTranslationTest URL: https://github.com/apache/beam/pull/4182 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign p

[GitHub] kennknowles commented on issue #4182: Add license header to SideInputTranslationTest

2017-11-27 Thread GitBox
kennknowles commented on issue #4182: Add license header to SideInputTranslationTest URL: https://github.com/apache/beam/pull/4182#issuecomment-347395117 CC: @jkff @tweise This is an automated message from the Apache Git Serv

[GitHub] kennknowles opened a new pull request #4182: Add license header to SideInputTranslationTest

2017-11-27 Thread GitBox
kennknowles opened a new pull request #4182: Add license header to SideInputTranslationTest URL: https://github.com/apache/beam/pull/4182 Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apach

[GitHub] kennknowles commented on a change in pull request #4153: [BEAM-3219] DataflowRunner: delegate @Setup and @Teardown in stateful ParDo

2017-11-27 Thread GitBox
kennknowles commented on a change in pull request #4153: [BEAM-3219] DataflowRunner: delegate @Setup and @Teardown in stateful ParDo URL: https://github.com/apache/beam/pull/4153#discussion_r153360051 ## File path: sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ParDoLi

[GitHub] kennknowles commented on a change in pull request #4153: [BEAM-3219] DataflowRunner: delegate @Setup and @Teardown in stateful ParDo

2017-11-27 Thread GitBox
kennknowles commented on a change in pull request #4153: [BEAM-3219] DataflowRunner: delegate @Setup and @Teardown in stateful ParDo URL: https://github.com/apache/beam/pull/4153#discussion_r153380128 ## File path: sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ParDoLi

[GitHub] jkff commented on issue #4074: [BEAM-3130] View.asMap() causes a ClassCastException in Apex runner

2017-11-27 Thread GitBox
jkff commented on issue #4074: [BEAM-3130] View.asMap() causes a ClassCastException in Apex runner URL: https://github.com/apache/beam/pull/4074#issuecomment-347389329 Fair enough, the current PR is an improvement in any case. Simplified the test a little bit and merged. -

[GitHub] asfgit closed pull request #4074: [BEAM-3130] View.asMap() causes a ClassCastException in Apex runner

2017-11-27 Thread GitBox
asfgit closed pull request #4074: [BEAM-3130] View.asMap() causes a ClassCastException in Apex runner URL: https://github.com/apache/beam/pull/4074 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance:

[GitHub] chamikaramj commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO

2017-11-27 Thread GitBox
chamikaramj commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO URL: https://github.com/apache/beam/pull/4169#discussion_r153376819 ## File path: sdks/java/io/file-based-io-tests/pom.xml ## @@ -139,6 +139,24 @@

[GitHub] chamikaramj commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO

2017-11-27 Thread GitBox
chamikaramj commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO URL: https://github.com/apache/beam/pull/4169#discussion_r153375847 ## File path: sdks/java/io/common/src/test/java/org/apache/beam/sdk/io/common/IOTestPipelineOpti

[GitHub] chamikaramj commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO

2017-11-27 Thread GitBox
chamikaramj commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO URL: https://github.com/apache/beam/pull/4169#discussion_r153376493 ## File path: sdks/java/io/file-based-io-tests/src/test/java/org/apache/beam/sdk/io/text/TextIOI

Re: SerializableCoder Structured Value

2017-11-27 Thread Kenneth Knowles
What I said is not quite right - there are accidental collisions allowed. The "all coders" spec for structural value only requires that encode(a) == encode(b) implies sv(a).equals(sv(b)). The converse is not required. For example, the nondeterministic SetCoder can use the Set objects themselves as

Re: SerializableCoder Structured Value

2017-11-27 Thread Kenneth Knowles
To add some flavor, *All coders:* structuralValue(a).equals(structuralValue(b)) if and only if encode(a) == encode(b) *"Consistent with equals" aka injective:* encode(a) == encode(b) implies a.equals(b) *Deterministic:* a.equals(b) implies structuralValue(a).equals(structuralValue(b)) (hence enc

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347381845 Run Python Performance Test This is an automated

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347381795 Run Python Performance Test This is an automa

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347381328 run seed job This is an automated message from th

[GitHub] robertwb closed pull request #4181: Fix typo in proto files.

2017-11-27 Thread GitBox
robertwb closed pull request #4181: Fix typo in proto files. URL: https://github.com/apache/beam/pull/4181 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347378789 Run Python Performance Test This is an automated

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347376775 Run Python Dataflow ValidatesRunner This is an au

[GitHub] herohde commented on issue #4181: Fix typo in proto files.

2017-11-27 Thread GitBox
herohde commented on issue #4181: Fix typo in proto files. URL: https://github.com/apache/beam/pull/4181#issuecomment-347376582 LGTM This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

2017-11-27 Thread GitBox
jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153365677 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +201,49 @@ publ

Re: SerializableCoder Structured Value

2017-11-27 Thread Mairbek Khadikov
I'm open to renaming *consistentWithEquals*. If I understand the code correctly, when consistentWithEquals returns true, org.apache.beam.sdk.util.MutationDetectors expects *a.equals(deserialize(serialize(a))* which I think is reasonable for SerializableCoder (assuming objects implement equals)*. *

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347374422 Run Python PostCommit This is an automated messag

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347372780 Run Python PostCommit This is an automated messag

[GitHub] robertwb commented on issue #4181: Fix typo in proto files.

2017-11-27 Thread GitBox
robertwb commented on issue #4181: Fix typo in proto files. URL: https://github.com/apache/beam/pull/4181#issuecomment-347372394 R: @herohde This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] robertwb opened a new pull request #4181: Fix typo in proto files.

2017-11-27 Thread GitBox
robertwb opened a new pull request #4181: Fix typo in proto files. URL: https://github.com/apache/beam/pull/4181 Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/i

Re: [Build] Enforce failed on project beam-sdks-java-io-tika

2017-11-27 Thread Manu Zhang
Thanks, the debug information pointed me to the corrupted geoapi-3.0.0.jar file. Removing it solves the problem for me. Manu On Tue, Nov 28, 2017 at 4:41 AM Lukasz Cwik wrote: > Try running with -X to get all the debug output when using Maven. It may > give more details. > > Alternatively, try

Re: [VOTE] Fixing @yyy.com.INVALID mailing addresses

2017-11-27 Thread Lukasz Cwik
I have confirmed that I am no longer impacted by the .INVALID suffix. On Mon, Nov 27, 2017 at 3:06 PM, Lukasz Cwik wrote: > Apache infra has mentioned that they have updated the mailing list to > prevent the ".INVALID" from appearing on the end. dev@ now matches user@ > in this regard. > > On Mo

Re: SerializableCoder Structured Value

2017-11-27 Thread Lukasz Cwik
I think the idea is that SerializableCoder should be updated to expect that all values it encodes do implement equals() since this seems to be the much more common case then classes that don't implement a useful equals. It would be possible to add a useful check to DirectRunner that any value that

Re: SerializableCoder Structured Value

2017-11-27 Thread Eugene Kirpichov
Not sure where you see the contradiction? consistentWithEquals says "Whenever the encoded bytes of two values are equal, then the original values are equal according to {@code Objects.equals()}." - which is clearly false for Serializable's in general: it's possible that serialized form of "a" and "

SerializableCoder Structured Value

2017-11-27 Thread Mairbek Khadikov
Hi all, Currently SerializableCoder#consistentWithEquals returns false, which contradicts it's own documentation "{@link SerializableCoder} does not guarantee a deterministic encoding, as Java serialization may produce different binary encodings for two equivalent objects". In practice, it leads

[GitHub] kennknowles commented on issue #4135: [BEAM-3194] Add @RequiresStableInput annotation

2017-11-27 Thread GitBox
kennknowles commented on issue #4135: [BEAM-3194] Add @RequiresStableInput annotation URL: https://github.com/apache/beam/pull/4135#issuecomment-347367940 Thanks! Squashed in the fixup and re-running tests since there looked like a failure that I can't imagine this PR caused.

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347364157 run seed job This is an automated message from th

[GitHub] kennknowles commented on issue #4105: [BEAM-2899] Fork FnDataService from runners-core

2017-11-27 Thread GitBox
kennknowles commented on issue #4105: [BEAM-2899] Fork FnDataService from runners-core URL: https://github.com/apache/beam/pull/4105#issuecomment-347361536 LGTM This is an automated message from the Apache Git Service. To re

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347359996 Run Python Performance Test This is an automated

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347359648 Run Python Dataflow ValidatesRunner This is an au

[GitHub] tweise commented on issue #4074: [BEAM-3130] View.asMap() causes a ClassCastException in Apex runner

2017-11-27 Thread GitBox
tweise commented on issue #4074: [BEAM-3130] View.asMap() causes a ClassCastException in Apex runner URL: https://github.com/apache/beam/pull/4074#issuecomment-347359334 @jkff I'm aware of that issue as well. It is certainly important, but at the same time it's not new and I don't see why

Re: [VOTE] Fixing @yyy.com.INVALID mailing addresses

2017-11-27 Thread Lukasz Cwik
Apache infra has mentioned that they have updated the mailing list to prevent the ".INVALID" from appearing on the end. dev@ now matches user@ in this regard. On Mon, Nov 27, 2017 at 11:30 AM, Lukasz Cwik wrote: > Vote Closed: > all +1s with at least 3 binding votes. > > On Fri, Nov 24, 2017 at

[GitHub] tgroh closed pull request #4127: [BEAM-2899] Add a Logging Service to FnExecution

2017-11-27 Thread GitBox
tgroh closed pull request #4127: [BEAM-2899] Add a Logging Service to FnExecution URL: https://github.com/apache/beam/pull/4127 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign

[GitHub] nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

2017-11-27 Thread GitBox
nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153350918 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67 @

[GitHub] tgroh closed pull request #4151: [BEAM-2899] Decompose Direct Execution Components

2017-11-27 Thread GitBox
tgroh closed pull request #4151: [BEAM-2899] Decompose Direct Execution Components URL: https://github.com/apache/beam/pull/4151 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreig

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347357063 Run Python PostCommit This is an automated messag

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347356163 run seed job This is an automated message from th

[GitHub] jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

2017-11-27 Thread GitBox
jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153348845 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67 @@ publ

[GitHub] jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

2017-11-27 Thread GitBox
jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153348529 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67 @@ publ

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347354797 run seed job This is an automated message fro

[GitHub] jkff commented on issue #4074: [BEAM-3130] View.asMap() causes a ClassCastException in Apex runner

2017-11-27 Thread GitBox
jkff commented on issue #4074: [BEAM-3130] View.asMap() causes a ClassCastException in Apex runner URL: https://github.com/apache/beam/pull/4074#issuecomment-347351587 Thank you! While trying to merge this, I noticed a much more serious issue that I believe should be resolved before this o

[GitHub] nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

2017-11-27 Thread GitBox
nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153343674 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67 @

Re: [discuss] java profile

2017-11-27 Thread Lukasz Cwik
Manually whitelisting/blacklisting sub-modules is error prone since it hides issues due to incorrectly maintaining that list is the same argument as if the build process doesn't correctly invoke an incremental build process. On Mon, Nov 27, 2017 at 1:45 PM, Romain Manni-Bucau wrote: > Well for v

[GitHub] chamikaramj closed pull request #4149: [BEAM-3060] Add Compressed TextIOIT

2017-11-27 Thread GitBox
chamikaramj closed pull request #4149: [BEAM-3060] Add Compressed TextIOIT URL: https://github.com/apache/beam/pull/4149 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull re

[GitHub] chamikaramj commented on issue #4149: [BEAM-3060] Add Compressed TextIOIT

2017-11-27 Thread GitBox
chamikaramj commented on issue #4149: [BEAM-3060] Add Compressed TextIOIT URL: https://github.com/apache/beam/pull/4149#issuecomment-347345709 Merged. Closing. This is an automated message from the Apache Git Service. To respo

Re: [DISCUSS] Move away from Apache Maven as build tool

2017-11-27 Thread Eugene Kirpichov
On Mon, Nov 27, 2017 at 11:52 AM Romain Manni-Bucau wrote: > 2017-11-27 20:26 GMT+01:00 Lukasz Cwik : > > Romain, as mentioned earlier, I identified that Maven was slower because > it > > needed to finish building the entire module before dependent modules > could > > start which included running

Re: [DISCUSS] Move away from Apache Maven as build tool

2017-11-27 Thread Lukasz Cwik
I don't believe that such a switch could happen immediately as the scope of the change is to replace 15k lines of pom files so anything which isn't gradual is unlikely to work. Below is a more thorough list of points which compare the two build systems beyond speed. Maven Java Support: Mature Pyt

[GitHub] jkff commented on issue #4139: Update cloud spanner library to 0.29.0

2017-11-27 Thread GitBox
jkff commented on issue #4139: Update cloud spanner library to 0.29.0 URL: https://github.com/apache/beam/pull/4139#issuecomment-347342624 (bumping grpc version by +5 seems welcome but quite risky, hence the ValidatesRunner tests) ---

[GitHub] jkff commented on issue #4139: Update cloud spanner library to 0.29.0

2017-11-27 Thread GitBox
jkff commented on issue #4139: Update cloud spanner library to 0.29.0 URL: https://github.com/apache/beam/pull/4139#issuecomment-347342562 Run Dataflow ValidatesRunner This is an automated message from the Apache Git Service.

[GitHub] jkff commented on issue #4139: Update cloud spanner library to 0.29.0

2017-11-27 Thread GitBox
jkff commented on issue #4139: Update cloud spanner library to 0.29.0 URL: https://github.com/apache/beam/pull/4139#issuecomment-347342299 retest this please This is an automated message from the Apache Git Service. To respond

[GitHub] jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

2017-11-27 Thread GitBox
jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153335970 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67 @@ publ

[GitHub] jkff commented on issue #4172: [BEAM-3243] support multiple anonymous classes from the same enclosing class in a pipeline

2017-11-27 Thread GitBox
jkff commented on issue #4172: [BEAM-3243] support multiple anonymous classes from the same enclosing class in a pipeline URL: https://github.com/apache/beam/pull/4172#issuecomment-347341595 That would be optimal, and in general I really like the idea of capturing the stack trace of transf

Re: [DISCUSS] Move away from Apache Maven as build tool

2017-11-27 Thread Romain Manni-Bucau
Le 27 nov. 2017 21:34, "Lukasz Cwik" a écrit : On Mon, Nov 27, 2017 at 11:51 AM, Romain Manni-Bucau wrote: > 2017-11-27 20:26 GMT+01:00 Lukasz Cwik : > > Romain, as mentioned earlier, I identified that Maven was slower because > it > > needed to finish building the entire module before dependen

[GitHub] nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

2017-11-27 Thread GitBox
nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153334683 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67 @

[GitHub] jkff commented on a change in pull request #4145: Many simplifications to WriteFiles

2017-11-27 Thread GitBox
jkff commented on a change in pull request #4145: Many simplifications to WriteFiles URL: https://github.com/apache/beam/pull/4145#discussion_r153293639 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/io/WriteFiles.java ## @@ -527,13 +521,8 @@ public void pr

[GitHub] jkff commented on a change in pull request #4145: Many simplifications to WriteFiles

2017-11-27 Thread GitBox
jkff commented on a change in pull request #4145: Many simplifications to WriteFiles URL: https://github.com/apache/beam/pull/4145#discussion_r153291649 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/io/WriteFiles.java ## @@ -672,8 +661,11 @@ public void pr

[GitHub] jkff commented on a change in pull request #4145: Many simplifications to WriteFiles

2017-11-27 Thread GitBox
jkff commented on a change in pull request #4145: Many simplifications to WriteFiles URL: https://github.com/apache/beam/pull/4145#discussion_r153291927 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/io/WriteFiles.java ## @@ -824,177 +826,78 @@ public void

[GitHub] jkff commented on a change in pull request #4145: Many simplifications to WriteFiles

2017-11-27 Thread GitBox
jkff commented on a change in pull request #4145: Many simplifications to WriteFiles URL: https://github.com/apache/beam/pull/4145#discussion_r153291513 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileBasedSink.java ## @@ -686,40 +756,38 @@ public in

[GitHub] jkff commented on a change in pull request #4145: Many simplifications to WriteFiles

2017-11-27 Thread GitBox
jkff commented on a change in pull request #4145: Many simplifications to WriteFiles URL: https://github.com/apache/beam/pull/4145#discussion_r153294218 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/io/WriteFiles.java ## @@ -1011,14 +1002,19 @@ public void

Re: [discuss] java profile

2017-11-27 Thread Romain Manni-Bucau
Well for validation builds- pre PR, incremental support is pointless since it easily hides issues die to caching so a solution saving half of the build without loosing anuyhing would still be good IMHO. Le 27 nov. 2017 21:12, "Lukasz Cwik" a écrit : > Incremental builds aren't correctly setup ri

[GitHub] jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

2017-11-27 Thread GitBox
jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153332162 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67 @@ publ

[GitHub] nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

2017-11-27 Thread GitBox
nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153330138 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67 @

[GitHub] nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

2017-11-27 Thread GitBox
nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153329942 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67 @

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347333086 Run Python PostCommit This is an automated messag

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347332806 Run Python PostCommit This is an automated me

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347331790 Thanks, that's helpful. This is an automated mess

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347331807 run seed job This is an automated message from th

Re: [DISCUSS] Move away from Apache Maven as build tool

2017-11-27 Thread Ismaël Mejía
I have been a little bit out of the discussion on maven vs gradle because I was expecting the technical proof of concepts to evaluate the best approach. I deeply appreciate all the effort that Lukasz has put into the gradle version, and I also think that during the discussion Romain and others have

[GitHub] pabloem opened a new pull request #4180: Updating dataflow API version to newer release.

2017-11-27 Thread GitBox
pabloem opened a new pull request #4180: Updating dataflow API version to newer release. URL: https://github.com/apache/beam/pull/4180 r: @bjchambers This new release contains the changes to the Dataflow API protos for side input inter transform IO. ---

Re: Azure(ADLS) compatibility on Beam with Spark runner

2017-11-27 Thread Lukasz Cwik
Out of curiosity, does using the DirectRunner with ADL work for you? If not, then you'll be able to debug locally why its failing. On Fri, Nov 24, 2017 at 8:09 PM, Milan Chandna < milan.chan...@microsoft.com.invalid> wrote: > Hi JB, > > Thanks for the updates. > BTW I am myself in Microsoft but I

[GitHub] reuvenlax commented on issue #4116: [BEAM-2953] Part 1 of Multipart advanced timeseries examples

2017-11-27 Thread GitBox
reuvenlax commented on issue #4116: [BEAM-2953] Part 1 of Multipart advanced timeseries examples URL: https://github.com/apache/beam/pull/4116#issuecomment-347321463 R: @kennknowles This is an automated message from the Apac

Re: [Build] Enforce failed on project beam-sdks-java-io-tika

2017-11-27 Thread Lukasz Cwik
Try running with -X to get all the debug output when using Maven. It may give more details. Alternatively, try using the gradle build (from root of project directory): ./gradlew build On Fri, Nov 24, 2017 at 9:46 PM, Manu Zhang wrote: > Hi all, > > Has anyone seen this issue when building lat

Re: [DISCUSS] Move away from Apache Maven as build tool

2017-11-27 Thread Lukasz Cwik
On Mon, Nov 27, 2017 at 11:51 AM, Romain Manni-Bucau wrote: > 2017-11-27 20:26 GMT+01:00 Lukasz Cwik : > > Romain, as mentioned earlier, I identified that Maven was slower because > it > > needed to finish building the entire module before dependent modules > could > > start which included runnin

Re: [discuss] java profile

2017-11-27 Thread Lukasz Cwik
Incremental builds aren't correctly setup right now so your likely to see Python/Go rebuild even if there were no changes. See https://issues.apache.org/jira/browse/BEAM-3253 On Mon, Nov 27, 2017 at 11:46 AM, Romain Manni-Bucau wrote: > that was the goal: validate there was no side effect of the

[GitHub] kennknowles commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
kennknowles commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347310170 You can `run seed job` and it will actually be installed right away. It can then be rolled back if broken. Other

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

2017-11-27 Thread GitBox
tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347309489 @kennknowles @jasonkuster @lukecwik - Do we have a process to test groovy changes before the commit?

Re: GitBox down

2017-11-27 Thread Reuven Lax
We need to update the contribution guide as well. On Nov 27, 2017 10:30 AM, "Jean-Baptiste Onofré" wrote: > No I don't think so, > > In your case, it's because you still refering git-wip-us (probably from > ): you have to use gitbox instead. > > I already updated master branch but I didn't updat

  1   2   >