Re: [DISCUSS] Move away from Apache Maven as build tool

Yeah, especially, I think it would have been great to have a vote before merging on master. Not a big deal, however, I'm really community focus ;) Regards JB On 11/28/2017 07:36 AM, Reuven Lax wrote: Agreed. I thinking having a formal vote before Luke had numbers and results would have been

Re: [DISCUSS] Move away from Apache Maven as build tool

Agreed. I thinking having a formal vote before Luke had numbers and results would have been too early. However now that we have such numbers, we should think about having a vote. Also, while I disagree with Romain that Gradle is not "enterprise ready" (it's heavily used by Netflix, LinkedIn,

Re: [DISCUSS] Move away from Apache Maven as build tool

Hi Luke, just curious (and maybe I missed it): did we do a formal vote to merge the gradle build ? Gradle is now on master, we have some Jira to update the release guide with gradle. It's fine, but I remember only a discussion, not a vote. In order to embrace the community and avoid to have

Re: [DISCUSS] Move away from Apache Maven as build tool

Le 27 nov. 2017 23:07, "Eugene Kirpichov" a écrit : On Mon, Nov 27, 2017 at 11:52 AM Romain Manni-Bucau wrote: > 2017-11-27 20:26 GMT+01:00 Lukasz Cwik : > > Romain, as mentioned earlier, I identified that Maven was

[GitHub] kennknowles commented on issue #3538: Add a test for Avro write with RVP; fix code

kennknowles commented on issue #3538: Add a test for Avro write with RVP; fix code URL: https://github.com/apache/beam/pull/3538#issuecomment-347408240 There are conflicts introduced - would you mind rebasing? That would also give more recent signal for our use of the GitHub merge button,

[GitHub] kennknowles commented on issue #4168: [BEAM-3238][SQL] Add BeamRecordSqlTypeBuilder

kennknowles commented on issue #4168: [BEAM-3238][SQL] Add BeamRecordSqlTypeBuilder URL: https://github.com/apache/beam/pull/4168#issuecomment-347407859 @xumingming what do you think now? This is an automated message from

[GitHub] kennknowles closed pull request #3677: Tez runner

kennknowles closed pull request #3677: Tez runner URL: https://github.com/apache/beam/pull/3677 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the

[GitHub] kennknowles commented on issue #3677: Tez runner

kennknowles commented on issue #3677: Tez runner URL: https://github.com/apache/beam/pull/3677#issuecomment-347407078 Now that we have migrated to gitbox, I can close this. I am going to do so, since it is on the actual target branch of `tez-runner`.

[GitHub] luke-zhu commented on issue #4176: [BEAM-3143] Type Inference Compatibility with Python 3

luke-zhu commented on issue #4176: [BEAM-3143] Type Inference Compatibility with Python 3 URL: https://github.com/apache/beam/pull/4176#issuecomment-347397998 After another look, I think that the base of this solution won't improve Python 3 compatibility in the long run. I'll submit a

[GitHub] luke-zhu closed pull request #4176: [BEAM-3143] Type Inference Compatibility with Python 3

luke-zhu closed pull request #4176: [BEAM-3143] Type Inference Compatibility with Python 3 URL: https://github.com/apache/beam/pull/4176 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is

[GitHub] asfgit closed pull request #4182: Add license header to SideInputTranslationTest

asfgit closed pull request #4182: Add license header to SideInputTranslationTest URL: https://github.com/apache/beam/pull/4182 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign

[GitHub] kennknowles commented on issue #4182: Add license header to SideInputTranslationTest

kennknowles commented on issue #4182: Add license header to SideInputTranslationTest URL: https://github.com/apache/beam/pull/4182#issuecomment-347395117 CC: @jkff @tweise This is an automated message from the Apache Git

[GitHub] kennknowles opened a new pull request #4182: Add license header to SideInputTranslationTest

kennknowles opened a new pull request #4182: Add license header to SideInputTranslationTest URL: https://github.com/apache/beam/pull/4182 Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

[GitHub] kennknowles commented on a change in pull request #4153: [BEAM-3219] DataflowRunner: delegate @Setup and @Teardown in stateful ParDo

kennknowles commented on a change in pull request #4153: [BEAM-3219] DataflowRunner: delegate @Setup and @Teardown in stateful ParDo URL: https://github.com/apache/beam/pull/4153#discussion_r153360051 ## File path:

[GitHub] kennknowles commented on a change in pull request #4153: [BEAM-3219] DataflowRunner: delegate @Setup and @Teardown in stateful ParDo

kennknowles commented on a change in pull request #4153: [BEAM-3219] DataflowRunner: delegate @Setup and @Teardown in stateful ParDo URL: https://github.com/apache/beam/pull/4153#discussion_r153380128 ## File path:

[GitHub] jkff commented on issue #4074: [BEAM-3130] View.asMap() causes a ClassCastException in Apex runner

jkff commented on issue #4074: [BEAM-3130] View.asMap() causes a ClassCastException in Apex runner URL: https://github.com/apache/beam/pull/4074#issuecomment-347389329 Fair enough, the current PR is an improvement in any case. Simplified the test a little bit and merged.

[GitHub] asfgit closed pull request #4074: [BEAM-3130] View.asMap() causes a ClassCastException in Apex runner

asfgit closed pull request #4074: [BEAM-3130] View.asMap() causes a ClassCastException in Apex runner URL: https://github.com/apache/beam/pull/4074 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance:

[GitHub] chamikaramj commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO

chamikaramj commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO URL: https://github.com/apache/beam/pull/4169#discussion_r153376819 ## File path: sdks/java/io/file-based-io-tests/pom.xml ## @@ -139,6 +139,24 @@

[GitHub] chamikaramj commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO

chamikaramj commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO URL: https://github.com/apache/beam/pull/4169#discussion_r153375847 ## File path:

[GitHub] chamikaramj commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO

chamikaramj commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO URL: https://github.com/apache/beam/pull/4169#discussion_r153376493 ## File path:

Re: SerializableCoder Structured Value

What I said is not quite right - there are accidental collisions allowed. The "all coders" spec for structural value only requires that encode(a) == encode(b) implies sv(a).equals(sv(b)). The converse is not required. For example, the nondeterministic SetCoder can use the Set objects themselves as

Re: SerializableCoder Structured Value

To add some flavor, *All coders:* structuralValue(a).equals(structuralValue(b)) if and only if encode(a) == encode(b) *"Consistent with equals" aka injective:* encode(a) == encode(b) implies a.equals(b) *Deterministic:* a.equals(b) implies structuralValue(a).equals(structuralValue(b)) (hence

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347381845 Run Python Performance Test This is an automated

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347381795 Run Python Performance Test This is an

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347381328 run seed job This is an automated message from

[GitHub] robertwb closed pull request #4181: Fix typo in proto files.

robertwb closed pull request #4181: Fix typo in proto files. URL: https://github.com/apache/beam/pull/4181 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347378789 Run Python Performance Test This is an automated

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347376775 Run Python Dataflow ValidatesRunner This is an

[GitHub] herohde commented on issue #4181: Fix typo in proto files.

herohde commented on issue #4181: Fix typo in proto files. URL: https://github.com/apache/beam/pull/4181#issuecomment-347376582 LGTM This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153365677 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +201,49 @@

Re: SerializableCoder Structured Value

I'm open to renaming *consistentWithEquals*. If I understand the code correctly, when consistentWithEquals returns true, org.apache.beam.sdk.util.MutationDetectors expects *a.equals(deserialize(serialize(a))* which I think is reasonable for SerializableCoder (assuming objects implement equals)*.

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347374422 Run Python PostCommit This is an automated

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347372780 Run Python PostCommit This is an automated

[GitHub] robertwb commented on issue #4181: Fix typo in proto files.

robertwb commented on issue #4181: Fix typo in proto files. URL: https://github.com/apache/beam/pull/4181#issuecomment-347372394 R: @herohde This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] robertwb opened a new pull request #4181: Fix typo in proto files.

robertwb opened a new pull request #4181: Fix typo in proto files. URL: https://github.com/apache/beam/pull/4181 Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

Re: [Build] Enforce failed on project beam-sdks-java-io-tika

Thanks, the debug information pointed me to the corrupted geoapi-3.0.0.jar file. Removing it solves the problem for me. Manu On Tue, Nov 28, 2017 at 4:41 AM Lukasz Cwik wrote: > Try running with -X to get all the debug output when using Maven. It may > give more

Re: [VOTE] Fixing @yyy.com.INVALID mailing addresses

I have confirmed that I am no longer impacted by the .INVALID suffix. On Mon, Nov 27, 2017 at 3:06 PM, Lukasz Cwik wrote: > Apache infra has mentioned that they have updated the mailing list to > prevent the ".INVALID" from appearing on the end. dev@ now matches user@ > in

Re: SerializableCoder Structured Value

I think the idea is that SerializableCoder should be updated to expect that all values it encodes do implement equals() since this seems to be the much more common case then classes that don't implement a useful equals. It would be possible to add a useful check to DirectRunner that any value that

Re: SerializableCoder Structured Value

Not sure where you see the contradiction? consistentWithEquals says "Whenever the encoded bytes of two values are equal, then the original values are equal according to {@code Objects.equals()}." - which is clearly false for Serializable's in general: it's possible that serialized form of "a" and

SerializableCoder Structured Value

Hi all, Currently SerializableCoder#consistentWithEquals returns false, which contradicts it's own documentation "{@link SerializableCoder} does not guarantee a deterministic encoding, as Java serialization may produce different binary encodings for two equivalent objects". In practice, it leads

[GitHub] kennknowles commented on issue #4135: [BEAM-3194] Add @RequiresStableInput annotation

kennknowles commented on issue #4135: [BEAM-3194] Add @RequiresStableInput annotation URL: https://github.com/apache/beam/pull/4135#issuecomment-347367940 Thanks! Squashed in the fixup and re-running tests since there looked like a failure that I can't imagine this PR caused.

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347364157 run seed job This is an automated message from

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347359996 Run Python Performance Test This is an automated

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347359648 Run Python Dataflow ValidatesRunner This is an

[GitHub] tweise commented on issue #4074: [BEAM-3130] View.asMap() causes a ClassCastException in Apex runner

tweise commented on issue #4074: [BEAM-3130] View.asMap() causes a ClassCastException in Apex runner URL: https://github.com/apache/beam/pull/4074#issuecomment-347359334 @jkff I'm aware of that issue as well. It is certainly important, but at the same time it's not new and I don't see why

Re: [VOTE] Fixing @yyy.com.INVALID mailing addresses

Apache infra has mentioned that they have updated the mailing list to prevent the ".INVALID" from appearing on the end. dev@ now matches user@ in this regard. On Mon, Nov 27, 2017 at 11:30 AM, Lukasz Cwik wrote: > Vote Closed: > all +1s with at least 3 binding votes. > > On

[GitHub] nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153350918 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67

[GitHub] tgroh closed pull request #4151: [BEAM-2899] Decompose Direct Execution Components

tgroh closed pull request #4151: [BEAM-2899] Decompose Direct Execution Components URL: https://github.com/apache/beam/pull/4151 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347357063 Run Python PostCommit This is an automated

[GitHub] jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153348845 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67 @@

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347356163 run seed job This is an automated message from

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347354797 run seed job This is an automated message

Re: [DISCUSS] Move away from Apache Maven as build tool

I don't believe that such a switch could happen immediately as the scope of the change is to replace 15k lines of pom files so anything which isn't gradual is unlikely to work. Below is a more thorough list of points which compare the two build systems beyond speed. Maven Java Support: Mature

[GitHub] jkff commented on issue #4139: Update cloud spanner library to 0.29.0

jkff commented on issue #4139: Update cloud spanner library to 0.29.0 URL: https://github.com/apache/beam/pull/4139#issuecomment-347342624 (bumping grpc version by +5 seems welcome but quite risky, hence the ValidatesRunner tests)

[GitHub] jkff commented on issue #4139: Update cloud spanner library to 0.29.0

jkff commented on issue #4139: Update cloud spanner library to 0.29.0 URL: https://github.com/apache/beam/pull/4139#issuecomment-347342299 retest this please This is an automated message from the Apache Git Service. To

[GitHub] jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153335970 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67 @@

[GitHub] jkff commented on issue #4172: [BEAM-3243] support multiple anonymous classes from the same enclosing class in a pipeline

jkff commented on issue #4172: [BEAM-3243] support multiple anonymous classes from the same enclosing class in a pipeline URL: https://github.com/apache/beam/pull/4172#issuecomment-347341595 That would be optimal, and in general I really like the idea of capturing the stack trace of

[GitHub] nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153334683 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67

[GitHub] jkff commented on a change in pull request #4145: Many simplifications to WriteFiles

jkff commented on a change in pull request #4145: Many simplifications to WriteFiles URL: https://github.com/apache/beam/pull/4145#discussion_r153291649 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/io/WriteFiles.java ## @@ -672,8 +661,11 @@ public void

[GitHub] jkff commented on a change in pull request #4145: Many simplifications to WriteFiles

jkff commented on a change in pull request #4145: Many simplifications to WriteFiles URL: https://github.com/apache/beam/pull/4145#discussion_r153291513 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileBasedSink.java ## @@ -686,40 +756,38 @@ public

Re: [discuss] java profile

Well for validation builds- pre PR, incremental support is pointless since it easily hides issues die to caching so a solution saving half of the build without loosing anuyhing would still be good IMHO. Le 27 nov. 2017 21:12, "Lukasz Cwik" a écrit : > Incremental

[GitHub] jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153332162 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67 @@

[GitHub] nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153330138 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67

[GitHub] nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

nevillelyh commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153329942 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347333086 Run Python PostCommit This is an automated

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347332806 Run Python PostCommit This is an automated

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347331790 Thanks, that's helpful. This is an automated

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347331807 run seed job This is an automated message from

Re: [DISCUSS] Move away from Apache Maven as build tool

I have been a little bit out of the discussion on maven vs gradle because I was expecting the technical proof of concepts to evaluate the best approach. I deeply appreciate all the effort that Lukasz has put into the gradle version, and I also think that during the discussion Romain and others

[GitHub] pabloem opened a new pull request #4180: Updating dataflow API version to newer release.

pabloem opened a new pull request #4180: Updating dataflow API version to newer release. URL: https://github.com/apache/beam/pull/4180 r: @bjchambers This new release contains the changes to the Dataflow API protos for side input inter transform IO.

Re: Azure(ADLS) compatibility on Beam with Spark runner

Out of curiosity, does using the DirectRunner with ADL work for you? If not, then you'll be able to debug locally why its failing. On Fri, Nov 24, 2017 at 8:09 PM, Milan Chandna < milan.chan...@microsoft.com.invalid> wrote: > Hi JB, > > Thanks for the updates. > BTW I am myself in Microsoft but

[GitHub] reuvenlax commented on issue #4116: [BEAM-2953] Part 1 of Multipart advanced timeseries examples

reuvenlax commented on issue #4116: [BEAM-2953] Part 1 of Multipart advanced timeseries examples URL: https://github.com/apache/beam/pull/4116#issuecomment-347321463 R: @kennknowles This is an automated message from the

Re: [Build] Enforce failed on project beam-sdks-java-io-tika

Try running with -X to get all the debug output when using Maven. It may give more details. Alternatively, try using the gradle build (from root of project directory): ./gradlew build On Fri, Nov 24, 2017 at 9:46 PM, Manu Zhang wrote: > Hi all, > > Has anyone seen

Re: [DISCUSS] Move away from Apache Maven as build tool

On Mon, Nov 27, 2017 at 11:51 AM, Romain Manni-Bucau wrote: > 2017-11-27 20:26 GMT+01:00 Lukasz Cwik : > > Romain, as mentioned earlier, I identified that Maven was slower because > it > > needed to finish building the entire module before

Re: [discuss] java profile

Incremental builds aren't correctly setup right now so your likely to see Python/Go rebuild even if there were no changes. See https://issues.apache.org/jira/browse/BEAM-3253 On Mon, Nov 27, 2017 at 11:46 AM, Romain Manni-Bucau wrote: > that was the goal: validate there

[GitHub] kennknowles commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

kennknowles commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347310170 You can `run seed job` and it will actually be installed right away. It can then be rolled back if broken.

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347309489 @kennknowles @jasonkuster @lukecwik - Do we have a process to test groovy changes before the commit?

Re: GitBox down

We need to update the contribution guide as well. On Nov 27, 2017 10:30 AM, "Jean-Baptiste Onofré" wrote: > No I don't think so, > > In your case, it's because you still refering git-wip-us (probably from > ): you have to use gitbox instead. > > I already updated master

Re: [DISCUSS] Move away from Apache Maven as build tool

2017-11-27 20:26 GMT+01:00 Lukasz Cwik : > Romain, as mentioned earlier, I identified that Maven was slower because it > needed to finish building the entire module before dependent modules could > start which included running tests, performing checkstyle, etc... > Gradle

[GitHub] tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn commented on issue #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179#issuecomment-347305706 Run Python PostCommit This is an automated

[GitHub] tvalentyn opened a new pull request #4179: Fix repository path for Python Jenkins builds. Source directory was c?

tvalentyn opened a new pull request #4179: Fix repository path for Python Jenkins builds. Source directory was c? URL: https://github.com/apache/beam/pull/4179 ?hanged in PR/3976 This should fix BEAM-3120. Follow this checklist to help us incorporate your contribution quickly

[GitHub] angoenka opened a new pull request #4178: [BEAM-3239] Adding debug server to sdk worker to get threaddumps

angoenka opened a new pull request #4178: [BEAM-3239] Adding debug server to sdk worker to get threaddumps URL: https://github.com/apache/beam/pull/4178 Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

Re: [discuss] java profile

that was the goal: validate there was no side effect of the changes on the whole project. Now the "not java" part of the build will not be impacted by java changed so this is the part i want to skip since it takes a lot of time and I have guarantees it is safe to skip them. Romain Manni-Bucau

Re: [VOTE] Fixing @yyy.com.INVALID mailing addresses

Vote Closed: all +1s with at least 3 binding votes. On Fri, Nov 24, 2017 at 6:52 AM, Aljoscha Krettek wrote: > +1 > > > On 23. Nov 2017, at 23:22, Manu Zhang wrote: > > > > +1 > > > > On Thu, Nov 23, 2017 at 11:32 PM Maximilian Michels

Re: [discuss] java profile

Romain, that will build the entire project. I think you want to execute (from the root of the project): ./gradlew :beam-sdks-parent:beam-sdks-python:build On Mon, Nov 27, 2017 at 11:25 AM, Romain Manni-Bucau wrote: > gradle build --no-daemon > > (with gradle 4.2) > >

Re: [DISCUSS] Move away from Apache Maven as build tool

Romain, as mentioned earlier, I identified that Maven was slower because it needed to finish building the entire module before dependent modules could start which included running tests, performing checkstyle, etc... Gradle is able to increase the parallelism of the build process since it has task

Re: [discuss] java profile

gradle build --no-daemon (with gradle 4.2) Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn 2017-11-27 20:21 GMT+01:00 Kenneth Knowles : > What is the gradle command you are using to build just the Python SDK? > > On Mon, Nov 27, 2017 at 11:19 AM,

[GitHub] szewi commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO

szewi commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO URL: https://github.com/apache/beam/pull/4169#discussion_r153296166 ## File path:

Re: [discuss] java profile

What is the gradle command you are using to build just the Python SDK? On Mon, Nov 27, 2017 at 11:19 AM, Romain Manni-Bucau wrote: > Hmm, > > issue is the same with gradle (locally python build takes 15mn alone > which is as much as the java build and it is not

[GitHub] szewi commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO

szewi commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO URL: https://github.com/apache/beam/pull/4169#discussion_r153293511 ## File path:

Re: [discuss] java profile

Hmm, issue is the same with gradle (locally python build takes 15mn alone which is as much as the java build and it is not parallelized I think) pl is not as smooth since it means doing it on each command whereas the proposal is automatically activated through settings.xml Romain Manni-Bucau

[GitHub] szewi commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO

szewi commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO URL: https://github.com/apache/beam/pull/4169#discussion_r153293998 ## File path: sdks/java/io/file-based-io-tests/pom.xml ## @@ -139,6 +139,24 @@

[GitHub] szewi commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO

szewi commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO URL: https://github.com/apache/beam/pull/4169#discussion_r153293998 ## File path: sdks/java/io/file-based-io-tests/pom.xml ## @@ -139,6 +139,24 @@

[GitHub] kamszPolidea commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO

kamszPolidea commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO URL: https://github.com/apache/beam/pull/4169#discussion_r153293304 ## File path:

[GitHub] kamszPolidea commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO

kamszPolidea commented on a change in pull request #4169: [BEAM-3060] Added support for multiple filesystems in TextIO URL: https://github.com/apache/beam/pull/4169#discussion_r153293304 ## File path:

Re: [discuss] java profile

I think you can already mostly do this with mvn -pl sdks/XYZ -am -amd. I think that we have other work (gradle support) underway that will make this a non-issue since gradle automatically does even better than the profile or -am -amd. On Mon, Nov 27, 2017 at 11:01 AM, Romain Manni-Bucau

[discuss] java profile

Hi guys, java/python/go/xxx support is great but as a developer you rarely hack on them all. For that reason I opened https://github.com/apache/beam/pull/4173. Goal is to give each developer a way to build the whole project and all the code he can impact at once but without caring of the code

Re: [DISCUSS] Move away from Apache Maven as build tool

Hi Lukasz, Did you manage to identify how maven was slower and test tesla stuff and potentially a few other fixes? Side note: figures without python can be interesting cause locally - = for me - python tends to flatten the figures whereas I get something close to your conclusions without python

Re: [DISCUSS] Move away from Apache Maven as build tool

I have collected data by running several builds against master using Gradle and Maven without using Gradle's support for incremental builds. Gradle (mins) min: 25.04 max: 160.14 median: 45.78 average: 52.19 stdev: 30.80 Maven (mins) min: 56.86 max: 216.55 (actually > 240 mins because this data

[GitHub] jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance

jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any performance URL: https://github.com/apache/beam/pull/4175#discussion_r153282640 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java ## @@ -209,29 +202,67 @@

  1   2   >