Build failed in Jenkins: beam_PerformanceTests_Dataflow #366

2017-05-03 Thread Apache Jenkins Server
See Changes: [robertwb] Deprecate .options usage [lcwik] Remove Coder.asCloudObject and related methods [altay] Updating Dataflow API protos and client [tgroh] Add Additional

Jenkins build is back to stable : beam_PostCommit_Java_ValidatesRunner_Flink #2630

2017-05-03 Thread Apache Jenkins Server
See

[2/4] beam git commit: Removed coder and formatFn from PubsubIO.Write

2017-05-03 Thread jkff
Removed coder and formatFn from PubsubIO.Write Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/429c6131 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/429c6131 Diff:

[3/4] beam git commit: Removed coder and parseFn from PubsubIO.Read

2017-05-03 Thread jkff
Removed coder and parseFn from PubsubIO.Read Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/25dc94bc Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/25dc94bc Diff:

[jira] [Commented] (BEAM-1415) PubsubIO should comply with PTransform style guide

2017-05-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994339#comment-15994339 ] ASF GitHub Bot commented on BEAM-1415: -- Github user asfgit closed the pull request at:

[4/4] beam git commit: This closes #2634

2017-05-03 Thread jkff
This closes #2634 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/7d6f8bde Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/7d6f8bde Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/7d6f8bde Branch:

[GitHub] beam pull request #2634: [BEAM-1415] PubsubIO should comply with PTransform ...

2017-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/2634 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[1/4] beam git commit: Introduces read/writePubsubMessages

2017-05-03 Thread jkff
Repository: beam Updated Branches: refs/heads/master dc0fdcb7e -> 7d6f8bde7 Introduces read/writePubsubMessages Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/eca553f2 Tree:

[jira] [Closed] (BEAM-1353) Beam should comply with PTransform style guide

2017-05-03 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov closed BEAM-1353. -- Resolution: Fixed > Beam should comply with PTransform style guide >

[jira] [Closed] (BEAM-1415) PubsubIO should comply with PTransform style guide

2017-05-03 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov closed BEAM-1415. -- Resolution: Fixed > PubsubIO should comply with PTransform style guide >

[GitHub] beam pull request #2854: [BEAM-2095] Made SourceRDD hasNext idempotent

2017-05-03 Thread staslev
GitHub user staslev opened a pull request: https://github.com/apache/beam/pull/2854 [BEAM-2095] Made SourceRDD hasNext idempotent Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted

[jira] [Commented] (BEAM-2095) SourceRDD hasNext not idempotent

2017-05-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994352#comment-15994352 ] ASF GitHub Bot commented on BEAM-2095: -- GitHub user staslev opened a pull request:

[jira] [Updated] (BEAM-2095) The hasNext method of the iterator returned by SourceRDD#compute is not idempotent

2017-05-03 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2095: - Summary: The hasNext method of the iterator returned by SourceRDD#compute is not idempotent (was:

[GitHub] beam pull request #2855: [BEAM-1888] ApexRunner SDF support

2017-05-03 Thread tweise
GitHub user tweise opened a pull request: https://github.com/apache/beam/pull/2855 [BEAM-1888] ApexRunner SDF support Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like:

[jira] [Commented] (BEAM-1888) Support Splittable DoFn in Apex Runner

2017-05-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994368#comment-15994368 ] ASF GitHub Bot commented on BEAM-1888: -- GitHub user tweise opened a pull request:

[jira] [Assigned] (BEAM-2150) Support for recursive wildcards in GcsPath

2017-05-03 Thread Devon Meunier (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devon Meunier reassigned BEAM-2150: --- Assignee: Devon Meunier > Support for recursive wildcards in GcsPath >

[GitHub] beam pull request #2860: [BEAM-59] FileChecksumMatcherTest: switch from IOCh...

2017-05-03 Thread dhalperi
GitHub user dhalperi opened a pull request: https://github.com/apache/beam/pull/2860 [BEAM-59] FileChecksumMatcherTest: switch from IOChannelUtils to java file api These are unit tests always running on local filesystem. You can merge this pull request into a Git repository by

[jira] [Updated] (BEAM-2150) Support for recursive wildcards in GcsPath

2017-05-03 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-2150: -- Component/s: sdk-java-core > Support for recursive wildcards in GcsPath >

[GitHub] beam pull request #2859: reusing combine in GroupIntoBatches

2017-05-03 Thread hepeknet
GitHub user hepeknet opened a pull request: https://github.com/apache/beam/pull/2859 reusing combine in GroupIntoBatches Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like:

[jira] [Updated] (BEAM-2151) Type mapping inconsistent in BeamSqlRow & BeamSqlRowCoder

2017-05-03 Thread James Xu (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Xu updated BEAM-2151: --- External issue URL: (was: https://issues.apache.org/jira/browse/BEAM-2079) > Type mapping inconsistent in

[GitHub] beam pull request #2775: [BEAM-59] Remove IOChannelUtils from PackageUtil

2017-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/2775 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (BEAM-59) Switch from IOChannelFactory to FileSystems

2017-05-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-59?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995036#comment-15995036 ] ASF GitHub Bot commented on BEAM-59: Github user asfgit closed the pull request at:

[jira] [Commented] (BEAM-1876) GroupIntoBatches may be able to use Combine.BinaryCombineLongFn

2017-05-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995068#comment-15995068 ] ASF GitHub Bot commented on BEAM-1876: -- GitHub user hepeknet opened a pull request:

[jira] [Updated] (BEAM-2151) Type mapping inconsistent in BeamSqlRow & BeamSqlRowCoder

2017-05-03 Thread James Xu (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Xu updated BEAM-2151: --- Description: In BeamSqlRow, sql FLOAT is mapping to java Float: {code:java} case FLOAT: if

[jira] [Updated] (BEAM-2151) Type mapping inconsistent in BeamSqlRow & BeamSqlRowCoder

2017-05-03 Thread James Xu (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Xu updated BEAM-2151: --- External issue URL: https://issues.apache.org/jira/browse/BEAM-2079 Priority: Blocker (was:

[jira] [Commented] (BEAM-59) Switch from IOChannelFactory to FileSystems

2017-05-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-59?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995106#comment-15995106 ] ASF GitHub Bot commented on BEAM-59: GitHub user dhalperi opened a pull request:

[jira] [Updated] (BEAM-2143) (Mis)Running Dataflow Wordcount gives non-helpful errors

2017-05-03 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-2143: -- Fix Version/s: First stable release > (Mis)Running Dataflow Wordcount gives non-helpful errors >

[jira] [Updated] (BEAM-2143) (Mis)Running Dataflow Wordcount gives non-helpful errors

2017-05-03 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-2143: -- Priority: Blocker (was: Major) > (Mis)Running Dataflow Wordcount gives non-helpful errors >

[jira] [Commented] (BEAM-2136) AvroCoderTest.testTwoClassLoaders fails on beam_PostCommit_Java_ValidatesRunner_Dataflow

2017-05-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995189#comment-15995189 ] ASF GitHub Bot commented on BEAM-2136: -- GitHub user kennknowles opened a pull request:

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Dataflow #3024

2017-05-03 Thread Apache Jenkins Server
See

[GitHub] beam pull request #2861: [BEAM-2136] Use Thread context ClassLoader as paren...

2017-05-03 Thread kennknowles
GitHub user kennknowles opened a pull request: https://github.com/apache/beam/pull/2861 [BEAM-2136] Use Thread context ClassLoader as parent in AvroCoderTest Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure

[jira] [Updated] (BEAM-2150) Support for recursive wildcards in GcsPath

2017-05-03 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-2150: -- Component/s: (was: sdk-ideas) sdk-java-gcp > Support for recursive

[jira] [Commented] (BEAM-2150) Support for recursive wildcards in GcsPath

2017-05-03 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995050#comment-15995050 ] Daniel Halperin commented on BEAM-2150: --- Take a look at {{GcsFileSystem}}. In general, {{FileSystem}}

Jenkins build became unstable: beam_PostCommit_Java_MavenInstall #3612

2017-05-03 Thread Apache Jenkins Server
See

[GitHub] beam pull request #2858: Use LinkedHashMap for step contexts in BaseExecutio...

2017-05-03 Thread kennknowles
GitHub user kennknowles opened a pull request: https://github.com/apache/beam/pull/2858 Use LinkedHashMap for step contexts in BaseExecutionContext This gives a performance improvement when all step contexts need to be traversed, such as for state cleanup operations. Be

[jira] [Created] (BEAM-2151) Type mapping inconsistent in BeamSqlRow & BeamSqlRowCoder

2017-05-03 Thread James Xu (JIRA)
James Xu created BEAM-2151: -- Summary: Type mapping inconsistent in BeamSqlRow & BeamSqlRowCoder Key: BEAM-2151 URL: https://issues.apache.org/jira/browse/BEAM-2151 Project: Beam Issue Type: Bug

[2/2] beam git commit: This closes #2775

2017-05-03 Thread dhalperi
This closes #2775 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/1b0400fe Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/1b0400fe Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/1b0400fe Branch:

[jira] [Commented] (BEAM-2136) AvroCoderTest.testTwoClassLoaders fails on beam_PostCommit_Java_ValidatesRunner_Dataflow

2017-05-03 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995034#comment-15995034 ] Kenneth Knowles commented on BEAM-2136: --- Another thing that comes out of this - the test suite should

[jira] [Commented] (BEAM-302) Add Scio Scala DSL to Beam

2017-05-03 Thread Neville Li (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995075#comment-15995075 ] Neville Li commented on BEAM-302: - Looks like Spark runner still depends on 1.6.3. Can you give Spark 1.6 a

[1/2] beam git commit: Use LinkedHashMap for step contexts in BaseExecutionContext

2017-05-03 Thread kenn
Repository: beam Updated Branches: refs/heads/master 1b0400fee -> f33039bfd Use LinkedHashMap for step contexts in BaseExecutionContext This gives a performance improvement when all step contexts need to be traversed, such as for state cleanup operations. Project:

[GitHub] beam pull request #2858: Use LinkedHashMap for step contexts in BaseExecutio...

2017-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/2858 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Created] (BEAM-2152) Authentication fails if there is an unauthenticated gcloud tool even if application default credentials are available

2017-05-03 Thread Ahmet Altay (JIRA)
Ahmet Altay created BEAM-2152: - Summary: Authentication fails if there is an unauthenticated gcloud tool even if application default credentials are available Key: BEAM-2152 URL:

[jira] [Commented] (BEAM-638) Add sink transform to write bounded data per window, pane, [and key] even when PCollection is unbounded

2017-05-03 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995167#comment-15995167 ] Eugene Kirpichov commented on BEAM-638: --- TextIO and AvroIO can now write unbounded data, producing one

[jira] [Commented] (BEAM-567) FlinkRunner pollutes beam-examples-java[8] modules with SLF4J backend dependencies which are not optional (e.g. test/provided/system scoped, or marked optional)

2017-05-03 Thread Aljoscha Krettek (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994781#comment-15994781 ] Aljoscha Krettek commented on BEAM-567: --- Thanks for the update! So we can close this issue once we

[jira] [Created] (BEAM-2150) Support for recursive wildcards in GcsPath

2017-05-03 Thread Devon Meunier (JIRA)
Devon Meunier created BEAM-2150: --- Summary: Support for recursive wildcards in GcsPath Key: BEAM-2150 URL: https://issues.apache.org/jira/browse/BEAM-2150 Project: Beam Issue Type: New Feature

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Dataflow #3023

2017-05-03 Thread Apache Jenkins Server
See

[jira] [Assigned] (BEAM-2150) Support for recursive wildcards in GcsPath

2017-05-03 Thread JIRA
[ https://issues.apache.org/jira/browse/BEAM-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Baptiste Onofré reassigned BEAM-2150: -- Assignee: (was: Davor Bonaci) > Support for recursive wildcards in GcsPath

[jira] [Commented] (BEAM-2150) Support for recursive wildcards in GcsPath

2017-05-03 Thread JIRA
[ https://issues.apache.org/jira/browse/BEAM-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994939#comment-15994939 ] Jean-Baptiste Onofré commented on BEAM-2150: Any chance you prepare a PR about that ? Thanks !

Build failed in Jenkins: beam_PerformanceTests_Dataflow #367

2017-05-03 Thread Apache Jenkins Server
See Changes: [vikasrk] Triggers for Java SDK post submit tests [kirpichov] Removed coder and parseFn from PubsubIO.Read [kirpichov] Removed coder and formatFn from PubsubIO.Write [kirpichov]

[jira] [Commented] (BEAM-1612) Support real Bundle in Flink runner

2017-05-03 Thread Aljoscha Krettek (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994837#comment-15994837 ] Aljoscha Krettek commented on BEAM-1612: It seems FLINK-6413 is not gonna make it so we need to

[jira] [Assigned] (BEAM-2023) BigQueryIO.Write needs a way of dynamically specifying table schemas

2017-05-03 Thread Davor Bonaci (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davor Bonaci reassigned BEAM-2023: -- Assignee: Reuven Lax (was: Davor Bonaci) > BigQueryIO.Write needs a way of dynamically

[jira] [Created] (BEAM-2153) Move connection management in JmsIO to setup/teardown methods

2017-05-03 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-2153: - Summary: Move connection management in JmsIO to setup/teardown methods Key: BEAM-2153 URL: https://issues.apache.org/jira/browse/BEAM-2153 Project: Beam Issue

[jira] [Assigned] (BEAM-2023) BigQueryIO.Write needs a way of dynamically specifying table schemas

2017-05-03 Thread Davor Bonaci (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davor Bonaci reassigned BEAM-2023: -- Assignee: Davor Bonaci (was: Reuven Lax) > BigQueryIO.Write needs a way of dynamically

[jira] [Updated] (BEAM-2154) Writing to large numbers of BigQuery tables causes out-of-memory

2017-05-03 Thread Davor Bonaci (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davor Bonaci updated BEAM-2154: --- Fix Version/s: First stable release > Writing to large numbers of BigQuery tables causes out-of-memory

[GitHub] beam pull request #2862: [BEAM-1871] Move GCP specific serialization CloudOb...

2017-05-03 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2862 [BEAM-1871] Move GCP specific serialization CloudObject and supporting translation code to Dataflow runner module Be sure to do all of the following to help us incorporate your contribution

Jenkins build is back to stable : beam_PostCommit_Java_MavenInstall #3613

2017-05-03 Thread Apache Jenkins Server
See

[jira] [Assigned] (BEAM-2053) Add HDFS file provider to wordcount examples

2017-05-03 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles reassigned BEAM-2053: - Assignee: Daniel Halperin > Add HDFS file provider to wordcount examples >

[jira] [Commented] (BEAM-2053) Add HDFS file provider to wordcount examples

2017-05-03 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995286#comment-15995286 ] Kenneth Knowles commented on BEAM-2053: --- I believe [~dhalp...@google.com] has changes done or

[GitHub] beam pull request #2863: Reduce visibility of DirectRunner classes

2017-05-03 Thread tgroh
GitHub user tgroh opened a pull request: https://github.com/apache/beam/pull/2863 Reduce visibility of DirectRunner classes Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like:

Build failed in Jenkins: beam_PerformanceTests_Dataflow #368

2017-05-03 Thread Apache Jenkins Server
See Changes: [klk] Use LinkedHashMap for step contexts in BaseExecutionContext [dhalperi] Remove IoChannelUtils from PackageUtil [dhalperi] [BEAM-59] FileChecksumMatcherTest: switch from

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Dataflow #3025

2017-05-03 Thread Apache Jenkins Server
See

[jira] [Commented] (BEAM-2145) Do not publish javadoc for Java SDK's transforms.reflect directory

2017-05-03 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995367#comment-15995367 ] Kenneth Knowles commented on BEAM-2145: --- This should be owned by the next release manager if not

[jira] [Resolved] (BEAM-1889) '=' character in pipeline options cause problems

2017-05-03 Thread Ahmet Altay (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmet Altay resolved BEAM-1889. --- Resolution: Fixed Fix Version/s: Not applicable > '=' character in pipeline options cause

[jira] [Created] (BEAM-2154) Writing to large numbers of BigQuery tables causes out-of-memory

2017-05-03 Thread Reuven Lax (JIRA)
Reuven Lax created BEAM-2154: Summary: Writing to large numbers of BigQuery tables causes out-of-memory Key: BEAM-2154 URL: https://issues.apache.org/jira/browse/BEAM-2154 Project: Beam Issue

[jira] [Commented] (BEAM-2124) Deprecate .options usage

2017-05-03 Thread Ahmet Altay (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995235#comment-15995235 ] Ahmet Altay commented on BEAM-2124: --- Hey [~mariagh] can we close this now? Is there more work to be done

[jira] [Created] (BEAM-2155) Investigate problem creating Map PCollection view on Dataflow

2017-05-03 Thread Kenneth Knowles (JIRA)
Kenneth Knowles created BEAM-2155: - Summary: Investigate problem creating Map PCollection view on Dataflow Key: BEAM-2155 URL: https://issues.apache.org/jira/browse/BEAM-2155 Project: Beam

[jira] [Commented] (BEAM-1612) Support real Bundle in Flink runner

2017-05-03 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995201#comment-15995201 ] Kenneth Knowles commented on BEAM-1612: --- I would have thought that Flink would require isomorphic

[jira] [Commented] (BEAM-2053) Add HDFS file provider to wordcount examples

2017-05-03 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995299#comment-15995299 ] Daniel Halperin commented on BEAM-2053: --- This is a likely dupe of BEAM-59 and BEAM-2005. > Add HDFS

[2/2] beam git commit: This closes #2860

2017-05-03 Thread dhalperi
This closes #2860 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/0d19d3cb Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/0d19d3cb Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/0d19d3cb Branch:

[GitHub] beam pull request #2860: [BEAM-59] FileChecksumMatcherTest: switch from IOCh...

2017-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/2860 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (BEAM-2145) Do not publish javadoc for Java SDK's transforms.reflect directory

2017-05-03 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995407#comment-15995407 ] Daniel Halperin commented on BEAM-2145: --- Given that Davor will not be actively working on it, it

[jira] [Updated] (BEAM-2122) Writing to partitioned BigQuery tables from Dataflow is causing errors

2017-05-03 Thread Davor Bonaci (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davor Bonaci updated BEAM-2122: --- Fix Version/s: (was: First stable release) > Writing to partitioned BigQuery tables from Dataflow

[1/2] beam git commit: [BEAM-59] FileChecksumMatcherTest: switch from IOChannelUtils to java file API

2017-05-03 Thread dhalperi
Repository: beam Updated Branches: refs/heads/master f33039bfd -> 0d19d3cb2 [BEAM-59] FileChecksumMatcherTest: switch from IOChannelUtils to java file API These are unit tests always running on local filesystem. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit:

[jira] [Updated] (BEAM-2052) Windowed file sinks should support dynamic sharding

2017-05-03 Thread Davor Bonaci (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davor Bonaci updated BEAM-2052: --- Fix Version/s: First stable release > Windowed file sinks should support dynamic sharding >

[jira] [Assigned] (BEAM-2153) Move connection management in JmsIO to setup/teardown methods

2017-05-03 Thread Davor Bonaci (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davor Bonaci reassigned BEAM-2153: -- Assignee: Davor Bonaci (was: Jean-Baptiste Onofré) > Move connection management in JmsIO to

[jira] [Commented] (BEAM-1199) Condense recordAsOutput, finishSpecifyingOutput from POutput

2017-05-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995343#comment-15995343 ] ASF GitHub Bot commented on BEAM-1199: -- GitHub user tgroh opened a pull request:

[GitHub] beam pull request #2864: [BEAM-1199] Remove POutput#recordAsOutput

2017-05-03 Thread tgroh
GitHub user tgroh opened a pull request: https://github.com/apache/beam/pull/2864 [BEAM-1199] Remove POutput#recordAsOutput Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like:

Jenkins build is back to normal : beam_PostCommit_Python_Verify #2100

2017-05-03 Thread Apache Jenkins Server
See

Build failed in Jenkins: beam_PostCommit_Python_Verify #2098

2017-05-03 Thread Apache Jenkins Server
See Changes: [dhalperi] Remove IoChannelUtils from PackageUtil -- [...truncated 634.27 KB...] { "kind": "ParallelDo", "name": "s30",

[jira] [Comment Edited] (BEAM-1612) Support real Bundle in Flink runner

2017-05-03 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995201#comment-15995201 ] Kenneth Knowles edited comment on BEAM-1612 at 5/3/17 4:54 PM: --- I would have

[jira] [Updated] (BEAM-2122) Writing to partitioned BigQuery tables from Dataflow is causing errors

2017-05-03 Thread Davor Bonaci (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davor Bonaci updated BEAM-2122: --- Fix Version/s: First stable release > Writing to partitioned BigQuery tables from Dataflow is causing

[jira] [Commented] (BEAM-1871) Thin Java SDK Core

2017-05-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995227#comment-15995227 ] ASF GitHub Bot commented on BEAM-1871: -- GitHub user lukecwik opened a pull request:

[jira] [Assigned] (BEAM-2113) Apex Runner is not able to submit any job to YARN

2017-05-03 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles reassigned BEAM-2113: - Assignee: Kenneth Knowles (was: Thomas Weise) > Apex Runner is not able to submit any

[jira] [Commented] (BEAM-59) Switch from IOChannelFactory to FileSystems

2017-05-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-59?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995302#comment-15995302 ] ASF GitHub Bot commented on BEAM-59: Github user asfgit closed the pull request at:

Build failed in Jenkins: beam_PostCommit_Python_Verify #2099

2017-05-03 Thread Apache Jenkins Server
See Changes: [klk] Use LinkedHashMap for step contexts in BaseExecutionContext -- [...truncated 508.52 KB...] Collecting funcsigs>=1 (from mock->-r

[GitHub] beam pull request #2862: [BEAM-1871] Move GCP specific serialization CloudOb...

2017-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/2862 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (BEAM-1871) Thin Java SDK Core

2017-05-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995515#comment-15995515 ] ASF GitHub Bot commented on BEAM-1871: -- Github user asfgit closed the pull request at:

[GitHub] beam pull request #2865: [BEAM-1902] Retry on correct error codes for datast...

2017-05-03 Thread vikkyrk
GitHub user vikkyrk opened a pull request: https://github.com/apache/beam/pull/2865 [BEAM-1902] Retry on correct error codes for datastoreio Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is

[jira] [Commented] (BEAM-1902) Datastore IO never retries on errors

2017-05-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995581#comment-15995581 ] ASF GitHub Bot commented on BEAM-1902: -- GitHub user vikkyrk opened a pull request:

[jira] [Created] (BEAM-2157) HadoopFileSystemModuleTest Failed in Some JDK Versions on Jenkins

2017-05-03 Thread Mark Liu (JIRA)
Mark Liu created BEAM-2157: -- Summary: HadoopFileSystemModuleTest Failed in Some JDK Versions on Jenkins Key: BEAM-2157 URL: https://issues.apache.org/jira/browse/BEAM-2157 Project: Beam Issue

[jira] [Commented] (BEAM-2157) HadoopFileSystemModuleTest Failed in Some JDK Versions on Jenkins

2017-05-03 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995617#comment-15995617 ] Luke Cwik commented on BEAM-2157: - It might be something simple like "contains" -> "containsInAnyOrder" on

[GitHub] beam pull request #2869: [BEAM-2152] Remove gcloud auth as application defau...

2017-05-03 Thread sb2nov
GitHub user sb2nov opened a pull request: https://github.com/apache/beam/pull/2869 [BEAM-2152] Remove gcloud auth as application default credentials manages it Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ]

[jira] [Commented] (BEAM-1642) Combine transformation evaluation fails on direct runner with Avro as a fallback coder

2017-05-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995736#comment-15995736 ] ASF GitHub Bot commented on BEAM-1642: -- Github user asfgit closed the pull request at:

[3/3] beam git commit: Closes #2771

2017-05-03 Thread robertwb
Closes #2771 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/57f449c4 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/57f449c4 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/57f449c4 Branch: refs/heads/master

[GitHub] beam pull request #2842: [BEAM-2051] Mark all PCollectionView methods intern...

2017-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/2842 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (BEAM-2157) HadoopFileSystemModuleTest Failed in Some JDK Versions on Jenkins

2017-05-03 Thread Mark Liu (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Liu updated BEAM-2157: --- Description: I'm running Java Postcommit test in multiple JDK versions and HadoopFileSystemModuleTest failed

[jira] [Commented] (BEAM-1871) Thin Java SDK Core

2017-05-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995649#comment-15995649 ] ASF GitHub Bot commented on BEAM-1871: -- GitHub user lukecwik opened a pull request:

[4/4] beam-site git commit: Regenerate website

2017-05-03 Thread kenn
Regenerate website Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/d7f46849 Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/d7f46849 Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/d7f46849

[3/4] beam-site git commit: Minor touch-ups to Gearpump listing on overview

2017-05-03 Thread kenn
Minor touch-ups to Gearpump listing on overview - Alphabetical order - Add a space Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/da7d6c0e Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/da7d6c0e Diff:

[2/3] beam git commit: Reduce visibility of DirectRunner classes

2017-05-03 Thread tgroh
http://git-wip-us.apache.org/repos/asf/beam/blob/9edd8599/runners/direct-java/src/test/java/org/apache/beam/runners/direct/StatefulParDoEvaluatorFactoryTest.java -- diff --git

  1   2   3   >