[jira] [Commented] (BEAM-3159) DoFnTester should be deprecated in favor of TestPipeline

[ https://issues.apache.org/jira/browse/BEAM-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278304#comment-16278304 ] Etienne Chauchot commented on BEAM-3159: I agree that UDF execution being linked to the runner, it

[jira] [Commented] (BEAM-2721) Augment BeamRecordType to do slicing and concatenation.

[ https://issues.apache.org/jira/browse/BEAM-2721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279131#comment-16279131 ] Anton Kedin commented on BEAM-2721: --- After looking further into this, we can likely just remove the whole

[beam] branch master updated (6474616 -> cf38ed3)

This is an automated email from the ASF dual-hosted git repository. chamikara pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 6474616 Merge pull request #4085: Introduce a property to influence Maven output location. add 5792602

[beam] 01/01: Merge pull request #4157 from mhsaul/vcf-record-loose-mode

This is an automated email from the ASF dual-hosted git repository. chamikara pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit cf38ed3fc8c03ecdb28c22ca32ae079b7f22d526 Merge: 6474616 5792602 Author: Chamikara Jayalath

[jira] [Commented] (BEAM-2400) Null pointer exception when creating a template

[ https://issues.apache.org/jira/browse/BEAM-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278936#comment-16278936 ] Tobi Vollebregt commented on BEAM-2400: --- My request: change PipelineResult / waitUntilFinish so you

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #3629

See

[jira] [Updated] (BEAM-2083) Develop a Go SDK for Beam

[ https://issues.apache.org/jira/browse/BEAM-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henning Rohde updated BEAM-2083: Due Date: (was: 31/Oct/17) > Develop a Go SDK for Beam > - > >

[jira] [Updated] (BEAM-3189) Python Fnapi - Worker speedup

[ https://issues.apache.org/jira/browse/BEAM-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-3189: --- External issue URL: https://docs.google.com/document/d/1mHFaNgHA71RVGLVNrGrHIlWHgJb4tKCJ2qQzS13REY8 >

[jira] [Updated] (BEAM-2083) Develop a Go SDK for Beam

[ https://issues.apache.org/jira/browse/BEAM-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henning Rohde updated BEAM-2083: Description: Allow users of the Go programming language (https://golang.org/) to write Beam

[beam] branch master updated: Fixes a bug in Sample.Any

This is an automated email from the ASF dual-hosted git repository. jkff pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new f3f80d8 Fixes a bug in Sample.Any new fd8ad27

[jira] [Created] (BEAM-3292) Remove BeamRecordSqlType

Anton Kedin created BEAM-3292: - Summary: Remove BeamRecordSqlType Key: BEAM-3292 URL: https://issues.apache.org/jira/browse/BEAM-3292 Project: Beam Issue Type: Bug Components: dsl-sql

[jira] [Created] (BEAM-3293) Add lazy map side input form

Henning Rohde created BEAM-3293: --- Summary: Add lazy map side input form Key: BEAM-3293 URL: https://issues.apache.org/jira/browse/BEAM-3293 Project: Beam Issue Type: Improvement

[jira] [Created] (BEAM-3295) Consider: make KV more convenient

Henning Rohde created BEAM-3295: --- Summary: Consider: make KV more convenient Key: BEAM-3295 URL: https://issues.apache.org/jira/browse/BEAM-3295 Project: Beam Issue Type: Improvement

[jira] [Commented] (BEAM-2774) Add I/O source for VCF files (python)

[ https://issues.apache.org/jira/browse/BEAM-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279144#comment-16279144 ] ASF GitHub Bot commented on BEAM-2774: -- chamikaramj closed pull request #4157: [BEAM-2774] Added loose

[jira] [Created] (BEAM-3294) Move to graph.External and remove Source/Sink

Henning Rohde created BEAM-3294: --- Summary: Move to graph.External and remove Source/Sink Key: BEAM-3294 URL: https://issues.apache.org/jira/browse/BEAM-3294 Project: Beam Issue Type: Task

[jira] [Commented] (BEAM-3297) StartBundle/FinishBundle should be called only if there are elements

[ https://issues.apache.org/jira/browse/BEAM-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279249#comment-16279249 ] Henning Rohde commented on BEAM-3297: - [~kenn] is my understanding correct here? >

[jira] [Created] (BEAM-3297) StartBundle/FinishBundle should be called only if there are elements

Henning Rohde created BEAM-3297: --- Summary: StartBundle/FinishBundle should be called only if there are elements Key: BEAM-3297 URL: https://issues.apache.org/jira/browse/BEAM-3297 Project: Beam

[jira] [Commented] (BEAM-2400) Null pointer exception when creating a template

[ https://issues.apache.org/jira/browse/BEAM-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278934#comment-16278934 ] Tobi Vollebregt commented on BEAM-2400: --- I'm running into this as well. Turns out that when staging

[jira] [Commented] (BEAM-3070) Add support for windowed filenames in Python SDK

[ https://issues.apache.org/jira/browse/BEAM-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278890#comment-16278890 ] Eugene Kirpichov commented on BEAM-3070: Thanks! However, I strongly recommend to wait until

[jira] [Comment Edited] (BEAM-3070) Add support for windowed filenames in Python SDK

[ https://issues.apache.org/jira/browse/BEAM-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278786#comment-16278786 ] Vilhelm von Ehrenheim edited comment on BEAM-3070 at 12/5/17 4:42 PM: --

Build failed in Jenkins: beam_PerformanceTests_Spark #1086

See -- Started by timer [EnvInject] - Loading node environment variables. Building remotely on beam2 (beam) in workspace

[jira] [Commented] (BEAM-2083) Develop a Go SDK for Beam

[ https://issues.apache.org/jira/browse/BEAM-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279088#comment-16279088 ] ASF GitHub Bot commented on BEAM-2083: -- herohde opened a new pull request #4217: [BEAM-2083] Add Go

[jira] [Created] (BEAM-3301) Go SplittableDoFn support

Henning Rohde created BEAM-3301: --- Summary: Go SplittableDoFn support Key: BEAM-3301 URL: https://issues.apache.org/jira/browse/BEAM-3301 Project: Beam Issue Type: Improvement

[jira] [Updated] (BEAM-3300) Portable flattens in Go SDK Harness

[ https://issues.apache.org/jira/browse/BEAM-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henning Rohde updated BEAM-3300: Priority: Minor (was: Major) > Portable flattens in Go SDK Harness >

Build failed in Jenkins: beam_PerformanceTests_Spark #1087

See Changes: [msaul] Added loose failure mode to allow individual VCF record reads to fail [kirpichov] Fixes a bug in Sample.Any -- Started by timer

[beam] 04/13: remove ShardAssignment

This is an automated email from the ASF dual-hosted git repository. jkff pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 54eacf4b79993f40b9034bf429e387faeffdbdba Author: Eugene Kirpichov AuthorDate: Wed Nov 15 18:58:28

[beam] 05/13: consolidates windowed/unwindowed finalize fns somewhat

This is an automated email from the ASF dual-hosted git repository. jkff pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 97df5e703d4a891ab63a40b46c4e87d7c373168b Author: Eugene Kirpichov AuthorDate: Wed Nov 15 19:51:10

[beam] 03/13: non-null window/pane in FileResult

This is an automated email from the ASF dual-hosted git repository. jkff pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit b2d0671185fa1bd7f100853c7921e555c84578e7 Author: Eugene Kirpichov AuthorDate: Wed Nov 15 18:25:14

[beam] 07/13: Refactors WriteFiles into sub-transforms

This is an automated email from the ASF dual-hosted git repository. jkff pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 2f73a9534a709dd1fe775ab31e0598d2d89f123c Author: Eugene Kirpichov AuthorDate: Fri Nov 17 12:25:45

[beam] 12/13: Reintroduces dynamic sharding with windowed writes for bounded collections

This is an automated email from the ASF dual-hosted git repository. jkff pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit d314339ed2f8d5ce385c7b40705ef13f6ea43b45 Author: Eugene Kirpichov AuthorDate: Thu Nov 30 13:05:00

[beam] 02/13: Merges Writer.openWindowed/Unwindowed and removes result of close()

This is an automated email from the ASF dual-hosted git repository. jkff pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 5795c32c91a6de02b6731eacb5eef8ae55f069f5 Author: Eugene Kirpichov AuthorDate: Tue Oct 17 17:06:43

[jira] [Created] (BEAM-3303) Go windowing support

Henning Rohde created BEAM-3303: --- Summary: Go windowing support Key: BEAM-3303 URL: https://issues.apache.org/jira/browse/BEAM-3303 Project: Beam Issue Type: Improvement Components:

[beam] 08/13: Converts WriteFiles to AutoValue

This is an automated email from the ASF dual-hosted git repository. jkff pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 5b600da86a21e2c4339261d73fa1c2588cb3ab8d Author: Eugene Kirpichov AuthorDate: Fri Nov 17 12:38:15

[beam] 09/13: Makes checkstyle and findbugs happy

This is an automated email from the ASF dual-hosted git repository. jkff pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 3ecf13b9fc889f1978efef57f14f610766242901 Author: Eugene Kirpichov AuthorDate: Fri Nov 17 15:41:20

[beam] 10/13: Renames spilled back to unwritten

This is an automated email from the ASF dual-hosted git repository. jkff pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 83837eb20c1e1e4793f8410de1dc6d5864586f7f Author: Eugene Kirpichov AuthorDate: Wed Nov 22 16:34:46

[beam] 13/13: This closes #4145: Many simplifications to WriteFiles

This is an automated email from the ASF dual-hosted git repository. jkff pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 761ec1af410f0ee153893f6e7082db85d0fdc3e7 Merge: b059664 d314339 Author: Eugene Kirpichov AuthorDate:

Jenkins build is back to stable : beam_PostCommit_Java_ValidatesRunner_Spark #3631

See

Build failed in Jenkins: beam_PerformanceTests_Python #643

See Changes: [msaul] Added loose failure mode to allow individual VCF record reads to fail [kirpichov] Fixes a bug in Sample.Any -- Started by timer

[beam] branch master updated: Adds a deduplication key to Watch, and uses it to handle growing files in FileIO.match

This is an automated email from the ASF dual-hosted git repository. jkff pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 38d94c8 Adds a deduplication key to Watch, and

[jira] [Created] (BEAM-3305) Consider: Go ViewFn support

Henning Rohde created BEAM-3305: --- Summary: Consider: Go ViewFn support Key: BEAM-3305 URL: https://issues.apache.org/jira/browse/BEAM-3305 Project: Beam Issue Type: Improvement

[jira] [Created] (BEAM-3299) Go SDK support for portable progress reporting

Henning Rohde created BEAM-3299: --- Summary: Go SDK support for portable progress reporting Key: BEAM-3299 URL: https://issues.apache.org/jira/browse/BEAM-3299 Project: Beam Issue Type: Sub-task

[jira] [Updated] (BEAM-3299) Go SDK support for portable progress reporting

[ https://issues.apache.org/jira/browse/BEAM-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henning Rohde updated BEAM-3299: Priority: Minor (was: Major) > Go SDK support for portable progress reporting >

[jira] [Updated] (BEAM-3298) Go SDK support for portable user state

[ https://issues.apache.org/jira/browse/BEAM-3298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henning Rohde updated BEAM-3298: Labels: portability (was: ) > Go SDK support for portable user state >

[jira] [Created] (BEAM-3306) Consider: Go coder registry

Henning Rohde created BEAM-3306: --- Summary: Consider: Go coder registry Key: BEAM-3306 URL: https://issues.apache.org/jira/browse/BEAM-3306 Project: Beam Issue Type: Improvement

[jira] [Created] (BEAM-3307) Harden Go harness runtime logging

Henning Rohde created BEAM-3307: --- Summary: Harden Go harness runtime logging Key: BEAM-3307 URL: https://issues.apache.org/jira/browse/BEAM-3307 Project: Beam Issue Type: Improvement

[jira] [Assigned] (BEAM-3298) Go SDK support for portable user state

[ https://issues.apache.org/jira/browse/BEAM-3298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henning Rohde reassigned BEAM-3298: --- Assignee: (was: Henning Rohde) > Go SDK support for portable user state >

[jira] [Created] (BEAM-3298) Go SDK support for portable user state

Henning Rohde created BEAM-3298: --- Summary: Go SDK support for portable user state Key: BEAM-3298 URL: https://issues.apache.org/jira/browse/BEAM-3298 Project: Beam Issue Type: Sub-task

[beam] 01/13: enforce fixed sharding

This is an automated email from the ASF dual-hosted git repository. jkff pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 90402e4f21dbbf60ed498a67a5dfc3e276dcb07c Author: Eugene Kirpichov AuthorDate: Wed Nov 15 18:06:39

[beam] branch master updated (b059664 -> 761ec1a)

This is an automated email from the ASF dual-hosted git repository. jkff pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from b059664 Merge pull request #4218: Build output redirection option new 90402e4 enforce fixed sharding new

[beam] 11/13: Fixes tests

This is an automated email from the ASF dual-hosted git repository. jkff pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 060f05c659920c3a48dbc67c38db770788802d06 Author: Eugene Kirpichov AuthorDate: Mon Nov 27 11:41:25

[beam] 06/13: Unifies windowed and unwindowed finalize.

This is an automated email from the ASF dual-hosted git repository. jkff pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit c6154382263a68d7eca893c7da3617d177e4c1df Author: Eugene Kirpichov AuthorDate: Wed Nov 15 20:19:09

[jira] [Created] (BEAM-3304) Go triggering support

Henning Rohde created BEAM-3304: --- Summary: Go triggering support Key: BEAM-3304 URL: https://issues.apache.org/jira/browse/BEAM-3304 Project: Beam Issue Type: Improvement Components:

[jira] [Closed] (BEAM-3030) watchForNewFiles() can emit a file multiple times if it's growing

[ https://issues.apache.org/jira/browse/BEAM-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov closed BEAM-3030. -- Resolution: Fixed > watchForNewFiles() can emit a file multiple times if it's growing >

[jira] [Commented] (BEAM-413) Mean$CountSum tests for floating point equality

[ https://issues.apache.org/jira/browse/BEAM-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279330#comment-16279330 ] ASF GitHub Bot commented on BEAM-413: - huygaa11 opened a new pull request #4219: [BEAM-413] Created

[beam] 01/01: Merge pull request #4218: Build output redirection option

This is an automated email from the ASF dual-hosted git repository. kenn pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit b0596642561c386ceb4b28f3d83befd2757d63be Merge: fd8ad27 5bc69dd Author: Kenn Knowles AuthorDate: Tue

[jira] [Created] (BEAM-3300) Portable flattens in Go SDK Harness

Henning Rohde created BEAM-3300: --- Summary: Portable flattens in Go SDK Harness Key: BEAM-3300 URL: https://issues.apache.org/jira/browse/BEAM-3300 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-3302) Go CoGBK support

Henning Rohde created BEAM-3302: --- Summary: Go CoGBK support Key: BEAM-3302 URL: https://issues.apache.org/jira/browse/BEAM-3302 Project: Beam Issue Type: Improvement Components:

[jira] [Commented] (BEAM-3030) watchForNewFiles() can emit a file multiple times if it's growing

[ https://issues.apache.org/jira/browse/BEAM-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279443#comment-16279443 ] ASF GitHub Bot commented on BEAM-3030: -- asfgit closed pull request #4190: [BEAM-3030] Adds a

[jira] [Created] (BEAM-3309) Multi-bundle execution in Go runtime

Henning Rohde created BEAM-3309: --- Summary: Multi-bundle execution in Go runtime Key: BEAM-3309 URL: https://issues.apache.org/jira/browse/BEAM-3309 Project: Beam Issue Type: Improvement

[jira] [Created] (BEAM-3308) Improve Go exec runtime error handling

Henning Rohde created BEAM-3308: --- Summary: Improve Go exec runtime error handling Key: BEAM-3308 URL: https://issues.apache.org/jira/browse/BEAM-3308 Project: Beam Issue Type: Improvement

[jira] [Commented] (BEAM-607) Add DistributedLog IO

[ https://issues.apache.org/jira/browse/BEAM-607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279614#comment-16279614 ] ASF GitHub Bot commented on BEAM-607: - jkff closed pull request #1464: BEAM-607: Add a bounded source

[jira] [Commented] (BEAM-2750) Read whole files as one PCollection element each

[ https://issues.apache.org/jira/browse/BEAM-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279615#comment-16279615 ] ASF GitHub Bot commented on BEAM-2750: -- jkff closed pull request #3717: [BEAM-2750][BEAM-2751]

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

[ https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279681#comment-16279681 ] Valentyn Tymofieiev commented on BEAM-3040: --- cc: [~robertwb] Another recent error: test_errors

[jira] [Commented] (BEAM-3297) StartBundle/FinishBundle should be called only if there are elements

[ https://issues.apache.org/jira/browse/BEAM-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279603#comment-16279603 ] Kenneth Knowles commented on BEAM-3297: --- Empty bundles are permitted, and I think it is permissible

[jira] [Commented] (BEAM-3297) StartBundle/FinishBundle should be called only if there are elements

[ https://issues.apache.org/jira/browse/BEAM-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279610#comment-16279610 ] Kenneth Knowles commented on BEAM-3297: --- BTW a bundle can have timers but no elements. >

Build failed in Jenkins: beam_PerformanceTests_Python #644

See Changes: [wcn] Revert "Introduce a property to influence output location." [wcn] Introduce a property to influence output location. [kirpichov] enforce fixed sharding [kirpichov] Merges

Build failed in Jenkins: beam_PerformanceTests_Spark #1088

See Changes: [wcn] Revert "Introduce a property to influence output location." [wcn] Introduce a property to influence output location. [kirpichov] enforce fixed sharding [kirpichov] Merges

Build failed in Jenkins: beam_PerformanceTests_Spark #1085

See -- Started by timer [EnvInject] - Loading node environment variables. Building remotely on beam2 (beam) in workspace

[jira] [Created] (BEAM-3290) Construct iterators directly if possible to allow spilling to disk

holdenk created BEAM-3290: - Summary: Construct iterators directly if possible to allow spilling to disk Key: BEAM-3290 URL: https://issues.apache.org/jira/browse/BEAM-3290 Project: Beam Issue Type:

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #3628

See

[jira] [Assigned] (BEAM-3070) Add support for windowed filenames in Python SDK

[ https://issues.apache.org/jira/browse/BEAM-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vilhelm von Ehrenheim reassigned BEAM-3070: --- Assignee: Vilhelm von Ehrenheim > Add support for windowed filenames in

[jira] [Commented] (BEAM-3070) Add support for windowed filenames in Python SDK

[ https://issues.apache.org/jira/browse/BEAM-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278786#comment-16278786 ] Vilhelm von Ehrenheim commented on BEAM-3070: - Sure! I'd be happy to. Might need some pointers

[jira] [Created] (BEAM-3291) Add Kinesis Sink

Ismaël Mejía created BEAM-3291: -- Summary: Add Kinesis Sink Key: BEAM-3291 URL: https://issues.apache.org/jira/browse/BEAM-3291 Project: Beam Issue Type: Improvement Components:

[jira] [Commented] (BEAM-3026) Improve retrying in ElasticSearch client

[ https://issues.apache.org/jira/browse/BEAM-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278613#comment-16278613 ] Etienne Chauchot commented on BEAM-3026: Maybe an exponential backoff > Improve retrying in