[jira] [Created] (BEAM-2958) Expose a top level user agent PipelineOption which can be communicated to external services

2017-09-15 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-2958: --- Summary: Expose a top level user agent PipelineOption which can be communicated to external services Key: BEAM-2958 URL: https://issues.apache.org/jira/browse/BEAM-2958

[jira] [Resolved] (BEAM-2790) Error while reading from Amazon S3 via Hadoop File System

2017-09-15 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2790. - Resolution: Fixed > Error while reading from Amazon S3 via Hadoop File System >

[jira] [Assigned] (BEAM-2808) Improve error message when DoFn @ProcessElement has the wrong window type

2017-09-14 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reassigned BEAM-2808: --- Assignee: Daniel Oliveira (was: Luke Cwik) > Improve error message when DoFn @ProcessElement has the

[jira] [Assigned] (BEAM-2808) Improve error message when DoFn @ProcessElement has the wrong window type

2017-09-14 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reassigned BEAM-2808: --- Assignee: Luke Cwik > Improve error message when DoFn @ProcessElement has the wrong window type >

[jira] [Commented] (BEAM-2500) Add support for S3 as a Apache Beam FileSystem

2017-09-14 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16166823#comment-16166823 ] Luke Cwik commented on BEAM-2500: - The GCS implementation uses a fixed size buffer of 32 or 64mbs (don't

[jira] [Commented] (BEAM-2500) Add support for S3 as a Apache Beam FileSystem

2017-09-14 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16166735#comment-16166735 ] Luke Cwik commented on BEAM-2500: - Performing the multipart download/upload will become important as 5GiBs

[jira] [Commented] (BEAM-2948) Unable to find registrar when restoring flink job from savepoint

2017-09-14 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1616#comment-1616 ] Luke Cwik commented on BEAM-2948: - FileSystems assumes that it has been initialized early on within each

[jira] [Updated] (BEAM-2956) DataflowRunner incorrectly reports the user agent for the Dataflow distribution

2017-09-13 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2956: Description: The DataflowRunner when distributed with the Dataflow SDK distribution may incorrectly submit

[jira] [Created] (BEAM-2956) DataflowRunner incorrectly reports the user agent for the Dataflow distribution

2017-09-13 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-2956: --- Summary: DataflowRunner incorrectly reports the user agent for the Dataflow distribution Key: BEAM-2956 URL: https://issues.apache.org/jira/browse/BEAM-2956 Project: Beam

[jira] [Commented] (BEAM-2868) Allow FileSystems.copy() to copy between different scheme

2017-09-13 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164876#comment-16164876 ] Luke Cwik commented on BEAM-2868: - We want to stay away from methods which only take a single thing to do

[jira] [Resolved] (BEAM-2952) How to use KV.OrderByKey

2017-09-12 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2952. - Resolution: Invalid Fix Version/s: Not applicable > How to use KV.OrderByKey >

[jira] [Commented] (BEAM-2952) How to use KV.OrderByKey

2017-09-12 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164085#comment-16164085 ] Luke Cwik commented on BEAM-2952: - The KV.OrderByKey is only meant as a comparator and not used to create

[jira] [Created] (BEAM-2948) Unable to find registrar when restoring flink job from savepoint

2017-09-12 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-2948: --- Summary: Unable to find registrar when restoring flink job from savepoint Key: BEAM-2948 URL: https://issues.apache.org/jira/browse/BEAM-2948 Project: Beam Issue

[jira] [Assigned] (BEAM-2868) Allow FileSystems.copy() to copy between different scheme

2017-09-12 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reassigned BEAM-2868: --- Assignee: (was: Kenneth Knowles) > Allow FileSystems.copy() to copy between different scheme >

[jira] [Assigned] (BEAM-2868) Allow FileSystems.copy() to copy between different scheme

2017-09-12 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reassigned BEAM-2868: --- Assignee: Kenneth Knowles (was: Luke Cwik) > Allow FileSystems.copy() to copy between different

[jira] [Commented] (BEAM-2868) Allow FileSystems.copy() to copy between different scheme

2017-09-12 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163539#comment-16163539 ] Luke Cwik commented on BEAM-2868: - Currently FileSystems.copy() enforces that the same scheme is used

[jira] [Created] (BEAM-2902) Add user state support for ParDo.Multi for the Dataflow runner

2017-09-11 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-2902: --- Summary: Add user state support for ParDo.Multi for the Dataflow runner Key: BEAM-2902 URL: https://issues.apache.org/jira/browse/BEAM-2902 Project: Beam Issue Type:

[jira] [Updated] (BEAM-2902) Add user state support for ParDo.Multi for the Dataflow runner

2017-09-11 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2902: Description: Add support for user state in a ParDo.Multi by: * adding expansion/conversion logic within

[jira] [Created] (BEAM-2863) Add support for Side Inputs over the Fn API

2017-09-07 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-2863: --- Summary: Add support for Side Inputs over the Fn API Key: BEAM-2863 URL: https://issues.apache.org/jira/browse/BEAM-2863 Project: Beam Issue Type: Improvement

[jira] [Commented] (BEAM-2862) Add support for User State over the Fn API

2017-09-07 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157426#comment-16157426 ] Luke Cwik commented on BEAM-2862: - A lot of the work for the Java SDK harness was captured underneath

[jira] [Updated] (BEAM-2862) Add support for User State over the Fn API

2017-09-07 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2862: Description: https://s.apache.org/beam-fn-api-state-api (was: Design:

[jira] [Created] (BEAM-2862) Add support for User State over the Fn API

2017-09-07 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-2862: --- Summary: Add support for User State over the Fn API Key: BEAM-2862 URL: https://issues.apache.org/jira/browse/BEAM-2862 Project: Beam Issue Type: Improvement

[jira] [Updated] (BEAM-2421) Migrate Apache Beam to use impulse primitive as the only root primitive

2017-09-07 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2421: Summary: Migrate Apache Beam to use impulse primitive as the only root primitive (was: Migrate Dataflow to

[jira] [Updated] (BEAM-2822) Add support for progress reporting in fn API

2017-09-07 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2822: Description: https://s.apache.org/beam-fn-api-progress-reporting (was: https) > Add support for progress

[jira] [Updated] (BEAM-2822) Add support for progress reporting in fn API

2017-09-07 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2822: Description: https > Add support for progress reporting in fn API >

[jira] [Commented] (BEAM-2837) Writing To Spanner From Google Cloud DataFlow - Failure

2017-09-05 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154812#comment-16154812 ] Luke Cwik commented on BEAM-2837: - Is there more to the stack trace in the description, usually buried

[jira] [Assigned] (BEAM-2829) Add ability to set job labels in DataflowPipelineOptions

2017-08-30 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reassigned BEAM-2829: --- Assignee: Zongwei Zhou (was: Thomas Groh) > Add ability to set job labels in DataflowPipelineOptions

[jira] [Comment Edited] (BEAM-2826) Need to generate a single XML file when write is performed on small amount of data

2017-08-30 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147446#comment-16147446 ] Luke Cwik edited comment on BEAM-2826 at 8/30/17 3:34 PM: -- What about doing this

[jira] [Commented] (BEAM-2826) Need to generate a single XML file when write is performed on small amount of data

2017-08-30 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147446#comment-16147446 ] Luke Cwik commented on BEAM-2826: - What about doing this in your pipeline: ``` PC -> DoFn(AssignVoidKey) ->

[jira] [Commented] (BEAM-2823) Beam Windows MavenInstall tests failing

2017-08-29 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146457#comment-16146457 ] Luke Cwik commented on BEAM-2823: - This is really an issue with the maven protoc plugin:

[jira] [Commented] (BEAM-2790) Error while reading from Amazon S3 via Hadoop File System

2017-08-24 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140321#comment-16140321 ] Luke Cwik commented on BEAM-2790: - That would fix it for S3A, but I can only expect that there are other

[jira] [Updated] (BEAM-2779) PipelineOptionsFactory should prevent non PipelineOptions interfaces from being constructed.

2017-08-18 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2779: Summary: PipelineOptionsFactory should prevent non PipelineOptions interfaces from being constructed. (was:

[jira] [Updated] (BEAM-2779) PipelineOptions should not serialize data for types on interfaces which aren't PipelineOptions

2017-08-18 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2779: Labels: starter (was: ) > PipelineOptions should not serialize data for types on interfaces which > aren't

[jira] [Updated] (BEAM-2779) PipelineOptions should not serialize data for types on interfaces which aren't PipelineOptions

2017-08-18 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2779: Affects Version/s: 2.0.0 > PipelineOptions should not serialize data for types on interfaces which > aren't

[jira] [Created] (BEAM-2779) PipelineOptions should not serialize data for types on interfaces which aren't PipelineOptions

2017-08-18 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-2779: --- Summary: PipelineOptions should not serialize data for types on interfaces which aren't PipelineOptions Key: BEAM-2779 URL: https://issues.apache.org/jira/browse/BEAM-2779

[jira] [Created] (BEAM-2777) InMemoryStateInternals MapState / SetState should use StructuralValue for membership / key comparison checks

2017-08-17 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-2777: --- Summary: InMemoryStateInternals MapState / SetState should use StructuralValue for membership / key comparison checks Key: BEAM-2777 URL: https://issues.apache.org/jira/browse/BEAM-2777

[jira] [Assigned] (BEAM-1373) Update Python SDK code to support both Python 2 and 3

2017-08-17 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reassigned BEAM-1373: --- Assignee: Holden Karau (was: Ahmet Altay) > Update Python SDK code to support both Python 2 and 3 >

[jira] [Assigned] (BEAM-2773) Add Apache RocketMQ IO plugin

2017-08-17 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reassigned BEAM-2773: --- Assignee: vongosling (was: Luke Cwik) > Add Apache RocketMQ IO plugin >

[jira] [Created] (BEAM-2771) Allow using side inputs as ValueProviders

2017-08-15 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-2771: --- Summary: Allow using side inputs as ValueProviders Key: BEAM-2771 URL: https://issues.apache.org/jira/browse/BEAM-2771 Project: Beam Issue Type: New Feature

[jira] [Updated] (BEAM-2726) Typo in programming guide

2017-08-04 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2726: Summary: Typo in programming guide (was: Document Error?) > Typo in programming guide >

[jira] [Assigned] (BEAM-2726) Document Error?

2017-08-04 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reassigned BEAM-2726: --- Assignee: Luke Cwik (was: Frances Perry) > Document Error? > --- > >

[jira] [Resolved] (BEAM-2658) SerializableCoder has higher precedence over ProtoCoder in CoderRegistry#getCoder

2017-07-26 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2658. - Resolution: Fixed Fix Version/s: 2.2.0 > SerializableCoder has higher precedence over ProtoCoder in

[jira] [Assigned] (BEAM-2658) SerializableCoder has higher precedence over ProtoCoder in CoderRegistry#getCoder

2017-07-26 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reassigned BEAM-2658: --- Assignee: Neville Li (was: Davor Bonaci) > SerializableCoder has higher precedence over ProtoCoder

[jira] [Commented] (BEAM-2676) move BeamSqlRow and BeamSqlRowType to sdk/java/core

2017-07-25 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099989#comment-16099989 ] Luke Cwik commented on BEAM-2676: - I would suggest moving this to *org.apache.beam.sdk.values* instead of

[jira] [Resolved] (BEAM-2442) DSL SQL: Public classes/methods should not expose/use calcite types

2017-07-24 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2442. - Resolution: Fixed Fix Version/s: 2.2.0 > DSL SQL: Public classes/methods should not expose/use

[jira] [Resolved] (BEAM-2440) DSL SQL: Reduce visibility to simplify backwards compatibility

2017-07-24 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2440. - Resolution: Fixed Fix Version/s: 2.2.0 > DSL SQL: Reduce visibility to simplify backwards

[jira] [Updated] (BEAM-2445) DSL SQL to use service locator pattern to automatically register UDFs

2017-07-24 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2445: Labels: (was: dsl_sql_merge) > DSL SQL to use service locator pattern to automatically register UDFs >

[jira] [Commented] (BEAM-2658) SerializableCoder has higher precedence over ProtoCoder in CoderRegistry#getCoder

2017-07-24 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098322#comment-16098322 ] Luke Cwik commented on BEAM-2658: - Arbitrary number based precedence orders are difficult to maintain. I

[jira] [Comment Edited] (BEAM-2658) SerializableCoder has higher precedence over ProtoCoder in CoderRegistry#getCoder

2017-07-24 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098322#comment-16098322 ] Luke Cwik edited comment on BEAM-2658 at 7/24/17 12:57 PM: --- Arbitrary number

[jira] [Comment Edited] (BEAM-2658) SerializableCoder has higher precedence over ProtoCoder in CoderRegistry#getCoder

2017-07-24 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098322#comment-16098322 ] Luke Cwik edited comment on BEAM-2658 at 7/24/17 12:57 PM: --- Arbitrary number

[jira] [Updated] (BEAM-2642) Upgrade to Google Auth 0.7.1

2017-07-24 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2642: Fix Version/s: (was: 2.2.0) > Upgrade to Google Auth 0.7.1 > > >

[jira] [Updated] (BEAM-2662) Quickstart for Spark Java not working

2017-07-24 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2662: Fix Version/s: (was: 2.2.0) > Quickstart for Spark Java not working >

[jira] [Commented] (BEAM-2441) DSL SQL maven module location and name

2017-07-23 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097796#comment-16097796 ] Luke Cwik commented on BEAM-2441: - The reason why I believe its important to choose a good name know is

[jira] [Comment Edited] (BEAM-2662) Quickstart for Spark Java not working

2017-07-23 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097794#comment-16097794 ] Luke Cwik edited comment on BEAM-2662 at 7/23/17 8:32 PM: --

[jira] [Comment Edited] (BEAM-2662) Quickstart for Spark Java not working

2017-07-23 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097794#comment-16097794 ] Luke Cwik edited comment on BEAM-2662 at 7/23/17 8:32 PM: --

[jira] [Commented] (BEAM-2662) Quickstart for Spark Java not working

2017-07-23 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097794#comment-16097794 ] Luke Cwik commented on BEAM-2662: -

[jira] [Updated] (BEAM-2662) Quickstart for Spark Java not working

2017-07-23 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2662: Description: Spark version is missing in generated examples archetype pom.xml: {code}

[jira] [Updated] (BEAM-2662) Quickstart for Spark Java not working

2017-07-23 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2662: Description: Spark version is missing in generated examples archetype pom.xml: {code}

[jira] [Created] (BEAM-2662) Quickstart for Spark Java not working

2017-07-23 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-2662: --- Summary: Quickstart for Spark Java not working Key: BEAM-2662 URL: https://issues.apache.org/jira/browse/BEAM-2662 Project: Beam Issue Type: Bug Components:

[jira] [Commented] (BEAM-2658) SerializableCoder has higher precedence over ProtoCoder in CoderRegistry#getCoder

2017-07-23 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097685#comment-16097685 ] Luke Cwik commented on BEAM-2658: - I think this issue will come up more often then this single case. Is

[jira] [Resolved] (BEAM-2642) Upgrade to Google Auth 0.7.1

2017-07-19 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2642. - Resolution: Fixed Fix Version/s: 2.2.0 > Upgrade to Google Auth 0.7.1 >

[jira] [Updated] (BEAM-2642) Upgrade to Google Auth 0.7.1

2017-07-19 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2642: Description: Looking up application default credentials on a GCE VM can fail due to VM metadata server

[jira] [Created] (BEAM-2642) Upgrade to Google Auth 0.7.1

2017-07-19 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-2642: --- Summary: Upgrade to Google Auth 0.7.1 Key: BEAM-2642 URL: https://issues.apache.org/jira/browse/BEAM-2642 Project: Beam Issue Type: Bug Components:

[jira] [Updated] (BEAM-2642) Upgrade to Google Auth 0.7.1

2017-07-19 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2642: Description: Looking up application default credentials on a GCE VM can fail due to VM metadata server

[jira] [Commented] (BEAM-2614) Harness doesn't build with Java7

2017-07-13 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086734#comment-16086734 ] Luke Cwik commented on BEAM-2614: - I think this is working as intended as I think our only goal is that

[jira] [Reopened] (BEAM-2430) Java FnApiDoFnRunner to share across runners

2017-07-11 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reopened BEAM-2430: - Assignee: Kenneth Knowles (was: Luke Cwik) PR/3432 was about writing a DoFnRunner for executing DoFns

[jira] [Commented] (BEAM-1286) DataflowRunner handling of missing filesToStage

2017-07-11 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082439#comment-16082439 ] Luke Cwik commented on BEAM-1286: - I think the point is that if there is nothing to stage (even after

[jira] [Assigned] (BEAM-2553) Update Maven exec plugin to 1.6.0 to incorporate messaging improvements

2017-07-07 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reassigned BEAM-2553: --- Assignee: Luke Cwik (was: Frances Perry) > Update Maven exec plugin to 1.6.0 to incorporate

[jira] [Resolved] (BEAM-2553) Update Maven exec plugin to 1.6.0 to incorporate messaging improvements

2017-07-07 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2553. - Resolution: Fixed Fix Version/s: 2.2.0 > Update Maven exec plugin to 1.6.0 to incorporate messaging

[jira] [Resolved] (BEAM-2552) Update Dataflow container version for 2.1.0 release

2017-07-06 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2552. - Resolution: Fixed Fix Version/s: 2.1.0 > Update Dataflow container version for 2.1.0 release >

[jira] [Commented] (BEAM-1757) Improve log configuration when using multiple logging frameworks in Pipeline

2017-07-05 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075231#comment-16075231 ] Luke Cwik commented on BEAM-1757: - Once the Runner/Fn API progresses to a point, this will just be

[jira] [Updated] (BEAM-1757) Improve log configuration when using multiple logging frameworks in Pipeline

2017-07-05 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-1757: Component/s: beam-model-runner-api beam-model-fn-api > Improve log configuration when using

[jira] [Created] (BEAM-2553) Update Maven exec plugin to 1.6.0 to incorporate messaging improvements

2017-07-05 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-2553: --- Summary: Update Maven exec plugin to 1.6.0 to incorporate messaging improvements Key: BEAM-2553 URL: https://issues.apache.org/jira/browse/BEAM-2553 Project: Beam

[jira] [Created] (BEAM-2552) Update Dataflow container version for 2.1.0 release

2017-07-05 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-2552: --- Summary: Update Dataflow container version for 2.1.0 release Key: BEAM-2552 URL: https://issues.apache.org/jira/browse/BEAM-2552 Project: Beam Issue Type: Task

[jira] [Resolved] (BEAM-2373) AvroSource: Premature End of stream Exception on SnappyCompressorInputStream

2017-06-29 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2373. - Resolution: Fixed > AvroSource: Premature End of stream Exception on SnappyCompressorInputStream >

[jira] [Resolved] (BEAM-2389) GcpCoreApiSurfaceTest isn't testing right module

2017-06-28 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2389. - Resolution: Fixed > GcpCoreApiSurfaceTest isn't testing right module >

[jira] [Updated] (BEAM-1187) GCP Transport not performing timed backoff after connection failure

2017-06-27 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-1187: Description: The http request retries are failing and seemingly being immediately retried if there is a

[jira] [Updated] (BEAM-1187) GCP Transport not performing timed backoff after connection failure

2017-06-27 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-1187: Description: The http request retries are failing and seemingly being immediately retried if there is a

[jira] [Resolved] (BEAM-2522) upgrading jackson

2017-06-27 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2522. - Resolution: Fixed Fix Version/s: 2.1.0 > upgrading jackson > - > >

[jira] [Updated] (BEAM-2522) upgrading jackson

2017-06-27 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2522: Affects Version/s: (was: 2.1.0) > upgrading jackson > - > > Key:

[jira] [Resolved] (BEAM-2392) Avoid use of proto builder clone

2017-06-27 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2392. - Resolution: Fixed Fix Version/s: 2.1.0 > Avoid use of proto builder clone >

[jira] [Updated] (BEAM-2392) Avoid use of proto builder clone

2017-06-27 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2392: Affects Version/s: 2.0.0 > Avoid use of proto builder clone > > >

[jira] [Updated] (BEAM-2392) Avoid use of proto builder clone

2017-06-27 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2392: Affects Version/s: (was: 2.1.0) > Avoid use of proto builder clone > >

[jira] [Resolved] (BEAM-2514) Improve error message for missing required options of Beam pipeline

2017-06-27 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2514. - Resolution: Fixed Fix Version/s: 2.1.0 > Improve error message for missing required options of Beam

[jira] [Assigned] (BEAM-2522) upgrading jackson

2017-06-27 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reassigned BEAM-2522: --- Assignee: Antony Mayi (was: Davor Bonaci) > upgrading jackson > - > >

[jira] [Commented] (BEAM-2523) GCP IO exposes protobuf on its API surface, causing user pain

2017-06-26 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064247#comment-16064247 ] Luke Cwik commented on BEAM-2523: - Based upon the dependency dump, it seems as though no protobuf-java is

[jira] [Commented] (BEAM-2522) upgrading jackson

2017-06-26 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064198#comment-16064198 ] Luke Cwik commented on BEAM-2522: - [~antonymayi] Do you want to propose a PR which does this change? >

[jira] [Updated] (BEAM-2522) upgrading jackson

2017-06-26 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2522: Priority: Minor (was: Major) > upgrading jackson > - > > Key: BEAM-2522 >

[jira] [Commented] (BEAM-2514) Improve error message for missing required options of Beam pipeline

2017-06-26 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16063397#comment-16063397 ] Luke Cwik commented on BEAM-2514: - This change assumes that all users are CLI users (compared to the

[jira] [Resolved] (BEAM-2439) Datastore writer can fail to progress if Datastore is slow

2017-06-22 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2439. - Resolution: Fixed Fix Version/s: 2.1.0 > Datastore writer can fail to progress if Datastore is slow

[jira] [Assigned] (BEAM-2439) Datastore writer can fail to progress if Datastore is slow

2017-06-22 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reassigned BEAM-2439: --- Assignee: Colin Phipps (was: Stephen Sisk) > Datastore writer can fail to progress if Datastore is

[jira] [Commented] (BEAM-2497) TextIO can't read concatenated gzip files

2017-06-22 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16059932#comment-16059932 ] Luke Cwik commented on BEAM-2497: - It turns out that many implementations of gzip get the spec wrong in

[jira] [Assigned] (BEAM-2500) Add support for S3 as a Apache Beam FileSystem

2017-06-22 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik reassigned BEAM-2500: --- Assignee: (was: Davor Bonaci) > Add support for S3 as a Apache Beam FileSystem >

[jira] [Resolved] (BEAM-994) Support for S3 file source and sink

2017-06-22 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-994. Resolution: Won't Fix Assignee: Luke Cwik Fix Version/s: Not applicable Superceded by

[jira] [Resolved] (BEAM-284) Add AWS S3 support to FileBasedSink

2017-06-22 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-284. Resolution: Won't Fix Fix Version/s: Not applicable Superceded by BEAM-2500. Roland, are you

[jira] [Created] (BEAM-2500) Add support for S3 as a Apache Beam FileSystem

2017-06-22 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-2500: --- Summary: Add support for S3 as a Apache Beam FileSystem Key: BEAM-2500 URL: https://issues.apache.org/jira/browse/BEAM-2500 Project: Beam Issue Type: Improvement

[jira] [Created] (BEAM-2492) Have PipelineOptions DisplayData filter out attributes marked with @org.apache.beam.sdk.options.Hidden

2017-06-21 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-2492: --- Summary: Have PipelineOptions DisplayData filter out attributes marked with @org.apache.beam.sdk.options.Hidden Key: BEAM-2492 URL: https://issues.apache.org/jira/browse/BEAM-2492

[jira] [Resolved] (BEAM-2443) BeamSqlRecordType should migrate to using a builder pattern via AutoValue/AutoBuilder

2017-06-20 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2443. - Resolution: Fixed Fix Version/s: Not applicable > BeamSqlRecordType should migrate to using a

[jira] [Resolved] (BEAM-2444) DSL SQL: Use standard java exceptions instead of custom ones

2017-06-20 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2444. - Resolution: Fixed Fix Version/s: Not applicable > DSL SQL: Use standard java exceptions instead of

[jira] [Created] (BEAM-2482) CodedValueMutationDetector should use the coders structural value

2017-06-20 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-2482: --- Summary: CodedValueMutationDetector should use the coders structural value Key: BEAM-2482 URL: https://issues.apache.org/jira/browse/BEAM-2482 Project: Beam Issue

<    2   3   4   5   6   7   8   9   10   >