[jira] [Commented] (BEAM-10068) Modify behavior of Dynamic Destinations

[ https://issues.apache.org/jira/browse/BEAM-10068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114533#comment-17114533 ] Reuven Lax commented on BEAM-10068: --- It is reasonable to allow specifying a per-destination numShards.

[jira] [Commented] (BEAM-10053) Timers exception on "Job Drain" while using stateful beam processing in global window

[ https://issues.apache.org/jira/browse/BEAM-10053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113286#comment-17113286 ] Reuven Lax commented on BEAM-10053: --- Something appears off here:

[jira] [Reopened] (BEAM-3489) Expose the message id of received messages within PubsubMessage

[ https://issues.apache.org/jira/browse/BEAM-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuven Lax reopened BEAM-3489: -- This feature does not work correctly on Dataflow. Dataflow only returns a PubSubMessage if the 

[jira] [Resolved] (BEAM-10015) output timestamp not properly propagated through the Dataflow runner

[ https://issues.apache.org/jira/browse/BEAM-10015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuven Lax resolved BEAM-10015. --- Fix Version/s: 2.21.0 Resolution: Fixed > output timestamp not properly propagated through

[jira] [Created] (BEAM-10015) output timestamp not properly propagated through the Dataflow runner

Reuven Lax created BEAM-10015: - Summary: output timestamp not properly propagated through the Dataflow runner Key: BEAM-10015 URL: https://issues.apache.org/jira/browse/BEAM-10015 Project: Beam

[jira] [Commented] (BEAM-9814) Error occurred when transforming from row to a new row without setCoder

[ https://issues.apache.org/jira/browse/BEAM-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108644#comment-17108644 ] Reuven Lax commented on BEAM-9814: -- all these cases, you need to call setRowSchema on the PCollection. I

[jira] [Commented] (BEAM-9863) AvroUtils is converting incorrectly LogicalType Timestamps from long into Joda DateTimes

[ https://issues.apache.org/jira/browse/BEAM-9863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097534#comment-17097534 ] reuven lax commented on BEAM-9863: -- My thought too. It might be worth looking up the commit that

[jira] [Commented] (BEAM-9759) Pipeline creation with large number of shards/streams takes long time

[ https://issues.apache.org/jira/browse/BEAM-9759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095092#comment-17095092 ] Reuven Lax commented on BEAM-9759: -- The best solution would be to create a new SplittableDoFn version of

[jira] [Commented] (BEAM-6566) SqlTransform does not work for beam version above 2.6.0 if RowCoder explicitly chosen

[ https://issues.apache.org/jira/browse/BEAM-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094666#comment-17094666 ] Reuven Lax commented on BEAM-6566: -- We've redefined RowCoder to be SchemaCoder, so this should work.

[jira] [Commented] (BEAM-9613) BigQuery IO not support convert double type for beam row

[ https://issues.apache.org/jira/browse/BEAM-9613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091742#comment-17091742 ] Reuven Lax commented on BEAM-9613: -- JSON is a textual format. The fact that that the Java JSON object

[jira] [Commented] (BEAM-9696) UnionCoder IndexOutOfBoundsException in schema-driven join transform

[ https://issues.apache.org/jira/browse/BEAM-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091735#comment-17091735 ] Reuven Lax commented on BEAM-9696: -- Is there an easy repro to run? This failure is happening deep inside

[jira] [Commented] (BEAM-9814) Error occurred when transforming from row to a new row without setCoder

[ https://issues.apache.org/jira/browse/BEAM-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091725#comment-17091725 ] Reuven Lax commented on BEAM-9814: -- But we should add. a schema-aware version of MapElements. Something

[jira] [Commented] (BEAM-9814) Error occurred when transforming from row to a new row without setCoder

[ https://issues.apache.org/jira/browse/BEAM-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091721#comment-17091721 ] Reuven Lax commented on BEAM-9814: -- Better to use setRowSchema(newSchema). You should't manipulate

[jira] [Commented] (BEAM-9613) BigQuery IO not support convert double type for beam row

[ https://issues.apache.org/jira/browse/BEAM-9613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091720#comment-17091720 ] Reuven Lax commented on BEAM-9613: -- A simple fix for the bug experienced here would be for the conversion

[jira] [Commented] (BEAM-9613) BigQuery IO not support convert double type for beam row

[ https://issues.apache.org/jira/browse/BEAM-9613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091719#comment-17091719 ] Reuven Lax commented on BEAM-9613: -- TableRow is a json object, so fundamentally its values are

[jira] [Commented] (BEAM-9613) BigQuery IO not support convert double type for beam row

[ https://issues.apache.org/jira/browse/BEAM-9613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091120#comment-17091120 ] Reuven Lax commented on BEAM-9613: -- No, looks to be a limitation of BigQueryUtils. It currently assumes

[jira] [Commented] (BEAM-9370) ParDoSchemaTests fail when run with Java 11 on Flink 1.10

[ https://issues.apache.org/jira/browse/BEAM-9370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088839#comment-17088839 ] Reuven Lax commented on BEAM-9370: -- Sent pr/11480 > ParDoSchemaTests fail when run with Java 11 on Flink

[jira] [Commented] (BEAM-9370) ParDoSchemaTests fail when run with Java 11 on Flink 1.10

[ https://issues.apache.org/jira/browse/BEAM-9370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088809#comment-17088809 ] Reuven Lax commented on BEAM-9370: -- Yes, the problem is because this code   COLLECTION_TYPE  

[jira] [Commented] (BEAM-9569) Coder inference should be disabled for Row types

[ https://issues.apache.org/jira/browse/BEAM-9569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086155#comment-17086155 ] Reuven Lax commented on BEAM-9569: -- This failure is not related to POJOs The actual problem is that

[jira] [Created] (BEAM-9670) CoGroup transform should allow widening nullability in key schemas

Reuven Lax created BEAM-9670: Summary: CoGroup transform should allow widening nullability in key schemas Key: BEAM-9670 URL: https://issues.apache.org/jira/browse/BEAM-9670 Project: Beam Issue

[jira] [Commented] (BEAM-9557) Error setting processing time timers near end-of-window

[ https://issues.apache.org/jira/browse/BEAM-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067014#comment-17067014 ] Reuven Lax commented on BEAM-9557: -- pr/11226 sent for review > Error setting processing time timers near

[jira] [Commented] (BEAM-9557) Error setting processing time timers near end-of-window

[ https://issues.apache.org/jira/browse/BEAM-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066892#comment-17066892 ] Reuven Lax commented on BEAM-9557: -- ok, this seems simple. I'll send out a PR. > Error setting

[jira] [Commented] (BEAM-9557) Error setting processing time timers near end-of-window

[ https://issues.apache.org/jira/browse/BEAM-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066364#comment-17066364 ] Reuven Lax commented on BEAM-9557: -- Ah yes, I was referring to processing-time timers. For event-time

[jira] [Commented] (BEAM-9557) Error setting processing time timers near end-of-window

[ https://issues.apache.org/jira/browse/BEAM-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066023#comment-17066023 ] Reuven Lax commented on BEAM-9557: -- I'm not sure why the simple solution I proposed does not work. For

[jira] [Commented] (BEAM-9578) Enumerating artifacts is too expensive in Java

[ https://issues.apache.org/jira/browse/BEAM-9578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17065247#comment-17065247 ] Reuven Lax commented on BEAM-9578: -- This is causing Java Precommit tests to be extremely flaky (often

[jira] [Commented] (BEAM-9557) Error setting processing time timers near end-of-window

[ https://issues.apache.org/jira/browse/BEAM-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17065112#comment-17065112 ] Reuven Lax commented on BEAM-9557: -- This looks like it might be a type bug? I think that 

[jira] [Created] (BEAM-9569) Coder inference should be disabled for Row types

Reuven Lax created BEAM-9569: Summary: Coder inference should be disabled for Row types Key: BEAM-9569 URL: https://issues.apache.org/jira/browse/BEAM-9569 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-9568) Move Beam SQL to use the schema join transforms

Reuven Lax created BEAM-9568: Summary: Move Beam SQL to use the schema join transforms Key: BEAM-9568 URL: https://issues.apache.org/jira/browse/BEAM-9568 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-9442) Schema Select does not properly handle nested nullable fields

Reuven Lax created BEAM-9442: Summary: Schema Select does not properly handle nested nullable fields Key: BEAM-9442 URL: https://issues.apache.org/jira/browse/BEAM-9442 Project: Beam Issue

[jira] [Resolved] (BEAM-7333) Select need the ability to rename fields

[ https://issues.apache.org/jira/browse/BEAM-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuven Lax resolved BEAM-7333. -- Fix Version/s: 2.20.0 Resolution: Fixed > Select need the ability to rename fields >

[jira] [Created] (BEAM-9393) support schemas in state API

Reuven Lax created BEAM-9393: Summary: support schemas in state API Key: BEAM-9393 URL: https://issues.apache.org/jira/browse/BEAM-9393 Project: Beam Issue Type: Sub-task Components:

[jira] [Created] (BEAM-9353) ByteBuddy Schema code does not properly handle null values

Reuven Lax created BEAM-9353: Summary: ByteBuddy Schema code does not properly handle null values Key: BEAM-9353 URL: https://issues.apache.org/jira/browse/BEAM-9353 Project: Beam Issue Type:

[jira] [Created] (BEAM-9331) The Row object needs better builders

Reuven Lax created BEAM-9331: Summary: The Row object needs better builders Key: BEAM-9331 URL: https://issues.apache.org/jira/browse/BEAM-9331 Project: Beam Issue Type: Sub-task

[jira] [Commented] (BEAM-8543) Dataflow streaming timers are not strictly time ordered when set earlier mid-bundle

[ https://issues.apache.org/jira/browse/BEAM-8543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17027918#comment-17027918 ] Reuven Lax commented on BEAM-8543: -- The problem is more general: if an input bundle contains both element

[jira] [Commented] (BEAM-1589) Add OnWindowExpiration method to Stateful DoFn

[ https://issues.apache.org/jira/browse/BEAM-1589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011071#comment-17011071 ] Reuven Lax commented on BEAM-1589: -- output timestamp support is now in (Flink support pending). I think

[jira] [Commented] (BEAM-9010) BigQuery TableRow's size is toString().length() ?

[ https://issues.apache.org/jira/browse/BEAM-9010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17001198#comment-17001198 ] Reuven Lax commented on BEAM-9010: -- Could you check to see whether the output of TableRowJsonCoder has

[jira] [Commented] (BEAM-9010) BigQuery TableRow's size is toString().length() ?

[ https://issues.apache.org/jira/browse/BEAM-9010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17001196#comment-17001196 ] Reuven Lax commented on BEAM-9010: -- This change could be a major performance regression for production

[jira] [Assigned] (BEAM-6756) Support lazy iterables in schemas

[ https://issues.apache.org/jira/browse/BEAM-6756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuven Lax reassigned BEAM-6756: Assignee: Reuven Lax (was: Shehzaad Nakhoda) > Support lazy iterables in schemas >

[jira] [Commented] (BEAM-3772) BigQueryIO - Can't use DynamicDestination with CREATE_IF_NEEDED for unbounded PCollection and FILE_LOADS

[ https://issues.apache.org/jira/browse/BEAM-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932718#comment-16932718 ] Reuven Lax commented on BEAM-3772: -- It's possible that there's a bug with time partitioning. If

[jira] [Commented] (BEAM-3772) BigQueryIO - Can't use DynamicDestination with CREATE_IF_NEEDED for unbounded PCollection and FILE_LOADS

[ https://issues.apache.org/jira/browse/BEAM-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932698#comment-16932698 ] Reuven Lax commented on BEAM-3772: -- Are you calling withTimePartitioning on BigQueryIO? Also, are you

[jira] [Assigned] (BEAM-3772) BigQueryIO - Can't use DynamicDestination with CREATE_IF_NEEDED for unbounded PCollection and FILE_LOADS

[ https://issues.apache.org/jira/browse/BEAM-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuven Lax reassigned BEAM-3772: Assignee: Reuven Lax (was: Chamikara Jayalath) > BigQueryIO - Can't use DynamicDestination with

[jira] [Commented] (BEAM-3772) BigQueryIO - Can't use DynamicDestination with CREATE_IF_NEEDED for unbounded PCollection and FILE_LOADS

[ https://issues.apache.org/jira/browse/BEAM-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929839#comment-16929839 ] Reuven Lax commented on BEAM-3772: -- As mentioned on the PR, triggers will be per key after a reshuffle,

[jira] [Commented] (BEAM-8102) Portable Flink and Spark failing testSideInputAnnotation[WithMultipleSideInputs]

[ https://issues.apache.org/jira/browse/BEAM-8102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920193#comment-16920193 ] Reuven Lax commented on BEAM-8102: -- sent pr/9461 to fix > Portable Flink and Spark failing >

[jira] [Commented] (BEAM-8102) Portable Flink and Spark failing testSideInputAnnotation[WithMultipleSideInputs]

[ https://issues.apache.org/jira/browse/BEAM-8102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920190#comment-16920190 ] Reuven Lax commented on BEAM-8102: -- Taking a look. I believe this is an easy fix. > Portable Flink and

[jira] [Commented] (BEAM-6858) Support side inputs injected into a DoFn

[ https://issues.apache.org/jira/browse/BEAM-6858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890370#comment-16890370 ] Reuven Lax commented on BEAM-6858: -- Inside ParDo, you will need to get the rest of the information of the

[jira] [Commented] (BEAM-6858) Support side inputs injected into a DoFn

[ https://issues.apache.org/jira/browse/BEAM-6858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890365#comment-16890365 ] Reuven Lax commented on BEAM-6858: -- Some more details: We need to add a way for the user to control the

[jira] [Resolved] (BEAM-6675) The JdbcIO sink should accept schemas

[ https://issues.apache.org/jira/browse/BEAM-6675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuven Lax resolved BEAM-6675. -- Resolution: Fixed Fix Version/s: 2.14.0 > The JdbcIO sink should accept schemas >

[jira] [Commented] (BEAM-6855) Side inputs are not supported when using the state API

[ https://issues.apache.org/jira/browse/BEAM-6855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885853#comment-16885853 ] Reuven Lax commented on BEAM-6855: -- [~kenn] it was never addressed. Can you confirm that the described

[jira] [Commented] (BEAM-6855) Side inputs are not supported when using the state API

[ https://issues.apache.org/jira/browse/BEAM-6855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885670#comment-16885670 ] Reuven Lax commented on BEAM-6855: -- I believe that the side-input code in 

[jira] [Resolved] (BEAM-6674) The JdbcIO source should produce schemas

[ https://issues.apache.org/jira/browse/BEAM-6674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuven Lax resolved BEAM-6674. -- Resolution: Fixed Fix Version/s: 2.14.0 > The JdbcIO source should produce schemas >

[jira] [Resolved] (BEAM-6673) BigQueryIO.Read should automatically produce schemas

[ https://issues.apache.org/jira/browse/BEAM-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuven Lax resolved BEAM-6673. -- Resolution: Fixed Fix Version/s: 2.14.0 > BigQueryIO.Read should automatically produce schemas >

[jira] [Commented] (BEAM-7425) Reading BigQuery Table Data into Java Classes(Pojo) Directly

[ https://issues.apache.org/jira/browse/BEAM-7425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861204#comment-16861204 ] Reuven Lax commented on BEAM-7425: -- BTW the above works. It would be easy to combine into BigQueryIO if

[jira] [Created] (BEAM-7356) Select should support nested schema selectors

Reuven Lax created BEAM-7356: Summary: Select should support nested schema selectors Key: BEAM-7356 URL: https://issues.apache.org/jira/browse/BEAM-7356 Project: Beam Issue Type: Sub-task

[jira] [Commented] (BEAM-7116) Remove KV from Schema transforms

[ https://issues.apache.org/jira/browse/BEAM-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16841777#comment-16841777 ] Reuven Lax commented on BEAM-7116: -- The problem is that Beam special cases KvCoder all over the place, so

[jira] [Created] (BEAM-7337) Allow selecting slices of arrays and maps

Reuven Lax created BEAM-7337: Summary: Allow selecting slices of arrays and maps Key: BEAM-7337 URL: https://issues.apache.org/jira/browse/BEAM-7337 Project: Beam Issue Type: Sub-task

[jira] [Resolved] (BEAM-6772) Select transform has non-intuitive semantics

[ https://issues.apache.org/jira/browse/BEAM-6772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuven Lax resolved BEAM-6772. -- Resolution: Fixed Fix Version/s: 2.12.0 > Select transform has non-intuitive semantics >

[jira] [Commented] (BEAM-7116) Remove KV from Schema transforms

[ https://issues.apache.org/jira/browse/BEAM-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16841051#comment-16841051 ] Reuven Lax commented on BEAM-7116: -- To make this seamless, the Convert transform should be able to

[jira] [Created] (BEAM-7336) KafkaIO should support inferring schemas when reading AVRO

Reuven Lax created BEAM-7336: Summary: KafkaIO should support inferring schemas when reading AVRO Key: BEAM-7336 URL: https://issues.apache.org/jira/browse/BEAM-7336 Project: Beam Issue Type:

[jira] [Created] (BEAM-7335) PubSubIO.writeAvros should infer beam schemas

Reuven Lax created BEAM-7335: Summary: PubSubIO.writeAvros should infer beam schemas Key: BEAM-7335 URL: https://issues.apache.org/jira/browse/BEAM-7335 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-7334) Create a better Schema builder

Reuven Lax created BEAM-7334: Summary: Create a better Schema builder Key: BEAM-7334 URL: https://issues.apache.org/jira/browse/BEAM-7334 Project: Beam Issue Type: Sub-task Components:

[jira] [Created] (BEAM-7333) Select need the ability to rename fields

Reuven Lax created BEAM-7333: Summary: Select need the ability to rename fields Key: BEAM-7333 URL: https://issues.apache.org/jira/browse/BEAM-7333 Project: Beam Issue Type: Sub-task

[jira] [Commented] (BEAM-7116) Remove KV from Schema transforms

[ https://issues.apache.org/jira/browse/BEAM-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16841044#comment-16841044 ] Reuven Lax commented on BEAM-7116: -- Transforms like Group that return KV today should instead return a

[jira] [Resolved] (BEAM-7174) Add transforms for modifying schemas

[ https://issues.apache.org/jira/browse/BEAM-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuven Lax resolved BEAM-7174. -- Resolution: Fixed Fix Version/s: 2.12.0 > Add transforms for modifying schemas >

[jira] [Resolved] (BEAM-6602) Support schemas in BigQueryIO.Write

[ https://issues.apache.org/jira/browse/BEAM-6602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuven Lax resolved BEAM-6602. -- Resolution: Fixed Fix Version/s: 2.12.0 > Support schemas in BigQueryIO.Write >

[jira] [Resolved] (BEAM-6536) ParDo should allow any type with a compatible registered schema in the @Element parameter

[ https://issues.apache.org/jira/browse/BEAM-6536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuven Lax resolved BEAM-6536. -- Resolution: Fixed Fix Version/s: 2.12.0 > ParDo should allow any type with a compatible

[jira] [Resolved] (BEAM-6240) Allow users to annotate POJOs and JavaBeans for richer functionality

[ https://issues.apache.org/jira/browse/BEAM-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuven Lax resolved BEAM-6240. -- Resolution: Fixed Fix Version/s: 2.12.0 > Allow users to annotate POJOs and JavaBeans for richer

[jira] [Created] (BEAM-7301) Beam transforms reorder fields

Reuven Lax created BEAM-7301: Summary: Beam transforms reorder fields Key: BEAM-7301 URL: https://issues.apache.org/jira/browse/BEAM-7301 Project: Beam Issue Type: Sub-task Components:

[jira] [Commented] (BEAM-7247) DoFnSignatures analysis of @ProcessElement with @Element fails when Java SDK is used from Kotlin

[ https://issues.apache.org/jira/browse/BEAM-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16836556#comment-16836556 ] Reuven Lax commented on BEAM-7247: -- I think the issue might be that our reflection analysis looks for an

[jira] [Created] (BEAM-7246) Create a Spanner IO for Python

Reuven Lax created BEAM-7246: Summary: Create a Spanner IO for Python Key: BEAM-7246 URL: https://issues.apache.org/jira/browse/BEAM-7246 Project: Beam Issue Type: Bug Components:

[jira] [Assigned] (BEAM-2535) Allow explicit output time independent of firing specification for all timers

[ https://issues.apache.org/jira/browse/BEAM-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuven Lax reassigned BEAM-2535: Assignee: Shehzaad Nakhoda (was: Batkhuyag Batsaikhan) > Allow explicit output time independent

[jira] [Commented] (BEAM-7210) Test ParDoSchemaTest#testInferredSchemaPipeline is flaky

[ https://issues.apache.org/jira/browse/BEAM-7210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831829#comment-16831829 ] Reuven Lax commented on BEAM-7210: -- I see the problem. I changed things from POJO to AutoValue. Since

[jira] [Created] (BEAM-7189) Remove FieldType metadata

Reuven Lax created BEAM-7189: Summary: Remove FieldType metadata Key: BEAM-7189 URL: https://issues.apache.org/jira/browse/BEAM-7189 Project: Beam Issue Type: Sub-task Components:

[jira] [Created] (BEAM-7174) Add transforms for modifying schemas

Reuven Lax created BEAM-7174: Summary: Add transforms for modifying schemas Key: BEAM-7174 URL: https://issues.apache.org/jira/browse/BEAM-7174 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-7168) Convert should support more boxing and unboxing

Reuven Lax created BEAM-7168: Summary: Convert should support more boxing and unboxing Key: BEAM-7168 URL: https://issues.apache.org/jira/browse/BEAM-7168 Project: Beam Issue Type: Sub-task

[jira] [Commented] (BEAM-7002) [beam_PostCommit_Java_Nexmark_Dataflow] [Nexmark] NullPointerException in Main

[ https://issues.apache.org/jira/browse/BEAM-7002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824560#comment-16824560 ] Reuven Lax commented on BEAM-7002: -- Currently they are only used as an optimization (to short-circuit

[jira] [Commented] (BEAM-7002) [beam_PostCommit_Java_Nexmark_Dataflow] [Nexmark] NullPointerException in Main

[ https://issues.apache.org/jira/browse/BEAM-7002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819565#comment-16819565 ] Reuven Lax commented on BEAM-7002: -- [~apilloud] fix is out for review. pr/8325 >

[jira] [Commented] (BEAM-7002) [beam_PostCommit_Java_Nexmark_Dataflow] [Nexmark] NullPointerException in Main

[ https://issues.apache.org/jira/browse/BEAM-7002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819533#comment-16819533 ] Reuven Lax commented on BEAM-7002: -- It appears the problem is this line: RunnerApi.Schema.Builder

[jira] [Commented] (BEAM-7042) beam-sdks-java-core leaks antlr4 dependency

[ https://issues.apache.org/jira/browse/BEAM-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814926#comment-16814926 ] Reuven Lax commented on BEAM-7042: -- All tests appear to pass. However we probably should try and vendor

[jira] [Commented] (BEAM-7042) beam-sdks-java-core leaks antlr4 dependency

[ https://issues.apache.org/jira/browse/BEAM-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814548#comment-16814548 ] Reuven Lax commented on BEAM-7042: -- It was added because some things were breaking without it. I don't

[jira] [Commented] (BEAM-7042) [regression] org.apache.beam:beam-sdks-java-core:jar:2.12.0 dependencies are way fatter and conflicting than before

[ https://issues.apache.org/jira/browse/BEAM-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814115#comment-16814115 ] Reuven Lax commented on BEAM-7042: -- I'm trying to figure this out. The last PR that changed the antlr

[jira] [Commented] (BEAM-7042) [regression] org.apache.beam:beam-sdks-java-core:jar:2.12.0 dependencies are way fatter and conflicting than before

[ https://issues.apache.org/jira/browse/BEAM-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813875#comment-16813875 ] Reuven Lax commented on BEAM-7042: -- Given that Antlr has been part of core since 2.10, I don't think this

[jira] [Commented] (BEAM-7042) [regression] org.apache.beam:beam-sdks-java-core:jar:2.12.0 dependencies are way fatter and conflicting than before

[ https://issues.apache.org/jira/browse/BEAM-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813669#comment-16813669 ] Reuven Lax commented on BEAM-7042: -- question for [~romain.manni-bucau]. The Antlr dependency went in in

[jira] [Commented] (BEAM-7042) [regression] org.apache.beam:beam-sdks-java-core:jar:2.12.0 dependencies are way fatter and conflicting than before

[ https://issues.apache.org/jira/browse/BEAM-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813662#comment-16813662 ] Reuven Lax commented on BEAM-7042: -- The current design of schemas is for them to be a type system for

[jira] [Commented] (BEAM-7042) [regression] org.apache.beam:beam-sdks-java-core:jar:2.12.0 dependencies are way fatter and conflicting than before

[ https://issues.apache.org/jira/browse/BEAM-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813547#comment-16813547 ] Reuven Lax commented on BEAM-7042: -- though I'm a bit confused as to why antlr depends on jsonp at all :P

[jira] [Commented] (BEAM-7042) [regression] org.apache.beam:beam-sdks-java-core:jar:2.12.0 dependencies are way fatter and conflicting than before

[ https://issues.apache.org/jira/browse/BEAM-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813541#comment-16813541 ] Reuven Lax commented on BEAM-7042: -- Yes, I do believe it needs to be in core. Schema select-statement

[jira] [Commented] (BEAM-6753) Create proto representation for schemas

[ https://issues.apache.org/jira/browse/BEAM-6753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812109#comment-16812109 ] Reuven Lax commented on BEAM-6753: -- Right now I'm only using this proto in the Dataflow runner. Before

[jira] [Created] (BEAM-6953) BigQueryIO has constants that should be PipelineOptions

Reuven Lax created BEAM-6953: Summary: BigQueryIO has constants that should be PipelineOptions Key: BEAM-6953 URL: https://issues.apache.org/jira/browse/BEAM-6953 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-6756) Support lazy iterables in schemas

[ https://issues.apache.org/jira/browse/BEAM-6756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16796558#comment-16796558 ] Reuven Lax commented on BEAM-6756: -- this would require adding a new ITERABLE schema type (would not

[jira] [Created] (BEAM-6858) Support side inputs injected into a DoFn

Reuven Lax created BEAM-6858: Summary: Support side inputs injected into a DoFn Key: BEAM-6858 URL: https://issues.apache.org/jira/browse/BEAM-6858 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-6857) Support dynamic timers

Reuven Lax created BEAM-6857: Summary: Support dynamic timers Key: BEAM-6857 URL: https://issues.apache.org/jira/browse/BEAM-6857 Project: Beam Issue Type: Bug Components:

[jira] [Commented] (BEAM-6856) Support dynamic MapState on Dataflow

[ https://issues.apache.org/jira/browse/BEAM-6856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16795509#comment-16795509 ] Reuven Lax commented on BEAM-6856: -- Beam's state API supports MapState, a way of storing a tag-value map

[jira] [Created] (BEAM-6856) Support dynamic MapState on Dataflow

Reuven Lax created BEAM-6856: Summary: Support dynamic MapState on Dataflow Key: BEAM-6856 URL: https://issues.apache.org/jira/browse/BEAM-6856 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-6855) Side inputs are not supported when using the state API

Reuven Lax created BEAM-6855: Summary: Side inputs are not supported when using the state API Key: BEAM-6855 URL: https://issues.apache.org/jira/browse/BEAM-6855 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-5645) RowCoder not equal after serialization

[ https://issues.apache.org/jira/browse/BEAM-5645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788257#comment-16788257 ] Reuven Lax commented on BEAM-5645: -- Do we simply need to add an equals() override? On Fri, Mar 8, 2019

[jira] [Created] (BEAM-6772) Select transform has non-intuitive semantics

Reuven Lax created BEAM-6772: Summary: Select transform has non-intuitive semantics Key: BEAM-6772 URL: https://issues.apache.org/jira/browse/BEAM-6772 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-6756) Support lazy iterables in schemas

Reuven Lax created BEAM-6756: Summary: Support lazy iterables in schemas Key: BEAM-6756 URL: https://issues.apache.org/jira/browse/BEAM-6756 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-6753) Create proto representation for schemas

Reuven Lax created BEAM-6753: Summary: Create proto representation for schemas Key: BEAM-6753 URL: https://issues.apache.org/jira/browse/BEAM-6753 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-6701) Create LogicalType for Schema fields

Reuven Lax created BEAM-6701: Summary: Create LogicalType for Schema fields Key: BEAM-6701 URL: https://issues.apache.org/jira/browse/BEAM-6701 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-6700) isSubType isSuperType methods do not belong in Schema.FieldType

Reuven Lax created BEAM-6700: Summary: isSubType isSuperType methods do not belong in Schema.FieldType Key: BEAM-6700 URL: https://issues.apache.org/jira/browse/BEAM-6700 Project: Beam Issue

[jira] [Created] (BEAM-6675) The JdbcIO sink should accept schemas

Reuven Lax created BEAM-6675: Summary: The JdbcIO sink should accept schemas Key: BEAM-6675 URL: https://issues.apache.org/jira/browse/BEAM-6675 Project: Beam Issue Type: Sub-task

  1   2   >