[jira] [Commented] (BEAM-11721) Cannot read array values with ParquetIO

2021-02-01 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17276435#comment-17276435 ] Ismaël Mejía commented on BEAM-11721: -  [~sekiforever] you can follow BEAM-11650

[jira] [Commented] (BEAM-11721) Cannot read array values with ParquetIO

2021-02-01 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17276538#comment-17276538 ] Ismaël Mejía commented on BEAM-11721: - Yes that is in my future to do but I am g

[jira] [Updated] (BEAM-7637) Migrate S3FileSystem to AWS SDK for Java 2

2021-02-10 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-7637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-7637: --- Fix Version/s: 2.29.0 > Migrate S3FileSystem to AWS SDK for Jav

[jira] [Work started] (BEAM-7637) Migrate S3FileSystem to AWS SDK for Java 2

2021-02-10 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-7637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on BEAM-7637 started by Sanamyan Raphael. -- > Migrate S3FileSystem to AWS SDK for Jav

[jira] [Updated] (BEAM-11735) Logging from DoFn doesn't work with Spark Runner in cluster mode

2021-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11735: Component/s: (was: sdk-java-core) > Logging from DoFn doesn't work with Spark R

[jira] [Commented] (BEAM-11815) fail to read more than 1M of items

2021-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17302334#comment-17302334 ] Ismaël Mejía commented on BEAM-11815: - We need to dig deeper into why this is

[jira] [Updated] (BEAM-11815) fail to read more than 1M of items

2021-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11815: Fix Version/s: (was: 2.29.0) > fail to read more than 1M of it

[jira] [Updated] (BEAM-11861) ParquetIO throws Coder not found when using parseGenericRecord or parseFilesGenericRecord

2021-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11861: Resolution: Fixed Status: Resolved (was: In Progress) > ParquetIO throws Coder not fo

[jira] [Commented] (BEAM-11227) Upgrade beam-vendor-grpc-1_26_0-0.3 to fix CVE-2020-27216

2021-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17302338#comment-17302338 ] Ismaël Mejía commented on BEAM-11227: - I don't think this issue should be

[jira] [Assigned] (BEAM-11323) Date type mapping from AVRO to Beam Schema produces invalid schema when used with Beam SQL

2021-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía reassigned BEAM-11323: --- Assignee: Steven van Rossum > Date type mapping from AVRO to Beam Schema produces inva

[jira] [Updated] (BEAM-11323) Date type mapping from AVRO to Beam Schema produces invalid schema when used with Beam SQL

2021-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11323: Status: Open (was: Triage Needed) > Date type mapping from AVRO to Beam Schema produces inva

[jira] [Assigned] (BEAM-11927) Parameter should be used instead of argument in the Annotation Type DoFn.ProcessElement documentation

2021-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía reassigned BEAM-11927: --- Assignee: (was: Aizhamal Nurmamat kyzy) > Parameter should be used instead of argum

[jira] [Updated] (BEAM-11313) FileIO azfs Stream mark expired

2021-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11313: Status: Open (was: Triage Needed) > FileIO azfs Stream mark expi

[jira] [Assigned] (BEAM-8717) Beam Dependency Update Request: org.apache.commons:commons-lang3

2021-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-8717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía reassigned BEAM-8717: -- Assignee: (was: Ismaël Mejía) > Beam Dependency Update Request: org.apache.commons:comm

[jira] [Assigned] (BEAM-7929) ParquetTable.buildIOReader should support column projection and filter predicate

2021-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-7929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía reassigned BEAM-7929: -- Assignee: Ismaël Mejía > ParquetTable.buildIOReader should support column projection

[jira] [Updated] (BEAM-7929) ParquetTable.buildIOReader should support column projection and filter predicate

2021-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-7929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-7929: --- Fix Version/s: 2.30.0 > ParquetTable.buildIOReader should support column projection and fil

[jira] [Updated] (BEAM-11055) Update log4j to version 2.14.1

2021-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11055: Summary: Update log4j to version 2.14.1 (was: Update log4j to version 2.14.0) > Update log4j

[jira] [Assigned] (BEAM-11905) GCP DataFlow not cleaning up GCP BigQuery temporary datasets

2021-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía reassigned BEAM-11905: --- Assignee: (was: Aizhamal Nurmamat kyzy) > GCP DataFlow not cleaning up GCP BigQu

[jira] [Commented] (BEAM-11992) beam_PostCommit_XVR_Spark is perma red due to "ConnectException"

2021-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17302733#comment-17302733 ] Ismaël Mejía commented on BEAM-11992: - We merged support for both Spark 2 and S

[jira] [Updated] (BEAM-11992) beam_PostCommit_XVR_Spark is perma red due to "ConnectException"

2021-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11992: Status: Open (was: Triage Needed) > beam_PostCommit_XVR_Spark is perma red due

[jira] [Commented] (BEAM-7093) Support Spark 3 in Spark runner

2021-03-17 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-7093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17303623#comment-17303623 ] Ismaël Mejía commented on BEAM-7093: Thanks [~tysonjh] I expect to get the web

[jira] [Assigned] (BEAM-2530) Make Beam compatible with next Java LTS version (Java 11)

2021-03-18 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-2530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía reassigned BEAM-2530: -- Assignee: Ismaël Mejía > Make Beam compatible with next Java LTS version (Java

[jira] [Commented] (BEAM-2530) Make Beam compatible with next Java LTS version (Java 11)

2021-03-18 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-2530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304051#comment-17304051 ] Ismaël Mejía commented on BEAM-2530: With the recent addition of official support

[jira] [Updated] (BEAM-11023) GroupByKeyTest testLargeKeys100MB and testGroupByKeyWithBadEqualsHashCode are failing on Spark Structured Streaming runner

2021-03-18 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11023: Fix Version/s: 2.30.0 > GroupByKeyTest testLargeKeys100MB and testGroupByKeyWithBadEqualsHashC

[jira] [Updated] (BEAM-11023) GroupByKeyTest testLargeKeys100MB and testGroupByKeyWithBadEqualsHashCode are failing on Spark Structured Streaming runner

2021-03-18 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11023: Resolution: Fixed Status: Resolved (was: Open) > GroupByKeyTest testLargeKeys100MB

[jira] [Commented] (BEAM-11979) Can't use ReadFromMongoDB with a datetime in filter

2021-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304961#comment-17304961 ] Gaël commented on BEAM-11979: - I saw you merged the fix on master. Can I have an estim

[jira] [Updated] (BEAM-11992) beam_PostCommit_XVR_Spark is perma red due to "ConnectException"

2021-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11992: Resolution: Fixed Status: Resolved (was: Open) > beam_PostCommit_XVR_Spark is perma

[jira] [Updated] (BEAM-11992) beam_PostCommit_XVR_Spark is perma red due to "ConnectException"

2021-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11992: Fix Version/s: Not applicable > beam_PostCommit_XVR_Spark is perma red due to "ConnectE

[jira] [Updated] (BEAM-9239) Dependency conflict with Spark using aws io

2021-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-9239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-9239: --- Fix Version/s: 2.29.0 Resolution: Fixed Status: Resolved (was: Open) > Depende

[jira] [Commented] (BEAM-9239) Dependency conflict with Spark using aws io

2021-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-9239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305195#comment-17305195 ] Ismaël Mejía commented on BEAM-9239: I am really sorry we did not pay attentio

[jira] [Commented] (BEAM-11620) PeriodicImpulse uses unspecified / operator on Durations

2021-03-20 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305388#comment-17305388 ] Mátyás Manninger commented on BEAM-11620: - [~whitewg77] I will take a look

[jira] [Commented] (BEAM-11862) Write To Kafka does not work

2021-03-21 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305928#comment-17305928 ] Dénes Bartha commented on BEAM-11862: - Hi [~chamikara], Thanks for your comment

[jira] [Commented] (BEAM-11862) Write To Kafka does not work

2021-03-23 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17306840#comment-17306840 ] Dénes Bartha commented on BEAM-11862: - Btw I am also having similar issues

[jira] [Created] (BEAM-12032) Spark warns on SparkBeamMetric already registered

2021-03-23 Thread Jira
Ismaël Mejía created BEAM-12032: --- Summary: Spark warns on SparkBeamMetric already registered Key: BEAM-12032 URL: https://issues.apache.org/jira/browse/BEAM-12032 Project: Beam Issue Type

[jira] [Updated] (BEAM-12032) Spark warns on SparkBeamMetric already registered

2021-03-23 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-12032: Description: {{Spark runner in some environments (Spark 3 and Databricks) reports a warning on

[jira] [Created] (BEAM-12044) JdbcIO should explicitly setAutoCommit to false

2021-03-24 Thread Jira
Sylvain Veyrié created BEAM-12044: - Summary: JdbcIO should explicitly setAutoCommit to false Key: BEAM-12044 URL: https://issues.apache.org/jira/browse/BEAM-12044 Project: Beam Issue Type

[jira] [Updated] (BEAM-12044) JdbcIO should explicitly setAutoCommit to false

2021-03-24 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Veyrié updated BEAM-12044: -- Description: Hello, Per [PostgreSQL JDBC documentation|https://jdbc.postgresql.org

[jira] [Commented] (BEAM-12044) JdbcIO should explicitly setAutoCommit to false

2021-03-24 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17307970#comment-17307970 ] Sylvain Veyrié commented on BEAM-12044: --- [~jkff] it looks like it. I must adm

[jira] [Comment Edited] (BEAM-12044) JdbcIO should explicitly setAutoCommit to false

2021-03-24 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17307970#comment-17307970 ] Sylvain Veyrié edited comment on BEAM-12044 at 3/24/21, 4:0

[jira] [Commented] (BEAM-12044) JdbcIO should explicitly setAutoCommit to false

2021-03-25 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17308552#comment-17308552 ] Sylvain Veyrié commented on BEAM-12044: --- Created https://github.com/apache/

[jira] [Updated] (BEAM-11815) Fail to read more than 1M of items

2021-03-25 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11815: Fix Version/s: 2.30.0 > Fail to read more than 1M of it

[jira] [Updated] (BEAM-11815) Fail to read more than 1M of items

2021-03-25 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11815: Resolution: Fixed Status: Resolved (was: In Progress) > Fail to read more than 1M

[jira] [Created] (BEAM-12053) FlinkRuner: Pipeline using KafkaIO seems not be able to terminate

2021-03-25 Thread Jira
Jan Lukavský created BEAM-12053: --- Summary: FlinkRuner: Pipeline using KafkaIO seems not be able to terminate Key: BEAM-12053 URL: https://issues.apache.org/jira/browse/BEAM-12053 Project: Beam

[jira] [Updated] (BEAM-12053) FlinkRuner: Pipeline using KafkaIO seems not be able to terminate

2021-03-25 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lukavský updated BEAM-12053: Description: {{2021-03-25 14:21:27,210 WARN org.apache.beam.sdk.io.kafka.KafkaUnboundedReader

[jira] [Updated] (BEAM-12053) FlinkRuner: Pipeline using KafkaIO seems not be able to terminate

2021-03-25 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lukavský updated BEAM-12053: Description: 2021-03-25 14:21:27,210 WARN org.apache.beam.sdk.io.kafka.KafkaUnboundedReader

[jira] [Created] (BEAM-12057) Add missing populateDisplayData methods to ParquetIO

2021-03-26 Thread Jira
Ismaël Mejía created BEAM-12057: --- Summary: Add missing populateDisplayData methods to ParquetIO Key: BEAM-12057 URL: https://issues.apache.org/jira/browse/BEAM-12057 Project: Beam Issue Type

[jira] [Created] (BEAM-12058) Add withMatchConfiguration to ParquetIO

2021-03-26 Thread Jira
Ismaël Mejía created BEAM-12058: --- Summary: Add withMatchConfiguration to ParquetIO Key: BEAM-12058 URL: https://issues.apache.org/jira/browse/BEAM-12058 Project: Beam Issue Type: Improvement

[jira] [Updated] (BEAM-12058) Add withMatchConfiguration + watchForNewFiles to ParquetIO

2021-03-26 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-12058: Summary: Add withMatchConfiguration + watchForNewFiles to ParquetIO (was: Add

[jira] [Updated] (BEAM-12054) Mutator.close() has to be moved to @FinishBundle in WriteFn and DeleteFn

2021-03-26 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-12054: Summary: Mutator.close() has to be moved to @FinishBundle in WriteFn and DeleteFn (was

[jira] [Commented] (BEAM-12044) JdbcIO should explicitly setAutoCommit to false

2021-03-26 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17309566#comment-17309566 ] Sylvain Veyrié commented on BEAM-12044: --- Updated PR: https://github.com/ap

[jira] [Commented] (BEAM-8212) StatefulDoFnRunner creates GC timers for every record

2021-03-29 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-8212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17310482#comment-17310482 ] Jan Lukavský commented on BEAM-8212: This looks related to BEAM-10760 (which

[jira] [Created] (BEAM-12070) Make ParquetIO splittable by default

2021-03-30 Thread Jira
Ismaël Mejía created BEAM-12070: --- Summary: Make ParquetIO splittable by default Key: BEAM-12070 URL: https://issues.apache.org/jira/browse/BEAM-12070 Project: Beam Issue Type: Improvement

[jira] [Updated] (BEAM-11650) Allow ParquetIO.read/parse without explicit schema

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11650: Status: Open (was: Triage Needed) > Allow ParquetIO.read/parse without explicit sch

[jira] [Assigned] (BEAM-11650) Allow ParquetIO.read/parse without explicit schema

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía reassigned BEAM-11650: --- Assignee: Ismaël Mejía > Allow ParquetIO.read/parse without explicit sch

[jira] [Updated] (BEAM-11911) Support dynamic class registration for Convert transform

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11911: Status: Open (was: Triage Needed) > Support dynamic class registration for Convert transf

[jira] [Updated] (BEAM-11703) Support apache-beam python install on ARM64

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11703: Status: Open (was: Triage Needed) > Support apache-beam python install on AR

[jira] [Updated] (BEAM-11704) Support Beam docker images on ARM64

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11704: Status: Open (was: Triage Needed) > Support Beam docker images on AR

[jira] [Updated] (BEAM-11030) Nexmark Query 3 produces inconsistent output size on Direct runner

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11030: Priority: P2 (was: P3) > Nexmark Query 3 produces inconsistent output size on Direct run

[jira] [Updated] (BEAM-11030) Nexmark Query 3 produces inconsistent output size on Direct runner

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11030: Status: Open (was: Triage Needed) > Nexmark Query 3 produces inconsistent output size on Dir

[jira] [Updated] (BEAM-2530) Make Beam compatible with next Java LTS version (Java 11)

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-2530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-2530: --- Fix Version/s: (was: Not applicable) > Make Beam compatible with next Java LTS version (Java

[jira] [Updated] (BEAM-2530) Make Beam compatible with next Java LTS version (Java 11)

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-2530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-2530: --- Fix Version/s: 2.29.0 > Make Beam compatible with next Java LTS version (Java

[jira] [Updated] (BEAM-2530) Make Beam compatible with next Java LTS version (Java 11)

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-2530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-2530: --- Resolution: Fixed Status: Resolved (was: Open) Finally ! Such a pleasure to resolve this

[jira] [Updated] (BEAM-11815) Fail to read more than 1M of items with DynamoDBIO

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11815: Summary: Fail to read more than 1M of items with DynamoDBIO (was: Fail to read more than 1M of

[jira] [Updated] (BEAM-11815) Fail to read more than 1M of items with DynamoDB

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11815: Summary: Fail to read more than 1M of items with DynamoDB (was: Fail to read more than 1M of

[jira] [Assigned] (BEAM-7093) Support Spark 3 in Spark runner

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-7093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía reassigned BEAM-7093: -- Assignee: Ismaël Mejía > Support Spark 3 in Spark run

[jira] [Updated] (BEAM-7093) Support Spark 3 in Spark runner

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-7093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-7093: --- Fix Version/s: (was: 2.30.0) 2.29.0 > Support Spark 3 in Spark run

[jira] [Updated] (BEAM-7093) Support Spark 3 in Spark runner

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-7093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-7093: --- Resolution: Fixed Status: Resolved (was: Open) > Support Spark 3 in Spark run

[jira] [Updated] (BEAM-11655) Update website with Spark 3 runner specific information

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11655: Fix Version/s: (was: 2.30.0) Not applicable > Update website with Spar

[jira] [Commented] (BEAM-11995) Implement FileIO/TextIO on top of Splittable DoFn

2021-03-30 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17311813#comment-17311813 ] Ismaël Mejía commented on BEAM-11995: - Interesting! [~boyuanz] I have a ques

[jira] [Commented] (BEAM-11671) Spark PortableRunner (Python SDK) low parallelism

2021-03-31 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312683#comment-17312683 ] Ismaël Mejía commented on BEAM-11671: - Just to clarify our current performance t

[jira] [Assigned] (BEAM-12081) AwsOptions protobuf (de)serialization bug

2021-04-01 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía reassigned BEAM-12081: --- Assignee: Bulat Shakirzyanov > AwsOptions protobuf (de)serialization

[jira] [Updated] (BEAM-12081) Fix AwsOptions Jackson (de)serialization of integer values

2021-04-01 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-12081: Summary: Fix AwsOptions Jackson (de)serialization of integer values (was: AwsOptions protobuf (de

[jira] [Updated] (BEAM-12081) Fix AwsOptions Jackson (de)serialization of integer values

2021-04-01 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-12081: Description: Minimal test case to reproduce: {code:java} AwsOptions awsOptions

[jira] [Updated] (BEAM-11213) Beam metrics should be displayed in Spark UI

2021-04-02 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11213: Fix Version/s: 2.29.0 > Beam metrics should be displayed in Spark

[jira] [Commented] (BEAM-11213) Beam metrics should be displayed in Spark UI

2021-04-02 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17313850#comment-17313850 ] Ismaël Mejía commented on BEAM-11213: - This is a new feature but it is introdu

[jira] [Updated] (BEAM-12088) Make file staging uniform among Spark Runners

2021-04-02 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-12088: Fix Version/s: 2.30.0 > Make file staging uniform among Spark Runn

[jira] [Created] (BEAM-12088) Make file staging uniform among Spark Runners

2021-04-02 Thread Jira
Ismaël Mejía created BEAM-12088: --- Summary: Make file staging uniform among Spark Runners Key: BEAM-12088 URL: https://issues.apache.org/jira/browse/BEAM-12088 Project: Beam Issue Type: Bug

[jira] [Comment Edited] (BEAM-11213) Beam metrics should be displayed in Spark UI

2021-04-02 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17313850#comment-17313850 ] Ismaël Mejía edited comment on BEAM-11213 at 4/2/21, 8:5

[jira] [Updated] (BEAM-12088) Make file staging uniform among Spark Runners

2021-04-02 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-12088: Description: Both the Spark Classic and Portable runners share the file staging logic, but the

[jira] [Assigned] (BEAM-4106) Merge staging file options between runners

2021-04-03 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía reassigned BEAM-4106: -- Assignee: Ismaël Mejía > Merge staging file options between runn

[jira] [Updated] (BEAM-4106) Merge staging file options between runners

2021-04-03 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-4106: --- Fix Version/s: 2.30.0 > Merge staging file options between runn

[jira] [Updated] (BEAM-4106) Merge staging file options between runners

2021-04-03 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-4106: --- Summary: Merge staging file options between runners (was: Merge staging file options between

[jira] [Created] (BEAM-12091) Make file staging uniform among runners

2021-04-03 Thread Jira
Ismaël Mejía created BEAM-12091: --- Summary: Make file staging uniform among runners Key: BEAM-12091 URL: https://issues.apache.org/jira/browse/BEAM-12091 Project: Beam Issue Type: Improvement

[jira] [Updated] (BEAM-12088) Make file staging uniform among Spark Runners

2021-04-03 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-12088: Resolution: Fixed Status: Resolved (was: Open) > Make file staging uniform among Sp

[jira] [Commented] (BEAM-12088) Make file staging uniform among Spark Runners

2021-04-03 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314256#comment-17314256 ] Ismaël Mejía commented on BEAM-12088: - [~ibzib] I opened BEAM-12091 as a follo

[jira] [Updated] (BEAM-12091) Make file staging uniform among runners

2021-04-03 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-12091: Fix Version/s: 2.30.0 > Make file staging uniform among runn

[jira] [Updated] (BEAM-4106) Merge staging file options between runners

2021-04-05 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-4106: --- Resolution: Fixed Status: Resolved (was: Open) > Merge staging file options between runn

[jira] [Work started] (BEAM-12091) Make file staging uniform among runners

2021-04-05 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on BEAM-12091 started by Ismaël Mejía. --- > Make file staging uniform among runn

[jira] [Commented] (BEAM-10083) Spark Runner Tests failing [Java 11]

2021-04-05 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-10083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315227#comment-17315227 ] Ismaël Mejía commented on BEAM-10083: - Yes, thanks for commenting I was not awar

[jira] [Commented] (BEAM-10083) Spark Runner Tests failing [Java 11]

2021-04-05 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-10083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315241#comment-17315241 ] Ismaël Mejía commented on BEAM-10083: - I have misread this issue. Now I realiz

[jira] [Commented] (BEAM-11398) #1 CoGBK Go load test is too slow

2021-04-06 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315363#comment-17315363 ] Ismaël Mejía commented on BEAM-11398: - Maybe related to BEAM-6110 'CoGbkR

[jira] [Commented] (BEAM-11735) Logging from DoFn doesn't work with Spark Runner in cluster mode

2021-04-06 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315457#comment-17315457 ] Ismaël Mejía commented on BEAM-11735: - I tested this on EMR and it works. You ma

[jira] [Commented] (BEAM-11735) Logging from DoFn doesn't work with Spark Runner in cluster mode

2021-04-06 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315458#comment-17315458 ] Ismaël Mejía commented on BEAM-11735: - Can you confirm if it works for

[jira] [Commented] (BEAM-11788) TextIO.read() will ignore custom AwsServiceEndpoint

2021-04-06 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315461#comment-17315461 ] Ismaël Mejía commented on BEAM-11788: - This should work, the current aws test

[jira] [Updated] (BEAM-11948) Drop support for Flink 1.8 and 1.9

2021-04-07 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-11948: Resolution: Fixed Status: Resolved (was: Open) > Drop support for Flink 1.8 and

[jira] [Created] (BEAM-12134) Add Table statistics / Row estimation for ParquetTable

2021-04-08 Thread Jira
Ismaël Mejía created BEAM-12134: --- Summary: Add Table statistics / Row estimation for ParquetTable Key: BEAM-12134 URL: https://issues.apache.org/jira/browse/BEAM-12134 Project: Beam Issue Type

[jira] [Updated] (BEAM-12134) Add Table statistics / Row estimation for ParquetTable

2021-04-08 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-12134: Status: Open (was: Triage Needed) > Add Table statistics / Row estimation for ParquetTa

[jira] [Updated] (BEAM-12134) Add Table statistics / Row estimation for ParquetTable

2021-04-08 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-12134: Description: Table statistics and Row estimation are used by the SQL implementation to optimize

[jira] [Created] (BEAM-12135) Batch optimized translation for Spark Runner

2021-04-08 Thread Jira
Ismaël Mejía created BEAM-12135: --- Summary: Batch optimized translation for Spark Runner Key: BEAM-12135 URL: https://issues.apache.org/jira/browse/BEAM-12135 Project: Beam Issue Type

[jira] [Updated] (BEAM-12135) Batch optimized translation for Spark Runner

2021-04-08 Thread Jira
[ https://issues.apache.org/jira/browse/BEAM-12135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-12135: Status: Open (was: Triage Needed) > Batch optimized translation for Spark Run

<    2   3   4   5   6   7   8   9   10   11   >