[jira] [Updated] (BEAM-13499) beam-sdks-java-io-hcatalog and beam-sdks-java-extensions-sql-hcatalog are vulnerable to CVE-2021-44228

2022-03-24 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13499: - Affects Version/s: 2.38.0 > beam-sdks-java-io-hcatalog and beam-sdks-java-extensions-sql-hcatalog

[jira] [Commented] (BEAM-13499) beam-sdks-java-io-hcatalog and beam-sdks-java-extensions-sql-hcatalog are vulnerable to CVE-2021-44228

2022-03-24 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511923#comment-17511923 ] Brian Hulette commented on BEAM-13499: -- 2.3.10 is still not available as of Beam 2.3

[jira] [Commented] (BEAM-13905) Apache Beam Python: Dataframe Transforms break when the option runtime_type_check is enabled.

2022-03-24 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511922#comment-17511922 ] Brian Hulette commented on BEAM-13905: -- Sorry I didn't get to this for 2.38.0. I'm g

[jira] [Updated] (BEAM-13905) Apache Beam Python: Dataframe Transforms break when the option runtime_type_check is enabled.

2022-03-24 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13905: - Affects Version/s: 2.38.0 > Apache Beam Python: Dataframe Transforms break when the option > run

[jira] [Updated] (BEAM-13905) Apache Beam Python: Dataframe Transforms break when the option runtime_type_check is enabled.

2022-03-24 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13905: - Fix Version/s: (was: 2.38.0) > Apache Beam Python: Dataframe Transforms break when the option

[jira] [Updated] (BEAM-13905) Apache Beam Python: Dataframe Transforms break when the option runtime_type_check is enabled.

2022-03-24 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13905: - Affects Version/s: 2.37.0 2.36.0 > Apache Beam Python: Dataframe Transform

[jira] [Commented] (BEAM-14157) Don't request work on a closed windmill GetWorkStream

2022-03-24 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511915#comment-17511915 ] Brian Hulette commented on BEAM-14157: -- Could you clarify this bug with the possible

[jira] [Updated] (BEAM-14148) ExternalPythonTransformTest.trivialPythonTransform flaky

2022-03-24 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-14148: - Status: Open (was: Triage Needed) > ExternalPythonTransformTest.trivialPythonTransform flaky > -

[jira] [Commented] (BEAM-14148) ExternalPythonTransformTest.trivialPythonTransform flaky

2022-03-24 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511911#comment-17511911 ] Brian Hulette commented on BEAM-14148: -- Is the action here to diagnose the flake and

[jira] [Commented] (BEAM-14162) DataFrame groupby does not support Grouper objects

2022-03-23 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511488#comment-17511488 ] Brian Hulette commented on BEAM-14162: -- Categorized this as a bug for now since I'm

[jira] [Updated] (BEAM-14162) DataFrame groupby does not support Grouper objects

2022-03-23 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-14162: - Status: Open (was: Triage Needed) > DataFrame groupby does not support Grouper objects > ---

[jira] [Updated] (BEAM-14107) DataFrame API: Support deferred GroupBy.__len__ and GroupBy.ngroups

2022-03-23 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-14107: - Description: We should be able to build a deferred operation that calculates the number of group

[jira] [Updated] (BEAM-14107) DataFrame API: Support deferred GroupBy.__len__ and GroupBy.ngroups

2022-03-23 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-14107: - Summary: DataFrame API: Support deferred GroupBy.__len__ and GroupBy.ngroups (was: DataFrame API

[jira] [Updated] (BEAM-14107) DataFrame API: Support deferred GroupBy.__len__ and GroupBy.ngroups

2022-03-23 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-14107: - Description: We should be able to build a deferred operation that calculates the number of groups

[jira] [Updated] (BEAM-14107) DataFrame API: Refactor GroupBy

2022-03-23 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-14107: - Description: * len(GroupBy) and ngroups ** if groupers are all categorical _and_ observed=Fals

[jira] [Commented] (BEAM-14107) DataFrame API: Refactor GroupBy

2022-03-23 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511485#comment-17511485 ] Brian Hulette commented on BEAM-14107: -- There are really two separate issues - I bro

[jira] [Created] (BEAM-14162) DataFrame groupby does not support Grouper objects

2022-03-23 Thread Brian Hulette (Jira)
Brian Hulette created BEAM-14162: Summary: DataFrame groupby does not support Grouper objects Key: BEAM-14162 URL: https://issues.apache.org/jira/browse/BEAM-14162 Project: Beam Issue Type: B

[jira] [Commented] (BEAM-12356) BigQueryWriteClient in DatasetServiceImpl is not closed, which causes "ManagedChannel allocation site" exceptions

2022-03-22 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-12356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17510716#comment-17510716 ] Brian Hulette commented on BEAM-12356: -- [~bsikander] [~joachim_isaksson] [~clairemcg

[jira] [Commented] (BEAM-7554) datetime and decimal should be logical types

2022-03-21 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17510048#comment-17510048 ] Brian Hulette commented on BEAM-7554: - Thanks! I'm happy to help out/provide context a

[jira] [Updated] (BEAM-7554) datetime and decimal should be logical types

2022-03-21 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-7554: Component/s: cross-language > datetime and decimal should be logical types > ---

[jira] [Updated] (BEAM-7554) datetime and decimal should be logical types

2022-03-21 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-7554: Priority: P2 (was: P3) > datetime and decimal should be logical types > ---

[jira] [Commented] (BEAM-7554) datetime and decimal should be logical types

2022-03-21 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17510017#comment-17510017 ] Brian Hulette commented on BEAM-7554: - Thanks [~ilango-mediaagility] that's a good poi

[jira] [Updated] (BEAM-13959) Unable to write to BigQuery tables with column named 'f'

2022-03-21 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13959: - Fix Version/s: 2.37.0 Resolution: Fixed Status: Resolved (was: Open) > Unable

[jira] [Updated] (BEAM-13955) PythonLint PreCommit is broken

2022-03-18 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13955: - Fix Version/s: Not applicable Resolution: Fixed Status: Resolved (was: Open) >

[jira] [Updated] (BEAM-13860) ParDo.infer_output_type expects DoFn.infer_output_type to return Iterable of element_type

2022-03-18 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13860: - Fix Version/s: 2.38.0 Resolution: Fixed Status: Resolved (was: Open) > ParDo.i

[jira] [Commented] (BEAM-13769) beam_PreCommit_Python_Cron failing on test_create_uses_coder_for_pickling

2022-03-15 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17507115#comment-17507115 ] Brian Hulette commented on BEAM-13769: -- Marked this as P2 since test has been sickba

[jira] [Updated] (BEAM-13769) beam_PreCommit_Python_Cron failing on test_create_uses_coder_for_pickling

2022-03-15 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13769: - Labels: sickbay (was: ) > beam_PreCommit_Python_Cron failing on test_create_uses_coder_for_pickl

[jira] [Updated] (BEAM-13769) beam_PreCommit_Python_Cron failing on test_create_uses_coder_for_pickling

2022-03-15 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13769: - Priority: P2 (was: P1) > beam_PreCommit_Python_Cron failing on test_create_uses_coder_for_pickli

[jira] [Commented] (BEAM-13905) Apache Beam Python: Dataframe Transforms break when the option runtime_type_check is enabled.

2022-03-14 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17506514#comment-17506514 ] Brian Hulette commented on BEAM-13905: -- I'll look into this soon now that 2.37.0 is

[jira] [Updated] (BEAM-13905) Apache Beam Python: Dataframe Transforms break when the option runtime_type_check is enabled.

2022-03-14 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13905: - Labels: (was: stale-assigned) > Apache Beam Python: Dataframe Transforms break when the option

[jira] [Updated] (BEAM-11751) Consider removing "Development Setup" from contribution guide

2022-03-14 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-11751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-11751: - Fix Version/s: Not applicable Assignee: Brian Hulette Resolution: Won't Fix

[jira] [Commented] (BEAM-14051) There should be single source of truth for the project version

2022-03-14 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17506386#comment-17506386 ] Brian Hulette commented on BEAM-14051: -- [~dmitryor] should we keep this open to trac

[jira] [Updated] (BEAM-14051) There should be single source of truth for the project version

2022-03-14 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-14051: - Status: Open (was: Triage Needed) > There should be single source of truth for the project versi

[jira] [Comment Edited] (BEAM-12164) SpannerIO Change Stream Connector

2022-03-08 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-12164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17503114#comment-17503114 ] Brian Hulette edited comment on BEAM-12164 at 3/8/22, 6:36 PM:

[jira] [Updated] (BEAM-12164) SpannerIO Change Stream Connector

2022-03-08 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-12164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-12164: - Resolution: Fixed Status: Resolved (was: Open) Marking this fixed, if there's still work

[jira] [Commented] (BEAM-13769) beam_PreCommit_Python_Cron failing on test_create_uses_coder_for_pickling

2022-03-04 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17501584#comment-17501584 ] Brian Hulette commented on BEAM-13769: -- Is there a jira for cloudpickle work that w

[jira] [Updated] (BEAM-14026) Using unnest on a array with rows having nested rows throws IllegalArgumentException

2022-03-04 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-14026: - Fix Version/s: 2.38.0 (was: Not applicable) Resolution: Fixed

[jira] [Updated] (BEAM-14005) Exceptions when reading from Spanner ignored

2022-03-03 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-14005: - Resolution: Fixed Status: Resolved (was: Open) > Exceptions when reading from Spanner ig

[jira] [Commented] (BEAM-12164) SpannerIO Change Stream Connector

2022-03-03 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-12164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17501104#comment-17501104 ] Brian Hulette commented on BEAM-12164: -- [~thiagotnunes] [~pabloem] can you confirm i

[jira] [Updated] (BEAM-14005) Exceptions when reading from Spanner ignored

2022-03-03 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-14005: - Status: Open (was: Triage Needed) > Exceptions when reading from Spanner ignored > -

[jira] [Updated] (BEAM-13997) Cherrypick to 2.37.0: S3 Client broken: missing get_object_metadata()

2022-03-03 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13997: - Resolution: Fixed Status: Resolved (was: Open) > Cherrypick to 2.37.0: S3 Client broken:

[jira] [Commented] (BEAM-13997) Cherrypick to 2.37.0: S3 Client broken: missing get_object_metadata()

2022-03-03 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17501103#comment-17501103 ] Brian Hulette commented on BEAM-13997: -- Cherrypicked in https://github.com/apache/be

[jira] [Commented] (BEAM-14026) Using unnest on a array with rows having nested rows throws IllegalArgumentException

2022-03-03 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17500991#comment-17500991 ] Brian Hulette commented on BEAM-14026: -- Thanks [~lostluck], we do have a dsl-sql com

[jira] [Updated] (BEAM-14026) Using unnest on a array with rows having nested rows throws IllegalArgumentException

2022-03-03 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-14026: - Component/s: dsl-sql > Using unnest on a array with rows having nested rows throws > IllegalArgu

[jira] [Updated] (BEAM-14026) Using unnest on a array with rows having nested rows throws IllegalArgumentException

2022-03-03 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-14026: - Status: Open (was: Triage Needed) > Using unnest on a array with rows having nested rows throws

[jira] [Updated] (BEAM-14017) beam_PreCommit_CommunityMetrics_Cron is failing.

2022-03-03 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-14017: - Status: Open (was: Triage Needed) > beam_PreCommit_CommunityMetrics_Cron is failing. > -

[jira] [Assigned] (BEAM-14017) beam_PreCommit_CommunityMetrics_Cron is failing.

2022-03-03 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette reassigned BEAM-14017: Assignee: Heejong Lee > beam_PreCommit_CommunityMetrics_Cron is failing. > ---

[jira] [Commented] (BEAM-13995) Apache beam is having vulnerable dependencies - Tensorflow, httplib2, pandas and numpy

2022-03-02 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17500374#comment-17500374 ] Brian Hulette commented on BEAM-13995: -- I see, sorry about that. I'm still not sure

[jira] [Updated] (BEAM-11587) Support pd.read_gbq and DataFrame.to_gbq

2022-03-01 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-11587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-11587: - Description: We should support [read_gbq|https://pandas.pydata.org/pandas-docs/stable/reference/a

[jira] [Commented] (BEAM-11587) Support pd.read_gbq and DataFrame.to_gbq

2022-03-01 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-11587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17499818#comment-17499818 ] Brian Hulette commented on BEAM-11587: -- Hi Svetak, I discussed this with [~robertwb]

[jira] [Comment Edited] (BEAM-11587) Support pd.read_gbq and DataFrame.to_gbq

2022-03-01 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-11587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17499818#comment-17499818 ] Brian Hulette edited comment on BEAM-11587 at 3/2/22, 12:20 AM: ---

[jira] [Updated] (BEAM-10708) InteractiveRunner cannot execute pipeline with cross-language transform

2022-03-01 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-10708: - Status: Open (was: Triage Needed) > InteractiveRunner cannot execute pipeline with cross-languag

[jira] [Reopened] (BEAM-10708) InteractiveRunner cannot execute pipeline with cross-language transform

2022-03-01 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette reopened BEAM-10708: -- Assignee: (was: Ning) I don't think we should close this. The beam_sql magic is a great wor

[jira] [Updated] (BEAM-10708) InteractiveRunner cannot execute pipeline with cross-language transform

2022-03-01 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-10708: - Fix Version/s: (was: Missing) > InteractiveRunner cannot execute pipeline with cross-language

[jira] [Commented] (BEAM-12181) Implement parallelized (approximate) mode

2022-02-28 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17499247#comment-17499247 ] Brian Hulette commented on BEAM-12181: -- > Could you elaborate on why it may be neces

[jira] [Updated] (BEAM-13693) beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming timing out at 9 hours

2022-02-28 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13693: - Fix Version/s: Not applicable Resolution: Fixed Status: Resolved (was: Open) >

[jira] [Commented] (BEAM-13693) beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming timing out at 9 hours

2022-02-28 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17499156#comment-17499156 ] Brian Hulette commented on BEAM-13693: -- Runtime down to ~6h30m in https://ci-beam.a

[jira] [Updated] (BEAM-14008) Java PreCommit failing at HEAD in sdks:java:fn-execution:analyzeClassesDepdencies

2022-02-28 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-14008: - Fix Version/s: Not applicable Resolution: Fixed Status: Resolved (was: Open) >

[jira] [Updated] (BEAM-14008) Java PreCommit failing at HEAD in sdks:java:fn-execution:analyzeClassesDepdencies

2022-02-28 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-14008: - Priority: P1 (was: P2) > Java PreCommit failing at HEAD in > sdks:java:fn-execution:analyzeClas

[jira] [Updated] (BEAM-14008) Java PreCommit failing at HEAD in sdks:java:fn-execution:analyzeClassesDepdencies

2022-02-28 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-14008: - Status: Open (was: Triage Needed) > Java PreCommit failing at HEAD in > sdks:java:fn-execution:

[jira] [Commented] (BEAM-14008) Java PreCommit failing at HEAD in sdks:java:fn-execution:analyzeClassesDepdencies

2022-02-28 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17499078#comment-17499078 ] Brian Hulette commented on BEAM-14008: -- First failure: https://ci-beam.apache.org/jo

[jira] [Created] (BEAM-14008) Java PreCommit failing at HEAD in sdks:java:fn-execution:analyzeClassesDepdencies

2022-02-28 Thread Brian Hulette (Jira)
Brian Hulette created BEAM-14008: Summary: Java PreCommit failing at HEAD in sdks:java:fn-execution:analyzeClassesDepdencies Key: BEAM-14008 URL: https://issues.apache.org/jira/browse/BEAM-14008 Proje

[jira] [Comment Edited] (BEAM-12181) Implement parallelized (approximate) mode

2022-02-25 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17498380#comment-17498380 ] Brian Hulette edited comment on BEAM-12181 at 2/26/22, 2:04 AM: ---

[jira] [Comment Edited] (BEAM-12181) Implement parallelized (approximate) mode

2022-02-25 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17498380#comment-17498380 ] Brian Hulette edited comment on BEAM-12181 at 2/26/22, 2:01 AM: ---

[jira] [Commented] (BEAM-12181) Implement parallelized (approximate) mode

2022-02-25 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17498380#comment-17498380 ] Brian Hulette commented on BEAM-12181: -- > IIUC, we can calculate the KDE for partiti

[jira] [Updated] (BEAM-12169) Allow non-deferred column operations on categorical columns

2022-02-25 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-12169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-12169: - Description: There are several operations that we currently disallow because they produce a vari

[jira] [Commented] (BEAM-12169) Allow non-deferred column operations on categorical columns

2022-02-25 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-12169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17498349#comment-17498349 ] Brian Hulette commented on BEAM-12169: -- Hey [~yeandy] thanks for pointing that out.

[jira] [Commented] (BEAM-13998) Build flakes - Timeout waiting to lock daemon addresses registry

2022-02-25 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17498295#comment-17498295 ] Brian Hulette commented on BEAM-13998: -- This is consistently breaking PythonLint on

[jira] [Commented] (BEAM-13997) Cherrypick to 2.37.0: S3 Client broken: missing get_object_metadata()

2022-02-24 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497739#comment-17497739 ] Brian Hulette commented on BEAM-13997: -- I see, so presumably reading from an s3 path

[jira] [Updated] (BEAM-13733) Make GroupBy.apply work in pandas 1.4.0

2022-02-24 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13733: - Fix Version/s: Not applicable Resolution: Fixed Status: Resolved (was: Open) >

[jira] [Commented] (BEAM-13997) Cherrypick to 2.37.0: S3 Client broken: missing get_object_metadata()

2022-02-24 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497697#comment-17497697 ] Brian Hulette commented on BEAM-13997: -- On second thought I think usage of this is p

[jira] [Commented] (BEAM-12551) BigQueryStorageReader should produce Rows, rather than GenericRecords

2022-02-24 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-12551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497656#comment-17497656 ] Brian Hulette commented on BEAM-12551: -- Yes still relevant > BigQueryStorageReader

[jira] [Commented] (BEAM-13997) Cherrypick to 2.37.0: S3 Client broken: missing get_object_metadata()

2022-02-24 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497620#comment-17497620 ] Brian Hulette commented on BEAM-13997: -- Looks like there is a workaround in BEAM-139

[jira] [Commented] (BEAM-13995) Apache beam is having vulnerable dependencies - Tensorflow, httplib2, pandas and numpy

2022-02-24 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497590#comment-17497590 ] Brian Hulette commented on BEAM-13995: -- The pandas versions in the message are odd.

[jira] [Commented] (BEAM-13506) hcatalog is broken in java 11

2022-02-22 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17496220#comment-17496220 ] Brian Hulette commented on BEAM-13506: -- I'm not sure we'll be able to close this any

[jira] [Comment Edited] (BEAM-13951) mass_comment.py has outdated list of Run strings

2022-02-17 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492311#comment-17492311 ] Brian Hulette edited comment on BEAM-13951 at 2/18/22, 2:58 AM: ---

[jira] [Comment Edited] (BEAM-13951) mass_comment.py has outdated list of Run strings

2022-02-17 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492311#comment-17492311 ] Brian Hulette edited comment on BEAM-13951 at 2/18/22, 2:57 AM: ---

[jira] [Comment Edited] (BEAM-13951) mass_comment.py has outdated list of Run strings

2022-02-17 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492311#comment-17492311 ] Brian Hulette edited comment on BEAM-13951 at 2/18/22, 2:51 AM: ---

[jira] [Commented] (BEAM-12674) SqlTransform IllegalArgumentException: key already present: SchemaCoder

2022-02-17 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-12674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17494307#comment-17494307 ] Brian Hulette commented on BEAM-12674: -- Yeah that fits with my understanding of the

[jira] [Commented] (BEAM-12674) SqlTransform IllegalArgumentException: key already present: SchemaCoder

2022-02-17 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-12674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17494274#comment-17494274 ] Brian Hulette commented on BEAM-12674: -- Looked at this offline with Andy. The proble

[jira] [Updated] (BEAM-13921) validatesCrossLanguageRunnerGoUsingJava failing for beam_PostCommit_XVR_Spark

2022-02-17 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13921: - Resolution: Fixed Status: Resolved (was: Open) > validatesCrossLanguageRunnerGoUsingJava

[jira] [Commented] (BEAM-13959) Unable to write to BigQuery tables with column named 'f'

2022-02-17 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17494065#comment-17494065 ] Brian Hulette commented on BEAM-13959: -- Hi [~joelw] I'm trying to get a sense for ho

[jira] [Updated] (BEAM-13921) validatesCrossLanguageRunnerGoUsingJava failing for beam_PostCommit_XVR_Spark

2022-02-16 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13921: - Status: Open (was: Triage Needed) > validatesCrossLanguageRunnerGoUsingJava failing for beam_Po

[jira] [Reopened] (BEAM-13921) validatesCrossLanguageRunnerGoUsingJava failing for beam_PostCommit_XVR_Spark

2022-02-16 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette reopened BEAM-13921: -- Assignee: Brian Hulette (was: Ritesh Ghorse) reopening because this didn't make the 2.37.0 cut

[jira] [Updated] (BEAM-13958) XVR Spark failing on 2.37.0 release branch

2022-02-16 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13958: - Fix Version/s: Not applicable (was: 2.37.0) Resolution: Duplicate

[jira] [Updated] (BEAM-13958) XVR Spark failing on 2.37.0 release branch

2022-02-16 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13958: - Priority: P1 (was: P2) > XVR Spark failing on 2.37.0 release branch > --

[jira] [Updated] (BEAM-13958) XVR Spark failing on 2.37.0 release branch

2022-02-16 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13958: - Fix Version/s: 2.37.0 > XVR Spark failing on 2.37.0 release branch >

[jira] [Updated] (BEAM-13958) XVR Spark failing on 2.37.0 release branch

2022-02-16 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13958: - Description: :runners:spark:2:job-server:validatesCrossLanguageRunnerGoUsingJava and :runners:sp

[jira] [Created] (BEAM-13958) XVR Spark failing on 2.37.0 release branch

2022-02-16 Thread Brian Hulette (Jira)
Brian Hulette created BEAM-13958: Summary: XVR Spark failing on 2.37.0 release branch Key: BEAM-13958 URL: https://issues.apache.org/jira/browse/BEAM-13958 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-13955) PythonLint PreCommit is broken

2022-02-15 Thread Brian Hulette (Jira)
Brian Hulette created BEAM-13955: Summary: PythonLint PreCommit is broken Key: BEAM-13955 URL: https://issues.apache.org/jira/browse/BEAM-13955 Project: Beam Issue Type: Bug Compone

[jira] [Updated] (BEAM-13694) beam_PostCommit_Java_Hadoop_Versions failing with ClassDefNotFoundError

2022-02-15 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13694: - Fix Version/s: Not applicable (was: 2.37.0) Resolution: Fixed

[jira] [Updated] (BEAM-13919) SparkStructuredStreaming VR permared

2022-02-15 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13919: - Fix Version/s: Not applicable (was: 2.37.0) Resolution: Fixed

[jira] [Updated] (BEAM-13931) BigQueryIO is sending rows that are too large to Deadletter Queue even on RETRY_ALWAYS

2022-02-15 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13931: - Resolution: Fixed Status: Resolved (was: Open) > BigQueryIO is sending rows that are too

[jira] [Assigned] (BEAM-13668) Java Spanner IO Request Count metrics broke backwards compatibility

2022-02-15 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette reassigned BEAM-13668: Assignee: Niel Markwick > Java Spanner IO Request Count metrics broke backwards compatibil

[jira] [Commented] (BEAM-13668) Java Spanner IO Request Count metrics broke backwards compatibility

2022-02-15 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492813#comment-17492813 ] Brian Hulette commented on BEAM-13668: -- [~nielm] could you confirm this was resolved

[jira] [Commented] (BEAM-13794) Update documentation for Java 17

2022-02-15 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492808#comment-17492808 ] Brian Hulette commented on BEAM-13794: -- Dropping the fix version since this isn't tr

[jira] [Updated] (BEAM-13794) Update documentation for Java 17

2022-02-15 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13794: - Fix Version/s: (was: 2.37.0) > Update documentation for Java 17 > ---

[jira] [Commented] (BEAM-13694) beam_PostCommit_Java_Hadoop_Versions failing with ClassDefNotFoundError

2022-02-15 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492792#comment-17492792 ] Brian Hulette commented on BEAM-13694: -- pr/16861 should address this, we just have t

[jira] [Updated] (BEAM-13950) PVR_Spark2_Streaming perma-red

2022-02-15 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette updated BEAM-13950: - Affects Version/s: 2.37.0 > PVR_Spark2_Streaming perma-red > -- > >

[jira] [Assigned] (BEAM-13694) beam_PostCommit_Java_Hadoop_Versions failing with ClassDefNotFoundError

2022-02-15 Thread Brian Hulette (Jira)
[ https://issues.apache.org/jira/browse/BEAM-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette reassigned BEAM-13694: Assignee: Brian Hulette > beam_PostCommit_Java_Hadoop_Versions failing with ClassDefNotFou

<    1   2   3   4   5   6   7   8   9   10   >