[jira] [Created] (BEAM-8850) Flink: Batch translation context respects provided PTransform during Input / Output lookups.

David Moravek created BEAM-8850: --- Summary: Flink: Batch translation context respects provided PTransform during Input / Output lookups. Key: BEAM-8850 URL: https://issues.apache.org/jira/browse/BEAM-8850

[jira] [Created] (BEAM-8849) Flink: Support multiple translation or a single URN in batch runner.

David Moravek created BEAM-8849: --- Summary: Flink: Support multiple translation or a single URN in batch runner. Key: BEAM-8849 URL: https://issues.apache.org/jira/browse/BEAM-8849 Project: Beam

[jira] [Created] (BEAM-8848) Flink: Memory efficient GBK implementation for batch runner

David Moravek created BEAM-8848: --- Summary: Flink: Memory efficient GBK implementation for batch runner Key: BEAM-8848 URL: https://issues.apache.org/jira/browse/BEAM-8848 Project: Beam Issue

[jira] [Updated] (BEAM-8577) FileSystems may have not be initialized during ResourceId deserialization

[ https://issues.apache.org/jira/browse/BEAM-8577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek updated BEAM-8577: Fix Version/s: (was: 2.17.0) > FileSystems may have not be initialized during ResourceId

[jira] [Commented] (BEAM-8577) FileSystems may have not be initialized during ResourceId deserialization

[ https://issues.apache.org/jira/browse/BEAM-8577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976838#comment-16976838 ] David Moravek commented on BEAM-8577: - This is broken in previous versions as well. Removing 2.17 tag

[jira] [Created] (BEAM-8648) Euphoria: Deprecate OutputHints from public API

David Moravek created BEAM-8648: --- Summary: Euphoria: Deprecate OutputHints from public API Key: BEAM-8648 URL: https://issues.apache.org/jira/browse/BEAM-8648 Project: Beam Issue Type:

[jira] [Commented] (BEAM-8608) Chain DoFns in Flink batch runner when possible.

[ https://issues.apache.org/jira/browse/BEAM-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16971687#comment-16971687 ] David Moravek commented on BEAM-8608: - Needed for optimal finishBundle implementation. > Chain DoFns

[jira] [Updated] (BEAM-8608) Chain DoFns in Flink batch runner when possible.

[ https://issues.apache.org/jira/browse/BEAM-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek updated BEAM-8608: Description: Right now, in Batch runner, DoFn is executed using MapPartition operator

[jira] [Updated] (BEAM-8608) Chain DoFns in Flink batch runner when possible.

[ https://issues.apache.org/jira/browse/BEAM-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek updated BEAM-8608: Attachment: Screen Shot 2019-11-07 at 10.35.22.png > Chain DoFns in Flink batch runner when

[jira] [Updated] (BEAM-8608) Chain DoFns in Flink batch runner when possible.

[ https://issues.apache.org/jira/browse/BEAM-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek updated BEAM-8608: Attachment: Screen Shot 2019-11-07 at 10.35.07.png > Chain DoFns in Flink batch runner when

[jira] [Created] (BEAM-8608) Chain DoFns in Flink batch runner when possible.

David Moravek created BEAM-8608: --- Summary: Chain DoFns in Flink batch runner when possible. Key: BEAM-8608 URL: https://issues.apache.org/jira/browse/BEAM-8608 Project: Beam Issue Type:

[jira] [Updated] (BEAM-8568) Local file system does not match relative path with wildcards

[ https://issues.apache.org/jira/browse/BEAM-8568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek updated BEAM-8568: Status: Open (was: Triage Needed) > Local file system does not match relative path with wildcards

[jira] [Created] (BEAM-8577) FileSystems may have not be initialized during ResourceId deserialization

David Moravek created BEAM-8577: --- Summary: FileSystems may have not be initialized during ResourceId deserialization Key: BEAM-8577 URL: https://issues.apache.org/jira/browse/BEAM-8577 Project: Beam

[jira] [Updated] (BEAM-8568) Local file system does not match relative path with wildcards

[ https://issues.apache.org/jira/browse/BEAM-8568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek updated BEAM-8568: Fix Version/s: 2.17.0 Affects Version/s: 2.16.0 Description: CWD structure:

[jira] [Comment Edited] (BEAM-8568) Local file system does not match relative path with wildcards

[ https://issues.apache.org/jira/browse/BEAM-8568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969012#comment-16969012 ] David Moravek edited comment on BEAM-8568 at 11/7/19 7:52 AM: --

[jira] [Commented] (BEAM-8568) Local file system does not match relative path with wildcards

[ https://issues.apache.org/jira/browse/BEAM-8568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969012#comment-16969012 ] David Moravek commented on BEAM-8568: - LocalFileSystem no longer support wildcard relative paths. >

[jira] [Assigned] (BEAM-7962) Drop support for Flink 1.5 and 1.6

[ https://issues.apache.org/jira/browse/BEAM-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek reassigned BEAM-7962: --- Assignee: Maximilian Michels (was: David Moravek) > Drop support for Flink 1.5 and 1.6 >

[jira] [Commented] (BEAM-7962) Drop support for Flink 1.5 and 1.6

[ https://issues.apache.org/jira/browse/BEAM-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926428#comment-16926428 ] David Moravek commented on BEAM-7962: - [~mxm] awesome, if you already have the PR ready, feel free to

[jira] [Assigned] (BEAM-7962) Drop support for Flink 1.5 and 1.6

[ https://issues.apache.org/jira/browse/BEAM-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek reassigned BEAM-7962: --- Assignee: David Moravek (was: Maximilian Michels) > Drop support for Flink 1.5 and 1.6 >

[jira] [Updated] (BEAM-8025) Cassandra IO classMethod test is flaky

[ https://issues.apache.org/jira/browse/BEAM-8025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek updated BEAM-8025: Affects Version/s: 2.16.0 Priority: Blocker (was: Major) > Cassandra IO classMethod

[jira] [Commented] (BEAM-7730) Add Flink 1.9 build target and Make FlinkRunner compatible with Flink 1.9

[ https://issues.apache.org/jira/browse/BEAM-7730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900109#comment-16900109 ] David Moravek commented on BEAM-7730: - I've managed to solve the serialization issues. Those were on

[jira] [Commented] (BEAM-7730) Add Flink 1.9 build target and Make FlinkRunner compatible with Flink 1.9

[ https://issues.apache.org/jira/browse/BEAM-7730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16897895#comment-16897895 ] David Moravek commented on BEAM-7730: - Successfully tested 1.9 runner with an internal pipeline.

[jira] [Resolved] (BEAM-6883) StreamingSourceMetricsTest takes too long to finish

[ https://issues.apache.org/jira/browse/BEAM-6883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek resolved BEAM-6883. - Resolution: Fixed Fix Version/s: 2.15.0 > StreamingSourceMetricsTest takes too long to

[jira] [Commented] (BEAM-7730) Add Flink 1.9 build target and Make FlinkRunner compatible with Flink 1.9

[ https://issues.apache.org/jira/browse/BEAM-7730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890219#comment-16890219 ] David Moravek commented on BEAM-7730: - Current progress: https://github.com/dmvk/beam/tree/BEAM-7730

[jira] [Commented] (BEAM-7730) Add Flink 1.9 build target and Make FlinkRunner compatible with Flink 1.9

[ https://issues.apache.org/jira/browse/BEAM-7730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890181#comment-16890181 ] David Moravek commented on BEAM-7730: - Already did some work on this as we need to start trying out

[jira] [Updated] (BEAM-7613) HadoopFileSystem can be only used with fs.defaultFS

[ https://issues.apache.org/jira/browse/BEAM-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek updated BEAM-7613: Summary: HadoopFileSystem can be only used with fs.defaultFS (was: HdfsFileSystem can be only used

[jira] [Created] (BEAM-7613) HdfsFileSystem can be only used with fs.defaultFS

David Moravek created BEAM-7613: --- Summary: HdfsFileSystem can be only used with fs.defaultFS Key: BEAM-7613 URL: https://issues.apache.org/jira/browse/BEAM-7613 Project: Beam Issue Type: Bug

[jira] [Resolved] (BEAM-7543) ReduceByKey.combineBy must accept BinaryFunction

[ https://issues.apache.org/jira/browse/BEAM-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek resolved BEAM-7543. - Resolution: Fixed Fix Version/s: 2.14.0 > ReduceByKey.combineBy must accept BinaryFunction

[jira] [Created] (BEAM-7591) Unify FileSystems tests across implementation

David Moravek created BEAM-7591: --- Summary: Unify FileSystems tests across implementation Key: BEAM-7591 URL: https://issues.apache.org/jira/browse/BEAM-7591 Project: Beam Issue Type:

[jira] [Commented] (BEAM-7561) HdfsFileSystem is unable to match a directory

[ https://issues.apache.org/jira/browse/BEAM-7561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866353#comment-16866353 ] David Moravek commented on BEAM-7561: - at the conference, but I’ll do my best to finish it tonight

[jira] [Created] (BEAM-7561) HdfsFileSystem is unable to match a directory

David Moravek created BEAM-7561: --- Summary: HdfsFileSystem is unable to match a directory Key: BEAM-7561 URL: https://issues.apache.org/jira/browse/BEAM-7561 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-6895) Upgrade joda-time to 2.10.1

David Moravek created BEAM-6895: --- Summary: Upgrade joda-time to 2.10.1 Key: BEAM-6895 URL: https://issues.apache.org/jira/browse/BEAM-6895 Project: Beam Issue Type: Improvement

[jira] [Resolved] (BEAM-6812) Convert keys to ByteArray in Combine.perKey for Spark

[ https://issues.apache.org/jira/browse/BEAM-6812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek resolved BEAM-6812. - Resolution: Fixed Fix Version/s: 2.12.0 > Convert keys to ByteArray in Combine.perKey for

[jira] [Resolved] (BEAM-6774) Euphoria - replace Distinct.mapped with Distinct.projected

[ https://issues.apache.org/jira/browse/BEAM-6774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek resolved BEAM-6774. - Resolution: Fixed Fix Version/s: 2.12.0 > Euphoria - replace Distinct.mapped with

[jira] [Assigned] (BEAM-6774) Euphoria - replace Distinct.mapped with Distinct.projected

[ https://issues.apache.org/jira/browse/BEAM-6774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek reassigned BEAM-6774: --- Assignee: Jan Lukavsky (was: Vaclav Plajt) > Euphoria - replace Distinct.mapped with

[jira] [Updated] (BEAM-6589) SDKs::Java::Extensions::Kryo is incorrectly shaded

[ https://issues.apache.org/jira/browse/BEAM-6589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek updated BEAM-6589: Affects Version/s: 2.10.0 > SDKs::Java::Extensions::Kryo is incorrectly shaded >

[jira] [Resolved] (BEAM-6589) SDKs::Java::Extensions::Kryo is incorrectly shaded

[ https://issues.apache.org/jira/browse/BEAM-6589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek resolved BEAM-6589. - Resolution: Fixed > SDKs::Java::Extensions::Kryo is incorrectly shaded >

[jira] [Commented] (BEAM-6589) SDKs::Java::Extensions::Kryo is incorrectly shaded

[ https://issues.apache.org/jira/browse/BEAM-6589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769083#comment-16769083 ] David Moravek commented on BEAM-6589: - yes, closing > SDKs::Java::Extensions::Kryo is incorrectly

[jira] [Resolved] (BEAM-3900) Introduce Euphoria Java 8 DSL

[ https://issues.apache.org/jira/browse/BEAM-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek resolved BEAM-3900. - Resolution: Fixed Fix Version/s: 2.9.0 > Introduce Euphoria Java 8 DSL >

[jira] [Created] (BEAM-6214) Spark: CombineByKey performance

David Moravek created BEAM-6214: --- Summary: Spark: CombineByKey performance Key: BEAM-6214 URL: https://issues.apache.org/jira/browse/BEAM-6214 Project: Beam Issue Type: Improvement

[jira] [Resolved] (BEAM-6174) Remove unnecesarry Kryo dependecny from euphoria

[ https://issues.apache.org/jira/browse/BEAM-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek resolved BEAM-6174. - Resolution: Fixed > Remove unnecesarry Kryo dependecny from euphoria >

[jira] [Updated] (BEAM-6174) Remove unnecesarry Kryo dependecny from euphoria

[ https://issues.apache.org/jira/browse/BEAM-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek updated BEAM-6174: Fix Version/s: 2.10.0 > Remove unnecesarry Kryo dependecny from euphoria >

[jira] [Updated] (BEAM-6062) Spark runner does not show Beam metrics in web UI

[ https://issues.apache.org/jira/browse/BEAM-6062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek updated BEAM-6062: Summary: Spark runner does not show Beam metrics in web UI (was: Spark runner do not show Beam

[jira] [Resolved] (BEAM-6062) Spark runner do not show Beam metrics in Spark Web Interface

[ https://issues.apache.org/jira/browse/BEAM-6062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek resolved BEAM-6062. - Resolution: Fixed > Spark runner do not show Beam metrics in Spark Web Interface >

[jira] [Updated] (BEAM-5857) Unnecessary submission of a new job in Combine.globally

[ https://issues.apache.org/jira/browse/BEAM-5857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek updated BEAM-5857: Affects Version/s: 2.8.0 > Unnecessary submission of a new job in Combine.globally >

[jira] [Updated] (BEAM-5987) Spark SideInputReader performance

[ https://issues.apache.org/jira/browse/BEAM-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek updated BEAM-5987: Description: We did some profiling of a spark job and 90% of the application time was spent on

[jira] [Updated] (BEAM-5987) Spark SideInputReader performance

[ https://issues.apache.org/jira/browse/BEAM-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek updated BEAM-5987: Description: We did some profiling of a spark job and 90% of the application time was spent on

[jira] [Updated] (BEAM-5987) Spark SideInputReader performance

[ https://issues.apache.org/jira/browse/BEAM-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek updated BEAM-5987: Attachment: Screen Shot 2018-11-06 at 13.05.36.png > Spark SideInputReader performance >

[jira] [Created] (BEAM-5987) Spark SideInputReader performance

David Moravek created BEAM-5987: --- Summary: Spark SideInputReader performance Key: BEAM-5987 URL: https://issues.apache.org/jira/browse/BEAM-5987 Project: Beam Issue Type: Bug