[
https://issues.apache.org/jira/browse/BEAM-14323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Moritz Mack updated BEAM-14323:
-------------------------------
Status: Open (was: Triage Needed)
> Improve IDE integration for Spark cross version builds
> ------------------------------------------------------
>
> Key: BEAM-14323
> URL: https://issues.apache.org/jira/browse/BEAM-14323
> Project: Beam
> Issue Type: Improvement
> Components: runner-spark
> Reporter: Moritz Mack
> Assignee: Moritz Mack
> Priority: P2
>
> With the current build setup, developer experience is fairly poor when
> working with cross version builds for Spark (but also similarly for Flink):
> * Sources for version specific overrides are copied to a new location and
> references as gradle sources from there. First of all, this is totally
> unnecessary. These sources are not shared and should be used in place. But
> much more troublesome, the actual sources won't be resolved / checked by any
> IDE anymore and can't be properly worked on that way. Sadly for no reason at
> all ...
> * The actual shared resources on the other hand are referenced (added to
> srcDirs) in place. The IDE will randomly assign them to one Spark version
> module. Typically, for IntelliJ at least, that's the first (lower) one and
> not the one developers are actively working on.
> The suggested changes is:
> * Don't copy version specific overrides
> * Only copy shared sources conditionally based on a flag. This allows
> developers to disable copying to pick a primary version they intend to work
> on.
> Note: This is primary a cosmetic flag to improve IDE integration and has no
> impact on builds, even if all modules disable copying.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
