[
https://issues.apache.org/jira/browse/SPARK-62147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicholas Chammas updated SPARK-62147:
-------------------------------------
Description:
We have a lower quality bar for dev tooling compared to Spark itself. This is
OK; the user base is much smaller, and in the case of the merge script the user
base is just committers.
However, the merge script is long overdue on a reorg to follow basic
engineering principles, like scoping functions to focused modules and
separating tests from the main code. On `master`, the merge script is currently
over 5K lines of code, and the `__test__` docstring alone is 1K lines!
A reorg into a library with proper unit tests will make it easier to understand
and maintain the script, and make it easier to reuse common utilities that
other dev scripts may need down the line.
> Split merge_spark_pr.py into a prmerge library
> ----------------------------------------------
>
> Key: SPARK-62147
> URL: https://issues.apache.org/jira/browse/SPARK-62147
> Project: Spark
> Issue Type: Improvement
> Components: Project Infra
> Affects Versions: 4.4.0
> Reporter: Nicholas Chammas
> Assignee: Nicholas Chammas
> Priority: Minor
>
> We have a lower quality bar for dev tooling compared to Spark itself. This is
> OK; the user base is much smaller, and in the case of the merge script the
> user base is just committers.
> However, the merge script is long overdue on a reorg to follow basic
> engineering principles, like scoping functions to focused modules and
> separating tests from the main code. On `master`, the merge script is
> currently over 5K lines of code, and the `__test__` docstring alone is 1K
> lines!
> A reorg into a library with proper unit tests will make it easier to
> understand and maintain the script, and make it easier to reuse common
> utilities that other dev scripts may need down the line.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]