[ 
https://issues.apache.org/jira/browse/SPARK-62147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Chammas updated SPARK-62147:
-------------------------------------
    Description: 
We have a lower quality bar for dev tooling compared to Spark itself. This is 
OK; the user base is much smaller, and in the case of the merge script the user 
base is just committers.

However, the merge script is long overdue on a reorg to follow basic 
engineering principles, like scoping functions to focused modules and 
separating tests from the main code. On `master`, the merge script is currently 
over 5K lines of code, and the `_{_}test{_}_` docstring alone is 1K lines!

A reorg into a library with proper unit tests will make it easier to understand 
and maintain the script, and make it easier to reuse common utilities that 
other dev scripts can use today or may need down the line.

The new `prmerge` library I envision for the merge script will have modules for 
various topics covered, including:
 * Jira
 * GitHub
 * PR title parsing and linting
 * Backports
 * Commit message construction

  was:
We have a lower quality bar for dev tooling compared to Spark itself. This is 
OK; the user base is much smaller, and in the case of the merge script the user 
base is just committers.

However, the merge script is long overdue on a reorg to follow basic 
engineering principles, like scoping functions to focused modules and 
separating tests from the main code. On `master`, the merge script is currently 
over 5K lines of code, and the `__test__` docstring alone is 1K lines!

A reorg into a library with proper unit tests will make it easier to understand 
and maintain the script, and make it easier to reuse common utilities that 
other dev scripts may need down the line.


> Split merge_spark_pr.py into a prmerge library
> ----------------------------------------------
>
>                 Key: SPARK-62147
>                 URL: https://issues.apache.org/jira/browse/SPARK-62147
>             Project: Spark
>          Issue Type: Improvement
>          Components: Project Infra
>    Affects Versions: 4.4.0
>            Reporter: Nicholas Chammas
>            Assignee: Nicholas Chammas
>            Priority: Minor
>
> We have a lower quality bar for dev tooling compared to Spark itself. This is 
> OK; the user base is much smaller, and in the case of the merge script the 
> user base is just committers.
> However, the merge script is long overdue on a reorg to follow basic 
> engineering principles, like scoping functions to focused modules and 
> separating tests from the main code. On `master`, the merge script is 
> currently over 5K lines of code, and the `_{_}test{_}_` docstring alone is 1K 
> lines!
> A reorg into a library with proper unit tests will make it easier to 
> understand and maintain the script, and make it easier to reuse common 
> utilities that other dev scripts can use today or may need down the line.
> The new `prmerge` library I envision for the merge script will have modules 
> for various topics covered, including:
>  * Jira
>  * GitHub
>  * PR title parsing and linting
>  * Backports
>  * Commit message construction



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to