[spark-docker] branch master updated: [SPARK-40520] Add support to generate DOI mainifest

[yikun]

This is an automated email from the ASF dual-hosted git repository.

yikun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark-docker.git


The following commit(s) were added to refs/heads/master by this push:
     new 7bb8661  [SPARK-40520] Add support to generate DOI mainifest
7bb8661 is described below

commit 7bb8661f7d57356f94fd5874696df1b1c058cb0b
Author: Yikun Jiang <yikunk...@gmail.com>
AuthorDate: Wed Dec 21 10:15:44 2022 +0800

    [SPARK-40520] Add support to generate DOI mainifest
    
    ### What changes were proposed in this pull request?
    This patch add support to generate DOI mainifest from versions.json.
    
    ### Why are the changes needed?
    To help generate DOI mainifest
    
    ### Does this PR introduce _any_ user-facing change?
    No
    
    ### How was this patch tested?
    ```shell
    $ flake8 ./tools/manifest.py --max-line-length=100
    $ black ./tools/manifest.py
    All done! ✨ 🍰 ✨
    1 file left unchanged.
    ```
    
    ```shell
    $ tools/manifest.py manifest
    Maintainers: Apache Spark Developers <devspark.apache.org> (ApacheSpark)
    GitRepo: https://github.com/apache/spark-docker.git
    
    Tags: 3.3.1-scala2.12-java11-python3-ubuntu, 3.3.1-python3, 3.3.1, python3, 
latest
    Architectures: amd64, arm64v8
    GitCommit: 496edb6dee0ade08bc5d180d7a6da0ff8b5d91ff
    Directory: ./3.3.1/scala2.12-java11-python3-ubuntu
    
    Tags: 3.3.1-scala2.12-java11-r-ubuntu, 3.3.1-r, r
    Architectures: amd64, arm64v8
    GitCommit: 496edb6dee0ade08bc5d180d7a6da0ff8b5d91ff
    Directory: ./3.3.1/scala2.12-java11-r-ubuntu
    
    // ... ...
    ```
    
    Closes #27 from Yikun/SPARK-40520.
    
    Authored-by: Yikun Jiang <yikunk...@gmail.com>
    Signed-off-by: Yikun Jiang <yikunk...@gmail.com>
---
 tools/manifest.py | 34 ++++++++++++++++++++++++++++++++--
 1 file changed, 32 insertions(+), 2 deletions(-)

diff --git a/tools/manifest.py b/tools/manifest.py
index fbfad6f..13bc631 100755
--- a/tools/manifest.py
+++ b/tools/manifest.py
@@ -19,7 +19,33 @@
 
 from argparse import ArgumentParser
 import json
-from statistics import mode
+import subprocess
+
+
+def run_cmd(cmd):
+    if isinstance(cmd, list):
+        return subprocess.check_output(cmd).decode("utf-8")
+    else:
+        return subprocess.check_output(cmd.split(" ")).decode("utf-8")
+
+
+def generate_manifest(versions):
+    output = (
+        "Maintainers: Apache Spark Developers <d...@spark.apache.org> 
(@ApacheSpark)\n"
+        "GitRepo: https://github.com/apache/spark-docker.git\n\n";
+    )
+    git_commit = run_cmd("git rev-parse HEAD").replace("\n", "")
+    content = (
+        "Tags: %s\n"
+        "Architectures: amd64, arm64v8\n"
+        "GitCommit: %s\n"
+        "Directory: ./%s\n\n"
+    )
+    for version in versions:
+        tags = ", ".join(version["tags"])
+        path = version["path"]
+        output += content % (tags, git_commit, path)
+    return output
 
 
 def parse_opts():
@@ -27,7 +53,7 @@ def parse_opts():
 
     parser.add_argument(
         dest="mode",
-        choices=["tags"],
+        choices=["tags", "manifest"],
         type=str,
         help="The print mode of script",
     )
@@ -76,6 +102,10 @@ def main():
             # Get matched version's tags
             tags = versions[0]["tags"] if versions else []
         print(",".join(["%s:%s" % (image, t) for t in tags]))
+    elif mode == "manifest":
+        with open(version_file, "r") as f:
+            versions = json.load(f).get("versions")
+            print(generate_manifest(versions))
 
 
 if __name__ == "__main__":


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org