arvindshmicrosoft commented on a change in pull request #427:
URL: https://github.com/apache/fluo-muchos/pull/427#discussion_r779975046
##########
File path: lib/muchos/config/base.py
##########
@@ -480,12 +505,26 @@ def checksum_ver(self, software, version):
)
key = "{0}:{1}".format(software, version)
+
if key not in self.checksums_d:
- exit(
- "ERROR - Failed to find checksums for {} {} in {}".format(
- software, version, self.checksums_path
+ local_tarball_path = self.get_local_tarball_path(software)
+ if local_tarball_path is None or not exists(local_tarball_path):
+ exit(
+ "ERROR - Failed to find either a valid checksum in {}, "
+ "or a local tarball to upload for {} {}.".format(
+ self.checksums_path, software, version
+ )
)
- )
+ else:
+ # compute and use the checksum for local tarball
+ local_tarball_sha512 = sha512()
+ with open(local_tarball_path, "rb") as tarball_contents:
+ file_buffer = tarball_contents.read(65536)
+ while len(file_buffer) > 0:
+ local_tarball_sha512.update(file_buffer)
+ file_buffer = tarball_contents.read(65536)
+ return f"sha512:{local_tarball_sha512.hexdigest()}"
+
Review comment:
Please review my subsequent commit. I believe it addresses the previous
comments. To summarize, these changes now offer added flexibility, without
sacrificing security:
* For archive releases, we can now use local tarballs from conf/upload,
**only** if they have either a checksum in conf/checksums, or a matching
.sha512 file present in conf/upload.
* I tested this with samples of archived Hadoop, Zookeeper and Spark
releases which do not have entries in conf/checksums - they are all good.
* For good measure, I also tested a case where I placed the released
Hadoop 3.1.4 tarball alone in the conf/upload folder (without a .sha512 file)
and validated that Mucho correctly used the matching checksum from
conf/checksums.
* For SNAPSHOT tarballs, we will compute a checksum locally and use that on
the Ansible side to ensure that the tarball was not corrupted in transit. This
is an enhancement over what we have currently in main.
By the way, in neither my original commit, nor in this one, will the code
download a tarball if it is already present and the checksum matches. This
behavior is also status quo from what is in main.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
