Re: Have makesum target display hashes in common format
On Mon, Sep 06, 2021 at 02:01:08PM +0100, Stuart Henderson wrote:
> On 2021/09/06 14:48, Marc Espie wrote:
> > On Fri, Sep 03, 2021 at 01:34:57PM -0600, Aaron Bieber wrote:
> > >
> > > Kurt Mosiejczuk writes:
> > >
> > > > On Fri, Sep 03, 2021 at 12:04:42PM -0600, Aaron Bieber wrote:
> > > >
> > > >> How many is multiple? :D - the Go stuff is gonna get ugly(er?)!
> > > >
> > > >> net/go-ipfs for example, the output is useless because it's so big :D
> > > >
> > > > Does go actually publish base64 SHA256 hashes for their modules?
> > > > If not, it's not the size of the output that makes that useless.
> > >
> > > They do, but checking by hand is redundant as the checks are all built
> > > into the tooling.
> > >
> > > For example: https://sum.golang.org/lookup/suah.dev/ogvt@v1.1.0
> > >
> > > (the h1:blablabl stuff is an encoded sha256 hash)
> > >
> > > >
> > > >> Maybe it can be conditional.. somehow?
> > > >
> > > > Why? How often do you do a "make makesum"? I do it like once per port
> > > > update and verify my distfiles.
> > > >
> > >
> > > It seems like I do it quite often. Though maybe it just stands out
> > > because the Go lists are always sofaking big!
> > >
> > > > --Kurt
> >
> > I think it would be more sense to append it to a file, having the name
> > be /dev/null by default, for those few individuals who actually need this
> >
>
> I really would like it to not calculate at all if it's not needed, some
> of the distfiles are huge and sha256 and md5 is definitely non-trivial,
> especially over NFS
>
>
So, yeah, empty instead of redirect to /dev/null
I think it's nicer to >>${SOMEFILE}... people can always do /dev/tty if they
want to see it on screen, and that way it doesn't clutter the display after
the diff.
.e.g,
.if !empty(SOMEVAR)
md6 >>${SOMEVAR}; sha256 >>${SOMEVAR}
.endif
(or the corresponding exec form)
only thing missing would be a variable name.
LEGACY_CHECKSUMS ?
Re: Have makesum target display hashes in common format
On 2021/09/06 14:48, Marc Espie wrote: > On Fri, Sep 03, 2021 at 01:34:57PM -0600, Aaron Bieber wrote: > > > > Kurt Mosiejczuk writes: > > > > > On Fri, Sep 03, 2021 at 12:04:42PM -0600, Aaron Bieber wrote: > > > > > >> How many is multiple? :D - the Go stuff is gonna get ugly(er?)! > > > > > >> net/go-ipfs for example, the output is useless because it's so big :D > > > > > > Does go actually publish base64 SHA256 hashes for their modules? > > > If not, it's not the size of the output that makes that useless. > > > > They do, but checking by hand is redundant as the checks are all built > > into the tooling. > > > > For example: https://sum.golang.org/lookup/suah.dev/ogvt@v1.1.0 > > > > (the h1:blablabl stuff is an encoded sha256 hash) > > > > > > > >> Maybe it can be conditional.. somehow? > > > > > > Why? How often do you do a "make makesum"? I do it like once per port > > > update and verify my distfiles. > > > > > > > It seems like I do it quite often. Though maybe it just stands out > > because the Go lists are always sofaking big! > > > > > --Kurt > > I think it would be more sense to append it to a file, having the name > be /dev/null by default, for those few individuals who actually need this > I really would like it to not calculate at all if it's not needed, some of the distfiles are huge and sha256 and md5 is definitely non-trivial, especially over NFS
Re: Have makesum target display hashes in common format
On Fri, Sep 03, 2021 at 01:34:57PM -0600, Aaron Bieber wrote: > > Kurt Mosiejczuk writes: > > > On Fri, Sep 03, 2021 at 12:04:42PM -0600, Aaron Bieber wrote: > > > >> How many is multiple? :D - the Go stuff is gonna get ugly(er?)! > > > >> net/go-ipfs for example, the output is useless because it's so big :D > > > > Does go actually publish base64 SHA256 hashes for their modules? > > If not, it's not the size of the output that makes that useless. > > They do, but checking by hand is redundant as the checks are all built > into the tooling. > > For example: https://sum.golang.org/lookup/suah.dev/ogvt@v1.1.0 > > (the h1:blablabl stuff is an encoded sha256 hash) > > > > >> Maybe it can be conditional.. somehow? > > > > Why? How often do you do a "make makesum"? I do it like once per port > > update and verify my distfiles. > > > > It seems like I do it quite often. Though maybe it just stands out > because the Go lists are always sofaking big! > > > --Kurt I think it would be more sense to append it to a file, having the name be /dev/null by default, for those few individuals who actually need this
Re: Have makesum target display hashes in common format
On 2021/09/03 15:20, Kurt Mosiejczuk wrote: > On Fri, Sep 03, 2021 at 06:30:01PM +, Klemens Nanni wrote: > > > Put it into a `show-checksums' target so the regular `makesum' does not > > get slowed down on already big/slow ports? > > I don't understand this suggestion. makesum only gets run when one updates > a port. The existing output it fairly useless for verifying distfiles. > Why are we worried about performance of a seldom-performed step that makes > verification easier? I'm with kn on this, updating a port isn't all that rare and this is really unwieldy on some ports (there isn't really much you can do when it spits out 300+ lines of hashes, and it's quite a slow operation on some ports on some machines). Maybe it could be controlled by a variable? Then it can be set one-off on the command line if wanted, or in mk.conf if you always want to use it.
Re: Have makesum target display hashes in common format
Kurt Mosiejczuk writes: > On Fri, Sep 03, 2021 at 12:04:42PM -0600, Aaron Bieber wrote: > >> How many is multiple? :D - the Go stuff is gonna get ugly(er?)! > >> net/go-ipfs for example, the output is useless because it's so big :D > > Does go actually publish base64 SHA256 hashes for their modules? > If not, it's not the size of the output that makes that useless. They do, but checking by hand is redundant as the checks are all built into the tooling. For example: https://sum.golang.org/lookup/suah.dev/ogvt@v1.1.0 (the h1:blablabl stuff is an encoded sha256 hash) > >> Maybe it can be conditional.. somehow? > > Why? How often do you do a "make makesum"? I do it like once per port > update and verify my distfiles. > It seems like I do it quite often. Though maybe it just stands out because the Go lists are always sofaking big! > --Kurt
Re: Have makesum target display hashes in common format
On Fri, Sep 03, 2021 at 06:30:01PM +, Klemens Nanni wrote: > Put it into a `show-checksums' target so the regular `makesum' does not > get slowed down on already big/slow ports? I don't understand this suggestion. makesum only gets run when one updates a port. The existing output it fairly useless for verifying distfiles. Why are we worried about performance of a seldom-performed step that makes verification easier? --Kurt
Re: Have makesum target display hashes in common format
On Fri, Sep 03, 2021 at 12:04:42PM -0600, Aaron Bieber wrote: > How many is multiple? :D - the Go stuff is gonna get ugly(er?)! > net/go-ipfs for example, the output is useless because it's so big :D Does go actually publish base64 SHA256 hashes for their modules? If not, it's not the size of the output that makes that useless. > Maybe it can be conditional.. somehow? Why? How often do you do a "make makesum"? I do it like once per port update and verify my distfiles. --Kurt
Re: Have makesum target display hashes in common format
On Fri, Sep 03, 2021 at 12:04:42PM -0600, Aaron Bieber wrote: > > Kurt Mosiejczuk writes: > > > Whenever I update a port or create a new one, to verify the hashes I have > > to run sha256(1) or md5(1) on the distfile. makesum does display our SHA256 > > has, but we use base64 format and everywhere else (that I've interacted > > with) > > uses hexadecimal format. > > > > This diff makes the makesum target spit out hexadecimal formatted MD5 and > > SHA256 hashes for each distfile. I included md5 because, sadly, a number of > > projects still only provide an MD5 hash for verification. > > > > I've tested this with ports with both a single distfile and multiple > > distfiles. > > How many is multiple? :D - the Go stuff is gonna get ugly(er?)! > > net/go-ipfs for example, the output is useless because it's so big :D > > Maybe it can be conditional.. somehow? Put it into a `show-checksums' target so the regular `makesum' does not get slowed down on already big/slow ports?
Re: Have makesum target display hashes in common format
Kurt Mosiejczuk writes:
> Whenever I update a port or create a new one, to verify the hashes I have
> to run sha256(1) or md5(1) on the distfile. makesum does display our SHA256
> has, but we use base64 format and everywhere else (that I've interacted with)
> uses hexadecimal format.
>
> This diff makes the makesum target spit out hexadecimal formatted MD5 and
> SHA256 hashes for each distfile. I included md5 because, sadly, a number of
> projects still only provide an MD5 hash for verification.
>
> I've tested this with ports with both a single distfile and multiple
> distfiles.
How many is multiple? :D - the Go stuff is gonna get ugly(er?)!
net/go-ipfs for example, the output is useless because it's so big :D
Maybe it can be conditional.. somehow?
>
> ok?
>
> --Kurt
>
> Index: bsd.port.mk
> ===
> RCS file: /cvs/ports/infrastructure/mk/bsd.port.mk,v
> retrieving revision 1.1555
> diff -u -p -r1.1555 bsd.port.mk
> --- bsd.port.mk 3 May 2021 17:53:15 - 1.1555
> +++ bsd.port.mk 3 Sep 2021 17:32:50 -
> @@ -2249,6 +2249,8 @@ makesum:
> done; \
> sort -u -o $$ck $$ck; \
> diff -Lold -Lnew -u ${CHECKSUM_FILE} $$ck 2>/dev/null|| true; \
> + md5 ${MAKESUMFILES}; \
> + sha256 ${MAKESUMFILES}; \
> mv -f $$ck ${CHECKSUM_FILE}
> .endif
>
Have makesum target display hashes in common format
Whenever I update a port or create a new one, to verify the hashes I have
to run sha256(1) or md5(1) on the distfile. makesum does display our SHA256
has, but we use base64 format and everywhere else (that I've interacted with)
uses hexadecimal format.
This diff makes the makesum target spit out hexadecimal formatted MD5 and
SHA256 hashes for each distfile. I included md5 because, sadly, a number of
projects still only provide an MD5 hash for verification.
I've tested this with ports with both a single distfile and multiple
distfiles.
ok?
--Kurt
Index: bsd.port.mk
===
RCS file: /cvs/ports/infrastructure/mk/bsd.port.mk,v
retrieving revision 1.1555
diff -u -p -r1.1555 bsd.port.mk
--- bsd.port.mk 3 May 2021 17:53:15 - 1.1555
+++ bsd.port.mk 3 Sep 2021 17:32:50 -
@@ -2249,6 +2249,8 @@ makesum:
done; \
sort -u -o $$ck $$ck; \
diff -Lold -Lnew -u ${CHECKSUM_FILE} $$ck 2>/dev/null|| true; \
+ md5 ${MAKESUMFILES}; \
+ sha256 ${MAKESUMFILES}; \
mv -f $$ck ${CHECKSUM_FILE}
.endif
