Re: How do I know if my 13-stable has security patches?

[Warner Losh]

On Sat, Feb 27, 2021 at 12:15 AM Kevin Oberman  wrote:

> Well, back to a full clone. Thanks for the "--unshallow" argument, but so
> far it has failed twice. I suspect that it's my urtwn interface is bad.
> I'll swap it out tomorrow and see it that fixes it.
>
> I am a bit surprised at how little more space the full clone takes. I was
> really expecting it to be much worse. Of course, it will only grow... as
> will the sources, themselves.
>
> Warner, I suggest an immediate update to your mini-git primer. It was
> already a bit out of date, but this fix is more urgent with RC on the
> horizon. In particular, the "Repositories" at least look old to me.
>

First quick pass done. Please let me know what you think. You can suggest
edits right on the github site (they turn into pull requests) or you can
send me suggestions directly.

Warner


> --
> Kevin Oberman, Part time kid herder and retired Network Engineer
> E-mail: rkober...@gmail.com
> PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683
>
>
> On Fri, Feb 26, 2021 at 10:55 AM Ed Maste  wrote:
>
> > On Fri, 26 Feb 2021 at 12:10, Helge Oldach  wrote:
> > >
> > > A shallow tree is about 1.6G. If you want to patch source (say, from
> > > a SA or EN) you certainly also need space for an object tree which is
> > > about 4.5G. The total is >6G.
> > >
> > > I'd say relative to the total required to build, the 1.1G "savings"
> from
> > > using a shallow versus a full tree (which is about 2.7G) isn't really
> > > worth the effort. Plus, you get the a few benefits like full commit
> > > history including comments.
> >
> > Indeed, this is a good point. We can update docs to state:
> >
> > At present a full clone is required to include the commit count in
> > uname. An existing shallow clone can be converted into a full clone by
> > running
> >
> > % git fetch --unshallow
> > ___
> > freebsd-stable@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> > To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org
> "
> >
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: How do I know if my 13-stable has security patches?

[Kevin Oberman]

Well, back to a full clone. Thanks for the "--unshallow" argument, but so
far it has failed twice. I suspect that it's my urtwn interface is bad.
I'll swap it out tomorrow and see it that fixes it.

I am a bit surprised at how little more space the full clone takes. I was
really expecting it to be much worse. Of course, it will only grow... as
will the sources, themselves.

Warner, I suggest an immediate update to your mini-git primer. It was
already a bit out of date, but this fix is more urgent with RC on the
horizon. In particular, the "Repositories" at least look old to me.
--
Kevin Oberman, Part time kid herder and retired Network Engineer
E-mail: rkober...@gmail.com
PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683


On Fri, Feb 26, 2021 at 10:55 AM Ed Maste  wrote:

> On Fri, 26 Feb 2021 at 12:10, Helge Oldach  wrote:
> >
> > A shallow tree is about 1.6G. If you want to patch source (say, from
> > a SA or EN) you certainly also need space for an object tree which is
> > about 4.5G. The total is >6G.
> >
> > I'd say relative to the total required to build, the 1.1G "savings" from
> > using a shallow versus a full tree (which is about 2.7G) isn't really
> > worth the effort. Plus, you get the a few benefits like full commit
> > history including comments.
>
> Indeed, this is a good point. We can update docs to state:
>
> At present a full clone is required to include the commit count in
> uname. An existing shallow clone can be converted into a full clone by
> running
>
> % git fetch --unshallow
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: How do I know if my 13-stable has security patches?

[Ed Maste]

On Fri, 26 Feb 2021 at 12:10, Helge Oldach  wrote:
>
> A shallow tree is about 1.6G. If you want to patch source (say, from
> a SA or EN) you certainly also need space for an object tree which is
> about 4.5G. The total is >6G.
>
> I'd say relative to the total required to build, the 1.1G "savings" from
> using a shallow versus a full tree (which is about 2.7G) isn't really
> worth the effort. Plus, you get the a few benefits like full commit
> history including comments.

Indeed, this is a good point. We can update docs to state:

At present a full clone is required to include the commit count in
uname. An existing shallow clone can be converted into a full clone by
running

% git fetch --unshallow
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: How do I know if my 13-stable has security patches?

[Warner Losh]

On Fri, Feb 26, 2021 at 9:35 AM Ed Maste  wrote:

> On Thu, 25 Feb 2021 at 15:58, Warner Losh  wrote:
> >
> > The problem, though, can happen when you run a shallow clone or gitup to
> > get the sources and build from that. In that case the v number is bogus
> > (hmmm, we should omit it when we have a shallow clone maybe).
>
> I want to clarify one point here - the commit count is already omitted
> from uname in the case of shallow clones (as Kevin Oberman
> discovered).
>
> Shallow clones certainly have the benefit of limiting the amount of
> disk space used by the clone. Does that outweigh the loss of the
> commit count?
>
> I had a look at the size of the .git directory with different --depth
> settings:
>
> 262Mstable-13-shallow-1/.git
> 262Mstable-13-shallow-10/.git
> 262Mstable-13-shallow-100/.git
> 281Mstable-13-shallow-1000/.git
> 807Mstable-13-shallow-1/.git
>
> I think we can provide a way to include the commit count as long as
> we're willing to require some minimum clone depth, and will pursue
> this in the next while. If this works out it will make it into
> stable/13, but probably not releng/13.0.
>

The count is a count back to the first commit in the repo. There's not even
a clear design for how to accomplish that with a subset of the repo that's
been floated. It might be possible, but until we have a clearly articulated
plan, it may be a bit premature to set expectations that it might happen.

I'm happy to be proven wrong, of course, and once there's a clear way to
accomplish this that's not fragile or a large maintenance burden on the
project, I'll happily support it...

Warner
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: How do I know if my 13-stable has security patches?

[Ed Maste]

On Thu, 25 Feb 2021 at 15:58, Warner Losh  wrote:
>
> The problem, though, can happen when you run a shallow clone or gitup to
> get the sources and build from that. In that case the v number is bogus
> (hmmm, we should omit it when we have a shallow clone maybe).

I want to clarify one point here - the commit count is already omitted
from uname in the case of shallow clones (as Kevin Oberman
discovered).

Shallow clones certainly have the benefit of limiting the amount of
disk space used by the clone. Does that outweigh the loss of the
commit count?

I had a look at the size of the .git directory with different --depth settings:

262Mstable-13-shallow-1/.git
262Mstable-13-shallow-10/.git
262Mstable-13-shallow-100/.git
281Mstable-13-shallow-1000/.git
807Mstable-13-shallow-1/.git

I think we can provide a way to include the commit count as long as
we're willing to require some minimum clone depth, and will pursue
this in the next while. If this works out it will make it into
stable/13, but probably not releng/13.0.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: How do I know if my 13-stable has security patches?

[Karl Denninger]

On 2/26/2021 10:22, Ed Maste wrote:

On Thu, 25 Feb 2021 at 16:57, Karl Denninger  wrote:

The time (and present items) on a given machine to know whether it is
covered by a given advisory under the "svn view of the world" is one
command, and no sources.  That is, if the advisory says "r123456" has
the fix, then if I do a "uname -v" and get something larger, it's safe.

Yes, as previously stated the commit count will be included in future
advisories.

On stable/13 today uname will include:
uname displays e.g. stable/13-n244688-66308a13dddc

The advisory would report stabl/13-n244572

244688 is greater than 244572 so will have the fix.

Sounds like the issue has been addressed -- thank you!
--
Karl Denninger
k...@denninger.net 
/The Market Ticker/
/[S/MIME encrypted email preferred]/


smime.p7s
Description: S/MIME Cryptographic Signature

Re: How do I know if my 13-stable has security patches?

[Ed Maste]

On Thu, 25 Feb 2021 at 16:57, Karl Denninger  wrote:
>
> The time (and present items) on a given machine to know whether it is
> covered by a given advisory under the "svn view of the world" is one
> command, and no sources.  That is, if the advisory says "r123456" has
> the fix, then if I do a "uname -v" and get something larger, it's safe.

Yes, as previously stated the commit count will be included in future
advisories.

On stable/13 today uname will include:
uname displays e.g. stable/13-n244688-66308a13dddc

The advisory would report stabl/13-n244572

244688 is greater than 244572 so will have the fix.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: How do I know if my 13-stable has security patches?

[Mark Millard via freebsd-stable]

Warner Losh imp at bsdimp.com wrote on
Fri Feb 26 00:23:15 UTC 2021 :

> Before I get into the blow by blow (which can sound nit-picky, despite my
> best efforts), I would like to apologize. It wasn't completely appreciated
> how clearly the dependencies that the nX number being generated needed
> to be communicated. And for that I apologize. When they are met, we have
> all the pieces we need to at build time to generate the nX number and
> none of the fallback methods are needed. I'll work to update the docs to
> clearly communicate this since it is completely absent from my current docs.
. . . (lots more later and in earlier notices) . . .

There is a fairly simple technique to figure out the relationships
based on using https://cgit.freebsd.org instead of a local .git
copy.

I'll give examples based on the notice's:

releng/13.0/ ce9af53d0897a1cb926bd244f499fc09b1626b27

and examples of before (earlier date/time) and after (later
date/time), presumably from some command like uname -apKU .

The date/time-increasing order for the below is (for reference):
8305d6906fe983a . . . ce9af53d0897a1c . . . ba27dd8be821792

I'll follow the same steps with the before value vs. the
after value substituted and report what results.

I go to: https://cgit.freebsd.org/src/log/?h=releng/13.0
to match that part of the notice. (I'm not trying to specify
which technique that one uses for this, just the result.)

I select range and enter: 8305d6906fe983a~1..ce9af53d0897a1c
(8305d6906fe983a happens to be a before ce9af53d0897a1c value):

(Spacing in the ranges seems to be important to avoid.)

The result is I see:

Commit message (Expand) Author  Age Files   Lines

(In other words an empty output/range meets the criteria
for when uname -apKU reported an earlier date/time's
commit.)

By contrast . . .

I again go to: https://cgit.freebsd.org/src/log/?h=releng/13.0 .

I select range and enter: ba27dd8be821792~1..ce9af53d0897a1c
(ba27dd8be821792 happens to be an after ce9af53d0897a1c value):

Commit message (Expand) Author  Age Files   Lines
*   zfs: merge OpenZFS master-9312e0fd1 Martin Matuska  4 days  40  
-248/+724
|\  
| * Update vendor/openzfs to master-9312e0fd1vendor/openzfs Martin Matuska  
4 days  36  -247/+716
* | Fix build after 2c7dc6bae9fd.   Alexander Motin 4 days  1   -0/+4
* | Refactor CTL datamove KPI.  Alexander Motin 4 days  12  -162/+94
* | jail: Add pr_state to struct prison Jamie Gritton   4 days  2   
-51/+65
* | vfs: shrink struct vnode to 448 bytes on LP64   Mateusz Guzik   4 days  
1   -1/+12
* | jail: fix build after the previous commit   Mateusz Guzik   4 days  
1   -1/+1
* | jail: Change the locking around pr_ref and pr_uref  Jamie Gritton   
4 days  6   -235/+232
* | sctp: improve computation of an alternate net   Michael Tuexen  5 days  
1   -36/+49
* | sctp: clear a pointer to a net which will be removedMichael Tuexen  
5 days  1   -0/+4
* | ext2fs: clear write cluster tracking on truncation  Konstantin 
Belousov 5 days  1   -0/+1
. . . (goes on indefinately) . . .

In other words it starts to list everything at
ba27dd8be821792 or before (in time) for the branch.
(Listing ba27dd8be821792 itself is why I use the ~1
part of the notation on the left hand hash-id.)

(In other words a non-empty output/range meets the
criteria for when uname -apKU reported a no-earlier
date/time's commit, normally a later date/time's
commit.)

So the empty vs. non-empty result indicates the time
relationship of the hash-ids on the branch.

No need for a local .git of any kind but access to
https://cgit.freebsd.org is needed for the technique.


Notes:

The order of the range specifications is deliberate
in order to make the output harder to misinterpret.
This is because . . .

Using: ce9af53d0897a1c~1..8305d6906fe983a
gives:

Commit message (Expand) Author  Age Files   Lines
*   loader: unload command should reset tg_kernel_supported in gfx_state
Toomas Soome3 days  1   -0/+2
*   Fix loader detection of vbefb support on !amd64 Dimitry Andric  3 days  
1   -2/+2
*   loader: start kernel in text mode when there is no vbefb vt driver  
Toomas Soome3 days  4   -9/+74
*   loader_lua: consider userboot console as serial Toomas Soome3 days  
1   -1/+4
*   Add UPDATING entries and bump version   Mark Johnston   2 days  2   
-1/+8
*   pam_login_access: Fix negative entry matching logic Mark Johnston   
2 days  1   -3/+3
*   xen-blkback: fix leak of grant maps on ring setup failure   Roger 
Pau Monné 2 days  1   -0/+21

and using: ce9af53d0897a1c~1..ba27dd8be821792
gives:

Commit message (Expand) Author  Age Files   Lines
*   zfs: merge OpenZFS master-9312e0fd1 Martin Matuska  4 days  40  
-248/+724
|\  
| * Update vendor/openzfs to 

Re: How do I know if my 13-stable has security patches?

[Warner Losh]

Before I get into the blow by blow (which can sound nit-picky, despite my
best efforts), I would like to apologize. It wasn't completely appreciated
how clearly the dependencies that the nX number being generated needed
to be communicated. And for that I apologize. When they are met, we have
all the pieces we need to at build time to generate the nX number and
none of the fallback methods are needed. I'll work to update the docs to
clearly communicate this since it is completely absent from my current docs.

On Thu, Feb 25, 2021 at 2:56 PM Karl Denninger  wrote:

> On 2/25/2021 15:56, Warner Losh wrote:
>
>
> On Thu, Feb 25, 2021 at 6:37 AM Karl Denninger  wrote:
>
>> On 2/25/2021 04:30, Olivier Certner wrote:
>> >> Neither command is what I'd call 'intuitive', so it would have taken
>> me a
>> >> long time to find either of them. I cut and pasted the 'git branch'
>> command
>> >> and it took me a moment to realize what that meant. Never ran "grep
>> -l" on
>> >> a pipe, I guess.
>> > You made me laugh! Apart from relatively simple commands, git's
>> interface is
>> > far from intuitive. That's the reason why I regret that it became the
>> hugely
>> > dominant DVCS.
>>
>> Regression doesn't have to come to a project, but if the tools you
>> choose do things like this then you have to work around them as a
>> project to avoid the issue, and that might wind up being somewhat of a
>> PITA.
>>
>> This specific issue is IMHO quite severe in terms of operational
>> impact.  I track -STABLE but don't load "new things" all the time.  For
>> security-related things it's more important to know if I've got
>> something out there in a specific instance where it may apply (and not
>> care in others where it doesn't; aka the recent Xen thing if you're not
>> using Xen.)  Otherwise if everything is running as it should do I wish
>> to risk introducing bugs along with improvements?  If not in a
>> security-related context, frequently not.
>>
>> Well, this used to be easy.  Is your "uname" r-number HIGHER than the
>> "when fixed" revision?  You're good.  Now, nope.  Now I have to go dig
>> source to know because there is no longer a "revision number" that
>> monotonically increments with each commit so there is no longer a way to
>> have a "point in time" view of the source, as-committed, for a given
>> checked-out version.
>>
>> IMHO that's a fairly serious regression for the person responsible for
>> keeping security-related things up to date and something the project
>> should find a way to fix before rolling the next -RELEASE. (Yeah, I know
>> that's almost-certain to not happen but it's not like this issue wasn't
>> known since moving things over to git.)
>>
>
> We should likely just publish the 'v' number in the advisories. It's
> basically a count back to the start of the project. We put that number in
> uname already.
>
> You can also  find out the 'v' number in the latest advisories by cloning
> the repo and doing the same thing we do in newvers.sh:
> % git rev-list --first-parent --count $HASH
> and that will tell you. This needn't be on the target machine since the
> hashes are stable across the world.
>
> (list of further "stuff")
>
> But that's my entire point Warner.
>
> The time (and present items) on a given machine to know whether it is
> covered by a given advisory under the "svn view of the world" is one
> command, and no sources.  That is, if the advisory says "r123456" has the
> fix, then if I do a "uname -v" and get something larger, it's safe.
>
> If I get something smaller it's not.
>
If you built from a full clone, then it's the same today. You can look at
the nX that's in the kernel string along with the branch and know if
you should upgrade or not.

However, if you built from a shallow clone, that's no longer possible. If
you need this functionality, you must build from sources that are from a
tree that has access to the full git repo. This detail was poorly
communicated, I'll be the first to admit. And to be fair, to get the
rXX number, however, you also needed to have a subversion metadata
around as well (it is much the same as having the full clone now).

And there's also the date which is added to the uname, if you didn't do a
reproducible build. If it is older than the security advisory, then you'll
need to update. And most ways that update the kernel preserve the build
time in the metadata for /boot/kernel/kernel if you did a reproducible
build (though this may not be true if you use makefs) . If the dates are
newer, though, you'll need to do the hash dance.


> I don't need the source on the machine, I don't need svn on the target or,
> for that matter, do I need to know if the source tree I have on a build
> machine is coherent with whatever is on the running machine.
>
Still don't need these things, completely. You just need the hash that's
reported. If you build from a pull of the full tree, you'll have the n
number and you have what you want. If you didn't, you'll 

Re: How do I know if my 13-stable has security patches?

[Tim Daneliuk]

On 2/25/21 3:56 PM, Karl Denninger wrote:
> On 2/25/2021 15:56, Warner Losh wrote:
>>

> 
> Unless I've missed something that's what was lost and IMHO needs to be 
> restored; a way to know that in seconds with nothing other than the operating 
> OS on the box (e.g. via uname) and the advisory with its "greater than X is 
> safe" from the mailing list.  Am I misunderstanding the current state of 
> things in this regard?
> 

One mechanism for doing this with git would be to use tags of some sort to
indicate which commits address which CVEs.  The problem with this is that
you still need a source tree.

I may be dense (I've certainly been told I am from time-to-time) but what's
wrong with this algo:

   - FreeBSD security team sends out notification of CVE and patches
 that address them AND _what date the patches went into the source tree_.

   - I do a 'uname -a' to see if my running system was built before- or after
 that date  (+- timezone variability, perhaps).

This does assume that people are pulling latest source for their branch prior 
to building.
This only addresses kernel fixes directly, however. A patch to, say, sshd would
not be reflected in the kernel build date.  But even there, it's kind of a hint.
If your instance of sshd is older than the patch date in question, you are for 
sure
not patched.  The uncertainty remains whether or not a file timestamped after 
the
patch date was build from the correct/new source.  But I would argue that this 
particular
problem also existed with kernel r notation.

git does get many things right, but this is an area that is kind of clunky.  I 
also
hate that it has no equivalent to $RCSID for me to embed in code and docs.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: How do I know if my 13-stable has security patches?

[George Mitchell]

On 2/25/21 3:56 PM, Warner Losh wrote:

[...]
We should likely just publish the 'v' number in the advisories. It's
basically a count back to the start of the project. We put that number in
uname already. []

+1 !!! -- George



OpenPGP_signature
Description: OpenPGP digital signature

Re: How do I know if my 13-stable has security patches?

[Karl Denninger]

On 2/25/2021 15:56, Warner Losh wrote:


On Thu, Feb 25, 2021 at 6:37 AM Karl Denninger > wrote:


On 2/25/2021 04:30, Olivier Certner wrote:
>> Neither command is what I'd call 'intuitive', so it would have
taken me a
>> long time to find either of them. I cut and pasted the 'git
branch' command
>> and it took me a moment to realize what that meant. Never ran
"grep -l" on
>> a pipe, I guess.
> You made me laugh! Apart from relatively simple commands, git's
interface is
> far from intuitive. That's the reason why I regret that it
became the hugely
> dominant DVCS.

Regression doesn't have to come to a project, but if the tools you
choose do things like this then you have to work around them as a
project to avoid the issue, and that might wind up being somewhat
of a PITA.

This specific issue is IMHO quite severe in terms of operational
impact.  I track -STABLE but don't load "new things" all the
time.  For
security-related things it's more important to know if I've got
something out there in a specific instance where it may apply (and
not
care in others where it doesn't; aka the recent Xen thing if
you're not
using Xen.)  Otherwise if everything is running as it should do I
wish
to risk introducing bugs along with improvements?  If not in a
security-related context, frequently not.

Well, this used to be easy.  Is your "uname" r-number HIGHER than the
"when fixed" revision?  You're good.  Now, nope.  Now I have to go
dig
source to know because there is no longer a "revision number" that
monotonically increments with each commit so there is no longer a
way to
have a "point in time" view of the source, as-committed, for a given
checked-out version.

IMHO that's a fairly serious regression for the person responsible
for
keeping security-related things up to date and something the project
should find a way to fix before rolling the next -RELEASE. (Yeah,
I know
that's almost-certain to not happen but it's not like this issue
wasn't
known since moving things over to git.)


We should likely just publish the 'v' number in the advisories. It's 
basically a count back to the start of the project. We put that number 
in uname already.


You can also  find out the 'v' number in the latest advisories by 
cloning the repo and doing the same thing we do in newvers.sh:

% git rev-list --first-parent --count $HASH
and that will tell you. This needn't be on the target machine since 
the hashes are stable across the world.


(list of further "stuff")

But that's my entire point Warner.

The time (and present items) on a given machine to know whether it is 
covered by a given advisory under the "svn view of the world" is one 
command, and no sources.  That is, if the advisory says "r123456" has 
the fix, then if I do a "uname -v" and get something larger, it's safe.


If I get something smaller it's not.

I don't need the source on the machine, I don't need svn on the target 
or, for that matter, do I need to know if the source tree I have on a 
build machine is coherent with whatever is on the running machine.  I 
simply need to know if the source that built the code that is running 
was updated *after* the commit that fixes the problem.  What if the 
source /isn't on that machine /because you build on some system and then 
distribute?  Does every machine now have to be coherent with your source 
repository in order to be able to figure out where you are or worse, it 
must keep the source from which that specific installation, 
individually, was built? /What if the source isn't there at all /because 
you run binary code and update with freebsd-update?


Unless I've missed something that's what was lost and IMHO needs to be 
restored; a way to know that in seconds with nothing other than the 
operating OS on the box (e.g. via uname) and the advisory with its 
"greater than X is safe" from the mailing list.  Am I misunderstanding 
the current state of things in this regard?


--
Karl Denninger
k...@denninger.net 
/The Market Ticker/
/[S/MIME encrypted email preferred]/


smime.p7s
Description: S/MIME Cryptographic Signature

Re: How do I know if my 13-stable has security patches?

[Warner Losh]

On Thu, Feb 25, 2021 at 6:37 AM Karl Denninger  wrote:

> On 2/25/2021 04:30, Olivier Certner wrote:
> >> Neither command is what I'd call 'intuitive', so it would have taken me
> a
> >> long time to find either of them. I cut and pasted the 'git branch'
> command
> >> and it took me a moment to realize what that meant. Never ran "grep -l"
> on
> >> a pipe, I guess.
> > You made me laugh! Apart from relatively simple commands, git's
> interface is
> > far from intuitive. That's the reason why I regret that it became the
> hugely
> > dominant DVCS.
>
> Regression doesn't have to come to a project, but if the tools you
> choose do things like this then you have to work around them as a
> project to avoid the issue, and that might wind up being somewhat of a
> PITA.
>
> This specific issue is IMHO quite severe in terms of operational
> impact.  I track -STABLE but don't load "new things" all the time.  For
> security-related things it's more important to know if I've got
> something out there in a specific instance where it may apply (and not
> care in others where it doesn't; aka the recent Xen thing if you're not
> using Xen.)  Otherwise if everything is running as it should do I wish
> to risk introducing bugs along with improvements?  If not in a
> security-related context, frequently not.
>
> Well, this used to be easy.  Is your "uname" r-number HIGHER than the
> "when fixed" revision?  You're good.  Now, nope.  Now I have to go dig
> source to know because there is no longer a "revision number" that
> monotonically increments with each commit so there is no longer a way to
> have a "point in time" view of the source, as-committed, for a given
> checked-out version.
>
> IMHO that's a fairly serious regression for the person responsible for
> keeping security-related things up to date and something the project
> should find a way to fix before rolling the next -RELEASE. (Yeah, I know
> that's almost-certain to not happen but it's not like this issue wasn't
> known since moving things over to git.)
>

We should likely just publish the 'v' number in the advisories. It's
basically a count back to the start of the project. We put that number in
uname already.

You can also  find out the 'v' number in the latest advisories by cloning
the repo and doing the same thing we do in newvers.sh:
% git rev-list --first-parent --count $HASH
and that will tell you. This needn't be on the target machine since the
hashes are stable across the world.

That's kinda the whole reason we did the 'v' number: to provide a stable,
monotonically increasing number that's unaffected by vendor merges (the
c number was affected by merges). If you have a 'c' number in your
uname the answer is super simple: you are affected.

The problem, though, can happen when you run a shallow clone or gitup to
get the sources and build from that. In that case the v number is bogus
(hmmm, we should omit it when we have a shallow clone maybe). In that case
you'll need to do the following on a clone somewhere (not necessarily on
the target machine):

% git log --max-count 10 --oneline  $UNAME_HASH | grep $ADVISORY_HASH

The other alternative: you can do a 'git fetch' to pull the new hashes
without doing a merge with what's on the machine. Then you can do

% git log --oneline stable/13..freebsd/stable/13 | grep $ADVISORY_HASH

and if you get a hit, you don't have the patch yet installed. The advantage
of this is that this is work you'll need to do eventually anyway. If you
don't have it, then a

% git merge --ff-only freebsd/stable/13

will pull it in. If it turns out you did have it in the history before the
shallow clone, then you can choose to update or not. If you choose to
update, do the merge. If you choose not, then do nothing. The next 'git
pull --ff-only' will do the right thing, as will repeating this process for
the next advisory. The only harm is a few extra bytes pulled and/or a few
extra compressed revs on the branch.

Of course, the above assumes that you're running the sources == system
binaries. If in doubt, substitute $UNAME_HASH for the bare stable/13 in the
above.

Warner
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: How do I know if my 13-stable has security patches?

[Kevin Oberman]

On Thu, Feb 25, 2021 at 6:10 AM Ed Maste  wrote:

> On Thu, 25 Feb 2021 at 02:42, Kevin Oberman  wrote:
> >
> > Thanks, Ed, but where do I find this? uname -a" gives me
> stable/13-007101f87. For a while I was seeing a hyphenated number prefixed
> with a 'c' and I had assumed that that number was the sequence.
>
> It is (was) - we changed from 'c' to avoid having it look like a hex value.
>
> To generate (this part of) uname the build script runs:
> if [ "$($git_cmd rev-parse --is-shallow-repository)" = false ] ;
> then
> git_cnt=$($git_cmd rev-list --first-parent --count
> HEAD 2>/dev/null)
> if [ -n "$git_cnt" ] ; then
> git="n${git_cnt}-${git}"
> fi
> fi
>
> Would you try running, at the top of your stable/13 src tree:
> git rev-parse --is-shallow-repository
> git rev-list --first-parent --count HEAD
>

I do run a shallow clone, as I suspect most non-developers will.
# git rev-parse --is-shallow-repository
true
# git rev-list --first-parent --count HEAD
133

As for an easy check for the presence of a patch by hash, I like the "git
log --pretty=oneline" | grep  

This assumes that the sources have NOT been updated since the system was
updated. If they have, you can do a "egrep -n ^" on both the partial
hash in "uname -a" and in the security announcement and see which is older
by line number.

My thanks to Jeremy Chadwick for this approach.
--
Kevin Oberman, Part time kid herder and retired Network Engineer
E-mail: rkober...@gmail.com
PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: How do I know if my 13-stable has security patches?

[Ed Maste]

On Thu, 25 Feb 2021 at 02:42, Kevin Oberman  wrote:
>
> Thanks, Ed, but where do I find this? uname -a" gives me stable/13-007101f87. 
> For a while I was seeing a hyphenated number prefixed with a 'c' and I had 
> assumed that that number was the sequence.

It is (was) - we changed from 'c' to avoid having it look like a hex value.

To generate (this part of) uname the build script runs:
if [ "$($git_cmd rev-parse --is-shallow-repository)" = false ] ; then
git_cnt=$($git_cmd rev-list --first-parent --count
HEAD 2>/dev/null)
if [ -n "$git_cnt" ] ; then
git="n${git_cnt}-${git}"
fi
fi

Would you try running, at the top of your stable/13 src tree:
git rev-parse --is-shallow-repository
git rev-list --first-parent --count HEAD
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: How do I know if my 13-stable has security patches?

[Karl Denninger]

On 2/25/2021 04:30, Olivier Certner wrote:

Neither command is what I'd call 'intuitive', so it would have taken me a
long time to find either of them. I cut and pasted the 'git branch' command
and it took me a moment to realize what that meant. Never ran "grep -l" on
a pipe, I guess.

You made me laugh! Apart from relatively simple commands, git's interface is
far from intuitive. That's the reason why I regret that it became the hugely
dominant DVCS.


Regression doesn't have to come to a project, but if the tools you 
choose do things like this then you have to work around them as a 
project to avoid the issue, and that might wind up being somewhat of a PITA.


This specific issue is IMHO quite severe in terms of operational 
impact.  I track -STABLE but don't load "new things" all the time.  For 
security-related things it's more important to know if I've got 
something out there in a specific instance where it may apply (and not 
care in others where it doesn't; aka the recent Xen thing if you're not 
using Xen.)  Otherwise if everything is running as it should do I wish 
to risk introducing bugs along with improvements?  If not in a 
security-related context, frequently not.


Well, this used to be easy.  Is your "uname" r-number HIGHER than the 
"when fixed" revision?  You're good.  Now, nope.  Now I have to go dig 
source to know because there is no longer a "revision number" that 
monotonically increments with each commit so there is no longer a way to 
have a "point in time" view of the source, as-committed, for a given 
checked-out version.


IMHO that's a fairly serious regression for the person responsible for 
keeping security-related things up to date and something the project 
should find a way to fix before rolling the next -RELEASE. (Yeah, I know 
that's almost-certain to not happen but it's not like this issue wasn't 
known since moving things over to git.)


--
Karl Denninger
k...@denninger.net 
/The Market Ticker/
/[S/MIME encrypted email preferred]/


smime.p7s
Description: S/MIME Cryptographic Signature

Re: How do I know if my 13-stable has security patches?

[Olivier Certner]

> Neither command is what I'd call 'intuitive', so it would have taken me a
> long time to find either of them. I cut and pasted the 'git branch' command
> and it took me a moment to realize what that meant. Never ran "grep -l" on
> a pipe, I guess.

You made me laugh! Apart from relatively simple commands, git's interface is 
far from intuitive. That's the reason why I regret that it became the hugely 
dominant DVCS.

Sure, arguably most of the complexity comes from the DAG of commits, and the 
need to distinguish what is local and what is remote, which are not specific 
to git at all. But its interface makes things unnecessarily more complicated. 
It seems it grew up from ad-hoc pieces without much planification and it took 
a long time before it finally started to get more attention. A typical case 
were the cathedral would have been much better than the bazaar: Research for a 
minimal set of concepts required to have it work and model commands after it. 
Yes, it's not trivial and it takes time, but is usually much better in the 
end.

But it's progressing somehow. Some time ago, 'git restore ' appeared, to 
replace the stance 'git checkout -- '. It seems they are also trying to 
do something with new, hopefully more natural, commands, such as 'git switch' 
to switch branches. I guess this will benefit the next generations of us, 
since we have to deal with what's available and works in the meantime (and 
habits). ;-)

And while I'm thinking about it: There is also 'git cherry' to check if some 
changes are in a branch. It doesn't match on the commit hashes but on the file 
contents. This is not necessary here because SAs specify the relevant commits 
per branches, but can be very useful to check that some changes were actually 
cherry-picked in another branch.

-- 
Olivier Certner


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: How do I know if my 13-stable has security patches?

[Kevin Oberman]

Thanks, Ed, but where do I find this? uname -a" gives me
stable/13-007101f87. For a while I was seeing a hyphenated number prefixed
with a 'c' and I had assumed that that number was the sequence. The full
hash from the logs just is a long hex number. As usual with git stuff, I'm
still very confused. After decades with a common paradigm with RCS CVS and
SVN, git is fundamentally very different and old terminology does not
really align as git is designed from a  very different perspective.

I have read the little mini-guide Warner wrote as well as a couple of web
tutorial, but the web tutorials are really about running your own repo on
github or gitlab, not using a repo as a source for distributions. I'm still
a long way from having a real clue.
--
Kevin Oberman, Part time kid herder and retired Network Engineer
E-mail: rkober...@gmail.com
PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683


On Wed, Feb 24, 2021 at 12:06 PM Ed Maste  wrote:

> On Wed, 24 Feb 2021 at 12:35, Kevin Oberman  wrote:
> >
> > In the svn days, I could just look at my svn revision to check on
> whether a
> > security patch was required. Now I have a git hash. I have no idea how to
> > tell if my system running 13-STABLE of a few days ago has the patch.
>
> Thanks for posting this question. I see some useful information in
> other replies to this thread and we'll want to make sure that makes
> its way to appropriate documentation.
>
> For future advisories we should also report the commit count
> associated with the fix; this is a monotonically-increasing number and
> is reported in the uname.
>
> If you build stable/13 right now you would get
> "stable/13-n244668-4664afc05402", and the fix in
> 894360bacd42f021551f76518edd445f6d299f2e corresponds to n244572.
> 244668 being larger than 244572 indicates that the fix is included.
>
> These counts are not unique across different branches; you can only
> compare counts for the same branch.
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: How do I know if my 13-stable has security patches?

[Ed Maste]

On Wed, 24 Feb 2021 at 12:35, Kevin Oberman  wrote:
>
> In the svn days, I could just look at my svn revision to check on whether a
> security patch was required. Now I have a git hash. I have no idea how to
> tell if my system running 13-STABLE of a few days ago has the patch.

Thanks for posting this question. I see some useful information in
other replies to this thread and we'll want to make sure that makes
its way to appropriate documentation.

For future advisories we should also report the commit count
associated with the fix; this is a monotonically-increasing number and
is reported in the uname.

If you build stable/13 right now you would get
"stable/13-n244668-4664afc05402", and the fix in
894360bacd42f021551f76518edd445f6d299f2e corresponds to n244572.
244668 being larger than 244572 indicates that the fix is included.

These counts are not unique across different branches; you can only
compare counts for the same branch.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: How do I know if my 13-stable has security patches?

[Kevin Oberman]

Thanks, Olivier, for the quick response. Now I don't have to do a system
build!

Neither command is what I'd call 'intuitive', so it would have taken me a
long time to find either of them. I cut and pasted the 'git branch' command
and it took me a moment to realize what that meant. Never ran "grep -l" on
a pipe, I guess.
--
Kevin Oberman, Part time kid herder and retired Network Engineer
E-mail: rkober...@gmail.com
PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683


On Wed, Feb 24, 2021 at 10:06 AM Olivier Certner 
wrote:

> Hi,
>
> In your base git repository, type:
> git rev-list  | grep -lF 
>
> This outputs something ("(standard input)") iff you have it in.
>
> In order to limit the search time in case of a false result, you'd better
> pass
> the --since= to git rev-list.
>
> There is an alternative if you have a branch pointing to your uname hash:
> git branch --contains  | grep -lF 
>
> Regards.
>
> --
> Olivier Certner
>
>
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: How do I know if my 13-stable has security patches?

[Olivier Certner]

Hi,

In your base git repository, type:
git rev-list  | grep -lF 

This outputs something ("(standard input)") iff you have it in.

In order to limit the search time in case of a false result, you'd better pass 
the --since= to git rev-list.

There is an alternative if you have a branch pointing to your uname hash:
git branch --contains  | grep -lF 

Regards.

-- 
Olivier Certner


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

How do I know if my 13-stable has security patches?

[Kevin Oberman]

In the svn days, I could just look at my svn revision to check on whether a
security patch was required. Now I have a git hash. I have no idea how to
tell if my system running 13-STABLE of a few days ago has the patch.

Branch/path  Revision
- -
stable/13/   894360bacd42f021551f76518edd445f6d299f2e
releng/13.0/ 9f00cb5fa8a438e7b9efb2158f2e2edc730badd1
stable/12/r369312
releng/12.2/  r369353

Is there a git command that can confirm whether a given hash is covered in
my system?
--
Kevin Oberman, Part time kid herder and retired Network Engineer
E-mail: rkober...@gmail.com
PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"