Re: [RFC] Proposal adding new sorting algorithm, bsort() to libc

[Hans Petter Selasky]

On 9/8/22 12:50, Hans Petter Selasky wrote:

See:
https://reviews.freebsd.org/D36493

Looking through base I see qsort() being used in places it shouldn't be 
used. For example in fts_open().


If for example you fill a directory with 64k simply numerical file names 
in the wrong order and ask fts_open() to sort these ascending for 
example, qsort() will end stuck for a long-long time. So either switch 
to mergesort, or if malloc() is unacceptable, use something like bsort() 
which I've implemented in the above review as a drop-in replacement for 
qsort(). The advantage with bsort() is that in can be CPU accelerated, 
due to fixed comparison patterns.


Quick sort is not always a quick sorting algorithm. Quick means simple, 
and not clever this time.


For the qsort's bad pattern, sorting 4096 entries 1024 times in a row took:

qsort: 15 seconds
bsort: 230 milliseconds (non-CPU accelerated)
mergesort: 30 milliseconds

The problem with qsort() is that as the array size grows, the time 
consumption just gets worse and worse for the bad-patterns.


Sorry there is no nice and fancy paper yet about this.

--HPS



Hi,

Also see the "kill_ls.c" test program I attached to D36493, which shows 
the direct affect of qsort() on the regular "ls" command!


--HPS

Re: A proposal

[Joe Nosay]

On Sat, Mar 29, 2014 at 12:14 PM, Allan Jude free...@allanjude.com wrote:

 On 2014-03-29 09:21, Joe Nosay wrote:
  On Sat, Mar 29, 2014 at 9:11 AM, Steve Kargl 
  s...@troutmask.apl.washington.edu wrote:
 
  On Sat, Mar 29, 2014 at 10:29:23AM +0100, Lars Engels wrote:
  On Fri, Mar 28, 2014 at 10:52:49PM -0700, Steve Kargl wrote:
  On Sat, Mar 29, 2014 at 01:46:15AM -0400, Joe Nosay wrote:
  On Sat, Mar 29, 2014 at 1:43 AM, Allan Jude free...@allanjude.com
  wrote:
 
  On 2014-03-29 01:22, Joe Nosay wrote:
  I have noticed that options VPS, VIMAGE, and MROUTING are not
  standard
  for
  the kernel with a base install. Is there any way that these can be
  made a
  part of the normal kernel so that jail(s) would get the full
 benefit
  without a kernel recompile?
 
 
  VIMAGE has some problems with PF. Martin Matu ska said at AsiaBSDCon
  that he had a patch that solves the issue.
 
  Really, you say? That's good news.
  So, will those options soon be standard for a first time install?
 
  Certainly, hope not.  I don't use any of options.
 
  So you use all the other devices and options of the GENERIC kernel?
 
  Of course, not.  Not sure how you inferred such a thing.
 
  IMHO, GENERIC should contain only those devices and options
  that are required to get FreeBSD booted on new hardware.
  VIMAGE and MROUTING aren't needed, and can be configured by
  the user after installation.  As for VPS, AFAICT, there isn't
  an option/device named VPS; at least 'find /sys/ -type f | xargs
  grep VPS' wasn't too enlightening.
 
 
  --
  Steve
 
 
 
  http://www.7he.at/freebsd/vps/announcements/
 
 
  Booyah. It exists.
  ___
  freebsd-current@freebsd.org mailing list
  http://lists.freebsd.org/mailman/listinfo/freebsd-current
  To unsubscribe, send any mail to 
 freebsd-current-unsubscr...@freebsd.org
 

 VPS only exists in a project branch, and is not nearly in a state to be
 included in GENERIC

 And I don't see a compelling reason to have MROUTING in GENERIC either.

 --
 Allan Jude



VIMAGE?
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: A proposal

[KIRIYAMA Kazuhiko]

Hi, Allan

At Sat, 29 Mar 2014 01:43:11 -0400,
Allan Jude wrote:
 
 On 2014-03-29 01:22, Joe Nosay wrote:
  I have noticed that options VPS, VIMAGE, and MROUTING are not standard for
  the kernel with a base install. Is there any way that these can be made a
  part of the normal kernel so that jail(s) would get the full benefit
  without a kernel recompile?
  ___
  freebsd-current@freebsd.org mailing list
  http://lists.freebsd.org/mailman/listinfo/freebsd-current
  To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
  
 
 VIMAGE has some problems with PF. Martin Matuska said at AsiaBSDCon
 that he had a patch that solves the issue.

Oh really? Is there any site above patch? The patch is not [1] is it?

# I'm too plagued with rebooting VIMAGE+pf server every midnight ;-(

[1] http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/160496

 
 -- 
 Allan Jude
 

---
Kazuhiko Kiriyama
k...@openedu.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: A proposal

[Lars Engels]

On Fri, Mar 28, 2014 at 10:52:49PM -0700, Steve Kargl wrote:
 On Sat, Mar 29, 2014 at 01:46:15AM -0400, Joe Nosay wrote:
  On Sat, Mar 29, 2014 at 1:43 AM, Allan Jude free...@allanjude.com wrote:
  
   On 2014-03-29 01:22, Joe Nosay wrote:
I have noticed that options VPS, VIMAGE, and MROUTING are not standard
   for
the kernel with a base install. Is there any way that these can be made 
a
part of the normal kernel so that jail(s) would get the full benefit
without a kernel recompile?
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
   freebsd-current-unsubscr...@freebsd.org
   
  
   VIMAGE has some problems with PF. Martin Matu ska said at AsiaBSDCon
   that he had a patch that solves the issue.
  
  Really, you say? That's good news.
  So, will those options soon be standard for a first time install?
 
 Certainly, hope not.  I don't use any of options.

So you use all the other devices and options of the GENERIC kernel?


pgpDFWhHRjTdf.pgp
Description: PGP signature

Re: A proposal

[Joe Nosay]

On Sat, Mar 29, 2014 at 5:29 AM, Lars Engels lars.eng...@0x20.net wrote:

 On Fri, Mar 28, 2014 at 10:52:49PM -0700, Steve Kargl wrote:
  On Sat, Mar 29, 2014 at 01:46:15AM -0400, Joe Nosay wrote:
   On Sat, Mar 29, 2014 at 1:43 AM, Allan Jude free...@allanjude.com
 wrote:
  
On 2014-03-29 01:22, Joe Nosay wrote:
 I have noticed that options VPS, VIMAGE, and MROUTING are not
 standard
for
 the kernel with a base install. Is there any way that these can be
 made a
 part of the normal kernel so that jail(s) would get the full
 benefit
 without a kernel recompile?
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to 
freebsd-current-unsubscr...@freebsd.org

   
VIMAGE has some problems with PF. Martin Matu ska said at AsiaBSDCon
that he had a patch that solves the issue.
  
   Really, you say? That's good news.
   So, will those options soon be standard for a first time install?
 
  Certainly, hope not.  I don't use any of options.

 So you use all the other devices and options of the GENERIC kernel?



I also want to know where this patch is.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: A proposal

[Steve Kargl]

On Sat, Mar 29, 2014 at 10:29:23AM +0100, Lars Engels wrote:
 On Fri, Mar 28, 2014 at 10:52:49PM -0700, Steve Kargl wrote:
 On Sat, Mar 29, 2014 at 01:46:15AM -0400, Joe Nosay wrote:
 On Sat, Mar 29, 2014 at 1:43 AM, Allan Jude free...@allanjude.com wrote:
 
 On 2014-03-29 01:22, Joe Nosay wrote:
 I have noticed that options VPS, VIMAGE, and MROUTING are not standard
 for
 the kernel with a base install. Is there any way that these can be made a
 part of the normal kernel so that jail(s) would get the full benefit
 without a kernel recompile?


 VIMAGE has some problems with PF. Martin Matu ska said at AsiaBSDCon
 that he had a patch that solves the issue.
 
 Really, you say? That's good news.
 So, will those options soon be standard for a first time install?
 
 Certainly, hope not.  I don't use any of options.
 
 So you use all the other devices and options of the GENERIC kernel?

Of course, not.  Not sure how you inferred such a thing.

IMHO, GENERIC should contain only those devices and options
that are required to get FreeBSD booted on new hardware.
VIMAGE and MROUTING aren't needed, and can be configured by
the user after installation.  As for VPS, AFAICT, there isn't
an option/device named VPS; at least 'find /sys/ -type f | xargs
grep VPS' wasn't too enlightening.


-- 
Steve
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: A proposal

[Joe Nosay]

On Sat, Mar 29, 2014 at 9:11 AM, Steve Kargl 
s...@troutmask.apl.washington.edu wrote:

 On Sat, Mar 29, 2014 at 10:29:23AM +0100, Lars Engels wrote:
  On Fri, Mar 28, 2014 at 10:52:49PM -0700, Steve Kargl wrote:
  On Sat, Mar 29, 2014 at 01:46:15AM -0400, Joe Nosay wrote:
  On Sat, Mar 29, 2014 at 1:43 AM, Allan Jude free...@allanjude.com
 wrote:
 
  On 2014-03-29 01:22, Joe Nosay wrote:
  I have noticed that options VPS, VIMAGE, and MROUTING are not
 standard
  for
  the kernel with a base install. Is there any way that these can be
 made a
  part of the normal kernel so that jail(s) would get the full benefit
  without a kernel recompile?
 
 
  VIMAGE has some problems with PF. Martin Matu ska said at AsiaBSDCon
  that he had a patch that solves the issue.
 
  Really, you say? That's good news.
  So, will those options soon be standard for a first time install?
 
  Certainly, hope not.  I don't use any of options.
 
  So you use all the other devices and options of the GENERIC kernel?

 Of course, not.  Not sure how you inferred such a thing.

 IMHO, GENERIC should contain only those devices and options
 that are required to get FreeBSD booted on new hardware.
 VIMAGE and MROUTING aren't needed, and can be configured by
 the user after installation.  As for VPS, AFAICT, there isn't
 an option/device named VPS; at least 'find /sys/ -type f | xargs
 grep VPS' wasn't too enlightening.


 --
 Steve



http://www.7he.at/freebsd/vps/announcements/


Booyah. It exists.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: A proposal

[Allan Jude]

On 2014-03-29 09:21, Joe Nosay wrote:
 On Sat, Mar 29, 2014 at 9:11 AM, Steve Kargl 
 s...@troutmask.apl.washington.edu wrote:
 
 On Sat, Mar 29, 2014 at 10:29:23AM +0100, Lars Engels wrote:
 On Fri, Mar 28, 2014 at 10:52:49PM -0700, Steve Kargl wrote:
 On Sat, Mar 29, 2014 at 01:46:15AM -0400, Joe Nosay wrote:
 On Sat, Mar 29, 2014 at 1:43 AM, Allan Jude free...@allanjude.com
 wrote:

 On 2014-03-29 01:22, Joe Nosay wrote:
 I have noticed that options VPS, VIMAGE, and MROUTING are not
 standard
 for
 the kernel with a base install. Is there any way that these can be
 made a
 part of the normal kernel so that jail(s) would get the full benefit
 without a kernel recompile?


 VIMAGE has some problems with PF. Martin Matu ska said at AsiaBSDCon
 that he had a patch that solves the issue.

 Really, you say? That's good news.
 So, will those options soon be standard for a first time install?

 Certainly, hope not.  I don't use any of options.

 So you use all the other devices and options of the GENERIC kernel?

 Of course, not.  Not sure how you inferred such a thing.

 IMHO, GENERIC should contain only those devices and options
 that are required to get FreeBSD booted on new hardware.
 VIMAGE and MROUTING aren't needed, and can be configured by
 the user after installation.  As for VPS, AFAICT, there isn't
 an option/device named VPS; at least 'find /sys/ -type f | xargs
 grep VPS' wasn't too enlightening.


 --
 Steve

 
 
 http://www.7he.at/freebsd/vps/announcements/
 
 
 Booyah. It exists.
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
 

VPS only exists in a project branch, and is not nearly in a state to be
included in GENERIC

And I don't see a compelling reason to have MROUTING in GENERIC either.

-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

Re: A proposal

[Lars Engels]

On Sat, Mar 29, 2014 at 06:11:10AM -0700, Steve Kargl wrote:
 On Sat, Mar 29, 2014 at 10:29:23AM +0100, Lars Engels wrote:
  On Fri, Mar 28, 2014 at 10:52:49PM -0700, Steve Kargl wrote:
  On Sat, Mar 29, 2014 at 01:46:15AM -0400, Joe Nosay wrote:
  On Sat, Mar 29, 2014 at 1:43 AM, Allan Jude free...@allanjude.com wrote:
  
  On 2014-03-29 01:22, Joe Nosay wrote:
  I have noticed that options VPS, VIMAGE, and MROUTING are not standard
  for
  the kernel with a base install. Is there any way that these can be made 
  a
  part of the normal kernel so that jail(s) would get the full benefit
  without a kernel recompile?
 
 
  VIMAGE has some problems with PF. Martin Matu ska said at AsiaBSDCon
  that he had a patch that solves the issue.
  
  Really, you say? That's good news.
  So, will those options soon be standard for a first time install?
  
  Certainly, hope not.  I don't use any of options.
  
  So you use all the other devices and options of the GENERIC kernel?
 
 Of course, not.  Not sure how you inferred such a thing.
 
 IMHO, GENERIC should contain only those devices and options
 that are required to get FreeBSD booted on new hardware.
 VIMAGE and MROUTING aren't needed, and can be configured by
 the user after installation.  As for VPS, AFAICT, there isn't
 an option/device named VPS; at least 'find /sys/ -type f | xargs
 grep VPS' wasn't too enlightening.

IMHO common use scenarios should be included in GENERIC if they're
stable and mature. E.g. I don't know why IPSEC still isn't in GENERIC.


pgpbdbYRYczJw.pgp
Description: PGP signature

Re: A proposal

[Gary Palmer]

On Sat, Mar 29, 2014 at 06:11:10AM -0700, Steve Kargl wrote:
 On Sat, Mar 29, 2014 at 10:29:23AM +0100, Lars Engels wrote:
  On Fri, Mar 28, 2014 at 10:52:49PM -0700, Steve Kargl wrote:
  On Sat, Mar 29, 2014 at 01:46:15AM -0400, Joe Nosay wrote:
  On Sat, Mar 29, 2014 at 1:43 AM, Allan Jude free...@allanjude.com wrote:
  
  On 2014-03-29 01:22, Joe Nosay wrote:
  I have noticed that options VPS, VIMAGE, and MROUTING are not standard
  for
  the kernel with a base install. Is there any way that these can be made 
  a
  part of the normal kernel so that jail(s) would get the full benefit
  without a kernel recompile?
 
 
  VIMAGE has some problems with PF. Martin Matu ska said at AsiaBSDCon
  that he had a patch that solves the issue.
  
  Really, you say? That's good news.
  So, will those options soon be standard for a first time install?
  
  Certainly, hope not.  I don't use any of options.
  
  So you use all the other devices and options of the GENERIC kernel?
 
 Of course, not.  Not sure how you inferred such a thing.
 
 IMHO, GENERIC should contain only those devices and options
 that are required to get FreeBSD booted on new hardware.
 VIMAGE and MROUTING aren't needed, and can be configured by
 the user after installation.  As for VPS, AFAICT, there isn't
 an option/device named VPS; at least 'find /sys/ -type f | xargs
 grep VPS' wasn't too enlightening.

Until freebsd-update deals with custom kernel configurations, people will
want all the possible non-conflicting flags enabled in GENERIC, or the
options made available as runtime loadable kernel modules.  It's not
an unreasonable request.

Regards,

Gary
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: A proposal

[Julian Elischer]

On 3/29/14, 4:52 PM, Steve Kargl wrote:

On Sat, Mar 29, 2014 at 01:46:15AM -0400, Joe Nosay wrote:

On Sat, Mar 29, 2014 at 1:43 AM, Allan Jude free...@allanjude.com wrote:


On 2014-03-29 01:22, Joe Nosay wrote:

I have noticed that options VPS, VIMAGE, and MROUTING are not standard

for

the kernel with a base install. Is there any way that these can be made a
part of the normal kernel so that jail(s) would get the full benefit
without a kernel recompile?
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 

freebsd-current-unsubscr...@freebsd.org
VIMAGE has some problems with PF. Martin Matu ska said at AsiaBSDCon
that he had a patch that solves the issue.

Really, you say? That's good news.
So, will those options soon be standard for a first time install?

Certainly, hope not.  I don't use any of options.
funnily, the only profiling speed differences we've seen with enabling 
netgraph is speedups :-)
especially with comparing many sessions on one vnet with the same 
number of sessions on several vnets.


Last time I tested it, putting several services on different vimage 
jails and assigning them different interfaces actually ran faster than 
having the same services on the same VM (machine).

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

RE: A proposal

[dteske]

 -Original Message-
 From: Julian Elischer [mailto:jul...@freebsd.org]
 Sent: Saturday, March 29, 2014 3:41 PM
 To: Steve Kargl; Joe Nosay
 Cc: freebsd-current
 Subject: Re: A proposal
 
 On 3/29/14, 4:52 PM, Steve Kargl wrote:
  On Sat, Mar 29, 2014 at 01:46:15AM -0400, Joe Nosay wrote:
  On Sat, Mar 29, 2014 at 1:43 AM, Allan Jude free...@allanjude.com
 wrote:
 
  On 2014-03-29 01:22, Joe Nosay wrote:
  I have noticed that options VPS, VIMAGE, and MROUTING are not
  standard
  for
  the kernel with a base install. Is there any way that these can be
  made a part of the normal kernel so that jail(s) would get the full
  benefit without a kernel recompile?
  ___
  freebsd-current@freebsd.org mailing list
  http://lists.freebsd.org/mailman/listinfo/freebsd-current
  To unsubscribe, send any mail to 
  freebsd-current-unsubscr...@freebsd.org
  VIMAGE has some problems with PF. Martin Matu ska said at
 AsiaBSDCon
  that he had a patch that solves the issue.
  Really, you say? That's good news.
  So, will those options soon be standard for a first time install?
  Certainly, hope not.  I don't use any of options.
 funnily, the only profiling speed differences we've seen with enabling
 netgraph is speedups :-) especially with comparing many sessions on one
 vnet with the same number of sessions on several vnets.
 
 Last time I tested it, putting several services on different vimage jails
and
 assigning them different interfaces actually ran faster than having the
same
 services on the same VM (machine).

I think we all owe this one gentleman a great show of gratitude for
his work on netgraph (initially @ whistle)
-- 
Devin (smiles)

_
The information contained in this message is proprietary and/or confidential. 
If you are not the intended recipient, please: (i) delete the message and all 
copies; (ii) do not disclose, distribute or use the message in any manner; and 
(iii) notify the sender immediately. In addition, please be aware that any 
message addressed to our domain is subject to archiving and review by persons 
other than the intended recipient. Thank you.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: A proposal

[Allan Jude]

On 2014-03-29 01:22, Joe Nosay wrote:
 I have noticed that options VPS, VIMAGE, and MROUTING are not standard for
 the kernel with a base install. Is there any way that these can be made a
 part of the normal kernel so that jail(s) would get the full benefit
 without a kernel recompile?
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
 

VIMAGE has some problems with PF. Martin Matuska said at AsiaBSDCon
that he had a patch that solves the issue.

-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

Re: A proposal

[Joe Nosay]

On Sat, Mar 29, 2014 at 1:43 AM, Allan Jude free...@allanjude.com wrote:

 On 2014-03-29 01:22, Joe Nosay wrote:
  I have noticed that options VPS, VIMAGE, and MROUTING are not standard
 for
  the kernel with a base install. Is there any way that these can be made a
  part of the normal kernel so that jail(s) would get the full benefit
  without a kernel recompile?
  ___
  freebsd-current@freebsd.org mailing list
  http://lists.freebsd.org/mailman/listinfo/freebsd-current
  To unsubscribe, send any mail to 
 freebsd-current-unsubscr...@freebsd.org
 

 VIMAGE has some problems with PF. Martin Matu ska said at AsiaBSDCon
 that he had a patch that solves the issue.

 --
 Allan Jude



Really, you say? That's good news.
So, will those options soon be standard for a first time install?
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: A proposal

[Steve Kargl]

On Sat, Mar 29, 2014 at 01:46:15AM -0400, Joe Nosay wrote:
 On Sat, Mar 29, 2014 at 1:43 AM, Allan Jude free...@allanjude.com wrote:
 
  On 2014-03-29 01:22, Joe Nosay wrote:
   I have noticed that options VPS, VIMAGE, and MROUTING are not standard
  for
   the kernel with a base install. Is there any way that these can be made a
   part of the normal kernel so that jail(s) would get the full benefit
   without a kernel recompile?
   ___
   freebsd-current@freebsd.org mailing list
   http://lists.freebsd.org/mailman/listinfo/freebsd-current
   To unsubscribe, send any mail to 
  freebsd-current-unsubscr...@freebsd.org
  
 
  VIMAGE has some problems with PF. Martin Matu ska said at AsiaBSDCon
  that he had a patch that solves the issue.
 
 Really, you say? That's good news.
 So, will those options soon be standard for a first time install?

Certainly, hope not.  I don't use any of options.

-- 
Steve
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[John-Mark Gurney]

Warner Losh wrote this message on Fri, Mar 07, 2014 at 22:30 -0700:
 On Mar 7, 2014, at 10:22 PM, Allan Jude free...@allanjude.com wrote:
  Performance for default, sha512 w/ 5k rounds:
  AMD A10-5700 3.4GHz3.8ms
  AMD Opteron 4228 HE 2.8Ghz 5.4ms
  Intel(R) Xeon(R) X5650 2.67GHz 4.0ms
  
  these times are aprox as the timing varies quite a bit, ~+/-10%?
 
 And what would that be on a RPi or other embedded device?

Ok, here you go, IXP425 533MHz is ~1465ms..  This is a fast AVILA
board, I have a slower 266MHz AVILA board next to me...  Yes, that is
1.5 seconds at the default number of rounds for sha512 which is now the
default for passwords...

For comparision, md5 is 44.5ms and sha256 is 405ms on the AVILA...

So, by making sha512, we just killed the performance of embedded
systems...  This is also with the default of 5000 rounds...

So, the autoscaling could either help on embedded because we let the
number of rounds drop below the default of 5k, or it stays the same, so,
no hit on embedded...

 And do the extra route have a peer-reviewed paper showing the increased 
 strength?

Well, if it doesn't increase the strength, then we might as well drop
the rounds down to 1000 (the min per spec)...  since clearly if
increasing rounds pass 5k doesn't increase strength, then the same can
be said for 1k...

As for papers, I don't think anyone wrote a peer-reviewed paper saying
that crypt-sha{256,512} is secure...

Plus, they clearly thought that changing the rounds would be helpful,
so, they added it as an option, well, actually, Drepper just copied
Sun for making rounds an option...

Per Drepper:
The more rounds are performed the higher the CPU
requirements are.  This is a safety mechanism which might help
countering brute-force attacks in the face of increasing computing
power.

Notice the might...

http://www.akkadia.org/drepper/SHA-crypt.txt

  One possible solution would be just setting the default login.conf
  number of rounds, based on a test in the installer. Although this won't
  help for systems that are deployed by imaging, or VM images (like EC2
  images) etc.

Does CPU time measuring work properly on VM's?  i.e. if I do a cpu
intesive task and measure it with getrusage, do I get how much I really
ran for?  By my understanding, you can't, since often the VM isn't aware
of the parent, so doesn't know when to stop the clock when it isn't
running...

Unless I'm missing something, you really can't do any cpu or profiling
on a VM and trust the results...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[Derek (freebsd lists)]

Hi all,

Thanks for your attention to the matter/threads.  I have thought 
a bit about this, and I hope I can add some value to the current 
conversation, below:


On 03/07/2014 07:36 PM, Xin Li wrote:

On 03/07/14 14:50, A.J. Kehoe IV (Nanoman) wrote:

Xin Li wrote:

On 03/07/14 13:52, A.J. Kehoe IV (Nanoman) wrote:

Allan Jude wrote:

On 2014-03-07 11:13, A.J. Kehoe IV (Nanoman) wrote:

Allan Jude wrote:

[...]


Honestly, my use case is just silently upgrading the
strength of the hashing algorithm (when combined with my
other feature request). Updating my bcrypt hashes from
$2a$04$ to $2b$12$ or something. Same applies for the
default sha512, maybe I want to update to rounds=15000


Like this?

http://www.freebsd.org/cgi/query-pr.cgi?pr=182518

Request for comments:

http://docs.freebsd.org/cgi/mid.cgi?20140106205156.GD4903


[...]


Speaking for adding rounds, the only problem that needs to be
fixed is that the proposed patch makes it possible to create
conflicting configuration (passwd_format and passwd_modular can
use different hashing algorithms) and need to be fixed and
polished.  I like the idea of making it possible to use more
rounds though.


This was deliberate for backward compatibility.  passwd_format will
be used by default if passwd_modular isn't defined.  If
passwd_modular is defined as disabled, then passwd_format will be
used.


Well, my point is that the two shouldn't be allowed to exist together
if they can mean something conflicting.  Allowing passwd_format=sha512
AND passwd_modular=$2a$08$ in the same configuration creates confusion
and it's not good.



Agreed.  My original intention was to create a patch that didn't 
touch a lot of code.


My reasons for this were first to see if there was any interest 
from a committer to take this further.  Much more likely to have 
a 15 or so line patch looked at, than one that touches stuff all 
over the place - I think.


We are now at least having a conversation about it.

It seemed to be a lot of work to specify rounds via 
login_setcryptfmt, with a bunch of changes also required in 
libcrypt.


I don't have the resources to test for regressions in libcrypt, 
beyond the scope of whether login.conf works as expected 
(specifically, the ports tree, yp, ldap, or any other areas that 
I don't know about).


If other developers were willing to work together on the api/abi 
changes, I would feel a lot better about spending my time there 
and doing it right.  Without support from other, more 
knowledgeable people (as far as what will break if we do XYZ), 
who will eventually merge productive changes, I would be wasting 
my time.


I don't want to be the libcrypt api changing pixie, scattering 
patches into /dev/null. :)



My suggestion is that we either have:

a) passwd_format and passwd_round (so that they don't conflict), or



I recommend against this.  By example, based on current scrypt 
modular crypt RFCs, there are multiple tunable parameters.  It's 
conceivable that other future algorithms will have different 
functional and named parameters.


Additionally, I think having all the parsing code for this 
scattered about actually makes things less clear.  For example, 
$2a$08$ means a lot more to people (across different *nix 
backgrounds) than blf, is concise, and is/already should be well 
documented in crypt(3). Likewise with sha512.  Looking at 
login.conf, you can't tell exactly what it means.


Modular crypt is something that developers are working to stay 
compatible with (e.g. $5$, $6$, $2y$, etc), is understood outside 
of the context of FreeBSD system administration, and would be 
understood by people who are knowledgeable enough to seek to 
change this aspect of their system.



b) extend passwd_format in a compatible manner to allow specifying a
round, or,

c) make passwd_format and passwd_modular conflict so we don't silently
accept it and instead bail out when doing pwd_mkdb.



As jmg suggested, by supplying the modular format for 
passwd_format, we eliminate this conflict, and make it obvious. 
I definitely support this notion.


That means touching login_setcryptfmt and friends, I think.


What do you think of the idea of putting this into libcrypt instead
of pam_unix.c, and then patching pam_unix.c and pw_user.c to
reference libcrypt?


Which part of the idea?  I think it's a bad idea to make libcrypt to
depend on libutil (for login_cap(3)) but we should probably provide
new wrappers in login_cap(3) to do the common things when requested
for various password manipulating tools to reduce duplicated code.



Specifically:

The makesalt aspect can/should be put into libcrypt, refined 
appropriately, and exposed publicly.  It is a terrible little 
piece of code as it is now, twice (or more!), and it could be 
cleaned up considerably.  This could be a nice little api.


Secondly, since the digests are used externally, I think it would 
be good to push the custom base64 code out to a library 
somewhere, so there is the 

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[John Baldwin]

On Wednesday, March 05, 2014 3:09:30 pm Matthew Rezny wrote:
   Password expiry is an orthogonal issue and should be up to administrator
  
  policy.
  
  Yes, but if you are moving to a different algorithm to improve security, not
  coupling it with an eventual expiration of non-migrated accounts gives a
  false sense of security.  Any admin worth his/her salt is going to want the
  option of enforcing that sort of policy along with the transparent update. 
  They should really be implemented together is all.
 
 Account expiration and password expiration are already present. There is 
 absolutely no reason that password algorithm upgrade should be tied in any 
 way 
 to expiration. A transparent algorithm upgrade as proposed is *far* better 
 than the forced password change method that is commonly employed. If the 
 administrator wants to force all accounts to migrate by a set deadline, we 
 already have the expiration facilities in place to accomplish that. Expiring 
 accounts that have not been used in a long time, regardless of algorithm 
 changes, should be part of general housekeeping and may be covered by 
 existing 
 policy. Password expiration serves no purpose, EVER. Password expiration 
 encourages users to choose bad passwords because they are throwaway items.
 
 Bruce states it well enough I need not elaborate further
 https://www.schneier.com/blog/archives/2010/11/changing_passwo.html
 
 Anyone who fails to understand the above should NOT be an administrator.

I think you failed to understand my point.  I am assuming that an administrator
wants the transparent upgrade (which I think is useful) because they are
assuming that the hash algorithm is compromised or inferior.  To that end,
they may wish to limit the time window for which they accept hashes generated
using the suspect algorithm.  This is separate (I believe) from the issue Bruce
raises above.  For example, in this case, the administrator is perfectly happy
for the actual plaintext to remain the same, the administrator simply wants to
enforce the new hash.

As far as I can tell, there is nothing in /etc/login.conf to allow for automatic
account expiration if an account is idle for more than N days.

OTOH, even that is probably not sufficient for the original case since a user 
might
login with a different authentication method (e.g. ssh key) that would reset the
idle timer without updating the hash.

I suppose if you really were paranoid about the hash what you would want is an
ability to set an expiration time on the hash algo itself where authentication
using that hash always fails after the expiration time.  This doesn't 
necessarily
expire the entire account (e.g. ssh key auth would still work), though it might
be a bit surprising to the user to find that the next time they attempt to use
password authentication it doesn't work.  (You would at least want a warning
about the hash being expired on login via another mechanism.)

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[Allan Jude]

On 2014-03-07 09:13, John Baldwin wrote:
 On Wednesday, March 05, 2014 3:09:30 pm Matthew Rezny wrote:
 Password expiry is an orthogonal issue and should be up to administrator

 policy.

 Yes, but if you are moving to a different algorithm to improve security, not
 coupling it with an eventual expiration of non-migrated accounts gives a
 false sense of security.  Any admin worth his/her salt is going to want the
 option of enforcing that sort of policy along with the transparent update. 
 They should really be implemented together is all.

 Account expiration and password expiration are already present. There is 
 absolutely no reason that password algorithm upgrade should be tied in any 
 way 
 to expiration. A transparent algorithm upgrade as proposed is *far* better 
 than the forced password change method that is commonly employed. If the 
 administrator wants to force all accounts to migrate by a set deadline, we 
 already have the expiration facilities in place to accomplish that. Expiring 
 accounts that have not been used in a long time, regardless of algorithm 
 changes, should be part of general housekeeping and may be covered by 
 existing 
 policy. Password expiration serves no purpose, EVER. Password expiration 
 encourages users to choose bad passwords because they are throwaway items.

 Bruce states it well enough I need not elaborate further
 https://www.schneier.com/blog/archives/2010/11/changing_passwo.html

 Anyone who fails to understand the above should NOT be an administrator.
 
 I think you failed to understand my point.  I am assuming that an 
 administrator
 wants the transparent upgrade (which I think is useful) because they are
 assuming that the hash algorithm is compromised or inferior.  To that end,
 they may wish to limit the time window for which they accept hashes generated
 using the suspect algorithm.  This is separate (I believe) from the issue 
 Bruce
 raises above.  For example, in this case, the administrator is perfectly happy
 for the actual plaintext to remain the same, the administrator simply wants to
 enforce the new hash.
 
 As far as I can tell, there is nothing in /etc/login.conf to allow for 
 automatic
 account expiration if an account is idle for more than N days.
 
 OTOH, even that is probably not sufficient for the original case since a user 
 might
 login with a different authentication method (e.g. ssh key) that would reset 
 the
 idle timer without updating the hash.
 
 I suppose if you really were paranoid about the hash what you would want is an
 ability to set an expiration time on the hash algo itself where authentication
 using that hash always fails after the expiration time.  This doesn't 
 necessarily
 expire the entire account (e.g. ssh key auth would still work), though it 
 might
 be a bit surprising to the user to find that the next time they attempt to use
 password authentication it doesn't work.  (You would at least want a warning
 about the hash being expired on login via another mechanism.)
 

Honestly, my use case is just silently upgrading the strength of the
hashing algorithm (when combined with my other feature request).
Updating my bcrypt hashes from $2a$04$ to $2b$12$ or something. Same
applies for the default sha512, maybe I want to update to rounds=15000

-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[Tom Evans]

On Fri, Mar 7, 2014 at 2:13 PM, John Baldwin j...@freebsd.org wrote:
 On Wednesday, March 05, 2014 3:09:30 pm Matthew Rezny wrote:
   Password expiry is an orthogonal issue and should be up to administrator
 
  policy.
 
  Yes, but if you are moving to a different algorithm to improve security, 
  not
  coupling it with an eventual expiration of non-migrated accounts gives a
  false sense of security.  Any admin worth his/her salt is going to want the
  option of enforcing that sort of policy along with the transparent update.
  They should really be implemented together is all.

 Account expiration and password expiration are already present. There is
 absolutely no reason that password algorithm upgrade should be tied in any 
 way
 to expiration. A transparent algorithm upgrade as proposed is *far* better
 than the forced password change method that is commonly employed. If the
 administrator wants to force all accounts to migrate by a set deadline, we
 already have the expiration facilities in place to accomplish that. Expiring
 accounts that have not been used in a long time, regardless of algorithm
 changes, should be part of general housekeeping and may be covered by 
 existing
 policy. Password expiration serves no purpose, EVER. Password expiration
 encourages users to choose bad passwords because they are throwaway items.

 Bruce states it well enough I need not elaborate further
 https://www.schneier.com/blog/archives/2010/11/changing_passwo.html

 Anyone who fails to understand the above should NOT be an administrator.

 I think you failed to understand my point.  I am assuming that an 
 administrator
 wants the transparent upgrade (which I think is useful) because they are
 assuming that the hash algorithm is compromised or inferior.  To that end,
 they may wish to limit the time window for which they accept hashes generated
 using the suspect algorithm.  This is separate (I believe) from the issue 
 Bruce
 raises above.  For example, in this case, the administrator is perfectly happy
 for the actual plaintext to remain the same, the administrator simply wants to
 enforce the new hash.

 As far as I can tell, there is nothing in /etc/login.conf to allow for 
 automatic
 account expiration if an account is idle for more than N days.

 OTOH, even that is probably not sufficient for the original case since a user 
 might
 login with a different authentication method (e.g. ssh key) that would reset 
 the
 idle timer without updating the hash.

 I suppose if you really were paranoid about the hash what you would want is an
 ability to set an expiration time on the hash algo itself where authentication
 using that hash always fails after the expiration time.  This doesn't 
 necessarily
 expire the entire account (e.g. ssh key auth would still work), though it 
 might
 be a bit surprising to the user to find that the next time they attempt to use
 password authentication it doesn't work.  (You would at least want a warning
 about the hash being expired on login via another mechanism.)


All of this is orthogonal to adding a way to upgrade hashes. Yes, all
of the points you mentioned are relevant to general password security,
but doesn't explain why a feature that provides transparent hash
upgrades cannot be added without first adding the features you are
asking for.

It's like trying to prevent people from shooting themselves in the
foot by only giving them rocks to throw.

Cheers

Tom
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[A.J. Kehoe IV (Nanoman)]

Allan Jude wrote:

[...]


Honestly, my use case is just silently upgrading the strength of the
hashing algorithm (when combined with my other feature request).
Updating my bcrypt hashes from $2a$04$ to $2b$12$ or something. Same
applies for the default sha512, maybe I want to update to rounds=15000


Like this?

http://www.freebsd.org/cgi/query-pr.cgi?pr=182518

Request for comments:

http://docs.freebsd.org/cgi/mid.cgi?20140106205156.GD4903

--
A.J. Kehoe IV (Nanoman) |  /\  ASCII Ribbon Campaign
Nanoman's Company   |  \ /   - No HTML/RTF in E-mail
E-mail: nano...@nanoman.ca  |   X- No proprietary attachments
WWW: http://www.nanoman.ca/ |  / \   - Respect for open standards


smime.p7s
Description: S/MIME cryptographic signature

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[RW]

On Fri, 7 Mar 2014 09:13:30 -0500
John Baldwin wrote:

 I am assuming that an
 administrator wants the transparent upgrade (which I think is useful)
 because they are assuming that the hash algorithm is compromised or
 inferior.

I'd expect it to be done well in advance of that to give plenty of
time for the transition. We are talking about brute force attacks
and GPU development is relatively predicable.

And lets not lose sight of the fact that we are only talking about
limited mitigation after an attacker has gained root access, not
front-line security.


 I suppose if you really were paranoid about the hash what you would
 want is an ability to set an expiration time on the hash algo itself
 where authentication using that hash always fails after the
 expiration time.

Whenever I've been required to change passwords it's always been
imposed immediately after a login.

Just locking-out an account sounds very heavy-handed to me. It seems
like it would be trivial to extract a list of accounts using the
old-style hashes from master.passwd - at least that way you can send
them an email.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[John Baldwin]

On Friday, March 07, 2014 10:34:40 am Tom Evans wrote:
 On Fri, Mar 7, 2014 at 2:13 PM, John Baldwin j...@freebsd.org wrote:
  On Wednesday, March 05, 2014 3:09:30 pm Matthew Rezny wrote:
Password expiry is an orthogonal issue and should be up to 
administrator
  
   policy.
  
   Yes, but if you are moving to a different algorithm to improve security, 
   not
   coupling it with an eventual expiration of non-migrated accounts gives a
   false sense of security.  Any admin worth his/her salt is going to want 
   the
   option of enforcing that sort of policy along with the transparent 
   update.
   They should really be implemented together is all.
 
  Account expiration and password expiration are already present. There is
  absolutely no reason that password algorithm upgrade should be tied in any 
  way
  to expiration. A transparent algorithm upgrade as proposed is *far* better
  than the forced password change method that is commonly employed. If the
  administrator wants to force all accounts to migrate by a set deadline, we
  already have the expiration facilities in place to accomplish that. 
  Expiring
  accounts that have not been used in a long time, regardless of algorithm
  changes, should be part of general housekeeping and may be covered by 
  existing
  policy. Password expiration serves no purpose, EVER. Password expiration
  encourages users to choose bad passwords because they are throwaway items.
 
  Bruce states it well enough I need not elaborate further
  https://www.schneier.com/blog/archives/2010/11/changing_passwo.html
 
  Anyone who fails to understand the above should NOT be an administrator.
 
  I think you failed to understand my point.  I am assuming that an 
  administrator
  wants the transparent upgrade (which I think is useful) because they are
  assuming that the hash algorithm is compromised or inferior.  To that end,
  they may wish to limit the time window for which they accept hashes 
  generated
  using the suspect algorithm.  This is separate (I believe) from the issue 
  Bruce
  raises above.  For example, in this case, the administrator is perfectly 
  happy
  for the actual plaintext to remain the same, the administrator simply wants 
  to
  enforce the new hash.
 
  As far as I can tell, there is nothing in /etc/login.conf to allow for 
  automatic
  account expiration if an account is idle for more than N days.
 
  OTOH, even that is probably not sufficient for the original case since a 
  user might
  login with a different authentication method (e.g. ssh key) that would 
  reset the
  idle timer without updating the hash.
 
  I suppose if you really were paranoid about the hash what you would want is 
  an
  ability to set an expiration time on the hash algo itself where 
  authentication
  using that hash always fails after the expiration time.  This doesn't 
  necessarily
  expire the entire account (e.g. ssh key auth would still work), though it 
  might
  be a bit surprising to the user to find that the next time they attempt to 
  use
  password authentication it doesn't work.  (You would at least want a warning
  about the hash being expired on login via another mechanism.)
 
 
 All of this is orthogonal to adding a way to upgrade hashes. Yes, all
 of the points you mentioned are relevant to general password security,
 but doesn't explain why a feature that provides transparent hash
 upgrades cannot be added without first adding the features you are
 asking for.

What I didn't say is that I think adding hash expirations is much more of an
edge case and not very useful generically, so probably not worth the hassle.
Also, I had originally suggested expiring accounts, but expiring accounts isn't
really a good proxy for expiring hashes.  Anyway, I think the original idea is
good so long as it is optional.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[Allan Jude]

On 2014-03-07 11:13, A.J. Kehoe IV (Nanoman) wrote:
 Allan Jude wrote:
 
 [...]
 
 Honestly, my use case is just silently upgrading the strength of the
 hashing algorithm (when combined with my other feature request).
 Updating my bcrypt hashes from $2a$04$ to $2b$12$ or something. Same
 applies for the default sha512, maybe I want to update to rounds=15000
 
 Like this?
 
 http://www.freebsd.org/cgi/query-pr.cgi?pr=182518
 
 Request for comments:
 
 http://docs.freebsd.org/cgi/mid.cgi?20140106205156.GD4903
 

This looks like what we wanted. In the feedback you talked about some
changes to your patch required to make it work, is there any progress on
those?

-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[A.J. Kehoe IV (Nanoman)]

Allan Jude wrote:

On 2014-03-07 11:13, A.J. Kehoe IV (Nanoman) wrote:

Allan Jude wrote:

[...]


Honestly, my use case is just silently upgrading the strength of the
hashing algorithm (when combined with my other feature request).
Updating my bcrypt hashes from $2a$04$ to $2b$12$ or something. Same
applies for the default sha512, maybe I want to update to rounds=15000


Like this?

http://www.freebsd.org/cgi/query-pr.cgi?pr=182518

Request for comments:

http://docs.freebsd.org/cgi/mid.cgi?20140106205156.GD4903



This looks like what we wanted. In the feedback you talked about some
changes to your patch required to make it work, is there any progress on
those?


Derek's patches worked perfectly for our needs, but we're the sort of people 
who use vipw and our own utilities for user management.  It wasn't until later 
that we discovered at least one other file would need patching to satisfy 
everyone.  We didn't want to employ the same copy-pasta method, so we asked for 
feedback about our proposed alternative.

secteam@, do you have any comments?  Before we put any more work into this, we 
want to be sure that our proposal is an acceptable one.

--
A.J. Kehoe IV (Nanoman) |  /\  ASCII Ribbon Campaign
Nanoman's Company   |  \ /   - No HTML/RTF in E-mail
E-mail: nano...@nanoman.ca  |   X- No proprietary attachments
WWW: http://www.nanoman.ca/ |  / \   - Respect for open standards


smime.p7s
Description: S/MIME cryptographic signature

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[Xin Li]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Hi,

On 03/07/14 13:52, A.J. Kehoe IV (Nanoman) wrote:
 Allan Jude wrote:
 On 2014-03-07 11:13, A.J. Kehoe IV (Nanoman) wrote:
 Allan Jude wrote:
 
 [...]
 
 Honestly, my use case is just silently upgrading the strength
 of the hashing algorithm (when combined with my other feature
 request). Updating my bcrypt hashes from $2a$04$ to $2b$12$
 or something. Same applies for the default sha512, maybe I
 want to update to rounds=15000
 
 Like this?
 
 http://www.freebsd.org/cgi/query-pr.cgi?pr=182518
 
 Request for comments:
 
 http://docs.freebsd.org/cgi/mid.cgi?20140106205156.GD4903
 
 
 This looks like what we wanted. In the feedback you talked about
 some changes to your patch required to make it work, is there any
 progress on those?
 
 Derek's patches worked perfectly for our needs, but we're the sort
 of people who use vipw and our own utilities for user management.
 It wasn't until later that we discovered at least one other file
 would need patching to satisfy everyone.  We didn't want to employ
 the same copy-pasta method, so we asked for feedback about our
 proposed alternative.
 
 secteam@, do you have any comments?  Before we put any more work
 into this, we want to be sure that our proposal is an acceptable
 one.
 

Did you mean adding rounds capability, or transparent upgrade of
crypt() algorithms, or both?

I need some time to digest the whole transparent upgrade idea but in
general I think it's good.

Speaking for adding rounds, the only problem that needs to be fixed is
that the proposed patch makes it possible to create conflicting
configuration (passwd_format and passwd_modular can use different
hashing algorithms) and need to be fixed and polished.  I like the
idea of making it possible to use more rounds though.

Cheers,
- -- 
Xin LI delp...@delphij.nethttps://www.delphij.net/
FreeBSD - The Power to Serve!   Live free or die
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (FreeBSD)

iQIcBAEBCgAGBQJTGkLzAAoJEJW2GBstM+nsaVoP/017iARGzd++lCsfqyFDozGk
nXJjatlrIcRjrbCmVRT0lsHiK/hoYJ4zZPeOu8EXU1Qs0/wggGHYePX7+zEVob2S
YCxhqUOdG/jqrHnH8bljzWE/OtI7Y4PvFOLpsWkOE/uulssQfGDMSy8WJzFriqzv
GXjAEyFrGXCT29gW6ozTRfDfPSOfd4MhwewbMYAUykSqucMfkG4FaDAgLxv/XdRi
YmLQZuxxTzEqMYanZGq/0e5CvOwOuncd0aVxncJC8ZRcsHs5cqbzcyDkkRwvw/YU
g1OsLXiO08zej0rOz1E4pud8O6q3unG5dNcz9Y96oNo0fJONMrk9IetCUCHBsR8N
eyWJQyHL7wwwNlC5k8U9cOnsL3zxBv54N6bfWuWNNDpJmNrvgMr9LdPso+AX0gLD
y4RhVJeLCQbLrkQawoM1+Ki5N0mQibk9BBGXH/ZPScP1pNqVt9tqXp94N5ZPLV54
Uu4cn/2uKjtTjl76YFlCTvfwwiuWgds1k6CnKZIW8luOp4cG5XOoOSztONqWr6S/
yd7SLDV4f8PC7Fi1iSkSuVW5MYz1I7RRVR1Z27oV3e3UwXwIgqRjHJawNZqIgVe1
4lk84+fm75ULLfiA6bgkMCjylyWHCzrdOQt/Zx+0vyZOer5x2p4gZmnYAyV2EQIP
TM611j1UES6OUGFkfbWa
=4Qur
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[Allan Jude]

On 2014-03-07 17:06, Xin Li wrote:
 Hi,
 
 On 03/07/14 13:52, A.J. Kehoe IV (Nanoman) wrote:
 Allan Jude wrote:
 On 2014-03-07 11:13, A.J. Kehoe IV (Nanoman) wrote:
 Allan Jude wrote:

 [...]

 Honestly, my use case is just silently upgrading the strength
 of the hashing algorithm (when combined with my other feature
 request). Updating my bcrypt hashes from $2a$04$ to $2b$12$
 or something. Same applies for the default sha512, maybe I
 want to update to rounds=15000

 Like this?

 http://www.freebsd.org/cgi/query-pr.cgi?pr=182518

 Request for comments:

 http://docs.freebsd.org/cgi/mid.cgi?20140106205156.GD4903


 This looks like what we wanted. In the feedback you talked about
 some changes to your patch required to make it work, is there any
 progress on those?
 
 Derek's patches worked perfectly for our needs, but we're the sort
 of people who use vipw and our own utilities for user management.
 It wasn't until later that we discovered at least one other file
 would need patching to satisfy everyone.  We didn't want to employ
 the same copy-pasta method, so we asked for feedback about our
 proposed alternative.
 
 secteam@, do you have any comments?  Before we put any more work
 into this, we want to be sure that our proposal is an acceptable
 one.
 
 
 Did you mean adding rounds capability, or transparent upgrade of
 crypt() algorithms, or both?

There are 2 separate but related threads

1) specify rounds for crypt()

2) transparent upgrade of crypt() algo (or more likely just number of
rounds)

 
 I need some time to digest the whole transparent upgrade idea but in
 general I think it's good.
 
 Speaking for adding rounds, the only problem that needs to be fixed is
 that the proposed patch makes it possible to create conflicting
 configuration (passwd_format and passwd_modular can use different
 hashing algorithms) and need to be fixed and polished.  I like the
 idea of making it possible to use more rounds though.
 
 Cheers,
 


-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[John-Mark Gurney]

Allan Jude wrote this message on Fri, Mar 07, 2014 at 17:53 -0500:
 On 2014-03-07 17:06, Xin Li wrote:
  Hi,
  
  On 03/07/14 13:52, A.J. Kehoe IV (Nanoman) wrote:
  Allan Jude wrote:
  On 2014-03-07 11:13, A.J. Kehoe IV (Nanoman) wrote:
  Allan Jude wrote:
 
  [...]
 
  Honestly, my use case is just silently upgrading the strength
  of the hashing algorithm (when combined with my other feature
  request). Updating my bcrypt hashes from $2a$04$ to $2b$12$
  or something. Same applies for the default sha512, maybe I
  want to update to rounds=15000
 
  Like this?
 
  http://www.freebsd.org/cgi/query-pr.cgi?pr=182518
 
  Request for comments:
 
  http://docs.freebsd.org/cgi/mid.cgi?20140106205156.GD4903
 
 
  This looks like what we wanted. In the feedback you talked about
  some changes to your patch required to make it work, is there any
  progress on those?
  
  Derek's patches worked perfectly for our needs, but we're the sort
  of people who use vipw and our own utilities for user management.
  It wasn't until later that we discovered at least one other file
  would need patching to satisfy everyone.  We didn't want to employ
  the same copy-pasta method, so we asked for feedback about our
  proposed alternative.
  
  secteam@, do you have any comments?  Before we put any more work
  into this, we want to be sure that our proposal is an acceptable
  one.
  
  
  Did you mean adding rounds capability, or transparent upgrade of
  crypt() algorithms, or both?
 
 There are 2 separate but related threads
 
 1) specify rounds for crypt()
 
 2) transparent upgrade of crypt() algo (or more likely just number of
 rounds)

Can't the two be merged...  where 2 becomes a flag in login.conf instead
of an algo fetch, and then if it's true, it does the algo fetch from
1?

I really would like us to get 1 in, and then on boot dynamicly adjust
the number of rounds depending upon CPU usage... obviously, a flag will
adjust how long/many rounds the admin wants, but it would allow an
automatic increase in security as faster CPUs are used...

Anyways, how many people are still using passwords instead of ssh keys?
Setting the time to be something like 100ms may seem long, but
considering few people should be using passwords these days, it's less
of an issue...

Xin Li, if you need help reviewing, testing, let me know...

  I need some time to digest the whole transparent upgrade idea but in
  general I think it's good.
  
  Speaking for adding rounds, the only problem that needs to be fixed is
  that the proposed patch makes it possible to create conflicting
  configuration (passwd_format and passwd_modular can use different
  hashing algorithms) and need to be fixed and polished.  I like the
  idea of making it possible to use more rounds though.

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[Xin Li]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 03/07/14 14:50, A.J. Kehoe IV (Nanoman) wrote:
 Xin Li wrote:
 Hi,
 
 On 03/07/14 13:52, A.J. Kehoe IV (Nanoman) wrote:
 Allan Jude wrote:
 On 2014-03-07 11:13, A.J. Kehoe IV (Nanoman) wrote:
 Allan Jude wrote:
 
 [...]
 
 Honestly, my use case is just silently upgrading the
 strength of the hashing algorithm (when combined with my
 other feature request). Updating my bcrypt hashes from
 $2a$04$ to $2b$12$ or something. Same applies for the
 default sha512, maybe I want to update to rounds=15000
 
 Like this?
 
 http://www.freebsd.org/cgi/query-pr.cgi?pr=182518
 
 Request for comments:
 
 http://docs.freebsd.org/cgi/mid.cgi?20140106205156.GD4903
 
 [...]
 
 Speaking for adding rounds, the only problem that needs to be
 fixed is that the proposed patch makes it possible to create
 conflicting configuration (passwd_format and passwd_modular can
 use different hashing algorithms) and need to be fixed and
 polished.  I like the idea of making it possible to use more
 rounds though.
 
 This was deliberate for backward compatibility.  passwd_format will
 be used by default if passwd_modular isn't defined.  If
 passwd_modular is defined as disabled, then passwd_format will be
 used.

Well, my point is that the two shouldn't be allowed to exist together
if they can mean something conflicting.  Allowing passwd_format=sha512
AND passwd_modular=$2a$08$ in the same configuration creates confusion
and it's not good.

My suggestion is that we either have:

a) passwd_format and passwd_round (so that they don't conflict), or

b) extend passwd_format in a compatible manner to allow specifying a
round, or,

c) make passwd_format and passwd_modular conflict so we don't silently
accept it and instead bail out when doing pwd_mkdb.

 What do you think of the idea of putting this into libcrypt instead
 of pam_unix.c, and then patching pam_unix.c and pw_user.c to
 reference libcrypt?

Which part of the idea?  I think it's a bad idea to make libcrypt to
depend on libutil (for login_cap(3)) but we should probably provide
new wrappers in login_cap(3) to do the common things when requested
for various password manipulating tools to reduce duplicated code.

Cheers,
- -- 
Xin LI delp...@delphij.nethttps://www.delphij.net/
FreeBSD - The Power to Serve!   Live free or die
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (FreeBSD)

iQIcBAEBCgAGBQJTGmYMAAoJEJW2GBstM+nsDJ8QAJ+SM9WuRCXo1KYERj+/NJsC
VoP8psjZDZ7+hGOnG7iSwREYTLpxSEAw+sPnIhMgEy1Tg5jCcPvnIhCN/n+XPvaR
HG0o0TdTXL5ZVU4HyKuhNH6JGF9sWWua7Ki/jFguqE+1rdmivcbhrHMZNqOy8Djc
N/dnoCD/eN8K2/FiwP+KjTsYeSyisKFMyiGimQNcuPA7boF4ZBgJmmJqPASHzO9M
DVccoVPrDUip/6BdM+CNx/rNTry1sW3lMFSAuJkx+LENgulbhFz5R0aRGglzwGnJ
LocXVCZlTv0QB37qp9VIHCtTO5n8GxOx43dEtgjWF1cjDs+s+iKjEylX8NguUi0x
SjYu5WOw8xXNdE48QtqpT0N5aHSw9+CCwbrocGaOVYy11voGzo+r3C7jXprhQl8a
pgeiXH5pyBpo9Eh7+/aZdN3WcBjpaOVDnX8We7A9my1lVjxyuLXFyhC3q2OqUjvl
dX4ywKIjiFHSOz0ivzi+uQPx6PD05UuyrWUDING2PvMD/oMtg/hHbR5IxOHdmgPq
j7brHNOk7gxu1f/NFft/yfJAKem6JXjlX68z6/9jMrwxZ8jwTWWAtHrVBjo9/u2i
7ShSZlsEi62GewoIKRRVKvKmdX7Xl+Of/p/DZMTNGCJ9K5NnhEnLKWSp+I5VF0LN
fVQkTqpRaXglMVa/iRkG
=xSx1
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[Xin Li]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 03/07/14 15:07, John-Mark Gurney wrote:
 Allan Jude wrote this message on Fri, Mar 07, 2014 at 17:53 -0500:
 On 2014-03-07 17:06, Xin Li wrote:
 Hi,
 
 On 03/07/14 13:52, A.J. Kehoe IV (Nanoman) wrote:
 Allan Jude wrote:
 On 2014-03-07 11:13, A.J. Kehoe IV (Nanoman) wrote:
 Allan Jude wrote:
 
 [...]
 
 Honestly, my use case is just silently upgrading the
 strength of the hashing algorithm (when combined with
 my other feature request). Updating my bcrypt hashes
 from $2a$04$ to $2b$12$ or something. Same applies for
 the default sha512, maybe I want to update to
 rounds=15000
 
 Like this?
 
 http://www.freebsd.org/cgi/query-pr.cgi?pr=182518
 
 Request for comments:
 
 http://docs.freebsd.org/cgi/mid.cgi?20140106205156.GD4903



 
This looks like what we wanted. In the feedback you talked about
 some changes to your patch required to make it work, is
 there any progress on those?
 
 Derek's patches worked perfectly for our needs, but we're the
 sort of people who use vipw and our own utilities for user
 management. It wasn't until later that we discovered at least
 one other file would need patching to satisfy everyone.  We
 didn't want to employ the same copy-pasta method, so we asked
 for feedback about our proposed alternative.
 
 secteam@, do you have any comments?  Before we put any more
 work into this, we want to be sure that our proposal is an
 acceptable one.
 
 
 Did you mean adding rounds capability, or transparent upgrade
 of crypt() algorithms, or both?
 
 There are 2 separate but related threads
 
 1) specify rounds for crypt()
 
 2) transparent upgrade of crypt() algo (or more likely just
 number of rounds)
 
 Can't the two be merged...  where 2 becomes a flag in login.conf
 instead of an algo fetch, and then if it's true, it does the algo
 fetch from 1?
 
 I really would like us to get 1 in, and then on boot dynamicly
 adjust the number of rounds depending upon CPU usage... obviously,
 a flag will adjust how long/many rounds the admin wants, but it
 would allow an automatic increase in security as faster CPUs are
 used...

Or by the installer/a tool that gets run when doing
mergemaster/etcupdate/freebsd-update: it's rare that CPU gets faster
after installation, and we can probably just write in the
configuration anyway?

Personally I'm not a big fan of making it something that changes over
time: the attacker may do offline attacker than doing it on the victim
system that revealed the salted hashes, so how fast the system CPU
runs doesn't really matter, except for how long a system administrator
is willing to have the user to wait.

 Anyways, how many people are still using passwords instead of ssh
 keys? Setting the time to be something like 100ms may seem long,
 but considering few people should be using passwords these days,
 it's less of an issue...

I'm currently using SSH key plus Google Authenticator for my systems
but all remote login via password is disabled.  I am aware of,
however, many people who refuse to use SSH key authentication because
they don't want to carry their keys, which is a bad idea but they do
it anyways.

 Xin Li, if you need help reviewing, testing, let me know...

Will do and thanks for the offer!

Cheers,
- -- 
Xin LI delp...@delphij.nethttps://www.delphij.net/
FreeBSD - The Power to Serve!   Live free or die
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (FreeBSD)

iQIcBAEBCgAGBQJTGme0AAoJEJW2GBstM+nsXnkQAIYplCr5wMtENLYMQCDSrOFJ
7oxKbW2Iy1qPbbrjAb6mG0TY4ugJu2T6Sg6Wp1B+um2sWqWkNMr+8auHokwuB8+1
8NpnbcZarFvmA5tgVEsh+JcAJF1qZRFQDku+DbL9f/ZXFn/4CtmLkw5NS/kIKf/0
TIeXykNie5nFCS8ifT5Ai7vEHImOTwS4OEVzXoTQSdFGuLnHrToCnV7LpOK2ceIo
ssZ/0Go49tSzW8y3u2a0TZYTqMnh+0EzQFusWkulyCIam0NYjYON/3UY0/TpRgZd
ik2QLqKXaMZBPmi4EsmgpQr97MS0PRag4lahZZad2CckZmhiwWrHLyECf0Xk5i1W
+ACqSfJAzq+NeyDBW05y31qALeyUhm7+ALolSMDFkQMj5B7ra8qnQsbXVyG+DLmg
itpCWfXUpKPxclkvirnDQx89BE1MOYGYBbw69IR5NWcvF3f4EF177xplwAMjHhn5
EXUVIeTwjHYoYgMiZKX8aFgyNR2EX/g6JvZS8236HUbskLQl5AAKM0RA4aQkAFGW
206DYokJW3TnXNArm8kKJCZrYAJb17XyzN6HcY89N+GA0oEkehy2qyQiBVqtpjgh
6WsslScxAnQM3LG84un98cdipOWwQerTwfeji1yqfmik5oNuCm4D/Jlt6rvJBFLb
S5fUd1BQv+0woAKndGhb
=rCdB
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[John-Mark Gurney]

Xin Li wrote this message on Fri, Mar 07, 2014 at 16:43 -0800:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA512
 
 On 03/07/14 15:07, John-Mark Gurney wrote:
  Allan Jude wrote this message on Fri, Mar 07, 2014 at 17:53 -0500:
  On 2014-03-07 17:06, Xin Li wrote:
  Hi,
  
  On 03/07/14 13:52, A.J. Kehoe IV (Nanoman) wrote:
  Allan Jude wrote:
  On 2014-03-07 11:13, A.J. Kehoe IV (Nanoman) wrote:
  Allan Jude wrote:
  
  [...]
  
  Honestly, my use case is just silently upgrading the
  strength of the hashing algorithm (when combined with
  my other feature request). Updating my bcrypt hashes
  from $2a$04$ to $2b$12$ or something. Same applies for
  the default sha512, maybe I want to update to
  rounds=15000
  
  Like this?
  
  http://www.freebsd.org/cgi/query-pr.cgi?pr=182518
  
  Request for comments:
  
  http://docs.freebsd.org/cgi/mid.cgi?20140106205156.GD4903
 
 
 
  
 This looks like what we wanted. In the feedback you talked about
  some changes to your patch required to make it work, is
  there any progress on those?
  
  Derek's patches worked perfectly for our needs, but we're the
  sort of people who use vipw and our own utilities for user
  management. It wasn't until later that we discovered at least
  one other file would need patching to satisfy everyone.  We
  didn't want to employ the same copy-pasta method, so we asked
  for feedback about our proposed alternative.
  
  secteam@, do you have any comments?  Before we put any more
  work into this, we want to be sure that our proposal is an
  acceptable one.
  
  
  Did you mean adding rounds capability, or transparent upgrade
  of crypt() algorithms, or both?
  
  There are 2 separate but related threads
  
  1) specify rounds for crypt()
  
  2) transparent upgrade of crypt() algo (or more likely just
  number of rounds)
  
  Can't the two be merged...  where 2 becomes a flag in login.conf
  instead of an algo fetch, and then if it's true, it does the algo
  fetch from 1?
  
  I really would like us to get 1 in, and then on boot dynamicly
  adjust the number of rounds depending upon CPU usage... obviously,
  a flag will adjust how long/many rounds the admin wants, but it
  would allow an automatic increase in security as faster CPUs are
  used...
 
 Or by the installer/a tool that gets run when doing
 mergemaster/etcupdate/freebsd-update: it's rare that CPU gets faster
 after installation, and we can probably just write in the
 configuration anyway?

It's just easier to throw something into /etc/rc.d w/ an enable/disable
switch than it is to update one/all of those tools to do it...  If you
update only one, then the users of the other tools won't get the benefit..
Or someone forgets to update the other tool...  or we could detect that
the CPU is the same, and keep the previous results...

 Personally I'm not a big fan of making it something that changes over
 time: the attacker may do offline attacker than doing it on the victim
 system that revealed the salted hashes, so how fast the system CPU
 runs doesn't really matter, except for how long a system administrator
 is willing to have the user to wait.

This is my point, there is currently the default number of rounds which
provides basic security, but if the sysadmin wants to provide
additional security, they can do so, either by fixing the number of
rounds to something larger, or by providing a time they are willing
to spend to do the work...

I'm tired of default security parameters not being ideal, or secure
enough...  Most sysadmins won't go and increase the number of rounds
since they don't know enough to (or couldn't before the other patch
was even presented), but they will continue to install FreeBSD on ever
faster machines, yet our only response so far is to switch algorithms,
instead of including more rounds, etc...  This feature would allow us
to provide better security out of the box, and continue to scale our
security as time goes on...

Performance for default, sha512 w/ 5k rounds:
AMD A10-5700 3.4GHz 3.8ms
AMD Opteron 4228 HE 2.8Ghz  5.4ms
Intel(R) Xeon(R) X5650 2.67GHz  4.0ms

these times are aprox as the timing varies quite a bit, ~+/-10%...

code available at: https://www.funkthat.com/~jmg/testcrypt.c

Most people won't notice a 50ms delay on login, yet it'll give us a
10x security benefit...  Just for fun, compare how long it takes to
run sleep .005 and sleep .05 from the command line...  Heck I think
most people would be fine w/ 100ms delay.. try it.. :)

and if they don't mind something similar to how geli does it, it
could be as long as 2 seconds, giving a 500x benefit! :)

and with the auto recrypt path, we could automatically downgrade
users passwords if system ends up w/ a slower CPU, or we could prevent
the downgrade...

  Anyways, how many people are still using passwords instead of ssh
  keys? Setting the time to be something like 100ms may seem long,
  but considering few people should be using passwords these days,
  it's less of an 

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[John-Mark Gurney]

Xin Li wrote this message on Fri, Mar 07, 2014 at 16:36 -0800:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA512
 
 On 03/07/14 14:50, A.J. Kehoe IV (Nanoman) wrote:
  Xin Li wrote:
  Hi,
  
  On 03/07/14 13:52, A.J. Kehoe IV (Nanoman) wrote:
  Allan Jude wrote:
  On 2014-03-07 11:13, A.J. Kehoe IV (Nanoman) wrote:
  Allan Jude wrote:
  
  [...]
  
  Honestly, my use case is just silently upgrading the
  strength of the hashing algorithm (when combined with my
  other feature request). Updating my bcrypt hashes from
  $2a$04$ to $2b$12$ or something. Same applies for the
  default sha512, maybe I want to update to rounds=15000
  
  Like this?
  
  http://www.freebsd.org/cgi/query-pr.cgi?pr=182518
  
  Request for comments:
  
  http://docs.freebsd.org/cgi/mid.cgi?20140106205156.GD4903
  
  [...]
  
  Speaking for adding rounds, the only problem that needs to be
  fixed is that the proposed patch makes it possible to create
  conflicting configuration (passwd_format and passwd_modular can
  use different hashing algorithms) and need to be fixed and
  polished.  I like the idea of making it possible to use more
  rounds though.
  
  This was deliberate for backward compatibility.  passwd_format will
  be used by default if passwd_modular isn't defined.  If
  passwd_modular is defined as disabled, then passwd_format will be
  used.
 
 Well, my point is that the two shouldn't be allowed to exist together
 if they can mean something conflicting.  Allowing passwd_format=sha512
 AND passwd_modular=$2a$08$ in the same configuration creates confusion
 and it's not good.
 
 My suggestion is that we either have:
 
 a) passwd_format and passwd_round (so that they don't conflict), or
 
 b) extend passwd_format in a compatible manner to allow specifying a
 round, or,

Couldn't we merge passwd_format and passwd_modular into one, being
either the name, or the begining salt string?  since the salt string
always begins w/ $, they wouldn't conflict, and could be properly
resolved...

 c) make passwd_format and passwd_modular conflict so we don't silently
 accept it and instead bail out when doing pwd_mkdb.
 
  What do you think of the idea of putting this into libcrypt instead
  of pam_unix.c, and then patching pam_unix.c and pw_user.c to
  reference libcrypt?
 
 Which part of the idea?  I think it's a bad idea to make libcrypt to
 depend on libutil (for login_cap(3)) but we should probably provide
 new wrappers in login_cap(3) to do the common things when requested
 for various password manipulating tools to reduce duplicated code.

So, the current code in pam_unix is:
login_setcryptfmt which calls crypt_set_format as necessary
makesalt
crypt

So, we could expand crypt_set_format to understand the two, and keep
a copy of the rounds data, or we could expose makesalt into maybe
crypt_makesalt (this would be nice) w/ an optional arg that provides
either the name or the beging salt string w/ rounds, i.e. what
passwd_format would contain...  and update pam_unix and friends to
use this...  This last method would make login_setcryptfmt unnecessary,
so I'm not sure how good/bad that is..

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[A.J. Kehoe IV (Nanoman)]

Xin Li wrote:

Hi,

On 03/07/14 13:52, A.J. Kehoe IV (Nanoman) wrote:

Allan Jude wrote:

On 2014-03-07 11:13, A.J. Kehoe IV (Nanoman) wrote:

Allan Jude wrote:

[...]


Honestly, my use case is just silently upgrading the strength
of the hashing algorithm (when combined with my other feature
request). Updating my bcrypt hashes from $2a$04$ to $2b$12$
or something. Same applies for the default sha512, maybe I
want to update to rounds=15000


Like this?

http://www.freebsd.org/cgi/query-pr.cgi?pr=182518

Request for comments:

http://docs.freebsd.org/cgi/mid.cgi?20140106205156.GD4903


[...]


Speaking for adding rounds, the only problem that needs to be fixed is
that the proposed patch makes it possible to create conflicting
configuration (passwd_format and passwd_modular can use different
hashing algorithms) and need to be fixed and polished.  I like the
idea of making it possible to use more rounds though.


This was deliberate for backward compatibility.  passwd_format will be used by default if 
passwd_modular isn't defined.  If passwd_modular is defined as disabled, then 
passwd_format will be used.

What do you think of the idea of putting this into libcrypt instead of 
pam_unix.c, and then patching pam_unix.c and pw_user.c to reference libcrypt?

--
A.J. Kehoe IV (Nanoman) |  /\  ASCII Ribbon Campaign
Nanoman's Company   |  \ /   - No HTML/RTF in E-mail
E-mail: nano...@nanoman.ca  |   X- No proprietary attachments
WWW: http://www.nanoman.ca/ |  / \   - Respect for open standards


smime.p7s
Description: S/MIME cryptographic signature

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[Allan Jude]

On 2014-03-07 21:15, John-Mark Gurney wrote:
 Xin Li wrote this message on Fri, Mar 07, 2014 at 16:43 -0800:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA512

 On 03/07/14 15:07, John-Mark Gurney wrote:
 Allan Jude wrote this message on Fri, Mar 07, 2014 at 17:53 -0500:
 On 2014-03-07 17:06, Xin Li wrote:
 Hi,

 On 03/07/14 13:52, A.J. Kehoe IV (Nanoman) wrote:
 Allan Jude wrote:
 On 2014-03-07 11:13, A.J. Kehoe IV (Nanoman) wrote:
 Allan Jude wrote:

 [...]

 Honestly, my use case is just silently upgrading the
 strength of the hashing algorithm (when combined with
 my other feature request). Updating my bcrypt hashes
 from $2a$04$ to $2b$12$ or something. Same applies for
 the default sha512, maybe I want to update to
 rounds=15000

 Like this?

 http://www.freebsd.org/cgi/query-pr.cgi?pr=182518

 Request for comments:

 http://docs.freebsd.org/cgi/mid.cgi?20140106205156.GD4903




 This looks like what we wanted. In the feedback you talked about
 some changes to your patch required to make it work, is
 there any progress on those?

 Derek's patches worked perfectly for our needs, but we're the
 sort of people who use vipw and our own utilities for user
 management. It wasn't until later that we discovered at least
 one other file would need patching to satisfy everyone.  We
 didn't want to employ the same copy-pasta method, so we asked
 for feedback about our proposed alternative.

 secteam@, do you have any comments?  Before we put any more
 work into this, we want to be sure that our proposal is an
 acceptable one.


 Did you mean adding rounds capability, or transparent upgrade
 of crypt() algorithms, or both?

 There are 2 separate but related threads

 1) specify rounds for crypt()

 2) transparent upgrade of crypt() algo (or more likely just
 number of rounds)

 Can't the two be merged...  where 2 becomes a flag in login.conf
 instead of an algo fetch, and then if it's true, it does the algo
 fetch from 1?

 I really would like us to get 1 in, and then on boot dynamicly
 adjust the number of rounds depending upon CPU usage... obviously,
 a flag will adjust how long/many rounds the admin wants, but it
 would allow an automatic increase in security as faster CPUs are
 used...

 Or by the installer/a tool that gets run when doing
 mergemaster/etcupdate/freebsd-update: it's rare that CPU gets faster
 after installation, and we can probably just write in the
 configuration anyway?
 
 It's just easier to throw something into /etc/rc.d w/ an enable/disable
 switch than it is to update one/all of those tools to do it...  If you
 update only one, then the users of the other tools won't get the benefit..
 Or someone forgets to update the other tool...  or we could detect that
 the CPU is the same, and keep the previous results...
 
 Personally I'm not a big fan of making it something that changes over
 time: the attacker may do offline attacker than doing it on the victim
 system that revealed the salted hashes, so how fast the system CPU
 runs doesn't really matter, except for how long a system administrator
 is willing to have the user to wait.
 
 This is my point, there is currently the default number of rounds which
 provides basic security, but if the sysadmin wants to provide
 additional security, they can do so, either by fixing the number of
 rounds to something larger, or by providing a time they are willing
 to spend to do the work...
 
 I'm tired of default security parameters not being ideal, or secure
 enough...  Most sysadmins won't go and increase the number of rounds
 since they don't know enough to (or couldn't before the other patch
 was even presented), but they will continue to install FreeBSD on ever
 faster machines, yet our only response so far is to switch algorithms,
 instead of including more rounds, etc...  This feature would allow us
 to provide better security out of the box, and continue to scale our
 security as time goes on...
 
 Performance for default, sha512 w/ 5k rounds:
 AMD A10-5700 3.4GHz   3.8ms
 AMD Opteron 4228 HE 2.8Ghz5.4ms
 Intel(R) Xeon(R) X5650 2.67GHz4.0ms
 
 these times are aprox as the timing varies quite a bit, ~+/-10%...
 
 code available at: https://www.funkthat.com/~jmg/testcrypt.c
 
 Most people won't notice a 50ms delay on login, yet it'll give us a
 10x security benefit...  Just for fun, compare how long it takes to
 run sleep .005 and sleep .05 from the command line...  Heck I think
 most people would be fine w/ 100ms delay.. try it.. :)
 
 and if they don't mind something similar to how geli does it, it
 could be as long as 2 seconds, giving a 500x benefit! :)
 
 and with the auto recrypt path, we could automatically downgrade
 users passwords if system ends up w/ a slower CPU, or we could prevent
 the downgrade...
 
 Anyways, how many people are still using passwords instead of ssh
 keys? Setting the time to be something like 100ms may seem long,
 but considering few people should be using passwords these days,
 it's less of 

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[Warner Losh]

On Mar 7, 2014, at 10:22 PM, Allan Jude free...@allanjude.com wrote:
 Performance for default, sha512 w/ 5k rounds:
 AMD A10-5700 3.4GHz  3.8ms
 AMD Opteron 4228 HE 2.8Ghz   5.4ms
 Intel(R) Xeon(R) X5650 2.67GHz   4.0ms
 
 these times are aprox as the timing varies quite a bit, ~+/-10%…

And what would that be on a RPi or other embedded device?

And do the extra route have a peer-reviewed paper showing the increased 
strength?

 One possible solution would be just setting the default login.conf
 number of rounds, based on a test in the installer. Although this won't
 help for systems that are deployed by imaging, or VM images (like EC2
 images) etc.

I’m not sure that’s a good idea.

Warner

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[Matthew Rezny]

  Password expiry is an orthogonal issue and should be up to administrator
 
 policy.
 
 Yes, but if you are moving to a different algorithm to improve security, not
 coupling it with an eventual expiration of non-migrated accounts gives a
 false sense of security.  Any admin worth his/her salt is going to want the
 option of enforcing that sort of policy along with the transparent update. 
 They should really be implemented together is all.

Account expiration and password expiration are already present. There is 
absolutely no reason that password algorithm upgrade should be tied in any way 
to expiration. A transparent algorithm upgrade as proposed is *far* better 
than the forced password change method that is commonly employed. If the 
administrator wants to force all accounts to migrate by a set deadline, we 
already have the expiration facilities in place to accomplish that. Expiring 
accounts that have not been used in a long time, regardless of algorithm 
changes, should be part of general housekeeping and may be covered by existing 
policy. Password expiration serves no purpose, EVER. Password expiration 
encourages users to choose bad passwords because they are throwaway items.

Bruce states it well enough I need not elaborate further
https://www.schneier.com/blog/archives/2010/11/changing_passwo.html

Anyone who fails to understand the above should NOT be an administrator.

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[John Baldwin]

On Friday, February 28, 2014 4:58:29 pm Eitan Adler wrote:
 On 27 February 2014 20:14, Allan Jude free...@allanjude.com wrote:
  With r262501
  (http://svnweb.freebsd.org/base?view=revisionrevision=262501) importing
  the upgraded bcrypt from OpenBSD and eventually changing the default
  identifier for bcrypt to $2b$ it reminded me of a feature that is often
  seen in Forum software and other web apps.
 
  Transparent algorithm upgrade.
 ...
 
 I would strongly support this
 
  I think Nick's point is you do want passwords using the old hash to 
expire
 are some point if they haven't been auto-converted.
 
 Password expiry is an orthogonal issue and should be up to administrator 
policy.

Yes, but if you are moving to a different algorithm to improve security, not 
coupling it with an eventual expiration of non-migrated accounts gives a false 
sense of security.  Any admin worth his/her salt is going to want the option
of enforcing that sort of policy along with the transparent update.  They 
should really be implemented together is all.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[Nick Hibma]

On 28 Feb 2014, at 02:14, Allan Jude free...@allanjude.com wrote:

 With r262501
 (http://svnweb.freebsd.org/base?view=revisionrevision=262501) importing
 the upgraded bcrypt from OpenBSD and eventually changing the default
 identifier for bcrypt to $2b$ it reminded me of a feature that is often
 seen in Forum software and other web apps.
 …
 This would make it much easier to transition a very large userbase from
 md5crypt to bcrypt or sha512crypt, rather than expiring the passwords or
 something.

The sleeping accounts won’t be upgraded, so be left at the ‘insecure’ 
algorithm. I do see the point of automatic updating of password hashes for a 
newer algorithm, but ‘not needing expiry’ isn’t the right argument. It is 
actually an argument opposing your change!

What you probably meant was: don’t hassle users with the change in algorithm, 
possibly only the users that haven’t ever logged in after 6 months.

Nick


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[Allan Jude]

On 2014-02-28 10:07, Nick Hibma wrote:
 
 On 28 Feb 2014, at 02:14, Allan Jude free...@allanjude.com wrote:
 
 With r262501
 (http://svnweb.freebsd.org/base?view=revisionrevision=262501) importing
 the upgraded bcrypt from OpenBSD and eventually changing the default
 identifier for bcrypt to $2b$ it reminded me of a feature that is often
 seen in Forum software and other web apps.
 …
 This would make it much easier to transition a very large userbase from
 md5crypt to bcrypt or sha512crypt, rather than expiring the passwords or
 something.
 
 The sleeping accounts won’t be upgraded, so be left at the ‘insecure’ 
 algorithm. I do see the point of automatic updating of password hashes for a 
 newer algorithm, but ‘not needing expiry’ isn’t the right argument. It is 
 actually an argument opposing your change!
 
 What you probably meant was: don’t hassle users with the change in algorithm, 
 possibly only the users that haven’t ever logged in after 6 months.
 
 Nick
 

The algorithm upgrade would upgrade everyone, including people who
changed their password just 5 days ago. If an account is dormant, and
never logs in, even a password expirey wouldn't force a password change,
because the user never logs in.

To better rephrase my point, the goal is to avoid having to adjust every
users password expirey to yesterday, in order to force them all to set
new passwords.

-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[John Baldwin]

On Friday, February 28, 2014 12:16:45 pm Allan Jude wrote:
 On 2014-02-28 10:07, Nick Hibma wrote:
  
  On 28 Feb 2014, at 02:14, Allan Jude free...@allanjude.com wrote:
  
  With r262501
  (http://svnweb.freebsd.org/base?view=revisionrevision=262501) importing
  the upgraded bcrypt from OpenBSD and eventually changing the default
  identifier for bcrypt to $2b$ it reminded me of a feature that is often
  seen in Forum software and other web apps.
  …
  This would make it much easier to transition a very large userbase from
  md5crypt to bcrypt or sha512crypt, rather than expiring the passwords or
  something.
  
  The sleeping accounts won’t be upgraded, so be left at the ‘insecure’ 
algorithm. I do see the point of automatic updating of password hashes for a 
newer algorithm, but ‘not needing expiry’ isn’t the right argument. It is 
actually an argument opposing your change!
  
  What you probably meant was: don’t hassle users with the change in 
algorithm, possibly only the users that haven’t ever logged in after 6 months.
  
  Nick
  
 
 The algorithm upgrade would upgrade everyone, including people who
 changed their password just 5 days ago. If an account is dormant, and
 never logs in, even a password expirey wouldn't force a password change,
 because the user never logs in.
 
 To better rephrase my point, the goal is to avoid having to adjust every
 users password expirey to yesterday, in order to force them all to set
 new passwords.

I think Nick's point is you do want passwords using the old hash to expire
are some point if they haven't been auto-converted.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Feature Proposal: Transparent upgrade of crypt() algorithms

[Eitan Adler]

On 27 February 2014 20:14, Allan Jude free...@allanjude.com wrote:
 With r262501
 (http://svnweb.freebsd.org/base?view=revisionrevision=262501) importing
 the upgraded bcrypt from OpenBSD and eventually changing the default
 identifier for bcrypt to $2b$ it reminded me of a feature that is often
 seen in Forum software and other web apps.

 Transparent algorithm upgrade.
...

I would strongly support this

 I think Nick's point is you do want passwords using the old hash to expire
are some point if they haven't been auto-converted.

Password expiry is an orthogonal issue and should be up to administrator policy.

 This might actually be more applicable with my next suggestion, exposing
 tuneables to control the number of rounds for bcrypt and sha512crypt. As
 this would make it easy to upgrade all existing bcrypt/sha512crypt
 hashes from the default number of rounds (10^4 and 5000 respectively) to
 higher values.

Another orthogonal issue: I'd like to see the results of the password
hashing competition (see: https://password-hashing.net/.


-- 
Eitan Adler
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Feature Proposal: 'rounds' tuneables for crypt() algorithms

[John-Mark Gurney]

Allan Jude wrote this message on Thu, Feb 27, 2014 at 20:28 -0500:
 Currently, you can change the password hashing algorithm used by crypt()
 with the passwd_format in /etc/login.conf
 
 However, as far as I could find, you cannot change the number of
 'rounds', the dynamic adjustment factor using in bcrypt, and
 sha256crypt, and sha512crypt.
 
 bcrypt uses a log number, the default is 4 (so 2^4 rounds). The minimum
 is currently 4, and the maximum 31
 
 sha256 and sha512crypt default to 5000, with a minimum of 1000 and a
 maximum of 9
 
 OpenBSD implements this in login.conf with 'localcipher' similar to our
 'passwd_format', except it takes an optional 2nd parameter, the number
 of log2() rounds.
 
 Arch implements this in pam_unix with rounds=
 
 For compatibility, it might make most sense to use a separate variable
 rather than adding the optional parameter to the existing passwd_format,
 so older boxes do not choke on it.
 
 Thoughts?

There is already a patch out there to do this..  It basicly adds a string
to login.conf that is the first part of the crypt that you want to
use which will provide the number of rounds too...  I think it was
posted to -current...

I've been meaning to look at adding it...  The reason I'm interested
in doing this is so that we can configure the number of rounds at boot
time...  Say always take 50ms to run the rounds or a minimum number of
rounds..  This way on faster boxes you get added security of extra


-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: [patch] Proposal: move getmntopts(3) into libutil

[Sergey Kandaurov]

On 26 February 2013 16:20, Konstantin Belousov kostik...@gmail.com wrote:
 On Tue, Feb 26, 2013 at 02:39:26PM +0300, Sergey Kandaurov wrote:
 Hi.

 The functions from sbin/mount/getmntopts.c are used in a bunch of other
 stuff like mount_* utilities which have to suck them in as their own
 functions in quite a hackish way. getmntopts.c copies are compiled in to
 an every utility-consumer instead of being present in one place.  Looks
 like getmntopts.c was brought together with mount_ufs.c in 4.4BSD-Lite.
 After that other mount_* were converted to use getmntopts().
 Yes, this is ugly. On the other hand, compiling the functions into
 mount binaries makes them not to depend on the yet another library.
 It cannot be an argument for rejecting your patch, only a point to
 consider.

I'm afraid this is the price for the change. No better thoughts.

 The attached patch moves them to the IMHO architecturally more correct
 place: a separate library -lutil.
 sbin/mount/mntopts.h- include/mntopts.h
 I think the mntopts.h should be moved to lib/libutil then, and installed
 by libutil Makefile.

That's reasonable, thanks. Patch is updated with your correction.
I have put it on freefall for convenience.
http://people.freebsd.org/~pluknet/patches/getmntopts.2.patch


 sbin/mount/getmntopts.[3c]  - lib/libutil/getmntopts.[3c]
 I assume that the move is done by 'svn mv' to preserve the history.

Yes. Unfortunately svn stat cannot nicely represent 'svn mv' ops.

-- 
wbr,
pluknet
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: [patch] Proposal: move getmntopts(3) into libutil

[Sergey Kandaurov]

On 26 February 2013 23:11, Craig Rodrigues rodr...@crodrigues.org wrote:
 On Tue, Feb 26, 2013 at 3:39 AM, Sergey Kandaurov pluk...@gmail.com wrote:

 Hi.



 External mount-like utilities may also have difficulties with building
 to get getmntopts.c source as this requires /usr/src presence which is
 in sync with installed world. Look how mount_fusefs from ports compiles:
 # mount_fusefs needs mntopts.h and getmntopts.c from src/sbin/mount/


 I have no object in moving getmntopts.c to libutil.

 I'll give some history to some of this stuff, to the best of my ability.
 A few years ago, phk@ made a big change by introducing the nmount() system
 call to replace mount().
 Look at all mount-related commits around this time:
 http://lists.freebsd.org/pipermail/cvs-src/2004-December/author.html#36373

 This required changing every file system, and for old file systems which
 supported the classic mount cmount(),
 in each file system, the cmount() call had to internally call the nmount()
 code.

 In addition, phk made a pass to clean up all the userland mount programs to
 use nmount().  There was a lot
 of duplicated (copy and pasted) code in various mount programs.  I helped
 in the effort to clean up some of the userland mount() programs.

 pjd@ proposed to make the main /sbin/mount load a shared library for each
 specific file system,
 and each shared library would have file-system specific mount logic.  For
 example:

 mount -t newfoofs  /dev/blah /mnt

 would internally do something like:

 dlopen(libmount_newfoofs,.);


 phk@ opposed this approach, saying that it could lead to ABI/API problems,
 library mismatches, etc.
 So we kept the existing approach.  I modified /sbin/mount to by default use
 nmount() and only
 for certain file systems, exec an external mount program.

 phk's ideas for getmntopts.c was always to keep it as a place where
 library-like functions for mounting
 file systems would be kept.  To avoid library mismatch problems, it was kept
 has a C file directly compiled
 into the mount programs.

Sure, keeping it directly compiled has its own benefits.
Reading your mail is very educational, thank you.


 I have no problems keeping getmntopts.c in a separate library.  libutil is
 fine, or even a separate libmntutil (or whatever).
 Just keep in mind the issues that could possibly come up if there is a
 mismatch between
 the userland mount programs, and the library which contains getmntopts.c

 Other than that, you proposal is quite reasonable, and I have no issue with
 it.

Although libutil is already used with such binaries like mount and mountd,
library mismatch is a real concern. I will need to think somewhat more.
Thanks for looking at this.

-- 
wbr,
pluknet
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: [patch] Proposal: move getmntopts(3) into libutil

[Konstantin Belousov]

On Wed, Feb 27, 2013 at 07:08:35PM +0300, Sergey Kandaurov wrote:
 I have put it on freefall for convenience.
 http://people.freebsd.org/~pluknet/patches/getmntopts.2.patch

Should the synopsis for getmntopts(3) use #include mntopts.h, instead
of '', as well as the example ?

For the mntopts.h itself, I suggest to add an include guard, it is required
for system header.

I would also take an opportunity and wrapped long lines in mntopts.h
in the same change.


pgpdztsVQqeiG.pgp
Description: PGP signature

Re: [patch] Proposal: move getmntopts(3) into libutil

[Konstantin Belousov]

On Tue, Feb 26, 2013 at 02:39:26PM +0300, Sergey Kandaurov wrote:
 Hi.
 
 The functions from sbin/mount/getmntopts.c are used in a bunch of other
 stuff like mount_* utilities which have to suck them in as their own
 functions in quite a hackish way. getmntopts.c copies are compiled in to
 an every utility-consumer instead of being present in one place.  Looks
 like getmntopts.c was brought together with mount_ufs.c in 4.4BSD-Lite.
 After that other mount_* were converted to use getmntopts().
Yes, this is ugly. On the other hand, compiling the functions into
mount binaries makes them not to depend on the yet another library.
It cannot be an argument for rejecting your patch, only a point to
consider.

 
 The utilities consuming getmntopts.c as currently present in HEAD:
 mount_smbfs
 fsck_ffs
 growfs
 mksnap_ffs
 mount
 mount_cd9660
 mount_ext2fs
 mount_fusefs
 mount_hpfs
 mount_msdosfs
 mount_nfs
 mount_nullfs
 mount_reiserfs
 mount_std
 mount_udf
 mount_unionfs
 mount_nwfs
 mount_portalfs
 mount_smbfs
 mountd
 
 External mount-like utilities may also have difficulties with building
 to get getmntopts.c source as this requires /usr/src presence which is
 in sync with installed world. Look how mount_fusefs from ports compiles:
 # mount_fusefs needs mntopts.h and getmntopts.c from src/sbin/mount/
 
 The attached patch moves them to the IMHO architecturally more correct
 place: a separate library -lutil.
 sbin/mount/mntopts.h- include/mntopts.h
I think the mntopts.h should be moved to lib/libutil then, and installed
by libutil Makefile.

 sbin/mount/getmntopts.[3c]  - lib/libutil/getmntopts.[3c]
I assume that the move is done by 'svn mv' to preserve the history.


pgpAdbjc1IgDi.pgp
Description: PGP signature

Re: [patch] Proposal: move getmntopts(3) into libutil

[Alfred Perlstein]

On 2/26/13 4:20 AM, Konstantin Belousov wrote:

On Tue, Feb 26, 2013 at 02:39:26PM +0300, Sergey Kandaurov wrote:

Hi.

The functions from sbin/mount/getmntopts.c are used in a bunch of other
stuff like mount_* utilities which have to suck them in as their own
functions in quite a hackish way. getmntopts.c copies are compiled in to
an every utility-consumer instead of being present in one place.  Looks
like getmntopts.c was brought together with mount_ufs.c in 4.4BSD-Lite.
After that other mount_* were converted to use getmntopts().

Yes, this is ugly. On the other hand, compiling the functions into
mount binaries makes them not to depend on the yet another library.
It cannot be an argument for rejecting your patch, only a point to
consider.


The utilities consuming getmntopts.c as currently present in HEAD:
mount_smbfs
fsck_ffs
growfs
mksnap_ffs
mount
mount_cd9660
mount_ext2fs
mount_fusefs
mount_hpfs
mount_msdosfs
mount_nfs
mount_nullfs
mount_reiserfs
mount_std
mount_udf
mount_unionfs
mount_nwfs
mount_portalfs
mount_smbfs
mountd

External mount-like utilities may also have difficulties with building
to get getmntopts.c source as this requires /usr/src presence which is
in sync with installed world. Look how mount_fusefs from ports compiles:
# mount_fusefs needs mntopts.h and getmntopts.c from src/sbin/mount/

The attached patch moves them to the IMHO architecturally more correct
place: a separate library -lutil.
sbin/mount/mntopts.h- include/mntopts.h

I think the mntopts.h should be moved to lib/libutil then, and installed
by libutil Makefile.


sbin/mount/getmntopts.[3c]  - lib/libutil/getmntopts.[3c]

I assume that the move is done by 'svn mv' to preserve the history.

Agree with all of Konstantin's review.

Looks very good.

-Alfred
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: [patch] Proposal: move getmntopts(3) into libutil

[Craig Rodrigues]

On Tue, Feb 26, 2013 at 3:39 AM, Sergey Kandaurov pluk...@gmail.com wrote:

 Hi.


 External mount-like utilities may also have difficulties with building
 to get getmntopts.c source as this requires /usr/src presence which is
 in sync with installed world. Look how mount_fusefs from ports compiles:
 # mount_fusefs needs mntopts.h and getmntopts.c from src/sbin/mount/


I have no object in moving getmntopts.c to libutil.

I'll give some history to some of this stuff, to the best of my ability.
A few years ago, phk@ made a big change by introducing the nmount() system
call to replace mount().
Look at all mount-related commits around this time:
http://lists.freebsd.org/pipermail/cvs-src/2004-December/author.html#36373

This required changing every file system, and for old file systems which
supported the classic mount cmount(),
in each file system, the cmount() call had to internally call the nmount()
code.

In addition, phk made a pass to clean up all the userland mount programs to
use nmount().  There was a lot
of duplicated (copy and pasted) code in various mount programs.  I helped
in the effort to clean up some of the userland mount() programs.

pjd@ proposed to make the main /sbin/mount load a shared library for each
specific file system,
and each shared library would have file-system specific mount logic.  For
example:

mount -t newfoofs  /dev/blah /mnt

would internally do something like:

dlopen(libmount_newfoofs,.);


phk@ opposed this approach, saying that it could lead to ABI/API problems,
library mismatches, etc.
So we kept the existing approach.  I modified /sbin/mount to by default use
nmount() and only
for certain file systems, exec an external mount program.

phk's ideas for getmntopts.c was always to keep it as a place where
library-like functions for mounting
file systems would be kept.  To avoid library mismatch problems, it was
kept has a C file directly compiled
into the mount programs.

I have no problems keeping getmntopts.c in a separate library.  libutil is
fine, or even a separate libmntutil (or whatever).
Just keep in mind the issues that could possibly come up if there is a
mismatch between
the userland mount programs, and the library which contains getmntopts.c

Other than that, you proposal is quite reasonable, and I have no issue with
it.
Thanks!
--
Craig


--
Craig
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: NETGRAPH (proposal. FINAL)

[Garrett Wollman]

On Tue, 29 Feb 2000 18:59:50 +0100 (CET), Luigi Rizzo [EMAIL PROTECTED] said:

 can you clarify this ? Looong ago i used the '586 on a bridge and it did let
 me write the MAC header...

The 82586 has a mode bit which selects one of two possibilities:

1) The transmit command specifies the destination address and
length/ethertype field; the source address is inserted by hardware.
The receive buffer descriptor gets the source address and
length/ethertype.

2) The transmit and receive buffers include a full Ethernet header.

I can't say off the top of my head which the `ie' driver uses, but I
would bet on (2) because that's easier for the driver to deal with.

These sorts of controllers are the reason why ether_input takes the
Ethernet header as a separate parameter.

-GAWollman

--
Garrett A. Wollman   | O Siem / We are all family / O Siem / We're all the same
[EMAIL PROTECTED]  | O Siem / The fires of freedom 
Opinions not those of| Dance in the burning flame
MIT, LCS, CRS, or NSA| - Susan Aglukark and Chad Irschick


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: NETGRAPH (proposal. FINAL)

[Archie Cobbs]

Julian Elischer writes:
  here is url: http://home.earthlink.net/~evmax/ng.tar.gz
  
  these are final patches for NETGRAPH.
  new features:
  - new hook ``divertin'' allows to put frame back to
  kernel stack.
  - new control message allows to set raw mode on
  ``divert'' hook. raw mode assumes that we have
  fully prepared frame and we do not have to update
  ``ether_shost'' field.
 
 This is good in theory, however the intel 82586 ethernet chip
 (and 596 in 586 mode) will overwrite anything you put there anyhow
 as it treats the header specially and fabricates it.
 (unless you are running in some mode that is not usually used).
 I don't know how many other chips do this but it may be misleading
 for the user who sets this on such a chip because the source 
 address he sets will not be put on the wire.
 
 The idea is however useful and I guess we'll try add it in 
 in the near future...
 What do you think Archie?
 Are we still in code freeze? (I think so).

Yes, I was going to take a look at this after 4.0-REL and then
commit something hopefully soon thereafter..

By the way, if the ethernet chip doesn't support manual source
address then BPF has the same problem that we do.. IMHO, we should
just punt and return an error in this case..

-Archie

___
Archie Cobbs   *   Whistle Communications, Inc.  *   http://www.whistle.com


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: NETGRAPH (proposal. FINAL)

[Julian Elischer]

Yevmenkin, Maksim N, CSCIO wrote:
 
 hello all,
 
 here is url: http://home.earthlink.net/~evmax/ng.tar.gz
 
 these are final patches for NETGRAPH.
 new features:
 - new hook ``divertin'' allows to put frame back to
 kernel stack.
 - new control message allows to set raw mode on
 ``divert'' hook. raw mode assumes that we have
 fully prepared frame and we do not have to update
 ``ether_shost'' field.

This is good in theory, however the intel 82586 ethernet chip
(and 596 in 586 mode) will overwrite anything you put there anyhow
as it treats the header specially and fabricates it.
(unless you are running in some mode that is not usually used).
I don't know how many other chips do this but it may be misleading
for the user who sets this on such a chip because the source 
address he sets will not be put on the wire.

The idea is however useful and I guess we'll try add it in 
in the near future...
What do you think Archie?
Are we still in code freeze? (I think so).

Julian

 
 thanks,
 emax
 
 To Unsubscribe: send mail to [EMAIL PROTECTED]
 with "unsubscribe freebsd-hackers" in the body of the message

-- 
  __--_|\  Julian Elischer
 /   \ [EMAIL PROTECTED]
(   OZ) World tour 2000
--- X_.---._/  presently in:  Perth
v


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

RE: NETGRAPH (proposal. FINAL)

[Yevmenkin, Maksim N, CSCIO]

[...]
  This is good in theory, however the intel 82586 ethernet chip
  (and 596 in 586 mode) will overwrite anything you put there anyhow
  as it treats the header specially and fabricates it.
  (unless you are running in some mode that is not usually used).
  I don't know how many other chips do this but it may be misleading
  for the user who sets this on such a chip because the source 
  address he sets will not be put on the wire.
  
  The idea is however useful and I guess we'll try add it in 
  in the near future...
  What do you think Archie?
  Are we still in code freeze? (I think so).
 
 Yes, I was going to take a look at this after 4.0-REL and then
 commit something hopefully soon thereafter..
 
 By the way, if the ethernet chip doesn't support manual source
 address then BPF has the same problem that we do.. IMHO, we should
 just punt and return an error in this case..

i think we still have this problem in BPF. as far as i know ``bpfwrite''
calls ``if_output'' which is ``ether_output''. in the same time
``ether_output'' updates ``ether_shost''. so, as far as i know,
it's imposible to send frame with custom ``ether_shost''.
please correct me if i wrong.

thanks,
emax


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: NETGRAPH (proposal. FINAL)

[Luigi Rizzo]

 This is good in theory, however the intel 82586 ethernet chip
 (and 596 in 586 mode) will overwrite anything you put there anyhow
 as it treats the header specially and fabricates it.
 (unless you are running in some mode that is not usually used).

can you clarify this ? Looong ago i used the '586 on a bridge and it did let
me write the MAC header...

cheers
luigi

 I don't know how many other chips do this but it may be misleading
 for the user who sets this on such a chip because the source 
 address he sets will not be put on the wire.
 
 The idea is however useful and I guess we'll try add it in 
 in the near future...
 What do you think Archie?
 Are we still in code freeze? (I think so).
 
 Julian
 
  
  thanks,
  emax
  
  To Unsubscribe: send mail to [EMAIL PROTECTED]
  with "unsubscribe freebsd-hackers" in the body of the message
 
 -- 
   __--_|\  Julian Elischer
  /   \ [EMAIL PROTECTED]
 (   OZ) World tour 2000
 --- X_.---._/  presently in:  Perth
 v
 
 
 To Unsubscribe: send mail to [EMAIL PROTECTED]
 with "unsubscribe freebsd-current" in the body of the message
 



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: NETGRAPH (proposal. FINAL)

[Poul-Henning Kamp]

In message [EMAIL PROTECTED], Julian Elischer writes:

 these are final patches for NETGRAPH.
 new features:
 - new hook ``divertin'' allows to put frame back to
 kernel stack.
 - new control message allows to set raw mode on
 ``divert'' hook. raw mode assumes that we have
 fully prepared frame and we do not have to update
 ``ether_shost'' field.

This is good in theory, however the intel 82586 ethernet chip
(and 596 in 586 mode) will overwrite anything you put there anyhow
as it treats the header specially and fabricates it.
(unless you are running in some mode that is not usually used).
I don't know how many other chips do this but it may be misleading
for the user who sets this on such a chip because the source 
address he sets will not be put on the wire.

But the driver would know and could issue an error or warning
in that case ?  I guess the packet should be failed with ENOCANDO ?

--
Poul-Henning Kamp FreeBSD coreteam member
[EMAIL PROTECTED]   "Real hackers run -current on their laptop."
FreeBSD -- It will take a long time before progress goes too far!


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message