Re: Technical Differences of *BSD and Linux

2003-02-04 Thread der Mouse
 To: Terry Lambert [EMAIL PROTECTED]
 Cc: [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED],
 [EMAIL PROTECTED], [EMAIL PROTECTED]

 [off-list]

Ugh.  My apologies to everyone involved.

I'll go crawl back in my hole now

/~\ The ASCII   der Mouse
\ / Ribbon Campaign
 X  Against HTML   [EMAIL PROTECTED]
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Re: Technical Differences of *BSD and Linux

2003-02-04 Thread Terry Lambert
der Mouse wrote:
 [off-list]

Uh, no it wasn't.  Do you want a reply?  On list or off?

8-).

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Missing commit bit [PATCH].

2003-02-04 Thread Pawel Jakub Dawidek

Submitter-Id:  current-users
Originator:Pawel Jakub Dawidek
Organization:  
Confidential:  no
Synopsis:  Missing commit bit [PATCH].
Severity:  critical
Priority:  high
Category:  misc
Class: change-request
Release:   All of them.
Environment:
Any.

Description:
There is commit bit for Matthew Dillon [EMAIL PROTECTED].
It was removed without any explanation.
How-To-Repeat:
Don't repeat it. Ever.
Fix:
Here is appropriate patch:

--- access.orig Tue Feb  4 10:44:28 2003
+++ access  Tue Feb  4 10:44:59 2003
@@ -1,5 +1,5 @@
 #
-# $FreeBSD: CVSROOT/access,v 1.364 2003/02/02 09:01:30 markm Exp $
+# $FreeBSD$
 #
 # Please try to keep the handbook (contrib/chapter.sgml authors.ent
 # staff/chapter.sgml) in in sync with this.
@@ -164,6 +164,7 @@
 andy
 imura
 jmas
+dillon
 bp
 wsanchez
 dan

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Re: Missing commit bit [PATCH].

2003-02-04 Thread Pawel Jakub Dawidek
On Tue, Feb 04, 2003 at 10:48:40AM +0100, Pawel Jakub Dawidek wrote:
+  There is commit bit for Matthew Dillon [EMAIL PROTECTED].

Grr:) s/is/ISN'T/

-- 
Pawel Jakub Dawidek
UNIX Systems Administrator
http://garage.freebsd.pl
Am I Evil? Yes, I Am.



msg39615/pgp0.pgp
Description: PGP signature


Re: vfork / execve / accept lock

2003-02-04 Thread rmkml
Thank for you answer.

It is difficult to find anything concerning the signal model
of BSD implementation. In particular, for threaded applications.
If you can give me some advise or documentation to read, it will
be very helpfull to me .

In the first part of the answer, do you want to say that a threaded
application can't use vfork/fork/rfork command because the result
will be undefined ?
In this case is there a solution to launch external cmd ?
In your opinion, can directly the application create a new thread,
this thread used execve, and the parent thread waitpid() ?

Thank you in advance.

Bests regards.




Terry Lambert wrote:

 rmkml wrote:
  here is a sample code (vfork_execve.c) to demonstrate a locking problem.

 The short answer is that your code is wrong.

 The longer answer is that your code is assuming implementation
 details in vfork() which are undefined in threaded programs,
 according to the standard, and is assuming that the threads
 implementation is in the kernel, and assuming that the threads
 are created via a process similar to vfork().

 Since the standard permits implementations to be in user space,
 and fork interaction is undefined, and signal delivery targeting
 is undefined, you should probably expect that any threaded
 program using signals or using fork/vfork/rfork will exhibit
 undefined behaviour.

 For example, the pthreads implementation on AIX will not end up
 delivering the signal to the expected thread, either.  This was
 a bug in MySQL on AIX, for a while.  I can't remember if it was
 me, Mark Peek, Paul Ozzello, or Jenifer Meyers that fixed the
 assumption, and sent the patch off to the MySQL folks, but it
 was one of us.

 FWIW, it's also incorrect to expect PTHREAD_SCOPE_SYSTEM to work
 on a user space pthreads implementation, which is permitted by the
 standard.

 It's also illegal set up your signal handler the way you have,
 given that the standard specifically states that it's unsafe to
 call some function in signal handlers, and you call one of them;
 realize that any system call you make will actually get the libc_r
 wrapped version, and therefore it should not be called in a signal
 handler in a threaded program, unless it's specifically allowed.

 Finally, signals are persistent conditions, not events.  If you
 intend to write code where reaping your SIGCHLD handler, you
 *will* have to loop, as you do, on waitpid, following receipt of
 the signal, but the waiting should be accomplished via the signal
 handler setting a global volatile flag, and then the flag should
 be checked in the main loop of the program or a thread, somewhere,
 and the waitpid called from there.

 If you want to signal specific threads, you need to trampoline
 the signal from the process scope to a particular thread scope,
 using pthread_kill() after receiving the signal (this was the AIX
 MySQL fix).

 -- Terry


To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Re: vfork / execve / accept lock

2003-02-04 Thread Terry Lambert
rmkml wrote:
 Thank for you answer.

Sorry that it probably was not the answer you wanted.  8-(.


 It is difficult to find anything concerning the signal model
 of BSD implementation. In particular, for threaded applications.
 If you can give me some advise or documentation to read, it will
 be very helpfull to me .

I recommend the POSIX standard, the Go Solo 2 book (I'm afraid
it's out of print, now), and the O'Reilly book.  The rule of thumb
is that anything that is undefined/implementation defined or
is an extension -- don't use it.

BSD attempts to implement strict POSIX 1003.1 signals.  It does
not implement the POSIX reliable signal delivery mechanism at
this time.

With regard to threads, signals are delivered to the process.  This
may mean that any thread that happens to be running at the time, or
the threads schduler, gets the actual signal.

In general, the user space threads implementation is what's called
a call conversion implementation.  The wayhe this operates is by
trading a blocking call for a non-blocking call, plus an entry into
the user space threads scheduler.  If there are other threads that
are pending execution (not blocked on resources), then the user
space threads scheduler schedules them to run.

The scheduler function is _thread_kern_sched(); this is the pthreads
kernel, in user space.  It's located in libc_r, in the source file
/usr/src/lib/libc_r/uthread/uthread_kern.c; if there is only one
thread, then it sets a timer and retries.  Because everything is
non-blocking, this is the only way to convert a non-blocking call
in a single thread to block pending completion of the operation
(really, it polls, and the timer is to keep it from swamping the
CPU with a buzz-loop in the scheduler).

The time for this wait is tiny, so it's not going to be the timeout
you saw.  It's tricky, because handling the signal is different in
the scheduler from elsewhere, since it's an interval timer signal,
and you might be using interval timers in your program.

But basically, this means that on return, when signals are unmasked,
they are delivered to the unmasking process.  This happens either
automatically, or as a result of the siglongjmp from the scheduler.

So signals which are not caught may end up delivered in the context
of any thread, at random.

Since signals run on their own stack, and have their own context
(sort of), you can do everything you'd normally do in a signal
handler, except assume context other than process context.  So
if you has a thread allocated (or auto) variable that you set a
global pointer to, you can't access it from a signal handler,
and assume that the handler will only fire in the thread that
originally registered it, etc..


The best documentation is Chris Provenzo's documentation on
MIT pthreads.  There are a couple of papers he wrote on the
pthreads system he wrote.  It's a distant ancestor of the
FreeBSD user space pthreads implementation.


 In the first part of the answer, do you want to say that a threaded
 application can't use vfork/fork/rfork command because the result
 will be undefined ?

No.  vfork() is just a wrapper for fork(), in threads.  THat's
a library implementation detail.  The source file is in:
/usr/src/lib/libc_r/uthread/uthread_vfork.c; the rfork()
call doesn't exist at all.

If you are expecting the main process execution to syspend from
the vfork() to the execve(), as documented for vfork(), well, it
won't.


 In this case is there a solution to launch external cmd ?

The fork()/execve() combination and system() work.  I really
recommend using fork/execve.  The reason for this is that the
system() is a cancellation point.  Basically, it will suspend
until the command returns.

For reapable status, use fork() and system(), rather than
trying to roll your own.


 In your opinion, can directly the application create a new thread,
 this thread used execve, and the parent thread waitpid() ?

Not without causing a cancellation point, which means that the
thread you care about isn't waiting on the status, and the SIGCLD
will interrupt it, and then your SIGCLD will hit the handler and
not the wait.

You need to use wait4().  This is probably a bug in the
implementation, since waitpid() is defined to be identical to
wait4(), with an rusage value of 0, so the threads waitpid()
needs to call the threads wait4(), which uses the WNOHANG to
convert from a blocking call to a non-blocking call, and avoid
the cancellation point.

Regular wait() also introduces a cancellation point.  I'm not
positive that it can be implemented in terms of wait4(), since
the man page doesn't note an equivalence.  I would *think* it
could be wait4(-1, status, 0, 0).  If so, this would avoid the
cancellation point there, too.  This is mostly a signal thing;
sigwait() is also suspect, but with nothing to do about it.

You can grep for _thread_enter_cancellation_point() in all the
source files in /usr/src/lib/libc_r/uthread/*.c; this will
basically tell you every place you 

Silly question regarding x86-64

2003-02-04 Thread Attila Nagy
Hello,

I have a very silly (because real hardware isn't yet ready) question to
x86-64 support.
According to the docs, the first implementation of x86-64 will use PAE to
map 48 bit address space to 40 bit, which is 128 GB.

Will FreeBSD support that 40 bit addressing or it will be similar to the
alpha platform due to some bugs and will use only a smaller portion of the
available memory?

BTW, what's the exact state of this project? On its webpage the latest
news is from 13 April 2001. and says David O'Brien got the software
simulator...

Thanks,
--[ Free Software ISOs - http://www.fsn.hu/?f=download ]--
Attila Nagy e-mail: [EMAIL PROTECTED]
Free Software Network (FSN.HU)phone @work: +361 210 1415 (194)
cell.: +3630 306 6758

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Re: replacing GNU grep with UNIX grep.

2003-02-04 Thread Eirik Nygaard
On Mon, Feb 03, 2003 at 11:39:47PM -0500, Munish Chopra wrote:
 On 2003-02-03 18:51 +, Lyndon Nerenberg wrote:
  The classic Unix including BSD4.4 UNIX is now under a
  BSD-like license too (finding it is another issue
  though ;).
  
  Kirk McKusick sells a CD-ROM collection containing all the CSRG
  distributions:
  
  https://www.mckusick.com/csrg/
  
  Pretty much everything that had to be GNUified in BSD/386 and friends
  (thanks to the Death Star's lawyers) can be found in all its glory on
  the CDs. And thanks to Caldera, it can now be freely distributed.
  
  A lot of the CSRG code is missing functionality that we've picked up
  along the way, and much of it is pre-ANSI C, so there's work to be done
  to bring it up to date. Regardless, the CSRG code is a good starting
  point for a lot of things. (For example, I'm currently about half way
  through getting troff updated.)
 
 I think tjr mentioned he had started converting a few utilities. Anyone
 picking things up may want to pick up where he left off.
 
 -- 
 Munish Chopra
 

I am working on a BSD lisenced grep, missing some functions. Checking if
file is a binary file and some other small things.
If anyone wants I can upload it somewhere.

-- 

Eirik Nygaard [EMAIL PROTECTED]
PGP Key: 83C55EDE



msg39619/pgp0.pgp
Description: PGP signature


Re: vfork / execve / accept lock

2003-02-04 Thread Daniel Eischen
On Tue, 4 Feb 2003, Terry Lambert wrote:

 rmkml wrote:
  Thank for you answer.
 
 Sorry that it probably was not the answer you wanted.  8-(.
 
 
  It is difficult to find anything concerning the signal model
  of BSD implementation. In particular, for threaded applications.
  If you can give me some advise or documentation to read, it will
  be very helpfull to me .
 
 I recommend the POSIX standard, the Go Solo 2 book (I'm afraid
 it's out of print, now), and the O'Reilly book.  The rule of thumb
 is that anything that is undefined/implementation defined or
 is an extension -- don't use it.
 
 BSD attempts to implement strict POSIX 1003.1 signals.  It does
 not implement the POSIX reliable signal delivery mechanism at
 this time.
 
 With regard to threads, signals are delivered to the process.  This
 may mean that any thread that happens to be running at the time, or
 the threads schduler, gets the actual signal.
 
 In general, the user space threads implementation is what's called
 a call conversion implementation.  The wayhe this operates is by
 trading a blocking call for a non-blocking call, plus an entry into
 the user space threads scheduler.  If there are other threads that
 are pending execution (not blocked on resources), then the user
 space threads scheduler schedules them to run.
 
 The scheduler function is _thread_kern_sched(); this is the pthreads
 kernel, in user space.  It's located in libc_r, in the source file
 /usr/src/lib/libc_r/uthread/uthread_kern.c; if there is only one
 thread, then it sets a timer and retries.  Because everything is
 non-blocking, this is the only way to convert a non-blocking call
 in a single thread to block pending completion of the operation
 (really, it polls, and the timer is to keep it from swamping the
 CPU with a buzz-loop in the scheduler).
 
 The time for this wait is tiny, so it's not going to be the timeout
 you saw.  It's tricky, because handling the signal is different in
 the scheduler from elsewhere, since it's an interval timer signal,
 and you might be using interval timers in your program.
 
 But basically, this means that on return, when signals are unmasked,
 they are delivered to the unmasking process.  This happens either
 automatically, or as a result of the siglongjmp from the scheduler.
 
 So signals which are not caught may end up delivered in the context
 of any thread, at random.
 
 Since signals run on their own stack, and have their own context
 (sort of), you can do everything you'd normally do in a signal
 handler, except assume context other than process context.  So
 if you has a thread allocated (or auto) variable that you set a
 global pointer to, you can't access it from a signal handler,
 and assume that the handler will only fire in the thread that
 originally registered it, etc..
 
 
 The best documentation is Chris Provenzo's documentation on
 MIT pthreads.  There are a couple of papers he wrote on the
 pthreads system he wrote.  It's a distant ancestor of the
 FreeBSD user space pthreads implementation.
 
 
  In the first part of the answer, do you want to say that a threaded
  application can't use vfork/fork/rfork command because the result
  will be undefined ?
 
 No.  vfork() is just a wrapper for fork(), in threads.  THat's
 a library implementation detail.  The source file is in:
 /usr/src/lib/libc_r/uthread/uthread_vfork.c; the rfork()
 call doesn't exist at all.
 
 If you are expecting the main process execution to syspend from
 the vfork() to the execve(), as documented for vfork(), well, it
 won't.
 
 
  In this case is there a solution to launch external cmd ?
 
 The fork()/execve() combination and system() work.  I really
 recommend using fork/execve.  The reason for this is that the
 system() is a cancellation point.  Basically, it will suspend
 until the command returns.
 
 For reapable status, use fork() and system(), rather than
 trying to roll your own.
 
 
  In your opinion, can directly the application create a new thread,
  this thread used execve, and the parent thread waitpid() ?
 
 Not without causing a cancellation point, which means that the
 thread you care about isn't waiting on the status, and the SIGCLD
 will interrupt it, and then your SIGCLD will hit the handler and
 not the wait.
 
 You need to use wait4().  This is probably a bug in the
 implementation, since waitpid() is defined to be identical to
 wait4(), with an rusage value of 0, so the threads waitpid()
 needs to call the threads wait4(), which uses the WNOHANG to
 convert from a blocking call to a non-blocking call, and avoid
 the cancellation point.
 
 Regular wait() also introduces a cancellation point.  I'm not
 positive that it can be implemented in terms of wait4(), since
 the man page doesn't note an equivalence.  I would *think* it
 could be wait4(-1, status, 0, 0).  If so, this would avoid the
 cancellation point there, too.  This is mostly a signal thing;
 sigwait() is also suspect, but with nothing to do about it.
 
 

Re: Modifying mergemaster behavior

2003-02-04 Thread clemens fischer
Garance A Drosihn [EMAIL PROTECTED]:

 I happen to be updating my system tonight, so when it came to the
 mergemaster step I first modified the script.  I added:
 -I '$FreeBSD:.*$'
 to the 'diff ${DIFF_FLAG}' command in diff_loop, and it seems to have
 worked the way I wanted it to work.

how about making this an option value to the `-c' option?  `-c'
normally just switches to context-diffs, why not allow users to
specify his custom diff(1) options here and in the ~/.mergemasterrc?

  clemens

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Re: Technical Differences of *BSD and Linux

2003-02-04 Thread Sean Davis
On Mon, Feb 03, 2003 at 10:00:48PM -0800, Terry Lambert wrote:
snip
 This is actually incorrect.  At one point in time, the email
 address of the driver authors was printed out in boot messsages,
 under Linux.  They had a flag day in which Linus removed all
 the printf's.  This flag day was actually commercially motivated.

Some (many?) Linux drivers still print out the e-mail addresses of the author.
I base this claim on the dmesg from a 2.4.something/alpha system I saw the other
day.

snip

-- 
/~\ The ASCII
\ / Ribbon Campaign   Sean Davis
 X  Against HTML   aka dive
/ \ Email!

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



He dares to Ask...

2003-02-04 Thread Steve Kudlak
Is there a place where these *nix questions
can be asked? Where I can ask why like how
BSD like is the new MAC-OS?  Like in the
MAC OS boot-up what part is handled by
BSD. If someone says my MAC hangs with
the Happy Mac can I go down to the # prompt
and start poking around and see what is wrong.

Note I dare to ask these things here because
as I lurk usually good information comes my way
via this group. I certainly feel better about BSD
and even giggle! Linux than WIndows which
many of us are stuck with.

Have Fun,
Sends Steve


To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Re: End-Of-Life announcement for M-Systems DiskOnChip driver(fla).

2003-02-04 Thread Yury Tarasievich
Nat Lanza wrote:


On Mon, 2003-02-03 at 18:06, Terry Lambert wrote:
 

But if that's the argument for removing it, then it's probably
time to remove the ability to use non-DMA IDE drives from the
ATA driver, and kill all the ethernet drivers that have alignment
requirements for their DMA engines, making m_pullup copies
necessary, and yanking all drivers that do destructive probes,
and getting rid of the F00F workaround, and yanking all support
for things hung off the floppy controller, etc. etc..

All that could be justified using exactly the same argument.
   


It's great to hear you volunteering to maintain this driver, Terry.
 

Is the code *broken*? If not, what is The Real Reason to axe it?

I'd also add that in existing environment relevance of having FreeBSD 
support for any brand new hardware is not *that* higher than relevance 
of maintaining support for existing hardware. Perhaps not higher at all.




To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message


Re: Technical Differences of *BSD and Linux

2003-02-04 Thread Terry Lambert
Sean Davis wrote:
 On Mon, Feb 03, 2003 at 10:00:48PM -0800, Terry Lambert wrote:
 snip
  This is actually incorrect.  At one point in time, the email
  address of the driver authors was printed out in boot messsages,
  under Linux.  They had a flag day in which Linus removed all
  the printf's.  This flag day was actually commercially motivated.
 
 Some (many?) Linux drivers still print out the e-mail addresses of the author.
 I base this claim on the dmesg from a 2.4.something/alpha system I saw the other
 day.

There is a boot option to make them less noisy.

I've had employers who would have liked an option to make the
FreeBSD boot less noisy.  Unfortunately, what they were actually
concerned about was obfuscating the identity of the code they
were running, rather than doing the right thing in this area,
which would be to flag the types of messages you wanted logged
to the console vs. logged to the console log, or the ability to
override the verbosity level very early in the boot.

It was possible to do what they wanted, but I was personally on
the hook, should a significant customer problem arise in the
field, because I was known to be the best diagnostician, so I
needed the minimal set of diagnostic information, and a way to
get the full set.  Knowing that the intent of the change was to
be obfuscation, which would make that job harder, failed to
motivate me to work on that, as long as there were other high
priority tasks on the task list which needed to be done, which
were technical in nature, and marketing eye candy.

I suppose if the number of priority tasks had ever drained out
of the queue sufficiently long for the work to be done, I would
have ended up doing it.  Luckily, startups, for some managers,
are all about creating crises, and then being seen by the VCs
to be masterfully resolving them (e.g. compressing release
cycles to once a quarter, etc.), and I never had enough free
time to actually be in a position to have to deal with the
otherwise absurd request.

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



RE: He dares to Ask...

2003-02-04 Thread Michael Semcheski
You have to understand that some of these people know a ton of unix because
for as long as I have been alive they have been working, hacking on Unix.
The reason questions like yours are generally not welcome in this crowd, is
because you are asking them to boil down the last 25 years of their career
so you can ask questions about OSX which have no good answers.  

Software as a product of engineering is more complex than anything else in
human history:  forget about cathedrals, suspension bridges, SR-71's.  Its
very interesting and astounding but not something you can pick up in a day.

Just realize that if you break something while working or hacking on your
system, people generally will help out of a spirit of camaraderie -- we've
all been there.  Read the docs, RFC's, whitepapers, read lists like this.
If you can grasp the concepts (not everyone can) they will eventually come
to you.  But no one is interested in spending their time satisfying
someone's morbid curiosity.  

Mike

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]] On Behalf Of Steve Kudlak
Sent: Tuesday, February 04, 2003 11:16 AM
To: '[EMAIL PROTECTED]'
Subject: He dares to Ask...


Is there a place where these *nix questions
can be asked? Where I can ask why like how
BSD like is the new MAC-OS?  Like in the
MAC OS boot-up what part is handled by
BSD. If someone says my MAC hangs with
the Happy Mac can I go down to the # prompt
and start poking around and see what is wrong.

Note I dare to ask these things here because
as I lurk usually good information comes my way
via this group. I certainly feel better about BSD
and even giggle! Linux than WIndows which
many of us are stuck with.

Have Fun,
Sends Steve


To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message


To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



[eugene@securityarchitects.com: Re: Preventing exploitation with rebasing]

2003-02-04 Thread Justin Lundy
Has similar work been done in FreeBSD been done? This would be a nice
feature in 5.0-CURRENT. We had SecureBSD, and the IBM port of propolice,
but both projects appear to be defunct at present. If we can integrate
MAC into the kernel, why not port over OpenBSD's rebasing implementation 
from /src/sys/kern/kern_exec.c? 

--jbl

- Forwarded message from Eugene Tsyrklevich [EMAIL PROTECTED] -

Mailing-List: contact [EMAIL PROTECTED]; run by ezmlm
Precedence: bulk
List-Id: bugtraq.list-id.securityfocus.com
List-Post: mailto:[EMAIL PROTECTED]
List-Help: mailto:[EMAIL PROTECTED]
List-Unsubscribe: mailto:[EMAIL PROTECTED]
List-Subscribe: mailto:[EMAIL PROTECTED]
Delivered-To: mailing list [EMAIL PROTECTED]
Delivered-To: moderator for [EMAIL PROTECTED]
Date: Tue, 4 Feb 2003 03:34:32 -0800
From: Eugene Tsyrklevich [EMAIL PROTECTED]
To: David Litchfield [EMAIL PROTECTED]
Subject: Re: Preventing exploitation with rebasing
User-Agent: Mutt/1.2.5i
In-Reply-To: 006b01c2cc0b$78d7cb70$2501010a@recovery; from [EMAIL PROTECTED] on 
Mon, Feb 03, 2003 at 09:08:35PM -0800

 Rebasing
 ***
 The problem with operating systems is that they all have pretty much the
 same genetic code which makes each and every one of them vulnerable to a
 new exploit. So we need to make them different and this can be achieved
 through rebasing. Rebasing is the process of changing the Image Base of an
 image file. By doing this the DLL/EXE is loaded into a different location in
 the virtual address space.


Similar idea, applied to the location of stack, was implemented in OpenBSD.
This is from OpenBSD CVS (August 2001):

Add a possibility to add a random offset to the stack on exec. This makes
it slightly harder to write generic buffer overflows. This doesn't really
give any real security, but it raises the bar for script-kiddies and it's
really cheap.

The range of the random offsets is controlled by the sysctl
kern.stackgap_random (must be a power of 2).


http://www.openbsd.org/cgi-bin/cvsweb/src/sys/kern/kern_exec.c.diff?r1=1.54r2=1.55

- End forwarded message 
-

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Re: [eugene@securityarchitects.com: Re: Preventing exploitation with rebasing]

2003-02-04 Thread Tim Kientzle
Justin Lundy wrote:


Add a possibility to add a random offset to the stack on exec. This makes
it slightly harder to write generic buffer overflows. This doesn't really
give any real security, but it raises the bar for script-kiddies and it's
really cheap.



This idea can also be used within applications.
Just use

  alloca(random value)

to randomize the stack location.  No kernel
changes needed.  There are more portable ways
to do this, of course, for those who dislike alloca.

Tim Kientzle


To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Re: Lower power SMP boxes?

2003-02-04 Thread Narvi

On Sat, 1 Feb 2003, Kurt J. Lidl wrote:

 On Sat, Feb 01, 2003 at 11:21:49AM -0800, Matthew Dillon wrote:
  :On Sat, Feb 01, 2003 at 10:17:23AM +0100, Mats Larsson wrote:
  : Via just recently announced their new Nehemiah processor capable of smp,
  : presumably slow as its precursor but also the lowest power consuming
  : processor at the market (at least with standard socket fcpga motherboard)
  :[...]
  : http://www.via.com.tw/en/viac3/c3.jsp
  :
  :It says IO/APIC support in future versions.  So, it's not an SMP option
  :today, as I understand it.
  :
  :-Kurt
 
  Although, this is more a deficiency in the way FreeBSD is designed.  Using
  an APIC is nice, but not absolutely necessary.  All we need are good
  specs on how VIA's SMP cpus interact with each other and we could
  support it.

 How else are you going to do the physical interrupt steering?
 Unless they have gone through the effort of implementing a whole
 new and different steering mechanism -- which would fly in the face
 of having off-the-shelf OS support from the people in Redmond, at
 the very least.


At least at some point there was a thing called OpenPIC which the then big
two alternative x86 processor vendors AMD and Cyrix promised to support.
In practice I believe it ever only got used on one or two PPC boards.

  I like the 11 watts specified in the paper.  That *is* low power for
  the class of system they are selling.  I don't see a clock specification
  but I assume it is going to be at least as fast as the ~900MHz M-9000.

 It says the new generation VIA C3 processor is available at speeds starting
 at 1GHz in the paragraph under Enhanced Digital Media Performance.

 -Kurt

 To Unsubscribe: send mail to [EMAIL PROTECTED]
 with unsubscribe freebsd-hackers in the body of the message



To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Re: Modifying mergemaster behavior

2003-02-04 Thread Dmitry Morozovsky
On Mon, 3 Feb 2003, Doug Barton wrote:

[snip]

DB  -I '$FreeBSD:.*$'
DB  to the 'diff ${DIFF_FLAG}' command in diff_loop, and it seems to have
DB  worked the way I wanted it to work.
DB
DB What do you think mm should do if the only diff between two files is the
DB cvs ID? This comes up inside a branch if a change is checked in, then
DB backed out. Of course, it happens a lot when upgrading from 4 to 5 for
DB example.

Moreover, and possibly much more frequent, when upgrading between security
branches of -stable, e.g. RELENG_4_6 - RELENG_4_7

Not that it were major PITA, but avoiding it would simplify our lives ;-)

Sincerely,
D.Marck   [DM5020, DM268-RIPE, DM3-RIPN]

*** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- [EMAIL PROTECTED] ***


To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Re: Lower power SMP boxes?

2003-02-04 Thread Benno Rice
On Wed, 2003-02-05 at 07:25, Narvi wrote:
 On Sat, 1 Feb 2003, Kurt J. Lidl wrote:

[snip]

  How else are you going to do the physical interrupt steering?
  Unless they have gone through the effort of implementing a whole
  new and different steering mechanism -- which would fly in the face
  of having off-the-shelf OS support from the people in Redmond, at
  the very least.
 
 
 At least at some point there was a thing called OpenPIC which the then big
 two alternative x86 processor vendors AMD and Cyrix promised to support.
 In practice I believe it ever only got used on one or two PPC boards.

One or two like every new world Macintosh. =)

In fact quite a lot of PowerPC boards use OpenPIC as it's specified in
the CHRP spec, which IBM and Apple follow for the most part.  I know
that Motorola's MPC10x host-pci bridge chipsets also have an
OpenPIC-compatible PIC in them.  I also wouldn't be surprised if the Mai
Logic chipset used on the Teron CX motherboards has one as well.

-- 
Benno Rice [EMAIL PROTECTED]



signature.asc
Description: This is a digitally signed message part


Re: [eugene@securityarchitects.com: Re: Preventing exploitation with rebasing]

2003-02-04 Thread Brandon D. Valentine
On Tue, Feb 04, 2003 at 11:51:14AM -0800, Justin Lundy wrote:
 Has similar work been done in FreeBSD been done? This would be a nice
 feature in 5.0-CURRENT. We had SecureBSD, and the IBM port of propolice,
 but both projects appear to be defunct at present. If we can integrate
 MAC into the kernel, why not port over OpenBSD's rebasing implementation 
 from /src/sys/kern/kern_exec.c? 
 
 - Forwarded message from Eugene Tsyrklevich [EMAIL PROTECTED] -
 Add a possibility to add a random offset to the stack on exec. This makes
 it slightly harder to write generic buffer overflows. This doesn't really
 give any real security, but it raises the bar for script-kiddies and it's
 really cheap.

AFAIK, no.  No similiar work has been done in FreeBSD.

Personally I think if one is going to expend effort in making the stack
more secure the proper way to do this is to follow NetBSD's example and
switch to a signal trampoline provided by libc so that stack pages can
be marked non-executable in the first place.  Adding random offsets to
the stack is never going to be more than a hack.

But, the surest way to test whether or not there is support for this
among actual FreeBSD developers (of which I am not one) is to post a
patch.  You'll know pretty quickly one way or the other.

Brandon D. Valentine
-- 
[EMAIL PROTECTED] http://www.geekpunk.net

We've been raised on replicas of fake and winding roads, and day after day up
on this beautiful stage we've been playing tambourine for minimum wage, but we
are real; I know we are real.  -- David Berman

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Re: [eugene@securityarchitects.com: Re: Preventing exploitation with rebasing]

2003-02-04 Thread Terry Lambert
Justin Lundy wrote:
 Add a possibility to add a random offset to the stack on exec. This makes
 it slightly harder to write generic buffer overflows. This doesn't really
 give any real security, but it raises the bar for script-kiddies and it's
 really cheap.

It's also security through obscurity.  All you have to do to get
around it is write PIC and use your own system calls.

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Re: [eugene@securityarchitects.com: Re: Preventing exploitationwith rebasing]

2003-02-04 Thread Garance A Drosihn
At 2:10 PM -0600 2/4/03, Brandon D. Valentine wrote:

On Tue, Feb 04, 2003 at 11:51:14AM -0800, Justin Lundy wrote:
  Has similar work been done in FreeBSD been done? This would be
  a nice feature in 5.0-CURRENT. We had SecureBSD, and the IBM
  port of propolice, but both projects appear to be defunct at
  present.


It would be much smarter to follow what OpenBSD is doing with
propolice, and revive a freebsd project of *that*.


  - Forwarded message from Eugene Tsyrklevich 
[EMAIL PROTECTED] -
  Add a possibility to add a random offset to the stack on exec.
  This makes it slightly harder to write generic buffer overflows.
  This doesn't really give any real security, but it raises the
  bar for script-kiddies and it's really cheap.

AFAIK, no.  No similiar work has been done in FreeBSD.

Personally I think if one is going to expend effort in making the
stack more secure the proper way to do this is to follow NetBSD's
example and switch to a signal trampoline provided by libc so that
stack pages can be marked non-executable in the first place.
Adding random offsets to the stack is never going to be more than
a hack.

I agree that random offsets will not buy much in the way of
security, but it might make some kinds of initialization errors
more obvious.  I'm thinking of the kind of errors where a routine
forgets to initialize a key variable, but everything seems to
work because the routine happens to always pick up the same
value off the stack.  By adding random offsets, the routine
*might* at least behave differently each time it's run.

Okay, I'll admit that even that is a bit of a long-shot...

--
Garance Alistair Drosehn=   [EMAIL PROTECTED]
Senior Systems Programmer   or  [EMAIL PROTECTED]
Rensselaer Polytechnic Instituteor  [EMAIL PROTECTED]

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Re: [eugene@securityarchitects.com: Re: Preventing exploitation with rebasing]

2003-02-04 Thread David Schultz
Thus spake Garance A Drosihn [EMAIL PROTECTED]:
 I agree that random offsets will not buy much in the way of
 security, but it might make some kinds of initialization errors
 more obvious.  I'm thinking of the kind of errors where a routine
 forgets to initialize a key variable, but everything seems to
 work because the routine happens to always pick up the same
 value off the stack.  By adding random offsets, the routine
 *might* at least behave differently each time it's run.

Nondeterminism is nearly always a bad thing when debugging.  Maybe
random stack offsets would be a useful component in some sort of
stress test, but I'm not sure I'd like to see such a feature used
in production.

As far as preventing buffer overflows goes, there are already
enough ad hoc techniques like Stack Guard out there that only
lessen the impact of a bug, and even then only sometimes.  A much
better approach is to develop better coding practices (better
language features) and use static checking for legacy code.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Re: Modifying mergemaster behavior

2003-02-04 Thread Doug Barton
On Tue, 4 Feb 2003, Garance A Drosihn wrote:

 At 10:36 PM -0800 2/3/03, Doug Barton wrote:
 On Mon, 3 Feb 2003, Garance A Drosihn wrote:
I added:
   -I '$FreeBSD:.*$'
to the 'diff ${DIFF_FLAG}' command in diff_loop, and it seems to
have worked the way I wanted it to work.
 
 How did you want it to work?  (This isn't a rhetorical question.)

 I do not care to see what the version-string is, and I certainly
 don't need to see the two un-changing lines before and after the
 version-string.  When you also count the diff-output line which
 gives the line numbers of the change, that's a total of six lines
 which appear on the screen and which give me (IMO) very little
 useful information.

Ok, thanks for that description.

 Try -I and watch how it behaves.

I have, actually.

I suppose we should also change the 'absolute diff first' part
to include that, or people will be asked about merging a file,
only to be shown a file which apparently has zero changes.
 
 I'm sorry, I don't understand exactly what you're getting at here.

 There's a section of mergemaster that starts out with the comment
 Do an absolute diff first to see if the files are actually different.

That's an option that users can enable, it's not the default behavior.

 What I was saying is we should add the same -I argument to that
 diff command,

That's not going to happen... the whole point of strict comparison mode is
that ALL differences are displayed, no matter what. Since users choose
this option, it's reasonable not to change its behavior.

 In thinking about it now, I realize that I (personally) would
 prefer that mergemaster would just install the file in that case,
 but the result of adding the -I to that diff command is that the
 file would be quietly skipped over.  I could live with that too,
 but I'd feel better to have it automatically installed.  JMO.

I'm really, really hesitant to do this. The first time that this goes
wrong, the peasants will be after my ass with torches and pitchforks, and
they will be well justified. I'll give some thought to it though...  it
may be worth adding this as an option, with a big fat warning.

If so, my gut reaction is that if a lot of FreeBSD users are
requesting this, then we should figure out how to make it
happen -- one way or another.

 What I was getting at is that mergemaster is going to be run fairly
 often by a significant number of freebsd users.  If we need make some
 changes to make that process more pleasant, then I think we should
 feel free to do that.

My primary design goal, and the one that I've jealously guarded from
people clamoring for options that increase the pleasantness of the mm
experience is that running mergemaster should always be SAFE. It never
alters any existing file without explicit user input. The -i option is a
slight bastardization of my original design, but one that I was willing to
compromise on because it doesn't touch things that already exist.

I'm snipping your discussion about adding options to diff, since I can't
code mm to deal with diff options that don't exist yet.  I already
added stat(1) to the base to deal with the demise of perl.  :)

What I can do in the short term is make a couple small changes to mm in
order to make it possible to add your -I$FreeBSD to the DIFF_FLAG variable
in .mergemasterrc so that folks can test this behavior easily, then give
me further feedback on it.

Thanks again for your feedback, I appreciate you carefully describing what
you want, it makes it a lot easier for me to respond, and improve the
program.

Doug

-- 

If it's moving, encrypt it. If it's not moving, encrypt
  it till it moves, then encrypt it some more.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Re: Modifying mergemaster behavior

2003-02-04 Thread Garance A Drosihn
At 11:43 PM -0800 2/4/03, Doug Barton wrote:

On Tue, 4 Feb 2003, Garance A Drosihn wrote:

  At 10:36 PM -0800 2/3/03, Doug Barton wrote:
  There's a section of mergemaster that starts out with the comment

 Do an absolute diff first to see if the files are actually different.


That's an option that users can enable, it's not the default behavior.


 What I was saying is we should add the same -I argument to that
 diff command,


That's not going to happen... the whole point of strict comparison
mode is that ALL differences are displayed, no matter what. Since
users choose this option, it's reasonable not to change its behavior.


Oh, I didn't realize that was an optional behavior, I thought
that was the default code path.  That request is my mistake then,
for skimming through the script a little too quickly.

--
Garance Alistair Drosehn=   [EMAIL PROTECTED]
Senior Systems Programmer   or  [EMAIL PROTECTED]
Rensselaer Polytechnic Instituteor  [EMAIL PROTECTED]

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message



Re: Modifying mergemaster behavior

2003-02-04 Thread Doug Barton
On Wed, 5 Feb 2003, Garance A Drosihn wrote:

 At 11:43 PM -0800 2/4/03, Doug Barton wrote:
 On Tue, 4 Feb 2003, Garance A Drosihn wrote:
 
At 10:36 PM -0800 2/3/03, Doug Barton wrote:
There's a section of mergemaster that starts out with the comment
   Do an absolute diff first to see if the files are actually different.
 
 That's an option that users can enable, it's not the default behavior.
 
   What I was saying is we should add the same -I argument to that
   diff command,
 
 That's not going to happen... the whole point of strict comparison
 mode is that ALL differences are displayed, no matter what. Since
 users choose this option, it's reasonable not to change its behavior.

 Oh, I didn't realize that was an optional behavior, I thought
 that was the default code path.  That request is my mistake then,
 for skimming through the script a little too quickly.

Yeah, I had the feeling that you thought it was the default... I meant to
say something to that effect. I confess that the style isn't 100% readable
to people who haven't spent as much time as I have working on it... my
only defense is that when that script started, it was a whole lot simpler
than it is now. :)  Of course, there are some aspects of the style that I
actually like, so I guess I'm not totally blameless here.

Doug

-- 

If it's moving, encrypt it. If it's not moving, encrypt
  it till it moves, then encrypt it some more.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message