Re: panic: random.ko
# [EMAIL PROTECTED] / 2004-04-04 19:02:07 -0700: On Mon, Apr 05, 2004 at 02:35:53AM +0200, Roman Neuhauser wrote: Booting a kernel with random compiled in, and load_random=YES in loader.conf causes a panic very similar to the one described here: http://lists.freebsd.org/pipermail/freebsd-alpha/2003-August/000656.html Hi Mark, please fix random.ko :-) Sounds like a generic module problem that is known to exist on FreeBSD. Don't do that is the solution. You can unload the preloaded kernel and/or modules from the loader to prevent it from panicking at boot, then modify loader.conf. The fun part (and reason I asked whether there was a way to tell loader to ignore loader.con) was that unload didn't help, it showed the kernel and modules were unloaded, but subsequent load kernel boot or boot kernel loaded the the modules again, and I had to disable-module for every individual module, about twelve times. Is that normal behavior, or was there a pilot error somewhere? -- If you cc me or remove the list(s) completely I'll most likely ignore your message.see http://www.eyrie.org./~eagle/faqs/questions.html ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: panic: random.ko
On Wed, Apr 21, 2004 at 10:20:14AM +0200, Roman Neuhauser wrote: # [EMAIL PROTECTED] / 2004-04-04 19:02:07 -0700: On Mon, Apr 05, 2004 at 02:35:53AM +0200, Roman Neuhauser wrote: Booting a kernel with random compiled in, and load_random=YES in loader.conf causes a panic very similar to the one described here: http://lists.freebsd.org/pipermail/freebsd-alpha/2003-August/000656.html Hi Mark, please fix random.ko :-) Sounds like a generic module problem that is known to exist on FreeBSD. Don't do that is the solution. You can unload the preloaded kernel and/or modules from the loader to prevent it from panicking at boot, then modify loader.conf. The fun part (and reason I asked whether there was a way to tell loader to ignore loader.con) was that unload didn't help, it showed the kernel and modules were unloaded, but subsequent load kernel boot or boot kernel loaded the the modules again, and I had to disable-module for every individual module, about twelve times. Is that normal behavior, or was there a pilot error somewhere? I think it's expected behaviour. My fingers know what to do to stop this, but my brain can't remember it right now :-) Kris pgp0.pgp Description: PGP signature
Re: Detecting 'floppy' like umass devices
On Wed, Apr 21, 2004 at 01:29:29PM +0930, Daniel O'Connor wrote: On Tue, 20 Apr 2004 18:28, Bernd Walter wrote: I am wondering if there is any way of telling if a given umass device is a floppy drive (or wants to look like one) - eg I have a USB FDD which I imagine should fall into the same basket. What do do you mean with wants to look like one. In which way does a floppy look different from other direct access drives? They all read and store direct access data. Partially to handle things like fdformat, and density selection, but also from a user point of view, ie it would be nice if it appeared as /dev/fdX. camcontrol format? OK - density has to be configured raw. Do USB floppies support different formats anyway? IMHO it's not obvious (and dangerous) to tell mtools that a: = /dev/da0 but that's what I have to do if I want to use my USB FDD with it. That's what I have to face with my MO drives since years. And it's the same story that you can overwrite the wrong streamer tape, ... The real point that is missing here is working hardwiring of USB mass storage devices. Plus I never liked ad0 vs. da0 problematic - typos can me made too quick. But that's the way is is: You are root. I note that you get wacky values from fdisk when you try and read partition table from them too.. On another note my USB floppy drive does 2k/sec in FreeBSD :( Sound like another instance of msdosfs does no clustering and drive is too stupid to get speed without. IIRC there were some work on this point, but I don't now the state. Check the speed with dd and different blocksizes. This IS with dd :) I did a few tests.. Block Size | Speed ===+=== 0.5k | 2.2k/sec 1 k | 4.4k/sec 2 k | 8.0k/sec 4 k | 12.6k/sec 8 k | 17.7k/sec 16 k | 21.8k/sec 32 k | 23.9k/sec 64 k | 26.2k/sec Bleh :( Yes that drive would massivly win from IO clustering. UFS should work fine, but msdosfs... -- B.Walter BWCThttp://www.bwct.de [EMAIL PROTECTED] [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: panic: random.ko
Kris Kennaway writes: loader to ignore loader.con) was that unload didn't help, it showed the kernel and modules were unloaded, but subsequent load kernel boot or boot kernel loaded the the modules again, and I had to disable-module for every individual module, about twelve times. Is that normal behavior, or was there a pilot error somewhere? I think it's expected behaviour. My fingers know what to do to stop this, but my brain can't remember it right now :-) unload load /boot/kernel/kernel boot does it for me. M -- Mark Murray iumop ap!sdn w,I idlaH ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [patch] Raw sockets in jails
Poul/group
The following patch makes raw sockets comply with prison IP addresses.
Some tools such as traceroute(8) may require that the prison IP address
be specified on the command line. I.E.
traceroute -s prison ip dest address
Otherwise it might fail.
(because of this we may want to get rid of the
create_raw_sockets MIB all together).
Anyway, take a gander at it (testers feedback welcome):
Regards
Christian S.J. Peron
--- sys/netinet/raw_ip.c.b Mon Apr 19 16:23:57 2004
+++ sys/netinet/raw_ip.cTue Apr 20 19:43:30 2004
@@ -40,6 +40,7 @@
#include opt_random_ip_id.h
#include sys/param.h
+#include sys/jail.h
#include sys/kernel.h
#include sys/lock.h
#include sys/mac.h
@@ -215,6 +216,11 @@
if (inp-inp_faddr.s_addr
inp-inp_faddr.s_addr != ip-ip_src.s_addr)
goto docontinue;
+ if (inp-inp_socket-so_cred-cr_prison) {
+ if (htonl(inp-inp_socket-so_cred-cr_prison-pr_ip)
+ != ip-ip_dst.s_addr)
+ goto docontinue;
+ }
if (last) {
struct mbuf *n;
@@ -270,7 +276,11 @@
ip-ip_off = 0;
ip-ip_p = inp-inp_ip_p;
ip-ip_len = m-m_pkthdr.len;
- ip-ip_src = inp-inp_laddr;
+ if (inp-inp_socket-so_cred-cr_prison)
+ ip-ip_src.s_addr =
+ htonl(inp-inp_socket-so_cred-cr_prison-pr_ip);
+ else
+ ip-ip_src = inp-inp_laddr;
ip-ip_dst.s_addr = dst;
ip-ip_ttl = inp-inp_ip_ttl;
} else {
@@ -279,6 +289,13 @@
return(EMSGSIZE);
}
ip = mtod(m, struct ip *);
+ if (inp-inp_socket-so_cred-cr_prison) {
+ if (ip-ip_src.s_addr !=
+ htonl(inp-inp_socket-so_cred-cr_prison-pr_ip)) {
+ m_freem(m);
+ return (EPERM);
+ }
+ }
/* don't allow both user specified and setsockopt options,
and don't allow packet length sizes that will crash */
if (((ip-ip_hl != (sizeof (*ip) 2))
@@ -505,6 +522,7 @@
}
}
+extern int jail_allow_raw_sockets;
u_long rip_sendspace = RIPSNDQ;
u_long rip_recvspace = RIPRCVQ;
@@ -527,7 +545,11 @@
INP_INFO_WUNLOCK(ripcbinfo);
return EINVAL;
}
- if (td (error = suser(td)) != 0) {
+ if (td jailed(td-td_ucred) !jail_allow_raw_sockets) {
+ INP_INFO_WUNLOCK(ripcbinfo);
+ return (EPERM);
+ }
+ if (td (error = suser_cred(td-td_ucred, PRISON_ROOT)) != 0) {
INP_INFO_WUNLOCK(ripcbinfo);
return error;
}
@@ -626,6 +648,11 @@
if (nam-sa_len != sizeof(*addr))
return EINVAL;
+
+ if (td-td_ucred-cr_prison)
+ if (htonl(td-td_ucred-cr_prison-pr_ip)
+ != addr-sin_addr.s_addr)
+ return (EADDRNOTAVAIL);
if (TAILQ_EMPTY(ifnet) ||
(addr-sin_family != AF_INET addr-sin_family != AF_IMPLINK) ||
--- sys/kern/kern_jail.c.bakMon Apr 19 16:55:40 2004
+++ sys/kern/kern_jail.cMon Apr 19 17:56:03 2004
@@ -53,6 +53,11 @@
jail_sysvipc_allowed, 0,
Processes in jail can use System V IPC primitives);
+intjail_allow_raw_sockets = 0;
+SYSCTL_INT(_security_jail, OID_AUTO, allow_raw_sockets, CTLFLAG_RW,
+jail_allow_raw_sockets, 0,
+Prison root can create raw sockets);
+
/* allprison, lastprid, and prisoncount are protected by allprison_mtx. */
struct prisonlist allprison;
struct mtx allprison_mtx;
On 20 Apr 2004 Poul-Henning Kamp wrote:
In message [EMAIL PROTECTED], Christian S.J. Peron w
rites:
Although RAW sockets can be used when specifying the source
address of packets (defeating one of the aspects of the jail)
some people may find it usefull to use utilities like ping(8)
or traceroute(8) from inside jails.
Enclosed is a patch I have written which gives you the option
of allowing prison-root to create raw sockets inside the prison,
so that programs various network debugging programs like ping
and traceroute etc can be used.
This patch will create the security.jail.allow_raw_sockets sysctl
MIB. I would appriciate any feed-back from testers
See PR #:
http://www.freebsd.org/cgi/query-pr.cgi?pr=65800
Could you take a peek and see how hard it would be to enforce source-IP
compliance with the jail restriction ?
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Re: Help: threading problem in non-threaded applications
On Wed, 21 Apr 2004, Joe Marcus Clarke wrote: On Tue, 2004-04-20 at 17:08, Daniel Eischen wrote: On Tue, 20 Apr 2004, Joe Marcus Clarke wrote: I have a problem I'm hoping someone can help me with. GTK+ 2.4 introduced a new file selection GUI which works just fine in threaded and non-threaded applications. However, GNOME 2.6 augmented this dialog with a dynamically loadable threaded shared object. The GNOME version is automatically used by all GTK+ apps when run under a GNOME desktop if libgnomeui is installed. Shared libraries shouldn't link with threading libraries unless they actually create threads behind the scenes. Actually, even so, they could force the (unthreaded) applications that link with them to explicitly supply the thread library in the link option. And that's the case here. The underlying libraries are creating and using threads. But what happens when a non-threaded application loads a thread library via dlopen() (which is the case here)? Here's the stack trace I see: In that case you're going to have to link the application to a threads library. -- Dan Eischen ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Help: threading problem in non-threaded applications
On Wed, 2004-04-21 at 09:18, Daniel Eischen wrote: On Wed, 21 Apr 2004, Joe Marcus Clarke wrote: On Tue, 2004-04-20 at 17:08, Daniel Eischen wrote: On Tue, 20 Apr 2004, Joe Marcus Clarke wrote: I have a problem I'm hoping someone can help me with. GTK+ 2.4 introduced a new file selection GUI which works just fine in threaded and non-threaded applications. However, GNOME 2.6 augmented this dialog with a dynamically loadable threaded shared object. The GNOME version is automatically used by all GTK+ apps when run under a GNOME desktop if libgnomeui is installed. Shared libraries shouldn't link with threading libraries unless they actually create threads behind the scenes. Actually, even so, they could force the (unthreaded) applications that link with them to explicitly supply the thread library in the link option. And that's the case here. The underlying libraries are creating and using threads. But what happens when a non-threaded application loads a thread library via dlopen() (which is the case here)? Here's the stack trace I see: In that case you're going to have to link the application to a threads library. I had another thought based on your comments. Since -pthread does the right thing in -CURRENT, why don't we make that the default PTHREAD_LIBS? Why do we explicitly link to -lpthread when that causes shared libs to be linked to threading libraries? If we did -pthread universally, this problem would go away because ORBit would do the right thing in the non-threaded case. Joe -- Joe Marcus Clarke FreeBSD GNOME Team :: [EMAIL PROTECTED] [EMAIL PROTECTED] FreeNode / #freebsd-gnome http://www.FreeBSD.org/gnome signature.asc Description: This is a digitally signed message part
Re: Help: threading problem in non-threaded applications
On Wed, 21 Apr 2004, Joe Marcus Clarke wrote: On Wed, 2004-04-21 at 09:18, Daniel Eischen wrote: On Wed, 21 Apr 2004, Joe Marcus Clarke wrote: On Tue, 2004-04-20 at 17:08, Daniel Eischen wrote: On Tue, 20 Apr 2004, Joe Marcus Clarke wrote: I have a problem I'm hoping someone can help me with. GTK+ 2.4 introduced a new file selection GUI which works just fine in threaded and non-threaded applications. However, GNOME 2.6 augmented this dialog with a dynamically loadable threaded shared object. The GNOME version is automatically used by all GTK+ apps when run under a GNOME desktop if libgnomeui is installed. Shared libraries shouldn't link with threading libraries unless they actually create threads behind the scenes. Actually, even so, they could force the (unthreaded) applications that link with them to explicitly supply the thread library in the link option. And that's the case here. The underlying libraries are creating and using threads. But what happens when a non-threaded application loads a thread library via dlopen() (which is the case here)? Here's the stack trace I see: In that case you're going to have to link the application to a threads library. I had another thought based on your comments. Since -pthread does the right thing in -CURRENT, why don't we make that the default PTHREAD_LIBS? Why do we explicitly link to -lpthread when that causes shared libs to be linked to threading libraries? If we did -pthread universally, this problem would go away because ORBit would do the right thing in the non-threaded case. I thought your example above was creating threads behind the scenes (in the library). That would either require linking the library with -lpthread or every application that used it with -lpthread. Actually, if the library is dynamically loaded, then you still need to explicitly link the application with the threads library (or else libc is going to get confused). I think most of the problem is that our synchronization types are private to each threads library. For FreeBSD-6, we'll change that and have the synchronization types uniformly defined so that libc can see them the same way that the threads library sees them. Perhaps pthread_t will also have to be uniformly defined. So a mutex initialized by libc can be used by libpthread... I don't have a problem with PTHREAD_LIBS being changed to -pthread. I don't know if that will actually solve your particular problem, but it would probably solve other problems and make it easier to mix'n match thread libraries with applications that use the same shared libraries. -- Dan Eischen ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: panic: random.ko
# [EMAIL PROTECTED] / 2004-04-21 12:03:36 +0100: Kris Kennaway writes: loader to ignore loader.con) was that unload didn't help, it showed the kernel and modules were unloaded, but subsequent load kernel boot or boot kernel loaded the the modules again, and I had to disable-module for every individual module, about twelve times. Is that normal behavior, or was there a pilot error somewhere? I think it's expected behaviour. My fingers know what to do to stop this, but my brain can't remember it right now :-) unload load /boot/kernel/kernel boot does it for me. Of course I run unload as the first step. It wrote it unloaded the modules, and the started loading them again booting the new kernel. -- If you cc me or remove the list(s) completely I'll most likely ignore your message.see http://www.eyrie.org./~eagle/faqs/questions.html ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
how to flush out cache.?
Ok so I have an application where I need to reread a file I have just written to ensure that it went to disk correctly.. Other than reading a few GB of data, is there a way to flush out the cache copy of a file I've written? possibilities include: a file flag saying don't keep a copy after it's written to disk? a syscall discard_cached_blocks(fd); ? any other suggestions? julian (BTW this would be for 4.x initially) ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
In article [EMAIL PROTECTED] you write: Ok so I have an application where I need to reread a file I have just written to ensure that it went to disk correctly.. Other than reading a few GB of data, is there a way to flush out the cache copy of a file I've written? You can flush the file to disk with fsync(). But I don't think that evicts the blocks from the cache, just forces them to be written. To the disk controller's cache, if you're lucky. possibilities include: a file flag saying don't keep a copy after it's written to disk? a syscall discard_cached_blocks(fd); ? any other suggestions? What are you hoping to accomplish? There are probably other ways to solve the larger problem. -- Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.8 / 37N 20' 14.9 Internet: steve @ Watt.COM Whois: SW32 Free time? There's no such thing. It just comes in varying prices... ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
On Wed, 21 Apr 2004, Steve Watt wrote: In article [EMAIL PROTECTED] you write: Ok so I have an application where I need to reread a file I have just written to ensure that it went to disk correctly.. Other than reading a few GB of data, is there a way to flush out the cache copy of a file I've written? You can flush the file to disk with fsync(). But I don't think that evicts the blocks from the cache, just forces them to be written. To the disk controller's cache, if you're lucky. possibilities include: a file flag saying don't keep a copy after it's written to disk? a syscall discard_cached_blocks(fd); ? any other suggestions? What are you hoping to accomplish? There are probably other ways to solve the larger problem. I thought I was being clear.. I need to remove all the pages from cache so that a reread of the file is forced to go to disk. and I don't want to go read a 2GB dummy file to force the flush.. Someone suggested that I read the file usign 'dump' through the raw device.. -- Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.8 / 37N 20' 14.9 Internet: steve @ Watt.COM Whois: SW32 Free time? There's no such thing. It just comes in varying prices... ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
On Wed, 21 Apr 2004, Julian Elischer wrote: On Wed, 21 Apr 2004, Steve Watt wrote: In article [EMAIL PROTECTED] you write: Ok so I have an application where I need to reread a file I have just written to ensure that it went to disk correctly.. Other than reading a few GB of data, is there a way to flush out the cache copy of a file I've written? You can flush the file to disk with fsync(). But I don't think that evicts the blocks from the cache, just forces them to be written. To the disk controller's cache, if you're lucky. possibilities include: a file flag saying don't keep a copy after it's written to disk? a syscall discard_cached_blocks(fd); Turns out O_DIRECT might do this, but I can't use this as apparently it's being written by a 3rd party program.. I can only check the file after the other program has written it.. ? any other suggestions? What are you hoping to accomplish? There are probably other ways to solve the larger problem. I thought I was being clear.. I need to remove all the pages from cache so that a reread of the file is forced to go to disk. and I don't want to go read a 2GB dummy file to force the flush.. Someone suggested that I read the file usign 'dump' through the raw device.. -- Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.8 / 37N 20' 14.9 Internet: steve @ Watt.COM Whois: SW32 Free time? There's no such thing. It just comes in varying prices... ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
On Apr 21, 13:28, Julian Elischer wrote: } Subject: Re: how to flush out cache.? } } On Wed, 21 Apr 2004, Steve Watt wrote: } } In article [EMAIL PROTECTED] you write: } } Ok so I have an application where I need to } reread a file I have just written to ensure that it went to disk } correctly.. } } What are you hoping to accomplish? There are probably other ways } to solve the larger problem. } } I thought I was being clear.. } I need to remove all the pages from cache so that a reread of the file } is forced to go to disk. } and I don't want to go read a 2GB dummy file to force the flush.. No, my question is what are you trying to accomplish with the reread, at a higher level than I want to know it's on disk. Is there some reason you have for not trusting the hardware? Are you trying to do a database commit protocol? Debugging the storage system? } Someone suggested that I read the file usign 'dump' through the raw } device.. Even doing that doesn't necessarily mean the bits have made it onto the rotating media. There can also be caches in the disk controller, and/or caches on the drive itself. If you're trying for a case where you want to pull the power, unmounting and remounting the filesystem will get it about as close as you can. -- Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.8 / 37N 20' 14.9 Internet: steve @ Watt.COM Whois: SW32 Free time? There's no such thing. It just comes in varying prices... ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
At 12:23 PM -0700 4/21/04, Julian Elischer wrote: Ok so I have an application where I need to reread a file I have just written to ensure that it went to disk correctly.. Other than reading a few GB of data, is there a way to flush out the cache copy of a file I've written? possibilities include: a file flag saying don't keep a copy after it's written to disk? a syscall discard_cached_blocks(fd); ? any other suggestions? julian (BTW this would be for 4.x initially) Hmm. That means you couldn't use a 'snapshot' to force the issue, right? If you *could* use snapshots, you could close the file, snapshot the partition, mount the snapshot, and read the file from it's copy in the snapshot-filesystem. -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
On Wed, 21 Apr 2004, Steve Watt wrote: On Apr 21, 13:28, Julian Elischer wrote: } Subject: Re: how to flush out cache.? } } On Wed, 21 Apr 2004, Steve Watt wrote: } } In article [EMAIL PROTECTED] you write: } } Ok so I have an application where I need to } reread a file I have just written to ensure that it went to disk } correctly.. } } What are you hoping to accomplish? There are probably other ways } to solve the larger problem. } } I thought I was being clear.. } I need to remove all the pages from cache so that a reread of the file } is forced to go to disk. } and I don't want to go read a 2GB dummy file to force the flush.. No, my question is what are you trying to accomplish with the reread, at a higher level than I want to know it's on disk. Is there some reason you have for not trusting the hardware? Are you trying to do a database commit protocol? Debugging the storage system? we are getting data curruptions occasionally and we are trying to track it down.. if we wait a half hour so the cache is flushed out, teh file sometimes checksums differently and has bad data in it but by then the original files have gone away so we have a tough time recreating the data.. This is also to help us figure out where the problem is... but since we have seen this several times we'd like to add a check that data on disk option to our application, to help track this down in the future if it appears to be happenning again. if it's happenned several times it can happen again. } Someone suggested that I read the file usign 'dump' through the raw } device.. Even doing that doesn't necessarily mean the bits have made it onto the rotating media. There can also be caches in the disk controller, and/or caches on the drive itself. If you're trying for a case where you want to pull the power, unmounting and remounting the filesystem will get it about as close as you can. -- Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.8 / 37N 20' 14.9 Internet: steve @ Watt.COM Whois: SW32 Free time? There's no such thing. It just comes in varying prices... ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
On Wed, 21 Apr 2004, Steve Watt wrote: On Apr 21, 13:28, Julian Elischer wrote: } Subject: Re: how to flush out cache.? } } On Wed, 21 Apr 2004, Steve Watt wrote: } } In article [EMAIL PROTECTED] you write: } } Ok so I have an application where I need to } reread a file I have just written to ensure that it went to disk } correctly.. } } What are you hoping to accomplish? There are probably other ways } to solve the larger problem. } } I thought I was being clear.. } I need to remove all the pages from cache so that a reread of the file } is forced to go to disk. } and I don't want to go read a 2GB dummy file to force the flush.. No, my question is what are you trying to accomplish with the reread, at a higher level than I want to know it's on disk. Is there some reason you have for not trusting the hardware? Are you trying to do a database commit protocol? Debugging the storage system? } Someone suggested that I read the file usign 'dump' through the raw } device.. Even doing that doesn't necessarily mean the bits have made it onto the rotating media. There can also be caches in the disk controller, and/or caches on the drive itself. If you're trying for a case where you want to pull the power, unmounting and remounting the filesystem will get it about as close as you can. The disk caches are small enough.. we are talking about multi gigabyte files getting a few blocks bad somewhere in the middle. (and yes the machines have enough RAM to cache the files). -- Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.8 / 37N 20' 14.9 Internet: steve @ Watt.COM Whois: SW32 Free time? There's no such thing. It just comes in varying prices... ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
On Wed, 21 Apr 2004, Garance A Drosihn wrote: At 12:23 PM -0700 4/21/04, Julian Elischer wrote: Ok so I have an application where I need to reread a file I have just written to ensure that it went to disk correctly.. Other than reading a few GB of data, is there a way to flush out the cache copy of a file I've written? possibilities include: a file flag saying don't keep a copy after it's written to disk? a syscall discard_cached_blocks(fd); ? any other suggestions? julian (BTW this would be for 4.x initially) Hmm. That means you couldn't use a 'snapshot' to force the issue, right? If you *could* use snapshots, you could close the file, snapshot the partition, mount the snapshot, and read the file from it's copy in the snapshot-filesystem. ingenious, but, no we can't do that.. As I mentionned.. someone suggested using 'dump' to extract the file from raw disk and checksumming that... :-) -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
On Wed, Apr 21, 2004 at 01:47:43PM -0700, Julian Elischer wrote: ingenious, but, no we can't do that.. As I mentionned.. someone suggested using 'dump' to extract the file from raw disk and checksumming that... :-) If it is SCSI, you could probably put a second controler on the SCSI bus, and when you want to checksum the file you mount the filesystem read-only using using the device for the same disk via the second SCSI controler. David. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
mmap() and msync(..MS_INVALIDATE..) should work. Stephan Ok so I have an application where I need to reread a file I have just written to ensure that it went to disk correctly.. Other than reading a few GB of data, is there a way to flush out the cache copy of a file I've written? possibilities include: a file flag saying don't keep a copy after it's written to disk? a syscall discard_cached_blocks(fd); ? any other suggestions? julian (BTW this would be for 4.x initially) ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
On Wed, 21 Apr 2004, Stephan Uphoff wrote: mmap() and msync(..MS_INVALIDATE..) should work. hmmm that is rather interesting.. I wonder if it would work Maybe a vm guru could confirm this.. (under 4.x) Stephan Ok so I have an application where I need to reread a file I have just written to ensure that it went to disk correctly.. Other than reading a few GB of data, is there a way to flush out the cache copy of a file I've written? possibilities include: a file flag saying don't keep a copy after it's written to disk? a syscall discard_cached_blocks(fd); ? any other suggestions? julian (BTW this would be for 4.x initially) ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: panic: random.ko
Roman Neuhauser writes: unload load /boot/kernel/kernel boot does it for me. Of course I run unload as the first step. It wrote it unloaded the modules, and the started loading them again booting the new kernel. Did you type the exact THREE commands as shown above? M -- Mark Murray iumop ap!sdn w,I idlaH ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: panic: random.ko
# [EMAIL PROTECTED] / 2004-04-21 23:54:43 +0100: Roman Neuhauser writes: unload load /boot/kernel/kernel boot does it for me. Of course I run unload as the first step. It wrote it unloaded the modules, and the started loading them again booting the new kernel. Did you type the exact THREE commands as shown above? Of course, it's been three weeks since I had that problem so I would be insane to claim I have exact recollection. I might have been typing unload boot /boot/kernel/kernel over and over again in an attempt to convince myself that I had made a typo, but I doubt that. Nevertheless, would that make a difference? It's more likely (= I'm quite sure) I tried both versions more than once; I *really* wanted to avoid typing disable-module foo twelve or so times, which I ended up doing. Also note that we're talking about CURRENT in case that could mean a difference. Hm, I still have the exact same kernel/world on the machine. I'll try it out in the morning, and will let you know. -- If you cc me or remove the list(s) completely I'll most likely ignore your message.see http://www.eyrie.org./~eagle/faqs/questions.html ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
: : : mmap() and msync(..MS_INVALIDATE..) should work. : :hmmm that is rather interesting.. :I wonder if it would work :Maybe a vm guru could confirm this.. (under 4.x) : Huh. If I hadn't looked at the code I would have said that MS_INVALIDATE doesn't work in FreeBSD, but when I look at the code it sure looks like it ought to work! But, alas, it does not. The invalidation request goes all the way through to the vnode pager but it looks like the vnode pager ignores it. MS_INVALIDATE - OBJPC_INVAL - VM_PAGER_PUT_INVAL - IO_INVAL - (ignored) IO_INVAL is defined to be 'invalidate after I/O completes', not 'throw away the dirty data', but the only place it appears to be implemented is in the NFS code. -Matt ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
On Wed, 21 Apr 2004, Matthew Dillon wrote: : : : mmap() and msync(..MS_INVALIDATE..) should work. : :hmmm that is rather interesting.. :I wonder if it would work :Maybe a vm guru could confirm this.. (under 4.x) : Huh. If I hadn't looked at the code I would have said that MS_INVALIDATE doesn't work in FreeBSD, but when I look at the code it sure looks like it ought to work! But, alas, it does not. The invalidation request goes all the way through to the vnode pager but it looks like the vnode pager ignores it. MS_INVALIDATE - OBJPC_INVAL - VM_PAGER_PUT_INVAL - IO_INVAL - (ignored) IO_INVAL is defined to be 'invalidate after I/O completes', not 'throw away the dirty data', but the only place it appears to be implemented is in the NFS code. Actually what I'm looking for is throw away clean data I want to dump the cached version of a file so that I can force a reread of the disk. -Matt ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
:Actually what I'm looking for is :throw away clean data : :I want to dump the cached version of a file so that I can force a reread :of the disk. MS_INVALIDATE doesn't do that. madvise()'s MADV_FREE does what you want, BUT it does not currently work (at least on 4.x or in DFly) on file-backed data, it only works with anonymous memory. I believe that on some systems MADV_FREE does what you expect, e.g. like on Solaris (though I am not 100% sure), so it would not be far-fetched to go and implement it. -Matt Matthew Dillon [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
Matthew Dillon wrote: : : : mmap() and msync(..MS_INVALIDATE..) should work. : :hmmm that is rather interesting.. :I wonder if it would work :Maybe a vm guru could confirm this.. (under 4.x) : Huh. If I hadn't looked at the code I would have said that MS_INVALIDATE doesn't work in FreeBSD, but when I look at the code it sure looks like it ought to work! But, alas, it does not. The invalidation request goes all the way through to the vnode pager but it looks like the vnode pager ignores it. MS_INVALIDATE - OBJPC_INVAL - VM_PAGER_PUT_INVAL - IO_INVAL - (ignored) IO_INVAL is defined to be 'invalidate after I/O completes', not 'throw away the dirty data', but the only place it appears to be implemented is in the NFS code. -Matt Yes - but FreeBSD then calls vm_object_page_remove to remove the pages from the vnode object. (vm_object_sync for 5.x or vm_map_clean for 4.x ) Mhhh .. 4.X removes the pages without flushing if the file is not write mapped. This means that anyone with read access can screw up a file. ( For example periodically throwing away pages from /var/log/wtmp ...) 5.x looks better - but there is a race condition that can cause the same thing. I think for msync() the vm_object_page_remove() function should be called with clean_only set to TRUE to avoid throwing away valid changes. ( I will send-pr tomorrow ) Stephan ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
Stephan Uphoff wrote: Matthew Dillon wrote: : : : mmap() and msync(..MS_INVALIDATE..) should work. : :hmmm that is rather interesting.. :I wonder if it would work :Maybe a vm guru could confirm this.. (under 4.x) : Huh. If I hadn't looked at the code I would have said that MS_INVALIDATE doesn't work in FreeBSD, but when I look at the code it sure looks like it ought to work! But, alas, it does not. The invalidation request goes all the way through to the vnode pager but it looks like the vnode pager ignores it. MS_INVALIDATE - OBJPC_INVAL - VM_PAGER_PUT_INVAL - IO_INVAL - (ignored) IO_INVAL is defined to be 'invalidate after I/O completes', not 'throw away the dirty data', but the only place it appears to be implemented is in the NFS code. -Matt Yes - but FreeBSD then calls vm_object_page_remove to remove the pages from the vnode object. (vm_object_sync for 5.x or vm_map_clean for 4.x ) Mhhh .. 4.X removes the pages without flushing if the file is not write mapped. This means that anyone with read access can screw up a file. A file with dirty (unflushed) pages. ( For example periodically throwing away pages from /var/log/wtmp ...) 5.x looks better - but there is a race condition that can cause the same thing. I think for msync() the vm_object_page_remove() function should be called with clean_only set to TRUE to avoid throwing away valid changes. ( I will send-pr tomorrow ) Stephan ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to flush out cache.?
:Yes - but FreeBSD then calls vm_object_page_remove to remove the pages :from the vnode object. (vm_object_sync for 5.x or vm_map_clean for 4.x ) : :Mhhh .. 4.X removes the pages without flushing if the file is not write :mapped. :This means that anyone with read access can screw up a file. :( For example periodically throwing away pages from /var/log/wtmp ...) :5.x looks better - but there is a race condition that can cause the same thing. :I think for msync() the vm_object_page_remove() function should be called with :clean_only set to TRUE to avoid throwing away valid changes. :( I will send-pr tomorrow ) : : Stephan I don't quite see that. Could you point out the code in question? (And, of course, a test program would tell us for sure whether that hole exists). There are two different things being removed... the page table entries are removed from pmap via the vm_map for the process, and the pages in the underlying object are being cleaned. Just removing pages from a pmap will not destroy the underlying pages. (I'm not saying that you are wrong, just that we need more information to figure out what is actually happening here). -Matt Matthew Dillon [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
