Re: [PATCH 1/2] KEYS: Implement a big key type that can save to tmpfs

2013-08-09 Thread Nico Williams
On Thu, Aug 8, 2013 at 9:46 AM, David Howells wrote: > Nico Williams wrote: > >> b) how to create tmpfs locations in which to store credentials (which >> can be unbounded in size, so storing them in the kernel is silly; > > Ummm... tmpfs stores them in the kernel too -

Re: [PATCH 1/2] KEYS: Implement a big key type that can save to tmpfs

2013-08-09 Thread Nico Williams
On Thu, Aug 8, 2013 at 9:46 AM, David Howells dhowe...@redhat.com wrote: Nico Williams n...@cryptonector.com wrote: b) how to create tmpfs locations in which to store credentials (which can be unbounded in size, so storing them in the kernel is silly; Ummm... tmpfs stores them in the kernel

Re: [PATCH 1/2] KEYS: Implement a big key type that can save to tmpfs

2013-08-02 Thread Nico Williams
On Fri, Aug 2, 2013 at 3:49 PM, Nico Williams wrote: > Solving (b) in a way that does not add a new ccache type (though > having a KEYRING: ccache type that means "find the ccache URI in my > keyring" is fine) is important because many of us run multiple > implementations of

Re: [PATCH 1/2] KEYS: Implement a big key type that can save to tmpfs

2013-08-02 Thread Nico Williams
I think this is the wrong design. There are two problems you're trying to solve: a) how rpc.gssd finds credentials for processes on behalf of which it's acting b) how to create tmpfs locations in which to store credentials (which can be unbounded in size, so storing them in the kernel is silly;

Re: [PATCH 2/2] KEYS: Add per-user_namespace registers for persistent per-UID kerberos caches

2013-08-02 Thread Nico Williams
On Fri, Aug 2, 2013 at 8:55 AM, Jeff Layton wrote: > Isn't it possible to have a valid uid of (unsigned int)-1? I know that > at least some sites use that for "nobody". Why not just require passing > in the correct UID? POSIX requires valid UIDs to be non-nengative. POSIX does not require uid_t

Re: [PATCH 2/2] KEYS: Add per-user_namespace registers for persistent per-UID kerberos caches

2013-08-02 Thread Nico Williams
On Fri, Aug 2, 2013 at 8:55 AM, Jeff Layton jlay...@redhat.com wrote: Isn't it possible to have a valid uid of (unsigned int)-1? I know that at least some sites use that for nobody. Why not just require passing in the correct UID? POSIX requires valid UIDs to be non-nengative. POSIX does not

Re: [PATCH 1/2] KEYS: Implement a big key type that can save to tmpfs

2013-08-02 Thread Nico Williams
I think this is the wrong design. There are two problems you're trying to solve: a) how rpc.gssd finds credentials for processes on behalf of which it's acting b) how to create tmpfs locations in which to store credentials (which can be unbounded in size, so storing them in the kernel is silly;

Re: [PATCH 1/2] KEYS: Implement a big key type that can save to tmpfs

2013-08-02 Thread Nico Williams
On Fri, Aug 2, 2013 at 3:49 PM, Nico Williams n...@cryptonector.com wrote: Solving (b) in a way that does not add a new ccache type (though having a KEYRING: ccache type that means find the ccache URI in my keyring is fine) is important because many of us run multiple implementations

Re: [sqlite] light weight write barriers

2012-11-26 Thread Nico Williams
Vlad, You keep saying that programmers don't understand "barriers". You've provided no evidence of this. Meanwhile memory barriers are generally well understood, and every programmer I know understands that a "barrier" is a synchronization primitive that says that all operations of a certain

Re: [sqlite] light weight write barriers

2012-11-26 Thread Nico Williams
Vlad, You keep saying that programmers don't understand barriers. You've provided no evidence of this. Meanwhile memory barriers are generally well understood, and every programmer I know understands that a barrier is a synchronization primitive that says that all operations of a certain type

Re: [sqlite] light weight write barriers

2012-11-13 Thread Nico Williams
On Tue, Nov 13, 2012 at 11:40 AM, Alan Cox wrote: >> > Barriers are pretty much universal as you need them for power off ! >> >> I'm afraid, no storage (drives, if you like this term more) at the moment >> supports >> barriers and, as far as I know the storage history, has never supported. > >

Re: [sqlite] light weight write barriers

2012-11-13 Thread Nico Williams
On Tue, Nov 13, 2012 at 11:40 AM, Alan Cox a...@lxorguk.ukuu.org.uk wrote: Barriers are pretty much universal as you need them for power off ! I'm afraid, no storage (drives, if you like this term more) at the moment supports barriers and, as far as I know the storage history, has never

Re: [sqlite] light weight write barriers

2012-10-30 Thread Nico Williams
[Dropping sqlite-users. Note that I'm not subscribed to any of the other lists cc'ed.] On Thu, Oct 25, 2012 at 1:02 AM, Theodore Ts'o wrote: > On Thu, Oct 25, 2012 at 12:18:47AM -0500, Nico Williams wrote: >> >> By trusting fsync(). And if you don't care about immediate Durabi

Re: [sqlite] light weight write barriers

2012-10-30 Thread Nico Williams
[Dropping sqlite-users. Note that I'm not subscribed to any of the other lists cc'ed.] On Thu, Oct 25, 2012 at 1:02 AM, Theodore Ts'o ty...@mit.edu wrote: On Thu, Oct 25, 2012 at 12:18:47AM -0500, Nico Williams wrote: By trusting fsync(). And if you don't care about immediate Durability you

Re: [sqlite] light weight write barriers

2012-10-24 Thread Nico Williams
On Wed, Oct 24, 2012 at 8:04 PM, wrote: > On Wed, 24 Oct 2012, Nico Williams wrote: >> COW is "copy on write", which is actually a bit of a misnomer -- all >> COW means is that blocks aren't over-written, instead new blocks are >> written. In particular this mea

Re: [sqlite] light weight write barriers

2012-10-24 Thread Nico Williams
On Wed, Oct 24, 2012 at 5:03 PM, wrote: > I'm doing some work with rsyslog and it's disk-baded queues and there is a > similar issue there. The good news is that we can have a version that is > linux specific (rsyslog is used on other OSs, but there is an existing queue > implementation that

Re: [sqlite] light weight write barriers

2012-10-24 Thread Nico Williams
On Tue, Oct 23, 2012 at 2:53 PM, Vladislav Bolkhovitin wrote: >> As most of the time the order we need do not involve too many blocks >> (certainly a lot less than all the cached blocks in the system or in >> the disk's cache), that topological order isn't likely to be very >> complicated, and I

Re: [sqlite] light weight write barriers

2012-10-24 Thread Nico Williams
On Tue, Oct 23, 2012 at 2:53 PM, Vladislav Bolkhovitin ...@gmail.com wrote: As most of the time the order we need do not involve too many blocks (certainly a lot less than all the cached blocks in the system or in the disk's cache), that topological order isn't likely to be very

Re: [sqlite] light weight write barriers

2012-10-24 Thread Nico Williams
On Wed, Oct 24, 2012 at 5:03 PM, da...@lang.hm wrote: I'm doing some work with rsyslog and it's disk-baded queues and there is a similar issue there. The good news is that we can have a version that is linux specific (rsyslog is used on other OSs, but there is an existing queue implementation

Re: [sqlite] light weight write barriers

2012-10-24 Thread Nico Williams
On Wed, Oct 24, 2012 at 8:04 PM, da...@lang.hm wrote: On Wed, 24 Oct 2012, Nico Williams wrote: COW is copy on write, which is actually a bit of a misnomer -- all COW means is that blocks aren't over-written, instead new blocks are written. In particular this means that inodes, indirect

Re: [sqlite] light weight write barriers

2012-10-11 Thread Nico Williams
To expand a bit, the on-disk format needs to allow the roots of N of the last transactions to be/remain reachable at all times. At open time you look for the latest transaction, verify that it has been written[0] completely, then use it, else look for the preceding transaction, verify it, and so

Re: [sqlite] light weight write barriers

2012-10-11 Thread Nico Williams
On Wed, Oct 10, 2012 at 12:48 PM, Richard Hipp wrote: >> Could you list the requirements of such a light weight barrier? >> i.e. what would it need to do minimally, what's different from >> fsync/fdatasync ? > > For SQLite, the write barrier needs to involve two separate inodes. The >

Re: [sqlite] light weight write barriers

2012-10-11 Thread Nico Williams
On Wed, Oct 10, 2012 at 12:48 PM, Richard Hipp d...@sqlite.org wrote: Could you list the requirements of such a light weight barrier? i.e. what would it need to do minimally, what's different from fsync/fdatasync ? For SQLite, the write barrier needs to involve two separate inodes. The

Re: [sqlite] light weight write barriers

2012-10-11 Thread Nico Williams
To expand a bit, the on-disk format needs to allow the roots of N of the last transactions to be/remain reachable at all times. At open time you look for the latest transaction, verify that it has been written[0] completely, then use it, else look for the preceding transaction, verify it, and so