Re: [HACKERS] better atomics - v0.6

2014-09-25 Thread Andres Freund
On 2014-09-25 21:02:28 -0400, Robert Haas wrote: > I feel like this could really use a developer README: what primitives > do we have, which ones are likely to be efficient or inefficient on > which platforms, how do atomics interact with barriers, etc. atomics.h has most of that. Including docume

Re: [HACKERS] better atomics - v0.6

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 6:03 PM, Andres Freund wrote: > On 2014-09-24 20:27:39 +0200, Andres Freund wrote: >> On 2014-09-24 21:19:06 +0300, Heikki Linnakangas wrote: >> I won't repost a version with it removed, as removing a function as the >> only doesn't seem to warrant reposting it. > > I've fi

Re: [HACKERS] better atomics - v0.6

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 3:23 PM, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: >> Heikki has marked the patch as 'ready for commiter' in the commitfest >> (conditional on his remarks being addressed) and I agree. There seems to >> be little benefit in waiting further. There *d

Re: [HACKERS] better atomics - v0.6

2014-09-25 Thread Stephen Frost
Andres, * Andres Freund (and...@anarazel.de) wrote: > Heikki has marked the patch as 'ready for commiter' in the commitfest > (conditional on his remarks being addressed) and I agree. There seems to > be little benefit in waiting further. There *definitely* will be some > platform dependant issues

Re: [HACKERS] better atomics - v0.6

2014-09-25 Thread Andres Freund
On 2014-09-24 20:27:39 +0200, Andres Freund wrote: > On 2014-09-24 21:19:06 +0300, Heikki Linnakangas wrote: > I won't repost a version with it removed, as removing a function as the > only doesn't seem to warrant reposting it. I've fixed (thanks Alvaro!) some minor additional issues besides the r

Re: [HACKERS] better atomics - v0.6

2014-09-24 Thread Andres Freund
On 2014-09-24 14:28:18 -0400, Tom Lane wrote: > Note that the spinlock code separates s_lock.h (hardware implementations) > from spin.h (a hardware-independent abstraction layer). Perhaps there's > room for a similar separation here. Luckily that separation exists ;). The hardware dependant bits

Re: [HACKERS] better atomics - v0.6

2014-09-24 Thread Tom Lane
Heikki Linnakangas writes: > On 09/24/2014 07:57 PM, Andres Freund wrote: >> On 2014-09-24 12:44:09 -0400, Tom Lane wrote: >>> I think the question is more like "what in the world happened to confining >>> ourselves to a small set of atomics". >> I fail to see why the existance of a wrapper aroun

Re: [HACKERS] better atomics - v0.6

2014-09-24 Thread Andres Freund
On 2014-09-24 21:19:06 +0300, Heikki Linnakangas wrote: > On 09/24/2014 07:57 PM, Andres Freund wrote: > >On 2014-09-24 12:44:09 -0400, Tom Lane wrote: > >>Andres Freund writes: > >>>On 2014-09-24 18:55:51 +0300, Heikki Linnakangas wrote: > There doesn't seem to be any hardware implementations

Re: [HACKERS] better atomics - v0.6

2014-09-24 Thread Heikki Linnakangas
On 09/24/2014 07:57 PM, Andres Freund wrote: On 2014-09-24 12:44:09 -0400, Tom Lane wrote: Andres Freund writes: On 2014-09-24 18:55:51 +0300, Heikki Linnakangas wrote: There doesn't seem to be any hardware implementations of that in the patch. Is there any architecture that has an instructio

Re: [HACKERS] better atomics - v0.6

2014-09-24 Thread Andres Freund
On 2014-09-24 12:44:09 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-09-24 18:55:51 +0300, Heikki Linnakangas wrote: > >> There doesn't seem to be any hardware implementations of that in the patch. > >> Is there any architecture that has an instruction or compiler intrinsic for > >> t

Re: [HACKERS] better atomics - v0.6

2014-09-24 Thread Tom Lane
Andres Freund writes: > On 2014-09-24 18:55:51 +0300, Heikki Linnakangas wrote: >> There doesn't seem to be any hardware implementations of that in the patch. >> Is there any architecture that has an instruction or compiler intrinsic for >> that? > You can implement it rather efficiently on ll/sc

Re: [HACKERS] better atomics - v0.6

2014-09-24 Thread Andres Freund
On 2014-09-24 18:55:51 +0300, Heikki Linnakangas wrote: > On 09/24/2014 03:37 PM, Andres Freund wrote: > >+/* > >+ * pg_fetch_add_until_u32 - saturated addition to variable > >+ * > >+ * Returns the the value of ptr after the arithmetic operation. > >+ * > >+ * Full ba

Re: [HACKERS] better atomics - v0.6

2014-09-24 Thread Heikki Linnakangas
On 09/24/2014 03:37 PM, Andres Freund wrote: > >+/* > >+ * pg_fetch_add_until_u32 - saturated addition to variable > >+ * > >+ * Returns the the value of ptr after the arithmetic operation. > >+ * > >+ * Full barrier semantics. > >+ */ > >+STATIC_IF_INLINE uint32 > >+pg_atomic_fetch_add_until_u32

Re: [HACKERS] better atomics - v0.6

2014-09-24 Thread Andres Freund
On 2014-09-24 14:59:19 +0300, Heikki Linnakangas wrote: > On 09/23/2014 12:01 AM, Andres Freund wrote: > >Hi, > > > >I've finally managed to incorporate (all the?) feedback I got for > >0.5. Imo the current version looks pretty good. > > Thanks! I agree it looks good. Some random comments after a

Re: [HACKERS] better atomics - v0.6

2014-09-24 Thread Heikki Linnakangas
On 09/23/2014 12:01 AM, Andres Freund wrote: Hi, I've finally managed to incorporate (all the?) feedback I got for 0.5. Imo the current version looks pretty good. Thanks! I agree it looks good. Some random comments after a quick read-through: There are some spurious whitespace changes in sp

Re: [HACKERS] better atomics - v0.6

2014-09-23 Thread Andres Freund
On 2014-09-24 00:27:25 +0300, Oskari Saarenmaa wrote: > 23.09.2014, 15:18, Andres Freund kirjoitti: > >On 2014-09-23 13:50:28 +0300, Oskari Saarenmaa wrote: > >>23.09.2014, 00:01, Andres Freund kirjoitti: > >>>The patches: > >>>0001: The actual atomics API > >> > >>I tried building PG on Solaris 10

Re: [HACKERS] better atomics - v0.6

2014-09-23 Thread Oskari Saarenmaa
23.09.2014, 15:18, Andres Freund kirjoitti: On 2014-09-23 13:50:28 +0300, Oskari Saarenmaa wrote: 23.09.2014, 00:01, Andres Freund kirjoitti: The patches: 0001: The actual atomics API I tried building PG on Solaris 10/Sparc using GCC 4.9.0 (buildfarm animal dingo) with this patch but regressi

Re: [HACKERS] better atomics - v0.6

2014-09-23 Thread Andres Freund
Hi, On 2014-09-23 13:50:28 +0300, Oskari Saarenmaa wrote: > 23.09.2014, 00:01, Andres Freund kirjoitti: > >I've finally managed to incorporate (all the?) feedback I got for > >0.5. Imo the current version looks pretty good. > > > >Most notable changes: > >* Lots of comment improvements > >* code m

Re: [HACKERS] better atomics - v0.6

2014-09-23 Thread Andres Freund
On 2014-09-23 13:50:28 +0300, Oskari Saarenmaa wrote: > 23.09.2014, 00:01, Andres Freund kirjoitti: > >I've finally managed to incorporate (all the?) feedback I got for > >0.5. Imo the current version looks pretty good. > > > >Most notable changes: > >* Lots of comment improvements > >* code moved

Re: [HACKERS] better atomics - v0.6

2014-09-23 Thread Oskari Saarenmaa
23.09.2014, 00:01, Andres Freund kirjoitti: I've finally managed to incorporate (all the?) feedback I got for 0.5. Imo the current version looks pretty good. Most notable changes: * Lots of comment improvements * code moved out of storage/ into port/ * separated the s_lock.h changes into its own

Re: [HACKERS] better atomics - v0.5

2014-08-20 Thread Andres Freund
Hi, On 2014-08-20 12:43:05 +0300, Heikki Linnakangas wrote: > Are you planning to continue working on this? Yes, I am! I've recently been on holiday and now I'm busy with catching up with everything that has happened since. I hope to have a next version ready early next week. Greetings, Andres

Re: [HACKERS] better atomics - v0.5

2014-08-20 Thread Heikki Linnakangas
Hi Andres, Are you planning to continue working on this? Summarizing the discussion so far: * Robert listed a bunch of little cleanup tasks (http://www.postgresql.org/message-id/ca+tgmozshvvqjakul6p3kdvzvpibtgkzoti3m+fvvjg5v+x...@mail.gmail.com). Amit posted yet more detailed commends. * W

Re: [HACKERS] better atomics - v0.5

2014-07-16 Thread Amit Kapila
On Mon, Jul 14, 2014 at 12:47 AM, Andres Freund wrote: > On 2014-07-08 11:51:14 +0530, Amit Kapila wrote: > > On Thu, Jul 3, 2014 at 10:56 AM, Amit Kapila > > wrote: > > Further review of patch: > > 1. > > /* > > * pg_atomic_test_and_set_flag - TAS() > > * > > * Acquire/read barrier semantics

Re: [HACKERS] better atomics - v0.5

2014-07-13 Thread Andres Freund
On 2014-07-10 08:46:55 +0530, Amit Kapila wrote: > As per my understanding of the general theory around barriers, > read and write are defined to avoid reordering due to compiler and > full memory barriers are defined to avoid reordering due to > processors. No, that's not the case. There's severa

Re: [HACKERS] better atomics - v0.5

2014-07-13 Thread Andres Freund
Hi Amit, On 2014-07-08 11:51:14 +0530, Amit Kapila wrote: > On Thu, Jul 3, 2014 at 10:56 AM, Amit Kapila > wrote: > > On Tue, Jul 1, 2014 at 4:10 PM, Andres Freund > wrote: > > Further review of patch: > 1. > /* > * pg_atomic_test_and_set_flag - TAS() > * > * Acquire/read barrier semantics.

Re: [HACKERS] better atomics - v0.5

2014-07-11 Thread Amit Kapila
On Sat, Jul 12, 2014 at 1:26 AM, Martijn van Oosterhout wrote: > On Thu, Jul 10, 2014 at 08:46:55AM +0530, Amit Kapila wrote: > > As per my understanding of the general theory around barriers, > > read and write are defined to avoid reordering due to compiler and > > full memory barriers are defin

Re: [HACKERS] better atomics - v0.5

2014-07-11 Thread Martijn van Oosterhout
On Thu, Jul 10, 2014 at 08:46:55AM +0530, Amit Kapila wrote: > As per my understanding of the general theory around barriers, > read and write are defined to avoid reordering due to compiler and > full memory barriers are defined to avoid reordering due to processors. > There are some processors th

Re: [HACKERS] better atomics - v0.5

2014-07-09 Thread Amit Kapila
On Wed, Jul 9, 2014 at 8:58 PM, Robert Haas wrote: > On Tue, Jul 8, 2014 at 2:21 AM, Amit Kapila wrote: > > On Thu, Jul 3, 2014 at 10:56 AM, Amit Kapila > > wrote: > > > > Further review of patch: > > 1. > > /* > > * pg_atomic_test_and_set_flag - TAS() > > * > > * Acquire/read barrier semanti

Re: [HACKERS] better atomics - v0.5

2014-07-09 Thread Robert Haas
On Tue, Jul 8, 2014 at 2:21 AM, Amit Kapila wrote: > On Thu, Jul 3, 2014 at 10:56 AM, Amit Kapila > wrote: >> On Tue, Jul 1, 2014 at 4:10 PM, Andres Freund >> wrote: > > Further review of patch: > 1. > /* > * pg_atomic_test_and_set_flag - TAS() > * > * Acquire/read barrier semantics. > */ >

Re: [HACKERS] better atomics - v0.5

2014-07-07 Thread Amit Kapila
On Thu, Jul 3, 2014 at 10:56 AM, Amit Kapila wrote: > On Tue, Jul 1, 2014 at 4:10 PM, Andres Freund wrote: Further review of patch: 1. /* * pg_atomic_test_and_set_flag - TAS() * * Acquire/read barrier semantics. */ STATIC_IF_INLINE_DECLARE bool pg_atomic_test_set_flag(volatile pg_atomic_flag

Re: [HACKERS] better atomics - v0.5

2014-07-02 Thread Amit Kapila
On Tue, Jul 1, 2014 at 4:10 PM, Andres Freund wrote: > On 2014-07-01 10:44:19 +0530, Amit Kapila wrote: > > > I think defining barrier support for some atomic ops and not for others > > will lead to inconsistency with specs and we need to additionally > > define rules for such atomic ops usage. >

Re: [HACKERS] better atomics - v0.5

2014-07-02 Thread Andres Freund
On 2014-06-26 00:42:37 +0300, Heikki Linnakangas wrote: > On 06/25/2014 11:36 PM, Andres Freund wrote: > > > >>>- I completely loathe the file layout you've proposed in > >>>src/include/storage. If we're going to have separate #include files > >>>for each architecture (and I'd rather we didn't go

Re: [HACKERS] better atomics - v0.5

2014-07-02 Thread Andres Freund
Hi, On 2014-06-30 20:28:52 +0200, Andres Freund wrote: > To quantify this, on my 2 socket xeon E5520 workstation - which is too > small to heavily show the contention problems in pgbench -S - the > numbers are: > > pgbench -M prepared -c 16 -j 16 -T 10 -S (best of 5, noticeably variability) > mast

Re: [HACKERS] better atomics - v0.5

2014-07-02 Thread Andres Freund
On 2014-07-02 09:27:52 +0300, Ants Aasma wrote: > On Fri, Jun 27, 2014 at 9:00 PM, Andres Freund wrote: > > Imagine the situation for the buffer header spinlock which is one of the > > bigger performance issues atm. We could aim to replace all usages of > > that with clever and complicated logic,

Re: [HACKERS] better atomics - v0.5

2014-07-01 Thread Ants Aasma
On Fri, Jun 27, 2014 at 9:00 PM, Andres Freund wrote: > Imagine the situation for the buffer header spinlock which is one of the > bigger performance issues atm. We could aim to replace all usages of > that with clever and complicated logic, but it's hard. > > The IMO reasonable (and prototyped) w

Re: [HACKERS] better atomics - v0.5

2014-07-01 Thread Andres Freund
On 2014-07-01 10:44:19 +0530, Amit Kapila wrote: > I think for such usage, we need to rely on barriers wherever there is a > need for synchronisation, recent example I have noticed is in your patch > where we have to use pg_write_barrier() during wakeup. However if we > go by atomic ops definition

Re: [HACKERS] better atomics - v0.5

2014-06-30 Thread Amit Kapila
On Mon, Jun 30, 2014 at 2:54 PM, Andres Freund wrote: > On 2014-06-30 11:04:53 +0530, Amit Kapila wrote: > > On Sun, Jun 29, 2014 at 2:54 PM, Andres Freund wrote: > > > Yes, intentionally so. It's often important to get/set the current value > > > without the cost of emitting a memory barrier. I

Re: [HACKERS] better atomics - v0.5

2014-06-30 Thread Noah Misch
On Mon, Jun 30, 2014 at 07:44:47PM +0200, Andres Freund wrote: > On 2014-06-30 13:15:23 -0400, Robert Haas wrote: > > I'm personally not convinced that we're approaching this topic in the > > right way. I'm not convinced that it's at all reasonable to try to > > emulate atomics on platforms that d

Re: [HACKERS] better atomics - v0.5

2014-06-30 Thread Andres Freund
On 2014-06-30 13:45:52 -0400, Tom Lane wrote: > Robert Haas writes: > > I'm personally not convinced that we're approaching this topic in the > > right way. I'm not convinced that it's at all reasonable to try to > > emulate atomics on platforms that don't have them. I would punt the > > problem

Re: [HACKERS] better atomics - v0.5

2014-06-30 Thread Andres Freund
On 2014-06-30 19:44:47 +0200, Andres Freund wrote: > On 2014-06-30 13:15:23 -0400, Robert Haas wrote: > > People who write code that uses atomics are not > > likely to think about how those algorithms will actually perform when > > those atomics are merely emulated, and I suspect that means that in

Re: [HACKERS] better atomics - v0.5

2014-06-30 Thread Andres Freund
On 2014-06-30 12:54:10 -0400, Tom Lane wrote: > Robert Haas writes: > > ... this again is my point: why can't we make the same argument about > > two spinlocks situated on the same cache line? I don't have a bit of > > trouble believing that doing the same thing with a couple of spinlocks > > cou

Re: [HACKERS] better atomics - v0.5

2014-06-30 Thread Tom Lane
Robert Haas writes: > I'm personally not convinced that we're approaching this topic in the > right way. I'm not convinced that it's at all reasonable to try to > emulate atomics on platforms that don't have them. I would punt the > problem into the next layer and force things like lwlock.c to h

Re: [HACKERS] better atomics - v0.5

2014-06-30 Thread Andres Freund
On 2014-06-30 13:15:23 -0400, Robert Haas wrote: > On Mon, Jun 30, 2014 at 12:54 PM, Tom Lane wrote: > > The existing coding rules also discourage spinlocking within a spinlock, > > and the reason for that is that there's no very clear upper bound to the > > time required to obtain a spinlock, so

Re: [HACKERS] better atomics - v0.5

2014-06-30 Thread Robert Haas
On Mon, Jun 30, 2014 at 12:54 PM, Tom Lane wrote: > The existing coding rules also discourage spinlocking within a spinlock, > and the reason for that is that there's no very clear upper bound to the > time required to obtain a spinlock, so that there would also be no clear > upper bound to the ti

Re: [HACKERS] better atomics - v0.5

2014-06-30 Thread Tom Lane
Robert Haas writes: > ... this again is my point: why can't we make the same argument about > two spinlocks situated on the same cache line? I don't have a bit of > trouble believing that doing the same thing with a couple of spinlocks > could sometimes work out well, too, but Tom is adamantly op

Re: [HACKERS] better atomics - v0.5

2014-06-30 Thread Andres Freund
On 2014-06-30 12:15:06 -0400, Robert Haas wrote: > More broadly, it doesn't seem consistent. I think that other projects > also sometimes write code that acquires a spinlock while holding > another spinlock, and we don't do that; in fact, we've elevated that > to a principle, regardless of whether

Re: [HACKERS] better atomics - v0.5

2014-06-30 Thread Robert Haas
On Sat, Jun 28, 2014 at 4:25 AM, Andres Freund wrote: >> > To make pin/unpin et al safe without acquiring the spinlock the pincount >> > and the flags had to be moved into one variable so that when >> > incrementing the pincount you automatically get the flagbits back. If >> > it's currently valid

Re: [HACKERS] better atomics - v0.5

2014-06-30 Thread Andres Freund
On 2014-06-30 11:04:53 +0530, Amit Kapila wrote: > On Sun, Jun 29, 2014 at 2:54 PM, Andres Freund wrote: > > On 2014-06-29 11:53:28 +0530, Amit Kapila wrote: > > > On Sat, Jun 28, 2014 at 1:48 PM, Andres Freund > > > I think it is better to have some discussion about it. Apart from this, > > > to

Re: [HACKERS] better atomics - v0.5

2014-06-29 Thread Amit Kapila
On Sun, Jun 29, 2014 at 2:54 PM, Andres Freund wrote: > On 2014-06-29 11:53:28 +0530, Amit Kapila wrote: > > On Sat, Jun 28, 2014 at 1:48 PM, Andres Freund > > I think it is better to have some discussion about it. Apart from this, > > today I noticed atomic read/write doesn't use any barrier sem

Re: [HACKERS] better atomics - v0.5

2014-06-29 Thread Andres Freund
On 2014-06-28 19:36:39 +0300, Heikki Linnakangas wrote: > On 06/27/2014 08:15 PM, Robert Haas wrote: > >On Thu, Jun 26, 2014 at 3:04 PM, Andres Freund > >wrote: > >>I don't really see usecases where it's not related data that's being > >>touched, but: The point is that the fastpath (not using a s

Re: [HACKERS] better atomics - v0.5

2014-06-29 Thread Andres Freund
On 2014-06-29 11:53:28 +0530, Amit Kapila wrote: > On Sat, Jun 28, 2014 at 1:48 PM, Andres Freund > > Two things: > > a) compare_exchange is a read/write operator and so far I've defined it > >to have full barrier semantics (documented in atomics.h). I'd be > >happy to discuss which semant

Re: [HACKERS] better atomics - v0.5

2014-06-28 Thread Amit Kapila
On Sat, Jun 28, 2014 at 1:48 PM, Andres Freund wrote: > On 2014-06-28 11:58:41 +0530, Amit Kapila wrote: > > 1. > > +pg_atomic_compare_exchange_u32_impl(volatile pg_atomic_uint32 *ptr, > > + uint32 *expected, uint32 newval) > > +{ > > + bool ret; > > + uint32 current; > > + current = InterlockedCo

Re: [HACKERS] better atomics - v0.5

2014-06-28 Thread Heikki Linnakangas
On 06/27/2014 08:15 PM, Robert Haas wrote: On Thu, Jun 26, 2014 at 3:04 PM, Andres Freund wrote: I don't really see usecases where it's not related data that's being touched, but: The point is that the fastpath (not using a spinlock) might touch the atomic variable, even while the slowpath has

Re: [HACKERS] better atomics - v0.5

2014-06-28 Thread Andres Freund
On 2014-06-27 22:47:02 -0400, Robert Haas wrote: > On Fri, Jun 27, 2014 at 2:00 PM, Andres Freund wrote: > > On 2014-06-27 13:15:25 -0400, Robert Haas wrote: > >> On Thu, Jun 26, 2014 at 3:04 PM, Andres Freund > >> wrote: > >> > I don't really see usecases where it's not related data that's bein

Re: [HACKERS] better atomics - v0.5

2014-06-28 Thread Andres Freund
On 2014-06-28 11:58:41 +0530, Amit Kapila wrote: > On Wed, Jun 25, 2014 at 10:44 PM, Andres Freund > wrote: > > > > Attached is a new version of the atomic operations patch. Lots has > > changed since the last post: > > > > * gcc, msvc work. acc, xlc, sunpro have blindly written support which > >

Re: [HACKERS] better atomics - v0.5

2014-06-27 Thread Amit Kapila
On Wed, Jun 25, 2014 at 10:44 PM, Andres Freund wrote: > > Attached is a new version of the atomic operations patch. Lots has > changed since the last post: > > * gcc, msvc work. acc, xlc, sunpro have blindly written support which > should be relatively easy to fix up. All try to implement TAS,

Re: [HACKERS] better atomics - v0.5

2014-06-27 Thread Robert Haas
On Fri, Jun 27, 2014 at 2:00 PM, Andres Freund wrote: > On 2014-06-27 13:15:25 -0400, Robert Haas wrote: >> On Thu, Jun 26, 2014 at 3:04 PM, Andres Freund >> wrote: >> > I don't really see usecases where it's not related data that's being >> > touched, but: The point is that the fastpath (not us

Re: [HACKERS] better atomics - v0.5

2014-06-27 Thread Andres Freund
On 2014-06-27 13:15:25 -0400, Robert Haas wrote: > On Thu, Jun 26, 2014 at 3:04 PM, Andres Freund wrote: > > I don't really see usecases where it's not related data that's being > > touched, but: The point is that the fastpath (not using a spinlock) might > > touch the atomic variable, even while

Re: [HACKERS] better atomics - v0.5

2014-06-27 Thread Robert Haas
On Thu, Jun 26, 2014 at 3:04 PM, Andres Freund wrote: > I don't really see usecases where it's not related data that's being > touched, but: The point is that the fastpath (not using a spinlock) might > touch the atomic variable, even while the slowpath has acquired the > spinlock. So the slowpath

Re: [HACKERS] better atomics - v0.5

2014-06-26 Thread Merlin Moncure
On Thu, Jun 26, 2014 at 1:47 PM, Tom Lane wrote: > Robert Haas writes: > Would it be practical to say that the coding rule is that you can only > use atomic ops on fields that are protected by the spinlock, ie, nobody > else is supposed to be touching those fields while you have the spinlock? > I

Re: [HACKERS] better atomics - v0.5

2014-06-26 Thread Andres Freund
On 2014-06-26 11:47:15 -0700, Tom Lane wrote: > Robert Haas writes: > > On Thu, Jun 26, 2014 at 6:20 AM, Andres Freund > > wrote: > >> On 2014-06-25 20:16:08 -0400, Robert Haas wrote: > >>> I think that's going to fall afoul of Tom's previously-articulated "no > >>> loops inside spinlocks" rule.

Re: [HACKERS] better atomics - v0.5

2014-06-26 Thread Tom Lane
Robert Haas writes: > On Thu, Jun 26, 2014 at 6:20 AM, Andres Freund wrote: >> On 2014-06-25 20:16:08 -0400, Robert Haas wrote: >>> I think that's going to fall afoul of Tom's previously-articulated "no >>> loops inside spinlocks" rule. Most atomics, by nature, are >>> loop-until-it-works. >> W

Re: [HACKERS] better atomics - v0.5

2014-06-26 Thread Robert Haas
On Thu, Jun 26, 2014 at 7:21 AM, Andres Freund wrote: > On 2014-06-25 20:22:31 -0400, Robert Haas wrote: >> On Wed, Jun 25, 2014 at 5:42 PM, Heikki Linnakangas >> wrote: >> > I think having a separate file for each architecture is nice. I totally >> > agree that they don't belong in src/include/s

Re: [HACKERS] better atomics - v0.5

2014-06-26 Thread Robert Haas
On Thu, Jun 26, 2014 at 6:20 AM, Andres Freund wrote: > On 2014-06-25 20:16:08 -0400, Robert Haas wrote: >> On Wed, Jun 25, 2014 at 4:36 PM, Andres Freund >> wrote: >> > Since it better be legal to manipulate a atomic variable while holding a >> > spinlock we cannot simply use an arbitrary spinl

Re: [HACKERS] better atomics - v0.5

2014-06-26 Thread Noah Misch
On Thu, Jun 26, 2014 at 12:20:06PM +0200, Andres Freund wrote: > On 2014-06-25 20:16:08 -0400, Robert Haas wrote: > > On Wed, Jun 25, 2014 at 4:36 PM, Andres Freund > > wrote: > > > Since it better be legal to manipulate a atomic variable while holding a > > > spinlock we cannot simply use an arb

Re: [HACKERS] better atomics - v0.5

2014-06-26 Thread Andres Freund
On 2014-06-26 07:44:14 -0500, Merlin Moncure wrote: > On Thu, Jun 26, 2014 at 5:20 AM, Andres Freund wrote: > > On 2014-06-25 20:16:08 -0400, Robert Haas wrote: > >> On Wed, Jun 25, 2014 at 4:36 PM, Andres Freund > >> wrote: > >> > Since it better be legal to manipulate a atomic variable while h

Re: [HACKERS] better atomics - v0.5

2014-06-26 Thread Merlin Moncure
On Thu, Jun 26, 2014 at 5:20 AM, Andres Freund wrote: > On 2014-06-25 20:16:08 -0400, Robert Haas wrote: >> On Wed, Jun 25, 2014 at 4:36 PM, Andres Freund >> wrote: >> > Since it better be legal to manipulate a atomic variable while holding a >> > spinlock we cannot simply use an arbitrary spinl

Re: [HACKERS] better atomics - v0.5

2014-06-26 Thread Andres Freund
On 2014-06-25 20:22:31 -0400, Robert Haas wrote: > On Wed, Jun 25, 2014 at 5:42 PM, Heikki Linnakangas > wrote: > > I think having a separate file for each architecture is nice. I totally > > agree that they don't belong in src/include/storage, though. s_lock.h has > > always been misplaced there,

Re: [HACKERS] better atomics - v0.5

2014-06-26 Thread Andres Freund
On 2014-06-25 20:16:08 -0400, Robert Haas wrote: > On Wed, Jun 25, 2014 at 4:36 PM, Andres Freund wrote: > > Since it better be legal to manipulate a atomic variable while holding a > > spinlock we cannot simply use an arbitrary spinlock as backing for > > atomics. That'd possibly cause us to wait

Re: [HACKERS] better atomics - v0.5

2014-06-25 Thread Robert Haas
On Wed, Jun 25, 2014 at 5:42 PM, Heikki Linnakangas wrote: > I think having a separate file for each architecture is nice. I totally > agree that they don't belong in src/include/storage, though. s_lock.h has > always been misplaced there, but we've let it be for historical reasons, but > now that

Re: [HACKERS] better atomics - v0.5

2014-06-25 Thread Robert Haas
On Wed, Jun 25, 2014 at 4:36 PM, Andres Freund wrote: > Since it better be legal to manipulate a atomic variable while holding a > spinlock we cannot simply use an arbitrary spinlock as backing for > atomics. That'd possibly cause us to wait on ourselves or cause > deadlocks. I think that's going

Re: [HACKERS] better atomics - v0.5

2014-06-25 Thread Alvaro Herrera
Heikki Linnakangas wrote: > On 06/25/2014 11:36 PM, Andres Freund wrote: > > > >>>- I completely loathe the file layout you've proposed in > >>>src/include/storage. If we're going to have separate #include files > >>>for each architecture (and I'd rather we didn't go that route, because > >>>it's

Re: [HACKERS] better atomics - v0.5

2014-06-25 Thread Heikki Linnakangas
On 06/25/2014 11:36 PM, Andres Freund wrote: >- I completely loathe the file layout you've proposed in >src/include/storage. If we're going to have separate #include files >for each architecture (and I'd rather we didn't go that route, because >it's messy and unlike what we have now), then sho

Re: [HACKERS] better atomics - v0.5

2014-06-25 Thread Andres Freund
Hi, On 2014-06-25 15:54:37 -0400, Robert Haas wrote: > - The new argument to s_init_lock_sema() isn't used. It's used when atomics fallback to spinlocks which fall back to semaphores. c.f. atomics.c. Since it better be legal to manipulate a atomic variable while holding a spinlock we cannot simp

Re: [HACKERS] better atomics - v0.5

2014-06-25 Thread Robert Haas
On Wed, Jun 25, 2014 at 1:14 PM, Andres Freund wrote: > [sorry for the second copy Robert] > > Attached is a new version of the atomic operations patch. Lots has > changed since the last post: > > * gcc, msvc work. acc, xlc, sunpro have blindly written support which > should be relatively easy t

Re: [HACKERS] better atomics - v0.5

2014-06-25 Thread Andres Freund
On 2014-06-25 10:39:53 -0700, Josh Berkus wrote: > On 06/25/2014 10:14 AM, Andres Freund wrote: > > Hi, > > > > [sorry for the second copy Robert] > > > > Attached is a new version of the atomic operations patch. Lots has > > changed since the last post: > > Is this at a state where we can perfo

Re: [HACKERS] better atomics - v0.5

2014-06-25 Thread Josh Berkus
On 06/25/2014 10:14 AM, Andres Freund wrote: > Hi, > > [sorry for the second copy Robert] > > Attached is a new version of the atomic operations patch. Lots has > changed since the last post: Is this at a state where we can performance-test it yet? -- Josh Berkus PostgreSQL Experts Inc. http:/

Re: [HACKERS] better atomics - v0.2

2014-01-21 Thread Andres Freund
On 2014-01-21 10:20:35 -0500, Robert Haas wrote: > On Sun, Jan 19, 2014 at 2:43 PM, Marti Raudsepp wrote: > > On Tue, Nov 19, 2013 at 6:38 PM, Andres Freund > > wrote: > >> The attached patches compile and make check successfully on linux x86, > >> amd64 and msvc x86 and amd64. This time and upd

Re: [HACKERS] better atomics - v0.2

2014-01-21 Thread Robert Haas
On Sun, Jan 19, 2014 at 2:43 PM, Marti Raudsepp wrote: > On Tue, Nov 19, 2013 at 6:38 PM, Andres Freund wrote: >> The attached patches compile and make check successfully on linux x86, >> amd64 and msvc x86 and amd64. This time and updated configure is >> included. The majority of operations stil

Re: [HACKERS] better atomics - v0.2

2014-01-19 Thread Marti Raudsepp
On Tue, Nov 19, 2013 at 6:38 PM, Andres Freund wrote: > The attached patches compile and make check successfully on linux x86, > amd64 and msvc x86 and amd64. This time and updated configure is > included. The majority of operations still rely on CAS emulation. Note that the atomics-generic-acc.h

Re: [HACKERS] better atomics - v0.2

2013-12-24 Thread Andres Freund
On 2013-12-23 15:04:08 -0500, Robert Haas wrote: > On Thu, Dec 5, 2013 at 6:39 AM, Andres Freund wrote: > > On 2013-11-19 10:37:35 -0500, Tom Lane wrote: > >> Or does the warning > >> mean code bloat (lots of useless copies of the inline function)? > > > > After thinking on that for a bit, yes tha

Re: [HACKERS] better atomics - v0.2

2013-12-23 Thread Robert Haas
On Thu, Dec 5, 2013 at 6:39 AM, Andres Freund wrote: > On 2013-11-19 10:37:35 -0500, Tom Lane wrote: >> Andres Freund writes: >> > The only animal we have that doesn't support quiet inlines today is >> > HP-UX/ac++, and I think - as in patch 1 in the series - we might be able >> > to simply suppr

Re: [HACKERS] better atomics - v0.2

2013-12-21 Thread Peter Eisentraut
This patch didn't make it out of the 2013-11 commit fest. You should move it to the next commit fest (probably with an updated patch) before January 15th. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailp

Re: [HACKERS] better atomics - v0.2

2013-12-05 Thread Andres Freund
On 2013-11-19 10:37:35 -0500, Tom Lane wrote: > Andres Freund writes: > > The only animal we have that doesn't support quiet inlines today is > > HP-UX/ac++, and I think - as in patch 1 in the series - we might be able > > to simply suppress the warning there. > > Or just not worry about it, if i

Re: [HACKERS] better atomics - v0.2

2013-11-19 Thread Andres Freund
On 2013-11-19 17:25:21 -0500, Bruce Momjian wrote: > On Tue, Nov 19, 2013 at 11:21:06PM +0100, Andres Freund wrote: > > On 2013-11-19 17:16:56 -0500, Bruce Momjian wrote: > > > On Tue, Nov 19, 2013 at 10:39:19PM +0100, Andres Freund wrote: > > Do you mean inline? Or atomics? If the former no, if th

Re: [HACKERS] better atomics - v0.2

2013-11-19 Thread Bruce Momjian
On Tue, Nov 19, 2013 at 11:21:06PM +0100, Andres Freund wrote: > On 2013-11-19 17:16:56 -0500, Bruce Momjian wrote: > > On Tue, Nov 19, 2013 at 10:39:19PM +0100, Andres Freund wrote: > > > On 2013-11-19 16:37:32 -0500, Bruce Momjian wrote: > > > > On Tue, Nov 19, 2013 at 04:34:59PM +0100, Andres Fr

Re: [HACKERS] better atomics - v0.2

2013-11-19 Thread Andres Freund
On 2013-11-19 17:16:56 -0500, Bruce Momjian wrote: > On Tue, Nov 19, 2013 at 10:39:19PM +0100, Andres Freund wrote: > > On 2013-11-19 16:37:32 -0500, Bruce Momjian wrote: > > > On Tue, Nov 19, 2013 at 04:34:59PM +0100, Andres Freund wrote: > > > > On 2013-11-19 10:30:24 -0500, Tom Lane wrote: > > >

Re: [HACKERS] better atomics - v0.2

2013-11-19 Thread Bruce Momjian
On Tue, Nov 19, 2013 at 10:39:19PM +0100, Andres Freund wrote: > On 2013-11-19 16:37:32 -0500, Bruce Momjian wrote: > > On Tue, Nov 19, 2013 at 04:34:59PM +0100, Andres Freund wrote: > > > On 2013-11-19 10:30:24 -0500, Tom Lane wrote: > > > > > I don't have an informed opinion about requiring inlin

Re: [HACKERS] better atomics - v0.2

2013-11-19 Thread Andres Freund
On 2013-11-19 16:37:32 -0500, Bruce Momjian wrote: > On Tue, Nov 19, 2013 at 04:34:59PM +0100, Andres Freund wrote: > > On 2013-11-19 10:30:24 -0500, Tom Lane wrote: > > > > I don't have an informed opinion about requiring inline support > > > > (although it would surely be nice). > > > > > > inli

Re: [HACKERS] better atomics - v0.2

2013-11-19 Thread Bruce Momjian
On Tue, Nov 19, 2013 at 04:34:59PM +0100, Andres Freund wrote: > On 2013-11-19 10:30:24 -0500, Tom Lane wrote: > > > I don't have an informed opinion about requiring inline support > > > (although it would surely be nice). > > > > inline is C99, and we've generally resisted requiring C99 features.

Re: [HACKERS] better atomics - v0.2

2013-11-19 Thread Andres Freund
On 2013-11-19 12:43:44 -0500, Robert Haas wrote: > > * To be useful they usually will need to be placed in memory shared between > > * processes or threads, most frequently by embedding them in structs. Be > > * careful to align atomic variables to their own size! > > What does that mean exactl

Re: [HACKERS] better atomics - v0.2

2013-11-19 Thread Robert Haas
On Tue, Nov 19, 2013 at 11:38 AM, Andres Freund wrote: > /*- > * > * atomics.h > *Generic atomic operations support. > * > * Hardware and compiler dependent functions for manipulating memory > * atomically and de

Re: [HACKERS] better atomics - v0.2

2013-11-19 Thread Andres Freund
On 2013-11-19 10:37:35 -0500, Tom Lane wrote: > Andres Freund writes: > > The only animal we have that doesn't support quiet inlines today is > > HP-UX/ac++, and I think - as in patch 1 in the series - we might be able > > to simply suppress the warning there. > > Or just not worry about it, if i

Re: [HACKERS] better atomics - v0.2

2013-11-19 Thread Tom Lane
Andres Freund writes: > The only animal we have that doesn't support quiet inlines today is > HP-UX/ac++, and I think - as in patch 1 in the series - we might be able > to simply suppress the warning there. Or just not worry about it, if it's only a warning? Or does the warning mean code bloat (

Re: [HACKERS] better atomics - v0.2

2013-11-19 Thread Andres Freund
On 2013-11-19 10:30:24 -0500, Tom Lane wrote: > > I don't have an informed opinion about requiring inline support > > (although it would surely be nice). > > inline is C99, and we've generally resisted requiring C99 features. > Maybe it's time to move that goalpost, and maybe not. But it's a part

Re: [HACKERS] better atomics - v0.2

2013-11-19 Thread Robert Haas
On Tue, Nov 19, 2013 at 10:31 AM, Andres Freund wrote: > On 2013-11-19 10:23:57 -0500, Robert Haas wrote: >> > The only fundamental thing that I don't immediately see how we can >> > support is the spinlock based memory barrier since that introduces a >> > circularity (atomics need barrier, barrie

Re: [HACKERS] better atomics - v0.2

2013-11-19 Thread Andres Freund
On 2013-11-19 10:23:57 -0500, Robert Haas wrote: > > The only fundamental thing that I don't immediately see how we can > > support is the spinlock based memory barrier since that introduces a > > circularity (atomics need barrier, barrier needs spinlocks, spinlock > > needs atomics). > > We've bee

Re: [HACKERS] better atomics - v0.2

2013-11-19 Thread Tom Lane
Peter Eisentraut writes: > On 11/19/13, 9:57 AM, Tom Lane wrote: >> Hm. Now that I think about it, isn't Peter proposing to break systems >> without working "inline" over here? >> http://www.postgresql.org/message-id/1384257026.8059.5.ca...@vanquo.pezone.net > No, that's about const, volatile, #

Re: [HACKERS] better atomics - v0.2

2013-11-19 Thread Robert Haas
On Tue, Nov 19, 2013 at 10:26 AM, Andres Freund wrote: > On 2013-11-19 09:12:42 -0500, Robert Haas wrote: >> On point #1, I dunno. It looks like a lot of rearrangement to me, and >> I'm not really sure what the final form of it is intended to be. > > Understandable. I am not that sure what parts

Re: [HACKERS] better atomics - v0.2

2013-11-19 Thread Andres Freund
On 2013-11-19 09:12:42 -0500, Robert Haas wrote: > On point #1, I dunno. It looks like a lot of rearrangement to me, and > I'm not really sure what the final form of it is intended to be. Understandable. I am not that sure what parts we want to rearange either. We very well could leave barrier.h

  1   2   >