Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-27 Thread Robert Haas
On Wed, Jan 27, 2016 at 3:57 AM, Aleksander Alekseev wrote: > I'm a bit concerned regarding assumption that sizeof int never exceeds 4 > bytes. While this could be true today for most C compilers, standard > [1][2] doesn't guarantee that. Perhaps we should add something like: > > StaticAssertStmt(

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-27 Thread Aleksander Alekseev
Hello, Tom. I'm a bit concerned regarding assumption that sizeof int never exceeds 4 bytes. While this could be true today for most C compilers, standard [1][2] doesn't guarantee that. Perhaps we should add something like: StaticAssertStmt(sizeof(int) <= sizeof(int32), "int size exceeds int32

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-26 Thread Tom Lane
Aleksander Alekseev writes: >> Um, that's not too surprising, because they're exactly the same patch? > Wrong diff. Here is correct one. This still had quite a few bugs, but I fixed them (hope I caught everything) and pushed it. I did some performance testing of the ResourceArray code in isola

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-25 Thread Aleksander Alekseev
> Um, that's not too surprising, because they're exactly the same patch? Wrong diff. Here is correct one. Everything else is right. I just re-checked :) step2a: number of transactions actually processed: 16325 latency average: 49.008 ms latency stddev: 8.780 ms tps = 163.182594 (including con

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-25 Thread Tom Lane
Aleksander Alekseev writes: > I compared two implementations - "always use hashing" (step2a.path) and > "use hashing only for large arrays" (step2b.path). Both patches give > the same performance according to benchmark I described in a first > message of this thread. Um, that's not too surprising

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-25 Thread Aleksander Alekseev
Hello, Tom > Also, there are defined ways to convert between Datum format and > other formats (DatumGetPointer() etc). This code isn't using 'em. Fixed. But I was not sure how to deal with File and Buffer types since they are ints (see fd.h and buf.h) and there is no DatumGetInt macro, only Datu

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-23 Thread Tom Lane
Aleksander Alekseev writes: > Oops, wrong patches - here are correct ones. This is certainly not doing what I had in mind for communication between ResourceOwnerGetAny and ResourceOwnerRelease, because the latter pays zero attention to "lastidx", but instead does a blind hash search regardless.

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-24 Thread Aleksander Alekseev
Oops, wrong patches - here are correct ones.diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 0e7acbf..3330c8d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -29,6 +29,156 @@ #include "utils/snapmgr.h

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-24 Thread Aleksander Alekseev
I believe I fixed all flaws mentioned so far (see attachment). Also I did a new benchmark to make sure that new patch makes PostgreSQL work faster and doesn't cause performance degradation in some cases. "usual pgbench" row corresponds to `pgbench -j 8 -c 8 -T 30 pgbench` performed on a 4-core P

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-18 Thread Tom Lane
Robert Haas writes: > I don't know, I'm still not very comfortable with this. And Tom > didn't like dictating that hash_any() must be no-fail, though I'm not > sure why. What I definitely didn't like was assuming at a distance that it would be no-fail. If we're to depend on that, the patch had

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-18 Thread Robert Haas
On Mon, Dec 14, 2015 at 6:47 AM, Aleksander Alekseev wrote: > Here is my fix for item 4. I don't know, I'm still not very comfortable with this. And Tom didn't like dictating that hash_any() must be no-fail, though I'm not sure why. Let's wait to see what others think. I kind of hope there's a

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-14 Thread Aleksander Alekseev
Hello, Robert Here is my fix for item 4. Best regards, Aleksander On Thu, 10 Dec 2015 11:37:23 -0500 Robert Haas wrote: > On Wed, Dec 9, 2015 at 8:30 AM, Aleksander Alekseev > wrote: > > Hello, Robert > > > > Thanks for your review. I believe I fixed items 1, 2 and 3 (see > > attachment). Als

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-11 Thread Aleksander Alekseev
> It would be InvalidBuffer for buffers, -1 for files and NULL for all > void*-types. Does such solution sounds OK? On second thought I believe there is no reason for storing anything for void*-like types. I could just hardcode NULL in PointerResourceArray. Best regards, Aleksander -- Sent v

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-11 Thread Aleksander Alekseev
>> To be honest, I think this patch is really ugly. [...] I'm not sure >> exactly what to do about that, but it seems like a problem. I have an idea. There are actually two types of resources - int-like (buffers, files) and void*-like (RelationRef, TupleDesc, ...). What if I split ResourceArray in

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-10 Thread Robert Haas
On Thu, Dec 10, 2015 at 2:11 PM, Tom Lane wrote: > Robert Haas writes: >> Well, sorta. To be honest, I think this patch is really ugly. If we >> were going to do this then, yes, I would want to split the patch into >> two parts along those lines. But actually I don't really want to do >> it th

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-10 Thread Tom Lane
Robert Haas writes: > Well, sorta. To be honest, I think this patch is really ugly. If we > were going to do this then, yes, I would want to split the patch into > two parts along those lines. But actually I don't really want to do > it this way at all. It's not that I don't want the performan

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-10 Thread Robert Haas
On Wed, Dec 9, 2015 at 8:30 AM, Aleksander Alekseev wrote: > Hello, Robert > > Thanks for your review. I believe I fixed items 1, 2 and 3 (see > attachment). Also I would like to clarify item 4. > >> 4. It mixes together multiple ideas in a single patch, not only >> introducing a hashing concept b

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-09 Thread Aleksander Alekseev
Hello, Robert Thanks for your review. I believe I fixed items 1, 2 and 3 (see attachment). Also I would like to clarify item 4. > 4. It mixes together multiple ideas in a single patch, not only > introducing a hashing concept but also striping a brand-new layer of > abstraction across the resourc

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-08 Thread Robert Haas
On Fri, Dec 4, 2015 at 7:15 AM, Aleksander Alekseev wrote: > Current implementation of ResourceOwner uses arrays to store resources > like TupleDescs, Snapshots, etc. When we want to release one of these > resources ResourceOwner finds it with linear scan. Granted, resource > array are usually sma

[HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-04 Thread Aleksander Alekseev
Hello all, Current implementation of ResourceOwner uses arrays to store resources like TupleDescs, Snapshots, etc. When we want to release one of these resources ResourceOwner finds it with linear scan. Granted, resource array are usually small so its not a problem most of the time. But it appears