Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-27 Thread Robert Haas
On Wed, Jan 27, 2016 at 3:57 AM, Aleksander Alekseev wrote: > I'm a bit concerned regarding assumption that sizeof int never exceeds 4 > bytes. While this could be true today for most C compilers, standard > [1][2] doesn't guarantee that. Perhaps we should add something like: > > StaticAssertStmt(

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-27 Thread Aleksander Alekseev
Hello, Tom. I'm a bit concerned regarding assumption that sizeof int never exceeds 4 bytes. While this could be true today for most C compilers, standard [1][2] doesn't guarantee that. Perhaps we should add something like: StaticAssertStmt(sizeof(int) <= sizeof(int32), "int size exceeds int32

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-26 Thread Tom Lane
Aleksander Alekseev writes: >> Um, that's not too surprising, because they're exactly the same patch? > Wrong diff. Here is correct one. This still had quite a few bugs, but I fixed them (hope I caught everything) and pushed it. I did some performance testing of the ResourceArray code in isola

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-25 Thread Aleksander Alekseev
> Um, that's not too surprising, because they're exactly the same patch? Wrong diff. Here is correct one. Everything else is right. I just re-checked :) step2a: number of transactions actually processed: 16325 latency average: 49.008 ms latency stddev: 8.780 ms tps = 163.182594 (including con

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-25 Thread Tom Lane
Aleksander Alekseev writes: > I compared two implementations - "always use hashing" (step2a.path) and > "use hashing only for large arrays" (step2b.path). Both patches give > the same performance according to benchmark I described in a first > message of this thread. Um, that's not too surprising

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-25 Thread Aleksander Alekseev
Hello, Tom > Also, there are defined ways to convert between Datum format and > other formats (DatumGetPointer() etc). This code isn't using 'em. Fixed. But I was not sure how to deal with File and Buffer types since they are ints (see fd.h and buf.h) and there is no DatumGetInt macro, only Datu

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-23 Thread Tom Lane
Aleksander Alekseev writes: > Oops, wrong patches - here are correct ones. This is certainly not doing what I had in mind for communication between ResourceOwnerGetAny and ResourceOwnerRelease, because the latter pays zero attention to "lastidx", but instead does a blind hash search regardless.

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-24 Thread Aleksander Alekseev
Oops, wrong patches - here are correct ones.diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 0e7acbf..3330c8d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -29,6 +29,156 @@ #include "utils/snapmgr.h

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-24 Thread Aleksander Alekseev
I believe I fixed all flaws mentioned so far (see attachment). Also I did a new benchmark to make sure that new patch makes PostgreSQL work faster and doesn't cause performance degradation in some cases. "usual pgbench" row corresponds to `pgbench -j 8 -c 8 -T 30 pgbench` performed on a 4-core P

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-18 Thread Tom Lane
Robert Haas writes: > I don't know, I'm still not very comfortable with this. And Tom > didn't like dictating that hash_any() must be no-fail, though I'm not > sure why. What I definitely didn't like was assuming at a distance that it would be no-fail. If we're to depend on that, the patch had

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-18 Thread Robert Haas
On Mon, Dec 14, 2015 at 6:47 AM, Aleksander Alekseev wrote: > Here is my fix for item 4. I don't know, I'm still not very comfortable with this. And Tom didn't like dictating that hash_any() must be no-fail, though I'm not sure why. Let's wait to see what others think. I kind of hope there's a

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-14 Thread Aleksander Alekseev
Hello, Robert Here is my fix for item 4. Best regards, Aleksander On Thu, 10 Dec 2015 11:37:23 -0500 Robert Haas wrote: > On Wed, Dec 9, 2015 at 8:30 AM, Aleksander Alekseev > wrote: > > Hello, Robert > > > > Thanks for your review. I believe I fixed items 1, 2 and 3 (see > > attachment). Als

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-11 Thread Aleksander Alekseev
> It would be InvalidBuffer for buffers, -1 for files and NULL for all > void*-types. Does such solution sounds OK? On second thought I believe there is no reason for storing anything for void*-like types. I could just hardcode NULL in PointerResourceArray. Best regards, Aleksander -- Sent v

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-11 Thread Aleksander Alekseev
>> To be honest, I think this patch is really ugly. [...] I'm not sure >> exactly what to do about that, but it seems like a problem. I have an idea. There are actually two types of resources - int-like (buffers, files) and void*-like (RelationRef, TupleDesc, ...). What if I split ResourceArray in

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-10 Thread Robert Haas
On Thu, Dec 10, 2015 at 2:11 PM, Tom Lane wrote: > Robert Haas writes: >> Well, sorta. To be honest, I think this patch is really ugly. If we >> were going to do this then, yes, I would want to split the patch into >> two parts along those lines. But actually I don't really want to do >> it th

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-10 Thread Tom Lane
Robert Haas writes: > Well, sorta. To be honest, I think this patch is really ugly. If we > were going to do this then, yes, I would want to split the patch into > two parts along those lines. But actually I don't really want to do > it this way at all. It's not that I don't want the performan

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-10 Thread Robert Haas
On Wed, Dec 9, 2015 at 8:30 AM, Aleksander Alekseev wrote: > Hello, Robert > > Thanks for your review. I believe I fixed items 1, 2 and 3 (see > attachment). Also I would like to clarify item 4. > >> 4. It mixes together multiple ideas in a single patch, not only >> introducing a hashing concept b

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-09 Thread Aleksander Alekseev
Hello, Robert Thanks for your review. I believe I fixed items 1, 2 and 3 (see attachment). Also I would like to clarify item 4. > 4. It mixes together multiple ideas in a single patch, not only > introducing a hashing concept but also striping a brand-new layer of > abstraction across the resourc

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-08 Thread Robert Haas
On Fri, Dec 4, 2015 at 7:15 AM, Aleksander Alekseev wrote: > Current implementation of ResourceOwner uses arrays to store resources > like TupleDescs, Snapshots, etc. When we want to release one of these > resources ResourceOwner finds it with linear scan. Granted, resource > array are usually sma