Re: [HACKERS] Move unused buffers to freelist

2013-06-30 Thread Amit kapila
On Friday, June 28, 2013 6:20 PM Robert Haas wrote: On Fri, Jun 28, 2013 at 12:52 AM, Amit Kapila wrote: >> Currently it wakes up based on bgwriterdelay config parameter which is by >> default 200ms, so you means we should >> think of waking up bgwriter based on allocations and number of elements

Re: [HACKERS] Move unused buffers to freelist

2013-06-30 Thread Amit kapila
On Friday, June 28, 2013 6:38 PM Robert Haas wrote: On Fri, Jun 28, 2013 at 8:50 AM, Robert Haas wrote: > On Fri, Jun 28, 2013 at 12:52 AM, Amit Kapila wrote: >>> Currently it wakes up based on bgwriterdelay config parameter which is by >>> default 200ms, so you means we should >>> think of waki

Re: [HACKERS] New regression test time

2013-06-30 Thread Amit kapila
On Sunday, June 30, 2013 11:37 AM Fabien COELHO wrote: >> If we had a different set of tests, that would be a valid argument. But >> we don't, so it's not. And nobody has offered to write a feature to >> split our tests either. >I have done a POC. See: > https://commitfest.postgresql.org/actio

Re: [HACKERS] Request for Patch Feedback: Lag & Lead Window Functions Can Ignore Nulls

2013-06-30 Thread Dean Rasheed
On 29 June 2013 17:30, Jeff Davis wrote: > > On Mon, 2013-06-24 at 18:01 +0100, Nicholas White wrote: >> Good catch - I've attached a patch to address your point 1. It now >> returns the below (i.e. correctly doesn't fill in the saved value if >> the index is out of the window. However, I'm not su

Re: [HACKERS] GIN improvements part2: fast scan

2013-06-30 Thread Heikki Linnakangas
On 28.06.2013 22:31, Alexander Korotkov wrote: Now, I got the point of three state consistent: we can keep only one consistent in opclasses that support new interface. exact true and exact false values will be passed in the case of current patch consistent; exact false and unknown will be passed

Re: [HACKERS] GIN improvements part 3: ordering in index

2013-06-30 Thread Heikki Linnakangas
On 25.06.2013 21:18, Alexander Korotkov wrote: On Tue, Jun 25, 2013 at 7:31 PM, Heikki Linnakangas wrote: In summary: The test case you presented as motivation for this patch is a bit of a worst-case scenario for the current tidbitmap implementation. The speedup from your patch comes from avoi

[HACKERS] plpython implementation

2013-06-30 Thread Szymon Guz
I'm reading through plperl and plpython implementations and I don't understand the way they work. Comments for plperl say that there are two interpreters (trusted and untrusted) for each user session, and they are stored in a hash. Plpython version looks quite different, there is no such global h

Re: [HACKERS] plpython implementation

2013-06-30 Thread Martijn van Oosterhout
On Sun, Jun 30, 2013 at 01:49:53PM +0200, Szymon Guz wrote: > I'm reading through plperl and plpython implementations and I don't > understand the way they work. > > Comments for plperl say that there are two interpreters (trusted and > untrusted) for each user session, and they are stored in a ha

Re: [HACKERS] plpython implementation

2013-06-30 Thread Andrew Dunstan
On 06/30/2013 07:49 AM, Szymon Guz wrote: I'm reading through plperl and plpython implementations and I don't understand the way they work. Comments for plperl say that there are two interpreters (trusted and untrusted) for each user session, and they are stored in a hash. Plpython version

Re: [HACKERS] plpython implementation

2013-06-30 Thread Szymon Guz
On 30 June 2013 14:13, Andrew Dunstan wrote: > > On 06/30/2013 07:49 AM, Szymon Guz wrote: > >> I'm reading through plperl and plpython implementations and I don't >> understand the way they work. >> >> Comments for plperl say that there are two interpreters (trusted and >> untrusted) for each us

[HACKERS] Randomisation for ensuring nlogn complexity in quicksort

2013-06-30 Thread Atri Sharma
Hi all, I have been reading the recent discussion and was researching a bit, and I think that we should really go with the idea of randomising the input data(if it is not completely presorted), to ensure that we do not get quadratic complexity. One easy way to do that could be to take a sample

Re: [HACKERS] plpython implementation

2013-06-30 Thread Martijn van Oosterhout
On Sun, Jun 30, 2013 at 02:18:07PM +0200, Szymon Guz wrote: > > python does not any any sort of reliable sandbox, so there is no plpython, > > only plpythonu - hence only one interpreter per backend is needed. > > > Is there any track of the discussion that there is no way to make the > sandbox? I

Re: [HACKERS] plpython implementation

2013-06-30 Thread Andrew Dunstan
On 06/30/2013 08:18 AM, Szymon Guz wrote: python does not any any sort of reliable sandbox, so there is no plpython, only plpythonu - hence only one interpreter per backend is needed. Is there any track of the discussion that there is no way to make the sandbox? I managed to cr

Re: [HACKERS] plpython implementation

2013-06-30 Thread Szymon Guz
On 30 June 2013 14:31, Martijn van Oosterhout wrote: > On Sun, Jun 30, 2013 at 02:18:07PM +0200, Szymon Guz wrote: > > > python does not any any sort of reliable sandbox, so there is no > plpython, > > > only plpythonu - hence only one interpreter per backend is needed. > > > > > Is there any tra

Re: [HACKERS] plpython implementation

2013-06-30 Thread Andres Freund
On 2013-06-30 14:42:24 +0200, Szymon Guz wrote: > On 30 June 2013 14:31, Martijn van Oosterhout wrote: > > > On Sun, Jun 30, 2013 at 02:18:07PM +0200, Szymon Guz wrote: > > > > python does not any any sort of reliable sandbox, so there is no > > plpython, > > > > only plpythonu - hence only one i

Re: [HACKERS] plpython implementation

2013-06-30 Thread Szymon Guz
On 30 June 2013 14:45, Andres Freund wrote: > On 2013-06-30 14:42:24 +0200, Szymon Guz wrote: > > On 30 June 2013 14:31, Martijn van Oosterhout wrote: > > > > > On Sun, Jun 30, 2013 at 02:18:07PM +0200, Szymon Guz wrote: > > > > > python does not any any sort of reliable sandbox, so there is no

Re: [HACKERS] review: Non-recursive processing of AND/OR lists

2013-06-30 Thread Gurjeet Singh
On Tue, Jun 18, 2013 at 3:01 PM, Pavel Stehule wrote: > > related to > > https://commitfest.postgresql.org/action/patch_view?id=1130 > > http://www.postgresql.org/message-id/cabwtf4v9rsjibwe+87pk83mmm7acdrg7sz08rq-4qyme8jv...@mail.gmail.com > > > * motivation: remove recursive procession of AND/OR

Re: [HACKERS] New regression test time

2013-06-30 Thread Fabien COELHO
https://commitfest.postgresql.org/action/patch_view?id=1170 I think it is better to submit for next commit fest which is at below link: https://commitfest.postgresql.org/action/commitfest_view?id=19 I put it there as the discussion whether to accept or not Robins patches because of their p

Re: [HACKERS] review: Non-recursive processing of AND/OR lists

2013-06-30 Thread Pavel Stehule
Hello just one small notices I dislike a name "root_bool_expr", because, there is not a expression, but expression type. Can you use "root_bool_expr_type" instead? It is little bit longer, but more correct. Same not best name is "root_char", maybe "root_bool_op_name" or root_expr_type and root_o

Re: [HACKERS] review: Non-recursive processing of AND/OR lists

2013-06-30 Thread Gurjeet Singh
On Sun, Jun 30, 2013 at 11:13 AM, Pavel Stehule wrote: > Hello > > just one small notices > > I dislike a name "root_bool_expr", because, there is not a expression, > but expression type. Can you use "root_bool_expr_type" instead? It is > little bit longer, but more correct. Same not best name is

Re: [HACKERS] New regression test time

2013-06-30 Thread Robins Tharakan
On 30 June 2013 02:33, Amit kapila wrote: > > On Sunday, June 30, 2013 11:37 AM Fabien COELHO wrote: > >> If we had a different set of tests, that would be a valid argument. But > >> we don't, so it's not. And nobody has offered to write a feature to > >> split our tests either. > > >I have don

Re: [HACKERS] review: Non-recursive processing of AND/OR lists

2013-06-30 Thread Pavel Stehule
2013/6/30 Gurjeet Singh : > On Sun, Jun 30, 2013 at 11:13 AM, Pavel Stehule > wrote: >> >> Hello >> >> just one small notices >> >> I dislike a name "root_bool_expr", because, there is not a expression, >> but expression type. Can you use "root_bool_expr_type" instead? It is >> little bit longer,

Re: [HACKERS] review: Non-recursive processing of AND/OR lists

2013-06-30 Thread Gurjeet Singh
On Sun, Jun 30, 2013 at 11:46 AM, Pavel Stehule wrote: > 2013/6/30 Gurjeet Singh : > > On Sun, Jun 30, 2013 at 11:13 AM, Pavel Stehule > > > wrote: > > > > How about naming those 3 variables as follows: > > > > root_expr_kind > > root_expr_name > > root_bool_expr_type > > +1 Thanks. Attached is

[HACKERS] Fwd: review: Non-recursive processing of AND/OR lists

2013-06-30 Thread Gurjeet Singh
On Sun, Jun 30, 2013 at 11:46 AM, Pavel Stehule wrote: > 2013/6/30 Gurjeet Singh : > > On Sun, Jun 30, 2013 at 11:13 AM, Pavel Stehule > > > wrote: > > > > How about naming those 3 variables as follows: > > > > root_expr_kind > > root_expr_name > > root_bool_expr_type > > +1 Thanks. Attached is

Re: [HACKERS] review: Non-recursive processing of AND/OR lists

2013-06-30 Thread Pavel Stehule
2013/6/30 Gurjeet Singh : > On Sun, Jun 30, 2013 at 11:46 AM, Pavel Stehule > wrote: >> >> 2013/6/30 Gurjeet Singh : >> > On Sun, Jun 30, 2013 at 11:13 AM, Pavel Stehule >> > >> > wrote: >> > >> > How about naming those 3 variables as follows: >> > >> > root_expr_kind >> > root_expr_name >> > roo

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-06-30 Thread Jeff Davis
On Tue, 2013-05-28 at 22:10 -0400, Greg Smith wrote: > I was just thinking of something to run in your test program, not > another build time check. Just run the new allocation sequence, and > then check the resulting WAL file for a) correct length, and b) 16K of > zero bytes. I would like to

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-06-30 Thread Jeff Davis
On Fri, 2013-06-28 at 11:38 -0700, Josh Berkus wrote: > Since Greg seems to be busy, what needs to be done to test this? As I understand it, he was mainly asking if posix_fallocate works at all. I tried to address that question with a simple test, which behaves as I expected it to: http://www.pos

Re: [HACKERS] [PATCH] big test separation POC

2013-06-30 Thread Fabien COELHO
Note about the POC patch limitations/questions: - is deriving a schedule with a piece of shell okay? or should perl/python/whatever scripting be better? - the big_schedule is assumed "sequential", i.e. one test per line. maybe it could/should be parallel? - I'm not sure of the "parall

Re: [HACKERS] [PATCH] big test separation POC

2013-06-30 Thread Andrew Dunstan
On 06/30/2013 02:54 PM, Fabien COELHO wrote: Note about the POC patch limitations/questions: - is deriving a schedule with a piece of shell okay? or should perl/python/whatever scripting be better? I would think all we need are the results, i.e. the schedule files, plus some Makefile e

Re: [HACKERS] New regression test time

2013-06-30 Thread Jeff Janes
On Sat, Jun 29, 2013 at 3:43 PM, Andrew Dunstan wrote: > > On 06/29/2013 05:59 PM, Josh Berkus wrote: > > Maybe there is a good case for these last two in a different set of tests. >>> >> If we had a different set of tests, that would be a valid argument. But >> we don't, so it's not. And nobo

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-06-30 Thread Jeff Davis
On Sun, 2013-06-30 at 11:11 -0700, Jeff Davis wrote: > Unless something surprising comes up, or someone thinks and objection > has been missed, I am going to commit this soon. Quick question to anyone who happens to know: What is the standard procedure for changes to pg_config.h.win32? I looked a

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-06-30 Thread Andrew Dunstan
On 06/30/2013 03:50 PM, Jeff Davis wrote: On Sun, 2013-06-30 at 11:11 -0700, Jeff Davis wrote: Unless something surprising comes up, or someone thinks and objection has been missed, I am going to commit this soon. Quick question to anyone who happens to know: What is the standard procedure fo

Re: [HACKERS] Request for Patch Feedback: Lag & Lead Window Functions Can Ignore Nulls

2013-06-30 Thread Nicholas White
> this should throw a FEATURE_NOT_SUPPORTED error if it is used for window functions that don't support it > arbitrary aggregate functions over a window ... should also throw a FEATURE_NOT_SUPPORTED error. Fixed (with test cases) in the attached patch. > because the same window may be shared by m

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-06-30 Thread Greg Smith
On 6/30/13 2:01 PM, Jeff Davis wrote: Simple test program attached, which creates two files and fills them: one by 2048 8KB writes; and another by 1 posix_fallocate of 16MB. Then, I just cmp the resulting files (and also "ls" them, to make sure they are 16MB). This makes platform level testing

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-06-30 Thread Jon Nelson
On Sun, Jun 30, 2013 at 5:55 PM, Greg Smith wrote: > > > pwrite(4, "\0", 1, 16769023)= 1 > pwrite(4, "\0", 1, 16773119)= 1 > pwrite(4, "\0", 1, 16777215)= 1 > > That's glibc helpfully converting your call to posix_fallocate into small > writes, because the OS do

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-06-30 Thread Greg Smith
On 5/28/13 10:00 PM, Jon Nelson wrote: A note: The attached test program uses *fsync* instead of *fdatasync* after calling fallocate (or writing out 16MB of zeroes), per an earlier suggestion. I tried this out on the RHEL5 platform I'm worried about now. There's something weird about the tes

Re: FILTER for aggregates [was Re: [HACKERS] Department of Redundancy Department: makeNode(FuncCall) division]

2013-06-30 Thread David Fetter
On Fri, Jun 28, 2013 at 09:22:52PM +0100, Dean Rasheed wrote: > On 21 June 2013 06:16, David Fetter wrote: > > Please find attached a patch which allows subqueries in the FILTER > > clause and adds regression testing for same. > > > > This needs re-basing/merging following Robert's recent commit

Re: [HACKERS] Department of Redundancy Department: makeNode(FuncCall) division

2013-06-30 Thread David Fetter
On Fri, Jun 28, 2013 at 01:28:35PM -0400, Peter Eisentraut wrote: > On 6/28/13 11:30 AM, Robert Haas wrote: > > On Fri, Jun 28, 2013 at 10:31 AM, Tom Lane wrote: > >> David Fetter writes: > >>> Please find attached the latest patch. > >> > >> I remain of the opinion that this is simply a bad idea

Re: [HACKERS] Review: query result history in psql

2013-06-30 Thread ian link
Not sure about all of your suggestions. Let me see if I can clarify what you're looking for. > * simply decision if content should be stored in history or not, Do you mean that the user should use a flag to place the result of a query into the history? like: --ans SELECT * FROM cities... Not su

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-06-30 Thread Jon Nelson
On Sun, Jun 30, 2013 at 6:49 PM, Greg Smith wrote: > On 5/28/13 10:00 PM, Jon Nelson wrote: > >> A note: The attached test program uses *fsync* instead of *fdatasync* >> after calling fallocate (or writing out 16MB of zeroes), per an >> earlier suggestion. > > > I tried this out on the RHEL5 platf

Re: [HACKERS] plpython implementation

2013-06-30 Thread Claudio Freire
On Sun, Jun 30, 2013 at 9:45 AM, Andres Freund wrote: > On 2013-06-30 14:42:24 +0200, Szymon Guz wrote: >> On 30 June 2013 14:31, Martijn van Oosterhout wrote: >> >> > On Sun, Jun 30, 2013 at 02:18:07PM +0200, Szymon Guz wrote: >> > > > python does not any any sort of reliable sandbox, so there i

Re: [HACKERS] Request for Patch Feedback: Lag & Lead Window Functions Can Ignore Nulls

2013-06-30 Thread Nicholas White
I've attached another iteration of the patch that fixes the multiple-window bug and adds (& uses) a function to create a Bitmapset using a custom allocator. I don't think there's any outstanding problems with it now. > Alternatively, it might be trivial to make all aggregate functions work with ig

[HACKERS] build postgresql-9.3beta2 on xubuntu 12.04 without failure

2013-06-30 Thread Boris Skegin
Hi. Name: postgresql-9.3 , ftp.postgresql.org/pub/source/v9.3beta2/postgresql-9.3beta2.tar.gz as of June 24, 2013, 7:03 p.m. Release: beta2 Test Type: build Platform: xubuntu 12.04 Installation Method: building from sourse, gmake install-world Platform Detail: 2 core , 3 GB RAM Test Proce

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-06-30 Thread Robert Haas
On Sat, Jun 29, 2013 at 11:24 AM, Robins wrote: > On 10 June 2013 00:17, Jeff Davis wrote: >> >> On Thu, 2013-05-30 at 10:07 -0700, Jeff Davis wrote: >> > > Come to think of it, even without the torn page & checksum issue, do >> > > we >> > > really want to actively clear the all-visible flags af

Re: [HACKERS] New regression test time

2013-06-30 Thread Josh Berkus
On 06/30/2013 12:33 AM, Amit kapila wrote: > > On Sunday, June 30, 2013 11:37 AM Fabien COELHO wrote: >>> If we had a different set of tests, that would be a valid argument. But >>> we don't, so it's not. And nobody has offered to write a feature to >>> split our tests either. > >> I have done

Re: [HACKERS] Support for RANGE ... PRECEDING windows in OVER

2013-06-30 Thread ian link
I found some time and I think I am up to speed now. I finally figured out how to add new operator strategies and made a little test operator for myself. It seems pretty clear that assuming '+' and '-' are addition and subtraction is a bad idea. I don't think it would be too tricky to add support f

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-06-30 Thread Josh Berkus
> I thought that Jeff withdrew this patch. > He did, but nobody removed it from the commitfest --- partly because of a change of subject line breaking the thread. Bounced to "returned with feedback" now. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-06-30 Thread Greg Smith
On 6/30/13 9:28 PM, Jon Nelson wrote: The performance of the latter (new) test sometimes seems to perform worse and sometimes seems to perform better (usually worse) than either of the other two. In all cases, posix_fallocate performs better, but I don't have a sufficiently old kernel to test wit

Re: [HACKERS] Support for RANGE ... PRECEDING windows in OVER

2013-06-30 Thread Josh Berkus
On 06/30/2013 08:54 PM, ian link wrote: > I found some time and I think I am up to speed now. I finally figured out > how to add new operator strategies and made a little test operator for > myself. > > It seems pretty clear that assuming '+' and '-' are addition and > subtraction is a bad idea. I

Re: [HACKERS] [PATCH] big test separation POC

2013-06-30 Thread Fabien COELHO
Note about the POC patch limitations/questions: - is deriving a schedule with a piece of shell okay? or should perl/python/whatever scripting be better? I would think all we need are the results, i.e. the schedule files, plus some Makefile entries for them. You can replicate data, but

Re: [HACKERS] [PATCH] big test separation POC

2013-06-30 Thread Fabien COELHO
- I do not understand why the makefile specifies $(srcdir) before local files in some places. For VPATH builds :-) Here is a v2 which is more likely to work under VPATH. -- Fabien.diff --git a/src/test/regress/GNUmakefile b/src/test/regress/GNUmakefile index 7309b00..5a6d0f9 100644 ---

Re: [HACKERS] plpython implementation

2013-06-30 Thread james
On 01/07/2013 02:43, Claudio Freire wrote: In essence, you'd have to use another implementation. CPython guys have left it very clear they don't intend to "fix" that, as they don't consider it a bug. It's just how it is. Given how useful it is to have a scripting language that can be used outsid

Re: [HACKERS] plpython implementation

2013-06-30 Thread Claudio Freire
On Mon, Jul 1, 2013 at 2:29 AM, james wrote: > On 01/07/2013 02:43, Claudio Freire wrote: >> >> In essence, you'd have to use another implementation. CPython guys >> have left it very clear they don't intend to "fix" that, as they don't >> consider it a bug. It's just how it is. > > Given how usef

Re: [HACKERS] Review: query result history in psql

2013-06-30 Thread Pavel Stehule
Hello 2013/7/1 ian link : > Not sure about all of your suggestions. Let me see if I can clarify what > you're looking for. > >> >> * simply decision if content should be stored in history or not, > > Do you mean that the user should use a flag to place the result of a query > into the history? >

Re: [HACKERS] plpython implementation

2013-06-30 Thread Hannu Krosing
On 07/01/2013 07:53 AM, Claudio Freire wrote: > On Mon, Jul 1, 2013 at 2:29 AM, james wrote: >> On 01/07/2013 02:43, Claudio Freire wrote: >>> In essence, you'd have to use another implementation. CPython guys >>> have left it very clear they don't intend to "fix" that, as they don't >>> consider

Re: [HACKERS] Request for Patch Feedback: Lag & Lead Window Functions Can Ignore Nulls

2013-06-30 Thread Dean Rasheed
On 1 July 2013 03:07, Nicholas White wrote: >> Alternatively, it might be trivial to make all aggregate functions work >> with ignore nulls in a window context > > This is a good idea, but I'd like to keep the scope of this patch limited > for the time being Agreed. > - I'll look at doing this