Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-09-21 Thread Kyotaro HORIGUCHI
At Thu, 21 Sep 2017 20:35:01 -0400, Robert Haas wrote in > On Thu, Sep 21, 2017 at 8:16 PM, Kyotaro HORIGUCHI > wrote: > > Though I don't see it's bug and agree that the message is not > > proper, currently we can create hash indexes without no warning > > on unlogged tables and it causes a pro

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-09-21 Thread Robert Haas
On Thu, Sep 21, 2017 at 8:16 PM, Kyotaro HORIGUCHI wrote: > Though I don't see it's bug and agree that the message is not > proper, currently we can create hash indexes without no warning > on unlogged tables and it causes a problem with replication. That's true, but I don't believe it's a suffic

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-09-21 Thread Kyotaro HORIGUCHI
At Thu, 21 Sep 2017 16:19:27 -0400, Robert Haas wrote in <694cb417-ef2c-4760-863b-aec4530c2...@gmail.com> > On Sep 21, 2017, at 8:59 AM, Amit Kapila wrote:. > > I think giving an error message like "hash indexes are not WAL-logged > > and .." for unlogged tables doesn't seem like a good behavior

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-09-21 Thread Robert Haas
On Sep 21, 2017, at 8:59 AM, Amit Kapila wrote:. > I think giving an error message like "hash indexes are not WAL-logged > and .." for unlogged tables doesn't seem like a good behavior. +1. This seems like deliberate behavior, not a bug. ...Robert -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-09-21 Thread Amit Kapila
On Thu, Sep 21, 2017 at 4:14 PM, Kyotaro HORIGUCHI wrote: > > postgres=# create table test (id int primary key, v text); > postgres=# create index on test using hash (id); > WARNING: hash indexes are not WAL-logged and their use is discouraged > > But not for for unlogged tables. > > postgres=# c

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-09-21 Thread Kyotaro HORIGUCHI
Hello, Following a bit older thread. At Tue, 18 Jul 2017 08:33:07 +0200, Michael Paquier wrote in > On Tue, Jul 18, 2017 at 4:18 AM, Amit Kapila wrote: > > Thanks. Do you have any suggestion for back-branches? As of now, it > > fails badly with below kind of error: > > > > test=> SELECT * F

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-17 Thread Michael Paquier
On Tue, Jul 18, 2017 at 4:18 AM, Amit Kapila wrote: > Thanks. Do you have any suggestion for back-branches? As of now, it > fails badly with below kind of error: > > test=> SELECT * FROM t_u_hash; > ERROR: could not open file "base/16384/16392": No such file or directory > > It is explained in

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-17 Thread Amit Kapila
On Mon, Jul 17, 2017 at 10:21 PM, Michael Paquier wrote: > On Mon, Jul 17, 2017 at 6:02 PM, Robert Haas wrote: >> It seems to me that this area might benefit from a broader rethink. >> It's not very nice to impose a restriction that init forks can only be >> constructed using log_newpage(); on th

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-17 Thread Michael Paquier
On Mon, Jul 17, 2017 at 6:02 PM, Robert Haas wrote: > It seems to me that this area might benefit from a broader rethink. > It's not very nice to impose a restriction that init forks can only be > constructed using log_newpage(); on the other hand, it is also not > very nice that Amit's patch need

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-17 Thread Robert Haas
On Sat, Jul 15, 2017 at 4:25 AM, Michael Paquier wrote: > On Sat, Jul 15, 2017 at 6:27 AM, Ashutosh Sharma > wrote: >> I do agree with Amit. I think hash index is slightly trickier (in >> terms of how it is organised) than other indexes and that could be the >> reason for maintaining common code

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-15 Thread Michael Paquier
On Sat, Jul 15, 2017 at 6:27 AM, Ashutosh Sharma wrote: > I do agree with Amit. I think hash index is slightly trickier (in > terms of how it is organised) than other indexes and that could be the > reason for maintaining common code for hashbuild and hashbuildempty. Well, you both and Robert wor

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-14 Thread Ashutosh Sharma
On Sat, Jul 15, 2017 at 8:20 AM, Amit Kapila wrote: > On Fri, Jul 14, 2017 at 6:02 PM, Michael Paquier > wrote: >> (catching up finally with this thread) >> >> On Mon, Jul 10, 2017 at 11:57 AM, Kyotaro HORIGUCHI >> wrote: >>> At Mon, 10 Jul 2017 14:58:13 +0530, Amit Kapila >>> wrote in >>> >

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-14 Thread Amit Kapila
On Fri, Jul 14, 2017 at 6:02 PM, Michael Paquier wrote: > (catching up finally with this thread) > > On Mon, Jul 10, 2017 at 11:57 AM, Kyotaro HORIGUCHI > wrote: >> At Mon, 10 Jul 2017 14:58:13 +0530, Amit Kapila >> wrote in >>> I am also not 100% comfortable with adding flush at two new place

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-14 Thread Michael Paquier
(catching up finally with this thread) On Mon, Jul 10, 2017 at 11:57 AM, Kyotaro HORIGUCHI wrote: > At Mon, 10 Jul 2017 14:58:13 +0530, Amit Kapila > wrote in >> I am also not 100% comfortable with adding flush at two new places, >> but that makes the code for fix simpler and fundamentally the

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-10 Thread Kyotaro HORIGUCHI
At Mon, 10 Jul 2017 18:37:34 +0530, Amit Kapila wrote in > On Mon, Jul 10, 2017 at 3:27 PM, Kyotaro HORIGUCHI > wrote: > > Hi, > > > > At Mon, 10 Jul 2017 14:58:13 +0530, Amit Kapila > > wrote in > > > > > >> I am also not 100% comfortable with adding flush at two new places, > >> but that

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-10 Thread Amit Kapila
On Mon, Jul 10, 2017 at 3:27 PM, Kyotaro HORIGUCHI wrote: > Hi, > > At Mon, 10 Jul 2017 14:58:13 +0530, Amit Kapila > wrote in > >> I am also not 100% comfortable with adding flush at two new places, >> but that makes the code for fix simpler and fundamentally there is no >> problem in doing so

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-10 Thread Kyotaro HORIGUCHI
Hi, At Mon, 10 Jul 2017 14:58:13 +0530, Amit Kapila wrote in > On Mon, Jul 10, 2017 at 10:39 AM, Kyotaro HORIGUCHI > wrote: > > At Sat, 8 Jul 2017 16:41:27 +0530, Amit Kapila > > wrote in > > > >> On Sat, Jul 8, 2017 at 9:08 AM, Robert Haas wrote: > >> > On Fri, Jul 7, 2017 at 11:36 PM, A

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-10 Thread Amit Kapila
On Mon, Jul 10, 2017 at 10:39 AM, Kyotaro HORIGUCHI wrote: > At Sat, 8 Jul 2017 16:41:27 +0530, Amit Kapila > wrote in >> On Sat, Jul 8, 2017 at 9:08 AM, Robert Haas wrote: >> > On Fri, Jul 7, 2017 at 11:36 PM, Amit Kapila >> > wrote: >> >> I think we should do that as a separate patch (I ca

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-09 Thread Kyotaro HORIGUCHI
At Sat, 8 Jul 2017 16:41:27 +0530, Amit Kapila wrote in > On Sat, Jul 8, 2017 at 9:06 AM, Amit Kapila wrote: > > On Sat, Jul 8, 2017 at 8:52 AM, Robert Haas wrote: > >> On Thu, Jul 6, 2017 at 5:54 AM, Kyotaro HORIGUCHI > >> wrote: > >>> Check for INIT_FORKNUM appears both accompanied and not

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-08 Thread Amit Kapila
On Sat, Jul 8, 2017 at 9:06 AM, Amit Kapila wrote: > On Sat, Jul 8, 2017 at 8:52 AM, Robert Haas wrote: >> On Thu, Jul 6, 2017 at 5:54 AM, Kyotaro HORIGUCHI >> wrote: >>> Check for INIT_FORKNUM appears both accompanied and not >>> accompanied by check for RELPER.._UNLOGGED, so I'm not sure which

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-07 Thread Amit Kapila
On Sat, Jul 8, 2017 at 9:08 AM, Robert Haas wrote: > On Fri, Jul 7, 2017 at 11:36 PM, Amit Kapila wrote: >> I think we should do that as a separate patch (I can write the same as >> well) because that is not new behavior introduced by this patch, ... > > True, although maybe hash indexes are the

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-07 Thread Robert Haas
On Fri, Jul 7, 2017 at 11:36 PM, Amit Kapila wrote: > I think we should do that as a separate patch (I can write the same as > well) because that is not new behavior introduced by this patch, ... True, although maybe hash indexes are the only way that happens today? -- Robert Haas EnterpriseDB:

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-07 Thread Amit Kapila
On Sat, Jul 8, 2017 at 8:52 AM, Robert Haas wrote: > On Thu, Jul 6, 2017 at 5:54 AM, Kyotaro HORIGUCHI > wrote: >> Check for INIT_FORKNUM appears both accompanied and not >> accompanied by check for RELPER.._UNLOGGED, so I'm not sure which >> is the convention here. > > Checking only for INIT_FOR

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-07 Thread Robert Haas
On Thu, Jul 6, 2017 at 5:54 AM, Kyotaro HORIGUCHI wrote: > Check for INIT_FORKNUM appears both accompanied and not > accompanied by check for RELPER.._UNLOGGED, so I'm not sure which > is the convention here. Checking only for INIT_FORKNUM seems logical to me. Also checking for RELPERSISTENCE_UN

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-07 Thread Robert Haas
On Wed, Jul 5, 2017 at 11:02 PM, Noah Misch wrote: > [Action required within three days. This is a generic notification.] > > The above-described topic is currently a PostgreSQL 10 open item. Robert, > since you committed the patch believed to have created it, you own this open > item. If some

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-06 Thread Amit Kapila
On Thu, Jul 6, 2017 at 3:24 PM, Kyotaro HORIGUCHI wrote: > Hello, > > At Tue, 4 Jul 2017 14:49:26 +0530, Amit Kapila > wrote in >> On Tue, Jul 4, 2017 at 1:23 PM, Michael Paquier >> wrote: >> > On Mon, Jul 3, 2017 at 7:40 PM, Amit Kapila >> > wrote: >> > >> > It seems to me that this qualifi

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-06 Thread Kyotaro HORIGUCHI
FWIW.. At Thu, 06 Jul 2017 18:54:47 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20170706.185447.256482539.horiguchi.kyot...@lab.ntt.co.jp> > > > + /* > > > +* Force the on-disk state of init forks to always be in sync > > > with the > > > +* state in shared b

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-06 Thread Kyotaro HORIGUCHI
Hello, At Tue, 4 Jul 2017 14:49:26 +0530, Amit Kapila wrote in > On Tue, Jul 4, 2017 at 1:23 PM, Michael Paquier > wrote: > > On Mon, Jul 3, 2017 at 7:40 PM, Amit Kapila wrote: > > > > It seems to me that this qualifies as an open item for 10. WAL-logging > > is new for hash tables. Amit, cou

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-05 Thread Noah Misch
On Mon, Jul 03, 2017 at 09:12:01AM +0530, Amit Kapila wrote: > While discussing the behavior of hash indexes with Bruce in the nearby > thread [1], it has been noticed that hash index on unlogged tables > doesn't behave as expected. Prior to 10, it has the different set of > problems (mainly becau

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-04 Thread Amit Kapila
On Tue, Jul 4, 2017 at 1:23 PM, Michael Paquier wrote: > On Mon, Jul 3, 2017 at 7:40 PM, Amit Kapila wrote: > > It seems to me that this qualifies as an open item for 10. WAL-logging > is new for hash tables. Amit, could you add one? > Added. > + use_wal = RelationNeedsWAL(rel) || > +

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-04 Thread Michael Paquier
On Mon, Jul 3, 2017 at 7:40 PM, Amit Kapila wrote: > There is already such a test, see create_index.sql. > CREATE UNLOGGED TABLE unlogged_hash_table (id int4); > CREATE INDEX unlogged_hash_index ON unlogged_hash_table USING hash (id > int4_ops); > > Do you have something else in mind? > > I think

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-03 Thread Ashutosh Sharma
Hi, On Mon, Jul 3, 2017 at 4:10 PM, Amit Kapila wrote: > > On Mon, Jul 3, 2017 at 3:24 PM, Ashutosh Sharma wrote: > > On Mon, Jul 3, 2017 at 9:12 AM, Amit Kapila wrote: > >> The basic issue was that the WAL logging for Create Index operation > >> was oblivion of the fact that for unlogged table

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-03 Thread Amit Kapila
On Mon, Jul 3, 2017 at 3:24 PM, Ashutosh Sharma wrote: > On Mon, Jul 3, 2017 at 9:12 AM, Amit Kapila wrote: >> The basic issue was that the WAL logging for Create Index operation >> was oblivion of the fact that for unlogged tables only INIT forks need >> to be logged. Another point which we nee

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-03 Thread Ashutosh Sharma
Hi, On Mon, Jul 3, 2017 at 9:12 AM, Amit Kapila wrote: > While discussing the behavior of hash indexes with Bruce in the nearby > thread [1], it has been noticed that hash index on unlogged tables > doesn't behave as expected. Prior to 10, it has the different set of > problems (mainly because h

[HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-02 Thread Amit Kapila
While discussing the behavior of hash indexes with Bruce in the nearby thread [1], it has been noticed that hash index on unlogged tables doesn't behave as expected. Prior to 10, it has the different set of problems (mainly because hash indexes are not WAL-logged) which were discussed on that thre