Re: Proposal: Global Index

2021-01-18 Thread 曾文旌
> 2021年1月12日 02:37,Robert Haas 写道: > > On Mon, Jan 11, 2021 at 12:46 PM Bruce Momjian wrote: >>> For 1) The DETACH old child table can be finished immediately, global index >>> can be kept valid after DETACH is completed, and the cleanup of garbage >>> data in global index can be deferred

Re: Proposal: Global Index

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 12:05:43PM -0800, Peter Geoghegan wrote: > On Mon, Jan 11, 2021 at 11:25 AM Bruce Momjian wrote: > > Once you layer on all the places a global index will be worse than just > > creating a single large table, or a partitioned table with an index per > > child, there might

Re: Proposal: Global Index

2021-01-11 Thread Peter Geoghegan
On Mon, Jan 11, 2021 at 11:25 AM Bruce Momjian wrote: > Once you layer on all the places a global index will be worse than just > creating a single large table, or a partitioned table with an index per > child, there might not be much usefulness left. A POC patch might tell > us that, and might

Re: Proposal: Global Index

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 11:01:20AM -0800, Peter Geoghegan wrote: > However, it probably would be okay if a global index feature performed > poorly in scenarios where partitions get lots of UPDATEs that produce > lots of index bloat and cause lots of LP_DEAD line pointers to > accumulate in heap

Re: Proposal: Global Index

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 01:37:02PM -0500, Robert Haas wrote: > However, there is a VACUUM amplification effect to worry about here ... > That's not necessarily a death sentence for every use case, but it's > going to be pretty bad for tables that are big and heavily updated. Yeah, I had not

Re: Proposal: Global Index

2021-01-11 Thread Peter Geoghegan
On Mon, Jan 11, 2021 at 10:37 AM Robert Haas wrote: > I actually think the idea of lazily deleting the index entries is > pretty good, but it won't work if the way the global index is > implemented is by adding a tableoid column. Perhaps there is an opportunity to apply some of the

Re: Proposal: Global Index

2021-01-11 Thread Robert Haas
On Mon, Jan 11, 2021 at 12:46 PM Bruce Momjian wrote: > > For 1) The DETACH old child table can be finished immediately, global index > > can be kept valid after DETACH is completed, and the cleanup of garbage > > data in global index can be deferred to VACUUM. > > This is similar to the global

Re: Proposal: Global Index

2021-01-11 Thread Tom Lane
Bruce Momjian writes: > On Mon, Jan 11, 2021 at 07:40:18PM +0800, 曾文旌 wrote: >> This is indeed a typical scenario for a partitioned table. >> there are two basic operations >> 1) Monthly DETACH old child table >> 2) Monthly ATTACH new child table >> >> For 1) The DETACH old child table can be

Re: Proposal: Global Index

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 07:40:18PM +0800, 曾文旌 wrote: > >> In addition you mentioned: "It is still unclear if these use-cases justify > >> the architectural changes needed to enable global indexes." > >> Please also describe the problems you see, I will confirm each specific > >> issue one by

Re: Proposal: Global Index

2021-01-11 Thread 曾文旌
> 2021年1月7日 23:04,Robert Haas 写道: > > On Thu, Jan 7, 2021 at 4:44 AM 曾文旌 wrote: >> I've been following this topic for a long time. It's been a year since the >> last response. >> It was clear that our customers wanted this feature as well, and a large >> number of them mentioned it. >> >>

Re: Proposal: Global Index

2021-01-11 Thread 曾文旌
> 2021年1月8日 16:26,Julien Rouhaud 写道: > > On Fri, Jan 8, 2021 at 4:02 PM 曾文旌 wrote: >> >>> 2021年1月7日 22:16,Bruce Momjian 写道: >>> >>> On Thu, Jan 7, 2021 at 05:44:01PM +0800, 曾文旌 wrote: I've been following this topic for a long time. It's been a year since the last response.

Re: Proposal: Global Index

2021-01-08 Thread Bruce Momjian
On Fri, Jan 8, 2021 at 11:26:48AM +0800, 曾文旌 wrote: > > On Thu, Jan 7, 2021 at 05:44:01PM +0800, 曾文旌 wrote: > >> I've been following this topic for a long time. It's been a year since the > >> last response. > >> It was clear that our customers wanted this feature as well, and a large > >>

Re: Proposal: Global Index

2021-01-08 Thread Julien Rouhaud
On Fri, Jan 8, 2021 at 4:02 PM 曾文旌 wrote: > > > 2021年1月7日 22:16,Bruce Momjian 写道: > > > > On Thu, Jan 7, 2021 at 05:44:01PM +0800, 曾文旌 wrote: > >> I've been following this topic for a long time. It's been a year since the > >> last response. > >> It was clear that our customers wanted this

Re: Proposal: Global Index

2021-01-08 Thread 曾文旌
> 2021年1月7日 22:16,Bruce Momjian 写道: > > On Thu, Jan 7, 2021 at 05:44:01PM +0800, 曾文旌 wrote: >> I've been following this topic for a long time. It's been a year since the >> last response. >> It was clear that our customers wanted this feature as well, and a large >> number of them mentioned

Re: Proposal: Global Index

2021-01-07 Thread Robert Haas
On Thu, Jan 7, 2021 at 4:44 AM 曾文旌 wrote: > I've been following this topic for a long time. It's been a year since the > last response. > It was clear that our customers wanted this feature as well, and a large > number of them mentioned it. > > So, I wish the whole feature to mature as soon as

Re: Proposal: Global Index

2021-01-07 Thread Bruce Momjian
On Thu, Jan 7, 2021 at 05:44:01PM +0800, 曾文旌 wrote: > I've been following this topic for a long time. It's been a year since the > last response. > It was clear that our customers wanted this feature as well, and a large > number of them mentioned it. > > So, I wish the whole feature to mature

Re: Proposal: Global Index

2021-01-07 Thread 曾文旌
I've been following this topic for a long time. It's been a year since the last response.It was clear that our customers wanted this feature as well, and a large number of them mentioned it.So, I wish the whole feature to mature as soon as possible.I summarized the scheme mentioned in the email

Re: Proposal: Global Index

2019-12-19 Thread Bruce Momjian
On Thu, Dec 19, 2019 at 11:28:55AM -0800, Jeremy Schneider wrote: > On 12/19/19 08:12, Bruce Momjian wrote: > > I don't see lossy BRIN indexes helping with the uniqueness use-case, and > > I am not sure they would help with the rare case either. They would > > help for range-based partitions, but

Re: Proposal: Global Index

2019-12-19 Thread Bruce Momjian
On Thu, Dec 19, 2019 at 09:48:40AM +0100, Jose Luis Tallon wrote: > On 19/12/19 4:03, Bruce Momjian wrote: > > On Mon, Nov 25, 2019 at 03:44:39PM -0800, Jeremy Schneider wrote: > > > On 11/25/19 15:05, Jeremy Schneider wrote: > > > > ... the cost of doing the individual index lookups across 180 >

Re: Proposal: Global Index

2019-12-19 Thread Jose Luis Tallon
On 19/12/19 4:03, Bruce Momjian wrote: On Mon, Nov 25, 2019 at 03:44:39PM -0800, Jeremy Schneider wrote: On 11/25/19 15:05, Jeremy Schneider wrote: ... the cost of doing the individual index lookups across 180 partitions (and 180 indexes) was very high, so they stored max and min txn id per

Re: Proposal: Global Index

2019-12-18 Thread Bruce Momjian
On Mon, Nov 25, 2019 at 03:44:39PM -0800, Jeremy Schneider wrote: > On 11/25/19 15:05, Jeremy Schneider wrote: > > ... the cost of doing the individual index lookups across 180 > > partitions (and 180 indexes) was very high, so they stored max and min > > txn id per partition and would generate a

Re: Proposal: Global Index

2019-10-31 Thread Tomas Vondra
On Thu, Oct 31, 2019 at 03:02:40PM -0400, Isaac Morland wrote: On Thu, 31 Oct 2019 at 14:50, Stephen Frost wrote: Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: [] Absolutely- our lack of such is a common point of issue when folks are considering using or migrating to

Re: Proposal: Global Index

2019-10-31 Thread Isaac Morland
On Thu, 31 Oct 2019 at 14:50, Stephen Frost wrote: > Greetings, > > * Peter Geoghegan (p...@bowt.ie) wrote: > [] > > Absolutely- our lack of such is a common point of issue when folks are > considering using or migrating to PostgreSQL. > Not sure how similar my situation really is, but I

Re: Proposal: Global Index

2019-10-31 Thread Stephen Frost
Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: > On Wed, Oct 30, 2019 at 9:23 AM Tom Lane wrote: > > Well, the *effects* of the feature seem desirable, but that doesn't > > mean that we want an implementation that actually has a shared index. > > As soon as you do that, you've thrown away

Re: Proposal: Global Index

2019-10-30 Thread Andres Freund
Hi, On 2019-10-30 13:05:57 -0400, Tom Lane wrote: > Peter Geoghegan writes: > > On Wed, Oct 30, 2019 at 9:23 AM Tom Lane wrote: > >> Well, the *effects* of the feature seem desirable, but that doesn't > >> mean that we want an implementation that actually has a shared index. > >> As soon as you

Re: Proposal: Global Index

2019-10-30 Thread Tom Lane
Peter Geoghegan writes: > On Wed, Oct 30, 2019 at 9:23 AM Tom Lane wrote: >> Well, the *effects* of the feature seem desirable, but that doesn't >> mean that we want an implementation that actually has a shared index. >> As soon as you do that, you've thrown away most of the benefits of >>

Re: Proposal: Global Index

2019-10-30 Thread Peter Geoghegan
On Wed, Oct 30, 2019 at 9:23 AM Tom Lane wrote: > Well, the *effects* of the feature seem desirable, but that doesn't > mean that we want an implementation that actually has a shared index. > As soon as you do that, you've thrown away most of the benefits of > having a partitioned data structure

Re: Proposal: Global Index

2019-10-30 Thread Tom Lane
Robert Haas writes: > On Wed, Oct 30, 2019 at 10:13 AM Tom Lane wrote: >> I believe that the current design of partitioning is explicitly intended >> to avoid the need for such a construct. It'd be absolutely disastrous >> to have such a thing from many standpoints, including the breadth of >>

Re: Proposal: Global Index

2019-10-30 Thread Robert Haas
On Wed, Oct 30, 2019 at 10:13 AM Tom Lane wrote: > I believe that the current design of partitioning is explicitly intended > to avoid the need for such a construct. It'd be absolutely disastrous > to have such a thing from many standpoints, including the breadth of > locking needed to work with

Re: Proposal: Global Index

2019-10-30 Thread Tom Lane
Ibrar Ahmed writes: > A global index by very definition is a single index on the parent table > that maps to many > underlying table partitions. I believe that the current design of partitioning is explicitly intended to avoid the need for such a construct. It'd be absolutely disastrous to have