Re: [HACKERS] WIP: Access method extendability

I think the variable "magick" should be "magic". Patch attached. bloom-magick.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: Access method extendability

Hello > Fixed. Thanks. I don't have any other suggestions at the moment. Let see whats committers opinion on this. -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] WIP: Access method extendability

Hi! On Fri, Apr 1, 2016 at 12:45 PM, Aleksander Alekseev < a.aleks...@postgrespro.ru> wrote: > Code looks much better now, thanks. Still I believe it could be improved. > > I don't think that using srand() / rand() in signValue procedure the > way you did is such a good idea. You create a side

Re: [HACKERS] WIP: Access method extendability

Hello, Alexander > Hi! > > New revision of patches is attached. Code looks much better now, thanks. Still I believe it could be improved. I don't think that using srand() / rand() in signValue procedure the way you did is such a good idea. You create a side affect (changing current randseed)

Re: [HACKERS] WIP: Access method extendability

On 03/31/16 17:29, Alexander Korotkov wrote: > New revision of patches is attached. > 1) API of generic xlog was changed: now there is no static variables, >GenericXLogStart() returns palloc'd struct. Attached are two trivial documentation editing fixes for this, as an incremental patch.

Re: [HACKERS] WIP: Access method extendability

Hi! New revision of patches is attached. Changes are following: 1) API of generic xlog was changed: now there is no static variables, GenericXLogStart() returns palloc'd struct. 2) Generic xlog use elog instead ereport since it reports internal errors which shouldn't happen normally. 3) Error

Re: [HACKERS] WIP: Access method extendability

Alexander Korotkov wrote: > I heard no objections. There is revision of patch where generic WAL > interface description was moved to documentation. This description > contains improvements by Petr Jelinek, Alvaro Herrera and Markus Nullmeier Attached are a

Re: [HACKERS] WIP: Access method extendability

> > http://www.postgresql.org/message-id/56eff347.20...@anastigmatix.net > That discussion is about SQL-level types which could be stored on > disk, not about in-memory structs I must respectfully disagree. That discussion is also about memory sanitizers and using them on buildfarms. Lets say you

Re: [HACKERS] WIP: Access method extendability

as we discussed recently [1] you should avoid leaving "holes" with uninitialized data in structures. Please fix this or provide a comment that describes why things are done here the way they are done. [1] http://www.postgresql.org/message-id/56eff347.20...@anastigmatix.net That discussion is

Re: [HACKERS] WIP: Access method extendability

GenericXLogStart(Relation relation) { ... if (genericXlogStatus != GXLOG_NOT_STARTED) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("GenericXLogStart: generic xlog is already started"))); Hmm, seems, generic wal whiil be

Re: [HACKERS] WIP: Access method extendability

Referenced by commit commit 473b93287040b20017cc25a157cffdc5b978c254 ("Support CREATE ACCESS METHOD"), commited by alvherre -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: Access method extendability

Hello I did a brief review of bloom contrib and I don't think I like it much. Here are some issues I believe should be fixed before committing it to PostgreSQL. 1) Most of the code is not commented. Every procedure should at least have a breif description of what it does, what arguments it

Re: [HACKERS] WIP: Access method extendability

On Tue, Mar 29, 2016 at 8:34 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Tue, Mar 29, 2016 at 8:29 PM, Teodor Sigaev wrote: > >> I incorporated your changes and did some additional refinements on top of >>> them >>> still. >>> >>> Attached is delta against

Re: [HACKERS] WIP: Access method extendability

On Tue, Mar 29, 2016 at 8:29 PM, Teodor Sigaev wrote: > I incorporated your changes and did some additional refinements on top of >> them >> still. >> >> Attached is delta against v12, that should cause less issues when merging >> for >> Teodor. >> > > But last version is 13th?

Re: [HACKERS] WIP: Access method extendability

On 29/03/16 19:29, Teodor Sigaev wrote: I incorporated your changes and did some additional refinements on top of them still. Attached is delta against v12, that should cause less issues when merging for Teodor. But last version is 13th? No, 12 is last version from Alexander afaics, I named

Re: [HACKERS] WIP: Access method extendability

I incorporated your changes and did some additional refinements on top of them still. Attached is delta against v12, that should cause less issues when merging for Teodor. But last version is 13th? BTW, it would be cool to add odcs in VII. Internals chapter, description should be similar to

Re: [HACKERS] WIP: Access method extendability

On 29/03/16 18:25, Alvaro Herrera wrote: + /*- >+ * API for construction of generic xlog records >+ * >+ * This API allows user to construct generic xlog records which describe >+ * difference between pages in a generic

Re: [HACKERS] WIP: Access method extendability

Petr Jelinek wrote: > And here it is. It's not perfect but it's better (I am not native speaker > either). It's same as v12, just changed comments somewhat. I think this can still be improved a bit more, in particular this large comment, which I edit inline for expediency. > +

Re: [HACKERS] WIP: Access method extendability

And here it is. It's not perfect but it's better (I am not native speaker either). It's same as v12, just changed comments somewhat. Thank you, but I have a problems with applying: % patch -p1 < ~/Downloads/0002-generic-xlog.13.patch Hmm... Looks like a new-style context diff to me... ...

Re: [HACKERS] WIP: Access method extendability

On 29/03/16 17:33, Teodor Sigaev wrote: Does that mean this patch should be closed or is there more remaining to commit? Petr promises to check english in comments/docs in generic-wal patch at this week, bloom patch depends on it. Bloom patch is ready from my point of view. And here it is.

Re: [HACKERS] WIP: Access method extendability

Does that mean this patch should be closed or is there more remaining to commit? Petr promises to check english in comments/docs in generic-wal patch at this week, bloom patch depends on it. Bloom patch is ready from my point of view. -- Teodor Sigaev

Re: [HACKERS] WIP: Access method extendability

On Tue, Mar 29, 2016 at 6:20 PM, David Steele wrote: > On 3/28/16 1:26 PM, Alvaro Herrera wrote: > >> Alexander Korotkov wrote: >> >>> On Thu, Mar 24, 2016 at 6:19 PM, Alvaro Herrera < >>> alvhe...@2ndquadrant.com> >>> wrote: >>> >>> .. Oh crap. I just noticed I forgot to

Re: [HACKERS] WIP: Access method extendability

On 3/28/16 1:26 PM, Alvaro Herrera wrote: Alexander Korotkov wrote: On Thu, Mar 24, 2016 at 6:19 PM, Alvaro Herrera wrote: .. Oh crap. I just noticed I forgot to update a comment in pg_dump's getAccessMethods. And we're missing psql tab-complete support for the

Re: [HACKERS] WIP: Access method extendability

Alexander Korotkov wrote: > On Thu, Mar 24, 2016 at 6:19 PM, Alvaro Herrera > wrote: > > > .. Oh crap. I just noticed I forgot to update a comment in pg_dump's > > getAccessMethods. And we're missing psql tab-complete support for the > > new commands. > > Attached

Re: [HACKERS] WIP: Access method extendability

Hi, Petr! On Thu, Mar 17, 2016 at 10:56 AM, Petr Jelinek wrote: > On 16/03/16 15:31, Teodor Sigaev wrote: > >> Good catch, thanks! Tests were added. >>> >> >> I don't see any objection, is consensus reached? I'm close to comiit >> that... >> >> > I did only cursory review

Re: [HACKERS] WIP: Access method extendability

On Thu, Mar 24, 2016 at 6:19 PM, Alvaro Herrera wrote: > .. Oh crap. I just noticed I forgot to update a comment in pg_dump's > getAccessMethods. And we're missing psql tab-complete support for the > new commands. Attached patches fix both these issues. --

Re: [HACKERS] WIP: Access method extendability

Alvaro Herrera wrote: > Teodor Sigaev wrote: > > > So, per patch status: > > create-am > > ready > > Teodor agreed to me committing this one instead of him; thanks. I just > pushed it after some mostly cosmetic adjustments. This was maybe too laconic. I actually changed the code a good

Re: [HACKERS] WIP: Access method extendability

Teodor Sigaev wrote: > So, per patch status: > create-am > ready Teodor agreed to me committing this one instead of him; thanks. I just pushed it after some mostly cosmetic adjustments. I pass the baton back to Teodor for the remaining patches in this series. Thanks, -- Álvaro Herrera

Re: [HACKERS] WIP: Access method extendability

I don't quite see how this is supposed to work: + #ifdef WAL_DEBUG + /* +* If xlog debug is enabled then check produced delta. Result of delta +* application to saved image should be the same as current page state. +*/ + if (XLOG_DEBUG) + { + chartmp[BLCKSZ]; +

Re: [HACKERS] WIP: Access method extendability

On Tue, Mar 22, 2016 at 11:53 PM, Alvaro Herrera wrote: > Alexander Korotkov wrote: > > On Tue, Mar 22, 2016 at 1:26 AM, Alvaro Herrera < > alvhe...@2ndquadrant.com> > > wrote: > > > > I noticed this state of affairs because I started reading the complete > > > thread

Re: [HACKERS] WIP: Access method extendability

Robert Haas wrote: > On Tue, Mar 22, 2016 at 5:50 AM, Alexander Korotkov > wrote: > >> And then Alexander posted this version, without any discussion that > >> evidenced that those old objections were overridden. What happened > >> here? Did somebody discuss this in

Re: [HACKERS] WIP: Access method extendability

On Tue, Mar 22, 2016 at 5:50 AM, Alexander Korotkov wrote: >> And then Alexander posted this version, without any discussion that >> evidenced that those old objections were overridden. What happened >> here? Did somebody discuss this in unarchived meetings? If so, it >>

Re: [HACKERS] WIP: Access method extendability

Alexander Korotkov wrote: > On Tue, Mar 22, 2016 at 1:26 AM, Alvaro Herrera > wrote: > > I noticed this state of affairs because I started reading the complete > > thread in order to verify whether a consensus had been reached regarding > > the move of IndexQualInfo and

Re: [HACKERS] WIP: Access method extendability

On Tue, Mar 22, 2016 at 1:26 AM, Alvaro Herrera wrote: > So. Before this version of the patch was posted in Nov 4th 2015, both > Tom and Heikki had said essentially "CREATE ACCESS METHOD is worthless, > let's pursue this stuff without those commands". >

Re: [HACKERS] WIP: Access method extendability

On 21/03/16 23:26, Alvaro Herrera wrote: Alexander Korotkov wrote: Hi! Thank you for review! So. Before this version of the patch was posted in Nov 4th 2015, both Tom and Heikki had said essentially "CREATE ACCESS METHOD is worthless, let's pursue this stuff without those commands".

Re: [HACKERS] WIP: Access method extendability

Alexander Korotkov wrote: > Hi! > > Thank you for review! So. Before this version of the patch was posted in Nov 4th 2015, both Tom and Heikki had said essentially "CREATE ACCESS METHOD is worthless, let's pursue this stuff without those commands".

Re: [HACKERS] WIP: Access method extendability

I don't see any objection, is consensus reached? I'm close to comiit that... I did only cursory review on the bloom contrib module and don't really have complaints there, but I know you can review that one. I'd like the English of the generic_xlog.c description improved but I won't get to it

Re: [HACKERS] WIP: Access method extendability

Teodor Sigaev wrote: > > >>I don't see any objection, is consensus reached? I'm close to comiit > >>that... > >> > > > >I did only cursory review on the bloom contrib module and don't really have > >complaints there, but I know you can review that one. I'd like the English of > >the

Re: [HACKERS] WIP: Access method extendability

On 16/03/16 15:31, Teodor Sigaev wrote: Good catch, thanks! Tests were added. I don't see any objection, is consensus reached? I'm close to comiit that... I did only cursory review on the bloom contrib module and don't really have complaints there, but I know you can review that one. I'd

Re: [HACKERS] WIP: Access method extendability

Good catch, thanks! Tests were added. I don't see any objection, is consensus reached? I'm close to comiit that... -- Teodor Sigaev E-mail: teo...@sigaev.ru WWW: http://www.sigaev.ru/ -- Sent via

Re: [HACKERS] WIP: Access method extendability

On Wed, Mar 9, 2016 at 8:31 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > Hi! > > On Wed, Mar 9, 2016 at 3:27 PM, Alvaro Herrera > wrote: > >> Hi. As I just said to Tomas Vondra: since your patch creates a new >> object type, please make sure to add a

Re: [HACKERS] WIP: Access method extendability

Hi. As I just said to Tomas Vondra: since your patch creates a new object type, please make sure to add a case to cover it in the object_address.sql test. That verifies some things such as pg_identify_object and related. Thanks, -- Álvaro Herrerahttp://www.2ndQuadrant.com/

Re: [HACKERS] WIP: Access method extendability

Hi, I went over the latest version of this, here are my notes. create-am.9: + case DO_ACCESS_METHOD: + snprintf(buf, bufsize, +""); + return; Missing the actual description. +

Re: [HACKERS] WIP: Access method extendability

On Mon, Feb 29, 2016 at 7:42 PM, Alexander Korotkov wrote: > On Fri, Feb 19, 2016 at 4:08 AM, Michael Paquier > wrote: >> This is basically a copy of RewindTest.pm. This is far from generic. >> If this patch gets committed first I would

Re: [HACKERS] WIP: Access method extendability

On Fri, Feb 19, 2016 at 12:51 AM, Alexander Korotkov wrote: >> 11 I'd really like to see regression tests (TAP-tests?) for replication >> with generic xlog. > > > TAP test for replication added to bloom contrib. This test run on additional > make target wal-check. Just putting my eyes on that...

Re: [HACKERS] WIP: Access method extendability

On Wed, Feb 17, 2016 at 11:56 PM, Teodor Sigaev wrote: > 11 I'd really like to see regression tests (TAP-tests?) for replication with > generic xlog. The recovery test suite can help to accomplish that. -- Michael -- Sent via pgsql-hackers mailing list

Re: [HACKERS] WIP: Access method extendability

Next version of patch is attached: * Documentation for CREATE ACCESS METHOD command is added. * Refactoring and comments for bloom contrib. Cool work, nice to see. Some comments 1 create-am.7.patch: function lookup_index_am_handler_func() why doesn't it emit error in case of emtpy input?

Re: [HACKERS] WIP: Access method extendability

Alexander Korotkov wrote: > Hi! > > Patches was rebased against master. > > In the attached version of patch access methods get support of pg_dump. Thanks. This patch came in just as the commitfest was ending, so I'm moving it to the next one. -- Álvaro Herrera

Re: [HACKERS] WIP: Access method extendability

The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, failed Spec compliant: not tested Documentation:not tested There are currently no docs or unit tests. I suspect this patch is

Re: [HACKERS] WIP: Access method extendability

Hi! Thank you for review! On Mon, Sep 7, 2015 at 6:41 PM, Teodor Sigaev wrote: > Some notices: > > 1) create-am.3.patch.gz > As I understand, you didn't add schema name to access method. Why? > Suppose, if we implement SQL-like interface for AM screation/dropping then > we

Re: [HACKERS] WIP: Access method extendability

On Mon, Sep 7, 2015 at 3:32 PM, Teodor Sigaev wrote: > Petr Jelinek wrote: >> >> On 2015-09-07 17:41, Teodor Sigaev wrote: >>> >>> Some notices: >>> >>> 1) create-am.3.patch.gz >>>As I understand, you didn't add schema name to access method. Why? >>> Suppose, if we implement

Re: [HACKERS] WIP: Access method extendability

On 2015-09-07 17:41, Teodor Sigaev wrote: Some notices: 1) create-am.3.patch.gz As I understand, you didn't add schema name to access method. Why? Suppose, if we implement SQL-like interface for AM screation/dropping then we should provide a schema qualification for them Why would access

Re: [HACKERS] WIP: Access method extendability

Some notices: 1) create-am.3.patch.gz As I understand, you didn't add schema name to access method. Why? Suppose, if we implement SQL-like interface for AM screation/dropping then we should provide a schema qualification for them 2) create-am.3.patch.gz get_object_address_am() +

Re: [HACKERS] WIP: Access method extendability

On Tue, Sep 1, 2015 at 6:50 PM, Teodor Sigaev wrote: > In general pattern of generic WAL usage is following. >> >> 1) Start using generic WAL: specify relation >> > > M-m, what about extensions which wants to use WAL but WAL record doesn't > connected to any relation? For

Re: [HACKERS] WIP: Access method extendability

Petr Jelinek wrote: On 2015-09-07 17:41, Teodor Sigaev wrote: Some notices: 1) create-am.3.patch.gz As I understand, you didn't add schema name to access method. Why? Suppose, if we implement SQL-like interface for AM screation/dropping then we should provide a schema qualification for

Re: [HACKERS] WIP: Access method extendability

In general pattern of generic WAL usage is following. 1) Start using generic WAL: specify relation M-m, what about extensions which wants to use WAL but WAL record doesn't connected to any relation? For example, transaction manager or kind of FDW. GenericXLogStart(index); 2) Register

Re: [HACKERS] WIP: Access method extendability

Hackers, there is next revision of patches providing access method extendability. Now it's based on another patch which reworks access method interface. http://www.postgresql.org/message-id/capphfdsxwzmojm6dx+tjnpyk27kt4o7ri6x_4oswcbyu1rm...@mail.gmail.com Besides access method interface, major

Re: [HACKERS] WIP: Access method extendability

On 11/10/2014 10:30 PM, Alexander Korotkov wrote: Don't allowing CREATE ACCESS METHOD command seems problematic for me. How could it work with pg_upgrade? pg_dump wouldn't dump extra pg_am records. So, pg_upgrade would break at creating operator classes on new cluster. So, I agree with dropping

Re: [HACKERS] WIP: Access method extendability

Hi, Heikki! Thank you for summarizing. In general, I agree with your notes with some exceptions. On Mon, Nov 24, 2014 at 1:52 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 11/10/2014 10:30 PM, Alexander Korotkov wrote: Don't allowing CREATE ACCESS METHOD command seems problematic

Re: [HACKERS] WIP: Access method extendability

Hi! Thanks everybody for discussion. Sorry for delay. I'll try to address most of questions arised in this discussion. In general, I'm agree with concept of marking index as invalid in certain cases. I see following possible consequences of buggy WAL-logged custom AM: 1) Server crash during

Re: [HACKERS] WIP: Access method extendability

On 28 October 2014 23:25, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-28 20:17:57 +, Simon Riggs wrote: On 28 October 2014 17:47, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-28 17:45:36 +, Simon Riggs wrote: I'd like to avoid all of the pain by making persistent

Re: [HACKERS] WIP: Access method extendability

On Tue, Oct 28, 2014 at 7:25 PM, Andres Freund and...@2ndquadrant.com wrote: To me this is a pretty independent issue. I quite agree. As Stephen was at pains to remind me last night on another thread, we cannot force people to write the patches we think they should write. They get to pursue

Re: [HACKERS] WIP: Access method extendability

On 10/28/14, 3:27 PM, Simon Riggs wrote: On 28 October 2014 17:50, Jim Nasby jim.na...@bluetreble.com wrote: On 10/28/14, 9:22 AM, Simon Riggs wrote: 2. Some additional code in Autovacuum to rebuild corrupt indexes at startup, using AV worker processes to perform a REINDEX CONCURRENTLY. I

Re: [HACKERS] WIP: Access method extendability

On 2014-10-29 14:33:00 -0500, Jim Nasby wrote: On 10/28/14, 3:27 PM, Simon Riggs wrote: On 28 October 2014 17:50, Jim Nasby jim.na...@bluetreble.com wrote: On 10/28/14, 9:22 AM, Simon Riggs wrote: 2. Some additional code in Autovacuum to rebuild corrupt indexes at startup, using AV worker

Re: [HACKERS] WIP: Access method extendability

On 15 October 2014 13:08, Alexander Korotkov aekorot...@gmail.com wrote: Postgres was initially designed to support access methods extendability. This extendability lives to present day. However, this is mostly internal in-core extendability. One can quite easily add new access method into

Re: [HACKERS] WIP: Access method extendability

On Tue, Oct 28, 2014 at 10:22 AM, Simon Riggs si...@2ndquadrant.com wrote: Or put it another way, it will be easier to write new index AMs because we'll be able to skip the WAL part until we know we want it. I like the feature you are proposing, but I don't think that we should block Alexander

Re: [HACKERS] WIP: Access method extendability

On 28 October 2014 14:53, Robert Haas robertmh...@gmail.com wrote: On Tue, Oct 28, 2014 at 10:22 AM, Simon Riggs si...@2ndquadrant.com wrote: Or put it another way, it will be easier to write new index AMs because we'll be able to skip the WAL part until we know we want it. I like the feature

Re: [HACKERS] WIP: Access method extendability

Hi, On 2014-10-15 16:08:38 +0400, Alexander Korotkov wrote: Postgres was initially designed to support access methods extendability. This extendability lives to present day. However, this is mostly internal in-core extendability. One can quite easily add new access method into PostgreSQL

Re: [HACKERS] WIP: Access method extendability

On Tue, Oct 28, 2014 at 7:57 PM, Simon Riggs si...@2ndquadrant.com wrote: On 28 October 2014 14:53, Robert Haas robertmh...@gmail.com wrote: On Tue, Oct 28, 2014 at 10:22 AM, Simon Riggs si...@2ndquadrant.com wrote: Or put it another way, it will be easier to write new index AMs because

Re: [HACKERS] WIP: Access method extendability

* Andres Freund (and...@2ndquadrant.com) wrote: The other thing I'm not sure about is that I'm unconvinced that we really want external AMs... I was wondering about this also and curious as to if there's been any prior on-list discussion about this proposal that I've simply missed..? Would be

Re: [HACKERS] WIP: Access method extendability

On 28 October 2014 16:19, Stephen Frost sfr...@snowman.net wrote: Would be happy to go back and review earlier discussions, of course, but I don't recall there being any. It depends how far back you go. I think I've had at least 2 tries at writing something, but not in last 5 years. --

Re: [HACKERS] WIP: Access method extendability

On 28 October 2014 14:22, Simon Riggs si...@2ndquadrant.com wrote: Or put it another way, it will be easier to write new index AMs because we'll be able to skip the WAL part until we know we want it. To be clear: I am suggesting you do *less* work, not more. By allowing AMs to avoid writing

Re: [HACKERS] WIP: Access method extendability

Stephen Frost sfr...@snowman.net writes: * Andres Freund (and...@2ndquadrant.com) wrote: The other thing I'm not sure about is that I'm unconvinced that we really want external AMs... I was wondering about this also and curious as to if there's been any prior on-list discussion about this

Re: [HACKERS] WIP: Access method extendability

On 2014-10-28 13:06:52 -0400, Tom Lane wrote: Stephen Frost sfr...@snowman.net writes: * Andres Freund (and...@2ndquadrant.com) wrote: The other thing I'm not sure about is that I'm unconvinced that we really want external AMs... I was wondering about this also and curious as to if

Re: [HACKERS] WIP: Access method extendability

Andres Freund and...@2ndquadrant.com writes: On 2014-10-28 13:06:52 -0400, Tom Lane wrote: But having said that, it's quite unclear to me that we need the CREATE/DROP ACCESS METHOD infrastructure proposed here. The traditional theory about that is that if you're competent to develop an AM at

Re: [HACKERS] WIP: Access method extendability

On 28 October 2014 17:06, Tom Lane t...@sss.pgh.pa.us wrote: My own thought is that allowing external AMs is simply a natural consequence of PG's general approach to extensibility, and it would be surprising if we were to decide we didn't want to allow that. If it wasn't clear from my two

Re: [HACKERS] WIP: Access method extendability

On 2014-10-28 17:45:36 +, Simon Riggs wrote: I'd like to avoid all of the pain by making persistent AMs that are recoverable after a crash, rather than during crash recovery. Besides the actual difficulities of supporting this, imo not being available on HS and directly after a failover

Re: [HACKERS] WIP: Access method extendability

Simon Riggs si...@2ndquadrant.com writes: On 28 October 2014 17:06, Tom Lane t...@sss.pgh.pa.us wrote: My own thought is that allowing external AMs is simply a natural consequence of PG's general approach to extensibility, and it would be surprising if we were to decide we didn't want to allow

Re: [HACKERS] WIP: Access method extendability

On 2014-10-28 13:37:33 -0400, Tom Lane wrote: I'm not at all sold on the idea that we need to support dropping AMs. I think it'd be fine to consider that installing an AM into a given database is a one-way operation. Then you just need to insert some pg_depend entries that pin the AM's

Re: [HACKERS] WIP: Access method extendability

On 10/28/14, 9:22 AM, Simon Riggs wrote: 2. Some additional code in Autovacuum to rebuild corrupt indexes at startup, using AV worker processes to perform a REINDEX CONCURRENTLY. I don't think loading more functionality into autovac is the right way to do that. -- Jim Nasby, Data Architect,

Re: [HACKERS] WIP: Access method extendability

On Tue, Oct 28, 2014 at 8:04 PM, Simon Riggs si...@2ndquadrant.com wrote: On 28 October 2014 14:22, Simon Riggs si...@2ndquadrant.com wrote: Or put it another way, it will be easier to write new index AMs because we'll be able to skip the WAL part until we know we want it. To be clear: I

Re: [HACKERS] WIP: Access method extendability

* Alexander Korotkov (aekorot...@gmail.com) wrote: Having access methods as extensions can significantly improves situations here. Imagine, GIN was an extension. One day we decide to change its binary format. Then we can issue new extension, GIN2 for instance. User can upgrade from GIN to GIN2

Re: [HACKERS] WIP: Access method extendability

On Tue, Oct 28, 2014 at 01:51:21PM -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On 28 October 2014 17:06, Tom Lane t...@sss.pgh.pa.us wrote: My own thought is that allowing external AMs is simply a natural consequence of PG's general approach to extensibility, and it

Re: [HACKERS] WIP: Access method extendability

On 28 October 2014 17:47, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-28 17:45:36 +, Simon Riggs wrote: I'd like to avoid all of the pain by making persistent AMs that are recoverable after a crash, rather than during crash recovery. Besides the actual difficulities of

Re: [HACKERS] WIP: Access method extendability

On 28 October 2014 17:58, Alexander Korotkov aekorot...@gmail.com wrote: Also, I'm not sure that many users have enough of courage to use unlogged AMs. Absence of durability is only half of trouble, another half is lack of streaming replication. I think if we have unlogged GIN then external

Re: [HACKERS] WIP: Access method extendability

On 28 October 2014 17:50, Jim Nasby jim.na...@bluetreble.com wrote: On 10/28/14, 9:22 AM, Simon Riggs wrote: 2. Some additional code in Autovacuum to rebuild corrupt indexes at startup, using AV worker processes to perform a REINDEX CONCURRENTLY. I don't think loading more functionality

Re: [HACKERS] WIP: Access method extendability

On 2014-10-28 20:17:57 +, Simon Riggs wrote: On 28 October 2014 17:47, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-28 17:45:36 +, Simon Riggs wrote: I'd like to avoid all of the pain by making persistent AMs that are recoverable after a crash, rather than during crash