Avoiding Tablespace path collision for primary and standby

Currently, if primary and standby are setup on same machine (which is always the case for development), CREATE TABLESPACE xyz LOCATION '/abc', primary and mirror both write to "/abc/TABLESPACE_VERSION_DIRECTORY" directory. Collision is certainly not an issue in any production deployment but seems

Re: Avoiding Tablespace path collision for primary and standby

On Fri, May 25, 2018 at 7:33 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Ashwin Agrawal <aagra...@pivotal.io> writes: > > Proposing to create directory with timestamp at time of creating > tablespace > > and create symbolic link to it instead. > > I'm skeptical

Re: Keeping temporary tables in shared buffers

On Thu, May 24, 2018 at 11:50 PM, Andres Freund wrote: > On 2018-05-25 09:40:10 +0300, Heikki Linnakangas wrote: > > On 25/05/18 09:25, Asim Praveen wrote: > > > My parochial vision of the overhead is restricted to 4 * NBuffers of > > > additional shared memory, as 4 bytes

Re: Avoiding Tablespace path collision for primary and standby

On Sat, May 26, 2018 at 7:08 AM, Tom Lane wrote: > Thomas Munro writes: > > I also wondered about this when trying to figure out how to write a > > TAP test for recovery testing with tablespaces, for my undo proposal. > > I was starting to wonder about either allowing relative paths or > >

Re: Recovery performance of DROP DATABASE with many tablespaces

On Mon, Jun 4, 2018 at 9:46 AM, Fujii Masao wrote: > Generally the recovery performance of DROP DATABASE is not critical > for many users. But unfortunately my colleague's project might need to > sometimes drop the database using multiple tablespaces, for some reasons. > So, if the fix is not so

Re: Avoiding Tablespace path collision for primary and standby

On Wed, Jun 20, 2018 at 10:50 AM Andres Freund wrote: > > > On June 20, 2018 10:31:05 AM PDT, Ashwin Agrawal > wrote: > >On Wed, Jun 20, 2018 at 9:39 AM Bruce Momjian wrote: > > > >> On Fri, May 25, 2018 at 02:17:23PM -0700, Ashwin Agrawal wrote: > >>

Re: Avoiding Tablespace path collision for primary and standby

On Wed, Jun 20, 2018 at 9:39 AM Bruce Momjian wrote: > On Fri, May 25, 2018 at 02:17:23PM -0700, Ashwin Agrawal wrote: > > > > On Fri, May 25, 2018 at 7:33 AM, Tom Lane wrote: > > > > Ashwin Agrawal writes: > > > Proposing to create director

Re: Bulk Insert into PostgreSQL

On Sat, Jun 30, 2018 at 6:27 AM Craig Ringer wrote: > > You can also gain a bit by running with wal_level = minimal. On newer > version you can use UNLOGGED tables then convert them to logged, but that > won't be an option for 9.4. > Curious to know more on this does with standby also its

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

On Fri, Jul 20, 2018 at 1:54 AM Heikki Linnakangas wrote: > On 19/07/18 23:13, Andres Freund wrote: > > On 2018-07-19 14:54:52 -0500, Nico Williams wrote: > >> On Thu, Jul 19, 2018 at 12:20:53PM -0700, Andres Freund wrote: > >>> On 2018-07-19 11:57:25 +0300, Heikki Linnakangas wrote: > Ugh.

make_etags: avoid recursive symbolic creation.

src/tools/make_etags creates link to TAGS file from each and every directory in source tree. It works fine without all those links in each and every directory in source code so remove the extra work and annoyance. diff --git a/src/tools/make_etags b/src/tools/make_etags index

Re: Function to promote standby servers

Just curious to know, is promotion via function only allowed for hot-standby or works for any standby? On Mon, Oct 15, 2018 at 7:16 AM Laurenz Albe wrote: > Michael Paquier wrote: > > > diff --git a/src/backend/access/transam/xlog.c > b/src/backend/access/transam/xlog.c > > > index

Re: Make drop database safer

On Mon, Feb 11, 2019 at 3:55 PM Ashwin Agrawal wrote: > > Thanks for the response and inputs. > > On Sat, Feb 9, 2019 at 4:51 AM Andres Freund wrote: > >> Hi, >> >> On 2019-02-08 16:36:13 -0800, Alexandra Wang wrote: >> > Current sequence of operations

Re: Make drop database safer

On Wed, Mar 6, 2019 at 7:56 AM Tomas Vondra wrote: > > > On 2/12/19 12:55 AM, Ashwin Agrawal wrote: > > > > Thanks for the response and inputs. > > > > On Sat, Feb 9, 2019 at 4:51 AM Andres Freund > <mailto:and...@anarazel.de>> wrote: > > &

Re: Pluggable Storage - Andres's take

Hi, While playing with the tableam, usage of which starts with commit v12-0023-tableam-Introduce-and-use-begin-endscan-and-do-i.patch, should we check for NULL function pointer before actually calling the same and ERROR out instead as NOT_SUPPORTED or something on those lines. Understand its

Re: Make drop database safer

Thanks for the response and inputs. On Sat, Feb 9, 2019 at 4:51 AM Andres Freund wrote: > Hi, > > On 2019-02-08 16:36:13 -0800, Alexandra Wang wrote: > > Current sequence of operations for drop database (dropdb()) > > 1. Start Transaction > > 2. Make catalog changes > > 3. Drop database buffers

Re: Zedstore - compressed in-core columnar storage

> On Apr 10, 2019, at 9:08 PM, Mark Kirkwood > wrote: > > >> On 11/04/19 4:01 PM, Mark Kirkwood wrote: >>> On 9/04/19 12:27 PM, Ashwin Agrawal wrote: >>> >>> Heikki and I have been hacking recently for few weeks to implement >>> in-c

Re: Zedstore - compressed in-core columnar storage

On Tue, Apr 9, 2019 at 11:29 AM Robert Haas wrote: > On Tue, Apr 9, 2019 at 11:51 AM Alvaro Herrera > wrote: > > This is not surprising, considering that columnar store is precisely the > > reason for starting the work on table AMs. > > > > We should certainly look into integrating some sort of

Re: Zedstore - compressed in-core columnar storage

On Tue, Apr 9, 2019 at 9:13 AM Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > > > On 09.04.2019 18:51, Alvaro Herrera wrote: > > On 2019-Apr-09, Konstantin Knizhnik wrote: > > > >> On 09.04.2019 3:27, Ashwin Agrawal wrote: > >>> Heikk

Re: Zedstore - compressed in-core columnar storage

On Mon, Apr 8, 2019 at 6:04 PM Andres Freund wrote: > Hi, > > On 2019-04-08 17:27:05 -0700, Ashwin Agrawal wrote: > > Heikki and I have been hacking recently for few weeks to implement > > in-core columnar storage for PostgreSQL. Here's the design and initial > >

Re: Zedstore - compressed in-core columnar storage

On Mon, Apr 15, 2019 at 11:18 AM Tomas Vondra wrote: > > Maybe. I'm not going to pretend I fully understand the internals. Does > that mean the container contains ZSUncompressedBtreeItem as elements? Or > just the plain Datum values? > First, your reading of code and all the comments/questions

Re: Zedstore - compressed in-core columnar storage

On Mon, Apr 15, 2019 at 10:33 AM Tomas Vondra wrote: > On Mon, Apr 15, 2019 at 09:29:37AM -0700, Ashwin Agrawal wrote: > > On Sun, Apr 14, 2019 at 9:40 AM Tomas Vondra > >wrote: > > > > On Thu, Apr 11, 2019 at 06:20:47PM +0300, Heikki Linnakangas wrote: &

Re: Zedstore - compressed in-core columnar storage

On Sun, Apr 14, 2019 at 9:40 AM Tomas Vondra wrote: > On Thu, Apr 11, 2019 at 06:20:47PM +0300, Heikki Linnakangas wrote: > >On 11/04/2019 17:54, Tom Lane wrote: > >>Ashwin Agrawal writes: > >>>Thank you for trying it out. Yes, noticed for certain patterns >

Re: Zedstore - compressed in-core columnar storage

On Sat, Apr 13, 2019 at 4:22 PM Peter Geoghegan wrote: > On Thu, Apr 11, 2019 at 6:06 AM Rafia Sabih > wrote: > > Reading about it reminds me of this work -- TAG column storage( >

Re: finding changed blocks using WAL scanning

On Thu, Apr 11, 2019 at 6:27 AM Robert Haas wrote: > On Thu, Apr 11, 2019 at 3:52 AM Peter Eisentraut > wrote: > > I had in mind that you could have different overlapping incremental > > backup jobs in existence at the same time. Maybe a daily one to a > > nearby disk and a weekly one to a

Re: finding changed blocks using WAL scanning

On Wed, Apr 10, 2019 at 2:50 PM Robert Haas wrote: > Over at > https://www.postgresql.org/message-id/CA%2BTgmobFVe4J4AA7z9OMUzKnm09Tt%2BsybhxeL_Ddst3q3wqpzQ%40mail.gmail.com > I mentioned parsing the WAL to extract block references so that > incremental backup could efficiently determine which

Re: Using the return value of strlcpy() and strlcat()

On Wed, Mar 13, 2019 at 9:51 AM Tom Lane wrote: > ilm...@ilmari.org (Dagfinn Ilmari =?utf-8?Q?Manns=C3=A5ker?=) writes: > > [ let's convert > > + strlcpy(buf + buflen, name, NAMEDATALEN); > > + buflen += strlen(buf + buflen); > > to > > + buflen += strlcpy(buf

Re: Adding a test for speculative insert abort case

inserting the tuple in unique index but before completing the speculative insert. Hence, helps to create the condition easily. I believe order of index insertion is helping here that unique index is inserted and then non-unique index is inserted too. Attaching patch with the test using the

Re: Pluggable Storage - Andres's take

On Wed, May 15, 2019 at 11:54 AM Andres Freund wrote: > Hi, > > On 2019-04-25 15:43:15 -0700, Andres Freund wrote: > > > > 3) nodeTidscan, skipping over too large tids > > >I think this should just be moved into the AMs, there's no need to > > >have this in nodeTidscan.c > > > > I think

Create TOAST table only if AM needs

. From fcbf6b8187ef8389cf38cd185b709bcb6ef1152a Mon Sep 17 00:00:00 2001 From: Ashwin Agrawal Date: Fri, 10 May 2019 17:42:06 -0700 Subject: [PATCH 1/1] Create toast table only if AM needs it. All AM may not need toast table, hence add property to TableAmRoutine to define if uses toast table

Re: Create TOAST table only if AM needs

On Fri, May 17, 2019 at 11:34 AM Andres Freund wrote: > Hi, > > On 2019-05-17 11:26:29 -0700, Ashwin Agrawal wrote: > > Currently TOAST table is always created (if needed based on data type > > properties) independent of table AM. How toasting is handled seems > >

Re: Create TOAST table only if AM needs

On Fri, May 17, 2019 at 1:51 PM Andres Freund wrote: > > > /* > > > * Planner related callbacks for the heap AM > > @@ -2558,6 +2615,8 @@ static const TableAmRoutine heapam_methods = { > > > >

Re: Create TOAST table only if AM needs

On Fri, May 17, 2019 at 2:42 PM Robert Haas wrote: > In any case, it bears saying that > tableam is a remarkable accomplishment regardless of whatever > shortcomings it has in this area or elsewhere. > Big +1 to this.

Re: Pluggable Storage - Andres's take

On Fri, May 17, 2019 at 12:54 PM Andres Freund wrote: > Hi, > > On 2019-05-15 23:00:38 -0700, Ashwin Agrawal wrote: > > Highlevel this looks good to me. Will look into full details tomorrow. > > Ping? > > I'll push the first of the patches soon, and unless you'll c

Re: Zedstore - compressed in-core columnar storage

On Thu, May 23, 2019 at 7:30 PM Ajin Cherian wrote: > Hi Ashwin, > > - how to pass the "column projection list" to table AM? (as stated in > initial email, currently we have modified table am API to pass the > projection to AM) > > We were working on a similar columnar storage using

Confusing error message for REINDEX TABLE CONCURRENTLY

current case, make it consistent, less confusing and leave it there. Attaching the patch to just do that. Thoughts? From dc8119abe180cc14b0720c4de79495de4c62bf12 Mon Sep 17 00:00:00 2001 From: Ashwin Agrawal Date: Fri, 24 May 2019 16:34:48 -0700 Subject: [PATCH v1] Avoid confusing error message for R

Re: Confusing error message for REINDEX TABLE CONCURRENTLY

On Sun, May 26, 2019 at 6:43 PM Michael Paquier wrote: > As you mention for reindex_relation() no indexes <=> nothing to do, > still let's not rely on that. Instead of making the error message > specific to concurrent operations, I would suggest to change it to > "table foo has no indexes to

Re: Confusing error message for REINDEX TABLE CONCURRENTLY

On Mon, Jun 3, 2019 at 6:27 PM Michael Paquier wrote: > On Mon, Jun 03, 2019 at 04:53:48PM -0700, Ashwin Agrawal wrote: > > Please check if the attached patch addresses and satisfies all the points > > discussed so far in this thread. > > It looks to be so, please see be

Re: Comment typo in tableam.h

There were few more minor typos I had collected for table am, passing them along here. Some of the required callback functions are missing Assert checking (minor thing), adding them in separate patch. From f32bdf5d0d3af5fd6ee6bf6430905f9c4bf5fefa Mon Sep 17 00:00:00 2001 From: Ashwin Agrawal

Re: Confusing error message for REINDEX TABLE CONCURRENTLY

On Tue, May 28, 2019 at 12:05 PM David Rowley wrote: > On Tue, 28 May 2019 at 01:23, Ashwin Agrawal wrote: > > I think we will need to separate out the NOTICE message for concurrent > and regular case. > > > > For example this doesn't sound correct > > WARN

Re: Comment typo in tableam.h

On Mon, Jun 3, 2019 at 6:24 PM Andres Freund wrote: > Hi, > > On 2019-06-03 18:21:56 -0700, Ashwin Agrawal wrote: > > On Mon, Jun 3, 2019 at 5:26 PM Andres Freund wrote: > > > > > Hi, > > > > > > Thanks for these! > > >

Re: Comment typo in tableam.h

On Mon, Jun 3, 2019 at 5:26 PM Andres Freund wrote: > Hi, > > Thanks for these! > > On 2019-06-03 17:24:15 -0700, Ashwin Agrawal wrote: > > /* > >* Estimate the size of shared memory needed for a parallel scan > of this > > - *

Re: Removing a few more lseek() calls

On Sat, Mar 30, 2019 at 2:14 AM Thomas Munro wrote: > Hello, > > Patch 0001 gets rid of the unconditional lseek() calls for SLRU I/O, > as a small follow-up to commit c24dcd0c. Patch 0002 gets rid of a few > places that usually do a good job of avoiding lseek() calls while > reading and writing

Re: Extracting only the columns needed for a query

On Sat, Jun 15, 2019 at 10:02 AM Tom Lane wrote: > > Approach B: after parsing and/or after planning > > If we wanted to do something about this, making the planner record > the set of used columns seems like the thing to do. We could avoid > the expense of doing it when it's not needed by

Re: heapam_index_build_range_scan's anyvisible

On Mon, Jun 10, 2019 at 2:56 PM Andres Freund wrote: > > Currently, all AM needs to build HeapTuple in > > index_build_range_scan function. I looked into all the callback functions > > and only htup->t_self is used from heaptuple in all the functions > (unless I > > missed something). So, if

Re: heapam_index_build_range_scan's anyvisible

On Fri, Jun 7, 2019 at 1:19 PM Robert Haas wrote: > I spent some time today studying heapam_index_build_range_scan and > quickly reached the conclusion that it's kind of a mess. > While at it might be helpful and better to also decouple HeapTuple dependency for IndexBuildCallback. Currently,

Re: Comment typo in tableam.h

On Mon, Jun 3, 2019 at 5:24 PM Ashwin Agrawal wrote: > There were few more minor typos I had collected for table am, passing them > along here. > > Some of the required callback functions are missing Assert checking (minor > thing), adding them in separate patch. > Curio

Re: errbacktrace

On Tue, Jun 25, 2019 at 4:08 AM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > New thread continuing from > < > https://www.postgresql.org/message-id/d4903af2-e7b7-b551-71f8-3e4a6bdc2...@2ndquadrant.com > >. > > Here is a extended version of Álvaro's patch that adds an

Remove HeapTuple and Buffer dependency for predicate locking functions

/CAEepm%3D2QbqQ_%2BKQQCnhKukF6NEAeq4SqiO3Qxe%2BfHza5-H-jKA%40mail.gmail.com From aac57c17f078f869bf360677556842d58d5d33ea Mon Sep 17 00:00:00 2001 From: Ashwin Agrawal Date: Fri, 21 Jun 2019 13:42:00 -0700 Subject: [PATCH v1] Remove HeapTuple dependency for predicate locking functions. Following

Re: Inconsistency between table am callback and table function names

On Fri, May 10, 2019 at 10:51 AM Andres Freund wrote: > > Hi, > > On 2019-05-10 10:43:44 -0700, Ashwin Agrawal wrote: > > On Thu, May 9, 2019 at 8:52 AM Andres Freund wrote: > > > The changes necessary for tableam were already huge. Changing naming > > > sch

Re: Inconsistency between table am callback and table function names

On Thu, May 9, 2019 at 8:52 AM Andres Freund wrote: > The changes necessary for tableam were already huge. Changing naming > schemes for functions that are used all over the backend (e.g. ~80 calls > to table_beginscan), and where there's other wrapper functions that also > widely used (237 calls

Re: Pluggable Storage - Andres's take

On Wed, May 8, 2019 at 2:46 PM Andres Freund wrote: > > Hi, > > On 2019-05-07 23:18:39 -0700, Ashwin Agrawal wrote: > > On Mon, May 6, 2019 at 1:39 PM Ashwin Agrawal wrote: > > > Also wish to point out, while working on Zedstore, we realized that > > >

Re: Inconsistency between table am callback and table function names

On Mon, May 13, 2019 at 12:51 PM Robert Haas wrote: > On Fri, May 10, 2019 at 3:43 PM Ashwin Agrawal > wrote: > > Meant to stick the question mark in that email, somehow missed. Yes > > not planning to spend any time on it if objections. Here is the list > > of r

Re: Inconsistency between table am callback and table function names

On Thu, May 23, 2019 at 4:32 PM Andres Freund wrote: > Hi, > > On 2019-05-14 12:11:46 -0700, Ashwin Agrawal wrote: > > Thank you. Please find the patch to rename the agreed functions. It would > > be good to make all consistent instead of applying exception to three &g

Re: Pluggable Storage - Andres's take

On Tue, Apr 9, 2019 at 6:17 AM Heikki Linnakangas wrote: > On 08/04/2019 20:37, Andres Freund wrote: > > On 2019-04-08 15:34:46 +0300, Heikki Linnakangas wrote: > >> There's a little bug in index-only scan executor node, where it mixes > up the > >> slots to hold a tuple from the index, and from

Re: Pluggable Storage - Andres's take

On Wed, May 15, 2019 at 11:54 AM Andres Freund wrote: > Attached is a prototype of a variation of this. I added a > table_tuple_tid_valid(TableScanDesc sscan, ItemPointer tid) > callback / wrapper. Currently it just takes a "plain" scan, but we could > add a separate table_beginscan variant too.

Re: Inconsistency between table am callback and table function names

On Wed, May 8, 2019 at 2:51 PM Andres Freund wrote: > > Hi, > > On 2019-05-08 00:32:22 -0700, Ashwin Agrawal wrote: > > The general theme for table function names seem to be > > "table_". For example table_scan_getnextslot() and its > > corresponding callba

Re: Inconsistency between table am callback and table function names

On Wed, May 8, 2019 at 5:05 PM Ashwin Agrawal wrote: > Not having consistency is the main aspect I wish to bring to > attention. Like for some callback functions the comment is > > /* see table_insert() for reference about parameters */ > void(*tuple_insert

Match table_complete_speculative() code to comment

Comment for table_complete_speculative() says /* * Complete "speculative insertion" started in the same transaction. If * succeeded is true, the tuple is fully inserted, if false, it's removed. */ static inline void table_complete_speculative(Relation rel, TupleTableSlot *slot,

Re: What is an item pointer, anyway?

On Fri, Apr 26, 2019 at 2:19 PM Peter Geoghegan wrote: > itemid.h introduces the struct ItemIdData as follows: > > /* > * An item pointer (also called line pointer) on a buffer page > > Meanwhile, itemptr.h introduces the struct ItemPointerData as follows: > > /* > * ItemPointer: > * > *

Re: Race conditions with checkpointer and shutdown

On Sat, Apr 27, 2019 at 5:57 PM Tom Lane wrote: > > I have spent a fair amount of time trying to replicate these failures > locally, with little success. I now think that the most promising theory > is Munro's idea in [1] that the walreceiver is hanging up during its > unsafe attempt to do

Re: Race conditions with checkpointer and shutdown

On Mon, Apr 29, 2019 at 10:36 AM Tom Lane wrote: > > Can you try applying a1a789eb5ac894b4ca4b7742f2dc2d9602116e46 > to see if it fixes the problem for you? Yes, will give it a try on greenplum and report back the result. Have we decided if this will be applied to back branches?

Re: Pluggable Storage - Andres's take

On Thu, Apr 25, 2019 at 3:43 PM Andres Freund wrote: > Hm. I think some of those changes would be a bit bigger than I initially > though. Attached is a more minimal fix that'd route > RelationGetNumberOfBlocksForFork() through tableam if necessary. I > think it's definitely the right answer for

Re: Calling PrepareTempTablespaces in BufFileCreateTemp

On Thu, Apr 25, 2019 at 11:53 PM Michael Paquier wrote: > > On Thu, Apr 25, 2019 at 12:45:03PM -0400, Tom Lane wrote: > > I still remain concerned that invoking catalog lookups from fd.c is a darn > > bad idea, even if we have a fallback for it to work (for some value of > > "work") in

Re: Pluggable Storage - Andres's take

On Mon, May 6, 2019 at 7:14 AM Andres Freund wrote: > > Hi, > > On May 6, 2019 3:40:55 AM PDT, Rafia Sabih wrote: > >I was trying the toyam patch and on make check it failed with > >segmentation fault at > > > >static void > >toyam_relation_set_new_filenode(Relation rel, > > char persistence, >

Re: Pluggable Storage - Andres's take

On Mon, May 6, 2019 at 1:39 PM Ashwin Agrawal wrote: > > Also wish to point out, while working on Zedstore, we realized that > TupleDesc from Relation object can be trusted at AM layer for > scan_begin() API. As for ALTER TABLE rewrite case (ATRewriteTables()), > catalog i

Inconsistency between table am callback and table function names

The general theme for table function names seem to be "table_". For example table_scan_getnextslot() and its corresponding callback scan_getnextslot(). Most of the table functions and callbacks follow mentioned convention except following ones table_beginscan table_endscan table_rescan

Re: An out-of-date comment in nodeIndexonlyscan.c

On Thu, Jun 27, 2019 at 4:33 PM Thomas Munro wrote: > Here's a patch I'd like to commit to fix the comment. We could look > at improving the actual code after > https://commitfest.postgresql.org/23/2169/ is done. > Change looks good to me. > I wonder if it might be possible to avoid page

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

On Wed, Jul 10, 2019 at 6:46 AM Tom Lane wrote: > Michael Paquier writes: > > On Wed, Jul 10, 2019 at 12:51:28PM +0530, Amit Kapila wrote: > >> It would be good if we can come up with something like that. It will > >> be helpful for zheap, where in some cases we get different row > >> ordering

Re: heapam_index_build_range_scan's anyvisible

On Mon, Jun 10, 2019 at 5:38 PM Ashwin Agrawal wrote: > > On Mon, Jun 10, 2019 at 2:56 PM Andres Freund wrote: > >> > Currently, all AM needs to build HeapTuple in >> > index_build_range_scan function. I looked into all the callback >> functions >> > an

Re: Comment typo in tableam.h

On Sat, Jul 6, 2019 at 12:05 AM Amit Kapila wrote: > On Tue, Jul 2, 2019 at 1:00 AM Ashwin Agrawal wrote: > > Please find attached v2 of patch 1 without objectionable comment change. > v1 of patch 2 attaching here just for convenience, no modifications made to > it. > >

Re: Declared but no defined functions

On Sat, Jul 6, 2019 at 4:32 PM Masahiko Sawada wrote: > Indeed. I've tried to search again with the following script and got > more such functions. > > for func in `git ls-files | egrep "\w+\.h$" | xargs cat | egrep -v > "(^typedef)|(DECLARE)|(BKI)" | egrep "^(extern )*[\_0-9A-Za-z]+ >

Re: Regression test PANICs with master-standby setup on same machine

On Wed, Apr 24, 2019 at 9:25 AM Andres Freund wrote: > The inability > to reasonably test master/standby setups on a single machine is pretty > jarring (yes, one can use basebackup tablespace maps - but that doesn't > work well for new tablespaces). +1 agree. Feature which can't be easily

Re: Zedstore - compressed in-core columnar storage

On Tue, Apr 16, 2019 at 9:15 AM Tomas Vondra wrote: > > I'm not sure it's that clear cut, actually. Sure, it's not the usual > (block,item) pair so it's not possible to jump to the exact location, so > it's not the raw physical identifier as regular TID. But the data are > organized in a btree,

Re: block-level incremental backup

On Wed, Apr 10, 2019 at 9:21 AM Robert Haas wrote: > I have a related idea, though. Suppose that, as Peter says upthread, > you have a replication slot that prevents old WAL from being removed. > You also have a background worker that is connected to that slot. It > decodes WAL and produces

Re: Calling PrepareTempTablespaces in BufFileCreateTemp

On Wed, Apr 24, 2019 at 5:48 PM Peter Geoghegan wrote: > On Wed, Apr 24, 2019 at 12:17 PM Tom Lane wrote: > > After a bit more thought it seemed like another answer would be to > > make all three of those functions assert that the caller did the > > right thing, as per attached. This addresses

Re: Calling PrepareTempTablespaces in BufFileCreateTemp

On Thu, Apr 25, 2019 at 9:45 AM Tom Lane wrote: > However, by that argument we should change all 3 of these functions to > set up the data. If we're eating the layering violation to the extent > of letting OpenTemporaryFile call into commands/tablespace, then there's > little reason for the

Re: Zedstore - compressed in-core columnar storage

On Mon, Apr 15, 2019 at 12:50 PM Peter Geoghegan wrote: > On Mon, Apr 15, 2019 at 9:16 AM Ashwin Agrawal > wrote: > > Would like to know more specifics on this Peter. We may be having > different context on hybrid row/column design. > > I'm confused about how clo

Re: Zedstore - compressed in-core columnar storage

On Thu, May 30, 2019 at 8:07 AM DEV_OPS wrote: > > it's really cool and very good progress, > > I'm interesting if SIDM/JIT will be supported > That's something outside of Zedstore work directly at least now. The intent is to work with current executor code or enhance it only wherever needed.

Re: Comment typo in tableam.h

On Fri, Jun 28, 2019 at 1:47 PM Amit Kapila wrote: > On Mon, Jun 24, 2019 at 11:26 PM Ashwin Agrawal > wrote: > > > > On Mon, Jun 3, 2019 at 5:24 PM Ashwin Agrawal > wrote: > >> > >> There were few more minor typos I had collected for table am, pa

Re: Zedstore - compressed in-core columnar storage

On Sun, Jun 30, 2019 at 7:59 PM Tsunakawa, Takayuki < tsunakawa.ta...@jp.fujitsu.com> wrote: > From: Ashwin Agrawal [mailto:aagra...@pivotal.io] > > The objective is to gather feedback on design and approach to the same. > > The implementation has core basic pieces

Re: C testing for Postgres

On Mon, Jul 1, 2019 at 11:26 PM Michael Paquier wrote: > On Fri, Jun 28, 2019 at 09:42:54AM -0400, Adam Berlin wrote: > > If we were to use this tool, would the community want to vendor the > > framework in the Postgres repository, or keep it in a separate repository > > that produces a

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

ed or not. Essentially HeapCheckForSerializableConflictOut() will become callback function instead. Due to void* callback argument aspect I didn't like that solution and felt AM performing checks and calling CheckForSerializableConflictOut() seems more straight forward. From 2516c592db97f285925667aafb34fc2de4286

Re: Zedstore - compressed in-core columnar storage

On Wed, Aug 14, 2019 at 2:51 AM Ashutosh Sharma wrote: > Hi Ashwin, > > I tried playing around with the zedstore code a bit today and there > are couple questions that came into my mind. > Great! Thank You. > > 1) Can zedstore tables be vacuumed? If yes, does VACUUM on zedstore > table set

Re: Zedstore - compressed in-core columnar storage

On Mon, Aug 26, 2019 at 5:36 AM Ashutosh Sharma wrote: > Thanks Ashwin and Heikki for your responses. I've one more query here, > > If BTree index is created on a zedstore table, the t_tid field of > Index tuple contains the physical tid that is not actually pointing to > the data block instead

Re: Comment in ginpostinglist.c doesn't match code

On Thu, Aug 22, 2019 at 1:14 AM Heikki Linnakangas wrote: > > The patch also includes a little unit test module to test this without > creating a 16 TB table. A whole new test module seems a bit like > overkill just for this, but clearly we were missing test coverage here. > And it will come

Re: heapam_index_build_range_scan's anyvisible

On Tue, Jul 16, 2019 at 10:22 AM Andres Freund wrote: > Hi, > > On 2019-07-11 17:27:46 -0700, Ashwin Agrawal wrote: > > Please find attached the patch to remove IndexBuildCallback's dependency > on > > HeapTuple, as discussed. Changed to have the argument as ItemPointer

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

On Fri, Aug 2, 2019 at 4:56 PM Ashwin Agrawal wrote: > > On Wed, Jul 31, 2019 at 2:06 PM Andres Freund wrote: > >> Looking at the code as of master, we currently have: >> > > Super awesome feedback and insights, thank you! > > - PredicateLockTuple() ca

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

On Tue, Jul 30, 2019 at 2:58 PM Thomas Munro wrote: > On Tue, Jun 25, 2019 at 6:02 AM Andres Freund wrote: > > > - CheckForSerializableConflictOut() no more takes HeapTuple nor > > > buffer, instead just takes xid. Push heap specific parts from > > > CheckForSerializableConflictOut() into

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

On Wed, Jul 31, 2019 at 12:37 PM Ashwin Agrawal wrote: > > On Wed, Jul 31, 2019 at 10:55 AM Andres Freund wrote: > >> Hi, >> >> On 2019-07-31 10:42:50 -0700, Ashwin Agrawal wrote: >> > On Tue, Jul 30, 2019 at 2:58 PM Thomas Munro >> wrote: >>

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

On Wed, Jul 31, 2019 at 10:55 AM Andres Freund wrote: > Hi, > > On 2019-07-31 10:42:50 -0700, Ashwin Agrawal wrote: > > On Tue, Jul 30, 2019 at 2:58 PM Thomas Munro > wrote: > > > > > On Tue, Jun 25, 2019 at 6:02 AM Andres Freund > wrote: > > > &g

Re: heapam_index_build_range_scan's anyvisible

s being treated as errors > : recipe for target 'verify_nbtree.o' failed > make[4]: *** [verify_nbtree.o] Error 1 > Thanks for reporting, I did indeed missed out contrib. Please find attached the v2 of the patch which includes the change required in contrib as well. From 873711e0601d503

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

On Thu, Nov 7, 2019 at 8:44 PM Thomas Munro wrote: > On Thu, Aug 8, 2019 at 6:53 AM Ashwin Agrawal wrote: > >>> - I wonder if CheckForSerializableConflictOutNeeded() shouldn't have a > >>> portion of it's code as a static inline. In particular, it's a shame > &

Re: Start Walreceiver completely before shut down it on standby server.

On Tue, Dec 10, 2019 at 3:06 AM jiankang liu wrote: > Start Walreceiver completely before shut down it on standby server. > > The walreceiver will be shut down, when read an invalid record in the > WAL streaming from master.And then, we retry from archive/pg_wal again. > > After that, we start

Missing error_context_stack = NULL in AutoVacWorkerMain()

I am not sure if this causes any potential problems or not, but for consistency of code seems we are missing below. All other places in code where sigsetjmp() exists for top level handling has error_context_stack set to NULL. diff --git a/src/backend/postmaster/autovacuum.c

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

On Sun, Nov 10, 2019 at 8:21 PM Thomas Munro wrote: > I pushed the first two, Thank You! but on another read-through of the main patch > I didn't like the comments for CheckForSerializableConflictOut() or > the fact that it checks SerializationNeededForRead() again, after I > thought a bit

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

On Fri, Oct 4, 2019 at 8:49 AM Tom Lane wrote: > Jacob Champion writes: > > On Fri, Oct 4, 2019 at 7:51 AM Tom Lane wrote: > >> I concur with Joe here. The reason why some of the existing > >> memset's use "false" is for symmetry with other places where we use > >> "memset(p, true, n)" to set

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

On Mon, Jan 27, 2020 at 4:47 PM Thomas Munro wrote: > OK, I kept only the small comment change from that little fixup patch, > and pushed this. > > > I had proposed as alternative way in initial email and also later, > > didn't receive comment on that, so re-posting. > > > typedef bool

Re: Extracting only the columns needed for a query

On Wed, Jan 15, 2020 at 7:54 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > For UPDATE, we need all of the columns in the table because of the > > table_lock() API's current expectation that the slot has all of the > > columns populated. If we want UPDATE to only need to insert the column >

Re: Use LN_S instead of "ln -s" in Makefile

On Fri, Feb 14, 2020 at 4:57 PM Tom Lane wrote: > Ashwin Agrawal writes: > > In general, the variable LN_S is 'ln -s', however, LN_S changes to 'cp > > -pR' if configure finds the file system does not support symbolic > > links. > > I was playing with 'ln' for some ot

Use LN_S instead of "ln -s" in Makefile

for 'ln -s' revealed three places it's used. Attaching the patch to fix the same. From 60b3374de1e4d0fd143c2a7b3bf78893e32579af Mon Sep 17 00:00:00 2001 From: Ashwin Agrawal Date: Fri, 14 Feb 2020 16:16:37 -0800 Subject: [PATCH v1] Use LN_S instead of "ln -s" in Makefile --- src/backen

  1   2   >