Re: parallel distinct union and aggregate support patch

2020-10-27 Thread Robert Haas
e hash aggregate can be put into the > respective batch. I am not sure if this would be a win if the typical group size is small and the transition state has to be serialized/deserialized. Possibly we need multiple strategies, but I guess we'd have to test performance to be sure. -- Robert Haas E

Re: "unix_socket_directories" should be GUC_LIST_INPUT?

2020-10-27 Thread Robert Haas
respect to GUC_LIST_QUOTE seems > > proper.) > > Right, I was definitely assuming that that would happen. If we change this, is it going to be a compatibility break for the contents of postgresql.conf files? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Disable WAL logging to speed up data loading

2020-11-02 Thread Robert Haas
RC - use LSNs to figure out whether a block has changed. For temporary and unlogged tables, we use "fake" LSNs that are generated using a counter, but that approach only works because such relations are never really WAL-logged. Mixing fake LSNs and real LSNs will break stuff, and not bump

Re: public schema default ACL

2020-11-02 Thread Robert Haas
g number. On the other hand, the only alternative is to continue shipping a configuration that, by default, is potentially insecure. It's hard to decide which thing we should care more about. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Online checksums verification in the backend

2020-11-02 Thread Robert Haas
check rather than core, though there could usefully be enabling functions in core. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: public schema default ACL

2020-11-03 Thread Robert Haas
s in > the public schema to make things dangerous for some other user poking > around at the tables or views that you are allowed to create there. I agree. Everything that can execute code is a risk, which also includes things like triggers and RLS policies. Noah's certainl

Re: [PATCH] remove pg_archivecleanup and pg_standby

2020-11-03 Thread Robert Haas
I certainly saw hand-assembled shell code fail with "too > many files" and things when it tried to act on large amount of WAL. Yeah, I see pg_archivecleanup used in customer environments all the time. Like just this morning, for example. -- Robert Haas EnterpriseDB: http://www.enterp

Re: Online checksums verification in the backend

2020-11-03 Thread Robert Haas
on disk, > > and then still check it. > > Don't think we need a share lock. That still allows the buffer to be > written out (and thus a torn read). What we need is to set > BM_IO_IN_PROGRESS on the buffer in question - only one backend can set > that. And then unset tha

Re: [HACKERS] xlog min recovery request ... is past current point ...

2020-11-12 Thread Robert Haas
to whether or not to replay VM actions fully conditional on the page LSN. The visibility map is so small that the additional volume of full-page writes seems unlikely to matter very much. There's an additional downside in that registering an additional buffer makes the WAL records bigger, an

Re: pg_proc.dat "proargmodes is not a 1-D char array"

2020-11-17 Thread Robert Haas
On Tue, Nov 17, 2020 at 10:32 AM Tom Lane wrote: > Adding the expected length to the error message might be OK though. Certainly seems like we should do at least that much. The current message is just wrong, right? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterpr

Re: Protect syscache from bloating with negative cache entries

2020-11-17 Thread Robert Haas
et accessed a bunch of times. If there are any meaningful thresholds here I'd expect us to be trying to distinguish things like 1000+ accesses vs. 100-1000 vs. 10-100 vs. 1-10. Or maybe we don't need to distinguish at all and can just have a single mark bit rather than a counter. -- R

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-18 Thread Robert Haas
ive strategy may be chosen anyway. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: new heapcheck contrib module

2020-11-19 Thread Robert Haas
e == NULL) + fatal("no table name on entry to check_index"); Again, probably these should be asserts, or if they're not, the error should be reported better and maybe elsewhere. Similarly in some other places, like expand_schema_name_patterns(). +

Re: new heapcheck contrib module

2020-11-19 Thread Robert Haas
t data on disk doesn't cause crashes, even for regular SELECT statements. I don't think we can take an arbitrary performance hit to get there, which is a challenge, but I do think that even a brief outage is nothing to take lightly. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: new heapcheck contrib module

2020-11-19 Thread Robert Haas
On Thu, Nov 19, 2020 at 12:06 PM Robert Haas wrote: > I originally intended to review the docs and regression tests in the > same email as the patch itself, but this email has gotten rather long > and taken rather longer to get together than I had hoped, so I'm going > to stop

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-20 Thread Robert Haas
, > which can be significant. Yeah, I think dirtying the page fewer times is a big win. However, a page may have tuples that are not yet all-visible, and we can't freeze those just because we are freezing others. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: xid wraparound danger due to INDEX_CLEANUP false

2020-11-20 Thread Robert Haas
between the index AM and the table AM being more than slightly unclear. Maybe we need to clear up the definitional problems first. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-20 Thread Robert Haas
On Fri, Nov 20, 2020 at 11:02 AM Alvaro Herrera wrote: > On 2020-Nov-20, Robert Haas wrote: > > Yeah, I think dirtying the page fewer times is a big win. However, a > > page may have tuples that are not yet all-visible, and we can't freeze > > those just because we a

Re: [Patch] ALTER SYSTEM READ ONLY

2020-11-20 Thread Robert Haas
s get reset in the catch block. Correct me if I am > missing something here. Good catch. I think you're right. Do you want to update accordingly? Andres, do you like the new loop better? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: enable_incremental_sort changes query behavior

2020-11-20 Thread Robert Haas
ins, because there are no joins. What I said was: "target lists for any nodes below the top of the join tree were previously always just Var nodes." The topmost join allowed non-Var nodes before, but not lower levels. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: enable_incremental_sort changes query behavior

2020-11-20 Thread Robert Haas
breaks stuff. Or, on the third hand, maybe I'm wrong about what the rule was in the first place. This is certainly a complicated topic. I don't think I'm wrong, or I wouldn't be bothering to write emails about it, but that doesn't mean I'm not wrong anyway. -- Robert Haas EDB: http://www.enterprisedb.com

Re: xid wraparound danger due to INDEX_CLEANUP false

2020-11-20 Thread Robert Haas
store TIDs anyway. So that's an example of an index AM that might want to do some work to keep performance up, but it's not actually required. This might be orthogonal to the 0.0-1.0 scale you were thinking about, but it might be good to factor it into the thinking somehow. -- Robert Haas EDB: http://www.enterprisedb.com

Re: xid wraparound danger due to INDEX_CLEANUP false

2020-11-20 Thread Robert Haas
ble to VACUUM > any one index" notwithstanding -- that's still necessary, of course, > but we still basically decouple table bloat from index bloat at the > conceptual level.) > > Does that make sense? I *think* so. For me the point is that the index never has a right to i

Re: [HACKERS] Custom compression methods

2020-11-20 Thread Robert Haas
compression setting must endure. At any rate, we need to make sure we have clear and sensible rules for when attcompression (a) must be valid, (b) may be valid, and (c) must be invalid. And those rules need to at least be documented in the comments, and maybe in the SGML docs. I'm out of time for today, so I'll have to look at this more another day. Hope this helps for a start. -- Robert Haas EDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2020-11-24 Thread Robert Haas
ale.c: errmsg("ICU is not supported in this build"), \ src/backend/utils/misc/guc.c: GUC_check_errmsg("Bonjour is not supported by this build"); src/backend/utils/misc/guc.c: GUC_check_errmsg("SSL is not supported by this build"); -- Robert Haas EDB: http://www.enterprisedb.com

Re: libpq compression

2020-11-24 Thread Robert Haas
robably a bridge too far, but at least I think this is a strong argument that the compression method shouldn't have to be the same in both directions. -- Robert Haas EDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2020-11-24 Thread Robert Haas
ct that the patch wasn't doing it that way. Hmm. -- Robert Haas EDB: http://www.enterprisedb.com

Re: libpq compression

2020-11-24 Thread Robert Haas
t certain things can be left for the future, that's fine. If we've have discussed how they could be added without breaking backward compatibility, even better. But we can't just skip over having that discussion. -- Robert Haas EDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2020-11-24 Thread Robert Haas
//postgr.es/m/20171130205155.7mgq2cuqv6zxi25a@alvherre.pgsql -- Robert Haas EDB: http://www.enterprisedb.com

Re: bug in pageinspect's "tuple data" feature

2020-11-24 Thread Robert Haas
ly really like to see us do the same sort of hardening here that is present in the recently-committed amcheck-for-heap support, which is robust against a wide variety of things of this sort rather than just this one particularly. Again, this is not to say that you should be on the hook for that; it's a general statement. -- Robert Haas EDB: http://www.enterprisedb.com

Re: parallel distinct union and aggregate support patch

2020-11-28 Thread Robert Haas
general there can be multiple nodes that cause waiting in the tree: any number of Parallel Redistribute nodes, plus a Gather, plus maybe other stuff. The cheap way out of that problem is to use a buffer that can grow arbitrarily large, but that's not terribly satisfying either. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Proposal: Global Index

2021-01-11 Thread Robert Haas
1024 = 141TB, which is a heck of a lot worse than the 1.125TB I required with the unpartitioned table or the locally partitioned table. That's not necessarily a death sentence for every use case, but it's going to be pretty bad for tables that are big and heavily updated. -- Robert Haas EDB: http://www.enterprisedb.com

Re: CheckpointLock needed in CreateCheckPoint()?

2021-01-14 Thread Robert Haas
INTERRUPTS() in the first place. It has its own machinery: HandleCheckpointerInterrupts(). Possibly that's something we should be looking to refactor somehow as well. -- Robert Haas EDB: http://www.enterprisedb.com

Re: vacuum_cost_page_miss default value and modern hardware

2021-01-14 Thread Robert Haas
don't know if that's the right idea. Still another approach would be to try to reduce the degree to which the cache gets trashed, or make it have less harmful effect on future performance by reading things back in more efficiently. I don't really know. TL;DR: This change is fine with me,

Re: adding wait_start column to pg_locks

2021-01-14 Thread Robert Haas
ng for more than one lock at the same time, because once it starts waiting to acquire the first one, it can't do anything else, and thus can't begin waiting for a second one. So I would have thought that this would be recorded in the PROC. But I haven't looked at the patch so maybe

Re: new heapcheck contrib module

2021-01-14 Thread Robert Haas
at function ends up doing something like TheGenericFunctionToExecuteOrDie(my_die_fn, the_query), or maybe pg_dump can just open-code it but have a my_die_fn to pass down to the glob-expansion stuff, or, well, I don't know. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Key management with tests

2021-01-15 Thread Robert Haas
without which any commit is always premature. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Key management with tests

2021-01-15 Thread Robert Haas
On Fri, Jan 15, 2021 at 4:47 PM Bruce Momjian wrote: > If people want changes, I need to hear about it here. I have address > everything people have mentioned in these threads so far. That does not match my perception of the situation. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Add primary keys to system catalogs

2021-01-17 Thread Robert Haas
nt about labelling some of the unique indexes as primary keys, but I think installing foreign keys that don't really enforce anything may lead to confusion. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Key management with tests

2021-01-17 Thread Robert Haas
ative effect on the perceptions of PostgreSQL's quality. But if the TDE itself is magic that only designated smart people on special calls can understand, then it's going to be far worse, because that'll mean when any kind of TDE problems comes up, nobody else can help debug anything. While I would like to have TDE in PostgreSQL, I would not like to have it on those terms. -- Robert Haas EDB: http://www.enterprisedb.com

Re: [PATCH] ProcessInterrupts_hook

2021-01-18 Thread Robert Haas
to be enough: I'm not sure backends have any business caring about why they are being asked to terminate. But they might well want to know whether a smart shutdown is in progress, and right now there's no way for them to know that. -- Robert Haas EDB: http://www.enterprisedb.com

Re: recovering from "found xmin ... from before relfrozenxid ..."

2021-01-18 Thread Robert Haas
of the bugs that are or have been present in those back branches. I'm not sure that I see the logic in telling people we'll try to prevent them from getting hosed in the future but if they're already hosed they can wait for v14 to fix it. They can't wait that long, and a dump-an

Re: recovering from "found xmin ... from before relfrozenxid ..."

2021-01-18 Thread Robert Haas
r individual users in need. Sure, but I don't see how that's better. > That is Assuming it does not need assorted server changes to go with; I > did not read the thread in detail but I was under the assumption it is a > client program? It's a server extension. It does not re

Re: support for MERGE

2021-01-18 Thread Robert Haas
;s my impression that the previous discussion concluded that their version needed pretty substantial design changes. Is there a plan to work on that? Or was some of that stuff done already? -- Robert Haas EDB: http://www.enterprisedb.com

Re: ResourceOwner refactoring

2021-01-18 Thread Robert Haas
fast, and that's important. The slow path > becomes 30% slower, but that's acceptable. > > - Heikki > > -- Robert Haas EDB: http://www.enterprisedb.com

Re: ResourceOwner refactoring

2021-01-18 Thread Robert Haas
On Mon, Jan 18, 2021 at 11:11 AM Robert Haas wrote: > On Mon, Jan 18, 2021 at 10:19 AM Heikki Linnakangas wrote: > > On 18/01/2021 16:34, Alvaro Herrera wrote: > > > So according to your performance benchmark, we're willing to accept a > > > 30% performance loss

Re: [PATCH] ProcessInterrupts_hook

2021-01-18 Thread Robert Haas
y to the problem, but exposing > the shutdown state more fully seems reasonable. Ah, I hadn't realized. -- Robert Haas EDB: http://www.enterprisedb.com

Re: [Patch] ALTER SYSTEM READ ONLY

2021-01-18 Thread Robert Haas
ateCheckPoint(), and therefore the 0003 patch is > marked WIP. Based on the favorable review comment from Andres upthread and also your feedback, I committed 0001. -- Robert Haas EDB: http://www.enterprisedb.com

Re: CheckpointLock needed in CreateCheckPoint()?

2021-01-18 Thread Robert Haas
On Thu, Jan 14, 2021 at 10:21 AM Robert Haas wrote: > Yeah, I think this lock is useless. In fact, I think it's harmful, > because it makes large sections of the checkpointer code, basically > all of it really, run with interrupts held off for no reason. > > It's not imp

Re: Key management with tests

2021-01-18 Thread Robert Haas
ion of %R is very clear, or adequate for someone to make effective use of it. If I wanted to use %R, how would I ensure that a value is available? * The changes to allfiles.sgml add pg_alterckey.sgml in the wrong place and include an incorrect whitespace change. * It's odd that "pg_alterc

Re: CheckpointLock needed in CreateCheckPoint()?

2021-01-18 Thread Robert Haas
's no longer possible for more than one > process to try to run that code concurrently, so we shouldn't need > the lock anymore. Interesting. I think that must have been a *very* long time ago. Perhaps 076a055acf3c55314de267c62b03191586d79cf6 from 2004? -- Robert Haas EDB: http://www.enterprisedb.com

Re: popcount

2021-01-19 Thread Robert Haas
what things like popcount() and fls() and ffs() and stuff like that are, but it's far from obvious and I often have to refresh my memory. Let's make it easy for someone to figure out, if they don't know already. Like just a comment that says "this returns the number of 1 bits in the integer supplied as an argument" or something can save somebody a lot of trouble. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Printing backtrace of postgres processes

2021-01-19 Thread Robert Haas
e implementation are multiplied. If it carries an 0.01% chance of crashing the system, we'll have people taking production systems down with this all the time. At that point I wouldn't want the feature, even if the gdb approach had the same risk (which I don't think it d

Re: [Patch] ALTER SYSTEM READ ONLY

2021-01-19 Thread Robert Haas
d ReqCheckpointerHandler() with no intention of requesting a checkpoint, because ReqCheckpointerHandler() would be gone. That problem could also be fixed by renaming ReqCheckpointerHandler() to something clearer, but that seems inferior. This is probably not a complete list of problems. Review from others would be appreciated. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Printing backtrace of postgres processes

2021-01-19 Thread Robert Haas
n some scenarios. That would be super-sad. I assume that the people who wrote the code tried to make sure that didn't happen but I don't really know what's reasonable to expect. -- Robert Haas EDB: http://www.enterprisedb.com

strange error reporting

2021-01-20 Thread Robert Haas
se (1) I was not attempting to connect to a database whose name is the empty string and (2) saying that it couldn't connect to the socket is wrong, else it would not also be showing a server message. I haven't investigated why this is happening; apologies if this is a known issue. -- Robert Haas EDB: http://www.enterprisedb.com

Re: strange error reporting

2021-01-20 Thread Robert Haas
On Wed, Jan 20, 2021 at 12:19 PM Tom Lane wrote: > Robert Haas writes: > > I just made the mistake of trying to run pgbench without first running > > createdb and got this: > > > pgbench: error: connection to database "" failed: could not connect to >

Re: strange error reporting

2021-01-20 Thread Robert Haas
quot;/tmp/.s.PGSQL.5432" failed: FATAL: > > database "rhaas" does not exist > > I'd be inclined to spell it "connection to server at ... failed", > but that sort of wording is surely also possible. "connection to server" rather than "connection to database" works for me; in fact, I think I like it slightly better. -- Robert Haas EDB: http://www.enterprisedb.com

Re: strange error reporting

2021-01-20 Thread Robert Haas
thing like that. I don't know whether the right thing is to use something like PQdb() to get the correct database name, or whether we should go with Tom's suggestion and omit that detail altogether, but I think showing the empty string when the user relied on the default is too con

Re: strange error reporting

2021-01-20 Thread Robert Haas
On Wed, Jan 20, 2021 at 1:54 PM Tom Lane wrote: > Alvaro Herrera writes: > > On 2021-Jan-20, Robert Haas wrote: > >> I figured it was something like that. I don't know whether the right > >> thing is to use something like PQdb() to get the correct database > &g

Re: Race condition in recovery?

2021-01-21 Thread Robert Haas
OR if 13 branches off of 12 at an earlier point in the WAL stream than where we are now, then that should be an error that shuts down the standby, because then there is no way for replay to get from where it is now to the correct timeline. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-01-21 Thread Robert Haas
face that is based on waiting will cause difficulty for tool-writers than one that does not. Other people may feel differently, of course... -- Robert Haas EDB: http://www.enterprisedb.com

cleaning up a few CLOG-related things

2021-01-25 Thread Robert Haas
s as a result of copy-and-paste work over the years. I plan to look at tidying that stuff up, too. However, I wanted to post (and probably commit) these patches first, partly to get some feedback, and also because all the cases are a little different and I want to make sure to do a proper analysis of

Re: Is Recovery actually paused?

2021-01-25 Thread Robert Haas
er? To me, when "is" exists > in a function, I expect a boolean output. Others may have better > thoughts. Maybe we should leave the existing function pg_is_wal_replay_paused() alone and add a new one with the name you suggest that returns text. That would create less bur

Re: Printing backtrace of postgres processes

2021-01-25 Thread Robert Haas
s > of things really only start to make sense when you're touching many different > postgres systems "in the wild" - such as commercial support services, helping > people on -general, -bugs or stackoverflow, etc. > > I really appreciate your help with it. Big +1 for

Re: ResourceOwner refactoring

2021-01-25 Thread Robert Haas
nt margin. That's pretty cool; thanks for working on it! -- Robert Haas EDB: http://www.enterprisedb.com

Re: CheckpointLock needed in CreateCheckPoint()?

2021-01-25 Thread Robert Haas
tion some of the relevant commits, and then committed this to master. -- Robert Haas EDB: http://www.enterprisedb.com

Re: [Patch] ALTER SYSTEM READ ONLY

2021-01-25 Thread Robert Haas
ate_gen, next_state_gen); + Assert(success); I am almost positive that this is not OK. I think on some platforms atomics just randomly fail some percentage of the time. You always need a retry loop. Anyway, what happens if two people enter this function at the same time and both read the same starting counter value before either does anything? + /* To be sure that any later reads of memory happen strictly after this. */ + pg_memory_barrier(); You don't need a memory barrier after use of an atomic. The atomic includes a barrier. -- Robert Haas EDB: http://www.enterprisedb.com

Re: {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2021-01-26 Thread Robert Haas
s area seems to have changed a lot since I last looked at it. So, I'm just commenting mostly on the specific question that was asked, and a little bit on the theory of the patch, but without expressing an opinion on the implementation. -- Robert Haas EDB: http://www.enterprisedb.com

Re: making update/delete of inheritance trees scale better

2021-01-26 Thread Robert Haas
way to further optimize accessing the same tuple multiple times in rapid succession to claw back some of the lost performance in the slow cases, but I don't have a specific idea. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Key management with tests

2021-01-26 Thread Robert Haas
rmountable. There's also still some question in my mind about whether the design choices here (one KEK, 2 DEKs, one for data and one for WAL) have enough consensus. I don't have a considered opinion on that, partly because I'm not quite sure what the reasonable alternatives are, but it seems that other people had some questions about it, IIUC. -- Robert Haas EDB: http://www.enterprisedb.com

Re: cleaning up a few CLOG-related things

2021-01-27 Thread Robert Haas
k whether XactCtl->shared->latest_page_number == FullTransactionIdToLatestPageNumber(nextXid) which turned out to be super-helpful in understanding things. -- Robert Haas EDB: http://www.enterprisedb.com v2-0001-In-clog_redo-don-t-set-XactCtl-shared-latest_page.patch Description: Binary data

Re: new heapcheck contrib module

2021-01-28 Thread Robert Haas
ok pretty boring; they are just moving code around. Is there any point in splitting the code from 0003 across two files? Maybe it's fine. If I run pg_amcheck --all -j4 do I get a serialization boundary across databases? Like, I have to completely finish db1 before I can go onto db2, even though maybe only one worker is still busy with it? -- Robert Haas EDB: http://www.enterprisedb.com

Re: vacuum_cost_page_miss default value and modern hardware

2021-01-28 Thread Robert Haas
sometimes get away with it until a large batch job like VACUUM gets involved. -- Robert Haas EDB: http://www.enterprisedb.com

Re: new heapcheck contrib module

2021-01-28 Thread Robert Haas
On Thu, Jan 28, 2021 at 12:40 PM Mark Dilger wrote: > > On Jan 28, 2021, at 9:13 AM, Robert Haas wrote: > > If I run pg_amcheck --all -j4 do I get a serialization boundary across > > databases? Like, I have to completely finish db1 before I can go onto > > db2, even thou

Re: [HACKERS] Custom compression methods

2021-02-01 Thread Robert Haas
ast_internals.h is that it should not be included in very many places at all. If we're adding stuff there that is going to be broadly needed, we're adding it in the wrong place. varlena.c shouldn't need toast_internals.h either, and if it did, it should be in alphabetical order. -- Robert Haas EDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-02-02 Thread Robert Haas
_RAWSIZE_MASK and then having toast_internals.h use those constants as well. Taken with the email I sent yesterday, I think this is a more or less complete review of 0001. Although there are a bunch of things to fix here still, I don't think this is that far from being committable. I don'

Re: Background writer and checkpointer in crash recovery

2021-02-02 Thread Robert Haas
recovery activity to disk." inside StartupXLOG() could really do with some simplification. Right now we have three cases: CreateEndOfRecoveryRecord(), RequestCheckpoint(), and CreateCheckpoint(). Maybe with this change we could get it down to just two, since RequestCheckpoint() already knows what to do about !IsUnderPostmaster. -- Robert Haas EDB: http://www.enterprisedb.com

Re: new heapcheck contrib module

2021-02-03 Thread Robert Haas
ot to get sucked into renaming things. It's a little strange that we end up with mutators to set the slot's handler and handler context when we elsewhere feel free to monkey with a slot's connection directly, but it's not a perfect world and I can't think of anything I'd like better. -- Robert Haas EDB: http://www.enterprisedb.com

Re: making update/delete of inheritance trees scale better

2021-02-04 Thread Robert Haas
and save a bunch of work. zheap, though, is always going to need to take another look at the tuple to do the update, unless you can pass up some values through hidden columns. I'm not exactly sure how expensive that really is, though. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-02-04 Thread Robert Haas
acquire and release is serving the function of a barrier; or perhaps to guard against some other hazard. But saying that it's because reading or writing a 1-byte value might not be atomic does not sound correct. -- Robert Haas EDB: http://www.enterprisedb.com

Re: new heapcheck contrib module

2021-02-04 Thread Robert Haas
ying the definitions of HEAP_TABLE_AM_OID and BTREE_AM_OID into pg_amcheck.h or anywhere else seems bad. I think you just be doing #include "catalog/pg_am_d.h". I think I'm out of steam for today but I'll try to look at this more soon. In general I think this patch and the whole series are pretty close to being ready to commit, even though there are still things I think need fixing here and there. Thanks, -- Robert Haas EDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-02-04 Thread Robert Haas
> that old slot. Or I am missing something. My guess is you want to leave the ScanState alone so that we keep fetching into the same slot as before and have an extra slot on the side someplace. -- Robert Haas EDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-02-05 Thread Robert Haas
esting > purposes*, now that the linux and BSD environments include that. My guess was that would annoy some hackers whose build environments got broken. If everyone thinks otherwise I'm willing to be persuaded, but it's going to take more than 1 vote... -- Robert Haas EDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-02-05 Thread Robert Haas
On Fri, Feb 5, 2021 at 11:07 AM Robert Haas wrote: > Personally, my preference is to just update the test outputs. It's not > important whether many people look closely to verify the differences; > we just need to look them over on a one-time basis to see if they seem > OK.

Re: making update/delete of inheritance trees scale better

2021-02-05 Thread Robert Haas
n the middle. Maybe there's a real reason that won't work, but the only reason I know about why it wouldn't work is that we don't have the bookkeeping to make it work. -- Robert Haas EDB: http://www.enterprisedb.com

Re: making update/delete of inheritance trees scale better

2021-02-05 Thread Robert Haas
an devise a scheme that has less overhead, > but it's not immediately obvious that any huge win would be available. I agree. I'm inclined to suspect that some benefit is possible, but that might be wrong and it sure doesn't look easy. -- Robert Haas EDB: http://www.enterprisedb.com

Re: new heapcheck contrib module

2021-02-05 Thread Robert Haas
enerally we prefer (*foo)(bar) when it can be confused with a direct function call, but wunk->foo(bar) is also considered acceptable. - Changed the return type of ParallelSlotResultHandler to be bool, because having it return PGresult * seemed to offer no advantages. -- Robert Haa

Re: repeated decoding of prepared transactions

2021-02-09 Thread Robert Haas
at the logical replication protocol is supposed to be designed in such a way that once a transaction is successfully confirmed, it won't be sent again. Now if something is not confirmed then it has to be sent again. But if it is confirmed then it shouldn't happen. -- Robert Haas EDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-02-09 Thread Robert Haas
patch. Maybe the possibility of compressing smaller values makes it slightly more important, but I'm not sure that it's worth getting excited about. If anyone feels otherwise on either point, it'd be good to hear about it. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Is txid_status() actually safe? / What is 011_crash_recovery.pl testing?

2021-02-09 Thread Robert Haas
d xid is shown as aborted, and that new xids are assigned after > the xid. > > But as far as I can tell there's no guarantee that that is the case. I think you are right. -- Robert Haas EDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-02-09 Thread Robert Haas
cts in other patches) > while adding HIDE_COMPRESSAM in 0002. Hmm, I guess that makes some sense, too. I'm not sure either one is completely critical, but it does make sense to me now. -- Robert Haas EDB: http://www.enterprisedb.com

Re: repeated decoding of prepared transactions

2021-02-10 Thread Robert Haas
mmit > time. > > OTOH, If by successfully confirmed, you mean that once the subscriber > has applied the complete transaction (including commit), then you are > right that it won't be sent again. I meant - once the subscriber has advanced the flush location. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Extensibility of the PostgreSQL wire protocol

2021-02-10 Thread Robert Haas
out having any code that actually uses them, so not only would the project get no benefit, but it would actually incur a small tax. I wouldn't say that's an absolutely show-stopper, but it definitely isn't my first choice. -- Robert Haas EDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-02-10 Thread Robert Haas
nsert into t1 values (1, repeat('foo', 1000)); insert into t2 values (1, repeat('foo', 1000)); insert into t3 select t1 from t1; insert into t3 select row(a, b)::t1 from t2; rhaas=# select pg_column_compression((t3.x).b) from t3; pg_column_compression --- pglz lz4 (2 rows) That's not good, because now -- Robert Haas EDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-02-10 Thread Robert Haas
On Wed, Feb 10, 2021 at 3:06 PM Robert Haas wrote: > I think you have a fairly big problem with row types. Consider this example: > > create table t1 (a int, b text compression pglz); > create table t2 (a int, b text compression lz4); > create table t3 (x t1); > insert into t1

Re: Extensibility of the PostgreSQL wire protocol

2021-02-11 Thread Robert Haas
ne other than Amazon. Which seems to bring us right back to expecting the community to maintain things for the benefit of third-party forks. -- Robert Haas EDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-02-11 Thread Robert Haas
something parts of which have already been compressed. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Extensibility of the PostgreSQL wire protocol

2021-02-11 Thread Robert Haas
ied, based on what was posted. I'm sorry that my tone seems to have aggravated you, but it wasn't intended to do so. -- Robert Haas EDB: http://www.enterprisedb.com

Re: repeated decoding of prepared transactions

2021-02-11 Thread Robert Haas
se the effects of the prepared transaction might otherwise be included neither in the initial synchronization of the data nor in any subsequently decoded transaction, thus leaving the replica out of sync? -- Robert Haas EDB: http://www.enterprisedb.com

<    1   2   3   4   5   6   7   8   9   10   >