On Tue, May 26, 2015 at 3:07 PM, Tom Lane wrote:
> Robert Haas writes:
> > Realistically, with merge.conflictstyle = diff3 (why is this not the
> > default?), resolving whitespace conflicts that occur when you try to
> > cherry-pick is typically not very difficult.
>
> Really? The problems I ha
mistake.
>
>
If all you want is to avoid the write storms when fsyncs start happening on
slow storage, can you not just adjust the kernel vm.dirty* tunables to
start making the kernel write out dirty buffers much sooner instead of
letting them accumulate until fsyncs force them out all at once?
olutely deprecate it first in that place. Preferrably
> visibly (e.g. with a log message when people use it). That could at least
> get those people who use it to let us know they do, to that way figure out
> if they do - and can de-deprecate it.
>
> Or if someone wants to fix it prope
count of 5. So I'm reluctant to rely on
> that for much of anything.
>
> --
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes
predictable error when it happens.
> E.g. a first step in the regression tests that just verifies what kind
> of line endings are in a file. Could maybe be as simple as checking
> the size of the file?
This leads to making sure you keep your "verification list" in source,
and
reason why I suggested up thread tring to decouple
the *starting* of the backend with the "options" to PQ connect...
A "Helper function" in libpq could easily start the backend, and
possibly return a conninfostring to give PQconnectdb...
But if they are decoupled, I could easily envisi
ute, and especially
> accounting for programs that need multiple backends.
>
> --
> fdr
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>
--
Aidan
On Sep 4, 2012 6:06 PM, "Andrew Dunstan" wrote:
>
>
> Frankly, I have had enough failures of parallel make that I think doing
this would generate a significant number of non-repeatable failures (I had
one just the other day that took three invocations of make to get right).
So I'm not sure doing t
http://enterprisedb.com
>
> + It's impossible for everything to be true. +
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>
ceptible to that, and defending against it, no? ;-)
And they are susceptible to that if they are on PostgreSQL, Oracle, MS
SQL, DB2, etc.
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a kin
quot;.
Sure, many people don't *really* want that data durability guarantee,
even though they would like the "maybe guaranteed" version of it.
But that fine line is actually a difficult (impossible?) one to define
if you don't know, at the moment of decision, w
plication" camp are there because the guarantees of a simple RAID 1
just isn't good enough for us ;-)
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http://www.highrise.ca/
t has to be worth their time *to
them* to use it.
Witness the hundreds of graves that are he thousands bugzilla bugs out
there filed against even active open-source projects.
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca
;re using operators, what would you think is an
appropriate name for the file the operator is dumped into?
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
htt
ns in one file (hopefully
with deterministic ordering) and a sane, simple filename, than have
every function in every database in a separate file with some strange
mess in the filename that makes me cringe every time I see it.
a.
--
Aidan Van Dyk
t;index only" because heap pages aren't
"all visible"...
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http://www.highrise.ca/
On Wed, Jun 20, 2012 at 3:49 PM, Andres Freund wrote:
> On Wednesday, June 20, 2012 09:41:03 PM Aidan Van Dyk wrote:
>> On Wed, Jun 20, 2012 at 3:27 PM, Andres Freund
> wrote:
>> >> OK, so in this case, I still don't see how the "origin_id" is even
>>
ange directly from A, how does it
>> know to *not* apply it again?
> The lsn of the change.
So why isn't the LSN good enough for when C propagates the change back to A?
Why does A need more information than C?
a.
--
Aidan Van Dyk
n it..
> Yuck. The aim is to improve on whats done today ;)
>
> --
> Andres Freund http://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Training & Services
>
> --
> Sent via pgsql-hackers mailing list (pgsq
100s of GB of data
in my pg directory, the *only* corruption is that a single file
pg_control file is missing?
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
htt
* the remote", or "it's a write *by*
the remote". But when combined with other terms, only one makes sense
in all cases.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
tages" of WAL processing on the remote...
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http://www.highrise.ca/ work like a slave.
--
Sent via p
of JSON <-> Row/setof in core, I
could see this being a very nice "RPC" mechanism for PostgreSQL.
Plain HTTP still give's you the session/transaction control problem of
stateless clients, but maybe coupled with PgPool you could cobbl
lay whatever FPWs it has in the
double-write buffer it has accumulated to make sure it's writes were
consistent. Exactly as the master would do.
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca
too that the reduction in
commit/WAL latency will be offset (hopefully not as much) by increased
query processing time as backends double-write dirty buffers.
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca
On Fri, Jan 6, 2012 at 6:48 PM, Aidan Van Dyk wrote:
> I think I've said it before, but I'm guessing OLTP style database
> rarely have pages written that are dirty that aren't covered by real
> changes (so have the FPW anyways) and OLAP type generally freeze after
>
at are dirty that aren't covered by real
changes (so have the FPW anyways) and OLAP type generally freeze after
loads to avoid the hint-bit-write penalty too...
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca
7;ld love a "hook" script that was run if sync-rep state ever changed
(heck, I'ld even like it if it just choose a new sync standby).
Even better, is there a way we could start injecting "notify" events
into the cluster on these types of changes? Especially now that
no
l interest in such a thing?
>
> I wanted such a thing mere two weeks ago ...
Generally when I've wanted these things, I just make a new "$HOME" in
my shared user home dir:
export HOME=$HOME/aidan
It's worked for things I've wanted, I haven't tried it for psql
has:
1) writes for hint-bit only buffers don't need to be durable
And the problem that optimization introduces:
1) Since they aren't guarenteed durable, we can't believe a checksum
--
Aidan Van Dyk Create like a god,
ai...@high
On Wed, Dec 21, 2011 at 1:59 PM, Alvaro Herrera
wrote:
> But, well, tuples that are succesfully hinted need no more hint bits.
Not only do they need no more hinting, they also allow the next
client-serving process that hits it avoid the clog lookup to determine
the hint.
a.
--
Aidan Van
nt-bit only changes made it dirty), a page that was
really messed up on the kernel panic that last happened causing this
whole mess, or an even older page that really is giving bitrot...
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca
us... And to make matters worse, we don't even
know when the perioud of "they may be bugus" ends, unless we have a
way to methodically force PG through ever buffer in the database after
the crash... And then that makes them very hard to consider
useful...
a.
--
Aidan Van
On Wed, Oct 26, 2011 at 9:57 AM, Florian Pflug wrote:
> On Oct26, 2011, at 15:12 , Simon Riggs wrote:
>> On Wed, Oct 26, 2011 at 12:54 PM, Aidan Van Dyk wrote:
>>
>>> The read fails because their is no data at the location it's trying to
>>> read from, be
od on the master.
The read fails because their is no data at the location it's trying to
read from, because clog hasn't been extended yet by recovery.
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca
p
> feels natural but the message might scroll offscreen due to errors...
Decorate them with a marker like:
\extension
And make the CREATE EXTENSION skip (or verify) it?
It will make psql stop on the \extension command.
a.
--
Aidan Van Dyk
s a *lot* when they heap pages are not in shared buffers
(both ways, saving IO, or causing lots of random IO)
Can we hope that if pages are not in shared buffers, they are not
recently modified, so hopefully both all visible, and have the VM
bit?set? Or does the table-based nat
py enough if pg_upgrade could easily upgrade given a
datadir that had a postgresql.conf in it, or possibly a
postgresql.conf that had data_directory set in it.
Anything else, and I say it's responsibility of whoever scripted the
startup to be able to provide all the necessary information to
pg_up
hance to run, and based on that,goes to look for something
in clog page that wasn't guarenteed to exists at the start of the
backup period, and bombing out before recovery has a chance to start
replaying WAL and write the new clog page.
a.
--
Aidan
On Wed, Sep 21, 2011 at 1:34 PM, Aidan Van Dyk wrote:
> And I think Tom touched on this point in the
> "recovery.conf/recovery.done" thread a bit too.
Doh! That's this thread
/me slinks away, ashamed for not even taking a close look at the to/cc list.
e different "when done do ..."
distinctions, and and using that distinction to help our nomenclature.
Both recovery/slave (both hot or cold) use the same retrieve/apply
machinery (and thus configuration options). But because of the
different "caught up
all needed to have their own
search_path and other settings set before login (yes, dumb apps,
mostly odbc), and be able to have the same "userid" for different
databases, using different settings...
a.
--
Aidan Van Dyk Create like a go
"real error" (errno should be set) is
one that could yet lead to confusion.
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http://www.highrise.ca/ w
test, I'm going to guess there will be lots of chance for
code to do:
if (pg_write_no_intr(...) < 0)
...
which will only catch some of the errors, and happily continue with the rest...
a.
--
Aidan Van Dyk Create like a god,
ai...@high
r very seriously pointing users at properly written
tools like omniptr, or ptrtools, walmgr, etc...
Neither of those cases should ever happen. If you're copying a file
into the archive, and making it appear non-atomically in your archive,
your doing something wrong.
Period.
No excuses.
a.
-
7;ve finally remove the BKL out of VFS/inode?
I mean, complaining about scalability in linux 2.6.18 is like
complaining about scalability in postgresql 8.2 ;-)
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca
an opaque "simple key" type
function, you haven't given the planner/executor anything better to be
able to pick partitions for queries, unless the query is an exact "key
=" type of operation.
So I'm failing to see the benefit of that "key based" partitioning,
even if
alks on it, but others might not bother to even start.
It's a double-edged sword. If nobody writes anything, because
everyone is afraid to possibly having to change things, nothing will
never need to be changed ;-)
a.
--
Aidan Van Dyk
On Mon, Jun 13, 2011 at 12:30 PM, Robert Haas wrote:
> Incidentally, are you planning to revive the PostgreSQL FDW for 9.2?
> That would be a killer feature.
Even more killer would be that it could be built/packaged as an
extension, and use for 9.1 too ;-)
a.
--
Aidan V
So, if SSI conflicts something on the UPDATE case, it would necessrily
have to conflict the DELETE+INSERT case as well, and vice-versa.
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a kin
nt into the C
code, there really isn't anything catalog-wise that you need to
"manage" for upgrades.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http://www.highrise.c
, that all depends on:
1) pgindent being "work everywhere", "exactly the same"
2) Discipline of all new published commits being "pgindent clean".
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca
neral form seems to be to declare things as:
typedef struct foo { ... } foo;
Is there any reason why we see any struct foo in the sources other
than in the typedef line?
"Legacy" and "invasive patch" are good enough reasons, if they are it...
a.
--
Aidan Van Dyk
On Tue, Apr 19, 2011 at 1:57 PM, Kevin Grittner
wrote:
> Aidan Van Dyk wrote:
>
>> And for the "first-hack-that-comes-to-mind", I find my self
>> pulling out the named fifo trick all the time, and just leaving my
>> for/loop/if logic in bash writing SQL com
to write an answer to a file that I then read back in bash
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http://www.highrise.ca/ work like a sla
transactions committed" for the older clog segments might mean a scan
on a *toast* heap might return tuples as committed when they might
have been aborted, but the real table heap would never refer to those,
right?
a.
--
Aidan Van Dyk Create like
psql ecom
>
> boom, I am in.
>
> Thoughts?
So you're really looking to make psql use "service" connection
definitions more easily, not just retrieve the password associated
with the given (maybe defaulted) host:port:database:user, right?
a.
--
Aidan Van Dyk
ge world,
"objects" in PG world) would definitely be nice, but my Makefile can
build those for me for now until 9.2 (or 9.3, 9.3, etc), if only I had
a way to track them with my installed extension ;-)
a.
--
Aidan Van Dyk Create
and been through the hell that is open
source package variants, all with the ability to turn on/off features
at configure/compile time, a just versions even with <, <=, =, >=, >
all mapped correctly isn't good enough.
Of course, I'ld love for extension in 9.1 to provide a ba
compatable
objects.
But checking
http://developer.postgresql.org/pgdocs/postgres/extend-extensions.html,
I don't see any "provides" mechanism. That might be something
actually needed if we are trying to avoid "version" comparisons and
want to be describing actual de
e master doesn't), you haven't changed any of the
guarantees sync rep can make (or not).
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http://www.highrise.ca/
that important, your logs/monitoring are *equally*
important, because they are what give you confidence your data is as
safe as you think it is...
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca co
On Mon, Mar 7, 2011 at 2:29 PM, Aidan Van Dyk wrote:
> They they are either already hosed or already using 2PC.
Sorry, to expand on my all too brief comment, even *without*
replication, they are hosed.
Once you issue commit, you have know knowledge if the commit is
durable, (or even posi
a transaction is not successfully
>> replicated for any reason, including crash, it is rolled back in the master
>> too. That would require 2PC.
>>
>
> My worry is that the stricter definition is what many people will expect,
> without reading the fine print.
They they are eith
fore
it is installed, so make sure it's installed, or be prepared to
cope with errors during the installation.
And make sure you don't try and drop pl/pgsql language when
the extension is installed either.
Maybe that's enough for 9.
o sync slave is available" is one possible way oof "dealing
with" them ;-)
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http://www.highrise.ca/
'm not forced to
*install* 9.0, because PG has decide that the proper way to release
ist o make a single release of all versions.
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
h
course, if I have slony 1.2.$x replicating one of my databases,
I'ld love to be able to try slony 2 and have it packaged on my system
too to test somethign else. And not have to upgrade my slony 2
instance just to get the critical bugfix for my production slony
1.2$x+1.
a.
--
Aidan Van
ion
"afoo-1" and "afoo-2" to be able to have them both co-exist on the
filesystem independantly, and at that point, *I* don't need multiple
versions of it anymore. I'm going to keep the same extension
objects/libraries backwards compatible, and I j
we can leave making it safer as a
> topic for future investigation.
Personally, I'ld rather be able to install the *same*
extension/version in different schemas at the same time then move an
extension from 1 schema to another, although I have no problems with
extensions moving out
EATE
..." staements for all previous 15 versions can succeed.
With having the $old -> $new scripts, the new .so only needs to have
functions enough that the DROPs work, and the new CREATE... work.
--
Aidan Van Dyk Create like a god,
ve fsync, write WAL, fsync WAL,
send WAL, wait for slave fsync". And it's expense is all the time,
rather than just when the "no slave no go" situations arise.
And it doesn't reduce the transactions I need to verify by hand
either, because tha
On Fri, Jan 21, 2011 at 1:03 PM, Tom Lane wrote:
> Robert Haas writes:
>> On Fri, Jan 21, 2011 at 12:23 PM, Aidan Van Dyk wrote:
>>> When no sync slave is connected, yes, I want to stop things hard.
>
>> What you're proposing is to fail things earlier than absolu
27;ld like it if PG couldn't do anything to generate
any user-initiated WAL unless there is a sync slave connected. Yes, I
understand that leads to hard-fail, and yes, I understand I'm in the
minority, maybe almost singular in that desire.
a.
--
Aidan Van
gs being backwards
compatible as long as there's a very easy way to know what I might be
looking at, so conversion is easy...
But then again, I don't have multiple gigabytes of logs to process either.
a.
--
Aidan Van Dyk Create like a
uot; is closed.
A FIFO would allow postmaster to not need as many file handles, and
clients reading the fifo would notice when the writer (postmaster)
closes it.
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca
> backups less dependent on CPU, among them:
>
> - Making the on-disk representation smaller
> - Making COPY more efficient
>
> As far as I know, none of this work is public yet.
pg_dump is another story. But it's not related to base backups for
PIT Recovery/Replication.
a.
--
A
there is a chance I (my database
system) confirmed a transaction that I can't recover.
So sync rep with 1st past post already makes my job easier. I'll take
it over nothing ;-)
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca
we have the problem even on a
single pg cluster on a single machine. But the point is that if
you've committed, any new transactions see *at least* that data or
newer. But no chance of older.
But personally, I'm not interested in that ;-)
--
Aidan Va
e commit packet stuffed out the
network, you're in the same boat. The data might be committed, even
though you didn't get the commit packet, and when your DB recovers,
it's got the committed data that you never "knew" was committed.
a.
e segment.
This get's you an archive synced as it's made (as long as streamrecv
is running), and my "verify"archive command would make sure that if
for some reason, the backup archive went "down", the wal segments
would be blocked on the master until it's up agai
On Wed, Dec 29, 2010 at 9:11 AM, Gurjeet Singh wrote:
> On Wed, Dec 29, 2010 at 8:31 AM, Joel Jacobson wrote:
>>
>>
>> 2010/12/29 Aidan Van Dyk
>>>
>>> On Wed, Dec 29, 2010 at 2:27 AM, Joel Jacobson
>>> wrote:
>>>
>>>
>>
On Wed, Dec 29, 2010 at 2:27 AM, Joel Jacobson wrote:
So, how different (or not) is this to the "directory" format that was
coming out of the desire of a parallel pg_dump?
a.
--
Aidan Van Dyk Create like a god,
ai...@h
ive of
anything else in the database. And path has to be encoding aware.
And you want names that glob well, so for instance, you could exclude
*.data (or a schema) from the diff.
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca
uot;search the bugtracker" is no less rude than "search the archives"...
And most of the bugtrackers I've had to search have way *less*
ease-of-use for searching than a good mailing list archive (I tend to
keep going back to gmane's search)
a.
--
Aidan Van Dyk
don't deny the rest of us airbags while you keep working on
teleportation ;-)
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http://www.highrise.ca/
d buffers.
Being able to arbitrary (i.e at any point in time) prove that the
shared buffers contents are exactly what they should be may be a
worthy goal, but that's many orders of magnitude more difficult than
verifying that the bytes we read from disk are the ones we wrote to
etwork, and the
admins have set it be very network tolerant.
The ACK might report that the salve is hopelessly behind on
fsyncing/applying it's WAL, but that's good too. At least then the
ACK comes back, and the master knows the slave is still churning away
on th
to have that error if I give a bad set of version matches.
If only have those 2 lines to manage, it's a lot more likely I won't
mess them up than if I have to manage 30 almost identical lines and
not miss/duplicate a version.
;-)
--
Aidan Van Dyk
convention on me to maitain/install an upgrade script for
every single version is way more than asking me to just specify an
upgrade script for versions.
Again, I'ld love for the "version" to support some sort of prefix or
wildcard matching, so I could do:
upgrade-1.* = $
>so wise. They would only be "bug
fixes" if I did something wrong in my stuff.. Anything not compatible
woudl bump the first number.
If it's a "prefix" type match, then the PG versionins woudl work too,
for intsance:
upgrade-9.0.=...
would mat
n't have some sort of
TAS/memory barrier/cache-coherency stuff in it ;-)
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http://www.highrise.ca/
7;s
guarding is in another cacheline, because that won't *necessarily*
force cache coherency in your local lock/variable memory.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http:/
PUs with different
caches that are incoherent to have those problems.
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http://www.highrise.ca/ work
nt is *better*
that giving everybody "yet another password" they have to manage, have
users not mis-manage, and make sure users don't mis-use...
So, yes, ident is only as secure as the *network and machines* it's
used on. Passwords are only as secure as
s when the "appside" knows the data's dispensable and
rebuild-able.
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http://www.highrise.ca/
t; song when people
complain about "pg being too slow"
;-)
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http://www.highrise.ca/ work lik
te penalty the 1st time they scan the
tables...
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http://www.highrise.ca/ work like a slave.
--
Sent v
not write that page, and loose the work the hint-bits did, or do
a full-page WAL of it, so the torn-page checksum is fixed
Both of these are theoretical performance tradeoffs. How badly do we
want to verify on read that it is *exactly* what we thought we wrote?
a.
--
Aidan Van D
that
write could be in-consistent.
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http://www.highrise.ca/ work like a slave.
--
Sent via pgsql-hackers maili
1 - 100 of 364 matches
Mail list logo
