On 1 June 2013 21:26, Robert Haas wrote:
> On Sat, Jun 1, 2013 at 3:22 PM, Simon Riggs wrote:
>> If we set a bit, surely we need to write the page. Isn't that what we
>> were trying to avoid?
>
> No, the bit only gets set in situations when we were going to dirty
> the page for some other reason
On Sat, Jun 1, 2013 at 1:49 PM, Simon Riggs wrote:
> On 30 May 2013 19:48, Josh Berkus wrote:
>
>> There's currently some great ideas bouncing around about eliminating the
>> overhead associated with FREEZE. However, I wanted to take a step back
>> and take a look at the big picture for VACUUM,
On Sat, Jun 01, 2013 at 09:41:13AM +0100, Simon Riggs wrote:
> FK checks can be expensive, especially when loading large volumes of
> data into an existing table or partition. A couple of ideas for
> improving performance are discussed here:
>
> 1. Use Case: Bulk loading
> COPY pgbench_accounts;
On Sat, Jun 1, 2013 at 3:22 PM, Simon Riggs wrote:
> If we set a bit, surely we need to write the page. Isn't that what we
> were trying to avoid?
No, the bit only gets set in situations when we were going to dirty
the page for some other reason anyway. Specifically, if a page
modification disco
On Sat, Jun 1, 2013 at 2:48 PM, Heikki Linnakangas
wrote:
>> We define a new page-level bit, something like PD_RECENTLY_FROZEN.
>> When this bit is set, it means there are no unfrozen tuples on the
>> page with XIDs that predate the current half-epoch. Whenever we know
>> this to be true, we set
On 1 June 2013 19:48, Heikki Linnakangas wrote:
> On 31.05.2013 06:02, Robert Haas wrote:
>>
>> On Thu, May 30, 2013 at 2:39 PM, Robert Haas
>> wrote:
>>>
>>> Random thought: Could you compute the reference XID based on the page
>>> LSN? That would eliminate the storage overhead.
>>
>>
>> After m
On 31.05.2013 06:02, Robert Haas wrote:
On Thu, May 30, 2013 at 2:39 PM, Robert Haas wrote:
Random thought: Could you compute the reference XID based on the page
LSN? That would eliminate the storage overhead.
After mulling this over a bit, I think this is definitely possible.
We begin a new
On 27 May 2013 15:31, Tom Lane wrote:
> Simon Riggs writes:
>> On 26 May 2013 17:10, Tom Lane wrote:
>>> More readable would be to invent an intermediate nonterminal falling
>>> between ColId and ColLabel, whose expansion would be "IDENT |
>>> unreserved_keyword | col_name_keyword | type_func_na
On 1 June 2013 19:25, Heikki Linnakangas wrote:
> Hmm, it would probably be much easier to squeeze, say, one byte from the
> tuple header, than full four bytes. Then you could store store a null bitmap
> for upto 16 columns, without crossing the 24 byte mark. That would already
> get you much of
On 01.06.2013 11:22, Simon Riggs wrote:
What is the benefit?
Merging xmin/xmax would save 4 bytes per row. On servers with 8 byte
word length, that means that we'd save 8 bytes per row for tables that
have between 9 and 40 columns. Which is the majority of tables.
Hmm, it would probably be much
On 1 June 2013 15:45, Tom Lane wrote:
> Simon Riggs writes:
>> The way to resolve this is to have two functions:
>> pg_is_in_backup() - which covers both/all kinds of backup
>> pg_is_in_exclusive_backup() - which covers just the exclusive backup mode
>
> What will you do with pg_backup_start_t
On 28 May 2013 15:15, Robert Haas wrote:
> On Sat, May 25, 2013 at 6:14 AM, Simon Riggs wrote:
>> I think the right way is actually to rethink and simplify all this
>> complexity of Freezing/Pruning/Hinting/Visibility
>
> I agree, but I think that's likely to have to wait until we get a
> plugga
On 30 May 2013 19:48, Josh Berkus wrote:
> There's currently some great ideas bouncing around about eliminating the
> overhead associated with FREEZE. However, I wanted to take a step back
> and take a look at the big picture for VACUUM, FREEZE and ANALYZE.
That is a very commendable approach.
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 06/01/2013 09:39 AM, Tom Lane wrote:
> I wrote:
>> Actually, I believe the answer is just that getSchemaData() is
>> doing things in the wrong order:
>
> BTW, I'm inclined to think it's also wrong that the
> getEventTriggers() call was just added a
I wrote:
> Actually, I believe the answer is just that getSchemaData() is doing
> things in the wrong order:
BTW, I'm inclined to think it's also wrong that the getEventTriggers()
call was just added at the end; those things are certainly not table
subsidiary objects. I don't know if we allow eve
Joe Conway writes:
> I can look at having pg_dump ignore these entries, but I suspect that
> will be quite a bit more invasive.
Actually, I believe the answer is just that getSchemaData() is doing
things in the wrong order:
if (g_verbose)
write_msg(NULL, "reading rewrite rules\n");
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 06/01/2013 08:32 AM, Andres Freund wrote:
> On 2013-06-01 11:31:05 -0400, Tom Lane wrote:
>> But in any case, making rules act differently from other table
>> properties for this purpose seems seriously wrong.
>
> What's your proposal to fix this
On 2013-06-01 11:31:05 -0400, Tom Lane wrote:
> Andres Freund writes:
> > On 2013-06-01 11:07:53 -0400, Tom Lane wrote:
> >> I don't like this approach much.
> >>
> >> 1. It does nothing to fix the issue in *existing* databases, which
> >> won't have pg_depend entries like this.
>
> > Well, you
Andres Freund writes:
> On 2013-06-01 11:07:53 -0400, Tom Lane wrote:
>> I don't like this approach much.
>>
>> 1. It does nothing to fix the issue in *existing* databases, which
>> won't have pg_depend entries like this.
> Well, you can now write an extension upgrade script that adds the
> miss
On 2013-06-01 08:27:42 -0700, Joe Conway wrote:
> > Uh. Why would you do the lock(file) thingy *after* calling
> > pg_start_backup? You should do lock before calling start backup
> > and remove the lock after calling stop backup. In that case I don't
> > see where the race condition is?
>
> No, t
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 06/01/2013 08:14 AM, Andres Freund wrote:
> On 2013-06-01 08:11:26 -0700, Joe Conway wrote:
>> -BEGIN PGP SIGNED MESSAGE- Hash: SHA1
>>
>> On 06/01/2013 02:43 AM, Christoph Moench-Tegeder wrote:
>>> ## Joe Conway (m...@joeconway.com):
>>>
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 06/01/2013 08:07 AM, Tom Lane wrote:
> Joe Conway writes:
>> On 05/31/2013 08:46 PM, Robert Haas wrote:
>>> Changing SQL syntax in the back-branches isn't normally
>>> something we do, but I confess I don't see any real reason not
>>> to do it in t
On 2013-06-01 11:07:53 -0400, Tom Lane wrote:
> Joe Conway writes:
> > On 05/31/2013 08:46 PM, Robert Haas wrote:
> >> Changing SQL syntax in the back-branches isn't normally something
> >> we do, but I confess I don't see any real reason not to do it in
> >> this case.
>
> > That was part of my
On 2013-06-01 08:11:26 -0700, Joe Conway wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 06/01/2013 02:43 AM, Christoph Moench-Tegeder wrote:
> > ## Joe Conway (m...@joeconway.com):
> >
> >> However there is a period of time after pg_start_backup() is
> >> first executed to when i
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 06/01/2013 02:43 AM, Christoph Moench-Tegeder wrote:
> ## Joe Conway (m...@joeconway.com):
>
>> However there is a period of time after pg_start_backup() is
>> first executed to when it completes, during which backup_label
>> file does not exist ye
Joe Conway writes:
> On 05/31/2013 08:46 PM, Robert Haas wrote:
>> Changing SQL syntax in the back-branches isn't normally something
>> we do, but I confess I don't see any real reason not to do it in
>> this case.
> That was part of my hesitation, but I don't see any better way to fix
> existing
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 05/31/2013 08:46 PM, Robert Haas wrote:
> On Wed, May 29, 2013 at 6:55 PM, Joe Conway
> wrote:
>>> OK, simple enough. New patch attached. I still need to do some
>>> testing to verify this does not break anything, but other than
>>> that, any com
Notes on a longer term idea...
An item pointer (also called line pointer) is used to allow an
external pointer to an item, while allowing us to place the tuple that
anywhere on the page. An ItemId is 4 bytes long and currently consists
of (see src/include/storage/itemid.h)...
typedef struct ItemI
Simon Riggs writes:
> The way to resolve this is to have two functions:
> pg_is_in_backup() - which covers both/all kinds of backup
> pg_is_in_exclusive_backup() - which covers just the exclusive backup mode
What will you do with pg_backup_start_time()?
regards, tom la
* Bruce Momjian (br...@momjian.us) wrote:
> OK.
Done.
Thanks,
Stephen
signature.asc
Description: Digital signature
On 30 May 2013 19:39, Robert Haas wrote:
> On Thu, May 30, 2013 at 9:33 AM, Heikki Linnakangas
> wrote:
>> The reason we have to freeze is that otherwise our 32-bit XIDs wrap around
>> and become ambiguous. The obvious solution is to extend XIDs to 64 bits, but
>> that would waste a lot space. Th
On 30 May 2013 14:33, Heikki Linnakangas wrote:
> Since we're bashing around ideas around freezing, let me write down the idea
> I've been pondering and discussing with various people for years. I don't
> think I invented this myself, apologies to whoever did for not giving
> credit.
>
> The reaso
On 31 May 2013 21:06, Andres Freund wrote:
> On 2013-05-31 22:53:14 +0300, Heikki Linnakangas wrote:
>> On 31.05.2013 22:36, Andres Freund wrote:
>> >On 2013-05-31 22:29:45 +0300, Heikki Linnakangas wrote:
>> >>Note that pg_is_in_backup() just checks for presence of
>> >>$PGDATA/backup_label. Also
Yes, I'm sure.
Difference of filenodes and new files changed w.r.t my first mail, because
I added a table.
I attached 3 files,
newfile.pg : list of numerical files in base/[db-oid], ls | grep
'[[:digit:]]\>'
filenode.pg : select distinct relfilenode from pg_class
newfile-filenode.pg : Set of oids
On 2013-06-01 13:04:55 +0200, Martijn van Oosterhout wrote:
> On Sat, Jun 01, 2013 at 03:27:40PM +0430, Soroosh Sardari wrote:
> > Yes, I have some files which is not in pg_class.relfilenode of any table or
> > index.
> > I want to know which table or index stored in such files.
>
> That shouldn't
2013/6/1 Martijn van Oosterhout
> On Sat, Jun 01, 2013 at 03:27:40PM +0430, Soroosh Sardari wrote:
> > Yes, I have some files which is not in pg_class.relfilenode of any table
> or
> > index.
> > I want to know which table or index stored in such files.
>
> That shouldn't happen. Are you sure you
On Sat, Jun 01, 2013 at 03:27:40PM +0430, Soroosh Sardari wrote:
> Yes, I have some files which is not in pg_class.relfilenode of any table or
> index.
> I want to know which table or index stored in such files.
That shouldn't happen. Are you sure you're looking in the right
database? Kan you list
Yes, I have some files which is not in pg_class.relfilenode of any table or
index.
I want to know which table or index stored in such files.
> From that page:
>
> Each table and index is stored in a separate file. For ordinary
> relations, these files are named after the table or index's filenod
On Sat, Jun 01, 2013 at 03:11:50PM +0430, Soroosh Sardari wrote:
> Dear Hackers
>
> I've created a new DB, and a bunch of files created in base/12054, 12054
> is oid of the new DB.
> I want to find what table stored in each file.
> BTW, I read this
> http://www.postgresql.org/docs/9.2/interactive
Dear Hackers
I've created a new DB, and a bunch of files created in base/12054, 12054
is oid of the new DB.
I want to find what table stored in each file.
BTW, I read this
http://www.postgresql.org/docs/9.2/interactive/storage-file-layout.html
I have 156 files with numerical names, vm and fsm fil
## Joe Conway (m...@joeconway.com):
> However there is a period of time after pg_start_backup() is first
> executed to when it completes, during which backup_label file does not
> exist yet, but the backup has essentially been started. Is there any
> way to detect this state?
When I did some rese
On Sat, Jun 01, 2013 at 09:22:05AM +0100, Simon Riggs wrote:
> When would this make sense?
> Frequently. Most of the time a tuple needs only one xid set. In most
> cases, we set xmin and xmax a long time apart. Very few cases end with
> both of them set inside the *same* xmin horizon. In a heavy
>
New submission for the next commit fest.
This new version also reports the average lag time, i.e. the delay between
scheduled and actual transaction start times. This may help detect whether
things went smothly, or if at some time some delay was introduced because
of the load and some catchup
FK checks can be expensive, especially when loading large volumes of
data into an existing table or partition. A couple of ideas for
improving performance are discussed here:
1. Use Case: Bulk loading
COPY pgbench_accounts; --> references pgbench_branches with many
repeated values
Proposal: Tran
As we get faster, we'll need to wrap the xid counter more quickly. At
current full speed, we could wrap the xid counter every 72 hours.
This is a concern for very large tables, since currently we have to
rescan the whole table. This has my interest for change in the next
release.
We could solve t
Currently, we hold xmin and xmax for each tuple.
For xmax, we have the multixact mechanism that allows us to represent
an array of xids with just a single pseudo xid.
So why not hold xmin and xmax as part of a multixact? Instead of
caching two xids on each tuple, why not hold just one and allow a
On 2013-06-01 17:05:57 +0900, Michael Paquier wrote:
> On Sat, Jun 1, 2013 at 4:55 AM, Alvaro Herrera
> wrote:
>
> > Heikki Linnakangas wrote:
> >
> > > The manual says:
> > > >pg_is_in_backup()boolTrue if an on-line exclusive backup is
> > still in progress.
> > >
> > > So clearly that i
On Sat, Jun 1, 2013 at 4:55 AM, Alvaro Herrera wrote:
> Heikki Linnakangas wrote:
>
> > The manual says:
> > >pg_is_in_backup()boolTrue if an on-line exclusive backup is
> still in progress.
> >
> > So clearly that is intentional. That could use some rephrasing,
> > though; a layman won't
48 matches
Mail list logo
