Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-26 Thread Alvaro Herrera
Bruce Momjian escribió:
 
 This has been saved for the next commit-fest:

I noticed you broke an URL that previously worked: what was
http://momjian.us/mhonarc/patches/[EMAIL PROTECTED]
is now
http://momjian.us/mhonarc/patches_hold/[EMAIL PROTECTED]

May I suggest that the URLs with Message-Ids are stored outside the
particular patch queue directory?  The script I showed you yesterday
could be used to do that.

Also I noticed that by moving it to the hold queue, the comments that
may have existed on the patch queue are now gone :-(  I'm not sure if
there were any in this case, but it's better if we're aware of that
fact.  I think this could be solved if the namespace of the comment
does not contain the patch queue name.

I moved it to the May commitfest on the wiki too.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-26 Thread Bruce Momjian
Alvaro Herrera wrote:
 Bruce Momjian escribi?:
  
  This has been saved for the next commit-fest:
 
 I noticed you broke an URL that previously worked: what was
 http://momjian.us/mhonarc/patches/[EMAIL PROTECTED]
 is now
 http://momjian.us/mhonarc/patches_hold/[EMAIL PROTECTED]
 
 May I suggest that the URLs with Message-Ids are stored outside the
 particular patch queue directory?  The script I showed you yesterday
 could be used to do that.

My email are added/removed so I am unsure how to do that easily becuase
the lists would share the same directory.  We don't have that problem
with the archives.

 Also I noticed that by moving it to the hold queue, the comments that
 may have existed on the patch queue are now gone :-(  I'm not sure if
 there were any in this case, but it's better if we're aware of that
 fact.  I think this could be solved if the namespace of the comment
 does not contain the patch queue name.

I specifically set things up so the comments should move with the email.


-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-26 Thread Alvaro Herrera
Bruce Momjian escribió:
 Alvaro Herrera wrote:

  I noticed you broke an URL that previously worked: what was
  http://momjian.us/mhonarc/patches/[EMAIL PROTECTED]
  is now
  http://momjian.us/mhonarc/patches_hold/[EMAIL PROTECTED]
  
  May I suggest that the URLs with Message-Ids are stored outside the
  particular patch queue directory?  The script I showed you yesterday
  could be used to do that.
 
 My email are added/removed so I am unsure how to do that easily becuase
 the lists would share the same directory.  We don't have that problem
 with the archives.

My point is that you should only _add_ Message-Ids, not remove them.
You can move the messages from one queue to the other to your heart's
content, but the Message-Id URL should continue to work with no changes.

  Also I noticed that by moving it to the hold queue, the comments that
  may have existed on the patch queue are now gone :-(  I'm not sure if
  there were any in this case, but it's better if we're aware of that
  fact.  I think this could be solved if the namespace of the comment
  does not contain the patch queue name.
 
 I specifically set things up so the comments should move with the email.

Yeah, I noticed that after sending the email -- the js-kit name seems to
be only msgid-foo.  I thought the permalink= attribute was part of
that, but perhaps not?  In case you added that permalink attribute
because of my request the other day, let me clarify that what I was
actually thinking was having something like

a href=http://momjian.us/msgid/[EMAIL PROTECTED]permalink/a

after the name of the poster, so that it would be visible on the index
page and the user didn't have to open the page to get it.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-26 Thread Bruce Momjian
Alvaro Herrera wrote:
 Bruce Momjian escribi?:
  Alvaro Herrera wrote:
 
   I noticed you broke an URL that previously worked: what was
   http://momjian.us/mhonarc/patches/[EMAIL PROTECTED]
   is now
   http://momjian.us/mhonarc/patches_hold/[EMAIL PROTECTED]
   
   May I suggest that the URLs with Message-Ids are stored outside the
   particular patch queue directory?  The script I showed you yesterday
   could be used to do that.
  
  My email are added/removed so I am unsure how to do that easily becuase
  the lists would share the same directory.  We don't have that problem
  with the archives.
 
 My point is that you should only _add_ Message-Ids, not remove them.
 You can move the messages from one queue to the other to your heart's
 content, but the Message-Id URL should continue to work with no changes.

Well, when an email is applied, it is deleted.  How do I update the
message-id for that in an automated manner.  Right now a rebuild deleted
all the links and recreates them.

   Also I noticed that by moving it to the hold queue, the comments that
   may have existed on the patch queue are now gone :-(  I'm not sure if
   there were any in this case, but it's better if we're aware of that
   fact.  I think this could be solved if the namespace of the comment
   does not contain the patch queue name.
  
  I specifically set things up so the comments should move with the email.
 
 Yeah, I noticed that after sending the email -- the js-kit name seems to
 be only msgid-foo.  I thought the permalink= attribute was part of
 that, but perhaps not?  In case you added that permalink attribute
 because of my request the other day, let me clarify that what I was
 actually thinking was having something like
 
 a href=http://momjian.us/msgid/[EMAIL PROTECTED]permalink/a

The permalink is for people who get email --- it tells them which
message got the comment.  (People were complaining before I fixed that.)

 after the name of the poster, so that it would be visible on the index
 page and the user didn't have to open the page to get it.

We can do that if people want.  I used to show the message id on the
thread page but several felt it was too cluttered-looking.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-26 Thread Alvaro Herrera
Bruce Momjian escribió:
 Alvaro Herrera wrote:

  My point is that you should only _add_ Message-Ids, not remove them.
  You can move the messages from one queue to the other to your heart's
  content, but the Message-Id URL should continue to work with no changes.
 
 Well, when an email is applied, it is deleted.  How do I update the
 message-id for that in an automated manner.  Right now a rebuild deleted
 all the links and recreates them.

Well, remove the part that deletes links, and keep the part that
creates links.  That way, links that used to work continue working.

I am assuming you use hard links -- obviously this doesn't work with
symlinks.  (My script creates hard links.)

  after the name of the poster, so that it would be visible on the index
  page and the user didn't have to open the page to get it.
 
 We can do that if people want.  I used to show the message id on the
 thread page but several felt it was too cluttered-looking.

Well, it would have helped me.  I suggested permalink as text because
it is less clutter than the full Message-Id.  Besides, the Message-Id by
itself is useless, whereas a link is useful.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-26 Thread Bruce Momjian
Alvaro Herrera wrote:
 Bruce Momjian escribi?:
  Alvaro Herrera wrote:
 
   My point is that you should only _add_ Message-Ids, not remove them.
   You can move the messages from one queue to the other to your heart's
   content, but the Message-Id URL should continue to work with no changes.
  
  Well, when an email is applied, it is deleted.  How do I update the
  message-id for that in an automated manner.  Right now a rebuild deleted
  all the links and recreates them.
 
 Well, remove the part that deletes links, and keep the part that
 creates links.  That way, links that used to work continue working.
 
 I am assuming you use hard links -- obviously this doesn't work with
 symlinks.  (My script creates hard links.)

I was using symlinks.  I try to avoid hardlinks that cross directories
--- not sure why, probably because I can't easily manage them to find
out what something is linked to without playing with inodes.  You are
right that if I used hard-links I put things in a separate directory,
and the hard links would still contain the email, though if the email
has been removed, should we still be returning it from a query?  I
suppose I could delete items with only one link.

Well, if I move the permanent links to another directory, as you
suggest, I am going to invalidate every link.  Maybe we should wait for
this commit fest to end and then I can create a permanent link directory
and use hard links.

Or is everyone OK with invalidating all the permanent links now.

   after the name of the poster, so that it would be visible on the index
   page and the user didn't have to open the page to get it.
  
  We can do that if people want.  I used to show the message id on the
  thread page but several felt it was too cluttered-looking.
 
 Well, it would have helped me.  I suggested permalink as text because
 it is less clutter than the full Message-Id.  Besides, the Message-Id by
 itself is useless, whereas a link is useful.

Are you talking about the permalink in the js-kit comment or the
permalink at the top of each message?  What text do you want to be the
permalink?  How is that created in an automated manner?

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-26 Thread Bruce Momjian
bruce wrote:
  I am assuming you use hard links -- obviously this doesn't work with
  symlinks.  (My script creates hard links.)
 
 I was using symlinks.  I try to avoid hardlinks that cross directories
 --- not sure why, probably because I can't easily manage them to find
 out what something is linked to without playing with inodes.  You are
 right that if I used hard-links I put things in a separate directory,
 and the hard links would still contain the email, though if the email
 has been removed, should we still be returning it from a query?  I
 suppose I could delete items with only one link.
 
 Well, if I move the permanent links to another directory, as you
 suggest, I am going to invalidate every link.  Maybe we should wait for
 this commit fest to end and then I can create a permanent link directory
 and use hard links.
 
 Or is everyone OK with invalidating all the permanent links now.

OK, I remember now.  The problem wasn't symlinks but that mhonarc
generates URLs relative to the current directory.  If we have message-id
files outside the directory, links like Thread Next will not work.  We
could try to change those to absolute with a script.  The community
archives are going to have the same problem.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-26 Thread Alvaro Herrera
Bruce Momjian escribió:

 OK, I remember now.  The problem wasn't symlinks but that mhonarc
 generates URLs relative to the current directory.  If we have message-id
 files outside the directory, links like Thread Next will not work.  We
 could try to change those to absolute with a script.  The community
 archives are going to have the same problem.

Ahh, thanks for pointing it out.  I think I will use an HTTP redirect.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-26 Thread Bruce Momjian
Alvaro Herrera wrote:
 Bruce Momjian escribi?:
 
  OK, I remember now.  The problem wasn't symlinks but that mhonarc
  generates URLs relative to the current directory.  If we have message-id
  files outside the directory, links like Thread Next will not work.  We
  could try to change those to absolute with a script.  The community
  archives are going to have the same problem.
 
 Ahh, thanks for pointing it out.  I think I will use an HTTP redirect.

How do you know which directory to redirect to?

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-26 Thread Bruce Momjian
Bruce Momjian wrote:
 Alvaro Herrera wrote:
  Bruce Momjian escribi?:
  
   OK, I remember now.  The problem wasn't symlinks but that mhonarc
   generates URLs relative to the current directory.  If we have message-id
   files outside the directory, links like Thread Next will not work.  We
   could try to change those to absolute with a script.  The community
   archives are going to have the same problem.
  
  Ahh, thanks for pointing it out.  I think I will use an HTTP redirect.
 
 How do you know which directory to redirect to?

I can't see how Apache redirects would work, but I suppose you could use
HTML redirect because you know at the time you are creating the message
file which directory to point to, but again the title bar will change
to relative once you do that.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-26 Thread Alvaro Herrera
Bruce Momjian wrote:
 Alvaro Herrera wrote:

  Ahh, thanks for pointing it out.  I think I will use an HTTP redirect.
 
 How do you know which directory to redirect to?

The script gets the directory as a parameter.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-21 Thread Simon Riggs
On Fri, 2007-03-30 at 12:28 +0530, NikhilS wrote:

 Please find attached the WIP version 1 of the auto partitioning patch.
 There was discussion on this a while back on -hackers at:
 http://archives.postgresql.org/pgsql-hackers/2007-03/msg00375.php
 
 Please note that this patch tries to automate the activities that
 currently are carried out manually. It does nothing fancy beyond that
 for now. There were a lot of good suggestions, I have noted them down
 but for now I have tried to stick to the initial goal of automating
 existing steps for providing partitioning. 
 
 Things that this patch does:

I think this patch is a reasonable first step and clearly written, but
not yet ready for application to Postgres in this commit fest.

I would say we need:

* Clear explanation of the new syntax, with examples of each permutation
so we can see how that would work. In light of recent discussions on
-hackers we need to take a view on whether we should go with Gavin's
suggested syntax or this syntax.

* There are some additional syntax items I don't understand the need
for. So these need to be explained.

* I would be against using the term PARTITION BY since it is already a
phrase that is part of the SQL Standard. Perhaps PARTITIONED BY?

* We need regression tests for any new command syntax

* No docs - that might be the same thing as the first item

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com 

  PostgreSQL UK 2008 Conference: http://www.postgresql.org.uk


-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-21 Thread NikhilS
Hi Simon,

On Fri, Mar 21, 2008 at 7:30 PM, Simon Riggs [EMAIL PROTECTED] wrote:

 On Fri, 2007-03-30 at 12:28 +0530, NikhilS wrote:

  Please find attached the WIP version 1 of the auto partitioning patch.
  There was discussion on this a while back on -hackers at:
  http://archives.postgresql.org/pgsql-hackers/2007-03/msg00375.php
 
  Please note that this patch tries to automate the activities that
  currently are carried out manually. It does nothing fancy beyond that
  for now. There were a lot of good suggestions, I have noted them down
  but for now I have tried to stick to the initial goal of automating
  existing steps for providing partitioning.
 
  Things that this patch does:

 I think this patch is a reasonable first step and clearly written, but
 not yet ready for application to Postgres in this commit fest.

 I would say we need:

 * Clear explanation of the new syntax, with examples of each permutation
 so we can see how that would work. In light of recent discussions on
 -hackers we need to take a view on whether we should go with Gavin's
 suggested syntax or this syntax.

 * There are some additional syntax items I don't understand the need
 for. So these need to be explained.

 * I would be against using the term PARTITION BY since it is already a
 phrase that is part of the SQL Standard. Perhaps PARTITIONED BY?

 * We need regression tests for any new command syntax

 * No docs - that might be the same thing as the first item

 --


Thanks for taking a look. But if I am not mistaken Gavin and co. are working
on a much exhaustive proposal. In light of that maybe this patch might not
be needed in the first place?

I will wait for discussion and a subsequent collective consensus here,
before deciding the further course of actions.

Regards,
Nikhils
-- 
EnterpriseDB http://www.enterprisedb.com


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-21 Thread Bruce Momjian
NikhilS wrote:
  * Clear explanation of the new syntax, with examples of each permutation
  so we can see how that would work. In light of recent discussions on
  -hackers we need to take a view on whether we should go with Gavin's
  suggested syntax or this syntax.
 
  * There are some additional syntax items I don't understand the need
  for. So these need to be explained.
 
  * I would be against using the term PARTITION BY since it is already a
  phrase that is part of the SQL Standard. Perhaps PARTITIONED BY?
 
  * We need regression tests for any new command syntax
 
  * No docs - that might be the same thing as the first item
 
 Thanks for taking a look. But if I am not mistaken Gavin and co. are working
 on a much exhaustive proposal. In light of that maybe this patch might not
 be needed in the first place?
 
 I will wait for discussion and a subsequent collective consensus here,
 before deciding the further course of actions.

I think it is unwise to wait on Gavin for a more complex implemention
---  we might end up with nothing for 8.4.  As long as your syntax is
compatible with whatever Gavin proposed Gavin can add on to your patch
once it is applied.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-21 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 NikhilS wrote:
 Thanks for taking a look. But if I am not mistaken Gavin and co. are working
 on a much exhaustive proposal. In light of that maybe this patch might not
 be needed in the first place?
 
 I will wait for discussion and a subsequent collective consensus here,
 before deciding the further course of actions.

 I think it is unwise to wait on Gavin for a more complex implemention
 ---  we might end up with nothing for 8.4.  As long as your syntax is
 compatible with whatever Gavin proposed Gavin can add on to your patch
 once it is applied.

It would be equally unwise to apply a stopgap patch if we're not certain
it will be upward compatible with what we want to do later.

I haven't been through the partitioning threads at all yet, but I think
what we probably want to have when we emerge from commit fest is some
consensus on what the road map is for partitioning.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-21 Thread Simon Riggs
On Fri, 2008-03-21 at 20:15 +0530, NikhilS wrote:

 Thanks for taking a look. But if I am not mistaken Gavin and co. are
 working on a much exhaustive proposal. In light of that maybe this
 patch might not be needed in the first place? 

We should wait to apply, but not wait to discuss. Somebody must take the
initiative.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com 

  PostgreSQL UK 2008 Conference: http://www.postgresql.org.uk


-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-21 Thread Simon Riggs
On Fri, 2008-03-21 at 11:18 -0400, Bruce Momjian wrote:

 I think it is unwise to wait on Gavin for a more complex implemention
 ---  we might end up with nothing for 8.4.  As long as your syntax is
 compatible with whatever Gavin proposed Gavin can add on to your patch
 once it is applied.

The patch as stands does little apart from bring together many DDL
statements into one. Partitioning is much much more than this so there
seems little reason to think we should rush to commit this, especially
before we get some good docs that explain what it does, and why.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com 

  PostgreSQL UK 2008 Conference: http://www.postgresql.org.uk


-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-21 Thread NikhilS
Hi,

On Fri, Mar 21, 2008 at 9:23 PM, Tom Lane [EMAIL PROTECTED] wrote:

 Bruce Momjian [EMAIL PROTECTED] writes:
  NikhilS wrote:
  Thanks for taking a look. But if I am not mistaken Gavin and co. are
 working
  on a much exhaustive proposal. In light of that maybe this patch might
 not
  be needed in the first place?
 
  I will wait for discussion and a subsequent collective consensus here,
  before deciding the further course of actions.

  I think it is unwise to wait on Gavin for a more complex implemention
  ---  we might end up with nothing for 8.4.  As long as your syntax is
  compatible with whatever Gavin proposed Gavin can add on to your patch
  once it is applied.

 It would be equally unwise to apply a stopgap patch if we're not certain
 it will be upward compatible with what we want to do later.

 I haven't been through the partitioning threads at all yet, but I think
 what we probably want to have when we emerge from commit fest is some
 consensus on what the road map is for partitioning.


+2

Regards,
Nikhils
-- 
EnterpriseDB http://www.enterprisedb.com


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-03-21 Thread Luke Lonergan
Hi all,

I think the intent of the syntax / parser patch from Gavin and Jeff was to get 
consensus from PG on the syntax prior to proceeding with the next chunk of work.

The next chunk of work is now well underway - with support for ALTER TABLE 
and partitioning, along with fast inserts into the parent table.  This 
involves changes to the catalog, so we'll also need to discuss this as part of 
a submission.

GP is in the middle of merging 8.3 into our product, so it will be a few weeks 
at least before we can push any more info to the list.

Was there consensus on the syntax?  IIRC, there was a cessation of contrary 
comments on the matter.  If so, the parser patch was provided earlier - we 
could posibly refresh it.  The way it works in our dev branch now is that the 
partition syntax is turned off by default using a GUC, but is fully functional 
wrt creating rules, etc.  This allows for experimentation.

- Original Message -
From: [EMAIL PROTECTED] [EMAIL PROTECTED]
To: Tom Lane [EMAIL PROTECTED]
Cc: Bruce Momjian [EMAIL PROTECTED]; Simon Riggs [EMAIL PROTECTED]; 
pgsql-patches@postgresql.org pgsql-patches@postgresql.org
Sent: Sat Mar 22 01:19:01 2008
Subject: Re: [PATCHES] Auto Partitioning Patch - WIP version 1

Hi, 


On Fri, Mar 21, 2008 at 9:23 PM, Tom Lane [EMAIL PROTECTED] wrote:


Bruce Momjian [EMAIL PROTECTED] writes:

 NikhilS wrote:
 Thanks for taking a look. But if I am not mistaken Gavin and co. are 
working
 on a much exhaustive proposal. In light of that maybe this patch 
might not
 be needed in the first place?

 I will wait for discussion and a subsequent collective consensus 
here,
 before deciding the further course of actions.

 I think it is unwise to wait on Gavin for a more complex implemention
 ---  we might end up with nothing for 8.4.  As long as your syntax is
 compatible with whatever Gavin proposed Gavin can add on to your patch
 once it is applied.


It would be equally unwise to apply a stopgap patch if we're not certain
it will be upward compatible with what we want to do later.

I haven't been through the partitioning threads at all yet, but I think
what we probably want to have when we emerge from commit fest is some
consensus on what the road map is for partitioning.



+2

Regards,
Nikhils

-- 
EnterpriseDB http://www.enterprisedb.com 


Re: [PATCHES] Auto Partitioning Patch - WIP version 1

2007-04-26 Thread Bruce Momjian

This has been saved for the 8.4 release:

http://momjian.postgresql.org/cgi-bin/pgpatches_hold

---

NikhilS wrote:
 Hi,
 
 Please find attached the WIP version 1 of the auto partitioning patch. There
 was discussion on this a while back on -hackers at:
 http://archives.postgresql.org/pgsql-hackers/2007-03/msg00375.php
 
 Please note that this patch tries to automate the activities that currently
 are carried out manually. It does nothing fancy beyond that for now. There
 were a lot of good suggestions, I have noted them down but for now I have
 tried to stick to the initial goal of automating existing steps for
 providing partitioning.
 
 Things that this patch does:
 
 i) Handle new syntax to provide partitioning:
 
 CREATE TABLE tabname (
  ...
   ) PARTITION BY
RANGE(ColId)
  | LIST(ColId)
  (
  PARTITION partition_name CHECK(...),
  PARTITION partition_name CHECK(...)
   ...
 );
 
 ii) Create master table.
 iii) Create children tables based on the number of partitions specified and
 make them inherit from the master table.
 
 The following things are TODOs:
 
 iv) Auto generate rules using the checks mentioned for the partitions, to
 handle INSERTs/DELETEs/UPDATEs to navigate them to the appropriate child.
 Note that checks specified directly on the master table will get inherited
 automatically.
 v) Based on the PRIMARY, UNIQUE information specified, pass it on to the
 children tables.
 vi) [stretch goal] Support HASH partitions
 
 Will try to complete the above mentioned TODOs as soon as is possible.
 
 Comments, feedback appreciated.
 
 Thanks and Regards,
 Nikhils
 --
 
 EnterpriseDB   http://www.enterprisedb.com

[ Attachment, skipping... ]

 
 ---(end of broadcast)---
 TIP 6: explain analyze is your friend

-- 
  Bruce Momjian  [EMAIL PROTECTED]  http://momjian.us
  EnterpriseDB   http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


[PATCHES] Auto Partitioning Patch - WIP version 1

2007-03-30 Thread NikhilS

Hi,

Please find attached the WIP version 1 of the auto partitioning patch. There
was discussion on this a while back on -hackers at:
http://archives.postgresql.org/pgsql-hackers/2007-03/msg00375.php

Please note that this patch tries to automate the activities that currently
are carried out manually. It does nothing fancy beyond that for now. There
were a lot of good suggestions, I have noted them down but for now I have
tried to stick to the initial goal of automating existing steps for
providing partitioning.

Things that this patch does:

i) Handle new syntax to provide partitioning:

CREATE TABLE tabname (
...
 ) PARTITION BY
  RANGE(ColId)
| LIST(ColId)
(
PARTITION partition_name CHECK(...),
PARTITION partition_name CHECK(...)
 ...
);

ii) Create master table.
iii) Create children tables based on the number of partitions specified and
make them inherit from the master table.

The following things are TODOs:

iv) Auto generate rules using the checks mentioned for the partitions, to
handle INSERTs/DELETEs/UPDATEs to navigate them to the appropriate child.
Note that checks specified directly on the master table will get inherited
automatically.
v) Based on the PRIMARY, UNIQUE information specified, pass it on to the
children tables.
vi) [stretch goal] Support HASH partitions

Will try to complete the above mentioned TODOs as soon as is possible.

Comments, feedback appreciated.

Thanks and Regards,
Nikhils
--

EnterpriseDB   http://www.enterprisedb.com
Index: src/backend/commands/tablecmds.c
===
RCS file: /repositories/postgreshome/cvs/pgsql/src/backend/commands/tablecmds.c,v
retrieving revision 1.218
diff -c -r1.218 tablecmds.c
*** src/backend/commands/tablecmds.c	19 Mar 2007 23:38:29 -	1.218
--- src/backend/commands/tablecmds.c	30 Mar 2007 06:31:37 -
***
*** 6864,6866 
--- 6864,6945 
  		}
  	}
  }
+ 
+ /* 
+  * --
+  *		DefinePartitions
+  *		Create new partitions. They end up inheriting from the parent
+  *		relation.
+  *		Once they have been created, rules need to be assigned to the parent to
+  *		provide the UPDATEs/INSERTs/DELETEs to percolate down to the children
+  * 		Callers expect this function to end with CommandCounterIncrement if it
+  * 		makes any changes.
+  * --
+  */
+ void
+ DefinePartitions(CreateStmt *stmt)
+ {
+ 	CreateStmt		*childStmt;
+ 	RangeVar 		inr;
+ 	Oid 			childOid;
+ 	PartitionAttrs	*partAttr;
+ 
+ 	if (stmt-partAttr == NULL) 
+ 		return;
+ 
+ 	Assert(IsA(stmt-partAttr, PartitionAttrs));
+ 	partAttr = (PartitionAttrs *)(stmt-partAttr);
+ 
+ 	/*
+ 	 * All the partitions will inherit from the parent, set the parent in the
+ 	 * inhRelations structure
+ 	 */
+ 	inr = *stmt-relation;
+ 
+ 
+ 	/* 
+ 	 * Create the children tables. The parser has already made sure that we
+ 	 * have atleast one partition in the list
+ 	 */
+ 	if (partAttr-partFunc == PART_LIST || partAttr-partFunc == PART_RANGE)
+ 	{
+ 		List *partitionList = partAttr-partitions;
+ 		ListCell *temp_part; 
+ 
+ 		Assert(list_length(partitionList)  0);
+ 		if (list_length(partitionList)  0) 
+ 		{
+ 			foreach(temp_part, partitionList)
+ 			{
+ Partition *temp_partition = lfirst(temp_part);
+ /*
+  * Create a working copy for each child
+  */
+ childStmt = (CreateStmt *)copyObject((void *)stmt);
+ 
+ /*
+  * Child has to use all columns from the parent, otherwise we will get
+  * unnecessary merging columns notices as part of the
+  * DefineRelation 
+  */
+ childStmt-tableElts = NIL;
+ 
+ childStmt-inhRelations = lappend(NULL, inr);
+ childStmt-relation-relname = temp_partition-partName-relname;
+ childOid = DefineRelation(childStmt, RELKIND_RELATION);
+ 			}
+ 		}
+ 	}
+ 	else
+ 		ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TABLE_DEFINITION),
+  errmsg(Invalid PARTITION type specified)));
+ 
+ 	/*
+ 	 *TODO: Add logic to create rules on the parent table
+ 	 */
+ 	/*
+ 	 * Make the changes carried out so far visible
+ 	 */
+ 	CommandCounterIncrement();
+ }
Index: src/backend/nodes/copyfuncs.c
===
RCS file: /repositories/postgreshome/cvs/pgsql/src/backend/nodes/copyfuncs.c,v
retrieving revision 1.372
diff -c -r1.372 copyfuncs.c
*** src/backend/nodes/copyfuncs.c	27 Mar 2007 23:21:09 -	1.372
--- src/backend/nodes/copyfuncs.c	30 Mar 2007 06:31:37 -
***
*** 2070,2075 
--- 2070,2078 
  	COPY_NODE_FIELD(options);
  	COPY_SCALAR_FIELD(oncommit);
  	COPY_STRING_FIELD(tablespacename);
+ 	/*
+ 	 * There is no need to copy partAttr as of now
+ 	 */
  
  	return newnode;
  }
Index: src/backend/parser/analyze.c
===
RCS file: