Hannu Krosing wrote:
After triggers can't change tuple, thus cant change routing.
An after trigger can always issue an update of the tuple but that should
be trapped by the regular mechanism that will deal with updates (when we
have it available).
Also the description omits the execution of
On Tue, 2009-11-24 at 10:08 -0500, Emmanuel Cecchet wrote:
Itagaki Takahiro wrote:
I just edited a wiki page for this discussion.
I hope it can be a help.
http://wiki.postgresql.org/wiki/Table_partitioning
I guess the problem of handling user triggers is still open.
If we allow
Hannu Krosing wrote:
On Tue, 2009-11-24 at 10:08 -0500, Emmanuel Cecchet wrote:
Itagaki Takahiro wrote:
I just edited a wiki page for this discussion.
I hope it can be a help.
http://wiki.postgresql.org/wiki/Table_partitioning
I guess the problem of handling user triggers is
On Wed, Nov 25, 2009 at 5:03 AM, Hannu Krosing ha...@2ndquadrant.com wrote:
I'd propose that triggers on both parent table and selected child are
executed.
I was thinking we should make the partitioning decision FIRST, before
any triggers are fired, and then fire only those triggers relevant to
Robert Haas wrote:
On Wed, Nov 25, 2009 at 5:03 AM, Hannu Krosing ha...@2ndquadrant.com wrote:
I'd propose that triggers on both parent table and selected child are
executed.
I was thinking we should make the partitioning decision FIRST, before
any triggers are fired, and then fire
On Wed, Nov 25, 2009 at 9:21 AM, Emmanuel Cecchet m...@asterdata.com wrote:
Robert Haas wrote:
On Wed, Nov 25, 2009 at 5:03 AM, Hannu Krosing ha...@2ndquadrant.com
wrote:
I'd propose that triggers on both parent table and selected child are
executed.
I was thinking we should make the
It seems like the easiest way to resolve this without weird corner
cases is to say that we fire triggers belonging to the parent table.
The individual partition child tables either shouldn't have triggers
at all, or we should restrict the cases in which those are considered
applicable.
As an
On Wed, 2009-11-25 at 08:39 -0500, Emmanuel Cecchet wrote:
Hannu Krosing wrote:
On Tue, 2009-11-24 at 10:08 -0500, Emmanuel Cecchet wrote:
Itagaki Takahiro wrote:
I just edited a wiki page for this discussion.
I hope it can be a help.
On Wed, 2009-11-25 at 11:30 -0500, Tom Lane wrote:
It seems like the easiest way to resolve this without weird corner
cases is to say that we fire triggers belonging to the parent table.
The individual partition child tables either shouldn't have triggers
at all, or we should restrict the
On Wed, Nov 25, 2009 at 11:30 AM, Tom Lane t...@sss.pgh.pa.us wrote:
It seems like the easiest way to resolve this without weird corner
cases is to say that we fire triggers belonging to the parent table.
The individual partition child tables either shouldn't have triggers
at all, or we should
On Mon, 2009-11-23 at 10:24 -0500, Emmanuel Cecchet wrote:
But it looks like it is a waste of everybody's time to continue this
discussion further. Just move the patch to the rejected patches and
let's wait for Itagaki's implementation.
Emmanuel, please try to work together with Itagaki san
Hannu Krosing ha...@2ndquadrant.com wrote:
Even though this patch will not get in, most of the effort in developing
it is not actual coding, but familiarizing yourself with the other code
involved.
I just edited a wiki page for this discussion.
I hope it can be a help.
On Tue, 2009-11-24 at 17:30 +0900, Itagaki Takahiro wrote:
Hannu Krosing ha...@2ndquadrant.com wrote:
Even though this patch will not get in, most of the effort in developing
it is not actual coding, but familiarizing yourself with the other code
involved.
I just edited a wiki page
Itagaki Takahiro wrote:
I just edited a wiki page for this discussion.
I hope it can be a help.
http://wiki.postgresql.org/wiki/Table_partitioning
I guess the problem of handling user triggers is still open.
If we allow triggers on partitions, badly written logic could lead to
infinite
Emmanuel Cecchet m...@asterdata.com wrote:
I guess the problem of handling user triggers is still open.
If we allow triggers on partitions, badly written logic could lead to
infinite loops in routing.
Infinite loops are not a partition-related problem, no?
We can also find infinite loops in
Itagaki Takahiro wrote:
Emmanuel Cecchet m...@asterdata.com wrote:
I guess the problem of handling user triggers is still open.
If we allow triggers on partitions, badly written logic could lead to
infinite loops in routing.
Infinite loops are not a partition-related problem, no?
We
On Wed, 2009-11-11 at 19:53 -0500, Emmanuel Cecchet wrote:
Hi,
I have extracted the partitioning option for COPY (removed the error
logging part) from the previous patch.
We can use an INSERT trigger to route tuples into partitions even now.
Why do you need an additional router
Simon,
I think you should read the thread and the patch before making any false
statements like you did in your email.
1. The patch does not use any trigger for routing.
2. This is just an option for COPY that is useful for loading operations
in the datawarehouse world. It is not meant to
On Mon, Nov 23, 2009 at 9:39 AM, Emmanuel Cecchet m...@asterdata.com wrote:
I think you should read the thread and the patch before making any false
statements like you did in your email.
1. The patch does not use any trigger for routing.
Whoa, whoa! I don't think that Simon said that it
Robert Haas wrote:
On Mon, Nov 23, 2009 at 9:39 AM, Emmanuel Cecchet m...@asterdata.com wrote:
I think you should read the thread and the patch before making any false
statements like you did in your email.
1. The patch does not use any trigger for routing.
Whoa, whoa! I don't think
On Mon, 2009-11-23 at 09:39 -0500, Emmanuel Cecchet wrote:
I think you should read the thread and the patch
I did read the thread and patch in full before posting. My opinions are
given to help you and the community towards a desirable common goal.
I was unaware you were developing these
On Mon, 2009-11-23 at 09:39 -0500, Emmanuel Cecchet wrote:
I think you should read the thread and the patch
I did read the thread and patch in full before posting. My opinions are
given to help you and the community towards a desirable common goal.
I was unaware you were developing these
Simon Riggs wrote:
I was unaware you were developing these ideas and so was unable to
provide comments until now.
The first patch was published to this list on September 10 (almost 2.5
months ago) along with the wiki page describing the problem and the
solution.
What should I have done to
On Mon, 2009-11-23 at 10:24 -0500, Emmanuel Cecchet wrote:
I think there is a misunderstanding between what Simon wants
('Anyway, I want data routing, as is the intention of this patch.') and
what this patch is about. This patch is just supposed to load tuples in
a hierarchy of tables as
On Mon, 2009-11-23 at 10:43 -0500, Emmanuel Cecchet wrote:
Simon Riggs wrote:
I was unaware you were developing these ideas and so was unable to
provide comments until now.
The first patch was published to this list on September 10 (almost 2.5
months ago) along with the wiki page
Simon Riggs si...@2ndquadrant.com writes:
Anyway, I want data routing, as is the intention of this patch. I just
don't think this patch is a useful way to do it. It is too narrow in its
scope and potentially buggy in its approach to developing a cache and
using trigger-like stuff.
FWIW, I
Simon Riggs wrote:
...Read my detailed comments in response to Kedar's patch and post
comments on that thread to say you didn't agree with that proposal and
that you were thinking of another way entirely.
Useful background here is:
http://wiki.postgresql.org/wiki/Table_partitioning
On Mon, 2009-11-23 at 12:23 -0500, Greg Smith wrote:
What would probably be helpful here is to take the mess of raw data
above and turn it into a simpler partitioning roadmap.
Thanks for summarising.
I briefly tried to do that on the thread for Itagaki-san's patch. That's
a first stab at
Tom Lane t...@sss.pgh.pa.us wrote:
What we need first is an explicit representation of partitioning, and
then to build routing code on top of that. I haven't looked at
Itagaki-san's syntax patch at all, but I think it's at least starting
in a sensible place.
I have the following
Hi,
What would probably be helpful here is to take the mess of raw data
above and turn it into a simpler partitioning roadmap.
Thanks for summarising.
Yeah, excellent summary Greg. As you rightly pointed out, partitioning
needs a broad roadmap so that the community can contribute in unison.
Emmanuel Cecchet wrote:
Jan,
A couple of nitpicks first:
o) the route_tuple_to_child recurses to child tables of child tables,
which is undocumented and requires a check_stack_depth() call if it's
really desirable
The recursive call is as deep as the inheritance hierarchy. I am not
On Sun, 22 Nov 2009, Emmanuel Cecchet wrote:
As I explained to Tom, if the after row trigger is called asynchronously
I get a relcache leak on the child table at the end of the copy
operation. If the trigger is called synchronously (like a before row
trigger) it works fine. Also calling the
Stephan Szabo wrote:
On Sun, 22 Nov 2009, Emmanuel Cecchet wrote:
As I explained to Tom, if the after row trigger is called asynchronously
I get a relcache leak on the child table at the end of the copy
operation. If the trigger is called synchronously (like a before row
trigger) it works
On Sun, 22 Nov 2009, Emmanuel Cecchet wrote:
Stephan Szabo wrote:
On Sun, 22 Nov 2009, Emmanuel Cecchet wrote:
As I explained to Tom, if the after row trigger is called asynchronously
I get a relcache leak on the child table at the end of the copy
operation. If the trigger is called
On Sun, Nov 22, 2009 at 12:35 PM, Jan Urbański wulc...@wulczer.org wrote:
I was thinking more about SGML docs. They could mention that BEFORE
triggers are fired both for the parent table and for the child table,
while AFTER triggers will only be called on the target table. I'd add a
sentence
Emmanuel Cecchet wrote:
Hi Jan,
Here is the updated patch.
Note that the new code in trigger is a copy/paste of the before row
insert trigger code modified to use the pointers of the after row
trigger functions.
Hi,
ok, this version applied, compiled and ran the regression tests fine. I
Jan Urbański wrote:
o) my main concern is still valid: the design was never agreed upon.
The approach of using inheritance info for automatic partitioning is, at
least IMHO, too restricted. Regular INSERTs won't get routed to child
tables. Data from writable CTEs won't get routed. People
Jan Urbański wrote:
o) my main concern is still valid: the design was never agreed upon.
The approach of using inheritance info for automatic partitioning is, at
least IMHO, too restricted. Regular INSERTs won't get routed to child
tables. Data from writable CTEs won't get routed. People
Emmanuel Cecchet wrote:
Hi Jan,
Here is a new version of the patch with the following modifications:
- used oid list from pg_list.h
- properly handles triggers and generate an error if needed (updated doc
as well)
- added your test cases + extra bad trigger cases
Hi,
that got broken by
=?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= wulc...@wulczer.org writes:
that got broken by the WHEN triggers patch
(c6e0a36243a54eff79b47b3a0cb119fb67a55165), which changed the
TriggerEnabled function signature, the code currently does not compile.
[ squint... ] What is that patch doing touching the
Tom Lane wrote:
=?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= wulc...@wulczer.org writes:
that got broken by the WHEN triggers patch
(c6e0a36243a54eff79b47b3a0cb119fb67a55165), which changed the
TriggerEnabled function signature, the code currently does not compile.
[ squint... ] What is that
Tom Lane wrote:
=?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= wulc...@wulczer.org writes:
Program received signal SIGSEGV, Segmentation fault.
0x0819368b in route_tuple_to_child (parent_relation=0xb5d93040,
tuple=0x873b08c, hi_options=0, parentResultRelInfo=0x871e204) at copy.c:1821
1821
Emmanuel Cecchet m...@asterdata.com writes:
Tom Lane wrote:
This looks like the patch is trying to create a data structure in a
memory context that's not sufficiently long-lived for the use of the
structure. If you do this in a non-cassert build, it will seem to
work, some of the time, if
Tom Lane wrote:
Emmanuel Cecchet m...@asterdata.com writes:
Tom Lane wrote:
This looks like the patch is trying to create a data structure in a
memory context that's not sufficiently long-lived for the use of the
structure. If you do this in a non-cassert build, it will seem to
work,
Emmanuel Cecchet m...@asterdata.com writes:
Actually the list is supposed to stay around between statement
executions. You don't want to restart with a cold cache at every
statement so I really want this structure to stay in memory at a more
global level.
Cache? Why do you need a cache
Tom Lane wrote:
Emmanuel Cecchet m...@asterdata.com writes:
Actually the list is supposed to stay around between statement
executions. You don't want to restart with a cold cache at every
statement so I really want this structure to stay in memory at a more
global level.
Cache? Why
Emmanuel Cecchet m...@asterdata.com writes:
Tom Lane wrote:
Cache? Why do you need a cache for COPY?
Actually the cache is only activated if you use the partitioning option.
It is just a list of oids of child tables where tuples were inserted.
Umm ... why is that useful enough to be
Hi,
I'll hopefully look at the next version of the patch tommorrow.
Emmanuel Cecchet wrote:
o test1.sql always segfaults for me, poking around with gdb suggests
it's a case of an uninitialised cache list (another reason to use the
builtin one).
I was never able to reproduce that
=?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= wulc...@wulczer.org writes:
Program received signal SIGSEGV, Segmentation fault.
0x0819368b in route_tuple_to_child (parent_relation=0xb5d93040,
tuple=0x873b08c, hi_options=0, parentResultRelInfo=0x871e204) at copy.c:1821
1821
Tom Lane wrote:
A good rule of thumb is to never do code development in a non-cassert
build.
And the same rule goes for review, too; I'll update the review
guidelines to spell that out more clearly. Basically, if you're doing
any work on new code, you should have cassert turned on, *except*
Emmanuel Cecchet wrote:
Hi all,
Hi!,
partitioning option for COPY
Here's the review:
== Submission ==
The patch is contextual, applies cleanly to current HEAD, compiles fine.
The docs build cleanly.
== Docs ==
They're reasonably clear, although they still mention ERROR_LOGGING,
which was
Jan Urbański wrote:
Emmanuel Cecchet wrote:
Hi all,
Hi!,
partitioning option for COPY
Attached are 3 files that demonstrate problems the patch has.
And the click-before-you-think prize winner is... me.
Test cases attached, see the comments for expected/actual results.
Jan
--
Emmanuel Cecchet m...@asterdata.com wrote:
I have extracted the partitioning option for COPY (removed the error
logging part) from the previous patch.
We can use an INSERT trigger to route tuples into partitions even now.
Why do you need an additional router for COPY? Also, it would be
Hi,
I have extracted the partitioning option for COPY (removed the error
logging part) from the previous patch.
We can use an INSERT trigger to route tuples into partitions even now.
Why do you need an additional router for COPY?
Tom has already explained on the list why using a trigger
Emmanuel Cecchet m...@asterdata.com wrote:
If you look at the code you will see that you can do optimizations in
the COPY code that you cannot do in the trigger.
Since the optimizations is nice, I hope it will work not only in COPY
but also in INSERT. An idea is moving the partitioning cache
We can use an INSERT trigger to route tuples into partitions even now.
Why do you need an additional router for COPY? Also, it would be nicer
that the router can works not only in COPY but also in INSERT.
Yeah, but performance on an insert trigger is impractical for large
volumes of data.
56 matches
Mail list logo
