[HACKERS] Update obsolete text in indexam.sgml

2012-11-04 Thread Etsuro Fujita
ISTM it would be better to update the text about index cost estimation in
indexam.sgml.  Please find attached a patch.

Thanks,

Best regards,
Etsuro Fujita


indexam.sgml.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH]Tablesample Submission

2012-11-04 Thread Qi Huang
Dear hackers Sorry for not replying the patch review. I didn't see the 
review until recently as my mail box is full of Postgres mails and I didn't 
notice the one for me, my mail box configuration problem. I am still kind 
of busy with my university final year project. I shall not have time to work on 
updating the patch until this semester finishes which is December. I will work 
on then.Thanks.
Best RegardsHuang Qi VictorComputer Science of National University of Singapore

From: huangq...@outlook.com
To: robertmh...@gmail.com
CC: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] [PATCH]Tablesample Submission
Date: Tue, 21 Aug 2012 23:08:41 +0800




> Please add your patch here:
> 
> https://commitfest.postgresql.org/action/commitfest_view/open
> 
> -- 
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company

Hi, Robert
I added it under "Miscellaneous".
https://commitfest.postgresql.org/action/patch_view?id=918

Best RegardsHuang Qi VictorComputer Science of National University of Singapore 

  

Re: [HACKERS] Synchronous commit not... synchronous?

2012-11-04 Thread Peter van Hardenberg
On Fri, Nov 2, 2012 at 11:16 AM, Peter Eisentraut wrote:

>
> Did the inserted row also arrive at the standby?
>

No, as there was no standby.

-- 
Peter van Hardenberg
San Francisco, California
"Everything was beautiful, and nothing hurt." -- Kurt Vonnegut


Re: [HACKERS] Synchronous commit not... synchronous?

2012-11-04 Thread Robert Haas
On Sat, Nov 3, 2012 at 5:44 PM, Florian Weimer  wrote:
> * Daniel Farina:
>> The idea of canceling a COMMIT statement causing a COMMIT seems pretty
>> strange to me.
>
> Canceling commits is inherently racy, so I'm not sure if this behavior
> so strange after all.

Yeah.  You can't make the local fsync() and the remote fsync() happen
at exactly the same moment in time.  No implementation can do that,
anywhere, ever.  Our implementation happens to require the local
fsync() to always be done first.  With sufficient smarts, that could
possibly be relaxed to allow them to run concurrently (if after a
crash the master tried to fetch remotely committed WAL from its
slave), but you still have the possibility that one fsync gets done
and the other does not, and that would actually allow more weird edge
cases than we have now - the main argument for it is performance.  And
because of the way our WAL stream works, once the commit record is
fsync'd, it's too late to get cold feet.  I daresay that will be true
in any system, too.  Once the local commit record hits the disk, the
absolute best you could do, even theoretically, is *try* to take it
back.  And you had better be prepared for that to fail, because the
system might crash or the take-back write might itself fail.

Atomicity in a distributed system is not an easy problem.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Bug in ALTER COLUMN SET DATA TYPE ?

2012-11-04 Thread Tom Lane
Pavan Deolasee  writes:
> On Sat, Nov 3, 2012 at 10:21 AM, Nikhil Sontakke  wrote:
 So coming back to the issue, do you think it's a good idea to teach
 ATAddCheckConstraint() that the call is coming from a late phase of ALTER
 TABLE ?

>>> +1

>> You mentioned AT_PASS_OLD_INDEX in your original mail, but I guess that
>> you meant that we should check for  AT_PASS_OLD_CONSTR and then not raise
>> an error?

> Yeah, I meant  AT_PASS_OLD_CONSTR.

It's clear that we need to pass down the information that this action is
coming from re-creation of a check constraint, but I think the above
proposal for how to do it is pretty wrong-headed.  The pass structure
has never meant anything for semantics of individual AlterTable actions;
it's only used to make sure those actions are done in an appropriate
order.  Furthermore, the pass number isn't available where it would be
needed --- you'd have to pass it down through several levels of
subroutine that don't have another reason to care about it.

I'm inclined to think the cleanest solution is to add another value of
enum AlterTableType, perhaps "AT_ReAddConstraint", to signal that we're
executing a re-add; and then add another bool parameter to
ATExecAddConstraint to tell the latter not to complain if child tables
exist.  This is more in line with pre-existing coding choices such as
the use of AT_AddConstraintRecurse.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Arguments to foreign tables?

2012-11-04 Thread Willem Leenen


> 
> what does that even mean?how would 'data' accept 'arguments' ??!
> 
> 

That's Sherlock Holmes!  "Don't argue with your data"


  

Re: [HACKERS] Arguments to foreign tables?

2012-11-04 Thread John R Pierce

On 11/04/12 11:59 AM, Jeff Davis wrote:

Is there any fundamental or philosophical reason why a foreign table
can't accept arguments? Should that be a TODO?


what does that even mean?how would 'data' accept 'arguments' ??!



--
john r pierceN 37, W 122
santa cruz ca mid-left coast



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Arguments to foreign tables?

2012-11-04 Thread Willem Leenen


I don't understand the question. A table accepting arguments? What do you mean 
with that? Can you give an code example in an other RDBMS (like oracle)?

Perhaps nothing came up because there are more people not understanding what 
you want to accomplish?

regards,
Willem

> Subject: [HACKERS] Arguments to foreign tables?
> From: pg...@j-davis.com
> To: pgsql-hackers@postgresql.org
> Date: Sun, 4 Nov 2012 11:59:20 -0800
> 
> Is there any fundamental or philosophical reason why a foreign table
> can't accept arguments? Should that be a TODO?
> 
> Right now, to accept arguments to a from-clause item, you have to use an
> SRF, which is much more limited than a foreign table. If foreign tables
> could accept arguments, then SRFs could just be a simplified special
> case of foreign tables.
> 
> I looked for some previous discussions on this topic and nothing turned
> up.
> 
> Regards,
>   Jeff Davis
> 
> 
> 
> -- 
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
  

Re: [HACKERS] Deparsing DDL command strings

2012-11-04 Thread Dimitri Fontaine
Dimitri Fontaine  writes:
> The current design for event triggers is to spit out several things:
>
>  - command tag  is already commited
>  - object id, can be null
>  - schema name, can be null
>  - object name
>  - operationeither ALTER, CREATE or DROP, …
>  - object type  TABLE, VIEW, FUNCTION, …
>  - normalized command string
>
> After some more thinking, it appears that in several case you want to
> have all those information filled in and you don't want to care if that
> means your trigger needs to run at ddl_command_start or ddl_command_end.
>
> The proposal I want to make here is to introduce a generic event (or an
> event alias) named ddl_command_trace that the system provides at the
> right spot where you have the information. That's useful when you don't
> intend to divert the execution of the DDL and need to know all about it.

Please find attached a patch that includes support for such a design.

> For a DROP operation, ddl_command_trace would be ddl_command_start, and
> for a CREATE operation, that would be ddl_command_end, so that the
> target object (still|already) exists when the trigger is fired.

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support



event_trigger_infos.1.patch.gz
Description: Added Information for Event Triggers

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Arguments to foreign tables?

2012-11-04 Thread Tom Lane
Jeff Davis  writes:
> Is there any fundamental or philosophical reason why a foreign table
> can't accept arguments?

That isn't a table; it's some sort of function.  Now that we have
LATERAL, there is no good reason to contort SQL's syntax and semantics
in the direction you suggest.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Arguments to foreign tables?

2012-11-04 Thread Jeff Davis
Is there any fundamental or philosophical reason why a foreign table
can't accept arguments? Should that be a TODO?

Right now, to accept arguments to a from-clause item, you have to use an
SRF, which is much more limited than a foreign table. If foreign tables
could accept arguments, then SRFs could just be a simplified special
case of foreign tables.

I looked for some previous discussions on this topic and nothing turned
up.

Regards,
Jeff Davis



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

2012-11-04 Thread Jeff Davis
On Fri, 2012-11-02 at 12:47 +0400, Alexander Korotkov wrote:

> Right version of patch is attached.
> 
* In bounds_adjacent, there's no reason to flip the labels back.
* Comment should indicate more explicitly that bounds_adjacent is
sensitive to the argument order.
* In bounds_adjacent, it appears that "bound1" corresponds to "B" in the
comment above, and "bound2" corresponds to "A" in the comment above. I
would have guessed from reading the comment that bound1 corresponded to
A. We should just use consistent names between the comment and the code
(e.g. boundA and boundB).
* I could be getting confused, but I think that line 645 of
rangetypes_spgist.c should be inverted (!bounds_adjacent(...)).
* I think needPrevious should have an explanatory comment somewhere. It
looks like you are using it to store some state as you descend the tree,
but it doesn't look like it's used to reconstruct the value (because we
already have the value anyway). Since it's being used for a purpose
other than what's intended, that should be explained.

Regards,
Jeff Davis




-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-04 Thread Tom Lane
Matthew Gerber  writes:
>> Here is the command that was executing when the 0xC409 exception was
>> raised:
>> INSERT INTO places (bounding_box,country,full_name,id,name,type,url)
>> VALUES
>> (st_transform_null(ST_GeometryFromText('POLYGON((-97.034085
>> 32.771786,-97.034085 32.953966,-96.888789 32.953966,-96.888789
>> 32.771786,-97.034085 32.771786))',4326),26918),'United States','Irving,
>> TX','dce44ec49eb788f5','Irving','city','
>> http://api.twitter.com/1/geo/id/dce44ec49eb788f5.json'),

Assuming that 0xC409 does actually represent a stack-overrun
condition, it doesn't seem like there are very many ways that could
happen on such a simple command.  The best theory that is coming to mind
is that you hit some corner case in the GiST code that is causing the
index-entry-insertion attempt to recurse infinitely, or at least enough
times to hit the stack limit.  (I exclude the theory of infinite
recursion in the btree indexes only on the grounds that those are so
much better tested than GiST that the idea seems improbable.)  It's not
clear yet whether the bug is in the generic GiST code or in the
PostGIS-provided index operator class.

If that is the explanation, then hitting the crash would likely depend
not only on the specific bounding_box value being inserted, but also on
the previous state of the places_bounding_box_idx index, which could
make it darn hard to reproduce.  If you can't easily create a reproducer
script, I think we'll have to ask you for a stack trace from the crash.
See
https://wiki.postgresql.org/wiki/Generating_a_stack_trace_of_a_PostgreSQL_backend

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-04 Thread Matthew Gerber
On Sun, Nov 4, 2012 at 3:39 AM, Craig Ringer  wrote:

>  On 11/04/2012 08:47 AM, Matthew Gerber wrote:
>
>
>  Here is the command that was executing when the 0xC409 exception was
> raised:
>
> INSERT INTO places (bounding_box,country,full_name,id,name,type,url)
>
> VALUES
>
> (st_transform_null(ST_GeometryFromText('POLYGON((-97.034085
> 32.771786,-97.034085 32.953966,-96.888789 32.953966,-96.888789
> 32.771786,-97.034085 32.771786))',4326),26918),'United States','Irving,
> TX','dce44ec49eb788f5','Irving','city','
> http://api.twitter.com/1/geo/id/dce44ec49eb788f5.json'),
>
>
> OK, so you're using PostGIS. What other extensions are loaded? PL/R? Any
> other PLs?
>

PostGIS is the only extension that I added to the default configuration. I
didn't change anything else.


>
> Can you show the definition of the table `places`, incuding any associated
> triggers, etc? Use `\d+` in psql for the table def'n.
>

Here the definition of the places table:

twitter=# \d+ places
   Table "public.places"
Column|  Type   | Modifiers | Storage  | Stats
target |Description
--+-+---+--+--+-
 bounding_box | geometry(Polygon,26918) |   | main
|  |
 country  | character varying   |   | extended
|  |
 full_name| character varying   |   | extended
|  |
 id   | character varying   | not null  | extended
|  |
 name | character varying   |   | extended
|  |
 type | character varying   |   | extended
|  |
 url  | character varying   |   | extended
|  |

Indexes:
"places_pkey" PRIMARY KEY, btree (id)
"places_bounding_box_idx" gist (bounding_box)
"places_type_idx" btree (type)
Referenced by:
TABLE "tweets" CONSTRAINT "tweets_place_id_fkey" FOREIGN KEY (place_id)
REFERENCES places(id) ON DELETE CASCADE
Has OIDs: no


>
> Please also post the query plan. http://explain.depesz.com/ is useful for
> this.
>

Here is the query plan:

QUERY PLAN

Insert on public.places  (cost=0.00..0.01 rows=1 width=0) (actual
time=1.000..1.000 rows=0 loops=1)
   ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.001..0.001
rows=1 loops=1)
 Output:
'0103202669010005005E8705A4E32C38C1EE553AE6E95E
4D41086A91990B1B38C11620AF9784874D41FCA5741676E437C19436654287814D41C43E11332BF6
37C17C863746F0584D415E8705A4E32C38C1EE553AE6E95E4D41'::geometry(Polygon,26918),
'United States'::character varying, 'Irving, TX'::character varying,
'dce44ec49e
b788f5'::character varying, 'Irving'::character varying, 'city'::character
varyi
ng, 'http://api.twitter.com/1/geo/id/dce44ec49eb788f5.json'::charactervarying
 Total runtime: 1.157 ms
(4 rows)


>
>
> So I attached the VS debugger, but the server died without raising an
> exception in VS. Not sure what's going on here.
>
>>
>>   Try creating a directory called "crashdumps" in the data directory, at
> the same level as "pg_xlog" and "pg_clog" etc. Give the "postgresql" user
> the "full control" permission on it. Then run the test again.
>

Running it now.


>
> Do any minidump files appear in the directory? If so, you can examine them
> with windbg or Visual Studio to see where the crash happened.
>

Will try it.

Thanks for your help so far, guys. Hopefully we get somewhere on this...

Matt


[HACKERS] proposal: fix corner use case of variadic fuctions usage

2012-11-04 Thread Pavel Stehule
Hello

here is patch, that enables using a variadic parameter modifier for
variadic "any" function.

Motivation for this patch is consistent behave of "format" function,
but it fixes behave of all this class variadic functions.

postgres=> -- check pass variadic argument
postgres=> select format('%s, %s', variadic array['Hello','World']);
format
──
 Hello, World
(1 row)

postgres=> -- multidimensional array is supported
postgres=> select format('%s, %s', variadic
array[['Nazdar','Svete'],['Hello','World']]);
format
───
 {Nazdar,Svete}, {Hello,World}
(1 row)

It respect Tom's comments - it is based on short-lived FmgrInfo
structures, that are created immediately before function invocation.

Note: there is unsolved issue - reusing transformed arguments - so it
is little bit suboptimal for VARIADIC RETURNING MultiFuncCall
functions, where we don't need to repeat a unpacking of array value.

Regards

Pavel


variadic_argument_for_variadic_any_function.diff
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-04 Thread Craig Ringer
On 11/04/2012 08:47 AM, Matthew Gerber wrote:
>
> Here is the command that was executing when the 0xC409 exception
> was raised:
>
> INSERT INTO places (bounding_box,country,full_name,id,name,type,url)
>
> VALUES
>
> (st_transform_null(ST_GeometryFromText('POLYGON((-97.034085
> 32.771786,-97.034085 32.953966,-96.888789 32.953966,-96.888789
> 32.771786,-97.034085 32.771786))',4326),26918),'United
> States','Irving,
> TX','dce44ec49eb788f5','Irving','city','http://api.twitter.com/1/geo/id/dce44ec49eb788f5.json'
> ),

OK, so you're using PostGIS. What other extensions are loaded? PL/R? Any
other PLs?

Can you show the definition of the table `places`, incuding any
associated triggers, etc? Use `\d+` in psql for the table def'n.

Please also post the query plan. http://explain.depesz.com/ is useful
for this.
>
> So I attached the VS debugger, but the server died without raising an
> exception in VS. Not sure what's going on here.
>
>
Try creating a directory called "crashdumps" in the data directory, at
the same level as "pg_xlog" and "pg_clog" etc. Give the "postgresql"
user the "full control" permission on it. Then run the test again.

Do any minidump files appear in the directory? If so, you can examine
them with windbg or Visual Studio to see where the crash happened.

--
Craig Ringer


Re: [HACKERS] RFC: Timing Events

2012-11-04 Thread Pavel Stehule
Hello

2012/11/4 Satoshi Nagayasu :
> (2012/11/03 10:44), Josh Berkus wrote:
>>
>>
>>> I don't see all that going into core without a much bigger push than I
>>> think people will buy.  What people really want for all these is a
>>> proper trending system, and that means graphs and dashboards and
>>> bling--not a history table.
>>
>>
>> Well, I'm particularly thinking for autoconfiguration.  For example, to
>> set vacuum_freeze_min_age properly, you have to know the XID "burn rate"
>> of the server, which is only available via history.  I really don't want
>> to be depending on a graphical monitoring utility to find these things
>> out.
>>
>>> This whole approach has the assumption that things are going to fall off
>>> sometimes.  To expand on that theme for a second, right now I'm more
>>> worried about the "99%" class of problems.  Neither pg_stat_statements
>>> nor this idea are very good for tracking the rare rogue problem down.
>>> They're both aimed to make things that happen a lot more statistically
>>> likely to be seen, by giving an easier UI to glare at them frequently.
>>> That's not ideal, but I suspect really fleshing the whole queue consumer
>>> -> table idea needs to happen to do much better.
>>
>>
>> I'm just concerned that for some types of incidents, it would be much
>> more than 1% *of what you want to look at* which fall off.  For example,
>> consider a server which does 95% reads at a very high rate, but has 2%
>> of its writes cronically having lock waits.  That's something you want
>> to solve, but it seems fairly probably that these relatively infrequent
>> queries would have fallen off the bottom of pg_stat_statements.  Same
>> thing with the relative handful of queries which do large on-disk sorts.
>>
>> The problem I'm worried about is that pg_stat_statements is designed to
>> keep the most frequent queries, but sometimes the thing you really need
>> to look at is not in the list of most frequent queries.
>
>
> I think auto_explain would help you solve such rare incidents
> if it could dump several statistics into server log, including lock
> waits and block reads/writes statistic per-session, for example.
>
> Do we have something to add to auto_explain?

Now I am working on expanding slow query record and auto_explain with
some locking times (lock on objects, lock on enhancing pages, other
locks).

Just statement time produces too less information in our complex and
"unpredictable" cloud environment with thousand databases and hundreds
servers.

Regards

Pavel Stehule

>
> Regards,
> --
> Satoshi Nagayasu 
> Uptime Technologies, LLC. http://www.uptime.jp
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] RFC: Timing Events

2012-11-04 Thread Satoshi Nagayasu

(2012/11/03 10:44), Josh Berkus wrote:



I don't see all that going into core without a much bigger push than I
think people will buy.  What people really want for all these is a
proper trending system, and that means graphs and dashboards and
bling--not a history table.


Well, I'm particularly thinking for autoconfiguration.  For example, to
set vacuum_freeze_min_age properly, you have to know the XID "burn rate"
of the server, which is only available via history.  I really don't want
to be depending on a graphical monitoring utility to find these things out.


This whole approach has the assumption that things are going to fall off
sometimes.  To expand on that theme for a second, right now I'm more
worried about the "99%" class of problems.  Neither pg_stat_statements
nor this idea are very good for tracking the rare rogue problem down.
They're both aimed to make things that happen a lot more statistically
likely to be seen, by giving an easier UI to glare at them frequently.
That's not ideal, but I suspect really fleshing the whole queue consumer
-> table idea needs to happen to do much better.


I'm just concerned that for some types of incidents, it would be much
more than 1% *of what you want to look at* which fall off.  For example,
consider a server which does 95% reads at a very high rate, but has 2%
of its writes cronically having lock waits.  That's something you want
to solve, but it seems fairly probably that these relatively infrequent
queries would have fallen off the bottom of pg_stat_statements.  Same
thing with the relative handful of queries which do large on-disk sorts.

The problem I'm worried about is that pg_stat_statements is designed to
keep the most frequent queries, but sometimes the thing you really need
to look at is not in the list of most frequent queries.


I think auto_explain would help you solve such rare incidents
if it could dump several statistics into server log, including lock
waits and block reads/writes statistic per-session, for example.

Do we have something to add to auto_explain?

Regards,
--
Satoshi Nagayasu 
Uptime Technologies, LLC. http://www.uptime.jp


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [v9.3] writable foreign tables

2012-11-04 Thread Kohei KaiGai
2012/11/2 Alexander Korotkov :
> On Mon, Sep 24, 2012 at 12:49 PM, Kohei KaiGai  wrote:
>>
>> 2012/9/23 Kohei KaiGai :
>> > 2012/8/29 Kohei KaiGai :
>> >> 2012/8/28 Kohei KaiGai :
>> >>> 2012/8/28 Tom Lane :
>>  Kohei KaiGai  writes:
>> >> Would it be too invasive to introduce a new pointer in
>> >> TupleTableSlot
>> >> that is NULL for anything but virtual tuples from foreign tables?
>> 
>> > I'm not certain whether the duration of TupleTableSlot is enough to
>> > carry a private datum between scan and modify stage.
>> 
>>  It's not.
>> 
>> > Is it possible to utilize ctid field to move a private pointer?
>> 
>>  UPDATEs and DELETEs do not rely on the ctid field of tuples to carry
>>  the
>>  TID from scan to modify --- in fact, most of the time what the modify
>>  step is going to get is a "virtual" TupleTableSlot that hasn't even
>>  *got* a physical CTID field.
>> 
>>  Instead, the planner arranges for the TID to be carried up as an
>>  explicit resjunk column named ctid.  (Currently this is done in
>>  rewriteTargetListUD(), but see also preptlist.c which does some
>>  related
>>  things for SELECT FOR UPDATE.)
>> 
>>  I'm inclined to think that what we need here is for FDWs to be able
>>  to
>>  modify the details of that behavior, at least to the extent of being
>>  able to specify a different data type than TID for the row
>>  identification column.
>> 
>> >>> Hmm. It seems to me a straight-forward solution rather than ab-use
>> >>> of ctid system column. Probably, cstring data type is more suitable
>> >>> to carry a private datum between scan and modify stage.
>> >>>
>> >>> One problem I noticed is how FDW driver returns an extra field that
>> >>> is in neither system nor regular column.
>> >>> Number of columns and its data type are defined with TupleDesc of
>> >>> the target foreign-table, so we also need a feature to extend it on
>> >>> run-time. For example, FDW driver may have to be able to extend
>> >>> a "virtual" column with cstring data type, even though the target
>> >>> foreign table does not have such a column.
>> >>>
>> >> I tried to investigate the related routines.
>> >>
>> >> TupleDesc of TupleTableSlot associated with ForeignScanState
>> >> is initialized at ExecInitForeignScan as literal.
>> >> ExecAssignScanType assigns TupleDesc of the target foreign-
>> >> table on tts_tupleDescriptor, "as-is".
>> >> It is the reason why IterateForeignScan cannot return a private
>> >> datum except for the columns being declared as regular ones.
>> >>
>> > The attached patch improved its design according to the upthread
>> > discussion. It now got away from ab-use of "ctid" field, and adopts
>> > a concept of pseudo-column to hold row-id with opaque data type
>> > instead.
>> >
>> > Pseudo-column is Var reference towards attribute-number larger
>> > than number of attributes on the target relation; thus, it is not
>> > a substantial object. It is normally unavailable to reference such
>> > a larger attribute number because TupleDesc of each ScanState
>> > associated with a particular relation is initialized at ExecInitNode.
>> >
>> > The patched ExecInitForeignScan was extended to generate its
>> > own TupleDesc including pseudo-column definitions on the fly,
>> > instead of relation's one, when scan-plan of foreign-table requires
>> > to have pseudo-columns.
>> >
>> > Right now, the only possible pseudo-column is "rowid" being
>> > injected at rewriteTargetListUD(). It has no data format
>> > restriction like "ctid" because of VOID data type.
>> > FDW extension can set an appropriate value on the "rowid"
>> > field in addition to contents of regular columns at
>> > IterateForeignScan method, to track which remote row should
>> > be updated or deleted.
>> >
>> > Another possible usage of this pseudo-column is push-down
>> > of target-list including complex calculation. It may enable to
>> > move complex mathematical formula into remote devices
>> > (such as GPU device?) instead of just a reference of Var node.
>> >
>> > This patch adds a new interface: GetForeignRelInfo being invoked
>> > from get_relation_info() to adjust width of RelOptInfo->attr_needed
>> > according to the target-list which may contain "rowid" pseudo-column.
>> > Some FDW extension may use this interface to push-down a part of
>> > target list into remote side, even though I didn't implement this
>> > feature on file_fdw.
>> >
>> > RelOptInfo->max_attr is a good marker whether the plan shall have
>> > pseudo-column reference. Then, ExecInitForeignScan determines
>> > whether it should generate a TupleDesc, or not.
>> >
>> > The "rowid" is fetched using ExecGetJunkAttribute as we are currently
>> > doing on regular tables using "ctid", then it shall be delivered to
>> > ExecUpdate or ExecDelete. We can never expect the fist argument of
>> > them now, so "ItemPointer tupleid" redefined to "Datum rowid", and