[HACKERS] validating foreign tables
The API for FDW validators doesn't appear to have any way that the validator function can check that the defined foreign table shape matches the FDW options sanely. Maybe it's a chicken and egg problem, but there seems to be something missing, unless I'm mistaken. We'll have the info when we come to make plan estimates, but that seems like the wrong place to be doing this sort of validation. Can we extend the validator API somehow to make this possible? cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] validating foreign tables
Andrew Dunstan and...@dunslane.net writes: The API for FDW validators doesn't appear to have any way that the validator function can check that the defined foreign table shape matches the FDW options sanely. Huh? The options ought to be orthogonal to the table column info. If they're not, maybe you need to rethink your option definitions. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] validating foreign tables
On 02/21/2011 06:56 PM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: The API for FDW validators doesn't appear to have any way that the validator function can check that the defined foreign table shape matches the FDW options sanely. Huh? The options ought to be orthogonal to the table column info. If they're not, maybe you need to rethink your option definitions. Well, let's take a couple of cases. 1. My old favorite, file as a text array. 2. A hypothetical RSS feed, where the options specify which RSS fields we want. Of course, we could just let these break or give odd results when we run a SELECT if the foreign table doesn't match what's expected. file_fdw will presumably break if the input file has rows with the wrong number of columns, just as COPY does. But there will be cases, like the two above, where a sanity check on the table shape could usefully be done at validation time as opposed to run time, and it would be nice to be able to do such a check. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] validating foreign tables
On Tue, Feb 22, 2011 at 10:12, Andrew Dunstan and...@dunslane.net wrote: The API for FDW validators doesn't appear to have any way that the validator function can check that the defined foreign table shape matches the FDW options sanely. Huh? The options ought to be orthogonal to the table column info. If they're not, maybe you need to rethink your option definitions. Well, let's take a couple of cases. 1. My old favorite, file as a text array. 2. A hypothetical RSS feed, where the options specify which RSS fields we want. I think we need to overhaul validators in 9.2 listening to FDW developers' opinions anyway. The text array is an example, but there should be many other requirements. Personally, I'd like to have a method to list available options from SQL. We should also consider column-level options for foreign tables then. -- Itagaki Takahiro -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] validating foreign tables
On 02/21/2011 08:59 PM, Itagaki Takahiro wrote: I think we need to overhaul validators in 9.2 listening to FDW developers' opinions anyway. The text array is an example, but there should be many other requirements. Personally, I'd like to have a method to list available options from SQL. We should also consider column-level options for foreign tables then. Ok, I guess. It just seems to me like it will be harder to extend the API later than now, so if we can reasonably foresee a likely need we should try to provide for it. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] validating foreign tables
Andrew Dunstan and...@dunslane.net writes: On 02/21/2011 06:56 PM, Tom Lane wrote: Huh? The options ought to be orthogonal to the table column info. If they're not, maybe you need to rethink your option definitions. Well, let's take a couple of cases. 1. My old favorite, file as a text array. 2. A hypothetical RSS feed, where the options specify which RSS fields we want. As above, I claim that an FDW that has such options is badly designed to begin with. Why can't you generate the RSS command on-the-fly from the table rowtype? Of course, we could just let these break or give odd results when we run a SELECT if the foreign table doesn't match what's expected. file_fdw will presumably break if the input file has rows with the wrong number of columns, just as COPY does. But there will be cases, like the two above, where a sanity check on the table shape could usefully be done at validation time as opposed to run time, and it would be nice to be able to do such a check. I can't get excited about this. For one thing, you'd then need to worry about involving the validator in random ALTER TABLE situations, not just when changing the options it's supposed to be checking. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] validating foreign tables
Andrew Dunstan and...@dunslane.net writes: On 02/21/2011 08:59 PM, Itagaki Takahiro wrote: I think we need to overhaul validators in 9.2 listening to FDW developers' opinions anyway. Ok, I guess. It just seems to me like it will be harder to extend the API later than now, so if we can reasonably foresee a likely need we should try to provide for it. Perhaps we should put a large friendly EXPERIMENTAL, SUBJECT TO CHANGE notice on all the FDW API stuff? Just tell people up front that we're not prepared to promise any API stability yet. There's stuff we *know* is lacking (it's read-only, the optimization support sucks) in addition to whatever we may later realize is misdesigned. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] validating foreign tables
2011/2/22 Tom Lane t...@sss.pgh.pa.us: Andrew Dunstan and...@dunslane.net writes: On 02/21/2011 08:59 PM, Itagaki Takahiro wrote: I think we need to overhaul validators in 9.2 listening to FDW developers' opinions anyway. Ok, I guess. It just seems to me like it will be harder to extend the API later than now, so if we can reasonably foresee a likely need we should try to provide for it. Perhaps we should put a large friendly EXPERIMENTAL, SUBJECT TO CHANGE notice on all the FDW API stuff? Just tell people up front that we're not prepared to promise any API stability yet. There's stuff we *know* is lacking (it's read-only, the optimization support sucks) in addition to whatever we may later realize is misdesigned. regards, tom lane +1 regards Pavel Stehule -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
