Re: [HACKERS] psql, remove include of psqlscan.c
Karl O. Pinc k...@meme.com writes: This patch (psql_remove_include.patch) eliminates the #include of psqlscan.c at the bottom of mainloop.c. I don't really see that this is enough of an improvement to justify depending on a non-portable flex feature. I'm thinking of exposing enough of the psql parser, moving it to libpq, that any client-side app can do what libpq does; given a bunch of sql separated by semi-colons get the results of all the statements. This should also allow the statement separation to be done on the client side in libpq. Although I don't imagine that this will have a performance impact on the server side it sounds like a first step toward pushing more of the parsing onto the client. Quite honestly, I don't like that idea at all either. It's bad enough that psql has to know so much low-level SQL syntax. If we start encouraging other programs to know that, we're going to be locked into never again making any lexical-level changes. Why exactly is pushing parsing onto the client a good idea? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql, remove include of psqlscan.c
On 09/27/2012 10:07:48 AM, Tom Lane wrote: Karl O. Pinc k...@meme.com writes: This patch (psql_remove_include.patch) eliminates the #include of psqlscan.c at the bottom of mainloop.c. I don't really see that this is enough of an improvement to justify depending on a non-portable flex feature. Ok. (I had no idea it was non-portable. The flex docs don't mention this.) I'm thinking of exposing enough of the psql parser, moving it to libpq, that any client-side app can do what libpq does; given a bunch of sql separated by semi-colons get the results of all the statements. This should also allow the statement separation to be done on the client side in libpq. Although I don't imagine that this will have a performance impact on the server side it sounds like a first step toward pushing more of the parsing onto the client. Quite honestly, I don't like that idea at all either. It's bad enough that psql has to know so much low-level SQL syntax. If we start encouraging other programs to know that, we're going to be locked into never again making any lexical-level changes. Why exactly is pushing parsing onto the client a good idea? There's really 2 ideas. I don't care about this one: pushing parsing/anything onto the client is a good idea because clients scale horizontally and so performance is improved. What I'm thinking of in libpq is the ability to give it big string with many sql statements and have it hand back each statement so the client can then submit it to the server for execution. What I really _want_ is to be able get a bit string of many sql statements from the user and return the results, statuses, etc. of executing each statement. Just what psql does when, say, fed a file from stdin. The reason I want this is because I don't want to have to rewrite the sql parser in PHP for inclusion in phpPgAdmin. (I did this once, and it was such a big ugly patch it never got around to getting into the mainline phpPgAdmin.) phpPgAdmin (pgAdmin/ front-end of your choice) should be able process a string of sql statements in the same way that psql does, but currently the only way to do this is to rewrite the parser in each application. Much better to have libpq provide the functionality, although I suppose it's possible to push this into the server. Karl k...@meme.com Free Software: You don't pay back, you pay forward. -- Robert A. Heinlein -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql, remove include of psqlscan.c
Excerpts from Karl O. Pinc's message of jue sep 27 12:29:53 -0300 2012: The reason I want this is because I don't want to have to rewrite the sql parser in PHP for inclusion in phpPgAdmin. (I did this once, and it was such a big ugly patch it never got around to getting into the mainline phpPgAdmin.) phpPgAdmin (pgAdmin/ front-end of your choice) should be able process a string of sql statements in the same way that psql does, but currently the only way to do this is to rewrite the parser in each application. Much better to have libpq provide the functionality, although I suppose it's possible to push this into the server. This seems a worthy goal to me. But I think I see what Tom objection to it is: if we export this capability to libpq applications, then we set it in stone to a certain extent: exactly how things are split would become part of the API, so to speak. Upgrading to a newer libpq could break application code that worked with the previous release's by splitting things differently. I don't currently have an opinion on whether this is a bad thing or not. Who wants to argue for/against? -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql, remove include of psqlscan.c
On 09/27/12 8:41 AM, Alvaro Herrera wrote: But I think I see what Tom objection to it is: if we export this capability to libpq applications, then we set it in stone to a certain extent: exactly how things are split would become part of the API, so to speak. Upgrading to a newer libpq could break application code that worked with the previous release's by splitting things differently. I don't currently have an opinion on whether this is a bad thing or not. Who wants to argue for/against? I wonder if it shouldn't be in a separate 'helper' library, as it has no direct ties to any libpq internals. -- john r pierceN 37, W 122 santa cruz ca mid-left coast -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql, remove include of psqlscan.c
Alvaro Herrera alvhe...@2ndquadrant.com writes: But I think I see what Tom objection to it is: if we export this capability to libpq applications, then we set it in stone to a certain extent: exactly how things are split would become part of the API, so to speak. Upgrading to a newer libpq could break application code that worked with the previous release's by splitting things differently. That's not exactly what the problem is: an upgrade could only break *existing* application code if we had made a non-backward-compatible change in SQL lexical rules, which hopefully we'd never do. Rather, the problem is that the server might know about some newer lexical feature, and so might the application, but if libpq is behind the times then it's broken. You can see an instance of this right now over in the pgsql-jdbc list, where they're arguing about fixing the JDBC driver to understand dollar quoting. Application authors not unreasonably think it should have heard of that by now. The JDBC example is sort of an argument in favor of Karl's idea, in that if client-side code is going to know this anyway then it would be better if it were at least localized in one place. But of course JDBC is never going to depend on libpq (wrong language) so moving this code into libpq isn't actually going to help them. A larger point is that I don't believe this is actually going to help anybody, because of mismatch of requirements not only implementation language. JDBC couldn't use a libpq lexer implementation even without the language issue, because the context in which they're arguing about this is finding and replacing JDBC-spec escape sequences, which libpq is not going to know about. I imagine PHP has got the same problem only different. Conversely, psql's lexer has a lot of psql-specific design decisions, such as the need to handle backslash commands and include files, that I don't think would belong in a libpq implementation. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql, remove include of psqlscan.c
On 09/27/2012 11:02:42 AM, Tom Lane wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: A larger point is that I don't believe this is actually going to help anybody, because of mismatch of requirements not only implementation language. JDBC couldn't use a libpq lexer implementation even without the language issue, because the context in which they're arguing about this is finding and replacing JDBC-spec escape sequences, which libpq is not going to know about. I imagine PHP has got the same problem only different. Conversely, psql's lexer has a lot of psql-specific design decisions, such as the need to handle backslash commands and include files, that I don't think would belong in a libpq implementation. Well no, I'm not at all interested in escape sequences. I want to take sql directly from the user and execute it. Right now I can take only one statement at a time. And this is true of any human-facing application. I'm not looking for a general purpose solution, although it did occur to me that the psql variable substitution mechanism could be exposed. But what I really want is not an exposed parser. What I really want is to be able to get results from all the statements passed to PQexec (et-al), not just the last statement. This could be done without exposing the parser, but it does mean having a parser in libpq. Since psql links to libpq anyway my plan was to move the parser entirely out of psql into libpq and have an undocumented internal interface so that psql can do the escaping/variable substitution stuff. Regards, Karl k...@meme.com Free Software: You don't pay back, you pay forward. -- Robert A. Heinlein -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql, remove include of psqlscan.c
On 09/27/2012 11:02:42 AM, Tom Lane wrote: Rather, the problem is that the server might know about some newer lexical feature, and so might the application, but if libpq is behind the times then it's broken. If the application knows about the newer feature and wants to use it, is it unreasonable to ask the application be linked against a newer libpq? Karl k...@meme.com Free Software: You don't pay back, you pay forward. -- Robert A. Heinlein -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql, remove include of psqlscan.c
Karl O. Pinc k...@meme.com writes: What I'm thinking of in libpq is the ability to give it big string with many sql statements and have it hand back each statement so the client can then submit it to the server for execution. What I really _want_ is to be able get a bit string of many sql statements from the user and return the results, statuses, etc. of executing each statement. Just what psql does when, say, fed a file from stdin. Just as a note --- I believe you can get that result today with PQsendQuery followed by a PQgetResult loop. There's no need to provide an API that splits the string first. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql, remove include of psqlscan.c
Karl O. Pinc k...@meme.com writes: On 09/27/2012 11:02:42 AM, Tom Lane wrote: Rather, the problem is that the server might know about some newer lexical feature, and so might the application, but if libpq is behind the times then it's broken. If the application knows about the newer feature and wants to use it, is it unreasonable to ask the application be linked against a newer libpq? Well, see the JDBC example: there is no newer driver with the desired feature for applications to link against. Even if all client-side code with such knowledge stays perfectly in sync as far as upstream sources are concerned, there are any number of reasons why particular installations might have copies of varying vintages. It's not really a situation that I want to get into more than necessary. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql, remove include of psqlscan.c
On 09/27/2012 02:28:49 PM, Tom Lane wrote: Karl O. Pinc k...@meme.com writes: What I really _want_ is to be able get a bit string of many sql statements from the user and return the results, statuses, etc. of executing each statement. Just what psql does when, say, fed a file from stdin. Just as a note --- I believe you can get that result today with PQsendQuery followed by a PQgetResult loop. There's no need to provide an API that splits the string first. Ah. *sigh* I should pay better attention. Sorry for the distraction. Karl k...@meme.com Free Software: You don't pay back, you pay forward. -- Robert A. Heinlein -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
