Re: [HACKERS] Attention PL authors: want to be listed in template table?
On 8-Sep-05, at 2:18 AM, Thomas Hallgren wrote: Peter Eisentraut wrote: Thomas Hallgren wrote: Well, yes. But use the word environment in singular please :-) To my knowledge the security is full-proof with all other VM's since they all use the standard runtime libraries. It's not quite as simple as that. There are a bunch of VMs and a bunch of libraries (and a bunch of compilers), and they can be combined in many permutations. Not all of them work with PL/Java at the moment, but we should not hardcode support for just one of them. AFAIK, there are only two flavors of the Java Runtime library out there. The one that Sun provides (and small variants of it, such as the ones that IBM, HP, and BEA) and the "classpath" clean-room implementation. All variants of the former are OK with respect to security and only GCJ has a working environment of the latter. In particular, only GCJ has a functional standards conformant Java Native Interface (JNI) API to the latter and PL/Java is built on JNI. Should however, someone come up with another Java environment built on "classpath" that has JNI support, then there will be another potential environment for PL/Java. TMK, there's no such environment and none in the making. I have serious doubts that there ever will be. IMO it would be perfectly safe to hard code support for a trusted "java". Actually the apache guys are doing another one (Harmony), and there is Kaffe. Hardly relevant to the conversation, just added for completion The GCJ support is as experimental as the GCJ in itself and cannot be trusted in production. You should not say that too loud when someone from Red Hat is listening. :-) To my knowledge GCJ is Ready(tm) as of version 4.0. And it's being used. Distributions such as Fedora and Ubuntu will ship (or do ship?) with everything compiled using GCJ to the extent possible. And there are people, in particular at or near Red Hat, who have been specifically charged for several years now to make sure that every piece of Java code out there compiles with GCJ. Don't get me wrong. I like GCJ and the idea of compiled Java executables but I try to look at it's potential and usefulness in a realistic way. If Red Hat wants to tout that it's production ready, that's up to them. I'm not a marketing guy. GCJ currently that has limited security. It is 2 years behind mainstream in versions (they don't have Java 5 yet and their Java 1.4 support is not complete). It is not stable and the performance is nowhere close to the commercial implementations. I think the GCJ team is aware of this and I seriously doubt that it is surprise to the people at Red Hat. Try using GCJ to run Java applets in a web browser. You can't really since such applets cannot be trusted. I doubt the browser vendors make attempts to prevent it though ;-) Regarding the security issue: Word from Andrew Haley of Red Hat is that it has simply been too much work to implement security up to now. This should not affect the judgement of the quality of GCJ, it's simply a missing feature. Security is some "feature" to "simply miss". Especially if we talk about a VM. Of course, I don't intend to undermine your judgement as the author about what you consider experimental or not, but you should expect that if you put your code out there, people will use it in whatever way they see fit, and in particular with whatever Java toolchain they see fit. I do indeed expect that. But the PostgreSQL community cannot take responsibility for all that may happen when people do that. PL/Java is designed to run perfectly safe with a JVM that has the correct features implemented. GCJ has serious issues with security and I don't see that PL/Java, nor PostgreSQL should make any attempt to fix them. How safe is PostgreSQL running on an unsafe operating system? Regards, Thomas Hallgren ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Attention PL authors: want to be listed in template
Dennis Bjorklund <[EMAIL PROTECTED]> writes: > Doesn't this make it hard for distributions to package up a language in a > rpm (or some other system) and have it just work? No, it makes it easier. Particularly when you think about altering the implementation details of that language across successive PG versions. > If the per language info was simply stored in a file then this file > can be included in a package and you can install a new language in a > simple way. That's not any easier from a packager's viewpoint than patching an appropriate entry into the standard contents of pg_pltemplate. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Attention PL authors: want to be listed in template table?
"Rupa Schomaker (lists)" <[EMAIL PROTECTED]> writes: > Situation: A user installs a third-party PL that installs itself by > updating the template catalog and then doing a createlang. > Can the user simply do a dumpall/restore and get back into working > order. Or will the user have to re-run the PL install script? So? The dumpall/restore doesn't guarantee that the third-party PL library is in place, so you still need to re-run the PL's install script. The point I think you are missing is that this change is an effort to separate the existing treatment of PLs into two levels. The handler and validator and library path properties of PLs are implementation details that are best left outside the scope of pg_dump. pg_dump has never been about exactly duplicating the original installation; it's about dumping the user data in a form that can be transported to other installations (possibly on different platforms) or to newer releases of Postgres. Thus for example pg_dump intentionally avoids dumping and restoring any built-in tables, functions, operators, etc. If it tried to do that then it'd be useless for moving user data across PG versions, because it'd wipe out whatever catalog updates the new version had made. The basic point of the template patch is that the appropriate level of abstraction for porting procedural languages across PG versions is simply the language name --- the pg_dump script should say "CREATE LANGUAGE plfoo", no more and no less. The other details such as the library path name or the presence of a validator function are items that can *and should* be allowed to vary across versions. So it should be up to the new installation to provide valid values. In the discussion yesterday, Peter provided some reasons why the valid values might not be the same for every installation; but I continue to think that they will be the same more often than not. On convenience grounds it seems to me that it's best to provide built-in template values for as many PLs as we can. When the destination installation does have correct template values, the argument that pg_dump failed to do it becomes moot. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Attention PL authors: want to be listed in template
On Thu, 8 Sep 2005, Tom Lane wrote: > I've committed the changes to have a system catalog in place of the > hard-wired table. In the initial commit, I listed only the languages > included in the core distribution. If I understand this correct you have created a system table that contain the settings for some of the languages. It basicly have one row for each language? And in this row there is for example a column that say what the validator function is called, right? Or what do the current solution look like? Doesn't this make it hard for distributions to package up a language in a rpm (or some other system) and have it just work? If the per language info was simply stored in a file then this file can be included in a package and you can install a new language in a simple way. -- /Dennis Björklund ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Attention PL authors: want to be listed in template
On 9/8/2005 9:20 PM, Tom Lane wrote: > "Rupa Schomaker (lists)" <[EMAIL PROTECTED]> writes: > >>How are dumps/restores of local modifications to the template going to >>be handled? > > > pg_dump has no business trying to save or restore the template data; > if it did so then we'd just have moved the original problem of obsolete > data to a different place. See my original proposal. > I must be missing something. Situation: A user installs a third-party PL that installs itself by updating the template catalog and then doing a createlang. Can the user simply do a dumpall/restore and get back into working order. Or will the user have to re-run the PL install script? Situation: A DBA changes something in the catalog as you've suggested is possible. Does the DBA now have to perform an extra step after a dump/restore to get things 'back the way they were?' The 'whats the point' was sort of implied in my original message. Rather than say 'whats the point' I was trying to point out a potential pitfall for end-users with this approach and suggest we should address it. You removed the pertinent part of your statement about how this catalog would be useful for more than just core PLs. If we're encouraging this as a general solution then the solution should be complete. > regards, tom lane -- -Rupa ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Attention PL authors: want to be listed in template table?
"Rupa Schomaker (lists)" <[EMAIL PROTECTED]> writes: > How are dumps/restores of local modifications to the template going to > be handled? pg_dump has no business trying to save or restore the template data; if it did so then we'd just have moved the original problem of obsolete data to a different place. See my original proposal. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Thomas Hallgren wrote: >> PL/Java is designed to run perfectly safe with a JVM that has the >> correct features implemented. GCJ has serious issues with security >> and I don't see that PL/Java, nor PostgreSQL should make any attempt >> to fix them. > Well, we had a similar discussion about the time when the Python > security support was decreed nonexistent by its author. Clearly, > people still use Python, and people still use PL/Python. It's really > easy to spread a panic by claiming that GCJ has "no security". That's > clearly wrong because GCJ can be used safely in many useful situations. Actually, I've just been discussing this with Red Hat's gcj people in connection with a different project. What they say is that the Java security manager is completely implemented now, but what is still missing is that it's possible to bypass Java security if you can execute untrusted bytecode. So if I understand correctly, a gcj environment is secure as long as you can prevent hacked-up class files from getting into your classpath. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Thomas Hallgren <[EMAIL PROTECTED]> writes: > That's great news for PL/Java (and for Java in general of course). Did > they mention a release date? You can have it today if you feel like downloading Fedora Rawhide. (Actually you might want to wait till tomorrow -- I hear rawhide is pretty borked at the moment because someone managed to turn off the setuid bit on "su" ;-)) I'm not sure what the schedule is for releasing Fedora 5, but you could probably find it out from fedora.redhat.com. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Attention PL authors: want to be listed in template table?
On 8-Sep-05, at 3:45 PM, Thomas Hallgren wrote: Tom Lane wrote: Actually, I've just been discussing this with Red Hat's gcj people in connection with a different project. What they say is that the Java security manager is completely implemented now, but what is still missing is that it's possible to bypass Java security if you can execute untrusted bytecode. So if I understand correctly, a gcj environment is secure as long as you can prevent hacked-up class files from getting into your classpath. Pretty tough to do, since you can read classes in your classpath, and modify the bytecode on the fly There's even a library to do it for you. That's great news for PL/Java (and for Java in general of course). Did they mention a release date? Regards, Thomas Hallgren ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Attention PL authors: want to be listed in template table?
I wrote: > Well, that's exactly the point Peter is arguing: he thinks (if I > understand correctly) that the template mechanism should only be used > for stuff that's included with the core distribution. I disagree; > I have seldom seen any good reason for restricting mechanisms to work > with only core components. It does seem apparent however that we need > the templates to be reconfigurable for local conditions ... so the > hardwired table is out, even as a one-release stopgap. I'll work on > building a catalog tomorrow. I've committed the changes to have a system catalog in place of the hard-wired table. In the initial commit, I listed only the languages included in the core distribution. It's still possible to add additional entries for non-core languages into the default table contents, and I'll be happy to do that for any PL authors who want them. However, given that it is now possible for superusers to alter the catalog contents, it's not so pressing to have a preloaded entry --- you could instead suggest that people create an entry while installing your language. Were any of the people who already asked for entries persuaded by Peter's arguments that non-core languages are better off without a standard template entry? Let me know if you still want one. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Peter Eisentraut wrote: The way I was reading your statements was that you concluded from this sitation that GCJ should not be used at all for real work. I see your point and my way of expressing it was probably too harsh. I agree that there are many applications where an unsafe VM can be safely used. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Tom Lane wrote: Actually, I've just been discussing this with Red Hat's gcj people in connection with a different project. What they say is that the Java security manager is completely implemented now, but what is still missing is that it's possible to bypass Java security if you can execute untrusted bytecode. So if I understand correctly, a gcj environment is secure as long as you can prevent hacked-up class files from getting into your classpath. That's great news for PL/Java (and for Java in general of course). Did they mention a release date? Regards, Thomas Hallgren ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Thomas Hallgren wrote: > >Frankly, that is all FUD. > > No, that's all facts. Those two are not mutually exclusive. > We where discussing a very specific situation here. Not GCJ in > general. As you pointed out yourself (and that's what started this > discussion), GCJ cannot be used for a trusted Java implementation. The way I was reading your statements was that you concluded from this sitation that GCJ should not be used at all for real work. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Peter Eisentraut wrote: Thomas Hallgren wrote: GCJ currently that has limited security. It is 2 years behind mainstream in versions (they don't have Java 5 yet and their Java 1.4 support is not complete). It is not stable and the performance is nowhere close to the commercial implementations. Frankly, that is all FUD. No, that's all facts. At least with the release I'm using (4.0). The security seems to come along well though. Classpath has all the security packages fully implemented at this point so this whole discussion is probably obsolete by the time GCJ does their next release. A lot of free software is limited or behind or claimed to be unstable in some way, but that has never stopped anyone from using it in the appropriate and expanding niches. That's very true. But it doesn't make my statement FUD nevertheless. Well, we had a similar discussion about the time when the Python security support was decreed nonexistent by its author. Clearly, people still use Python, and people still use PL/Python. It's really easy to spread a panic by claiming that GCJ has "no security". That's clearly wrong because GCJ can be used safely in many useful situations. We where discussing a very specific situation here. Not GCJ in general. As you pointed out yourself (and that's what started this discussion), GCJ cannot be used for a trusted Java implementation. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Thomas Hallgren wrote: > GCJ currently that has limited security. It is 2 years behind > mainstream in versions (they don't have Java 5 yet and their Java 1.4 > support is not complete). It is not stable and the performance is > nowhere close to the commercial implementations. Frankly, that is all FUD. A lot of free software is limited or behind or claimed to be unstable in some way, but that has never stopped anyone from using it in the appropriate and expanding niches. Just look at PostgreSQL. Now, if the commercial Java implementations had acceptable licensing terms, I would consider using them, but they don't, so I won't. > PL/Java is designed to run perfectly safe with a JVM that has the > correct features implemented. GCJ has serious issues with security > and I don't see that PL/Java, nor PostgreSQL should make any attempt > to fix them. Well, we had a similar discussion about the time when the Python security support was decreed nonexistent by its author. Clearly, people still use Python, and people still use PL/Python. It's really easy to spread a panic by claiming that GCJ has "no security". That's clearly wrong because GCJ can be used safely in many useful situations. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Well don't forget sablevm, and jamvm. there's quite a few around. Dave On 8-Sep-05, at 9:48 AM, Thomas Hallgren wrote: Dave Cramer wrote: Actually the apache guys are doing another one (Harmony), and there is Kaffe. Hardly relevant to the conversation, just added for completion I stand corrected. I forgot Kaffe. It also uses the "classpath" stuff and have the same issues as GCJ. Isn't Harmony is just an incubation project at Apache? To my knowledge they have not produced anything yet and it will be years before they have something useful. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Attention PL authors: want to be listed in template table?
On 8-Sep-05, at 2:18 AM, Thomas Hallgren wrote: Peter Eisentraut wrote: Thomas Hallgren wrote: Well, yes. But use the word environment in singular please :-) To my knowledge the security is full-proof with all other VM's since they all use the standard runtime libraries. It's not quite as simple as that. There are a bunch of VMs and a bunch of libraries (and a bunch of compilers), and they can be combined in many permutations. Not all of them work with PL/Java at the moment, but we should not hardcode support for just one of them. AFAIK, there are only two flavors of the Java Runtime library out there. The one that Sun provides (and small variants of it, such as the ones that IBM, HP, and BEA) and the "classpath" clean-room implementation. All variants of the former are OK with respect to security and only GCJ has a working environment of the latter. In particular, only GCJ has a functional standards conformant Java Native Interface (JNI) API to the latter and PL/Java is built on JNI. Should however, someone come up with another Java environment built on "classpath" that has JNI support, then there will be another potential environment for PL/Java. TMK, there's no such environment and none in the making. I have serious doubts that there ever will be. IMO it would be perfectly safe to hard code support for a trusted "java". Actually the apache guys are doing another one (Harmony), and there is Kaffe. Hardly relevant to the conversation, just added for completion The GCJ support is as experimental as the GCJ in itself and cannot be trusted in production. You should not say that too loud when someone from Red Hat is listening. :-) To my knowledge GCJ is Ready(tm) as of version 4.0. And it's being used. Distributions such as Fedora and Ubuntu will ship (or do ship?) with everything compiled using GCJ to the extent possible. And there are people, in particular at or near Red Hat, who have been specifically charged for several years now to make sure that every piece of Java code out there compiles with GCJ. Don't get me wrong. I like GCJ and the idea of compiled Java executables but I try to look at it's potential and usefulness in a realistic way. If Red Hat wants to tout that it's production ready, that's up to them. I'm not a marketing guy. GCJ currently that has limited security. It is 2 years behind mainstream in versions (they don't have Java 5 yet and their Java 1.4 support is not complete). It is not stable and the performance is nowhere close to the commercial implementations. I think the GCJ team is aware of this and I seriously doubt that it is surprise to the people at Red Hat. Try using GCJ to run Java applets in a web browser. You can't really since such applets cannot be trusted. I doubt the browser vendors make attempts to prevent it though ;-) Regarding the security issue: Word from Andrew Haley of Red Hat is that it has simply been too much work to implement security up to now. This should not affect the judgement of the quality of GCJ, it's simply a missing feature. Security is some "feature" to "simply miss". Especially if we talk about a VM. Of course, I don't intend to undermine your judgement as the author about what you consider experimental or not, but you should expect that if you put your code out there, people will use it in whatever way they see fit, and in particular with whatever Java toolchain they see fit. I do indeed expect that. But the PostgreSQL community cannot take responsibility for all that may happen when people do that. PL/Java is designed to run perfectly safe with a JVM that has the correct features implemented. GCJ has serious issues with security and I don't see that PL/Java, nor PostgreSQL should make any attempt to fix them. How safe is PostgreSQL running on an unsafe operating system? Regards, Thomas Hallgren ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Dave Cramer wrote: Actually the apache guys are doing another one (Harmony), and there is Kaffe. Hardly relevant to the conversation, just added for completion I stand corrected. I forgot Kaffe. It also uses the "classpath" stuff and have the same issues as GCJ. Isn't Harmony is just an incubation project at Apache? To my knowledge they have not produced anything yet and it will be years before they have something useful. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Peter Eisentraut wrote: Thomas Hallgren wrote: Well, yes. But use the word environment in singular please :-) To my knowledge the security is full-proof with all other VM's since they all use the standard runtime libraries. It's not quite as simple as that. There are a bunch of VMs and a bunch of libraries (and a bunch of compilers), and they can be combined in many permutations. Not all of them work with PL/Java at the moment, but we should not hardcode support for just one of them. AFAIK, there are only two flavors of the Java Runtime library out there. The one that Sun provides (and small variants of it, such as the ones that IBM, HP, and BEA) and the "classpath" clean-room implementation. All variants of the former are OK with respect to security and only GCJ has a working environment of the latter. In particular, only GCJ has a functional standards conformant Java Native Interface (JNI) API to the latter and PL/Java is built on JNI. Should however, someone come up with another Java environment built on "classpath" that has JNI support, then there will be another potential environment for PL/Java. TMK, there's no such environment and none in the making. I have serious doubts that there ever will be. IMO it would be perfectly safe to hard code support for a trusted "java". The GCJ support is as experimental as the GCJ in itself and cannot be trusted in production. You should not say that too loud when someone from Red Hat is listening. :-) To my knowledge GCJ is Ready(tm) as of version 4.0. And it's being used. Distributions such as Fedora and Ubuntu will ship (or do ship?) with everything compiled using GCJ to the extent possible. And there are people, in particular at or near Red Hat, who have been specifically charged for several years now to make sure that every piece of Java code out there compiles with GCJ. Don't get me wrong. I like GCJ and the idea of compiled Java executables but I try to look at it's potential and usefulness in a realistic way. If Red Hat wants to tout that it's production ready, that's up to them. I'm not a marketing guy. GCJ currently that has limited security. It is 2 years behind mainstream in versions (they don't have Java 5 yet and their Java 1.4 support is not complete). It is not stable and the performance is nowhere close to the commercial implementations. I think the GCJ team is aware of this and I seriously doubt that it is surprise to the people at Red Hat. Try using GCJ to run Java applets in a web browser. You can't really since such applets cannot be trusted. I doubt the browser vendors make attempts to prevent it though ;-) Regarding the security issue: Word from Andrew Haley of Red Hat is that it has simply been too much work to implement security up to now. This should not affect the judgement of the quality of GCJ, it's simply a missing feature. Security is some "feature" to "simply miss". Especially if we talk about a VM. Of course, I don't intend to undermine your judgement as the author about what you consider experimental or not, but you should expect that if you put your code out there, people will use it in whatever way they see fit, and in particular with whatever Java toolchain they see fit. I do indeed expect that. But the PostgreSQL community cannot take responsibility for all that may happen when people do that. PL/Java is designed to run perfectly safe with a JVM that has the correct features implemented. GCJ has serious issues with security and I don't see that PL/Java, nor PostgreSQL should make any attempt to fix them. How safe is PostgreSQL running on an unsafe operating system? Regards, Thomas Hallgren ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Greg Stark <[EMAIL PROTECTED]> writes: > I'm not sure how that relates to the template work you're > discussing. Are all of the templates you're creating for components > that are built out of core and therefore can be expected to be > packaged? Well, that's exactly the point Peter is arguing: he thinks (if I understand correctly) that the template mechanism should only be used for stuff that's included with the core distribution. I disagree; I have seldom seen any good reason for restricting mechanisms to work with only core components. It does seem apparent however that we need the templates to be reconfigurable for local conditions ... so the hardwired table is out, even as a one-release stopgap. I'll work on building a catalog tomorrow. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Tom Lane <[EMAIL PROTECTED]> writes: > Now this certainly discriminates against installing PLs outside $libdir, > but I haven't heard any good reason why people would need to do that. Well one example would be distributions that package up some PLs but want to allow you to install some on your own. The normal model for that is to have a directory in /usr/lib// for the packaged software and a separate place in /usr/local/lib// for the locally installed add-ons. I'm not sure how that relates to the template work you're discussing. Are all of the templates you're creating for components that are built out of core and therefore can be expected to be packaged? -- greg ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> In any case, "java" has not been put forward as one of the template >> entries, and as long as we don't accept a template for it, we have >> not made the situation any worse. > Hmm, Thomas Hallgren sent in a template using "java" as name and you > answered "OK", so we're already there if it's already committed. Oh, I hadn't noticed. That seems like rather a bad idea --- shouldn't it be "pljava"? ("javaU" isn't going to work either, because of case_translate_language_name.) >> Yes, I am assuming that, and I challenge you to supply examples of >> PLs that won't require at least a recompile before there's any hope >> of their working on 8.1. > There is no hope of that, but a mere recompilation does not change the > validator or the schema or any other property that may be under > consideration. The current code will force a *version* upgrade of all > PLs with every PostgreSQL upgrade. I need to download new code and > deal with it. That is currently not required. Really? See the oidvector changes. I think that will force at least minor source changes on every PL. Now there may be people out there who will prefer making a few small changes by hand to downloading a new version ... but they can probably manage throwing in a stub validator function too. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Tom Lane wrote: > (What the PL template approach *would* do is make it difficult to > create a language that is trusted but named pljavau, or untrusted and > named pljava. But the latter is exactly what I would like to do. > Personally I don't see that as a bad thing, however. > The opportunity for confusion is far too great if you go against the > established naming conventions.) Extensibility means you don't control the naming. I guess if you want to say that this whole idea of extensibility in the language handler area is hereby withdrawn, doesn't work, never existed, then let's make that clear. Then we can hardcode everything, tell people, if you want to write a language handler, you should talk to us so we can arrange the hooks. That is the direction we're headed in. The PostgreSQL developers and the language handler authors dictate to the user what language he can use in what mode. If you don't like it, here's a way to do manual surgery to change it. Of course you can always change everything with varying effort. So yeah, that would work, but then it should be called that. But sacrificing user options to reduce "confusion" is hardly our game. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Tom Lane wrote: > Are you seriously suggesting that it's a good idea for the single > language name "java" to mean different things at different > installations? I can't believe that that wouldn't lead to chaos. There is a standard for the SQL integration of a "java" language, and I don't see why there can't be multiple implementations of that same specification. It in fact appears that there are. Of course we would not want the programming interfaces to differ, but they don't have to. > In any case, "java" has not been put forward as one of the template > entries, and as long as we don't accept a template for it, we have > not made the situation any worse. Hmm, Thomas Hallgren sent in a template using "java" as name and you answered "OK", so we're already there if it's already committed. > Yes, I am assuming that, and I challenge you to supply examples of > PLs that won't require at least a recompile before there's any hope > of their working on 8.1. There is no hope of that, but a mere recompilation does not change the validator or the schema or any other property that may be under consideration. The current code will force a *version* upgrade of all PLs with every PostgreSQL upgrade. I need to download new code and deal with it. That is currently not required. And considering the general breakage of PLs out there, I don't think it's acceptable to require it. What if the new PL/R also requires a new R? What if the new R requires a new GCC? We don't know that, we can't control that, we should not interfere in that. I'm not making this up; these problems are real (although not necessarily in PL/R). -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Thomas Hallgren wrote: > Well, yes. But use the word environment in singular please :-) To my > knowledge the security is full-proof with all other VM's since they > all use the standard runtime libraries. It's not quite as simple as that. There are a bunch of VMs and a bunch of libraries (and a bunch of compilers), and they can be combined in many permutations. Not all of them work with PL/Java at the moment, but we should not hardcode support for just one of them. > The GCJ support is as > experimental as the GCJ in itself and cannot be trusted in > production. You should not say that too loud when someone from Red Hat is listening. :-) To my knowledge GCJ is Ready(tm) as of version 4.0. And it's being used. Distributions such as Fedora and Ubuntu will ship (or do ship?) with everything compiled using GCJ to the extent possible. And there are people, in particular at or near Red Hat, who have been specifically charged for several years now to make sure that every piece of Java code out there compiles with GCJ. Regarding the security issue: Word from Andrew Haley of Red Hat is that it has simply been too much work to implement security up to now. This should not affect the judgement of the quality of GCJ, it's simply a missing feature. Of course, I don't intend to undermine your judgement as the author about what you consider experimental or not, but you should expect that if you put your code out there, people will use it in whatever way they see fit, and in particular with whatever Java toolchain they see fit. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> In any case, not one of these arguments seems to me to favor the >> alternative of preloading definitions into pg_language: > That is not the alternative I am proposing. Your template idea, > hard-coded even, is just fine for core languages. For non-core > languages I am proposing that we simply do nothing because the problem > we are solving does not exist, or at least has a wildly different > nature. Ah. We have been talking at cross-purposes then, because I thought you were arguing for putting exactly the same template information into a different place. Given that, I think we can just agree to disagree: whether a particular PL could benefit from a template entry is up to the author of that PL to decide. Depending on the extent of the changes needed for 8.1, it might or might not make sense to assume that a source code update is needed, and if there isn't then maybe adding a template is a bad idea. It does seem though that your arguments indicate a possible need for local adjustment of the template info ... which means the hard-wired approach is not good enough, and we have to take the next step of creating a system catalog. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Tom Lane wrote: > Now this certainly discriminates against installing PLs outside > $libdir, but I haven't heard any good reason why people would need to > do that. Development, testing, staging, ... > > If I have PL/foo 1.0 > > installed and upgrade from PostgreSQL 8.0 to 8.1, the validator in > > PL/foo 1.0 will not magically appear or disappear. > > As a general rule, PLs have to at least be recompiled to move to a > new release of the backend, and usually need some source code > tweaking as well. You keep saying that, but the fact is that most of the non-core PLs will work just fine across several releases, if only because they have some #ifdef's. So, no, you do not need to upgrade your PLs when you upgrade PostgreSQL. You really don't. You just recompile them and put them back in the same place. There are certainly constraints and exceptions, but as a general rule it simply is not the case. If it were, I would not have started this discussion. Taking a step back and considering this "validator" business in a more general context: Basically, the validator is just a property that may or may not be applicable to a C function hidden inside a shared library. We don't know what's in the shared library, so we (currently) have to rely on the meta information in the dump to find out. This is a special case of a more general problem, which is why I mentioned PostGIS earlier. Any "plug-in" will evolve, and if it's just a bit sophisticated it will have fairly tight dependencies on the server version, which is certainly the case for much of the GIS and GiST stuff. So here we have the same problem. The old dump will have the meta-information on the old plug-in version. Under your theory, which is surely true in certain cases, the user will have to upgrade the plug-in at the same time. So the restore of the dump will have the old, now wrong information. I don't suppose hard-coding the PostGIS schema and ignoring special cases of CREATE FUNCTION will be the answer. One might object that these cases -- procedural languages and PostGIS sort of things -- are different, but they are not. The chance that an upgrade of PostgreSQL will require a version upgrade of the plug-in is about the same. Certainly, the number of users of PostGIS and the various GiST modules (FTS et al.) is comparable to, if not higher than that of some of the PLs. So that means two things: 1. The problem is much worse. 2. The problem is really much less bad because external plug-ins, PL or not, are often coded to work with multple server versions. And they should be, because otherwise the work forced upon the development teams to synchronize and the users to figure all that out will grow more than linearly. > In any case, not one of these arguments seems to me to favor the > alternative of preloading definitions into pg_language: That is not the alternative I am proposing. Your template idea, hard-coded even, is just fine for core languages. For non-core languages I am proposing that we simply do nothing because the problem we are solving does not exist, or at least has a wildly different nature. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Thomas Hallgren <[EMAIL PROTECTED]> writes: > GCJ is a clean house implementation of Java. They don't use the runtime > libraries from Sun and they are not really there yet in their efforts to > copy the functionality. One of the things that lag behind is security. > They hope to have a better security implementation before the year end > but there's no promise. OK, so that is a transient limitation of the GCJ work, not something fundamental. Thanks for clarifying. In that case I agree that trying to restrict it mechanically isn't a good idea --- the code restriction would still be around after the problem was gone. I still think this is irrelevant to the PL template discussion, however, since neither our past approach nor either of the proposals will make it the least bit difficult for a user to mislabel pljava as TRUSTED when the underlying implementation isn't really trustworthy. (What the PL template approach *would* do is make it difficult to create a language that is trusted but named pljavau, or untrusted and named pljava. Personally I don't see that as a bad thing, however. The opportunity for confusion is far too great if you go against the established naming conventions.) regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Peter Eisentraut <[EMAIL PROTECTED]> writes: > The case in reality is this: First of all, the language name "java" is > fixed by the SQL standard, so we ought to allow alternative > implementations to use that name. I'm not sure what kind of interface > the PL/J people are working on, but if they also lay claim to the name > "java", then we have a problem. Second, Java is not, in fact, always > Java, so different quality variants of the same implementations exist. > The Debian package of pljava is compiled using gcj, but it is also > planned to provide an alternative version that is compiled using the > Sun JDK. That way, users can trade off quality/features vs. licensing > freedom. Are you seriously suggesting that it's a good idea for the single language name "java" to mean different things at different installations? I can't believe that that wouldn't lead to chaos. In any case, "java" has not been put forward as one of the template entries, and as long as we don't accept a template for it, we have not made the situation any worse. > I think you are assuming all the way through this discussion that a > PostgreSQL upgrade will also entail an upgrade of all procedural > languages. Yes, I am assuming that, and I challenge you to supply examples of PLs that won't require at least a recompile before there's any hope of their working on 8.1. In a quick look through the CVS logs, I note that heap_openr/index_openr are gone, the representation of pg_proc entries is quite a bit different than it was in 8.0, and there are incompatible changes in the APIs of spi.c and dynahash.c. The pg_proc changes in particular practically guarantee a need for a source-code update. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Tom Lane wrote: Peter Eisentraut <[EMAIL PROTECTED]> writes: The other problem I see emerging here is that in certain environments, the "java" language may not be trusted, such as when it is compiled with GCJ. Well, yes. But use the word environment in singular please :-) To my knowledge the security is full-proof with all other VM's since they all use the standard runtime libraries. The GCJ support is as experimental as the GCJ in itself and cannot be trusted in production. Hmm, is that really the case? I thought Java is Java. GCJ is a clean house implementation of Java. They don't use the runtime libraries from Sun and they are not really there yet in their efforts to copy the functionality. One of the things that lag behind is security. They hope to have a better security implementation before the year end but there's no promise. Then, this built-in template will override the CREATE LANGUAGE specification and introduce a security hole. But it's exactly the same hole the user would create by manually saying CREATE TRUSTED LANGUAGE in error. I don't think that's a reasonable argument against the template idea --- it just says that you have to be aware of what you're doing. (An appropriate solution, in my mind, would be to drop the trusted call handler from the shared library if it's built with gcj --- then there's really no possibility of doing the wrong thing.) That's a though although I'm not sure we would gain anything. No PL/Java binaries are provided that run with GCJ. You have to compile from source using some specific settings. If you are skilled enough to experiment with that, then there's a good chance you are able to tweak the source to enable the trusted call handler also. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> We know we have a problem with hard-coded paths >> in old dumps. > This is irrelevant for non-core languages. If I have PL/foo 1.0 > installed and upgrade from PostgreSQL 8.0 to 8.1, then I have to > manually recompile PL/foo and install the shared library. So manual > intervention is required in any case and no template system of any kind > will help. Certainly you have to install new software, but that is a task exactly equivalent to installing the new version of Postgres itself. The problem being addressed here is fixing wrong information in your pg_dump scripts, and that is just as real for non-core as core PLs. The proposal *will* fix the need for manual intervention while reloading dump data. > Moreover, the hard-coded paths will not be solved by your proposal. On what basis do you assert that? The proposal will replace those paths by "$libdir/plfoo" (or whatever is in the template), which is something we should be able to go forward with indefinitely ... and if we can't, a template update will fix it in some future release. Now this certainly discriminates against installing PLs outside $libdir, but I haven't heard any good reason why people would need to do that. >> We know we have a problem with missing validators in >> old dumps > This is irrelevant for non-core languages. Again, I see no basis for that assertion. It is only true to the extent that non-core languages haven't created validators yet. > If I have PL/foo 1.0 > installed and upgrade from PostgreSQL 8.0 to 8.1, the validator in > PL/foo 1.0 will not magically appear or disappear. As a general rule, PLs have to at least be recompiled to move to a new release of the backend, and usually need some source code tweaking as well. So I think your premise is faulty. You'll need to install the 8.1 version of that PL, and that very possibly will have a validator where none existed before. > ... The information in > the dump is the only accurate information that exists about the > existence of a validator. Except that the information in the dump is likely *not* to be accurate. It tells you the way things were some number of releases ago, and as we've already seen, people frequently propagate their dumps forward across multiple releases --- indeed, one would hope that they'd be able to do so. We need to fix things so that that Just Works in the face of improvements to PLs. > You are assuming that during the > update from PostgreSQL 8.0 to PostgreSQL 8.1 there will also be an > upgrade from PL/foo 1.0 to PL/foo ${latest-at-time-of-8.1-release}, but > there is simply no basis for assuming that. How many of the past major releases have allowed a previous PL to be used without any changes? Hardly any AFAIR, and 8.1 is certainly not far behind the previous ones when it comes to making internal API changes that affect PLs. In any case, not one of these arguments seems to me to favor the alternative of preloading definitions into pg_language: that preloaded definition is going to be just as wrong as the template information, if either one is wrong. Now it's true that a hardwired template table is harder to fix than a system catalog, but we can get rid of the hardwired table at the same cost (viz, a forced initdb) as adding preloaded definitions. I don't see any other advantage to doing it that way. One other response to your point about surprise or lack of it: in my proposal, in a release or so everyone will just be doing "CREATE LANGUAGE foo" and there will be no surprise that the system knows how to fill in the low-level details. The handler functions will be relegated to what they should have been all along: an implementation detail. But I think that having CREATE LANGUAGE alter the permissions of an existing entry would be a permanent security risk; it will never not surprise people, because other forms of CREATE don't do that. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Tom Lane wrote: > But it's exactly the same hole the user would create by manually > saying CREATE TRUSTED LANGUAGE in error. I don't think that's a > reasonable argument against the template idea --- it just says that > you have to be aware of what you're doing. > > (An appropriate solution, in my mind, would be to drop the trusted > call handler from the shared library if it's built with gcj --- then > there's really no possibility of doing the wrong thing.) The case in reality is this: First of all, the language name "java" is fixed by the SQL standard, so we ought to allow alternative implementations to use that name. I'm not sure what kind of interface the PL/J people are working on, but if they also lay claim to the name "java", then we have a problem. Second, Java is not, in fact, always Java, so different quality variants of the same implementations exist. The Debian package of pljava is compiled using gcj, but it is also planned to provide an alternative version that is compiled using the Sun JDK. That way, users can trade off quality/features vs. licensing freedom. This is the kind of freedom of choice that the current situation allows. Now the template solution will buy nothing here, because there has never been a release of PL/Java that (a) used a hard-coded shared library path, or (b) was ambiguous about its installation schema in the first place. And the validator upgrade issue will also not be solved unless there is a major synchronization effort between the PostgreSQL upstream, the PL/Java upstream, the PostgreSQL packager, and the PL/Java packager (4 different parties) (and in worse cases: the GCC packagers, etc.). In other words, it is not possible and could only be solved effectively by putting PL/Java in the core both upstream and downstream. I think you are assuming all the way through this discussion that a PostgreSQL upgrade will also entail an upgrade of all procedural languages. People are putting dummy validator functions into their code right now, but (1) will they also release that new version at the same time as PostgreSQL 8.1 comes out?, and (2) will users be willing to upgrade at the same time? Point 1 is questionable at best, and point 2 is completely out of the question, not only because I've seen too many PL releases just plain break, but simply because people may want to upgrade one thing at a time. Again, we could work around both of these points by putting the PL in question into the core, in which case I am willing to support the template idea, but for languages out of the core I don't think the implicitly required synchronization efforts have been taking into account. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Tom Lane wrote: > These are straw men. No, these are real problems that exist today. > We know we have a problem with hard-coded paths > in old dumps. This is irrelevant for non-core languages. If I have PL/foo 1.0 installed and upgrade from PostgreSQL 8.0 to 8.1, then I have to manually recompile PL/foo and install the shared library. So manual intervention is required in any case and no template system of any kind will help. Moreover, the hard-coded paths will not be solved by your proposal. I don't suppose that in a release or three will start hard coding postgis parameters to work around hard coded paths in general plug-in modules? > We know we have a problem with missing validators in > old dumps This is irrelevant for non-core languages. If I have PL/foo 1.0 installed and upgrade from PostgreSQL 8.0 to 8.1, the validator in PL/foo 1.0 will not magically appear or disappear. The information in the dump is the only accurate information that exists about the existence of a validator. The information in the template is completely unrelated to reality. You are assuming that during the update from PostgreSQL 8.0 to PostgreSQL 8.1 there will also be an upgrade from PL/foo 1.0 to PL/foo ${latest-at-time-of-8.1-release}, but there is simply no basis for assuming that. > And I think that a template is much more likely to > prevent than introduce the sorts of mis-definition problems you > suggest No, the template system introduces misdefinitions that would otherwise not exist. > --- all of them can easily happen now from simple user > errors, whereas with a template it would be right every time. As you can see above, that is not true. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Tom Lane wrote: > > How is this different from > > > 1. register language in pg_language without privileges > > 2. activate language by granting privileges > > Because you can't create a language without first creating the support > procedures, which ordinarily requires having the shared library > present. We are only talking about well-known procedural languages. The issue of completely new languages added by the user is addressed by neither proposal. > Also, ISTM your proposal is to cause "CREATE LANGUAGE foo" on an > already-existing language to execute "GRANT USAGE ON LANGUAGE foo TO > PUBLIC" instead, rather than erroring out. That doesn't seem to pass > the least-surprise test at all. Clearly, there's going to be some surprise element. The surprise element proposed by you is that the command does something completely different than specified (which possibly introduces security holes, see other mail). My proposal is that the command does only a subset of what it would normally do, which amounts to some sort of implicit "OR REPLACE", which people are familiar with. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Tom Lane wrote: > Barring further changes, we'll have a hard-wired template list for > 8.1 and a real system catalog in 8.2. So there's a choice now for > PLs that are not part of the core distribution: do you want to be > listed in the hard-wired template? Another note: PLs that are not in the core distribution are not created with createlang in the first place, so the issues that led up to this proposal don't apply AFAICT, that is: - "pg_dump dumps them at a pretty low semantic level" --> pg_dump dumps them at exactly the level they were created at. - "problem with adding a validator" --> An update of PostgreSQL does not change the properties of external modules - "issues with hardwired paths to the shared libraries" --> This is an issue with all loadable modules and needs a more general solution. Basically, on a major version upgrade, we need to make the user recompile all shared libraries. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Peter Eisentraut <[EMAIL PROTECTED]> writes: > We are only talking about well-known procedural languages. The issue of > completely new languages added by the user is addressed by neither > proposal. Sure it is: in my proposal, you can add new languages to the template catalog. (Which admittedly we don't have today. We could have it tomorrow, though, if an initdb for beta2 is OK.) >> Also, ISTM your proposal is to cause "CREATE LANGUAGE foo" on an >> already-existing language to execute "GRANT USAGE ON LANGUAGE foo TO >> PUBLIC" instead, rather than erroring out. That doesn't seem to pass >> the least-surprise test at all. > Clearly, there's going to be some surprise element. The surprise > element proposed by you is that the command does something completely > different than specified (which possibly introduces security holes, see > other mail). Haven't seen this other mail yet... > My proposal is that the command does only a subset of > what it would normally do, which amounts to some sort of implicit "OR > REPLACE", which people are familiar with. No, because what the command would normally do in that situation (the language already exists) is error out. Silently granting rights is a security risk. What if the DBA already created the language, adjusted its permissions the way he wants, and then accidentally did another CREATE LANGUAGE (or more likely, loaded a dump file containing same)? Now he's got language usage granted to PUBLIC, and he doesn't even know it. The CREATE OR REPLACE variants that we have don't muck with the object's permissions, so I don't see them as good precedent. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Attention PL authors: want to be listed in template table?
I wrote: > Another note: PLs that are not in the core distribution are not > created with createlang in the first place, so the issues that led up > to this proposal don't apply AFAICT, What I'm trying to say here is that it might be OK to hardcode the properties of core languages in the server, because considering that the only officially supported way to activate those is "createlang" (rather than using SQL), the properties of them are in effect already hardcoded, and from the point of view of a user who is using createlang, nothing changes. For languages that are not shipped in the core, we gain pretty much nothing by this approach as currently implemented and potentially introduce more problems than we solve (e.g., language is compiled in an untrusted way, but template says it's trusted; template says language has validator, but user uses old version that has none; template points to $libdir, user has it installed elsewhere). -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Peter Eisentraut <[EMAIL PROTECTED]> writes: > The other problem I see emerging here is that in certain environments, > the "java" language may not be trusted, such as when it is compiled > with GCJ. Hmm, is that really the case? I thought Java is Java. > Then, this built-in template will override the CREATE > LANGUAGE specification and introduce a security hole. But it's exactly the same hole the user would create by manually saying CREATE TRUSTED LANGUAGE in error. I don't think that's a reasonable argument against the template idea --- it just says that you have to be aware of what you're doing. (An appropriate solution, in my mind, would be to drop the trusted call handler from the shared library if it's built with gcj --- then there's really no possibility of doing the wrong thing.) regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Peter Eisentraut <[EMAIL PROTECTED]> writes: > What I'm trying to say here is that it might be OK to hardcode the > properties of core languages in the server, because considering that > the only officially supported way to activate those is > "createlang" (rather than using SQL), the properties of them are in > effect already hardcoded, and from the point of view of a user who is > using createlang, nothing changes. Please note that one of the benefits of a template catalog is that createlang will work for everything in the catalog, not only the core languages. > For languages that are not shipped > in the core, we gain pretty much nothing by this approach as currently > implemented and potentially introduce more problems than we solve > (e.g., language is compiled in an untrusted way, but template says it's > trusted; template says language has validator, but user uses old > version that has none; template points to $libdir, user has it > installed elsewhere). These are straw men. We know we have a problem with hard-coded paths in old dumps. We know we have a problem with missing validators in old dumps (which will get worse as more languages acquire validators). And I think that a template is much more likely to prevent than introduce the sorts of mis-definition problems you suggest --- all of them can easily happen now from simple user errors, whereas with a template it would be right every time. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Thomas Hallgren wrote: > If my assumptions are correct, then please add: > > { "java", true, "sqlj.java_call_handler", NULL, > "libpljava" }, > { "javaU", false, "sqlj.javau_call_handler", NULL, > "libpljava" }, In the interest of uniformity, please rename your libraries to omit the "lib" prefix. The other problem I see emerging here is that in certain environments, the "java" language may not be trusted, such as when it is compiled with GCJ. Then, this built-in template will override the CREATE LANGUAGE specification and introduce a security hole. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Thomas Hallgren <[EMAIL PROTECTED]> writes: > Wouldn't it be better if all PL's used the > dynamic_library_path setting at all times? I wouldn't think so at all. That's just another way to shoot yourself in the foot; $libdir is the correct place by definition, and any other directory is not the correct place, by definition. This is certainly true for the PLs that are part of the distribution, but I don't see that it would be any less true for add-on PLs. As an example of how to shoot yourself in the foot, consider someone setting dynamic_library_path to point to a version-specific directory: /home/postgres/version74/lib and then updating to a newer release without changing that. If there were some upside to letting people load alternate versions of PLs by changing dynamic_library_path then you might have a point ... but I really don't see any value there, just risk of breakage. >> Do you want to drop in a stub? It's only a one-line function. >> > Yes, that's a good idea. I'll call them "java_validator" and > "javau_validator" respectively. OK, I'll add that to the template info. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Tom Lane wrote: Thomas Hallgren <[EMAIL PROTECTED]> writes: I assume that the path of the shared library will be somehow relative to the GUC dynamic_library_path? Well, whatever you put in the template is what will be in the probin field of the support functions. I suppose it does not *have* to use $libdir, but I would definitely recommend using $libdir rather than depending on dynamic_library_path. I'm not I understand this. The default setting for the dynamic_library_path is $libdir, isn't it? So why have another hardwired setting here? Wouldn't it be better if all PL's used the dynamic_library_path setting at all times? I also assume that the handler name can be prefixed with a schema name? All PL/Java support functions reside in the sqlj schema. Not if you use the template facility, they won't. The handler and validator are hard-wired to live in pg_catalog under this scheme. Ok. That's fine. They're not covered by the SQL standard anyway. I have a lot of other "support functions" for managing jar files, classpath, etc. in the database. They all live in the sqlj schema but they will not be affected by this. The validator for PL/Java will have to wait until 8.2. Do you want to drop in a stub? It's only a one-line function. Yes, that's a good idea. I'll call them "java_validator" and "javau_validator" respectively. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Thomas Hallgren <[EMAIL PROTECTED]> writes: > I assume that the path of the shared library will be somehow relative to > the GUC dynamic_library_path? Well, whatever you put in the template is what will be in the probin field of the support functions. I suppose it does not *have* to use $libdir, but I would definitely recommend using $libdir rather than depending on dynamic_library_path. > I also assume that the handler name can be prefixed with a schema name? > All PL/Java support functions reside in the sqlj schema. Not if you use the template facility, they won't. The handler and validator are hard-wired to live in pg_catalog under this scheme. > The validator for PL/Java will have to wait until 8.2. Do you want to drop in a stub? It's only a one-line function. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Tom, I assume that the path of the shared library will be somehow relative to the GUC dynamic_library_path? If not, the lanlibrary should be changed to "$libdir/libpljava". That requires that PL/Java is installed within the PostgreSQL distribution. I also assume that the handler name can be prefixed with a schema name? All PL/Java support functions reside in the sqlj schema. If my assumptions are correct, then please add: { "java", true, "sqlj.java_call_handler", NULL, "libpljava" }, { "javaU", false, "sqlj.javau_call_handler", NULL, "libpljava" }, The validator for PL/Java will have to wait until 8.2. Regards, Thomas Hallgren Tom Lane wrote: I've committed changes to implement the cut-down form of this proposal: http://archives.postgresql.org/pgsql-hackers/2005-08/msg01185.php discussed here: http://archives.postgresql.org/pgsql-hackers/2005-09/msg00138.php Barring further changes, we'll have a hard-wired template list for 8.1 and a real system catalog in 8.2. So there's a choice now for PLs that are not part of the core distribution: do you want to be listed in the hard-wired template? The advantages of being listed are: 1. Reloading old dumps that involve your language should be easier, since problems like version-specific paths to shared libraries will go away. 2. Your PL support functions will end up in pg_catalog instead of the public schema, which will please people who'd like to remove public from their installations. The main disadvantage I can see is that you won't easily be able to change your PL creation parameters (eg, add a validator function) over the lifespan of the 8.1 release. So depending on your development roadmap you might think this a bad tradeoff. If you want to be listed, let me know. What I need to know to list you is values for this table: typedef struct { char *lanname; /* PL name */ boollantrusted; /* trusted? */ char *lanhandler; /* name of handler function */ char *lanvalidator;/* name of validator function, or NULL */ char *lanlibrary; /* path of shared library */ } PLTemplate; As examples, the entries for the core PLs are { "plpgsql", true, "plpgsql_call_handler", "plpgsql_validator", "$libdir/plpgsql" }, { "pltcl", true, "pltcl_call_handler", NULL, "$libdir/pltcl" }, { "pltclu", false, "pltclu_call_handler", NULL, "$libdir/pltcl" }, { "plperl", true, "plperl_call_handler", "plperl_validator", "$libdir/plperl" }, { "plperlu", false, "plperl_call_handler", "plperl_validator", "$libdir/plperl" }, { "plpythonu", false, "plpython_call_handler", NULL, "$libdir/plpython" }, regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Peter Eisentraut <[EMAIL PROTECTED]> writes: > So your proposal is to enable a new language by doing: > 1. register a template > 2. activate template using CREATE LANGUAGE (which would copy it to > pg_language) That's the long-term idea, yes. > How is this different from > 1. register language in pg_language without privileges > 2. activate language by granting privileges Because you can't create a language without first creating the support procedures, which ordinarily requires having the shared library present. (This is why I proposed text names for the support procedures in pltemplate, rather than OID references.) This is perhaps not an issue for entries wired in by initdb, but it's definitely an issue for manual addition of template entries. Also, ISTM your proposal is to cause "CREATE LANGUAGE foo" on an already-existing language to execute "GRANT USAGE ON LANGUAGE foo TO PUBLIC" instead, rather than erroring out. That doesn't seem to pass the least-surprise test at all. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Tom Lane wrote: > I don't see it as an exception list. The direction I see for this is > that the parameters to CREATE LANGUAGE are obsolete and will > eventually be removed altogether, with "CREATE LANGUAGE foo" using an > existing template as the only recommended way to do it. So your proposal is to enable a new language by doing: 1. register a template 2. activate template using CREATE LANGUAGE (which would copy it to pg_language) How is this different from 1. register language in pg_language without privileges 2. activate language by granting privileges This already works and uses only well-known concepts. > > Why don't we just put all PLs that we > > know of into pg_language to begin with, revoke the permissions, and > > just let CREATE LANGUAGE fail when the dump is restored? > > To do that we'd have to force an initdb, in which case we might as > well add the proposed pltemplate catalog and have done with it. I don't have a strong opinion on initdb, but a difference would be that this solution would not *require* an initdb but only offer the improvement if initdb were done while continuing to work as before without initdb. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Attention PL authors: want to be listed in template
Peter Eisentraut wrote: Andrew Dunstan wrote: Anyway, clearly there are cases where a validator make no sense or very little sense (pl/sh and pl/{j,java} spring to mind). PL/sh has a validator. :-) Really? Curious. I must look more closely. cheers andrew ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Peter Eisentraut <[EMAIL PROTECTED]> writes: > I don't doubt that, but I just have a stomach ache with this entire > notion of having a hard-coded exception list of the sort "if the user > requires this, do this instead". I don't see it as an exception list. The direction I see for this is that the parameters to CREATE LANGUAGE are obsolete and will eventually be removed altogether, with "CREATE LANGUAGE foo" using an existing template as the only recommended way to do it. > Why don't we just put all PLs that we > know of into pg_language to begin with, revoke the permissions, and > just let CREATE LANGUAGE fail when the dump is restored? To do that we'd have to force an initdb, in which case we might as well add the proposed pltemplate catalog and have done with it. I'm entirely willing to add the catalog if people are willing to accept an initdb now. As I said earlier, I don't want to mess with adding a usage privilege just yet, but we could add the catalog with a nonfunctional ACL column and write that code later. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Tom Lane wrote: > There is deliberately not a way to override (other than using a > different name for the PL). One of the points of this facility is to > fix up broken PL definitions being imported from old dump files, so > believing what the CREATE LANGUAGE command says is exactly what we > don't want. I don't doubt that, but I just have a stomach ache with this entire notion of having a hard-coded exception list of the sort "if the user requires this, do this instead". Why don't we just put all PLs that we know of into pg_language to begin with, revoke the permissions, and just let CREATE LANGUAGE fail when the dump is restored? (The dump should restore the permissions, shouldn't it?) At least that would let you alter the "template" using existing means. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Andrew Dunstan wrote: > Anyway, clearly there are cases where a validator make no sense or > very little sense (pl/sh and pl/{j,java} spring to mind). PL/sh has a validator. :-) -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Attention PL authors: want to be listed in template
Tom Lane wrote: For instance, if you think you might want a validator later, you can set up a no-op validator procedure today, and then the template doesn't need to change when you make the validator do something. Similarly, you could future-proof yourself against adding a trusted (or untrusted) variant by putting in some stubs now. Only necessary for one release, right? Anyway, clearly there are cases where a validator make no sense or very little sense (pl/sh and pl/{j,java} spring to mind). cheers andrew ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Is there any way to create the language and not use the template (an > override option of some kind)? There is deliberately not a way to override (other than using a different name for the PL). One of the points of this facility is to fix up broken PL definitions being imported from old dump files, so believing what the CREATE LANGUAGE command says is exactly what we don't want. This will definitely be more palatable with a system catalog that you could alter, of course. Still, I don't see that there's any big problem. For instance, if you think you might want a validator later, you can set up a no-op validator procedure today, and then the template doesn't need to change when you make the validator do something. Similarly, you could future-proof yourself against adding a trusted (or untrusted) variant by putting in some stubs now. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Attention PL authors: want to be listed in template table?
Tom Lane wrote: > Barring further changes, we'll have a hard-wired template list for > 8.1 and a real system catalog in 8.2. So there's a choice now for > PLs that are not part of the core distribution: do you want to be > listed in the hard-wired template? Is there any way to create the language and not use the template (an override option of some kind)? -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Attention PL authors: want to be listed in template
Tom Lane wrote: If you want to be listed, let me know. What I need to know to list you is values for this table: typedef struct { char *lanname; /* PL name */ boollantrusted; /* trusted? */ char *lanhandler; /* name of handler function */ char *lanvalidator;/* name of validator function, or NULL */ char *lanlibrary; /* path of shared library */ } PLTemplate; Hi Tom, Please include PL/R: { "plr", false, "plr_call_handler", NULL, "$libdir/plr" }, Thanks, Joe p.s. my (supposedly) static IP changed this summer -- I think my direct mail might be getting blocked by your spam filters again. ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] Attention PL authors: want to be listed in template table?
I've committed changes to implement the cut-down form of this proposal: http://archives.postgresql.org/pgsql-hackers/2005-08/msg01185.php discussed here: http://archives.postgresql.org/pgsql-hackers/2005-09/msg00138.php Barring further changes, we'll have a hard-wired template list for 8.1 and a real system catalog in 8.2. So there's a choice now for PLs that are not part of the core distribution: do you want to be listed in the hard-wired template? The advantages of being listed are: 1. Reloading old dumps that involve your language should be easier, since problems like version-specific paths to shared libraries will go away. 2. Your PL support functions will end up in pg_catalog instead of the public schema, which will please people who'd like to remove public from their installations. The main disadvantage I can see is that you won't easily be able to change your PL creation parameters (eg, add a validator function) over the lifespan of the 8.1 release. So depending on your development roadmap you might think this a bad tradeoff. If you want to be listed, let me know. What I need to know to list you is values for this table: typedef struct { char *lanname; /* PL name */ boollantrusted; /* trusted? */ char *lanhandler; /* name of handler function */ char *lanvalidator;/* name of validator function, or NULL */ char *lanlibrary; /* path of shared library */ } PLTemplate; As examples, the entries for the core PLs are { "plpgsql", true, "plpgsql_call_handler", "plpgsql_validator", "$libdir/plpgsql" }, { "pltcl", true, "pltcl_call_handler", NULL, "$libdir/pltcl" }, { "pltclu", false, "pltclu_call_handler", NULL, "$libdir/pltcl" }, { "plperl", true, "plperl_call_handler", "plperl_validator", "$libdir/plperl" }, { "plperlu", false, "plperl_call_handler", "plperl_validator", "$libdir/plperl" }, { "plpythonu", false, "plpython_call_handler", NULL, "$libdir/plpython" }, regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org