Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Peter Eisentraut
On Sunday 28 June 2009 20:38:59 Robert Haas wrote: The only other use case I can think of for functionality of this type is some kind of dashboard view on a system with very long-running queries, where you want to see how far you have yet to go on each one (maybe to approximate when you can

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Scara Maccai
You might want to take a look at this: http://wiki.postgresql.org/wiki/Submitting_a_Patch I will; I'm sorry it wasn't in the proper format. It was just a proof of concept, I guess I should have talked about it before even sending the patch. As to the content of the patch, I think that what

Re: [HACKERS] Proposal: More portable way to support 64bit platforms

2009-06-29 Thread Tsutomu Yamada
Peter Eisentraut pete...@gmx.net wrote: On Friday 26 June 2009 12:07:24 Tsutomu Yamada wrote: Proposal: More portable way to support 64bit platforms Short description: Current PostgreSQL implementation has some portability issues to support 64bit platforms: pointer calculations

Re: [HACKERS] Extensions User Design

2009-06-29 Thread Peter Eisentraut
On Thursday 25 June 2009 01:09:17 Andrew Dunstan wrote: Well, I think in our case that would be going too far. I think there is a very good case for keeping a few key extensions in core both as exemplars and to make it easy to validate the extension mechanism itself. There have been

Re: [HACKERS] Extensions User Design

2009-06-29 Thread Richard Huxton
Peter Eisentraut wrote: Another thing we might want to consider once we have a robust extension mechanism is to move some things out of the backend into extensions. Candidates could be uuid, legacy geometry types, inet/cidr, for example. These extensions would still be available and

[HACKERS] [PATCH] user mapping extension to pg_ident.conf

2009-06-29 Thread Lars Kanis
Hi all, this patch adds the possibility to map the login-rolename to a different rolename actually used for permissions. What is it used for? I'm working with smartcard based TLS-authentication to connect to the PG server. Authentication is done with the keys and certificates from the card

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Peter Hunsberger
On Mon, Jun 29, 2009 at 3:47 AM, Scara Maccaim_li...@yahoo.it wrote: Is anyone interested in such a progress indicator??? I'm relatively new to Postgres and just starting to look at starting to look at what we might do with it for handling large genomic datasets. I've used Toad for Oracle to

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Robert Haas
On Mon, Jun 29, 2009 at 4:47 AM, Scara Maccaim_li...@yahoo.it wrote: As to the content of the patch, I think that what you are doing is comparing the actual number of operations with the expected number of operations.  If that's correct, I'm not sure it's really all that useful, because it

Re: [HACKERS] Proposal: More portable way to support 64bit platforms

2009-06-29 Thread Tom Lane
Tsutomu Yamada tsut...@sraoss.co.jp writes: Yes, I have read through the discusion but it seems somewhat faded out. This is because no platform other than Windows has 64bit pointer issues IMO. I think using intptr_t is cleaner and will bring more portability. Moreover it will solve Windows

Re: [HACKERS] [PATCH] user mapping extension to pg_ident.conf

2009-06-29 Thread Tom Lane
Lars Kanis ka...@comcard.de writes: this patch adds the possibility to map the login-rolename to a different rolename actually used for permissions. This seems like an ugly addition with a very narrow use case. Can't you accomplish what you want with the existing usermap facility?

Re: [HACKERS] Proposal: More portable way to support 64bit platforms

2009-06-29 Thread Peter Eisentraut
On Monday 29 June 2009 17:20:09 Tom Lane wrote: The problem with this is that it's barely the tip of the iceberg. One point I recall is that there are lots of places where %lu is assumed to be the correct format to print Datums with. Hmm. I tried this out. typedef Datum to be long long int

Re: [HACKERS] Proposal: More portable way to support 64bit platforms

2009-06-29 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On Monday 29 June 2009 17:20:09 Tom Lane wrote: If it were actually possible to support Win64 with only a couple of dozen lines of changes, we would have done it long since. Possibly, or everyone was too confused and didn't know where to start. Well,

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread m_lists
It's easy to have estimates that are off by a factor of two or three, though, so I think you'd frequently have situations when the query completed when the progress estimater was at 40% or 250%. I thought about implementing a given perfect estimates indicator at first then, as a second

Re: [HACKERS] [PATCH] user mapping extension to pg_ident.conf

2009-06-29 Thread Lars Kanis
Am Montag, 29. Juni 2009 16:26:56 schrieben Sie: Lars Kanis ka...@comcard.de writes: this patch adds the possibility to map the login-rolename to a different rolename actually used for permissions. This seems like an ugly addition with a very narrow use case. Can't you accomplish what you

Re: [HACKERS] [PATCH] user mapping extension to pg_ident.conf

2009-06-29 Thread Tom Lane
Lars Kanis ka...@comcard.de writes: The problem I have, is that I want to use an ordinary windows application, which connects to an arbitrary ODBC data source. This application stores a fixed username und password for the connection within it's own binary data file. It doesn't know anything

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Scara Maccai
IMO any diagnostics you can provide for a low cost are useful.  The more detail, the better.  Step 1 of 10 is good, 80% complete on step 1 of 10 is better.  80% complete on step 1, 10% complete on 10 steps is even better. Well, I guess Step 1 of 10 would be pretty trivial to implement

[HACKERS] Multi-Dimensional Histograms

2009-06-29 Thread David Fetter
Folks, For things like PostGIS, which will want to index in 4 dimensions (x, y, z, t), we might want to have multi-dimensional selectivity histograms and some way to use same. Anybody here qualified to check out this paper on the subject, please speak up :)

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: David Fetter wrote: There have been previous discussions of prospective permissions changes. Are we restarting them here? It's not on the TODO list. I recall it being raised from time to time but I certainly don't recall a consensus that it should

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Jeff Davis
On Mon, 2009-06-29 at 12:55 -0400, Tom Lane wrote: I think it has to be looked at in comparison to more general prospective-permissions schemes; When I searched google for prospective permissions, all I found were links to messages in this thread ;) Can you refer me to a general

[HACKERS] pg_restore -t table concerns

2009-06-29 Thread Mike Toews
Hey hackers, I have a few concerns with the usability and documentation for pg_restore (note: I'm on 8.3, but I've checked the documentation for 8.4). In my attempts to restore a table (using -t) from a backup file with -v (verbose) selected, I noticed a few things: 1. If a non-existent

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: Can you refer me to a general prospective-permissions scheme that is more widely accepted? Well, the point of my post was that nothing's gotten to the point of being widely accepted. But there are people working on a default ACLs scheme that would cover

Re: [HACKERS] pg_restore -t table concerns

2009-06-29 Thread Tom Lane
Mike Toews mwto...@sfu.ca writes: I have a few concerns with the usability and documentation for pg_restore (note: I'm on 8.3, but I've checked the documentation for 8.4). There's a TODO item about that already... * Add support for multiple pg_restore -t options, like pg_dump

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Andrew Dunstan
Jeff Davis wrote: On Mon, 2009-06-29 at 12:55 -0400, Tom Lane wrote: I think it has to be looked at in comparison to more general prospective-permissions schemes; When I searched google for prospective permissions, all I found were links to messages in this thread ;) Can you refer

Re: [HACKERS] pg_restore -t table concerns

2009-06-29 Thread Andrew Dunstan
Tom Lane wrote: Mike Toews mwto...@sfu.ca writes: I have a few concerns with the usability and documentation for pg_restore (note: I'm on 8.3, but I've checked the documentation for 8.4). There's a TODO item about that already... * Add support for multiple pg_restore -t options,

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Josh Berkus
All, First, let me talk about the problem: it's been my observation that the majority of users, including public commercial web sites, which I run into in the field do not employ permissions in any useful way to protect their data. An awful lot of these applications are running as the

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Josh Berkus
All, Actually, an indicator of even just what step of the query was being executed would be very useful for checking on stuck queries. If a DBA checks once that the query is on bitmapscan on table_x(index_y), and it's still on that 15 minutes later, he/she can guess that the query is

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: So, while an actual % completed indicator would be perfect, a query steps completed, current step = would still be very useful and a large improvement over what we have now. I think this is pretty much nonsense --- most queries run all their plan nodes

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Jeff Davis
On Mon, 2009-06-29 at 10:52 -0700, Josh Berkus wrote: 1) ALTER SCHEMA SET DEFAULT PRIVILEGES statements which sets default permissions, by ROLE and object type, on new objects. 2) a statement to set privs on all existing objects by type and role within a schema. I don't see why either of

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Robert Haas
On Mon, Jun 29, 2009 at 11:15 AM, m_li...@yahoo.it wrote: So the code that I have right now works pretty well for the 10 queries of my project, but I guess won't work for general queries :( I think that's probably right. So, I'm all in favor of what you're trying to conceptually; I just

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Joshua Tolley
On Mon, Jun 29, 2009 at 02:07:23PM -0400, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: So, while an actual % completed indicator would be perfect, a query steps completed, current step = would still be very useful and a large improvement over what we have now. I think this is

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: On Mon, 2009-06-29 at 10:52 -0700, Josh Berkus wrote: 1) ALTER SCHEMA SET DEFAULT PRIVILEGES statements which sets default permissions, by ROLE and object type, on new objects. 2) a statement to set privs on all existing objects by type and role within

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Tom Lane
Joshua Tolley eggyk...@gmail.com writes: On Mon, Jun 29, 2009 at 02:07:23PM -0400, Tom Lane wrote: I think this is pretty much nonsense --- most queries run all their plan nodes concurrently to some extent. You can't usefully say that a query is on some node, nor measure progress by whether

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Josh Berkus
Jeff, I don't see why either of these things should be properties of the schema. It seems to make much more sense for these defaults to be a property of the user who creates the objects. The main reason is existing practice. Currently, most applications I see in the field which bother with

Re: [HACKERS] [PATCH] [v8.5] Security checks on largeobjects

2009-06-29 Thread Bernd Helmle
--On 29. Juni 2009 08:32:29 +0900 KaiGai Kohei kai...@ak.jp.nec.com wrote: Yes, it intends to assign an identifier string not only numeric large object identifier. The identifier string can be qualified with a certain namespace as follows. E.g) SELECT lo_open('my_picture01', x'4'::int);

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: The second, and bigger problem I can see is that this opens a whole new set of security holes by allowing an end-run around the existing access control structure with attackers can try to exploit. Yeah. I'm very concerned about any scheme that invents

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Peter Hunsberger
On Mon, Jun 29, 2009 at 1:07 PM, Tom Lanet...@sss.pgh.pa.us wrote: Josh Berkus j...@agliodbs.com writes: So, while an actual % completed indicator would be perfect, a query steps completed, current step = would still be very useful and a large improvement over what we have now. I think this

Re: [HACKERS] [PATCH] [v8.5] Security checks on largeobjects

2009-06-29 Thread Tom Lane
Bernd Helmle maili...@oopsware.de writes: It might be interesting to dig into your proposal deeper in conjunction with TOAST (you've already mentioned this TODO). Having serial access with a nice interface into TOAST would be eliminating the need for pg_largeobject completely (i'm not a big

Re: [HACKERS] [PATCH] user mapping extension to pg_ident.conf

2009-06-29 Thread Stephen Frost
Lars, * Lars Kanis (ka...@comcard.de) wrote: The problem I have, is that I want to use an ordinary windows application, which connects to an arbitrary ODBC data source. This application stores a fixed username und password for the connection within it's own binary data file. It doesn't

Re: [HACKERS] Problem building from source on Windows

2009-06-29 Thread Ms swati chande
Thanks for your help. It worked. Regards Swati -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] facing problem with createdb.

2009-06-29 Thread Ms swati chande
Hi, I built postgresql from source using Windows 2005. After installation, I am trying to create a database using createdb. It then asks for a password. What password is it? There are now passwords set on my system. Also tried editing pg_hba.conf. But could not solve the problem. How do I

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Josh Berkus
Tom, From what I recall of prior discussions, there is rough consensus that the two types of facilities you mentioned (setting up default ACLs to be applied at creation of objects created later, and providing a way to change multiple objects' permissions with one GRANT) are desirable, though

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Greg Stark
On Mon, Jun 29, 2009 at 7:41 PM, Josh Berkusj...@agliodbs.com wrote: The main reason is existing practice. I haven't followed the entire conversation so i'm not sure who I'm going to be disagreeing with or agreeing with here. But I wanted to mention that existing practice may not be a very

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Josh Berkus
Greg, In particular, one early question was whether to use wildcard patterns or schema names. People were saying wildcard patterns would be more flexible because people don't always set up their objects in different schemas. But if we had a mechanism someone wanted to use which depended on

Re: [HACKERS] Multi-Dimensional Histograms

2009-06-29 Thread Nathan Boley
For things like PostGIS, which will want to index in 4 dimensions (x, y, z, t), we might want to have multi-dimensional selectivity histograms and some way to use same. Another use case is cross column statistics. Anybody here qualified to check out this paper on the subject, please speak

Re: [HACKERS] Multi-Dimensional Histograms

2009-06-29 Thread David Fetter
On Mon, Jun 29, 2009 at 01:28:01PM -0700, Nathan Boley wrote: For things like PostGIS, which will want to index in 4 dimensions (x, y, z, t), we might want to have multi-dimensional selectivity histograms and some way to use same. Another use case is cross column statistics. Good to see

Re: [HACKERS] facing problem with createdb.

2009-06-29 Thread Filip Rembiałkowski
2009/6/29 Ms swati chande swat...@yahoo.com Hi, I built postgresql from source using Windows 2005. Did you read http://wiki.postgresql.org/wiki/Running_%26_Installing_PostgreSQL_On_Native_Windows? Why not use prebuilt versions (see http://www.postgresql.org/download/windows)? After

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Simon Riggs
On Mon, 2009-06-29 at 14:07 -0400, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: So, while an actual % completed indicator would be perfect, a query steps completed, current step = would still be very useful and a large improvement over what we have now. I think this is pretty

Re: [HACKERS] Multi-Dimensional Histograms

2009-06-29 Thread Tom Lane
David Fetter da...@fetter.org writes: On Mon, Jun 29, 2009 at 01:28:01PM -0700, Nathan Boley wrote: ... They dismiss singular value decomposition and the discrete wavelet transform as being too parametric ( which is silly, IMHO ) Should we have a separate discussion about eigenvalues?

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Mon, 2009-06-29 at 14:07 -0400, Tom Lane wrote: I think this is pretty much nonsense --- most queries run all their plan nodes concurrently to some extent. You can't usefully say that a query is on some node, nor measure progress by whether some

Re: [HACKERS] Multi-Dimensional Histograms

2009-06-29 Thread David Fetter
On Mon, Jun 29, 2009 at 06:43:35PM -0400, Tom Lane wrote: David Fetter da...@fetter.org writes: On Mon, Jun 29, 2009 at 01:28:01PM -0700, Nathan Boley wrote: ... They dismiss singular value decomposition and the discrete wavelet transform as being too parametric ( which is silly, IMHO )

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Greg Stark
On Mon, Jun 29, 2009 at 9:02 PM, Josh Berkusj...@agliodbs.com wrote: What I'm saying is that there are many users currently using schema for security classes.  I personally haven't ever encountered a DBA who used role ownership of objects as a mechanism for security context.  There's nothing

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Greg Stark
On Mon, 2009-06-29 at 14:07 -0400, Tom Lane wrote: I think this is pretty much nonsense --- most queries run all their plan nodes concurrently to some extent.  You can't usefully say that a query is on some node, nor measure progress by whether some node is done. Right, that was why my

Re: [HACKERS] Multi-Dimensional Histograms

2009-06-29 Thread Nathan Boley
Finally, this creates the partition but ( AFAICT ) it doesn't describe a method for locating the histogram estimate given a point ( although that doesn't seem too difficult ). Is that not difficult, in terms of the math that needs doing, or not difficult, in terms of how well PostgreSQL is

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Ron Mayer
Greg Stark wrote: Right, that was why my proposed interface was to dump out the explain plan with the number of loops, row counts seen so far, and approximate percentage progress. My thinking was that a human could interpret that to understand where the bottleneck is if, say you're still on the

Re: [HACKERS] Multi-Dimensional Histograms

2009-06-29 Thread Nathan Boley
On Mon, Jun 29, 2009 at 3:43 PM, Tom Lanet...@sss.pgh.pa.us wrote: David Fetter da...@fetter.org writes: On Mon, Jun 29, 2009 at 01:28:01PM -0700, Nathan Boley wrote: ... They dismiss singular value decomposition and the discrete wavelet transform as being too parametric ( which is silly,

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Josh Berkus
Greg, And there's I just created a new table, I want www and www-backend to get their usual privileges without thinking about it. You want to be able to specify default grants that an object gets based on the schema? That seems mostly reasonable though it might be a good idea to have a WITH

[HACKERS] How to register my function into backend?

2009-06-29 Thread Bruce YUAN
Hi All, My function is to collect some backend information for user anlysis. How to register my function into backend? It make that we can called it via libpg/PQfn(). Thanks! Best regards, Bruce

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Robert Haas
On Mon, Jun 29, 2009 at 8:15 PM, Ron Mayerrm...@cheapcomplexdevices.com wrote: Greg Stark wrote: Right, that was why my proposed interface was to dump out the explain plan with the number of loops, row counts seen so far, and approximate percentage progress. My thinking was that a human

Re: [HACKERS] How to register my function into backend?

2009-06-29 Thread Robert Haas
On Mon, Jun 29, 2009 at 4:53 AM, Bruce YUANsua...@gmail.com wrote: My function is to collect some backend information for user anlysis. How to register my function into backend? It make that we can called it via libpg/PQfn(). Thanks! I'm not sure this is -hackers question; seems like -general

Re: [HACKERS] Multi-Dimensional Histograms

2009-06-29 Thread Robert Haas
On Mon, Jun 29, 2009 at 8:17 PM, Nathan Boleynpbo...@gmail.com wrote: On Mon, Jun 29, 2009 at 3:43 PM, Tom Lanet...@sss.pgh.pa.us wrote: David Fetter da...@fetter.org writes: On Mon, Jun 29, 2009 at 01:28:01PM -0700, Nathan Boley wrote: ... They dismiss singular value decomposition and the

Re: [HACKERS] [PATCH] [v8.5] Security checks on largeobjects

2009-06-29 Thread KaiGai Kohei
Tom Lane wrote: Bernd Helmle maili...@oopsware.de writes: It might be interesting to dig into your proposal deeper in conjunction with TOAST (you've already mentioned this TODO). Having serial access with a nice interface into TOAST would be eliminating the need for pg_largeobject

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Greg Stark
On Tue, Jun 30, 2009 at 1:51 AM, Josh Berkusj...@agliodbs.com wrote: It does seems slightly silly since surely anyone creating a new object would just paste in their grants from another object or some common source anyways, but I suppose that's the way with convenience features. That works

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Josh Berkus
Greg, Well I don't understand how you get them wrong if you're just pasting them from a file. I mean, sure you can pick the wrong template but nothing can help you there. You could just as easily pick the wrong template if it's a database feature instead of a text file. I really have to

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread David Fetter
On Tue, Jun 30, 2009 at 04:24:40AM +0100, Greg Stark wrote: On Tue, Jun 30, 2009 at 1:51 AM, Josh Berkusj...@agliodbs.com wrote: It does seems slightly silly since surely anyone creating a new object would just paste in their grants from another object or some common source anyways, but I

Re: [HACKERS] pre-proposal: permissions made easier

2009-06-29 Thread Greg Stark
On Tue, Jun 30, 2009 at 4:39 AM, Josh Berkusj...@agliodbs.com wrote: Well I don't understand how you get them wrong if you're just pasting them from a file. I mean, sure you can pick the wrong template but nothing can help you there. You could just as easily pick the wrong template if it's a

Re: [HACKERS] Multi-Dimensional Histograms

2009-06-29 Thread Joshua Tolley
On Mon, Jun 29, 2009 at 10:22:15PM -0400, Robert Haas wrote: I'm finding myself unable to follow all the terminology on this thead. What's dimension reduction? For instance, ask a bunch of people a bunch of survey questions, in hopes of predicting some value (for instance, whether or not

[HACKERS] foreign.h is not installed

2009-06-29 Thread Itagaki Takahiro
Header file include/foreign/foreign.h is not installed in 8.4.0. Did we forget to add subdir foreign to Makefile? Index: src/include/Makefile === --- src/include/Makefile(8.4.0) +++ src/include/Makefile(fixed) @@

Re: [HACKERS] Query progress indication - an implementation

2009-06-29 Thread Dimitri Fontaine
Le 30 juin 2009 à 01:34, Greg Stark gsst...@mit.edu a écrit : Basically I disagree that imperfect progress reports annoy users. I think we can do better than reporting 250% done or having a percentage that goes backward though. It would be quite tolerable (though perhaps for no logical reason)

[HACKERS] 8.5 development schedule

2009-06-29 Thread Peter Eisentraut
Now that 8.4.0 is out the door, development for 8.5devel will be opened any day now. But we haven't discussed the development timeline so far. The core team has several proposals: CommitFest Alpha Aug. 1 Sept. 1 Oct. 1 Nov. 1 Dec. 1 Jan ~~ 5 Feb. 1