Re: [HACKERS] Re: [Pljava-dev] Should creating a new base type require superuser status?
Tom Lane wrote: This is a non-issue in PL/Java. An integer parameter is never passed by reference and there's no way the PL/Java user can get direct access to backend memory. So what exactly does happen when the user deliberately specifies wrong typlen/typbyval/typalign info when creating a type based on PL/Java functions? Everything is converted into instances of Java classes such as String, byte[], etc. I think that assumption is without ground. Java doesn't permit you to access memory unless you use Java classes (java.nio stuff) that is explicitly designed to do that and you need native code to set such things up. A PL/Java user can not do that unless he is able to link in other shared objects or dll's to the backend process. PL/Java itself must be doing "unsafe" things in order to interface with PG at all. So what your argument really is is that you have managed to securely sandbox the user-written code you are calling. That might or might not be true, but I don't think that worrying about it is without foundation. I would be presumptuous to claim that I provide the sandbox. All PL/Java does is to provide the type mapping. The sandbox as such is implicit in Java, much in the same way that it does it for web-browsers etc. Regardless of that, I think there's some difference in expressing a worry that might or might not have a foundation versus claiming that there indeed must be a security hole a mile wide ;-) - thomas -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
Robert Treat <[EMAIL PROTECTED]> writes: > Certainly there isn't any reason to allow a reload of a file that is just > going to break things when the first connection happens. For that matter, > why would we ever not want to parse it at HUP time rather than connect time? Two or three reasons why not were already mentioned upthread, but for the stubborn, here's another one: are you volunteering to write the code that backs out the config-file reload after the checks have determined it was bad? Given the amount of pain we suffered trying to make GUC do something similar, any sane person would run screaming from the prospect. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
"Joshua D. Drake" <[EMAIL PROTECTED]> writes: > True enough but perhaps that is a problem in itself. IMO, we should be > encouraging people to never touch the postgres binary. I don't buy that at all. pg_ctl is useful for some people and not so useful for others; in particular, from the perspective of a system startup script it tends to get in the way more than it helps. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] WITH RECURSIVE patches 0803
Hi, Included are the latest WITH RECURSIVE patches against CVS HEAD. The main differences from previous patches include: - Allow multiple query names (mutual recursion is not still allowed) These are some examples from the regression test: WITH RECURSIVE x(id) AS (SELECT * FROM y UNION ALL SELECT id+1 FROM x WHERE id < 5), y(id) AS (values (1)) SELECT * FROM x; WITH RECURSIVE x(id) AS (SELECT 1 UNION ALL SELECT id+1 FROM x WHERE id < 3 ), y(id) AS (SELECT * FROM x UNION ALL SELECT * FROM x), z(id) AS (SELECT * FROM y UNION ALL SELECT id+1 FROM z WHERE id < 10) SELECT * FROM z; - Fix some cases where target list has subquries WITH RECURSIVE t(id) AS ( SELECT (VALUES(1)) UNION ALL SELECT id+1 FROM t WHERE id < 5 ) SELECT * FROM t; Remaining works: 1) write sgml docs 2) write README Currently I'm writing 2) based on the one posted before. BTW, I'm traveling to the United States from Aug 4 to Aug 12. I'm going to join Linux World, PGDay and pgpool party at Bruce's. I hope to meet many community poeple soon! -- Tatsuo Ishii SRA OSS, Inc. Japan recursive_query.patch.gz Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
On Saturday 02 August 2008 13:36:40 Magnus Hagander wrote:
> Josh Berkus wrote:
> > Tom,
> >
> >> Seems a lot better to me to just train people to run the check-config
> >> code by hand before pulling the trigger to load the settings for real.
> >
> > Or just have pg_ctl --check-first as an option. Cautious DBAs would use
> > that. In 2-3 versions, we can make --check-first the default, and
> > require the option --dont-check for unattended restarts.
>
> If the config file is known to be broken, why should you *ever* allow it
> to try to restart it?
>
Certainly there isn't any reason to allow a reload of a file that is just
going to break things when the first connection happens. For that matter,
why would we ever not want to parse it at HUP time rather than connect time?
--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
Joshua D. Drake wrote: > Alvaro Herrera wrote: >> The problem with pg_ctl is that it's indirectly calling postgres, and it >> doesn't have a lot of a way to know what happened after calling it; >> consider the mess we have with pg_ctl -w. > > True enough but perhaps that is a problem in itself. IMO, we should be > encouraging people to never touch the postgres binary. If that means > pg_ctl becomes a lot smarter, then we have to consider that as well. That's a great idea, but I don't think we should weigh down this idea of config checking with that responsability. We could discuss solutions to this problem, but I think it's going to be quite a lot harder than you seem to think. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Review: DTrace probes (merged version) ver_03
On Fri, 1 Aug 2008, Alvaro Herrera wrote: Sounds like the thing to do would be to pass CheckpointStats into the DONE probe. I thought it would be more useful to demarcate where the two phases of the checkpoint process were at clearly--the actual times themselves are helpful but dtrace can do more than that. The write and sync phases have very different I/O characteristics, and it really should be easy for people to analyze them independantly. I'm not familiar enough with dtrace to know if that's easy with the patch as it stands, I think there needs to be another probe in the middle there to make that straightforward. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Mini improvement: statement_cost_limit
On Aug 2, 2008, at 8:38 PM, Tom Lane wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: Hans-Jürgen Schönig wrote: i introduced a GUC called statement_cost_limit which can be used to error out if a statement is expected to be too expensive. You clearly have far more faith in the cost estimates than I do. Wasn't this exact proposal discussed and rejected awhile back? regards, tom lane i don't remember precisely. i have seen it on simon's wiki page and it is something which would have been useful in some cases in the past. many thanks, hans -- Cybertec Schönig & Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: www.postgresql-support.de -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
Alvaro Herrera wrote: Joshua D. Drake wrote: Tom Lane wrote: Doesn't it seem reasonable that it should be pg_ctl? You should never run postgres directly unless it is for DR. What on earth is DR? Disaster Recovery. The problem with pg_ctl is that it's indirectly calling postgres, and it doesn't have a lot of a way to know what happened after calling it; consider the mess we have with pg_ctl -w. True enough but perhaps that is a problem in itself. IMO, we should be encouraging people to never touch the postgres binary. If that means pg_ctl becomes a lot smarter, then we have to consider that as well. Comparatively if I do a apachectl configtest it tells me if it is correct. Now I assume it is actually calling httpd (I haven't checked) but the point is, I am not calling httpd. Sincerely, Joshua D. Drake -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
Joshua D. Drake wrote: > Tom Lane wrote: >> I'd go for just >> >> postgres --check-config -D $PGDATA >> >> (In a reload scenario, you'd edit the files in-place and then do this >> before issuing SIGHUP.) Sounds good. > Doesn't it seem reasonable that it should be pg_ctl? You should never > run postgres directly unless it is for DR. What on earth is DR? The problem with pg_ctl is that it's indirectly calling postgres, and it doesn't have a lot of a way to know what happened after calling it; consider the mess we have with pg_ctl -w. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WITH RECUSIVE patches 0723
On Sat, Aug 02, 2008 at 12:33:38AM -0400, Tom Lane wrote: > grammar. Right now the situation is that Michael Meskes makes a manual > cleanup pass every so often :-(. I would like to see that get automated > sooner rather than later, but in the near term it is not the I received a very promising first version of an automation script from Mike Auberry. Waiting for the next one now. I'm hopeful that we are not too far away. Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: [EMAIL PROTECTED] Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Mini improvement: statement_cost_limit
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Hans-Jürgen Schönig wrote: >> i introduced a GUC called statement_cost_limit which can be used to >> error out if a statement is expected to be too expensive. > You clearly have far more faith in the cost estimates than I do. Wasn't this exact proposal discussed and rejected awhile back? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WITH RECUSIVE patches 0723
On Sat, Aug 02, 2008 at 12:35:55AM +0100, Andrew Gierth wrote: > One more oversight: the patch isn't updating the ECPG preproc.y other > than trivially, so WITH queries aren't working in ecpg: I'll take care of this as soon as the patch settles down enough, gets included into our CVS or into its own accessible source archive. It doesn't make sense IMO to bloat the patch with code not needed for the backend functionality at this point in time. Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: [EMAIL PROTECTED] Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Mini improvement: statement_cost_limit
Hans-Jürgen Schönig wrote: hello ... i picked up csaba nagy's idea and implemented a very simple yet very useful extension. i introduced a GUC called statement_cost_limit which can be used to error out if a statement is expected to be too expensive. the advantage over statement_timeout is that we are actually able to error out before spending many seconds which is killed by statement_timeout anyway. You clearly have far more faith in the cost estimates than I do. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Mini improvement: statement_cost_limit
hello ... i picked up csaba nagy's idea and implemented a very simple yet very useful extension. i introduced a GUC called statement_cost_limit which can be used to error out if a statement is expected to be too expensive. the advantage over statement_timeout is that we are actually able to error out before spending many seconds which is killed by statement_timeout anyway. best regards, hans statement_cost_limit1.patch Description: Binary data -- Cybertec Schönig & Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: www.postgresql-support.de -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
Josh Berkus wrote: > Tom, > >> Seems a lot better to me to just train people to run the check-config >> code by hand before pulling the trigger to load the settings for real. > > Or just have pg_ctl --check-first as an option. Cautious DBAs would use > that. In 2-3 versions, we can make --check-first the default, and > require the option --dont-check for unattended restarts. If the config file is known to be broken, why should you *ever* allow it to try to restart it? //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
Tom, Seems a lot better to me to just train people to run the check-config code by hand before pulling the trigger to load the settings for real. Or just have pg_ctl --check-first as an option. Cautious DBAs would use that. In 2-3 versions, we can make --check-first the default, and require the option --dont-check for unattended restarts. --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
Tom Lane wrote: Alvaro Herrera <[EMAIL PROTECTED]> writes: Tom Lane wrote: Idle thought: maybe what would really make sense here is a "lint" for PG config files, I like this idea. postgres --check-hba-file /path/to/hba.conf postgres --check-conf-file /path/to/postgresql.conf (I think it's better to reuse the same postmaster executable, because that way it's easier to have the same parsing routines.) I'd go for just postgres --check-config -D $PGDATA (In a reload scenario, you'd edit the files in-place and then do this before issuing SIGHUP.) regards, tom lane Doesn't it seem reasonable that it should be pg_ctl? You should never run postgres directly unless it is for DR. Joshua D. Drake -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
Alvaro Herrera wrote: Tom Lane wrote: Idle thought: maybe what would really make sense here is a "lint" for PG config files, which you'd run as a standalone program and which would look for not only clear errors but questionable things to warn about. For instance it might notice multiple pg_hba.conf entries for the same IP addresses, check whether an LDAP server can be connected to, check that all user/group/database names used in the file actually exist, etc. These are things that we'd certainly not put into any load- or reload-time tests. I like this idea. postgres --check-hba-file /path/to/hba.conf postgres --check-conf-file /path/to/postgresql.conf (I think it's better to reuse the same postmaster executable, because that way it's easier to have the same parsing routines.) Change that to pg_ctl and you have a deal :) Joshua D. Drake -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
Magnus Hagander <[EMAIL PROTECTED]> writes: >> The good way to solve this would be to have independant command line >> utilities which check pg_hba.conf, pg_ident.conf and postgresql.conf for >> errors. Then DBAs could run a check *before* restarting the server. > While clearly useful, it'd still leave the fairly large foot-gun that is > editing the hba file and HUPing things which can leave you with a > completely un-connectable database because of a small typo. That will *always* be possible, just because software is finite and human foolishness is not ;-). Now, we could ameliorate it a bit given a "postgres --check-config" mode by having pg_ctl automatically run that mode before any start, restart, or reload command, and then refusing to proceed if the check detects any indubitable errors. On the other hand, that would leave us with the scenario where the checking code warns about stuff that it can't be sure is wrong, but then we go ahead and install the borked config anyway. (Nobody is going to put up with code that refuses to install config settings that aren't 100% clean, unless the checks are so weak that they miss a lot of possibly-useful warnings.) Seems a lot better to me to just train people to run the check-config code by hand before pulling the trigger to load the settings for real. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Idle thought: maybe what would really make sense here is a "lint" >> for PG config files, > I like this idea. > postgres --check-hba-file /path/to/hba.conf > postgres --check-conf-file /path/to/postgresql.conf > (I think it's better to reuse the same postmaster executable, because > that way it's easier to have the same parsing routines.) I'd go for just postgres --check-config -D $PGDATA (In a reload scenario, you'd edit the files in-place and then do this before issuing SIGHUP.) Reasons: 1. Easier to use: one command gets you all the checks, and you can't accidentally forget to check the one config file that's gonna give you problems. 2. An in-place check is the only way to be sure that, for instance, relative-path 'include' directives are okay. 3. To implement the suggested check on role/database validity, the code is going to need to pull in the flat-file copies of pg_database etc. (Or try to contact a live postmaster, but that won't work when you don't have one.) So it needs access to $PGDATA in any case. 4. There might be usefulness in cross-checking different config files, so examining a single one out of context just seems like the wrong mindset. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
Josh Berkus wrote: > Magnus, > >> However it would be nice to throw an error or at least a warning when >> parsing >> the file instead of pretending everything's ok. Perhaps authentication >> methods >> should have a function to check whether the method is supported which is >> called when the file is parsed. >> > > The good way to solve this would be to have independant command line > utilities which check pg_hba.conf, pg_ident.conf and postgresql.conf for > errors. Then DBAs could run a check *before* restarting the server. While clearly useful, it'd still leave the fairly large foot-gun that is editing the hba file and HUPing things which can leave you with a completely un-connectable database because of a small typo. The difference in the "could run" vs "must run, thus runs automatically" part... //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
Tom Lane wrote: > Magnus Hagander <[EMAIL PROTECTED]> writes: >> Is there any actual gain by not doing the parsing in the postmaster, > > Define "parsing". There's quite a lot of possible errors in pg_hba > that it would be totally unreasonable for the postmaster to detect. Parsing as in turning into a struct with clearly defined parts. Like what type it is (host/local/hostssl), CIDR mask, auth method and parameters. > We could catch some simple problems at file load time, perhaps, > but those usually aren't the ones that cause trouble for people. It would catch things like typos, invalid CIDR address/mask and specifying an auth method that doesn't exist. This is the far most common errors I've seen - which ones are you referring to? > On the whole, I am against putting any more functionality into the > main postmaster process than absolutely has to be there. Every > new function you put onto it is another potential source of > service-outage-inducing bugs. True. But as a counterexample, we have a whole lot of code in there to do the same for GUC. Which can even call user code (custom variables), no? Are you also proposing we should look at getting rid of that? >> I've also noticed that authentication methods error out in different >> ways when they are not supported. > > Yeah, that's something that should be made more consistent. > > > Idle thought: maybe what would really make sense here is a "lint" > for PG config files, which you'd run as a standalone program and > which would look for not only clear errors but questionable things > to warn about. For instance it might notice multiple pg_hba.conf > entries for the same IP addresses, check whether an LDAP server > can be connected to, check that all user/group/database names > used in the file actually exist, etc. These are things that we'd > certainly not put into any load- or reload-time tests. That would also be a valuable tool, but IMHO for a slightly different purpose. To me that sounds more in the line of the tool to "tune/suggest certain postgresql.conf parameters" that has been discussed earlier. It would have to be implemented as a SQL callable function or so in order to make it usable for people doing remote admin, but that could certainly be done. It would still leave a fairly large hole open for anybody editing the config file and just HUPing the postmaster (which a whole lot of people do, since they're used to doing that to their daemon processes) //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
Magnus, However it would be nice to throw an error or at least a warning when parsing the file instead of pretending everything's ok. Perhaps authentication methods should have a function to check whether the method is supported which is called when the file is parsed. The good way to solve this would be to have independant command line utilities which check pg_hba.conf, pg_ident.conf and postgresql.conf for errors. Then DBAs could run a check *before* restarting the server. --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [Pljava-dev] Should creating a new base type require superuser status?
Thomas Hallgren <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> The problem that we've seen in the past shows up when the user lies in >> the CREATE TYPE command, specifying type representation properties that >> are different from what the underlying functions expect. > This is a non-issue in PL/Java. An integer parameter is never passed by > reference and there's no way the PL/Java user can get direct access to > backend memory. So what exactly does happen when the user deliberately specifies wrong typlen/typbyval/typalign info when creating a type based on PL/Java functions? > I think that assumption is without ground. Java doesn't permit you to > access memory unless you use Java classes (java.nio stuff) that is > explicitly designed to do that and you need native code to set such > things up. A PL/Java user can not do that unless he is able to link in > other shared objects or dll's to the backend process. PL/Java itself must be doing "unsafe" things in order to interface with PG at all. So what your argument really is is that you have managed to securely sandbox the user-written code you are calling. That might or might not be true, but I don't think that worrying about it is without foundation. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] Fragments in tsearch2 headline
Sorry for the delay. Here is the patch with FragmentDelimiter option.
It requires an extra option in HeadlineParsedText and uses that option
during generateHeadline.
Implementing notion of fragments in HeadlineParsedText and a separate
function to join them seems more complicated. So for the time being I
just dump a FragmentDelimiter whenever a new fragment (other than the
first one) starts.
The patch also contains the updated regression tests/results and also a
new test for FragmentDelimiter option. It also contains the
documentation for the new options.
I have also attached a separate file that tests different aspects of the
new headline generation function.
Let me know if anything else is needed.
-Sushant.
On Thu, 2008-07-24 at 00:28 +0400, Oleg Bartunov wrote:
> On Wed, 23 Jul 2008, Sushant Sinha wrote:
>
> > I guess it is more readable to add cover separator at the end of a fragment
> > than in the front. Let me know what you think and I can update it.
>
> FragmentsDelimiter should *separate* fragments and that says all.
> Not very difficult algorithmic problem, it's like perl's
> join(FragmentsDelimiter, @array)
>
> >
> > I think the right place for cover separator is in the structure
> > HeadlineParsedText just like startsel and stopsel. This will enable users to
> > specify their own cover separators. But this will require changes to the
> > structure as well as to the generateHeadline function. This option will not
> > also play well with the default headline generation function.
>
> As soon as we introduce FragmentsDelimiter we should make it
> configurable.
>
> >
> > The default MaxWords = 35 seems a bit high for this headline generation
> > function and 20 seems to be more reasonable. Any thoughts?
>
> I think we should not change default value because it could change
> behaviour of existing applications. I'm not sure if it'd be useful and
> possible to define default values in CREATE TEXT SEARCH PARSER
>
> >
> > -Sushant.
> >
> > On Wed, Jul 23, 2008 at 7:44 AM, Oleg Bartunov <[EMAIL PROTECTED]> wrote:
> >
> >> btw, is it intentional to have '' in headline ?
> >>
> >> =# select ts_headline('1 2 3 4 5 1 2 3 1','1&4'::tsquery,'MaxFragments=1');
> >> ts_headline
> >> -
> >> ... 4 5 1
> >>
> >>
> >>
> >> Oleg
> >>
> >> On Wed, 23 Jul 2008, Teodor Sigaev wrote:
> >>
> >> Let me know of any other changes that are needed.
>
> >>>
> >>> Looks like ready to commit, but documentation is needed.
> >>>
> >>>
> >>>
> >>Regards,
> >>Oleg
> >> _
> >> Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
> >> Sternberg Astronomical Institute, Moscow University, Russia
> >> Internet: [EMAIL PROTECTED],
> >> http://www.sai.msu.su/~megera/
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
Tom Lane wrote: > Idle thought: maybe what would really make sense here is a "lint" > for PG config files, which you'd run as a standalone program and > which would look for not only clear errors but questionable things > to warn about. For instance it might notice multiple pg_hba.conf > entries for the same IP addresses, check whether an LDAP server > can be connected to, check that all user/group/database names > used in the file actually exist, etc. These are things that we'd > certainly not put into any load- or reload-time tests. I like this idea. postgres --check-hba-file /path/to/hba.conf postgres --check-conf-file /path/to/postgresql.conf (I think it's better to reuse the same postmaster executable, because that way it's easier to have the same parsing routines.) -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
Magnus Hagander <[EMAIL PROTECTED]> writes: > Is there any actual gain by not doing the parsing in the postmaster, Define "parsing". There's quite a lot of possible errors in pg_hba that it would be totally unreasonable for the postmaster to detect. We could catch some simple problems at file load time, perhaps, but those usually aren't the ones that cause trouble for people. On the whole, I am against putting any more functionality into the main postmaster process than absolutely has to be there. Every new function you put onto it is another potential source of service-outage-inducing bugs. > I've also noticed that authentication methods error out in different > ways when they are not supported. Yeah, that's something that should be made more consistent. Idle thought: maybe what would really make sense here is a "lint" for PG config files, which you'd run as a standalone program and which would look for not only clear errors but questionable things to warn about. For instance it might notice multiple pg_hba.conf entries for the same IP addresses, check whether an LDAP server can be connected to, check that all user/group/database names used in the file actually exist, etc. These are things that we'd certainly not put into any load- or reload-time tests. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS][PATCHES] odd output in restore mode
Simon Riggs wrote: Well, this is a strange conclusion, leaving me slightly bemused. The discussion between Andrew and I at PGcon concluded that we would * document which other tools to use * remove the delay Now we have rejected the patch which does that, but then re-requested the exact same thing again. The patch interprets "remove the delay" as "remove the delay in a way which will not screw up existing users of pg_standby when they upgrade". Doing that requires us to have a configurable delay, which defaults to the current behaviour, but that can be set to zero (the recommended way). Which is what the patch implements. Andrew, Heikki: ISTM its time to just make the changes yourselves. This is just going round and round to no benefit. This doesn't warrant such a long discussion and review process. You ought to know by now that the length and ferocity of the discussion bears no relation at all to the importance of the subject ;-) Personally, I think it's reasonable to provide the delay as long as it's switchable, although I would have preferred zero to be the default. If we remove it altogether then we force bigger changes on people who are currently using Windows copy. But I can live with that since changing their archive_command is the better path by far anyway, either to use Gnu cp or the copy / rename trick. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [WIP] patch - Collation at database level
On Sat, Aug 02, 2008 at 03:39:18PM +0200, Radek Strnad wrote: > > I also think that the clauses you have attached to your CREATE > > COLLATION statement (case-insensitive, accent-insensitive) are an > > oversimplification of reality. I suggest you look up the Unicode > > collation algorithm to learn about who collations work in practice. > > I already did in the very beginning of the development. The reason why I'm > not implementing the whole Unicode collation algorithm is that this patch > shold be sort of framework. You'll be able to use different collation > functions not only POSIX locales so further development towards full Unicode > collation algorithm is possible. Agreed. Ofcourse it's a simplification of reality. POSIX locales are a simplification of reality, but its the only form of collation currently available to us. And quite frankly, I don't beleive postgresql should be in the business of writing collation algorithms, we don't have the expertese. FWIW, I think case-insensitive and accent-insensitive are useful modifiers that we should aim to support in the future. > At the end of the next week I'll publish my bachelor thesis concerning this > topic where everything will be explained in details so stay tuned. Good luck! Have a nice day, -- Martijn van Oosterhout <[EMAIL PROTECTED]> http://svana.org/kleptog/ > Please line up in a tree and maintain the heap invariant while > boarding. Thank you for flying nlogn airlines. signature.asc Description: Digital signature
Re: [HACKERS] [WIP] patch - Collation at database level
Hello, the main reason why I've submitted the patch was to start a discussion and know other people's opinion on this problem. On Tue, Jul 29, 2008 at 10:41 AM, Peter Eisentraut <[EMAIL PROTECTED]> wrote: > > Where are the collations going to come from? There will be two new catalogs - pg_collate and pg_charset. Each of them will be filled with ANSI standard collations and charsets (ISO8BIT, LATIN1, UTF-8..) and alternatively with default collation set when creating. For instance if you create database cluster with initdb and specify en_US.utf8 there will be standard rows (ISO8BIT, LATIN1, UTF-8..) + one row with en_US.utf8 in template0. Then you can connect to template0 and create other collations if your POSIX locales support them and use them one per each database. Have the various build and distributions issues been thought about? Yes. Since POSIX locales doesn't guarantee any collation there will be hard-coded collations implemented regarding ANSI collation standard. Others can be set by command CREATE COLLATION. How are they going to be configured (not the SQL syntax, but how will the > configuration be applied)? pg_type, pg_attribute, pg_namespace of each database will be extended with collation oid column that will be specifying collation. How are the collations going to be applied at run-time? Collation will be set when connecting to the database with setlocale(LC_COLLATION, XXX) and setlocale(LC_CTYPE, XXX) > How are you going to handle locale and encoding conflicts? Since I'm currently implementing collation support per database I don't think this is an issue. (It will be in the future I know.) > I also think that the clauses you have attached to your CREATE COLLATION > statement (case-insensitive, > accent-insensitive) are an oversimplification of reality. I suggest you > look > up the Unicode collation algorithm to learn about who collations work in > practice. I already did in the very beginning of the development. The reason why I'm not implementing the whole Unicode collation algorithm is that this patch shold be sort of framework. You'll be able to use different collation functions not only POSIX locales so further development towards full Unicode collation algorithm is possible. At the end of the next week I'll publish my bachelor thesis concerning this topic where everything will be explained in details so stay tuned. Regards Radek Strnad
Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
"Magnus Hagander" <[EMAIL PROTECTED]> writes: > I've also noticed that authentication methods error out in different > ways when they are not supported. For example, if I try to use Kerberos > without having it compiled in, I get an error when a client tries to > connect (because we compile in stub functions for the authentication > that just throw an error). But if I use pam, I get an "missing or > erroneous pg_hba.conf file" error (because we #ifdef out the entire > option all over the place). I'd like to make these consistent - but > which one of them do people prefer? Generally I prefer having stub functions which error out because it means other code doesn't need lots of ifdef's around the call sites. However it would be nice to throw an error or at least a warning when parsing the file instead of pretending everything's ok. Perhaps authentication methods should have a function to check whether the method is supported which is called when the file is parsed. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's PostGIS support! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Parsing of pg_hba.conf and authentication inconsistencies
The way pg_hba.conf is set up to be loaded/parsed now, we have: postmaster: open and tokenize file (load_hba(), tokenize_file()). backend: parse lines (parse_hba) and check for matches (check_hba/hba_getauthmethod) This means that the code in the postmaster is very simple, and it's shared with the caching of the role and ident files. It also means that we don't catch any syntax errors in the hba file until a client connects. For example, if I misspell "local" on the first line of the file (or just leave a bogus character on a line by mistake), no client will be able to connect. But I can still do a pg_ctl reload loading the broken file into the backend, thus making it impossible for anybody to connect to the database. (when there's a broken line, we won't continue to look at further lines in the file, obviously) Is there any actual gain by not doing the parsing in the postmaster, other than the fact that it's slightly less shared code with the other two files? If not, then I'd like to move that parsing there for above reasons. I've also noticed that authentication methods error out in different ways when they are not supported. For example, if I try to use Kerberos without having it compiled in, I get an error when a client tries to connect (because we compile in stub functions for the authentication that just throw an error). But if I use pam, I get an "missing or erroneous pg_hba.conf file" error (because we #ifdef out the entire option all over the place). I'd like to make these consistent - but which one of them do people prefer? //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS][PATCHES] odd output in restore mode
On Thu, 2008-07-31 at 12:32 -0400, Tom Lane wrote: > "Heikki Linnakangas" <[EMAIL PROTECTED]> writes: > > Martin Zaun wrote: > >> With these avenues to be explored, can the pg_standby patch on the > >> CommitFest wiki be moved to the "Returned with Feedback" section? > > > Yes, I think we can conclude that we don't want this patch as it is. > > Instead, we want a documentation patch that describes the problem, > > mentioning that GNU cp is safe, or you can use the copy+rename trick. > > Right, after which we remove the presently hacked-in delay. > > I've updated the commitfest page accordingly. Well, this is a strange conclusion, leaving me slightly bemused. The discussion between Andrew and I at PGcon concluded that we would * document which other tools to use * remove the delay Now we have rejected the patch which does that, but then re-requested the exact same thing again. The patch interprets "remove the delay" as "remove the delay in a way which will not screw up existing users of pg_standby when they upgrade". Doing that requires us to have a configurable delay, which defaults to the current behaviour, but that can be set to zero (the recommended way). Which is what the patch implements. Andrew, Heikki: ISTM its time to just make the changes yourselves. This is just going round and round to no benefit. This doesn't warrant such a long discussion and review process. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Should creating a new base type require superuser status?
On Thu, 2008-07-31 at 09:39 +0100, Andrew Sullivan wrote: > On Wed, Jul 30, 2008 at 06:07:53PM -0400, Alvaro Herrera wrote: > > > I do agree that creating base types should require a superuser though. > > It too seems dangerous just on principle, even if today there's no > > actual hole (that we already know of). > > I agree. +1 -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
