Re: [HACKERS] PL/perl should fail on configure, not make
On Fri, 11 Jan 2013, Tom Lane wrote: pgbuildf...@jdrake.com writes: Well, that's darn interesting in itself, because the error message looks like it should be purely a linker issue. (And I note that your other buildfarm animal mongoose uses icc but is working anyway, so that's definitely not the whole story ...) mongoose is 32-bit, and a really old version of icc. okapi is 64-bit, and a merely moderately old icc. I should set up a dedicated buildfarm VM with the latest version... Please note Aaron Swenson's offer of help too -- he's probably a lot better qualified than anybody else here to figure out what is going on with this. I'm sorry, I didn't see this. It must not have been CC'd to me, I don't subscribe to -hackers anymore, I just couldn't keep up with the traffic after I got a new job that wasn't postgres-related. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Refactor flex and bison make rules
On Wed, 28 Nov 2012, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 11/28/2012 02:14 PM, Alvaro Herrera wrote: Okapi has been failing sporadically on ecpg, and I wonder if it's related to this change. Well, it looks like the make is broken and missing a clear dependency requirement. I think we need to ask Jeremy to turn off parallel build for okapi. Yeah, we already know that unpatched make 3.82 has got serious parallelism bugs: http://archives.postgresql.org/pgsql-hackers/2012-09/msg00397.php I wonder whether adding another .NOTPARALLEL directive would be a better idea than insisting people get hold of patched versions. While we're talking about odd issues that only seem to happen on Okapi, does anyone know of anything I can do to diagnose the pg_upgrade failure on the 9.2 branch? There are no rogue (non-buildfarm-related) postmaster/postgres processes running on the machine. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Refactor flex and bison make rules
On Wed, 28 Nov 2012, Tom Lane wrote: Jeremy Drake pgbuildf...@jdrake.com writes: While we're talking about odd issues that only seem to happen on Okapi, does anyone know of anything I can do to diagnose the pg_upgrade failure on the 9.2 branch? There are no rogue (non-buildfarm-related) postmaster/postgres processes running on the machine. [ digs around ... ] It looks like the failure is coming from here: if (strlen(path) = sizeof(unp-sun_path)) return EAI_FAIL; What's the size of the sun_path member of struct sockaddr_un on your machine? I count 115 characters in your socket path ... maybe you just need a less deeply nested test directory. (If that is the problem, seems like we need to return something more helpful than EAI_FAIL here.) /usr/include/sys/un.h:char sun_path[108]; /* Path name. */ That seems to be it. This may be just the excuse I needed to set up dedicated users for my buildfarm animals. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove fmgr.h include in cube contrib --- caused crash on a Ge
On Sun, 4 Sep 2011, Tom Lane wrote: What I would suggest is to see whether a more recent x86 version shows the problem or not. If not, let's just write it off as an already-fixed compiler bug. I have installed the most recent version in the home directory of a purpose-made user on that machine. configure:3252: icc --version 5 icc (ICC) 12.0.5 20110719 Copyright (C) 1985-2011 Intel Corporation. All rights reserved. I did git checkout 6416a82a62db4e66b2edb0fa8fc83a580c3f1931 to get a revision I knew was right in the broken range for mongoose. Apparently they deprecated one of my compiler flags: -xN (N is for Nocona), seems they renamed it to -xSSE2. Since this is a one-off run, I ignored that warning. The result is no crash in the cube test. I think tomorrow I'll try to get the 9.0 compiler set up on a clean VM, and if the issue duplicates there, I can see about setting up SSH access if anyone is still interested in investigating this further. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove fmgr.h include in cube contrib --- caused crash on a Ge
On Mon, 5 Sep 2011, Bruce Momjian wrote: Jeremy Drake wrote: I think tomorrow I'll try to get the 9.0 compiler set up on a clean VM, and if the issue duplicates there, I can see about setting up SSH access if anyone is still interested in investigating this further. What would we investigate except a compiler bug? To me, simply chalking it up to some uncharacterized compiler bug is still quite a bit of black magic. But, if that explanation is good enough for you, I've certainly got better things to do with my holiday than spending time on this :) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: [COMMITTERS] pgsql: Remove fmgr.h include in cube contrib --- caused crash on a Ge
On Sun, 4 Sep 2011, Tom Lane wrote: Jeremy Drake jere...@jdrake.com writes: I didn't see any changes that looked like they affected CurrentMemoryContext, but I attached the compressed context diff in case you want to look at it. Right now I have a feeling that this is a compiler bug. That's my feeling, also. Don't know whether you have the interest/energy to try to reduce it to a reportable test case. If you mean reporting it to the compiler vendor (Intel), I doubt that would be worthwhile. The version of the compiler on this machine is very out of date. It is version 9.0 20060222. I would bet that if I did track down and report an issue in a 5-year-old compiler version, their first question would be, does the issue duplicate in the current version. Given that my other buildfarm member is running 11.1 20100414 (albeit for the x64 platform instead of the x86) and had no issue, I would expect that the current x86 version would also have no problem. I have intentionally been keeping the compiler versions on my buildfarm members pretty much fixed, for the benefit of reproducable results. However, I would be interested in hearing any guidelines on how old is too old for buildfarm member versions. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extending varlena
On Mon, 18 Aug 2008, Tom Lane wrote:
What would make more sense is to redesign the large-object stuff to be
somewhat modern and featureful, and provide stream-access APIs (think
lo_read, lo_seek, etc) that allow offsets wider than 32 bits.
A few years ago, I was working on such a project for a company I used to
work for. The company changed directions shortly thereafter, and the
project was dropped, but perhaps the patch might still be useful as a
starting point for someone else.
The original patch is
http://archives.postgresql.org/pgsql-hackers/2005-09/msg01026.php, and the
advice I was working on implementing was in
http://archives.postgresql.org/pgsql-hackers/2005-09/msg01063.php
I am attaching the latest version of the patch I found around. As it was
almost 3 years ago, I am a little fuzzy on where I left off, but I do
remember that I was trying to work through the suggestions Tom Lane gave
in that second linked email. I would recommend discarding the libpq
changes, since that seemed to not pass muster.
Note that this patch was against 8.0.3. There only seem to be a few
issues applying it to the current head, but I haven't really dug into them
to see how difficult it would be to update. Luckily, the large object
code is fairly slow-moving, so there aren't too many conflicts. One thing
I did notice is that it looks like someone extracted one of the functions
I wrote in this patch and applied it as a 32-bit version. Good for them.
I'm glad someone got some use out of this project, and perhaps more use
will come of it.
--
At the source of every error which is blamed on the computer you will
find at least two human errors, including the error of blaming it on
the computer.diff -Nur postgresql-8.0.3-orig/src/backend/libpq/be-fsstubs.c
postgresql-8.0.3/src/backend/libpq/be-fsstubs.c
--- postgresql-8.0.3-orig/src/backend/libpq/be-fsstubs.c2004-12-31
13:59:50.0 -0800
+++ postgresql-8.0.3/src/backend/libpq/be-fsstubs.c 2005-10-03
11:43:36.0 -0700
@@ -233,6 +233,34 @@
PG_RETURN_INT32(status);
}
+
+Datum
+lo_lseek64(PG_FUNCTION_ARGS)
+{
+ int32 fd = PG_GETARG_INT32(0);
+ int64 offset = PG_GETARG_INT64(1);
+ int32 whence = PG_GETARG_INT32(2);
+ MemoryContext currentContext;
+ int64 status;
+
+ if (fd 0 || fd = cookies_size || cookies[fd] == NULL)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+errmsg(invalid large-object descriptor: %d,
fd)));
+ PG_RETURN_INT64(-1);
+ }
+
+ Assert(fscxt != NULL);
+ currentContext = MemoryContextSwitchTo(fscxt);
+
+ status = inv_seek(cookies[fd], offset, whence);
+
+ MemoryContextSwitchTo(currentContext);
+
+ PG_RETURN_INT64(status);
+}
+
Datum
lo_creat(PG_FUNCTION_ARGS)
{
@@ -283,6 +311,165 @@
PG_RETURN_INT32(inv_tell(cookies[fd]));
}
+
+Datum
+lo_tell64(PG_FUNCTION_ARGS)
+{
+ int32 fd = PG_GETARG_INT32(0);
+
+ if (fd 0 || fd = cookies_size || cookies[fd] == NULL)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+errmsg(invalid large-object descriptor: %d,
fd)));
+ PG_RETURN_INT64(-1);
+ }
+
+ /*
+* We assume we do not need to switch contexts for inv_tell. That is
+* true for now, but is probably more than this module ought to
+* assume...
+*/
+ PG_RETURN_INT64(inv_tell(cookies[fd]));
+}
+
+Datum
+lo_length(PG_FUNCTION_ARGS)
+{
+ int32 fd = PG_GETARG_INT32(0);
+ int32 sz = 0;
+ MemoryContext currentContext;
+
+ if (fd 0 || fd = cookies_size || cookies[fd] == NULL)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+errmsg(invalid large-object descriptor: %d,
fd)));
+ PG_RETURN_INT32(-1);
+ }
+ Assert(fscxt != NULL);
+ currentContext = MemoryContextSwitchTo(fscxt);
+
+ sz = inv_length(cookies[fd]);
+
+ MemoryContextSwitchTo(currentContext);
+
+ PG_RETURN_INT32(sz);
+}
+
+Datum
+lo_length64(PG_FUNCTION_ARGS)
+{
+ int32 fd = PG_GETARG_INT32(0);
+ int64 sz = 0;
+ MemoryContext currentContext;
+
+ if (fd 0 || fd = cookies_size || cookies[fd] == NULL)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+errmsg(invalid large-object descriptor: %d,
fd)));
+ PG_RETURN_INT64(-1);
+ }
+ Assert(fscxt != NULL);
+ currentContext = MemoryContextSwitchTo(fscxt);
+
+ sz = inv_length(cookies[fd]);
+
+ MemoryContextSwitchTo(currentContext);
+
+
Re: [HACKERS] What in the world is happening on spoonbill?
On Sat, 17 May 2008, Tom Lane wrote: Does anyone know how to get the child process exit status on Windows? GetExitCodeProcess, if you've got the process handle handy (which I assume you do, since you most likely were calling one of the WaitFor...Object family of functions. http://msdn.microsoft.com/en-us/library/ms683189(VS.85).aspx regards, tom lane -- Then a man said: Speak to us of Expectations. He then said: If a man does not see or hear the waters of the Jordan, then he should not taste the pomegranate or ply his wares in an open market. If a man would not labour in the salt and rock quarries then he should not accept of the Earth that which he refuses to give of himself. Such a man would expect a pear of a peach tree. Such a man would expect a stone to lay an egg. Such a man would expect Sears to assemble a lawnmower. -- Kehlog Albran, The Profit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] modules
On Thu, 3 Apr 2008, Peter Eisentraut wrote: Am Donnerstag, 3. April 2008 schrieb Andrew Dunstan: If this were at all true we would not not have seen the complaints from people along the lines of My ISP won't install contrib. But we have, and quite a number of times. We have concrete evidence that calling it contrib actually works against us. ISPs also won't install additional Perl modules, for example. Yet, CPAN does exist successfully. ISPs don't necessarily HAVE to install additional perl modules. If I have my own home directory and shell access, I can run perl Makefile.PL PREFIX=/home/myuser/perlstuff, and just tweak PERL5LIB (or use lib) and I can install modules without any superuser intervention. This is where the CPAN comparison breaks down. I can install any perl module I want (native perl or even XS/C modules) without superuser privileges. With postgres, super user privileges are REQUIRED to install any module, whatever it is called (contrib, modules, pgfoundry, gborg)... IMHO, this is the Achilles heel of Postgres extensibility. Look at this library of plugins out there that do all of these nifty things, and if you can't find one that fits your needs, you can always write a little C code to do the job exactly how you want. Too bad you can't use them if you can't afford your own dedicated database server instance... This was the most frustrating thing for me as a developer. I know that there are all of these fine modules out there, and I even have a few of my own. I have been spoiled by the extensibility of Postgres, only to have it taken away when I want to move my databases from my own machine into production on the hosting provider. If I want to put geographical data in a database, I know PostGIS is out there, but I can't install it. I could use cube/earthdistance, but I can't install that either. So much for the geographical data. How about text search? Nope, can't have that either, at least until 8.3 finds its way into OpenBSD ports and the hosting provider gets around to installing it. At least I have that to look forward to. My opinion is, it doesn't matter what you call the modules/contrib stuff if I can't use it, and I can't use it if it is not loaded in my database, and I can't load it without superuser privileges. -- Never put off till tomorrow what you can avoid all together. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Including PL/PgSQL by default
On Fri, 22 Feb 2008, D'Arcy J.M. Cain wrote: On Fri, 22 Feb 2008 07:37:55 + Dave Page [EMAIL PROTECTED] wrote: I know I'm gonna regret wading in on this, but in my mind this is akin to one of the arguments for including tsearch in the core server - namely that too many brain dead hosting providers won't add a contrib module or anything else in a customer's database because they don't So their clients will go somewhere PLUG URL=http://www.Vex.Net/; / that does understand what they are installing and can support their users properly. How far are we supposed to go to support the clueless? Being someone on one of these clueless providers, I wrote the patch (which made it into 8.3) which allows database owners to create trusted languages. For me, this was just far enough. The clueless tend to CREATE DATABASE %s OWNER %s, so then I can CREATE LANGUAGE plpgsql if I want it. This does not provide any detriment to the clueful, who can always REVOKE the privilege to create any PL (the patch also added ACL stuff for this). And, since the clueful tend to run web apps and such as non-database owners, if the web app was compromised and the db did not explicitly load plpgsql, the attacker could not use it. understand that just because it's not there by default doesn't mean it's in any way second rate. Including pl/pgsql in template1 will help those folks who forwhatever reason use such providers, whilst more savvy providers can easily disable it post-initdb if thats what they want to do. And the first time someone uses pl/pgsql to do harm, even if it is due to their mis-configuration, who gets blamed? -- The primary theme of SoupCon is communication. The acronym LEO represents the secondary theme: Law Enforcement Officials The overall theme of SoupCon shall be: Avoiding Communication with Law Enforcement Officials -- M. Gallaher ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Severe regression in autoconf 2.61
On Mon, 18 Feb 2008, Tom Lane wrote: There seems to have been a bit of a brain cramp upstream :-(. Previously, AC_FUNC_FSEEKO did this to test if fseeko was available: return !fseeko; Now it does this: return fseeko (stdin, 0, 0) (fseeko) (stdin, 0, 0); Unfortunately, that gives the compiler enough of a syntactic clue to guess that fseeko is probably an undeclared function, and therefore *it will not error out*, only generate a warning, if it's not seen a declaration for fseeko. So that's what that was. I had the same problem in another project I was working on (which I used some PostgreSQL configure code in). I had to add this in the gcc section of configure: PGAC_PROG_CC_CFLAGS_OPT([-Werror-implicit-function-declaration]) But it would be nice to find a better fix. I don't understand how calling a function that has not been defined yet is ever not an error. -- In 1915 pancake make-up was invented but most people still preferred syrup. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] The question of LOCALE at the time of a regression test.
On Sat, 9 Feb 2008, Hiroshi Saito wrote: Um, I was flipped off by you You shouldn't go around flipping people off: it's rude :) http://www.merriam-webster.com/dictionary/flip%20off ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] Re: [COMMITTERS] pgsql: GIN index build's allocatedMemory counter needs to be long, not
On Fri, 16 Nov 2007, Tom Lane wrote: GIN index build's allocatedMemory counter needs to be long, not uint32. Else, in a 64-bit machine with maintenance_work_mem set to above 4Gb, the counter overflows I don't know if this has been discussed before, but you are aware that it is not dictated by the C standard that sizeof(long) == sizeof(void*)? The best counter-example I know is Windows x64, where sizeof(long) == 4 while sizeof(void*) == 8. The standards-compliant way to deal with this IIRC is using size_t or ptrdiff_t, depending on whether or not you need it to be signed. Sorry if this has been discussed before, but this commit just struck me as someone who has just been working at porting some software to Win64... ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Open items for 8.3
On Mon, 5 Nov 2007, Gregory Stark wrote: How many developers have even jumped through the hoops to get wiki accounts? According to http://developer.postgresql.org/index.php?title=Special:Listusersgroup=pgdevlimit=500 there are currently 51 members of the group pgdev on the wiki. -- Spare no expense to save money on this one. -- Samuel Goldwyn ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Build farm failure
On Tue, 2 Oct 2007, Gregory Stark wrote: (we don't seem to have a recent icc ia32 build farm member). Sorry about that, my buildfarm member (mongoose) is down with hardware problems, and probably will be for the forseeable future. For some reason, it suddenly decided to stop recognizing its RAID card... -- In the beginning was the word. But by the time the second word was added to it, there was trouble. For with it came syntax ... -- John Simon ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] buildfarm failure after ICC configure change
I just saw that my buildfarm member (running ICC 9.0 on linux) failed after the latest change to configure http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=mongoosedt=2007-09-11%2020:45:01 I was the one who sent in the first patch to configure to add the check for ICC, and as I recall at the time, the docs said -fno-strict-aliasing was the default (which is why I said not sure if this is needed in the comment). I don't know what -fno-alias does, but I think it means that the program does not do aliasing at all. The docs say You must specify -fno-alias if you do not want aliasing to be assumed in the program The true option for -fno-strict-aliasing is -ansi-alias- disable use of ANSI aliasing rules in optimizations. But this is the default... -- Help me, I'm a prisoner in a Fortune cookie file! ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] buildfarm failure after ICC configure change
On Wed, 12 Sep 2007, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: I just saw that my buildfarm member (running ICC 9.0 on linux) failed after the latest change to configure Argh! Can someone quote chapter and verse from the ICC manual about this? I was just following what Sergey said was the approved spelling of the switch ... The docs are not particularly clear about these options, at least not in 9.0 (which is the version I have). I figured -ansi-alias- was what -fno-strict-aliasing meant, that the gcc people decided to follow the ISO C standard strictly unless you gave -fno-strict-aliasing, which could result in additional optimization. This seems to match what the docs for -ansi-alias, but with a different default. The -fno-alias seems to go entirely the other way, saying you are not going to be doing any aliasing, standard permitted or otherwise, so it can optimize better accordingly. Here are the docs for the two options. == ansi-alias, Qansi-alias Enable use of ANSI aliasing rules in optimizations. Syntax Linux: -ansi-alias -ansi-alias- Windows: /Qansi-alias /Qansi-alias- Default -ansi-alias-Disable use of ANSI aliasing rules in optimizations. This option tells the compiler to assume that the program adheres to ISO C Standard aliasability rules. If your program adheres to these rules, then this option allows the compiler to optimize more aggressively. If it doesn't adhere to these rules, then it can cause the compiler to generate incorrect code. == falias Specifies that aliasing should be assumed in the program. Syntax Linux: -falias -fno-alias Windows: None Default -falias Aliasing is assumed in the program. This option specifies that aliasing should be assumed in the program. You must specify -fno-alias if you do not want aliasing to be assumed in the program. Alternate Options Linux: None Windows: /Oa -- Remember: Silly is a state of Mind, Stupid is a way of Life. -- Dave Butler ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [EMAIL PROTECTED]: Re: [GENERAL] array_to_set functions]
On Tue, 7 Aug 2007, Decibel! wrote:
ISTM that having a built-in array_to_set function would be awfully
useful... Is the aggregate method below an acceptable way to do it?
Umm, the array_to_set function is not an aggregate. Personally, when I
need this functionality, I use this function conveniently present in the
default install:
select * from information_schema._pg_expandarray(ARRAY['foo', 'bar', 'baz']);
x | n
-+---
foo | 1
bar | 2
baz | 3
(3 rows)
Not exactly well documented or well known, but it works.
- Forwarded message from Merlin Moncure [EMAIL PROTECTED] -
On 8/3/07, Guy Fraser [EMAIL PROTECTED] wrote:
On Wed, 2007-08-01 at 07:14 +0530, Merlin Moncure wrote:
On 8/1/07, Decibel! [EMAIL PROTECTED] wrote:
David Fetter and I just came up with these, perhaps others will find
them useful:
CREATE OR REPLACE FUNCTION array_to_set(anyarray, int) RETURNS SETOF
anyelement LANGUAGE SQL AS $$
SELECT $1[i] from generate_series(array_lower($1, $2),
array_upper($1, $2)) i
$$;
CREATE OR REPLACE FUNCTION array_to_set(anyarray) RETURNS SETOF
anyelement LANGUAGE SQL AS $$
SELECT array_to_set($1, 1)
$$;
very nice, although IMO there is a strong justification for these
functions to be in core and written in C for efficiency (along with
array_accum, which I have hand burn from copying and pasting out of
the documentation).
merlin
Excellent timing guys. :^)
I was trying to build a function to list the items of an array, but
ran into problems and was going to post what I had been working on.
Your functions work great.
In case you don't have the function to generate an array from a set
here is one I have been using :
CREATE AGGREGATE array_accum (
BASETYPE = anyelement,
SFUNC = array_append,
STYPE = anyarray,
INITCOND = '{}'
);
I think that's what just about everyone uses. Unfortunately the
reverse of the function (array_to_set above) AFAIK does not map
directly to the C array API.
merlin
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
- End forwarded message -
--
Mollison's Bureaucracy Hypothesis:
If an idea can survive a bureaucratic review and be implemented
it wasn't worth doing.
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] compiler warnings on the buildfarm
On Thu, 12 Jul 2007, Stefan Kaltenbrunner wrote:
What would probably be useful if you want to pursue this is to filter
out the obvious spam like statement-not-reached, and see what's left.
I had gone through and looked at the warnings on mongoose before, but I am
running it against the current code now. Let me know if you want line
numbers on any of these...
count | msgtype | msgno |
msg
---+-+---+
552 | warning | 1292 | attribute warn_unused_result ignored
This is due to perl headers, so don't worry about this one
77 | warning | 188 | enumerated type mixed with another type
16 | warning | 186 | pointless comparison of unsigned integer with zero
9 | warning | 167 | argument of type int * is incompatible with
parameter of type socklen_t={__socklen_t={unsigned int}} *restrict
2 | warning | 300 | const variable all_zeroes requires an initializer
1 | warning | 556 | a value of type void * cannot be assigned to an
entity of type rl_completion_func_t *
(6 rows)
--
Give thought to your reputation. Consider changing name and moving to
a new town.
---(end of broadcast)---
TIP 6: explain analyze is your friend
Re: [HACKERS] Bugtraq: Having Fun With PostgreSQL
On Tue, 26 Jun 2007, Andrew Dunstan wrote: Jeremy Drake wrote: 2. If you cannot tell what process is connecting on a local socket (which I suspect you cannot portably), See ident_unix() in hba.c. It might not be 100% portable but I think it's fairly close for platforms that actually have unix sockets. It looks to me (looking at docs on the various functions used there) that only Linux supports getting the PID of the connecting process. The other various *BSD methods tend only to give the uid and gid, which will not be helpful if the connection is coming from another backend in the same cluster. In the linux case, it looks like one would need to get the client pid, try to get the PGPROC entry for it, if it exists get the roleid out of that and allow connections as that role. For any other case, some sort of painful protocol hack would be in order. The best way I can see is to see if the client process is owned by the same user as the database cluster, and if so send an auth request (like the SCM_CRED one), which would be responded to with the pid and a random sequence stored in the PGPROC entry. The server then proves the backend really is the one it claims to be by looking up the PID's PGPROC entry, and making sure the token matches. This is all just thinking out loud, of course... I have no plans to implement this in the short-term, but it may be an interesting project in the future. -- I like to believe that people in the long run are going to do more to promote peace than our governments. Indeed, I think that people want peace so much that one of these days governments had better get out of the way and let them have it. -- Dwight D. Eisenhower ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Bugtraq: Having Fun With PostgreSQL
On Tue, 26 Jun 2007, Tom Lane wrote: Gregory Stark [EMAIL PROTECTED] writes: All that really has to happen is that dblink should by default not be callable by any user other than Postgres. Yeah, that is not an unreasonable change. Someone suggested it far upthread, but we seem to have gotten distracted :-( An idea came to me while thinking about this. The particular use-case that I use dblink for is connecting to another database in the same database cluster. ISTM (without looking at any code) that the postmaster could keep track of who is properly authenticated in each backend, and see if a connection is being created from that backend to allow connections as the user in that backend. I had a couple ideas about this: 1. If you can tell what process is connecting on a local socket, store a mapping of pid to userid in the postmaster shmem and if a connection is originating from a pid in this table and is attempting to authenticate as the corresponding userid, allow it. 2. If you cannot tell what process is connecting on a local socket (which I suspect you cannot portably), generate a random token and stash it in shared memory mapping it to a userid, and then on authentication, send this token to the postmaster to prove that you have already authenticated. This has the downside of turning an exploit where a non-privileged user can read arbitrary postgres memory, they could potentially gain the privilieges of any logged on user, but the best idea is to not have that kind of bug ;) I know this is not the time for thinking about such things, but it may be an idea for 8.4... -- It's really quite a simple choice: Life, Death, or Los Angeles. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Bugtraq: Having Fun With PostgreSQL
On Sat, 16 Jun 2007, Michael Fuhr wrote: A message entitled Having Fun With PostgreSQL was posted to Bugtraq today. I haven't read through the paper yet so I don't know if the author discusses security problems that need attention or if the article is more like a compilation of Stupid PostgreSQL Tricks. http://www.securityfocus.com/archive/1/471541/30/0/threaded The crux of this seems to be two-fold: 1. If dblink is installed, an untrusted user could use it to gain privileges, either using trust/ident auth (you have a superuser named after the account the postmaster is runing as), or can be scripted to brute force passwords. 2. If you are a superuser, you can gain access to the external system, ie, by creating C language functions. Neither of these are news to me, but maybe some new postgres admin will read it and figure out to disable trust auth and not to let untrusted users call dblink (either not install it or REVOKE the rights to call it). -- Around computers it is difficult to find the correct unit of time to measure progress. Some cathedrals took a century to complete. Can you imagine the grandeur and scope of a program that would take as long? -- Epigrams in Programming, ACM SIGPLAN Sept. 1982 ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] is_array_type vs type_is_array
On Thu, 7 Jun 2007, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: Was there some change in functionality reason for renaming is_array_type to type_is_array? Just to sync style with type_is_enum ... there were more of the latter than the former. OK, so it is safe to just #define one to the other, right? It broke compilation of fulldisjunctions, Sorry, but we change internal APIs every day, and twice on Sundays. Deal with it. This is why I build fulldisjunctions in my sandbox, because when I decided to try it out one time, I could not get it to compile. After much effort getting it happy with HEAD and sending the changes back to the maintainer of fulldisjunctions, I thought a good thing for me to contribute is to make sure it continues to compile cleanly against HEAD and send patches when it breaks. I just wanted to make sure that the functionality of this function did not change in some way that I did not see before sending a patch to the maintainer of fulldisjunctions. Deal with it was not the response I was expecting, as that is exactly what I am trying to do ;) -- It is generally agreed that Hello is an appropriate greeting because if you entered a room and said Goodbye, it could confuse a lot of people. -- Dolph Sharp, I'm O.K., You're Not So Hot ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] is_array_type vs type_is_array
Was there some change in functionality reason for renaming is_array_type to type_is_array? It broke compilation of fulldisjunctions, which I build and run regression tests on in my sandbox to keep it getting too horribly broken with respect to current HEAD. I got it to build and pass its regression tests by adding this: + #if !defined(is_array_type) defined(type_is_array) + #define is_array_type(x) type_is_array(x) + #endif to the beginning of the one file which uses is_array_type. Is this reasonable to send back to the fulldisjunctions maintainer, or is there some subtle change that prompted the name change to make uses of this function immediately apparent? -- Ducharme's Axiom: If you view your problem closely enough you will recognize yourself as part of the problem. ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] ERROR: index row size
Just glancing at this, a couple things stand out to me:
On Mon, 4 Jun 2007, Rodrigo Sakai wrote:
Datum
periodo_in(PG_FUNCTION_ARGS)
{
char*str = PG_GETARG_CSTRING(0);
chartvi_char[MAXDATEFIELDS];
chartvf_char[MAXDATEFIELDS];
tvi_char = (char *) palloc(strlen(MAXDATEFIELDS));
What are you doing here? This is completely broken. I think you meant to
say:
char *tvi_char;
tvi_char = palloc(MAXDATEFIELDS);
Or:
char tvi_char[MAXDATEFIELDS];
and no palloc.
tvf_char = (char *) palloc(strlen(MAXDATEFIELDS));
Same as above.
Periodo *result;
if (sscanf(str, ( %s , %s ), tvi_char, tvf_char) != 2)
This is asking for trouble if arbitrary input can be fed to this.
ereport(ERROR,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
errmsg(invalid input syntax for periodo: \%s\, str)));
result-tvi = StringToDateADT(tvi_char);
result-tvi = StringToDateADT(tvf_char);
result = (Periodo *) palloc(sizeof(Periodo));
if (result-tvi result-tvf)
ereport(ERROR,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
errmsg(Initial date (TVi) must be smaller than final date
(TVf;
PG_RETURN_POINTER(result);
}
Please help me!
Thanks in advance!
Hope this helps.
--
My love, he's mad, and my love, he's fleet,
And a wild young wood-thing bore him!
The ways are fair to his roaming feet,
And the skies are sunlit for him.
As sharply sweet to my heart he seems
As the fragrance of acacia.
My own dear love, he is all my dreams --
And I wish he were in Asia.
-- Dorothy Parker
---(end of broadcast)---
TIP 6: explain analyze is your friend
[HACKERS] buildfarm failures after pgstat patch
The buildfarm appears to be failing after the recent pgstat patch. The failure seems to be caused by this failed assertion, which appears to occur fairly consistently in the ECPG tests, in the postmaster log: TRAP: FailedAssertion(!(entry-trans == 0L), File: pgstat.c, Line: 696) -- Disco is to music what Etch-A-Sketch is to art. ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] tsearch2 in 8.3
On Tue, 24 Apr 2007, Bruce Momjian wrote: Naz Gassiep wrote: A few of us on IRC were wondering what the status of tsearch2 is in 8.3 ? Was it decided to include it in core or did we decide to keep FTS as a plugin? Some brief comments from anyone on the inside of the whole FTS issue would be greatly appreciated by us mere end users. Regards, The patch is in the patch queue and we will try to get it into 8.3. Let me just say, that for me this is the most anticipated feature for 8.3. Along with the patch to allow the database owner to create trusted PLs, this will allow me to move all but one of my databases to my hosting provider's PostgreSQL instance from my own instance running in my home directory (the one I cannot move also requires dblink). I can only imagine there are other users out there in similar circumstances to mine. I was lucky enough to find a hosting provider with shell access where I can run a postgres instance and that I already had the know-how to do so. Without running my own instance, my only other option was to choose the lesser of two evils: do without FTS, or use MySQL. ;) Sorry for the rant, I just wanted to make sure that people knew that this is not just cosmetic, or a restructure for its own sake, but will actually help real world users. -- The cow is nothing but a machine which makes grass fit for us people to eat. -- John McNulty ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Patch queue concern
On Wed, 28 Mar 2007, Simon Riggs wrote: On Wed, 2007-03-28 at 17:12 -0400, Bruce Momjian wrote: If everybody knows where everybody stands then we'll all be better off. There may be other dependencies that need resolution, or last minute decisions required to allow authors to finish. Wasn't this the purpose of the wiki page that was set up? I notice it has not been updated in a while... http://developer.postgresql.org/index.php/Todo:WishlistFor83 -- If the aborigine drafted an IQ test, all of Western civilization would presumably flunk it. -- Stanley Garn ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Buildfarm feature request: some way to track/classify failures
On Sun, 18 Mar 2007, Tom Lane wrote: another icc crash| 2007-02-03 10:50:01 | 1 icc internal error | 2007-03-16 16:30:01 |29 These on mongoose are most likely a result of flaky hardware. They tend to occur most often when either a) I am doing something else on the box when the build runs, or b) the ambient temperature in the room is ~72degF I need to bring down this box at some point and try to figure out if it is bad memory or what. Anyway, ICC seems to be one of the few things that are really succeptable to hardware issues (on this box at least, it is mostly ICC and firefox), and I apologize for the noise this caused in the buildfarm logs... -- American business long ago gave up on demanding that prospective employees be honest and hardworking. It has even stopped hoping for employees who are educated enough that they can tell the difference between the men's room and the women's room without having little pictures on the doors. -- Dave Barry, Urine Trouble, Mister ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Buildfarm feature request: some way to track/classify failures
On Fri, 16 Mar 2007, Andrew Dunstan wrote: OK, for anyone that wants to play, I have created an extract that contains a summary of every non-CVS-related failure we've had. It's a single table looking like this: CREATE TABLE mfailures ( sysname text, snapshot timestamp without time zone, stage text, conf_sum text, branch text, changed_this_run text, changed_since_success text, log_archive_filenames text[], build_flags text[] ); Sweet. Should be interesting to look at. The dump is just under 1Mb and can be downloaded from http://www.pgbuildfarm.org/mfailures.dump Sure about that? --14:45:45-- http://www.pgbuildfarm.org/mfailures.dump = `mfailures.dump' Resolving www.pgbuildfarm.org... 207.173.203.146 Connecting to www.pgbuildfarm.org|207.173.203.146|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 9,184,142 (8.8M) [text/plain] -- BOO! We changed Coke again! BLEAH! BLEAH! ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] hex integer input
On Fri, 2 Mar 2007, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: On several occasions I have wanted to input integers in hexadecimal rather than in decimal in PostgreSQL. I notice that there is a to_hex function, but there is not (AFAIK) a way to provide an integer in hexadecimal. regression=# select x'abcd'::int; int4 --- 43981 (1 row) Hmm. I actually used the bit(N) type on a project quite some time ago, but it did not occur to me to use the bit string input syntax to input an integer. Well, I guess there is a way after all. So in the immortal words of Emily Litella, Never mind. -- In an organization, each person rises to the level of his own incompetency -- The Peter Principle ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[HACKERS] hex integer input
On several occasions I have wanted to input integers in hexadecimal rather than in decimal in PostgreSQL. I notice that there is a to_hex function, but there is not (AFAIK) a way to provide an integer in hexadecimal. I have written a pure-sql implementation of some functions to input integers in hexadecimal (http://postgresql.jdrake.com/hex-int-in.sql.html), but it is less than efficient ;) Is there some reason why hex input is not currently supported? What would be the preferred way to implement it? 1. A function (or one for each int[248]), essentially from_hex(text) 2. Support in the input functions for int[248] for integers that start with '0[Xx]' as with strtol with base set to 0. 3. Support in the grammar for integral constants 0[Xx][0-9A-Fa-f]+ Thoughts? Is this a legitimate TODO? -- Mr. Cole's Axiom: The sum of the intelligence on the planet is a constant; the population is growing. ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] error in cvs head: bogus varattno for OUTER var: 5
On Sun, 25 Feb 2007, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: psql:bogus_varattno_error.sql:23: ERROR: bogus varattno for OUTER var: 5 Any ideas what is causing this? This looks pretty nearly related to stuff I've been hacking on recently, so I suppose I broke something :-(. Will take a closer look tomorrow. This should help. I managed to whiddle the example down even further. CREATE TABLE a ( a_a text ); CREATE TABLE b ( b_a text ); CREATE TABLE b_chld () INHERITS (b); EXPLAIN SELECT a_a FROM a LEFT JOIN b ON a_a = lower(b_a); Results in: ERROR: bogus varattno for OUTER var: 2 The function call and the inheritance are both required, remove either and it works fine. -- This is the LAST time I take travel suggestions from Ray Bradbury! ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
[HACKERS] error in cvs head: bogus varattno for OUTER var: 5
The attached sql file creates some table infrastructure and then tries to explain a query. I get the following error on CVS HEAD: psql:bogus_varattno_error.sql:23: ERROR: bogus varattno for OUTER var: 5 In my real data, when I attempt to run the query I get the error: ERROR: invalid attribute number 5 But in the test setup it just returns an empty set. Any ideas what is causing this? Is it my query, or is something broken somewhere? I tend to think the latter, since this error message does not feel like one that a user would get and be expected to know what to do with... -- If God had intended Man to Watch TV, He would have given him Rabbit Ears.CREATE TABLE filenames ( recdate timestamp NOT NULL, season smallint NOT NULL, partno smallint, station text NOT NULL, eptitle text NOT NULL ); CREATE TABLE episodes ( epnum text NOT NULL, season smallint NOT NULL, title text NOT NULL, partno smallint ); CREATE TABLE episodes_chld () INHERITS (episodes); \set tsearch_sql `pg_config --sharedir`/contrib/tsearch2.sql \i :tsearch_sql EXPLAIN SELECT epnum, filenames.* FROM filenames LEFT JOIN episodes ON ( filenames.season = episodes.season AND strip(to_tsvector(filenames.eptitle)) = strip(to_tsvector(episodes.title)) AND filenames.partno IS NOT DISTINCT FROM episodes.partno) WHERE epnum IS NULL; -- vim: set ft=psql ts=2 sw=2 expandtab : ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] SCMS question
On Sat, 24 Feb 2007, Warren Turkal wrote: The interesting thing about Git is that is has two way sync support for a SVN repository also. You could run a Git repository pushing changes in real time to a SVN repository and present a CVS frontend also. I would like to try converting the CVS repository of PostgreSQL to Git and try setting some of this stuff up. Does anyone know how I could get the CVS repository files? Use cvsup, or if you don't want to go through the effort of getting that set up, use rsync: rsync -avzCH --delete rsync.postgresql.org::pgsql-cvs cvsroot/ wt -- Man is the only animal that can remain on friendly terms with the victims he intends to eat until he eats them. -- Samuel Butler (1835-1902) ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] SCMS question
On Sat, 24 Feb 2007, Warren Turkal wrote: On Saturday 24 February 2007 00:32, Jeremy Drake wrote: Use cvsup, or if you don't want to go through the effort of getting that set up, use rsync: rsync -avzCH --delete rsync.postgresql.org::pgsql-cvs cvsroot/ Thanks for this. Is this documented somewhere that I should have looked? CVSup is: http://developer.postgresql.org/pgdocs/postgres/cvsup.html rsync is a fairly new (and AFAIK, undocumented) method. I had to go back in the mailing list archives to find the reference: http://archives.postgresql.org/pgsql-hackers/2006-03/msg01081.php -- Chicken Little was right. ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] [ANNOUNCE] == PostgreSQL Weekly News - February 11 2007 ==
I made some notes about what you said about my patch, just so that I can be sure that it is clear what it does. On Sun, 11 Feb 2007, David Fetter wrote: == PostgreSQL Weekly News - February 11 2007 == == Pending Patches == Jeremy Drake sent in a patch which implements regexp_replace with multiple atoms, I don't know what you mean here. The only change I made to regexp_replace was fairly incedental: I split out the flag parsing code so that regexp_matches and regexp_split could use it as well, and in the process added support for some new flags which before could only be specified using the metasyntax (http://developer.postgresql.org/pgdocs/postgres/functions-matching.html#POSIX-METASYNTAX). Also, the error message for invalid flags to regexp_replace changed. I did not touch anything relating to what atoms are allowed (http://developer.postgresql.org/pgdocs/postgres/functions-matching.html#POSIX-ATOMS-TABLE). regexp_matches, a set-returning function, and regexp_split. Perl weenies rejoice! regexp_matches will only return a set if the 'g' flag was given. The no-flags version is not even declared as set-returning, it just returns a straight text[]. regexp_split is more of a set-returning function... :) -- The human mind ordinarily operates at only ten percent of its capacity -- the rest is overhead for the operating system. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] pgsql: Fix backend crash in parsing incorrect tsquery.
On Mon, 12 Feb 2007, Teodor Sigaev wrote: Log Message: --- Fix backend crash in parsing incorrect tsquery. Per report from Jon Rosebaugh [EMAIL PROTECTED] Is this a security issue? Does it need a new security release? I hope that the answer is not this is contrib, it isn't as important since I have been trying to convince others that contrib is not less secure or well supported than core. If this is a security issue (and if it can crash the backend due to a function parameter, it probably is) the community response to it will be particularly compelling evidence. Modified Files: -- pgsql/contrib/tsearch2: query.c (r1.30 - r1.31) (http://developer.postgresql.org/cvsweb.cgi/pgsql/contrib/tsearch2/query.c.diff?r1=1.30r2=1.31) -- Colvard's Logical Premises: All probabilities are 50%. Either a thing will happen or it won't. Colvard's Unconscionable Commentary: This is especially true when dealing with someone you're attracted to. Grelb's Commentary Likelihoods, however, are 90% against you. ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pgsql: Fix backend crash in parsing incorrect tsquery.
On Tue, 13 Feb 2007, Peter Eisentraut wrote: We don't treat crashes to be security issues of the kind that calls for the full security exercise. But if a security issue, by whatever definition of the term applies to core, is found in contrib, it would result in the full security exercise, correct? Of course, the people I am trying to convince that contrib is not insecure have yet to update their server with the latest security release (still running 8.1.3), so it is probably pretty much moot :) -- Anyone can hold the helm when the sea is calm. -- Publius Syrus ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] pgsql: Fix backend crash in parsing incorrect tsquery.
On Mon, 12 Feb 2007, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: On Mon, 12 Feb 2007, Teodor Sigaev wrote: Fix backend crash in parsing incorrect tsquery. Is this a security issue? Does it need a new security release? We looked at this and determined that the worst that could be done with it is crash the backend. Which is annoying, but if we treated every such bug as a security exercise then we'd be having a new release every week or so. Core's current policy is that we'll consider a bug worthy of a security release if it can be used to force execution of arbitrary code, access otherwise-unavailable information, etc. A simple crash is at worst a momentary denial of service to other DB users, and if you've got the ability to issue arbitrary SQL there are lots of ways to create denial-of-service situations of one magnitude or another. Also, recent history should impress on you the disadvantages of treating problems as security exercises: patches that go in without any public review or testing are far more likely to create new problems than those that go through the normal process. So setting a low bar for what constitutes a security issue is likely to decrease the system's overall reliability. I understand. This is reasonable. I am glad that this was considered, and weighed against the same policy as core. -- Andrea: Unhappy the land that has no heroes. Galileo: No, unhappy the land that _needs heroes. -- Bertolt Brecht, Life of Galileo ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] [HACKERS] writing new regexp functions
On Sun, 4 Feb 2007, David Fetter wrote:
On Fri, Feb 02, 2007 at 07:01:33PM -0800, Jeremy Drake wrote:
Let me know if you see any bugs or issues with this code, and I am
open to suggestions for further regression tests ;)
I have not heard anything, so I guess at this point I should figure out
where to go next with this. I see a couple options:
* Set this up as a pgfoundry project or contrib. This would require
merging the patch to expose some functions from regexp.c outside that
file, which has raised some concerns about maintainability.
* Put together a patch to add these functions to core. I could put them
directly in regexp.c, so the support functions could stay static. My
concern here is that I don't know if there are any functions currently
in core with OUT parameters. I don't know the acceptable style for
handling this: OUT parameters, a named composite type, ...?
Does anyone have any opinions either way, as to how I should proceed
from here?
* maybe a join function that works as an aggregate
SELECT join(',', col) FROM tbl
currently can be written as
SELECT array_to_string(ARRAY(SELECT col FROM tbl), ',')
The array_accum() aggregate in the docs works OK for this purpose.
I have decided not to pursue this function, I think the array construct,
or the array_accum option, is about the best possible currently. If it
should become possible in the future to write aggregates with a non-sql
state type (structs with pointers) it may be worthwhile to re-evaluate
this.
--
The cost of living hasn't affected its popularity.
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
Re: [PATCHES] [HACKERS] writing new regexp functions
On Wed, 7 Feb 2007, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: * Put together a patch to add these functions to core. I could put them directly in regexp.c, so the support functions could stay static. My concern here is that I don't know if there are any functions currently in core with OUT parameters. As of 8.2 there are. Could you give me the name of one in pg_proc.h so I can see how I should go about adding one there? If we are going to include these I would vote for core not contrib status, exactly to avoid having to export those functions. OK, this patch will be my next project. -- History is curious stuff You'd think by now we had enough Yet the fact remains I fear They make more of it every year. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] quick SRF question
If I have a multi-call SRF and a user_fctx struct allocated in the
multi_call_memory_ctx, and in the if(SRF_IS_FIRSTCALL()) block while still
in the multi_call_memory_ctx I use PG_GETARG_TEXT_P(n) to get an argument
to my function, and stash the result of this in my user_fctx struct, am I
guaranteed that this pointer will remain valid throughout the remaining
calls to this SRF, or should I instead use PG_GETARG_TEXT_P_COPY(n)?
Here is an example of what I am talking about
typedef struct testfunc_ctx {
text * txt;
} testfunc_ctx;
Datum testfunc(PG_FUNCTION_ARGS)
{
FuncCallContext *funcctx;
testfunc_ctx *userctx;
MemoryContext oldcontext;
if (SRF_IS_FIRSTCALL())
{
funcctx = SRF_FIRSTCALL_INIT();
oldcontext = MemoryContextSwitchTo(funcctx-multi_call_memory_ctx);
userctx = palloc(sizeof(testfunc_ctx));
/* XXX does this need to be PG_GETARG_TEXT_P_COPY, or is this ok like this */
userctx-txt = PG_GETARG_TEXT_P(0);
MemoryContextSwitchTo(oldcontext);
funcctx-user_fctx = userctx;
}
funcctx = SRF_PERCALL_SETUP();
userctx = funcctx-user_fctx;
/* do something with userctx-txt */
if (done)
SRF_RETURN_DONE(funcctx);
else
SRF_RETURN_NEXT(funcctx, result);
}
--
The New Testament offers the basis for modern computer coding theory,
in the form of an affirmation of the binary number system.
But let your communication be Yea, yea; nay, nay: for
whatsoever is more than these cometh of evil.
-- Matthew 5:37
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
Re: [HACKERS] Proposal: TABLE functions
On Tue, 6 Feb 2007, Pavel Stehule wrote: Hello, Currently PostgreSQL support set returning functions. ANSI SQL 2003 goes with new type of functions - table functions. With this syntax CREATE FUNCTION foo() RETURNS TABLE (c1 t1, ... ) PostgreSQL equal statements are: CREATE TYPE tmptype AS (c1 t1, ...) CREATE FUNCTION ... RETURNS SETOF tmptype AS ... or you can do CREATE FUNCTION foo(OUT c1 t1, OUT ...) RETURNS SETOF record AS ... But I think this would be nice, I think the OUT parameters make less sense than saying RETURNS TABLE(...). But what about functions not returning SETOF? -- The Schwine-Kitzenger Institute study of 47 men over the age of 100 showed that all had these things in common: (1) They all had moderate appetites. (2) They all came from middle class homes (3) All but two of them were dead. ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] SRF optimization question
I am writing a set returning function in C. There are cases where I can know definitively, upfront, that this function will only return one row. I have noticed, through happenstance of partially converted function, that I can mark a normal, non-set returning function as returning SETOF something, while not utilizing the SRF macros and using PG_RETURN_DATUM, and it still works as returning one row. I am wondering, if it is an acceptable optimization that if I know up-front that a function will only return one row, to avoid all of the SRF overhead of setting up a new memory context, and a function context struct, and requiring an extra call to my function to tell Postgres that I am done sending rows, to simply not use the SRF stuff and interact with Postgres as though I was not returning SETOF? Is this a sane idea, or did I just stumble into an accidental feature when I changed my CREATE FUNCTION statement without changing my C code? -- UNIX was half a billion (5) seconds old on Tue Nov 5 00:53:20 1985 GMT (measuring since the time(2) epoch). -- Andy Tannenbaum ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] writing new regexp functions
On Thu, 1 Feb 2007, David Fetter wrote:
Yes, although it might have the same name, as in regex_match(pattern
TEXT, string TEXT, return_pre_and_post BOOL).
The data structure could be something like
TYPE matches (
prematch TEXT,
matchTEXT[],
postmatch TEXT
)
I just coded up for this:
CREATE FUNCTION regexp_matches(IN str text, IN pattern text) RETURNS
text[]
AS 'MODULE_PATHNAME', 'regexp_matches'
LANGUAGE C IMMUTABLE STRICT;
CREATE FUNCTION regexp_matches(
IN str text, IN pattern text, IN return_pre_and_post bool,
OUT prematch text, OUT fullmatch text, OUT matches text[], OUT
postmatch text) RETURNS record
AS 'MODULE_PATHNAME', 'regexp_matches'
LANGUAGE C IMMUTABLE STRICT;
Which works like this:
jeremyd=# \pset null '\\N'
Null display is \N.
jeremyd=# select * from regexp_matches('foobarbequebaz',
$re$(bar)(beque)$re$);
regexp_matches
{bar,beque}
(1 row)
jeremyd=# select * from regexp_matches('foobarbequebaz',
$re$(bar)(beque)$re$, false);
prematch | fullmatch | matches | postmatch
--+---+-+---
\N | \N| {bar,beque} | \N
(1 row)
jeremyd=# select * from regexp_matches('foobarbequebaz',
$re$(bar)(beque)$re$, true);
prematch | fullmatch | matches | postmatch
--+---+-+---
foo | barbeque | {bar,beque} | baz
(1 row)
And then you also have this behavior in the matches array:
jeremyd=# select * from regexp_matches('foobarbequebaz',
$re$(bar)(.*)(beque)$re$);
regexp_matches
{bar,,beque}
(1 row)
jeremyd=# select * from regexp_matches('foobarbequebaz',
$re$(bar)(.+)(beque)$re$);
regexp_matches
\N
(1 row)
jeremyd=# select * from regexp_matches('foobarbequebaz',
$re$(bar)(.+)?(beque)$re$);
regexp_matches
--
{bar,NULL,beque}
(1 row)
Reasonable?
--
A.A.A.A.A.:
An organization for drunks who drive
---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] writing new regexp functions
On Fri, 2 Feb 2007, Jeremy Drake wrote:
jeremyd=# select * from regexp_matches('foobarbequebaz',
$re$(bar)(beque)$re$, false);
prematch | fullmatch | matches | postmatch
--+---+-+---
\N | \N| {bar,beque} | \N
(1 row)
I just changed this to fill in fullmatch when the bool is false, so this
one would look like:
prematch | fullmatch | matches | postmatch
--+---+-+---
\N | barbeque | {bar,beque} | \N
(1 row)
I also removed my check for capture groups, since in this setup you could
get useful output without any. I am still trying to decide whether or not
to add back an error if you called the no-bool version which just returns
the array, and you do not have any capture groups. ISTM this is likely an
oversight on the query author's part, and it would be helpful to alert him
to this.
If you have no capture groups, the matches array is empty (not null). If
the match happened at the start of the string, the prematch is an empty
string, and if the match happened at the end of the string, the postmatch
is an empty string.
Reasonable?
--
It's odd, and a little unsettling, to reflect upon the fact that
English is the only major language in which I is capitalized; in many
other languages You is capitalized and the i is lower case.
-- Sydney J. Harris
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
Re: [HACKERS] writing new regexp functions
On Fri, 2 Feb 2007, Jeremy Drake wrote: I just coded up for this: CREATE FUNCTION regexp_matches(IN str text, IN pattern text) RETURNS text[] AS 'MODULE_PATHNAME', 'regexp_matches' LANGUAGE C IMMUTABLE STRICT; CREATE FUNCTION regexp_matches( IN str text, IN pattern text, IN return_pre_and_post bool, OUT prematch text, OUT fullmatch text, OUT matches text[], OUT postmatch text) RETURNS record AS 'MODULE_PATHNAME', 'regexp_matches' LANGUAGE C IMMUTABLE STRICT; I wanted to put out there the question of what order the parameters to these regex functions should go. ISTM most people expect them to go (pattern, string), but I made these functions consistant with substring(text,text) which takes (string, pattern). Now I have been working on a regexp_split function, which takes (pattern, string), which is what someone familiar with the function from perl would expect, but is not consistant with substring or now with my regexp_matches function. I want to ask, should I break with following substring's precedent, and put the pattern first (as most people probably would expect), or should I break with perl's precedent and put the pattern second (to behave like substring)? -- We cannot put the face of a person on a stamp unless said person is deceased. My suggestion, therefore, is that you drop dead. -- James E. Day, Postmaster General ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] writing new regexp functions
I am wanting to write some new C functions which leverage postgresql's existing regexp code in an extension module. I notice that the functions RE_compile_and_cache and RE_compile_and_execute in src/backend/util/regexp.c contain the code necessary to connect the regexp code in src/backend/regex with the postgresql string conversion, error reporting, and memory management infrastructure, as well as providing caching of regexes which would probably be a win to any regex function in postgres. It would seem that these functions would be useful to any C function dealing with regexp matching in postgresql, but they are static functions, so they cannot be used outside of src/backend/utils/adt/regexp.c. Since all of the core regexp functions are in this file, this has been ok, but it is my opinion that these functions should be made visible and added to a header file so that extensions can make use of them, because any add-on functions that want to use the regex code in postgres in some new way would need to basically duplicate that same code in order to do so. Is there some specific reason that these functions are static, or would it be ok to make them non-static and add them to a header (say, src/include/utils/regexp.h) so that extensions could use them as well? I could put together a patch for this if desired, or it seems simple enough that someone could just do it... -- I can't decide whether to commit suicide or go bowling. -- Florence Henderson ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] writing new regexp functions
On Thu, 1 Feb 2007, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: Is there some specific reason that these functions are static, Yeah: not cluttering the global namespace. Is there a reason for not putting your new code itself into regexp.c? Not really, I just figured it would be cleaner/easier to write it as an extension. I also figure that it is unlikely that every regexp function that anyone could possibly want will be implemented in core in that one file. If anyone writes an extension like this, they would need to duplicate a good amount of code in order to do so, that would make more difficulty in maintaining the code if it should need to change. It also makes developing a new function a lot easier, no need to re-initdb to add the function, no need to relink the postmaster and restart it every time the function changes. Anyway, the particular thing I was writing was a function like substring(str FROM pattern) which instead of returning just the first match group, would return an array of text containing all of the match groups. I exported the functions in my sandbox, and wrote a module with a function that does this. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org -- Calling J-Man Kink. Calling J-Man Kink. Hash missile sighted, target Los Angeles. Disregard personal feelings about city and intercept. ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] writing new regexp functions
On Thu, 1 Feb 2007, David Fetter wrote: On Thu, Feb 01, 2007 at 05:11:30PM -0800, Jeremy Drake wrote: Anyway, the particular thing I was writing was a function like substring(str FROM pattern) which instead of returning just the first match group, would return an array of text containing all of the match groups. If you are subscribed to -patches, I sent my code to date there earlier this evening. I also said that I wanted to make a function that split on a pattern (like perl split) and returned setof text. That'd be great! People who use dynamic languages like Perl would feel much more at home having access to all the matches. While you're at it, could you could make pre-match and post-match (optionally--I know it's expensive) available? I could, but I'm not sure how someone would go about accessing such a thing. What I just wrote would be most like this perl: @foo = ($str=~/pattern/); Where would pre and post match fit into this? Are you talking about a different function? Or sticking prematch at the beginning of the array and postmatch at the end? I could also put the whole match somewhere also, but I did not in this version. The code I wrote returns a text[] which is one-dimensional, has a lower bound of 1 (as most postgres arrays do), where if there are n capture groups, ra[1] has the first capture group and ra[n] has the last one. Since postgres has an option to make different lower bounds, I suppose I could have an option to put the prematch in [-1], the entire match in [0], and the postmatch in [n+1]. This seems to be odd to me though. I guess I'm saying, I agree that the entire match, prematch, and postmatch would be helpful, but how would you propose to present these to the user? Cheers, D -- To err is human, to forgive, beyond the scope of the Operating System. ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [pgsql-patches] [HACKERS] unprivileged pl install
On Wed, 24 Jan 2007, Tom Lane wrote: * For an untrusted language: must be superuser to either create or use the language (no change from current rules). Ownership of the pg_language entry is really irrelevant, as is its ACL. * For a trusted language: * if pg_pltemplate.something is ON: either a superuser or the current DB's owner can CREATE the language. In either case the pg_language entry will be marked as owned by the DB owner (pg_database.datdba), which means that subsequently he (or a superuser) can grant or deny USAGE within his DB. What happens on ALTER DATABASE ALTER OWNER? Does the ownership of the language change to the new datdba or stay the old one? If the CREATE LANGUAGE results in creating the handler and validation funcs, who should own them? At the moment it is the user doing the CREATE LANGUAGE, but what does that mean? Can they then do odd things to the permissions of the procs, such as denying execute on them, to break other user's usage of the language, or does the perms on a language pre-empt the perms on the func? What happens if pg_pltemplate.something changes after the language is created? The datdba would continue to own the language, and can change permissions and drop it, but could not recreate it. I assume if the superuser wanted to revoke the ability for database owners to create that language they would remove it from people's databases who already have it. * if pg_pltemplate.something is OFF: must be superuser to CREATE the language; subsequently it will be owned by you, so only you or another superuser can grant or deny USAGE (same behavior as currently). What if pg_pltemplate.something is OFF, the language is CREATEd by a superuser, and then pg_pltemplate is set ON? The language is now owned by a superuser, so the db owner could not manipulate it. The patch I put together adds an owner to pg_language. Should there be an ALTER LANGUAGE OWNER TO command added as well. Thinking about these conditions I have described here, it seems to me there should be. Or there could not be an owner for a language and who the owner is depends on the conditions listed. But then permissions checks for languages would depend on pg_pltemplate, which seems less than clear or ideal to me. Besides which, when the acl is initalized from NULL to a value, it depends on who the owner is. It would need to be changed as well when the owner changing conditions change. I think that an ALTER LANGUAGE OWNER TO is the proper response to these things, and unless I hear otherwise I will attempt to add this to my patch. -- Checkuary, n.: The thirteenth month of the year. Begins New Year's Day and ends when a person stops absentmindedly writing the old year on his checks. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] tsearch in core patch, for inclusion
On Wed, 24 Jan 2007, Peter Eisentraut wrote: Teodor Sigaev wrote: If there aren't objections then we plan commit patch tomorrow or after tomorrow. I still haven't heard any argument for why this would be necessary or desirable at all, other than that it looks better for marketing reasons, which I will counter by saying that it looks worse for marketing reasons because our hailed plugin mechanism is apparently so poor that it can't support some practical extension module such as this. I for one am greatly looking forward to tsearch2 being in core. I was very fond of the plugin mechanism, until I signed up with a hosting provider. I do not have superuser privileges on the database cluster, and they will not install any plugins due to unspecified security concerns. So ATM if I want full text indexing, my only choice would be to avail myself of their mysql instance which has it built in. So I have been jaded, and my opinion of optional plugins has gone from wow, this is neat to man, this is a pain. They do not install plpgsql so I cannot write any triggers, they don't install tsearch2 so I don't get full text indexing, so all of the great features of postgres I have come to enjoy on my own box are suddenly taken away :( Sorry for the rant, I am just looking forward to 8.3 so I could get full text indexing... -- ARCHDUKE FERDINAND FOUND ALIVE -- FIRST WORLD WAR A MISTAKE ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] tsearch in core patch, for inclusion
On Wed, 24 Jan 2007, Martijn van Oosterhout wrote: On Wed, Jan 24, 2007 at 09:38:06PM +0100, Stefan Kaltenbrunner wrote: sure that ISP is a bit stupid(especially wrt plpgsql) - but tsearch2 in the current version is actually imposing some additional(often non-trivial) complexity for things like database restores and upgrades so I can see an ISP wanting to avoid that altogether. Something I've wondered about before is the concept of having installed Modules in the system. Let's say for example that while compiling postgres it compiled the modules in contrib also and installed them in a modules directory. Once installed there, unpriviledged users could say INSTALL foo and it would install the module, even if they do not have the permissions to create them themselves. That would be great, and also it would be great to be able to CREATE LANGUAGE as a regular user for a trusted pl that is already compiled/installed. That way you don't clutter the catalogs with external projects, and there is some indication from the postgres team of some trust in these modules. After all, if the installation made it easy to use for users, it must be safe, right? Essentially, I think they are just pretty reluctant to run commands as a superuser on behalf of a user... -- It is better never to have been born. But who among us has such luck? One in a million, perhaps. ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [pgsql-patches] [HACKERS] unprivileged contrib and pl install
On Wed, 24 Jan 2007, Tom Lane wrote: [ redirecting thread from -patches to -hackers for wider comment ] Jeremy Drake [EMAIL PROTECTED] writes: On Wed, 24 Jan 2007, Tom Lane wrote: Note I'm not arguing against allowing it to be on by default, I just want to be sure there is a way for paranoid DBAs to turn it off. Maybe it'd be sufficient if the flag bit was there but UPDATE pg_pltemplate was the only way to manipulate it --- we've gotten along with treating datistemplate and datallowconn that way. That sounds reasonable to me. I'll try to put together a patch like this (adding a boolean column to pg_pltemplate) and see if this is acceptable. I assume that only superusers can modify pg_pltemplate already ;) I had a further thought about this: if we allow random users to create languages, then without any further tweaking the instance of the language in their DB would be owned by them and they could grant or deny USAGE on it to others in their DB. This is probably not good. Given the current structure of pg_language, a language is effectively a one-time-per-DB resource and so random users could obstruct others from using a language. Perhaps it'd make sense to limit this to the DB owner, who would then be able to grant or deny language usage to the other users in his database. I am digging through the code looking at this, and I have a question. As far as I can tell, there is currently no owner for a pg_language entry. Is this correct or is ownership information stored somewhere other than the pg_language relation? Are you suggesting that a lanowner column would need to be added? As far as the column name referred to below as pg_pltemplate.something, for now I am calling it tmpldbaallowed. I am not particularly attached to nor fond of that name, however, and am open to naming suggestions. In detail, it'd look something like: * For an untrusted language: must be superuser to either create or use the language (no change from current rules). Ownership of the pg_language entry is really irrelevant, as is its ACL. * For a trusted language: * if pg_pltemplate.something is ON: either a superuser or the current DB's owner can CREATE the language. In either case the pg_language entry will be marked as owned by the DB owner (pg_database.datdba), which means that subsequently he (or a superuser) can grant or deny USAGE within his DB. * if pg_pltemplate.something is OFF: must be superuser to CREATE the language; subsequently it will be owned by you, so only you or another superuser can grant or deny USAGE (same behavior as currently). Comments? The bit about assigning the datdba as the owner might seem a bit odd, but I'm worried about the case where someone has the DBA privilege as a role but issues the create under his own ID. If it's owned directly by him, you'd end up in a situation where other holders of the DBA role couldn't manipulate the language, which seems undesirable. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate -- Save the Whales -- Harpoon a Honda. ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [pgsql-patches] [HACKERS] unprivileged contrib and pl install
On Wed, 24 Jan 2007, Jeremy Drake wrote: I am digging through the code looking at this, and I have a question. As far as I can tell, there is currently no owner for a pg_language entry. Is this correct or is ownership information stored somewhere other than the pg_language relation? Are you suggesting that a lanowner column would need to be added? Sort of answered my own question, found this comment: * Note: for now, languages are treated as owned by the bootstrap * user. We should add an owner column to pg_language instead. So in the course of implementing this, an owner column would probably need to be added to pg_language, I guess. -- If a 6600 used paper tape instead of core memory, it would use up tape at about 30 miles/second. -- Grishman, Assembly Language Programming ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] msvc failure in largeobject regression test
On Tue, 23 Jan 2007, Magnus Hagander wrote: On Tue, Jan 23, 2007 at 09:31:40AM -0500, Andrew Dunstan wrote: Magnus Hagander wrote: Hi! I get failures for the largeobject regression tests on my vc++ build. I don't think this has ever worked, given that those tests are fairly new. Any quick ideas on what's wrong before I dig deeper? [snip] I wonder if this is a line-end issue? Assuming you are working from CVS, does your client turn \n into \r\n ? I see that other windows boxes are happily passing this test on the buildfarm, and of course the mingw cvs doesn't adjust line endings. Bingo! That's it. I copeid the file in binary mode from a linux box and now it passes. I thought about that when I wrote it, and thus tried it under mingw and cygwin without issue ;) I don't think the regression tests were in a position of running on the msvc build at the time... My thought for what to do if this did run into a problem would be an alternate output file that is also acceptable (I don't know what they're called but other tests have them IIRC). -- Fifth Law of Procrastination: Procrastination avoids boredom; one never has the feeling that there is nothing important to do. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] pltcl regression failures with ORDER BY ... USING change
It looks like pltcl regression tests are failing due to the recent ORDER BY ... USING change. http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=mongoosedt=2007-01-09%2002:30:01 -- Horse sense is the thing a horse has which keeps it from betting on people. -- W. C. Fields ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] contrib regression failures after recent money type changes
Seems that the contrib regression tests, namely the cash and oid tests of the btree_gist contrib module, are failing after the recent commit to widen the money type to 64 bits. Example: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=mongoosedt=2007-01-03%2005:30:01 Also, on a slightly off-topic note, I am subscribed to the -committers list and did not receive an automated commit message for this commit. It is also not in the archives... -- Experience is what causes a person to make new mistakes instead of old ones. ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] contrib regression failures after recent money type
On Tue, 2 Jan 2007, Jeremy Drake wrote: Seems that the contrib regression tests, namely the cash and oid tests of the btree_gist contrib module, are failing after the recent commit to widen the money type to 64 bits. Example: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=mongoosedt=2007-01-03%2005:30:01 Almost forgot, I saw that this commit modified src/include/catalog/pg_type.h, should this have required a catversion change? Also, on a slightly off-topic note, I am subscribed to the -committers list and did not receive an automated commit message for this commit. It is also not in the archives... -- Whom the gods wish to destroy they first call promising. ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] float8 width_bucket function
I came across this when looking through the patches_hold queue link that Bruce sent out. http://momjian.us/mhonarc/patches_hold/msg00162.html There is no patch or anything associated with it, just the suggestion that it be put in when 8.3 devel starts up. Just thought I'd put this back out there now that 8.3 devel has started, since I had just about forgotten about it until seeing it on that list... -- Putt's Law: Technology is dominated by two types of people: Those who understand what they do not manage. Those who manage what they do not understand. ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Doc bug
On Sun, 31 Dec 2006, Gurjeet Singh wrote: On 12/31/06, Andrew Dunstan [EMAIL PROTECTED] wrote: Gurjeet Singh wrote: BTW, I don't know how to make sure that the effect of a doc patch looks fine in a browser. I mean, how to view the doc/src/sgml/*.sgml in a browser, nicely formatted as we see on our website! Docs for CVS HEAD can aleways be seen here: http://momjian.us/main/writings/pgsql/sgml/ I meant, before submitting the patch, or while even working on it, I would like to see it's rendering on my local machine, from the sources. One would like to be sure that he used the correct (or more appropriate) tags while writing the patch. cd pgsql/doc/src/sgml make html See http://developer.postgresql.org/pgdocs/postgres/docguide-build.html ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] recent --with-libxml support
On Sat, 23 Dec 2006, Tom Lane wrote: Ah-hah, I've sussed it. sqlchar_to_unicode() calls the mb2wchar_with_len converters, which are defined to return a *null terminated* pg_wchar string. So even if you only ask for the conversion of a single character, you need a 2-pg_wchar array to hold the result, and that wasn't being allocated. So the code was clobbering a word of stack space that evidently is critical on some machines and not others. Fix committed, please see if it works for you now. The buildfarm run seems to be green now. I am compiling a sandbox to test it manually now, but I'm pretty sure that fixed it. Yep, the make check works there too... -- As long as war is regarded as wicked, it will always have its fascination. When it is looked upon as vulgar, it will cease to be popular. -- Oscar Wilde ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[HACKERS] recent --with-libxml support
I adjusted my buildfarm config (mongoose) to attempt to build HEAD
--with-libxml. I added the following to build-farm.conf:
if ($branch eq 'HEAD' || $branch ge 'REL8_3')
{
push(@{$conf{config_opts}},
--with-includes=/usr/include/et:/usr/include/libxml2);
push(@{$conf{config_opts}}, --with-libxml);
}
As seen, I needed to add an include dir for configure to pass. However,
make check fails now with the backend crashing. This can be seen in the
buildfarm results for mongoose.
According to gentoo portage, I have libxml2 version 2.6.26 installed on my
system.
I am not clear if I should have pointed it at libxml version 1 or 2, but
configure seemed to be happy with libxml2. If it needs version 1, perhaps
configure should do something to keep it from using version 2.
Here is the diff for the xml regression test:
*** ./expected/xml.out Thu Dec 21 16:47:22 2006
--- ./results/xml.out Thu Dec 21 16:59:32 2006
***
*** 58,68
SELECT xmlelement(name element,
xmlattributes (1 as one, 'deuce' as two),
'content');
!xmlelement
!
! element one=1 two=deucecontent/element
! (1 row)
!
SELECT xmlelement(name element,
xmlattributes ('unnamed and wrong'));
ERROR: unnamed attribute value must be a column reference
--- 58,64
SELECT xmlelement(name element,
xmlattributes (1 as one, 'deuce' as two),
'content');
! ERROR: cache lookup failed for type 0
SELECT xmlelement(name element,
xmlattributes ('unnamed and wrong'));
ERROR: unnamed attribute value must be a column reference
***
*** 73,145
(1 row)
SELECT xmlelement(name employee, xmlforest(name, age, salary as pay)) FROM
emp;
! xmlelement
! --
! employeenamesharon/nameage25/agepay1000/pay/employee
! employeenamesam/nameage30/agepay2000/pay/employee
! employeenamebill/nameage20/agepay1000/pay/employee
! employeenamejeff/nameage23/agepay600/pay/employee
! employeenamecim/nameage30/agepay400/pay/employee
! employeenamelinda/nameage19/agepay100/pay/employee
! (6 rows)
!
! SELECT xmlelement(name wrong, 37);
! ERROR: argument of XMLELEMENT must be type xml, not type integer
! SELECT xmlpi(name foo);
! xmlpi
! -
! ?foo?
! (1 row)
!
! SELECT xmlpi(name xmlstuff);
! ERROR: invalid XML processing instruction
! DETAIL: XML processing instruction target name cannot start with xml.
! SELECT xmlpi(name foo, 'bar');
! xmlpi
! -
! ?foo bar?
! (1 row)
!
! SELECT xmlpi(name foo, 'in?valid');
! ERROR: invalid XML processing instruction
! DETAIL: XML processing instruction cannot contain ?.
! SELECT xmlroot (
! xmlelement (
! name gazonk,
! xmlattributes (
! 'val' AS name,
! 1 + 1 AS num
! ),
! xmlelement (
! NAME qux,
! 'foo'
! )
! ),
! version '1.0',
! standalone yes
! );
! xmlroot
!
--
! ?xml version=1.0 standalone=yes?gazonk name=val
num=2quxfoo/qux/gazonk
! (1 row)
!
! SELECT xmlserialize(content data as character varying) FROM xmltest;
! data
!
! valueone/value
! valuetwo/value
! (2 rows)
!
! -- Check mapping SQL identifier to XML name
! SELECT xmlpi(name :::_xml_abc135.%-_);
! xmlpi
! -
! ?_x003A_::_x005F_xml_abc135._x0025_-_x0026__?
! (1 row)
!
! SELECT xmlpi(name 123);
! xmlpi
! ---
! ?_x0031_23?
! (1 row)
!
--- 69,75
(1 row)
SELECT xmlelement(name employee, xmlforest(name, age, salary as pay)) FROM
emp;
! server closed the connection unexpectedly
! This probably means the server terminated abnormally
! before or while processing the request.
! connection to server was lost
--
The very powerful and the very stupid have one thing in common.
Instead of altering their views to fit the facts, they alter the facts
to fit their views ... which can be very uncomfortable if you happen to
be one of the facts that needs altering.
-- Doctor Who, Face of Evil
---(end of broadcast)---
TIP 6: explain analyze is your friend
Re: [HACKERS] recent --with-libxml support
On Fri, 22 Dec 2006, Tom Lane wrote:
Jeremy Drake [EMAIL PROTECTED] writes:
As seen, I needed to add an include dir for configure to pass. However,
make check fails now with the backend crashing. This can be seen in the
buildfarm results for mongoose.
Can you provide a stack trace for that crash?
#0 0xb7c4dc85 in memcpy () from /lib/tls/libc.so.6
#1 0x08190f59 in appendBinaryStringInfo (str=0xbfd87f90,
data=0x841ffc0 qux, datalen=138543040) at stringinfo.c:192
#2 0x0828377f in map_sql_identifier_to_xml_name (ident=0x841ffc0 qux,
fully_escaped=0 '\0') at xml.c:933
#3 0x0811ce83 in transformXmlExpr (pstate=0x84202b8, x=0x8420034)
at parse_expr.c:1426
#4 0x0811ac91 in transformExpr (pstate=0x84202b8, expr=0x8420034)
at parse_expr.c:238
#5 0x0811ceb4 in transformXmlExpr (pstate=0x84202b8, x=0x8420174)
at parse_expr.c:1456
#6 0x0811ac91 in transformExpr (pstate=0x84202b8, expr=0x8420174)
at parse_expr.c:238
#7 0x081288a4 in transformTargetEntry (pstate=0x84202b8, node=0x8420174,
expr=0x0, colname=0x0, resjunk=0 '\0') at parse_target.c:74
#8 0x0812890e in transformTargetList (pstate=0x84202b8, targetlist=0x1)
at parse_target.c:146
#9 0x080ffcef in transformStmt (pstate=0x84202b8, parseTree=0x84201fc,
extras_before=0xbfd882c4, extras_after=0xbfd882c8) at analyze.c:2102
#10 0x08101421 in do_parse_analyze (parseTree=0x841ffc0, pstate=0x84202b8)
at analyze.c:251
#11 0x0810227a in parse_analyze (parseTree=0x84201fc,
sourceText=0x841ffc0 qux, paramTypes=0x841ffc0, numParams=138543040)
at analyze.c:173
#12 0x0820b66e in pg_analyze_and_rewrite (parsetree=0x84201fc,
query_string=0x841fb74 SELECT xmlconcat(xmlcomment('hello'),\n, ' '
repeats 17 times, xmlelement(NAME qux, 'foo'),\n, ' ' repeats 17
times, xmlcomment('world'));, paramTypes=0x0, numParams=0) at
postgres.c:567
#13 0x0820b91e in exec_simple_query (
query_string=0x841fb74 SELECT xmlconcat(xmlcomment('hello'),\n, ' '
repeats 17 times, xmlelement(NAME qux, 'foo'),\n, ' ' repeats 17
times, xmlcomment('world'));) at postgres.c:875
#14 0x0820d72b in PostgresMain (argc=4, argv=0x83c5c2c,
username=0x83c5bfc jeremyd) at postgres.c:3418
#15 0x081dfbd7 in ServerLoop () at postmaster.c:2924
#16 0x081e132c in PostmasterMain (argc=3, argv=0x83c4550) at
postmaster.c:958
#17 0x081991e0 in main (argc=3, argv=0x83c4550) at main.c:188
--
In Tennessee, it is illegal to shoot any game other than whales from a
moving automobile.
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
Re: [HACKERS] recent --with-libxml support
On Fri, 22 Dec 2006, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: Can you provide a stack trace for that crash? #0 0xb7c4dc85 in memcpy () from /lib/tls/libc.so.6 #1 0x08190f59 in appendBinaryStringInfo (str=0xbfd87f90, data=0x841ffc0 qux, datalen=138543040) at stringinfo.c:192 #2 0x0828377f in map_sql_identifier_to_xml_name (ident=0x841ffc0 qux, fully_escaped=0 '\0') at xml.c:933 Hmm ... it seems to work for me here, using Fedora 5's libxml. Are you by any chance running this with a non-C locale? The trace suggests an encoding-mismatch sort of issue... Nope. I saw another buildfarm member that looks like it croaked in the same place: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=spongedt=2006-12-22%2022:30:02 So I guess it is not just me... -- If you think education is expensive, try ignorance. -- Derek Bok, president of Harvard ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] recent --with-libxml support
On Sat, 23 Dec 2006, Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: Jeremy Drake wrote: #0 0xb7c4dc85 in memcpy () from /lib/tls/libc.so.6 #1 0x08190f59 in appendBinaryStringInfo (str=0xbfd87f90, data=0x841ffc0 qux, datalen=138543040) at stringinfo.c:192 #2 0x0828377f in map_sql_identifier_to_xml_name (ident=0x841ffc0 qux, fully_escaped=0 '\0') at xml.c:933 Obviously the datalen has gone off the map. I wouldn't put 100% faith in that display, unless Jeremy built with -O0. I built this one with gcc 3.4.5 using --enable-debug --enable-cassert configure options. I will try with -O0 and see what I get... -- NAPOLEON: What shall we do with this soldier, Guiseppe? Everything he says is wrong. GUISEPPE: Make him a general, Excellency, and then everything he says will be right. -- G. B. Shaw, The Man of Destiny ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] recent --with-libxml support
On Fri, 22 Dec 2006, Jeremy Drake wrote: On Sat, 23 Dec 2006, Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: Jeremy Drake wrote: #0 0xb7c4dc85 in memcpy () from /lib/tls/libc.so.6 #1 0x08190f59 in appendBinaryStringInfo (str=0xbfd87f90, data=0x841ffc0 qux, datalen=138543040) at stringinfo.c:192 #2 0x0828377f in map_sql_identifier_to_xml_name (ident=0x841ffc0 qux, fully_escaped=0 '\0') at xml.c:933 Obviously the datalen has gone off the map. I wouldn't put 100% faith in that display, unless Jeremy built with -O0. I built this one with gcc 3.4.5 using --enable-debug --enable-cassert configure options. I will try with -O0 and see what I get... I just tried the same thing, but passing CFLAGS=-g -O0 to configure and the xml test passed. Maybe a '\0' termination issue? I also recompiled everything with the defaults again (-O2) and the xml test crashed in the same place. So it is an issue of -O0 works vs -O2 does not. Hate those... -- When I get real bored, I like to drive downtown and get a great parking spot, then sit in my car and count how many people ask me if I'm leaving. -- Steven Wright ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] psql: core dumped
On Wed, 20 Dec 2006, Philip Yarra wrote: Mario wrote: Even if you get a core dumped every time you press CTRL+\ ? why? Try ulimit -c 0, then run it (you should get no core dump) Then ulimit -c 50, then run it (you should get a core dump) SIGQUIT is supposed to dump core. Ulimit settings can suppress generation of core files. The difference between your machine and your friend's is likely just the ulimit settings. If you want to type CTRL+\ you can redefine what char generates SIGQUIT with stty quit command. For instance, stty quit ^@ -- fortune's Contribution of the Month to the Animal Rights Debate: I'll stay out of animals' way if they'll stay out of mine. Hey you, get off my plate -- Roger Midnight ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Proposal: syntax of operation with tsearch's configuration
On Fri, 17 Nov 2006, Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: I don't see any comparable arguments about this full-text search stuff. In particular I don't see any arguments why a change would necessary at all, including why moving to core would be necessary in the first place. AFAIR the only argument in favor of that is basically a marketing one: users perceive a feature as more real, or more supported, if it's in core. I don't find this argument especially compelling myself. I am currently in the position that my hosting provider is apprehensive about installing modules in contrib because they believe they are less secure. They cited (real or imagined) security holes as the reason they would not install tsearch2, or any other contrib module. This leaves me without any fulltext indexing option, as it requires a superuser to install. I have currently worked around this by running my own postgres instance from my home directory, as they provide shell access and allow running background processes, but I was really happy when I heard that tsearch2 was going to be integrated into core in 8.3. I think I would settle for some sort of assurance somewhere by someone who sounds authoritative that the contrib modules are not less secure than postgres core, and are fully supported by the developers. I think if I could point them at that, I may be able to convince them that it is safe. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] beta3 CFLAGS issue on openbsd
I was trying to compile 8.2beta3 on openbsd, and ran into an interesting issue. My account on the particular openbsd box has some restrictive ulimit settings, so I don't have a lot of memory to work with. I was getting an out of memory issue linking postgres, while I did not before. I figured out that the -g flag was being surreptitiously added to my CFLAGS. It was like pulling teeth trying to get the -g flag out. I tried --disable-debug to configure, which did not work. I had to do CFLAGS=-O2 ./configure ... Is this a known feature in the betas to get people running with -g in case things break, or is this a configure bug, or expected? Here is the first bit from configure, note the -g in the using CFLAGS line at the end. [EMAIL PROTECTED](~/build/postgres/postgresql-8.2beta3)$ ./configure --prefix=/home/jeremyd/progs/pg82 --with-perl --with-openssl --with-pgport=54322 checking build system type... x86_64-unknown-openbsd3.9 checking host system type... x86_64-unknown-openbsd3.9 checking which template to use... openbsd checking whether to build with 64-bit integer date/time support... no checking whether NLS is wanted... no checking for default port number... 54322 checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether cc accepts -g... yes checking for cc option to accept ANSI C... none needed checking if cc supports -Wdeclaration-after-statement... no checking if cc supports -Wendif-labels... yes checking if cc supports -fno-strict-aliasing... yes configure: using CFLAGS=-O2 -g -pipe -Wall -Wmissing-prototypes -Wpointer-arith -Winline -Wendif-labels -fno-strict-aliasing -- It's odd, and a little unsettling, to reflect upon the fact that English is the only major language in which I is capitalized; in many other languages You is capitalized and the i is lower case. -- Sydney J. Harris ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] plperl/plperlu interaction
On Thu, 26 Oct 2006, Alvaro Herrera wrote: Jeff Trout wrote: On Oct 26, 2006, at 3:23 PM, Martijn van Oosterhout wrote: On Thu, Oct 26, 2006 at 03:15:00PM -0400, Andrew Dunstan wrote: Perhaps people who use other platforms could look for these flags in the output of perl -e 'use Config qw(myconfig config_sh config_vars config_re); print config_sh();' OSX 10.4.8: usemultiplicity='define' usethreads='define' useithreads='define' Same here on Debian unstable (stock Perl packages). On my current Gentoo box: useithreads='undef' usemultiplicity='undef' usethreads='undef' My USE flags have ithreads disabled, since the description of the feature is Enable Perl threads, has some compatibility problems -- Whether you can hear it or not The Universe is laughing behind your back -- National Lampoon, Deteriorata ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] New CRC algorithm: Slicing by 8
On Mon, 23 Oct 2006, Mark Kirkwood wrote:
Tom Lane wrote:
Yah, I checked. Several times... but if anyone else wants to repeat
the experiment, please do. Or look for bugs in either my test case
or Gurjeet's.
Just for fun, I tried it out with both GCC and with Intel's C compiler
with some agressive platform-specific flags on my 2.8Ghz Xeon running
Gentoo.
Std crc Slice-8 crc
Intel P4 Xeon 2.8Ghz (Gentoo, gcc-3.4.5, -O2)
8192 bytes 4.6975729.806341
1024 bytes 0.5974291.181828
64 bytes0.0466360.086984
Intel P4 Xeon 2.8Ghz (Gentoo, icc-9.0.032, -O2 -xN -ipo -parallel)
8192 bytes 0.040.001085
1024 bytes 0.040.001292
64 bytes0.030.001078
So at this point I realize that intel's compiler is optimizing the loop
away, at least for the std crc and probably for both. So I make mycrc an
array of 2, and substript mycrc[j1] in the loop.
Std crc Slice-8 crc
Intel P4 Xeon 2.8Ghz (Gentoo, gcc-3.4.5, -O2)
8192 bytes 51.397146 9.523182
1024 bytes 6.4309861.229043
64 bytes0.4000620.128579
Intel P4 Xeon 2.8Ghz (Gentoo, icc-9.0.032, -O2 -xN -ipo -parallel)
8192 bytes 29.881708 0.001432
1024 bytes 3.7503130.001432
64 bytes0.2385830.001431
So it looks like something fishy is still going on with the slice-8 with
the intel compiler.
I have attached my changed testcrc.c file.
FWIW - FreeBSD and Linux results using Tom's test program on almost identical
hardware[1]:
Std crc Slice-8 crc
Intel P-III 1.26Ghz (FreeBSD 6.2)
8192 bytes 12.975314 14.503810
1024 bytes 1.6335571.852322
64 bytes0.1115800.206975
Intel P-III 1.26Ghz (Gentoo 2006.1)
8192 bytes 12.967997 28.363876
1024 bytes 1.6323173.626230
64 bytes0.1115130.326557
Interesting that the slice-8 algorithm seems to work noticeably better on
FreeBSD than Linux - but still not as well as the standard one (for these
tests anyway)...
Cheers
Mark
[1] Both boxes have identical mobos, memory and CPUs (same sspec nos).
---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings
--
You can tune a piano, but you can't tuna fish.#include postgres.h
#include time.h
#include sys/time.h
#include pg_crc.h
int
main()
{
charbuffer[TESTSIZE];
pg_crc32mycrc[2];
int j;
struct timeval tstart;
struct timeval tstop;
srand(time(NULL));
for (j = 0; j TESTSIZE; ++j)
buffer[j] = (char) (255 * (rand() / (RAND_MAX + 1.0)));
gettimeofday(tstart, NULL);
for (j = 0; j NTESTS; j++)
{
INIT_CRC32(mycrc[j1]);
COMP_CRC32(mycrc[j1], buffer, TESTSIZE);
FIN_CRC32(mycrc[j1]);
}
gettimeofday(tstop, NULL);
if (tstop.tv_usec tstart.tv_usec)
{
tstop.tv_sec--;
tstop.tv_usec += 100;
}
printf(bufsize = %d, loops = %d, elapsed = %ld.%06ld\n,
TESTSIZE, NTESTS,
(long) (tstop.tv_sec - tstart.tv_sec),
(long) (tstop.tv_usec - tstart.tv_usec));
return 0;
}
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
Re: [HACKERS] New CRC algorithm: Slicing by 8
On Mon, 23 Oct 2006, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: So at this point I realize that intel's compiler is optimizing the loop away, at least for the std crc and probably for both. So I make mycrc an array of 2, and substript mycrc[j1] in the loop. That's not a good workaround, because making mycrc expensive to access means your inner loop timing isn't credible at all. Instead try making the buffer array nonlocal --- malloc it, perhaps. That did not make any difference. The way I see it, the only way to convince the compiler it really needs to do this loop more than once is to make it think it is not overwriting the same variable every time. The subscript was the cheapest way I could think of to do that. Any other suggestions on how to do this are welcome. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend -- I like being single. I'm always there when I need me. -- Art Leo ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] New CRC algorithm: Slicing by 8
On Mon, 23 Oct 2006, Tom Lane wrote:
Hmm. Maybe store the CRCs into a global array somewhere?
uint32 results[NTESTS];
for ...
{
INIT/COMP/FIN_CRC32...
results[j] = mycrc;
}
This still adds a bit of overhead to the outer loop, but not much.
That seems to have worked.
Std crc Slice-8 crc
Intel P4 Xeon 2.8Ghz (Gentoo, gcc-3.4.5, -O2)
8192 bytes 26.765317 10.511143
1024 bytes 3.3578431.280890
64 bytes0.2232130.103767
Intel P4 Xeon 2.8Ghz (Gentoo, icc-9.0.032, -O2 -xN -ipo -parallel)
8192 bytes 29.495836 0.007107
1024 bytes 3.7086650.012183
64 bytes0.2425790.008700
So the gcc times are reasonable, but the icc times for the slice-by-8 are
still too fast to be believed. I will have to take a look at the
generated assembly later and see what gives.
My changed testcrc.c is attached, again.
--
I'd love to go out with you, but I did my own thing and now I've got
to undo it.#include postgres.h
#include time.h
#include sys/time.h
#include pg_crc.h
int
main()
{
volatile char buffer[TESTSIZE];
pg_crc32results[NTESTS];
pg_crc32mycrc;
int j;
struct timeval tstart;
struct timeval tstop;
srand(time(NULL));
for (j = 0; j TESTSIZE; ++j)
buffer[j] = (char) (255 * (rand() / (RAND_MAX + 1.0)));
gettimeofday(tstart, NULL);
for (j = 0; j NTESTS; j++)
{
INIT_CRC32(mycrc);
COMP_CRC32(mycrc, buffer, TESTSIZE);
FIN_CRC32(mycrc);
results[j] = mycrc;
}
gettimeofday(tstop, NULL);
if (tstop.tv_usec tstart.tv_usec)
{
tstop.tv_sec--;
tstop.tv_usec += 100;
}
printf(bufsize = %d, loops = %d, elapsed = %ld.%06ld\n,
TESTSIZE, NTESTS,
(long) (tstop.tv_sec - tstart.tv_sec),
(long) (tstop.tv_usec - tstart.tv_usec));
return 0;
}
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
[HACKERS] row-wise comparison question/issue
I noticed something odd when trying to use the row-wise comparison mentioned in the release notes for 8.2 and in the docs http://developer.postgresql.org/pgdocs/postgres/functions-comparisons.html#ROW-WISE-COMPARISON This sets up a suitable test: create type myrowtype AS (a integer, b integer); create table myrowtypetable (rowval myrowtype); insert into myrowtypetable select (a, b)::myrowtype from generate_series(1,5) a, generate_series(1,5) b; First I get this error: select rowval rowval from myrowtypetable ; ERROR: operator does not exist: myrowtype myrowtype LINE 1: select rowval rowval from myrowtypetable ; ^ HINT: No operator matches the given name and argument type(s). You may need to add explicit type casts. OK, I guess I can live with that. I did create a new type, and there are no operators for it... Now, I can do the following (pointless) query select ROW((rowval).*) ROW((rowval).*) from myrowtypetable ; and I get 25 rows of 'f'. So far so good. But if I try to do select rowval from myrowtypetable ORDER BY ROW((rowval).*); ERROR: could not identify an ordering operator for type record HINT: Use an explicit ordering operator or modify the query. or even select rowval from myrowtypetable ORDER BY ROW((rowval).*) USING ; ERROR: operator does not exist: record record HINT: No operator matches the given name and argument type(s). You may need to add explicit type casts. I know that that less-than operator exists, because I just used it in the query that worked above. It seems that ORDER BY just can't find it for some reason. Is it supposed to not work in order by? That doesn't really make sense to me why order by should be special for this. -- All extremists should be taken out and shot. ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] row-wise comparison question/issue
On Fri, 20 Oct 2006, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: select rowval from myrowtypetable ORDER BY ROW((rowval).*) USING ; ERROR: operator does not exist: record record This isn't required by the spec, and it's not implemented. I don't see that it'd give any new functionality anyway, since you can always do ORDER BY rowval.f1, rowval.f2, ... The cases that are implemented are comparisons of explicit row constructors, eg (a,b,c) (d,e,f) --- which I think is all you'll find support for in the spec. I just think it is quite unexpected that the operator is defined in some places and not in others. And the way I wrote the order by, it should have been comparing explicit row constructors (compare the explicitly constructed row for each rowval in order to sort). I don't understand how the operator in a where clause would be different than the operator used by the order by. If I were to make a custom type in C, and write these same operators for it, they would work in both places, right? Why then would this be any different? -- If someone had told me I would be Pope one day, I would have studied harder. -- Pope John Paul I ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Postgresql Caching
On Mon, 16 Oct 2006, [EMAIL PROTECTED] wrote: On Sun, Oct 15, 2006 at 06:33:36PM -0700, Jeremy Drake wrote: 2) When updating a PostgreSQL record, I updated the memcache record to the new value. If another process comes along in parallel before I commit, that is still looking at an older view, cross-referencing may not work as expected. Shouldn't you be able to use 2-stage commit for this? Prepare to commit, update the memcache record, then commit? Or am I thinking of something else? Two stage commits makes the window of error smaller, it can't eliminate it. Right, I was thinking there was still some raciness there. I think what I remembered is that if you updated the cache and then the transaction failed (or rolled back for whatever reason) later on, the cache would have data that was never committed. The two-phase commit thing is intended to deal with that eventuality. Which is also a possibility for a consistency issue. -- Oh, I have slipped the surly bonds of earth, And danced the skies on laughter silvered wings; Sunward I've climbed and joined the tumbling mirth Of sun-split clouds and done a hundred things You have not dreamed of -- Wheeled and soared and swung High in the sunlit silence. Hovering there I've chased the shouting wind along and flung My eager craft through footless halls of air. Up, up along delirious, burning blue I've topped the wind-swept heights with easy grace, Where never lark, or even eagle flew; And, while with silent, lifting mind I've trod The high untrespassed sanctity of space, Put out my hand, and touched the face of God. -- John Gillespie Magee Jr., High Flight ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Postgresql Caching
On Sun, 15 Oct 2006, [EMAIL PROTECTED] wrote: On Sun, Oct 15, 2006 at 08:31:36PM +0530, Merlin Moncure wrote: On 10/15/06, Anon Mous [EMAIL PROTECTED] wrote: Would it be possible to combine a special memcache implementation of memcache with a Postgresql interface wrapper? have you seen http://people.freebsd.org/~seanc/pgmemcache/ Interesting. I note that they don't address the view consistency problem any better than an application using memcached directly. And that's the real problem with memcached, and why people are tempted to 'indulge' by relying on PostgreSQL. Some people value the consistency. Others don't. memcached, whether application-side, or whether automatically invoked by triggers (pgmemcache) is a decision to ignore the consistency. Using memcache, I've had problems with consistency brought right to the front. Both of these have failed me: 1) When updating a PostgreSQL record, I invalidate the memcache record. If another process comes along in parallel before I commit, notices that the memcache record is invalidated, it queries the data from SQL, and updates the memcache record back to the old value. :-( 2) When updating a PostgreSQL record, I updated the memcache record to the new value. If another process comes along in parallel before I commit, that is still looking at an older view, cross-referencing may not work as expected. Shouldn't you be able to use 2-stage commit for this? Prepare to commit, update the memcache record, then commit? Or am I thinking of something else? I'm currently settled on 2), but setting a short timeout (5 seconds) on the data. Still an imperfect compromise between speed and accuracy, but it isn't causing me problems... yet. -- Fortune's Real-Life Courtroom Quote #32: Q: Do you know how far pregnant you are right now? A: I will be three months November 8th. Q: Apparently then, the date of conception was August 8th? A: Yes. Q: What were you and your husband doing at that time? ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] constraints in query plans
I set up the following experiment: CREATE DOMAIN m_or_p AS char CHECK (VALUE = 'm' OR VALUE = 'p'); CREATE TABLE test_domain ( fkey integer not null, kinteger not null, x1 integer not null, x2 integer, mp m_or_p not null ); CREATE INDEX test_domain_k_x1_x2_m ON test_domain (k, x1, x2) WHERE mp = 'm'; CREATE INDEX test_domain_k_x1_x2_p ON test_domain (k, x1, x2) WHERE mp = 'p'; then added about 375000 rows, half where mp = 'm' and half where mp = 'p' Now, I do analyze verbose test_domain jeremyd=# ANALYZE verbose test_domain; INFO: analyzing public.test_domain INFO: test_domain: scanned 2379 of 2379 pages, containing 375226 live rows and 0 dead rows; 3000 rows in sample, 375226 estimated total rows ANALYZE Now, take a look at this jeremyd=# explain select * from test_domain where k = 1255; QUERY PLAN - Seq Scan on test_domain (cost=0.00..7069.32 rows=127 width=17) Filter: (k = 1255) (2 rows) I turn constraint_exclusion on and I still get the same plan. I tried adding the domain's constraint to the table as well ALTER TABLE test_domain ADD CHECK (mp = 'm' OR mp = 'p'); and I still get the same plan. It seems the constraint is not incorporated into the plan, since I get a different plan if I include the constraint in the WHERE clause explicitly: jeremyd=# explain select * from test_domain where k = 1255 AND (mp = 'm' OR mp = 'p'); QUERY PLAN -- Bitmap Heap Scan on test_domain (cost=9.97..423.26 rows=95 width=17) Recheck Cond: (((k = 1255) AND ((mp)::char = 'm'::char)) OR ((k = 1255) AND ((mp)::char = 'p'::char))) - BitmapOr (cost=9.97..9.97 rows=127 width=0) - Bitmap Index Scan on test_domain_k_x1_x2_m (cost=0.00..4.98 rows=60 width=0) Index Cond: (k = 1255) - Bitmap Index Scan on test_domain_k_x1_x2_p (cost=0.00..4.98 rows=67 width=0) Index Cond: (k = 1255) (7 rows) And the explain analyze for each: jeremyd=# explain analyze select * from test_domain where k = 1255 AND (mp = 'm' OR mp = 'p'); QUERY PLAN --- Bitmap Heap Scan on test_domain (cost=9.97..423.26 rows=95 width=17) (actual time=0.325..2.397 rows=261 loops=1) Recheck Cond: (((k = 1255) AND ((mp)::char = 'm'::char)) OR ((k = 1255) AND ((mp)::char = 'p'::char))) - BitmapOr (cost=9.97..9.97 rows=127 width=0) (actual time=0.269..0.269 rows=0 loops=1) - Bitmap Index Scan on test_domain_k_x1_x2_m (cost=0.00..4.98 rows=60 width=0) (actual time=0.150..0.150 rows=129 loops=1) Index Cond: (k = 1255) - Bitmap Index Scan on test_domain_k_x1_x2_p (cost=0.00..4.98 rows=67 width=0) (actual time=0.101..0.101 rows=132 loops=1) Index Cond: (k = 1255) Total runtime: 3.238 ms (8 rows) jeremyd=# explain analyze select * from test_domain where k = 1255; QUERY PLAN --- Seq Scan on test_domain (cost=0.00..7069.32 rows=127 width=17) (actual time=0.427..125.057 rows=261 loops=1) Filter: (k = 1255) Total runtime: 125.878 ms (3 rows) ISTM that with the constraint_exclusion flag on, it should see from the constraints that all values but 'm' or 'p' are excluded for the column mp, and thus the two queries I gave are exactly equivalent. I noticed that the docs said it looked at table constraints, so I added the check to the table constraint as well, but it made no difference. I'm not sure if this is a bug or a limitation of the planner, but it seems that these two queries are equivalent. I wonder how it would work out with boolean instead of the char column, it should definitely know that there are only 2 possible values for a boolean not null column, true or false. DROP INDEX test_domain_k_x1_x2_p; DROP INDEX test_domain_k_x1_x2_m; ALTER TABLE test_domain DROP CONSTRAINT test_domain_mp_check; ALTER TABLE test_domain ALTER COLUMN mp TYPE boolean USING (CASE WHEN mp = 'm' THEN false ELSE true END); CREATE INDEX test_domain_k_x1_x2_p ON test_domain (k, x1, x2) WHERE mp; CREATE INDEX test_domain_k_x1_x2_m ON test_domain (k, x1, x2) WHERE NOT mp; jeremyd=# ANALYZE verbose test_domain; INFO: analyzing public.test_domain INFO: test_domain: scanned 2379 of 2379 pages, containing 375226 live rows and 0 dead rows; 3000 rows in sample, 375226 estimated total rows ANALYZE jeremyd=# explain analyze select * from test_domain where k = 1255; QUERY PLAN
Re: [HACKERS] constraints in query plans
On Sun, 15 Oct 2006, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: CREATE TABLE test_domain ( fkey integer not null, kinteger not null, x1 integer not null, x2 integer, mp m_or_p not null ); CREATE INDEX test_domain_k_x1_x2_m ON test_domain (k, x1, x2) WHERE mp = 'm'; CREATE INDEX test_domain_k_x1_x2_p ON test_domain (k, x1, x2) WHERE mp = 'p'; Perhaps you need a non-partial index. I just tried that, CREATE INDEX test_domain_k_x1_x2_mp ON test_domain (k, x1, x2, mp); and dropped the others. That actually works properly. jeremyd=# explain analyze select * from test_domain where k = 1255 and mp; QUERY PLAN -- Bitmap Heap Scan on test_domain (cost=5.37..237.21 rows=66 width=17) (actual time=0.115..0.707 rows=132 loops=1) Recheck Cond: (k = 1255) Filter: mp - Bitmap Index Scan on test_domain_k_x1_x2_mp (cost=0.00..5.37 rows=66 width=0) (actual time=0.081..0.081 rows=132 loops=1) Index Cond: ((k = 1255) AND (mp = true)) Total runtime: 1.137 ms (6 rows) I thought I had to refer to all of the columns in order for this to work, that I could not skip some in the middle, but it seems to work. -- If you can survive death, you can probably survive anything. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] width_bucket function for timestamps
On Mon, 9 Oct 2006, Tom Lane wrote: It's not clear to me why we have width_bucket operating on numeric and not float8 --- that seems like an oversight, if not outright misunderstanding of the type hierarchy. Would that make the below a lot faster? But if we had the float8 version, I think Jeremy's problem would be solved just by applying the float8 version to extract(epoch from timestamp). I don't really see the use-case for putting N versions of the function in there. I found the function I used before I implemented the C version. It was significantly slower, which is why I wrote the C version. -- given a date range and a number of buckets, round the given date to one -- of the buckets such that any number of dates within the date range passed -- in to this function will only return up to the number of buckets unique -- values CREATE OR REPLACE FUNCTION date_width_bucket (tm TIMESTAMP WITHOUT TIME ZONE, low TIMESTAMP WITHOUT TIME ZONE, high TIMESTAMP WITHOUT TIME ZONE, nbuckets INTEGER ) RETURNS TIMESTAMP WITHOUT TIME ZONE AS $$ SELECT ((EXTRACT(epoch FROM $3) - EXTRACT(epoch FROM $2)) / $4) * (width_bucket(EXTRACT(epoch FROM $1)::NUMERIC, EXTRACT(epoch FROM $2)::NUMERIC, EXTRACT(epoch FROM $3)::NUMERIC, $4) - 1) * '1 second'::INTERVAL + $2; $$ LANGUAGE sql IMMUTABLE STRICT; -- I don't think they could put him in a mental hospital. On the other hand, if he were already in, I don't think they'd let him out. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] width_bucket function for timestamps
I just came across this code I wrote about a year ago which implements a
function equivilant to width_bucket for timestamps.
I wrote this when I was trying to plot some data over time, and I had more
points than I needed. This function allowed me to create a pre-determined
number of bins to average the data inside of so that I could get a sane
number of points. Part of the problem was that there were so many data
points, that a sql implementation of the function (or plpgsql, I forget,
it was a year ago) was painfully slow. This C function provided much
better performance than any other means at my disposal.
I wanted to share this code since it may be useful for someone else, but I
don't know exactly what to do with it. So I am putting it out there, and
asking what the proper home for such a function might be. I believe it
would be generally useful for people, but it is so small that it hardly
seems like a reasonable pgFoundry project. Maybe there is a home for such
a thing in the core distribution in a future release?
The code can be found at
http://www.jdrake.com/postgresql/bintimestamp.tar.gz for a buildable PGXS
module, or I attached just the C code. There is no documentation, the
parameters work the same as the width_bucket function. The code is not
necessarily the most readable in the world, I was trying to get as much
speed out of it as possible, since I was calling it over a million times
as a group by value.
Thanks for any pointers...
--
Fortune's Office Door Sign of the Week:
Incorrigible punster -- Do not incorrige./*
* file:$RCSfile: bintimestamp.c,v $ $Revision: 1.1 $
* module: timestamp
* authors: jeremyd
* last mod:$Author: jeremyd $ at $Date: 2005/10/28 20:26:38 $
*
* created: Fri Oct 28 13:26:38 PDT 2005
*
*/
#include string.h
#include math.h
#include postgres.h
#include fmgr.h
#include libpq/pqformat.h
#include utils/builtins.h
#include funcapi.h
#include utils/timestamp.h
#ifndef JROUND
# define JROUND(x) (x)
#endif
Datum timestamp_get_bin_size(PG_FUNCTION_ARGS);
Datum timestamp_bin(PG_FUNCTION_ARGS);
PG_FUNCTION_INFO_V1(timestamp_get_bin_size);
Datum
timestamp_get_bin_size(PG_FUNCTION_ARGS)
{
Timestamp start = PG_GETARG_TIMESTAMP(0);
Timestamp stop = PG_GETARG_TIMESTAMP(1);
int32 nbuckets = PG_GETARG_INT32(2);
Interval * retval = (Interval *)palloc (sizeof(Interval));
if (!retval)
{
ereport(ERROR, (errcode(ERRCODE_OUT_OF_MEMORY),
errmsg(insufficient memory for Interval allocation)));
PG_RETURN_NULL();
}
memset (retval, 0, sizeof(Interval));
retval-time = JROUND ((stop - start) / nbuckets);
PG_RETURN_INTERVAL_P(retval);
}
PG_FUNCTION_INFO_V1(timestamp_bin);
Datum
timestamp_bin(PG_FUNCTION_ARGS)
{
/*Timestamp op = PG_GETARG_TIMESTAMP(0);*/
Timestamp start = PG_GETARG_TIMESTAMP(1);
/*Timestamp stop = PG_GETARG_TIMESTAMP(2);*/
Timestamp binsz;
/*int32 nbuckets = PG_GETARG_INT32(3)*/;
binsz = (PG_GETARG_TIMESTAMP(2) - start) / PG_GETARG_INT32(3);
PG_RETURN_TIMESTAMP(JROUND((int)((PG_GETARG_TIMESTAMP(0) - start) /
binsz) * binsz + start));
}
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [HACKERS] src/tools/msvc usage instructions
On Tue, 3 Oct 2006, Magnus Hagander wrote: Looks like the gendef script is failing. Check the contents of release\postgres\postgres.def - it should have thousands of symbols, but I'm willing to bet it's empty... It contains one word: EXPORTS. I assume this means it is empty. What should I do about it? Is there something I can check to see why this is failing? //Magnus -- Honesty is the best policy, but insanity is a better defense. ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] src/tools/msvc usage instructions
On Tue, 3 Oct 2006, Magnus Hagander wrote: Looks like the gendef script is failing. Check the contents of release\postgres\postgres.def - it should have thousands of symbols, but I'm willing to bet it's empty... It contains one word: EXPORTS. I assume this means it is empty. What should I do about it? Is there something I can check to see why this is failing? Yup. Delete the DEF file and run the gendef command manually (see the project file for commandline, IIRC there are no parameters, but just to be sure). I'm wondering if you're seeing the samre problem as Joachim Wieland (off-list conversation) where the output from dumpbin.exe goes to the console instead of the pipe in the perl program... I was just checking this, I read the gendef script, and saw it would short-circut if postgres.def existed, so I deleted the file and ran a build in visual studio again and it printed all kinds of dumpbin output into the visual stuio output window, which I remember it did before. Since you have seen this before, what was the fix (or was there one)? -- It's raisins that make Post Raisin Bran so raisiny ... ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] src/tools/msvc usage instructions
On Tue, 3 Oct 2006, Magnus Hagander wrote:
Funky.
Can you try having it run the dumpbin command into a tempfile, and then
open-and-read that tempfile, to see if that makes a difference?
(Assuming you know enough perl to do that, of course)
Doing it as
system(dumpbin /symbols $_ $tmpfn)
still output to the console.
But, I got it to work with the attached patch to the script. Note the use
of the handy /out:FILE parameter to dumpbin for redirecting the output ;)
Also, I changed the file glob to *.obj from * since I got an error trying
to run dumpbin on BuildLog.htm which is obviously not an object file.
Hopefully this is correct?
--
Q: Why do mountain climbers rope themselves together?
A: To prevent the sensible ones from going home.Index: gendef.pl
===
RCS file: X:\\postgres\\cvsuproot/pgsql/src/tools/msvc/gendef.pl,v
retrieving revision 1.1
diff -c -r1.1 gendef.pl
*** gendef.pl 4 Sep 2006 21:30:40 - 1.1
--- gendef.pl 3 Oct 2006 07:20:26 -
***
*** 10,18
print Generating $defname.DEF from directory $ARGV[0]\n;
! while ($ARGV[0]/*) {
print .;
! open(F,dumpbin /symbols $_|) || die Could not open $_\n;
while (F) {
s/\(\)//g;
next unless /^\d/;
--- 10,23
print Generating $defname.DEF from directory $ARGV[0]\n;
! while ($ARGV[0]/*.obj) {
print .;
! #open(F,dumpbin /symbols $_|) || die Could not open $_\n;
! s/\//\\/g;
! system(dumpbin /symbols $_ /out:$_.syms) == 0 or die Could not
dumpbin $_\n;
! my $tmpfn = $_.syms;
! open(F, $tmpfn) || die Could not open $tmpfn\n;
!
while (F) {
s/\(\)//g;
next unless /^\d/;
***
*** 31,36
--- 36,42
push @def, $pieces[6];
}
close(F);
+ unlink $tmpfn;
}
print \n;
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
[HACKERS] buildfarm failures in ECPG-Check
It looks like something broke the ECPG-Check recently. A number of buildfarm members are failing. On Tue, 3 Oct 2006, PG Build Farm wrote: The PGBuildfarm member mongoose had the following event on branch HEAD: Failed at Stage: ECPG-Check The snapshot timestamp for the build that triggered this notification is: 2006-10-03 22:30:01 The specs of this machine are: OS: Gentoo / 1.6.14 Arch: i686 Comp: icc / 9.0.032 For more information, see http://www.pgbuildfarm.org/cgi-bin/show_history.pl?nm=mongoosebr=HEAD -- Lewis's Law of Travel: The first piece of luggage out of the chute doesn't belong to anyone, ever. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] src/tools/msvc usage instructions
On Mon, 2 Oct 2006, Tom Lane wrote:
Jeremy Drake [EMAIL PROTECTED] writes:
I grabbed flex and bison from GNUwin32
(http://gnuwin32.sourceforge.net/packages/bison.htm)
This appears to not work out well. If I copy the generated files from
bison from a linux box, then they are ok, but if I try to use ones
generated using that version of bison, it does not compile.
Hm, have you tried diff'ing the output files from the two cases? This
is really probably a question for the flex and bison maintainers, not
us, but it seems like it should work for moderately up-to-date versions
of those tools. What compile failures do you get exactly?
I was just going to chalk it up to a bad matching of tool ports or
something and try to find a different bison, but if you are really
interested...
I get errors on any bison generated file. For simplicity of the diff, I
did not use the first failure I got, which was gram.c, but instead used
the much smaller bootparse.c file. I grabbed the bootparse.c files
generated on windows and on linux, did a diff -cw between them, and tarred
up the three files, which you can get from
http://www.jdrake.com/postgresql/bison-files-win32.tar.gz
The errors I got on this file were:
1-- Build started: Project: postgres, Configuration: Release Win32 --
1Compiling...
1bootparse.c
1bootparse.tab.c(1065) : error C2449: found '{' at file scope (missing
function header?)
1bootparse.tab.c(1858) : error C2059: syntax error : '}'
and then a whole lot of random, uninteresting errors of the kind you get
when the compiler is so confused it no longer knows what it is doing.
I am currently trying to build a newer version of bison using mingw and
use it, but I am running into issues with that also.
Oh, I just got the email from Magnus which says do not use v2.1, but
1.875, so I guess that's what I did wrong. Oops!
regards, tom lane
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
--
Drive defensively. Buy a tank.
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
Re: [HACKERS] src/tools/msvc usage instructions
On Mon, 2 Oct 2006, Magnus Hagander wrote: This appears to not work out well. If I copy the generated files from bison from a linux box, then they are ok, but if I try to use ones generated using that version of bison, it does not compile. I'll look around for a different one. That's the onw I'm using. However, be sure to get version 1.875-4, and *not* version 2.1. Oops, that was it. If you do build solution it should build all project sin the correct order - there are dependency references set between them that should take care of this automatically. If I do build solution it tells me Project not selected to build for this solution configuration for all projects, then 55 skipped at the end. Do you have any idea how to get the environment to know where windows.h is? I even explicitly added the directory to the INCLUDE environment variable, but it did not work. I will try switching to short paths in there in case it is an issue of paths with spaces. In my environment, that gets set when I start the Visual Studio command prompt - that's the whole point abou tusing the VS commandprompt and not a normal one. I think you get a question about integrating the Platform SDK with Visual studio when you install it - any chance you missed that one? Well, it works in the gui, so I thought I got that integrated correctly. One of the deals with the visual c express thing is that it does not come with the headers and libraries and that you have to use the platform sdk instead. I already have ActivePerl and ActivePython installed, so those should work out. I am not really concerned about krb5 and ldap, so as long as commenting them out will disable them, that is good. You can safely leave LDAP in, because it uses only the builtin functionality in the OS and no external dependencies. //Magnus ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq -- I don't know what you mean by `glory,' Alice said Humpty Dumpty smiled contemptuously. Of course you don't -- till I tell you. I meant `there's a nice knock-down argument for you!' But glory doesn't mean `a nice knock-down argument,' Alice objected. When I use a word, Humpty Dumpty said, in a rather scornful tone, it means just what I choose it to mean -- neither more nor less. The question is, said Alice, whether you can make words mean so many different things. The question is, said Humpty Dumpty, which is to be master-- that's all. -- Lewis Carroll, Through the Looking Glass ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] src/tools/msvc usage instructions
On Sun, 1 Oct 2006, Jeremy Drake wrote: On Mon, 2 Oct 2006, Magnus Hagander wrote: If you do build solution it should build all project sin the correct order - there are dependency references set between them that should take care of this automatically. If I do build solution it tells me Project not selected to build for this solution configuration for all projects, then 55 skipped at the end. I clicked around a little, selected the postgres project in the project list, and switched to the release configuration, and now build solution works. Hmm. Do you have any idea how to get the environment to know where windows.h is? I even explicitly added the directory to the INCLUDE environment variable, but it did not work. I will try switching to short paths in there in case it is an issue of paths with spaces. I switched to short paths in the INCLUDE env var, but it seems to just ignore it. I'll have to look around for how to deal with this, but for now perhaps the gui will work ok. Is there anything that needs to happen post-compile that may not get done if I use the gui? -- H. L. Mencken suffers from the hallucination that he is H. L. Mencken -- there is no cure for a disease of that magnitude. -- Maxwell Bodenheim ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] src/tools/msvc usage instructions
On Mon, 2 Oct 2006, Tom Lane wrote:
Jeremy Drake [EMAIL PROTECTED] writes:
The errors I got on this file were:
1bootparse.tab.c(1065) : error C2449: found '{' at file scope (missing
function header?)
I looked at this. Line 1065 is the left brace starting yyparse(). On
my Fedora Core 5 box with Bison 2.1 installed, the stuff leading up to
it is
#ifdef YYPARSE_PARAM
... some uninteresting stuff, since we don't define YYPARSE_PARAM ...
#else /* ! YYPARSE_PARAM */
#if defined (__STDC__) || defined (__cplusplus)
int
yyparse (void)
#else
int
yyparse ()
#endif
#endif
{
But lookee here, your Windows-built version has
#ifdef YYPARSE_PARAM
...
#else /* ! YYPARSE_PARAM */
#if defined (__STDC__) || defined (__cplusplus)
int
yyparse (void)
#else
int
yyparse ()
;
#endif
#endif
{
So that semicolon is the source of the trouble. That's clearly a bison
bug, and in fact digging in Red Hat's SRPM shows that they are carrying
a patch for it:
2005-10-05 Paul Eggert [EMAIL PROTECTED]
* data/m4sugar/m4sugar.m4 (_m4_map): New macro.
(m4_map, m4_map_sep): Use it. Handle the empty list correctly.
snip patch
Presumably bison 2.2 includes this fix. But I guess the real question
is why the devil doesn't MSVC define __STDC__ ? Are they that far
removed from spec compliance?
In the bison 2.2 generated code, the #if check is
#if (defined __STDC__ || defined __C99__FUNC__ \
|| defined __cplusplus || defined _MSC_VER)
which looks like they figured out that they needed to check for MicroSoft
C explicitly. I have no idea why they do not define __STDC__ however.
regards, tom lane
--
A person is just about as big as the things that make him angry.
---(end of broadcast)---
TIP 6: explain analyze is your friend
Re: [HACKERS] src/tools/msvc usage instructions
I now get things to compile, but now I get linker errors on any dll which needs to access symbols from postgres.exe via postgres.lib. For example: 1-- Build started: Project: autoinc, Configuration: Release Win32 -- 1Generate DEF file 1Not re-generating AUTOINC.DEF, file already exists. 1Linking... 1 Creating library Release\autoinc\autoinc.lib and object Release\autoinc\autoinc.exp 1autoinc.obj : error LNK2019: unresolved external symbol _SPI_modifytuple referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _pfree referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _pg_detoast_datum referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _nextval referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _DirectFunctionCall1 referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _textin referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _SPI_getbinval referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _SPI_gettypeid referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _errfinish referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol ___msvc_errcode referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _errmsg referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _errstart referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _SPI_fnumber referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _MemoryContextAlloc referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _SPI_getrelname referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _elog_finish referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _elog_start referenced in function _autoinc 1.\Release\autoinc\autoinc.dll : fatal error LNK1120: 17 unresolved externals I checked the project properties for linker options and it does list Release\postgres\postgres.lib in the additional dependencies list. Any ideas? Am I missing something? -- A penny saved is ridiculous. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] src/tools/msvc usage instructions
I was just trying to build using the src/tools/msvc scripts on windows, and I was wondering if there were any instructions on how to do this, what prerequisites there are, where to get them, etc. I couldn't find any, but I may not know the correct place to look. Sorry if this is the wrong list for this question. -- People need good lies. There are too many bad ones. -- Bokonon, Cat's Cradle by Kurt Vonnegut, Jr. ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Developer's Wiki
On Wed, 27 Sep 2006, Lukas Kahwe Smith wrote: Dave Page wrote: I have now moved the wiki installation to: http://developer.postgresql.org/ BTW: I am wondering if there is an RSS feed of the changes? On my wiki I have an RSS feed for every page, subwiki (aka area) and the entire wiki people can subscribe to: http://oss.backendmedia.com/rss.php?area=PHPTODOpage=HomePage http://oss.backendmedia.com/rss.php?area=PHPTODO http://oss.backendmedia.com/rss.php I only really know of the entire wiki one, but that's the only one I have ever wanted to do. I think it may be able to limit to namespaces, but I am not sure about that. http://developer.postgresql.org/index.php?title=Special:Recentchangesfeed=rss There are a bunch of knobs on the Special:Recentchanges page which could apply also to the rss version, but I have never tried it and they may not, I don't know. regards, Lukas -- Besides the device, the box should contain: * Eight little rectangular snippets of paper that say WARNING * A plastic packet containing four 5/17 inch pilfer grommets and two club-ended 6/93 inch boxcar prawns. YOU WILL NEED TO SUPPLY: a matrix wrench and 60,000 feet of tram cable. IF ANYTHING IS DAMAGED OR MISSING: You IMMEDIATELY should turn to your spouse and say: Margaret, you know why this country can't make a car that can get all the way through the drive-through at Burger King without a major transmission overhaul? Because nobody cares, that's why. WARNING: This is assuming your spouse's name is Margaret. -- Dave Barry, Read This First! ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] [HACKERS] large object regression tests
On Sun, 24 Sep 2006, Jeremy Drake wrote: On Thu, 21 Sep 2006, Tom Lane wrote: I suggest that instead of testing the server-side lo_import/lo_export functions, perhaps you could test the psql equivalents and write and read a file in psql's working directory. snip In the mean time, I will alter the test to also test the psql backslash commands based on how the copy equivalents are tested, since I had forgotten them and they need to be tested also. I just tried using the \lo_import command in a regression test, and I think I figured out why this will not work: $ make check ... largeobject ... FAILED ... $ cat regression.diffs *** ./expected/largeobject.out Sun Sep 24 19:55:25 2006 --- ./results/largeobject.out Sun Sep 24 19:55:58 2006 *** *** 188,194 (1 row) \lo_import 'results/lotest.txt' ! lo_import 31138 \set newloid :LASTOID -- This is a hack to test that export/import are reversible -- This uses knowledge about the inner workings of large object mechanism --- 188,194 (1 row) \lo_import 'results/lotest.txt' ! lo_import 31199 \set newloid :LASTOID -- This is a hack to test that export/import are reversible -- This uses knowledge about the inner workings of large object mechanism == Yes, that's the large object OID in the output there, and it is different each run (as I expect). If you look at src/bin/psql/large_obj.c line 192, you see: fprintf(pset.queryFout, lo_import %u\n, loid); Which is executed unconditionally whenever the lo_import is successful. While in a normal circumstance, it is quite necessary to know the loid, since it does change each call, in this case it serves to break the diffs, and so I guess it is impossible to use the \lo_import command in a regression test. -- The first time, it's a KLUDGE! The second, a trick. Later, it's a well-established technique! -- Mike Broido, Intermetrics ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] [HACKERS] large object regression tests
On Mon, 25 Sep 2006, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: I just tried using the \lo_import command in a regression test, and I think I figured out why this will not work: ... Yes, that's the large object OID in the output there, and it is different each run (as I expect). Right. I'd suggest temporarily setting ECHO off to hide the unpredictable part of the output. There are similar measures taken in many of the contrib tests. I tried this: jeremyd=# \set QUIET jeremyd=# \set ECHO off jeremyd=# BEGIN; jeremyd=# \lo_import results/lotest.txt lo_import 84951 jeremyd=# ROLLBACK; From what I could tell in the code, the message is printed regardless of setting. It looks like the large_obj.c output is missing much of the output settings handling which is in the PrintQueryStatus function in common.c, such as handling quiet mode, and html output. I will try to dig around and try to put together a patch to make it respect the settings like other commands... -- You are old, said the youth, and your programs don't run, And there isn't one language you like; Yet of useful suggestions for help you have none -- Have you thought about taking a hike? Since I never write programs, his father replied, Every language looks equally bad; Yet the people keep paying to read all my books And don't realize that they've been had. ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] [HACKERS] large object regression tests
On Sun, 24 Sep 2006, Jeremy Drake wrote: On Thu, 21 Sep 2006, Tom Lane wrote: I think we could do without the Moby Dick extract too ... I am open to suggestions. I saw one suggestion that I use an image of an elephant, but I suspect that was tongue-in-cheek. I am not very fond of the idea of generating repetitious data, as I think it would be more difficult to determine whether or not the loseek/tell functions put me in the right place in the middle of the file. I just had the idea that I could use one of the existing data files which are used for testing COPY instead of the Moby Dick extract. They are already there, a few of them are pretty good sized, they have data in the file which is not just simple repetition so it would be pretty obvious if the seek function broke, and they are very unlikely to change. I am considering changing the test I put together to use tenk.data as the input file tomorrow and send in what I have again, since I also am doing a test of \lo_import (which also requires a patch to psql I sent in earlier to fix the output of the \lo_* commands to respect the output settings). -- When does summertime come to Minnesota, you ask? Well, last year, I think it was a Tuesday. ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] [HACKERS] large object regression tests
On Thu, 21 Sep 2006, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: I put together a patch which adds a regression test for large objects, hopefully attached to this message. I would like some critique of it, to see if I have gone about it the right way. Also I would be happy to hear any additional tests which should be added to it. I'd prefer it if we could arrange not to need any absolute paths embedded into the test, because maintaining tests that require such is a real PITA --- instead of just committing the actual test output, one has to reverse-convert it to a .source file. I just copied how the test for COPY worked, since I perceived a similarity in what I needed to do (use external files to load data). I suggest that instead of testing the server-side lo_import/lo_export functions, perhaps you could test the psql equivalents and write and read a file in psql's working directory. I did not see any precedent for that when I was looking around in the existing tests for an example of how to do things. I am not even sure where the cwd of psql is, so I can put an input file there. Could you provide an example of how this might look, by telling me where to put a file in the src/test/regress tree and the path to give to \lo_import? Besides which, shouldn't both the server-side and psql versions be tested? When I was looking at the copy tests, it looked like the server-side ones were tested, and then the psql ones were tested by exporting and then importing data which was originally loaded from the server-side method. Am I correctly interpreting the precedent, or are you suggesting that the precedent be changed? I was trying to stay as close to the copy tests as possible since the functionality is so similar (transferring data to/from files in the filesystem, either via server-side functions which require absolute paths or via psql \ commands (which I forgot about for the lo funcs)). I think we could do without the Moby Dick extract too ... I am open to suggestions. I saw one suggestion that I use an image of an elephant, but I suspect that was tongue-in-cheek. I am not very fond of the idea of generating repetitious data, as I think it would be more difficult to determine whether or not the loseek/tell functions put me in the right place in the middle of the file. Perhaps if there was a way to generate deterministic pseudo-random data, that would work (has to be deterministic so the diffs of the output come out right). Anyone have a good example of seeding a random number generator and generating a bunch of bytea which is deterministic cross-platform? regards, tom lane In the mean time, I will alter the test to also test the psql backslash commands based on how the copy equivalents are tested, since I had forgotten them and they need to be tested also. -- Any sufficiently advanced technology is indistinguishable from a rigged demo. ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[HACKERS] pls disregard, testing majordomo settings
I just messed with a bunch of my majordomo settings and I wanted to make sure things are working the way I thought. Please disregard. Sorry to bother everyone -- I'll defend to the death your right to say that, but I never said I'd listen to it! -- Tom Galloway with apologies to Voltaire ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] [HACKERS] large object regression tests
On Sun, 24 Sep 2006, Jeremy Drake wrote: On Thu, 21 Sep 2006, Tom Lane wrote: I suggest that instead of testing the server-side lo_import/lo_export functions, perhaps you could test the psql equivalents and write and read a file in psql's working directory. I did not see any precedent for that when I was looking around in the existing tests for an example of how to do things. snip When I was looking at the copy tests, it looked like the server-side ones were tested, and then the psql ones were tested by exporting and then importing data which was originally loaded from the server-side method. I just went back and looked at the tests again. The only time the psql \copy command was used was in the (quite recent IIRC) copyselect test, and then only via stdout (never referring to psql working directory, or to files at all). Did I misunderstand, and you are proposing a completely new way of doing things in the regression tests? I am not particularly fond of the sed substitution stuff myself, but it seems to be the only currently supported/used method in the regression tests... I do think that making the large object test and the copy test consistent would make a lot of sense, since as I said before, the functionality of file access is so similar... -- We demand rigidly defined areas of doubt and uncertainty! -- Vroomfondel ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
