Re: [HACKERS] compiler warnings on the buildfarm

2007-07-13 Thread Stefan Kaltenbrunner
Tom Lane wrote:
 Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
 animal: lionfishwarnings: 16
 scan.l:180: warning, the character range [80-FF] is ambiguous in a
 case-insensitive scanner
 scan.l:180: warning, the character range [80-FF] is ambiguous in a
 case-insensitive scanner
 scan.l:302: warning, the character range [80-FF] is ambiguous in a
 case-insensitive scanner
 
 This is evidently complaining about plpgsql's scan.l, which specifies
 %option case-insensitive
 and then defines
 ident_start   [A-Za-z\200-\377_]
 which is the way we do it in the main grammar too.  But I've never
 seen this message in any of the flex versions I've used with PG.
 (Which flex version is installed on lionfish anyway?)

$ flex -V
flex 2.5.31



 
 I find some relevant points in the flex manual:
 http://flex.sourceforge.net/manual/Patterns.html
 
   Character classes are expanded immediately when seen in the flex
   input. This means the character classes are sensitive to the locale in
   which flex is executed, and the resulting scanner will not be sensitive
   to the runtime locale. This may or may not be desirable.
   
   Character classes with ranges, such as `[a-Z]', should be used with
   caution in a case-insensitive scanner if the range spans upper or
   lowercase characters. Flex does not know if you want to fold all upper
   and lowercase characters together, or if you want the literal numeric
   range specified (with no case folding). When in doubt, flex will assume
   that you meant the literal numeric range, and will issue a warning. The
   exception to this rule is a character range such as `[a-z]' or `[S-W]'
   where it is obvious that you want case-folding to occur.
 
 What I suspect is happening is that lionfish is running the buildfarm
 script in a non-C locale, in which flex finds that some high-bit-set
 characters are case-folded by tolower() and accordingly issues this
 complaint.  Now the statements that it assumes you meant the literal
 numeric range and that the behavior is fully determined at compile time
 (ie, no run-time invocations of tolower(), as indeed are not to be seen
 in pl_scan.c) seem to mean that we'll get the behavior we want anyway.
 But the warning is a bit nervous-making.

hmmm - note that lionfish is not the only box reporting that kind of
warning - also affected are:

rosella (which is definitly running in a non-C locale as all the errors
are in german there)
wildebeest

Stefan

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] compiler warnings on the buildfarm

2007-07-13 Thread Stefan Kaltenbrunner
Tom Lane wrote:
 Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
 some more(I have removed duplicates and ones that should be fixed by
 your latest commits though):
 
 I did what I could with this batch.  Some comments:
 
 animal: salamander  warnings: 27
 cash.c: In function `cash_in':
 cash.c:244: warning: subscript has type `char'
 
 I wish we could promote this one to be a hard error :-(.  It typically
 indicates (and did in this case) that someone has unportably forgotten
 to cast the argument of a ctype.h macro to unsigned char.

:-( we can promote certain warnings to an error on sun studio vor
example but salamander is running gcc ...

 
 pg_lzcompress.c: In function `pglz_compress':
 pg_lzcompress.c:378: warning: inlining failed in call to `pglz_find_match'
 
 This is not an error condition, it just means that gcc decided not to do
 inlining because the called function was too big.  IIRC we had some
 discussion whether to specify -Winline or not, and decided to do so in
 order to gather some info about whether we were overstressing inline.
 We could live with it as-is, or document somewhere (where?) that it's
 fine as long as you don't see very many of 'em, or decide that the
 experiment is finished and we should take out -Winline.  Comments?

we have about 5 boxes on the farm with that exact warning (inlining
failed in pglz_find_match) there are no other inline related warnings at
all afaiks.

 animal: lionfishwarnings: 16
 /tmp/cclwN8N9.s: Assembler messages:
 /tmp/cclwN8N9.s:109082: Warning: Macro instruction expanded into
 multiple instructions
 [multiple occurrences]
 
 This is pretty strange.  It seems to occur only in files generated
 from bison and/or flex.  Anybody have a clue?

other than that lionfish is a weird mips box - no :-)


Stefan

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [HACKERS] compiler warnings on the buildfarm

2007-07-13 Thread Stefan Kaltenbrunner
Tom Lane wrote:
[...]
 animal: clownfish   warnings: 12
 dynloader.c, line 4: warning: empty translation unit
 postgres.c, line 3758: warning: loop not entered at top
 
 The first of these is not a bug, the second seems to be some weird
 aberration in their statement-not-reached detection.

will see about filtering out those

 
 animal: grebe   warnings: 45
 xlog.c:651: warning: implicit declaration of function '_check_lock'
 xlog.c:654: warning: implicit declaration of function '_clear_lock'
 hba.c:1449: warning: implicit declaration of function 'getpeereid'
 
 Someone needs to find out which system headers declare these functions
 on AIX.
 
 ip.c: In function 'getaddrinfo_unix':
 ip.c:254: warning: large integer implicitly truncated to unsigned type
 
 This is complaining about
 
 #ifdef HAVE_STRUCT_SOCKADDR_STORAGE_SS_LEN
   unp-sun_len = sizeof(struct sockaddr_un);
 #endif
 
 I don't know how wide sun_len is on this platform.  It's probably uint8,
 but if we explicitly cast the sizeof to 8 bits, we could conceivably
 break things on other platforms.  Are there any where sockaddr_un is
 longer than 255 bytes?  Anyway I'm inclined to leave this alone.

no idea on AIX but I have added christopher to the CC list - maybe he
can shed some light on those things.

 
 guc.c:2866: warning: 'guc_get_index' defined but not used
 Extra instructions are being generated for each reference to a TOC
 symbol if the symbol is in the TOC overflow area.
 
 This is fairly bizarre, since 'guc_get_index' *is* used in guc-file.c,
 which is included into this same file.  However I don't much like the
 coding method used here (it is certainly not better than using a
 temporary flag bit), so when I get a chance I'll rewrite it out of
 existence.
 
 connect.c:23: warning: missing braces around initializer
 connect.c:23: warning: (near initialization for
 'actual_connection_key_once.__on_word')
 misc.c:67: warning: missing braces around initializer
 misc.c:67: warning: (near initialization for 'sqlca_key_once.__on_word')
 
 I think these are a platform bug.  The spec clearly says that
 
 static pthread_once_t actual_connection_key_once = PTHREAD_ONCE_INIT;
 
 is exactly how you are supposed to do it.  If pthread_once_t is a struct
 on a given platform, that platform ought to be defining
 PTHREAD_ONCE_INIT with the appropriate braces included.  If we added
 braces ourselves we'd break it for platforms where the macro is correct
 already.  Hence, not our problem.

I see


Stefan

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [HACKERS] compiler warnings on the buildfarm

2007-07-13 Thread Stefan Kaltenbrunner
Tom Lane wrote:
 Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
 ok I did that for a few members (removing all the statement not reached
 ones as well as some purely informal notices and all the flex related
 warnings) and came up with something similiar to:
 [snip]
 
 Yeah, this looks like a good list.  I can't readily check the ones from
 eel as they appear to be in Windows-specific code; anyone else want to
 fix those?

and this is the initial list for contrib(excluding a lot of duplicate
warnings and stuff that is a result of invalid compiler flags which I
will mention seperatly):

animal: salamander  warnings: 6
stopword.c: In function `readstoplist':
stopword.c:51: warning: subscript has type `char'

animal: dragonfly   warnings: 4
pgbench.c: In function `main':
pgbench.c:1445: warning: int format, pid_t arg (arg 4)
stopword.c: In function `readstoplist':
stopword.c:51: warning: subscript has type `char'


animal: clownfish   warnings: 12
crc32.c, line 93: warning: initializer does not fit or is out of range: -1
crc32.c, line 102: warning: initializer does not fit or is out of
range: -1
imath.c, line 3202: warning: integer overflow detected: op 
imath.c, line 3206: warning: integer overflow detected: op 
query_cleanup.c, line 179: warning: macro redefined: V_FALSE
crc32.c, line 95: warning: initializer does not fit or is out of range: -1
query_support.c, line 199: warning: syntax error:  empty declaration
query_support.c, line 200: warning: syntax error:  empty declaration
query_support.c, line 201: warning: syntax error:  empty declaration
query_support.c, line 202: warning: syntax error:  empty declaration
query_support.c, line 203: warning: syntax error:  empty declaration
query_support.c, line 204: warning: syntax error:  empty declaration


animal: kuduwarnings: 13
crc32.c, line 93: warning: initializer does not fit or is out of range: -1
crc32.c, line 102: warning: initializer does not fit or is out of
range: -1
oid2name.c, line 579: warning: Function has no return statement : main
pg_standby.c, line 622: warning: Function has no return statement : main
imath.c, line 3202: warning: integer overflow detected: op 
dict_thesaurus.c, line 699: warning: non-constant initializer: op NAME
crc32.c, line 95: warning: initializer does not fit or is out of range: -1
query_support.c, line 199: warning: syntax error:  empty declaration
query_support.c, line 200: warning: syntax error:  empty declaration
query_support.c, line 201: warning: syntax error:  empty declaration
query_support.c, line 202: warning: syntax error:  empty declaration
query_support.c, line 203: warning: syntax error:  empty declaration
query_support.c, line 204: warning: syntax error:  empty declaration

animal: warthog warnings: 396
UX:acomp: WARNING: btreefuncs.c, line 59: no macro replacement within
a string literal
UX:acomp: WARNING: pgstatindex.c, line 50: no macro replacement within
a string literal
UX:acomp: WARNING: xpath.c, line 212: argument #1 incompatible with
prototype: strlen()
UX:acomp: WARNING: xpath.c, line 268: argument #2 incompatible with
prototype: xmlBufferWriteChar()
UX:acomp: WARNING: xpath.c, line 607: argument #1 incompatible with
prototype: xmlStrdup()
UX:acomp: WARNING: xpath.c, line 612: argument #1 incompatible with
prototype: strlen()
UX:acomp: WARNING: xpath.c, line 663: assignment type mismatch
UX:acomp: WARNING: xpath.c, line 738: assignment type mismatch
UX:acomp: WARNING: xpath.c, line 742: argument #1 incompatible with
prototype: strstr()
UX:acomp: WARNING: xpath.c, line 742: argument #2 incompatible with
prototype: strstr()
UX:acomp: WARNING: xpath.c, line 742: assignment type mismatch
UX:acomp: WARNING: xpath.c, line 896: argument #1 incompatible with
prototype: xmlStrdup()
UX:acomp: WARNING: xpath.c, line 904: assignment type mismatch
UX:acomp: WARNING: xslt_proc.c, line 105: argument #1 incompatible
with prototype: xsltParseStylesheetFile()


animal: emperor_mothwarnings: 11
pgbench.c: In function `main':
pgbench.c:1445: warning: int format, pid_t arg (arg 4)
query_cleanup.c:179:1: warning: V_FALSE redefined
In file included from /usr/include/sys/stream.h:22,
 from /usr/include/netinet/in.h:66,
 from /usr/include/netdb.h:98,
 from ../../src/include/port.h:17,
 from ../../src/include/c.h:839,
 from ../../src/include/postgres.h:48,
 from query_cleanup.c:6:
/usr/include/sys/vnode.h:505:1: warning: this is the location of the
previous definition

animal: cuckoo  warnings: 9
y.tab.c: In function 'yy_reduce_print':
y.tab.c:764: warning: passing argument 3 of 'yy_symbol_print' from
incompatible pointer type
y.tab.c: In function 'yydestruct':
y.tab.c:1036: warning: passing argument 3 of 'yy_symbol_print' from
incompatible pointer type
y.tab.c: In function 'cube_yyparse':
y.tab.c:1277: warning: passing argument 3 of 'yy_symbol_print' from

[HACKERS] stored procedure stats in collector

2007-07-13 Thread Martin Pihlak

Howdy,

I'm working on a patch to extend the stats collector to handle stored procedure
statistics (call counts, duration etc). The goal is to make this information
visible via pg_stat functions/views. The collection would be controllable via
stats_function_level GUC and will have minimal overhead when turned off. At
our company we depend heavily on stored procedures and such a tool would be of
great help. Perhaps others could also find it somewhat useful.

So far I have a working patch against 8.2.4 (attached) that places some
instrumentation in the executor (ExecMakeFunctionResult,
ExecMakeFunctionResultNoSets and ExecMakeTableFunctionResult) and send the
stats to the collector (handled similarly to table stats).  The results are
visible through pg_stat_user_functions:


martinp=# set stats_function_level = on;
SET
Time: 0.172 ms

martinp=# select count(f3()) from generate_series(1, 1000);
 count
---
  1000
(1 row)

Time: 40059.713 ms

martinp=# select * from pg_stat_user_functions ;
 procid | schemaname | procname | nargs | calls | total_time | total_cpu | 
self_time | self_cpu
++--+---+---++---+---+--
  16388 | public | f1   | 0 |  4000 |  14978 |  8352 | 
14978 | 8352
  16389 | public | f2   | 0 |  2000 |  40044 |  8364 | 
25066 |   12
  16390 | public | f3   | 0 |  1000 |  40054 |  8364 |  
   9 |0
(3 rows)

Time units are in milliseconds. Only functions with oid = FirstNormalObjectId
are accounted.

This is of course still very experimental - some work is required to get this
into production shape. I was hoping to get some feedback before I continue
though. Maybe there are some obvious flaws that I'm not seeing, maybe something
needs to be changed to make this more generally useful. Feedback is greatly
appreciated.

Regards,
Martin


8.2.4-procstat.patch.gz
Description: GNU Zip compressed data

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [HACKERS] compiler warnings on the buildfarm

2007-07-13 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes:
 Zdenek Kotala wrote:
 I don't see any const keyword there.

 Right after that:

 where int conv(int num_msg, const struct pam_message **msg, struct 
 pam_response **resp, void *appdata_ptr); 

 How confusing...

And the pam_start page he cited earlier has a different set of typos in
its version of the struct :-(.  Still, that's two out of three places
that say it's const, and Solaris appears to be the only implementation
that has chosen to read it as not const.

regards, tom lane

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


[HACKERS] has anyone looked at burstsort ?

2007-07-13 Thread Hannu Krosing
has anyone looked at burstsort
https://sourceforge.net/projects/burstsort

they claim that Copy-Burstsort is a sorting algorithm for strings that
is cache-efficient. Burstsort and its variants are much faster than
Quicksort and Radixsort especially on large datasets. Copy-Burstsort
works best for sorting short strings such as genomes and words

if the speed claim is true, and there are no other bad effects, like for
example very bad memory use,  we could try to talk the author into
allowing us to include it under BSD licens (currently it is GPL)


Hannu



---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] compiler warnings on the buildfarm

2007-07-13 Thread Zdenek Kotala

Heikki Linnakangas wrote:

Zdenek Kotala wrote:
If I look there 
http://www.opengroup.org/onlinepubs/008329799/chap5.htm#tagcjh_06


in Call Back Information section. The structure is defined as

struct pam_conv{ int (*conv) (int, struct pam_message **, struct 
pam_response **, void *); void *appdata_ptr; };



I don't see any const keyword there.


Right after that:

 where int conv(int num_msg, const struct pam_message **msg, 
struct pam_response **resp, void *appdata_ptr); 


Ups, I overlooked it.


How confusing...


Yes agree.

Zdenek



---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

  http://www.postgresql.org/docs/faq


Re: [HACKERS] compiler warnings on the buildfarm

2007-07-13 Thread Kris Jurka



On Fri, 13 Jul 2007, Zdenek Kotala wrote:


Tom Lane wrote:

Kris Jurka [EMAIL PROTECTED] writes:

So pam_message ** isn't const.


Ah, thanks.  I see luna_moth is giving the same warning, so it's still
not const in Solaris 11 either.

Is it worth working around this?  It's strictly cosmetic AFAICS.

The main issue in my mind would be how to determine whether to use
const or not.  If all Solaris releases are like this, and can be
expected to stay that way,


I think yes. It is defined as X/Open standard says.



Not according to the link you sent earlier.  My reading says that Solaris 
has it defined wrong and pg has it right.


Kris Jurka

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match


Re: [HACKERS] compiler warnings on the buildfarm

2007-07-13 Thread Zdenek Kotala

Stefan Kaltenbrunner wrote:

Zdenek Kotala wrote:



For sun studio -erroff=E_STATEMENT_NOT_REACHED is useful there. If you
want to determine warning tags for each warning add -errtags.


Is that supported on all versions of sun studio(Sun WorkShop 6, Sun
Studio 8,11) we have on the farm ?


Yes. Also on SS12.

Zdenek

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match


Re: [HACKERS] compiler warnings on the buildfarm

2007-07-13 Thread Heikki Linnakangas

Zdenek Kotala wrote:
If I look there 
http://www.opengroup.org/onlinepubs/008329799/chap5.htm#tagcjh_06


in Call Back Information section. The structure is defined as

struct pam_conv{ int (*conv) (int, struct pam_message **, struct 
pam_response **, void *); void *appdata_ptr; };



I don't see any const keyword there.


Right after that:

 where int conv(int num_msg, const struct pam_message **msg, struct pam_response **resp, void *appdata_ptr); 


How confusing...

--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] compiler warnings on the buildfarm

2007-07-13 Thread Zdenek Kotala

Kris Jurka wrote:



On Fri, 13 Jul 2007, Zdenek Kotala wrote:


Tom Lane wrote:

Kris Jurka [EMAIL PROTECTED] writes:

So pam_message ** isn't const.


Ah, thanks.  I see luna_moth is giving the same warning, so it's still
not const in Solaris 11 either.

Is it worth working around this?  It's strictly cosmetic AFAICS.

The main issue in my mind would be how to determine whether to use
const or not.  If all Solaris releases are like this, and can be
expected to stay that way,


I think yes. It is defined as X/Open standard says.



Not according to the link you sent earlier.  My reading says that 
Solaris has it defined wrong and pg has it right.


If I look there 
http://www.opengroup.org/onlinepubs/008329799/chap5.htm#tagcjh_06


in Call Back Information section. The structure is defined as

struct pam_conv{ int (*conv) (int, struct pam_message **, struct 
pam_response **, void *); void *appdata_ptr; };



I don't see any const keyword there.


Zdenek


---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match


Re: [HACKERS] compiler warnings on the buildfarm

2007-07-13 Thread Zdenek Kotala

Tom Lane wrote:

Kris Jurka [EMAIL PROTECTED] writes:

So pam_message ** isn't const.


Ah, thanks.  I see luna_moth is giving the same warning, so it's still
not const in Solaris 11 either.

Is it worth working around this?  It's strictly cosmetic AFAICS.

The main issue in my mind would be how to determine whether to use
const or not.  If all Solaris releases are like this, and can be
expected to stay that way,


I think yes. It is defined as X/Open standard says.


I'd be inclined to just put a #define
PAM_CONV_PROC_NOT_CONST in include/port/solaris.h and drive the
function declaration off that.  If there's a version dependency
involved then it gets a lot more complicated, and might not be worth
the trouble.


Following patch works for me, but I did not test it on other platform.


retrieving revision 1.153
diff -r1.153 auth.c
61c61
   pam_passwd_conv_proc,
---
   (int (*)())pam_passwd_conv_proc,


Zdenek

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] compiler warnings on the buildfarm

2007-07-13 Thread Zdenek Kotala

Stefan Kaltenbrunner wrote:

Peter Eisentraut wrote:

Am Donnerstag, 12. Juli 2007 15:25 schrieb Stefan Kaltenbrunner:

a lot of those are simply noise (like the LOOP VECTORIZED stuff from the
icc boxes or the statement not reached spam from the sun compilers)
but others might indicate real issues.
To find warnings that might be a real problem we might want to look into
 suppressing those - if possible -  using compiler switches.
It would be good to determine an appropriate set of compiler switches to 
reduce the warnings to a reasonable level.


yeah once we have determined that this whole experiment is useful it
should be pretty easy to tweak the compiler switches for the non-gcc
compilers (mostly icc and sun studio seem to be the ones that generate
excessive output).



For sun studio -erroff=E_STATEMENT_NOT_REACHED is useful there. If you 
want to determine warning tags for each warning add -errtags.



Zdenek

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] compiler warnings on the buildfarm

2007-07-13 Thread Zdenek Kotala

Tom Lane wrote:

Heikki Linnakangas [EMAIL PROTECTED] writes:

Zdenek Kotala wrote:

I don't see any const keyword there.



Right after that:


where int conv(int num_msg, const struct pam_message **msg, struct pam_response **resp, void *appdata_ptr); 



How confusing...


And the pam_start page he cited earlier has a different set of typos in
its version of the struct :-(.  Still, that's two out of three places
that say it's const, and Solaris appears to be the only implementation
that has chosen to read it as not const.


Yes, Agree. I try to send request to security team for explanation. 
Maybe original author also overlooked it as me today :-).


Zdenek

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

   http://www.postgresql.org/about/donate


Re: [HACKERS] has anyone looked at burstsort ?

2007-07-13 Thread Tom Lane
Hannu Krosing [EMAIL PROTECTED] writes:
 has anyone looked at burstsort
 https://sourceforge.net/projects/burstsort
 they claim that Copy-Burstsort is a sorting algorithm for strings that
 is cache-efficient.

If its reason for living is cache efficiency, then I wonder

(1) how well does it work on data types other than strings, or for
that matter even strings when the comparison function is strcoll()
rather than memcmp().  A heavyweight comparison function is likely
to play hob with any assumptions about memory access patterns.

(2) does it scale to deal with problems larger than memory (ie,
how do you make it spill to disk).

 if the speed claim is true, and there are no other bad effects, like for
 example very bad memory use,  we could try to talk the author into
 allowing us to include it under BSD licens (currently it is GPL)

If we wrote our own implementation ... and realistically, fitting it
into postgres would likely require rewriting much of the code anyway
... then we don't have to worry about the copyright on someone else's
implementation.  What we do have to worry about is whose patent(s)
we might be infringing.

regards, tom lane

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] has anyone looked at burstsort ?

2007-07-13 Thread Gregory Stark

Hannu Krosing [EMAIL PROTECTED] writes:

 has anyone looked at burstsort
 https://sourceforge.net/projects/burstsort

 they claim that Copy-Burstsort is a sorting algorithm for strings that
 is cache-efficient. Burstsort and its variants are much faster than
 Quicksort and Radixsort especially on large datasets. Copy-Burstsort
 works best for sorting short strings such as genomes and words

 if the speed claim is true, and there are no other bad effects, like for
 example very bad memory use,  we could try to talk the author into
 allowing us to include it under BSD licens (currently it is GPL)

The actual implementation isn't very interesting. It's C++ code written for
Visual C++ and it's pretty primitive, has no comments, and only supports
sorting ASCII strings containing only 26 letters...

In any case we can't make use of it for sorting strings unless we want to
special-case text/varchar in the tuplesort code. That might be something to
consider at some point but right now there's still a lot to do with the
generic tuplesort that requires only a comparison operator.

On the other hand it does seem like it might be possible to adapt this
algorithm to sorting multi-column keys. 

The key to the algorithm is that it uses a trie to bin rows with common
leading prefixes together. This avoids performing redundant comparisons
between those columns later.

If you have long keys with each column in the key being relatively low
cardinality you could get some mileage out of doing something like this
algorithm does on a column-by-column basis rather than on a
character-by-character basis.

But can you see any way to adapt this to a disk-sort?

-- 
  Gregory Stark
  EnterpriseDB  http://www.enterprisedb.com


---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] Regarding Distributed Database features

2007-07-13 Thread Richard Huxton

Chidambaram Janamaddi wrote:

Hello!
 Can you please let me know whether postgreSQL has
Distributed Database supporting features?
Which of the Distributed database functionalities are totally or partially
supported among the following ..


This is the hackers list, and it's intended for discussion of 
developers of the PostgreSQL database itself.


I'd suggest asking your question on the general mailing list.

You might also want to read the online manuals to see if they cover your 
questions, since I'm not familiar with the terms you're using.



  -   Database independence


In particular, this one makes no sense to me.

--
  Richard Huxton
  Archonet Ltd

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

   http://www.postgresql.org/about/donate


[HACKERS] Regarding Distributed Database features

2007-07-13 Thread Chidambaram Janamaddi

Hello!
 Can you please let me know whether postgreSQL has
Distributed Database supporting features?
Which of the Distributed database functionalities are totally or partially
supported among the following ..

  -   Local autonomy
  -   No reliance on a central site
  -   Continuous operation
  -   Location independence
  -   Fragmentation independence
  -   Replication independence
  -   Distributed query independence
  -   Distributed transaction processing
  -   Hardware independence
  -   Operating system independence
  -   Network independence
  -   Database independence

Hope you reply to me at the earliest possible.

 Thanks and regards.


--Chidambaram


Re: [HACKERS] compiler warnings on the buildfarm

2007-07-13 Thread Zdenek Kotala

Kris Jurka wrote:



On Thu, 12 Jul 2007, Tom Lane wrote:

static int pam_passwd_conv_proc(int num_msg, const struct pam_message 
** msg,

struct pam_response ** resp, void *appdata_ptr);

which exactly matches what my Fedora 6 pam header file says it should
be.  What is it on those Solaris machines?


struct pam_conv {
int (*conv)(int, struct pam_message **,
struct pam_response **, void *);
void *appdata_ptr;  /* Application data ptr */
};

So pam_message ** isn't const.



Yes, according to X/Open XSSO Standard - see 
http://www.opengroup.org/onlinepubs/008329799/

http://www.opengroup.org/onlinepubs/008329799/pam_start.htm#tagcjh_07_32

Zdenek

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] compiler warnings on the buildfarm

2007-07-13 Thread Stefan Kaltenbrunner
Zdenek Kotala wrote:
 Stefan Kaltenbrunner wrote:
 Peter Eisentraut wrote:
 Am Donnerstag, 12. Juli 2007 15:25 schrieb Stefan Kaltenbrunner:
 a lot of those are simply noise (like the LOOP VECTORIZED stuff from
 the
 icc boxes or the statement not reached spam from the sun compilers)
 but others might indicate real issues.
 To find warnings that might be a real problem we might want to look
 into
  suppressing those - if possible -  using compiler switches.
 It would be good to determine an appropriate set of compiler switches
 to reduce the warnings to a reasonable level.

 yeah once we have determined that this whole experiment is useful it
 should be pretty easy to tweak the compiler switches for the non-gcc
 compilers (mostly icc and sun studio seem to be the ones that generate
 excessive output).

 
 For sun studio -erroff=E_STATEMENT_NOT_REACHED is useful there. If you
 want to determine warning tags for each warning add -errtags.

Is that supported on all versions of sun studio(Sun WorkShop 6, Sun
Studio 8,11) we have on the farm ?


Stefan

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [HACKERS] has anyone looked at burstsort ?

2007-07-13 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes:
 The key to the algorithm is that it uses a trie to bin rows with common
 leading prefixes together. This avoids performing redundant comparisons
 between those columns later.

Interesting, but doesn't that make it utterly useless for sorting in
non-C locales?

I'm not that thrilled with introducing datatype-specific paths into the
sort code anyway; seems like a maintenance nightmare.

regards, tom lane

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] has anyone looked at burstsort ?

2007-07-13 Thread Alvaro Herrera
Tom Lane wrote:
 Gregory Stark [EMAIL PROTECTED] writes:
  The key to the algorithm is that it uses a trie to bin rows with common
  leading prefixes together. This avoids performing redundant comparisons
  between those columns later.
 
 Interesting, but doesn't that make it utterly useless for sorting in
 non-C locales?

It seems so.  But on the other hand it might prove helpful for
multicolumn sorts (which removes the datatype-specific objection).

-- 
Alvaro Herrera   Valdivia, Chile   ICBM: S 39º 49' 18.1, W 73º 13' 56.4
No hay cielo posible sin hundir nuestras raíces
 en la profundidad de la tierra(Malucha Pinto)

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


[HACKERS] GSSAPI on Solaris - psql segfault

2007-07-13 Thread Stefan Kaltenbrunner
I just took a look at adding gssapi build support on solaris (solaris
10/x86_64, sun studio 10, 64bit build) which seemed easy enough by
educating configure to look for -lgss but while it compiles just fine
the resulting tree will not be able to complete a make check due to
psql(!) segfaulting as soon as it tries to connect to the backend.

the backtrace looks similiar to:


program terminated by signal SEGV (no mapping at the fault address)
0x7fd3d401: _memcpy+0x00e1: movq %rax,(%rdi)
Current function is pqPutMsgBytes
  475   memcpy(conn-outBuffer + conn-outMsgEnd, buf, len);
(dbx) where
  [1] _memcpy(0x4, 0x4709a0, 0x67, 0x2, 0x646c697562677000,
0x7361626174616400), at 0x7fd3d401
=[2] pqPutMsgBytes(buf = 0x4709a0, len = 103U, conn = 0x4682f0), line
475 in fe-misc.c
  [3] pqPutnchar(s = 0x4709a0 , len = 103U, conn = 0x4682f0), line 189
in fe-misc.c
  [4] pqPacketSend(conn = 0x4682f0, pack_type = '\0', buf = 0x4709a0,
buf_len = 103U), line 2439 in fe-connect.c
  [5] PQconnectPoll(conn = 0x4682f0), line 1299 in fe-connect.c
  [6] connectDBComplete(conn = 0x4682f0), line 936 in fe-connect.c
  [7] PQsetdbLogin(pghost = (nil), pgport = (nil), pgoptions = (nil),
pgtty = (nil), dbName = 0xfd7fffdffbeb postgres, login = (nil),
pwd = (nil)), line 660 in fe-connect.c
  [8] main(argc = 3, argv = 0xfd7fffdff9b8), line 211 in startup.c


ideas ?


Stefan

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] GSSAPI on Solaris - psql segfault

2007-07-13 Thread Bruce Momjian
Stefan Kaltenbrunner wrote:
 I just took a look at adding gssapi build support on solaris (solaris
 10/x86_64, sun studio 10, 64bit build) which seemed easy enough by
 educating configure to look for -lgss but while it compiles just fine
 the resulting tree will not be able to complete a make check due to
 psql(!) segfaulting as soon as it tries to connect to the backend.

Can we add the segfault to the Solaris expected regression results?  ;-)

---


 
 the backtrace looks similiar to:
 
 
 program terminated by signal SEGV (no mapping at the fault address)
 0x7fd3d401: _memcpy+0x00e1: movq %rax,(%rdi)
 Current function is pqPutMsgBytes
   475   memcpy(conn-outBuffer + conn-outMsgEnd, buf, len);
 (dbx) where
   [1] _memcpy(0x4, 0x4709a0, 0x67, 0x2, 0x646c697562677000,
 0x7361626174616400), at 0x7fd3d401
 =[2] pqPutMsgBytes(buf = 0x4709a0, len = 103U, conn = 0x4682f0), line
 475 in fe-misc.c
   [3] pqPutnchar(s = 0x4709a0 , len = 103U, conn = 0x4682f0), line 189
 in fe-misc.c
   [4] pqPacketSend(conn = 0x4682f0, pack_type = '\0', buf = 0x4709a0,
 buf_len = 103U), line 2439 in fe-connect.c
   [5] PQconnectPoll(conn = 0x4682f0), line 1299 in fe-connect.c
   [6] connectDBComplete(conn = 0x4682f0), line 936 in fe-connect.c
   [7] PQsetdbLogin(pghost = (nil), pgport = (nil), pgoptions = (nil),
 pgtty = (nil), dbName = 0xfd7fffdffbeb postgres, login = (nil),
 pwd = (nil)), line 660 in fe-connect.c
   [8] main(argc = 3, argv = 0xfd7fffdff9b8), line 211 in startup.c
 
 
 ideas ?
 
 
 Stefan
 
 ---(end of broadcast)---
 TIP 7: You can help support the PostgreSQL project by donating at
 
 http://www.postgresql.org/about/donate

-- 
  Bruce Momjian  [EMAIL PROTECTED]  http://momjian.us
  EnterpriseDB   http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] stored procedure stats in collector

2007-07-13 Thread Neil Conway
On Fri, 2007-07-13 at 14:11 +0300, Martin Pihlak wrote:
 I'm working on a patch to extend the stats collector to handle stored 
 procedure
 statistics (call counts, duration etc). The goal is to make this information
 visible via pg_stat functions/views. The collection would be controllable via
 stats_function_level GUC and will have minimal overhead when turned off. At
 our company we depend heavily on stored procedures and such a tool would be of
 great help. Perhaps others could also find it somewhat useful.

Very cool, certainly sounds like a useful feature to me.

 martinp=# select * from pg_stat_user_functions ;
   procid | schemaname | procname | nargs | calls | total_time | total_cpu | 
 self_time | self_cpu
 ++--+---+---++---+---+--
16388 | public | f1   | 0 |  4000 |  14978 |  8352 |   
   14978 | 8352
16389 | public | f2   | 0 |  2000 |  40044 |  8364 |   
   25066 |   12
16390 | public | f3   | 0 |  1000 |  40054 |  8364 |   
   9 |0
 (3 rows)

(schemaname, procname, nargs) is still ambiguous in the face of function
overloading. Although the presence of procid uniquely identifies each
function anyway, if you're going to include the name and argument
information, it might be worth including the argument types as well (as
an array of regtype, perhaps).

 Only functions with oid = FirstNormalObjectId are accounted.

Sounds reasonable to me; adding profiling to every DirectFunctionCall
invocation is likely to be too expensive anyway.

From looking quickly at the patch, I don't think the current coding
handles set-returning functions (ExecMakeTableFunctionResult).

-Neil



---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] GSSAPI on Solaris - psql segfault

2007-07-13 Thread Zdenek Kotala

Stefan Kaltenbrunner wrote:

I just took a look at adding gssapi build support on solaris (solaris
10/x86_64, sun studio 10, 64bit build) which seemed easy enough by
educating configure to look for -lgss but while it compiles just fine
the resulting tree will not be able to complete a make check due to
psql(!) segfaulting as soon as it tries to connect to the backend.
the backtrace looks similiar to:


program terminated by signal SEGV (no mapping at the fault address)
0x7fd3d401: _memcpy+0x00e1: movq %rax,(%rdi)
Current function is pqPutMsgBytes
  475   memcpy(conn-outBuffer + conn-outMsgEnd, buf, len);
(dbx) where
  [1] _memcpy(0x4, 0x4709a0, 0x67, 0x2, 0x646c697562677000,
0x7361626174616400), at 0x7fd3d401
=[2] pqPutMsgBytes(buf = 0x4709a0, len = 103U, conn = 0x4682f0), line
475 in fe-misc.c
  [3] pqPutnchar(s = 0x4709a0 , len = 103U, conn = 0x4682f0), line 189
in fe-misc.c
  [4] pqPacketSend(conn = 0x4682f0, pack_type = '\0', buf = 0x4709a0,
buf_len = 103U), line 2439 in fe-connect.c
  [5] PQconnectPoll(conn = 0x4682f0), line 1299 in fe-connect.c
  [6] connectDBComplete(conn = 0x4682f0), line 936 in fe-connect.c
  [7] PQsetdbLogin(pghost = (nil), pgport = (nil), pgoptions = (nil),
pgtty = (nil), dbName = 0xfd7fffdffbeb postgres, login = (nil),
pwd = (nil)), line 660 in fe-connect.c
  [8] main(argc = 3, argv = 0xfd7fffdff9b8), line 211 in startup.c


ideas ?


Do you use also 64bit psql? Can you send me your ./configure setup? 
However, I see there potential integer overflow, because len is size_t 
and  conn-outMsgEnd is defined as int.


 Zdenek

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

   http://www.postgresql.org/about/donate


[HACKERS] non-blocking CREATE INDEX in 8.2??

2007-07-13 Thread Jim C. Nasby
According to http://developer.postgresql.org/index.php/Feature_Matrix,
8.2 has non-blocking CREATE INDEX, which is news to me. Is it correct?

http://developer.postgresql.org/index.php?title=Feature_Matrixoldid=1734
is the version where that was added (very early in the history of the
page).
-- 
Jim Nasby  [EMAIL PROTECTED]
EnterpriseDB  http://enterprisedb.com  512.569.9461 (cell)


pgpbHQz24XMTQ.pgp
Description: PGP signature


Re: [HACKERS] non-blocking CREATE INDEX in 8.2??

2007-07-13 Thread Neil Conway
On Fri, 2007-07-13 at 15:38 -0500, Jim C. Nasby wrote:
 According to http://developer.postgresql.org/index.php/Feature_Matrix,
 8.2 has non-blocking CREATE INDEX, which is news to me. Is it correct?

http://www.postgresql.org/docs/8.2/static/sql-createindex.html

See the CONCURRENTLY clause.

-Neil



---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] non-blocking CREATE INDEX in 8.2??

2007-07-13 Thread Joshua D. Drake

Jim C. Nasby wrote:

According to http://developer.postgresql.org/index.php/Feature_Matrix,
8.2 has non-blocking CREATE INDEX, which is news to me. Is it correct?


CREATE INDEX CONCURRENTLY

:)

Joshua D. Drake



http://developer.postgresql.org/index.php?title=Feature_Matrixoldid=1734
is the version where that was added (very early in the history of the
page).



--

  === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive  PostgreSQL solutions since 1997
 http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/


---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] non-blocking CREATE INDEX in 8.2??

2007-07-13 Thread Jim C. Nasby
On Fri, Jul 13, 2007 at 01:45:18PM -0700, Joshua D. Drake wrote:
 Jim C. Nasby wrote:
 According to http://developer.postgresql.org/index.php/Feature_Matrix,
 8.2 has non-blocking CREATE INDEX, which is news to me. Is it correct?
 
 CREATE INDEX CONCURRENTLY

Well, I guess it's a good thing someone created that cheat-sheet, 'cause
I can't keep which feature went into what version straight anymore. :)
-- 
Jim Nasby  [EMAIL PROTECTED]
EnterpriseDB  http://enterprisedb.com  512.569.9461 (cell)


pgpOpzG4z9Zs9.pgp
Description: PGP signature


Re: [HACKERS] compiler warnings on the buildfarm

2007-07-13 Thread Chris Browne
[EMAIL PROTECTED] (Stefan Kaltenbrunner) writes:

 Tom Lane wrote:
 [...]
 animal: clownfish   warnings: 12
 dynloader.c, line 4: warning: empty translation unit
 postgres.c, line 3758: warning: loop not entered at top
 
 The first of these is not a bug, the second seems to be some weird
 aberration in their statement-not-reached detection.

 will see about filtering out those

 
 animal: grebe   warnings: 45
 xlog.c:651: warning: implicit declaration of function '_check_lock'
 xlog.c:654: warning: implicit declaration of function '_clear_lock'
 hba.c:1449: warning: implicit declaration of function 'getpeereid'
 
 Someone needs to find out which system headers declare these functions
 on AIX.

Hmm.  Logging onto grebe:

/usr/include/sys/socket.h:int getpeereid(int, uid_t *__restrict__, gid_t 
*__restrict__);

ydb1.int.libertyrms.com(cbbrowne): /home/cbbrowne # egrep '_(check|clear)_lock' 
/usr/include/*/*.h
/usr/include/sys/atomic_op.h:boolean_t _check_lock();
/usr/include/sys/atomic_op.h:void _clear_lock();
/usr/include/sys/atomic_op.h:void _clear_lock_mem();
/usr/include/sys/atomic_op.h:boolean_t _check_lock(atomic_p, int, int);
/usr/include/sys/atomic_op.h:void _clear_lock(atomic_p, int);
/usr/include/sys/atomic_op.h:void _clear_lock_mem(atomic_p, int);

Do those seem apropos?

 ip.c: In function 'getaddrinfo_unix':
 ip.c:254: warning: large integer implicitly truncated to unsigned type
 
 This is complaining about
 
 #ifdef HAVE_STRUCT_SOCKADDR_STORAGE_SS_LEN
  unp-sun_len = sizeof(struct sockaddr_un);
 #endif
 
 I don't know how wide sun_len is on this platform.  It's probably uint8,
 but if we explicitly cast the sizeof to 8 bits, we could conceivably
 break things on other platforms.  Are there any where sockaddr_un is
 longer than 255 bytes?  Anyway I'm inclined to leave this alone.

 no idea on AIX but I have added christopher to the CC list - maybe he
 can shed some light on those things.

/* According to RFC3493 sockaddr_storage structure should be greater than or
equal to the largest sockaddr struct. The size of sockaddr_un structure changed 
to 1025 in order to support long user names. Change _SS_MAXSIZE accordingly
inorder to main compliance to the RFC */
#define _SS_MAXSIZE 1280 /* Implementation specific max size */

Actually, you can take a look at doc/FAQ_AIX; that reports that the
size was updated to 1028 back in 2005, as a result, in fact, of my bug
submission :-).

The comment in the #include seems somewhat nonsensical; the reason for
increasing sockaddr_un was to support IPv6 addresses.  I didn't think
it had anything to do with user names...

[Aside: Sorry, I don't have any flames about EDB/CMD today.  Boy, you
miss reading -advocacy for half a day, and sometimes you miss
something big...]
-- 
output = reverse(gro.mca @ enworbbc)
http://www3.sympatico.ca/cbbrowne/linuxxian.html
One  of my most  often repeated quips was  the one I made when former
Presidents Carter, Ford and Nixon stood by each other at a White House
event. 'There they are,' I said. 'See  no evil, hear  no evil, and ...
evil.' -- Bob Dole, 1983

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] FK Deferred RI Triggers SAVEPOINTs --- ACID violation

2007-07-13 Thread Affan Salman

Patch submitted.

--
Affan Salman
EnterpriseDB Corporation  http://www.enterprisedb.com


---(end of broadcast)---
TIP 6: explain analyze is your friend