[HACKERS] ECPG thread-safety

[Lee Kindness]

Philip,

You can find the "patch so far" at:

 http://services.csl.co.uk/postgresql/

along with a libpq thread-safe patch.

The thread referenced by Bruce's email details some of the issues
still to be looked at. The main work is integration with the build
system (testing for and linking in POSIX threads) and testing.

I'm more than happy for you to look at at, I haven't had the time I
hoped to complete this.

Thanks, Lee.

Philip Yarra writes:
 > As I found out recently, ECPG is not thread-safe. This is a problem for us, as 
 > we really need to be able to use ECPG* in a multi-threaded C application. 
 > Michael mentioned that Lee is working on fixing these problems, but was not 
 > sure when it would be complete. 
 > 
 > So, my questions are: 
 > Lee/anyone, do you have an estimated completion date? 
 > If not, can we (myself plus a "real" C programmer) help you?
 > 
 > Regards, Philip Yarra.
 > 
 > * Yes, we could use libpq, but ECPG minimises our porting from Informix. 
 > Rewriting everything to use libpq would be too expensive.
 > 


---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Re: [HACKERS] DEFAULT in

[Rod Taylor]

I'll look into it...

On Mon, 2003-03-24 at 20:14, Christopher Kings-Lynne wrote:
> >  (the list of SET expressions in an UPDATE statement;
> > section 14.12) allows a  on the
> > right-hand side of SET assignments. One of the possibilities for a
> >  is DEFAULT (section 6.5).
> >
> > In other words, this syntax should be legal:
> >
> > UPDATE sometable SET somecolumn = DEFAULT;
> >
> > but it doesn't appear to be implemented as of CVS tip.
> 
> Hmmm...I ran into a situation where I wanted that just the other day.
> INSERT INTO...SELECT FROM kind of thing...
> 
> Chris
-- 
Rod Taylor <[EMAIL PROTECTED]>

PGP Key: http://www.rbt.ca/rbtpub.asc


signature.asc
Description: This is a digitally signed message part

[HACKERS] PostgreSQL and SOAP, version 7.4/8.0

[mlw]

I have been working on moving some of my software to a more SOAP 
compatible interface. As I was doing it, it occured to me that a generic 
function could be written, in PostgreSQL's new function manager that 
allows multiple columns to be returned, that is a generic SOAP interface.

All one would need do is define what is expected from the SOAP call in 
the "CREATE FUNCTION" statement. Then the generic SOAP function could 
then read what is expected and return the XML/SOAP data as a set of 
results as if it were a subquery.

What is needed is an efficient way to find the data types and names from 
the functions definition. Does anyone know how to do that?

A small program could also parse a WSDL file and write a "CREATE 
FUNCTION" script for the XML as well.

On the flip side, I am also working on a PostgreSQL SOAP interface, 
where one does this:

http://somehost/postgresql?query="select * from table"

And a SOAP compatible resultset is returned.

On a more advanced horizon, one should be able to do this:

select * from localtable, 
mysoap('http://remotehost/postgresql?query=select * from foo') as soap 
where soap.field = localtable.field;

If we can do that, PostgreSQL could fit into almost ANY service 
environment. What do you guys think? Anyone want to help out?

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly

Re: [HACKERS] PostgreSQL and SOAP, version 7.4/8.0

[Jason M. Felice]

First, a SOAP query should be posted in SOAP message format, not using the
query string as you do.  Second, I like the idea of calling external SOAP
services, but consider creating a language 'soap' you could do with a CREATE
FUNCTION type thing. e.g.

CREATE FUNCTION "foo" (TEXT) RETURNS INTEGER AS
'http://somewhere.com/path/to/.wsdl', 'foo'
LANGUAGE 'soap';

(hmm, it is unclear if this is what you are suggesting or not...)

Second, I hate SOAP because it is too bloated (have you read the spec(s)?).
If you can support xmlrpc instead, you'll save yourself a lot of headaches.
If you got SOAP working, though, I'd use it.  It's more an implementation
thing.


On Fri, Mar 28, 2003 at 09:01:08AM -0500, mlw wrote:
> I have been working on moving some of my software to a more SOAP 
> compatible interface. As I was doing it, it occured to me that a generic 
> function could be written, in PostgreSQL's new function manager that 
> allows multiple columns to be returned, that is a generic SOAP interface.
> 
> All one would need do is define what is expected from the SOAP call in 
> the "CREATE FUNCTION" statement. Then the generic SOAP function could 
> then read what is expected and return the XML/SOAP data as a set of 
> results as if it were a subquery.
> 
> What is needed is an efficient way to find the data types and names from 
> the functions definition. Does anyone know how to do that?
> 
> A small program could also parse a WSDL file and write a "CREATE 
> FUNCTION" script for the XML as well.
> 
> On the flip side, I am also working on a PostgreSQL SOAP interface, 
> where one does this:
> 
> http://somehost/postgresql?query="select * from table"
> 
> And a SOAP compatible resultset is returned.
> 
> On a more advanced horizon, one should be able to do this:
> 
> select * from localtable, 
> mysoap('http://remotehost/postgresql?query=select * from foo') as soap 
> where soap.field = localtable.field;
> 
> If we can do that, PostgreSQL could fit into almost ANY service 
> environment. What do you guys think? Anyone want to help out?
> 

I have no time to volunteer for projects, but what the hell...!  It's too
cool.  I can't spend much time on it but bounce things off me and I'll
do whatever hacking I can squeeze in.  What soap implementation would you
use for the PostgreSQL plugin?  libsoap, last I checked, is a wee bit 
out of date.  And not documented.

-Jason


---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]

Re: [HACKERS] PostgreSQL and SOAP, version 7.4/8.0

[cbbrowne]

Jason wrote:
> If you can support xmlrpc instead, you'll save yourself a lot of headaches.

XML-RPC has three merits over SOAP:

 1.  It's a simple specification, and thus readily implemented.

 2.  Microsoft and IBM aren't fighting over control over it, so it's
 not suffering from the "we keep adding pseudo-standards to it"
 problem.  (Which further complicates the specifications.)
 You can have a /complete/ implementation of XML-RPC, whereas,
 for SOAP, you can hold ghastly long arguments as to what SOAP
 means, anyways.

 3.  There's a (perhaps not "standard", but definitely widely
 implemented) scheme for bundling multiple XML-RPC requests into
 one message, which improves latency a LOT for small messages.

Of course, CORBA has actually been quite formally standardized, suffers
from many fairly interoperable implementations, and is rather a lot less
bloated than any of the XML-based schemes.  It might be worth trying,
too...
--
If this was helpful,  rate me
http://cbbrowne.com/info/soap.html
I just got skylights put in my place. The people who live above me are
furious.


---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])

Re: [HACKERS] PostgreSQL and SOAP, version 7.4/8.0

[Jason M. Felice]

On Fri, Mar 28, 2003 at 01:36:43PM -0500, [EMAIL PROTECTED] wrote:
> Of course, CORBA has actually been quite formally standardized, suffers
> from many fairly interoperable implementations, and is rather a lot less
> bloated than any of the XML-based schemes.  It might be worth trying,
> too...

The ability to use the HTTP transport has it's advantages with web services--
You can throw something together with a few lines of PHP, you don't have to
worry about how to activate objects, I've never been able to figure out how
to handle transport-layer security and authentication with CORBA (of course,
this was all fairly new stuff when I was working with it), all this stuff
comes for free with the HTTP transport.

I like CORBA, though, and I'd probably find a CORBA module useful, but it
doesn't solve all the same problems.

Hrm, I wonder if the overhead of XML-RPC wouldn't be too bad for the new
PostgreSQL protocol... it probably would, but it would be entirely useful.
You can make XML-RPC calls from mozilla javascript, so you could do some
pretty sweet tweaking to keep your addresses in a pgsql database.

As an "additional" protocol which postmaster can listen to it would rule.
I'm making a habit of putting all the business logic into stored procedures,
and this would basically publish the business logic in a very useful way.

-Jason


---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Re: [HACKERS] Detecting corrupted pages earlier

[Tom Lane]

Hiroshi Inoue <[EMAIL PROTECTED]> writes:
> How about adding a new option to skip corrupted pages ?

I have committed changes to implement checking for damaged page headers,
along the lines of last month's discussion.  It includes a GUC variable
to control the response as suggested by Hiroshi.

Given the number of data-corruption reports we've seen recently, I am
more than half tempted to commit the change into the 7.3.* branch too.
However the GUC variable makes it seem a little like a new feature.

I could backpatch the whole change, or backpatch it without the GUC
variable (so the only possible response is elog(ERROR)), or leave
well enough alone.  Any votes?

The patch is pretty small; I include the non-boilerplate parts below.

regards, tom lane


*** /home/tgl/pgsql/src/backend/storage/buffer/bufmgr.c.origTue Mar 25 09:06:11 
2003
--- /home/tgl/pgsql/src/backend/storage/buffer/bufmgr.c Fri Mar 28 11:54:48 2003
***
*** 59,64 
--- 60,69 
(*((XLogRecPtr*) MAKE_PTR((bufHdr)->data)))
  
  
+ /* GUC variable */
+ bool  zero_damaged_pages = false;
+ 
+ 
  static void WaitIO(BufferDesc *buf);
  static void StartBufferIO(BufferDesc *buf, bool forInput);
  static void TerminateBufferIO(BufferDesc *buf);
***
*** 217,222 
--- 222,241 
{
status = smgrread(DEFAULT_SMGR, reln, blockNum,
  (char *) MAKE_PTR(bufHdr->data));
+   /* check for garbage data */
+   if (status == SM_SUCCESS &&
+   !PageHeaderIsValid((PageHeader) MAKE_PTR(bufHdr->data)))
+   {
+   if (zero_damaged_pages)
+   {
+   elog(WARNING, "Invalid page header in block %u of %s; 
zeroing out page",
+blockNum, RelationGetRelationName(reln));
+   MemSet((char *) MAKE_PTR(bufHdr->data), 0, BLCKSZ);
+   }
+   else
+   elog(ERROR, "Invalid page header in block %u of %s",
+blockNum, RelationGetRelationName(reln));
+   }
}
  
if (isLocalBuf)
*** /home/tgl/pgsql/src/backend/storage/page/bufpage.c.orig Tue Mar 25 09:06:12 
2003
--- /home/tgl/pgsql/src/backend/storage/page/bufpage.c  Fri Mar 28 11:38:43 2003
***
*** 48,53 
--- 46,96 
  }
  
  
+ /*
+  * PageHeaderIsValid
+  *Check that the header fields of a page appear valid.
+  *
+  * This is called when a page has just been read in from disk.  The idea is
+  * to cheaply detect trashed pages before we go nuts following bogus item
+  * pointers, testing invalid transaction identifiers, etc.
+  *
+  * It turns out to be necessary to allow zeroed pages here too.  Even though
+  * this routine is *not* called when deliberately adding a page to a relation,
+  * there are scenarios in which a zeroed page might be found in a table.
+  * (Example: a backend extends a relation, then crashes before it can write
+  * any WAL entry about the new page.  The kernel will already have the
+  * zeroed page in the file, and it will stay that way after restart.)  So we
+  * allow zeroed pages here, and are careful that the page access macros
+  * treat such a page as empty and without free space.  Eventually, VACUUM
+  * will clean up such a page and make it usable.
+  */
+ bool
+ PageHeaderIsValid(PageHeader page)
+ {
+   char   *pagebytes;
+   int i;
+ 
+   /* Check normal case */
+   if (PageGetPageSize(page) == BLCKSZ &&
+   PageGetPageLayoutVersion(page) == PG_PAGE_LAYOUT_VERSION &&
+   page->pd_lower >= SizeOfPageHeaderData &&
+   page->pd_lower <= page->pd_upper &&
+   page->pd_upper <= page->pd_special &&
+   page->pd_special <= BLCKSZ &&
+   page->pd_special == MAXALIGN(page->pd_special))
+   return true;
+ 
+   /* Check all-zeroes case */
+   pagebytes = (char *) page;
+   for (i = 0; i < BLCKSZ; i++)
+   {
+   if (pagebytes[i] != 0)
+   return false;
+   }
+   return true;
+ }
+ 
+ 
  /* 
   *PageAddItem
   *


---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]

[HACKERS] EXPLAIN ANALYZE in comparable units

[Jason M. Felice]

I'm curious if anyone's considered adding logic to count actual disk/cache
hits to report for EXPLAIN ANALYZE so that we get a more apples-to-apples
comparison?

The other question is whether anyone has got scripts or tools or what not
for testing and getting accurate numbers for the following tuning variables:

random_page_cost
cpu_tuple_cost
cpu_index_tuple_cost
cpu_operator_cost

I've seen the code and the last one is pretty gross, but the first three
could definitely swing the optimizer, especially with the hardware I might
be getting ;-)

If noone has scripts, can anyone suggest how to implement.

-Jason


---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster

Re: [HACKERS] PostgreSQL and SOAP, version 7.4/8.0

[mlw]

Jason M. Felice wrote:

  First, a SOAP query should be posted in SOAP message format, not using the
query string as you do.  Second, I like the idea of calling external SOAP
services, but consider creating a language 'soap' you could do with a CREATE
FUNCTION type thing. e.g.

CREATE FUNCTION "foo" (TEXT) RETURNS INTEGER AS
	'http://somewhere.com/path/to/.wsdl', 'foo'
	LANGUAGE 'soap';

(hmm, it is unclear if this is what you are suggesting or not...)

Second, I hate SOAP because it is too bloated (have you read the spec(s)?).
If you can support xmlrpc instead, you'll save yourself a lot of headaches.
If you got SOAP working, though, I'd use it.  It's more an implementation
thing.


Here's the thing, yes I know there are a "lot" of alternatives to SOAP, all
with varying levels of "being better than SOAP." It still stands that a SOAP
interface would be useful for people.


  


On Fri, Mar 28, 2003 at 09:01:08AM -0500, mlw wrote:
  
  
I have been working on moving some of my software to a more SOAP 
compatible interface. As I was doing it, it occured to me that a generic 
function could be written, in PostgreSQL's new function manager that 
allows multiple columns to be returned, that is a generic SOAP interface.

All one would need do is define what is expected from the SOAP call in 
the "CREATE FUNCTION" statement. Then the generic SOAP function could 
then read what is expected and return the XML/SOAP data as a set of 
results as if it were a subquery.

What is needed is an efficient way to find the data types and names from 
the functions definition. Does anyone know how to do that?

A small program could also parse a WSDL file and write a "CREATE 
FUNCTION" script for the XML as well.

On the flip side, I am also working on a PostgreSQL SOAP interface, 
where one does this:

http://somehost/postgresql?query="select * from table"

And a SOAP compatible resultset is returned.

On a more advanced horizon, one should be able to do this:

select * from localtable, 
mysoap('http://remotehost/postgresql?query=select * from foo') as soap 
where soap.field = localtable.field;

If we can do that, PostgreSQL could fit into almost ANY service 
environment. What do you guys think? Anyone want to help out?


  
  
I have no time to volunteer for projects, but what the hell...!  It's too
cool.  I can't spend much time on it but bounce things off me and I'll
do whatever hacking I can squeeze in.  What soap implementation would you
use for the PostgreSQL plugin?  libsoap, last I checked, is a wee bit 
out of date.  And not documented.


I was thinking of using SOAP over HTTP as the protocol, and a minimalist
version at best. If the people want "more" let them add it.

I have an HTTP service class in my open source library. It would br trivial
to accept a SQL query formatted as a GET request, and then execute the query
and, using libpq, format the result as XML. It should be simple enough to
do. 

  

-Jason


---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]

  

Re: [HACKERS] PostgreSQL and SOAP, version 7.4/8.0

[Steve Wampler]

On Fri, 2003-03-28 at 14:39, mlw wrote:

> I was thinking of using SOAP over HTTP as the protocol, and a
> minimalist version at best. If the people want "more" let them add it.
> 
> I have an HTTP service class in my open source library. It would br
> trivial to accept a SQL query formatted as a GET request, and then
> execute the query and, using libpq, format the result as XML. It
> should be simple enough to do. 

It would be easy.  I've done something similar (using ODBC to
get to PostgreSQL) - but using a language none of the rest of
you are likely to be interested in (Unicon).  Works just fine,
though the implementation (deliberately, by personal preference)
avoids accepting arbitrary SQL statements from SOAP clients,
instead forcing the clients to use an RPC interface so I can
do sanity checking in the Unicon [which I know better than I know
PostgreSQL...] SOAP servers.

I, too, opted for a 'minimal-SOAP' implementation.  A 'real'
implementation boggles the mind.

-- 
Steve Wampler <[EMAIL PROTECTED]>
National Solar Observatory


---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster

Re: [HACKERS] What's a good PostgreSQL guide book?

[Michael Alan Dorman]

"Gary Hendricks" <[EMAIL PROTECTED]> writes:

> I'm thinking of buying "Practical PostgreSQL" from O'Reilly.
>
> Has anyone got any comments on this book?

As Christopher Browne pointes out, some of the information is
outdated.

My real criticism, though, would be that *far* too much of a book
about PostgreSQL is an advertisment for the authors' company's
application framework thing.  This necessarily means fewer pages for
what you're supposedly there for, PostgreSQL.

> Can anyone suggest alternatives?

I'm on a plane right at the momen , but the _PostgreSQL Developers
Reference_ (I believe that's the title) from SAMS, by Geschwinde &
someone else, is clear, comprehensive, and, on everything I've ever
checked, correct.

I would buy it again in a heartbeat.

Mike
-- 
I'd write a song called "Senorita with a Necklace of Tears" -- Paul Simon


---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]

Re: [HACKERS] updateable cursors & visibility

[Bruce Momjian]

Hiroshi Inoue wrote:
> > -Original Message-
> > From: Bruce Momjian [mailto:[EMAIL PROTECTED]
> > 
> > Hiroshi Inoue wrote:
> > > > > If the cursor is INSENSITIVE, it mustn't see the row ?
> > > > 
> > > > Right.
> > > 
> > > If so, isn't the difference between SENSITIVE and INSENSITIVE extreme ?
> > 
> > Yes.
> > 
> > > Why do you or Peter refer to ASENSITIVE little ?
> > 
> > Not sure --- ASENSITIVE seems to be "do whatever you want", which is
> > always good.
> > 
> > > And what does the following mean ? It is placed just before the 
> > sentences
> > > you quoted first.
> > > 
> > >   If a cursor is open, and the SQL-transaction in which the cursor was
> > >   opened makes a significant change to SQL-data, then whether that
> > >   change is visible through that cursor before it is closed is 
> > determined
> > >   as follows:
> > 
> > There seem to be two sections, one dealing with seeing change made by
> > the same transaction:
> 
> May I ask again ?
> Must a SENSITIVE cursor see other applications' changes made
> while the cursor is open ?

I think I see what you are saying, that the spec talks about seeing your
own transactions changes, and about seeing things after your transaction
commits, but doesn't really talk about seeing other transactions changes
while you are in the cursor's transaction.  Is that the point?

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073


---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

[HACKERS] Changing behavior of BEGIN...sleep...do something...COMMIT

[Tom Lane]

A conversation with Andrew Sullivan led me to the following idea:

We have a number of frontends that like to issue BEGIN immediately
after COMMIT; so that if the client does nothing for awhile after
finishing one transaction, the backend nonetheless sees it as being
in a transaction.  This creates problems for VACUUM since the open
transaction prevents it from removing dead tuples.

This has been known for a long time, and so far the hackers' opinion
has been that those frontends are broken and should be fixed.  But
AFAIK they are not getting fixed.  And some important frontends are
involved (I believe JDBC and DBI, and perhaps also ODBC, behave
this way).  Maybe it's time to think about a fix on the backend side.

It seems to me that it'd be fairly easy to make BEGIN cause only
a local state change in the backend; the actual transaction need not
start until the first subsequent command is received.  It's already
true that the transaction snapshot is not frozen at BEGIN time, but
only when the first DML or DDL command is received; so this would
have no impact on the client-visible semantics.  But a BEGIN-then-
sleep-for-awhile client wouldn't interfere with VACUUM anymore.

The other thing that could be thought about here is when to freeze the
value of now().  Currently now() is frozen when BEGIN is received.
We could keep doing that, but it seems to me it would make more sense
to freeze now() when the transaction snapshot is established.  In a
very real sense, the transaction snapshot defines "when the transaction
starts" --- so shouldn't now() agree?

If we did both of these things, then the negatives of doing an early
BEGIN would pretty much vanish, and we'd not need to complain that these
client libraries are broken.

Comments?

regards, tom lane


---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster

Re: [HACKERS] Changing behavior of BEGIN...sleep...do something...COMMIT

[Jon Jensen]

On Fri, 28 Mar 2003, Tom Lane wrote:

> It seems to me that it'd be fairly easy to make BEGIN cause only
> a local state change in the backend; the actual transaction need not
> start until the first subsequent command is received.
[snip]
> In a very real sense, the transaction snapshot defines "when the
> transaction starts" --- so shouldn't now() agree?
> 
> If we did both of these things, then the negatives of doing an early
> BEGIN would pretty much vanish, and we'd not need to complain that these
> client libraries are broken.
> 
> Comments?

Both ideas sound like a win to me.

Jon


---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly

[HACKERS] extract(timezone_hour) problem

[Key88 SF]

I understand that internally, Postgres converts timestamps to a GMT format 
and then redisplays them in the local timezone. cf: 
http://groups.google.com/groups?selm=Pine.LNX.4.30.0110100104320.672-10%40peter.localdomain&oe=UTF-8&output=gplain

However --> doesn't this make the extract(timezone_hour...) function 
basically useless? May as well call it "get_local_timezone_hour". How can 
one extract the timezone offset of a timestamp w/timezone (ie. get the 
intended behaviour of the extract call) ???

blackcherry=# SELECT extract(timezone_hour from TIMESTAMP WITH TIME ZONE 
'2001-02-16 20:38:40-05');
date_part
---
8
(1 row)

blackcherry=# SELECT extract(timezone_hour from TIMESTAMP WITH TIME ZONE 
'2001-02-16 20:38:40-00');
date_part
---
8
(1 row)



-Dave



_
Add photos to your e-mail with MSN 8. Get 2 months FREE*.  
http://join.msn.com/?page=features/featuredemail

---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org

Re: [HACKERS] Detecting corrupted pages earlier

[Kris Jurka]

On Fri, 28 Mar 2003, Tom Lane wrote:

> Hiroshi Inoue <[EMAIL PROTECTED]> writes:
> > How about adding a new option to skip corrupted pages ?
>
> I have committed changes to implement checking for damaged page headers,
> along the lines of last month's discussion.  It includes a GUC variable
> to control the response as suggested by Hiroshi.

Is zeroing the pages the only / best option?  Hiroshi suggested skipping
the pages as I recall.  Is there any chance of recovering data from a
trashed page manually?  If so perhaps the GUC variable should allow three
options: error, zero, and skip.

Kris Jurka


---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])

Re: [HACKERS] Detecting corrupted pages earlier

[Tom Lane]

Kris Jurka <[EMAIL PROTECTED]> writes:
> Is zeroing the pages the only / best option?

It's the only way to avoid a core dump when the system tries to process
the page.  And no, I don't want to propagate the notion that "this page
is broken" beyond the buffer manager, so testing elsewhere isn't an
acceptable answer.

Basically, one should only turn this variable on after giving up on the
possibility of getting any data out of the broken page itself.  It would
be folly to run with it turned on as a normal setting.

regards, tom lane


---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]

Re: [HACKERS] updateable cursors & visibility

[Hiroshi Inoue]

> -Original Message-
> From: Bruce Momjian [mailto:[EMAIL PROTECTED]
> 
> Hiroshi Inoue wrote:
> > > > I don't understand what you two are discussing.
> > > > What's is SENSITIVE, INSENSITIVE or ASESNSITIVE ?
> > > 
> > > In SQL99 standard, I see:
> > > 
> > >  -  If the cursor is insensitive, then significant 
> changes are not
> > > visible.
> > > 
> > >  -  If the cursor is sensitive, then significant changes are
> > > visible.
> > > 
> > >  -  If the cursor is asensitive, then the visibility 
> of significant
> > > changes is implementation-dependent.
> > 
> > While a cursor is open, another application inserted a
> > row which satisfies the condition to be contained in
> > the cursor and committed. Then
> > If the cursor is SENSITIVE, must it see the row ?
> 
> Right.
> 
> > If the cursor is INSENSITIVE, it mustn't see the row ?
> 
> Right.

If so, isn't the difference between SENSITIVE and INSENSITIVE extreme ?
Why do you or Peter refer to ASENSITIVE little ?
And what does the following mean ? It is placed just before the sentences
you quoted first.

  If a cursor is open, and the SQL-transaction in which the cursor was
  opened makes a significant change to SQL-data, then whether that
  change is visible through that cursor before it is closed is determined
  as follows:

regards,
Hiroshi Inoue 


---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])

Re: [HACKERS] updateable cursors & visibility

[Bruce Momjian]

Hiroshi Inoue wrote:
> > > If the cursor is INSENSITIVE, it mustn't see the row ?
> > 
> > Right.
> 
> If so, isn't the difference between SENSITIVE and INSENSITIVE extreme ?

Yes.

> Why do you or Peter refer to ASENSITIVE little ?

Not sure --- ASENSITIVE seems to be "do whatever you want", which is
always good.

> And what does the following mean ? It is placed just before the sentences
> you quoted first.
> 
>   If a cursor is open, and the SQL-transaction in which the cursor was
>   opened makes a significant change to SQL-data, then whether that
>   change is visible through that cursor before it is closed is determined
>   as follows:

There seem to be two sections, one dealing with seeing change made by
the same transaction:

 If a cursor is open, and the SQL-transaction in which the cursor
 was opened makes a significant change to SQL-data, then whether
 that change is visible through that cursor before it is closed is
 determined as follows:

and then another talking about holdable cursors after the cursor
transaction completes:

 If a holdable cursor is open during an SQL-transaction T and it
 is held open for a subsequent SQL-transaction, then whether any
 significant changes made to SQL-data (by T or any subsequent SQL-
 transaction in which the cursor is held open) are visible through
 that cursor in the subsequent SQL-transaction before that cursor is
 closed is determined as follows:


-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073


---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]

Re: [HACKERS] updateable cursors & visibility

[Hiroshi Inoue]

> -Original Message-
> From: Bruce Momjian [mailto:[EMAIL PROTECTED]
> 
> Hiroshi Inoue wrote:
> > > > If the cursor is INSENSITIVE, it mustn't see the row ?
> > > 
> > > Right.
> > 
> > If so, isn't the difference between SENSITIVE and INSENSITIVE extreme ?
> 
> Yes.
> 
> > Why do you or Peter refer to ASENSITIVE little ?
> 
> Not sure --- ASENSITIVE seems to be "do whatever you want", which is
> always good.
> 
> > And what does the following mean ? It is placed just before the 
> sentences
> > you quoted first.
> > 
> >   If a cursor is open, and the SQL-transaction in which the cursor was
> >   opened makes a significant change to SQL-data, then whether that
> >   change is visible through that cursor before it is closed is 
> determined
> >   as follows:
> 
> There seem to be two sections, one dealing with seeing change made by
> the same transaction:

May I ask again ?
Must a SENSITIVE cursor see other applications' changes made
while the cursor is open ?

regards,
Hiroshi Inoue 


---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

[HACKERS] new release of contrib/tsearch V2

[Oleg Bartunov]

Hi there,

we just released new version of tsearch V2. It's available for
testing form http://www.sai.msu.su/~megera/postgres/gist/

Changes:

* new function reset_tsearch() - reset all tsearch's caches on dictionary, parser 
and config. Useful for debugging.
* It's possible to get words statistics.
  select * from stat('select txtidx_field from test_txtidx') order by ndoc desc;
  For each unique word it returns:
  word, number of documents with it and total number of word occurencies in 
indexed collection.
  (getting statisctics could be very slow !)

Regards,
Oleg
_
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83


---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]

Re: [HACKERS] updateable cursors & visibility

[Bruce Momjian]

Hiroshi Inoue wrote:
> > > I don't understand what you two are discussing.
> > > What's is SENSITIVE, INSENSITIVE or ASESNSITIVE ?
> > 
> > In SQL99 standard, I see:
> > 
> >  -  If the cursor is insensitive, then significant changes are not
> > visible.
> > 
> >  -  If the cursor is sensitive, then significant changes are
> > visible.
> > 
> >  -  If the cursor is asensitive, then the visibility of significant
> > changes is implementation-dependent.
> 
> While a cursor is open, another application inserted a
> row which satisfies the condition to be contained in
> the cursor and committed. Then
> If the cursor is SENSITIVE, must it see the row ?

Right.

> If the cursor is INSENSITIVE, it mustn't see the row ?

Right.

What I don't understand is how the row would automatically appear in a
sensitive cursor.  If the cursor has an ORDER BY, and there are 100
rows, and I am on row 50, and someone inserts a row that is ordered just
before 50, is my current row now 51, and 50 is the new added row, and if
I do a MOVE -1, I see the most recently inserted row and not the row I
saw as 49 before.  Seems quite confusing.

Anyway, my idea is to have the existing cursor rows appear as updated to
the UPDATE, but to remain insensitive in the cursor itself.  We could
change that, but it would require us to somehow mark the cursor at
declare time to indicate that it will be used with WHERE CURRENT OF.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073


---(end of broadcast)---
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Re: [HACKERS] ECPG thread-safety

[Lee Kindness]

I must have spent at least a day looking at configure threads tests
the past few months. Look at the tests in Python, OpenLDAP, MySQL,
Apache 2.0, Perl, ... each and everyone of them is different!
Ouch. Way above my experience in configure.

One of the packages (sorry cannot find my notes on which one!) I
looked at down the test on a per-OS level - supplying the tread
compile and link flags used on that OS and CC to see if they
worked... Probably the simplest way to go?

L.

Bruce Momjian writes:
 > 
 > Would someone take those patches and hunt around for proper 'configure'
 > tests?  I can do the configure coding, but I don't know what tests to
 > make.
 > 
 >  http://services.csl.co.uk/postgresql/
 > 
 > 
 > ---
 > 
 > Larry Rosenman wrote:
 > > 
 > > 
 > > --On Friday, March 28, 2003 10:54:23 -0500 Bruce Momjian 
 > > <[EMAIL PROTECTED]> wrote:
 > > 
 > > >
 > > > I have the libpq patch in my mailbox that just needs configure thread
 > > > testing code.  I will get that in for 7.4.
 > > I can help with the UnixWare threads stuff.  Also, Micheal Meskes now has 
 > > an account
 > > on my UnixWare box.
 > > 
 > > The threads are native on UW, but you need to invoke the cc -Kpthread 
 > > option.
 > > 
 > > LER
 > > 
 > > >
 > > > -
 > > > --
 > > >
 > > > Lee Kindness wrote:
 > > >> Philip,
 > > >>
 > > >> You can find the "patch so far" at:
 > > >>
 > > >>  http://services.csl.co.uk/postgresql/
 > > >>
 > > >> along with a libpq thread-safe patch.
 > > >>
 > > >> The thread referenced by Bruce's email details some of the issues
 > > >> still to be looked at. The main work is integration with the build
 > > >> system (testing for and linking in POSIX threads) and testing.
 > > >>
 > > >> I'm more than happy for you to look at at, I haven't had the time I
 > > >> hoped to complete this.
 > > >>
 > > >> Thanks, Lee.
 > > >>
 > > >> Philip Yarra writes:
 > > >>  > As I found out recently, ECPG is not thread-safe. This is a problem
 > > >>  > for us, as  we really need to be able to use ECPG* in a
 > > >>  > multi-threaded C application.  Michael mentioned that Lee is working
 > > >>  > on fixing these problems, but was not  sure when it would be
 > > >>  > complete.
 > > >>  >
 > > >>  > So, my questions are:
 > > >>  > Lee/anyone, do you have an estimated completion date?
 > > >>  > If not, can we (myself plus a "real" C programmer) help you?
 > > >>  >
 > > >>  > Regards, Philip Yarra.
 > > >>  >
 > > >>  > * Yes, we could use libpq, but ECPG minimises our porting from
 > > >>  > Informix.  Rewriting everything to use libpq would be too expensive.
 > > >>  >
 > > >>
 > > >
 > > > --
 > > >   Bruce Momjian|  http://candle.pha.pa.us
 > > >   [EMAIL PROTECTED]   |  (610) 359-1001
 > > >   +  If your life is a hard drive, |  13 Roberts Road
 > > >   +  Christ can be your backup.|  Newtown Square, Pennsylvania
 > > > 19073
 > > >
 > > >
 > > > ---(end of broadcast)---
 > > > TIP 6: Have you searched our list archives?
 > > >
 > > > http://archives.postgresql.org
 > > >
 > > 
 > > 
 > > 
 > > -- 
 > > Larry Rosenman http://www.lerctr.org/~ler
 > > Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED]
 > > US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
 > > 
 > > 
 > > 
 > > 
 > 
 > -- 
 >   Bruce Momjian|  http://candle.pha.pa.us
 >   [EMAIL PROTECTED]   |  (610) 359-1001
 >   +  If your life is a hard drive, |  13 Roberts Road
 >   +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073


---(end of broadcast)---
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Re: [HACKERS] ECPG thread-safety

[Shridhar Daithankar]

On Friday 28 March 2003 21:30, you wrote:
> Would someone take those patches and hunt around for proper 'configure'
> tests?  I can do the configure coding, but I don't know what tests to
> make.
>
>   http://services.csl.co.uk/postgresql/

I also need configure tests for thread for one of my other application and 
don't know how to put them. So I have left those as additional ldflags 
options to users.

I suggest we somehow test following config to start with

1) Linuxthreads on linux
Threads are usually installed in /usr/ So pthread.h and -lpthread should work 
on most setups.

2) Native freeBSD threads
pthread.h in /usr/include and lc_r

3)linuxthreads on freeBSD, ports collection
/usr/local/include/pthreads/linuxthreads/pthread.h
/usr/local/lib/libpthread.so

For testing the library, I think it is enough to init an thread attr. like 
pthread_init_attr, that should be enough to test header and library.

I don't how we are going to let people specify thread implementation where 
there are multiple thread libraries are available .e.g. freeBSD.

HTH

 Shridhar


---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])

Re: [HACKERS] ECPG thread-safety

[Shridhar Daithankar]

On Friday 28 March 2003 21:45, you wrote:
> For testing the library, I think it is enough to init an thread attr. like
> pthread_init_attr, that should be enough to test header and library.

And compiler flags, I forgot.. -pthread on freeBSD/gcc and -D_REENTRANT on 
linux at least. FreeBSD needs _THREAD_SAFE as well IIRC.

 Shridhar


---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])

Re: [HACKERS] ECPG thread-safety

[Bruce Momjian]

Yep.  I need someone who knows these OS's to get some tests together.

---

Shridhar Daithankar wrote:
> On Friday 28 March 2003 21:45, you wrote:
> > For testing the library, I think it is enough to init an thread attr. like
> > pthread_init_attr, that should be enough to test header and library.
> 
> And compiler flags, I forgot.. -pthread on freeBSD/gcc and -D_REENTRANT on 
> linux at least. FreeBSD needs _THREAD_SAFE as well IIRC.
> 
>  Shridhar
> 
> 
> ---(end of broadcast)---
> TIP 2: you can get off all lists at once with the unregister command
> (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
> 

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073


---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html