Re: [HACKERS] Assertion failure on hot standby

2010-11-25 Thread Fujii Masao
On Wed, Nov 24, 2010 at 1:27 PM, Fujii Masao masao.fu...@gmail.com wrote:
 Hi,

 http://archives.postgresql.org/pgsql-hackers/2010-11/msg01303.php

 When I did unusual operations (e.g., suspend bgwriter by gdb,
 pgbench -i and issue txid_current many times) on the master
 in order to try to reproduce the above HS error, I encountered
 the following assertion error.

 Since I compiled the standby postgres with WAL_DEBUG and
 ran it with wal_debug = on, all the replayed WAL records were
 logged.

 
 sby LOG:  REDO @ 0/134C0490; LSN 0/134C04D0: prev 0/134C0450; xid
 23253; len 32: Transaction - commit: 2010-11-24 12:15:02.315634+09
 sby LOG:  REDO @ 0/134C04D0; LSN 0/134C0510: prev 0/134C0490; xid
 23254; len 32: Transaction - commit: 2010-11-24 12:15:02.325252+09
 sby LOG:  consistent recovery state reached at 0/134C0510
 sby LOG:  REDO @ 0/134C0510; LSN 0/134C0550: prev 0/134C04D0; xid
 23255; len 32: Transaction - commit: 2010-11-24 12:15:09.224343+09
 sby LOG:  REDO @ 0/134C0550; LSN 0/134C0580: prev 0/134C0510; xid 0;
 len 16: Standby - AccessExclusive locks: xid 0 db 11910 rel 16409
 sby LOG:  REDO @ 0/134C0580; LSN 0/134C05B8: prev 0/134C0550; xid 0;
 len 20: Standby -  running xacts: nextXid 23256 latestCompletedXid
 23255 oldestRunningXid 23256
 TRAP: FailedAssertion(!(((xid) != ((TransactionId) 0))), File:
 twophase.c, Line: 1209)
 sby LOG:  database system is ready to accept read only connections
 sby LOG:  startup process (PID 32666) was terminated by signal 6: Aborted
 sby LOG:  terminating any other active server processes
 

 Does anyone know what the cause of the problem is?

I was able to reproduce this problem. This happens because CHECKPOINT
can write the WAL record indicating that the transaction with XID = 0 has taken
the AccessExclusive lock. This WAL record causes that assertion failure in the
standby.

Here is the procedure to reproduce the problem:

---
1. Execute DROP TABLE and suspend the execution before calling

 RemoveRelations - LockRelationOid - LockAcquire -
LockAcquireExtended - LogAccessExclusiveLock

 by, for example, using gdb.

2. While DROP TABLE is being suspended, execute CHECKPOINT.
 This CHECKPOINT will generate the above-mentioned WAL record.
---

To solve the problem, ISTM that XID should be assigned before the information
about AccessExclusive lock becomes visible to another process. Or CHECKPOINT
(i.e., GetRunningTransactionLocks) should ignore the locks with XID = 0.

Thought?

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] SQL/MED - file_fdw

2010-11-25 Thread Shigeru HANADA
Hi, hackers,

Attached is a patch that adds file_fdw, FDW which reads records from
files on the server side, as a contrib module.  This patch is based on
SQL/MED core functionality patch.

[SQL/MED - core functionality]
http://archives.postgresql.org/pgsql-hackers/2010-11/msg01698.php

File_fdw can be installed with the steps similar to other contrib
modules, and you can create FDW with the script:
$SHAREDIR/contrib/file_fdw.sql
Note that you need to create file_fdw for each database.

Document for file_fdw is included in the patch, although the contents
might not be enough.

Any comments and questions are welcome.

Regards,
--
Shigeru Hanada


file_fdw.patch.gz
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] SQL/MED - postgresql_fdw

2010-11-25 Thread Shigeru HANADA
Hi, hackers,

Attached is a patch that adds postgresql_fdw, FDW which retrieves
tuples from external PostgreSQL server, as a contrib module.  This
patch is based on SQL/MED core functionality patch.

[SQL/MED - core functionality]
http://archives.postgresql.org/pgsql-hackers/2010-11/msg01698.php

Postgresql_fdw can be installed with the steps similar to other
contrib modules, and you can create FDW with the script:
$SHAREDIR/contrib/postgresql_fdw.sql
Note that you need to create postgresql_fdw for each database.

Document for postgresql_fdw is included in the patch, although the
contents might not be enough.

Any comments and questions are welcome.

Regards,
--
Shigeru Hanada


pgsql_fdw.patch.gz
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [COMMITTERS] pgsql: Remove useless whitespace at end of lines

2010-11-25 Thread Magnus Hagander
On Wed, Nov 24, 2010 at 23:54, Tom Lane t...@sss.pgh.pa.us wrote:
 Magnus Hagander mag...@hagander.net writes:
 On Wed, Nov 24, 2010 at 23:45, Tom Lane t...@sss.pgh.pa.us wrote:
 Alvaro Herrera alvhe...@commandprompt.com writes:
 This was stuck in the moderation queue because of message size limit (30
 kB).  Is it worth increasing that value?

 Evidently we should.  pgindent and copyright-update commits are likely
 to be at least this long.

 That's twice a year only - I don't see a big problem moderating those
 when it happens...

 Its not so much the moderation load, as I don't like being blindsided by
 commits that touch everything in sight.  Finding out only when you try
 to do git push (as indeed happened to me just this afternoon because of
 this patch) is annoying.

fair enough. I'm not sure the size limit actually protects us against
anything, so I'm not against increasing it.


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SQL/MED - file_fdw

2010-11-25 Thread Shigeru HANADA
On Thu, 25 Nov 2010 17:12:44 +0900
Shigeru HANADA han...@metrosystems.co.jp wrote:
 Attached is a patch that adds file_fdw, FDW which reads records from
 files on the server side, as a contrib module.  This patch is based on
 SQL/MED core functionality patch.
 
 [SQL/MED - core functionality]
 http://archives.postgresql.org/pgsql-hackers/2010-11/msg01698.php

I'm going to add new CommitFest items for this patch and SQL/MED -
postgresql_fdw patch which have been split from SQL/MED patch.  Can
I add them to CF 2010-11 which original SQL/MED item is in?  Or
should I add them to CF 2011-01?

Regards,
--
Shigeru Hanada



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Horizontal Write Scaling

2010-11-25 Thread Markus Wanner
Eliot,

On 11/23/2010 09:43 PM, Eliot Gable wrote:
 I know there has been a lot of talk about replication getting built into
 Postgres and I know of many projects that aim to fill the role. However,
 I have not seen much in the way of a serious attempt at multi-master
 write scaling.

Postgres-XC and Postgres-R are two pretty serious projects, IMO.

 I understand the fundamental problem with write scaling
 across multiple nodes is Disk I/O and inter-node communication latency
 and that in the conventional synchronous, multi-master replication type
 setup you would be limited to the speed of the slowest node,

That's not necessarily true for Postgres-R, which is why I call it an
'eager' solution (as opposed to fully synchronous). While it guarantees
that all transactions that got committed *will* be committable on all
nodes at some time in the future, nodes may still lag behind others.

Thus, even a slower / busy node doesn't hold back the others, but may
serve stale data. Ideally, your load balancer accounts for that and
gives that node a break or at least reduces the amount of transactions
going to that node, so it can catch up again.

Anyway, that's pretty Postgres-R specific.

 plus the
 communication protocol overhead and latency. However, it occurs to me
 that if you had a shared disk system via either iSCSI, Fiber Channel,
 NFS, or whatever (which also had higher I/O capabilities than a single
 server could utilize), if you used a file system that supported locks on
 a particular section (extent) of a file, it should theoretically be
 possible for multiple Postgres instances on multiple systems sharing the
 database to read and write to the database without causing corruption.

Possible, yes. Worthwile to do, probably not.

 Has anyone put any thought into what it would take to do this in
 Postgres? Is it simply a matter of making the database file interaction
 code aware of extent locking, or is it considerably more involved than
 that? It also occurs to me that you probably need some form of
 transaction ordering mechanism across the nodes based on synchronized
 timestamps, but it seems Postgres-R has the required code to do that
 portion already written.

If you rely on such an ordering, why use additional locks. That seems
like a waste of resources compared to Postgres-R. Not to mention the
introduction of a SPOF with the SAN.

 Wouldn't this type of setup be far
 simpler to implement

That's certainly debatable, yes. I obviously think that the benefit per
cost ratio for Postgres-R is better :-)

 and provide better scalability than trying to do
 multi-master replication using log shipping or binary object shipping or
 any other techniques?

It's more similar to replication using two phase commit, which provably
doesn't scale (see for example [1]) And using a SAN for locking
certainly doesn't beat 2PC via an equally modern/expensive interconnect.

 Wouldn't it also be far more efficient since you
 don't need to have a copy of your data on each master node and therefor
 also don't have to ship your data to each node and have each node
 process it?

You have to ship it from the SAN to the node, so I definitely don't
think so, but see this as an argument against it. Each having a local
copy and only exchange locking information and transactional changes
sounds like much less traffic overall.

Regards

Markus Wanner


[1]: The Dangers of Replication and a Solution, Gray et al, In Proc. of
the SIGMOD Conf., 1996,
http://research.microsoft.com/apps/pubs/default.aspx?id=68247

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Horizontal Write Scaling

2010-11-25 Thread Koichi Suzuki
Hi,

2010/11/25 Markus Wanner mar...@bluegap.ch:
 Eliot,

 On 11/23/2010 09:43 PM, Eliot Gable wrote:
 I know there has been a lot of talk about replication getting built into
 Postgres and I know of many projects that aim to fill the role. However,
 I have not seen much in the way of a serious attempt at multi-master
 write scaling.

 Postgres-XC and Postgres-R are two pretty serious projects, IMO.

Yes.  Please visit http://postgres-xc.sourceforge.net/ for details.

 I understand the fundamental problem with write scaling
 across multiple nodes is Disk I/O and inter-node communication latency
 and that in the conventional synchronous, multi-master replication type
 setup you would be limited to the speed of the slowest node,

 That's not necessarily true for Postgres-R, which is why I call it an
 'eager' solution (as opposed to fully synchronous). While it guarantees
 that all transactions that got committed *will* be committable on all
 nodes at some time in the future, nodes may still lag behind others.

 Thus, even a slower / busy node doesn't hold back the others, but may
 serve stale data. Ideally, your load balancer accounts for that and
 gives that node a break or at least reduces the amount of transactions
 going to that node, so it can catch up again.

 Anyway, that's pretty Postgres-R specific.

Right.   In the case of Postgres-XC, tables can be partitioned (we
call distributed) among cluster nodes so that writing can be done in
parallel.


 plus the
 communication protocol overhead and latency. However, it occurs to me
 that if you had a shared disk system via either iSCSI, Fiber Channel,
 NFS, or whatever (which also had higher I/O capabilities than a single
 server could utilize), if you used a file system that supported locks on
 a particular section (extent) of a file, it should theoretically be
 possible for multiple Postgres instances on multiple systems sharing the
 database to read and write to the database without causing corruption.

 Possible, yes. Worthwile to do, probably not.

We may be suffered from synchronizing cache on each database.


 Has anyone put any thought into what it would take to do this in
 Postgres? Is it simply a matter of making the database file interaction
 code aware of extent locking, or is it considerably more involved than
 that? It also occurs to me that you probably need some form of
 transaction ordering mechanism across the nodes based on synchronized
 timestamps, but it seems Postgres-R has the required code to do that
 portion already written.

 If you rely on such an ordering, why use additional locks. That seems
 like a waste of resources compared to Postgres-R. Not to mention the
 introduction of a SPOF with the SAN.

 Wouldn't this type of setup be far
 simpler to implement

 That's certainly debatable, yes. I obviously think that the benefit per
 cost ratio for Postgres-R is better :-)

 and provide better scalability than trying to do
 multi-master replication using log shipping or binary object shipping or
 any other techniques?

Postgres-XC uses combination of replicated table and distributed
(partitioned) table, not just simple replication.


 It's more similar to replication using two phase commit, which provably
 doesn't scale (see for example [1]) And using a SAN for locking
 certainly doesn't beat 2PC via an equally modern/expensive interconnect.

 Wouldn't it also be far more efficient since you
 don't need to have a copy of your data on each master node and therefor
 also don't have to ship your data to each node and have each node
 process it?

 You have to ship it from the SAN to the node, so I definitely don't
 think so, but see this as an argument against it. Each having a local
 copy and only exchange locking information and transactional changes
 sounds like much less traffic overall.

 Regards

 Markus Wanner


 [1]: The Dangers of Replication and a Solution, Gray et al, In Proc. of
 the SIGMOD Conf., 1996,
 http://research.microsoft.com/apps/pubs/default.aspx?id=68247

 --
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers


Cheers;
---
Koichi Suzuki

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-25 Thread Maciek Sakrejda
Haven't really gotten much further, but an interesting note: the named
/ unnamed prepared statement and portal stuff seems to be a red
herring. I can add a name to the portal, or move to an unnamed
prepared statement, and I still see the same thing. Which is
interesting, since that's not what Radosław saw (his original test
only failed once named prepared statements kicked in)...
---
Maciek Sakrejda | System Architect | Truviso

1065 E. Hillsdale Blvd., Suite 215
Foster City, CA 94404
(650) 242-3500 Main
www.truviso.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] function(contants) evaluated for every row

2010-11-25 Thread pasman pasmański
I've seen this as well be a performance issue, in particular with partitioned 
tables.
Out of habit I now write functions that always cache the value of the function 
in
a variable and use the variable in the actual query to avoid this particular 
gotcha.

subquery may be used to cache constants:

SELECT a_date
  FROM test
  WHERE a_date=(SELECT current_date);


Seq Scan on test1  (cost=0.01..10.76 rows=5 width=4)
  Filter: (a_date = $0)
  InitPlan 1 (returns $0)
-  Result  (cost=0.00..0.01 rows=1 width=0)



pasman

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] dblink versus long connection strings

2010-11-25 Thread Itagaki Takahiro
On Tue, Nov 23, 2010 at 02:21, Tom Lane t...@sss.pgh.pa.us wrote:
 In any case I don't see an argument why warning on connection creation
 isn't sufficient.

I'll check all versions of dblink.  truncate_identifier() will be called
with warn=false in all cases except dblink_coneect() - createNewConnection().

-- 
Itagaki Takahiro

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-25 Thread Radosław Smogura
I checked against other parameter bindings and it looks like problem is
connected with oid=0.
In those cases: 
1. Executing statement with parameter sent as varchar, int, long, with
text and binary format is ok.
2. Executing statement with oid=0 fail always; I've sent parameter in text
mode (encoded '2'), and in binary mode encoded int4 2 - both failed.

On Thu, 25 Nov 2010 01:56:02 -0800, Maciek Sakrejda
msakre...@truviso.com
wrote:
 Haven't really gotten much further, but an interesting note: the named
 / unnamed prepared statement and portal stuff seems to be a red
 herring. I can add a name to the portal, or move to an unnamed
 prepared statement, and I still see the same thing. Which is
 interesting, since that's not what Radosław saw (his original test
 only failed once named prepared statements kicked in)...
 ---
 Maciek Sakrejda | System Architect | Truviso
 
 1065 E. Hillsdale Blvd., Suite 215
 Foster City, CA 94404
 (650) 242-3500 Main
 www.truviso.com

-- 
--
Radosław Smogura
http://www.softperience.eu

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [COMMITTERS] pgsql: Remove useless whitespace at end of lines

2010-11-25 Thread Alvaro Herrera
Excerpts from Magnus Hagander's message of jue nov 25 05:46:49 -0300 2010:
 On Wed, Nov 24, 2010 at 23:54, Tom Lane t...@sss.pgh.pa.us wrote:

  Its not so much the moderation load, as I don't like being blindsided by
  commits that touch everything in sight.  Finding out only when you try
  to do git push (as indeed happened to me just this afternoon because of
  this patch) is annoying.
 
 fair enough. I'm not sure the size limit actually protects us against
 anything, so I'm not against increasing it.

So, given that this commit touched 517 files and resulted in a 47341b
message, the minimum size we should use is about 332kB for the 3633
files present in the source tree (I counted all the files, not just the
ones with copyright lines, out of pure laziness).

I've set the limit to 400 kB to have a wide safety margin.  Large commits
should now pass through unmoderated.

-- 
Álvaro Herrera alvhe...@commandprompt.com
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SQL/MED - core functionality

2010-11-25 Thread Heikki Linnakangas

On 25.11.2010 09:34, Shigeru HANADA wrote:

Attached is a patch that adds core functionality of SQL/MED.  This
patch provides:

* new option HANDLER for FOREIGN DATA WRAPPER
   * CREATE/ALTER DDLs are supported
   * psql \dew command shows handler option too
   * pg_dump can dump HANDLER option

* new object type FOREIGN TABLE
   * CREATE/ALTER/DROP DDLs are supported
   * system columns except TABLEOID are not supported
   * inheriting normal table is supported
   * psql \d shows detail of foreign tables
   * psql \det lists foreign tables
   * psql \dE lists foreign tables in \d format
   * pg_dump can dump the definition
   * information_schema views added
   * foreign table is read-only, so INSERT/UPDATE/DELETE are denied
   * ANALYZE and VACUUM skips foreign tables

* new executor node ForeignScan
   * it's a counterpart of SeqScan
   * this node scans one foreign table at a time
   * FDW HANDLER is necessary to execute SELECT statement

Patches for FDWs which can be used to execute SELECT statement will be
posted in their own thread soon.

 SQL/MED - file_fdw   : FDW for external PostgreSQL
 SEL/MED - postgresql_fdw : FDW for server-side file (CSV, TEXT)

I would reuse existing CommitFest item SQL/MED for this patch, and
add new item for each FDW patch.


Looking at the API again, there's a few things I don't like about it:

* It's tied to the ForeignScanState, so all the executor state 
structures are exposed to the FDW implementation. It feels like a 
modularity violation that the FDW Iterate function returns the tuple by 
storing it directly in scanstate-ss.ss_ScanTupleSlot for example. And 
it's not going to work for remote scans that don't go through the 
executor, for example if you wanted to rewrite contrib/dblink to use 
foreign data wrappers. Or the SQL/MED passthrough mode.


* There's no clear Plan stage in the API. Except for EstimateCosts, 
which just fills in the estimated costs in RelOptInfo, so it needs to 
understand quite a lot of the planner data structures to come up with a 
reasonable estimate. But if it e.g wants to apply a qual remotely, like 
the PostgreSQL FDW does, it has to check for such quals at execution 
time. And as I complained before, you don't get any meaningful EXPLAIN 
output.


I propose the attached API instead. This has a clear separation between 
plan and execution. I'm sure we'll have to extend the API in the future 
FDWs want tighter integration, but I think this is a good start. It 
makes it quite straightforward to write simple FDW like the file FDW, 
without having to know anything about the executor or planner internals, 
but provides enough flexibility to cover the functionality in your 
PostgreSQL FDW.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com
/*-
 *
 * fdwapi.h
 *	  API for foreign-data wrappers
 *
 * Portions Copyright (c) 1996-2010, PostgreSQL Global Development Group
 *
 * src/include/foreign/fdwapi.h
 *
 *-
 */
#ifndef FDWAPI_H
#define FDWAPI_H

#include nodes/pg_list.h
#include nodes/relation.h

typedef struct
{
	TupleDesc *tupdesc;

	/*
	 * Free-form text shown in EXPLAIN. The SQL to be sent to the remote
	 * server is typically shown here.
	 */
	char *explainInfo;

	/*
	 * Cost estimation info. The startup_cost should include the cost of
	 * connecting to the remote host and sending over the query, as well as
	 * the cost of starting up the query so that it returns the first result
	 * row.
	 */
	double startup_cost;
	double total_cost;
	double rows;
	double width;

	/* FDW-private data */
	void *private;
} FdwPlan;

typedef struct
{
	/* FDW-private data */
	void *private;
} FdwExecutionState;

/*
 * Common interface routines of FDW, inspired by the FDW API in the SQL/MED
 * standard, but adapted to the PostgreSQL world.
 *
 * A foreign-data wrapper implements these routines. At a minimum, it must
 * implement BeginScan, Iterate and EndScan, and either PlanNative or
 * PlanRelScan.
 *
 * The Plan* functions return an FdwPlan struct that can later be executed
 * with BeginScan. The implementation should fill in the cost estimates in
 * FdwPlan, as well as a tuple descriptor that describes the result set.
 */
struct FdwRoutine
{
#ifdef IN_THE_FUTURE
	/*
	 * Plan a query of arbitrary native SQL (or other query language supported
	 * by the foreign server). This is used for SQL/MED passthrough mode, or
	 * e.g contrib/dblink.
	 */
	FdwPlan (*PlanNative)(Oid serverid, char *query);

	/*
	 * Plan a whole subquery. This is used for example to execute an aggregate
	 * query remotely without pulling all the rows to the local server.
	 *
	 * The implementation can return NULL if it cannot satisfy the whole
	 * subquery, in which case the planner will break down the query into
	 * smaller parts and call PlanRelScan for the foreign tables involved.
	 *
	 * The 

Re: [HACKERS] libpq changes for synchronous replication

2010-11-25 Thread Fujii Masao
On Sat, Nov 20, 2010 at 2:04 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 Alvaro Herrera alvhe...@commandprompt.com writes:
 Excerpts from Tom Lane's message of vie nov 19 12:25:13 -0300 2010:
 Yeah.  You're adding a new fundamental state to the protocol; it's not
 enough to bury that in the description of a message format.  I don't
 think a whole lot of new verbiage is needed, but the COPY section needs
 to point out that this is a different state that allows both send and
 receive, and explain what the conditions are for getting into and out of
 that state.

 Is it sane that the new message has so specific a name?

 Yeah, it might be better to call it something generic like CopyBoth.

Thanks for the review!

The attached patch s/CopyXLog/CopyBoth/g and adds the description
about CopyBoth into the COPY section.

While modifying the code, it occurred to me that we might have to add new
ExecStatusType like PGRES_COPY_BOTH and use that for CopyBoth mode,
for the sake of consistency. But since it's just alias of PGRES_COPY_BOTH
for now, i.e., there is no specific behavior for that ExecStatusType, I don't
think that it's worth adding that yet.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


libpqrcv_send_v3.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] reporting reason for certain locks

2010-11-25 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mar nov 23 00:08:54 -0300 2010:

 How about publishing additional details to pg_stat_activity via
 pgstat_report_waiting()?

I'm not sure what you mean here.  Are you suggesting we should create a
new function with that name to report the reason for the lock?

If so, that's great -- but we'd be getting a bit ahead of ourselves.
Because while we do have the locking details available some of the time,
it's not easy to figure out what they are; and the rest of the time, it
just isn't available.

So I'm proposing a mechanism for this information to be available in the
first place.  When this is done we can talk about some nice user
interface for it.

-- 
Álvaro Herrera alvhe...@commandprompt.com
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SQL/MED - core functionality

2010-11-25 Thread Itagaki Takahiro
On Thu, Nov 25, 2010 at 22:03, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
 I propose the attached API instead. This has a clear separation between plan
 and execution.

The APIs seem to be cleaner. The previous ones might be too straight
implementation of the SQL standard.

But I have some questions about the new APIs:
  1. Doesn't FdwPlan need to inherit Plan struct?
  2. Doesn't FdwPlan need to support copyObject()?
  3. If Datum *values, bool *isnulls is the better interface,
 why do we use TupleTableSlot? We might have the similar issue
 in the index-only scan; it also handles virtual tuples.

-- 
Itagaki Takahiro

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] reporting reason for certain locks

2010-11-25 Thread Alvaro Herrera
Excerpts from Tom Lane's message of lun nov 22 20:51:09 -0300 2010:
 Alvaro Herrera alvhe...@alvh.no-ip.org writes:
  A much more common ocurrence is tuple locks.  We block in an Xid in that
  case; and this has been a frequent question in the mailing lists and
  IRC.
 
  I think it would be very nice to be able to report something to the
  user; however, I'm not seeing the mechanism.
 
 At least for tuple locks, the information is already visible, because we
 have a real lock on the target tuple before we try to lock the current
 holder's VXID.  So I think this isn't so much a question of needing more
 low-level mechanism as one of providing a more useful view --- some kind
 of self-join on pg_locks is needed.

Hmm, that's true, but it seems ugly: if we are blocking on a
transactionid, then go back to pg_locks and extract a lock of type
tuple; if it's there, you know you're waiting for that; if it's not,
you have to guess that you're waiting on something else (what?).
(Right now, it seems the only other thing that could wait is CREATE
INDEX CONCURRENTLY, but I don't want to bet that we're not going to
create something else in the future.  There's no way to figure out
what's happening from pg_locks, in any case.)

So what I want is something a bit more trustworthy than that.

On the other hand, pg_locks is already rather unwieldy to use.  We
already have a self-join that tells us the details of what's locking
processes: you need to join pg_locks like this:

FROM
pg_catalog.pg_locks l1
JOIN
pg_catalog.pg_locks l2 ON (
(
l1.locktype, l1.database, l1.relation, l1.page,
l1.tuple, l1.virtualxid, l1.transactionid, l1.classid,
l1.objid, l1.objsubid
)
IS NOT DISTINCT FROM
(
l2.locktype, l2.database, l2.relation, l2.page,
l2.tuple, l2.virtualxid, l2.transactionid, l2.classid,
l2.objid, l2.objsubid
)
)

and throw in a bunch of left joins to see the details of database,
relation, etc.  This works fine for all kinds of locks except xid and
vxid ones.  I don't think it's fair to users to expect that they need to 
deal with that mess *plus* the details of tuple locks.

-- 
Álvaro Herrera alvhe...@commandprompt.com
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SQL/MED - core functionality

2010-11-25 Thread Heikki Linnakangas

On 25.11.2010 16:16, Itagaki Takahiro wrote:

On Thu, Nov 25, 2010 at 22:03, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com  wrote:

I propose the attached API instead. This has a clear separation between plan
and execution.


The APIs seem to be cleaner. The previous ones might be too straight
implementation of the SQL standard.

But I have some questions about the new APIs:
   1. Doesn't FdwPlan need to inherit Plan struct?
   2. Doesn't FdwPlan need to support copyObject()?


No. You'll need a ForeignScan object in the planner that supports 
copyObject(), just like in your patch. ForeignScan points to the 
FdwPlan, but the FDW doesn't need to know anything about that stuff.


I left out some details on what exactly FdwPlan should contain and what 
it's lifecycle should be. I'm thinking that it should be allocated in 
the CurrentMemoryContext that's active when the FDW Plan routine is 
called, which would be the same context where we store all the Plan 
objects. It should not be modified after creation, so that it doesn't 
need to be copied when the ForeignScan is copied with copyObject(). It 
should not contain transient state information like connection objects, 
or references to a remotely prepared cursor etc. It must be possible to 
call BeginScan multiple times with the same FdwPlan object, so that it 
can be stored in a prepared plan that is executed multiple times.


For a typical case like the PostgreSQL FDW, it would contain the foreign 
server's OID, and the constructed SQL query that will be sent to the 
remote server on execution. For the file FDW, it will probably contain 
the filename, and the format options in some pre-parsed format.



   3. If Datum *values, bool *isnulls is the better interface,
  why do we use TupleTableSlot?


I'm not wedded to that part, but in general, the less the FDW needs to 
know about PostgreSQL internals the better. There's performance gain 
from passing a TupleTableSlot to the FDW, but the ForeignScan node will 
certainly store the datums/isnulls array to a TupleTableSlot to pass on 
the tuple.



We might have the similar issue
  in the index-only scan; it also handles virtual tuples.


Index-only scans are a very different story, that's going to be tightly 
internal to the planner and executor, there's no externally-visible API 
there.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] reporting reason for certain locks

2010-11-25 Thread Robert Haas
On Thu, Nov 25, 2010 at 9:00 AM, Alvaro Herrera
alvhe...@commandprompt.com wrote:
 Excerpts from Robert Haas's message of mar nov 23 00:08:54 -0300 2010:

 How about publishing additional details to pg_stat_activity via
 pgstat_report_waiting()?

 I'm not sure what you mean here.  Are you suggesting we should create a
 new function with that name to report the reason for the lock?

No, what I was suggesting was taking the existing function:

extern void pgstat_report_waiting(bool waiting);

...and instead doing something like this:

extern void pgstat_report_waiting(char *reason);

...and then arrange to pass the reason via the eponymous argument.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] reporting reason for certain locks

2010-11-25 Thread Alvaro Herrera
Excerpts from Robert Haas's message of jue nov 25 11:56:27 -0300 2010:

 No, what I was suggesting was taking the existing function:
 
 extern void pgstat_report_waiting(bool waiting);
 
 ...and instead doing something like this:
 
 extern void pgstat_report_waiting(char *reason);
 
 ...and then arrange to pass the reason via the eponymous argument.

Ah, I was looking for a SQL-callable function :-)  Hmm, maybe this would
work.

-- 
Álvaro Herrera alvhe...@commandprompt.com
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Horizontal Write Scaling

2010-11-25 Thread Robert Haas
On Thu, Nov 25, 2010 at 4:45 AM, Koichi Suzuki koichi@gmail.com wrote:
 plus the
 communication protocol overhead and latency. However, it occurs to me
 that if you had a shared disk system via either iSCSI, Fiber Channel,
 NFS, or whatever (which also had higher I/O capabilities than a single
 server could utilize), if you used a file system that supported locks on
 a particular section (extent) of a file, it should theoretically be
 possible for multiple Postgres instances on multiple systems sharing the
 database to read and write to the database without causing corruption.

 Possible, yes. Worthwile to do, probably not.

 We may be suffered from synchronizing cache on each database.

That's putting it mildly.  You have to worry about the database buffer
cache, the shared invalidation queue, the control file data, the
ProcArray, the lock manager, the LWLock tables, and probably some
other things I'm forgetting about.  Everything in shared memory, in
short.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] reporting reason for certain locks

2010-11-25 Thread Robert Haas
On Thu, Nov 25, 2010 at 10:05 AM, Alvaro Herrera
alvhe...@commandprompt.com wrote:
 Excerpts from Robert Haas's message of jue nov 25 11:56:27 -0300 2010:

 No, what I was suggesting was taking the existing function:

 extern void pgstat_report_waiting(bool waiting);

 ...and instead doing something like this:

 extern void pgstat_report_waiting(char *reason);

 ...and then arrange to pass the reason via the eponymous argument.

 Ah, I was looking for a SQL-callable function :-)  Hmm, maybe this would
 work.

I was just thinking it might be a good place for it because we're
already doing some nontrivial work at that point anyway.  Although,
lock acquisition (especially of AccessShareLocks) is already a bit of
a hotspot, so we definitely have to be careful what we put in there no
matter how we do it.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] problem with Win32 buildfarm

2010-11-25 Thread Bruce Momjian
Win32 buildfarm members are red because of my inet_pton changes.  I will
look into this in the next day, and also improve how we include C files
from /port for libpq.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] reporting reason for certain locks

2010-11-25 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes:
 On the other hand, pg_locks is already rather unwieldy to use.  We
 already have a self-join that tells us the details of what's locking
 processes: you need to join pg_locks like this:
 ...
 and throw in a bunch of left joins to see the details of database,
 relation, etc.

Sure.  I'm just suggesting one more left join to see if there's a tuple
lock.

 This works fine for all kinds of locks except xid and
 vxid ones.  I don't think it's fair to users to expect that they need to 
 deal with that mess *plus* the details of tuple locks.

Well, what was in the back of my mind was that we should create a join
of this sort as a stock system view, which would certainly improve
usability across the board.  Getting to consensus on exactly what the
view should contain might be hard though.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SQL/MED - core functionality

2010-11-25 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
 I left out some details on what exactly FdwPlan should contain and what 
 it's lifecycle should be. I'm thinking that it should be allocated in 
 the CurrentMemoryContext that's active when the FDW Plan routine is 
 called, which would be the same context where we store all the Plan 
 objects. It should not be modified after creation, so that it doesn't 
 need to be copied when the ForeignScan is copied with copyObject(). It 
 should not contain transient state information like connection objects, 
 or references to a remotely prepared cursor etc. It must be possible to 
 call BeginScan multiple times with the same FdwPlan object, so that it 
 can be stored in a prepared plan that is executed multiple times.

The above statements seem mutually contradictory.  In particular,
I think you're proposing that copyObject copy only a pointer and not the
whole plan tree when copying ForeignScan.  That is entirely
unworkable/unacceptable: quite aside from the semantic ugliness, it will
fail altogether for cached plans.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] reporting reason for certain locks

2010-11-25 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 No, what I was suggesting was taking the existing function:
 extern void pgstat_report_waiting(bool waiting);
 ...and instead doing something like this:
 extern void pgstat_report_waiting(char *reason);
 ...and then arrange to pass the reason via the eponymous argument.

The question is how many cycles are we willing to expend on preparing a
reason string that (in approximately 99.9% of the calls) will not be
of any use.  It would be much better to avoid doing this and instead
expend the extra work on the inspection side.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SQL/MED - core functionality

2010-11-25 Thread Heikki Linnakangas

On 25.11.2010 18:18, Tom Lane wrote:

Heikki Linnakangasheikki.linnakan...@enterprisedb.com  writes:

I left out some details on what exactly FdwPlan should contain and what
it's lifecycle should be. I'm thinking that it should be allocated in
the CurrentMemoryContext that's active when the FDW Plan routine is
called, which would be the same context where we store all the Plan
objects. It should not be modified after creation, so that it doesn't
need to be copied when the ForeignScan is copied with copyObject(). It
should not contain transient state information like connection objects,
or references to a remotely prepared cursor etc. It must be possible to
call BeginScan multiple times with the same FdwPlan object, so that it
can be stored in a prepared plan that is executed multiple times.


The above statements seem mutually contradictory.  In particular,
I think you're proposing that copyObject copy only a pointer and not the
whole plan tree when copying ForeignScan.


Right.


 That is entirely
unworkable/unacceptable: quite aside from the semantic ugliness, it will
fail altogether for cached plans.


Hmm, I see, cached plans are planned in a shorter-lived context first, 
and copied to permanent storage afterwards. Needs more thought then. 
Maybe the FDW needs to provide a copyFdwPlan() function to copy FdwPlans 
returned by that FDW.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SQL/MED - core functionality

2010-11-25 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
 Hmm, I see, cached plans are planned in a shorter-lived context first, 
 and copied to permanent storage afterwards. Needs more thought then. 
 Maybe the FDW needs to provide a copyFdwPlan() function to copy FdwPlans 
 returned by that FDW.

Or just specify a format for the extra information.  Perhaps it could be
thought of as being a value of type bytea?  Obviously we can't just have
a fixed amount of info, but maybe a blob with a length word is enough.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] reporting reason for certain locks

2010-11-25 Thread Alvaro Herrera
Excerpts from Tom Lane's message of jue nov 25 13:23:42 -0300 2010:
 Robert Haas robertmh...@gmail.com writes:
  No, what I was suggesting was taking the existing function:
  extern void pgstat_report_waiting(bool waiting);
  ...and instead doing something like this:
  extern void pgstat_report_waiting(char *reason);
  ...and then arrange to pass the reason via the eponymous argument.
 
 The question is how many cycles are we willing to expend on preparing a
 reason string that (in approximately 99.9% of the calls) will not be
 of any use.  It would be much better to avoid doing this and instead
 expend the extra work on the inspection side.

I'm all for making this cheap -- and your proposal works for tuple
locks (ugly however it may be).  But it doesn't work for snapshot
locks such as the ones CREATE INDEX CONCURRENTLY takes.

-- 
Álvaro Herrera alvhe...@commandprompt.com
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SQL/MED - core functionality

2010-11-25 Thread Heikki Linnakangas

On 25.11.2010 18:28, Tom Lane wrote:

Heikki Linnakangasheikki.linnakan...@enterprisedb.com  writes:

Hmm, I see, cached plans are planned in a shorter-lived context first,
and copied to permanent storage afterwards. Needs more thought then.
Maybe the FDW needs to provide a copyFdwPlan() function to copy FdwPlans
returned by that FDW.


Or just specify a format for the extra information.  Perhaps it could be
thought of as being a value of type bytea?  Obviously we can't just have
a fixed amount of info, but maybe a blob with a length word is enough.


That seems quite awkward to work with. Let's at least make it a Node *, 
so that you can store a Value or List there, or anything else that 
already has copyObject support.


I think the PostgreSQL FDW would want to store the remote query there. 
But it's not a stretch that you want to use parameter markers in the 
remote query, with the parameter values determined at runtime. In that 
case you'd also store a list of Exprs for the parameter values (Hmm, 
BeginScan needs an ExprContext for that..). This is very hand-wavy, but 
I think we'll hit the wall with a single blob pretty quickly.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] security hooks on object creation

2010-11-25 Thread Robert Haas
2010/11/25 KaiGai Kohei kai...@ak.jp.nec.com:
 The attached patch is a revised patch.

 - The utils/hooks.h was renamed to catalog/objectaccess.h
 - Numeric in the tail of InvokeObjectAccessHook0() has gone.
 - Fixed bug in ATExecAddColumn; it gave AttributeRelationId
  to the hook instead of RelationRelationId.

 In addition, I found that we didn't put post-creation hook
 on foreign data wrapper, foreign server and user mapping
 exceptionally. So, I put this hook around their command
 handler like any other object classes.

Committed with minor, mostly cosmetic adjustments.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Regression Tests (opr) Sanity

2010-11-25 Thread Dimitri Fontaine
Hi,

Trying to fix a regression test problem I've left for better days while
developping the extensions, some help is needed.

The pg_execute_from_file() function is designed to work with either a
filename as its sole argument, or the filename and a VARIADIC text list
of arguments containing placeholder names and values. It works fine with
two entries in pg_proc using the same backend function, and it looks
like the following from a psql shell:

  List of functions
   Schema   | Name | Result data type | Argument data types |  
Type  
+--+--+-+
 pg_catalog | pg_execute_from_file | void | text| 
normal
 pg_catalog | pg_execute_from_file | void | text, VARIADIC text | 
normal
(2 rows)

Now the opr_sanity check includes the following query, which is expected
not to return any row:

=# SELECT p1.oid, p1.proname, p2.oid, p2.proname
-# FROM pg_proc AS p1, pg_proc AS p2
-# WHERE p1.oid  p2.oid AND
-# p1.prosrc = p2.prosrc AND
-# p1.prolang = 12 AND p2.prolang = 12 AND
-# (p1.proisagg = false OR p2.proisagg = false) AND
-# (p1.prolang != p2.prolang OR
(#  p1.proisagg != p2.proisagg OR
(#  p1.prosecdef != p2.prosecdef OR
(#  p1.proisstrict != p2.proisstrict OR
(#  p1.proretset != p2.proretset OR
(#  p1.provolatile != p2.provolatile OR
(#  p1.pronargs != p2.pronargs);
 oid  |   proname| oid  |   proname
--+--+--+--
 3927 | pg_execute_from_file | 3928 | pg_execute_from_file
(1 row)

Oops. I'm not granted to do it this way. So I've been trying to setup
pg_proc.h with a single entry and the default arguments. That's a weird
thing in there, pg_node_tree. So I've tried to copy/paste what I get
from pg_proc when I create a function in SQL with the same prototype:

create or replace function foo(text, variadic text[] default '{}'::text[])
  returns text 
  language sql
as $$
  select $1 || coalesce(',' || (select array_to_string(array_agg(x), ',') from 
unnest($2) x), '')
$$;

({CONST :consttype 1009 :consttypmod -1 :constlen -1 :constbyval false 
:constisnull false :location 61 :constvalue 16 [ 64 0 0 0 0 0 0 0 0 0 0 0 25 0 
0 0 ]})

Then initdb says FATAL: cannot accept a value of type pg_node_tree.

So, should I fix the opr_sanity check, and if so, what would be the
right approach? Or should we get the proargdefaults supported in the
bootstrap mode somehow? Or should I create the function in a SQL script
that initdb will use, somewhere?


Of course having a single entry in pg_proc without default values for
the placeholders won't fly, because the user is expected to be able to
actually use the 1-argument version of the function (no placeholder).

And I don't think having 2 names is a great answer, but if it comes to
that, of course, it's easy to do.

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-25 Thread Tom Lane
Maciek Sakrejda msakre...@truviso.com writes:
 Since triggering the set of FEBE messages that leads to this was tied
 deep into the guts of JDBC, I opted for raw wire protocol. It looks
 like the following sequence of messages from the client leads to this
 result format mixup:

 1. Parse, with statement name S_1, sql SELECT $1::int, and
 parameter type oid 0 (ask server to guess)
 2. Describe S_1
 3. Bind S_1 to anonymous portal, with param formats [ 0 (text) ],
 param values [ '2' ], result formats [ 1 (binary) ]
 4. Execute anonymous portal (returning all rows)
 5. Sync

 I have not tried to narrow this further yet. This essentially
 reproduces what Radosław was seeing but outside of the JDBC driver.
 That is, the server responds with:

 1. Parse completion
 2. Parameter description, with type oids [ 23 ]
 3. Row description, with empty table data, type oids [ 23 ], type
 length 4, type mod -1, and format 0 (text)
 4. Bind completion
 5. DataRow, with the result in binary, as requested, but not as
 described in message (3) above
 6. Command completion  RFQ

AFAICS this isn't a bug.  What you're issuing Describe against is the
prepared statement, not the portal.  The result column formats are not
specified by a prepared statement, so Describe just returns zeroes for
them.  Result column formats are specified by the Bind command, which
creates a Portal.  If you'd issued the Describe against the Portal, you
should get back the correct format codes.  Per the protocol
specification:

The Describe message (statement variant) specifies the name of an
existing prepared statement (or an empty string for the unnamed
prepared statement). The response is a ParameterDescription message
describing the parameters needed by the statement, followed by a
RowDescription message describing the rows that will be returned
when the statement is eventually executed (or a NoData message if
the statement will not return rows). ErrorResponse is issued if
there is no such prepared statement. Note that since Bind has not
yet been issued, the formats to be used for returned columns are not
yet known to the backend; the format code fields in the
RowDescription message will be zeroes in this case.

Now, if there's something in the JDBC driver that expects
DescribeStatement to return useful result format codes, that'd be
a bug in the driver.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SQL/MED - core functionality

2010-11-25 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
 On 25.11.2010 18:28, Tom Lane wrote:
 Or just specify a format for the extra information.  Perhaps it could be
 thought of as being a value of type bytea?  Obviously we can't just have
 a fixed amount of info, but maybe a blob with a length word is enough.

 That seems quite awkward to work with. Let's at least make it a Node *, 
 so that you can store a Value or List there, or anything else that 
 already has copyObject support.

Yeah, that works.  A struct could be emulated by using a List with a
known order of elements.  If someone did need a binary blob, they could
represent it as a Const of type bytea.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Regression Tests (opr) Sanity

2010-11-25 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes:
 The pg_execute_from_file() function is designed to work with either a
 filename as its sole argument, or the filename and a VARIADIC text list
 of arguments containing placeholder names and values. It works fine with
 two entries in pg_proc using the same backend function, and it looks
 like the following from a psql shell:

Just make two pg_proc entries that are pointing at two C functions.
The C functions can call a common subroutine after extracting their
arguments.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-25 Thread Maciek Sakrejda
Interesting. I think you're right. Looking at the Wireshark traffic
again, the driver seems to issue a portal-variant Describe when using
unnamed prepared statements, but as soon as the named prepared
statements kick in (per prepare threshold), the Describe is a
statement-variant Describe with the expected behavior you've noted.
Actually, the driver sends Parse / Bind / Describe with the unnamed
statement, but for some reason switches to Parse / Describe / Bind
with named ones.
---
Maciek Sakrejda | System Architect | Truviso

1065 E. Hillsdale Blvd., Suite 215
Foster City, CA 94404
(650) 242-3500 Main
www.truviso.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Regression Tests (opr) Sanity

2010-11-25 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes:
 Just make two pg_proc entries that are pointing at two C functions.
 The C functions can call a common subroutine after extracting their
 arguments.

Mmmm, ok, will adapt the idea to the current code, where the extracting
is mingled into the processing. Thanks for the idea, that's much simpler
this way. Cleaner ain't always better :)

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] problem with Win32 buildfarm

2010-11-25 Thread Bruce Momjian
Bruce Momjian wrote:
 Win32 buildfarm members are red because of my inet_pton changes.  I will
 look into this in the next day, and also improve how we include C files
 from /port for libpq.

OK, I have accomplished both goals with the two attached, applied
patches.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 74ae79a..0be6a72 100644
*** /tmp/kBcRnb_Makefile	Thu Nov 25 12:48:37 2010
--- src/interfaces/libpq/Makefile	Thu Nov 25 12:44:41 2010
*** override CFLAGS += $(PTHREAD_CFLAGS)
*** 25,48 
  endif
  
  # Need to recompile any libpgport object files because we need these
! # object files to use the same compile flags as libpq.  If we used
! # the object files from libpgport, this would not be true on all
! # platforms.  We filter some object files so we only use object
! # files configure says we need.
  LIBS := $(LIBS:-lpgport=)
  
  OBJS=	fe-auth.o fe-connect.o fe-exec.o fe-misc.o fe-print.o fe-lobj.o \
  	fe-protocol2.o fe-protocol3.o pqexpbuffer.o pqsignal.o fe-secure.o \
  	libpq-events.o \
! 	md5.o ip.o wchar.o encnames.o inet_net_ntop.o noblock.o pgstrcasecmp.o thread.o \
! 	$(filter crypt.o getaddrinfo.o inet_aton.o open.o snprintf.o strerror.o strlcpy.o win32error.o, $(LIBOBJS))
  
  ifeq ($(PORTNAME), cygwin)
  override shlib = cyg$(NAME)$(DLSUFFIX)
  endif
  
  ifeq ($(PORTNAME), win32)
! OBJS += win32.o pgsleep.o libpqrc.o
  
  libpqrc.o: libpq.rc
  	$(WINDRES) -i $ -o $@
--- 25,57 
  endif
  
  # Need to recompile any libpgport object files because we need these
! # object files to use the same compile flags as libpq; some
! # platforms require special flags for all libpq object files.
  LIBS := $(LIBS:-lpgport=)
  
+ # external object files that are always used by libpq
+ BACKEND_LIBPQ = md5 ip
+ UTILS_MB = encnames wchar
+ PERM_PGPORT = inet_net_ntop noblock pgstrcasecmp thread
+ ifeq ($(PORTNAME), win32)
+ PERM_PGPORT += pgsleep
+ endif
+ 
+ 
+ # pgport object files are used by libpq if identified by configure
+ OPT_PGPORT = $(filter $(addsuffix .o, crypt getaddrinfo inet_aton open snprintf strerror strlcpy win32error), $(LIBOBJS))
+ 
  OBJS=	fe-auth.o fe-connect.o fe-exec.o fe-misc.o fe-print.o fe-lobj.o \
  	fe-protocol2.o fe-protocol3.o pqexpbuffer.o pqsignal.o fe-secure.o \
  	libpq-events.o \
! 	$(addsuffix .o, $(PERM_PGPORT) $(OPT_PGPORT) $(BACKEND_LIBPQ) $(UTILS_MB))
  
  ifeq ($(PORTNAME), cygwin)
  override shlib = cyg$(NAME)$(DLSUFFIX)
  endif
  
  ifeq ($(PORTNAME), win32)
! OBJS += win32.o libpqrc.o
  
  libpqrc.o: libpq.rc
  	$(WINDRES) -i $ -o $@
*** backend_src = $(top_srcdir)/src/backend
*** 77,93 
  # We use several backend modules verbatim, but since we need to
  # compile with appropriate options to build a shared lib, we can't
  # necessarily use the same object files as the backend uses. Instead,
! # symlink the source files in here and build our own object file.
! # For port modules, this only happens if configure decides the module
! # is needed (see filter hack in OBJS, above).
  
! crypt.c getaddrinfo.c inet_aton.c inet_net_ntop.c noblock.c open.c pgstrcasecmp.c snprintf.c strerror.c strlcpy.c thread.c win32error.c pgsleep.c: % : $(top_srcdir)/src/port/%
  	rm -f $@  $(LN_S) $ .
  
! md5.c ip.c: % : $(backend_src)/libpq/%
  	rm -f $@  $(LN_S) $ .
  
! encnames.c wchar.c : % : $(backend_src)/utils/mb/%
  	rm -f $@  $(LN_S) $ .
  
  
--- 86,100 
  # We use several backend modules verbatim, but since we need to
  # compile with appropriate options to build a shared lib, we can't
  # necessarily use the same object files as the backend uses. Instead,
! # we symlink the source files in here and build our own object files.
  
! $(addsuffix .c, $(PERM_PGPORT) $(OPT_PGPORT)): % : $(top_srcdir)/src/port/%
  	rm -f $@  $(LN_S) $ .
  
! $(addsuffix .c, $(BACKEND_LIBPQ)): % : $(backend_src)/libpq/%
  	rm -f $@  $(LN_S) $ .
  
! $(addsuffix .c, $(UTILS_MB)): % : $(backend_src)/utils/mb/%
  	rm -f $@  $(LN_S) $ .
  
  
*** uninstall: uninstall-lib
*** 124,130 
  	rm -f '$(DESTDIR)$(datadir)/pg_service.conf.sample'
  
  clean distclean: clean-lib
! 	rm -f $(OBJS) pg_config_paths.h crypt.c getaddrinfo.c inet_aton.c noblock.c open.c pgstrcasecmp.c snprintf.c strerror.c strlcpy.c thread.c md5.c ip.c encnames.c wchar.c win32error.c pgsleep.c pthread.h libpq.rc
  # Might be left over from a Win32 client-only build
  	rm -f pg_config_paths.h
  
--- 131,137 
  	rm -f '$(DESTDIR)$(datadir)/pg_service.conf.sample'
  
  clean distclean: clean-lib
! 	rm -f $(OBJS) pg_config_paths.h pthread.h libpq.rc $(addsuffix .c, $(BACKEND_LIBPQ) $(UTILS_MB) $(PERM_PGPORT) $(OPT_PGPORT))
  # Might be left over from a Win32 client-only build
  	rm -f pg_config_paths.h
  
diff --git a/src/port/getaddrinfo.c b/src/port/getaddrinfo.c

Re: [HACKERS] Re: [BUGS] BUG #5650: Postgres service showing as stopped when in fact it is running

2010-11-25 Thread Bruce Momjian
Bruce Momjian wrote:
   BTW, it is annoying that we can't definitively distinguish postmaster
   is not running from a connectivity problem, but I can't see a way
   around that.
  
  Agreed.  I will research this.
 
 I have researched this and developed the attached patch.  It implements
 PGping() and PGpingParams() in libpq, and has pg_ctl use it for pg_ctl
 -w server status detection.
 
 The new output for cases where .pgpass is not allowing for a connection
 is:
 
   $ pg_ctl -w -l /dev/null start
   waiting for server to start done
   server started
   However, could not connect, perhaps due to invalid authentication or
   misconfiguration.
 
 The code basically checks the connection status between PQconnectStart()
 and connectDBComplete() to see if the server is running but we failed to
 connect for some reason.

I have applied this patch, with modified wording of the cannot connect
case:

$ pg_ctl -w -l /dev/null start
waiting for server to start done
server started
warning:  could not connect, perhaps due to invalid authentication or
misconfiguration.

I assume having the warning as the last printed things is appropriate. 
This is my second patch this week that got little feedback --- I am
getting a little spooked.  ;-)

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-25 Thread Maciek Sakrejda
Okay, looking at the JDBC side of things, I think JDBC doesn't
actually need that information (since, it always used text results
before Radosław's patch--the previous binary support was for
parameters only, right?). From looking at QueryExecutorImpl
(specifically sendOneQuery), it's clear that it *does* need the
Describe before the Bind when using named prepared statements. I can't
quite follow why, but I think the upshot of this is that there is no
bug in either the JDBC driver or the backend. This is just a
limitation of the existing implementation in the JDBC driver that
Radosław would need to work around for binary result processing
(probably by storing the requested format somewhere and using that
instead of what comes back from Describe).

I think the lessons are (1) the bug is probably *not* in the
established library or backend code and (2) it's a little confusing
that the result codes for I don't know what format this will be in
and this will be in text format are aliased.

It's too late to do anything about the second one and it's only a minor quibble.

Thanks for the help in making sense of this.
---
Maciek Sakrejda | System Architect | Truviso

1065 E. Hillsdale Blvd., Suite 215
Foster City, CA 94404
(650) 242-3500 Main
www.truviso.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-25 Thread Radosław Smogura
Hi,

Thank you for your response.

I would only ask to be sure...
So, to summarise, I shouldn't believe server DescribeRow (in context of
format), in this situation, but only I should look at this what I asked
for, isn't it? If I asked for columns in binary format, I need to do binary
reading regarding what server has responded? If I asked for odd columns in
text, even in binary do I need to choose proper format basing only on my
request?

But to the last part of cited protocol specification, when I've sent
message with statement parameter's type int4, int8, varchar the format
field wasn't set to 0, but 1.

Kind regards,
Radosław Smogura

On Thu, 25 Nov 2010 12:23:03 -0500, Tom Lane t...@sss.pgh.pa.us wrote:
 AFAICS this isn't a bug.  What you're issuing Describe against is the
 prepared statement, not the portal.  The result column formats are not
 specified by a prepared statement, so Describe just returns zeroes for
 them.  Result column formats are specified by the Bind command, which
 creates a Portal.  If you'd issued the Describe against the Portal, you
 should get back the correct format codes.  Per the protocol
 specification:
 
 The Describe message (statement variant) specifies the name of an
 existing prepared statement (or an empty string for the unnamed
 prepared statement). The response is a ParameterDescription message
 describing the parameters needed by the statement, followed by a
 RowDescription message describing the rows that will be returned
 when the statement is eventually executed (or a NoData message if
 the statement will not return rows). ErrorResponse is issued if
 there is no such prepared statement. Note that since Bind has not
 yet been issued, the formats to be used for returned columns are not
 yet known to the backend; the format code fields in the
 RowDescription message will be zeroes in this case.
 
 Now, if there's something in the JDBC driver that expects
 DescribeStatement to return useful result format codes, that'd be
 a bug in the driver.
 
   regards, tom lane

-- 
--
Radosław Smogura
http://www.softperience.eu

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] reporting reason for certain locks

2010-11-25 Thread Robert Haas
On Thu, Nov 25, 2010 at 11:23 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 Robert Haas robertmh...@gmail.com writes:
 No, what I was suggesting was taking the existing function:
 extern void pgstat_report_waiting(bool waiting);
 ...and instead doing something like this:
 extern void pgstat_report_waiting(char *reason);
 ...and then arrange to pass the reason via the eponymous argument.

 The question is how many cycles are we willing to expend on preparing a
 reason string that (in approximately 99.9% of the calls) will not be
 of any use.  It would be much better to avoid doing this and instead
 expend the extra work on the inspection side.

I'd much rather have the information be pulled than pushed, if there's
a way to make that work.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [COMMITTERS] How strings are sorted by LC_COLLATE specifically?

2010-11-25 Thread Robert Haas
On Thu, Nov 25, 2010 at 1:40 AM, Chang Chao charleschung...@gmail.com wrote:
 How strings are sorted when LC_COLLATE = ja_JP.UTF-8.
 I tried to read the documention on that,but there are just a few words,
 like LC_COLLATE determines string sort order,
 Is there a specific reference about this?
 So I can implement an equivalent string sort function in JAVA.
 because some of the sort logic is here.
 Any clue will be greatly appreciated.

As far as I know, we just inherit whatever behavior the operating
system has for that collation.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-25 Thread Maciek Sakrejda
 So, to summarise, I shouldn't believe server DescribeRow (in context of 
 format), in this situation, but only I should look at this what I asked
 for, isn't it? If I asked for columns in binary format, I need to do binary 
 reading regarding what server has responded?

Yes, because in this case 0 doesn't mean the result will be in
text, it means, you issued the statement-variant of Describe, so I'm
not sure what the result format will be yet.

 If I asked for odd columns in text, even in binary do I need to choose proper 
 format basing only on my request?

I don't quite understand this question, but I think so. I don't think
there's ever a situation where the server will ignore your result
format requests.

 But to the last part of cited protocol specification, when I've sent message 
 with statement parameter's type int4, int8, varchar the format
 field wasn't set to 0, but 1.

I wasn't able to reproduce that with my standalone test case. When I
changed the parameter oid to 23, I still got the same behavior. Can
you alter my test case to reproduce the error?

---
Maciek Sakrejda | System Architect | Truviso

1065 E. Hillsdale Blvd., Suite 215
Foster City, CA 94404
(650) 242-3500 Main
www.truviso.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-25 Thread Radosław Smogura
On Thu, 25 Nov 2010 11:28:02 -0800, Maciek Sakrejda
msakre...@truviso.com
wrote:
 So, to summarise, I shouldn't believe server DescribeRow (in context of
 format), in this situation, but only I should look at this what I asked
 for, isn't it? If I asked for columns in binary format, I need to do
 binary reading regarding what server has responded?
 
 Yes, because in this case 0 doesn't mean the result will be in
 text, it means, you issued the statement-variant of Describe, so I'm
 not sure what the result format will be yet.
 
 If I asked for odd columns in text, even in binary do I need to choose
 proper format basing only on my request?
 
 I don't quite understand this question, but I think so. I don't think
 there's ever a situation where the server will ignore your result
 format requests.
 
 But to the last part of cited protocol specification, when I've sent
 message with statement parameter's type int4, int8, varchar the format
 field wasn't set to 0, but 1.
 
 I wasn't able to reproduce that with my standalone test case. When I
 changed the parameter oid to 23, I still got the same behavior. Can
 you alter my test case to reproduce the error?
 
I will do it tomorrow.

In this situation I need to test portals as well.

 ---
 Maciek Sakrejda | System Architect | Truviso
 
 1065 E. Hillsdale Blvd., Suite 215
 Foster City, CA 94404
 (650) 242-3500 Main
 www.truviso.com

-- 
--
Radosław Smogura
http://www.softperience.eu

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-25 Thread Tom Lane
Maciek Sakrejda msakre...@truviso.com writes:
 But to the last part of cited protocol specification, when I've sent message 
 with statement parameter's type int4, int8, varchar the format
 field wasn't set to 0, but 1.

 I wasn't able to reproduce that with my standalone test case. When I
 changed the parameter oid to 23, I still got the same behavior. Can
 you alter my test case to reproduce the error?

I'd be really surprised if that affected the server-side behavior.
OTOH, it seems possible that the JDBC driver might behave differently
depending on whether parameter types were prespecified or not --- it
might issue Describe earlier in order to get the parameter types,
perhaps.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-25 Thread Maciek Sakrejda
 OTOH, it seems possible that the JDBC driver might behave differently
 depending on whether parameter types were prespecified or not --- it
 might issue Describe earlier in order to get the parameter types,
 perhaps.

Ah. Bingo:

boolean describeStatement = describeOnly || (!oneShot 
paramsHasUnknown  queryHasUnknown  !query.isStatementDescribed());


---
Maciek Sakrejda | System Architect | Truviso

1065 E. Hillsdale Blvd., Suite 215
Foster City, CA 94404
(650) 242-3500 Main
www.truviso.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-25 Thread Radosław Smogura
Hm... I moved Bind before Describe, I now have
// Construct a new portal if needed.
Portal portal = null;
if (usePortal)
{
String portalName = C_ + (nextUniqueID++);
portal = new Portal(query, portalName);
}

sendBind(query, params, portal, noBinaryTransfer);

if (describeStatement) {
sendDescribeStatement(query, params, describeOnly);
if (describeOnly)
return;
}
still nothing
21:43:02.263 (26) simple execute,
handler=org.postgresql.jdbc2.abstractjdbc2statement$statementresulthand...@337ec9f7,
maxRows=0, fetchSize=0, flags=16
21:43:02.264 (26)  FE= Parse(stmt=S_1,query=SELECT $1::int,oids={0})
21:43:02.264 (26)  FE= Bind(stmt=S_1,portal=null,$1='2')
21:43:02.264 (26)  FE= Describe(statement=S_1)
21:43:02.264 (26)  FE= Execute(portal=null,limit=0)
21:43:02.265 (26)  FE= Sync
21:43:02.265 (26)  =BE ParseComplete [S_1]
21:43:02.265 (26)  =BE BindComplete [null]
21:43:02.266 (26)  =BE ParameterDescription
21:43:02.266 (26)  =BE RowDescription(1)
21:43:02.266 (26) Field(,INT4,4,T)
21:43:02.266 (26)  =BE DataRow(len=4)
21:43:02.267 (26)  =BE CommandStatus(SELECT 1)
21:43:02.267 (26)  =BE ReadyForQuery(I)
If I've understood well I should get proper result after 1st bind...?

On Thu, 25 Nov 2010 12:21:39 -0800, Maciek Sakrejda
msakre...@truviso.com
wrote:
 OTOH, it seems possible that the JDBC driver might behave differently
 depending on whether parameter types were prespecified or not --- it
 might issue Describe earlier in order to get the parameter types,
 perhaps.
 
 Ah. Bingo:
 
 boolean describeStatement = describeOnly || (!oneShot 
 paramsHasUnknown  queryHasUnknown  !query.isStatementDescribed());
 
 
 ---
 Maciek Sakrejda | System Architect | Truviso
 
 1065 E. Hillsdale Blvd., Suite 215
 Foster City, CA 94404
 (650) 242-3500 Main
 www.truviso.com

-- 
--
Radosław Smogura
http://www.softperience.eu


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Patch to add a primary key using an existing index

2010-11-25 Thread Steve Singer

On 10-11-22 03:24 PM, Steve Singer wrote:

On 10-11-22 09:37 AM, Gurjeet Singh wrote:

On Sat, Nov 20, 2010 at 9:00 AM, Steve Singer ssinger...@sympatico.ca



Almost fixed.
I still get an unexpected difference.

! DETAIL: cannot create PRIMARY KEY/UNIQUE constraint with a non-unique
index.
CREATE UNIQUE INDEX rpi_idx2 ON rpi_test(a , b);
-- should fail; WITH INDEX option specified more than once.
ALTER TABLE rpi_test ADD PRIMARY KEY (a, b)
--- 35,41 
-- should fail; non-unique
ALTER TABLE rpi_test ADD primary key(a, b) WITH (INDEX = 'rpi_idx1');
ERROR: rpi_idx1 is not a unique index
! DETAIL: Cannot create PRIMARY KEY/UNIQUE constraint using a non-unique
index.


The attached version of the patch gets your regression tests to pass.

I'm going to mark this as ready for a committer.






replace_pkey_index.revised2.patch.gz
Description: GNU Zip compressed data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-25 Thread Maciek Sakrejda
 21:43:02.264 (26)  FE= Describe(statement=S_1)

You're still doing the statement-flavor Describe. As Tom pointed out,
this won't tell you the result types because it doesn't know them.
Actually, technically if you issue a statement-flavor Describe *after*
a Bind, the server does have this information, but I'm not surprised
that it doesn't send it correctly, since it seems pointless to send
the statement variation after already doing a Bind.

You may be able to change the JDBC code to call sendDescribePortal() instead.


---
Maciek Sakrejda | System Architect | Truviso

1065 E. Hillsdale Blvd., Suite 215
Foster City, CA 94404
(650) 242-3500 Main
www.truviso.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, this time with a patch

2010-11-25 Thread Dimitri Fontaine
Itagaki Takahiro itagaki.takah...@gmail.com writes:
 Thanks. I'll move the patch to Ready for Committer.

Thanks!
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_execute_from_file review

2010-11-25 Thread Dimitri Fontaine
Joshua Tolley eggyk...@gmail.com writes:
 I've just looked at pg_execute_from_file[1]. The idea here is to execute all
 the SQL commands in a given file. My comments:

Thanks for your review. Please find attached a revised patch where I've
changed the internals of the function so that it's split in two and that
the opr_sanity check passes, per comments from David Wheeler and Tom Lane.

 * I'd like to see the docs slightly expanded, specifically to describe
   parameter replacement. I wondered for a while if I needed to set of
   parameters in any specific way, before reading the code and realizing they
   can be whatever I want.

My guess is that you knew that in the CREATE EXTENSION context, it has
been proposed to use the notation @extschema@ as a placeholder, and
you've then been confused. I've refrained from imposing any style with
respect to what the placeholder would look like in the mecanism-patch.

Do we still want to detail in the docs that there's nothing expected
about the placeholder syntax of format?

 * Does anyone think it needs representation in the test suite?

Well the patch will get its tests with the arrival of the extension main
patch, where all contribs are installed using it.

 * Is it at all bad to include spi.h in genfile.c? IOW should this function
   live elsewhere? It seems reasonable to me to do it as written, but I thought
   I'd ask.

Well, using spi at this place has been asked by Álvaro and Tom, so my
guess is that's ok :)

 * In the snippet below, it seems best just to use palloc0():
 query_string = (char *)palloc((fsize+1)*sizeof(char));
 memset(query_string, 0, fsize+1);

Edited.

 * Shouldn't it include SPI_push() and SPI_pop()?

ENOCLUE

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

*** a/doc/src/sgml/func.sgml
--- b/doc/src/sgml/func.sgml
***
*** 14461,14466  postgres=# SELECT * FROM pg_xlogfile_name_offset(pg_stop_backup());
--- 14461,14475 
 entrytyperecord/type/entry
 entryReturn information about a file/entry
/row
+   row
+entry
+ literalfunctionpg_execute_from_file(parameterfilename/ typetext/
+ [, parametervariable/parameter typetext/type, parametervalue/parameter typetext/type
+ [, ...] ]) )/function/literal
+/entry
+entrytypevoid/type/entry
+entryExecutes the acronymSQL/ commands contained in a file, replacing given placeholders./entry
+   /row
   /tbody
  /tgroup
 /table
***
*** 14499,14504  SELECT (pg_stat_file('filename')).modification;
--- 14508,14527 
  /programlisting
 /para
  
+indexterm
+ primarypg_execute_from_file/primary
+/indexterm
+para
+ functionpg_execute_from_file/ makes the server
+ execute acronymSQL/ commands to be found in a file. This function is
+ reserved to superusers.
+/para
+para
+ The script might contain placeholders that will be replaced by the
+ values given in the literalVARIADIC/literal arguments, which must be
+ a pair of variable names and values.
+/para
+ 
 para
  The functions shown in xref linkend=functions-advisory-locks manage
  advisory locks.  For details about proper use of these functions, see
***
*** 14521,14526  SELECT (pg_stat_file('filename')).modification;
--- 14544,14550 
 entrytypevoid/type/entry
 entryObtain exclusive advisory lock/entry
/row
+ 
row
 entry
  literalfunctionpg_advisory_lock(parameterkey1/ typeint/, parameterkey2/ typeint/)/function/literal
*** a/src/backend/utils/adt/genfile.c
--- b/src/backend/utils/adt/genfile.c
***
*** 7,12 
--- 7,13 
   * Copyright (c) 2004-2010, PostgreSQL Global Development Group
   *
   * Author: Andreas Pflug pgad...@pse-consulting.de
+  * Dimitri Fontaine dimi...@2ndquadrant.fr
   *
   * IDENTIFICATION
   *	  src/backend/utils/adt/genfile.c
***
*** 21,31 
--- 22,34 
  #include dirent.h
  
  #include catalog/pg_type.h
+ #include executor/spi.h
  #include funcapi.h
  #include mb/pg_wchar.h
  #include miscadmin.h
  #include postmaster/syslogger.h
  #include storage/fd.h
+ #include utils/array.h
  #include utils/builtins.h
  #include utils/memutils.h
  #include utils/timestamp.h
***
*** 264,266  pg_ls_dir(PG_FUNCTION_ARGS)
--- 267,441 
  
  	SRF_RETURN_DONE(funcctx);
  }
+ 
+ /*
+  * Support functions for pg_execute_from_file and its variant,
+  * pg_execute_from_file_with_placeholders.
+  */
+ static char *
+ read_query_string_from_file(const char *filename)
+ {
+ 	FILE   *file;
+ 	int64   fsize = -1, nbytes;
+ 	struct stat fst;
+ 	char   *query_string = NULL;
+ 
+ 	/*
+ 	 * Only superuser can call pg_execute_from_file, and CREATE EXTENSION
+ 	 * uses that too. Don't double check the PATH. Also note that
+ 	 * extension's install files are not in $PGDATA but `pg_config
+ 	 * 

Re: [HACKERS] ALTER OBJECT any_name SET SCHEMA name

2010-11-25 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes:
 Please do.  Tab completion support should really be included in the
 patch - adding it as a separate patch is better than not having it, of
 course.

Please find attached version 9 of the patch, which includes psql
completion support of the SET SCHEMA variant of already supported
ALTER commands.

That means I didn't add ALTER OPERATOR [CLASS,FAMILY] completion
support, my guess being there's no demand here, or the existing syntax
variants would be there already. And if there's demand, I don't feel
like it should be implemented as part of this very patch.

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

*** a/doc/src/sgml/ref/alter_conversion.sgml
--- b/doc/src/sgml/ref/alter_conversion.sgml
***
*** 23,28  PostgreSQL documentation
--- 23,29 
  synopsis
  ALTER CONVERSION replaceablename/replaceable RENAME TO replaceablenew_name/replaceable
  ALTER CONVERSION replaceablename/replaceable OWNER TO replaceablenew_owner/replaceable
+ ALTER CONVERSION replaceablename/replaceable SET SCHEMA replaceablenew_schema/replaceable
  /synopsis
   /refsynopsisdiv
  
***
*** 75,80  ALTER CONVERSION replaceablename/replaceable OWNER TO replaceablenew_owner
--- 76,90 
   /para
  /listitem
 /varlistentry
+ 
+varlistentry
+ termreplaceable class=parameternew_schema/replaceable/term
+ listitem
+  para
+   The new schema of the conversion.
+  /para
+ /listitem
+/varlistentry
/variablelist
   /refsect1
  
*** a/doc/src/sgml/ref/alter_opclass.sgml
--- b/doc/src/sgml/ref/alter_opclass.sgml
***
*** 23,28  PostgreSQL documentation
--- 23,29 
  synopsis
  ALTER OPERATOR CLASS replaceablename/replaceable USING replaceable class=parameterindex_method/replaceable RENAME TO replaceablenew_name/replaceable
  ALTER OPERATOR CLASS replaceablename/replaceable USING replaceable class=parameterindex_method/replaceable OWNER TO replaceablenew_owner/replaceable
+ ALTER OPERATOR CLASS replaceablename/replaceable USING replaceable class=parameterindex_method/replaceable SET SCHEMA replaceablenew_schema/replaceable
  /synopsis
   /refsynopsisdiv
  
***
*** 85,90  ALTER OPERATOR CLASS replaceablename/replaceable USING replaceable class=p
--- 86,100 
   /para
  /listitem
 /varlistentry
+ 
+varlistentry
+ termreplaceable class=parameternew_schema/replaceable/term
+ listitem
+  para
+   The new schema for the operator class.
+  /para
+ /listitem
+/varlistentry
   /variablelist
   /refsect1
  
*** a/doc/src/sgml/ref/alter_operator.sgml
--- b/doc/src/sgml/ref/alter_operator.sgml
***
*** 22,27  PostgreSQL documentation
--- 22,28 
   refsynopsisdiv
  synopsis
  ALTER OPERATOR replaceablename/replaceable ( { replaceableleft_type/replaceable | NONE } , { replaceableright_type/replaceable | NONE } ) OWNER TO replaceablenew_owner/replaceable
+ ALTER OPERATOR replaceablename/replaceable ( { replaceableleft_type/replaceable | NONE } , { replaceableright_type/replaceable | NONE } ) SET SCHEMA replaceablenew_schema/replaceable
  /synopsis
   /refsynopsisdiv
  
***
*** 85,90  ALTER OPERATOR replaceablename/replaceable ( { replaceableleft_type/repla
--- 86,100 
   /para
  /listitem
 /varlistentry
+ 
+varlistentry
+ termreplaceable class=parameternew_schema/replaceable/term
+ listitem
+  para
+   The new schema of the operator.
+  /para
+ /listitem
+/varlistentry
/variablelist
   /refsect1
  
*** a/doc/src/sgml/ref/alter_opfamily.sgml
--- b/doc/src/sgml/ref/alter_opfamily.sgml
***
*** 31,36  ALTER OPERATOR FAMILY replaceablename/replaceable USING replaceable class=
--- 31,37 
} [, ... ]
  ALTER OPERATOR FAMILY replaceablename/replaceable USING replaceable class=parameterindex_method/replaceable RENAME TO replaceablenew_name/replaceable
  ALTER OPERATOR FAMILY replaceablename/replaceable USING replaceable class=parameterindex_method/replaceable OWNER TO replaceablenew_owner/replaceable
+ ALTER OPERATOR FAMILY replaceablename/replaceable USING replaceable class=parameterindex_method/replaceable SET SCHEMA replaceablenew_schema/replaceable
  /synopsis
   /refsynopsisdiv
  
***
*** 216,221  ALTER OPERATOR FAMILY replaceablename/replaceable USING replaceable class=
--- 217,231 
   /para
  /listitem
 /varlistentry
+ 
+varlistentry
+ termreplaceable class=parameternew_schema/replaceable/term
+ listitem
+  para
+   The new schema for the operator family.
+  /para
+ /listitem
+/varlistentry
   /variablelist
  
para
*** a/doc/src/sgml/ref/alter_tsconfig.sgml
--- b/doc/src/sgml/ref/alter_tsconfig.sgml
***
*** 33,38  ALTER TEXT SEARCH CONFIGURATION replaceablename/replaceable
--- 33,39 
  DROP MAPPING [ IF EXISTS ] 

[HACKERS] improving foreign key locks

2010-11-25 Thread Alvaro Herrera
Hi,

So I've been working on improving locks for foreign key checks, as
discussed in a thread started by Joel Jacobson a while ago.  I've posted
about this:
http://www.commandprompt.com/blogs/alvaro_herrera/2010/11/fixing_foreign_key_deadlocks/
http://www.commandprompt.com/blogs/alvaro_herrera/2010/11/fixing_foreign_key_deadlocks_part_2/
(Note [1] below).

There's a question that arose in internal CMD discussion, which is: is
there an use case for keeping SELECT FOR SHARE locks, with the semantics
that they currently have, if we replace the FK checks with some other
lock implementation?

I've been assuming that we would keep FOR SHARE, because it's a feature
that's been exposed to users directly as a SQL command for five
releases, and so presumably someone may be depending on it.  So the new
code would be triggered by a different SQL option, and it needs to work
in conjunction with FOR SHARE.

Now, if the majority opinion here is that we don't need to keep the
current FOR SHARE semantics, a few things would be different.

Thoughts on keeping vs. removing FOR SHARE?

I will be posting more extensively on the implementation of this on this
list, later.


[1] The blog posts says that FOR SHARE would conflict with FOR KEY LOCK,
but I'm having second thoughts about this for various reasons; so they
will not conflict (in other words, transaction A can take a FOR SHARE
lock in a tuple, and transaction B can take FOR KEY LOCK, and they both
can continue).  Please consider this if you want to comment on the
design presented in those articles.

-- 
Álvaro Herrera alvhe...@alvh.no-ip.org

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Suggested easy TODO: pg_dump --from-list

2010-11-25 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes:
 One thing I've often wished for is the ability to dump a specific
 function

See getddl from OmniTI, or the alternative version I kept forgetting to
put online somewhere:

  https://labs.omniti.com/labs/pgtreats/wiki/getddl
  https://github.com/dimitri/getddl

The OmniTI version will output a single file with all objects into a
single file, and my fork will do that in a directory structure with a
file per object or about (a single file containing all functions sharing
the same name, e.g.).

Both project goal is to make it easy to version (as in git) your DDL and
check for changes.

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Assertion failure on hot standby

2010-11-25 Thread Simon Riggs
On Thu, 2010-11-25 at 16:59 +0900, Fujii Masao wrote:
 To solve the problem, ISTM that XID should be assigned before the
 information about AccessExclusive lock becomes visible to another
 process. Or CHECKPOINT (i.e., GetRunningTransactionLocks) should
 ignore the locks with XID = 0. 

First, thanks for pursuing this. I realise I made the mistake of
assuming there was just one bug; I see that the bug Heikki was
discussing is a separate issue.

As to solutions, it cannot be acceptable to ignore some locks just
because an xid has not been assigned.

If I understand you correctly, it seems possible to generate an
AccessExclusiveLock before an xid is assigned, so that its possible to
log that situation before the transaction assigns an xid slightly later.
So there's a narrow window where we can generate a lock WAL record with
xid 0. The sensible resolution is to ensure that all
AccessExclusiveLocks have an xid assigned prior to them registering
their proclock.

That would mean running GetCurrentTransactionId() inside LockAcquire()

if (lockmode = AccessExclusiveLock 
locktag-locktag_type == LOCKTAG_RELATION 
!RecoveryInProgress())
(void) GetCurrentTransactionId();

Any objections to that fix?

-- 
 Simon Riggs   http://www.2ndQuadrant.com/books/
 PostgreSQL Development, 24x7 Support, Training and Services
 


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [COMMITTERS] How strings are sorted by LC_COLLATE specifically?

2010-11-25 Thread Chang Chao
Hi,Robert.
Thanks for your reply.

As far as I looked into  postgre's source,
I came to know that It seems that it uses strcoll to compare strings.
So it depends on the underlying operating system,like you said.

Charles.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [GENERAL] column-level update privs + lock table

2010-11-25 Thread KaiGai Kohei
(2010/10/16 4:49), Josh Kupershmidt wrote:
 [Moving to -hackers]
 
 On Fri, Oct 15, 2010 at 3:43 AM, Simon Riggssi...@2ndquadrant.com  wrote:
 On Mon, 2010-10-11 at 09:41 -0400, Josh Kupershmidt wrote:
 On Thu, Oct 7, 2010 at 7:43 PM, Josh Kupershmidtschmi...@gmail.com  wrote:

 I noticed that granting a user column-level update privileges doesn't
 allow that user to issue LOCK TABLE with any mode other than Access
 Share.

 Anyone think this could be added as a TODO?

 Seems so to me, but you raise on Hackers.
 
 Thanks, Simon. Attached is a simple patch to let column-level UPDATE
 privileges allow a user to LOCK TABLE in a mode higher than Access
 Share. Small doc. update and regression test update are included as
 well. Feedback is welcome.
 

I checked your patch, then I'd like to mark it as ready for committer.

The point of this patch is trying to solve an incompatible behavior
between SELECT ... FOR SHARE/UPDATE and LOCK command.

On ExecCheckRTEPerms(), it allows the required accesses when no columns
are explicitly specified in the query and the current user has necessary
privilege on one of columns within the target relation.
If we stand on the perspective that LOCK command should take same
privileges with the case when we use SELECT ... FOR SHARE/UPDATE without
specifying explicit columns, like COUNT(*), the existing LOCK command
seems to me odd.

I think this patch fixes the behavior as we expected.

BTW, how about backporting this patch?
It seems to me a bug fix, although it contains user visible changes.

Thanks,
-- 
KaiGai Kohei kai...@ak.jp.nec.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Assertion failure on hot standby

2010-11-25 Thread Fujii Masao
On Fri, Nov 26, 2010 at 7:40 AM, Simon Riggs si...@2ndquadrant.com wrote:
 As to solutions, it cannot be acceptable to ignore some locks just
 because an xid has not been assigned.

Even if GetRunningTransactionLocks ignores such a lock, it's eventually
WAL-logged by LogAccessExclusiveLock, isn't it?

 If I understand you correctly, it seems possible to generate an
 AccessExclusiveLock before an xid is assigned, so that its possible to
 log that situation before the transaction assigns an xid slightly later.
 So there's a narrow window where we can generate a lock WAL record with
 xid 0.

Right.

 The sensible resolution is to ensure that all
 AccessExclusiveLocks have an xid assigned prior to them registering
 their proclock.

 That would mean running GetCurrentTransactionId() inside LockAcquire()

 if (lockmode = AccessExclusiveLock 
    locktag-locktag_type == LOCKTAG_RELATION 
    !RecoveryInProgress())
        (void) GetCurrentTransactionId();

s/GetCurrentTransactionId/GetTopTransactionId?

 Any objections to that fix?

Or can we call LogAccessExclusiveLock before registering the lock?

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_execute_from_file review

2010-11-25 Thread Joshua Tolley
On Thu, Nov 25, 2010 at 10:24:51PM +0100, Dimitri Fontaine wrote:
 Joshua Tolley eggyk...@gmail.com writes:
  I've just looked at pg_execute_from_file[1]. The idea here is to execute all
  the SQL commands in a given file. My comments:
 
 Thanks for your review. Please find attached a revised patch where I've
 changed the internals of the function so that it's split in two and that
 the opr_sanity check passes, per comments from David Wheeler and Tom Lane.

I'll take a look ASAP.

  * I'd like to see the docs slightly expanded, specifically to describe
parameter replacement. I wondered for a while if I needed to set of
parameters in any specific way, before reading the code and realizing they
can be whatever I want.
 
 My guess is that you knew that in the CREATE EXTENSION context, it has
 been proposed to use the notation @extschema@ as a placeholder, and
 you've then been confused. I've refrained from imposing any style with
 respect to what the placeholder would look like in the mecanism-patch.
 
 Do we still want to detail in the docs that there's nothing expected
 about the placeholder syntax of format?

Perhaps such docs will show up with the rest of the EXTENSION work, but I'd
like a brief mention somewhere.

  * Does anyone think it needs representation in the test suite?
 
 Well the patch will get its tests with the arrival of the extension main
 patch, where all contribs are installed using it.

Works for me.

  * Shouldn't it include SPI_push() and SPI_pop()?
 
 ENOCLUE

My guess is yes, because that was widely hailed as a good idea when I did
PL/LOLCODE. I suspect it would only matter if someone were using
pg_execute_from_file within some other function, which isn't entirely
unlikely.

--
Joshua Tolley / eggyknap
End Point Corporation
http://www.endpoint.com


signature.asc
Description: Digital signature


Re: [HACKERS] contrib: auth_delay module

2010-11-25 Thread Fujii Masao
On Thu, Nov 25, 2010 at 3:18 PM, KaiGai Kohei kai...@ak.jp.nec.com wrote:
 The attached patch is revised version.

 - Logging part within auth_delay was removed. This module now focuses on
  injection of a few seconds delay on authentication failed.
 - Documentation parts were added like any other contrib modules.

Something like the following is not required? Though I'm not sure
if there is the case where auth_delay is unload.


void
_PG_fini(void)
{
/* Uninstall hooks. */
ClientAuthentication_hook = original_client_auth_hook;
}


+   if (status != STATUS_OK)
+   {
+   sleep(auth_delay_seconds);
+   }

We should use pg_usleep rather than sleep?

+   DefineCustomIntVariable(auth_delay.seconds,
+   Seconds to be delayed 
on authentication failed,
+   NULL,
+   auth_delay_seconds,
+   2,
+   0, INT_MAX,
+   PGC_POSTMASTER,
+   GUC_UNIT_S,
+   NULL,
+   NULL);

Can we relax the context from PGC_POSTMASTER to PGC_SIGHUP?

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SQL/MED - file_fdw

2010-11-25 Thread David Fetter
On Thu, Nov 25, 2010 at 05:51:11PM +0900, Shigeru HANADA wrote:
 On Thu, 25 Nov 2010 17:12:44 +0900
 Shigeru HANADA han...@metrosystems.co.jp wrote:
  Attached is a patch that adds file_fdw, FDW which reads records from
  files on the server side, as a contrib module.  This patch is based on
  SQL/MED core functionality patch.
  
  [SQL/MED - core functionality]
  http://archives.postgresql.org/pgsql-hackers/2010-11/msg01698.php
 
 I'm going to add new CommitFest items for this patch and SQL/MED -
 postgresql_fdw patch which have been split from SQL/MED patch.  Can
 I add them to CF 2010-11 which original SQL/MED item is in?  Or
 should I add them to CF 2011-01?

The original.

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] contrib: auth_delay module

2010-11-25 Thread KaiGai Kohei

(2010/11/26 11:35), Fujii Masao wrote:

On Thu, Nov 25, 2010 at 3:18 PM, KaiGai Koheikai...@ak.jp.nec.com  wrote:

The attached patch is revised version.

- Logging part within auth_delay was removed. This module now focuses on
  injection of a few seconds delay on authentication failed.
- Documentation parts were added like any other contrib modules.


Something like the following is not required? Though I'm not sure
if there is the case where auth_delay is unload.


void
_PG_fini(void)
{
/* Uninstall hooks. */
 ClientAuthentication_hook = original_client_auth_hook;
}



I'm not also sure whether we have situation libraries are unloaded.
Right now, internal_unload_library() is just a placeholder, so
it seems to me _PG_fini() is never invoked.


+   if (status != STATUS_OK)
+   {
+   sleep(auth_delay_seconds);
+   }

We should use pg_usleep rather than sleep?


Indeed, pg_usleep() is mainly used rather than sleep().


+   DefineCustomIntVariable(auth_delay.seconds,
+   Seconds to be delayed on 
authentication failed,
+   NULL,
+   auth_delay_seconds,
+   2,
+   0, INT_MAX,
+   PGC_POSTMASTER,
+   GUC_UNIT_S,
+   NULL,
+   NULL);

Can we relax the context from PGC_POSTMASTER to PGC_SIGHUP?


It seems to me reasonable change.

I'll revise my patch. How about _PG_fini()?

Thanks,
--
KaiGai Kohei kai...@ak.jp.nec.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SQL/MED - file_fdw

2010-11-25 Thread Shigeru HANADA
On Thu, 25 Nov 2010 18:40:09 -0800
David Fetter da...@fetter.org wrote:
 On Thu, Nov 25, 2010 at 05:51:11PM +0900, Shigeru HANADA wrote:
  I'm going to add new CommitFest items for this patch and SQL/MED -
  postgresql_fdw patch which have been split from SQL/MED patch.  Can
  I add them to CF 2010-11 which original SQL/MED item is in?  Or
  should I add them to CF 2011-01?
 
 The original.

Thanks, added them to CF 2010-11.

--
Shigeru Hanada



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: [BUGS] BUG #5650: Postgres service showing as stopped when in fact it is running

2010-11-25 Thread Fujii Masao
On Fri, Nov 26, 2010 at 3:11 AM, Bruce Momjian br...@momjian.us wrote:
 I have applied this patch, with modified wording of the cannot connect
 case:

        $ pg_ctl -w -l /dev/null start
        waiting for server to start done
        server started
        warning:  could not connect, perhaps due to invalid authentication or
        misconfiguration.

This patch breaks the behavior that pg_ctl -w start waits until the standby
has been ready to accept read-only queries. IOW, pg_ctl without this patch
continues to check the connection even if the connection is rejected because
the database has not been consistent yet. But pg_ctl with this patch treats
that rejection as success of the standby starting and prints the above
messages.

I agree to treat the receipt of password request from the server as success
of the server starting. But I don't think that we should treat other rejection
cases that way and change the existing behavior.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-25 Thread Tom Lane
Maciek Sakrejda msakre...@truviso.com writes:
 21:43:02.264 (26)  FE= Describe(statement=S_1)
 You're still doing the statement-flavor Describe. As Tom pointed out,
 this won't tell you the result types because it doesn't know them.
 Actually, technically if you issue a statement-flavor Describe *after*
 a Bind, the server does have this information, but I'm not surprised
 that it doesn't send it correctly, since it seems pointless to send
 the statement variation after already doing a Bind.

In principle you could open more than one Portal off a Statement
at the same time, so it wouldn't necessarily be well-defined anyway.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Assertion failure on hot standby

2010-11-25 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes:
 That would mean running GetCurrentTransactionId() inside LockAcquire()

 if (lockmode = AccessExclusiveLock 
 locktag-locktag_type == LOCKTAG_RELATION 
 !RecoveryInProgress())
   (void) GetCurrentTransactionId();

 Any objections to that fix?

Could we have a wal level test in there too please?  It's pretty awful
in any case...

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-25 Thread Radosław Smogura
Thank you, but I think about this last night. Opening unnecessary portals
isn't good idea, similarly sending 2nd describe when statement was
prepared. Currently JDBC drivers doesn't make this. I think better will be
to store what format we had requested on stack, and then coerce those
formats when results are handled.

Kind regards,
Radosław Smogura

On Fri, 26 Nov 2010 01:02:25 -0500, Tom Lane t...@sss.pgh.pa.us wrote:
 Maciek Sakrejda msakre...@truviso.com writes:
 21:43:02.264 (26)  FE= Describe(statement=S_1)
 You're still doing the statement-flavor Describe. As Tom pointed out,
 this won't tell you the result types because it doesn't know them.
 Actually, technically if you issue a statement-flavor Describe *after*
 a Bind, the server does have this information, but I'm not surprised
 that it doesn't send it correctly, since it seems pointless to send
 the statement variation after already doing a Bind.
 
 In principle you could open more than one Portal off a Statement
 at the same time, so it wouldn't necessarily be well-defined anyway.
 
   regards, tom lane

-- 
--
Radosław Smogura
http://www.softperience.eu

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SQL/MED - core functionality

2010-11-25 Thread Shigeru HANADA
Thanks for the comments.

I'll revise the patch along the discussion.  Before starting code work,
please let me summarize the discussion.

* Generally, we should keep FDWs away from PostgreSQL internals,
such as TupleTableSlot.

* FDW should have planner hook which allows FDW to create FDW-specific
plan (FdwPlan in Heikki's proposal) for a scan on a foreign table.

* FdwPlan, a part of ForeignScan plan node, should be able to be
copied in generic way because plans would be copied into another
memory context during caching.  It might be better to represent
FdwPlan with Node or List.

* FdwExecutionState, a part of ForeignScanState, should be used
instead of ForeignScanState to remove executor details from FDW
implementation.
# ISTM that FdwExecutionState would be replace FdwReply.

Regards,
--
Shigeru Hanada



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers