Re: [GENERAL] LPI-Japan to start PostgreSQL certfication

2011-06-15 Thread Tatsuo Ishii
 Hi,
 
 LPI-Japan, a non-profit distributor of LPIC(Linux Professional
 Institute Certification) in Japan will start OSS-DB exam from July
 1st, 2011. LPI-Japan is known as one of the largest distributor of
 LPIC in the world(according to LPI-Japan they have distributed 164k
 LPIC so far).
 
 http://www.oss-db.jp/news/press/20110608_04.shtml
 
 According to LPI-Japan, OSS-DB will be ready for several open source
 databases in the future. However the initial version will only support
 PostgreSQL(!)
 
 To develop OSS-DB, many companies, including NEC, Hitachi, Fujitsu,
 NTT and SRA OSS, have been working with LPI-Japan.

Correction to this:

Representatives from Fujitsu, Hitach, Miracle Linux, NEC, NEC Soft.,
NTT, and SRA OSS attended the press announcement event.

 
 I hope OSS-DB will significantly contribute to making PostgreSQL more
 popular in Japan.
 --
 Tatsuo Ishii
 SRA OSS, Inc. Japan
 English: http://www.sraoss.co.jp/index_en.php
 Japanese: http://www.sraoss.co.jp
 
 -- 
 Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-general

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] psql \s not working - OS X

2011-06-15 Thread Joel Jacobson
I'm trying the new 9.1b2 release and got a weird problem with the \s
command to show the latest history, for some reason it's trying to
write to /dev/tty instead of printing out to stdout:

Joel-Jacobsons-MacBook-Pro:~ joel$ uname -a
Darwin Joel-Jacobsons-MacBook-Pro.local 10.7.4 Darwin Kernel Version
10.7.4: Mon Apr 18 21:24:17 PDT 2011;
root:xnu-1504.14.12~3/RELEASE_X86_64 x86_64
Joel-Jacobsons-MacBook-Pro:~ joel$ export
declare -x Apple_PubSub_Socket_Render=/tmp/launch-pOTElL/Render
declare -x COMMAND_MODE=unix2003
declare -x DISPLAY=/tmp/launch-QNeAJR/org.x:0
declare -x HOME=/Users/joel
declare -x LC_CTYPE=UTF-8
declare -x LOGNAME=joel
declare -x OLDPWD=/Users/joel/Downloads/postgresql-9.1beta2
declare -x 
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/git/bin:/opt/local/bin:/usr/X11/bin
declare -x PWD=/Users/joel
declare -x SHELL=/bin/bash
declare -x SHLVL=1
declare -x SSH_AUTH_SOCK=/tmp/launch-ejCA6e/Listeners
declare -x TERM=xterm-color
declare -x TERM_PROGRAM=Apple_Terminal
declare -x TERM_PROGRAM_VERSION=273.1
declare -x TMPDIR=/var/folders/FZ/FZOPIjkcF2GR0xFiNEkxME+++TI/-Tmp-/
declare -x USER=joel
declare -x __CF_USER_TEXT_ENCODING=0x1F5:0:0
Joel-Jacobsons-MacBook-Pro:~ joel$ /Library/PostgreSQL/9.1b2/bin/psql glue
psql (9.1beta2)
Type help for help.

glue=# SELECT version();

version
-
 PostgreSQL 9.1beta2 on x86_64-apple-darwin10.7.4, compiled by
i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5666) (dot
3), 64-bit
(1 row)

glue=# \s
could not save history to file /dev/tty: Operation not permitted

glue=#

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] random backend crashes - Needed Information included

2011-06-15 Thread Craig Ringer

On 06/15/2011 05:52 PM, BangarRaju Vadapalli wrote:


7. Attached the database side and application side logs.


Yeah, there's definitely a crash, you just chopped it off in the 
abbreviated logs you sent earlier. With this kind of log volume that's 
easy enough to do.


Anyway, the crash is:

2011-06-15 13:55:59 IST postgres epimart PANIC:  XX000: cannot abort 
transaction 19773146, it was already committed


from:

2011-06-15 13:55:59 IST postgres epimart ERROR:  XX000: could not open 
relation base/2850136/3344343_vm: A blocking operation was interrupted 
by a call to WSACancelBlockingCall.^M
2011-06-15 13:55:59 IST postgres epimart LOCATION:  mdopen, 
.\src\backend\storage\smgr\md.c:526

2011-06-15 13:55:59 IST postgres epimart STATEMENT:  COMMIT
2011-06-15 13:55:59 IST postgres epimeta DEBUG:  0: CommitTransaction
2011-06-15 13:55:59 IST postgres epimeta LOCATION: 
ShowTransactionState, .\src\backend\access\transam\xact.c:4074
2011-06-15 13:55:59 IST postgres epimart WARNING:  01000: 
AbortTransaction while in COMMIT state
2011-06-15 13:55:59 IST postgres epimart LOCATION:  AbortTransaction, 
.\src\backend\access\transam\xact.c:2011

2011-06-15 13:55:59 IST postgres epimart DEBUG:  0: StartTransaction
2011-06-15 13:55:59 IST postgres epimart LOCATION: 
ShowTransactionState, .\src\backend\access\transam\xact.c:4074
2011-06-15 13:55:59 IST postgres epimart PANIC:  XX000: cannot abort 
transaction 19773146, it was already committed
2011-06-15 13:55:59 IST postgres epimart LOCATION: 
RecordTransactionAbort, .\src\backend\access\transam\xact.c:1200



While I recall some known issues that resulted in panics because of 
cannot abort transaction errors, I seem to recall that they were 
related to autovacuum, which this doesn't particularly seem to be.


I'd still recommend updating to 8.4.8, as 8.4.2 was patched six more 
times for good reasons.


--
Craig Ringer

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] Setting up tablepace

2011-06-15 Thread Malm Paul
Hi all,
I have a problem with setting up tablespace on a ramdisk.
the ramdisk shall be used temporary for storing unprotected map data, that is 
normaly protected.

I'm working in Linux.
I have an ordirnary Linux user called normaluser, a user called chartuser (the 
ramdisk owner, the password is not known to the normal users), and a ordinary 
postgres user.

Chartuser and root are the only ones that has access to the ramdisk.

I have succeeded to store (as normaluser) the unprotected map files on the 
ramdisk, only accessable for chartuser.

I would now like to import the map files to a GIS db that has the tabespace on 
the ramdisk. But I'm not able to create the tablespace on the ramdisk.
Trying logged in as chartuser to run: psql -c CREATE TABLESPACE ramspc 
LOCATION '/tmp/ramdisk0/tblspace'
 could not set permission on directory /tmp/ramdisk0/tblspace: premission 
denied...

Any ideas, anyone?
Thanks,
Paul


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Cursors

2011-06-15 Thread Merlin Moncure
On Tue, Jun 14, 2011 at 11:54 PM, Andy Chambers achamb...@mcna.net wrote:
 Hi,

 What happens to cursors when new data is added to a table after you
 start iterating
 over its rows?

 For example, given the following loop...

 for rule in select tc.sid, tc.s, td.rule, td.returns
                     from tcell tc
               inner join tcelldef td on (tc.p = td.p)
                    where tc.iasid = current_audit_sid()
                       or committed_sid in ( select committed
                                               from tcellread tcr
                                              where tc.sid = tcr.tcell )
                         for update of tc loop
  ...
 end loop;

 some code in the loop might add a record into tcellread that causes the where
 condition to become true for a row in which it was previously false.
 Will the cursor
 eventually see it?

nope!

merlin

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] configure error... please help 9.0.4

2011-06-15 Thread akp geek
Thanks all for the responses. Based on your suggestion, I could able to
configure it .  The way I fixed it was, I configured the uuid, readline and
flex into the same path, then ran the configure command for postgres

./configure --prefix=/opt/postgres/9.0.4 --with-perl --with-libxml
--with-openssl  --with-readline --with-ossp-uuid
--with-includes=/opt/postgres/software/include
LDFLAGS=-L/opt/postgres/software/lib


every thing went well from there , gmake and gmake install


Regards



On Tue, Jun 14, 2011 at 9:48 AM, Adrian Klaver adrian.kla...@gmail.comwrote:

 On Tuesday, June 14, 2011 5:51:59 am akp geek wrote:
  Thanks all for the responses.
 
 
 
  $./configure --prefix=/opt/postgres/9.0.4 --with-readline --with-perl
  --with-libxml --with-openssl --with-ossp-uuid
  LDFLAGS=-L/opt/postgres/gis/lib
  --with-includes=/opt/postgres/software/include/readline/
 
  *I bypassed the uuid error.  Now getting this  .. I have installed
  readline. I have this under /opt/postgres/software/lib, include ,share *

 Did you install readline-dev also?

 
  *checking readline/readline.h usability... no*
  *checking readline/readline.h presence... no*
  *checking for readline/readline.h... no*
  *checking readline.h usability... no*
  *checking readline.h presence... no*
  *checking for readline.h... no*
  *configure: error: readline header not found*
  *If you have readline already installed, see config.log for details on
 the*
  *failure.  It is possible the compiler isn't looking in the proper
  directory.*
  *Use --without-readline to disable readline support.*
 
 

 --
 Adrian Klaver
 adrian.kla...@gmail.com



Re: [GENERAL] Cursors

2011-06-15 Thread Grzegorz Jaśkiewicz
Cursors only see the data that is the effect of the query. That output
doesn't get updated. It would actually be pretty bad if that was the
case.

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Out of tree build issue

2011-06-15 Thread pedz
This appears to be an issue with GNU Make 3.82.  I've opened a bug for postgres 
because you might want to figure out a work around.  I also sent it to the make 
bug report.

The postgresql bug report is 
http://archives.postgresql.org/pgsql-bugs/2011-06/msg00109.php

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] Encryption For Specific Column- Where to store the key

2011-06-15 Thread Manuel Gysin
Hello


I'm currently designing a database layout where some columns are encrypted.
Some tables contains sensitive user data which needs a special protection.

I used http://www.postgresql.org/docs/8.1/static/encryption-options.html as a 
guide.

- For the password field I just used a hash algorithm with some loops to 
protect the passwords (Password Storage Encryption with bcrypt).
- For the sensitive columns I used Encryption For Specific Columns, here I 
have later some questions.
- For general data encryption I used luks (crypsetup) Data Partition 
Encryption
- Connection is secured like desc. in Encrypting Data Across A Network with 
SSL Host Authentication

Much attack use cases are covered with this but I see one problem:

1. There is a frontend (webserver) and a backend (database)
   - backend must be configured to not allow to much queries in a given time, 
else there is a possibility to get around the whole security stuff
   - frontend needs too some protection against brute force
2. When encrypting some columns I need to save somewhere the key.
   - Frontend (very bad idea, first point of failure)
   - Backend (when someone can dump the database, he got the key too, 
encryption is in this use case useless)
   - Remote database (when someone can hack to the first db, it's not far away 
to the second db I think, but there is more time to register an attack and 
force shutdown everything)
   - Write an dedicated application (when someone hacked this server, it's only 
a matter of time before he can find out where the key is stored in the RAM)

So it seems there is no protection when someone gained access to the database 
server. Or is there a way? I can't see any.
I'm not fit enough in attack a database server, but I think when someone has 
access to the database, he can simply dump the whole tables, while the key is 
stored in the table, he has full access to everything in the database. At the 
end the question is, where and how I should store the key to decrypt the 
columns?

A discussion about this topic can be found under 
http://www.experts-exchange.com/Database/PostgreSQL/Q_21934798.html (answers 
are not all the time displayed...)
But there were no final solution at all.


Kind regards
Manuel Gysin

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Encryption For Specific Column- Where to store the key

2011-06-15 Thread Pavel Stehule
Hello

try to use a security definer functions

http://www.postgresql.org/docs/current/static/sql-createfunction.html

inside this function you can access to resourcess thats are no
available from outer for web user

Regards

Pavel Stehuke

2011/6/15 Manuel Gysin manuel.gy...@quantum-bytes.com:
 Hello


 I'm currently designing a database layout where some columns are encrypted.
 Some tables contains sensitive user data which needs a special protection.

 I used http://www.postgresql.org/docs/8.1/static/encryption-options.html as a 
 guide.

 - For the password field I just used a hash algorithm with some loops to 
 protect the passwords (Password Storage Encryption with bcrypt).
 - For the sensitive columns I used Encryption For Specific Columns, here I 
 have later some questions.
 - For general data encryption I used luks (crypsetup) Data Partition 
 Encryption
 - Connection is secured like desc. in Encrypting Data Across A Network with 
 SSL Host Authentication

 Much attack use cases are covered with this but I see one problem:

 1. There is a frontend (webserver) and a backend (database)
   - backend must be configured to not allow to much queries in a given time, 
 else there is a possibility to get around the whole security stuff
   - frontend needs too some protection against brute force
 2. When encrypting some columns I need to save somewhere the key.
   - Frontend (very bad idea, first point of failure)
   - Backend (when someone can dump the database, he got the key too, 
 encryption is in this use case useless)
   - Remote database (when someone can hack to the first db, it's not far away 
 to the second db I think, but there is more time to register an attack and 
 force shutdown everything)
   - Write an dedicated application (when someone hacked this server, it's 
 only a matter of time before he can find out where the key is stored in the 
 RAM)

 So it seems there is no protection when someone gained access to the database 
 server. Or is there a way? I can't see any.
 I'm not fit enough in attack a database server, but I think when someone has 
 access to the database, he can simply dump the whole tables, while the key is 
 stored in the table, he has full access to everything in the database. At the 
 end the question is, where and how I should store the key to decrypt the 
 columns?

 A discussion about this topic can be found under 
 http://www.experts-exchange.com/Database/PostgreSQL/Q_21934798.html (answers 
 are not all the time displayed...)
 But there were no final solution at all.


 Kind regards
 Manuel Gysin

 --
 Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-general


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Encryption For Specific Column- Where to store the key

2011-06-15 Thread Merlin Moncure
On Wed, Jun 15, 2011 at 1:07 AM, Manuel Gysin
manuel.gy...@quantum-bytes.com wrote:
 Hello
 I'm currently designing a database layout where some columns are encrypted.
 Some tables contains sensitive user data which needs a special protection.

 I used http://www.postgresql.org/docs/8.1/static/encryption-options.html as a 
 guide.

 - For the password field I just used a hash algorithm with some loops to 
 protect the passwords (Password Storage Encryption with bcrypt).
 - For the sensitive columns I used Encryption For Specific Columns, here I 
 have later some questions.
 - For general data encryption I used luks (crypsetup) Data Partition 
 Encryption
 - Connection is secured like desc. in Encrypting Data Across A Network with 
 SSL Host Authentication

 Much attack use cases are covered with this but I see one problem:

 1. There is a frontend (webserver) and a backend (database)
   - backend must be configured to not allow to much queries in a given time, 
 else there is a possibility to get around the whole security stuff
   - frontend needs too some protection against brute force
 2. When encrypting some columns I need to save somewhere the key.
   - Frontend (very bad idea, first point of failure)
   - Backend (when someone can dump the database, he got the key too, 
 encryption is in this use case useless)
   - Remote database (when someone can hack to the first db, it's not far away 
 to the second db I think, but there is more time to register an attack and 
 force shutdown everything)
   - Write an dedicated application (when someone hacked this server, it's 
 only a matter of time before he can find out where the key is stored in the 
 RAM)

 So it seems there is no protection when someone gained access to the database 
 server. Or is there a way? I can't see any.
 I'm not fit enough in attack a database server, but I think when someone has 
 access to the database, he can simply dump the whole tables, while the key is 
 stored in the table, he has full access to everything in the database. At the 
 end the question is, where and how I should store the key to decrypt the 
 columns?

 A discussion about this topic can be found under 
 http://www.experts-exchange.com/Database/PostgreSQL/Q_21934798.html (answers 
 are not all the time displayed...)
 But there were no final solution at all.

securing the backend from the dba is basically impossible.  you can
make the client pretty secure, but the only way your encryption can be
reasonably enforced is for both the encryption and decryption to
happen on the client side -- the key cannot and should not be
possessed by anyone who is not trustworthy.

merlin

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Setting up tablepace

2011-06-15 Thread John R Pierce

On 06/15/11 3:34 AM, Malm Paul wrote:

Hi all,
I have a problem with setting up tablespace on a ramdisk.
the ramdisk shall be used temporary for storing unprotected map data, that is 
normaly protected.

I'm working in Linux.
I have an ordirnary Linux user called normaluser, a user called chartuser (the 
ramdisk owner, the password is not known to the normal users), and a ordinary 
postgres user.

Chartuser and root are the only ones that has access to the ramdisk.

I have succeeded to store (as normaluser) the unprotected map files on the 
ramdisk, only accessable for chartuser.

I would now like to import the map files to a GIS db that has the tabespace on 
the ramdisk. But I'm not able to create the tablespace on the ramdisk.
Trying logged in as chartuser to run: psql -c CREATE TABLESPACE ramspc LOCATION 
'/tmp/ramdisk0/tblspace'
  could not set permission on directory /tmp/ramdisk0/tblspace: premission 
denied...



I hope you drop this tablespace and everything in it before 
rebooting...  and that your system never reboots unexpectedly



that said, its the postgres server process that needs write access to 
create the tablespace.  I'd create a directory in the ramdisk called 
pgsql or something, chown postgres  chmod 700, then put your 
tablespace in that.




--
john r pierceN 37, W 122
santa cruz ca mid-left coast


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] blks_read/blks_hit stats

2011-06-15 Thread Nigel Heron

Hi everyone,

I'm playing with the stats views and functions to graph them in cacti..
Adding up *_blks_hit (heap, idx, toast and tidx) from pg_statio doesn't 
match blks_hit in pg_stat_database.
Sometimes the sum is higher, sometimes lower. Do they have similar names 
but represent different metrics?

Same issue with blks_read.

eg.
SELECT
SUM(pg_statio_all_tables.heap_blks_hit)::bigint +
SUM(pg_statio_all_tables.idx_blks_hit)::bigint +
SUM(pg_statio_all_tables.toast_blks_hit)::bigint +
SUM(pg_statio_all_tables.tidx_blks_hit)::bigint AS blks_hit
FROM pg_statio_all_tables;
  blks_hit

 1275299563
(1 row)


SELECT blks_hit
FROM pg_stat_database
where datname='mydb';
  blks_hit
---
 674295210
(1 row)


here's 2 graphs from different databases on the same cluster (8.4.2).
first 4 stacked graph items are from pg_statio_all_tables and the red 
line is from pg_stat_database.


blks_hit is way under the sum:
http://www.psycode.com/gallery/d/88438-1/blks_read1.png
blks_hit seems pretty close to the sum of table+idx (but no toast):
http://www.psycode.com/gallery/d/88440-1/blks_read2.png


thanks,
-nigel.


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] pgadmin Running VACUUM recommended

2011-06-15 Thread Anibal David Acosta
Hello, I have installed postgres 9.0 that is supposed has by default auto
vacuum enabled

 

But sometimes when I click over a table in the pgadmin, this application
recommend me to vacuum 

The message is:

The estimated rowcount on the table XXX deviates significantly from the
actual rowcount. You should run VACUUM ANALYZE on this table.

 

Why this happened? The auto vacuum is not doing a good job?

 

Thanks



Re: [GENERAL] pgadmin Running VACUUM recommended

2011-06-15 Thread Guillaume Lelarge
On Wed, 2011-06-15 at 16:01 -0400, Anibal David Acosta wrote:
 Hello, I have installed postgres 9.0 that is supposed has by default
 auto vacuum enabled
 
  
 
 But sometimes when I click over a table in the pgadmin, this
 application recommend me to vacuum 
 
 The message is:
 
 “The estimated rowcount on the table XXX deviates significantly from
 the actual rowcount. You should run VACUUM ANALYZE on this table.
 
  
 
 Why this happened? The auto vacuum is not doing a good job?
 

Nope. pgAdmin detects that with a really simple algorithm. Not sure it's
still a good one. There are better chances pgAdmin is wrong than
autovacuum.


-- 
Guillaume
  http://blog.guillaume.lelarge.info
  http://www.dalibo.com


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] You could be a PostgreSQL Patch Reviewer!

2011-06-15 Thread Josh Berkus
PostgreSQL community members:

Do you love the PostgreSQL project?  Do you want to contribute to it?
Do you want to help produce the next version of PostgreSQL? (9.2)

Well, you *can*.  You can be a patch reviewer -- one of the single most
valuable things you can contribute to the project.

You do not have to be a C coder to be a patch reviewer.  Pretty much all
you need to know is:
- how to checkout PostgreSQL from Git
- how to build PostgreSQL from source
- how to apply a patch

If you know those three things, you can help with patch review.  Of
course, if you do know C, you can be even more help ... and learn the
PostgreSQL source in the process.

We especially need folks who are able to build PostgreSQL on Windows, as
we have several Windows-specific patches and no reviewers for them.

The First CommitFest for version 9.2 is underway *right now* and we need
your help. Help PostgreSQL ... review a patch!

http://wiki.postgresql.org/wiki/Reviewing_a_Patch

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] You could be a PostgreSQL Patch Reviewer!

2011-06-15 Thread Merlin Moncure
On Wed, Jun 15, 2011 at 3:49 PM, Josh Berkus j...@agliodbs.com wrote:
 PostgreSQL community members:

 Do you love the PostgreSQL project?  Do you want to contribute to it?
 Do you want to help produce the next version of PostgreSQL? (9.2)

 Well, you *can*.  You can be a patch reviewer -- one of the single most
 valuable things you can contribute to the project.

 You do not have to be a C coder to be a patch reviewer.  Pretty much all
 you need to know is:
 - how to checkout PostgreSQL from Git
 - how to build PostgreSQL from source
 - how to apply a patch

 If you know those three things, you can help with patch review.  Of
 course, if you do know C, you can be even more help ... and learn the
 PostgreSQL source in the process.

 We especially need folks who are able to build PostgreSQL on Windows, as
 we have several Windows-specific patches and no reviewers for them.

 The First CommitFest for version 9.2 is underway *right now* and we need
 your help. Help PostgreSQL ... review a patch!

 http://wiki.postgresql.org/wiki/Reviewing_a_Patch

This sounds suspiciously like you are trying to get me to join the army...

merlin

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] Invalid byte sequence for encoding UTF8: 0xedbebf

2011-06-15 Thread BRUSSER Michael
This is a follow-up on my previous message 
http://archives.postgresql.org/pgsql-general/2011-06/msg00054.php

I think I have now some understanding of what's causing the problem, but I 
don't have a good solution, instead more questions.
The release notes for v8.1 at 
http://www.postgresql.org/docs/current/interactive/release-8-1.html
make a good suggestion on using iconv to convert the plain-text dump file into 
utf8
On Linux this did not work, the input and output file were identical. The iconv 
on Solaris  refused to open the input file
(probably too big),  although it worked with a chunk of it and reported 
conversion error.

Unless there's no other options I don't want to use sed or break file into 
pieces, if possible, I would prefer to identify the bad records on the database.
I tried SELECT with everything  I could think of:  ~*, SIMILAR TO, and the 
likes of them, but I never got it right.

Is there a way to find the records with the text field containing Unicode bytes 
0xedbebf?
Unfortunately this is a very old version 7.3.10

Thank you.
Michael.

This email and any attachments are intended solely for the use of the 
individual or entity to whom it is addressed and may be confidential and/or 
privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email 
and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or 
responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer


Re: [GENERAL] Invalid byte sequence for encoding UTF8: 0xedbebf

2011-06-15 Thread Alan Hodgson
On June 15, 2011 01:18:27 PM BRUSSER Michael wrote:
 Unless there's no other options I don't want to use sed or break file into
 pieces, if possible,

iconv loads everything into RAM. You can use split, convert the pieces, and 
then recombine, I did that when converting a large database to utf-8 and it 
worked.

-- 
Obama has now fired more cruise missiles than all other Nobel Peace prize 
winners combined.

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Invalid byte sequence for encoding UTF8: 0xedbebf

2011-06-15 Thread BRUSSER Michael
-Original Message-
From: pgsql-general-ow...@postgresql.org 
[mailto:pgsql-general-ow...@postgresql.org] On Behalf Of Alan Hodgson
Sent: Wednesday, June 15, 2011 5:37 PM
To: pgsql-general@postgresql.org
Subject: Re: [GENERAL] Invalid byte sequence for encoding UTF8: 0xedbebf

On June 15, 2011 01:18:27 PM BRUSSER Michael wrote:
 Unless there's no other options I don't want to use sed or break file into
 pieces, if possible,

--

 iconv loads everything into RAM. You can use split, convert the pieces, and
 then recombine, I did that when converting a large database to utf-8 and it
 worked.

-

- Thanks, but this is exactly what I am trying to avoid!
Using split is good if you have one database to upgrade and no external 
customers.
(Not to mention other problems with this approach)

This email and any attachments are intended solely for the use of the 
individual or entity to whom it is addressed and may be confidential and/or 
privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email 
and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or 
responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] { SELECT *-NOT(column1, column2) FROM table } syntax idea

2011-06-15 Thread David Johnston
Is there, or has there ever been, a discussion about introducing syntax to
handle specifying which columns you do NOT want to output in the SELECT
list?

 

The use case I am running into is mostly within VIEWS.  I want to specify
SELECT * FROM table but there are a couple of fields that I do NOT want to
output (for instance a password hash field for a user table).  I guess it
would probably be better form to move those columns to a separate enhanced
permissions table but since PostgreSQL allows for per-column permissions
that is not strictly necessary.  Listing all the wanted columns is not
desirable though obviously possible.  The main reason to avoid doing so is
to allow for a view to output all the columns of the underlying tables.  If
I drop/create the view after altering the underlying tables the new view
will have the additional columns without any direct change to the view being
required.

 

David J.

 

 

 



[GENERAL] = ANY (SELECT ..) and type casts, what's going on here?

2011-06-15 Thread Russell Smith
Hi,

Is anybody able to explain the following behaviour?

Server is 8.4.7  RHEL5 build.  Also happens on 8.4.8 Ubuntu x64 package.

mr-russ=# SELECT 'BU'::varchar = ANY ((select '{BU,CI}'::varchar[]));
ERROR:  operator does not exist: character varying = character varying[]
LINE 1: SELECT 'BU'::varchar = ANY ((select '{BU,CI}'::varchar[]));
 ^
HINT:  No operator matches the given name and argument type(s). You might need 
to add explicit type casts.
mr-russ=# SELECT 'BU'::varchar = ANY ((select '{BU,CI}'::varchar[])::varchar[]);
 ?column?
--
 t
(1 row)

mr-russ=#


What I don't understand is what happens to the single SELECT's type, is it 
because select returns a row?  The error doesn't seem to match what I would 
expect?

Thanks

Russell

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] Further details on cursors.

2011-06-15 Thread Matthew A. R. Sherian
I am running into an odd problem when fetching from multiple cursors in
Perl. However, I do not feel knowledgeable enough to ask an intelligent
question.
Can you in the community direct me at some deeper documentation. Perhaps a
developers guide (for Postgres itself), that might discuss the internal
handling
of  cursors, queries prepared within the cursor and what happens when the
end of a block of fetched data is reached.


Re: [GENERAL] { SELECT *-NOT(column1, column2) FROM table } syntax idea

2011-06-15 Thread Derrick Rice
On Wed, Jun 15, 2011 at 6:08 PM, David Johnston pol...@yahoo.com wrote:

   The main reason to avoid doing so is to allow for a view to output all
 the columns of the underlying tables.  If I drop/create the view after
 altering the underlying tables the new view will have the additional columns
 without any direct change to the view being required.



 David J.


Even with CREATE VIEW foo AS SELECT * FROM bar you do not get this
behavior.  Try that, then use \dv foo and you'll see that it has expanded
the set of columns at CREATE VIEW time.  It will not get any new columns you
add to the underlying table.

(tested on 8.4)

So this is a deeper issue than just being able to exclude certain tables.

Derrick


Re: [GENERAL] { SELECT *-NOT(column1, column2) FROM table } syntax idea

2011-06-15 Thread David Johnston

On Wed, Jun 15, 2011 at 6:08 PM, David Johnston pol...@yahoo.com wrote:
  The main reason to avoid doing so is to allow for a view to output all
the columns of the underlying tables.  If I drop/create the view after
altering the underlying tables the new view will have the additional columns
without any direct change to the view being required.
 
David J.

Even with CREATE VIEW foo AS SELECT * FROM bar you do not get this
behavior.  Try that, then use \dv foo and you'll see that it has expanded
the set of columns at CREATE VIEW time.  It will not get any new columns you
add to the underlying table.
 (tested on 8.4)
So this is a deeper issue than just being able to exclude certain tables.
Derrick

First: I intentionally said (DROP/CREATE the VIEW) because of this fact -
and that does not bother me that much; but I'd rather not have to change the
view definition in addition to dropping and recreating it.

What I am basically requesting is that the rewriter that handles evaluation
of  *  within a Select List be able to be told that specific columns, by
name, are not to be included in the resulting expansion.  In the case where
the name would be ambiguous neither/none of the fields would be output -
though if you prefix the column with the table you could just exclude that
specific column.  This would help when you want to use:

SELECT *
FROM t1
JOIN t2 ON (t1.id = t2.id)

And you do not want a duplicate (and auto-named) id column.  You can just
do:

SELECT *-NOT(t2.id)
FROM t1 
JOIN t2 ON (t1.id = t2.id)

And only the id field from column 1 will display

David J.





-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] { SELECT *-NOT(column1, column2) FROM table } syntax idea

2011-06-15 Thread Susan Cassidy
Wouldn't it be easy enough to write a little program to suck in the column 
names from the information schema, and output the CREATE VIEW statement, 
excluding all the columns you want to exclude?

Then, if the tables have changed, just run the program, let it fetch the 
information from the information schema, do the DROP and CREATE, and you're 
done.

Susan C.


From: pgsql-general-ow...@postgresql.org 
[mailto:pgsql-general-ow...@postgresql.org] On Behalf Of David Johnston
Sent: Wednesday, June 15, 2011 3:08 PM
To: pgsql-general@postgresql.org
Subject: [GENERAL] { SELECT *-NOT(column1, column2) FROM table } syntax idea

Is there, or has there ever been, a discussion about introducing syntax to 
handle specifying which columns you do NOT want to output in the SELECT list?

The use case I am running into is mostly within VIEWS.  I want to specify 
SELECT * FROM table but there are a couple of fields that I do NOT want to 
output (for instance a password hash field for a user table).  I guess it would 
probably be better form to move those columns to a separate enhanced 
permissions table but since PostgreSQL allows for per-column permissions that 
is not strictly necessary.  Listing all the wanted columns is not desirable 
though obviously possible.  The main reason to avoid doing so is to allow for a 
view to output all the columns of the underlying tables.  If I drop/create the 
view after altering the underlying tables the new view will have the additional 
columns without any direct change to the view being required.

David J.





Re: [GENERAL] { SELECT *-NOT(column1, column2) FROM table } syntax idea

2011-06-15 Thread David Johnston
In this particular use-case that is doable; but the general idea of wanting
to exclude specific columns from an output occurs in different areas
including adhoc queries.

 

I'm not saying this feature is going to compete with sliced bread for
coolness but more than once I've desired this ability - both adhoc and now
as I am building more infrastructure around my database creation.  SQL is a
declarative language and the idea of Include everything except X, Y, Z is
not a new concept.  Having to resort to a procedural language to implement
what could be a simple declarative syntax construct seems overkill.

 

I wouldn't even care if you silently ignore an attempt to specify a
non-present column for exclusion...if I see a column that I didn't want
explicitly the first (and pretty much only) thing to check will be spelling.


 

I'm just curious if anyone else has considered this and/or whether the
implementers have difficulty concerns that make inhibit implementation or
whether it is more the fact that the demand is too low to spend the effort.

 

David J.

 

 

From: Susan Cassidy [mailto:scass...@edgewave.com] 
Sent: Wednesday, June 15, 2011 7:15 PM
To: David Johnston; pgsql-general@postgresql.org
Subject: RE: [GENERAL] { SELECT *-NOT(column1, column2) FROM table } syntax
idea

 

Wouldn't it be easy enough to write a little program to suck in the column
names from the information schema, and output the CREATE VIEW statement,
excluding all the columns you want to exclude?

 

Then, if the tables have changed, just run the program, let it fetch the
information from the information schema, do the DROP and CREATE, and you're
done.

 

Susan C.

 

  _  

From: pgsql-general-ow...@postgresql.org
[mailto:pgsql-general-ow...@postgresql.org] On Behalf Of David Johnston
Sent: Wednesday, June 15, 2011 3:08 PM
To: pgsql-general@postgresql.org
Subject: [GENERAL] { SELECT *-NOT(column1, column2) FROM table } syntax
idea

 

Is there, or has there ever been, a discussion about introducing syntax to
handle specifying which columns you do NOT want to output in the SELECT
list?

 

The use case I am running into is mostly within VIEWS.  I want to specify
SELECT * FROM table but there are a couple of fields that I do NOT want to
output (for instance a password hash field for a user table).  I guess it
would probably be better form to move those columns to a separate enhanced
permissions table but since PostgreSQL allows for per-column permissions
that is not strictly necessary.  Listing all the wanted columns is not
desirable though obviously possible.  The main reason to avoid doing so is
to allow for a view to output all the columns of the underlying tables.  If
I drop/create the view after altering the underlying tables the new view
will have the additional columns without any direct change to the view being
required.

 

David J.

 

 

 



Re: [GENERAL] = ANY (SELECT ..) and type casts, what's going on here?

2011-06-15 Thread Tom Lane
Russell Smith mr-r...@pws.com.au writes:
 Is anybody able to explain the following behaviour?

 mr-russ=# SELECT 'BU'::varchar = ANY ((select '{BU,CI}'::varchar[]));
 ERROR:  operator does not exist: character varying = character varying[]
 LINE 1: SELECT 'BU'::varchar = ANY ((select '{BU,CI}'::varchar[]));
  ^
 HINT:  No operator matches the given name and argument type(s). You might 
 need to add explicit type casts.
 mr-russ=# SELECT 'BU'::varchar = ANY ((select 
 '{BU,CI}'::varchar[])::varchar[]);
  ?column?
 --
  t
 (1 row)

For ANY (or ALL) with a sub-select, the sub-select is expected to return
rows, and the left-hand value is compared to each row's contained value.
This is required behavior per SQL standard.

If the right-hand argument of ANY/ALL is *not* a sub-select, then it's
expected to be an expression yielding an array value, and  the left-hand
value is compared to each array element.  AFAIR, this is not in the SQL
standard but is a Postgres extension.

In your second example, the RHS is a cast expression, not directly a
sub-select, so it behaves as per the second rule.  The sub-select
embedded within it doesn't count.

There isn't any provision for ANY/ALL with a sub-select returning a
series of array values; that would require iteration in two
dimensions, and we don't do that.  It would be contrary to spec in any
case, I think, and would break existing use cases where the ANY/ALL
operator is one that takes a scalar on the left and an array on the
right.

regards, tom lane

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] Symbols and versioning of binary releases; running a symbol server

2011-06-15 Thread Craig Ringer

Hi (EnterpriseDB) folks

I've been working with someone off list to get some information about a 
crash they encounter during a batch run. We're generating a crash dump, 
but I'm having some issues getting matching symbols so I can examine it.


One thing that would help with this would be if the EnterpriseDB 
releases included their build revision in the output of SELECT 
version(), so it's always clear exactly what build is in use.


I've also noticed in this process that the File version on 
postgres.exe bears no apparent relationship to the EnterpriseDB release 
number. For example, postgresql 8.4.2-2 has a File Version of 8.4.2-104 
while 8.4.2-1 has a file version of (IIRC) 8.4.2-9343 . Is there any way 
that can be improved?


It's always possible to get the user to send their symbols directory, or 
to just debug it locally using windbg.exe, but it'd be really nice if it 
were easier to reliably match releases to symbol sets.



Even better would be to put zipped symbols directories onto the EDB 
download site, arranged by Pg version. Bonus points for having symlinks 
from the md5sum of postgres.exe to the matching symbols. Better again 
would be to run a public symbol server with symbols for all builds 
EnterpriseDB releases:


http://chadaustin.me/2009/03/reporting-crashes-in-imvu-creating-your-very-own-symbol-server/

... so there's no need to play version guessing games, you just point 
your debugger at the symbol server and it fetches what it needs on demand.


Come to think of it, I can probably run a public symbol server myself if 
the EDB folks don't want to, but it'd be lovely if they were willing to 
do so because it could be integrated into the release process to ensure 
symbols were never missing for a build that hit public release.


--
Craig Ringer

Tech-related writing at http://soapyfrogs.blogspot.com/

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] You could be a PostgreSQL Patch Reviewer!

2011-06-15 Thread sunpeng
Cool, I hope I could be a member of patch reviewers.
peng sun

On Thu, Jun 16, 2011 at 4:58 AM, Merlin Moncure mmonc...@gmail.com wrote:

 On Wed, Jun 15, 2011 at 3:49 PM, Josh Berkus j...@agliodbs.com wrote:
  PostgreSQL community members:
 
  Do you love the PostgreSQL project?  Do you want to contribute to it?
  Do you want to help produce the next version of PostgreSQL? (9.2)
 
  Well, you *can*.  You can be a patch reviewer -- one of the single most
  valuable things you can contribute to the project.
 
  You do not have to be a C coder to be a patch reviewer.  Pretty much all
  you need to know is:
  - how to checkout PostgreSQL from Git
  - how to build PostgreSQL from source
  - how to apply a patch
 
  If you know those three things, you can help with patch review.  Of
  course, if you do know C, you can be even more help ... and learn the
  PostgreSQL source in the process.
 
  We especially need folks who are able to build PostgreSQL on Windows, as
  we have several Windows-specific patches and no reviewers for them.
 
  The First CommitFest for version 9.2 is underway *right now* and we need
  your help. Help PostgreSQL ... review a patch!
 
  http://wiki.postgresql.org/wiki/Reviewing_a_Patch

 This sounds suspiciously like you are trying to get me to join the army...

 merlin

 --
 Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-general



Re: [GENERAL] Encryption For Specific Column- Where to store the key

2011-06-15 Thread Craig Ringer
On 15/06/11 14:07, Manuel Gysin wrote:

 - For the password field I just used a hash algorithm with some loops to 
 protect the passwords (Password Storage Encryption with bcrypt).

Make sure you use a salt value and store the salt as well as the
password. If not salted, your passwords will be easily cracked with a
rainbow table if someone manages to dump them.

 1. There is a frontend (webserver) and a backend (database)
- backend must be configured to not allow to much queries in a given time, 
 else there is a possibility to get around the whole security stuff

You won't have much luck with that. It only takes one SELECT * FROM
... to bypass your query rate limiting. You could force everything
through stored procedures, but that'll be slow and clumsy.

Good intrusion detection and system monitoring so that you detect
unusual events is probably a safter bet than trying to make your system
fail when load spikes.

- frontend needs too some protection against brute force

Yep. There you can do per-IP or per-range rate limiting, among other
things, which will help. I'd probably want to do this using an intrusion
detection/prevention system probably running as a reverse http proxy.

 2. When encrypting some columns I need to save somewhere the key.
- Frontend (very bad idea, first point of failure)
- Backend (when someone can dump the database, he got the key too, 
 encryption is in this use case useless)
- Remote database (when someone can hack to the first db, it's not far 
 away to the second db I think, but there is more time to register an attack 
 and force shutdown everything)
- Write an dedicated application (when someone hacked this server, it's 
 only a matter of time before he can find out where the key is stored in the 
 RAM)
 
 So it seems there is no protection when someone gained access to the database 
 server. Or is there a way? I can't see any.

Not much, no. The main benefit to encrypting some data in the database
is to make it harder to use a stolen dump or extracted content. It can't
make it impossible so long as your system can use the content too.

I'd certainly avoid keeping the key in the database.

If you don't need the capability for unattended web front-end re-start,
you can have your web frontend store the key encrypted on disk and
require interactive password entry to decrypt the key before it can
start. It can then keep the key in RAM and forget the password. If your
frontend is utterly compromised and someone has the time to do the
analysis you're still busted, but it'll slow them down a bunch. If you
want to be viewed as a frothing paranoid you can even store the
encrypted key on the web frontend's HDD but on a separate SD card or
thumb drive that requires physical insertion. I wouldn't.

Personally, I'd probably just keep the key on the web frontend machine
either unencrypted or encrypted with a passphrase coded into the
frontend app. At least that way someone who manages to dump some of your
tables or steal a database dump will need to steal - and know they need
to steal - something completely different as well before they can use
the stolen information.

 I'm not fit enough in attack a database server, but I think when someone has 
 access to the database, he can simply dump the whole tables, while the key is 
 stored in the table, he has full access to everything in the database. At the 
 end the question is, where and how I should store the key to decrypt the 
 columns?

It depends a lot on what the trade-off between convenience/performance
and security is. How often is the data you want to encrypt accessed? Is
it acceptable to require interactive authorization or input before
encrypted data can be decrypted? Are the people who add sensitive data
the same ones who need to be able to read it back out again, or can you
restrict the group who can read it to a smaller group of users?

--
Craig Ringer

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Encryption For Specific Column- Where to store the key

2011-06-15 Thread Craig Ringer
On 15/06/11 14:07, Manuel Gysin wrote:

 A discussion about this topic can be found under 
 http://www.experts-exchange.com/Database/PostgreSQL/Q_21934798.html

Use Stack Overflow instead ;-)

Anyway: Given the additional detail you provided in that post, where it
becomes clear that you only need to be able to *read* the CCNs
occasionally and only in batches, perhaps you should consider using
public key crypto.

Store the last 4 digits of the CCN unencrypted but not the CVV or expiry
time. That way you can show a hint to the user about which card you're
using without them (or anyone else) being able to extract the full details.

Encrypt the full details using a public key when you store them in the
database. The web front end only needs to know the public key to encrypt
data. It doesn't need to know the private key, and without it it cannot
*decrypt* the data again.

Now your batch invoicing program can load the private key off a USB key
or SD card - or just keep it on disk and decrypt it using a strong
passphrase that is hand-entered by a user. When you're not doing
invoicing and billing, your system doesn't have any access to the
private key and cannot decrypt the stored data.

If you switch to rolling billing later on, you'll need to adjust this
process, but it still allows you to keep the part that can read the
credit card numbers very separate from the part that interacts with
untrusted users over the Internet.

--
Craig Ringer

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] You could be a PostgreSQL Patch Reviewer!

2011-06-15 Thread Craig Ringer
On 16/06/11 04:49, Josh Berkus wrote:

 We especially need folks who are able to build PostgreSQL on Windows, as
 we have several Windows-specific patches and no reviewers for them.

Urrrggh. Does that mean I have to volunteer myself? ;-)

I kind of regret learning to develop on Windows, because I find it a
horrid environment*, but so does everybody else, most of whom have been
clever enough not to learn it. They can say I don't know Windows and
they're safe.

Any chance of flagging patches on the commitfest when they're platform
specific? I'm hurting for time but will check out Windows-specific stuff
if it's an area the project particularly needs help with.

* Except for Visual C++'s debugger, which is a godsend, and so good it's
almost worth learning to code on Windows to use. I will never use gdb
for c++ again given the choice. gdb is a C debugger with bizarre
delusions that it's also a C++ debugger. Most of my C++ code is
cross-platform primarily so I can code on Linux, interactively debug on
Windows, and use Linux for valgrind memory debugging.

--
Craig Ringer

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] You could be a PostgreSQL Patch Reviewer!

2011-06-15 Thread Tom Lane
Craig Ringer cr...@postnewspapers.com.au writes:
 Any chance of flagging patches on the commitfest when they're platform
 specific? I'm hurting for time but will check out Windows-specific stuff
 if it's an area the project particularly needs help with.

Well, a quick look through
https://commitfest.postgresql.org/action/commitfest_view/inprogress
suggests that these could use attention:

Latch implementation: wake on postmaster death, reduce archiver wakeups
Has a Windows-specific implementation, which needs testing

POSIX shared memory
Does this work on Windows?

Allow multiple Postgres clusters running on the same machine to distinguish 
themselves in the event log
Windows-only feature, I assume

Add Support for building with Visual Studio 2010
Needs testing not only with VS2010, but older versions

libpq SSL with non-blocking sockets (WIP)
Might need testing on Windows, not sure


regards, tom lane

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] PGP encrypt/decrypt - Prereqistes

2011-06-15 Thread Vikram A
Dear Ringer, 

I was out of station; i could not reply on time; sorry. 

I shall follow all sort for ideas that you have given. I have decided to use 
the encrypt() and decrypt()functions. i could not found exact steps for 
using pgp_sym_encrypt() and pgp_sym_decrypt()

Can I have your commend on my decision and about the use of PGP functions.


Thank you.
Vikram A



From: Craig Ringer cr...@postnewspapers.com.au
To: Vikram A vikkiatb...@yahoo.in
Cc: PGSQL - Genearal pgsql-general@postgresql.org
Sent: Tuesday, 14 June 2011 12:52 PM
Subject: Re: [GENERAL] PGP encrypt/decrypt - Prereqistes

On 14/06/11 14:29, Vikram A wrote:
 My application work in a LAN. It will not with across internet. Number
 users also less than 25. Only certain information to be  cipher. Also I
 do not want such a complicated public and private key as PGP defines. As
 you said, I would like to go for simple[ Symmetric] method with our own
 key.

OK, so you can still use pgcrypto, but just using the pgp_sym_encrypt
and pgp_sym_decrypt functions.

Are you aware, though, that by encrypting your data field-by-field you
prevent it from being usefully indexed or otherwise processed by the
database? You will make a lot of things harder - and slower - than they
would otherwise be. Choose what you encrypt carefully.

You also need to do your key storage and access right. Encrypting data
is no use if you store the encryption/decryption key alongside the data,
after all.

 I agree that, we can not keep the key secret from the developers. My
 question is If the developer/or one is knowing the key is left the
 organization It is ultimately waste of doing encryption know? Is there
 any way to avoid such things? 

You can have a batch process that decrypts the data in the database and
re-encrypts it with a newly generated key unknown to that person. This
isn't a bad idea to have ready, because keys can be compromised for all
sorts of reasons including network intrusions.

That won't stop a currently-active developer from dumping and decypting
all your data, of course. As Alban Hertroys just pointed out, what it
comes down to is that if you can not trust your developers then  you're
screwed. You can prevent casual access and abuse, but not planned
attacks using a well-hidden trojan in the source code that's hidden in
an otherwise unobtrusive patch.

Nonetheless, you need to have a way to re-key if an old key is compromised.

On 14/06/11 14:57, Alban Hertroys wrote:
 So what exactly is the encryption supposed to solve? Do you really
 need it?

Usually people are trying to satisfy privacy rules or other
legislative/policy compliance requirements.

There *are* uses to encrypting data in a DB- at the very least, you can
reduce the amount of your infrastructure that knows how to decrypt the
data of interest to a small subset of your system. That makes it harder
for Joe Script Kiddie to exploit some dumb SQL injection vulnerability
to dump all the sensitive bits of your database in one go.

You never store your keys with your database dumps and they never travel
over the wire with replication traffic, so you're helped out in those
areas too. Of course, your replication traffic should be over SSL or on
a very secure LAN and your backups should be encrypted anyway, so
neither of those are as big a help... but they can't hurt.

--
Craig Ringer

[GENERAL] Constraint to ensure value does NOT exist in another table?

2011-06-15 Thread Mike Christensen
I know I can setup a FK constraint to make sure Table1.ColA exists in
Table2.Key, however what if I want to do the reverse?

I want to ensure Table1.ColA does NOT exist in Table2.Key..  Can I do
this with any sort of CHECK constraint, trigger, custom function, etc?
 Thanks!

Mike

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] You could be a PostgreSQL Patch Reviewer!

2011-06-15 Thread Mike Christensen
Did anyone ever fix the annoying thing where uuid_generate_v4()
doesn't work on Windows 64bit?

On Wed, Jun 15, 2011 at 9:46 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Craig Ringer cr...@postnewspapers.com.au writes:
 Any chance of flagging patches on the commitfest when they're platform
 specific? I'm hurting for time but will check out Windows-specific stuff
 if it's an area the project particularly needs help with.

 Well, a quick look through
 https://commitfest.postgresql.org/action/commitfest_view/inprogress
 suggests that these could use attention:

 Latch implementation: wake on postmaster death, reduce archiver wakeups
        Has a Windows-specific implementation, which needs testing

 POSIX shared memory
        Does this work on Windows?

 Allow multiple Postgres clusters running on the same machine to distinguish 
 themselves in the event log
        Windows-only feature, I assume

 Add Support for building with Visual Studio 2010
        Needs testing not only with VS2010, but older versions

 libpq SSL with non-blocking sockets (WIP)
        Might need testing on Windows, not sure


                        regards, tom lane

 --
 Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-general


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] You could be a PostgreSQL Patch Reviewer!

2011-06-15 Thread Tom Lane
Mike Christensen m...@kitchenpc.com writes:
 Did anyone ever fix the annoying thing where uuid_generate_v4()
 doesn't work on Windows 64bit?

AFAIR that was an issue with the uuid-ossp library, so it would be a
matter for that upstream group to deal with.

regards, tom lane

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] You could be a PostgreSQL Patch Reviewer!

2011-06-15 Thread Mike Christensen
Wouldn't it be faster/better/easier if Postgres just had its own built
in UUID generator?  Last I tested generating a bunch of UUIDs, it was
quite slow (well compared to MS SQL anyway)..

On Wed, Jun 15, 2011 at 10:23 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Mike Christensen m...@kitchenpc.com writes:
 Did anyone ever fix the annoying thing where uuid_generate_v4()
 doesn't work on Windows 64bit?

 AFAIR that was an issue with the uuid-ossp library, so it would be a
 matter for that upstream group to deal with.

                        regards, tom lane


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] You could be a PostgreSQL Patch Reviewer!

2011-06-15 Thread Tom Lane
Mike Christensen m...@kitchenpc.com writes:
 Wouldn't it be faster/better/easier if Postgres just had its own built
 in UUID generator?

Why would it be?  If you think you can easily improve on uuid-ossp,
you should go help them.

regards, tom lane

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] You could be a PostgreSQL Patch Reviewer!

2011-06-15 Thread Mike Christensen
On Wed, Jun 15, 2011 at 10:34 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Mike Christensen m...@kitchenpc.com writes:
 Wouldn't it be faster/better/easier if Postgres just had its own built
 in UUID generator?

 Why would it be?  If you think you can easily improve on uuid-ossp,
 you should go help them.

I have no knowledge on the subject one way or the other, that's why
I'm asking..  If the answer is no, there would be no benefit then
that's the answer..  However, it seems like generating UUIDs in Win64
has been broken, perhaps, forever which is why I question if there's a
better library to use on the Windows platform (or having a single UUID
generator built in for all platforms)..

I'd love to help out with Postgres (and several other open source
projects), perhaps some day though I'm more of a web/.NET guy..

Mike

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general