Basic stuff:
------------
- Rebase of Patch is required.
- Compiles cleanly without any errors/warnings
- Regression tests pass.
What it does:
---------------------
This patch is useful when COPY command input/output are stored in
compression format or in any command/script uses these output/input in any
means; without generating intermediate temporary files.
This feature can be used in server side using "COPY statement" by
administrator. Or can be used in psql internal "\copy" command by any user.
Code Review comments:
---------------------
1. Modify the comment in function header of: parse_slash_copy (needs to
modify for new syntax)
2. Comments for functions OpenPipeStream & ClosePipeStream are missing.
3. Any Script errors are not directly visible to user; If there problems in
script no way to cleanup.
Shouldn't this be mentioned in User Manual.
Test case issues:
------------------
1. "Broken pipe" is not handled in case of psql "\copy" command;
Issue are as follows:
Following are verified on SuSE-Linux 10.2.
1) psql is exiting when "\COPY xxx TO" command is issued and
command/script is not found
When popen is called in write mode it is creating valid
file descriptor and when it tries to write to file "Broken pipe" error is
coming which is not handled.
psql# \copy pgbench_accounts TO PROGRAM
'../compress.sh pgbench_accounts4.txt'
2) When "\copy" command is in progress then program/command is
killed/"crashed due to any problem"
psql is exiting.
Script used in testcases:
------------------
1. compress.sh
echo 'cat > $1' > compress.sh
echo 'bzip2 -z $1' >> compress.sh
chmod +x compress.sh
2. decompress.sh
echo 'bzip2 -d -c -k $*' > decompress.sh
chmod +x decompress.sh
Testcases executed are attached with this mail.
With Regards,
Amit Kapila.
--1. check success case admin can use this "COPY statement" feature.
select count(*) from pgbench_accounts;
copy pgbench_accounts TO PROGRAM '/home/kiran/installation/bin/compress.sh
/home/kiran/installation/bin/pgbench_accounts1.txt';
delete from pgbench_accounts;
copy pgbench_accounts FROM PROGRAM '/home/kiran/installation/bin/decompress.sh
/home/kiran/installation/bin/pgbench_accounts1.txt.bz2';
select count(*) from pgbench_accounts;
--2. check commands are running with relative path to database directory.
copy pgbench_accounts TO PROGRAM '../compress.sh ../pgbench_accounts4.txt';
delete from pgbench_accounts;
copy pgbench_accounts FROM PROGRAM '../decompress.sh
../pgbench_accounts4.txt.bz2';
select count(*) from pgbench_accounts;
--3. invalid case; where command not found
select count(*) from pgbench_accounts;
copy pgbench_accounts TO PROGRAM 'compress.sh pgbench_accounts1.txt';
copy pgbench_accounts FROM PROGRAM 'decompress.sh pgbench_accounts1.txt.bz2';
--4. invalid case; where file / script having problem.
select count(*) from pgbench_accounts;
copy pgbench_accounts TO PROGRAM '../compress.sh abc/pgbench_accounts1.txt';
copy pgbench_accounts FROM PROGRAM '../decompress.sh
abc/pgbench_accounts1.txt.bz2';
--repeat all above cases from psql command.
--5. check success case admin can use this "\COPY" psql feature.
select count(*) from pgbench_accounts;
\copy pgbench_accounts TO PROGRAM '/home/kiran/installation/bin/compress.sh
/home/kiran/installation/bin/pgbench_accounts1.txt';
delete from pgbench_accounts;
\copy pgbench_accounts FROM PROGRAM '/home/kiran/installation/bin/decompress.sh
/home/kiran/installation/bin/pgbench_accounts1.txt.bz2';
select count(*) from pgbench_accounts;
--6. check commands are running with relative path to psql current working
directory.
\copy pgbench_accounts TO PROGRAM '../compress.sh ../pgbench_accounts4.txt';
delete from pgbench_accounts;
\copy pgbench_accounts FROM PROGRAM '../decompress.sh
../pgbench_accounts4.txt.bz2';
select count(*) from pgbench_accounts;
--7. invalid case; where command not found
select count(*) from pgbench_accounts;
\copy pgbench_accounts TO PROGRAM 'compress.sh pgbench_accounts1.txt';
\copy pgbench_accounts FROM PROGRAM 'decompress.sh pgbench_accounts1.txt.bz2';
--8. invalid case; where file / script having problem.
select count(*) from pgbench_accounts;
\copy pgbench_accounts TO PROGRAM '../compress.sh abc/pgbench_accounts1.txt';
\copy pgbench_accounts FROM PROGRAM '../decompress.sh
abc/pgbench_accounts1.txt.bz2';
--9. Normal user can't execute "COPY statement"
create user abc;
\c abc
select count(*) from pgbench_accounts;
copy pgbench_accounts TO PROGRAM '/home/kiran/installation/bin/compress.sh
/home/kiran/installation/bin/pgbench_accounts1.txt';
delete from pgbench_accounts;
copy pgbench_accounts FROM PROGRAM '/home/kiran/installation/bin/decompress.sh
/home/kiran/installation/bin/pgbench_accounts1.txt.bz2';
select count(*) from pgbench_accounts;
--10. Normal user can execute "\COPY" psql command
select count(*) from pgbench_accounts;
\copy pgbench_accounts TO PROGRAM '/home/kiran/installation/bin/compress.sh
/home/kiran/installation/bin/pgbench_accounts1.txt';
delete from pgbench_accounts;
\copy pgbench_accounts FROM PROGRAM '/home/kiran/installation/bin/decompress.sh
/home/kiran/installation/bin/pgbench_accounts1.txt.bz2';
select count(*) from pgbench_accounts;
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
