Re: [PERFORM] autocommit (true/false) for more than 1 million records

Hello All, I learned a lot by inputs from all of you. To share one more thing about java_JDBC bypassing autocommit that I tried: (1) Read/save source data into f1.csv, f2.csv, .. (2) Copy/load into dest psql.DB CopyManager cm = null; FileReader fileReader = null;

Re: [PERFORM] autocommit (true/false) for more than 1 million records

Alex Goncharov wrote: > Kevin Grittner wrote: >> The rows will all be in the table, but not visible to any other >> transaction. > > How much data can I fit there while doing COPY?  Not 1 TB? As has already been said, why not?  This is not some special section of the table -- the data is writte

Re: [PERFORM] autocommit (true/false) for more than 1 million records

[about loadling large amounts of data] Felipe Santos wrote: > This might also help: > http://www.postgresql.org/docs/9.1/static/populate.html > > > Bulk load tables from text files in almost all RDMS are "log free" (Postgres' > COPY is one of them). > > The reason is that the database doesn't

Re: [PERFORM] autocommit (true/false) for more than 1 million records

This might also help: http://www.postgresql.org/docs/9.1/static/populate.html Bulk load tables from text files in almost all RDMS are "log free" (Postgres' COPY is one of them). The reason is that the database doesn't need to waste resources by writing the log because there's no risk of data loss

Re: [PERFORM] autocommit (true/false) for more than 1 million records

Alex Goncharov wrote: > Thank you, Kevin -- this is helpful. > > But it still leaves questions for me. >> Alex Goncharov wrote: > >>> The whole thing is aborted then, and the good 99 records are not >>> making it into the target table. >> >> Right. This is one reason people often batch such co

Re: [PERFORM] autocommit (true/false) for more than 1 million records

On Wed, Aug 27, 2014 at 1:02 AM, Alex Goncharov < alex.goncharov@gmail.com> wrote: > > Thank you, Kevin -- this is helpful. > > Thank you David, too. > > > > But it still leaves questions for me. > > Still... > > > Alex Goncharov wrote: > > >>> How do I decide, before starting a COPY data loa

Re: [PERFORM] autocommit (true/false) for more than 1 million records

> Thank you, Kevin -- this is helpful. Thank you David, too. > But it still leaves questions for me. Still... Alex Goncharov wrote: >>> How do I decide, before starting a COPY data load, whether such a load >>> protection ("complexity") makes sense ("is necessary")? This is *the* practical q

Re: [PERFORM] autocommit (true/false) for more than 1 million records

On Tue, Aug 26, 2014 at 9:21 PM, Alex Goncharov-2 [via PostgreSQL] < ml-node+s1045698n5816426...@n5.nabble.com> wrote: > Thank you, Kevin -- this is helpful. > > But it still leaves questions for me. > > > Kevin Grittner <[hidden email] > > wro

Re: [PERFORM] autocommit (true/false) for more than 1 million records

Thank you, Kevin -- this is helpful. But it still leaves questions for me. Kevin Grittner wrote: > Alex Goncharov wrote: > > The whole thing is aborted then, and the good 99 records are not > > making it into the target table. > > Right. This is one reason people often batch such copies or c

Re: [PERFORM] autocommit (true/false) for more than 1 million records

Alex Goncharov wrote: > Suppose I COPY a huge amount of data, e.g. 100 records. > > My 99 records are fine for the target, and the 100-th is not -- > it comes with a wrong record format or a target constraint > violation. > > The whole thing is aborted then, and the good 99 records are not > maki

Re: [PERFORM] autocommit (true/false) for more than 1 million records

On the COPY's atomicity -- looking for a definitive answer from a core developer, not a user's guess, please. Suppose I COPY a huge amount of data, e.g. 100 records. My 99 records are fine for the target, and the 100-th is not -- it comes with a wrong record format or a target constraint violatio

Re: [PERFORM] autocommit (true/false) for more than 1 million records

On Fri, Aug 22, 2014 at 1:49 PM, Emi Lu wrote: > Hello, > > Trying to insert into one table with 1 million records through java JDBC > into psql8.3. May I know (1) or (2) is better please? > > (1) set autocommit(true) > (2) set autocommit(false) > commit every n records (e.g., 100, 500, 1000

Re: [PERFORM] autocommit (true/false) for more than 1 million records

Hi Emi, Databases that comply to the ACID standard ( http://en.wikipedia.org/wiki/ACID) ensure that that are no data loss by first writing the data changes to the database log in opposition to updating the actual data on the filesystem first (on the datafiles). Each database has its own way of do

Re: [PERFORM] autocommit (true/false) for more than 1 million records

On Mon, Aug 25, 2014 at 9:40 AM, Emi Lu wrote: > > By the way, could someone let me know why set autocommit(false) is for > sure faster than true please? Or, some online docs talk about this. > > ​Not sure about the docs specifically but: Commit is expensive because as soon as it is issued all o

Re: [PERFORM] autocommit (true/false) for more than 1 million records

Good morning, Trying to insert into one table with 1 million records through java JDBC into psql8.3. May I know (1) or (2) is better please? (1) set autocommit(true) (2) set autocommit(false) commit every n records (e.g., 100, 500, 1000, etc) It depends on what you need. Data will be ava

Re: [PERFORM] autocommit (true/false) for more than 1 million records

* Emi Lu (em...@encs.concordia.ca) wrote: > >* > >>Trying to insert into one table with 1 million records through java > >>JDBC into psql8.3. May I know (1) or (2) is better please? > >> > >>(1) set autocommit(true) > >>(2) set autocommit(false) > >> commit every n records (e.g., 100, 500, 100

Re: [PERFORM] autocommit (true/false) for more than 1 million records

* Trying to insert into one table with 1 million records through java JDBC into psql8.3. May I know (1) or (2) is better please? (1) set autocommit(true) (2) set autocommit(false) commit every n records (e.g., 100, 500, 1000, etc) It depends on what you need. Data will be available to co

Re: [PERFORM] autocommit (true/false) for more than 1 million records

* Emi Lu (em...@encs.concordia.ca) wrote: > Hello, > > Trying to insert into one table with 1 million records through java > JDBC into psql8.3. May I know (1) or (2) is better please? > > (1) set autocommit(true) > (2) set autocommit(false) > commit every n records (e.g., 100, 500, 1000, etc

Re: [PERFORM] autocommit (true/false) for more than 1 million records

Emi Lu-2 wrote > Hello, > > Trying to insert into one table with 1 million records through java JDBC > into psql8.3. May I know (1) or (2) is better please? > > (1) set autocommit(true) > (2) set autocommit(false) > commit every n records (e.g., 100, 500, 1000, etc) > > Thanks a lot! > Em

[PERFORM] autocommit (true/false) for more than 1 million records

Hello, Trying to insert into one table with 1 million records through java JDBC into psql8.3. May I know (1) or (2) is better please? (1) set autocommit(true) (2) set autocommit(false) commit every n records (e.g., 100, 500, 1000, etc) Thanks a lot! Emi -- Sent via pgsql-performance

Re: [PERFORM] Autocommit in libpq

"Dengler, Michael" <[EMAIL PROTECTED]> writes: > Thanks for the reply. Your advice to upgrade sounds urgent. Are there > critical reasons I need to go to 7.4.16? Read the release notes between 7.4.1 and 7.4.16 and judge for yourself: http://developer.postgresql.org/pgdocs/postgres/release.html

Re: [PERFORM] Autocommit in libpq

@postgresql.org Subject: Re: [PERFORM] Autocommit in libpq Heikki Linnakangas wrote: > Dengler, Michael wrote: >> Please forgive me if I missed something (I have been searching for a >> definitive answer for this for 2 days). >> >> Is there any way to disable autocommit in

Re: [PERFORM] Autocommit in libpq

Heikki Linnakangas wrote: > Dengler, Michael wrote: >> Please forgive me if I missed something (I have been searching for a >> definitive answer for this for 2 days). >> >> Is there any way to disable autocommit in libpq? (PG 7.4.1) > > Just call BEGIN to start a transaction, and COMMIT to commit

Re: [PERFORM] Autocommit in libpq

Dengler, Michael wrote: Please forgive me if I missed something (I have been searching for a definitive answer for this for 2 days). Is there any way to disable autocommit in libpq? (PG 7.4.1) Just call BEGIN to start a transaction, and COMMIT to commit it. Other than that, no. -- Heikki

[PERFORM] Autocommit in libpq

Hi, Please forgive me if I missed something (I have been searching for a definitive answer for this for 2 days). Is there any way to disable autocommit in libpq? (PG 7.4.1) Thanks Mike

Re: [PERFORM] Autocommit

On Mon, Feb 14, 2005 at 04:58:31PM +0800, Hasnul Fadhly bin Hasan wrote: > Thanks for the reply. I am using postgres 7.4.5 client. There's one > that is using 7.4.1 client. I'm not sure if there would be any difference. > When i use psql and check the status of autocommit, it is set to > enab

Re: [PERFORM] Autocommit

Hi Micheal, Thanks for the reply.  I am using postgres 7.4.5 client.  There's one that is using 7.4.1 client.  I'm not sure if there would be any difference. When i use psql and check the status of autocommit, it is set to enable.  I'm not sure if libpq and psql uses the same defaults. Thanks

Re: [PERFORM] Autocommit

On Mon, Feb 14, 2005 at 04:01:20PM +0800, Hasnul Fadhly bin Hasan wrote: > > I am just wondering, by default, autocommit is enabled for every client > connection. The documentations states that we have to use BEGIN > and END or COMMIT so to increase performance by not using autocommit. > My qu

[PERFORM] Autocommit

Hi, I am just wondering, by default, autocommit is enabled for every client connection. The documentations states that we have to use BEGIN and END or COMMIT so to increase performance by not using autocommit. My question is, when we use the BEGIN and END statements, is autocommit unset/disabl