Hi Kevin,
I did another try with following additional changes based on our discussion:
1. use the tcp connection
2. turn off autovacuum
3. turn off full_page_writes
I could import more than 30G data in about 2 hours. That's totally
acceptable performance to me with the current server
On Fri, Sep 2, 2011 at 9:30 PM, Rural Hunter ruralhun...@gmail.com wrote:
Hi Kevin,
I did another try with following additional changes based on our discussion:
1. use the tcp connection
2. turn off autovacuum
3. turn off full_page_writes
I could import more than 30G data in about 2 hours.
well, thank you for the quick reply but actually I'm not concerning the
performance as of now. My problem is related to the bulk insert of
client side program psql. Or, it's a problem of migrating vast data from
mysql to pgsql.
于2011年8月29日 21:25:29,Julio Leyva写到:
check this out
Rural Hunter ruralhun...@gmail.com wrote:
it's a problem of migrating vast data from mysql to pgsql.
I don't know how helpful you'll find this, but when we migrated to
PostgreSQL a few years ago, we had good luck with using Java to
stream from one database to another. Be sure to include a
yes, the size of my database(around 200g) is very close to yours. How
long did it take you to migrate the data(100-200G)? I thought about to
write java code to transfer the data but it's a bit trivial. It would be
the best if I can complete this with regular export/import way. if that
won't
Rural Hunter ruralhun...@gmail.com wrote:
yes, the size of my database(around 200g) is very close to yours.
How long did it take you to migrate the data(100-200G)?
I don't remember off-hand, and I'm not sure it's worth digging for
the info because so much has changed in the intervening six
Hi Kevin,
Thank you very much for the quick and detailed answers/suggestions. I
will check and try them.
于 2011/8/29 23:18, Kevin Grittner 写道:
Good (but don't forget to change that once the bulk load is done). You
should probably also turn off full_page_writes and synchronous_commit.
I've
Rural Hunter ruralhun...@gmail.com wrote:
2011/8/29 23:18, Kevin Grittner:
I also recommend a VACUUM FREEZE ANALYZE on the database unless
most of these rows will be deleted or updated before you run a
billion database transactions. Otherwise you will get a painful
anti-wraparound
Thank you. I didn't understand what 'vacuum freeze' actually does. I
will check the detail to see if it's good for my situation. and I will
also test the load by tcp connection. Thanks again for all your advices
and they are really very helpful to me!
于 2011/8/30 0:06, Kevin Grittner 写道:
2011/8/29 Rural Hunter ruralhun...@gmail.com:
Hi all,
I'm a newbie here. I'm trying to test pgsql with my mysql data. If the
performance is good, I will migrate from mysql to pgsql.
I installed pgsql 9.1rc on my Ubuntu server. I'm trying to import a large
sql file dumped from mysql into pgsql
Yes, seems so. I separated one of the tables having the problem and did
a test. First, loaded it with original file and saw the same: invalid
command then follows out of memory. After I fixed those escape
characters, the table could be loaded successfully.
于 2011/8/30 10:14, Scott Marlowe 写道:
11 matches
Mail list logo
