Re: [PHP] Re: Issue with the huge import script
On Sun, 2009-08-16 at 04:06 +0200, Ralph Deffke wrote:
Hi,
this sounds huge, and cries for a sql version of the import.
Are both databases the same? MySQL?
I give u a draft for MySQL
u export the data u have, then u got a textfile with 10+ sql statments
in the php script u open the file and iterate over it by line (carefull it
could be also ; in case its a Unix created file on a windows platform)
line == one SQL insert in table bla bla...
in the loop then just mysq_query with this line
if the the someid is an unique index the insert will fail, so only those
records are inserted beeing not already in the database.
but I think as of the amount off records it doesn't sound like a every 10
minutes job, if it is a rara job, just do it with phpMyAdmin
sorry not pulling out the code, but was a long day behind the keyboard, need
some sleep
ralph_def...@yahoo.de
Devendra Jadhav devendra...@gmail.com wrote in message
news:be4b00cf0908151815r1c7430d2j8a6cb0da1f10a...@mail.gmail.com...
Hi,
I have to import data from one database to another, I have to import
around
10(1Lac) records.
First I need to check if the record is already imported or not and import
only those records which are not imported.
Here is my logic
$already_imported = get_already_imported_records();
format of the $already_imported is $already_imported[someid] = 'imported';
Now i take all records from another db and iterating through it.
if (!key_exists($already_imported[$new_id])){
import_function($new_id)
}else{
echo 'allready imported'.$already_imported[$new_id];
}
Now my script is importing same records for more than one time. I am not
able to get through this issue
Is it because of the size of the records or something else...?
Please suggest me some solution which is faster, safe and easy to code :D
Thanks in advance
--
Devendra Jadhav
You cry for a MySQL version and then revert back to PHP?! Why not just
keep the whole thing in MySQL? You can use SQL statements to check
whether a record exists before attempting to shove it in the database
using a WHERE clause in the INSERT statement or by making one field
unique and hiding notices about inserts that are attempting to overwrite
that.
Thanks,
Ash
http://www.ashleysheridan.co.uk
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: Issue with the huge import script
because I assume always that a requester got some intelligence, so in that
case there must be a reasonsble reason why he wants to do it in PHP
im not like u assuming everybody is a thumb
ralph
Ashley Sheridan a...@ashleysheridan.co.uk wrote in message
news:1250413427.2344.51.ca...@localhost...
On Sun, 2009-08-16 at 04:06 +0200, Ralph Deffke wrote:
Hi,
this sounds huge, and cries for a sql version of the import.
Are both databases the same? MySQL?
I give u a draft for MySQL
u export the data u have, then u got a textfile with 10+ sql
statments
in the php script u open the file and iterate over it by line (carefull
it
could be also ; in case its a Unix created file on a windows platform)
line == one SQL insert in table bla bla...
in the loop then just mysq_query with this line
if the the someid is an unique index the insert will fail, so only those
records are inserted beeing not already in the database.
but I think as of the amount off records it doesn't sound like a every
10
minutes job, if it is a rara job, just do it with phpMyAdmin
sorry not pulling out the code, but was a long day behind the keyboard,
need
some sleep
ralph_def...@yahoo.de
Devendra Jadhav devendra...@gmail.com wrote in message
news:be4b00cf0908151815r1c7430d2j8a6cb0da1f10a...@mail.gmail.com...
Hi,
I have to import data from one database to another, I have to import
around
10(1Lac) records.
First I need to check if the record is already imported or not and
import
only those records which are not imported.
Here is my logic
$already_imported = get_already_imported_records();
format of the $already_imported is $already_imported[someid] =
'imported';
Now i take all records from another db and iterating through it.
if (!key_exists($already_imported[$new_id])){
import_function($new_id)
}else{
echo 'allready imported'.$already_imported[$new_id];
}
Now my script is importing same records for more than one time. I am
not
able to get through this issue
Is it because of the size of the records or something else...?
Please suggest me some solution which is faster, safe and easy to code
:D
Thanks in advance
--
Devendra Jadhav
You cry for a MySQL version and then revert back to PHP?! Why not just
keep the whole thing in MySQL? You can use SQL statements to check
whether a record exists before attempting to shove it in the database
using a WHERE clause in the INSERT statement or by making one field
unique and hiding notices about inserts that are attempting to overwrite
that.
Thanks,
Ash
http://www.ashleysheridan.co.uk
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: Issue with the huge import script
On Sun, 2009-08-16 at 11:05 +0200, Ralph Deffke wrote:
because I assume always that a requester got some intelligence, so in that
case there must be a reasonsble reason why he wants to do it in PHP
im not like u assuming everybody is a thumb
ralph
Ashley Sheridan a...@ashleysheridan.co.uk wrote in message
news:1250413427.2344.51.ca...@localhost...
On Sun, 2009-08-16 at 04:06 +0200, Ralph Deffke wrote:
Hi,
this sounds huge, and cries for a sql version of the import.
Are both databases the same? MySQL?
I give u a draft for MySQL
u export the data u have, then u got a textfile with 10+ sql
statments
in the php script u open the file and iterate over it by line (carefull
it
could be also ; in case its a Unix created file on a windows platform)
line == one SQL insert in table bla bla...
in the loop then just mysq_query with this line
if the the someid is an unique index the insert will fail, so only those
records are inserted beeing not already in the database.
but I think as of the amount off records it doesn't sound like a every
10
minutes job, if it is a rara job, just do it with phpMyAdmin
sorry not pulling out the code, but was a long day behind the keyboard,
need
some sleep
ralph_def...@yahoo.de
Devendra Jadhav devendra...@gmail.com wrote in message
news:be4b00cf0908151815r1c7430d2j8a6cb0da1f10a...@mail.gmail.com...
Hi,
I have to import data from one database to another, I have to import
around
10(1Lac) records.
First I need to check if the record is already imported or not and
import
only those records which are not imported.
Here is my logic
$already_imported = get_already_imported_records();
format of the $already_imported is $already_imported[someid] =
'imported';
Now i take all records from another db and iterating through it.
if (!key_exists($already_imported[$new_id])){
import_function($new_id)
}else{
echo 'allready imported'.$already_imported[$new_id];
}
Now my script is importing same records for more than one time. I am
not
able to get through this issue
Is it because of the size of the records or something else...?
Please suggest me some solution which is faster, safe and easy to code
:D
Thanks in advance
--
Devendra Jadhav
You cry for a MySQL version and then revert back to PHP?! Why not just
keep the whole thing in MySQL? You can use SQL statements to check
whether a record exists before attempting to shove it in the database
using a WHERE clause in the INSERT statement or by making one field
unique and hiding notices about inserts that are attempting to overwrite
that.
Thanks,
Ash
http://www.ashleysheridan.co.uk
There are rare occasions on this list where the best answer is not PHP,
and I believe this is one of them.
Thanks,
Ash
http://www.ashleysheridan.co.uk
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: Issue with the huge import script
so then tell me tell me what my first sentence means
this sounds huge, and cries for a sql version of the import.
it looks like u have no experience in working as consultant hotlines Ash
first the folk is asking for a sulotion of HIS php implying a general
solution in PHP with a hint to the prof solution
I think thats what I did
Ashley Sheridan a...@ashleysheridan.co.uk wrote in message
news:1250414284.2344.55.ca...@localhost...
On Sun, 2009-08-16 at 11:05 +0200, Ralph Deffke wrote:
because I assume always that a requester got some intelligence, so in
that
case there must be a reasonsble reason why he wants to do it in PHP
im not like u assuming everybody is a thumb
ralph
Ashley Sheridan a...@ashleysheridan.co.uk wrote in message
news:1250413427.2344.51.ca...@localhost...
On Sun, 2009-08-16 at 04:06 +0200, Ralph Deffke wrote:
Hi,
this sounds huge, and cries for a sql version of the import.
Are both databases the same? MySQL?
I give u a draft for MySQL
u export the data u have, then u got a textfile with 10+ sql
statments
in the php script u open the file and iterate over it by line
(carefull
it
could be also ; in case its a Unix created file on a windows
platform)
line == one SQL insert in table bla bla...
in the loop then just mysq_query with this line
if the the someid is an unique index the insert will fail, so only
those
records are inserted beeing not already in the database.
but I think as of the amount off records it doesn't sound like a
every
10
minutes job, if it is a rara job, just do it with phpMyAdmin
sorry not pulling out the code, but was a long day behind the
keyboard,
need
some sleep
ralph_def...@yahoo.de
Devendra Jadhav devendra...@gmail.com wrote in message
news:be4b00cf0908151815r1c7430d2j8a6cb0da1f10a...@mail.gmail.com...
Hi,
I have to import data from one database to another, I have to
import
around
10(1Lac) records.
First I need to check if the record is already imported or not and
import
only those records which are not imported.
Here is my logic
$already_imported = get_already_imported_records();
format of the $already_imported is $already_imported[someid] =
'imported';
Now i take all records from another db and iterating through it.
if (!key_exists($already_imported[$new_id])){
import_function($new_id)
}else{
echo 'allready imported'.$already_imported[$new_id];
}
Now my script is importing same records for more than one time. I
am
not
able to get through this issue
Is it because of the size of the records or something else...?
Please suggest me some solution which is faster, safe and easy to
code
:D
Thanks in advance
--
Devendra Jadhav
You cry for a MySQL version and then revert back to PHP?! Why not just
keep the whole thing in MySQL? You can use SQL statements to check
whether a record exists before attempting to shove it in the database
using a WHERE clause in the INSERT statement or by making one field
unique and hiding notices about inserts that are attempting to
overwrite
that.
Thanks,
Ash
http://www.ashleysheridan.co.uk
There are rare occasions on this list where the best answer is not PHP,
and I believe this is one of them.
Thanks,
Ash
http://www.ashleysheridan.co.uk
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: Issue with the huge import script
On Sun, 2009-08-16 at 11:25 +0200, Ralph Deffke wrote:
so then tell me tell me what my first sentence means
this sounds huge, and cries for a sql version of the import.
it looks like u have no experience in working as consultant hotlines Ash
first the folk is asking for a sulotion of HIS php implying a general
solution in PHP with a hint to the prof solution
I think thats what I did
Ashley Sheridan a...@ashleysheridan.co.uk wrote in message
news:1250414284.2344.55.ca...@localhost...
On Sun, 2009-08-16 at 11:05 +0200, Ralph Deffke wrote:
because I assume always that a requester got some intelligence, so in
that
case there must be a reasonsble reason why he wants to do it in PHP
im not like u assuming everybody is a thumb
ralph
Ashley Sheridan a...@ashleysheridan.co.uk wrote in message
news:1250413427.2344.51.ca...@localhost...
On Sun, 2009-08-16 at 04:06 +0200, Ralph Deffke wrote:
Hi,
this sounds huge, and cries for a sql version of the import.
Are both databases the same? MySQL?
I give u a draft for MySQL
u export the data u have, then u got a textfile with 10+ sql
statments
in the php script u open the file and iterate over it by line
(carefull
it
could be also ; in case its a Unix created file on a windows
platform)
line == one SQL insert in table bla bla...
in the loop then just mysq_query with this line
if the the someid is an unique index the insert will fail, so only
those
records are inserted beeing not already in the database.
but I think as of the amount off records it doesn't sound like a
every
10
minutes job, if it is a rara job, just do it with phpMyAdmin
sorry not pulling out the code, but was a long day behind the
keyboard,
need
some sleep
ralph_def...@yahoo.de
Devendra Jadhav devendra...@gmail.com wrote in message
news:be4b00cf0908151815r1c7430d2j8a6cb0da1f10a...@mail.gmail.com...
Hi,
I have to import data from one database to another, I have to
import
around
10(1Lac) records.
First I need to check if the record is already imported or not and
import
only those records which are not imported.
Here is my logic
$already_imported = get_already_imported_records();
format of the $already_imported is $already_imported[someid] =
'imported';
Now i take all records from another db and iterating through it.
if (!key_exists($already_imported[$new_id])){
import_function($new_id)
}else{
echo 'allready imported'.$already_imported[$new_id];
}
Now my script is importing same records for more than one time. I
am
not
able to get through this issue
Is it because of the size of the records or something else...?
Please suggest me some solution which is faster, safe and easy to
code
:D
Thanks in advance
--
Devendra Jadhav
You cry for a MySQL version and then revert back to PHP?! Why not just
keep the whole thing in MySQL? You can use SQL statements to check
whether a record exists before attempting to shove it in the database
using a WHERE clause in the INSERT statement or by making one field
unique and hiding notices about inserts that are attempting to
overwrite
that.
Thanks,
Ash
http://www.ashleysheridan.co.uk
There are rare occasions on this list where the best answer is not PHP,
and I believe this is one of them.
Thanks,
Ash
http://www.ashleysheridan.co.uk
You did say this cries for a pure SQL solution, which you then went on
to say involved PHP. Call me a pedant, but PHP is not SQL. And please,
try to leave insults out of the list in future, it makes you look
unprofessional.
Thanks,
Ash
http://www.ashleysheridan.co.uk
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Re: Issue with the huge import script
On Sun, 16 Aug 2009 04:15:12 +0300, Devendra Jadhav devendra...@gmail.com wrote: Is it because of the size of the records or something else...? Please suggest me some solution which is faster, safe and easy to code :D What about using insert ignore ? :) 1. add UNIQUE constraint on a key you want to add, so you won't be able to add row with same ID twice. 2. then use INSERT IGNORE into = easy to code, easy to use. Of course, this aproach won't work if database will grow larger and larger. (if you adding new rows all the time) Then it is better to use this aproach: 1. add auto_increment column to original table. 2. before getting new rows from old table, check what is maximum ID from new table. 3. get only rows where old_table.id new_table.max_id remember to ORDER BY ID ASC. -- -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: Issue with the huge import script
better consider if u r insulting I've seen a view posts of u falling into
that category i w'ld recomment respect posters first approaches even if they
r stupid and incorporate the respect in the way u answer
the comment you r crying ... is an insult.
Ashley Sheridan a...@ashleysheridan.co.uk wrote in message
news:1250417781.2344.58.ca...@localhost...
On Sun, 2009-08-16 at 11:25 +0200, Ralph Deffke wrote:
so then tell me tell me what my first sentence means
this sounds huge, and cries for a sql version of the import.
it looks like u have no experience in working as consultant hotlines
Ash
first the folk is asking for a sulotion of HIS php implying a general
solution in PHP with a hint to the prof solution
I think thats what I did
Ashley Sheridan a...@ashleysheridan.co.uk wrote in message
news:1250414284.2344.55.ca...@localhost...
On Sun, 2009-08-16 at 11:05 +0200, Ralph Deffke wrote:
because I assume always that a requester got some intelligence, so
in
that
case there must be a reasonsble reason why he wants to do it in PHP
im not like u assuming everybody is a thumb
ralph
Ashley Sheridan a...@ashleysheridan.co.uk wrote in message
news:1250413427.2344.51.ca...@localhost...
On Sun, 2009-08-16 at 04:06 +0200, Ralph Deffke wrote:
Hi,
this sounds huge, and cries for a sql version of the import.
Are both databases the same? MySQL?
I give u a draft for MySQL
u export the data u have, then u got a textfile with 10+ sql
statments
in the php script u open the file and iterate over it by line
(carefull
it
could be also ; in case its a Unix created file on a windows
platform)
line == one SQL insert in table bla bla...
in the loop then just mysq_query with this line
if the the someid is an unique index the insert will fail, so
only
those
records are inserted beeing not already in the database.
but I think as of the amount off records it doesn't sound like a
every
10
minutes job, if it is a rara job, just do it with phpMyAdmin
sorry not pulling out the code, but was a long day behind the
keyboard,
need
some sleep
ralph_def...@yahoo.de
Devendra Jadhav devendra...@gmail.com wrote in message
news:be4b00cf0908151815r1c7430d2j8a6cb0da1f10a...@mail.gmail.com...
Hi,
I have to import data from one database to another, I have to
import
around
10(1Lac) records.
First I need to check if the record is already imported or not
and
import
only those records which are not imported.
Here is my logic
$already_imported = get_already_imported_records();
format of the $already_imported is $already_imported[someid] =
'imported';
Now i take all records from another db and iterating through
it.
if (!key_exists($already_imported[$new_id])){
import_function($new_id)
}else{
echo 'allready imported'.$already_imported[$new_id];
}
Now my script is importing same records for more than one
time. I
am
not
able to get through this issue
Is it because of the size of the records or something else...?
Please suggest me some solution which is faster, safe and easy
to
code
:D
Thanks in advance
--
Devendra Jadhav
You cry for a MySQL version and then revert back to PHP?! Why not
just
keep the whole thing in MySQL? You can use SQL statements to check
whether a record exists before attempting to shove it in the
database
using a WHERE clause in the INSERT statement or by making one
field
unique and hiding notices about inserts that are attempting to
overwrite
that.
Thanks,
Ash
http://www.ashleysheridan.co.uk
There are rare occasions on this list where the best answer is not
PHP,
and I believe this is one of them.
Thanks,
Ash
http://www.ashleysheridan.co.uk
You did say this cries for a pure SQL solution, which you then went on
to say involved PHP. Call me a pedant, but PHP is not SQL. And please,
try to leave insults out of the list in future, it makes you look
unprofessional.
Thanks,
Ash
http://www.ashleysheridan.co.uk
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Re: Issue with the huge import script
Hi,
this sounds huge, and cries for a sql version of the import.
Are both databases the same? MySQL?
I give u a draft for MySQL
u export the data u have, then u got a textfile with 10+ sql statments
in the php script u open the file and iterate over it by line (carefull it
could be also ; in case its a Unix created file on a windows platform)
line == one SQL insert in table bla bla...
in the loop then just mysq_query with this line
if the the someid is an unique index the insert will fail, so only those
records are inserted beeing not already in the database.
but I think as of the amount off records it doesn't sound like a every 10
minutes job, if it is a rara job, just do it with phpMyAdmin
sorry not pulling out the code, but was a long day behind the keyboard, need
some sleep
ralph_def...@yahoo.de
Devendra Jadhav devendra...@gmail.com wrote in message
news:be4b00cf0908151815r1c7430d2j8a6cb0da1f10a...@mail.gmail.com...
Hi,
I have to import data from one database to another, I have to import
around
10(1Lac) records.
First I need to check if the record is already imported or not and import
only those records which are not imported.
Here is my logic
$already_imported = get_already_imported_records();
format of the $already_imported is $already_imported[someid] = 'imported';
Now i take all records from another db and iterating through it.
if (!key_exists($already_imported[$new_id])){
import_function($new_id)
}else{
echo 'allready imported'.$already_imported[$new_id];
}
Now my script is importing same records for more than one time. I am not
able to get through this issue
Is it because of the size of the records or something else...?
Please suggest me some solution which is faster, safe and easy to code :D
Thanks in advance
--
Devendra Jadhav
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
