RE: [PHP] How Much Data Can Be Written to a Database In One Instance?
> Date: Wed, 4 Mar 2009 15:29:19 -0500 > Subject: Re: [PHP] How Much Data Can Be Written to a Database In One Instance? > From: danbr...@php.net > To: rob...@interjinn.com > CC: aj...@alumni.iu.edu; php-general@lists.php.net > > On Wed, Mar 4, 2009 at 15:26, Robert Cummings wrote: > > > > See this setting in your php.ini: > > > > memory_limit = ???M > > Also let us know what you're using for the database and how you're > doing it. You mention that the .dat files are 4-5MB, but don't say if > that's the database you're using, or if you're reading from those and > entering the data in MySQL, or what. > > For example, if you're inserting data from there into MySQL, it's > probably a MySQL issue that could be fixed by adjusting the packet > size in your /etc/my.cnf (or equivalent) configuration file. > > -- > > daniel.br...@parasane.net || danbr...@php.net > http://www.parasane.net/ || http://www.pilotpig.net/ > 50% Off All Shared Hosting Plans at PilotPig: Use Coupon DOW1 Hi, Daniel: I went back to my problem and looks like I could change the output_buffering and the query timeout time. Alice _ Use Messenger to talk to your IM friends, even those on Yahoo! http://ideas.live.com/programpage.aspx?versionId=7adb59de-a857-45ba-81cc-685ee3e858fe
RE: [PHP] How Much Data Can Be Written to a Database In One Instance?
_ Use Messenger to talk to your IM friends, even those on Yahoo! http://ideas.live.com/programpage.aspx?versionId=7adb59de-a857-45ba-81cc-685ee3e858fe -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] How Much Data Can Be Written to a Database In One Instance?
> Subject: Re: [PHP] How Much Data Can Be Written to a Database In One > Instance? > From: rob...@interjinn.com > To: aj...@alumni.iu.edu > CC: php-general@lists.php.net > Date: Wed, 4 Mar 2009 15:26:38 -0500 > > On Wed, 2009-03-04 at 15:05 -0500, Alice Wei wrote: > > Hi, > > > > I am working on a series of long files that contain 4 or 5 MB each, each > > of them is encoded in UTF-8 saved as .dat format. > > However, I have noticed that when I do insert statements, the file only > > reads up to around 1MB and then would stop. If I split the file into 3 or 4 > > files, that would hold no more than 1.2 MB each, then I won't have > > problems. > > > > Is there a limitation on how much data can be read and written in PHP > > within one instance? > > See this setting in your php.ini: > > memory_limit = ???M > > Cheers, > Rob. > -- > http://www.interjinn.com > Application and Templating Framework for PHP > Weird, I had memory_limit = 128M ; Maximum amount of memory a script may consume (128MB) However, the file that I was trying to read from was 5MB of text file. I tried to read it line by line and also ran one query for each line along the way before I got it to the database. If I have a file that is around 1MB, I have no problems, but once it passes that, a lot of lines go missing. Alice _ Check the weather nationwide with MSN Search: Try it now! http://search.msn.com/results.aspx?q=weather&FORM=WLMTAG
RE: [PHP] How Much Data Can Be Written to a Database In One Instance?
> Subject: Re: [PHP] How Much Data Can Be Written to a Database In One > Instance? > From: rob...@interjinn.com > To: aj...@alumni.iu.edu > CC: php-general@lists.php.net > Date: Wed, 4 Mar 2009 15:26:38 -0500 > > On Wed, 2009-03-04 at 15:05 -0500, Alice Wei wrote: > > Hi, > > > > I am working on a series of long files that contain 4 or 5 MB each, each > > of them is encoded in UTF-8 saved as .dat format. > > However, I have noticed that when I do insert statements, the file only > > reads up to around 1MB and then would stop. If I split the file into 3 or 4 > > files, that would hold no more than 1.2 MB each, then I won't have > > problems. > > > > Is there a limitation on how much data can be read and written in PHP > > within one instance? > > See this setting in your php.ini: > > memory_limit = ???M > > Cheers, > Rob. > -- > http://www.interjinn.com > Application and Templating Framework for PHP > Weird, I had memory_limit = 128M ; Maximum amount of memory a script may consume (128MB) However, the file that I was trying to read from was 5MB of text file. I tried to read it line by line and also ran one query for each line along the way before I got it to the database. If I have a file that is around 1MB, I have no problems, but once it passes that, a lot of lines go missing. Alice _ Express yourself with gadgets on Windows Live Spaces http://discoverspaces.live.com?source=hmtag1&loc=us
Re: [PHP] How Much Data Can Be Written to a Database In One Instance?
On Wed, Mar 4, 2009 at 15:26, Robert Cummings wrote: > > See this setting in your php.ini: > > memory_limit = ???M Also let us know what you're using for the database and how you're doing it. You mention that the .dat files are 4-5MB, but don't say if that's the database you're using, or if you're reading from those and entering the data in MySQL, or what. For example, if you're inserting data from there into MySQL, it's probably a MySQL issue that could be fixed by adjusting the packet size in your /etc/my.cnf (or equivalent) configuration file. -- daniel.br...@parasane.net || danbr...@php.net http://www.parasane.net/ || http://www.pilotpig.net/ 50% Off All Shared Hosting Plans at PilotPig: Use Coupon DOW1 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] How Much Data Can Be Written to a Database In One Instance?
On Wed, 2009-03-04 at 15:05 -0500, Alice Wei wrote: > Hi, > > I am working on a series of long files that contain 4 or 5 MB each, each of > them is encoded in UTF-8 saved as .dat format. > However, I have noticed that when I do insert statements, the file only > reads up to around 1MB and then would stop. If I split the file into 3 or 4 > files, that would hold no more than 1.2 MB each, then I won't have problems. > > Is there a limitation on how much data can be read and written in PHP > within one instance? See this setting in your php.ini: memory_limit = ???M Cheers, Rob. -- http://www.interjinn.com Application and Templating Framework for PHP -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
