Re: [PHP] Read directory; store filenames found in mySQL table?
Kenneth Sande wrote: > I use the glob function in my little homemade "web cam" page, which can > really swell up in memory when used against a large amount of files (in > my case around 30k files). +1 for glob() -- Thanks! -Shawn http://www.spidean.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Read directory; store filenames found in mySQL table?
On Wed, Jan 13, 2010 at 10:50 AM, Rahul S. Johari
wrote:
>
> On Jan 13, 2010, at 10:40 AM, Rahul S. Johari wrote:
>
>>
>> On Jan 13, 2010, at 9:50 AM, Warren Windvogel wrote:
>>
>>> On 2010/01/13 04:25 PM, Rahul S. Johari wrote:
Ave,
This is what I'm trying to do; I want to read a directory (eg: W:\Test\)
and take all the filenames found in the directory (eg: 1.vox, 2.wav, 3.txt)
and store them in a simple mySQL table.
Can I do this?
>>>
>>> I tried to very quickly convert something I've done. It may need some
>>> work. Will work in linux env.
>>>
>>> $origin = "Path"
>>>
>>> #load file listing into an array
>>> $shell = shell_exec("du $origin");
>>> $array = array_reverse(explode("\n",$shell));
>>> $contIdArr = array();
>>>
>>> $newArr = array();
>>> foreach($array as $elem){
>>> $newDir = "";
>>> $pathArray = explode("/", $elem);
>>> $nodeDepth = count($pathArray);
>>> for($count=1; $count<$nodeDepth; $count++){
>>> $newDir = $newDir.$pathArray[$count].'/';
>>> }
>>> $newArr[] = '/'.$newDir;
>>> }
>>> sort($newArr);
>>>
>>>
>>> foreach($newArr as $dir){
>>> $pathArray = explode("/", $dir);
>>>
>>> $fileListArr = dirList($dir);
>>>
>>> foreach($fileListArr as $file){
>>> //Insert file($file) and current dir/path($dir) into db
>>> }
>>> }
>>>
>>> Kind regards
>>> Warren
>>>
>>
>>
>> Warren,
>>
>> I tried using your code and it definitely is very efficient & fast;
>> however I'm running into a small problem and I'm not sure how to correct it.
>> I'm getting the array with filenames from the folder I'm searching in PLUS
>> all the root folders of the machine as well.
>>
>> This is the code I'm using (note that I'm just echoing the array right
>> now; I'll move to inserting data into mySQL after):
>>
>> function dirList ($directory) {
>> $results = array();
>> $handler = opendir($directory);
>> while ($file = readdir($handler)) {
>> if ($file != '.' && $file != '..')
>> $results[] = $file;
>> }
>> closedir($handler);
>> return $results;
>> }
>>
>> $origin = "/Library/WebServer/Documents/folder1/folder2/images/";
>>
>> #load file listing into an array
>> $shell = shell_exec("du $origin");
>> $array = array_reverse(explode("\n",$shell));
>> $contIdArr = array();
>>
>> $newArr = array();
>> foreach($array as $elem){
>> $newDir = "";
>> $pathArray = explode("/", $elem);
>> $nodeDepth = count($pathArray);
>> for($count=1; $count<$nodeDepth; $count++){
>> $newDir = $newDir.$pathArray[$count].'/';
>> }
>> $newArr[] = '/'.$newDir;
>> }
>> sort($newArr);
>>
>> foreach($newArr as $dir){
>> $pathArray = explode("/", $dir);
>> $fileListArr = dirList($dir);
>>
>> foreach($fileListArr as $file){
>> echo $file."";
>> //Insert file($file) and current dir/path($dir) into db
>> }
>> }
>>
>>
>> As an output ... i get a list of all the files in the "images" folder
>> preceeded by the all the list of root folders on my machine!! How do I
>> eliminate the list of root folders?
>
>
> Nevermind, I was looking at the wrong output. I got it!! I've got all my
> filenames in my $fileListArr[] array!!
> Now I just to get the values in a mySQL table.
>
>
> ---
> Rahul Sitaram Johari
> Founder, Internet Architects Group, Inc.
>
> [Email] sleepwal...@rahulsjohari.com
> [Web] http://www.rahulsjohari.com
>
>
>
>
>
> --
> PHP General Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>
consider stacking the insert statements in sql to allow for a certain
number of inserts in one connect.
insert into my_table (field1, field2...fieldn)
values('field1','field2'...fieldn),('field1','field2'...fieldn),('field1','field2'...fieldn),('field1','field2'...fieldn)...
keep to something like 100 to avoid buffer overflows and it should
make the inserts much faster
--
Bastien
Cat, the other other white meat
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Read directory; store filenames found in mySQL table? :: SOLVED!!
Thanks All (Especially Warren); Final Code:
function dirList ($directory) {
$results = array();
$handler = opendir($directory);
while ($file = readdir($handler)) {
if ($file != '.' && $file != '..')
$results[] = $file;
}
closedir($handler);
return $results;
}
$origin = "pathto/images/";
#load file listing into an array
$shell = shell_exec("du $origin");
$array = array_reverse(explode("\n",$shell));
$contIdArr = array();
$newArr = array();
foreach($array as $elem){
$newDir = "";
$pathArray = explode("/", $elem);
$nodeDepth = count($pathArray);
for($count=1; $count<$nodeDepth; $count++){
$newDir = $newDir.$pathArray[$count].'/';
}
$newArr[] = '/'.$newDir;
}
sort($newArr);
foreach($newArr as $dir){
$pathArray = explode("/", $dir);
$fileListArr = dirList($dir);
}
$db = mysql_connect("localhost","usr","pwd");
mysql_select_db("db",$db);
foreach($fileListArr AS $value) {
$sql="INSERT INTO r2 (ID, RECORDING) VALUES('','$value')";
$result = mysql_query($sql) or die (mysql_error());
}
---
Rahul Sitaram Johari
Founder, Internet Architects Group, Inc.
[Email] sleepwal...@rahulsjohari.com
[Web] http://www.rahulsjohari.com
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Read directory; store filenames found in mySQL table?
On Jan 13, 2010, at 10:40 AM, Rahul S. Johari wrote:
On Jan 13, 2010, at 9:50 AM, Warren Windvogel wrote:
On 2010/01/13 04:25 PM, Rahul S. Johari wrote:
Ave,
This is what I'm trying to do; I want to read a directory (eg: W:
\Test\) and take all the filenames found in the directory (eg:
1.vox, 2.wav, 3.txt) and store them in a simple mySQL table.
Can I do this?
I tried to very quickly convert something I've done. It may need
some work. Will work in linux env.
$origin = "Path"
#load file listing into an array
$shell = shell_exec("du $origin");
$array = array_reverse(explode("\n",$shell));
$contIdArr = array();
$newArr = array();
foreach($array as $elem){
$newDir = "";
$pathArray = explode("/", $elem);
$nodeDepth = count($pathArray);
for($count=1; $count<$nodeDepth; $count++){
$newDir = $newDir.$pathArray[$count].'/';
}
$newArr[] = '/'.$newDir;
}
sort($newArr);
foreach($newArr as $dir){
$pathArray = explode("/", $dir);
$fileListArr = dirList($dir);
foreach($fileListArr as $file){
//Insert file($file) and current dir/path($dir) into db
}
}
Kind regards
Warren
Warren,
I tried using your code and it definitely is very efficient & fast;
however I'm running into a small problem and I'm not sure how to
correct it. I'm getting the array with filenames from the folder I'm
searching in PLUS all the root folders of the machine as well.
This is the code I'm using (note that I'm just echoing the array
right now; I'll move to inserting data into mySQL after):
function dirList ($directory) {
$results = array();
$handler = opendir($directory);
while ($file = readdir($handler)) {
if ($file != '.' && $file != '..')
$results[] = $file;
}
closedir($handler);
return $results;
}
$origin = "/Library/WebServer/Documents/folder1/folder2/images/";
#load file listing into an array
$shell = shell_exec("du $origin");
$array = array_reverse(explode("\n",$shell));
$contIdArr = array();
$newArr = array();
foreach($array as $elem){
$newDir = "";
$pathArray = explode("/", $elem);
$nodeDepth = count($pathArray);
for($count=1; $count<$nodeDepth; $count++){
$newDir = $newDir.$pathArray[$count].'/';
}
$newArr[] = '/'.$newDir;
}
sort($newArr);
foreach($newArr as $dir){
$pathArray = explode("/", $dir);
$fileListArr = dirList($dir);
foreach($fileListArr as $file){
echo $file."";
//Insert file($file) and current dir/path($dir) into db
}
}
As an output ... i get a list of all the files in the "images"
folder preceeded by the all the list of root folders on my machine!!
How do I eliminate the list of root folders?
Nevermind, I was looking at the wrong output. I got it!! I've got all
my filenames in my $fileListArr[] array!!
Now I just to get the values in a mySQL table.
---
Rahul Sitaram Johari
Founder, Internet Architects Group, Inc.
[Email] sleepwal...@rahulsjohari.com
[Web] http://www.rahulsjohari.com
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Read directory; store filenames found in mySQL table?
On Jan 13, 2010, at 9:50 AM, Warren Windvogel wrote:
On 2010/01/13 04:25 PM, Rahul S. Johari wrote:
Ave,
This is what I'm trying to do; I want to read a directory (eg: W:
\Test\) and take all the filenames found in the directory (eg:
1.vox, 2.wav, 3.txt) and store them in a simple mySQL table.
Can I do this?
I tried to very quickly convert something I've done. It may need
some work. Will work in linux env.
$origin = "Path"
#load file listing into an array
$shell = shell_exec("du $origin");
$array = array_reverse(explode("\n",$shell));
$contIdArr = array();
$newArr = array();
foreach($array as $elem){
$newDir = "";
$pathArray = explode("/", $elem);
$nodeDepth = count($pathArray);
for($count=1; $count<$nodeDepth; $count++){
$newDir = $newDir.$pathArray[$count].'/';
}
$newArr[] = '/'.$newDir;
}
sort($newArr);
foreach($newArr as $dir){
$pathArray = explode("/", $dir);
$fileListArr = dirList($dir);
foreach($fileListArr as $file){
//Insert file($file) and current dir/path($dir) into db
}
}
Kind regards
Warren
Warren,
I tried using your code and it definitely is very efficient & fast;
however I'm running into a small problem and I'm not sure how to
correct it. I'm getting the array with filenames from the folder I'm
searching in PLUS all the root folders of the machine as well.
This is the code I'm using (note that I'm just echoing the array right
now; I'll move to inserting data into mySQL after):
function dirList ($directory) {
$results = array();
$handler = opendir($directory);
while ($file = readdir($handler)) {
if ($file != '.' && $file != '..')
$results[] = $file;
}
closedir($handler);
return $results;
}
$origin = "/Library/WebServer/Documents/folder1/folder2/images/";
#load file listing into an array
$shell = shell_exec("du $origin");
$array = array_reverse(explode("\n",$shell));
$contIdArr = array();
$newArr = array();
foreach($array as $elem){
$newDir = "";
$pathArray = explode("/", $elem);
$nodeDepth = count($pathArray);
for($count=1; $count<$nodeDepth; $count++){
$newDir = $newDir.$pathArray[$count].'/';
}
$newArr[] = '/'.$newDir;
}
sort($newArr);
foreach($newArr as $dir){
$pathArray = explode("/", $dir);
$fileListArr = dirList($dir);
foreach($fileListArr as $file){
echo $file."";
//Insert file($file) and current dir/path($dir) into db
}
}
As an output ... i get a list of all the files in the "images" folder
preceeded by the all the list of root folders on my machine!! How do I
eliminate the list of root folders?
---
Rahul Sitaram Johari
Founder, Internet Architects Group, Inc.
[Email] sleepwal...@rahulsjohari.com
[Web] http://www.rahulsjohari.com
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Read directory; store filenames found in mySQL table?
On Jan 13, 2010, at 10:07 AM, Kenneth Sande wrote: Ashley Sheridan wrote: On Wed, 2010-01-13 at 09:25 -0500, Rahul S. Johari wrote: Ave, This is what I'm trying to do; I want to read a directory (eg: W: \Test \) and take all the filenames found in the directory (eg: 1.vox, 2.wav, 3.txt) and store them in a simple mySQL table. Can I do this? --- Rahul Sitaram Johari Founder, Internet Architects Group, Inc. [Email] sleepwal...@rahulsjohari.com [Web] http://www.rahulsjohari.com You'll probably want to look at the readdir() function. The manual page also has dozens of different example scripts that would be easy to tweak for your purpose. http://php.net/manual/en/function.readdir.php Thanks, Ash http://www.ashleysheridan.co.uk I use the glob function in my little homemade "web cam" page, which can really swell up in memory when used against a large amount of files (in my case around 30k files). = $imgdir = 'img/south*'; $files = glob( $imgdir ); // Sort files by modified time, latest to earliest // Use SORT_ASC in place of SORT_DESC for earliest to latest array_multisort( array_map( 'filemtime', $files ), SORT_NUMERIC, SORT_DESC, $files ); = http://www.php.net/manual/en/function.glob.php DISCLAIMER: I found this code on a how-to somewhere out there and modified it to fit my need. Quite possibly there are much better means to this end. Ken Sande/KC8QNI Considering that I have over 80K files in the folder, would this be a faster/efficient then the readdir() method? Or should I stick to what I'm doing (other email)? --- Rahul Sitaram Johari Founder, Internet Architects Group, Inc. [Email] sleepwal...@rahulsjohari.com [Web] http://www.rahulsjohari.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Read directory; store filenames found in mySQL table?
On Jan 13, 2010, at 9:56 AM, Warren Windvogel wrote:
On 2010/01/13 04:25 PM, Rahul S. Johari wrote:
Ave,
This is what I'm trying to do; I want to read a directory (eg: W:
\Test\) and take all the filenames found in the directory (eg:
1.vox, 2.wav, 3.txt) and store them in a simple mySQL table.
Sorry. Forgot to include this.
function dirList ($directory)
{
// create an array to hold directory list
$results = array();
// create a handler for the directory
$handler = opendir($directory);
// keep going until all files in directory have been read
while ($file = readdir($handler)) {
// if $file isn't this directory or its parent,
// add it to the results array
if ($file != '.' && $file != '..')
$results[] = $file;
}
// tidy up: close the handler
closedir($handler);
// done!
return $results;
}
If you're dealing with 1 directory you can use it to read all files
in it to an array.
Kind regards
Warren
This is an interesting approach. Following is what I came up with to
scan a directory and store the filenames into an array ... very
similar to your example:
$listDir = array();
$dir = "../mounts/wd/IDT/IDT/";
if($handler = opendir($dir)) {
while (($sub = readdir($handler)) !== FALSE) {
if ($sub != "." && $sub != ".." && $sub !=
"Thumb.db") {
if(is_file($dir."/".$sub)) {
$listDir[] = $sub;
}elseif(is_dir($dir."/".$sub)){
$listDir[$sub] = $this-
>ReadFolderDirectory($dir."/".$sub);
}
}
}
closedir($handler);
}
and this is what I'm trying to implement in order to store the array
into a mysql table ..
$db = mysql_connect("localhost","usr","pwd");
mysql_select_db("db",$db);
$colors=serialize($listDir); //takes the data from a post
operation...
$sql="INSERT INTO recordings (ID, RECORDING, ADDED)
VALUES('','$colors','')";
$result = mysql_query($sql) or die (mysql_error());
I'm not sure if this the best or fastest approach ... but it's running
in the background as I write this (I should have tested on a smaller
folder). The folder I'm scanning has literally over 80,000 files ...
so it's taking LOOONG!!
---
Rahul Sitaram Johari
Founder, Internet Architects Group, Inc.
[Email] sleepwal...@rahulsjohari.com
[Web] http://www.rahulsjohari.com
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Read directory; store filenames found in mySQL table?
Ashley Sheridan wrote: On Wed, 2010-01-13 at 09:25 -0500, Rahul S. Johari wrote: Ave, This is what I'm trying to do; I want to read a directory (eg: W:\Test \) and take all the filenames found in the directory (eg: 1.vox, 2.wav, 3.txt) and store them in a simple mySQL table. Can I do this? --- Rahul Sitaram Johari Founder, Internet Architects Group, Inc. [Email] sleepwal...@rahulsjohari.com [Web] http://www.rahulsjohari.com You'll probably want to look at the readdir() function. The manual page also has dozens of different example scripts that would be easy to tweak for your purpose. http://php.net/manual/en/function.readdir.php Thanks, Ash http://www.ashleysheridan.co.uk I use the glob function in my little homemade "web cam" page, which can really swell up in memory when used against a large amount of files (in my case around 30k files). = $imgdir = 'img/south*'; $files = glob( $imgdir ); // Sort files by modified time, latest to earliest // Use SORT_ASC in place of SORT_DESC for earliest to latest array_multisort( array_map( 'filemtime', $files ), SORT_NUMERIC, SORT_DESC, $files ); = http://www.php.net/manual/en/function.glob.php DISCLAIMER: I found this code on a how-to somewhere out there and modified it to fit my need. Quite possibly there are much better means to this end. Ken Sande/KC8QNI -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Read directory; store filenames found in mySQL table?
On Wed, 2010-01-13 at 09:25 -0500, Rahul S. Johari wrote: > Ave, > > This is what I'm trying to do; I want to read a directory (eg: W:\Test > \) and take all the filenames found in the directory (eg: 1.vox, > 2.wav, 3.txt) and store them in a simple mySQL table. > > Can I do this? > > --- > Rahul Sitaram Johari > Founder, Internet Architects Group, Inc. > > [Email] sleepwal...@rahulsjohari.com > [Web] http://www.rahulsjohari.com > > > > > You'll probably want to look at the readdir() function. The manual page also has dozens of different example scripts that would be easy to tweak for your purpose. http://php.net/manual/en/function.readdir.php Thanks, Ash http://www.ashleysheridan.co.uk
