I have to add that, in his initial description, Sergey pointed out the
search takes 10-20 seconds before the output can be displayed. He was
looking for ways to improve performance. If he cares about "performance" in
terms of "latency" then a second thread that starts printing results as
soon as they are available might be all he needs! :)
Cheers,
Jafar
On Mon, Jan 12, 2015 at 3:48 PM, Jafar Al-Gharaibeh <[email protected]>
wrote:
> Well, stat() works fine, it is the one used by the example code at
> RosettaCode which Sergey used. We are just spoiled by the the ability of
> open() to filter based on names with wild cards. To get all of Unicon
> source files in a given directory I suggested
>
> open("*.icn")
>
> But if you have to do it recursively for sub-directories you still have
> to loop through all files (traversing .icn again) to get directory names,
> which kind of defeats the purpose of the initial open("*.icn") in this
> case. That is why open("directories only") would be useful here. I have
> seen a lot of situation where such feature is useful. Of course it can all
> be done using stat(), but this whole discussion started on the basis of
> performance where the current implementation of the problem was not "fast
> enough". open()'s built-in filtering might help a little but it doesn't
> provide everything needed to completely rely on it for filtering.
>
> Cheers,
> Jafar
>
>
> On Mon, Jan 12, 2015 at 2:22 PM, Jeffery, Clint ([email protected]) <
> [email protected]> wrote:
>
>> Hi,
>>
>>
>> I am all for implementing a library procedure, or class, to do this as
>> part of the Unicon distribution, if one is not there already. I am
>> wondering why I haven't noticed someone chime in with the stat() function,
>> it is the building block one uses to read file attributes, determine if
>> something is a directory, etc. I do know the Unicon book has an example
>> program that walks through directories recursively.
>>
>>
>> Cheers,
>>
>> Clint
>> ------------------------------
>> *From:* Sergey Logichev <[email protected]>
>> *Sent:* Monday, January 12, 2015 3:05 AM
>> *To:* David Gamey; Jafar Al-Gharaibeh
>> *Cc:* Unicon group
>>
>> *Subject:* Re: [Unicon-group] Walk of file directory
>>
>> David,
>>
>> Ok, I will try to implement it myself at first. If I will be in trouble I
>> ask you help, sure. In any case I will inform Unicon group about it.
>> It would be great to implement file search directly in Unicon. As Jafar
>> mentioned, we already may do it within current directory with help of
>> open() function. For example, open("*.icn") does filtering of icn files in
>> the directory. Next step would be logical to do search recursively in all
>> subdirectories. For example, something like
>> open("rootdir/*.icn","recursively").
>>
>> Sergey
>>
>> 11.01.2015, 22:25, "David Gamey" <[email protected]>:
>>
>> Sergey,
>>
>> I am responsible for much of the Rosetta code contributions (thanks also
>> to Steve, Andrew, Matt, Peter, and about 4 others) and this one in
>> particular dating from 2010. As I recall this was before the
>> multi-threading versions were widely available. I think multi-threading is
>> underrepresented in Rosetta/Unicon.
>>
>> If you come up with a multi-threading version, we should add it to the
>> post as an alternative version. If you don't feel comfortable doing this,
>> post the code and I can add it.
>>
>> David
>>
>>
>> ------------------------------
>> *From:* Sergey Logichev <[email protected]>
>> *To:* Jafar Al-Gharaibeh <[email protected]>
>> *Cc:* Unicon group <[email protected]>
>> *Sent:* Sunday, January 11, 2015 1:16 AM
>> *Subject:* Re: [Unicon-group] Walk of file directory
>>
>> Jafar,
>>
>> Thank you for a whole bundle of advices and suggestions! Threads are
>> worth to try. The thought of search by file attributes is very useful too.
>> Your suggestion about slow I/O partly is right. For UNIX I tried the
>> program on Raspberry Pi with 6 Class microSD as HDD (it's slow, agree). But
>> for Windows it was quite fast HDD. It would be interesting to compare
>> performance of the program on Windows with classic approach based on Win32
>> _FINDFIRST, _FINDNEXT functions. I have threaded Delphi/Lazarus
>> implementations of this algorithm. Feel that it will be faster but in which
>> degree?
>>
>> Sergey
>>
>> 10.01.2015, 21:50, "Jafar Al-Gharaibeh" <[email protected]>:
>>
>>
>> Sergey,
>>
>> There are so many things that came to mind when I saw your program.
>>
>> 1- At the end of your email, sourceforge ad says "Go Parallel", Which is
>> not a bad idea for this highly parallel application.
>>
>> There is a similar program "wordcount" listed in my dissertation
>> (available on unicon.org) that go through directories and count words in
>> every file using threads (Chapter 7, page 107)
>>
>> 2- Unicon open() already supports " pattern matching that would greatly
>> (I believe) speedup your program. For example you can do this:
>> L := open("*.icn")
>>
>> to get a list of all of Unicon source files in the current directory.
>>
>> Note: It would be nice if there were a way to tell open() to return
>> files not only based on a pattern, but also on file attribute to allow
>> something like "get me all directories in the current directory", or "get
>> me all read only file". There are a lot of situations where filtering
>> directory names for example is very useful - like this program
>>
>> 3- The program on Rosetta Code is not optimized for speed. You can
>> minimize the number of lists created and put() by careful rewriting of the
>> code.
>>
>> 4- Depending on how deep the directory tree is, there might be a lot of
>> I/O going on. A slow disk might limit how fast you can go regardless of how
>> optimized your code is.
>>
>> I will share results if get around trying any of these options.
>>
>> Cheers,
>> Jafar
>>
>>
>>
>> On Sat, Jan 10, 2015 at 5:51 AM, Sergey Logichev <[email protected]>
>> wrote:
>>
>> Hello all!
>>
>> Now I investigate the best approach to get list of files in specified
>> directory and beneath in Unicon.
>> I found excellent example at rosettacode.org:
>> http://rosettacode.org/wiki/Walk_a_directory/Recursively#Icon_and_Unicon
>>
>> I reconstructed this one to implement matching of filenames to specified
>> pattern (regular expression). My program recursively walks a directory and
>> prints appropriate filenames. The same as dir (ls) does. All working fine
>> except performance. If directory has a lot of subdirs the search may took
>> 10-20 seconds before starting output. Could you provide some advices how to
>> enchance the performance?
>>
>> Some notes how to make and use. Unpack content of udir.zip to your local
>> directory. Define which environment you use in env.icn file - uncomment
>> line "$define _UNIX 1" in the case of UNIX. Nothing to do in the case of
>> Windows.
>> Make udir program:
>> unicon -c futils.icn
>> unicon -c options.icn
>> unicon -c regexp.icn
>> unicon udir.icn
>>
>> Usage: udir -f<filemask>
>> for example: udir -f*.icn
>> shall list of icn files in the current dir and all its subdirectories.
>>
>> Best regards,
>> Sergey Logichev
>>
>>
>> ------------------------------------------------------------------------------
>> Dive into the World of Parallel Programming! The Go Parallel Website,
>> sponsored by Intel and developed in partnership with Slashdot Media, is
>> your
>> hub for all things parallel software development, from weekly thought
>> leadership blogs to news, videos, case studies, tutorials and more. Take a
>> look and join the conversation now. http://goparallel.sourceforge.net
>> _______________________________________________
>> Unicon-group mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/unicon-group
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Dive into the World of Parallel Programming! The Go Parallel Website,
>> sponsored by Intel and developed in partnership with Slashdot Media, is
>> your
>> hub for all things parallel software development, from weekly thought
>> leadership blogs to news, videos, case studies, tutorials and more. Take a
>> look and join the conversation now. http://goparallel.sourceforge.net
>>
>> _______________________________________________
>> Unicon-group mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/unicon-group
>>
>>
>>
>
------------------------------------------------------------------------------
New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
GigeNET is offering a free month of service with a new server in Ashburn.
Choose from 2 high performing configs, both with 100TB of bandwidth.
Higher redundancy.Lower latency.Increased capacity.Completely compliant.
www.gigenet.com
_______________________________________________
Unicon-group mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/unicon-group
