Re: Check whether a range is empty
On Sunday, 15 July 2018 at 12:18:27 UTC, Steven Schveighoffer wrote: On 7/15/18 7:45 AM, vino.B wrote: [...] I still don't know why you are using chain here as it equates to the identity function in this instance: https://github.com/dlang/phobos/blob/3d41721141f31059ca6e77dec2da390fad737955/std/range/package.d#L900 [...] So I'm assuming from your assertion that this works, that the range is not empty, but yields no elements when it's iterated? Seems like a bug to me. -Steve Hi Steve, Initially i thought the using "chain" it will merge several ranges for arrays into single range of array, but it doesn't merge into single range of array so i have removed the same from my code. From, Vino.B
Re: Check whether a range is empty
On Saturday, 14 July 2018 at 17:20:52 UTC, Ali Çehreli wrote:
First, please show us code that demonstrates the issue.
On 07/14/2018 07:47 AM, vino.B wrote:
>The reason it never prints the text "Empty" is that the
out of the
> "r" is just an empty array.
>
> OUTPUT:
> []
> []
If that's the output of r, then r is not empty but has two
elements and those elements are likely arrays. If they are in
fact arrays, them being empty does not change r: it still has
two elements.
If you want to treat the range as empty when all its elements
are empty, then perhaps your problem needs
std.algorithm.joiner. The following program demonstrates your
issue with the first assert and the fix with the second assert:
import std.algorithm;
import std.range;
void main() {
int[][] r = [ [], [] ];
assert(!r.empty);
auto joined_r = r.joiner;
assert(joined_r.empty);
}
joiner joins elements that are ranges themselves. For example,
joiner([ [1], [2] ]) is equal to [ 1, 2 ].
Ali
HI Ali,
Thank you very much, but unfortunately the above solution did
not work as the variable PFResult contains the output from the
workerLocalStorgage which is prited as PFResult.toRange , but was
able to find a solution as below
void ptThreadManager(alias coRoutine, T...)(Array!string Dirlst,
T params)
{
alias scRType = typeof(coRoutine(string.init, T.init));
auto PFresult = taskPool.workerLocalStorage!scRType();
PFresult.get ~= coRoutine(FFs, params); }
int a = 0;
if (!(PFresult.toRange).empty) {
foreach(i; chain(PFresult.toRange)) { writeln(i[]); a = a
+1;} }
if(a == 0) { writeln("No files");
}
From,
Vino.B
rmdirRecurse equivalent Windows API
Hi All, The D function rmdirRecurse on windows works 80% and rest of the time it complains the the "The filename, directory name, or volume label syntax is incorrect." while accessing the file using UNC path, so is there any alternate such as Windows API to remove folder irrespective of whether the folder is empty or contains files. From, Vino.B
Re: setAttributes Issue.
On Sunday, 15 July 2018 at 10:07:49 UTC, vino.B wrote: Hi All, Request your help, can some one find what is the issue with the below code, as this is throwing the error "Access is denied". as the below code is supposed to to remove the read-only permission if set. [...] Hi All, Was able to resolve this issue by removing the filter criteria auto sDir = Array!string (dirEntries(d[0], SpanMode.depth).filter!(a => !a.isSymlink) .map!(a => > (a.name))); From, Vino.B
setAttributes Issue.
Hi All,
Request your help, can some one find what is the issue with the
below code, as this is throwing the error "Access is denied". as
the below code is supposed to to remove the read-only permission
if set.
Code:
import std.algorithm;
import std.array;
import std.container.array;
import std.datetime.systime;
import std.file;
import std.stdio;
import std.typecons;
auto coAgedDirClean (string FFs) {
auto dFiles = Array!(Tuple!(string,
SysTime))(dirEntries(join([`\\?\`, FFs]),
SpanMode.shallow).filter!(a => a.isDir).map!(a => tuple(a.name,
a.timeCreated)));
foreach(d; dFiles) {
auto sDir = Array!string (dirEntries(d[0],
SpanMode.depth).filter!(a => a.isDir && !a.isSymlink).map!(a =>
(a.name)));
foreach(DirEntry sd; sDir) {
import core.sys.windows.windows;
if ((sd.attributes & FILE_ATTRIBUTE_READONLY) != 0) {
setAttributes(sd, sd.attributes & ~FILE_ATTRIBUTE_READONLY); }
rmdirRecurse(sd);
}}
dFiles.each!(f => rmdirRecurse(f[0]));
return dFiles;
}
void main () {
string FFs = "C:\\Temp\\SAPNAS1\\BACKUP1";
writeln(coAgedDirClean(FFs)[]);
}
From,
Vino.B
Re: Check whether a range is empty
On Saturday, 14 July 2018 at 14:28:52 UTC, vino.B wrote:
On Friday, 13 July 2018 at 19:45:03 UTC, Steven Schveighoffer
wrote:
On 7/13/18 3:29 PM, vino.B wrote:
[...]
Well, empty is how you detect whether any range is empty, and
as far as ranges are concerned, your code is correctly
checking for empty.
A couple comments:
1. Why are you using chain with a single parameter? That just
returns its parameter.
2. You may want to try grabbing the range ONCE. i.e.:
auto r = PFresult.toRange;
if(!r.empty)
{
foreach(i; r) ...
}
I'm not familiar with how taskPool works, so I don't know the
real answer.
-Steve
Hi Steve,
i Tried your method no luck, it just prints blank lines.
From,
Vino.B
Hi Steve,
The reason it never prints the text "Empty" is that the out of
the "r" is just an empty array.
OUTPUT:
[]
[]
From,
Vino.B
Re: Check whether a range is empty
On Friday, 13 July 2018 at 19:45:03 UTC, Steven Schveighoffer
wrote:
On 7/13/18 3:29 PM, vino.B wrote:
[...]
Well, empty is how you detect whether any range is empty, and
as far as ranges are concerned, your code is correctly checking
for empty.
A couple comments:
1. Why are you using chain with a single parameter? That just
returns its parameter.
2. You may want to try grabbing the range ONCE. i.e.:
auto r = PFresult.toRange;
if(!r.empty)
{
foreach(i; r) ...
}
I'm not familiar with how taskPool works, so I don't know the
real answer.
-Steve
Hi Steve,
i Tried your method no luck, it just prints blank lines.
From,
Vino.B
Re: Check whether a range is empty
On Friday, 13 July 2018 at 19:05:20 UTC, Steven Schveighoffer
wrote:
On 7/13/18 2:37 PM, vino.B wrote:
Hi All,
How do i check whether a range is empty. eg.
(!PFResutl.toRange).empty. I tired the below, but it is no
printing Empty if the range is empty it just prints blank line.
if (!(!PFResutl.toRange).empty) { writeln("Empty"); }
Without knowing what PFResutl is, let me simplify a bit:
if( ! (expr).empty) { writeln("Empty"); }
That exclamation point means "not". So you are first checking
if the range is NOT empty, and if so, printing "Empty". Is that
what you meant?
-Steve
Hi Steve,
Sorry there was a typo mistake, so the PFResult contain the
results of "taskPool.workerLocalStorage" which i print using
writeln(PFResult.toRange) so the requirement is that if the rage
is empty it has to print "Empty" else it should print the result.
Eg:
if (!(PFresult.toRange).empty) {
foreach(i; chain(PFresult.toRange)) { writeln(i[]); }
} else { writeln("Empty"); }
From,
Vino.B
Check whether a range is empty
Hi All,
How do i check whether a range is empty. eg.
(!PFResutl.toRange).empty. I tired the below, but it is no
printing Empty if the range is empty it just prints blank line.
if (!(!PFResutl.toRange).empty) { writeln("Empty"); }
From,
Vino.B
Re: Passing function(whose parameter would be dynamic and the type is unknown) as a parameter to another function.
On Tuesday, 10 July 2018 at 14:50:53 UTC, Alex wrote:
On Tuesday, 10 July 2018 at 14:38:03 UTC, vino.B wrote:
Hi Alex,
The reason the I am storing the output of "PFresult.toRange"
to another array "rData" is that the output of the
PFresult.toRange is different each time we execute the
code.(Data is correct) but the way the it output is different.
Is there any way to get the result in a single array - Whole
Data.
Single array - Whole Data
["C:\\Temp\\TEAM1\\Test Result-Team1.docx", 2018-Jun-28
17:37:45.9376229,
"C:\\Temp\\TEAM2\\Test Result-Team2.docx", 2018-Jun-28
17:37:45.9376229,
"C:\\Temp\\SAPNAS3\\TEAM3\\Test Result-Team3.docx",
2018-Jun-28 17:37:45.9376229
]
One array - For Each Data
[ C:\\Temp\\TEAM1\\Test Result-Team1.docx", 2018-Jun-28
17:37:45.9376229] - arr1
[ C:\\Temp\\TEAM2\\Test Result-Team2.docx", 2018-Jun-28
17:37:45.9376229] - arr2
[ C:\\Temp\\TEAM3\\Test Result-Team3.docx", 2018-Jun-28
17:37:45.9376229] - arr3
The code in the program.
foreach(i; PFresult.toRange) { rData ~= i[][]; }
if (!rData[].empty) { rData[].sort!((a,b) => a[1] <
b[1]).each!(e => logF.writefln!"%-83s
%.20s"(e[0].replace(`\\?\`, ""), e[1].to!string)); }
From,
Vino.B
Not sure, if I get your point, but are you aware of tuples?
https://dlang.org/phobos/std_typecons.html#tuple
Hi Alex,
I am getting the output as tuples of multiple arrays, but the
requirement is to get the all the tuple in a single array like
the below so that we can perform sorting and printing the output
is easy.
[
"C:\\Temp\\TEAM1\\Test Result-Team1.docx", 2018-Jun-28
17:37:45.9376229,
"C:\\Temp\\TEAM2\\Test Result-Team2.docx", 2018-Jun-28
17:37:45.9376229,
"C:\\Temp\\SAPNAS3\\TEAM3\\Test Result-Team3.docx", 2018-Jun-28
17:37:45.9376229
]
if (!rData[].empty)
{
rData[].sort!((a,b) => a[1] < b[1]).each!(e =>
logF.writefln!"%-83s %.20s"(e[0],
e[1].to!string));
}
From,
Vino.B
Re: Passing function(whose parameter would be dynamic and the type is unknown) as a parameter to another function.
On Monday, 9 July 2018 at 18:07:49 UTC, Alex wrote:
On Monday, 9 July 2018 at 17:26:30 UTC, vino.B wrote:
Request Help:
void process(alias coRoutine, T...)(Array!string Dirlst, T
params)
{
ReturnType!coRoutine rData; / This line is not
working
alias scRType = typeof(coRoutine(string.init, T.init));
auto PFresult = taskPool.workerLocalStorage!scRType();
foreach (string FFs; parallel(Dirlst[],1)) {
PFresult.get ~= coRoutine(FFs, params); }
foreach(i; PFresult.toRange) { rData ~= i[][]; }
}
Error:
test.d(206): Error: template instance
`std.traits.ReturnType!(coAgedDirClean)` does not match
template declaration ReturnType(func...) if (func.length == 1
&& isCallable!func)
Yeah... for ReturnType to work, you need a function, but you
have only a template.
The easy solution is to execute the template and to ask the
result for its type:
´´´
void main()
{
process!fun();
}
void process(alias coRoutine, T...)(T params)
{
auto res = coRoutine(params);
pragma(msg, typeof(res));
}
auto fun(T...)(T params)
{
return 42;
}
´´´
If you need it in advance... It is a little bit longer. There
was a place, where I used this once...
See
https://run.dlang.io/is/Xy6Xf4
However, I wonder why you need this, especially as your process
is void. Why not just using auto for results of the coroutines?
Hi Alex,
The reason the I am storing the output of "PFresult.toRange" to
another array "rData" is that the output of the PFresult.toRange
is different each time we execute the code.(Data is correct) but
the way the it output is different. Is there any way to get the
result in a single array - Whole Data.
Single array - Whole Data
["C:\\Temp\\TEAM1\\Test Result-Team1.docx", 2018-Jun-28
17:37:45.9376229,
"C:\\Temp\\TEAM2\\Test Result-Team2.docx", 2018-Jun-28
17:37:45.9376229,
"C:\\Temp\\SAPNAS3\\TEAM3\\Test Result-Team3.docx", 2018-Jun-28
17:37:45.9376229
]
One array - For Each Data
[ C:\\Temp\\TEAM1\\Test Result-Team1.docx", 2018-Jun-28
17:37:45.9376229] - arr1
[ C:\\Temp\\TEAM2\\Test Result-Team2.docx", 2018-Jun-28
17:37:45.9376229] - arr2
[ C:\\Temp\\TEAM3\\Test Result-Team3.docx", 2018-Jun-28
17:37:45.9376229] - arr3
The code in the program.
foreach(i; PFresult.toRange) { rData ~= i[][]; }
if (!rData[].empty) { rData[].sort!((a,b) => a[1] <
b[1]).each!(e => logF.writefln!"%-83s %.20s"(e[0].replace(`\\?\`,
""), e[1].to!string)); }
From,
Vino.B
Re: Passing function(whose parameter would be dynamic and the type is unknown) as a parameter to another function.
On Monday, 9 July 2018 at 15:49:50 UTC, Alex wrote:
On Monday, 9 July 2018 at 15:40:53 UTC, vino.B wrote:
On Sunday, 8 July 2018 at 19:10:24 UTC, Alex wrote:
On Sunday, 8 July 2018 at 18:46:31 UTC, vino.B wrote:
Request you help, in the below code we pass the function
"Testfun" as a parameter to another function "process" in
order for the function "process" to work we have to specify
the type of the parameter that is passed to the function "(T
function(string, int) coRoutine, string Test, int Size) ",
so now how do we pass a function whose parameter would be
dynamic and the type is unknown.
What do you mean with a "function with dynamic parameters"
and "unknown type"?
But how about
´´´
void main()
{
process!fun();
}
void process(alias coRoutine, T...)(T params)
{
coRoutine(params);
}
auto fun(T...)(T params)
{
}
´´´
HI Alex,
I tried you method, but it not working as expected, the
process!fun1(Test1) works but the process!fun2(Test1, Size )
does not work, instead of displaying the value (Test, 1) it
output's the parameter "Test1".
Output:
Test
Test1
If I comment process!fun1(Test1); then the output is same
Output :
Test1
import std.stdio: writeln;
void main()
{
string Test1 = "Test";
int Size = 1;
process!fun1(Test1);
process!fun2(Test1, Size );
}
void process(alias coRoutine, T...)(T params)
{
coRoutine(params);
}
auto fun1(T...)(T params)
{
writeln(params);
}
auto fun2(T...)(T params)
{
writeln(params);
}
From,
Vino.B
As I can see, this is the expected output.
The "1" after "Test" is the value of parameter "Size"...
Hi Alex,
Thank you a lot, i was able to improvise using your logic, and i
need more help, the below code is working as expected using your
logic, now how do i get the return type of the function
Request Help:
void process(alias coRoutine, T...)(Array!string Dirlst, T params)
{
ReturnType!coRoutine rData; / This line is not working
alias scRType = typeof(coRoutine(string.init, T.init));
auto PFresult = taskPool.workerLocalStorage!scRType();
foreach (string FFs; parallel(Dirlst[],1)) { PFresult.get
~= coRoutine(FFs, params); }
foreach(i; PFresult.toRange) { rData ~= i[][]; }
}
Error:
test.d(206): Error: template instance
`std.traits.ReturnType!(coAgedDirClean)` does not match template
declaration ReturnType(func...) if (func.length == 1 &&
isCallable!func)
Working Code:
auto coAgedDirClean(T...)(string FFs, T params) {
auto dFiles = Array!(Tuple!(string,
SysTime))(dirEntries(FFs, SpanMode.shallow).filter!(a => a.isFile
&& a.size > params[0]).map!(a => tuple(a.name,
a.timeLastModified)));
return dFiles;
}
auto coAgedDirDelete(T...)(string FFs, T params) {
auto dFiles = Array!(Tuple!(string,
SysTime))(dirEntries(FFs, SpanMode.shallow).filter!(a => a.isFile
&& a.size > params[0]).map!(a => tuple(a.name,
a.timeLastModified)));
if (params[1] == "run" && !dFiles.empty) { foreach(d;
parallel(dFiles[], 1)) { remove(d[0]); }}
return dFiles;
}
void process(alias coRoutine, T...)(Array!string Dirlst, T params)
{
alias scRType = typeof(coRoutine(string.init, T.init));
auto PFresult = taskPool.workerLocalStorage!scRType();
foreach (string FFs; parallel(Dirlst[],1)) { PFresult.get
~= coRoutine(FFs, params); }
foreach(i; PFresult.toRange) { writeln(i[][]); }
}
void main()
{
Array!string AgedDir, DeleteDir;
AgedDir.insert("C:\\Temp\\TEAM1");
DeleteDir.insert("C:\\Temp\\BACKUP1");
ulong Size = 102;
string Step = "run";
process!coAgedDirClean(AgedDir, Size );
process!coAgedDirDelete(DeleteDir, Size, Step);
}
From,
Vino.B
Re: Passing function(whose parameter would be dynamic and the type is unknown) as a parameter to another function.
On Sunday, 8 July 2018 at 19:10:24 UTC, Alex wrote:
On Sunday, 8 July 2018 at 18:46:31 UTC, vino.B wrote:
Request you help, in the below code we pass the function
"Testfun" as a parameter to another function "process" in
order for the function "process" to work we have to specify
the type of the parameter that is passed to the function "(T
function(string, int) coRoutine, string Test, int Size) ", so
now how do we pass a function whose parameter would be dynamic
and the type is unknown.
What do you mean with a "function with dynamic parameters" and
"unknown type"?
But how about
´´´
void main()
{
process!fun();
}
void process(alias coRoutine, T...)(T params)
{
coRoutine(params);
}
auto fun(T...)(T params)
{
}
´´´
HI Alex,
I tried you method, but it not working as expected, the
process!fun1(Test1) works but the process!fun2(Test1, Size ) does
not work, instead of displaying the value (Test, 1) it output's
the parameter "Test1".
Output:
Test
Test1
If I comment process!fun1(Test1); then the output is same
Output :
Test1
import std.stdio: writeln;
void main()
{
string Test1 = "Test";
int Size = 1;
process!fun1(Test1);
process!fun2(Test1, Size );
}
void process(alias coRoutine, T...)(T params)
{
coRoutine(params);
}
auto fun1(T...)(T params)
{
writeln(params);
}
auto fun2(T...)(T params)
{
writeln(params);
}
From,
Vino.B
Re: Passing function(whose parameter would be dynamic and the type is unknown) as a parameter to another function.
On Sunday, 8 July 2018 at 19:22:32 UTC, Timoses wrote:
On Sunday, 8 July 2018 at 18:46:31 UTC, vino.B wrote:
Hi All,
Request you help, in the below code we pass the function
"Testfun" as a parameter to another function "process" in
order for the function "process" to work we have to specify
the type of the parameter that is passed to the function "(T
function(string, int) coRoutine, string Test, int Size) ", so
now how do we pass a function whose parameter would be dynamic
and the type is unknown.
void process(T)(T function(string, int) coRoutine, string
Test, int Size) {
This would templetize the return type of the coRoutine, thus
within that function you could do
T returnedValue = coRoutine(string.init, int.init);
alias scRType = typeof(coRoutine(string.init, int.init));
Eg:
Run1 : process(, Test, Size);
void process(T ...)(T function(string, int) coRoutine, string
Test) {
alias scRType = typeof(coRoutine(string.init, int.init));
Run2 : process(, Test, Size, Str1);
void process(T)(T function(string, int, string) coRoutine,
string Test, int Size, string Str1) {
alias scRType = typeof(coRoutine(string.init, int.init,
string.int));
Run3 : process(, Test);
void process(T)(T function(string, string) coRoutine, string
Test, int Size) {
alias scRType = typeof(coRoutine(string.init));
PFresult.get = coRoutine(args);
Some what like this
auto Testfun (string FFs, int Size) { return tuple(FFs, Size);
}
void process(T ...)(T function(T args) coRoutine, T args) {
This would mean that if you pass a function that returns for
example an int, it must also
- take an int as argument
- and process has to accept another int
e.g.
process((int i) => i+1, 3);
if T would be (int, string) you would have to pass something
like
process((int i, string s) {
return AliasSeq!(int, string); }, // error: not even
sure how to express this
3, "hello");
where I'm not sure how to express the return type of (int,
string)... Does anybody know this? Would the Tuple Dip make
this possible?
(https://forum.dlang.org/post/p3bdp1$2b4e$1...@digitalmars.com)
alias scRType = typeof(coRoutine(T.init));
PFresult.get = coRoutine(T);
void main() {
string Test ="C:\\Temp\\BACKUP1"; int Size = 1;
process(, Test, Size);
}
Code : Working
import std.stdio: writeln;
import std.container.array;
import std.typecons: tuple;
import std.parallelism: taskPool;
auto Testfun (string FFs, int Size) { return tuple(FFs, Size);
}
void process(T)(T function(string, int) coRoutine, string
Test, int Size) {
alias scRType = typeof(coRoutine(string.init, int.init));
auto PFresult = taskPool.workerLocalStorage!scRType();
PFresult.get = coRoutine(Test, Size);
foreach(i; PFresult.toRange) { writeln(i[][]); }
}
void main() {
string Test ="C:\\Temp\\BACKUP1"; int Size = 1;
process(, Test, Size);
}
From,
Vino.B
I suggest taking a look at
https://dlang.org/phobos/std_traits.html .
E.g. ReturnType and Parameters:
int func() { return 3; }
assert(is(ReturnType!func == int));
void gunc(int i, double j) {}
import std.meta : AliasSeq;
assert(is(Parameters!gunc == AliasSeq!(int, double)));
Perhaps you could tell us what your goal is. People here might
come up with a nice solution. Why do you feel like having to
use templated functions in the first place? That is, what is
the generic goal of the functions you are trying to define?
Hi Timoses,
We are converting a Power shell script to D in phased manner;
the PS script has many functions and we converted few function to
D in Phase 1.
Phase 1:
Structure of the Program
Main -> Thread Manager->CoFunction1(Fs1,2,3,4,5)
Main -> Thread Manager->CoFunction2(Fs1,2,3,4,5)
The thread manager will call the Cofunctions and the function
gets executed on “N” of file systems each of size 5-10 TB.
The function that we transformed all has the same number of
parameters (3) and the type was same (string, string, int), so we
wrote a static thread manager as below
void ptManager (T)(T function(string, string, int) coRoutine,
Array!string Dirlst, string Step, int Size) {
alias scRType = typeof(coRoutine(string.init, string.init,
int.init));
auto PFresult = taskPool.workerLocalStorage!scRType();
ReturnType!coRoutine rData;
foreach (string FFs; parallel(Dirlst[0 .. $],1)) {
PFresult.get ~= coRoutine(FFs.strip, Step); }
foreach(i; PFresult.toRange) { writeln(i[][]); }
}
void main () {
ptManager(, Fn1Dirlst, Step, Size);
ptManager(, Fn2Dirlst, Step, Age);
}
Phase 2:
In phase 2 we are transferring few more function to the existing
D code, and these functions has variable number of parameter and
different type eg: Function3(string, string, string),
Function(string, int, string, int).
Initially I tried to re-write the ptManager function for each
type of function which ended with 8 ptManager functions, Eg :
ptManager1(string, string, int), ptManager2(string, string,
string),
Passing function(whose parameter would be dynamic and the type is unknown) as a parameter to another function.
Hi All,
Request you help, in the below code we pass the function
"Testfun" as a parameter to another function "process" in order
for the function "process" to work we have to specify the type of
the parameter that is passed to the function "(T function(string,
int) coRoutine, string Test, int Size) ", so now how do we pass a
function whose parameter would be dynamic and the type is unknown.
void process(T)(T function(string, int) coRoutine, string Test,
int Size) {
alias scRType = typeof(coRoutine(string.init, int.init));
Eg:
Run1 : process(, Test, Size);
void process(T ...)(T function(string, int) coRoutine, string
Test) {
alias scRType = typeof(coRoutine(string.init, int.init));
Run2 : process(, Test, Size, Str1);
void process(T)(T function(string, int, string) coRoutine, string
Test, int Size, string Str1) {
alias scRType = typeof(coRoutine(string.init, int.init,
string.int));
Run3 : process(, Test);
void process(T)(T function(string, string) coRoutine, string
Test, int Size) {
alias scRType = typeof(coRoutine(string.init));
PFresult.get = coRoutine(args);
Some what like this
auto Testfun (string FFs, int Size) { return tuple(FFs, Size); }
void process(T ...)(T function(T args) coRoutine, T args) {
alias scRType = typeof(coRoutine(T.init));
PFresult.get = coRoutine(T);
void main() {
string Test ="C:\\Temp\\BACKUP1"; int Size = 1;
process(, Test, Size);
}
Code : Working
import std.stdio: writeln;
import std.container.array;
import std.typecons: tuple;
import std.parallelism: taskPool;
auto Testfun (string FFs, int Size) { return tuple(FFs, Size); }
void process(T)(T function(string, int) coRoutine, string Test,
int Size) {
alias scRType = typeof(coRoutine(string.init, int.init));
auto PFresult = taskPool.workerLocalStorage!scRType();
PFresult.get = coRoutine(Test, Size);
foreach(i; PFresult.toRange) { writeln(i[][]); }
}
void main() {
string Test ="C:\\Temp\\BACKUP1"; int Size = 1;
process(, Test, Size);
}
From,
Vino.B
Re: Outside array bounds
On Saturday, 7 July 2018 at 12:13:21 UTC, Alex wrote:
On Saturday, 7 July 2018 at 11:22:38 UTC, Timoses wrote:
Aw, got it. So args is actually a tuple type where accessing
beyond the defined tuple (T) is invalid?
auto a = [1, 2, 4];
// works
pragma(msg, typeof(a[3]));
auto t = tuple(3, 4, 5.3);
// ERROR:
// pragma(msg, typeof(t[3]));
Yes. The absence of the next element in the array, doesn't
matter, while asking its type, while the absence of the next
tuple element means also the absence of any type, as I
understand it.
Hi All,
If we replace the statement as args[$ -1] the program works are
expected, if we apply the same logic in different approach it
does not work, in the below code if we command the first block
"if (fnID == "ListFilesNames") {} " then the second "if (fnID ==
"ListFilesSize") {}" works fine, and vice versa but if both the
"if" block is un-commented it does not work , rather it passes
wrong parameter to the function. Tried change the args as args[$]
, args[1], args[2],
Error:
Test.d(31): Error: function Test.ListFilesSize(string FFs, int
Size) is not callable using argument types (string, Array!string)
VarTest.d(31):cannot pass argument _param_1 of type
Array!string to parameter int Size
Test.d(42): Error: template instance
`Test.process!(Array!string)` error instantiating
Code:
import std.stdio: writeln;
import std.container.array;
import std.algorithm: filter, map;
import std.typecons: Tuple, tuple;
import std.file: dirEntries, isFile, SpanMode;
import std.conv: to;
import std.datetime.systime: Clock, SysTime;
import std.parallelism: parallel, taskPool, TaskPool;
import std.array: empty;
auto ListFilesNames (string FFs) {
auto dFiles = Array!(Tuple!(string, SysTime))(dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isFile).map!(a => tuple(a.name,
a.timeCreated)));
return dFiles;
}
auto ListFilesSize(string FFs, int Size) {
writeln(Size);
auto dFiles = Array!(Tuple!(string, ulong))(dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isFile).map!(a => tuple(a.name,
(a.size > Size).to!ulong)));
return dFiles;
}
void process(T ...)(string fnID, T args) {
if (fnID == "ListFilesNames") {
alias scRType = typeof(ListFilesNames(string.init));
auto PFresult = taskPool.workerLocalStorage!scRType();
foreach (string FFs; args[0]) { PFresult.get ~=
ListFilesNames(FFs); }
foreach(i; PFresult.toRange) { writeln(!i[][].empty); }
}
else if (fnID == "ListFilesSize") {
alias scRType = typeof(ListFilesSize(string.init, ulong.init));
auto PFresult = taskPool.workerLocalStorage!scRType();
foreach (string FFs; args[0]) { PFresult.get ~=
ListFilesSize(FFs, args[$ - 1]); }
foreach(i; PFresult.toRange) { writeln(!i[][].empty); }
}
}
void main() {
Array!string NameDir, SizeDir;
NameDir.insert("C:\\Temp\\BACKUP1");
SizeDir.insert("C:\\Temp\\TEAM1");
int Size = 1;
string fnID1 = "ListFilesNames", fnID2 = "ListFilesSize";
process(fnID1, NameDir);
process(fnID2, SizeDir, Size);
}
From,
Vino.B
Outside array bounds
Hi All,
Request you help, on the below code
import std.stdio: writeln;
void process(T ...)(string ID, T args) {
if (ID == "I1") { writeln(args.length, "\t", args[0]); }
else if (ID == "I2") { writeln(args.length, "\t", args[1]);}
}
void main() {
string S1 = "Test1", S2 = "Test2", ID1 = "I1", ID2 = "I2";
int Size = 1;
process(ID1, S1);
process(ID2, S2, Size);
}
Error:
Test.d(5): Error: array index [1] is outside array bounds [0 .. 1]
Test.d(11): Error: template instance `Test.process!string` error
instantiating
From,
Vino.B
Re: Function Template for Dynamic Parameter
On Sunday, 1 July 2018 at 12:46:30 UTC, Timoses wrote:
On Sunday, 1 July 2018 at 11:58:30 UTC, vino.B wrote:
On Sunday, 1 July 2018 at 11:52:19 UTC, Alex wrote:
NewType.d(19): Error: function declaration without return
type. (Note that constructors are always named this)
[...]
auto coCleanFiles(T ...)(T args) {
auto dFiles = Array!(Tuple!(string, SysTime))(dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isFile).map!(a =>
tuple(a.name, a.timeLastModified)));
return dFiles;
}
void ptManagecoRoutine(T)(T function(T ...)(T args),
Array!string Dirlst) {
alias scRType = typeof(coRoutine(args.init));
auto PFresult = taskPool.workerLocalStorage!scRType;
ReturnType!coRoutine rData;
foreach (string FFs; Dirlst[]) { PFresult.get ~=
coRoutine(FFs.strip); }
foreach(i; PFresult.toRange) { rData ~= i[][]; }
if (!rData[].empty) { rData[].sort!((a,b) => a[1] <
b[1]).each!(e => writefln!"%-83s %.20s"(e[0],
e[1].to!string)); }
}
void main () {
Array!string CleanDirlst;
CleanDirlst.insertBack("C:\\Temp\\BACKUP1");
ptManagecoRoutine(, CleanDirlst);
}
auto coCleanFiles(T ...)(T args)
{ ... }
void ptManagecoRoutine(T)(T fun, Array!string DirList)
{
foreach (dir; DirList)
fun(dir);
}
or
void ptManagecoRoutine2(alias func)(Array!string DirList)
if (is (typeof(func!(typeof(DirList[0]))) == function))
{
alias t = typeof(DirList[0]);
ptManagecoRoutine(!t, DirList);
}
callable via
Array!string CleanDirlst;
ptManagecoRoutine(!string, CleanDirlst);
ptManagecoRoutine2!coCleanFiles(CleanDirlst);
Hi All,
Request your help on the below code
auto coCleanFiles(T ...) (T FFs) {
auto dFiles = Array!(Tuple!(string, SysTime))(dirEntries(FFs,
SpanMode.depth).map!(a => tuple(a.name, a.timeCreated)));
return dFiles;
}
void process(T)(T pfunction, Array!string Dirlst) {
alias wlsType = typeof(pfunction(T));
auto Result = taskPool.workerLocalStorage!wlsType();
foreach (FFs; parallel(Dirlst[],1)) { Result.get ~=
pfunction(FFs); }
foreach(i; Result.toRange) { writeln(i[][]); }
}
void main() {
Array!string Cleanlst;
Cleanlst.insert("C:\\Temp\\BACKUP1");
process(, Cleanlst);
}
Error : Error: coCleanFiles(T...)(T FFs) is not an lvalue and
cannot be modified
From,
Vino.B
Re: Function Template for Dynamic Parameter
On Sunday, 1 July 2018 at 11:52:19 UTC, Alex wrote:
On Sunday, 1 July 2018 at 11:19:50 UTC, vino.B wrote:
Hi Timoses,
Thank you very much, can you help me on how to rewrite the
below using Variadic template
Passing function as a parameter to another function:
void ptFun(T)(T function(string, string, int) coRoutine,
Array!string Dirlst, ) {
alias scRType = typeof(coRoutine(string.init, string.init,
int.init));
where the "function(string, string, int) coRoutine" should be
a variadic function
From,
Vino.B
I'm not sure, if get your question right, is this what you are
looking for?
´´´
import std.stdio;
import std.traits;
void main()
{
alias instantiation = ptFun!(size_t, fun!string);
instantiation(4);
alias instantiation2 = ptFun2!(fun!string);
instantiation2(4);
}
auto fun(T...)(T input)
{
return size_t(42);
}
void ptFun(T, alias funToCall)(size_t definedParam)
if(is(T == ReturnType!(funToCall)))
{
"ptFun called".writeln;
assert(42 == funToCall("some string"));
}
void ptFun2(alias funToCall)(size_t definedParam)
if(__traits(isSame, TemplateOf!(funToCall), fun))
{
"ptFun2 called".writeln;
assert(42 == funToCall("some string"));
}
´´´
Hi Alex,
Something similar to the below code, when compling the below
code i get an error
NewType.d(19): Error: function declaration without return type.
(Note that constructors are always named this)
import core.time: days;
import std.algorithm: each, filter, map, sort, strip;
import std.container.array;
import std.conv: to;
import std.datetime.systime: Clock, SysTime;
import std.file: dirEntries, isFile, SpanMode;
import std.parallelism: parallel;
import std.stdio: File, writefln, writeln;
import std.string: strip;
import std.traits: ReturnType;
import std.typecons: Tuple, tuple;
auto coCleanFiles(T ...)(T args) {
auto dFiles = Array!(Tuple!(string, SysTime))(dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isFile).map!(a => tuple(a.name,
a.timeLastModified)));
return dFiles;
}
void ptManagecoRoutine(T)(T function(T ...)(T args), Array!string
Dirlst) {
alias scRType = typeof(coRoutine(args.init));
auto PFresult = taskPool.workerLocalStorage!scRType;
ReturnType!coRoutine rData;
foreach (string FFs; Dirlst[]) { PFresult.get ~=
coRoutine(FFs.strip); }
foreach(i; PFresult.toRange) { rData ~= i[][]; }
if (!rData[].empty) { rData[].sort!((a,b) => a[1] <
b[1]).each!(e => writefln!"%-83s %.20s"(e[0], e[1].to!string)); }
}
void main () {
Array!string CleanDirlst;
CleanDirlst.insertBack("C:\\Temp\\BACKUP1");
ptManagecoRoutine(, CleanDirlst);
}
From,
Vino.B
Re: Function Template for Dynamic Parameter
On Sunday, 1 July 2018 at 09:55:34 UTC, Timoses wrote:
On Sunday, 1 July 2018 at 09:46:32 UTC, vino.B wrote:
All,
Request your help, the D document states that "Template
functions are useful for avoiding code duplication - instead
of writing several copies of a function, each with a different
parameter type, a single function template can be sufficient"
which mean we can passing any type of parameter using function
template, similarly who we we pass in any number of parameter
of any type(dynamic parameters) , like in python
Python
def myfunc(*arg)
def myfunc(**kwargs) // multiple key-value
def myfunc(*args, **kwargs): mix
Do we have any thing similar to the above python in D
void (func(T)( T *args) // Example
Examples:
void func(T)(T x)
{
writeln(x);
}
void main()
{ func("x"); // pass a string }
void func(T)(T n3)
{
writeln(n); // where n = 3 parameters (1, string, char)
}
void func(T)(T n2)
{
writeln(n); // where n = 3 parameters (1, string)
}
void func(T)(T n1)
{
writeln(n); // where n = 3 parameters (1 or string or char)
}
From,
Vino.B
Perhaps https://dlang.org/spec/template.html#variadic-templates
void func(T ...)(T args)
{
static assert(T.length <= 3);
static assert(is(T[0] == int)); // 1
pragma(msg, T.stringof);
foreach (arg; args)
{
writeln(arg);
}
}
void main()
{
func(3, "s", 1.3);
//func(1.3); // error, first argument is not an int,
see // 1
}
Hi Timoses,
Thank you very much, can you help me on how to rewrite the
below using Variadic template
Passing function as a parameter to another function:
void ptFun(T)(T function(string, string, int) coRoutine,
Array!string Dirlst, ) {
alias scRType = typeof(coRoutine(string.init, string.init,
int.init));
where the "function(string, string, int) coRoutine" should be a
variadic function
From,
Vino.B
Function Template for Dynamic Parameter
All,
Request your help, the D document states that "Template
functions are useful for avoiding code duplication - instead of
writing several copies of a function, each with a different
parameter type, a single function template can be sufficient"
which mean we can passing any type of parameter using function
template, similarly who we we pass in any number of parameter of
any type(dynamic parameters) , like in python
Python
def myfunc(*arg)
def myfunc(**kwargs) // multiple key-value
def myfunc(*args, **kwargs): mix
Do we have any thing similar to the above python in D
void (func(T)( T *args) // Example
Examples:
void func(T)(T x)
{
writeln(x);
}
void main()
{ func("x"); // pass a string }
void func(T)(T n3)
{
writeln(n); // where n = 3 parameters (1, string, char)
}
void func(T)(T n2)
{
writeln(n); // where n = 3 parameters (1, string)
}
void func(T)(T n1)
{
writeln(n); // where n = 3 parameters (1 or string or char)
}
From,
Vino.B
Re: E-mail attachment with scrambled text.
On Thursday, 28 June 2018 at 12:36:11 UTC, Simen Kjærås wrote:
On Thursday, 28 June 2018 at 11:46:31 UTC, vino.B wrote:
Output in Linux
Server Details**
Server Name : 1 IP: 1XX
Server Name : 2 IP: 2XX
Server Name : 3 IP: 3XX
The output in Windows(Email attachment) all are in single line
Server Details**Server Name : 1
IP: 1XXServer Name : 2 IP: 2XX Server Name :
3 IP: 3XX
Looks to be an issue with newlines. In linux, a newline is
simply \n. In Windows it's \r\n, and some Windows programs get
confused when they just see a \n, notably notepad. Notepad++
and basically any other editor will handle Unix newlines
correctly.
--
Simen
Hi Simen,
Thank you very much, after replacing all the '\n' with '\r\n' it
resolved 99% of the formatting issue expect for the below
function, can you help me on the same.
auto getAvailableDiskSpace(Array!string UtilDrive, File logF) {
auto result = ["/usr/bin/df", "-h", UtilDrive,
"--output=target,size,used,avail,pcent"].execute;
enforce(result.status == 0);
logF.writeln(result.output);
}
The output of the above code is as below(Single line)
Mounted on Size Used Avail Use%/backup 3.0T 2.6T 393G 87%
From,
Vino.B
E-mail attachment with scrambled text.
Hi All, Request your help, i have a D code which generates a log file with below text, in Linux, when i send this log file(text file) as an mail attachment the text in the attachment are scrambled so request your help on this. Tried the below Options (no luck): Content-Type: text/plain Content-Transfer-Encoding: base64 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 Output in Linux Server Details** Server Name : 1 IP: 1XX Server Name : 2 IP: 2XX Server Name : 3 IP: 3XX The output in Windows(Email attachment) all are in single line Server Details**Server Name : 1 IP: 1XXServer Name : 2 IP: 2XX Server Name : 3 IP: 3XX From, Vino.B
Re: Getting Source code from complied code.
On Thursday, 28 June 2018 at 08:21:20 UTC, Basile B. wrote: On Thursday, 28 June 2018 at 08:01:42 UTC, vino.B wrote: Hi All, Request your help on how to get the source code, i wrote a program named clean.d, complied it and by mistake deleted the source code(clean.d), so can we get back the source using the complied program(clean), if yes, can you any one help on the same. From, Vino.B You cant. You can disasm that's all (with Hopper or IDA Free 7). Hi Basile, Thank you very much, luckily the file was backed up by the backup scheduler, so was able to restore the same. From, Vino.B
Getting Source code from complied code.
Hi All, Request your help on how to get the source code, i wrote a program named clean.d, complied it and by mistake deleted the source code(clean.d), so can we get back the source using the complied program(clean), if yes, can you any one help on the same. From, Vino.B
Re: Deleting a file with extsion *.FIFO in Windows
On Thursday, 24 May 2018 at 11:31:15 UTC, bauss wrote: On Thursday, 24 May 2018 at 06:59:47 UTC, Vino wrote: Hi All, Request your help on how to delete a file which has the extension .fifo (.javast.fifo) in Windows. From, Vino.B What exactly is your issue with it? Hi Bauss, We have a java program which creates a file with extension .fifo and we have another program written in D to clean up the old file, so the D code is not able to delete these files using any of the D function provided it states "Access Denied" we tried to provide the full access manually even then it is not able to delete such files. From, Vino.B
Re: E-mail attachment with unwanted characters
On Saturday, 28 April 2018 at 16:37:26 UTC, Vino.B wrote: On Friday, 27 April 2018 at 18:20:46 UTC, Adam D. Ruppe wrote: [...] Hi Adam, Thank you very much, after removing the dot the unwanted characters disappeared, The earlier program (as function) is working as expected without any issue, but if I change the program from function to Classes, then the programing is executing without any errors, able to get the attachment wihout any unwanted characters, but not able to get the body text, tries passing the body text as Array!sting and normal string, even then the body message is not appearing when I receive the mails. [...] Hi Adam, Thank you very much, was able to resolve the issue. From, Vino.B
Re: E-mail attachment with unwanted characters
On Friday, 27 April 2018 at 18:20:46 UTC, Adam D. Ruppe wrote:
On Friday, 27 April 2018 at 17:57:26 UTC, Vino.B wrote:
headers.insert(to!string(Base64.encode(Content)) ~
".\r\n");
headers.insert("--" ~ boundary ~ ".");
what are those random dots for?
Hi Adam,
Thank you very much, after removing the dot the unwanted
characters disappeared, The earlier program (as function) is
working as expected without any issue, but if I change the
program from function to Classes, then the programing is
executing without any errors, able to get the attachment wihout
any unwanted characters, but not able to get the body text, tries
passing the body text as Array!sting and normal string, even then
the body message is not appearing when I receive the mails.
Code:
import std.array: join;
import std.base64: Base64;
import std.container.array;
import std.conv : to;
import std.file: read, getSize;
import std.format : format;
import std.net.curl;
import std.path : baseName;
import std.uuid: randomUUID;
pragma(lib, "curl");
class EmailMessage {
static string Boundary;
static this() { Boundary = randomUUID().toString(); }
string From, Subject, cid, Filename, msg, crlf = "\r\n";
Array!string To, Body, headers, attach;
int Size;
void Attachment (string Filename) { attach ~= Filename; }
string BuildMail () {
string[] tos;
foreach (e; To) { tos ~= e; }
headers.insert("From: " ~ From );
headers.insert("To: " ~ join(tos, ","));
headers.insert("Subject: " ~ Subject);
headers.insert("MIME-Version: 1.0");
headers.insert(format("Content-Type: multipart/alternative;
boundary=\"%s\"\r\n", Boundary));
headers.insert("--" ~ Boundary);
headers.insert("Content-Type: text/plain; charset=utf-8");
headers ~ Body; //Array!string does not work
headers.insert(Body); //string does not work
headers.insert("--" ~ Boundary);
headers.insert("Content-Type: text/plain");
headers ~ ((cid !is null) ? "Content-ID: <" ~ cid ~ ">" : "");
headers.insert("Content-Transfer-Encoding: base64");
foreach (File; attach) {
string Fname = baseName(File);
ubyte[] Content = cast(ubyte[])read(File);
Size = to!int(getSize(File) + Body.length);
headers.insert("Content-Disposition: attachment;
filename=\"" ~ Fname ~ "\"");
headers.insert(crlf);
headers.insert(to!string(Base64.encode(Content)) ~ ".\r\n");
}
headers.insert("--" ~ Boundary);
msg.reserve(Size);
foreach(header; headers) { msg ~= header ~ "\r\n"; }
if(msg.length > 0) { msg ~= "\r\n";}
return(msg);
}
void Send(string server) {
const(char)[][] allRecipients;
foreach (e; To) { allRecipients ~= e; }
auto smtp = SMTP(server);
smtp.mailTo(allRecipients);
smtp.mailFrom = From;
smtp.message = BuildMail();
smtp.perform();
}
}
void main () {
string Filename = "D:\\DScript\\Test.txt";
Array!string To, Body;
To.insert("us...@ask.com");
To.insert("us...@ask.com");
Body.insert("This is Test1");
Body.insert("This is Test2");
auto message = new EmailMessage();
message.To = To;
message.From = "ad...@ask.com";
message.Subject = "My Subject";
message.Body ~= Body; //Array!string does not work
mesagae.Body = "Test Body"; //string does not work
message.Attachment = Filename;
message.Send = "smtp://ask.com";
}
From,
Vino.B
E-mail attachment with unwanted characters
Hi All,
Request your help, the below code is working as expected, but
when I receive the attachment, the attachment contains the
orginal text plus some unwanted characters like below, can
someone help me how to remove these unwanted characters.
Unwanted characters
This is a test documentoÞ óÎ}ã¿xÛ]Zõ§ûwN¶÷ÆÝy·
Code:
import std.base64: Base64;
import std.container.array;
import std.conv : to;
import std.file: read;
import std.format : format;
import std.net.curl;
import std.path : baseName;
import std.uuid: randomUUID;
pragma(lib, "curl");
string Message(string To, string From, string Body, string
Subject, string Filename, ubyte[] Content) {
Array!string headers;
string cid, msg, boundary = randomUUID().toString(), Fname =
baseName(Filename);
const string crlf = "\r\n";
int Size = to!int(getSize(Filename));
headers.insert("From: " ~ From );
headers.insert("To: " ~ To );
headers.insert("Subject: " ~ "Subject" );
headers.insert("MIME-Version: 1.0");
headers.insert(format("Content-Type: multipart/mixed;
boundary=\"%s\"\r\n", boundary));
headers.insert("--" ~ boundary);
headers.insert("Content-Type: text/html; charset=utf-8");
headers.insert(crlf);
headers.insert(Body);
headers.insert(crlf);
headers.insert("--" ~ boundary);
headers.insert("Content-Type: text/plain");
headers ~ ((cid !is null) ? "Content-ID: <" ~ cid ~ ">" : "");
headers.insert("Content-Transfer-Encoding: base64");
headers.insert("Content-Disposition: attachment; filename=\""
~ Fname ~ "\"");
headers.insert(crlf);
headers.insert(to!string(Base64.encode(Content)) ~ ".\r\n");
headers.insert(crlf);
headers.insert("--" ~ boundary ~ ".");
msg.reserve(Size + Body.length);
foreach(header; headers) { msg ~= header ~ "\r\n"; }
if(msg.length > 0) { msg ~= "\r\n";}
return(msg);
}
void main () {
auto Filename = "C:\\Script\\New\\new.txt";
auto Con = cast(ubyte[])read(Filename);
auto smtp = SMTP("smtp://xxx.com");
smtp.mailTo = "u...@ask.com";
smtp.mailFrom = "ad...@ask.com";
smtp.message = Message("u...@ask.com", "ad...@ask.com", "Test",
"TestMail", Filename, Con);
smtp.perform();
}
From,
Vino.B
Re: SMTP Mail
On Monday, 9 April 2018 at 13:02:06 UTC, Adam D. Ruppe wrote: On Sunday, 8 April 2018 at 15:45:48 UTC, Vino wrote: I am trying your email.d programming, and i am getting the below errors, can you please help me, i just used these programs (characterencodings.d, color.d, dom.d, htmltotext.d, email.d) What is your build command? It looks like a module is just missing in the build. If you are using rdmd, what is your directory layout too. (I don't work Sundays btw so that's why I am so slow to respond.) Hi Adam, Thank you very much, I copied your folder arsd under the phobes folder in c:\D\... and the program was placed on my desktop and tried to execute it from the desktop via rdmd. From, Vino.B
Array merge and sort
Hi All,
My code output's the below so can any one help me on hot to
merege all tese array and sort the same.
Output :
[ Tuple!(string, string)("C:\\Temp\\TEST1\\BACKUP\\DND1.pdf",
"2017-Sep-06 16:06:42") ]
[ Tuple!(string, string)("C:\\Temp\\TEST2\\EXPORT\\DND1.pdf",
"2017-Sep-06 16:06:43")]
[ Tuple!(string, string)("C:\\Temp\\TEST3\\PROD_TEAM\\DND1.pdf",
"2017-Sep-06 16:06:43")]
[ Tuple!(string, string)("C:\\Temp\\TEST4\\TEAM\\DND1.pdf",
"2017-Sep-06 16:06:44") ]
Code :
foreach (string FFs; parallel(CleanDirlst[0 .. $], 1)) {
MCresult.get ~= coCleanFiles(FFs.strip, Step);
}
foreach(i; MCresult.toRange)
if (!i.empty) { writefln("%(%-(%-63s %s %)\n%)", i[]); }
From,
Vino.B
Re: Assertion Error
On Wednesday, 13 September 2017 at 15:27:30 UTC, Moritz Maxeiner
wrote:
On Wednesday, 13 September 2017 at 15:12:57 UTC, Vino.B wrote:
On Wednesday, 13 September 2017 at 11:03:38 UTC, Moritz
Maxeiner wrote:
On Wednesday, 13 September 2017 at 07:39:46 UTC, Vino.B wrote:
[...]
[...]
---
foreach (string Fs; parallel(SizeDirlst[0 .. $], 1))
{
MSresult.get ~= coSizeDirList(Fs.strip, SizeDir);
}
---
Hi Max,
It's Moritz, not Max. ;)
Below is the explanation of the above code.
[...]
AFAICT that's a reason why you want parallelization of
coSizeDirList, but not why you need to spawn another thread
inside of an *already parallelelized" task. Try my shortened
parallel foreach loop vs your longer one and monitor system
load (threads, memory, etc).
Hi Moritz,
Thank you very much, it was very helpful and time saving and
fast.
From,
Vino.B
Question on Container Array.
Hi All, Can some one explain me on the below question. Q1: void main (Array!string args) : Why can't we use container array in void main? Q2: What is the difference between the below? insert, insertBack stableInsert, stableInsertBack linearInsert, stableLinearInsert, stableLinearInsert Q3: Storing the data in a container array store's the data in memory which is managed by malloc/free, where as operation such as appending data using any of the above nor "~=" is managed by gc, is my understanding correct. From, Vino.B
Using Parallel prints duplicates nor misses.
Hi,
Request your help, the below code sometime prints duplicate and
some time miss the entry. due to which any code written below the
line "foreach (d; parallel(dFiles[], 1))" are some time
duplicated and some time not executed(skips). tired adding sort
and uniq to the writeln but no luck.
Code:
import std.algorithm: filter, map, sort, uniq;
import std.container;
import std.file: SpanMode, dirEntries, isDir;
import std.stdio: writeln;
import std.parallelism: parallel;
void main () {
auto SizeDirlst = Array!string ("C:\\Temp\\TEAM\\BACKUP",
"C:\\Temp\\TEAM\\EXPORT");
foreach (FFs; SizeDirlst[]) {
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; parallel(dFiles[], 1)) { writeln(d); }
}
}
Output 1:
C:\Temp\TEAM\BACKUP\dir1
C:\Temp\TEAM\BACKUP\DND3
C:\Temp\TEAM\BACKUP\dir2 - Duplicate
C:\Temp\TEAM\BACKUP\dir2 - Duplicate
C:\Temp\TEAM\EXPORT\DND6
Output 2:
C:\Temp\TEAM\BACKUP\dir1
C:\Temp\TEAM\BACKUP\DND3
C:\Temp\TEAM\BACKUP\dir2 - Duplicate
C:\Temp\TEAM\BACKUP\dir2 - Duplicate
C:\Temp\TEAM\EXPORT\DND6 - Missing
From,
Vino.B
Re: Assertion Error
On Wednesday, 13 September 2017 at 11:03:38 UTC, Moritz Maxeiner
wrote:
On Wednesday, 13 September 2017 at 07:39:46 UTC, Vino.B wrote:
On Tuesday, 12 September 2017 at 21:01:26 UTC, Moritz Maxeiner
wrote:
On Tuesday, 12 September 2017 at 19:44:19 UTC, vino wrote:
Hi All,
I have a small piece of code which executes perfectly 8 out
of 10 times, very rarely it throws an assertion error, so is
there a way to find which line of code is causing this error.
You should be getting the line number as part of the crash,
like here:
--- test.d ---
void main(string[] args)
{
assert(args.length > 1);
}
--
-
$ dmd -run test.d
core.exception.AssertError@test.d(3): Assertion failure
[Stack trace]
-
If you don't what are the steps to reproduce?
Hi Max,
I tried to run the code for at least 80+ time the code ran
without any issue, will let you know in case if I hit the same
issue in feature, Below is the piece of code, plese do let me
know if you find any issue with the below code.
Program Code:
[...]
foreach (string Fs; parallel(SizeDirlst[0 .. $], 1))
{
auto FFs = Fs.strip;
auto MSizeDirList = task(, FFs, SizeDir);
MSizeDirList.executeInNewThread();
auto MSizeDirListData = MSizeDirList.workForce;
MSresult.get ~= MSizeDirListData;
}
From reading I don't see anything that I would expect to
assert, but I am wondering why you first parallelize your work
with a thread pool (`parallel(...)`) and then inside each
(implicitly created) task (that is already being serviced by a
thread in the thread pool) you create another task, have it
executed in a new thread, and make the thread pool thread wait
for that thread to complete servicing that new task.
This should yield the same result, but without the overhead of
spawning additional threads:
---
foreach (string Fs; parallel(SizeDirlst[0 .. $], 1))
{
MSresult.get ~= coSizeDirList(Fs.strip, SizeDir);
}
---
Hi Max,
Below is the explanation of the above code.
The Fs that is passed to the function ptSizeDirList is any array
of 10 -15 file system (NetApp Filers)from 15 different servers
mounted on a single server (Network share).
The function ptSizeDirList pass each of the FS to the function
coSizeDirList in parallel thread to find the folder size under
each of these Fs,
Each of this FS has around 1000+ folders. so the requirement is
that we need the size of each of the folders under each of these
15 Fs along with the folder name in less than an hour or two as
this script is about to be schedule to run once every 3 hours,
and at present we are able to achieve the same in 10 mins / fs
Each Fs is of about 10-15 TB's
Hence this code was written.
Re: Assertion Error
On Tuesday, 12 September 2017 at 21:01:26 UTC, Moritz Maxeiner
wrote:
On Tuesday, 12 September 2017 at 19:44:19 UTC, vino wrote:
Hi All,
I have a small piece of code which executes perfectly 8 out of
10 times, very rarely it throws an assertion error, so is
there a way to find which line of code is causing this error.
You should be getting the line number as part of the crash,
like here:
--- test.d ---
void main(string[] args)
{
assert(args.length > 1);
}
--
-
$ dmd -run test.d
core.exception.AssertError@test.d(3): Assertion failure
[Stack trace]
-
If you don't what are the steps to reproduce?
Hi Max,
I tried to run the code for at least 80+ time the code ran
without any issue, will let you know in case if I hit the same
issue in feature, Below is the piece of code, plese do let me
know if you find any issue with the below code.
Program Code:
import core.stdc.stdlib: exit;
import std.algorithm: all, among, filter, map, setDifference,
sort, uniq, each, joiner;
import std.array: appender, join;
import std.container.array;
import std.conv: to;
import std.datetime.systime: Clock, days, SysTime;
import std.file: SpanMode, dirEntries, exists, isFile, mkdir,
remove, rmdirRecurse;
import std.getopt;
import std.parallelism: parallel, task, taskPool;
import std.path: absolutePath, baseName, dirName,
isValidFilename, isValidPath, globMatch;
import std.range: empty, zip, chain, chunks;
import std.stdio: File, writefln, writeln;
import std.string: chomp, chop, isNumeric, split, strip;
import std.typecons: tuple, Tuple;
import std.uni: isAlpha, toLower, isWhite;
import std.conv;
auto coSizeDirList (string FFs, int SizeDir) {
int subdirTotal;
int subdirTotalGB;
Array!string Subdir;
Array!string Subsize;
Array!string Result;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong(dirEntries(d,
SpanMode.depth).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024);
if (subdirTotalGB > SizeDir) { Result ~= d; Result ~=
to!string(subdirTotalGB); }
subdirTotal = 0;
}
return Result;
}
void ptSizeDirList (Array!string SizeDirlst, int SizeDir) {
alias DirSizeList = typeof(coSizeDirList(string.init, int.init));
auto MSresult = taskPool.workerLocalStorage!DirSizeList();
foreach (string Fs; parallel(SizeDirlst[0 .. $], 1)) {
auto FFs = Fs.strip;
auto MSizeDirList = task(, FFs, SizeDir);
MSizeDirList.executeInNewThread();
auto MSizeDirListData = MSizeDirList.workForce;
MSresult.get ~= MSizeDirListData;
}
foreach(i; MSresult.toRange)
chain(i[]).chunks(2).each!(e => writefln!"%-60s %s"(e[0],
e[1]));
}
void main () {
auto SizeDirlst = Array!string( "C:\\Temp\\TEST2\\BACKUP",
"C:\\Temp\\TEST2\\EXPORT", "C:\\Temp\\TEST2\\PROD_TEAM",
"C:\\Temp\\TEST2\\TEAM", "C:\\Temp\\TEST3\\BACKUP",
"C:\\Temp\\TEST3\\EXPORT" );
int SizeDir = 10;
ptSizeDirList(SizeDirlst, SizeDir);
}
From,
Vino.B
Re: Array Printing
On Tuesday, 12 September 2017 at 07:28:00 UTC, Anton Fediushin
wrote:
On Tuesday, 12 September 2017 at 06:29:53 UTC, Vino.B wrote:
Hi All,
Request your help in printing the below array output as per
the below required output
Array Output:
["C:\\Temp\\TEST2\\BACKUP\\dir1", "34",
"C:\\Temp\\TEST2\\BACKUP\\dir2", "36",
"C:\\Temp\\TEST3\\BACKUP\\dir1", "69"]
["C:\\Temp\\TEST2\\PROD_TEAM\\dir1", "34",
"C:\\Temp\\TEST2\\PROD_TEAM\\DND1", "34"]
["C:\\Temp\\TEST2\\TEAM\\DND1", "34"]
Required output:
C:\Temp\TEST2\BACKUP\dir1 34
C:\Temp\TEST2\BACKUP\dir2 36
C:\Temp\TEST3\BACKUP\\dir1 69
C:\Temp\TEST2\PROD_TEAM\\dir1 34
C:\Temp\TEST2\PROD_TEAM\\DND1 34
C:\Temp\TEST2\TEAM\\DND134
From,
Vino.B
Try this:
writefln("%(%s\n%)", array);
See std.format's documentation for more
Hi,
Sorry, it didn't work, the genrated out is as below
Output:
"C:\Temp\TEST2\BACKUP\dir1"
"34"
"C:\Temp\TEST2\BACKUP\dir2"
"36"
"C:\Temp\TEST3\BACKUP\\dir1"
"69"
"C:\Temp\TEST2\PROD_TEAM\dir1"
"34"
"C:\Temp\TEST2\PROD_TEAM\DND1"
"34"
"C:\Temp\TEST2\TEAM\\DND1"
"34"
From,
Vino.B
Array Printing
Hi All, Request your help in printing the below array output as per the below required output Array Output: ["C:\\Temp\\TEST2\\BACKUP\\dir1", "34", "C:\\Temp\\TEST2\\BACKUP\\dir2", "36", "C:\\Temp\\TEST3\\BACKUP\\dir1", "69"] ["C:\\Temp\\TEST2\\PROD_TEAM\\dir1", "34", "C:\\Temp\\TEST2\\PROD_TEAM\\DND1", "34"] ["C:\\Temp\\TEST2\\TEAM\\DND1", "34"] Required output: C:\Temp\TEST2\BACKUP\dir1 34 C:\Temp\TEST2\BACKUP\dir2 36 C:\Temp\TEST3\BACKUP\\dir1 69 C:\Temp\TEST2\PROD_TEAM\\dir1 34 C:\Temp\TEST2\PROD_TEAM\\DND1 34 C:\Temp\TEST2\TEAM\\DND134 From, Vino.B
Re: Deprecation: std.container.array.RangeT(A) is not visible from module Size
On Monday, 11 September 2017 at 08:55:21 UTC, Vino.B wrote:
On Sunday, 10 September 2017 at 23:10:20 UTC, Ali Çehreli wrote:
On 09/10/2017 09:53 AM, Vino.B wrote:
> auto coSizeDirList (string FFs, int SizeDir) {
> //alias DirSizeList = typeof(coSizeDirList());
I worked with a version of coSizeDirList() that did not take
any parameters. (Could be from an earlier post of yours.)
In this case, you can must use a compilable expression. Since
coSizeDirList above takes a string and int, you can get its
return value like this:
alias DirSizeList = typeof(coSizeDirList(string.init,
int.init))
The following can work as well:
alias DirSizeList = typeof(coSizeDirList("", 42))
However, there are other compilation errors when I do that.
Ali
Hi Ali,
The issue occurs at the line "MSresult.get ~= MSizeDirListData"
when we try to append data the error message is as below, if we
change this line to "MSresult.get = MSizeDirListData" then the
code complies without any issue but the output is empty.
T2.d(41): Error: cannot append type
Tuple!(RangeT!(Array!string), RangeT!(Array!ulong)) to type
Tuple!(RangeT!(Array!string), RangeT!(Array!ulong))
Code:
import core.stdc.stdlib: exit;
import std.algorithm: all, among, filter, map, setDifference,
sort, uniq, each, joiner;
import std.array: appender, join;
import std.container.array;
import std.conv: to;
import std.datetime.systime: Clock, days, SysTime;
import std.file: SpanMode, dirEntries, exists, isFile, mkdir,
remove, rmdirRecurse;
import std.getopt;
import std.parallelism: parallel, task, taskPool;
import std.path: absolutePath, baseName, dirName,
isValidFilename, isValidPath, globMatch;
import std.range: empty;
import std.stdio: File, writefln, writeln;
import std.string: chomp, chop, isNumeric, split, strip;
import std.typecons: tuple, Tuple;
import std.uni: isAlpha, toLower, isWhite;
auto coSizeDirList (string FFs, int SizeDir) {
ulong subdirTotal;
ulong subdirTotalGB;
Array!string Subdir;
Array!ulong Subsize;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong(dirEntries(d,
SpanMode.depth).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024); { Subdir ~= d;
Subsize ~= subdirTotalGB; }
if (subdirTotalGB > SizeDir)
subdirTotal = 0;
}
return tuple (Subdir[], Subsize[]);
}
void ptSizeDirList (string[] SizeDirlst, int SizeDir) {
alias DirSizeList = typeof(coSizeDirList(string.init,
ulong.init));
auto MSresult = taskPool.workerLocalStorage!DirSizeList();
foreach (string Fs; parallel(SizeDirlst[0 .. $], 1)) {
auto FFs = Fs.strip;
auto MSizeDirList = task(, FFs, SizeDir);
MSizeDirList.executeInNewThread();
auto MSizeDirListData = MSizeDirList.workForce;
//MSresult.get ~= MSizeDirListData;
MSresult.get = MSizeDirListData;
}
foreach(i; MSresult.tupleof)
//if (!i.empty) { writefln("%(%-(%-63s %)\n%)",
i[].sort!((a,b) => a[0] < b[0]).uniq); }
writeln(i);
}
void main () {
string SizeDirlst = "C:\\Temp\\TEAM\\BACKUP";
int SizeDir = 1;
coSizeDirList(SizeDirlst, SizeDir);
}
From,
Vino.B
Hi Ali,
At last was able to resolve the issue, thank you very much for
all your help. below is the working code.
Program:
import core.stdc.stdlib: exit;
import std.algorithm: all, among, filter, map, setDifference,
sort, uniq, each, joiner;
import std.array: appender, join;
import std.container.array;
import std.conv: to;
import std.datetime.systime: Clock, days, SysTime;
import std.file: SpanMode, dirEntries, exists, isFile, mkdir,
remove, rmdirRecurse;
import std.getopt;
import std.parallelism: parallel, task, taskPool;
import std.path: absolutePath, baseName, dirName,
isValidFilename, isValidPath, globMatch;
import std.range: empty, zip;
import std.stdio: File, writefln, writeln;
import std.string: chomp, chop, isNumeric, split, strip;
import std.typecons: tuple, Tuple;
import std.uni: isAlpha, toLower, isWhite;
auto coSizeDirList (string FFs, ulong SizeDir) {
ulong subdirTotal;
ulong subdirTotalGB;
Array!string Subdir;
Array!ulong Subsize;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong(dirEntries(d,
SpanMode.depth).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024);
if (subdirTotalGB > SizeDir) { Subdir ~= d;
Re: Passing array as an function argument.
On Monday, 11 September 2017 at 12:44:00 UTC, wobbles wrote: On Monday, 11 September 2017 at 12:20:08 UTC, Vino.B wrote: On Monday, 11 September 2017 at 12:03:32 UTC, wobbles wrote: On Monday, 11 September 2017 at 11:58:18 UTC, Vino.B wrote: [...] The type returned from Test1() is a `RangeT!(Array!string)`. This is due to the `[]` on the end of `Fs[]`. Remove the `[]` to just return a `Array!string`. Hi, If i remove the `[]` at the end of `Fs[]` I am getting the same error, if i remove the `[]` from the file "auto t1 = Test1[];" the out is empty. From, Vino.B This works: https://dpaste.dzfl.pl/1fd9021739ad Hi Wobbles, Thank you very much, that solved my problem. From, Vino.B
Re: Passing array as an function argument.
On Monday, 11 September 2017 at 12:03:32 UTC, wobbles wrote:
On Monday, 11 September 2017 at 11:58:18 UTC, Vino.B wrote:
Hi All,
Can some one help me on how to pass a container array as a
function argument , the below code throws an error,
Error: Error: function T3.Test2 (Array!string t1) is not
callable using argument types (RangeT!(Array!string))
import std.stdio: writeln;
import std.container;
auto Test2 (Array!string t1) {
return t1;
}
auto Test1 () {
auto Fs = Array!string("C:\\Temp\\TEST1\\BACKUP",
"C:\\Temp\\TEST2\\EXPORT");
return Fs[];
}
void main () {
auto t1 = Test1[];
Test2(t1);
}
From,
Vino.B
The type returned from Test1() is a `RangeT!(Array!string)`.
This is due to the `[]` on the end of `Fs[]`.
Remove the `[]` to just return a `Array!string`.
Hi,
If i remove the `[]` at the end of `Fs[]` I am getting the same
error, if i remove the `[]` from the file "auto t1 = Test1[];"
the out is empty.
From,
Vino.B
Passing array as an function argument.
Hi All,
Can some one help me on how to pass a container array as a
function argument , the below code throws an error,
Error: Error: function T3.Test2 (Array!string t1) is not callable
using argument types (RangeT!(Array!string))
import std.stdio: writeln;
import std.container;
auto Test2 (Array!string t1) {
return t1;
}
auto Test1 () {
auto Fs = Array!string("C:\\Temp\\TEST1\\BACKUP",
"C:\\Temp\\TEST2\\EXPORT");
return Fs[];
}
void main () {
auto t1 = Test1[];
Test2(t1);
}
From,
Vino.B
Re: Deprecation: std.container.array.RangeT(A) is not visible from module Size
On Sunday, 10 September 2017 at 23:10:20 UTC, Ali Çehreli wrote:
On 09/10/2017 09:53 AM, Vino.B wrote:
> auto coSizeDirList (string FFs, int SizeDir) {
> //alias DirSizeList = typeof(coSizeDirList());
I worked with a version of coSizeDirList() that did not take
any parameters. (Could be from an earlier post of yours.)
In this case, you can must use a compilable expression. Since
coSizeDirList above takes a string and int, you can get its
return value like this:
alias DirSizeList = typeof(coSizeDirList(string.init,
int.init))
The following can work as well:
alias DirSizeList = typeof(coSizeDirList("", 42))
However, there are other compilation errors when I do that.
Ali
Hi Ali,
The issue occurs at the line "MSresult.get ~= MSizeDirListData"
when we try to append data the error message is as below, if we
change this line to "MSresult.get = MSizeDirListData" then the
code complies without any issue but the output is empty.
T2.d(41): Error: cannot append type Tuple!(RangeT!(Array!string),
RangeT!(Array!ulong)) to type Tuple!(RangeT!(Array!string),
RangeT!(Array!ulong))
Code:
import core.stdc.stdlib: exit;
import std.algorithm: all, among, filter, map, setDifference,
sort, uniq, each, joiner;
import std.array: appender, join;
import std.container.array;
import std.conv: to;
import std.datetime.systime: Clock, days, SysTime;
import std.file: SpanMode, dirEntries, exists, isFile, mkdir,
remove, rmdirRecurse;
import std.getopt;
import std.parallelism: parallel, task, taskPool;
import std.path: absolutePath, baseName, dirName,
isValidFilename, isValidPath, globMatch;
import std.range: empty;
import std.stdio: File, writefln, writeln;
import std.string: chomp, chop, isNumeric, split, strip;
import std.typecons: tuple, Tuple;
import std.uni: isAlpha, toLower, isWhite;
auto coSizeDirList (string FFs, int SizeDir) {
ulong subdirTotal;
ulong subdirTotalGB;
Array!string Subdir;
Array!ulong Subsize;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong(dirEntries(d,
SpanMode.depth).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024); { Subdir ~= d;
Subsize ~= subdirTotalGB; }
if (subdirTotalGB > SizeDir)
subdirTotal = 0;
}
return tuple (Subdir[], Subsize[]);
}
void ptSizeDirList (string[] SizeDirlst, int SizeDir) {
alias DirSizeList = typeof(coSizeDirList(string.init,
ulong.init));
auto MSresult = taskPool.workerLocalStorage!DirSizeList();
foreach (string Fs; parallel(SizeDirlst[0 .. $], 1)) {
auto FFs = Fs.strip;
auto MSizeDirList = task(, FFs, SizeDir);
MSizeDirList.executeInNewThread();
auto MSizeDirListData = MSizeDirList.workForce;
//MSresult.get ~= MSizeDirListData;
MSresult.get = MSizeDirListData;
}
foreach(i; MSresult.tupleof)
//if (!i.empty) { writefln("%(%-(%-63s %)\n%)", i[].sort!((a,b)
=> a[0] < b[0]).uniq); }
writeln(i);
}
void main () {
string SizeDirlst = "C:\\Temp\\TEAM\\BACKUP";
int SizeDir = 1;
coSizeDirList(SizeDirlst, SizeDir);
}
From,
Vino.B
Re: Deprecation: std.container.array.RangeT(A) is not visible from module Size
On Sunday, 10 September 2017 at 15:46:46 UTC, Ali Çehreli wrote:
On 09/10/2017 04:54 AM, Vino.B wrote:
> Thank you very much, as stated by you i used the auto
function and now
> i am get the output without any warnings
That's because now you're taking advantage of D's type
inference. Although it doesn't cover the entire story, you may
want to read about D's Voldemort types. (Your case did not
involve Voldemort types though; in your case it was just a
private symbol.)
So, you can give a name to that return value yourself:
alias DirSizeList = typeof(coSizeDirList());
> Sub Function:
Unrelated: They are all called "functions" in D.
> //Array!string MStext;
> string[][] MStext; // Getting Error on this line, while
trying to
> change it to auto;
> auto MSresult = taskPool.workerLocalStorage(MStext);
Now you can use DirSizeList there:
auto MSresult = taskPool.workerLocalStorage!DirSizeList();
Ali
Hi Ali,
I tried to add/replace the above line's but still not working.
Error: function T2.coSizeDirList (string FFs, int SizeDir) is not
callable using argument types ()
import core.stdc.stdlib: exit;
import std.algorithm: all, among, filter, map, setDifference,
sort, uniq, each, joiner;
import std.array: appender, join;
import std.container.array;
import std.conv: to;
import std.datetime.systime: Clock, days, SysTime;
import std.file: SpanMode, dirEntries, exists, isFile, mkdir,
remove, rmdirRecurse;
import std.getopt;
import std.parallelism: parallel, task, taskPool;
import std.path: absolutePath, baseName, dirName,
isValidFilename, isValidPath, globMatch;
import std.range: empty;
import std.stdio: File, writefln, writeln;
import std.string: chomp, chop, isNumeric, split, strip;
import std.typecons: tuple, Tuple;
import std.uni: isAlpha, toLower, isWhite;
auto coSizeDirList (string FFs, int SizeDir) {
ulong subdirTotal;
ulong subdirTotalGB;
Array!string Subdir;
Array!ulong Subsize;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong(dirEntries(d,
SpanMode.depth).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024); { Subdir ~= d;
Subsize ~= subdirTotalGB; }
if (subdirTotalGB > SizeDir)
subdirTotal = 0;
}
return tuple (Subdir[], Subsize[]);
// alias DirSizeList = typeof(coSizeDirList());
}
//alias DirSizeList = typeof(coSizeDirList());
void ptSizeDirList (string[] SizeDirlst, int SizeDir) {
try {
//alias DirSizeList = typeof(coSizeDirList());
auto MSresult = taskPool.workerLocalStorage!DirSizeList();
writeln("Function \t : List the Folder whose Size greater then
", SizeDir, " GB");
writeln("Dir. Scanned \t :", SizeDirlst);
writeln("");
writefln("%-63s %.20s", "File Name", "Size (GB)");
writeln("");
foreach (string Fs; parallel(SizeDirlst[0 .. $], 1)) {
auto FFs = Fs.strip;
auto MSizeDirList = task(, FFs, SizeDir);
MSizeDirList.executeInNewThread();
auto MSizeDirListData = MSizeDirList.workForce;
MSresult.get ~= MSizeDirListData;
}
foreach(i; MSresult.toRange)
if (!i.empty) {
writefln("%(%-(%-63s %)\n%)", i[].sort!((a,b) => a[0] <
b[0]).uniq); }
writeln("");
} catch (Exception e) { writeln(e.msg); }
}
void main () {
string SizeDirlst = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 10;
coSizeDirList(SizeDirlst, SizeDir);
}
Re: Deprecation: std.container.array.RangeT(A) is not visible from module Size
On Friday, 8 September 2017 at 23:48:14 UTC, Ali Çehreli wrote:
On 09/08/2017 11:21 AM, Vino.B wrote:
> One final help on how to print the below
> output , just in case if this issue is fixed in next release,
>
> Output:
> [Tuple!string("C:\\Temp\\sapnas2\\BACKUP\\dir1"),
> Tuple!string("C:\\Temp\\sapnas2\\BACKUP\\DND5")][34, 4]
>
> Required output:
> C:\\Temp\\sapnas2\\BACKUP\\dir1 34
> C:\\Temp\\sapnas2\\BACKUP\\DND5 4
std.algorithm.zip can help:
void main () {
auto results = coSizeDirList();
auto dirs = results[][0];// A range
auto sizes = results[][1]; // Another range
auto combined = zip(dirs, sizes);// Corresponding
elements linked
foreach (result; combined) {
auto dir = result[0]; // The element from the first
range
auto size = result[1];// The element from the
second range
writefln("%-40s %20s", dir, size);
}
}
Ali
Hi Ali,
Thank you very much, as stated by you i used the auto function
and now i am get the output without any warnings, but this do not
help me in my overall program, meaning, when i call this sub
function from the main thread function program it is not working
as expected and throwing an error.
Sub Function:
auto coSizeDirList (string FFs, int SizeDir) {
ulong subdirTotal;
ulong subdirTotalGB;
Array!string Subdir;
Array!ulong Subsize;
//Tuple!((Array!string), (Array!string)) Result;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong(dirEntries(d,
SpanMode.depth).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024); { Subdir ~= d;
Subsize ~= subdirTotalGB; }
if (subdirTotalGB > SizeDir)
subdirTotal = 0;
}
return tuple (Subdir[], Subsize[]);
}
Main Thread function :
void ptSizeDirList (string[] SizeDirlst, int SizeDir) {
try {
//Array!string MStext;
string[][] MStext; // Getting Error on this line, while
trying to change it to auto;
auto MSresult = taskPool.workerLocalStorage(MStext);
logF.writeln("Function \t : List the Folder whose Size greater
then ", SizeDir, " GB");
logF.writeln("Dir. Scanned \t :", SizeDirlst);
logF.writeln("");
logF.writefln("%-63s %.20s", "File Name", "Size (GB)");
logF.writeln("");
foreach (string Fs; parallel(SizeDirlst[0 .. $], 1)) {
auto FFs = Fs.strip;
auto MSizeDirList = task(, FFs, SizeDir);
MSizeDirList.executeInNewThread();
auto MSizeDirListData = MSizeDirList.workForce;
MSresult.get ~= MSizeDirListData;
}
foreach(i; MSresult.toRange)
if (!i.empty) {
writefln("%(%-(%-63s %)\n%)", i[].sort!((a,b) => a[0] <
b[0]).uniq); }
writeln("");
} catch (Exception e) { writeln(e.msg); }
}
void main () {
string SizeDirlst = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ptSizeDirList(SizeDirlst, SizeDir);
}
Error:
Error: no identifier for declarator MStext
Deprecation: use { } for an empty statement, not ;
Re: Deprecation: std.container.array.RangeT(A) is not visible from module Size
On Friday, 8 September 2017 at 16:58:35 UTC, Ali Çehreli wrote:
On 09/08/2017 07:48 AM, Vino.B wrote:
> if
> std.container.array.RangeT(A) is deprecated
No, it's not deprecated. It's a private symbol of the
std.container.array module. You shouldn't be able to use it at
all. The fact that you are able to use it is due to a bug fix,
which hasn't been fully activated yet. The bug is fixed but the
compiler is still letting you use the symbol until the bug is
effective in some future date.
In summary, you should not refer to RangeT in your code at all.
Use auto return types if you have to but I'm happy to see that
you've solved the remaining issues.
> subdirTotalGB = (subdirTotal/1024/1024); {
Subdir ~= d;
That should be MB, right? Kilo, mega, giga, etc.
Ali
Hi Ali,
Sure, will wait for the fix, and for the line "subdirTotalGB =
(subdirTotal/1024/1024); {
Subdir ~= d;" that's ok the issue was rising from the line
"string[][]" as you know std.container array do not have the
capability of multi dimensional array hence i need to change it
back from "Tuple!((Array!string), (Array!string)) Result;" which
was causing the issue, will try it once again once the issue is
fixed. Once again thank you very much for you help. One final
help on how to print the below output , just in case if this
issue is fixed in next release,
Output:
[Tuple!string("C:\\Temp\\sapnas2\\BACKUP\\dir1"),
Tuple!string("C:\\Temp\\sapnas2\\BACKUP\\DND5")][34, 4]
Required output:
C:\\Temp\\sapnas2\\BACKUP\\dir1 34
C:\\Temp\\sapnas2\\BACKUP\\DND5 4
From,
Vino.B
Re: Deprecation: std.container.array.RangeT(A) is not visible from module Size
On Friday, 8 September 2017 at 15:47:39 UTC, Vino.B wrote:
On Friday, 8 September 2017 at 14:48:38 UTC, Vino.B wrote:
Hi All,
The below code output's the below warning, so if
std.container.array.RangeT(A) is deprecated then what is the
equivalent for this, request your help on this.
Warning :
Size.d(10): Deprecation: std.container.array.RangeT(A) is not
visible from module Size
Size.d(10): Deprecation: std.container.array.RangeT(A) is not
visible from module Size
Size.d(10): Deprecation: std.container.array.RangeT(A) is not
visible from module Size
Size.d(10): Deprecation: std.container.array.RangeT(A) is not
visible from module Size
["C:\\Temp\\sapnas2\\BACKUP\\dir1",
"C:\\Temp\\sapnas2\\BACKUP\\DND3",
"C:\\Temp\\sapnas2\\BACKUP\\DND5"][34, 1, 5]
Code:
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir, isFile;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
import std.range;
Tuple!(RangeT!(Array!string), RangeT!(Array!ulong))
coSizeDirList () {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!(string) Subdir;
Array!(ulong) Subsize;
Tuple!((Array!string), (Array!string)) Result;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong(dirEntries(d,
SpanMode.depth).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024); { Subdir ~= d;
Subsize ~= subdirTotalGB; }
if (subdirTotalGB > SizeDir)
subdirTotal = 0;
}
return tuple (Subdir[], Subsize[]);
}
void main () {
writeln(coSizeDirList[]);
}
From,
Vino.B
Hi All,
Was able to resolve the above issue but not sure whether it is
correct and now i am getting the output as below, request your
help.
Output:
C:\Temp\sapnas2\BACKUP\dir1, 34, C:\Temp\sapnas2\BACKUP\DND3,
1, C:\Temp\sapnas2\BACKUP\DND5, 5
Required Output:
C:\Temp\sapnas2\BACKUP\dir134
C:\Temp\sapnas2\BACKUP\DND3 1
C:\Temp\sapnas2\BACKUP\DND5 5
Program:
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir, isFile;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
import std.range;
Array!string coSizeDirList () {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!(string) Subsize;
Array!string Result;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong(dirEntries(d,
SpanMode.depth).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024); { Result ~= d;
Result ~= to!string(subdirTotalGB); }
if (subdirTotalGB > SizeDir)
subdirTotal = 0;
}
return Result;
}
void main () {
writefln("%-(%s, %)", coSizeDirList[]);
}
From,
Vino.B
Hi All,
At last was able to resolve the issue including the output too,
thank you very much for your help, please let me know in case if
you find any issue with the below code.
import std.algorithm: filter, map;
import std.container;
import std.file: SpanMode, dirEntries, isDir, isFile;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
import std.range;
string[][] coSizeDirList () {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!(string) Subsize;
string[][] Result;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong(dirEntries(d,
SpanMode.depth).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024);
if (subdirTotalGB > SizeDir) { Result ~= [[d] ~
[to!string(subdirTotalGB)]]; }
subdirTotal = 0;
}
return Result;
}
void main () {
writefln("%(%-(%-63s %)\n%)", coSizeDirList[]);
}
From,
Vino.B
Re: Container Array
On Friday, 8 September 2017 at 15:48:47 UTC, Vino.B wrote:
On Friday, 8 September 2017 at 12:14:46 UTC, Vino.B wrote:
On Friday, 8 September 2017 at 09:51:38 UTC, Ali Çehreli wrote:
[...]
Hi Ali,
As stated earlier my release 1 code are still using
std.array, so now in release 2 i am converting all my standard
array to container array. My program has 5 function and I was
able to adopt 4 function to container array, and facing issue
with this 1 function. I would like to have all my function to
be neither in standard array nor in container array, more over
I am facing gc issue in standard array and this is the main
reason I would like to convert my function to container array,
as this function would find the size of folder which are
greater then a specified size in a 5 file system each with 10
TB, so i have added the thread(with Local storage) and
parallelism to my function as container array give's me the
option of reserving memory, so that i would bump it up as and
when it is required with any gc issues. All help to resolve
this issue would be appreciated.
Updated Code:
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
import std.range;
Tuple!(RangeT!(Array!string), RangeT!(Array!ulong))
coSizeDirList () {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!(string) Subdir;
Array!(ulong) Subsize;
Tuple!((Array!string), (Array!string)) Result;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong ((dirEntries(d,
SpanMode.depth)).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024); { Subdir ~= d;
Subsize ~= subdirTotalGB; }
if (subdirTotalGB > SizeDir)
subdirTotal = 0;
}
return tuple (Subdir[], Subsize[]);
}
void main () {
writeln(coSizeDirList[]);
}
From,
Vino.B
Hi Ali,
Was able to resolve the above issue but not sure whether it is
correct and now i am getting the output as below, request your
help.
Output:
C:\Temp\sapnas2\BACKUP\dir1, 34, C:\Temp\sapnas2\BACKUP\DND3,
1, C:\Temp\sapnas2\BACKUP\DND5, 5
Required Output:
C:\Temp\sapnas2\BACKUP\dir134
C:\Temp\sapnas2\BACKUP\DND3 1
C:\Temp\sapnas2\BACKUP\DND5 5
Program:
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir, isFile;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
import std.range;
Array!string coSizeDirList () {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!(string) Subsize;
Array!string Result;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong(dirEntries(d,
SpanMode.depth).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024); { Result ~= d;
Result ~= to!string(subdirTotalGB); }
if (subdirTotalGB > SizeDir)
subdirTotal = 0;
}
return Result;
}
void main () {
writefln("%-(%s, %)", coSizeDirList[]);
}
From,
Vino.B
Hi Ali,
At last was able to resolve the issue including the output too,
thank you very much for your help, please let me know in case if
you find any issue with the below code.
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir, isFile;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
import std.range;
string[][] coSizeDirList () {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!(string) Subsize;
string[][] Result;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong(dirEntries(d,
SpanMode.depth).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024);
if (subdirTotalGB > SizeDir) { Result ~=
Re: Container Array
On Friday, 8 September 2017 at 12:14:46 UTC, Vino.B wrote:
On Friday, 8 September 2017 at 09:51:38 UTC, Ali Çehreli wrote:
On 09/07/2017 11:21 PM, Vino.B wrote:
> At last was able to print the output, but i am getting some
> "Deprecation" warnings like below and also can you help me in
formating
> the output to display ulong.
>
> Output:
> Size.d(9): Deprecation: std.container.array.RangeT(A) is not
visible
> from module Size
That's due to std.container.array.RangeT being private. The
deprecation warning is about a bug that leaked such private
symbols when they were imported selectively (I think). Now the
bug is fixed, you won't be able to access the symbol at the
end of the deprecation period.
> import std.algorithm: filter, map, fold;
> import std.container;
> import std.file: SpanMode, dirEntries, isDir;
> import std.stdio: File, writefln, writeln;
> import std.typecons: tuple, Tuple;
> import std.parallelism: parallel;
> import std.conv;
> import std.range;
> Tuple!(RangeT!(Array!(Tuple!string)), RangeT!(Array!ulong))
> coSizeDirList () {
> string FFs = "C:\\Temp\\sapnas2\\BACKUP";
> int SizeDir = 1;
> ulong subdirTotal;
> ulong subdirTotalGB;
> Array!(ulong) Subdata;
> auto dFiles = Array!(Tuple!(string)) (dirEntries(FFs,
> SpanMode.shallow).filter!(a => a.isDir).map!(a =>
tuple(a.name)));
> foreach (d; dFiles[]) {
> auto SdFiles = Array!(Tuple!(ulong))
(dirEntries(d[0],
> SpanMode.depth).map!(a => tuple(a.size)));
> foreach(f; SdFiles[]) { subdirTotal +=
f.fold!((a, b) =>
> a + b); }
> subdirTotalGB = (subdirTotal/1024/1024);
> if (subdirTotalGB > SizeDir) { Subdata ~=
> subdirTotalGB; }
> subdirTotal = 0;
> }
> return tuple (dFiles[], Subdata[]);
> }
>
> void main () {
> writeln(coSizeDirList[]);
> //writefln("%(%-(%-63s %)\n%)", coSizeDirList[]);
> }
I apologize for not really having time to look at what you're
trying to achieve. I gave you advice which ended up trying to
solve compilation errors.
I think the main problem here is to determine directories
above a certain size. So, I think Array should enter the
picture only if built-in arrays are not usable for some
reason. Even better, one should stay with lazy range
algorithms as long as it's possible.
How about the following approach, which you can either use
directly or populate an Array with:
import std.algorithm: filter, map, sum;
import std.file: SpanMode, dirEntries, isDir, DirEntry;
import std.stdio: writeln;
auto dFiles(string dirName) {
return dirEntries(dirName, SpanMode.shallow).filter!(a =>
a.isDir);
}
auto totalSize(DirEntry dir) {
return dirEntries(dir, SpanMode.depth).map!(a =>
a.size).sum;
}
struct DirInfo {
string name;
ulong size;
}
auto coSizeDirList (string dirName, ulong sizeLimit) {
return dFiles(dirName).map!(dir => DirInfo(dir.name,
dir.totalSize)).filter!(info => info.size > sizeLimit);
}
void main () {
writeln(coSizeDirList("./deleteme", 1));
// Only if Array is really needed:
import std.container : Array;
auto arr = Array!DirInfo(coSizeDirList("./deleteme", 42));
writeln(arr[]);
}
Ali
Hi Ali,
As stated earlier my release 1 code are still using
std.array, so now in release 2 i am converting all my standard
array to container array. My program has 5 function and I was
able to adopt 4 function to container array, and facing issue
with this 1 function. I would like to have all my function to
be neither in standard array nor in container array, more over
I am facing gc issue in standard array and this is the main
reason I would like to convert my function to container array,
as this function would find the size of folder which are
greater then a specified size in a 5 file system each with 10
TB, so i have added the thread(with Local storage) and
parallelism to my function as container array give's me the
option of reserving memory, so that i would bump it up as and
when it is required with any gc issues. All help to resolve
this issue would be appreciated.
Updated Code:
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
import std.range;
Tuple!(RangeT!(Array!string), RangeT!(Array!ulong))
coSizeDirList () {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!(string) Subdir;
Array!(ulong) Subsize;
Tuple!((Array!string), (Array!string)) Result;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong ((dirEntries(d,
SpanMode.depth)).map!(a => a.size));
Re: Deprecation: std.container.array.RangeT(A) is not visible from module Size
On Friday, 8 September 2017 at 14:48:38 UTC, Vino.B wrote:
Hi All,
The below code output's the below warning, so if
std.container.array.RangeT(A) is deprecated then what is the
equivalent for this, request your help on this.
Warning :
Size.d(10): Deprecation: std.container.array.RangeT(A) is not
visible from module Size
Size.d(10): Deprecation: std.container.array.RangeT(A) is not
visible from module Size
Size.d(10): Deprecation: std.container.array.RangeT(A) is not
visible from module Size
Size.d(10): Deprecation: std.container.array.RangeT(A) is not
visible from module Size
["C:\\Temp\\sapnas2\\BACKUP\\dir1",
"C:\\Temp\\sapnas2\\BACKUP\\DND3",
"C:\\Temp\\sapnas2\\BACKUP\\DND5"][34, 1, 5]
Code:
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir, isFile;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
import std.range;
Tuple!(RangeT!(Array!string), RangeT!(Array!ulong))
coSizeDirList () {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!(string) Subdir;
Array!(ulong) Subsize;
Tuple!((Array!string), (Array!string)) Result;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong(dirEntries(d,
SpanMode.depth).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024); { Subdir ~= d;
Subsize ~= subdirTotalGB; }
if (subdirTotalGB > SizeDir)
subdirTotal = 0;
}
return tuple (Subdir[], Subsize[]);
}
void main () {
writeln(coSizeDirList[]);
}
From,
Vino.B
Hi All,
Was able to resolve the above issue but not sure whether it is
correct and now i am getting the output as below, request your
help.
Output:
C:\Temp\sapnas2\BACKUP\dir1, 34, C:\Temp\sapnas2\BACKUP\DND3, 1,
C:\Temp\sapnas2\BACKUP\DND5, 5
Required Output:
C:\Temp\sapnas2\BACKUP\dir134
C:\Temp\sapnas2\BACKUP\DND3 1
C:\Temp\sapnas2\BACKUP\DND5 5
Program:
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir, isFile;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
import std.range;
Array!string coSizeDirList () {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!(string) Subsize;
Array!string Result;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong(dirEntries(d,
SpanMode.depth).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024); { Result ~= d;
Result ~= to!string(subdirTotalGB); }
if (subdirTotalGB > SizeDir)
subdirTotal = 0;
}
return Result;
}
void main () {
writefln("%-(%s, %)", coSizeDirList[]);
}
From,
Vino.B
Deprecation: std.container.array.RangeT(A) is not visible from module Size
Hi All,
The below code output's the below warning, so if
std.container.array.RangeT(A) is deprecated then what is the
equivalent for this, request your help on this.
Warning :
Size.d(10): Deprecation: std.container.array.RangeT(A) is not
visible from module Size
Size.d(10): Deprecation: std.container.array.RangeT(A) is not
visible from module Size
Size.d(10): Deprecation: std.container.array.RangeT(A) is not
visible from module Size
Size.d(10): Deprecation: std.container.array.RangeT(A) is not
visible from module Size
["C:\\Temp\\sapnas2\\BACKUP\\dir1",
"C:\\Temp\\sapnas2\\BACKUP\\DND3",
"C:\\Temp\\sapnas2\\BACKUP\\DND5"][34, 1, 5]
Code:
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir, isFile;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
import std.range;
Tuple!(RangeT!(Array!string), RangeT!(Array!ulong)) coSizeDirList
() {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!(string) Subdir;
Array!(ulong) Subsize;
Tuple!((Array!string), (Array!string)) Result;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong(dirEntries(d,
SpanMode.depth).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024); { Subdir ~= d;
Subsize ~= subdirTotalGB; }
if (subdirTotalGB > SizeDir)
subdirTotal = 0;
}
return tuple (Subdir[], Subsize[]);
}
void main () {
writeln(coSizeDirList[]);
}
From,
Vino.B
Re: Container Array
On Friday, 8 September 2017 at 09:51:38 UTC, Ali Çehreli wrote:
On 09/07/2017 11:21 PM, Vino.B wrote:
> At last was able to print the output, but i am getting some
> "Deprecation" warnings like below and also can you help me in
formating
> the output to display ulong.
>
> Output:
> Size.d(9): Deprecation: std.container.array.RangeT(A) is not
visible
> from module Size
That's due to std.container.array.RangeT being private. The
deprecation warning is about a bug that leaked such private
symbols when they were imported selectively (I think). Now the
bug is fixed, you won't be able to access the symbol at the end
of the deprecation period.
> import std.algorithm: filter, map, fold;
> import std.container;
> import std.file: SpanMode, dirEntries, isDir;
> import std.stdio: File, writefln, writeln;
> import std.typecons: tuple, Tuple;
> import std.parallelism: parallel;
> import std.conv;
> import std.range;
> Tuple!(RangeT!(Array!(Tuple!string)), RangeT!(Array!ulong))
> coSizeDirList () {
> string FFs = "C:\\Temp\\sapnas2\\BACKUP";
> int SizeDir = 1;
> ulong subdirTotal;
> ulong subdirTotalGB;
> Array!(ulong) Subdata;
> auto dFiles = Array!(Tuple!(string)) (dirEntries(FFs,
> SpanMode.shallow).filter!(a => a.isDir).map!(a =>
tuple(a.name)));
> foreach (d; dFiles[]) {
> auto SdFiles = Array!(Tuple!(ulong))
(dirEntries(d[0],
> SpanMode.depth).map!(a => tuple(a.size)));
> foreach(f; SdFiles[]) { subdirTotal +=
f.fold!((a, b) =>
> a + b); }
> subdirTotalGB = (subdirTotal/1024/1024);
> if (subdirTotalGB > SizeDir) { Subdata ~=
> subdirTotalGB; }
> subdirTotal = 0;
> }
> return tuple (dFiles[], Subdata[]);
> }
>
> void main () {
> writeln(coSizeDirList[]);
> //writefln("%(%-(%-63s %)\n%)", coSizeDirList[]);
> }
I apologize for not really having time to look at what you're
trying to achieve. I gave you advice which ended up trying to
solve compilation errors.
I think the main problem here is to determine directories above
a certain size. So, I think Array should enter the picture only
if built-in arrays are not usable for some reason. Even better,
one should stay with lazy range algorithms as long as it's
possible.
How about the following approach, which you can either use
directly or populate an Array with:
import std.algorithm: filter, map, sum;
import std.file: SpanMode, dirEntries, isDir, DirEntry;
import std.stdio: writeln;
auto dFiles(string dirName) {
return dirEntries(dirName, SpanMode.shallow).filter!(a =>
a.isDir);
}
auto totalSize(DirEntry dir) {
return dirEntries(dir, SpanMode.depth).map!(a =>
a.size).sum;
}
struct DirInfo {
string name;
ulong size;
}
auto coSizeDirList (string dirName, ulong sizeLimit) {
return dFiles(dirName).map!(dir => DirInfo(dir.name,
dir.totalSize)).filter!(info => info.size > sizeLimit);
}
void main () {
writeln(coSizeDirList("./deleteme", 1));
// Only if Array is really needed:
import std.container : Array;
auto arr = Array!DirInfo(coSizeDirList("./deleteme", 42));
writeln(arr[]);
}
Ali
Hi Ali,
As stated earlier my release 1 code are still using std.array,
so now in release 2 i am converting all my standard array to
container array. My program has 5 function and I was able to
adopt 4 function to container array, and facing issue with this 1
function. I would like to have all my function to be neither in
standard array nor in container array, more over I am facing gc
issue in standard array and this is the main reason I would like
to convert my function to container array, as this function would
find the size of folder which are greater then a specified size
in a 5 file system each with 10 TB, so i have added the
thread(with Local storage) and parallelism to my function as
container array give's me the option of reserving memory, so that
i would bump it up as and when it is required with any gc issues.
All help to resolve this issue would be appreciated.
Updated Code:
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
import std.range;
Tuple!(RangeT!(Array!string), RangeT!(Array!ulong)) coSizeDirList
() {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!(string) Subdir;
Array!(ulong) Subsize;
Tuple!((Array!string), (Array!string)) Result;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong ((dirEntries(d,
SpanMode.depth)).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
Re: Container Array
On Thursday, 7 September 2017 at 20:47:43 UTC, Ali Çehreli wrote:
On 09/07/2017 10:39 AM, Vino.B wrote:
> Array!(Tuple!(string, ulong)) coSizeDirList () {
You stated the return type explicitly above.
> return tuple (dFiles[], Subdata[]);
According to the error message, what is being returned does not
have the same type:
> Test1.d(27): Error: cannot implicitly convert expression
> (tuple(dFiles.opSlice(), Subdata.opSlice())) of type
> Tuple!(RangeT!(Array!(Tuple!string)), RangeT!(Array!ulong)) to
> Array!(Tuple!(string, ulong))
The actual return type is
Tuple!(RangeT!(Array!(Tuple!string)), RangeT!(Array!ulong))
There needs to be some transformations to match the two.
Ali
Hi Ali,
At last was able to print the output, but i am getting some
"Deprecation" warnings like below and also can you help me in
formating the output to display ulong.
Output:
Size.d(9): Deprecation: std.container.array.RangeT(A) is not
visible from module Size
Size.d(9): Deprecation: std.container.array.RangeT(A) is not
visible from module Size
Size.d(9): Deprecation: std.container.array.RangeT(A) is not
visible from module Size
Size.d(9): Deprecation: std.container.array.RangeT(A) is not
visible from module Size
[Tuple!string("C:\\Temp\\sapnas2\\BACKUP\\dir1"),
Tuple!string("C:\\Temp\\sapnas2\\BACKUP\\DND3"),
Tuple!string("C:\\Temp\\sapnas2\\BACKUP\\DND5")][34, 4]
Program:
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
import std.range;
Tuple!(RangeT!(Array!(Tuple!string)), RangeT!(Array!ulong))
coSizeDirList () {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!(ulong) Subdata;
auto dFiles = Array!(Tuple!(string)) (dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir).map!(a => tuple(a.name)));
foreach (d; dFiles[]) {
auto SdFiles = Array!(Tuple!(ulong)) (dirEntries(d[0],
SpanMode.depth).map!(a => tuple(a.size)));
foreach(f; SdFiles[]) { subdirTotal += f.fold!((a, b) => a +
b); }
subdirTotalGB = (subdirTotal/1024/1024);
if (subdirTotalGB > SizeDir) { Subdata
~= subdirTotalGB; }
subdirTotal = 0;
}
return tuple (dFiles[], Subdata[]);
}
void main () {
writeln(coSizeDirList[]);
//writefln("%(%-(%-63s %)\n%)", coSizeDirList[]);
}
Re: Container Array
On Thursday, 7 September 2017 at 17:12:14 UTC, Vino.B wrote:
On Thursday, 7 September 2017 at 15:07:56 UTC, Vino.B wrote:
On Thursday, 7 September 2017 at 14:26:08 UTC, Ali Çehreli
wrote:
On 09/07/2017 03:56 AM, Vino.B wrote:
writeln(coCleanFiles);
Access the elements by taking a slice of the container:
writeln(coCleanFiles[]);
Ali
Hi Ali,
Thank you very much, was ablee to resolve this issue and now
facing a new issue as the below code is not working as
expected. The below code has to list the folders and their
size's.
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
Array!(Tuple!(string, ulong)) coSizeDirList () {
string FFs = "C:\\Temp\\TEST1\\BACKUP";
int SizeDir = 10;
ulong subdirTotal;
ulong subdirTotalGB;
auto dFiles = Array!(Tuple!(string)) (dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir).map!(a =>
tuple(a.name)));
foreach (d; dFiles) {
auto SdFiles = Array!(Tuple!(ulong)) (dirEntries(d[0],
SpanMode.depth).map!(a => tuple(a.size)));
foreach(f; parallel(SdFiles, 1)) { subdirTotal +=
f.fold!((a, b) => a + b); }
subdirTotalGB = (subdirTotal/1024/1024);
if (subdirTotalGB > SizeDir) { auto Subdata =
Array!(Tuple!(string, ulong))(dFiles ~ subdirTotalGB); }
subdirTotal = 0;
}
return Subdata;
}
void main () {
writeln (coSizeDirList[]);
}
Hi,
Few updates,
If i change the function to main i am able to print the
required output, but if i change the main to function and call
this function from another main then i am not able to return
the result from the function.
Updated Code:
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
Array!(Tuple!(string, ulong)) coSizeDirList () {
//void main () {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!(Tuple!(ulong)) Subdata;
auto dFiles = Array!(Tuple!(string)) (dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir).map!(a =>
tuple(a.name)));
foreach (d; dFiles[]) {
auto SdFiles = Array!(Tuple!(ulong)) (dirEntries(d[0],
SpanMode.depth).map!(a => tuple(a.size)));
foreach(f; SdFiles[]) { subdirTotal += f.fold!((a, b) => a
+ b); }
subdirTotalGB = (subdirTotal/1024/1024);
if (subdirTotalGB > SizeDir) { Subdata
~= subdirTotalGB; }
subdirTotal = 0;
}
//writeln(dFiles);
//writeln(Subdata);
return dFiles[];
return Subdata[];
}
void main () {
writeln (coSizeDirList[]);
}
From,
Vino.B
Few Update:
Update Code :
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
Array!(Tuple!(string, ulong)) coSizeDirList () {
//void main () {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!ulong Subdata;
auto dFiles = Array!(Tuple!(string)) (dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir).map!(a => tuple(a.name)));
foreach (d; dFiles[]) {
auto SdFiles = Array!(Tuple!(ulong)) (dirEntries(d[0],
SpanMode.depth).map!(a => tuple(a.size)));
foreach(f; SdFiles[]) { subdirTotal += f.fold!((a, b) => a +
b); }
subdirTotalGB = (subdirTotal/1024/1024);
if (subdirTotalGB > SizeDir) { Subdata
~= subdirTotalGB; }
subdirTotal = 0;
}
//writeln(dFiles[]);
//writeln(Subdata[]);
return tuple (dFiles[], Subdata[]);
//return Subdata[];
//return Result;
}
void main () {
writeln (coSizeDirList[]);
}
Error Output
Test1.d(27): Error: cannot implicitly convert expression
(tuple(dFiles.opSlice(), Subdata.opSlice())) of type
Tuple!(RangeT!(Array!(Tuple!string)), RangeT!(Array!ulong)) to
Array!(Tuple!(string, ulong))
Failed: ["dmd", "-v", "-o-", "Test1.d", "-I."]
From,
Vino.B
Re: Container Array
On Thursday, 7 September 2017 at 15:07:56 UTC, Vino.B wrote:
On Thursday, 7 September 2017 at 14:26:08 UTC, Ali Çehreli
wrote:
On 09/07/2017 03:56 AM, Vino.B wrote:
writeln(coCleanFiles);
Access the elements by taking a slice of the container:
writeln(coCleanFiles[]);
Ali
Hi Ali,
Thank you very much, was ablee to resolve this issue and now
facing a new issue as the below code is not working as
expected. The below code has to list the folders and their
size's.
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
Array!(Tuple!(string, ulong)) coSizeDirList () {
string FFs = "C:\\Temp\\TEST1\\BACKUP";
int SizeDir = 10;
ulong subdirTotal;
ulong subdirTotalGB;
auto dFiles = Array!(Tuple!(string)) (dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir).map!(a =>
tuple(a.name)));
foreach (d; dFiles) {
auto SdFiles = Array!(Tuple!(ulong)) (dirEntries(d[0],
SpanMode.depth).map!(a => tuple(a.size)));
foreach(f; parallel(SdFiles, 1)) { subdirTotal +=
f.fold!((a, b) => a + b); }
subdirTotalGB = (subdirTotal/1024/1024);
if (subdirTotalGB > SizeDir) { auto Subdata =
Array!(Tuple!(string, ulong))(dFiles ~ subdirTotalGB); }
subdirTotal = 0;
}
return Subdata;
}
void main () {
writeln (coSizeDirList[]);
}
Hi,
Few updates,
If i change the function to main i am able to print the required
output, but if i change the main to function and call this
function from another main then i am not able to return the
result from the function.
Updated Code:
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
Array!(Tuple!(string, ulong)) coSizeDirList () {
//void main () {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!(Tuple!(ulong)) Subdata;
auto dFiles = Array!(Tuple!(string)) (dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir).map!(a => tuple(a.name)));
foreach (d; dFiles[]) {
auto SdFiles = Array!(Tuple!(ulong)) (dirEntries(d[0],
SpanMode.depth).map!(a => tuple(a.size)));
foreach(f; SdFiles[]) { subdirTotal += f.fold!((a, b) => a +
b); }
subdirTotalGB = (subdirTotal/1024/1024);
if (subdirTotalGB > SizeDir) { Subdata
~= subdirTotalGB; }
subdirTotal = 0;
}
//writeln(dFiles);
//writeln(Subdata);
return dFiles[];
return Subdata[];
}
void main () {
writeln (coSizeDirList[]);
}
From,
Vino.B
Re: Container Array
On Thursday, 7 September 2017 at 14:26:08 UTC, Ali Çehreli wrote:
On 09/07/2017 03:56 AM, Vino.B wrote:
writeln(coCleanFiles);
Access the elements by taking a slice of the container:
writeln(coCleanFiles[]);
Ali
Hi Ali,
Thank you very much, was ablee to resolve this issue and now
facing a new issue as the below code is not working as expected.
The below code has to list the folders and their size's.
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
Array!(Tuple!(string, ulong)) coSizeDirList () {
string FFs = "C:\\Temp\\TEST1\\BACKUP";
int SizeDir = 10;
ulong subdirTotal;
ulong subdirTotalGB;
auto dFiles = Array!(Tuple!(string)) (dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir).map!(a => tuple(a.name)));
foreach (d; dFiles) {
auto SdFiles = Array!(Tuple!(ulong)) (dirEntries(d[0],
SpanMode.depth).map!(a => tuple(a.size)));
foreach(f; parallel(SdFiles, 1)) { subdirTotal += f.fold!((a,
b) => a + b); }
subdirTotalGB = (subdirTotal/1024/1024);
if (subdirTotalGB > SizeDir) { auto Subdata =
Array!(Tuple!(string, ulong))(dFiles ~ subdirTotalGB); }
subdirTotal = 0;
}
return Subdata;
}
void main () {
writeln (coSizeDirList[]);
}
Re: Container Array
On Wednesday, 6 September 2017 at 16:41:06 UTC, Vino.B wrote:
HI All,
Can some one provide me a example of how to use the
std.container.array for the below code.
import std.algorithm: filter, map;
import std.file: SpanMode, dirEntries, isDir;
import std.stdio: writeln;
import std.typecons: tuple;
import std.array: array;
void main () {
string[] Filesys = ["C:\\Temp\\TEST1\\BACKUP",
"C:\\Temp\\TEST2\\EXPORT"];
foreach(FFs; Filesys) {
auto dFiles = dirEntries("C:\\Temp\\TEST1\\BACKUP",
SpanMode.shallow).filter!(a => a.isDir).map!(a => tuple(a.name,
a.size));
foreach(d; dFiles)
writeln(d[0], "\t", d[1]);
}
}
From,
Vino.B
Hi,
I tried a small code using container array, and the output i
get form the code is a below, so can one help me on this issue.
Program:
import std.algorithm: filter, map;
import std.file: SpanMode, dirEntries, isDir;
import std.stdio: writeln;
import std.typecons: tuple, Tuple;
import std.container;
Array!(Tuple!(string, string)) coCleanFiles() {
auto dFiles = make!Array(dirEntries("C:\\Temp\\TEST1\\BACKUP",
SpanMode.shallow).filter!(a => a.isFile).map!(a => tuple(a.name,
a.timeCreated.toSimpleString[0 .. 20])));
return dFiles;
}
void main () {
writeln(coCleanFiles);
}
Output:
Array!(Tuple!(string, string))(RefCounted!(Payload,
cast(RefCountedAutoInitialize)0)(RefCountedStore(62D818)))
From,
Vino.B
Re: Performance Issue
On Wednesday, 6 September 2017 at 18:44:26 UTC, Azi Hassan wrote:
On Wednesday, 6 September 2017 at 18:21:44 UTC, Azi Hassan
wrote:
I tried to create a similar file structure on my Linux
machine. Here's the result of ls -R TEST1:
TEST1:
BACKUP
...
Upon further inspection it looks like I messed up the output.
[31460] - Array 1 for folder 1(all files in Folder 1) of the
FS C:\\Temp\\TEST1\\BACKUP
[138] - Array 2 for folder 2(all files in Folder 2) of
the FS C:\\Temp\\TEST1\\BACKUP
[2277663, 2277663] - Array 3 for folder 1(all files in Folder
1) of the FS C:\\Temp\\TEST2\\EXPOR
[31460] - Array 4 for folder 2(all files in Folder 2) the FS
C:\\Temp\\TEST2\\EXPORT
What files do these sizes correspond to ? Shouldn't there be
two elements in the first array because
C:\Temp\TEST1\BACKUP\FOLDER1 contains two files ?
Hi Azi,
Was able to implement "fold", below is the update code,
regarding container array, I have almost completed my
program(Release 1), so it is not a good idea to convert the
program from standard array to container array at this point.
Some staring tomorrow i would be working on(Release 2) where in
this release i plan to make the above changes. I have not reached
my study on container array, so can you help me on how to
implement the container array for the below code.
Note : I have raised another thread "Container Array" asking the
same.
string[][] coSizeDirList (string FFs, int SizeDir) {
ulong subdirTotal = 0;
ulong subdirTotalGB;
auto Subdata = appender!(string[][]); Subdata.reserve(100);
auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a =>
a.isDir).map!(a => tuple(a.name, a.size)).array;
foreach (d; dFiles) {
auto SdFiles = dirEntries(join(["?\\", d[0]]),
SpanMode.depth).map!(a => tuple(a.size)).array;
foreach(f; parallel(SdFiles, 1)) { subdirTotal += f.fold!((a,
b) => a + b); }
subdirTotalGB = (subdirTotal/1024/1024);
if (subdirTotalGB > SizeDir) { Subdata ~= [d[0],
to!string(subdirTotalGB)]; }
subdirTotal = 0;
}
return Subdata.data;
}
Note To All :
I am basically a Admin guy, so started learning D a few months
ago and found it very interesting, hence i raise so many
question, so request you to adjust with me for a while.
From,
Vino.B
Container Array
HI All,
Can some one provide me a example of how to use the
std.container.array for the below code.
import std.algorithm: filter, map;
import std.file: SpanMode, dirEntries, isDir;
import std.stdio: writeln;
import std.typecons: tuple;
import std.array: array;
void main () {
string[] Filesys = ["C:\\Temp\\TEST1\\BACKUP",
"C:\\Temp\\TEST2\\EXPORT"];
foreach(FFs; Filesys) {
auto dFiles = dirEntries("C:\\Temp\\TEST1\\BACKUP",
SpanMode.shallow).filter!(a => a.isDir).map!(a => tuple(a.name,
a.size));
foreach(d; dFiles)
writeln(d[0], "\t", d[1]);
}
}
From,
Vino.B
Re: Performance Issue
On Wednesday, 6 September 2017 at 14:38:39 UTC, Vino.B wrote:
On Wednesday, 6 September 2017 at 10:58:25 UTC, Azi Hassan
wrote:
[...]
Hi Azi,
Your are correct, i tried to implement the fold in a separate
small program as below, but not able to get the the required
output, when you execute the below program the output you get
is as below
Output:
[31460]
[31460, 138]
[31460, 138, 2277663]
[31460, 138, 2277663, 2277663]
[31460, 138, 2277663, 2277663, 31460]
Setup:
C:\\Temp\\TEST1\\BACKUP : This has 2 folder and 2 files in each
folder
C:\\Temp\\TEST2\\EXPORT : This has 2 folder and 2 files in one
folder and 1 file in another folder
Total files : 5
Required output:
[31460, 138] - Array 1 for the FS C:\\Temp\\TEST1\\BACKUP
[2277663, 2277663, 31460] - Array 2 for the
C:\\Temp\\TEST2\\EXPORT
import std.algorithm: filter, map, fold;
import std.parallelism: parallel;
import std.file: SpanMode, dirEntries, isDir;
import std.stdio: writeln;
import std.typecons: tuple;
import std.path: globMatch;
import std.array;
void main () {
ulong[] Alternate;
string[] Filesys = ["C:\\Temp\\TEST1\\BACKUP",
"C:\\Temp\\TEST2\\EXPORT"];
foreach(FFs; Filesys)
{
auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a =>
a.isDir).map!(a => tuple(a.name, a.size)).array;
foreach (d; dFiles) {
auto SdFiles = dirEntries(join(["?\\", d[0]]),
SpanMode.depth).map!(a => tuple(a.size)).array;
foreach (f;
parallel(SdFiles,1)) {
Alternate ~=
f[0]; writeln(Alternate);
}
}
}
}
From,
Vino.B
Hi Azi,
The required out is like below
[31460] - Array 1 for folder 1(all files in Folder 1) of the FS
C:\\Temp\\TEST1\\BACKUP
[138] - Array 2 for folder 2(all files in Folder 2) of the
FS C:\\Temp\\TEST1\\BACKUP
[2277663, 2277663] - Array 3 for folder 1(all files in Folder 1)
of the FS C:\\Temp\\TEST2\\EXPOR
[31460] - Array 4 for folder 2(all files in Folder 2) the FS
C:\\Temp\\TEST2\\EXPORT
Re: Performance Issue
On Wednesday, 6 September 2017 at 10:58:25 UTC, Azi Hassan wrote:
On Wednesday, 6 September 2017 at 08:10:35 UTC, Vino.B wrote:
in the next line of the code i say to list only folders that
are greater than 10 Mb but this now is listing all folder
(folder whose size is less than 10 MB are getting listed, not
sure why.
Is the size in GB ? If so, then subdirTotalGB =
(subdirTotal/1024/1024); needs to become subdirTotalGB =
(subdirTotal/1024/1024/1024); for it to take effect. But do
correct me if I'm wrong, I still haven't had my morning coffee.
Hi Azi,
Your are correct, i tried to implement the fold in a separate
small program as below, but not able to get the the required
output, when you execute the below program the output you get is
as below
Output:
[31460]
[31460, 138]
[31460, 138, 2277663]
[31460, 138, 2277663, 2277663]
[31460, 138, 2277663, 2277663, 31460]
Setup:
C:\\Temp\\TEST1\\BACKUP : This has 2 folder and 2 files in each
folder
C:\\Temp\\TEST2\\EXPORT : This has 2 folder and 2 files in one
folder and 1 file in another folder
Total files : 5
Required output:
[31460, 138] - Array 1 for the FS C:\\Temp\\TEST1\\BACKUP
[2277663, 2277663, 31460] - Array 2 for the
C:\\Temp\\TEST2\\EXPORT
import std.algorithm: filter, map, fold;
import std.parallelism: parallel;
import std.file: SpanMode, dirEntries, isDir;
import std.stdio: writeln;
import std.typecons: tuple;
import std.path: globMatch;
import std.array;
void main () {
ulong[] Alternate;
string[] Filesys = ["C:\\Temp\\TEST1\\BACKUP",
"C:\\Temp\\TEST2\\EXPORT"];
foreach(FFs; Filesys)
{
auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a =>
a.isDir).map!(a => tuple(a.name, a.size)).array;
foreach (d; dFiles) {
auto SdFiles = dirEntries(join(["?\\", d[0]]),
SpanMode.depth).map!(a => tuple(a.size)).array;
foreach (f;
parallel(SdFiles,1)) {
Alternate ~=
f[0]; writeln(Alternate);
}
}
}
}
From,
Vino.B
Re: Performance Issue
On Tuesday, 5 September 2017 at 10:28:28 UTC, Stefan Koch wrote:
On Tuesday, 5 September 2017 at 09:44:09 UTC, Vino.B wrote:
Hi,
The below code is consume more memory and slower can you
provide your suggestion on how to over come these issues.
[...]
Much slower then ?
Hi,
This code is used to get the size of folders on a NetApp NAS
Filesystem , so the NetApp have their own tool to perform such
task which is faster than this code, the difference is about
15-20 mins. While going through this website i was able to findd
that we can use the "fold" from std.algorithm.iteration which
would be faster that use the normal "+=", so tried replacing the
line "{ subdirTotal += f[0]; }" with { subdirTotal = f[0].fold!(
(a, b) => a + b); }, and this produces the required output+
additional output , in the next line of the code i say to list
only folders that are greater than 10 Mb but this now is listing
all folder (folder whose size is less than 10 MB are getting
listed, not sure why.
Program:
string[][] coSizeDirList (string FFs, int SizeDir) {
ulong subdirTotal = 0;
ulong subdirTotalGB;
auto Subdata = appender!(string[][]); Subdata.reserve(100);
auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a =>
a.isDir && !globMatch(a.baseName, "*DND*")).map!(a =>
tuple(a.name, a.size)).array;
foreach (d; dFiles) {
auto SdFiles = dirEntries(join(["?\\", d[0]]),
SpanMode.depth).map!(a => tuple(a.size)).array;
foreach (f; parallel(SdFiles,1))
{ subdirTotal = f[0].fold!( (a, b) => a
+ b); }
subdirTotalGB =
(subdirTotal/1024/1024);
if (subdirTotalGB > SizeDir) { Subdata ~= [d[0],
to!string(subdirTotalGB)]; }
subdirTotal = 0;
}
return Subdata.data;
}
OutPut
C:\Temp\TEAM1\dir1 - > Sieze greater than 10MB
C:\Temp\TEAM1\dir2 -> Size lesser than 10MB.
From,
Vino.B
Re: Using closure causes GC allocation
On Monday, 4 September 2017 at 14:42:45 UTC, Azi Hassan wrote:
On Monday, 4 September 2017 at 05:45:18 UTC, Vino.B wrote:
In order to resolve the issue "Using closure causes GC
allocation" it was stated that we need to use delegates
Alternatively you can drop the functional style and use a
foreach loop that doesn't require delegates, but you'd still
need the GC to store the result in an array. And even then you
could still use Array (from std.container).
Hi All,
Was able to resolve this issue, thank you for your help, below
is the changes that i did to resolve the issue.
import std.stdio: File,writeln;
import std.datetime.systime: Clock, days, SysTime;
import std.file: SpanMode, dirEntries, exists, isFile, mkdir,
remove;
import std.typecons: tuple, Tuple;
import std.algorithm: filter, map, each;
import std.array: array;
Tuple!(string)[] logClean (string[] Lglst, int LogAge) {
if (!Lglst[0].exists) { mkdir(Lglst[0]); }
auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a
=a.exists && a.isFile && a.timeCreated < dtLogAge).map!(a
=tuple(a.name)).array;
dFiles.each!(a =a[0].remove);
return dFiles;
}
SysTime dtLogAge () {
int LogAge = mParams[1];
auto ct2 = Clock.currTime();
auto st2 = ct2 + days(-LogAge);
return st2;
}
void main () {
string[] LogDir =
["C:\\Users\\admin\\Desktop\\Current\\Script\\D\\Logs"];
logClean(LogDir);
}
"mParams" is another function that reads the value from the
configuration file and returns the value for the LogAge as
defined in the configuration file.
From,
Vino.B
Performance Issue
Hi,
The below code is consume more memory and slower can you provide
your suggestion on how to over come these issues.
string[][] csizeDirList (string FFs, int SizeDir) {
ulong subdirTotal = 0;
ulong subdirTotalGB;
auto Subdata = appender!(string[][]);
auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a =>
a.isDir && !globMatch(a.baseName, "*DND*")).map!(a =>
tuple(a.name, a.size)).array;
foreach (d; dFiles) {
auto SdFiles = dirEntries(join(["?\\", d[0]]),
SpanMode.depth).map!(a => tuple(a.size)).array;
foreach (f; parallel(SdFiles,1))
{ subdirTotal += f[0]; }
subdirTotalGB =
(subdirTotal/1024/1024);
if (subdirTotalGB > SizeDir) { Subdata ~= [d[0],
to!string(subdirTotalGB)]; }
subdirTotal = 0;
}
return Subdata.data;
}
From,
Vino.B
Re: Returning multiple values from a function
On Monday, 4 September 2017 at 07:40:23 UTC, crimaniak wrote:
On Monday, 4 September 2017 at 07:27:12 UTC, Vino.B wrote:
Hi,
Can you help me in how to return multiple values from a
function, the below code is throwing an error as below
import std.stdio: writeln;
import std.typecons: tuple, Tuple;
Tuple!(int, string[]) Params () {
return tuple(1, ["C:\\Temp\\TEAM1\\BACKUP",
"C:\\Temp\\TEAM2\\ARCHIVE"]);
}
void main (){
Params.writeln;
}
Hi,
Thank you very much, i have used your idea and was able to
resolve, and i need one more favor. the below code outputs the
value but i need the name of the variable + value as below.
Output :
1
2
["C:\\Temp\\TEAM1\\BACKUP", "C:\\Temp\\TEAM2\\ARCHIVE"]
Required Output:
Test1 = 1
Test2 = 2
Path = ["C:\\Temp\\TEAM1\\BACKUP", "C:\\Temp\\TEAM2\\ARCHIVE"]
Program:
import std.stdio: writeln;
import std.typecons: tuple, Tuple;
import std.array: appender;
Tuple!(int,int, string[]) Params () {
int Test1;
int Test2;
string[] File1;
string[] File2;
auto Path = appender!(string[]);
Test1 = 1;
Test2 = 2;
File1 = ["C:\\Temp\\TEAM1\\BACKUP"];
File2 = ["C:\\Temp\\TEAM2\\ARCHIVE"];
Path ~= File1;
Path ~= File2;
return tuple (Test1, Test2, Path.data);
}
void main (){
writeln(Params[0]);
writeln(Params[1]);
writeln(Params[2]);
}
return tuple (Test1, Test1Test2, Path.data);
}
void main (){
writeln(Params[0]);
writeln(Params[1]);
}
From,
Vino.B
Returning multiple values from a function
Hi,
Can you help me in how to return multiple values from a
function, the below code is throwing an error as below
Program:
import std.stdio: writeln;
import std.typecons: tuple, Tuple;
Tuple!(int, string[]) Params () {
int Test1;
string[] Path;
Test1 = 1;
Path = ["C:\\Temp\\TEAM1\\BACKUP", "C:\\Temp\\TEAM2\\ARCHIVE"];
return Test1;
return Path;
}
void main (){
int Test1;
string[] Path;
Params;
writeln(Test1);
writeln(Path);
}
Error:
TEx1.d(9): Error: cannot implicitly convert expression Test1 of
type int to Tuple!(int, string[])
TEx1.d(10): Error: cannot implicitly convert expression Path of
type string[] to Tuple!(int, string[])
From,
Vino.B
Re: Using closure causes GC allocation
On Saturday, 2 September 2017 at 20:54:03 UTC, Vino.B wrote:
On Saturday, 2 September 2017 at 20:10:58 UTC, Moritz Maxeiner
wrote:
On Saturday, 2 September 2017 at 18:59:30 UTC, Vino.B wrote:
[...]
Cannot reproduce under Linux with dmd 2.076.0 (with commented
out Windows-only check). I'll try to see what happens on
Windows once I have a VM setup.
[...]
You changed the type of dFiles, which you return from
cleanFiles, without changing the return type of cleanFiles.
Change the return type of cleanFiles to the type the compiler
error above tells you it should be (`Tuple!(string, string)[]`
instead of `string[][]`), or let the compiler infer it via
auto (`auto cleanFiles(...`).
Hi,
Thank you very much, was able to resolve the second code issue
by changing the return type of the function.
Hi,
In order to resolve the issue "Using closure causes GC
allocation" it was stated that we need to use delegates, can you
please help me on how to as i have not gone that much far in D
programming.
import std.stdio: File,writeln;
import std.datetime.systime: Clock, days, SysTime;
import std.file: SpanMode, dirEntries, exists, isFile, mkdir,
remove;
import std.typecons: tuple, Tuple;
import std.algorithm: filter, map, each;
import std.array: array;
Tuple!(string)[] logClean (string[] Lglst, int LogAge) {
if (!Lglst[0].exists) { mkdir(Lglst[0]); }
auto ct1 = Clock.currTime();
auto st1 = ct1 + days(-LogAge);
auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a
=> a.exists && a.isFile && a.timeCreated < st1).map!(a =>
tuple(a.name)).array;
dFiles.each!(a => a[0].remove);
return dFiles;
}
void main () {
string[] LogDir =
["C:\\Users\\bheev1\\Desktop\\Current\\Script\\D\\Logs"];
int LogAge = 1;
logClean(LogDir,LogAge);
}
From,
Vino.B
Re: Help required on Array appender
On Saturday, 2 September 2017 at 22:39:33 UTC, Nicholas Wilson
wrote:
On Saturday, 2 September 2017 at 21:11:17 UTC, Vino.B wrote:
On Saturday, 2 September 2017 at 15:47:31 UTC, Vino.B wrote:
On Saturday, 2 September 2017 at 12:54:48 UTC, Nicholas
Wilson wrote:
[...]
Hi,
[...]
Hi,
Was able to resolve the above issue, but again getting the
same for other lines such as below when i tried to add the
appender.
auto CleanDirlst = appender([]);
CleanDirlst ~= PVStore[1][i].to!string.split(",");
Error: cannot implicitly convert expression appender([]) of
type Appender!(void[]) to string[].
You want `Appender!(string[])`
Hi,
Thank you very much, was able to resolve the issue.
Re: Help required on Array appender
On Saturday, 2 September 2017 at 15:47:31 UTC, Vino.B wrote:
On Saturday, 2 September 2017 at 12:54:48 UTC, Nicholas Wilson
wrote:
[...]
Hi,
[...]
Hi,
Was able to resolve the above issue, but again getting the same
for other lines such as below when i tried to add the appender.
auto CleanDirlst = appender([]);
CleanDirlst ~= PVStore[1][i].to!string.split(",");
Error: cannot implicitly convert expression appender([]) of type
Appender!(void[]) to string[].
Re: Using closure causes GC allocation
On Saturday, 2 September 2017 at 20:10:58 UTC, Moritz Maxeiner wrote: On Saturday, 2 September 2017 at 18:59:30 UTC, Vino.B wrote: [...] Cannot reproduce under Linux with dmd 2.076.0 (with commented out Windows-only check). I'll try to see what happens on Windows once I have a VM setup. [...] You changed the type of dFiles, which you return from cleanFiles, without changing the return type of cleanFiles. Change the return type of cleanFiles to the type the compiler error above tells you it should be (`Tuple!(string, string)[]` instead of `string[][]`), or let the compiler infer it via auto (`auto cleanFiles(...`). Hi, Thank you very much, was able to resolve the second code issue by changing the return type of the function.
Re: Using closure causes GC allocation
On Saturday, 2 September 2017 at 18:32:55 UTC, Moritz Maxeiner
wrote:
On Saturday, 2 September 2017 at 18:08:19 UTC, vino.b wrote:
On Saturday, 2 September 2017 at 18:02:06 UTC, Moritz Maxeiner
wrote:
On Saturday, 2 September 2017 at 17:43:08 UTC, Vino.B wrote:
[...]
Line 25 happens because of `[a.name]`. You request a new
array: the memory for this has to be allocated (the reason
why the compiler says "may" is because sometimes, e.g. if the
array literal itself contains only literals, the allocations
needn't happen at runtime and no GC call is necessary). Since
you don't actually use the array, get rid of it:
[...]
Hi,
Thank you for your help and the DMD version that i am using
is DMD 2.076.0 and yes I am on windows.
Please post a compilable, minimal example including how that
function gets called that yields you that compiler output.
Hi,
Please find the example code below,
import std.stdio: File,writeln;
import std.datetime.systime: Clock, days, SysTime;
import std.file: SpanMode, dirEntries, exists, isFile, mkdir,
remove;
import std.typecons: tuple;
import std.algorithm: filter, map, each;
import std.array: array;
void logClean (string[] Lglst, int LogAge) {
if (!Lglst[0].exists) { mkdir(Lglst[0]); }
auto ct1 = Clock.currTime();
auto st1 = ct1 + days(-LogAge);
auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a
=> a.exists && a.isFile && a.timeCreated < st1).map!(a =>
tuple(a.name)).array;
dFiles.each!(a => a[0].remove);
}
void main () {
string[] LogDir =
["C:\\Users\\bheev1\\Desktop\\Current\\Script\\D\\Logs"];
int LogAge = 1;
logClean(LogDir,LogAge);
}
Another similar issue :
I removed the [a.name] and the issue in line 25 has resolved,
but for another function i am getting the same error
string[][] cleanFiles(string FFs, string Step) {
auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a =>
a.isFile).map!(a =>[a.name , a.timeCreated.toSimpleString[0 ..
20]]).array; -> Issue in this line
if (Step == "run")
dFiles.each!(a => a[0].remove);
return dFiles;
}
if the replace the line in error as below then i am getting the
error "Error: cannot implicitly convert expression dFiles of type
Tuple!(string, string)[] to string[][]"
auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a =>
a.isFile).map!(a => tuple(a.name, a.timeCreated.toSimpleString[0
.. 20])).array;
Re: Using closure causes GC allocation
On Saturday, 2 September 2017 at 18:02:06 UTC, Moritz Maxeiner wrote: On Saturday, 2 September 2017 at 17:43:08 UTC, Vino.B wrote: [...] Line 25 happens because of `[a.name]`. You request a new array: the memory for this has to be allocated (the reason why the compiler says "may" is because sometimes, e.g. if the array literal itself contains only literals, the allocations needn't happen at runtime and no GC call is necessary). Since you don't actually use the array, get rid of it: [...] Hi, Thank you for your help and the DMD version that i am using is DMD 2.076.0 and yes I am on windows. From, Vino.B
Using closure causes GC allocation
Hi All,
Request your help on how to solve the issue in the below code
as when i execute the program with -vgc it state as below:
NewTD.d(21): vgc: using closure causes GC allocation
NewTD.d(25): vgc: array literal may cause GC allocation
void logClean (string[] Lglst, int LogAge) { //Line 21
if (!Lglst[0].exists) { mkdir(Lglst[0]); }
auto ct1 = Clock.currTime();
auto st1 = ct1 + days(-LogAge);
auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a
=> a.exists && a.isFile && a.timeCreated < st1).map!(a
=>[a.name]).array; // Line 25
dFiles.each!(f => f[0].remove);
}
From,
Vino.B
Re: Help required on Array appender
On Saturday, 2 September 2017 at 12:54:48 UTC, Nicholas Wilson
wrote:
On Saturday, 2 September 2017 at 10:15:04 UTC, Vino.B wrote:
Hi All,
Can you please guide me how can i use array appender for the
below piece of code
string[][] cleanFiles (string FFs, string Step) {
auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a =>
a.isFile).map!(a => tuple(a.name , a.timeCreated)).array;
foreach (d; dFiles) {
if (Step == "dryrun")
{
Subdata ~= [d[0], d[1].toSimpleString[0
.. 20]];
}
else if (Step == "run")
{
remove(d[0]);
if (!d[0].exists)
Subdata ~= [d[0], d[1].toSimpleString[0 ..
20]];
}
}
return Subdata;
}
From,
Vino.B
If you're wanting to use appender just make an appender and
replace the ~= to calls to appender.put(data);
if you're trying to make it faster, consider that Step could be
a bool, your return type could be string[2][], the `if
(!d[0].exists)` is redundant since `remove` will throw if it
fails. That leaves you with
string[2][] cleanFiles(string FFs, bool dryrun)
{
auto dFiles = dirEntries(FFs, SpanMode.shallow)
.filter!(a => a.isFile)
.map!(a =>[a.name ,
a.timeCreated.toSimpleString[0 .. 20])
.array;
if (! dryrun)
dFiles.each!(f => f[0].remove);
}
Hi,
Thank you very much, and your idea help a lot, and the reason to
use appender is for both faster and performance as my program
use's many ~= function and found that using appender is the best
way when compared with the ~= e.g;(MCresult.get ~=
MCleanTaskData;)
void mCleanFiles (string[] Dirlist, File logF, File logE, string
Step) {
try {
string[][] MCtext;
string[][] MCEresult;
auto MCresult = taskPool.workerLocalStorage(MCtext);
logF.writeln("Function \t : List of the File's which are not
placed in correct Location and list those deleted");
logF.writeln("Dir. Scanned \t :", Dirlist);
logF.writeln("");
logF.writefln("%-63s %.20s", "File Name", "CreationTime");
logF.writeln("");
foreach (string Fs; parallel(Dirlist[0 .. $], 1)) {
auto FFs = Fs.strip;
auto MCleanTask = task(, FFs, Step);
MCleanTask.executeInNewThread();
auto MCleanTaskData = MCleanTask.workForce;
MCresult.get ~= MCleanTaskData;
}
foreach(i; MCresult.toRange)
logF.writefln("%(%-(%-63s %)\n%)", i.sort!((a,b) => a[0] <
b[0]).uniq);
logF.writeln("");
} catch (Exception e) { logE.writeln(e.msg); }
}
From,
Vino.B
Help required on Array appender
Hi All,
Can you please guide me how can i use array appender for the
below piece of code
string[][] cleanFiles (string FFs, string Step) {
auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a =>
a.isFile).map!(a => tuple(a.name , a.timeCreated)).array;
foreach (d; dFiles) {
if (Step == "dryrun")
{ Subdata ~= [d[0], d[1].toSimpleString[0 .. 20]];
}
else if (Step == "run") {
remove(d[0]);
if (!d[0].exists)
Subdata ~= [d[0],
d[1].toSimpleString[0 .. 20]];
}
}
return Subdata;
}
From,
Vino.B
Re: Help Required on Getopt
On Friday, 1 September 2017 at 17:23:01 UTC, Jon Degenhardt wrote: On Friday, 1 September 2017 at 13:13:39 UTC, Vino.B wrote: Hi All, When i run the below program without any arguments "D1.d -r" it is throwing error, but i need it to show the help menu [snip...] Hi Vino, To get good error message behavior you need to put the construct in a try-catch block. Then you can choose how to respond. An example here: https://github.com/eBay/tsv-utils-dlang/blob/master/tsv-append/src/tsv-append.d#L138-L194. This code prints outs the error message from the exception. In your case: "Missing value for argument -r.". But, you could also print out the help text as well. There is an example of that as well in the above code block, look for the 'if (r.helpWanted)' test. --Jon Hi, Thank you very much, that helped me to resolve the issue.
Help Required on Getopt
Hi All,
When i run the below program without any arguments "D1.d -r" it
is throwing error, but i need it to show the help menu
Program:
import std.stdio;
import std.getopt;
string r;
void main (string[] args)
{
getopt(args, std.getopt.config.caseInsensitive,
std.getopt.config.stopOnFirstNonOption, "r" , );
switch (r) {
case "Test1" : Test1; break;
case "Test2" : Test2; break;
default : writeln("Unknown operation"); Help;
}
}
void Test1 ()
{ writeln("This is Test 1"); }
void Test2 ()
{ writeln("This is Test 2"); }
void Help ()
{ writeln("This is Help Menu"); }
Execution:
Works fine if i execute the program with below options
rdmd D1.d
rdmd D1.d -r Test1
rdmd D1.d -r Test2
rdmd D1.d -r Help
but fails with below error when i execute the program as rdmd
D1.d -r
Requirement :
If i execute the program as "rdmd D1.d -r" is should show me the
"Help" Menu rather than the below error/
Output:
C:\Admin\Desktop\Current\Script\D>rdmd D1.d -r
object.Exception@C:\D\dmd2\windows\bin\..\..\src\phobos\std\getopt.d(880):
Missing value for argument -r.
0x00402493
0x004043AC
0x00402999
0x004025FE
0x004025C5
0x0040239B
0x00402236
0x0040B47F
0x0040B443
0x0040B344
0x00405E07
0x76CB336A in BaseThreadInitThunk
0x77CF9902 in RtlInitializeExceptionChain
0x77CF98D5 in RtlInitializeExceptionChain
From,
Vino.B
Re: Valid File Path
On Thursday, 31 August 2017 at 23:45:01 UTC, Jonathan M Davis wrote: On Thursday, August 31, 2017 23:23:17 Vino via Digitalmars-d-learn wrote: [...] And why would that not be valid? isValidPath and isValidFilename are quite specific about what they think are valid path/file names, and having a # in a file name is perfectly legitimate. They do have some extra restrictions for Windows, since Windows is a lot pickier about its filenames than the rest of the world, but # is not one of the characters listed as invalid: https://dlang.org/phobos/std_path.html#isValidPath https://dlang.org/phobos/std_path.html#isValidFilename It would be invalid to have # as a drive name, but it's perfectly legal in a filename or directory name. - Jonathan M Davis Hi, Thank you very much for the explanation, was able to resolve the issue.
Re: Missing array element
On Tuesday, 29 August 2017 at 18:39:03 UTC, Ali Çehreli wrote:
On 08/29/2017 11:20 AM, Vino.B wrote:
string[] a = ["test1", "test2", "test4"];
string[] b = ["test2", "test4"];
Required output: "test1"
You're looking for setDifference:
https://dlang.org/phobos/std_algorithm_setops.html#.setDifference
Ali
Hi,
I tried the setDifference but does seem to be working as expected
import std.stdio, std.array, std.algorithm;
string[] a = ["test4", "test3", "test2", "test1"];
string[] b = ["test2", "test4"];
void main ()
{
auto m = setDifference(a,b);
writeln(m);
}
Output : Required output ["test3", "test1"]
["test3", "test2", "test1"]
From,
Vino.B
Missing array element
Hi,
Can any one help me on the below program, as I need the missing
element for array "a" to be printed when compared with array "b"
Program:
import std.stdio, std.array, std.algorithm;
string[] a = ["test1", "test2", "test4"];
string[] b = ["test2", "test4"];
void main ()
{
auto m = mismatch(a,b);
writeln(m[0]);
writeln(m[1]);
}
Output:
["test1", "test2", "test4"]
["test2", "test4"]
Required output: "test1"
From,
Vino.B
Re: 2 Dimensional Array Sorting
On Sunday, 27 August 2017 at 11:53:29 UTC, Vino.B wrote: On Saturday, 26 August 2017 at 10:53:03 UTC, Vino.B wrote: [...] Hi, After analyzing a bit further was able to find that the out data before sorting is like below(4 - 2 dimensional array) hence the sorting is not working, so may i know how do i make this as a single 2 dimensional array like below [...] Hi, Thank you very much, was able to resolve the issue. From, Vino.B
Re: 2 Dimensional Array Sorting
On Saturday, 26 August 2017 at 10:53:03 UTC, Vino.B wrote: On Saturday, 26 August 2017 at 10:45:13 UTC, Vino.B wrote: On Saturday, 26 August 2017 at 10:07:53 UTC, user1234 wrote: [...] Hi, Now there is no duplicate , but the sequence is still not correct [...] Hi, If I execute the script several time's i still get the duplicate entries. From, Vino.B Hi, After analyzing a bit further was able to find that the out data before sorting is like below(4 - 2 dimensional array) hence the sorting is not working, so may i know how do i make this as a single 2 dimensional array like below Required: [["C:\\Temp\\TEAM2\\TEAM\\DIR1", "40"], ["C:\\Temp\\TEAM2\\TEAM\\DIR2", "2228"], ["C:\\Temp\\TEAM3\\EXPORT\\dir2", "61"], ["C:\\Temp\\TEAM2\\PROD_TEAM\\dir1", "35772"], ["C:\\Temp\\TEAM2\\BACKUP\\dir1", "35732"], ["C:\\Temp\\TEAM3\\BACKUP\\dir1", "71465"], ["C:\\Temp\\TEAM2\\EXPORT\\dir2", "30"]] Output: [["C:\\Temp\\TEAM2\\TEAM\\DIR1", "40"], ["C:\\Temp\\TEAM2\\TEAM\\DIR2", "2228"], ["C:\\Temp\\TEAM3\\EXPORT\\dir2", "61"]] - 2darray 1 [["C:\\Temp\\TEAM2\\PROD_TEAM\\dir1", "35772"]] - 2darray 2 [["C:\\Temp\\TEAM2\\BACKUP\\dir1", "35732"], ["C:\\Temp\\TEAM3\\BACKUP\\dir1", "71465"]] - 2darray 3 [["C:\\Temp\\TEAM2\\EXPORT\\dir2", "30"]] - 2darray 4 From, Vino.B
Re: 2 Dimensional Array Sorting
On Saturday, 26 August 2017 at 10:45:13 UTC, Vino.B wrote: On Saturday, 26 August 2017 at 10:07:53 UTC, user1234 wrote: [...] Hi, Now there is no duplicate , but the sequence is still not correct [...] Hi, If I execute the script several time's i still get the duplicate entries. From, Vino.B
Re: 2 Dimensional Array Sorting
On Saturday, 26 August 2017 at 10:07:53 UTC, user1234 wrote: On Saturday, 26 August 2017 at 09:53:44 UTC, Vino.B wrote: On Saturday, 26 August 2017 at 06:12:57 UTC, user1234 wrote: [...] Hi, I tired you logic, but doesn't seem to be working, as every time i execute the order of the file list is different as in the below program i have used the sort based on the file name. [...] Try with (a,b) => a[1].to!int < b[1].to!int as predicate. T Hi, Now there is no duplicate , but the sequence is still not correct Output: the File C:\Temp\TEAM2\PROD_TEAM\dir1 is after C:\Temp\TEAM3\EXPORT\dir2 C:\Users\admin\Desktop\Script\D>rdmd Ftest.d C:\Temp\TEAM2\BACKUP\dir1 35732 C:\Temp\TEAM2\TEAM\DIR1 40 C:\Temp\TEAM2\TEAM\DIR2 2228 C:\Temp\TEAM2\EXPORT\dir2 30 C:\Temp\TEAM3\BACKUP\dir1 71465 C:\Temp\TEAM3\EXPORT\dir2 61 C:\Temp\TEAM2\PROD_TEAM\dir1 35772 Required output C:\Temp\TEAM2\BACKUP\dir1 35732 C:\Temp\TEAM2\TEAM\DIR1 40 C:\Temp\TEAM2\TEAM\DIR2 2228 C:\Temp\TEAM2\EXPORT\dir2 30 C:\Temp\TEAM2\PROD_TEAM\dir1 35772 C:\Temp\TEAM3\BACKUP\dir1 71465 C:\Temp\TEAM3\EXPORT\dir2 61 From, Vino.B
Re: 2 Dimensional Array Sorting
On Saturday, 26 August 2017 at 06:12:57 UTC, user1234 wrote:
On Saturday, 26 August 2017 at 06:11:37 UTC, user1234 wrote:
On Saturday, 26 August 2017 at 06:01:15 UTC, Vino.B wrote:
Hi,
Can someone provide me a example of sorting 2 Dimensional
Array containing Filename and Size, and should be sorted by
Size.
From,
Vino.B
void main(string[] args)
{
import std.stdio;
import std.algorithm.sorting;
string[2][] nameAndSize = [["b", "2"], ["a", "1"]];
auto s = nameAndSize.sort!((a,b) => a[0] < b[0] && a[1] <
b[1]);
writeln(s);
}
I missed the "by Size" directive. So it's just:
void main(string[] args)
{
import std.stdio;
import std.algorithm.sorting;
string[2][] nameAndSize = [["b", "2"], ["a", "1"]];
auto s = nameAndSize.sort!((a,b) => a[1] < b[1]);
writeln(s);
}
Hi,
I tired you logic, but doesn't seem to be working, as every
time i execute the order of the file list is different as in the
below program i have used the sort based on the file name.
Program:
import std.file: dirEntries, isFile, SpanMode;
import std.stdio: writeln, writefln;
import std.algorithm: filter, map, sort;
import std.path: globMatch, baseName;
import std.array: array, replace;
import std.typecons: tuple;
import std.parallelism;
import std.conv;
int SizeDir = 10;
string[][] Subdata;
auto Dirlst = [ "C:\\Temp\\TEAM2\\TEAM",
"C:\\Temp\\TEAM2\\PROD_TEAM", "C:\\Temp\\TEAM3\\BACKUP",
"C:\\Temp\\TEAM3\\EXPORT"];
string[][] SizeDirList (string Fs)
{
ulong subdirTotal = 0;
auto unc = "?\\"~Fs;
auto dFiles = dirEntries(unc, SpanMode.shallow).filter!(a =>
a.isDir && !globMatch(a.baseName, "*DND*")).map!(a =>
tuple(a.name, a.size)).array;
foreach (d; dFiles)
{
auto SdFiles = dirEntries(d[0], SpanMode.depth).map!(a =>
tuple(a.size)).array;
foreach (f; parallel(SdFiles,1))
{ subdirTotal += f[0]; }
ulong subdirTotalGB = (subdirTotal/1024);
if (subdirTotalGB > SizeDir)
{ Subdata ~= [d[0].replace("?\\", ""),
to!string(subdirTotalGB)]; }
subdirTotal = 0;
}
return Subdata;
}
void main ()
{
foreach (string Fs; parallel(Dirlst[0 .. $],1))
{
auto SizeDirListTask = task(, Fs);
SizeDirListTask.executeInNewThread();
auto SizeDirListTaskData = SizeDirListTask.yieldForce;
auto sortedresult = SizeDirListTaskData.sort!((a,b) => a[0] <
b[0]);
foreach(i; sortedresult[0 .. $])
writefln("%-(%-63s %)", i);
}
}
Output :The sequence is not correct and there are duplicates.
C:\Users\admin\Desktop\Script\D>rdmd Ftest.d
C:\Temp\TEAM2\TEAM\DIR1 40
C:\Temp\TEAM2\TEAM\DIR2 2228
C:\Temp\TEAM2\TEAM\DIR2 2228
C:\Temp\TEAM2\PROD_TEAM\dir1
35772
C:\Temp\TEAM2\TEAM\DIR2 2228
C:\Temp\TEAM2\PROD_TEAM\dir1
35772
C:\Temp\TEAM3\BACKUP\dir1
71465
C:\Temp\TEAM3\EXPORT\dir2 61
C:\Users\admin\Desktop\Script\D>rdmd Ftest.d
C:\Temp\TEAM2\TEAM\DIR1 40
C:\Temp\TEAM2\TEAM\DIR2 2228
C:\Temp\TEAM2\PROD_TEAM\dir1
35772
C:\Temp\TEAM3\BACKUP\dir1
71465
C:\Temp\TEAM3\EXPORT\dir2 61
C:\Users\admin\Desktop\Script\D>rdmd Ftest.d
C:\Temp\TEAM2\TEAM\DIR1 40
C:\Temp\TEAM2\TEAM\DIR2 2228
C:\Temp\TEAM3\BACKUP\dir1
71465
C:\Temp\TEAM3\EXPORT\dir2 61
C:\Temp\TEAM2\PROD_TEAM\dir1
35772
Required Output :C:\Temp\TEAM2\..., C:\Temp\TEAM3...
C:\Temp\TEAM2\TEAM\DIR1 40
C:\Temp\TEAM2\TEAM\DIR2 2228
C:\Temp\TEAM2\PROD_TEAM\dir1
35772
C:\Temp\TEAM3\BACKUP\dir1
71465
C:\Temp\TEAM3\EXPORT\dir2 61
From,
Vino.B
2 Dimensional Array Sorting
Hi, Can someone provide me a example of sorting 2 Dimensional Array containing Filename and Size, and should be sorted by Size. From, Vino.B
Re: Multi dimensional array format priting
On Friday, 25 August 2017 at 17:41:31 UTC, Vino.B wrote:
On Friday, 25 August 2017 at 17:02:53 UTC, Jonathan M Davis
wrote:
On Friday, August 25, 2017 16:45:16 Vino.B via
Digitalmars-d-learn wrote:
Hi,
Request your help on the below issue,
Issue : While appending data to a array the data is getting
duplicated.
Program:
import std.file: dirEntries, isFile, SpanMode;
import std.stdio: writeln, writefln;
import std.algorithm: filter, map;
import std.array: array;
import std.typecons: tuple;
string[] Subdata;
void main ()
{
auto dFiles = dirEntries("C:\\Temp\\TEAM",
SpanMode.shallow).filter!(a => a.isFile).map!(a =>
tuple(a.name ,
a.timeCreated)).array;
foreach (d; dFiles)
{
Subdata ~= d[0];
Subdata ~= d[1].toSimpleString;
writeln(Subdata);
}
}
Output:
["C:\\Temp\\TEAM\\test1.pdf", "2017-Aug-24 18:23:00.8946851"]
-
duplicate line
["C:\\Temp\\TEAM\\test1.pdf", "2017-Aug-24 18:23:00.8946851",
"C:\\TempTEAM\\test5.xlsx", "2017-Aug-25 23:38:14.486421"]
You keep printing out the entire array on every iteration of
the loop, so of course, you're going to see stuff output
multiple times. If you did something like
import std.stdio;
void main()
{
int[] arr;
foreach(i; 0 .. 5)
{
arr ~= i * 10;
writeln(arr);
}
}
then you'd get the output
[0]
[0, 10]
[0, 10, 20]
[0, 10, 20, 30]
[0, 10, 20, 30, 40]
whereas if you did
import std.stdio;
void main()
{
int[] arr;
foreach(i; 0 .. 5)
arr ~= i * 10;
writeln(arr);
}
then you'd just get
[0, 10, 20, 30, 40]
- Jonathan M Davis
Hi All,
Thank you very much, that was my mistake. The main idea is to
implement parallelism and now i get only single data as there
are 2 files in each of the folders, but it is listing only 1
per folder.
import std.file: dirEntries, isFile, SpanMode;
import std.stdio: writeln, writefln;
import std.algorithm: filter, map;
import std.array: array;
import std.typecons: tuple;
import std.parallelism;
string[] Subdata;
auto Dirlst = [ "C:\\Temp\\TEAM", "C:\\Temp\\PROD_TEAM"];
string[] Test (string Fs)
{
auto dFiles = dirEntries(Fs, SpanMode.shallow).filter!(a =>
a.isFile).map!(a => tuple(a.name , a.timeCreated)).array;
foreach (d; dFiles)
{ Subdata ~= d[0]; Subdata ~= d[1].toSimpleString; }
return Subdata;
}
void main ()
{
foreach (string Fs; Dirlst[0 .. $])
{
auto TestTask = task(, Fs);
TestTask.executeInNewThread();
auto TestTaskData = TestTask.yieldForce;
writefln("%-63s %.20s", TestTaskData[0], TestTaskData[1]);
}
}
Output:
C:\Temp\TEAM\Test.pdf
2017-Aug-24 18:23:00
C:\Temp\\PROD_TEAM\DND1.pdf
2017-Aug-25 23:38:04
The folder C:\Temp\TEAM contains 2 files and folder
C:\Temp\\PROD_TEAM contain 4 files but it display only 1 file
per folder.
Hi,
I was able to find the solution, thank you very much, please let
me know if there are any good logic than below,
import std.file: dirEntries, isFile, SpanMode;
import std.stdio: writeln, writefln;
import std.algorithm: filter, map;
import std.array: array;
import std.typecons: tuple;
import std.parallelism;
string[][] Subdata;
auto Dirlst = [ "C:\\Temp\\TEAM" ];
string[][] CleanFiles (string Fs)
{
auto dFiles = dirEntries(Fs, SpanMode.shallow).filter!(a =>
a.isFile).map!(a => tuple(a.name , a.timeCreated)).array;
foreach (d; dFiles)
{
Subdata ~= [d[0], d[1].toSimpleString[0 .. 20]];
}
return Subdata;
}
void main ()
{
foreach (string Fs; Dirlst[0 .. $])
{
auto MCleanTask = task(, Fs);
MCleanTask.executeInNewThread();
auto MCleanTaskData = MCleanTask.yieldForce;
foreach(i; MCleanTaskData[0 .. $])
writefln("%-(%-63s %)", i);
}
}
From,
Vino.B
Re: Appending data to array results in duplicate's.
On Friday, 25 August 2017 at 17:02:53 UTC, Jonathan M Davis wrote:
On Friday, August 25, 2017 16:45:16 Vino.B via
Digitalmars-d-learn wrote:
Hi,
Request your help on the below issue,
Issue : While appending data to a array the data is getting
duplicated.
Program:
import std.file: dirEntries, isFile, SpanMode;
import std.stdio: writeln, writefln;
import std.algorithm: filter, map;
import std.array: array;
import std.typecons: tuple;
string[] Subdata;
void main ()
{
auto dFiles = dirEntries("C:\\Temp\\TEAM",
SpanMode.shallow).filter!(a => a.isFile).map!(a =>
tuple(a.name ,
a.timeCreated)).array;
foreach (d; dFiles)
{
Subdata ~= d[0];
Subdata ~= d[1].toSimpleString;
writeln(Subdata);
}
}
Output:
["C:\\Temp\\TEAM\\test1.pdf", "2017-Aug-24 18:23:00.8946851"]
-
duplicate line
["C:\\Temp\\TEAM\\test1.pdf", "2017-Aug-24 18:23:00.8946851",
"C:\\TempTEAM\\test5.xlsx", "2017-Aug-25 23:38:14.486421"]
You keep printing out the entire array on every iteration of
the loop, so of course, you're going to see stuff output
multiple times. If you did something like
import std.stdio;
void main()
{
int[] arr;
foreach(i; 0 .. 5)
{
arr ~= i * 10;
writeln(arr);
}
}
then you'd get the output
[0]
[0, 10]
[0, 10, 20]
[0, 10, 20, 30]
[0, 10, 20, 30, 40]
whereas if you did
import std.stdio;
void main()
{
int[] arr;
foreach(i; 0 .. 5)
arr ~= i * 10;
writeln(arr);
}
then you'd just get
[0, 10, 20, 30, 40]
- Jonathan M Davis
Hi All,
Thank you very much, that was my mistake. The main idea is to
implement parallelism and now i get only single data as there are
2 files in each of the folders, but it is listing only 1 per
folder.
import std.file: dirEntries, isFile, SpanMode;
import std.stdio: writeln, writefln;
import std.algorithm: filter, map;
import std.array: array;
import std.typecons: tuple;
import std.parallelism;
string[] Subdata;
auto Dirlst = [ "C:\\Temp\\TEAM", "C:\\Temp\\PROD_TEAM"];
string[] Test (string Fs)
{
auto dFiles = dirEntries(Fs, SpanMode.shallow).filter!(a =>
a.isFile).map!(a => tuple(a.name , a.timeCreated)).array;
foreach (d; dFiles)
{ Subdata ~= d[0]; Subdata ~= d[1].toSimpleString; }
return Subdata;
}
void main ()
{
foreach (string Fs; Dirlst[0 .. $])
{
auto TestTask = task(, Fs);
TestTask.executeInNewThread();
auto TestTaskData = TestTask.yieldForce;
writefln("%-63s %.20s", TestTaskData[0], TestTaskData[1]);
}
}
Output:
C:\Temp\TEAM\Test.pdf
2017-Aug-24 18:23:00
C:\Temp\\PROD_TEAM\DND1.pdf
2017-Aug-25 23:38:04
The folder C:\Temp\TEAM contains 2 files and folder
C:\Temp\\PROD_TEAM contain 4 files but it display only 1 file per
folder.
Appending data to array results in duplicate's.
Hi,
Request your help on the below issue,
Issue : While appending data to a array the data is getting
duplicated.
Program:
import std.file: dirEntries, isFile, SpanMode;
import std.stdio: writeln, writefln;
import std.algorithm: filter, map;
import std.array: array;
import std.typecons: tuple;
string[] Subdata;
void main ()
{
auto dFiles = dirEntries("C:\\Temp\\TEAM",
SpanMode.shallow).filter!(a => a.isFile).map!(a => tuple(a.name ,
a.timeCreated)).array;
foreach (d; dFiles)
{
Subdata ~= d[0];
Subdata ~= d[1].toSimpleString;
writeln(Subdata);
}
}
Output:
["C:\\Temp\\TEAM\\test1.pdf", "2017-Aug-24 18:23:00.8946851"] -
duplicate line
["C:\\Temp\\TEAM\\test1.pdf", "2017-Aug-24 18:23:00.8946851",
"C:\\TempTEAM\\test5.xlsx", "2017-Aug-25 23:38:14.486421"]
From,
Vino.B
Re: Long File path Exception:The system cannot find the path specified
On Friday, 25 August 2017 at 09:08:44 UTC, zabruk70 wrote:
On Thursday, 24 August 2017 at 18:02:24 UTC, vino wrote:
Thanks for your support, was able to resolve this issue.
Hello.
IMHO, it will be better, if you will share your solution for
other peoples :)
Hi,
Please find the solution below, basically i converted the path
to UNC path as below
Solution:
auto unc = "?\\"~i;
auto dFiles = dirEntries(unc, SpanMode.shallow).filter!(a =>
a.isDir && !globMatch(a.baseName, "*DND*")).array;
Program:
import std.file: dirEntries, isFile, SpanMode, remove,
rmdirRecurse, exists, mkdir;
import std.stdio: writeln, writefln, File;
import std.algorithm: filter;
import std.array: array;
import std.path: globMatch, baseName;
/**/
/* Global Valiables */
/**/
int SizeDir = 10;
/**/
/* Folder Lists */
/**/
auto SizeDirlst = [ "N:\\PROD_TEAM", "P:\\TEAM" ];
/**/
/* Function : Size of Non DND Dir List*/
/**/
void SizeDirList (string[] SzDNDlst)
{
ulong subdirTotal = 0;
foreach (string i; SzDNDlst[0 .. $])
{
auto unc = "?\\"~i;
auto dFiles = dirEntries(unc, SpanMode.shallow).filter!(a =>
a.isDir && !globMatch(a.baseName, "*DND*")).array;
foreach (d; dFiles)
{
auto SdFiles = dirEntries(d, SpanMode.breadth).array;
foreach (f; SdFiles)
{
subdirTotal += f.size;
}
ulong subdirTotalGB = (subdirTotal/1024/1024/1024);
if (subdirTotalGB > SizeDir)
{
writefln("%-63s %s", d[0].replace("?\\", ""),
subdirTotalGB);
}
subdirTotal = 0;
}
}
}
//
/* Main
*/
//
void main ()
{
SizeDirList(SizeDirlst);
}
Re: Long File path Exception:The system cannot find the path specified
On Wednesday, 23 August 2017 at 13:50:18 UTC, Vino.B wrote: On Wednesday, 23 August 2017 at 13:14:31 UTC, Moritz Maxeiner wrote: [...] Hi, [...] Hi, Any idea of what is causing this issue.
Re: Long File path Exception:The system cannot find the path specified
On Wednesday, 23 August 2017 at 13:14:31 UTC, Moritz Maxeiner
wrote:
On Wednesday, 23 August 2017 at 13:04:28 UTC, Vino.B wrote:
The line it complains is
std.file.FileException@std\file.d(3713):even after enabling
debug it points to the same
Output:
D:\DScript>rdmd -debug Test.d -r dryrun
std.file.FileException@std\file.d(3713):
N:\PROD_TEAM\TST_BACKUP\abcyf0\TST_BATS\j2ee_backup\cluster\states0\apps\bat.com\tc~bat~agent~application~e2emai~std~collectors\servlet_jsp\tc~bat~agent~application~e2emai~std~collectors\root\WEB-INF\entities\DataCollectionPushFileContentScannerTypeBuilder: The system cannot find the path specified.
0x00431A56
0x00429801
You need to compile with debug info (option `-g`), not compile
in debug code (option `-debug`).
What's the (full) stack trace when compiling with debug info?
Hi,
Please find the output and the entire program below after
executing with option -g.
Program:
import std.file: dirEntries, isFile, SpanMode, remove,
rmdirRecurse, exists, mkdir;
import std.stdio: writeln, writefln, File;
import std.algorithm: filter;
import std.array: array;
import std.path: globMatch, baseName;
/**/
/* Global Valiables */
/**/
int SizeDir = 10;
/**/
/* Folder Lists */
/**/
auto SizeDirlst = [ "N:\\PROD_TEAM", "P:\\TEAM" ];
/**/
/* Function : Size of Non DND Dir List*/
/**/
void SizeDirList (string[] SzDNDlst)
{
ulong subdirTotal = 0;
foreach (string i; SzDNDlst[0 .. $])
{
auto dFiles = dirEntries(i, SpanMode.shallow).filter!(a =>
a.isDir && !globMatch(a.baseName, "*DND*")).array;
foreach (d; dFiles)
{
auto SdFiles = dirEntries(d,
SpanMode.breadth).array;
foreach (f; SdFiles)
{
subdirTotal += f.size;
}
ulong subdirTotalGB =
(subdirTotal/1024/1024/1024);
if (subdirTotalGB > SizeDir)
{
writefln("%-63s %s", d, subdirTotalGB);
}
subdirTotal = 0;
}
}
}
//
/* Main
*/
//
void main ()
{
SizeDirList(SizeDirlst);
}
Output:
D:\\DScript>rdmd -g Test.d -r dryrun
N:\PROD_TEAM\PREBACKUP
97
N:\PROD_TEAM\TST_BACKUP\abcyf0\TST_BATS\j2ee_backup\cluster\states0\apps\bat.com\tc~bat~agent~application~e2emai~std~collectors\servlet_jsp\tc~bat~agent~application~e2emai~std~collectors\root\WEB-INF\entities\DataCollectionPushFileContentScannerTypeBuilder:
The system cannot find the path specified.
0x00415742 in @safe bool std.file.cenforce!(bool).cenforce(bool,
lazy const(char)[], immutable(char)[], uint)
0x0040EA79 in void std.file.DirIteratorImpl.popFront()
0x00404948 in void
std.array.Appender!(std.file.DirEntry[]).Appender.put!(std.file.DirEntry).put(std.file.DirEntry) at C:\D\dmd2\windows\bin\..\
..\src\phobos\std\array.d(2919)
0x00404B94 in std.file.DirEntry[]
std.array.array!(std.file.DirIterator).array(std.file.DirIterator) at C:\D\dmd2\windows\bin\..\..\src\phobos\s
td\array.d(137)
0x00402363 in void Size.SizeDirList(immutable(char)[][]) at
D:\DScript\Test.d(26)
0x00402468 in _Dmain at D:\DScript\Test.d(46)
0x0040E323 in
D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFZv
0x0040E2E7 in void rt.dmain2._d_run_main(int, char**, extern (C)
int function(char[][])*).runAll()
0x0040E1E8 in _d_run_main
0x0040DCB8 in main at D:\DScript\Test.d(7)
0x00459DA9 in mainCRTStartup
0x75CE336A in BaseThreadInitThunk
0x775F9902 in RtlInitializeExceptionChain
0x775F98D5 in RtlInitializeExceptionChain
Lines in the script.
D:\DScript\Test.d(26) : auto SdFiles = dirEntries(d,
SpanMode.breadth).array;
D:\DScript\Test.d(46) : SizeDirList(SizeDirlst);
D:\DScript\Test.d(7) : /* Global Valiables
*/
Re: Long File path Exception:The system cannot find the path specified
On Wednesday, 23 August 2017 at 12:12:47 UTC, Moritz Maxeiner wrote: On Wednesday, 23 August 2017 at 12:01:20 UTC, Vino.B wrote: On Wednesday, 23 August 2017 at 11:29:07 UTC, Moritz Maxeiner wrote: On which line do you get the Exception? Does it happen with shorter paths, as well? Assuming it happens with all paths: Just to be sure, is each of those backslashes actually encoded as a backslash? If you specified the path in the D source like `path = "N:\PROD_TEAM..."`, then it won't be, because backslash is an escape character (you would need to write `path = "N:\\PROD_TEAM..."`, or better yet path = "N:/PROD_TEAM..."`). The above program scan for files/directories under the main folder N:\PROD_TEAM\ and reports the size of each of the sub folders eg: "TST_BACKUP", under the main folder "N:\PROD_TEAM\" there are more than 9000+ files/directories, eg: (N:\PROD_TEAM\TST_BACKUP,N:\PROD_TEAM\PRD_BACKUP\) and the above program will output the size of the sub folders "TST_BACKUP,PRD_BACKUP", there is no issue is the path is shorter, the issue arises only when the path is bigger, eg the program prints the size of the sub folder PRD_BACKUP but when it tries to scan the sub folder TST_BACKUP the issue arises and the program terminates with the exception "The system cannot find the path specified", hence it not not be possible to provide the path explicitly, so can you help me on this. While that is good to know, you still haven't answered my initial question: On which line do you get the Exception? If your program terminates because of an uncaught exception (as you stated), then you should've received a stack trace containing the line number on which the exception was thrown (remember to compile with debug info). You should also consider providing a compilable, minimal example (with test data) that can be used to reproduce the issue. The line it complains is std.file.FileException@std\file.d(3713): even after enabling debug it points to the same Output: D:\DScript>rdmd -debug Test.d -r dryrun std.file.FileException@std\file.d(3713): N:\PROD_TEAM\TST_BACKUP\abcyf0\TST_BATS\j2ee_backup\cluster\states0\apps\bat.com\tc~bat~agent~application~e2emai~std~collectors\servlet_jsp\tc~bat~agent~application~e2emai~std~collectors\root\WEB-INF\entities\DataCollectionPushFileContentScannerTypeBuilder: The system cannot find the path specified. 0x00431A56 0x00429801 From, Vino.B
Re: ore.exception.RangeError
On Wednesday, 23 August 2017 at 11:18:14 UTC, Moritz Maxeiner
wrote:
On Wednesday, 23 August 2017 at 05:53:46 UTC, ag0aep6g wrote:
On 08/23/2017 07:45 AM, Vino.B wrote:
Execution :
rdmd Summary.d - Not working
rdmd Summary.d test - Working
Program:
void main (string[] args)
{
if(args.length != 2 )
writefln("Unknown operation: %s", args[1]);
}
When args.length == 1, then the one element is args[0], not
args[1].
args[1] only exists when args.length >= 2.
To expand on that: argv[0] is what is passed to the process the
D program runs at in the system call it was spawned from (e.g.
execve), which usually corresponds to the program's name.
Hi All,
Thank you very much, was able to resolve this issue.
From,
Vino.B
Re: Long File path Exception:The system cannot find the path specified
On Wednesday, 23 August 2017 at 11:29:07 UTC, Moritz Maxeiner wrote: On Wednesday, 23 August 2017 at 05:06:50 UTC, Vino.B wrote: Hi All, When i run the below code in windows i am getting "The system cannot find the path specified" even though the path exist , the length of the path is 516 as below, request your help. Path : N:\PROD_TEAM\TST_BACKUP\abcyf0\TST_BATS\j2ee_backup\cluster\states0\apps\bat.com\tc~bat~agent~application~e2emai~std~collectors\servlet_jsp\tc~bat~agent~application~e2emai~std~collectors\root\WEB-INF\entities\DataCollectionPushFileContentScannerTypeBuilder Program: [...] On which line do you get the Exception? Does it happen with shorter paths, as well? Assuming it happens with all paths: Just to be sure, is each of those backslashes actually encoded as a backslash? If you specified the path in the D source like `path = "N:\PROD_TEAM..."`, then it won't be, because backslash is an escape character (you would need to write `path = "N:\\PROD_TEAM..."`, or better yet path = "N:/PROD_TEAM..."`). Hi, The above program scan for files/directories under the main folder N:\PROD_TEAM\ and reports the size of each of the sub folders eg: "TST_BACKUP", under the main folder "N:\PROD_TEAM\" there are more than 9000+ files/directories, eg: (N:\PROD_TEAM\TST_BACKUP,N:\PROD_TEAM\PRD_BACKUP\) and the above program will output the size of the sub folders "TST_BACKUP,PRD_BACKUP", there is no issue is the path is shorter, the issue arises only when the path is bigger, eg the program prints the size of the sub folder PRD_BACKUP but when it tries to scan the sub folder TST_BACKUP the issue arises and the program terminates with the exception "The system cannot find the path specified", hence it not not be possible to provide the path explicitly, so can you help me on this. From, Vino.B
Parameter File reading
Hi All, Can anyone provide me a example code on how to read a parameter file and use those parameter in the program. From, Vino.B
ore.exception.RangeError
Hi All,
Can any one guide me what is wrong with the below code , whil i
run the code without any arguments is am getting the below
exception
Execution :
rdmd Summary.d - Not working
rdmd Summary.d test - Working
Program:
void main (string[] args)
{
if(args.length != 2 )
writefln("Unknown operation: %s", args[1]);
}
core.exception.RangeError@Summary.d(197): Range violation
0x00416218
0x0041AA8F
0x0041AA53
0x0041A954
0x00413C0F
0x75EB336A in BaseThreadInitThunk
0x776B9902 in RtlInitializeExceptionChain
0x776B98D5 in RtlInitializeExceptionChain
From,
Vino.B
Long File path Exception:The system cannot find the path specified
Hi All,
When i run the below code in windows i am getting "The system
cannot find the path specified" even though the path exist , the
length of the path is 516 as below, request your help.
Path :
N:\PROD_TEAM\TST_BACKUP\abcyf0\TST_BATS\j2ee_backup\cluster\states0\apps\bat.com\tc~bat~agent~application~e2emai~std~collectors\servlet_jsp\tc~bat~agent~application~e2emai~std~collectors\root\WEB-INF\entities\DataCollectionPushFileContentScannerTypeBuilder
Program:
void SizeDirList (string[] SzDNDlst)
{
auto logF = File(LFpath, "a");
ulong subdirTotal = 0;
foreach (string i; SzDNDlst[0 .. $])
{
auto dFiles = dirEntries(i, SpanMode.shallow).filter!(a =>
a.isDir && !globMatch(a.baseName, "*DND*")).array;
foreach (d; dFiles)
{
auto SdFiles = dirEntries(d,
SpanMode.breadth).array;
foreach (f; SdFiles)
{
subdirTotal += f.size;
}
ulong subdirTotalGB =
(subdirTotal/1024/1024/1024);
if (subdirTotalGB > SizeDir)
{
writefln("%-63s %s", d, subdirTotalGB);
}
subdirTotal = 0;
}
}
}
From,
Vino.B
SMTP Mail
Hi All,
Request your help on sending Mails, I am able to receive mails
with empty body the line "smtp.message ="Example Message" doesn't
seem to be working and also please let me know how do i send a
file as a attachment in a email.
import std.net.curl;
void main ()
{
auto smtp = SMTP("smtp://server.com");
smtp.mailTo = ["x...@xxx.com"];
smtp.mailFrom = "x...@xxx.com";
smtp.message = "Example Message";
smtp.perform();
}
From,
Vino.B
Re: Folder Size
On Monday, 21 August 2017 at 08:57:52 UTC, Aravinda VK wrote:
On Saturday, 19 August 2017 at 14:19:39 UTC, Vino.B wrote:
[...]
Keep a variable to add the sizes of subdirs
auto dFiles = dirEntries(i, SpanMode.shallow).filter!(a =>
a.isDir && !globMatch(a.baseName, "*DND*")).array;
ulong subdirTotal = 0;
foreach (d; dFiles)
{
subdirTotal = 0;
auto SdFiles = dirEntries(d, SpanMode.depth).array;
foreach(f; SdFiles) {
subdirTotal += f.size;
}
writeln(d, "\t", subdirTotal);
}
--
Aravinda
http://aravindavk.in
Hi Aravinda,
Thank you for the idea, but i have to tweek my code toas below
to get the required output and now I am able to get the required
output
auto dFiles = dirEntries(i, SpanMode.shallow).filter!(a =>
a.isDir && !globMatch(a.baseName, "*DND*")).array;
foreach (d; dFiles)
{
auto SdFiles = dirEntries(d, SpanMode.depth).array;
//auto entry = SdFiles.front;
foreach (f; SdFiles)
{
subdirTotal += f.size;
}
ulong subdirTotalGB = (subdirTotal/1000/1000);
writefln("%-63s %s %s", d, subdirTotalGB, "
MB");
subdirTotal = 0;
}
From,
Vino.B
