File difference module similar to the Linux command "comm"

2021-09-11 Thread Vino via Digitalmars-d-learn

Hi All,

  May i know whether there is any module similar to  the Linux 
command "comm" (finding difference between 2 files), if present 
can you please guide me through link to the page nor do let me 
know  if there is any other solution.


Eg
File1
list1
list2
list3
lsit5

File2
list1
list3
lsit6

File1 : difference: list6
File2 : difference: list2,list5

From,
Vino


Re: Array permutations

2021-09-11 Thread Vino via Digitalmars-d-learn

On Saturday, 11 September 2021 at 23:04:29 UTC, Vino wrote:

On Saturday, 11 September 2021 at 19:57:26 UTC, jfondren wrote:

[...]


Hi,

   Thank you very much, let me try to explain the actual 
requirement, the actual requirement is to find all the 
permutations of a given array, I modified your code and it 
gives me the exact output , but i need help to remove the array 
which has the same value as below


[...]


Hi,

  Was able to resolve the issue by adding the filter 
".filter!"a[0] != a[1]"


From,
Vino


Re: Array permutations

2021-09-11 Thread Vino via Digitalmars-d-learn

On Saturday, 11 September 2021 at 19:57:26 UTC, jfondren wrote:

On Saturday, 11 September 2021 at 19:37:42 UTC, Vino wrote:

Hi All,

   Request your help on the below to print the below array as 
"Required output", Was able to get these values 
"[1,2],[2,3],[3,4],[4,5]" by using list.slide(2), need your 
help to get values "1,3],[1,4],[1,5],[2,4],[2,5],[3,5]"


auto list[] = [1,2,3,4,5]

Required output
[1,2],[2,3],[3,4],[4,5],[1,3],[1,4],[1,5],[2,4],[2,5],[3,5]

From,
Vino


I don't see the pattern just from the required output. If you 
have an English description of what you want, it might be 
easier.


Anyway, take a look at

```d
unittest {
import std.algorithm : cartesianProduct, filter, map, sort;
import std.range : drop;
import std.array : array;

auto list = [1,2,3,4,5];
auto required = 
[[1,2],[2,3],[3,4],[4,5],[1,3],[1,4],[1,5],[2,4],[2,5],[3,5]];

auto pairs = list.cartesianProduct(list.drop(1))
.filter!"a[0] < a[1]"
.map!array
.array;
assert(pairs == [[1, 2], [1, 3], [1, 4], [1, 5], [2, 3], 
[2, 4], [2, 5], [3, 4], [3, 5], [4, 5]]);

assert(required.sort.array == pairs);
}
```


Hi,

   Thank you very much, let me try to explain the actual 
requirement, the actual requirement is to find all the 
permutations of a given array, I modified your code and it gives 
me the exact output , but i need help to remove the array which 
has the same value as below


Example:
void main() {
import std.algorithm : cartesianProduct, filter, map, sort;
import std.range : drop;
import std.array : array;
import std.stdio : writeln;

auto list = [1,2,3,4,5];
auto pairs = list.cartesianProduct(list)
.map!array
.array;
 writeln(pairs);
}

Output
[
[1, 1], /* shodule not print this array */
[1, 2],
[1, 3],
[1, 4],
[1, 5],
[2, 1],
[2, 2], /* should not print this array */
[2, 3],
[2, 4],
[2, 5],
[3, 1],
[3, 2],
[3, 3], /* should not print this array */
[3, 4],
[3, 5],
[4, 1],
[4, 2],
[4, 3],
[4, 4], /* should not print this array */
[4, 5],
[5, 1],
[5, 2],
[5, 3],
[5, 4],
[5, 5] /* should not print this array */
]

From,
Vino


Array permutations

2021-09-11 Thread Vino via Digitalmars-d-learn

Hi All,

   Request your help on the below to print the below array as 
"Required output", Was able to get these values 
"[1,2],[2,3],[3,4],[4,5]" by using list.slide(2), need your help 
to get values "1,3],[1,4],[1,5],[2,4],[2,5],[3,5]"


auto list[] = [1,2,3,4,5]

Required output
[1,2],[2,3],[3,4],[4,5],[1,3],[1,4],[1,5],[2,4],[2,5],[3,5]

From,
Vino


Re: Calling function within class.

2020-11-19 Thread Vino via Digitalmars-d-learn

On Wednesday, 18 November 2020 at 21:33:58 UTC, Ali Çehreli wrote:

On 11/18/20 7:01 AM, Vino wrote:

>Request your help on how to call a function(listFile) from
another
> function(getFilelist) within the same class(GetDirlist),
below is an
> example code.

That code looks unnecessarily complex to me. First of all, 
parallel() already executes the loop body in separate threads, 
so I don't see any reason for Task in that code.


std.parallel will appear during my DConf presentation on 
Saturday. The following program applies ideas from some of my 
slides and just works in parallel:


import std.process;
import std.exception;
import std.format;
import std.parallelism;
import std.stdio;

class GetDirlist {

  @system private auto listFile(immutable string st)
  {
auto fl = execute(["ls","-l"]);
enforce(fl.status == 0, format("File not Found: %s", 
fl.status));

return fl.output;
  }

  @system public auto getFilelist()
  {
// I am using D dynamic arrays for simpliticy
auto flstore = [ "/path1/Dir1", "path2/Dir2" ];

// Note preallocated slot for each result:
auto amidata = new string[flstore.length];

// Taking advantage of automatic loop counter
foreach(i, st; parallel(flstore,1)) {
  // Each execution assigns to its own slot
  amidata[i] = listFile(st);
}
return amidata[];
  }
}

void main() {
  // Need an object to call a non-static member function:
  auto g = new GetDirlist();
  writeln(g.getFilelist());
}

Ali


Hi Ali,

Thank you very much, your solution works for my example, but it 
does not work for the main goal, let me explain what we are 
trying to perform.


Nut shell: Try to execute an aws command on sever accounts in 
parallel to get some data.


Noe: each account has as separate username and password store in 
a database table(encrypted).


Cod Logic
Fetch the username/ password from the table for each account.

Get the “awssecrete” key and “accesskey” for each account by 
calling an aws api using the above username/password.


Set the fetched key’s as an environment variable.

Execute the aws command and get the data for each of the account

  As we have many accounts what we are trying is to get the data 
in parallel(execute the aws command in parallel for each account 
and store the result in a array). At present our code is working 
fine(without parallel), the moment we enable parallelism, it is 
throwing an error on the SQL part (Fetch the username/ password 
from the table for each account), as it could not execute the SQL 
query in parallel for different account. If there is any other 
logic please do let me know will give it a try. Below is the SQL 
code.


@trusted public auto getAwsconf(immutable string account)
{
 auto con = new GetConnections();
 Statement stmt = con.db.prepare("SELECT 
username,AES_DECRYPT(b.userpass,b.key,b.vector) AS passwd FROM 
config WHERE account = :account");

 stmt.setParameter("account", account);
 RowSet awsaccount = stmt.query();
 scope(exit) con.db.close();
 return awsaccount;
}

From,
Vino.B


Re: Calling function within class.

2020-11-18 Thread Vino via Digitalmars-d-learn

On Wednesday, 18 November 2020 at 18:24:59 UTC, frame wrote:

On Wednesday, 18 November 2020 at 17:55:36 UTC, Vino wrote:



  I made the changes as below , still not working

auto fltask = task!listFile(st);
to
auto fltask = 
task!({listFile(st);})(this).executeInNewThread();


The syntax is just wrong:

auto fltask = task!({listFile(st);})(this).executeInNewThread();

Should be:

auto fltask = task!((GetDirlist 
obj){obj.listFile(st);})(this).executeInNewThread();


It's the anonymous function syntax in one line compressed - 
it's equal to:


auto fltask = task!(
  function(GetDirlist obj) {
 obj.listFile(st);
  }
)(this).executeInNewThread();


But with your context it must look like this:

auto fltask = task!(
  function(GetDirlist obj, string foo) {
 obj.listFile(foo);
  }
)(this, st);

Still your code need to be fixed at lot to get working.


Hi,

  The above code is a sample code, but the logic is same, correct 
me if my understanding is wrong, in the above code "obj" is a an 
object for the class GetDirlist, so we are accessing the class 
member using "obj.listFile(st)" , so why do we need the "(this)" 
and also why is this so complicated in D where as in PHP it is 
simple like below


PHP Code:
class PHPclass {
  function test1 ($st) { return $st; }
  function test2 ()  { return $this->test1("Test"); }
}

$obj = new PHPclass();
print_r($obj->test2());







Re: Calling function within class.

2020-11-18 Thread Vino via Digitalmars-d-learn

On Wednesday, 18 November 2020 at 16:53:44 UTC, frame wrote:

On Wednesday, 18 November 2020 at 15:01:53 UTC, Vino wrote:

Hi All,

  Request your help on how to call a function(listFile) from 
another function(getFilelist) within the same 
class(GetDirlist), below is an example code.


I think it's basically the same issue like that recently opened 
topic:


https://forum.dlang.org/thread/ddxasuvusgibppccl...@forum.dlang.org

You need to pass your class object as argument to a static 
method or anonymous function.


Hi,

  I made the changes as below , still not working

auto fltask = task!listFile(st);
to
auto fltask = task!({listFile(st);})(this).executeInNewThread();

Error:
/DTECH/LDC/dlang/ldc-1.24.0/bin/../import/std/parallelism.d(516,34): Error: 
function literal __lambda2() is not callable using argument types (GetDirlist)
/DTECH/LDC/dlang/ldc-1.24.0/bin/../import/std/parallelism.d(516,34):
expected 0 argument(s), not 1
/DTECH/LDC/dlang/ldc-1.24.0/bin/../import/std/parallelism.d(842,16): Error: 
template instance GetDirlist.GetDirlist.getFilelist.__foreachbody2.
Task!(delegate () @system { this.listFile(st); } , GetDirlist) 
error instantiating instantiated from here: task!(delegate () 
@system { this.listFile(st); } , GetDirlist)


From,
Vino.B


Re: Executing AWS commands

2020-11-18 Thread Vino via Digitalmars-d-learn

On Tuesday, 17 November 2020 at 21:08:21 UTC, Paul Backus wrote:

On Tuesday, 17 November 2020 at 19:07:42 UTC, Vino wrote:
auto pid = execute(["/usr/bin/aws ec2 describe-images 
--filters 'Name=state,Values=available' --query 
'Images[*].[ImageId]'"]);

[...]
auto pid = execute(["/usr/bin/aws ec2 describe-images 
--filters 'Name=state,Values=available' --query 
'Images[*].[ImageId]'"]);


You need to break up your command line so that each argument is 
in a separate array element. In the commands above you have 
multiple arguments grouped together into each array element.


Alternately, you can pass everything in a single string to 
`executeShell`.


Hi Paul,

 Thank you very much was able to execute the aws commands using 
`executeShell`.


From,
Vino.B


Calling function within class.

2020-11-18 Thread Vino via Digitalmars-d-learn

Hi All,

  Request your help on how to call a function(listFile) from 
another function(getFilelist) within the same class(GetDirlist), 
below is an example code.


Code:

class GetDirlist {

@system private auto listFile(immutable string st)
{
 auto fl = execute(["ls","-l"]);
 enforce(fl.status == 0, format("File not Found: %s", fl.status));
 return fl.output;
}

@system public auto getFilelist()
{
  Array!string amidata;
  auto flstore = Array!string("/path1/Dir1", "path2/Dir2");
  foreach(st; parallel(flstore,1)) {
 auto fltask = task!listFile(st);
 fltask.executeInNewThread();
 amidata.insert(fltask.yieldForce);
  }
  return amidata[];
}
}

void main() {
writeln(getFilelist());
}

From,
Vino.B


Executing AWS commands

2020-11-17 Thread Vino via Digitalmars-d-learn

Hi All,

  Request your help on how to execute aws commands, below is an 
example code, and this code is not working, tried several options 
nothing seem to be working.


Code:
import std.process: environment, execute;
import std.stdio: writeln;

void main() {
environment["AWS_DEFAULT_REGION"] = "eu-west-1";
auto pid = execute(["/usr/bin/aws ec2 describe-images --filters 
'Name=state,Values=available' --query 'Images[*].[ImageId]'"]);
if (pid.status != 0) { writeln("Failed"); } else { 
writeln(pid.output); }

}

Tried the below(execute, executeShell,spawnProcess,execv)

auto pid = execute(["/usr/bin/aws ec2 describe-images --filters 
'Name=state,Values=available' --query 'Images[*].[ImageId]'"]);


auto pid = execute(["/usr/bin/aws", "ec2 describe-images 
--filters 'Name=state,Values=available' --query 
'Images[*].[ImageId]'"]);


auto pid = execute(["/usr/bin/aws", "ec2 describe-images", 
"--filters 'Name=state,Values=available'", "--query 
'Images[*].[ImageId]'"]);


auto pid = execute(["/usr/bin/aws", "ec2", "describe-images", 
"--filters 'Name=state,Values=available'", "--query 
'Images[*].[ImageId]'"]);


From,
Vino.B



Re: Request our suggestio: better way to insert data from Array!string[string] into a database table

2020-11-16 Thread Vino via Digitalmars-d-learn

On Monday, 16 November 2020 at 18:30:01 UTC, Max Haughton wrote:

On Monday, 16 November 2020 at 17:44:08 UTC, Vino wrote:

Hi All,

  Request your suggestion, we have a program which call's an 
api, the output of the api is parsed using json parser and the 
result is stored in an array(Array!string[string] data), then 
these stored result are inserted into MySQL table, for 
inserting the data into the table we use the below code, this 
code is a small code which just contains 2 data items 
(Type,Hostname) and we have similar api's which  contains 
15-20 data items, hence request your suggestion on is there 
any better way than the below code, using the below logic the 
foreach line is will run into multiple lines eg:


[...]


What are you looking to improve? Do you want to make the code 
prettier or faster?


It doesn't look too bad to my eye although my personal style 
would be to unpack t and h inside the foreach loop.


Hi Max,

 I am looking to improve the performance and any quicks that can 
replace the above code, similar to the below topic  
https://forum.dlang.org/post/kkyzysjpcrdovafmr...@forum.dlang.org


Request our suggestio: better way to insert data from Array!string[string] into a database table

2020-11-16 Thread Vino via Digitalmars-d-learn

Hi All,

  Request your suggestion, we have a program which call's an api, 
the output of the api is parsed using json parser and the result 
is stored in an array(Array!string[string] data), then these 
stored result are inserted into MySQL table, for inserting the 
data into the table we use the below code, this code is a small 
code which just contains 2 data items (Type,Hostname) and we have 
similar api's which  contains 15-20 data items, hence request 
your suggestion on is there any better way than the below code, 
using the below logic the foreach line is will run into multiple 
lines eg:


foreach(i, ref a, ref b,...ref 20, 
lockstep(data["D1"][],data["D2"][],.data["D20"])


Code:

 Statement stmt = con.db.prepare("INSERT INTO test(Type,Hostname) 
VALUES(:Type,:Hostname");
 foreach(i,ref t,ref h; lockstep(data["Type"][], 
data["Hostname"][]) {

stmt.setParameter("Type", data["Type"][i]);
stmt.setParameter("Hostname", data["Hostname"][i]);
result =  stmt.execute();
}


From,
Vino.B


Re: canFind all elements in a array.

2020-11-10 Thread Vino via Digitalmars-d-learn

On Tuesday, 10 November 2020 at 09:47:06 UTC, sarn wrote:

On Tuesday, 10 November 2020 at 08:19:15 UTC, Vino wrote:

[...]


This is iterating over all the elements in data2 and outputting 
some of them, so the output will never be longer than data2.


[...]


Hi Sarn,

  Thank you very much


canFind all elements in a array.

2020-11-10 Thread Vino via Digitalmars-d-learn

Hi All,

   Request your help, the below code output's as below hence 
request your help on hot to get the output as below(Required 
Output).


Output
DEV Cluster
QAS Cluster

Required Output

DEV Cluster
DEV Cluster
DEV Cluster
QAS Cluster

Code
import std.container.array;
import std.stdio: writeln;
import std.algorithm: canFind;
import std.typecons: Tuple, tuple;

void main () {
Array!string data1;
Array!(Tuple!(string,string)) data2;
Array!string rs;

data1.insertBack("DEV Systems");
data1.insertBack("DEV Systems");
data1.insertBack("DEV Systems");
data1.insertBack("QAS Systems");

data2.insertBack(tuple("DEV Systems","DEV Cluster"));
data2.insertBack(tuple("QAS Systems","QAS Cluster"));

foreach(i; data2[]) {
   if(data1[].canFind(i[0])) {
 writeln(i[1]);
  }
}
}

From,
Vino.B


Re: std.net.curl : Performance

2020-11-09 Thread Vino via Digitalmars-d-learn

On Monday, 9 November 2020 at 20:57:33 UTC, Daniel Kozak wrote:
On Mon, Nov 9, 2020 at 9:50 PM rinfz via Digitalmars-d-learn < 
digitalmars-d-learn@puremagic.com> wrote:



On Monday, 9 November 2020 at 20:40:59 UTC, rinfz wrote:
> On Monday, 9 November 2020 at 19:55:07 UTC, Vino wrote:
>> ...
>
> The only curl option you need to set within the loop is the 
> CurlOption.url. So your foreach block should look more like:

>
> foreach (...) {
> string url = chain(apihost, only(':'), 
> to!string(apiport),

> apiuri).to!string;
> https.handle.set(CurlOption.url, url);
> https.perform();
> scope(failure) exit(-4);
> scope(exit) https.shutdown;
> apidata.insert(tuple(seq, cast(string) content));
> content = [];
> }
>
> Every other line can be placed before the foreach.

In fact, you don't need url in there either since it's not 
dependent on loop variables, and you don't need the scope 
guards if this is running in main (you can move them out of 
the loop too).


foreach (...) {
 https.handle.set(CurlOption.url, url);
 https.perform();
 apidata.insert(tuple(seq, cast(string) content));
 content = [];
}



In fact he does not need foreach. Because he use it on empty 
result


Hi All,

  The reason that the above code is within foreach loop is 
because there are several api's and each of the api's have 
separate configuration. These configuration are maintained in a 
table in MySQL database, the result contains the configuration 
details for each of these api's, below is the complete code.


Sequence
The code collects the data from several api's whose output is 
json.
The output of the api's are parsed, filtered, validated and 
stored in a array using another function.(using asdf json parser)
The data stores in array is then inserted into the table in MySQL 
database.(using hunt-database).


Code:
auto getData ()
{
 Array!(Tuple!(int,string)) apidata;
 Row[] result;
 result = getApiconf();  \\ fetch's the api configuration 
from the database table.

 foreach(i, k; parallel(result,1))
{

  string apihost = result[i][0].get!(string);
  int apiport = result[i][1].get!(int);
  string apiuser = result[i][2].get!(string);
  string apipass = result[i][3].get!(string);
  int seq = result[i][4].get!int;
  string apiuri = result[i][5].get!(string);
  int connecttimeout = result[i][6].get!(int);
  int tcp_nodelay = result[i][7].get!(int);
  int http_version = result[i][8].get!(int);
  int sslversion = result[i][9].get!(int);
  int use_ssl = result[i][10].get!(int);
  int ssl_verifypeer = result[i][11].get!(int);

  string url = chain(apihost, only(':'), to!string(apiport), 
apiuri).to!string;
  string usrpass = chain(apiuser, only(':'), 
apipass).to!string;


  auto https = HTTP();
  https.handle.set(CurlOption.buffersize, 512000);
  https.handle.set(CurlOption.userpwd, usrpass);
  https.handle.set(CurlOption.connecttimeout, connecttimeout);
  https.handle.set(CurlOption.tcp_nodelay, tcp_nodelay);
  https.handle.set(CurlOption.http_version, http_version);
  https.handle.set(CurlOption.sslversion,  sslversion);
  https.handle.set(CurlOption.use_ssl,  use_ssl);
  https.handle.set(CurlOption.ssl_verifypeer, ssl_verifypeer);
  https.handle.set(CurlOption.url, url);
  https.method(HTTP.Method.get);
  https.StatusLine st;
  https.onReceiveStatusLine = (https.StatusLine st) { if 
(st.code != 200)

 {
   throw new Exception(st.reason);
 } };
  ubyte[] content;
  https.onReceive = (ubyte[] data) { content ~= data; return 
data.length; };

  https.perform();
  scope(failure) { https.shutdown; exit(-4); } scope(exit) 
https.shutdown;

  apidata.insert(tuple(seq, cast(string) content));
 }
 return apidata[].sort;
}

 From,
Vino.B



std.net.curl : Performance

2020-11-09 Thread Vino via Digitalmars-d-learn

Hi All,

  Request your help to on how to improve the performance of the 
below code.


import std.conv: to;
import std.net.curl : get, HTTP, CurlOption;
import std.parallelism: parallel;
import std.range: chain, only;
import std.typecons: Tuple, tuple;

void main ()
{
 Array!(Tuple!(int,string)) apidata;
 Row[] result;
 string apihost = "abc.com"; int apiport = 1830; string apiuri = 
/getdata;

 string apiuser = "user"; string apipass = "pass";
 foreach(i, k; parallel(result,1))
{
  string url = chain(apihost, only(':'), to!string(apiport), 
apiuri).to!string;
  string usrpass = chain(apiuser, only(':'), 
apipass).to!string;

  auto https = HTTP();
  https.handle.set(CurlOption.buffersize, 512000);
  https.handle.set(CurlOption.userpwd, usrpass);
  https.handle.set(CurlOption.connecttimeout, 600);
  https.handle.set(CurlOption.tcp_nodelay, 1);
  https.handle.set(CurlOption.http_version, 2);
  https.handle.set(CurlOption.sslversion,  1;
  https.handle.set(CurlOption.use_ssl,  3);
  https.handle.set(CurlOption.ssl_verifypeer, 0);
  https.handle.set(CurlOption.url, url);
  https.method(HTTP.Method.get);
  https.StatusLine st;
  https.onReceiveStatusLine = (https.StatusLine st) { if 
(st.code != 200) { throw new Exception(st.reason); } };

  ubyte[] content;
  https.onReceive = (ubyte[] data) { content ~= data; return 
data.length; };

  https.perform();
  scope(failure) { https.shutdown; exit(-4); } scope(exit) 
https.shutdown;

  apidata.insert(tuple(seq, cast(string) content));
 }
 return apidata[].sort;
}

From,
Vino.B


Re: Extract sub string from a string

2020-11-09 Thread Vino via Digitalmars-d-learn

On Monday, 9 November 2020 at 18:55:44 UTC, Ali Çehreli wrote:

On 11/9/20 9:53 AM, k2aj wrote:

> string text = "welcome2worldinfo";
> string hg = toUpper(text[0..7] ~ "-" ~ text[7..8] ~ "-" ~
text[8..13]);

If those concatenations with the ~ operators prove to be costly 
at runtime, the following range expression may be faster 
because it does not allocate any memory:


import std.string;
import std.range;
import std.algorithm;
import std.stdio;

void main() {

  string text = "welcome2worldinfo";
  auto hg = chain(text[0..7], only('-'), text[7..8], only('-'), 
text[8..13]).map!toUpper;

  writeln(hg);
}

Ali


Hi Both,

  Thank you very much, your solution resolved our issue.




Extract sub string from a string

2020-11-09 Thread Vino via Digitalmars-d-learn

Hi All,

   Request your help on how to extract sub string from a string, 
below is an example in PHP, need your help on how to do the same 
in D.


$text = "welcome2worldinfo";
$hg = strtoupper(substr($text , 0, 7).'-'.substr($text, 7, 
1).'-'.substr($text, 8, 5));

print_r($hg) \\ Output : WELCOME-2-WORLD

From,
Vino.B




Re: asdf get first value from a json string.

2020-11-09 Thread Vino via Digitalmars-d-learn

On Sunday, 8 November 2020 at 19:31:50 UTC, frame wrote:

On Sunday, 8 November 2020 at 19:29:39 UTC, frame wrote:

On Sunday, 8 November 2020 at 16:30:40 UTC, Vino wrote:
   Request your help on how to get the first value of "type" 
from the below json, the expected output required is as below,


You need a data structure to work with, eg:

static struct S {
   string characteristicValue;
}

foreach (ref j; parseJson(data)["items"].byElement()) {
   auto sArr = j["type"].deserialize!(S[]);
   writefln("value: %s", s[0].characteristicValue);
}

Then split your S.characteristicValue into tokens with 
std.array.split


writefln("value: %s", sArr[0].characteristicValue);


Hi,

 Thank you very much, your solution resolved the issue.


asdf get first value from a json string.

2020-11-08 Thread Vino via Digitalmars-d-learn

Hi All,

   Request your help on how to get the first value of "type" from 
the below json, the expected output required is as below,


{"characteristicValue":"TT,t...@dev.com,DEV"}

output1: TT
output2: t...@dev.com

Code:

/+dub.sdl:
dependency "asdf" version="~>0.6.6"
+/
import std;
import asdf;

void main()
{
 string data = `{"items":[{
  "hostname":"test01",
  "type":[
  {"characteristicValue":"TT,t...@dev.com,DEV"},
  {"characteristicValue":"000"}
]}]}`;

 foreach (ref j; parseJson(data)["items"].byElement()) {
writeln(j["type"]) ;
 }
}

From
Vino.B


Re: Comparison : mysql-native + asdf and hunt-database + asdf

2020-11-07 Thread Vino via Digitalmars-d-learn

On Saturday, 7 November 2020 at 15:26:48 UTC, frame wrote:

On Saturday, 7 November 2020 at 14:57:39 UTC, Vino wrote:



  After further analysis we suspect that the issue is at the 
package std.net.curl the flow of the program is as below


Establishing a new connection may vary in duration of +200ms or 
more.


Hi All,

 Trying to improve the above code hence request your help on how 
to use array container instead of array, tried as per the example 
below but not working.


Code:
  ubyte[] content;
  https.onReceive = (ubyte[] data) { content ~= data; return 
data.length; };



Example
  Array!ubyte content;
  https.onReceive = (Array!ubyte data) { content.insert(data); 
return data.length; };


From,
Vino.B


Re: Comparison : mysql-native + asdf and hunt-database + asdf

2020-11-07 Thread Vino via Digitalmars-d-learn

On Saturday, 7 November 2020 at 14:38:35 UTC, Vino wrote:

On Saturday, 7 November 2020 at 14:16:46 UTC, Vino wrote:

On Saturday, 7 November 2020 at 12:29:46 UTC, Andre Pany wrote:

On Friday, 6 November 2020 at 04:58:05 UTC, Vino wrote:

[...]


While doing performance measurements you should do each test 
multiple time (at least 5 times). There could be for example 
an effect that executing a db query the first time is a lot 
slower than afterwards.


Maybe mysql native is slower, but maybe this isn't the case, 
just the way performance measurements was done was 
incorrectly.


Kind regards
Andre


Hi Andre,

  The benchmark was performed with 100 executions as below

void main()
{
 void f() {
 Array!string[string] data = getdata("TEST");
 foreach(i; data) { writeln(i[]); }
 }
 auto r = benchmark!(f)(100);
 Duration t = r[0];
 writeln(t);
}


Hi All,

  Upon further testing, below are the observations, based on 
different compliers (DMD/LDC), using LDC the size of the 
executable is reduced but the run time is increased from 5 to 6 
sec , the document states that if we use the complier option 
dub --"build=release --compiler=ldmd2" it would enhance the 
performance  but as per our analysis we do not see and 
improvement, below is the runtime


LDC  : 6 secs, 136 ms, 97 
μs, and 3 hnsecs
LDC(dub --build=release --compiler=ldmd2): 6 secs, 203 ms, 283 
μs, and 6 hnsecs



Observations

Component   :  mysql-native + asdf  
Complier:  DMD
Size:  17MB
Duration:  10 secs, 189 ms, 919 μs, and 3 hnsecs

Component   :  mysql-native + asdf  
Complier:  LDC
Size:  20MB
Duration:  10 secs, 526 ms, 350 μs, and 6 hnsecs

Component   :  mysql-native + asdf  
Complier:  LDC(dub --build=release --compiler=ldmd2)
Size:  3.7MB
Duration:  10 secs, 411 ms, 793 μs, and 8 hnsecs
*
Component   :  hunt-database + asdf 
Complier:  DMD
Size:  81MB
Duration:  5 secs, 916 ms, 418 μs, and 3 hnsecs

Component   :  hunt-database + asdf 
Complier:  LDC
Size:  50MB
Duration:  6 secs, 136 ms, 97 μs, and 3 hnsecs

Component   :  hunt-database + asdf 
Complier:  LDC(dub --build=release --compiler=ldmd2)
Size:  17MB
Duration:  6 secs, 136 ms, 97 μs, and 3 hnsecs

From,
Vino.B


Hi All,

  After further analysis we suspect that the issue is at the 
package std.net.curl the flow of the program is as below


hunt.database : queries a single table(12 column's and 3 rows) 
which contains the 3 api url details(getApiconf)


std.net.curl  : call those api's (parallel) and the send the data 
to json parser


asdf  : Parse the output of the api's and store the data 
in a array Array!string[string] data


Code of the API calls
import common.GetApiconf;
import core.stdc.stdlib: exit;
import hunt.database;
import std.algorithm: joiner, sort;
import std.container.array;
import std.conv: to;
import std.net.curl : get, HTTP, CurlOption;
import std.parallelism: parallel;
import std.typecons: Tuple, tuple;

void main ()
{
 Array!(Tuple!(int,string)) apidata;
 Row[] result;
 result = getApiconf("TST");
 foreach(i, k; parallel(result,1))
{

  string apihost = result[i][0].get!(string);
  int apiport = result[i][1].get!(int);
  string apiuser = result[i][2].get!(string);
  string apipass = result[i][3].get!(string);
  int seq = result[i][4].get!int;
  string apiuri = result[i][5].get!(string);
  int connecttimeout = result[i][6].get!(int);
  int tcp_nodelay = result[i][7].get!(int);
  int http_version = result[i][8].get!(int);
  int sslversion = result[i][9].get!(int);
  int use_ssl = result[i][10].get!(int);
  int ssl_verifypeer = result[i][11].get!(int);

  string url = to!string(([apihost, ":",  to!string(apiport), 
apiuri]).joiner);
  string usrpass = to!string(([apiuser, ":", 
apipass]).joiner);


  auto https = HTTP();
  https.handle.set(CurlOption.userpwd, usrpass);
  https.handle.set(CurlOption.connecttimeout, connecttimeout);
  https.handle.set(CurlOption.tcp_nodelay, tcp_nodelay);
  https.handle.set(CurlOption.http_version, http_version);
  https.handle.set(CurlOption.sslversion,  sslversion);
  https.handle.set(CurlOption.use_ssl,  use_ssl);
  https.handle.set(CurlOption.ssl_verifypeer, ssl_verifypeer);
  https.handle.set(CurlOption.url, url);
  https.method(HTTP.Method.get);
  https.StatusLine st;
  https.onReceiveStatusLine = (https.StatusLine st) { if 
(st.code != 200)

{
 throw new Exception(st.reason);
}
  };
  ubyte[] content;
  https.onReceive = (ubyte[] data) {
 content ~= data; return 
data.length;

   

Re: Comparison : mysql-native + asdf and hunt-database + asdf

2020-11-07 Thread Vino via Digitalmars-d-learn

On Saturday, 7 November 2020 at 14:16:46 UTC, Vino wrote:

On Saturday, 7 November 2020 at 12:29:46 UTC, Andre Pany wrote:

On Friday, 6 November 2020 at 04:58:05 UTC, Vino wrote:

[...]


While doing performance measurements you should do each test 
multiple time (at least 5 times). There could be for example 
an effect that executing a db query the first time is a lot 
slower than afterwards.


Maybe mysql native is slower, but maybe this isn't the case, 
just the way performance measurements was done was incorrectly.


Kind regards
Andre


Hi Andre,

  The benchmark was performed with 100 executions as below

void main()
{
 void f() {
 Array!string[string] data = getdata("TEST");
 foreach(i; data) { writeln(i[]); }
 }
 auto r = benchmark!(f)(100);
 Duration t = r[0];
 writeln(t);
}


Hi All,

  Upon further testing, below are the observations, based on 
different compliers (DMD/LDC), using LDC the size of the 
executable is reduced but the run time is increased from 5 to 6 
sec , the document states that if we use the complier option dub 
--"build=release --compiler=ldmd2" it would enhance the 
performance  but as per our analysis we do not see and 
improvement, below is the runtime


LDC  : 6 secs, 136 ms, 97 μs, 
and 3 hnsecs
LDC(dub --build=release --compiler=ldmd2): 6 secs, 203 ms, 283 
μs, and 6 hnsecs



Observations

Component   :  mysql-native + asdf  
Complier:  DMD
Size:  17MB
Duration:  10 secs, 189 ms, 919 μs, and 3 hnsecs

Component   :  mysql-native + asdf  
Complier:  LDC
Size:  20MB
Duration:  10 secs, 526 ms, 350 μs, and 6 hnsecs

Component   :  mysql-native + asdf  
Complier:  LDC(dub --build=release --compiler=ldmd2)
Size:  3.7MB
Duration:  10 secs, 411 ms, 793 μs, and 8 hnsecs
*
Component   :  hunt-database + asdf 
Complier:  DMD
Size:  81MB
Duration:  5 secs, 916 ms, 418 μs, and 3 hnsecs

Component   :  hunt-database + asdf 
Complier:  LDC
Size:  50MB
Duration:  6 secs, 136 ms, 97 μs, and 3 hnsecs

Component   :  hunt-database + asdf 
Complier:  LDC(dub --build=release --compiler=ldmd2)
Size:  17MB
Duration:  6 secs, 136 ms, 97 μs, and 3 hnsecs

From,
Vino.B


Re: Comparison : mysql-native + asdf and hunt-database + asdf

2020-11-07 Thread Vino via Digitalmars-d-learn

On Saturday, 7 November 2020 at 12:29:46 UTC, Andre Pany wrote:

On Friday, 6 November 2020 at 04:58:05 UTC, Vino wrote:

[...]


While doing performance measurements you should do each test 
multiple time (at least 5 times). There could be for example an 
effect that executing a db query the first time is a lot slower 
than afterwards.


Maybe mysql native is slower, but maybe this isn't the case, 
just the way performance measurements was done was incorrectly.


Kind regards
Andre


Hi Andre,

  The benchmark was performed with 100 executions as below

void main()
{
 void f() {
 Array!string[string] data = getdata("TEST");
 foreach(i; data) { writeln(i[]); }
 }
 auto r = benchmark!(f)(100);
 Duration t = r[0];
 writeln(t);
}



Re: Vibe.d build on LDC error

2020-11-06 Thread Vino via Digitalmars-d-learn

On Friday, 6 November 2020 at 10:30:03 UTC, Mathias LANG wrote:

On Friday, 6 November 2020 at 05:52:56 UTC, Vino wrote:

[...]


Which Linux distribution ? Which version of Vibe.d ?
A recent enough Vibe.d should detect OpenSSL based on 1) 
pkg-config 2) the openssl binary. Make sure you have the 
development version of OpenSSL installed.
Additionally, v1.0.2 is quite old (and subject to security 
issues), so you might want consider upgrading. But even with 
that version, it should work.

You can force the usage of a certain configuration using:

```
"dependencies": {
"vibe-d": "~>0.9",
"vibe-d:tls": "*"
},
"subConfigurations": {
"vibe-d:tls": "openssl-1.0"
},
```

See: 
https://github.com/vibe-d/vibe.d/blob/70b50fdb9cd4144f1a5007b36e6ac39d4731c140/tls/dub.sdl#L99-L103


Hi Mathias,

  Thank you very much, it resolved the issue.

From,
Vino.B


Vibe.d build on LDC error

2020-11-05 Thread Vino via Digitalmars-d-learn

Hi All,

  When we try to build vide.d using ldc (dub build) we are 
getting the below error, openssl is already been installed 
(OpenSSL 1.0.2j-fips  26 Sep 2016), hence request your help on 
the same.


openssl.d:84: error: undefined reference to 'OPENSSL_init_ssl'
openssl.d:121: error: undefined reference to 'OPENSSL_sk_num'
openssl.d:128: error: undefined reference to 'OPENSSL_sk_value'
openssl.d:243: error: undefined reference to 'BIO_set_init'
openssl.d:244: error: undefined reference to 'BIO_set_data'
openssl.d:245: error: undefined reference to 'BIO_set_shutdown'
openssl.d:1382: error: undefined reference to 'BIO_get_new_index'
openssl.d:1382: error: undefined reference to 'BIO_meth_new'
openssl.d:1384: error: undefined reference to 'BIO_meth_set_write'
openssl.d:1385: error: undefined reference to 'BIO_meth_set_read'
openssl.d:1386: error: undefined reference to 'BIO_meth_set_ctrl'
openssl.d:1387: error: undefined reference to 
'BIO_meth_set_create'
openssl.d:1388: error: undefined reference to 
'BIO_meth_set_destroy'
openssl.d:899: error: undefined reference to 
'BN_get_rfc3526_prime_2048'

openssl.d:1288: error: undefined reference to 'BIO_set_init'
openssl.d:1290: error: undefined reference to 'BIO_set_data'
openssl.d:1298: error: undefined reference to 'BIO_get_shutdown'
openssl.d:1300: error: undefined reference to 'BIO_set_init'
openssl.d:1302: error: undefined reference to 'BIO_set_data'
openssl.d:1309: error: undefined reference to 'BIO_get_data'
openssl.d:1323: error: undefined reference to 'BIO_get_data'
openssl.d:1339: error: undefined reference to 'BIO_get_shutdown'
openssl.d:1342: error: undefined reference to 'BIO_set_shutdown'
openssl.d:1335: error: undefined reference to 'BIO_get_data'
collect2: error: ld returned 1 exit status
Error: /usr/bin/cc failed with status: 1
/Project/dlang/ldc-1.24.0/bin/ldc2 failed with exit code 1.

From,
Vino.B


Comparison : mysql-native + asdf and hunt-database + asdf

2020-11-05 Thread Vino via Digitalmars-d-learn

Hi All,

  We recently tested the below components and the test results 
are as below, even though hunt-database is faster than 
mysql-native it is hard to use this package as it lacks on 
documentation, non of the example provided in the documentation 
works, one has to go through the code and find how things works 
to use this package. We have raised this issue in the github and 
it been 3 day's still no update and the last documentation 
(README) was updated 14 month ago, but the latest release of this 
package was released on 2020-Oct-11, version 2.1.9. May I know 
whether is there is any plan for updating the documentation if so 
may know when would the documentation be updated, so that we can 
explore other features of hunt packages (hunt-framework vs vib.d 
framework).


Component : mysql-native + asdf
Executable size : 17 MB
Execution time : 10 secs, 189 ms, 919 μs, and 3 hnsecs

Component : hunt-database + asdf
Executable size : 81 MB
Execution time : 5 secs, 916 ms, 418 μs, and 3 hnsecs


From,
Vino.B



Re: Hunt database

2020-11-03 Thread Vino via Digitalmars-d-learn

On Tuesday, 3 November 2020 at 18:14:33 UTC, Andre Pany wrote:

On Tuesday, 3 November 2020 at 14:05:18 UTC, Vino wrote:

Hi All,

  Currently testing Hunt database, and facing an issue as 
below, hence request your help


File : GetConnections.d
###
module common.GetConnections;
import hunt.database;

class Connections
{
  public Database conn;
  this() { conn = new 
Database("mysql://username:password@localhost:3910/testdb"); }

}

###
File: GetConfig.d
###
import common.GetConnections;
import hunt.database;

auto getConfig(int Seq)
{
 auto mdb = new Connections();
 Statement stmt = mdb.conn.prepare("SELECT * FROM settings 
WHERE Seq = :Sq");

   stmt.setParameter("Sq", Seq);
   stmt.execute();
 RowSet rs = stmt.query();
 mdb.conn.close();
 return rs;
}
##
File : app.d
###
import std.stdio;
imoprt common.GetConfig;

void main() {
writeln(getConfig(1));
}

Error:
GetConfig.d
Error: function 
hunt.database.Database.Database.prepare(SqlConnection conn, 
string sql) is not callable using argument types (string)
cannot pass argument "SELECT * FROM settings WHERE Seq = :Sq" 
of type string to parameter SqlConnection conn
Error: template hunt.database.Statement.Statement.setParameter 
cannot deduce function from argument types !()(int, int), 
candidates are:


From,
Vino.B


The hunt database class has a prepare method which expects a 
SqlConnection as first argument and a string as second 
argument. You are passing only a string, therefore the first 
error.


Ad you name your database instance also connection (conn) and 
your Connections class database (mdb) it makes the source code 
quite hard to understand :)


Kind regards
Andre


Hi Andre,

  We have also tried to change the connection (con) as dbconnect 
(con) as below, as per the example provided in the link 
https://code.dlang.org/packages/hunt-database, we dont see the 
prepare method needs SqlConnection as first argument and a string 
as second where as we can see the need from the source code, so 
is the example provide in the link is wrong? if yes can you 
please provide an example nor point me to the correct 
documentation link


"Statement stmt = db.prepare("SELECT * FROM user where username = 
:username and age = :age LIMIT 10");"


File : GetConnections.d
###
module common.GetConnections;
import hunt.database;

class dbconnect
{
  public Database con;
  this() { con = new 
Database("mysql://username:password@localhost:3910/testdb"); }

}

###
File: GetConfig.d
###
import common.GetConnections;
import hunt.database;

auto getConfig(int Seq)
{
 auto mdb = new dbconnect();
 Statement stmt = mdb.con.prepare("SELECT * FROM settings WHERE 
Seq = :Sq");

   stmt.setParameter("Sq", Seq);
   stmt.execute();
 RowSet rs = stmt.query();
 mdb.conn.close();
 return rs;
}
##
File : app.d
###
import std.stdio;
imoprt common.GetConfig;

void main() {
writeln(getConfig(1));
}



Re: Hunt database

2020-11-03 Thread Vino via Digitalmars-d-learn

On Tuesday, 3 November 2020 at 14:47:01 UTC, Imperatorn wrote:

On Tuesday, 3 November 2020 at 14:05:18 UTC, Vino wrote:

Hi All,

  Currently testing Hunt database, and facing an issue as 
below, hence request your help


[...]


What datatype is Seq in your settings table?


Hi,

The filed Seq is int.


Hunt database

2020-11-03 Thread Vino via Digitalmars-d-learn

Hi All,

  Currently testing Hunt database, and facing an issue as below, 
hence request your help


File : GetConnections.d
###
module common.GetConnections;
import hunt.database;

class Connections
{
  public Database conn;
  this() { conn = new 
Database("mysql://username:password@localhost:3910/testdb"); }

}

###
File: GetConfig.d
###
import common.GetConnections;
import hunt.database;

auto getConfig(int Seq)
{
 auto mdb = new Connections();
 Statement stmt = mdb.conn.prepare("SELECT * FROM settings WHERE 
Seq = :Sq");

   stmt.setParameter("Sq", Seq);
   stmt.execute();
 RowSet rs = stmt.query();
 mdb.conn.close();
 return rs;
}
##
File : app.d
###
import std.stdio;
imoprt common.GetConfig;

void main() {
writeln(getConfig(1));
}

Error:
GetConfig.d
Error: function 
hunt.database.Database.Database.prepare(SqlConnection conn, 
string sql) is not callable using argument types (string)
cannot pass argument "SELECT * FROM settings WHERE Seq = :Sq" of 
type string to parameter SqlConnection conn
Error: template hunt.database.Statement.Statement.setParameter 
cannot deduce function from argument types !()(int, int), 
candidates are:


From,
Vino.B


Removind duplicates for JSON string

2020-11-01 Thread Vino via Digitalmars-d-learn

Hi All,

   Request your help on how to remove duplicates in JSON.

Code:
import asdf;
import std.algorithm : map, filter, uniq;
import std.container.array;
import std.stdio : writeln;
import std.typecons : Tuple, tuple;
import std.array;

void main() {
string apidata = `{
"items":
  [
{ "name":"DEV", "Configuration":{"type":"D1"} },
{ "name":"DEV" },
{ "name":"DEV", "Configuration":{"type":"D1"} },
{ "name":"QAS", "Configuration":{"type":"Q1"} },
{ "name":"QAS", "Configuration":{"type":"Q1"} },
{ "name":"QAS" },
{ "name":"PRD", "Configuration":{"type":"P1"} },
{ "name":"PRD", "Configuration":{"type":"P1"} },
{ "name":"PRD" }
  ]
}`;

Array!(Tuple!(string, string)) data =
  parseJson(apidata)["items"]
  .byElement
  .map!(item => tuple(
item["name"].get!string("default"),
item["Configuration","type"].get!string("default")
));

writeln(data[]);
}

Output: The above code produces the below output

[
 Tuple!(string, string)("DEV", "D1"),
 Tuple!(string, string)("DEV", "default"),
 Tuple!(string, string)("DEV", "D1"),
 Tuple!(string, string)("QAS", "Q1"),
 Tuple!(string, string)("QAS", "Q1"),
 Tuple!(string, string)("QAS", "default"),
 Tuple!(string, string)("PRD", "P1"),
 Tuple!(string, string)("PRD", "P1"),
 Tuple!(string, string)("PRD", "default")
]

Required Output
[
 Tuple!(string, string)("DEV", "D1"),
 Tuple!(string, string)("QAS", "Q1"),
 Tuple!(string, string)("PRD", "P1"),
]

From,
Vino.B




Re: merging container arrays

2020-10-31 Thread Vino via Digitalmars-d-learn

On Saturday, 31 October 2020 at 15:16:22 UTC, Vino wrote:

Hi All,

   Request your help on the below code, the requirement is that 
result's are stored in one single container.


Code:
import asdf;
import std.algorithm: map;
import std.container.array;
import std.stdio: writeln;
import std.typecons: Tuple, tuple;
import std.range: lockstep;

auto api1()
{
 string apidata1 = `{"items": [
{"name":"T01","hostname":"test01","pool":"Development"},
{"name":"T02","hostname":"test02","pool":"Quality"},
{"name":"T03","hostname":"test03","pool":"Production"}
  ]}`;

 Array!(Tuple!(string, string, string)) data1 = 
parseJson(apidata1)["items"].byElement

 .map!(item => tuple(
 
item["name"].get!string("default"),
 
item["hostname"].get!string("default"),
 
item["pool"].get!string("default")

 ));
 return data1[];
}

auto api2()
{
 string apidata2 = `{"items": [
{"hostname":"test01","type":"Development"},
{"hostname":"test02","type":"Quality"},
{"hostname":"test03","type":"Production"}
  ]}`;

 Array!(Tuple!(string, string)) data2 = 
parseJson(apidata2)["items"].byElement

 .map!(item => tuple(
 
item["hostname"].get!string("default"),
 
item["type"].get!string("default")

 ));
 return data2[];
}

auto api3()
{
 string apidata3 = `{"items": [
{"type":"Development","location":"L1"},
{"type":"Quality","location":"L2"},
{"type":"Production","location":"L3"}
  ]}`;

 Array!(Tuple!(string, string)) data3 = 
parseJson(apidata3)["items"].byElement

.map!(item => tuple(

item["type"].get!string("default"),

item["location"].get!string("default")

 ));
 return data3[];
}



void main()
{
 auto apidata1 = api1;
 auto apidata2 = api2;
 auto apidata3 = api3;
 if(!apidata1.empty) {
foreach(ref x , y; lockstep(apidata1[], apidata2[])) {
   if(x[1] == y[0]){
  writeln(y[1]);  // the output needs to be merged with 
apidata1

   }
}
 }
 if(!apidata2.empty) {
foreach(ref x, y; lockstep(apidata2[], apidata3[])) {
   if(x[1] == y[0]) {
  writeln(y[1]); // the output needs to be merged with 
apidata1

   }
}
 }
 writeln(apidata1[]); // Should 
contain(name,hostname,pool,type,location)

}

From,
Vino.B


Hi All,

   I was able to find a solution using a different approach, the 
code is in the link
https://run.dlang.io/is/Jx4NLw, request your help on to get the 
value using the Key


In PHP we can get the value of the associative array using the 
key as below


PHP:
foreach($array as $k) { print_r($k["Name"]); }

So request you help on how the same in d

Tired the below , no luck

1 > foreach(i; data.byKey) { writeln(i["Name"]); }

Error:
onlineapp.d(53,37): Error: cannot implicitly convert expression 
`"Name"` of type `string` to `ulong`


2 > foreach (ref i; data) writeln(i["Name"]);
3 > foreach(i; data.byValue) { writeln(i["Name"]); }
4 > foreach(i; data) { writeln(get(i["Name"])); }

Error:
onlineapp.d(53,38): Error: function 
`std.container.array.Array!string.Array.opIndex(ulong i) inout` 
is not callable using argument types `(string)`
onlineapp.d(53,38):cannot pass argument `"Name"` of type 
`string` to parameter `ulong i`


From,
Vino.B



merging container arrays

2020-10-31 Thread