Re: is it std.datetime bug?
On 02.04.2015 09:19, Jonathan M Davis via Digitalmars-d-learn wrote:
On Tuesday, March 31, 2015 12:47:34 anonymous via Digitalmars-d-learn wrote:
On Tuesday, 31 March 2015 at 11:51:26 UTC, drug wrote:
import std.datetime;
import std.stdio;
void main()
{
long.max.SysTime.toISOExtString.writeln;
}
dmd 2.065 (dpaste.dzfl.pl):
+29228-09-14T02:48:05.4775807
dmd v2.067-devel-c6b489b (using Digger):
-29227-04-20T00:11:54.5224191
could somebody confirm it?
The difference is in time zones. So it's no surprise that the
output is different.
The negative value is probably because the internal `long` wraps
around when the difference from your time zone is added to the
UTC time. I don't know if this is acceptable.
A difference in time zones would result in a different value being printed,
since the default is LocalTime. And of course, it's going to wrap if you
start dealing with long.min or long.max, and the adjustment for your time
zone causes something to be added to long.max or something to be added to
long.min. There really isn't a way around that. About the only thing that I
could think of would be to check for overlow and just force it back to
long.max when it would have gone past it, or force it back to long.min if it
would have gone past that. But that's extra overhead for a use case that's
really never going to happen. Those dates are _well_ outside of the range
that most any program will need. And it's doing the math correctly at the
limits. It's just that that entails wraparound, because we're dealing with
fixed-length integers.
This isn't a bug.
- Jonathan M Davis
Thank you all for your answers. I use long.max as an initializer for my
data sorted by time to place new data to back of the queue. So the only
inconvinience for me is a diagnostic message with negative date. With
your help I've solved it using UTC as the time zone.
Re: Speed of horizontal flip
On 2/04/2015 2:52 a.m., tchaloupka wrote:
Hi,
I have a bunch of square r16 and png images which I need to flip
horizontally.
My flip method looks like this:
void hFlip(T)(T[] data, int w)
{
import std.datetime : StopWatch;
StopWatch sw;
sw.start();
foreach(int i; 0..w)
{
auto row = data[i*w..(i+1)*w];
row.reverse();
}
sw.stop();
writeln(Img flipped in: , sw.peek().msecs, [ms]);
}
With simple r16 file format its pretty fast, but with RGB PNG
files (2048x2048) I noticed its somewhat slow so I tried to
compare it with C# and was pretty surprised by the results.
C#:
PNG load - 90ms
PNG flip - 10ms
PNG save - 380ms
D using dlib (http://code.dlang.org/packages/dlib):
PNG load - 500ms
PNG flip - 30ms
PNG save - 950ms
D using imageformats
(http://code.dlang.org/packages/imageformats):
PNG load - 230ms
PNG flip - 30ms
PNG save - 1100ms
I used dmd-2.0.67 with -release -inline -O
C# was just with debug and VisualStudio attached to process for
debugging and even with that it is much faster.
I know that System.Drawing is using Windows GDI+, that can be
used with D too, but not on linux.
If we ignore the PNG loading and saving (didn't tried libpng
yet), even flip method itself is 3 times slower - I don't know D
enough to be sure if there isn't some more effecient way to make
the flip. I like how the slices can be used here.
For a C# user who is expecting things to just work as fast as
possible from a system level programming language this can be
somewhat disappointing to see that pure D version is about 3
times slower.
Am I doing something utterly wrong?
Note that this example is not critical for me, it's just a simple
hobby script I use to move and flip some images - I can wait. But
I post it to see if this can be taken somewhat closer to what can
be expected from a system level programming language.
dlib:
auto im = loadPNG(name);
hFlip(cast(ubyte[3][])im.data, cast(int)im.width);
savePNG(im, newName);
imageformats:
auto im = read_image(name);
hFlip(cast(ubyte[3][])im.pixels, cast(int)im.w);
write_image(newName, im.w, im.h, im.pixels);
C# code:
static void Main(string[] args)
{
var files = Directory.GetFiles(args[0]);
foreach (var f in files)
{
var sw = Stopwatch.StartNew();
var img = Image.FromFile(f);
Debug.WriteLine(Img loaded in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Restart();
img.RotateFlip(RotateFlipType.RotateNoneFlipX);
Debug.WriteLine(Img flipped in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Restart();
img.Save(Path.Combine(args[0], test_ +
Path.GetFileName(f)));
Debug.WriteLine(Img saved in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Stop();
}
}
Assuming I've done it correctly, Devisualization.Image takes around 8ms
in debug mode to flip horizontally using dmd. But 3ms for release.
module test;
void main() {
import devisualization.image;
import devisualization.image.mutable;
import devisualization.util.core.linegraph;
import std.stdio;
writeln(===\nREAD\n===);
Image img = imageFromFile(test/large.png);
img = new MutableImage(img);
import std.datetime : StopWatch;
StopWatch sw;
sw.start();
foreach(i; 0 .. 1000) {
img.flipHorizontal;
}
sw.stop();
writeln(Img flipped in: , sw.peek().msecs / 1000, [ms]);
}
I was planning on doing this earlier. But I discovered a PR I pulled
which fixed for 2.067 broke chunk types reading.
How to POST data with net.curl
I have next request, that work fine with curl:
curl -X POST -F upload=@wgs84_latlon.zip
http://ogre.adc4gis.com/convert
I wrote next code:
void main()
{
auto binfile = cast(ubyte[])
read(`D:\Apps\curl\wgs84_latlon.zip`);
auto http = HTTP(http://ogre.adc4gis.com/convert;);
http.setPostData(binfile, -X POST -F upload=@);
http.onReceive = (ubyte[] data) { writeln((cast(string)data));
return data.length; };
http.perform();
}
But I have got trouble with http.setPostData. I do not know how
to properly write text part, and attaches binary zip file.
Re: is it std.datetime bug?
On Tuesday, March 31, 2015 12:47:34 anonymous via Digitalmars-d-learn wrote:
On Tuesday, 31 March 2015 at 11:51:26 UTC, drug wrote:
import std.datetime;
import std.stdio;
void main()
{
long.max.SysTime.toISOExtString.writeln;
}
dmd 2.065 (dpaste.dzfl.pl):
+29228-09-14T02:48:05.4775807
dmd v2.067-devel-c6b489b (using Digger):
-29227-04-20T00:11:54.5224191
could somebody confirm it?
The difference is in time zones. So it's no surprise that the
output is different.
The negative value is probably because the internal `long` wraps
around when the difference from your time zone is added to the
UTC time. I don't know if this is acceptable.
A difference in time zones would result in a different value being printed,
since the default is LocalTime. And of course, it's going to wrap if you
start dealing with long.min or long.max, and the adjustment for your time
zone causes something to be added to long.max or something to be added to
long.min. There really isn't a way around that. About the only thing that I
could think of would be to check for overlow and just force it back to
long.max when it would have gone past it, or force it back to long.min if it
would have gone past that. But that's extra overhead for a use case that's
really never going to happen. Those dates are _well_ outside of the range
that most any program will need. And it's doing the math correctly at the
limits. It's just that that entails wraparound, because we're dealing with
fixed-length integers.
This isn't a bug.
- Jonathan M Davis
Re: USB Controller
On Thursday, 2 April 2015 at 17:38:17 UTC, Josh Phillips wrote: Hey Guys! I've started using Arduino, specifically with Processing. I'm using it essentially as a usb controller. Is there a good way to translate this to D? There doesn't seem to be any standard library for usb usage. Are there pre-built libraries or is there C libraries I should use? Thanks Josh Have a look at libusb.org and dstep binaries under releases on github to do the .h translation. Port of tango to d2 supposedly does USB also.
Re: Speed of horizontal flip
On Wednesday, 1 April 2015 at 14:00:52 UTC, bearophile wrote: If you have to perform performance benchmarks then use ldc or gdc. Also disable bound tests with your compilation switches. Add the usual pure/nothrow/@nogc/@safe annotations where you can (they don't increase speed much, usually). if you are using classes don't forget to make the method final. Profile the code and look for the performance bottlenecks. This very text should be placed somewhere prominent at the D homepage if we don't want to constantly dissapoint people who come with the impession that D should be at the same speed level as C/C++ but their test programs aren't.
Re: lambda code
On Wednesday, 1 April 2015 at 23:29:00 UTC, Vlad Levenfeld wrote: On Tuesday, 31 March 2015 at 13:25:47 UTC, John Colvin wrote: On Tuesday, 31 March 2015 at 12:49:36 UTC, Vlad Levenfeld wrote: Is there any way (or could there be any way, in the future) of getting the code from lambda expressions as a string? I've noticed that if I have an error with a lambda that looks like, say x=x+a the error message will come up referring to it as (x) = x + a so some level of processing has already been done on the expression. Can I get at any of it during compilation? It would be useful for automatic program rewriting. Short answer: no. .codeof for functions is something I've wanted for ages, but no movement so far. :( On a more positive note, there's probably an OK way of achieving your particular goal without this. Do you have an example?
Re: Structs as template parameters: weird error message
On Thursday, 2 April 2015 at 23:12:25 UTC, biozic wrote:
The code below doesn't compile. Why this error message?
---
struct Item {
int i;
}
struct Params {
Item* item;
this(int i) {
item = new Item(i); // line 9
}
}
struct Foo(Params params) {}
enum foo = Foo!(Params(1));
---
test.d(9): Error: Item(1) is not an lvalue
this doesn't work because struct Foo is parameterised by in
instance of Params. (like if you were implementing a fixed size
array: struct FixedSizeArray(size_t len) {...}. here
FixedSizeArray is parameterised by in instance of a size_t).
Item is a pointer in Params and thus if it were to compile the
compiler would need to know the value returned by `new`at compile
time.
There are two ways to get that to compile change `Item* item;` to
`Item item;` or change `item = new Item(1);` to `item = null;`
tl;dr you can't have r-value (pointers) in instance template
parameter lists.
i.e. struct foo(my_reference_type_or_type_containg_pointers
instance) { ... }
Re: Speed of horizontal flip
On 3/04/2015 4:27 a.m., John Colvin wrote:
On Thursday, 2 April 2015 at 11:49:44 UTC, Rikki Cattermole wrote:
On 3/04/2015 12:29 a.m., John Colvin wrote:
On Thursday, 2 April 2015 at 09:55:15 UTC, Rikki Cattermole wrote:
On 2/04/2015 10:47 p.m., Rikki Cattermole wrote:
On 2/04/2015 2:52 a.m., tchaloupka wrote:
Hi,
I have a bunch of square r16 and png images which I need to flip
horizontally.
My flip method looks like this:
void hFlip(T)(T[] data, int w)
{
import std.datetime : StopWatch;
StopWatch sw;
sw.start();
foreach(int i; 0..w)
{
auto row = data[i*w..(i+1)*w];
row.reverse();
}
sw.stop();
writeln(Img flipped in: , sw.peek().msecs, [ms]);
}
With simple r16 file format its pretty fast, but with RGB PNG
files (2048x2048) I noticed its somewhat slow so I tried to
compare it with C# and was pretty surprised by the results.
C#:
PNG load - 90ms
PNG flip - 10ms
PNG save - 380ms
D using dlib (http://code.dlang.org/packages/dlib):
PNG load - 500ms
PNG flip - 30ms
PNG save - 950ms
D using imageformats
(http://code.dlang.org/packages/imageformats):
PNG load - 230ms
PNG flip - 30ms
PNG save - 1100ms
I used dmd-2.0.67 with -release -inline -O
C# was just with debug and VisualStudio attached to process for
debugging and even with that it is much faster.
I know that System.Drawing is using Windows GDI+, that can be
used with D too, but not on linux.
If we ignore the PNG loading and saving (didn't tried libpng
yet), even flip method itself is 3 times slower - I don't know D
enough to be sure if there isn't some more effecient way to make
the flip. I like how the slices can be used here.
For a C# user who is expecting things to just work as fast as
possible from a system level programming language this can be
somewhat disappointing to see that pure D version is about 3
times slower.
Am I doing something utterly wrong?
Note that this example is not critical for me, it's just a simple
hobby script I use to move and flip some images - I can wait. But
I post it to see if this can be taken somewhat closer to what can
be expected from a system level programming language.
dlib:
auto im = loadPNG(name);
hFlip(cast(ubyte[3][])im.data, cast(int)im.width);
savePNG(im, newName);
imageformats:
auto im = read_image(name);
hFlip(cast(ubyte[3][])im.pixels, cast(int)im.w);
write_image(newName, im.w, im.h, im.pixels);
C# code:
static void Main(string[] args)
{
var files = Directory.GetFiles(args[0]);
foreach (var f in files)
{
var sw = Stopwatch.StartNew();
var img = Image.FromFile(f);
Debug.WriteLine(Img loaded in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Restart();
img.RotateFlip(RotateFlipType.RotateNoneFlipX);
Debug.WriteLine(Img flipped in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Restart();
img.Save(Path.Combine(args[0], test_ +
Path.GetFileName(f)));
Debug.WriteLine(Img saved in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Stop();
}
}
Assuming I've done it correctly, Devisualization.Image takes around
8ms
in debug mode to flip horizontally using dmd. But 3ms for release.
module test;
void main() {
import devisualization.image;
import devisualization.image.mutable;
import devisualization.util.core.linegraph;
import std.stdio;
writeln(===\nREAD\n===);
Image img = imageFromFile(test/large.png);
img = new MutableImage(img);
import std.datetime : StopWatch;
StopWatch sw;
sw.start();
foreach(i; 0 .. 1000) {
img.flipHorizontal;
}
sw.stop();
writeln(Img flipped in: , sw.peek().msecs / 1000, [ms]);
}
I was planning on doing this earlier. But I discovered a PR I pulled
which fixed for 2.067 broke chunk types reading.
My bad, forgot I decreased test image resolution to 256x256. I'm
totally out of the running. I have some serious work to do by the
looks.
Have you considered just being able to grab an object with changed
iteration order instead of actually doing the flip? The same goes for
transposes and 90º rotations. Sure, sometimes you do need actually
rearrange the memory and in a subset of those cases you need it to be
done fast, but a lot of the time you're better off* just using a
different iteration scheme (which, for ranges, should probably be part
of the type to avoid checking the scheme every iteration).
*for speed and memory reasons. Need to keep the original and the
transpose? No need to for any duplicates
Note that this is what numpy does with transposes. The .T and .transpose
methods of ndarray don't actually modify the data, they just set the
memory order** whereas the transpose function actually moves memory
around.
**using a runtime flag, which is ok for them because internal iteration
lets you only branch once on it.
I've got it down to ~ 12ms using dmd now. But
Re: How to POST data with net.curl
On Thursday, 2 April 2015 at 07:45:16 UTC, Suliman wrote:
I have next request, that work fine with curl:
curl -X POST -F upload=@wgs84_latlon.zip
http://ogre.adc4gis.com/convert
I wrote next code:
void main()
{
auto binfile = cast(ubyte[])
read(`D:\Apps\curl\wgs84_latlon.zip`);
auto http = HTTP(http://ogre.adc4gis.com/convert;);
http.setPostData(binfile, -X POST -F upload=@);
http.onReceive = (ubyte[] data) { writeln((cast(string)data));
return data.length; };
http.perform();
}
But I have got trouble with http.setPostData. I do not know how
to properly write text part, and attaches binary zip file.
I had trouble with that too. Take a look at
https://github.com/Zalastax/AvanzaFundSimulator/blob/master/source/funds/fundfetcher.d
and
https://github.com/Zalastax/zalastaxlibd/blob/master/source/std/extra/http.d
to see if that can help.
Structs as template parameters: weird error message
The code below doesn't compile. Why this error message?
---
struct Item {
int i;
}
struct Params {
Item* item;
this(int i) {
item = new Item(i); // line 9
}
}
struct Foo(Params params) {}
enum foo = Foo!(Params(1));
---
test.d(9): Error: Item(1) is not an lvalue
Re: Speed of horizontal flip
On Thursday, 2 April 2015 at 09:55:15 UTC, Rikki Cattermole wrote:
On 2/04/2015 10:47 p.m., Rikki Cattermole wrote:
On 2/04/2015 2:52 a.m., tchaloupka wrote:
Hi,
I have a bunch of square r16 and png images which I need to
flip
horizontally.
My flip method looks like this:
void hFlip(T)(T[] data, int w)
{
import std.datetime : StopWatch;
StopWatch sw;
sw.start();
foreach(int i; 0..w)
{
auto row = data[i*w..(i+1)*w];
row.reverse();
}
sw.stop();
writeln(Img flipped in: , sw.peek().msecs, [ms]);
}
With simple r16 file format its pretty fast, but with RGB PNG
files (2048x2048) I noticed its somewhat slow so I tried to
compare it with C# and was pretty surprised by the results.
C#:
PNG load - 90ms
PNG flip - 10ms
PNG save - 380ms
D using dlib (http://code.dlang.org/packages/dlib):
PNG load - 500ms
PNG flip - 30ms
PNG save - 950ms
D using imageformats
(http://code.dlang.org/packages/imageformats):
PNG load - 230ms
PNG flip - 30ms
PNG save - 1100ms
I used dmd-2.0.67 with -release -inline -O
C# was just with debug and VisualStudio attached to process
for
debugging and even with that it is much faster.
I know that System.Drawing is using Windows GDI+, that can be
used with D too, but not on linux.
If we ignore the PNG loading and saving (didn't tried libpng
yet), even flip method itself is 3 times slower - I don't
know D
enough to be sure if there isn't some more effecient way to
make
the flip. I like how the slices can be used here.
For a C# user who is expecting things to just work as fast as
possible from a system level programming language this can be
somewhat disappointing to see that pure D version is about 3
times slower.
Am I doing something utterly wrong?
Note that this example is not critical for me, it's just a
simple
hobby script I use to move and flip some images - I can wait.
But
I post it to see if this can be taken somewhat closer to what
can
be expected from a system level programming language.
dlib:
auto im = loadPNG(name);
hFlip(cast(ubyte[3][])im.data, cast(int)im.width);
savePNG(im, newName);
imageformats:
auto im = read_image(name);
hFlip(cast(ubyte[3][])im.pixels, cast(int)im.w);
write_image(newName, im.w, im.h, im.pixels);
C# code:
static void Main(string[] args)
{
var files = Directory.GetFiles(args[0]);
foreach (var f in files)
{
var sw = Stopwatch.StartNew();
var img = Image.FromFile(f);
Debug.WriteLine(Img loaded in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Restart();
img.RotateFlip(RotateFlipType.RotateNoneFlipX);
Debug.WriteLine(Img flipped in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Restart();
img.Save(Path.Combine(args[0], test_ +
Path.GetFileName(f)));
Debug.WriteLine(Img saved in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Stop();
}
}
Assuming I've done it correctly, Devisualization.Image takes
around 8ms
in debug mode to flip horizontally using dmd. But 3ms for
release.
module test;
void main() {
import devisualization.image;
import devisualization.image.mutable;
import devisualization.util.core.linegraph;
import std.stdio;
writeln(===\nREAD\n===);
Image img = imageFromFile(test/large.png);
img = new MutableImage(img);
import std.datetime : StopWatch;
StopWatch sw;
sw.start();
foreach(i; 0 .. 1000) {
img.flipHorizontal;
}
sw.stop();
writeln(Img flipped in: , sw.peek().msecs / 1000,
[ms]);
}
I was planning on doing this earlier. But I discovered a PR I
pulled
which fixed for 2.067 broke chunk types reading.
My bad, forgot I decreased test image resolution to 256x256.
I'm totally out of the running. I have some serious work to do
by the looks.
Have you considered just being able to grab an object with
changed iteration order instead of actually doing the flip? The
same goes for transposes and 90º rotations. Sure, sometimes you
do need actually rearrange the memory and in a subset of those
cases you need it to be done fast, but a lot of the time you're
better off* just using a different iteration scheme (which, for
ranges, should probably be part of the type to avoid checking the
scheme every iteration).
*for speed and memory reasons. Need to keep the original and the
transpose? No need to for any duplicates
Note that this is what numpy does with transposes. The .T and
.transpose methods of ndarray don't actually modify the data,
they just set the memory order** whereas the transpose function
actually moves memory around.
**using a runtime flag, which is ok for them because internal
iteration lets you only branch once on it.
Re: Speed of horizontal flip
On 3/04/2015 12:29 a.m., John Colvin wrote:
On Thursday, 2 April 2015 at 09:55:15 UTC, Rikki Cattermole wrote:
On 2/04/2015 10:47 p.m., Rikki Cattermole wrote:
On 2/04/2015 2:52 a.m., tchaloupka wrote:
Hi,
I have a bunch of square r16 and png images which I need to flip
horizontally.
My flip method looks like this:
void hFlip(T)(T[] data, int w)
{
import std.datetime : StopWatch;
StopWatch sw;
sw.start();
foreach(int i; 0..w)
{
auto row = data[i*w..(i+1)*w];
row.reverse();
}
sw.stop();
writeln(Img flipped in: , sw.peek().msecs, [ms]);
}
With simple r16 file format its pretty fast, but with RGB PNG
files (2048x2048) I noticed its somewhat slow so I tried to
compare it with C# and was pretty surprised by the results.
C#:
PNG load - 90ms
PNG flip - 10ms
PNG save - 380ms
D using dlib (http://code.dlang.org/packages/dlib):
PNG load - 500ms
PNG flip - 30ms
PNG save - 950ms
D using imageformats
(http://code.dlang.org/packages/imageformats):
PNG load - 230ms
PNG flip - 30ms
PNG save - 1100ms
I used dmd-2.0.67 with -release -inline -O
C# was just with debug and VisualStudio attached to process for
debugging and even with that it is much faster.
I know that System.Drawing is using Windows GDI+, that can be
used with D too, but not on linux.
If we ignore the PNG loading and saving (didn't tried libpng
yet), even flip method itself is 3 times slower - I don't know D
enough to be sure if there isn't some more effecient way to make
the flip. I like how the slices can be used here.
For a C# user who is expecting things to just work as fast as
possible from a system level programming language this can be
somewhat disappointing to see that pure D version is about 3
times slower.
Am I doing something utterly wrong?
Note that this example is not critical for me, it's just a simple
hobby script I use to move and flip some images - I can wait. But
I post it to see if this can be taken somewhat closer to what can
be expected from a system level programming language.
dlib:
auto im = loadPNG(name);
hFlip(cast(ubyte[3][])im.data, cast(int)im.width);
savePNG(im, newName);
imageformats:
auto im = read_image(name);
hFlip(cast(ubyte[3][])im.pixels, cast(int)im.w);
write_image(newName, im.w, im.h, im.pixels);
C# code:
static void Main(string[] args)
{
var files = Directory.GetFiles(args[0]);
foreach (var f in files)
{
var sw = Stopwatch.StartNew();
var img = Image.FromFile(f);
Debug.WriteLine(Img loaded in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Restart();
img.RotateFlip(RotateFlipType.RotateNoneFlipX);
Debug.WriteLine(Img flipped in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Restart();
img.Save(Path.Combine(args[0], test_ +
Path.GetFileName(f)));
Debug.WriteLine(Img saved in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Stop();
}
}
Assuming I've done it correctly, Devisualization.Image takes around 8ms
in debug mode to flip horizontally using dmd. But 3ms for release.
module test;
void main() {
import devisualization.image;
import devisualization.image.mutable;
import devisualization.util.core.linegraph;
import std.stdio;
writeln(===\nREAD\n===);
Image img = imageFromFile(test/large.png);
img = new MutableImage(img);
import std.datetime : StopWatch;
StopWatch sw;
sw.start();
foreach(i; 0 .. 1000) {
img.flipHorizontal;
}
sw.stop();
writeln(Img flipped in: , sw.peek().msecs / 1000, [ms]);
}
I was planning on doing this earlier. But I discovered a PR I pulled
which fixed for 2.067 broke chunk types reading.
My bad, forgot I decreased test image resolution to 256x256. I'm
totally out of the running. I have some serious work to do by the looks.
Have you considered just being able to grab an object with changed
iteration order instead of actually doing the flip? The same goes for
transposes and 90º rotations. Sure, sometimes you do need actually
rearrange the memory and in a subset of those cases you need it to be
done fast, but a lot of the time you're better off* just using a
different iteration scheme (which, for ranges, should probably be part
of the type to avoid checking the scheme every iteration).
*for speed and memory reasons. Need to keep the original and the
transpose? No need to for any duplicates
Note that this is what numpy does with transposes. The .T and .transpose
methods of ndarray don't actually modify the data, they just set the
memory order** whereas the transpose function actually moves memory around.
**using a runtime flag, which is ok for them because internal iteration
lets you only branch once on it.
I've got it down to ~ 12ms using dmd now. But if the image was much
bigger (lets say a height of ushort.max). I
Re: Speed of horizontal flip
On 2/04/2015 10:47 p.m., Rikki Cattermole wrote:
On 2/04/2015 2:52 a.m., tchaloupka wrote:
Hi,
I have a bunch of square r16 and png images which I need to flip
horizontally.
My flip method looks like this:
void hFlip(T)(T[] data, int w)
{
import std.datetime : StopWatch;
StopWatch sw;
sw.start();
foreach(int i; 0..w)
{
auto row = data[i*w..(i+1)*w];
row.reverse();
}
sw.stop();
writeln(Img flipped in: , sw.peek().msecs, [ms]);
}
With simple r16 file format its pretty fast, but with RGB PNG
files (2048x2048) I noticed its somewhat slow so I tried to
compare it with C# and was pretty surprised by the results.
C#:
PNG load - 90ms
PNG flip - 10ms
PNG save - 380ms
D using dlib (http://code.dlang.org/packages/dlib):
PNG load - 500ms
PNG flip - 30ms
PNG save - 950ms
D using imageformats
(http://code.dlang.org/packages/imageformats):
PNG load - 230ms
PNG flip - 30ms
PNG save - 1100ms
I used dmd-2.0.67 with -release -inline -O
C# was just with debug and VisualStudio attached to process for
debugging and even with that it is much faster.
I know that System.Drawing is using Windows GDI+, that can be
used with D too, but not on linux.
If we ignore the PNG loading and saving (didn't tried libpng
yet), even flip method itself is 3 times slower - I don't know D
enough to be sure if there isn't some more effecient way to make
the flip. I like how the slices can be used here.
For a C# user who is expecting things to just work as fast as
possible from a system level programming language this can be
somewhat disappointing to see that pure D version is about 3
times slower.
Am I doing something utterly wrong?
Note that this example is not critical for me, it's just a simple
hobby script I use to move and flip some images - I can wait. But
I post it to see if this can be taken somewhat closer to what can
be expected from a system level programming language.
dlib:
auto im = loadPNG(name);
hFlip(cast(ubyte[3][])im.data, cast(int)im.width);
savePNG(im, newName);
imageformats:
auto im = read_image(name);
hFlip(cast(ubyte[3][])im.pixels, cast(int)im.w);
write_image(newName, im.w, im.h, im.pixels);
C# code:
static void Main(string[] args)
{
var files = Directory.GetFiles(args[0]);
foreach (var f in files)
{
var sw = Stopwatch.StartNew();
var img = Image.FromFile(f);
Debug.WriteLine(Img loaded in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Restart();
img.RotateFlip(RotateFlipType.RotateNoneFlipX);
Debug.WriteLine(Img flipped in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Restart();
img.Save(Path.Combine(args[0], test_ +
Path.GetFileName(f)));
Debug.WriteLine(Img saved in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Stop();
}
}
Assuming I've done it correctly, Devisualization.Image takes around 8ms
in debug mode to flip horizontally using dmd. But 3ms for release.
module test;
void main() {
import devisualization.image;
import devisualization.image.mutable;
import devisualization.util.core.linegraph;
import std.stdio;
writeln(===\nREAD\n===);
Image img = imageFromFile(test/large.png);
img = new MutableImage(img);
import std.datetime : StopWatch;
StopWatch sw;
sw.start();
foreach(i; 0 .. 1000) {
img.flipHorizontal;
}
sw.stop();
writeln(Img flipped in: , sw.peek().msecs / 1000, [ms]);
}
I was planning on doing this earlier. But I discovered a PR I pulled
which fixed for 2.067 broke chunk types reading.
My bad, forgot I decreased test image resolution to 256x256. I'm totally
out of the running. I have some serious work to do by the looks.
Re: C++ to D
On Wednesday, 1 April 2015 at 17:51:40 UTC, John Colvin wrote:
Don't really see the point. Here's a neat thing that's
definitely cheating because although it stores the results in
the type system, the arithmetic is done in constant-folding:
struct Integer(int a){}
template Value(T)
{
static if (is(T == Integer!a, int a))
enum Value = a;
else static assert(false, Can't get Value for ~
T.stringof);
}
alias Inc(T) = Integer!(Value!T + 1);
But if you really insist on it being all type-system (until you
wan't the answer of course):
struct Zero{}
template Succ(T)
{
static if (is(T == Pred!A, A))
alias Succ = A;
else
struct Succ{}
}
template Pred(T)
{
static if (is(T == Succ!A, A))
alias Pred = A;
else
struct Pred{}
}
enum isPositive(T) = is(T == Succ!A, A);
enum isNegative(T) = is(T == Pred!A, A);
enum isZero(T) = is(T == Zero);
template Add(A, B)
{
static if (isZero!B)
alias Add = A;
else static if (isPositive!B)
alias Add = Add!(Succ!A, Pred!B);
else
alias Add = Add!(Pred!A, Succ!B);
}
template Value(T, int seed = 0)
{
static if (isZero!T)
enum Value = seed;
else static if (isPositive!T)
enum Value = Value!(Pred!T, seed+1);
else
enum Value = Value!(Succ!T, seed-1);
}
unittest
{
alias One = Succ!Zero;
alias Two = Succ!One;
alias MinusThree = Pred!(Pred!(Pred!Zero));
static assert (Value!Zero == 0);
static assert (Value!One == 1);
static assert (Value!Two == 2);
static assert (Value!MinusThree == -3);
static assert (Value!(Add!(One, MinusThree)) == -2);
static assert (Value!(Add!(One, Two)) == 3);
static assert (is(Add!(Add!(One, Two), MinusThree) == Zero));
}
Thanks.
Re: Speed of horizontal flip
On Thursday, 2 April 2015 at 05:21:08 UTC, thedeemon wrote: std.algorithm.reverse uses ranges, and shamefully DMD is really bad at optimizing away range-induced costs. The specialisation of reverse selected for slices does not use the range interface, it's all just indexing. The only overheads come from: a) function calls, if the inliner isn't doing its job (which it really should be in these cases). b) a check for aliasing in swapAt, which is only done for ranges of static arrays. Again, should be optimised away in this case, but it's possible DMD doesn't manage it. Either way, it's a trivially predictable branch and should be effectively free at the CPU level. Once you've got past those, it's just straight loop I posted before.
Re: Speed of horizontal flip
On Thursday, 2 April 2015 at 11:49:44 UTC, Rikki Cattermole wrote:
On 3/04/2015 12:29 a.m., John Colvin wrote:
On Thursday, 2 April 2015 at 09:55:15 UTC, Rikki Cattermole
wrote:
On 2/04/2015 10:47 p.m., Rikki Cattermole wrote:
On 2/04/2015 2:52 a.m., tchaloupka wrote:
Hi,
I have a bunch of square r16 and png images which I need to
flip
horizontally.
My flip method looks like this:
void hFlip(T)(T[] data, int w)
{
import std.datetime : StopWatch;
StopWatch sw;
sw.start();
foreach(int i; 0..w)
{
auto row = data[i*w..(i+1)*w];
row.reverse();
}
sw.stop();
writeln(Img flipped in: , sw.peek().msecs, [ms]);
}
With simple r16 file format its pretty fast, but with RGB
PNG
files (2048x2048) I noticed its somewhat slow so I tried to
compare it with C# and was pretty surprised by the results.
C#:
PNG load - 90ms
PNG flip - 10ms
PNG save - 380ms
D using dlib (http://code.dlang.org/packages/dlib):
PNG load - 500ms
PNG flip - 30ms
PNG save - 950ms
D using imageformats
(http://code.dlang.org/packages/imageformats):
PNG load - 230ms
PNG flip - 30ms
PNG save - 1100ms
I used dmd-2.0.67 with -release -inline -O
C# was just with debug and VisualStudio attached to process
for
debugging and even with that it is much faster.
I know that System.Drawing is using Windows GDI+, that can
be
used with D too, but not on linux.
If we ignore the PNG loading and saving (didn't tried libpng
yet), even flip method itself is 3 times slower - I don't
know D
enough to be sure if there isn't some more effecient way to
make
the flip. I like how the slices can be used here.
For a C# user who is expecting things to just work as fast
as
possible from a system level programming language this can
be
somewhat disappointing to see that pure D version is about 3
times slower.
Am I doing something utterly wrong?
Note that this example is not critical for me, it's just a
simple
hobby script I use to move and flip some images - I can
wait. But
I post it to see if this can be taken somewhat closer to
what can
be expected from a system level programming language.
dlib:
auto im = loadPNG(name);
hFlip(cast(ubyte[3][])im.data, cast(int)im.width);
savePNG(im, newName);
imageformats:
auto im = read_image(name);
hFlip(cast(ubyte[3][])im.pixels, cast(int)im.w);
write_image(newName, im.w, im.h, im.pixels);
C# code:
static void Main(string[] args)
{
var files = Directory.GetFiles(args[0]);
foreach (var f in files)
{
var sw = Stopwatch.StartNew();
var img = Image.FromFile(f);
Debug.WriteLine(Img loaded in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Restart();
img.RotateFlip(RotateFlipType.RotateNoneFlipX);
Debug.WriteLine(Img flipped in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Restart();
img.Save(Path.Combine(args[0], test_ +
Path.GetFileName(f)));
Debug.WriteLine(Img saved in {0}[ms],
(int)sw.Elapsed.TotalMilliseconds);
sw.Stop();
}
}
Assuming I've done it correctly, Devisualization.Image takes
around 8ms
in debug mode to flip horizontally using dmd. But 3ms for
release.
module test;
void main() {
import devisualization.image;
import devisualization.image.mutable;
import devisualization.util.core.linegraph;
import std.stdio;
writeln(===\nREAD\n===);
Image img = imageFromFile(test/large.png);
img = new MutableImage(img);
import std.datetime : StopWatch;
StopWatch sw;
sw.start();
foreach(i; 0 .. 1000) {
img.flipHorizontal;
}
sw.stop();
writeln(Img flipped in: , sw.peek().msecs / 1000,
[ms]);
}
I was planning on doing this earlier. But I discovered a PR
I pulled
which fixed for 2.067 broke chunk types reading.
My bad, forgot I decreased test image resolution to 256x256.
I'm
totally out of the running. I have some serious work to do by
the looks.
Have you considered just being able to grab an object with
changed
iteration order instead of actually doing the flip? The same
goes for
transposes and 90º rotations. Sure, sometimes you do need
actually
rearrange the memory and in a subset of those cases you need
it to be
done fast, but a lot of the time you're better off* just using
a
different iteration scheme (which, for ranges, should probably
be part
of the type to avoid checking the scheme every iteration).
*for speed and memory reasons. Need to keep the original and
the
transpose? No need to for any duplicates
Note that this is what numpy does with transposes. The .T and
.transpose
methods of ndarray don't actually modify the data, they just
set the
memory order** whereas the transpose function actually moves
memory around.
**using a runtime flag, which is ok for them because internal
iteration
lets you only branch once on it.
I've got it down to ~ 12ms using dmd now. But
USB Controller
Hey Guys! I've started using Arduino, specifically with Processing. I'm using it essentially as a usb controller. Is there a good way to translate this to D? There doesn't seem to be any standard library for usb usage. Are there pre-built libraries or is there C libraries I should use? Thanks Josh
