Re: Reading text (I mean "real" text...)
On Saturday, 20 June 2020 at 01:41:50 UTC, Paul Backus wrote: It sounds like maybe what you are looking for is Unicode character categories: https://en.wikipedia.org/wiki/Unicode_character_property#General_Category The character validation step could indeed be expressed using Unicode properties: Allow Unicode White_Space Reject Unicode Control Allow everything else
Re: Reading text (I mean "real" text...)
On Saturday, 20 June 2020 at 01:35:56 UTC, Denis wrote: THE OBJECTIVE The objective is to read a file one line at a time (reading each line into a string), while checking for human-readable text character by character. Invalid characters (control and UTF-8) should generate an exception. Unless there's already an existing function that works as described, I'd like to write one. I expect that this will require combining an existing read-by-UTF8-char or read-by-byte function with the additional validation. It sounds like maybe what you are looking for is Unicode character categories: https://en.wikipedia.org/wiki/Unicode_character_property#General_Category
Reading text (I mean "real" text...)
THE PROBLEM UTF-8 validation alone is insufficient for ensuring that a file contains only human-readable text, because control characters are UTF-8 valid. Apart from tab, newline, carriage return, and a few less commonly used others considered to be whitespace, human-readable text files should not normally contain embedded control characters. In the standard library, the read functions for "text" files (as opposed to binary files) that I looked at are not actually based on "human readable text", but on "characters". For example: - In std.stdio, readln accepts everything. Lines are simply separated by the occurrence of a newline or user-designated character. - In std.file, readText accepts all valid UTF-8 characters. This means, for example, that all of these functions will happily try to read an enormous file of zeroes in its entirety (something that should not even be considered "text") into a string variable, on the very first call to the read function. Not good... Whereas a function that reads only "human-readable text" should instead generate an exception immediately upon encountering an invalid control character or invalid UTF-8 character. THE OBJECTIVE The objective is to read a file one line at a time (reading each line into a string), while checking for human-readable text character by character. Invalid characters (control and UTF-8) should generate an exception. Unless there's already an existing function that works as described, I'd like to write one. I expect that this will require combining an existing read-by-UTF8-char or read-by-byte function with the additional validation. Q1: Which existing functions (D or C) would you suggest leveraging? For example, there are quite a few variants of "read" and in different libraries too. For a newcomer, it can be difficult to intuit which one is best suited for what. Q2: Any source code (D or C) you might suggest I look at, to get ideas for how parts of this could be written? Thanks for your help.
Re: Arduino and MCU Support
On Friday, 19 June 2020 at 11:57:01 UTC, frasdoge wrote: I am looking to use D for microcontroller programming due to its benefits over C in workflow and general language features. I was wondering what the current state of this is, especially with regards to AVR. An example of the MCUs I would like to develop with include anything from 8 bit ATmega328p to 32 bit ESP32. The ESP32 can be programmed in C, C++, micropython and Lua, so I'm hoping there's at least some compatibility there. Once you get it to work, please document something for use by others. Its quite unfortunate this is not well documented.
Re: Should the compiler be failing to infer template args here?
On 6/19/20 12:38 PM, SealabJaster wrote:
On Friday, 19 June 2020 at 16:31:50 UTC, Paul Backus wrote:
This is a known issue:
https://issues.dlang.org/show_bug.cgi?id=1807
"Reported: 2008"... yikes.
Thanks anyway, glad to know I wasn't just going mad :)
It's somewhat difficult to solve, because reverse-templating is not
necessarily one-to-one, or even have any tractible relationship at all.
Consider, for instance:
struct Foo(T) {}
alias Bar(T) = Foo!int;
void takefoo(X)(Bar!X) {}
takefoo(Foo!int());
How does the compiler decide what type T should be? It could be anything.
But this could be special cased for a *direct* alias translation. How to
define that is tricky, but should match the pattern:
alias Template1(T) = Template2!(T, and, possibly, other, things)
Where Template2!(T, and, possibly, other, things) can be deduced for
Template2 and T using IFTI directly.
As the age of that bug report shows, it's not easy to define or solve.
-Steve
Re: Should the compiler be failing to infer template args here?
On Friday, 19 June 2020 at 16:31:50 UTC, Paul Backus wrote: This is a known issue: https://issues.dlang.org/show_bug.cgi?id=1807 "Reported: 2008"... yikes. Thanks anyway, glad to know I wasn't just going mad :)
Re: Should the compiler be failing to infer template args here?
On Friday, 19 June 2020 at 16:16:55 UTC, SealabJaster wrote: If you take a look at this code here: https://godbolt.org/z/4T3uLh You can see that when using a templated alias, the compiler fails to infer the T template parameter, but only when using the function that also asks for the alias, instead of the original type. I was just wondering if this is correct behavior, since it's a slightly annoying, but easy to deal with difference. This is a known issue: https://issues.dlang.org/show_bug.cgi?id=1807
Should the compiler be failing to infer template args here?
If you take a look at this code here: https://godbolt.org/z/4T3uLh You can see that when using a templated alias, the compiler fails to infer the T template parameter, but only when using the function that also asks for the alias, instead of the original type. I was just wondering if this is correct behavior, since it's a slightly annoying, but easy to deal with difference.
Re: Parallel array append using std.parallelism?
On Fri, Jun 19, 2020 at 06:48:18AM +, Simen Kjærås via Digitalmars-d-learn
wrote:
[...]
> There's an example of exactly this in std.parallelism:
> https://dlang.org/phobos/std_parallelism.html#.TaskPool.workerIndex
>
> In short:
>
> Item[] targetArray = ...; // already contains data
> // Get thread count from taskPool
> Item[][] tmp = new Item[][taskPool.size+1];
> foreach (elem; input.parallel) {
> if (condition(elem)) {
> auto output = expensiveComputation(elem);
> // Use workerIndex as index
> tmp[taskPool.workerIndex] ~= output;
> }
> }
> foreach (a; tmp)
> targetArray ~= a;
[...]
Yes, that's exactly what I was looking for. Thanks!!
T
--
The best way to destroy a cause is to defend it poorly.
Re: Arduino and MCU Support
On Friday, 19 June 2020 at 14:14:07 UTC, frasdoge wrote: though the -mcpu does not have any AVR options out-of-the-box as you mentioned. I guess you mean `ldc2 -mcpu=help` doesn't list any AVR CPUs. Use `ldc2 -mtriple=avr -mcpu=help`.
Re: Arduino and MCU Support
On Friday, 19 June 2020 at 14:14:07 UTC, frasdoge wrote: I'm having a bit of trouble understanding how to actually get started even with those links. I've installed the latest LDC and LLVM releases for Windows, though the -mcpu does not have any AVR options out-of-the-box as you mentioned. Perhaps I've missed something, or will I need to build the binary myself using the flags for AVR? Compiling the sample from the AVR wiki works here on Win64 with the official LDC 1.22 package and: ldc2 -betterC -Oz -mtriple=avr -mcpu=atmega328p -c test.d For linking, you'll need an avr-gcc toolchain, and add `-gcc= -Xcc=-mmcu=atmega328p` as mentioned on the Wiki page. I don't have such a toolchain. The ESP32 process seems more involved. Due to my inexperience in this area, I'm wondering that, if I need to build the binaries myself anyway, how I would set it such that it'll work with AVR and ESP32 at the same time with one installation? Simply using -mtriple and tweaking etc\ldc2.conf as explained here: https://wiki.dlang.org/Cross-compiling_with_LDC
Re: Arduino and MCU Support
On Friday, 19 June 2020 at 12:20:52 UTC, kinke wrote: AVR: https://wiki.dlang.org/D_on_AVR With recent official LDC packages, you don't need to build LLVM and LDC yourself, AVR is supported out-of-the-box. ESP32: https://wiki.dlang.org/D_on_esp32/esp8266(llvm-xtensa%2Bldc)_and_how_to_get_started This arch hasn't been upstreamed to LLVM yet and so needs their LLVM fork, i.e., building it and LDC yourself. For some more infos, use the search function of this forum. Thanks for the reply. I'm having a bit of trouble understanding how to actually get started even with those links. I've installed the latest LDC and LLVM releases for Windows, though the -mcpu does not have any AVR options out-of-the-box as you mentioned. Perhaps I've missed something, or will I need to build the binary myself using the flags for AVR? If you could explain in greater detail I would very much appreciate it, as there is *very* little documentation elsewhere or even on this forum. The ESP32 process seems more involved. Due to my inexperience in this area, I'm wondering that, if I need to build the binaries myself anyway, how I would set it such that it'll work with AVR and ESP32 at the same time with one installation? I'm hoping to have an analogous experience as with C, where I simply write my code, specify my cpu target, and then program over serial.
Re: Why is there no std.stream anymore?
On 6/18/20 11:11 PM, Jesse Phillips wrote: On Thursday, 18 June 2020 at 14:53:58 UTC, aberba wrote: On Tuesday, 12 December 2017 at 20:51:30 UTC, Steven Schveighoffer wrote: On 12/11/17 6:33 PM, Seb wrote: [...] Since iopipe was mentioned several times, I will say a couple things: [...] I should really try iopipe this time round. I think I avoided toying with it because the making conventions put me off. Don't remember about the docs and available examples though. I too was trying to utilize iopipe and asked questions earlier this year[1] and I made a file writer util[2]. I haven't really taken advantage of the power yet, though it does appear that be a really nice abstraction. 1. https://forum.dlang.org/thread/faawejguebluwodfl...@forum.dlang.org 2. https://gitlab.com/jessephillips/devarticlator/-/blob/master/source/util/file.d I reread my comment and in particular this: "I was going to write an ascii art concept to show how the pushing works, but I think I'll maybe draw an actual picture. I need some time to accomplish this, though." I still haven't done this. I need to make this happen, probably with a blog post (Mike, keep bugging me on this). -Steve
Re: Arduino and MCU Support
On Friday, 19 June 2020 at 11:57:01 UTC, frasdoge wrote: I am looking to use D for microcontroller programming due to its benefits over C in workflow and general language features. I was wondering what the current state of this is, especially with regards to AVR. An example of the MCUs I would like to develop with include anything from 8 bit ATmega328p to 32 bit ESP32. The ESP32 can be programmed in C, C++, micropython and Lua, so I'm hoping there's at least some compatibility there. AVR: https://wiki.dlang.org/D_on_AVR With recent official LDC packages, you don't need to build LLVM and LDC yourself, AVR is supported out-of-the-box. ESP32: https://wiki.dlang.org/D_on_esp32/esp8266(llvm-xtensa%2Bldc)_and_how_to_get_started This arch hasn't been upstreamed to LLVM yet and so needs their LLVM fork, i.e., building it and LDC yourself. For some more infos, use the search function of this forum.
Arduino and MCU Support
I am looking to use D for microcontroller programming due to its benefits over C in workflow and general language features. I was wondering what the current state of this is, especially with regards to AVR. An example of the MCUs I would like to develop with include anything from 8 bit ATmega328p to 32 bit ESP32. The ESP32 can be programmed in C, C++, micropython and Lua, so I'm hoping there's at least some compatibility there.
Re: "if not" condition check (for data validation)
On Thursday, 18 June 2020 at 17:39:44 UTC, Denis wrote: I should add that this one made me laugh though, giving flashbacks to that horrible "not speak" of the early 90s: if ( configfile.isFile.not ) ... LOL Approve Yoda does.
