On Tuesday, 16 January 2018 at 23:01:19 UTC, H. S. Teoh wrote:
On Tue, Jan 16, 2018 at 05:49:11PM +0000, Dmitry Olshansky via
Digitalmars-d wrote:
On Monday, 15 January 2018 at 19:52:07 UTC, H. S. Teoh wrote:
[...]
> One thing I'm seeking help with, and this is mainly directed
> at Dmitry Olshansky but can be anyone here who knows the
> internal workings of std.uni well enough, is how to
> transform the Trie generated by the static ctor into
> compile-time TrieNode declarations. This is one blocker for
> my turning this code into a Phobos PR, because I don't want
> to incur the cost of initializing this trie at runtime.
Checkout my horribly named repo gsoc-uni-benchmark:
https://github.com/DmitryOlshansky/gsoc-bench-2012/blob/master/gen_uni.d
This is what generates unicode tables.
Need to revise it, as folks were delicate enough to hand-patch
auto-generated code in Phobos.
Maybe make some of that user-acessible.
[...]
Whoa. There's some pretty cool stuff in there! Thanks, I've
started experimenting with pre-generating the width table.
Pretty neat. There's a lot of hidden gems in std.uni that I
never knew existed, hidden away under `private`. :-D
The intent is to open that up somehow, to allow folks to make
their own extended versions of std.uni. Unicode is all about
“tailoring” - adjusting algorithm to your specific regional
preferences hy messing with tables.
I think there is at least 1 bug in Bugzilla on this.
One thing, though: I think it would benefit us all if we could
import at least gen_uni into Phobos, so that in the future when
we need to update std.uni to a new version of Unicode, it can
be (mostly) automated. It's better to have the tools to
generate the tables in Phobos itself, than to be dependent on
an external repo that may go out-of-sync eventually.
Yes but it’s non-trivial at the moment, if you take a look at
script to generate stuff it takes both 32-bit and 64-bit
executables to populate tables.
I think having it in tools repo should be fine though. Last time
I tried to update to Unicode 10, I found one table in Phobos that
is missing from generator (ooops!).
When I get around to making a PR for strwidth AKA displayWidth,
the plan is to check-in compileWidth.d in some form into Phobos
somewhere, so that somebody else can pick it up and improve the
implementation in the future if I'm not around / unavailable.
If we can get gen_uni into Phobos, perhaps we can even include
the displayWidth table generation in gen_uni too, so that all
the table generation code is in one place.
Right. A good step would be to move it to tools, then add your
code.
T
