Re: [dev] Suckless CMatrix clone
On Fri, Jun 17, 2022 at 3:00 AM Laslo Hunhold wrote: > I pushed line-break-detection[0][1] just yesterday. The functions > > size_t grapheme_next_line_break(const uint_least32_t *, size_t); > > and > > size_t grapheme_next_line_break_utf8(const char *, size_t); > > should be just the ones you'll need, depending on your internal > representation. It will give you the respective next line break > opportunity. Thank you. I'll look into adding it as an alternative to Tuxmath dependencies once it's released. I also added libgrapheme to the Delightful C Software list on Codeberg.
Re: [dev] Suckless CMatrix clone
On Tue, 31 May 2022 08:35:11 -0400 LM wrote: Dear LM, > On Mon, May 30, 2022 at 11:33 AM Laslo Hunhold wrote: > > What functions do you need in the context of Tuxmath? > > From what I remember, it just needs to figure out where to do a clean > break for text wrapping with internationalized strings. Don't believe > it uses the other features of libunistring. I've personally been > trying to avoid gettext and use a port/fork of BSD's Citrus Project ( > http://citrus.bsdclub.org/ ) for accessing the translated strings. > Just need a way to break the strings up in the appropriate locations > for wrapping text and displaying on screen. If I get that working, I > can replace or at least offer an alternative to the libunistring > dependency. I pushed line-break-detection[0][1] just yesterday. The functions size_t grapheme_next_line_break(const uint_least32_t *, size_t); and size_t grapheme_next_line_break_utf8(const char *, size_t); should be just the ones you'll need, depending on your internal representation. It will give you the respective next line break opportunity. After a bit of cleanup, refactoring and writing manuals for all functions I'll tag a release. With best regards Laslo [0]:https://git.suckless.org/libgrapheme/commit/0aaf3fcc7bb84fe97b8d75c9d5d14305d54d4dfd.html [1]:https://git.suckless.org/libgrapheme/file/grapheme.h.html
Re: [dev] Suckless CMatrix clone
On Mon, May 30, 2022 at 11:33 AM Laslo Hunhold wrote: > What functions do you need in the context of Tuxmath? >From what I remember, it just needs to figure out where to do a clean break for text wrapping with internationalized strings. Don't believe it uses the other features of libunistring. I've personally been trying to avoid gettext and use a port/fork of BSD's Citrus Project ( http://citrus.bsdclub.org/ ) for accessing the translated strings. Just need a way to break the strings up in the appropriate locations for wrapping text and displaying on screen. If I get that working, I can replace or at least offer an alternative to the libunistring dependency. Thanks.
Re: [dev] Suckless CMatrix clone
On Mon, 30 May 2022 07:33:24 -0400 LM wrote: Dear LM, > Thanks for the reviews. That's really helpful to know. As mentioned, > I haven't tried them myself. you're welcome! > That looks really useful. I noticed the break testing in libgrapheme. > Is it possible to use this as a replacement for libunistring? I've > been doing some work on Tuxmath. I updated the code from what they'd > copied from an old version of gettext to using the libunistring > package which was supposed to replace it. Would be nice to offer an > alternative to libunistring as another compilation option. They > mostly use the code to figure out where to put line breaks when > dealing with internationalized text. libunistring's Unicode implementation sadly is incorrect in some aspects, because they based their data-structures and API on a very old Unicode standard where, for example, grapheme cluster break detection was still stateless. It has become stateful since. Other than that, it depends on what you want to do, given libunistring offers much more than text segmentation. I'm still reflecting on how I want to implement case conversion and detection in libgrapheme and hadn't had the time to work on it yet. What functions do you need in the context of Tuxmath? With best regards Laslo
Re: [dev] Suckless CMatrix clone
On Mon, May 30, 2022 at 4:56 AM Laslo Hunhold wrote: > having dove deep into UTF-8 and Unicode, I can at least say that > libutf8proc has an unsafe UTF-8-decoder, as it doesn't catch overlong > encodings. There are also multiple other pitfalls. Thanks for the reviews. That's really helpful to know. As mentioned, I haven't tried them myself. > I can shamelessly recommend you my UTF-8-codec[0] that's part of my > libgrapheme[1]-library, which also allows you to directly count > grapheme clusters (i.e. visible character units made up of one or more > codepoints). That looks really useful. I noticed the break testing in libgrapheme. Is it possible to use this as a replacement for libunistring? I've been doing some work on Tuxmath. I updated the code from what they'd copied from an old version of gettext to using the libunistring package which was supposed to replace it. Would be nice to offer an alternative to libunistring as another compilation option. They mostly use the code to figure out where to put line breaks when dealing with internationalized text. Thanks again.
Re: [dev] Suckless CMatrix clone
On Sun, 29 May 2022 13:48:49 -0400 LM wrote: Dear LM, > I like that point. Not a fan of glib and I try to avoid software > that uses it. > > Don't know how good they are, but I've run across several lighter > utf-8 C libraries: > https://github.com/cls/libutf > https://github.com/JuliaStrings/utf8proc > https://github.com/skeeto/branchless-utf8 > https://github.com/sheredom/utf8.h > https://github.com/JulienPalard/is_utf8 > > I wrote my own and use it, so I haven't tested these. Thought they > were interesting though. having dove deep into UTF-8 and Unicode, I can at least say that libutf8proc has an unsafe UTF-8-decoder, as it doesn't catch overlong encodings. There are also multiple other pitfalls. I can shamelessly recommend you my UTF-8-codec[0] that's part of my libgrapheme[1]-library, which also allows you to directly count grapheme clusters (i.e. visible character units made up of one or more codepoints). libutf8proc also offers grapheme cluster counting (among other things, but also has the unsafe UTF-8-decoder) and used to be the fastest library out there, but with a few tricks (much smaller LUTs) I managed to make libgrapheme twice as fast. I did a lot of benchmarking and tweaking and don't see any more room for improvement in the codec, given you have branches for all the edge-cases. The branchless UTF-8-decoder is very interesting, but may lead to a buffer overrun. With best regards Laslo [0]:https://git.suckless.org/libgrapheme/file/src/utf8.c.html [1]:https://git.suckless.org/libgrapheme/
Re: [dev] Suckless CMatrix clone
On Sun, May 29, 2022 at 12:34 PM Kyryl Melekhin wrote: > Cool, but why use Glib? You can come up with some random utf-8 character > generator function, it's pretty trivial and there are plently of examples > in many suckless programs on how to do utf-8 validation. I like that point. Not a fan of glib and I try to avoid software that uses it. Don't know how good they are, but I've run across several lighter utf-8 C libraries: https://github.com/cls/libutf https://github.com/JuliaStrings/utf8proc https://github.com/skeeto/branchless-utf8 https://github.com/sheredom/utf8.h https://github.com/JulienPalard/is_utf8 I wrote my own and use it, so I haven't tested these. Thought they were interesting though. Also, Codeberg ( https://codeberg.org/ and https://docs.codeberg.org/getting-started/what-is-codeberg/ ) was recommended to me as an alternative to github for hosting code. So far, it's been easier to work with than github. They were very responsive to my support related questions. Might be worth checking out. Best of luck on the project, Ryan.
Re: [dev] Suckless CMatrix clone
Hello Ryan, Ryan Raymond wrote: > Hello all. I'm working on a suckless cmatrix clone. > It's not done yet, and it still suffers from constant memory leakage, but > I'm having fun working on it. > It already shows 50% reduced cpu/ram usage vs cmatrix, despite greatly > improved fps. Furthermore, because it doesn't rely on ncurses, it respects > terminal transparency which is super cool. It works especially well with > our own st/background-image patch. > I just wanted to know if it would be possible to host this on suckless.org > in the future, and what I would have to do to do that. > I still have so much work to do. but I'm having a blast. This is by far my > largest original computer program, and my first attempt at an open-source > project. > Thanks. Cool, but why use Glib? You can come up with some random utf-8 character generator function, it's pretty trivial and there are plently of examples in many suckless programs on how to do utf-8 validation. Oh and yeah, the program works but it's weird, the screen is flickering and not smooth like cmatrix. So yeah, report back when you master the program and make it perfect. You're on the right track, though maybe a few years til the art of programming in C really starts to click in. Regards, Kyryl
Re: [dev] Suckless CMatrix clone
Hi Ryan, On Sun, May 29, 2022 at 03:04:00AM -0400, Ryan Raymond wrote: > Hello all. I'm working on a suckless cmatrix clone. > It's not done yet, and it still suffers from constant memory leakage, but > I'm having fun working on it. > It already shows 50% reduced cpu/ram usage vs cmatrix, despite greatly > improved fps. Furthermore, because it doesn't rely on ncurses, it respects > terminal transparency which is super cool. It works especially well with > our own st/background-image patch. > I just wanted to know if it would be possible to host this on suckless.org > in the future, and what I would have to do to do that. For reasons, like security I prefer to not host it. You could host it on your website and link to it from the suckless wiki on the related projects page. > I still have so much work to do. but I'm having a blast. This is by far my > largest original computer program, and my first attempt at an open-source > project. > Thanks. -- Kind regards, Hiltjo
