Re: [dev] [st] UTF-8 not working
On 22/04/28 01:44, Robert Winkler wrote: > > usually, I am using the fish https://www.youtube.com/watch?v=8AQobpqySAU signature.asc Description: PGP signature
Re: [dev] [st] UTF-8 not working
On Thu Apr 28, 2022 at 11:48 AM CDT, Страхиња Радић wrote: > On 22/04/15 07:08, Robert Winkler wrote: > > Hi, according to the st Status, UTF-8 should be working. Much needed for > > multilingual typing with ú, ü, ß, µ, ¿ etc. > > However, I only get 00e9 if I type é; anything, I need to specify in > > config.h? > > Best, Robert > > Liks I said, I only use: > > - login through getty (no display manager like lightdm, sddm, gdm...) > - no desktop environment, pure X.Org > - dwm as a window manager > - X.Org's native XKB configured with: > setxkbmap -layout us,rs -option grp:alt_shift_toggle > which is executed by dwm on startup > > and I can input various characters in Cyrillic, as well as color emoji and > other symbols using my own script[1]. > > May I ask what shell are you using inside st? The only problem I noticed so > far > with my script, which uses xdotool(1) to type characters, is when I am using > it > while st is specifically executing mksh as a shell. With bash, dash and zsh > emoji are inserted correctly. This is undoubtedly some misconfiguration of > mksh > on my part, which I have yet to figure out in detail. > > However, when I type ordinary text, regardless if it is using Cyrillic > characters, using Serbian keyboard, or those characters you listed in your > original message, using Spanish keyboard, they are entered and displayed > correctly, even in mksh. I have captured a screenshot[2] as a proof of > concept. > > In the first line in the screenshot, I have typed a command to configure XKB: > > setxkbmap -layout us,es -option grp:alt_shift_toggle > > In the second line, I have typed characters after switching to Spanish layout > by pressing left Alt+Shift. > > In the third line, I have typed the characters I used to get characters in > the > previous line, after switching back to US layout. > > The other screenshot[3] demonstrates pasting a text containing color emoji > from > a web page into Bash running under st. I do have libxft-bgra patch for Xft > installed through an AUR package to show color emoji. > > > [1]: https://git.sr.ht/~strahinja/dwm/tree/master/item/pickchar > [2]: https://i.imgur.com/yWd2nPd.png > [3]: https://i.imgur.com/7yTYHUY.png > [4]: https://aur.archlinux.org/libxft-bgra.git usually, I am using the fish
Re: [dev] [st] UTF-8 not working
On 22/04/28 06:48, Страхиња Радић wrote: > May I ask what shell are you using inside st? The only problem I noticed so > far > with my script, which uses xdotool(1) to type characters, is when I am using > it > while st is specifically executing mksh as a shell. With bash, dash and zsh > emoji are inserted correctly. This is undoubtedly some misconfiguration of > mksh > on my part, which I have yet to figure out in detail. After some investigation, I discovered the following paragraph in mksh FAQ[1]: > The shell’s utf8-mode before mksh R60 supported only the BMP (Basic > Multilingual Plane) of UCS and mapped raw (extended ASCII) octets, i.e. these > which are not valid UTF-8 BMP codepoints) into the U+EF80‥U+EFFF range, which > is allocated at the CSUR for this purpose. (It otherwise lies in the PUA; > however, there is ambiguity if encountering those UTF-8-encoded, so it > changed > for R60.) The Arithmetic expressions and CAVEATS sections in mksh(1) contain > more details about encoding and mapping. > > As of R60, utf8-mode maps “raw octets” to U-1080‥U-10FF, which is > outside the UCS and therefore collision-free. There’s work underway to make > the > shell support the full 21-bit UCS range for R60. Since I'm currently using mksh R59, that part of the mystery is solved as well. **Definitive conclusion: st does not need GNOME, ibus or other bloat (aside from good old native X.Org bloat itself) to support UTF-8 input/output.** [1]: http://www.mirbsd.org/mksh-faq.htm#posix-mode signature.asc Description: PGP signature
Re: [dev] [st] UTF-8 not working
On 22/04/15 07:08, Robert Winkler wrote: > Hi, according to the st Status, UTF-8 should be working. Much needed for > multilingual typing with ú, ü, ß, µ, ¿ etc. > However, I only get 00e9 if I type é; anything, I need to specify in > config.h? > Best, Robert Liks I said, I only use: - login through getty (no display manager like lightdm, sddm, gdm...) - no desktop environment, pure X.Org - dwm as a window manager - X.Org's native XKB configured with: setxkbmap -layout us,rs -option grp:alt_shift_toggle which is executed by dwm on startup and I can input various characters in Cyrillic, as well as color emoji and other symbols using my own script[1]. May I ask what shell are you using inside st? The only problem I noticed so far with my script, which uses xdotool(1) to type characters, is when I am using it while st is specifically executing mksh as a shell. With bash, dash and zsh emoji are inserted correctly. This is undoubtedly some misconfiguration of mksh on my part, which I have yet to figure out in detail. However, when I type ordinary text, regardless if it is using Cyrillic characters, using Serbian keyboard, or those characters you listed in your original message, using Spanish keyboard, they are entered and displayed correctly, even in mksh. I have captured a screenshot[2] as a proof of concept. In the first line in the screenshot, I have typed a command to configure XKB: setxkbmap -layout us,es -option grp:alt_shift_toggle In the second line, I have typed characters after switching to Spanish layout by pressing left Alt+Shift. In the third line, I have typed the characters I used to get characters in the previous line, after switching back to US layout. The other screenshot[3] demonstrates pasting a text containing color emoji from a web page into Bash running under st. I do have libxft-bgra patch for Xft installed through an AUR package to show color emoji. [1]: https://git.sr.ht/~strahinja/dwm/tree/master/item/pickchar [2]: https://i.imgur.com/yWd2nPd.png [3]: https://i.imgur.com/7yTYHUY.png [4]: https://aur.archlinux.org/libxft-bgra.git signature.asc Description: PGP signature
Re: [dev] [st] UTF-8 not working
On 22/04/24 02:53, Robert Winkler wrote: > Hi, thanks for all the hints I got from the [st] [dev]! > Anyway, I still cannot sort it out; hopefully the following information > helps that someone can spot the problem: > As I mentioned, the UTF-8 symbols work in other terminals such as > lxterminal. How have you set up your input method? Try using setxkbmap(1) to set X input method. Also make sure that st's terminfo entries are installed (make && sudo make install from st repository) and you have export TERM=st-256color signature.asc Description: PGP signature
Re: [dev] [st] UTF-8 not working
On Wed Apr 20, 2022 at 2:29 AM CDT, Hiltjo Posthuma wrote:
> On Tue, Apr 19, 2022 at 07:09:57PM -0500, Robert Winkler wrote:
> >
> >
> > On April 15, 2022 2:23:36 PM CDT, Quentin Rameau wrote:
> > >Hi Robert,
> > >
> > >> Hi, according to the st Status, UTF-8 should be working. Much needed for
> > >> multilingual typing with ú, ü, ß, µ, ¿ etc.
> > >> However, I only get 00e9 if I type é; anything, I need to specify in
> > >> config.h?
> > >
> > >No, you just should only need to declare correctly your LC_CTYPE
> > >
> >
> > Sorry, but I don't get it fixed. I rebuilt the locals with dpkg-reconfigure
> > locales, with the US UTF-8 as default.
> >
> > On lxterminal, the characters é, ö etc. work. In st they don't.
> >
> >
> >
>
> It works in st. It's a misconfiguration on your system.
>
> If you provide more information, not the bare minimum, maybe some people are
> kind enough to help you.
>
> --
> Kind regards,
> Hiltjo
Hi, thanks for all the hints I got from the [st] [dev]!
Anyway, I still cannot sort it out; hopefully the following information
helps that someone can spot the problem:
As I mentioned, the UTF-8 symbols work in other terminals such as
lxterminal.
1) The locales seem to be fine:
~~~
rob@pop-os ~> locale
LANG=en_US.UTF-8
LANGUAGE=en_US
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
rob@pop-os ~>
~~~
2) Fonts. The liberation font is installed (see `config.h` below).
~~~
rob@pop-os ~> apt search liberation
Sorting... Done
Full Text Search... Done
fonts-liberation/impish,impish,now 1:1.07.4-11 all [installed,automatic]
Fonts with the same metrics as Times, Arial and Courier
fonts-liberation2/impish,impish,now 2.1.3-1 all [installed,automatic]
Fonts with the same metrics as Times, Arial and Courier (v2)
~~~
btw. I tested Liberation Mono in `lxterminal`, and UTF-8 works.
3) `config.h`
~~~
/* See LICENSE file for copyright and license details. */
/*
* appearance
*
* font: see http://freedesktop.org/software/fontconfig/fontconfig-user.html
*/
static char *font = "Liberation Mono:pixelsize=18:antialias=true:autohint=true";
static int borderpx = 2;
/*
* What program is execed by st depends of these precedence rules:
* 1: program passed with -e
* 2: scroll and/or utmp
* 3: SHELL environment variable
* 4: value of shell in /etc/passwd
* 5: value of shell in config.h
*/
static char *shell = "/bin/sh";
char *utmp = NULL;
/* scroll program: to enable use a string like "scroll" */
char *scroll = NULL;
char *stty_args = "stty raw pass8 nl -echo -iexten -cstopb 38400";
/* identification sequence returned in DA and DECID */
char *vtiden = "\033[?6c";
/* Kerning / character bounding-box multipliers */
static float cwscale = 1.0;
static float chscale = 1.0;
/*
* word delimiter string
*
* More advanced example: L" `'\"()[]{}"
*/
wchar_t *worddelimiters = L" ";
/* selection timeouts (in milliseconds) */
static unsigned int doubleclicktimeout = 300;
static unsigned int tripleclicktimeout = 600;
/* alt screens */
int allowaltscreen = 1;
/* allow certain non-interactive (insecure) window operations such as:
setting the clipboard text */
int allowwindowops = 0;
/*
* draw latency range in ms - from new content/keypress/etc until drawing.
* within this range, st draws when content stops arriving (idle). mostly it's
* near minlatency, but it waits longer for slow updates to avoid partial draw.
* low minlatency will tear/flicker more, as it can "detect" idle too early.
*/
static double minlatency = 8;
static double maxlatency = 33;
/*
* blinking timeout (set to 0 to disable blinking) for the terminal blinking
* attribute.
*/
static unsigned int blinktimeout = 800;
/*
* thickness of underline and bar cursors
*/
static unsigned int cursorthickness = 2;
/*
* bell volume. It must be a value between -100 and 100. Use 0 for disabling
* it
*/
static int bellvolume = 0;
/* default TERM value */
char *termname = "st-256color";
/*
* spaces per tab
*
* When you are changing this value, don't forget to adapt the »it« value in
* the st.info and appropriately install the st.info in the environment where
* you use this st version.
*
* it#$tabspaces,
*
* Secondly make sure your kernel is not expanding tabs. When running `stty
* -a` »tab0« should appear. You can tell the terminal to not expand tabs by
* running following command:
*
* stty tabs
*/
unsigned int tabspaces = 8;
/* Terminal colors (16 first used in escape sequence) */
static const char *colorname[] = {
/* 8 normal colors */
"black",
"red3",
"green3",
"yellow3",
"blue2",
"magenta3",
"cyan3",
"gray90",
/* 8 bright colors */
"gray50",
Re: [dev] [st] UTF-8 not working
On Wed, Apr 20, 2022 at 08:46:46AM -0500, Robert Winkler wrote:
> I had to remove color emojis due to crashes of my email client aerc in
> st, as advised in this mailing list.
I think it's better to reject color emojies in ST rather than removing
them system-wide. The FAQ already mentions how to do it.
diff --git a/x.c b/x.c
index f70e3fb..6e94f76 100644
--- a/x.c
+++ b/x.c
@@ -1015,6 +1015,7 @@ xloadfonts(const char *fontstr, double fontsize)
}
defaultfontsize = usedfontsize;
}
+ FcPatternAddBool(pattern, FC_COLOR, FcFalse);
if (xloadfont(, pattern))
die("can't open font %s\n", fontstr);
And on a sidenote, I support not adding such workaround in mainline ST.
It's an XFT issue which should've been fixed long ago. A merge request
fixing this exists for 2 years now [0].
The thread suggests that the patch isn't perfect, but it's certainly
better than crashing the application as it does right now.
[0]: https://gitlab.freedesktop.org/xorg/lib/libxft/-/merge_requests/1
- NRK
Re: [dev] [st] UTF-8 not working
On Wed Apr 20, 2022 at 8:39 AM CDT, Страхиња Радић wrote: > On 22/04/19 07:09, Robert Winkler wrote: > > > > Sorry, but I don't get it fixed. I rebuilt the locals with dpkg-reconfigure > > locales, with the US UTF-8 as default. > > > > On lxterminal, the characters é, ö etc. work. In st they don't. > > With > > export LC_ALL=en_US.UTF-8 > > properly configured fonts in fontconfig[1][2][3] and properly configured > config.h, Unicode characters, and even emoji, work in st. See the > documentation > for fontconfig for information on how to properly configure fonts in X.Org. > > st works here perfectly with Unicode characters and color emoji. > > > [1]: https://www.freedesktop.org/wiki/Software/fontconfig/ > [2]: https://freedesktop.org/software/fontconfig/fontconfig-user.html > [3]: https://wiki.archlinux.org/title/Font_configuration Cheers, Страхиња Радић! I had to remove color emojis due to crashes of my email client aerc in st, as advised in this mailing list. But you give me confidence that st can do all modern fonts; I just need to learn more about using fontconfig. As a scientist/writer, I usually install all LaTeX and fonts that I can find and hope things go well ;-). Best, Robert
Re: [dev] [st] UTF-8 not working
On Wed Apr 20, 2022 at 2:29 AM CDT, Hiltjo Posthuma wrote: > On Tue, Apr 19, 2022 at 07:09:57PM -0500, Robert Winkler wrote: > > > > > > On April 15, 2022 2:23:36 PM CDT, Quentin Rameau wrote: > > >Hi Robert, > > > > > >> Hi, according to the st Status, UTF-8 should be working. Much needed for > > >> multilingual typing with ú, ü, ß, µ, ¿ etc. > > >> However, I only get 00e9 if I type é; anything, I need to specify in > > >> config.h? > > > > > >No, you just should only need to declare correctly your LC_CTYPE > > > > > > > Sorry, but I don't get it fixed. I rebuilt the locals with dpkg-reconfigure > > locales, with the US UTF-8 as default. > > > > On lxterminal, the characters é, ö etc. work. In st they don't. > > > > > > > > It works in st. It's a misconfiguration on your system. > > If you provide more information, not the bare minimum, maybe some people are > kind enough to help you. > > -- > Kind regards, > Hiltjo With pleasure. Which information would be relevant? My keyboards are DIY (Corne, Kyria, Dactyl Manuform), programmed with QMK; the system is Pop OS; i.e. Debianish. UTF-8 works on lxterminal, but not in st, why I would suggest that I need to modify something related to the st.
Re: [dev] [st] UTF-8 not working
On 22/04/19 07:09, Robert Winkler wrote: > > Sorry, but I don't get it fixed. I rebuilt the locals with dpkg-reconfigure > locales, with the US UTF-8 as default. > > On lxterminal, the characters é, ö etc. work. In st they don't. With export LC_ALL=en_US.UTF-8 properly configured fonts in fontconfig[1][2][3] and properly configured config.h, Unicode characters, and even emoji, work in st. See the documentation for fontconfig for information on how to properly configure fonts in X.Org. st works here perfectly with Unicode characters and color emoji. [1]: https://www.freedesktop.org/wiki/Software/fontconfig/ [2]: https://freedesktop.org/software/fontconfig/fontconfig-user.html [3]: https://wiki.archlinux.org/title/Font_configuration signature.asc Description: PGP signature
Re: [dev] [st] UTF-8 not working
On Tue, Apr 19, 2022 at 07:09:57PM -0500, Robert Winkler wrote: > > > On April 15, 2022 2:23:36 PM CDT, Quentin Rameau wrote: > >Hi Robert, > > > >> Hi, according to the st Status, UTF-8 should be working. Much needed for > >> multilingual typing with ú, ü, ß, µ, ¿ etc. > >> However, I only get 00e9 if I type é; anything, I need to specify in > >> config.h? > > > >No, you just should only need to declare correctly your LC_CTYPE > > > > Sorry, but I don't get it fixed. I rebuilt the locals with dpkg-reconfigure > locales, with the US UTF-8 as default. > > On lxterminal, the characters é, ö etc. work. In st they don't. > > > It works in st. It's a misconfiguration on your system. If you provide more information, not the bare minimum, maybe some people are kind enough to help you. -- Kind regards, Hiltjo
Re: [dev] [st] UTF-8 not working
On April 15, 2022 2:23:36 PM CDT, Quentin Rameau wrote: >Hi Robert, > >> Hi, according to the st Status, UTF-8 should be working. Much needed for >> multilingual typing with ú, ü, ß, µ, ¿ etc. >> However, I only get 00e9 if I type é; anything, I need to specify in >> config.h? > >No, you just should only need to declare correctly your LC_CTYPE > Sorry, but I don't get it fixed. I rebuilt the locals with dpkg-reconfigure locales, with the US UTF-8 as default. On lxterminal, the characters é, ö etc. work. In st they don't.
Re: [dev] [st] UTF-8 not working
Hi Robert, > Hi, according to the st Status, UTF-8 should be working. Much needed for > multilingual typing with ú, ü, ß, µ, ¿ etc. > However, I only get 00e9 if I type é; anything, I need to specify in > config.h? No, you just should only need to declare correctly your LC_CTYPE
[dev] [st] UTF-8 not working
Hi, according to the st Status, UTF-8 should be working. Much needed for multilingual typing with ú, ü, ß, µ, ¿ etc. However, I only get 00e9 if I type é; anything, I need to specify in config.h? Best, Robert
