Re: build softraid(4) in powerpc64 RAMDISK
On Tue, 19 Sep 2023 21:48:57 + Klemens Nanni wrote: > distrib/powerpc64/ramdisk builds and fits; I did not have a free disk > (partition) to run-test, but completing the config should be fine. I moved my /usr/xobj filesystem to a softraid CONCAT. The old bsd.rd failed to mount it, the new bsd.rd succeeded. ok gkoehler@ > Index: sys/arch/powerpc64/conf/RAMDISK > === > RCS file: /cvs/src/sys/arch/powerpc64/conf/RAMDISK,v > retrieving revision 1.12 > diff -u -p -r1.12 RAMDISK > --- sys/arch/powerpc64/conf/RAMDISK 26 Jun 2022 20:05:06 - 1.12 > +++ sys/arch/powerpc64/conf/RAMDISK 16 Sep 2023 16:32:42 - > @@ -25,6 +25,7 @@ option CRYPTO > config bsd root on rd0a swap on rd0b > > mainbus0 at root > +softraid0at root > cpu0 at mainbus? > opal0at fdt? > opalcons*at fdt? >
ypldap: stop flattening trees
ypldap currently packs all the user and group lines into contiguous blocks
of memory so it can move from one entry to the next by pointer arithmetic.
This doesn't make much sense because the entries are also in red black trees
(that's how it looks up the entry in the first place) and RB_NEXT() is not
slow.
The one piece of the tree flattening code that seems worth keeping is
strdup()ing the netid lines so they don't take 1kB per user.
ok?
Index: entries.c
===
RCS file: /cvs/src/usr.sbin/ypldap/entries.c,v
retrieving revision 1.6
diff -u -p -u -p -r1.6 entries.c
--- entries.c 18 Jul 2023 13:06:33 - 1.6
+++ entries.c 20 Sep 2023 07:17:00 -
@@ -34,86 +34,6 @@
#include
#include "ypldap.h"
-#include "log.h"
-
-void
-flatten_entries(struct env *env)
-{
- size_t len;
- char*linep;
- char*endp;
- char*tmp;
- struct userent *ue;
- struct groupent *ge;
-
- log_debug("flattening trees");
- /*
-* This takes all the line pointers in RB elements and
-* concatenates them in a single string, to be able to
-* implement next element lookup without tree traversal.
-*
-* An extra octet is alloced to make space for an additional NUL.
-*/
- if ((linep = calloc(1, env->sc_user_line_len + 1)) == NULL) {
- /*
-* XXX: try allocating a smaller chunk of memory
-*/
- fatal("out of memory");
- }
- endp = linep;
-
- RB_FOREACH(ue, user_name_tree, env->sc_user_names) {
- /*
-* we convert the first nul back to a column,
-* copy the string and then convert it back to a nul.
-*/
- ue->ue_line[strlen(ue->ue_line)] = ':';
- log_debug("pushing line: %s", ue->ue_line);
- len = strlen(ue->ue_line) + 1;
- memcpy(endp, ue->ue_line, len);
- endp[strcspn(endp, ":")] = '\0';
- free(ue->ue_line);
- ue->ue_line = endp;
- endp += len;
-
- /*
-* To save memory strdup(3) the netid_line which originally used
-* LINE_WIDTH bytes
-*/
- tmp = ue->ue_netid_line;
- ue->ue_netid_line = strdup(tmp);
- if (ue->ue_netid_line == NULL) {
- fatal("out of memory");
- }
- free(tmp);
- }
- env->sc_user_lines = linep;
- log_debug("done pushing users");
-
- if ((linep = calloc(1, env->sc_group_line_len + 1)) == NULL) {
- /*
-* XXX: try allocating a smaller chunk of memory
-*/
- fatal("out of memory");
- }
- endp = linep;
- RB_FOREACH(ge, group_name_tree, env->sc_group_names) {
- /*
-* we convert the first nul back to a column,
-* copy the string and then convert it back to a nul.
-*/
- ge->ge_line[strlen(ge->ge_line)] = ':';
- log_debug("pushing line: %s", ge->ge_line);
- len = strlen(ge->ge_line) + 1;
- memcpy(endp, ge->ge_line, len);
- endp[strcspn(endp, ":")] = '\0';
- free(ge->ge_line);
- ge->ge_line = endp;
- endp += len;
- }
- env->sc_group_lines = linep;
- log_debug("done pushing groups");
-}
int
userent_name_cmp(struct userent *ue1, struct userent *ue2)
Index: yp.c
===
RCS file: /cvs/src/usr.sbin/ypldap/yp.c,v
retrieving revision 1.22
diff -u -p -u -p -r1.22 yp.c
--- yp.c18 Jul 2023 13:06:33 - 1.22
+++ yp.c20 Sep 2023 07:17:00 -
@@ -557,21 +557,25 @@ ypresp_key_val *
ypproc_first_2_svc(ypreq_nokey *arg, struct svc_req *req)
{
static struct ypresp_key_valres;
+ struct userent *ue;
+ struct groupent *ge;
if (yp_valid_domain(arg->domain, (struct ypresp_val *)) == -1)
return ();
if (strcmp(arg->map, "passwd.byname") == 0 ||
strcmp(arg->map, "master.passwd.byname") == 0) {
- if (env->sc_user_lines == NULL)
+ ue = RB_MIN(user_name_tree, env->sc_user_names);
+ if (ue == NULL)
return (NULL);
- yp_make_keyval(, env->sc_user_lines, env->sc_user_lines);
+ yp_make_keyval(, ue->ue_line, ue->ue_line);
} else if (strcmp(arg->map, "group.byname") == 0) {
- if (env->sc_group_lines == NULL)
+ ge = RB_MIN(group_name_tree, env->sc_group_names);
+ if (ge == NULL)
return (NULL);
-
Re: Send international text with mail(1) - proposal and patches
Steffen Nurpmeso wrote in <20230920214009.w5mrf%stef...@sdaoden.eu>: |Ingo Schwarze wrote in | : | ... ||I just checked - even though i'm using the higer-level mutt(1) MUA ||most of the time and even though the shell i'm starting mutt(1) from ||has LC_CTYPE=C.UTF-8 set on that particular machine, the last sixteen ||mails i sent all contained the explicit header || || Content-Type: text/plain; charset=us-ascii || ||and intentionally so. Yes, i do occasionally send UTF-8 mail on | |To be a hundred percent correct: MIME is not needed at all in that That is to say, to be correct myself: like RFC 2045 says, "MIME defines a number of new RFC 822 header fields that are used to describe the content of a MIME entity". Yet if there is no MIME entity but only a plain RFC 822/2822/5322 internet message format, there is nothing to describe. [.]there are still circumstances in which it might be desirable for a mail-processing agent to know whether a message was composed with the new standard in mind. Therefore, this document defines a new header field, "MIME-Version", which is to be used to declare the version of the Internet message body format standard in use. Messages composed in accordance with this document MUST include such a header field, with the following verbatim text: But normally OpenBSD Mail does not, so no "MIME-Version: 1.0", because no The presence of this header field is an assertion that the message has been composed in compliance with this document. |case, unless a transfer-encoding had to be used (you do not show |that header), maybe because of overlong lines to-be-folded, or for |whatever reason. (But it is swallowed by consumers of course.) That would at least be my point of view. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: Send international text with mail(1) - proposal and patches
Ingo Schwarze wrote in : ... |I just checked - even though i'm using the higer-level mutt(1) MUA |most of the time and even though the shell i'm starting mutt(1) from |has LC_CTYPE=C.UTF-8 set on that particular machine, the last sixteen |mails i sent all contained the explicit header | | Content-Type: text/plain; charset=us-ascii | |and intentionally so. Yes, i do occasionally send UTF-8 mail on To be a hundred percent correct: MIME is not needed at all in that case, unless a transfer-encoding had to be used (you do not show that header), maybe because of overlong lines to-be-folded, or for whatever reason. (But it is swallowed by consumers of course.) --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: Send international text with mail(1) - proposal and patches
Hi, Ingo • Ingo Schwarze [2023-09-20 13:55]: Hi Kirill, Kirill Miazine wrote on Wed, Sep 20, 2023 at 12:52:52PM +0200: you may not even need -m, and instead inspect LC_CTYPE environment variable and add appropriate headers for UTF-8. according to locale(1), LC_CTYPE may be set to indicate UTF-8: If the value of LC_CTYPE ends in ‘.UTF-8’, programs in the OpenBSD base system ignore the beginning of it, treating for example zh_CN.UTF-8 exactly like en_US.UTF-8. This is definitely very bad advice I am sorry! I was thinking that a user who sets appropriate LC_CTYPE is thus instructing programs that input and output is UTF-8 and such instruction could be used instead of a flag, as per locale(1) presence of .UTF-8 in LC_CTYPE is an instruction to treat input and output as UTF-8 encoded text: "The character encoding locale LC_CTYPE instructs programs which character encoding to assume for text input and to use for text output." After all, LC_CTYPE=en_US.UTF-8 has to be set by a user and thus signals a preference to programs, and thus it wouldn't be unexpected or surprising to treat text as UTF-8 and also set appropriate MIME-headers. After all, by setting LC_CTYPE=en_US.UTF-8 -- according to locale(1) -- user says that input text is UTF-8, and then mail(1) would have to figure out how to make sure that text is transmitted properly. Whether the user uses an UTF-8 locale for their shell and terminal has nothing to do with whether they want to be send UTF-8 encoded mail with MIME headers. For example, i'm using LC_CTYPE=en_US.UTF-8 for my shells and terminals most of the time, but i do not want the low-level mail(1) MUA to suddenly start sending UTF-8 mail without being specifically asked to. My understanding of purpose of LC_TYPE was that by setting it, user specifically asks to treat input as UTF-8, and then the programs have to handle encoding appropriately. So I wouldn't be surprised if mail(1) started sending UTF-8 mail with LC_CTYPE=en_US.UTF-8. In fact, I'd be happy if it did so. I just checked - even though i'm using the higer-level mutt(1) MUA most of the time and even though the shell i'm starting mutt(1) from has LC_CTYPE=C.UTF-8 set on that particular machine, the last sixteen mails i sent all contained the explicit header Content-Type: text/plain; charset=us-ascii and intentionally so. Yes, i do occasionally send UTF-8 mail on purpose, mostly in highly technical messages that need to display particular Unicode characters in addition to mentioning their codepoints in the U+[XX] form, and rarely, sending UTF-8 happens inadvertently because mutt(1) contains some weird autodetection logic - but what you set your terminal to and what you use for sending mail are clearly completely unrelated topics. Mutt has indeed a logic to see which character set a text can be converted into: it tries US-ASCII, then ISO-8859-1 and then UTF-8. Yours, Ingo
Re: Send international text with mail(1) - proposal and patches
On Wed, Sep 20, 2023 at 07:44:12PM +0200, Walter Alejandro Iglesias wrote: > And this new idea simplifies all to this: In case anyone else is worried. Crystal Kolipe already pointed me out that a better UTF-8 checking is needed, I know, I'll get to that tomorrow.
Re: Send international text with mail(1) - proposal and patches
On Wed, Sep 20, 2023 at 06:13:10PM +0200, Walter Alejandro Iglesias wrote:
> Now I was investigating exactly that :-) (like Mutt also does): to make
> mail(1) automatically set the appropiate MIME headers when it detects
> any utf8 characters in the body text. So, you don't like this idea?
>
And this new idea simplifies all to this:
Index: send.c
===
RCS file: /cvs/src/usr.bin/mail/send.c,v
retrieving revision 1.26
diff -u -p -r1.26 send.c
--- send.c 8 Mar 2023 04:43:11 - 1.26
+++ send.c 20 Sep 2023 17:40:22 -
@@ -33,6 +33,8 @@
#include "rcv.h"
#include "extern.h"
+char utf8 = 0;
+
static volatile sig_atomic_t sendsignal; /* Interrupted by a signal? */
/*
@@ -341,6 +343,13 @@ mail1(struct header *hp, int printheader
else
puts("Null message body; hope that's ok");
}
+ /* Check for non ascii characters */
+ int ch;
+while ((ch = getc(mtf)) != EOF)
+if (ch > 0x7f)
+ utf8 = 1;
+ rewind(mtf);
+
/*
* Now, take the user names from the combined
* to and cc lists and do all the alias
@@ -525,6 +534,10 @@ puthead(struct header *hp, FILE *fo, int
fmt("To:", hp->h_to, fo, w), gotcha++;
if (hp->h_subject != NULL && w & GSUBJECT)
fprintf(fo, "Subject: %s\n", hp->h_subject), gotcha++;
+ if (utf8)
+ fprintf(fo, "MIME-Version: 1.0\nContent-Type: text/plain;
charset=utf-8\nContent-Transfer-Encoding: 8bit\n"), gotcha++;
+ else
+ fprintf(fo, "MIME-Version: 1.0\nContent-Type: text/plain;
charset=us-ascii\nContent-Transfer-Encoding: 7bit\n"), gotcha++;
if (hp->h_cc != NULL && w & GCC)
fmt("Cc:", hp->h_cc, fo, w), gotcha++;
if (hp->h_bcc != NULL && w & GBCC)
--
Walter
OpenBSD Errata: September 21, 2023 (npppd)
Errata patches for npppd have been released for OpenBSD 7.2 and 7.3. Binary updates for the amd64, arm64 and i386 platform are available via the syspatch utility. Source code patches can be found on the respective errata page: https://www.openbsd.org/errata72.html https://www.openbsd.org/errata73.html
Re: Send international text with mail(1) - proposal and patches
On Wed, Sep 20, 2023 at 05:30:08PM +0200, Ingo Schwarze wrote: > Hi, > > i checked the following points: > > * Even though RFC 2049 section 2 bullet point 1 only *requires* >MIME-conformant MUAs to always write the header "MIME-Version: >1.0" - and mail(1) is most certainly not MIME-conformant - RFC 2049 >section 2 bullet point 8 explicitly *recommends* that even non-MIME >MUAs always set appropriate MIME headers. RFC 2046 section 4.1.2 >paragraph 8 also "strongly" recommends the explicit inclusion of a >"charset" parameter even for us-ascii. > >Consequently, i believe that when sending a message in US-ASCII, >mail(1) should include these headers: > >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Type: text/plain; charset=us-ascii I already thought about adding this, it's what Mutt does by default, But I thought, Ingo is going to scold me for complicating things. :-) > > * Adding a "Content-Transfer-Encoding: ..." header is indeed required >for sending UTF-8 messages, see RFC 2049 section 2 bullet point 2. >"8bit" is one of the valid values that MUAs must support for >receiving messages by default. >Using it seems sane because it is most likely to work with receiving >MUAs that are not MIME-conformant, like our mail(1) itself. >I think nowadays, that's a bigger concern than MTAs that are not >8-bit clean, in particular when maintaining a low-level program >like our mail(1). >Consequently, i think using 8bit is indeed better for our mail(1) >than quoted-printable or base64. Well, this also saves you the conversion, especially with the subject, which is tricky. > > * Adding "Content-Type: text/plain; charset=utf-8" is required by >RFC 2049 section 2 bullet point 4 (for the simplest kind of UTF-8 >encoded messages). > > * The Content-Disposition: header is defined in RFC 2183, clearly >optional, and not useful in single-part messages. Consequently, >mail(1) should not write it. Yeah, I read that, that's why I didn't add that header. > > So apart from writing the headers for us-ascii, i think you are > almost there. > > Given that the charset cannot be inferred from the environment > and that setting it per-system or per-user in a configuration file > is also inadequate - it shouldn't be uncommon for users to sometimes > send US-ASCII and sometimes UTF-8 mail - i think that a new option > is indeed needed. > > Regarding the naming of the option, compatibility with POSIX > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/mailx.html > is paramount, which kills the tentative idea to use -u for "UTF-8" > because -u already means "user". > > Compatibility with other mailx(1) implementations is also a > consideration. See, for example, > https://linux.die.net/man/1/mail > and -m is indeed among the very few options still available over there. > I would document it focussing on a "multibyte character encoding" > mnemonic. The "mime" mnemonic feels far too broad because MIME can > be used for lots of other purposes besides specifying a character > encoding. > > The -m option is also free here: > https://man.freebsd.org/cgi/man.cgi?query=mail(1) > https://man.netbsd.org/mail.1 > https://docs.oracle.com/cd/E88353_01/html/E37839/mailx-1.html > https://www.ibm.com/docs/en/aix/7.3?topic=m-mail-command-1 > None of those appears to support command line selection of the > character set for sending mail, so i don't see any immediate > logioc clashes either. > > The -m option does clash with this one: > https://www.sdaoden.eu/code-nail.html > But i think dismissing Steffen Daode Nurpmeso as a lunatic is obviously > the way to go. Try to listen to that person and you will never get > anything done. > > The mailx(1) documented on die.net appears to be the Heirloom one. > It does not have an option to select sending US-ASCII or UTF-8. > Instead, it has a "sendcharsets" configuration variable. That's > clearly overengineering, but even when hardcoding the equivalent of > > sendcharsets=utf-8 > > which is also the default, that's nasty because it silently switches to > UTF-8 as soon as a non-ASCII character appears in the input. I think > at least in interactive mode, explicit confirmation from the user would > be required to send UTF-8, instead writing dead.letter if the user > rejects the request, such that they can clean up the file and try again. > > That would certainly be more complicated than requiring an option > up front, not only from the implementation perspective, but arguably > also from the user perspective. So unless other developers think this > should be fully automatic with confirmation rather than controlled > by an option, i suggest staying with Walter's idea of using an option. Now I was investigating exactly that :-) (like Mutt also does): to make mail(1) automatically set the appropiate MIME headers when it detects any utf8 characters
Re: Send international text with mail(1) - proposal and patches
Hi, i checked the following points: * Even though RFC 2049 section 2 bullet point 1 only *requires* MIME-conformant MUAs to always write the header "MIME-Version: 1.0" - and mail(1) is most certainly not MIME-conformant - RFC 2049 section 2 bullet point 8 explicitly *recommends* that even non-MIME MUAs always set appropriate MIME headers. RFC 2046 section 4.1.2 paragraph 8 also "strongly" recommends the explicit inclusion of a "charset" parameter even for us-ascii. Consequently, i believe that when sending a message in US-ASCII, mail(1) should include these headers: MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii * Adding a "Content-Transfer-Encoding: ..." header is indeed required for sending UTF-8 messages, see RFC 2049 section 2 bullet point 2. "8bit" is one of the valid values that MUAs must support for receiving messages by default. Using it seems sane because it is most likely to work with receiving MUAs that are not MIME-conformant, like our mail(1) itself. I think nowadays, that's a bigger concern than MTAs that are not 8-bit clean, in particular when maintaining a low-level program like our mail(1). Consequently, i think using 8bit is indeed better for our mail(1) than quoted-printable or base64. * Adding "Content-Type: text/plain; charset=utf-8" is required by RFC 2049 section 2 bullet point 4 (for the simplest kind of UTF-8 encoded messages). * The Content-Disposition: header is defined in RFC 2183, clearly optional, and not useful in single-part messages. Consequently, mail(1) should not write it. So apart from writing the headers for us-ascii, i think you are almost there. Given that the charset cannot be inferred from the environment and that setting it per-system or per-user in a configuration file is also inadequate - it shouldn't be uncommon for users to sometimes send US-ASCII and sometimes UTF-8 mail - i think that a new option is indeed needed. Regarding the naming of the option, compatibility with POSIX https://pubs.opengroup.org/onlinepubs/9699919799/utilities/mailx.html is paramount, which kills the tentative idea to use -u for "UTF-8" because -u already means "user". Compatibility with other mailx(1) implementations is also a consideration. See, for example, https://linux.die.net/man/1/mail and -m is indeed among the very few options still available over there. I would document it focussing on a "multibyte character encoding" mnemonic. The "mime" mnemonic feels far too broad because MIME can be used for lots of other purposes besides specifying a character encoding. The -m option is also free here: https://man.freebsd.org/cgi/man.cgi?query=mail(1) https://man.netbsd.org/mail.1 https://docs.oracle.com/cd/E88353_01/html/E37839/mailx-1.html https://www.ibm.com/docs/en/aix/7.3?topic=m-mail-command-1 None of those appears to support command line selection of the character set for sending mail, so i don't see any immediate logioc clashes either. The -m option does clash with this one: https://www.sdaoden.eu/code-nail.html But i think dismissing Steffen Daode Nurpmeso as a lunatic is obviously the way to go. Try to listen to that person and you will never get anything done. The mailx(1) documented on die.net appears to be the Heirloom one. It does not have an option to select sending US-ASCII or UTF-8. Instead, it has a "sendcharsets" configuration variable. That's clearly overengineering, but even when hardcoding the equivalent of sendcharsets=utf-8 which is also the default, that's nasty because it silently switches to UTF-8 as soon as a non-ASCII character appears in the input. I think at least in interactive mode, explicit confirmation from the user would be required to send UTF-8, instead writing dead.letter if the user rejects the request, such that they can clean up the file and try again. That would certainly be more complicated than requiring an option up front, not only from the implementation perspective, but arguably also from the user perspective. So unless other developers think this should be fully automatic with confirmation rather than controlled by an option, i suggest staying with Walter's idea of using an option. > Index: extern.h > === > RCS file: /cvs/src/usr.bin/mail/extern.h,v > retrieving revision 1.29 > diff -u -p -r1.29 extern.h > --- extern.h 16 Sep 2018 02:38:57 - 1.29 > +++ extern.h 20 Sep 2023 10:44:41 - > @@ -261,3 +261,4 @@ intwriteback(FILE *); > extern char *__progname; > extern char *tmpdir; > extern const struct cmd *com; /* command we are running */ > +extern char mime; /* Add MIME headers */ Likely not best mnemonic naming. > Index: mail.1 > === > RCS file: /cvs/src/usr.bin/mail/mail.1,v > retrieving revision 1.83 > diff -u -p
Re: malloc write after free error checking
On Wed, Sep 20, 2023 at 03:02:27PM +0200, Matthieu Herrb wrote: > On Wed, Sep 20, 2023 at 08:08:23AM +0200, Otto Moerbeek wrote: > > > > The other, a write after free that crashed the X server when running > > picard was diagnosed by me. This one was a bit nasty, as it required > > instrumenting malloc to print some extra info to find the root cause. > > > > The bug is that the call in > > https://github.com/openbsd/xenocara/blob/master/xserver/Xext/xvdisp.c#L1002 > > overwrites the first 4 bytes of the chunk next to the one allocated on > > line 995. > > > > A workaround is to allocate 4 bytes extra, matthieu@ will be looking > > for a proper fix, as it requires knowledge of the X internals. > > > > Hi, > > Thanks again for finding it. Can you try this patch ? > > === > Fix overflow in glamor_xv_query_image_attributes for NV12 image format > > This is a format with num_planes == 2, we have only 2 elements in > offsets[] and pitches[] > > Bug found by Otto Moerbeek on OpenBSD using his strict malloc checking. > --- > glamor/glamor_xv.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/glamor/glamor_xv.c b/glamor/glamor_xv.c > index a3d6b3bc3..e0e8e0ba9 100644 > --- a/glamor/glamor_xv.c > +++ b/glamor/glamor_xv.c > @@ -291,10 +291,10 @@ glamor_xv_query_image_attributes(int id, > pitches[0] = size; > size *= *h; > if (offsets) > -offsets[1] = offsets[2] = size; > +offsets[1] = size; > tmp = ALIGN(*w, 4); > if (pitches) > -pitches[1] = pitches[2] = tmp; > +pitches[1] = tmp; > tmp *= (*h >> 1); > size += tmp; > break; > --- > 2.42.0 Yes, that works for me, -Otto
Re: malloc write after free error checking
On Wed, Sep 20, 2023 at 08:08:23AM +0200, Otto Moerbeek wrote: > > The other, a write after free that crashed the X server when running > picard was diagnosed by me. This one was a bit nasty, as it required > instrumenting malloc to print some extra info to find the root cause. > > The bug is that the call in > https://github.com/openbsd/xenocara/blob/master/xserver/Xext/xvdisp.c#L1002 > overwrites the first 4 bytes of the chunk next to the one allocated on > line 995. > > A workaround is to allocate 4 bytes extra, matthieu@ will be looking > for a proper fix, as it requires knowledge of the X internals. > Hi, Thanks again for finding it. Can you try this patch ? === Fix overflow in glamor_xv_query_image_attributes for NV12 image format This is a format with num_planes == 2, we have only 2 elements in offsets[] and pitches[] Bug found by Otto Moerbeek on OpenBSD using his strict malloc checking. --- glamor/glamor_xv.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/glamor/glamor_xv.c b/glamor/glamor_xv.c index a3d6b3bc3..e0e8e0ba9 100644 --- a/glamor/glamor_xv.c +++ b/glamor/glamor_xv.c @@ -291,10 +291,10 @@ glamor_xv_query_image_attributes(int id, pitches[0] = size; size *= *h; if (offsets) -offsets[1] = offsets[2] = size; +offsets[1] = size; tmp = ALIGN(*w, 4); if (pitches) -pitches[1] = pitches[2] = tmp; +pitches[1] = tmp; tmp *= (*h >> 1); size += tmp; break; --- 2.42.0 -- Matthieu Herrb
Re: Send international text with mail(1) - proposal and patches
Hi Kirill, Kirill Miazine wrote on Wed, Sep 20, 2023 at 12:52:52PM +0200: > you may not even need -m, and instead inspect LC_CTYPE environment > variable and add appropriate headers for UTF-8. according to locale(1), > LC_CTYPE may be set to indicate UTF-8: > > If the value of LC_CTYPE ends in ‘.UTF-8’, programs in the OpenBSD base > system ignore the beginning of it, treating for example zh_CN.UTF-8 > exactly like en_US.UTF-8. This is definitely very bad advice. Whether the user uses an UTF-8 locale for their shell and terminal has nothing to do with whether they want to be send UTF-8 encoded mail with MIME headers. For example, i'm using LC_CTYPE=en_US.UTF-8 for my shells and terminals most of the time, but i do not want the low-level mail(1) MUA to suddenly start sending UTF-8 mail without being specifically asked to. I just checked - even though i'm using the higer-level mutt(1) MUA most of the time and even though the shell i'm starting mutt(1) from has LC_CTYPE=C.UTF-8 set on that particular machine, the last sixteen mails i sent all contained the explicit header Content-Type: text/plain; charset=us-ascii and intentionally so. Yes, i do occasionally send UTF-8 mail on purpose, mostly in highly technical messages that need to display particular Unicode characters in addition to mentioning their codepoints in the U+[XX] form, and rarely, sending UTF-8 happens inadvertently because mutt(1) contains some weird autodetection logic - but what you set your terminal to and what you use for sending mail are clearly completely unrelated topics. Yours, Ingo
Re: Send international text with mail(1) - proposal and patches
On Wed, Sep 20, 2023 at 10:30:31AM +, Klemens Nanni wrote:
> Except for mandoc(1) and other manuals where "utf8" is a literal keyword,
> our manuals consistently use upper-case UTF-8 for what is an abbreviation,
> so this should do as wlel.
>
> > .It Fl n
> > Inhibits reading
> > .Pa /etc/mail.rc
>
> You forgot SYNOPSIS:
> $ man -h mail
> mail [-dEIinv] [-b list] [-c list] [-r from-addr] [-s subject] to-addr
> ...
> mail [-dEIiNnv] -f [file]
> mail [-dEIiNnv] [-u user]
>
> Otherwise looks sane.
>
Thank you!
Index: extern.h
===
RCS file: /cvs/src/usr.bin/mail/extern.h,v
retrieving revision 1.29
diff -u -p -r1.29 extern.h
--- extern.h16 Sep 2018 02:38:57 - 1.29
+++ extern.h20 Sep 2023 10:44:41 -
@@ -261,3 +261,4 @@ int writeback(FILE *);
extern char *__progname;
extern char *tmpdir;
extern const struct cmd *com; /* command we are running */
+extern char mime; /* Add MIME headers */
Index: mail.1
===
RCS file: /cvs/src/usr.bin/mail/mail.1,v
retrieving revision 1.83
diff -u -p -r1.83 mail.1
--- mail.1 31 Mar 2022 17:27:25 - 1.83
+++ mail.1 20 Sep 2023 10:44:41 -
@@ -40,7 +40,7 @@
.Sh SYNOPSIS
.Nm mail
.Bk -words
-.Op Fl dEIinv
+.Op Fl dEIimnv
.Op Fl b Ar list
.Op Fl c Ar list
.Op Fl r Ar from-addr
@@ -106,6 +106,8 @@ on noisy phone lines.
.It Fl N
Inhibits initial display of message headers
when reading mail or editing a mail folder.
+.It Fl m
+Add MIME headers to send UTF-8 encoded messages.
.It Fl n
Inhibits reading
.Pa /etc/mail.rc
Index: main.c
===
RCS file: /cvs/src/usr.bin/mail/main.c,v
retrieving revision 1.35
diff -u -p -r1.35 main.c
--- main.c 26 Jan 2021 18:21:47 - 1.35
+++ main.c 20 Sep 2023 10:44:41 -
@@ -79,6 +79,8 @@ int realscreenheight; /* the real scree
intuflag; /* Are we in -u mode? */
sigset_t intset; /* Signal set that is just SIGINT */
+char mime = 0; /* Add MIME headers */
+
/*
* The pointers for the string allocation routines,
* there are NSPACE independent areas.
@@ -136,7 +138,7 @@ main(int argc, char **argv)
smopts = NULL;
fromaddr = NULL;
subject = NULL;
- while ((i = getopt(argc, argv, "EINb:c:dfinr:s:u:v")) != -1) {
+ while ((i = getopt(argc, argv, "EINb:c:dfimnr:s:u:v")) != -1) {
switch (i) {
case 'u':
/*
@@ -171,6 +173,10 @@ main(int argc, char **argv)
*/
subject = optarg;
break;
+ case 'm':
+ /* Add MIME headers */
+ mime = 1;
+ break;
case 'f':
/*
* User is specifying file to "edit" with Mail,
@@ -337,7 +343,7 @@ __dead void
usage(void)
{
- fprintf(stderr, "usage: %s [-dEIinv] [-b list] [-c list] "
+ fprintf(stderr, "usage: %s [-dEIimnv] [-b list] [-c list] "
"[-r from-addr] [-s subject] to-addr ...\n", __progname);
fprintf(stderr, " %s [-dEIiNnv] -f [file]\n", __progname);
fprintf(stderr, " %s [-dEIiNnv] [-u user]\n", __progname);
Index: send.c
===
RCS file: /cvs/src/usr.bin/mail/send.c,v
retrieving revision 1.26
diff -u -p -r1.26 send.c
--- send.c 8 Mar 2023 04:43:11 - 1.26
+++ send.c 20 Sep 2023 10:44:41 -
@@ -525,6 +525,8 @@ puthead(struct header *hp, FILE *fo, int
fmt("To:", hp->h_to, fo, w), gotcha++;
if (hp->h_subject != NULL && w & GSUBJECT)
fprintf(fo, "Subject: %s\n", hp->h_subject), gotcha++;
+ if (mime)
+ fprintf(fo, "MIME-Version: 1.0\nContent-Type: text/plain;
charset=utf-8\nContent-Transfer-Encoding: 8bit\n"), gotcha++;
if (hp->h_cc != NULL && w & GCC)
fmt("Cc:", hp->h_cc, fo, w), gotcha++;
if (hp->h_bcc != NULL && w & GBCC)
--
Walter
Re: Send international text with mail(1) - proposal and patches
• Walter Alejandro Iglesias [2023-09-20 12:21]:
Hi Ingo,
I did what you suggested me, I investigated a bit and you were right in
that the MIME-Version header was necessary. This new set of patches
add the following headers (hardcoded as you suggested me):
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
I modified the code the less as possible, just a '-m' option:
$ mail -m -s Hello d...@ext.net < body_message_in_utf8
you may not even need -m, and instead inspect LC_CTYPE environment
variable and add appropriate headers for UTF-8. according to locale(1),
LC_CTYPE may be set to indicate UTF-8:
If the value of LC_CTYPE ends in ‘.UTF-8’, programs in the OpenBSD base
system ignore the beginning of it, treating for example zh_CN.UTF-8
exactly like en_US.UTF-8.
Although, to tell the truth, I'm not really convinced if this change is
worth it. Feel free to ignore it.
Index: extern.h
===
RCS file: /cvs/src/usr.bin/mail/extern.h,v
retrieving revision 1.29
diff -u -p -r1.29 extern.h
--- extern.h16 Sep 2018 02:38:57 - 1.29
+++ extern.h20 Sep 2023 09:55:06 -
@@ -261,3 +261,4 @@ int writeback(FILE *);
extern char *__progname;
extern char *tmpdir;
extern const struct cmd *com; /* command we are running */
+extern char mime; /* Add MIME headers */
Index: mail.1
===
RCS file: /cvs/src/usr.bin/mail/mail.1,v
retrieving revision 1.83
diff -u -p -r1.83 mail.1
--- mail.1 31 Mar 2022 17:27:25 - 1.83
+++ mail.1 20 Sep 2023 09:55:06 -
@@ -106,6 +106,8 @@ on noisy phone lines.
.It Fl N
Inhibits initial display of message headers
when reading mail or editing a mail folder.
+.It Fl m
+Add MIME headers to send utf-8 encoded messages.
.It Fl n
Inhibits reading
.Pa /etc/mail.rc
Index: main.c
===
RCS file: /cvs/src/usr.bin/mail/main.c,v
retrieving revision 1.35
diff -u -p -r1.35 main.c
--- main.c 26 Jan 2021 18:21:47 - 1.35
+++ main.c 20 Sep 2023 09:55:06 -
@@ -79,6 +79,8 @@ int realscreenheight; /* the real scree
int uflag; /* Are we in -u mode? */
sigset_t intset; /* Signal set that is just SIGINT */
+char mime = 0;/* Add MIME headers */
+
/*
* The pointers for the string allocation routines,
* there are NSPACE independent areas.
@@ -136,7 +138,7 @@ main(int argc, char **argv)
smopts = NULL;
fromaddr = NULL;
subject = NULL;
- while ((i = getopt(argc, argv, "EINb:c:dfinr:s:u:v")) != -1) {
+ while ((i = getopt(argc, argv, "EINb:c:dfimnr:s:u:v")) != -1) {
switch (i) {
case 'u':
/*
@@ -171,6 +173,10 @@ main(int argc, char **argv)
*/
subject = optarg;
break;
+ case 'm':
+ /* Add MIME headers */
+ mime = 1;
+ break;
case 'f':
/*
* User is specifying file to "edit" with Mail,
@@ -337,7 +343,7 @@ __dead void
usage(void)
{
- fprintf(stderr, "usage: %s [-dEIinv] [-b list] [-c list] "
+ fprintf(stderr, "usage: %s [-dEIimnv] [-b list] [-c list] "
"[-r from-addr] [-s subject] to-addr ...\n", __progname);
fprintf(stderr, " %s [-dEIiNnv] -f [file]\n", __progname);
fprintf(stderr, " %s [-dEIiNnv] [-u user]\n", __progname);
Index: send.c
===
RCS file: /cvs/src/usr.bin/mail/send.c,v
retrieving revision 1.26
diff -u -p -r1.26 send.c
--- send.c 8 Mar 2023 04:43:11 - 1.26
+++ send.c 20 Sep 2023 09:55:06 -
@@ -525,6 +525,8 @@ puthead(struct header *hp, FILE *fo, int
fmt("To:", hp->h_to, fo, w), gotcha++;
if (hp->h_subject != NULL && w & GSUBJECT)
fprintf(fo, "Subject: %s\n", hp->h_subject), gotcha++;
+ if (mime)
+ fprintf(fo, "MIME-Version: 1.0\nContent-Type: text/plain;
charset=utf-8\nContent-Transfer-Encoding: 8bit\n"), gotcha++;
if (hp->h_cc != NULL && w & GCC)
fmt("Cc:", hp->h_cc, fo, w), gotcha++;
if (hp->h_bcc != NULL && w & GBCC)
Re: Send international text with mail(1) - proposal and patches
Hi Ingo,
I did what you suggested me, I investigated a bit and you were right in
that the MIME-Version header was necessary. This new set of patches
add the following headers (hardcoded as you suggested me):
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
I modified the code the less as possible, just a '-m' option:
$ mail -m -s Hello d...@ext.net < body_message_in_utf8
Although, to tell the truth, I'm not really convinced if this change is
worth it. Feel free to ignore it.
Index: extern.h
===
RCS file: /cvs/src/usr.bin/mail/extern.h,v
retrieving revision 1.29
diff -u -p -r1.29 extern.h
--- extern.h16 Sep 2018 02:38:57 - 1.29
+++ extern.h20 Sep 2023 09:55:06 -
@@ -261,3 +261,4 @@ int writeback(FILE *);
extern char *__progname;
extern char *tmpdir;
extern const struct cmd *com; /* command we are running */
+extern char mime; /* Add MIME headers */
Index: mail.1
===
RCS file: /cvs/src/usr.bin/mail/mail.1,v
retrieving revision 1.83
diff -u -p -r1.83 mail.1
--- mail.1 31 Mar 2022 17:27:25 - 1.83
+++ mail.1 20 Sep 2023 09:55:06 -
@@ -106,6 +106,8 @@ on noisy phone lines.
.It Fl N
Inhibits initial display of message headers
when reading mail or editing a mail folder.
+.It Fl m
+Add MIME headers to send utf-8 encoded messages.
.It Fl n
Inhibits reading
.Pa /etc/mail.rc
Index: main.c
===
RCS file: /cvs/src/usr.bin/mail/main.c,v
retrieving revision 1.35
diff -u -p -r1.35 main.c
--- main.c 26 Jan 2021 18:21:47 - 1.35
+++ main.c 20 Sep 2023 09:55:06 -
@@ -79,6 +79,8 @@ int realscreenheight; /* the real scree
intuflag; /* Are we in -u mode? */
sigset_t intset; /* Signal set that is just SIGINT */
+char mime = 0; /* Add MIME headers */
+
/*
* The pointers for the string allocation routines,
* there are NSPACE independent areas.
@@ -136,7 +138,7 @@ main(int argc, char **argv)
smopts = NULL;
fromaddr = NULL;
subject = NULL;
- while ((i = getopt(argc, argv, "EINb:c:dfinr:s:u:v")) != -1) {
+ while ((i = getopt(argc, argv, "EINb:c:dfimnr:s:u:v")) != -1) {
switch (i) {
case 'u':
/*
@@ -171,6 +173,10 @@ main(int argc, char **argv)
*/
subject = optarg;
break;
+ case 'm':
+ /* Add MIME headers */
+ mime = 1;
+ break;
case 'f':
/*
* User is specifying file to "edit" with Mail,
@@ -337,7 +343,7 @@ __dead void
usage(void)
{
- fprintf(stderr, "usage: %s [-dEIinv] [-b list] [-c list] "
+ fprintf(stderr, "usage: %s [-dEIimnv] [-b list] [-c list] "
"[-r from-addr] [-s subject] to-addr ...\n", __progname);
fprintf(stderr, " %s [-dEIiNnv] -f [file]\n", __progname);
fprintf(stderr, " %s [-dEIiNnv] [-u user]\n", __progname);
Index: send.c
===
RCS file: /cvs/src/usr.bin/mail/send.c,v
retrieving revision 1.26
diff -u -p -r1.26 send.c
--- send.c 8 Mar 2023 04:43:11 - 1.26
+++ send.c 20 Sep 2023 09:55:06 -
@@ -525,6 +525,8 @@ puthead(struct header *hp, FILE *fo, int
fmt("To:", hp->h_to, fo, w), gotcha++;
if (hp->h_subject != NULL && w & GSUBJECT)
fprintf(fo, "Subject: %s\n", hp->h_subject), gotcha++;
+ if (mime)
+ fprintf(fo, "MIME-Version: 1.0\nContent-Type: text/plain;
charset=utf-8\nContent-Transfer-Encoding: 8bit\n"), gotcha++;
if (hp->h_cc != NULL && w & GCC)
fmt("Cc:", hp->h_cc, fo, w), gotcha++;
if (hp->h_bcc != NULL && w & GBCC)
--
Walter
Re: malloc write after free error checking
On Sun, Sep 03, 2023 at 09:21:18AM +0200, Otto Moerbeek wrote: > Hello, > > I'm seeing some reports of "write after free" reported by malloc by > peolpe running current. Malloc has become more strict since begining > of June. Let me explain: > > Small allocations share a page. e.g. a 4k page will hold 8 512 byte > allocations. > > When one such allocation is freed, it will be filled with a "free > junk" pattern, stay for a short while in the delayed free list (and be > subhject to write after free checks) and after that it will be marked > free and it might be allocated again. On a new allocation the write > after free check will also be done. That is the new part of my commit > in June. Another change is that on the allocation of the page > containing chunks, all chunks wil be filled with a "free junk" (0xdf) > patttern. > > The change has a consequence: the time span of the "write after free" > checks is much longer, as it can take a long time for a chunk to get > used again. > > I do believe the changes itself are correct and can help findiung > bugs in other code. > > You can also be set upon a wrong foot: if an out of bounds write on a > adjacent chunk happens and lands in (another) free chunk, upon > allocation of that free chunk it will be reported as a "write after > free" case. It might even be that an allocation that was never > allocated and never freed is still "write after free" because of > "close" out of bounds writes. > > I'm thinking how to change the error message to reflect that. > > If these extra checks hinder progress, it is possible to swicth them > off using malloc option f (small F). That should be j (small J). > For general debugging I recommend using malloc option S, is has all > the extra strictness that can be enabled while still conforming to the > malloc API. > > Please be aware of these things while hunting for bugs. I'm happy to say two of the more complex ones are (being) fixed: one turned out to be a reference counting bug in firefox. See http://undeadly.org/cgi?action=article;sid=20230912094727 The other, a write after free that crashed the X server when running picard was diagnosed by me. This one was a bit nasty, as it required instrumenting malloc to print some extra info to find the root cause. The bug is that the call in https://github.com/openbsd/xenocara/blob/master/xserver/Xext/xvdisp.c#L1002 overwrites the first 4 bytes of the chunk next to the one allocated on line 995. A workaround is to allocate 4 bytes extra, matthieu@ will be looking for a proper fix, as it requires knowledge of the X internals. I am pondering if the instrumentation hack I used could be modified to be always available and print the function that did the original allocation when detecting a write after free. The conclusion is that in these two cases, malloc helped in detecting/finding the bugs (and it was not a bug in malloc itself :-). Coincidentally this was the topioc of my talk during EuroBSD2023 last weekend, see http://www.openbsd.org/events.html I hope to look at the reports I got for the Wayland port (which is WIP) as well, see https://github.com/openbsd/ports/blob/master/wayland/TODO-Wayland.md -Otto
