I had a curious bug appear in my server logs when using a unicode
Transformer:
transform unicode "wind-Pa\x00\x00\x00" to ascii: transform: short
destination buffer
Here's the simplified code that caused the error (Gist
<https://gist.github.com/jschaf/78933954cba1ae6fc75b3d29a4c6fc8d> and Go
Playground <https://go.dev/play/p/6Rb_sZBTs2P>). I assumed that converting
from unicode to ascii would always have an equal or smaller length, hence
the panic. Here's the essential bits of the simplified code:
cs := []byte("wind-Pa\x00\x00\x00")
chars := make([]byte, len(cs))
t := transform.Chain(norm.NFD, runes.Remove(runes.In(unicode.Mn)),
norm.NFC)
nDst, _, err := t.Transform(chars, cs, true)
I suspect the error is thrown by text/runes.go:149
<https://github.com/golang/text/blob/master/runes/runes.go#L149> (or
perhaps on line 165) by the remove transformer. It looks like the form
transformers never throw ErrShortDestination.
I haven't been able to reproduce the error on my dev mac or on the
playground and there's only been a single occurrence of the error in my
server logs. The server binary was compiled with Bazel for
the @io_bazel_rules_go//go/toolchain:linux_amd64 toolchain using Go
version 1.18.4.
I'd like to understand when ErrShortDestination is thrown by the
Transformer. My code allocates a buffer the same length as the input so I
thought I'd avoid the short destination error.
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/6355b258-3c25-485c-8b42-d9cd6a85a15dn%40googlegroups.com.
