Subject: kate: A NUL byte in not an invalid utf8 character.
Package: kate
Version: 4:18.04.0-1
Severity: normal
Dear Maintainer,
While trying to edit a file I got a "opened with UTF-8 encoding but
contained invalid characters." error.
There was no indication of which or where in the file were this invalid
characters.
Filtering the file with:
iconv -c -t utf8 testfile.txt >testfile.txt.new
Made no modifications to the file (all characters were accepted).
Then I tested the file with:
isutf8 -v testfile.txt
Which also reported no error.
Then I used:
grep -axv '.*' testfile.txt
Which also found no error.
The only solution was to open the file in kate, force editing and force to
write a new file.
The difference bettween both files revealed that the removed characters
were NULs.
That is bytes wit value of zero: 0x00 a.k.a \0
You can reproduce by creating a file in the command line with:
printf 'A good line\nA line with a NUL\x00\n' >testfile.txt
Then, try to edit with kate.
A zero byte is a valid UTF-8 character as reported by the command:
$ unicode -x 0
U+0000 <control>
UTF-8: 00 UTF-16BE: 0000 Decimal: � Octal: \00
Category: Cc (Other, Control)
Unicode block: 0000..007F; Basic Latin
Bidi: BN (Boundary Neutral)
It can be tested with some other tools also.
Please:
1.- Provide a way to "see" or list the characters reported as wrong by
kate
2.- Do not include a NUL as invalid utf8.
-- System Information:
Debian Release: buster/sid
APT prefers testing
APT policy: (500, 'testing'), (500, 'stable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 4.16.0-1-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_US.utf8, LC_CTYPE=en_US.utf8 (charmap=UTF-8), LANGUAGE=
(charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages kate depends on:
ii kate5-data 4:18.04.0-1
ii kio 5.44.0-2
ii ktexteditor-katepart 5.44.0-1
ii libc6 2.27-3
ii libkf5activities5 5.44.0-1
ii libkf5bookmarks5 5.44.0-1
ii libkf5completion5 5.44.0-1
ii libkf5configcore5 5.44.0-1
ii libkf5configgui5 5.44.0-1
ii libkf5configwidgets5 5.44.0-1
ii libkf5coreaddons5 5.44.0-1
ii libkf5crash5 5.44.0-1
ii libkf5dbusaddons5 5.44.0-1
ii libkf5guiaddons5 5.44.0-1
ii libkf5i18n5 5.44.0-1
ii libkf5iconthemes5 5.44.0-1
ii libkf5itemmodels5 5.44.0-1
ii libkf5jobwidgets5 5.44.0-1
ii libkf5kiocore5 5.44.0-2
ii libkf5kiofilewidgets5 5.44.0-2
ii libkf5kiowidgets5 5.44.0-2
ii libkf5newstuff5 5.44.0-1
ii libkf5parts5 5.44.0-1
ii libkf5plasma5 5.44.0-1
ii libkf5service-bin 5.44.0-1
ii libkf5service5 5.44.0-1
ii libkf5texteditor5 5.44.0-1
ii libkf5textwidgets5 5.44.0-1
ii libkf5threadweaver5 5.44.0-1
ii libkf5wallet-bin 5.44.0-1
ii libkf5wallet5 5.44.0-1
ii libkf5widgetsaddons5 5.44.0-1
ii libkf5windowsystem5 5.44.0-1
ii libkf5xmlgui5 5.44.0-2+b1
ii libqt5core5a 5.10.1+dfsg-6
ii libqt5dbus5 5.10.1+dfsg-6
ii libqt5gui5 5.10.1+dfsg-6
ii libqt5sql5 5.10.1+dfsg-6
ii libqt5widgets5 5.10.1+dfsg-6
ii libqt5xml5 5.10.1+dfsg-6
ii libstdc++6 8.1.0-1
ii plasma-framework 5.44.0-1
ii qml-module-org-kde-kquickcontrolsaddons 5.44.0-1+b1
ii qml-module-qtquick-layouts 5.10.1-4
ii qml-module-qtquick2 5.10.1-4
Versions of packages kate recommends:
ii sonnet-plugins 5.44.0-1
Versions of packages kate suggests:
pn darcs <none>
pn exuberant-ctags <none>
pn git <none>
ii khelpcenter 4:17.08.3-1
ii konsole-kpart 4:18.04.0-1
pn mercurial <none>
pn subversion <none>
-- no debconf information
