> But thats only loading files as UTF-8, any other encoding is going to depend > on the system iconv implementation underlying the `g_convert()`
Indeed, but I feel like those are somewhat likely to support whichever bytes, as it's basically the point in converting encodings -- but indeed I don't know. And on my system it seems like it does work fine for what I tested. > But why saving it? Its readonly, so it won't change. All I did is fix it so it works, mostly because it was so easy -- and there was a very simple but weird bug in there not only easy to fix but also very confusing when looking at the code. So no, my idea was not to suggest we can work fully with files embedding NUL bytes and that then we need to be able to save them, but that why not fix this very important part for any future possibility in that area if it's so easy anyway. And encoding issue when saving will be the very same with NULs as any other byte: if the converter is happy with them it'll work, otherwise it won't. The problem was that UTF-8 files were artificially truncated at the first NUL, but actually non-UTF-8 were fine embedding NULs – which actually is likely to be important because I would imagine some encodings might generate NUL bytes in their encoded stream anyway (or at least could anyway). Anyway: what I fixed was simply that now we properly write the whole thing in any case, with no corner case doing otherwise. > The obvious one that comes to mind is actually fixing the file by removing > the NUL, which IIRC Scintilla helpfully displays using a special little > icon/glyph. Yes (like #1708), and yes (it displays it like any other control character, in a little box with the character name). And again, regex search actually works pretty OK with NULs, you can actually search for NUL bytes with them. Non-regex search doesn't, however (could be interesting to fix, but for now it's out of the scope of this). Anyway, despite the "WIP" label, I don't really plan on working on fixing everything so it accepts NULs. I started it to see how hard it would be to at least simply load such files properly so one could look at them, also thinking that if that part is done it's more likely anyone would work on fixing some other code to handle those. So @elextr I think the goal of this PR matches your hopes: loading files, but nothing else -- and warning users it's not properly supported and can cause issues. But if one day we end up with full support for those, I don't think it would be a bad thing. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/geany/geany/pull/1709#issuecomment-349418916
