Steven Atreju wrote:
Once more i want to point out that on Unix/POSIX systems the file
content can be seen as a whole, and i hope and think that this
will not change. This situation is completely different than on
Windows, which had textfiles with appended (separated by ^Z or so)
meta information that was invisible in normal text editors already
in the ninetees (or even earlier, but i don't know).
^Z as an EOF marker for text files was part of the MS-DOS legacy from
CP/M, where all files were written to a multiple of the disk block size
(I think 128 for CP/M and 512 for MS-DOS 1.x), and there had to be some
way to tell where the real text content ended. New stream-based I/O
calls in MS-DOS 2.0 made this mechanism unnecessary. Unix systems had no
legacy from CP/M, so they never had this problem.
I.e., this is why we do have this messy text OR binary file I/O
distinction like O_BINARY (for open(2)), "b" (for fopen(3)) or
binmode (perl(1)). Because without those a text file will see
End-Of-File at the ^Z, not at the real end of the file.
The reason for the text/binary distinction on DOS and Windows is
conversion between Unix-standard LF and Windows (DOS, CP/M)-standard
CRLF. It might be true that library calls to read a file in text mode
will stop at ^Z, but Notepad and Wordpad don't. I know the library
doesn't automatically write ^Z. Almost nobody in the MS world uses the
^Z convention on purpose any more; many don't even know about it.
(Which rises the immediate question why the Microsoft programmers did
not embed the meta information in this section at the end of the file.
But i don't really want to know.)
See above. The intent of ^Z was never to distinguish data from metadata,
as with the Mac data and resource forks.
But of course none of this has anything to do with U+FEFF.
So do the programmers have to face the same conditions? I don't
really think so. They prefer driving plain text readers up the wall.
Successfully.
Again, we don't really have this kind of evil intent, though it's often
fun and convenient for people to imagine we do.
--
Doug Ewell | Thornton, Colorado, USA
http://www.ewellic.org | @DougEwell
