Re: Can we get rid of non-raw write?
Vetoed after several years of nothing: https://issues.dlang.org/show_bug.cgi?id=9776#c7 I'm getting really fucking tired of D making up excuses to throw "do the right thing by default" straight into the gutter. D steering didn't used to be this way, and that was exactly what make D into something worthwhile in the first place. Now we're just diving head-first into C++-management (minus the committes).
Re: Can we get rid of non-raw write?
On Wednesday, 15 April 2015 at 14:47:46 UTC, armando sano wrote:
Reviving old topic... It is possible to force stdout to write
in binary mode on Windows, see
https://msdn.microsoft.com/en-us/library/tw4k6df8.aspx
In C, the solution is:
-
#include stdio.h
#include fcntl.h
#include io.h
/*...*/
int result = _setmode( _fileno( stdout ), _O_BINARY );
if ( result == -1 )
perror (Cannot set stdout to binary mode);
else
perror (stdout set to binary mode);
--
In Python, the solution is:
--
import platform
if platform.system() == Windows:
import os, msvcrt
msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
--
Since D can interface C, it must be possible to do the same in
D? (how, I am not sure)
my humble solution:
void setFileModeBinary(File f)
{
import std.c.stdlib;
version(Windows) {
immutable fd = _fileno(f.getFP);
f.flush();
_setmode(fd, _O_BINARY);
version(DigitalMars) {
// @@@BUG@@@ 4243
immutable info = __fhnd_info[fd];
__fhnd_info[fd] = ~FHND_TEXT;
}
}
}
Re: Can we get rid of non-raw write?
Reviving old topic... It is possible to force stdout to write in binary mode on Windows, see https://msdn.microsoft.com/en-us/library/tw4k6df8.aspx In C, the solution is: - #include stdio.h #include fcntl.h #include io.h /*...*/ int result = _setmode( _fileno( stdout ), _O_BINARY ); if ( result == -1 ) perror (Cannot set stdout to binary mode); else perror (stdout set to binary mode); -- In Python, the solution is: -- import platform if platform.system() == Windows: import os, msvcrt msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY) -- Since D can interface C, it must be possible to do the same in D? (how, I am not sure)
Re: Can we get rid of non-raw write?
On 4/15/15 10:47 AM, armando sano wrote: Reviving old topic... It is possible to force stdout to write in binary mode on Windows, see https://msdn.microsoft.com/en-us/library/tw4k6df8.aspx In C, the solution is: - #include stdio.h #include fcntl.h #include io.h /*...*/ int result = _setmode( _fileno( stdout ), _O_BINARY ); if ( result == -1 ) perror (Cannot set stdout to binary mode); else perror (stdout set to binary mode); -- Just a warning, binary mode is a C feature, not an OS feature. So you have to call the functions that are relevant to the C library you are using. On Windows, this could be DMC or MSVCRT. I'm not 100% sure the DMC way would be the same as above. -Steve
Re: Can we get rid of non-raw write?
On Wednesday, 15 April 2015 at 15:17:27 UTC, Jürgen Reichmann
wrote:
On Wednesday, 15 April 2015 at 14:47:46 UTC, armando sano wrote:
Reviving old topic... It is possible to force stdout to write
in binary mode on Windows, see
https://msdn.microsoft.com/en-us/library/tw4k6df8.aspx
In C, the solution is:
-
#include stdio.h
#include fcntl.h
#include io.h
/*...*/
int result = _setmode( _fileno( stdout ), _O_BINARY );
if ( result == -1 )
perror (Cannot set stdout to binary mode);
else
perror (stdout set to binary mode);
--
In Python, the solution is:
--
import platform
if platform.system() == Windows:
import os, msvcrt
msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
--
Since D can interface C, it must be possible to do the same in
D? (how, I am not sure)
my humble solution:
void setFileModeBinary(File f)
{
import std.c.stdlib;
version(Windows) {
immutable fd = _fileno(f.getFP);
f.flush();
_setmode(fd, _O_BINARY);
version(DigitalMars) {
// @@@BUG@@@ 4243
immutable info = __fhnd_info[fd];
__fhnd_info[fd] = ~FHND_TEXT;
}
}
}
Sorry, solution above is no longer valid.
Version below works for DMD 2.066 and 2.067 (tested for X86).
void setFileModeBinary(File f, bool setBinary = true)
{
// extracted from phobos stdio rawWrite
version(Windows) {
import std.stdio, std.c.stdlib;
f.flush(); // before changing translation mode
immutable fd = _fileno(f.getFP);
if (setBinary) {
_setmode(fd, _O_BINARY);
version(CRuntime_DigitalMars) { // D2.067
import core.atomic;
// @@@BUG@@@ 4243
atomicOp!=(__fhnd_info[fd], ~FHND_TEXT);
} else version(DigitalMars) { // D2.066
version (Win32) {
import core.atomic;
// @@@BUG@@@ 4243
atomicOp!=(__fhnd_info[fd],
~FHND_TEXT);
}
}
} else {
version (MICROSOFT_STDIO) {}
else {
enum _O_TEXT = 0x4000;
}
_setmode(fd, _O_TEXT);
version(CRuntime_DigitalMars) { // D2.067
import core.atomic;
// @@@BUG@@@ 4243
atomicOp!=(__fhnd_info[fd], ~FHND_TEXT);
} else version(DigitalMars) { // D2.066
version (Win32) {
import core.atomic;
// @@@BUG@@@ 4243
atomicOp!|=(__fhnd_info[fd],
FHND_TEXT);
}
}
}
}
}
IMO a function like this belongs in Phobos stdio
jürgen
Re: Can we get rid of non-raw write?
On Wednesday, 15 April 2015 at 17:53:24 UTC, Jürgen Reichmann
wrote:
On Wednesday, 15 April 2015 at 15:17:27 UTC, Jürgen Reichmann
wrote:
On Wednesday, 15 April 2015 at 14:47:46 UTC, armando sano
wrote:
Reviving old topic... It is possible to force stdout to write
in binary mode on Windows, see
https://msdn.microsoft.com/en-us/library/tw4k6df8.aspx
In C, the solution is:
-
#include stdio.h
#include fcntl.h
#include io.h
/*...*/
int result = _setmode( _fileno( stdout ), _O_BINARY );
if ( result == -1 )
perror (Cannot set stdout to binary mode);
else
perror (stdout set to binary mode);
--
In Python, the solution is:
--
import platform
if platform.system() == Windows:
import os, msvcrt
msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
--
Since D can interface C, it must be possible to do the same
in D? (how, I am not sure)
my humble solution:
void setFileModeBinary(File f)
{
import std.c.stdlib;
version(Windows) {
immutable fd = _fileno(f.getFP);
f.flush();
_setmode(fd, _O_BINARY);
version(DigitalMars) {
// @@@BUG@@@ 4243
immutable info = __fhnd_info[fd];
__fhnd_info[fd] = ~FHND_TEXT;
}
}
}
Sorry, solution above is no longer valid.
Version below works for DMD 2.066 and 2.067 (tested for X86).
void setFileModeBinary(File f, bool setBinary = true)
{
// extracted from phobos stdio rawWrite
version(Windows) {
import std.stdio, std.c.stdlib;
f.flush(); // before changing translation mode
immutable fd = _fileno(f.getFP);
if (setBinary) {
_setmode(fd, _O_BINARY);
version(CRuntime_DigitalMars) { // D2.067
import core.atomic;
// @@@BUG@@@ 4243
atomicOp!=(__fhnd_info[fd], ~FHND_TEXT);
} else version(DigitalMars) { // D2.066
version (Win32) {
import core.atomic;
// @@@BUG@@@ 4243
atomicOp!=(__fhnd_info[fd],
~FHND_TEXT);
}
}
} else {
version (MICROSOFT_STDIO) {}
else {
enum _O_TEXT = 0x4000;
}
_setmode(fd, _O_TEXT);
version(CRuntime_DigitalMars) { // D2.067
import core.atomic;
// @@@BUG@@@ 4243
atomicOp!=(__fhnd_info[fd], ~FHND_TEXT);
} else version(DigitalMars) { // D2.066
version (Win32) {
import core.atomic;
// @@@BUG@@@ 4243
atomicOp!|=(__fhnd_info[fd],
FHND_TEXT);
}
}
}
}
}
IMO a function like this belongs in Phobos stdio
jürgen
Thanks for posting your solution Jürgen, works great! I agree it
would be a nice addition to stdio (certainly one I was looking
for for a while)
Re: Can we get rid of non-raw write?
On Thursday, 21 March 2013 at 05:01:06 UTC, Kagamin wrote: On Wednesday, 20 March 2013 at 14:34:21 UTC, Nick Sabalausky wrote: Since *at least* as far back as XP, Windows has handled \n newlines perfectly fine. The command line displays them properly, .BAT scripts handle them properly, every code editor in existence handles them properly. The *only* thing I've found that doesn't is Windows Notepad, but really, whoTF uses that anyway? Notepad is used as a log viewer. When you're presented a machine you don't run, you usually don't have time to setup your preferred work environment, notepad comes in handy to just view the log and diagnose an error. My workaround in such cases is to use Write, which is also installed by default and handles the files correctly. I think Notepad still uses the same code from Windows 3.x days! :) -- Paulo
Re: Can we get rid of non-raw write?
On 2013-03-20 22:08, Nick Sabalausky wrote: I doubt there's a bugzilla entry for this since it is, unfortunately, the intended behavior. I didn't want to go posting a 'zilla issue for it before discussing here because I figured that might just end up INVALID or WONTFIX. If you make it an enhancement request it shouldn't at least end up as INVALID. -- /Jacob Carlborg
Re: Can we get rid of non-raw write?
On 2013-03-20 23:00, Graham Fawcett wrote: Is anyone still using MacOS earlier than version 10 (OSX)? Mac OS 9 was discontinued in 2002. On OSX, there's certainly no problem with Unix line endings. But I guess if we include ancient Windows support as an option, then ancient Mac support should in be there too. But writeln/writefln should emit '\n' as a line terminator, by default, on all platforms. Ancient terminators should be always opt-in, regardless of platform. I agree. -- /Jacob Carlborg
Re: Can we get rid of non-raw write?
On Thu, 21 Mar 2013 10:41:35 +0100 Jacob Carlborg d...@me.com wrote: On 2013-03-20 22:08, Nick Sabalausky wrote: I doubt there's a bugzilla entry for this since it is, unfortunately, the intended behavior. I didn't want to go posting a 'zilla issue for it before discussing here because I figured that might just end up INVALID or WONTFIX. If you make it an enhancement request it shouldn't at least end up as INVALID. http://d.puremagic.com/issues/show_bug.cgi?id=9776
Re: Can we get rid of non-raw write?
21-Mar-2013 09:01, Kagamin пишет: On Wednesday, 20 March 2013 at 14:34:21 UTC, Nick Sabalausky wrote: Since *at least* as far back as XP, Windows has handled \n newlines perfectly fine. The command line displays them properly, .BAT scripts handle them properly, every code editor in existence handles them properly. The *only* thing I've found that doesn't is Windows Notepad, but really, whoTF uses that anyway? Notepad is used as a log viewer. One word - Wordpad. Any relatively interesting log file is measured in MBs thus hanging the notepad on as said random machine. When you're presented a machine you don't run, you usually don't have time to setup your preferred work environment, notepad comes in handy to just view the log and diagnose an error. -- Dmitry Olshansky
Re: Can we get rid of non-raw write?
On 20.03.2013 15:34, Nick Sabalausky wrote:
Since *at least* as far back as XP, Windows has handled \n newlines
perfectly fine. The command line displays them properly, .BAT scripts
handle them properly, every code editor in existence handles them
properly. The *only* thing I've found that doesn't is Windows Notepad,
but really, whoTF uses that anyway?
So, why are silently and forcefully converting \n to \r\n on
windows by default? All it does is cause bugs. For example:
https://github.com/repeatedly/mustache-d/issues/3
And that's definitely not the first time I've run into problems due
using the write* functions instead of rawWrite.
Consider this straightforward code:
--
import std.file;
import std.stdio;
void transform(string str)
{
/+ ...perform some modification of 'str'... +/
return str;
}
void main()
{
auto str = cast(string) read(args[1]);
str = transform(str);
write(str);
}
--
That simple code is *wrong*:
It works correctly for all input on Unix: Output newlines match input
newlines. Always. The code never asks for newlines to be messed with,
and therefore they never are.
You're mixing binary and text mode functions. read() is binary,
stdout.write() is text mode. And yes, you are asking for newlines to be
messed with, as File.write is documented to write in text mode.
But I agree that the docs need improvement. And maybe the API.
Re: Can we get rid of non-raw write?
On Thu, 21 Mar 2013 23:37:06 +0100 torhu no@spam.invalid wrote: You're mixing binary and text mode functions. read() is binary, stdout.write() is text mode. And yes, you are asking for newlines to be messed with, as File.write is documented to write in text mode. But I agree that the docs need improvement. And maybe the API. You're missing the point. The point is that the text mode is bug-prone, grossly obsolete, and completely useless and therefore should absolutely not be the default, *if* it has any reason to even exist at all. We could toss a function output into Phobos and document it as being fubar mode, which converts every third word into 'DERP', but obviously just because its behavior matches the description doesn't justify its existence or its usage of such a generic name. The current write function is every bit as useless as this hypothetical output function, but it's more dangerous because it's harder to notice you're getting the wrong result.
Re: Can we get rid of non-raw write?
On 22.03.2013 00:36, Nick Sabalausky wrote: On Thu, 21 Mar 2013 23:37:06 +0100 torhu no@spam.invalid wrote: You're mixing binary and text mode functions. read() is binary, stdout.write() is text mode. And yes, you are asking for newlines to be messed with, as File.write is documented to write in text mode. But I agree that the docs need improvement. And maybe the API. You're missing the point. The point is that the text mode is bug-prone, grossly obsolete, and completely useless and therefore should absolutely not be the default, *if* it has any reason to even exist at all. Text mode isn't going away, I don't know where you'd get that idea from. I'm sure Microsoft wants Linux to go away, too. If you want a to finance a FUD campaign to hurt the reputation of text mode, be my guest. Or keep on ranting, who's gonna stop you.
Re: Can we get rid of non-raw write?
20-Mar-2013 18:34, Nick Sabalausky пишет:
Since *at least* as far back as XP, Windows has handled \n newlines
perfectly fine. The command line displays them properly, .BAT scripts
handle them properly, every code editor in existence handles them
properly. The *only* thing I've found that doesn't is Windows Notepad,
but really, whoTF uses that anyway?
So, why are silently and forcefully converting \n to \r\n on
windows by default? All it does is cause bugs. For example:
https://github.com/repeatedly/mustache-d/issues/3
And that's definitely not the first time I've run into problems due
using the write* functions instead of rawWrite.
Consider this straightforward code:
--
import std.file;
import std.stdio;
void transform(string str)
{
/+ ...perform some modification of 'str'... +/
return str;
}
void main()
{
auto str = cast(string) read(args[1]);
str = transform(str);
write(str);
}
--
That simple code is *wrong*:
It works correctly for all input on Unix: Output newlines match input
newlines. Always. The code never asks for newlines to be messed with,
and therefore they never are.
But on Windows the behavior is just plain weird: Unix-style newlines in
the input are silently and forcefully converted to Windows-style
newlines behind the user's back. And even worse yet, *Windows*-style
newlines on the input are converted to a bizarre Mac9 plus
Windows-style combination of \r\r\n (not only is that wrong period,
but this sequence is often interpreted as two newlines which makes it
even worse). Using rawWrite fixes the problem, and creates *no* problem.
I feel like the current write* behavior is a design Steve Jobs would
have come up with.
So how is this useful to anyone, and how is it worthy of being the
default output behavior?
No. Let's obliterate it. The only problem is compatibility and the fact
that Windows has had this text mode for I/O in e.g. MSVCRT for a long
time. It never helped me in any sensible way except that yay! I can
view it in notepad! but broken many things far too often.
Even with Windows as my primary system, there hasn't been one single
time write*'s output-altering feature has helped me by doing
something more correctly than rawWrite would have done. And that's just
when I'm dealing with pure text. I'm literally better off just using
rawWrite everywhere, which is exactly what I intend to do from now on
(and I'll be viewing uses of write* with suspicion as latent bugs) -
unlike write*, rawWrite *never* does the wrong thing.
Can we please get rid of this text-mode output? Or at least eliminate
it as the default?
--
Dmitry Olshansky
Re: Can we get rid of non-raw write?
On Wednesday, 20 March 2013 at 14:34:21 UTC, Nick Sabalausky wrote: Can we please get rid of this text-mode output? Or at least eliminate it as the default? +1. I've had to work around this issue several times by doing strange things like stripping the line ending and putting a \n at the end. I figured it was something I was just missing but seeing someone else mention the problem makes me think this is erroneous behavior. Is this in bugzilla?
Re: Can we get rid of non-raw write?
On Wed, 20 Mar 2013 21:13:54 +0100 Chris Cain clc...@uncg.edu wrote: On Wednesday, 20 March 2013 at 14:34:21 UTC, Nick Sabalausky wrote: Can we please get rid of this text-mode output? Or at least eliminate it as the default? +1. I've had to work around this issue several times by doing strange things like stripping the line ending and putting a \n at the end. I figured it was something I was just missing but seeing someone else mention the problem makes me think this is erroneous behavior. Is this in bugzilla? There *is* stdout.rawWrite(...) (ya gotta include the stdout. part), but unfortunately it doesn't come in *ln, *f or *fln varieties. And it should be the default. I doubt there's a bugzilla entry for this since it is, unfortunately, the intended behavior. I didn't want to go posting a 'zilla issue for it before discussing here because I figured that might just end up INVALID or WONTFIX.
Re: Can we get rid of non-raw write?
On Wednesday, 20 March 2013 at 14:34:21 UTC, Nick Sabalausky wrote: Since *at least* as far back as XP, Windows has handled \n newlines perfectly fine. The command line displays them properly, .BAT scripts handle them properly, every code editor in existence handles them properly. The *only* thing I've found that doesn't is Windows Notepad, but really, whoTF uses that anyway? Can we please get rid of this text-mode output? Or at least eliminate it as the default? +1. Leave an option in their for ancient Windows support if necessary, but take it out as the default. Graham
Re: Can we get rid of non-raw write?
On Wed, Mar 20, 2013 at 10:20:08PM +0100, Graham Fawcett wrote: On Wednesday, 20 March 2013 at 14:34:21 UTC, Nick Sabalausky wrote: Since *at least* as far back as XP, Windows has handled \n newlines perfectly fine. The command line displays them properly, .BAT scripts handle them properly, every code editor in existence handles them properly. The *only* thing I've found that doesn't is Windows Notepad, but really, whoTF uses that anyway? Can we please get rid of this text-mode output? Or at least eliminate it as the default? +1. Leave an option in their for ancient Windows support if necessary, but take it out as the default. [...] What about MacOS? T -- He who does not appreciate the beauty of language is not worthy to bemoan its flaws.
Re: Can we get rid of non-raw write?
On Wednesday, 20 March 2013 at 21:33:48 UTC, H. S. Teoh wrote: On Wed, Mar 20, 2013 at 10:20:08PM +0100, Graham Fawcett wrote: On Wednesday, 20 March 2013 at 14:34:21 UTC, Nick Sabalausky wrote: Since *at least* as far back as XP, Windows has handled \n newlines perfectly fine. The command line displays them properly, .BAT scripts handle them properly, every code editor in existence handles them properly. The *only* thing I've found that doesn't is Windows Notepad, but really, whoTF uses that anyway? Can we please get rid of this text-mode output? Or at least eliminate it as the default? +1. Leave an option in their for ancient Windows support if necessary, but take it out as the default. [...] What about MacOS? Is anyone still using MacOS earlier than version 10 (OSX)? Mac OS 9 was discontinued in 2002. On OSX, there's certainly no problem with Unix line endings. But I guess if we include ancient Windows support as an option, then ancient Mac support should in be there too. But writeln/writefln should emit '\n' as a line terminator, by default, on all platforms. Ancient terminators should be always opt-in, regardless of platform. Graham T
Re: Can we get rid of non-raw write?
On Wednesday, 20 March 2013 at 21:09:03 UTC, Nick Sabalausky wrote: There *is* stdout.rawWrite(...) Sure, I saw that in your post. Thanks for the heads up. That'll be sometimes helpful for when I want to use it later. But still, rawWrite isn't exactly a replacement for write (just like, as you noted it's not a replacement for writef and ln varieties). In particular, it doesn't handle things like ranges as seemlessly as you'd want. Also, things such as write(my range = , myRange) or writeln(This thing is , myDesc, !) are more difficult to do appropriately (though, std.string.format is pretty helpful here) Furthermore, lockingTextWriter displays the same sort of behavior and rawWrite doesn't help at all there because the use-case is completely different for an OutputRange. That's why the default behavior should be the way that works on modern systems, as you suggest.
Re: Can we get rid of non-raw write?
On Wed, 20 Mar 2013 23:00:18 +0100 Graham Fawcett fawc...@uwindsor.ca wrote: On Wednesday, 20 March 2013 at 21:33:48 UTC, H. S. Teoh wrote: On Wed, Mar 20, 2013 at 10:20:08PM +0100, Graham Fawcett wrote: On Wednesday, 20 March 2013 at 14:34:21 UTC, Nick Sabalausky wrote: Since *at least* as far back as XP, Windows has handled \n newlines perfectly fine. The command line displays them properly, .BAT scripts handle them properly, every code editor in existence handles them properly. The *only* thing I've found that doesn't is Windows Notepad, but really, whoTF uses that anyway? Can we please get rid of this text-mode output? Or at least eliminate it as the default? +1. Leave an option in their for ancient Windows support if necessary, but take it out as the default. [...] What about MacOS? Is anyone still using MacOS earlier than version 10 (OSX)? Mac OS 9 was discontinued in 2002. It was more than discontinued, it was more or less obliterated. Even OSX 10.3 and below are basically unusable anymore (unless you don't expect to be able to install anything). Probably 10.4, too. \r as a line ending is long dead. Might be worth supporting *reading* it in certain cases (old text files can live on for a long time), but not writing. On OSX, there's certainly no problem with Unix line endings. But I guess if we include ancient Windows support as an option, then ancient Mac support should in be there too. But writeln/writefln should emit '\n' as a line terminator, by default, on all platforms. Ancient terminators should be always opt-in, regardless of platform. Yea, and by Ancient it's not as if we're even calling XP ancient. We're talking circa-Win9x line here. DMD and Phobos don't even try to support those anyway, and yet that's exactly what write* are essentially catering to. I know Walter has said in the past that there are places where Windows still expects \r\n, but if even if that's true, such places are rare and are best handled as special cases as-needed.
Re: Can we get rid of non-raw write?
On Thu, 21 Mar 2013 00:44:56 -0400 Nick Sabalausky seewebsitetocontac...@semitwist.com wrote: I know Walter has said in the past that there are places where Windows still expects \r\n... Actually, I might be confusing that with forward-slashes vs backslashes in filepaths...
Re: Can we get rid of non-raw write?
On Wednesday, 20 March 2013 at 14:34:21 UTC, Nick Sabalausky wrote: Since *at least* as far back as XP, Windows has handled \n newlines perfectly fine. The command line displays them properly, .BAT scripts handle them properly, every code editor in existence handles them properly. The *only* thing I've found that doesn't is Windows Notepad, but really, whoTF uses that anyway? Notepad is used as a log viewer. When you're presented a machine you don't run, you usually don't have time to setup your preferred work environment, notepad comes in handy to just view the log and diagnose an error.
