Re: phobos: What type to use instead of File when doing I/O on streams?
On Monday, 9 November 2015 at 19:42:53 UTC, J.Frank wrote: - Can you flush() a range? - Can you use select() on a range? Maybe you should factor out a function that does pure data processing on arbitrary ranges and manage sources of the ranges - opening, flushing and closing files - in the caller code?
Re: phobos: What type to use instead of File when doing I/O on streams?
On 11/09/2015 01:48 AM, J.Frank wrote:
My question is now:
How can I make this work without using deprecated stuff?
import std.cstream;
void foo(InputStream in_stream, OutputStream out_stream)
{
//in_stream.seek(3); // compile error - good :)
char[] line;
while ((line = in_stream.readLine()) !is null)
out_stream.writeLine(line);
}
void main(string[] args)
{
foo(din, dout);
}
You don't need the template constraints but it is good practice:
import std.stdio;
import std.range;
void foo(I, O)(I in_stream, O out_stream)
if (isInputRange!I &&
isOutputRange!(O, ElementType!I)) {
// in_stream.seek(3); // compile error - good :)
foreach (element; in_stream) {
out_stream.put(element);
}
}
void main(string[] args)
{
// Also consider .byLine, which is faster and risky
foo(stdin.byLineCopy,
stdout.lockingTextWriter);
}
Ali
Re: phobos: What type to use instead of File when doing I/O on streams?
On Monday, 9 November 2015 at 19:42:53 UTC, J.Frank wrote: On Monday, 9 November 2015 at 18:44:00 UTC, Alex Parrill wrote: Ranges are streams. file.byLine(Copy) and byChunk are effectively streams that are ranges. I might be wrong, but from what I read so far I don't think that "ranges are streams": - Can you read an arbitrary length of bytes from a range? (Reading by line in my code was just an example.) The only way I see is to make it a byte range. But then reading n bytes would result in n calls to popFront(), which is out of the question for performance reasons. Is this correct? `myrange.take(array_size).array` front/popFront are very good candidates for optimization; the compiler should be able to inline them, removing all of the overhead (GCD and LDC will likely be better at this than DMD, though). - Can you flush() a range? - Can you use select() on a range? No, but you can't do either to anything other than file descriptors anyway (at least on Linux, dunno about Windows), so you may as well pass in a File.
Re: phobos: What type to use instead of File when doing I/O on streams?
On Monday, 9 November 2015 at 18:18:19 UTC, J.Frank wrote:
On Monday, 9 November 2015 at 14:48:35 UTC, Ali Çehreli wrote:
import std.stdio;
import std.range;
void foo(I, O)(I in_stream, O out_stream)
if (isInputRange!I &&
isOutputRange!(O, ElementType!I)) {
// in_stream.seek(3); // compile error - good :)
foreach (element; in_stream) {
out_stream.put(element);
}
}
void main(string[] args)
{
// Also consider .byLine, which is faster and risky
foo(stdin.byLineCopy,
stdout.lockingTextWriter);
}
Ali
Uhm... no. That's not a solution to my problem. Ranges are not
(I/O-)streams.
Ranges are streams. file.byLine(Copy) and byChunk are effectively
streams that are ranges.
The only problem with implementing your code in ranges is that
`splitter` requires a forward range, which file-backed ranges
aren't. Otherwise you could do
`file.byChunk(4096).joiner.drop(3).splitter('\n')` to get a
"stream" of lines.
Re: phobos: What type to use instead of File when doing I/O on streams?
On Monday, 9 November 2015 at 14:48:35 UTC, Ali Çehreli wrote:
import std.stdio;
import std.range;
void foo(I, O)(I in_stream, O out_stream)
if (isInputRange!I &&
isOutputRange!(O, ElementType!I)) {
// in_stream.seek(3); // compile error - good :)
foreach (element; in_stream) {
out_stream.put(element);
}
}
void main(string[] args)
{
// Also consider .byLine, which is faster and risky
foo(stdin.byLineCopy,
stdout.lockingTextWriter);
}
Ali
Uhm... no. That's not a solution to my problem. Ranges are not
(I/O-)streams.
Re: phobos: What type to use instead of File when doing I/O on streams?
On Monday, 9 November 2015 at 19:38:06 UTC, Ali Çehreli wrote: As far as I understand, the issue is to prevent the seek() call at compile time, right? If so, the range types that byLine() and byLineCopy() return does not have such a member function. I think the code achieves that goal because we are not dealing with File objects anymore. Yes and no. Again, seek() was just an example. I'm looking for a replacement for the deprecated inferfaces InputStream and OutputStream.
Re: phobos: What type to use instead of File when doing I/O on streams?
On Monday, 9 November 2015 at 19:49:15 UTC, J.Frank wrote: I'm looking for a replacement for the deprecated inferfaces InputStream and OutputStream. Just keep using them. The deprecated note isn't really important... they will still be in Phobos for another year and you can copy the file out yourself to keep using it beyond that. As of right now, there is no exact official replacement.
Re: phobos: What type to use instead of File when doing I/O on streams?
On Monday, 9 November 2015 at 18:44:00 UTC, Alex Parrill wrote: Ranges are streams. file.byLine(Copy) and byChunk are effectively streams that are ranges. I might be wrong, but from what I read so far I don't think that "ranges are streams": - Can you read an arbitrary length of bytes from a range? (Reading by line in my code was just an example.) The only way I see is to make it a byte range. But then reading n bytes would result in n calls to popFront(), which is out of the question for performance reasons. Is this correct? - Can you flush() a range? - Can you use select() on a range? etc. (BTW. Where do I find select()/poll() in phobos?)
Re: phobos: What type to use instead of File when doing I/O on streams?
On Monday, 9 November 2015 at 19:42:53 UTC, J.Frank wrote: (BTW. Where do I find select()/poll() in phobos?) They are in `core.sys.posix.sys.select` In general, OS headers for Posix in C like "sys/select.h" can be gotten by "import core.sys.posix.sys.select;" is in "core.sys.posix.unistd" Linux-specific ones like sys/epoll.h are in `core.sys.linux.sys.epoll` and so on
Re: phobos: What type to use instead of File when doing I/O on streams?
On Monday, 9 November 2015 at 08:49:08 UTC, J.Frank wrote: Hm. "Maybe the stream is seekable, maybe it is not" is not really an option for a language that is supposed to be type safe. It just isn't known at compile time. Files, especially on Unix, are interchangeable at compile time but offer different capabilities at run time. Thank you for your solution, but this sounds more lika a hack. Wrapper structs with alias this are basically how D does inheritance on a struct.
Re: phobos: What type to use instead of File when doing I/O on streams?
On 11/09/2015 10:18 AM, J.Frank wrote:
On Monday, 9 November 2015 at 14:48:35 UTC, Ali Çehreli wrote:
import std.stdio;
import std.range;
void foo(I, O)(I in_stream, O out_stream)
if (isInputRange!I &&
isOutputRange!(O, ElementType!I)) {
// in_stream.seek(3); // compile error - good :)
foreach (element; in_stream) {
out_stream.put(element);
}
}
void main(string[] args)
{
// Also consider .byLine, which is faster and risky
foo(stdin.byLineCopy,
stdout.lockingTextWriter);
}
Ali
Uhm... no. That's not a solution to my problem. Ranges are not
(I/O-)streams.
As far as I understand, the issue is to prevent the seek() call at
compile time, right? If so, the range types that byLine() and
byLineCopy() return does not have such a member function.
I think the code achieves that goal because we are not dealing with File
objects anymore.
Ali
Re: phobos: What type to use instead of File when doing I/O on streams?
On Sunday, 8 November 2015 at 23:50:58 UTC, Adam D. Ruppe wrote: You don't, in general. stdin is sometimes seekable and the compiler doesn't know if it is or not until you try at runtime. Hm. "Maybe the stream is seekable, maybe it is not" is not really an option for a language that is supposed to be type safe. I just found std.stream which looks very good. But it is deprecated?? You could write a wrapper struct for File though that @disables the seek function. It would have a File member, alias file this;, a constructor that forwards to it (optionally, you could also construct it as a File and assign it), and then a @disabled function with the same signature as the seek call in File. Then, it will no longer compile as long as you use your wrapper. Thank you for your solution, but this sounds more lika a hack.
Re: phobos: What type to use instead of File when doing I/O on streams?
My question is now:
How can I make this work without using deprecated stuff?
import std.cstream;
void foo(InputStream in_stream, OutputStream out_stream)
{
// in_stream.seek(3); // compile error - good :)
char[] line;
while ((line = in_stream.readLine()) !is null)
out_stream.writeLine(line);
}
void main(string[] args)
{
foo(din, dout);
}
phobos: What type to use instead of File when doing I/O on streams?
Hello again, I'm a bit puzzled by the "File" type. Looking at the implementation, it seems that all I/O functions were stuffed into a single class^H^H^H^H^Hstruct. What I expected to find is some kind of "Stream" class (with read(), write(), eof()), which is extended by a "File" class (with seek(), mmap(), etc.). So, assuming my program reads from stdin and is supposed to work on a file as well as on a pipe (which is not seekable) - how can I make the compiler bark when I accidently use stdin.seek()? Am I missing something here, too?
Re: phobos: What type to use instead of File when doing I/O on streams?
On Sunday, 8 November 2015 at 21:57:55 UTC, Alex Parrill wrote:
Post what you're trying to do if you want an example.
$ cat test.d
import std.stdio;
//void foo(Stream in_stream, Stream out_stream) // I want
something like this
void foo(File in_stream, File out_stream)
{
in_stream.seek(3); // BUG
string line;
while ((line = in_stream.readln()) !is null)
out_stream.write(line);
}
void main(string[] args)
{
foo(stdin, stdout);
}
$ cat test.txt
Line 0
Line 1
Line 2
$ gdc -o test test.d
$ ./test
Re: phobos: What type to use instead of File when doing I/O on streams?
On Sunday, 8 November 2015 at 20:47:08 UTC, J.Frank wrote: Hello again, I'm a bit puzzled by the "File" type. Looking at the implementation, it seems that all I/O functions were stuffed into a single class^H^H^H^H^Hstruct. What I expected to find is some kind of "Stream" class (with read(), write(), eof()), which is extended by a "File" class (with seek(), mmap(), etc.). So, assuming my program reads from stdin and is supposed to work on a file as well as on a pipe (which is not seekable) - how can I make the compiler bark when I accidently use stdin.seek()? Am I missing something here, too? Use ubyte/char input ranges. Post what you're trying to do if you want an example.
Re: phobos: What type to use instead of File when doing I/O on streams?
On Sunday, 8 November 2015 at 20:47:08 UTC, J.Frank wrote: So, assuming my program reads from stdin and is supposed to work on a file as well as on a pipe (which is not seekable) - how can I make the compiler bark when I accidently use stdin.seek()? You don't, in general. stdin is sometimes seekable and the compiler doesn't know if it is or not until you try at runtime. You could write a wrapper struct for File though that @disables the seek function. It would have a File member, alias file this;, a constructor that forwards to it (optionally, you could also construct it as a File and assign it), and then a @disabled function with the same signature as the seek call in File. Then, it will no longer compile as long as you use your wrapper.
