Re: Strange behavior with clojure.java.io/copy

[Matching Socks]

A bit awkward to clutter a standard library with too much speculative 
forgiveness of other software's potential failure to follow a spec.  Where 
would it end?

Anyway, the best improvement here would help with all InputStream 
consumers, not just io/copy.  

If you suffer from a faulty InputStream that returns zero when it simply 
does not feel like blocking, couldn't you either fix it or wrap it in a 
"TolerateZeroBytesReadByRetryingInputStream" that blocks until it gets 
something other than zero?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/84da2491-2a3c-4b0c-9406-49f63ed6caf4%40googlegroups.com.

Re: Strange behavior with clojure.java.io/copy

['Dirk Wetzel' via Clojure]

You mentioned an interesting point there:

*If implementations actually follow that documentation, [...]*
>

I know that at least some libraries (f.e. "org.apache.commons.io") only 
stop reading when -1 is returned and will continue reading if 0 is returned.
That behaviour is definitely more friendly towards implementations that 
don't follow the java-doc exactly.

@ Alex Miller:
To cover both cases, maybe take the buffer-size into account when making 
the decision about whether or not to continue reading?
Something along those lines:

(or (pos? size)
(and (= 0 size)
 (pos? buffer-size)))


Am Montag, 26. August 2019 07:01:01 UTC+2 schrieb Andy Fingerhut:
>
> Ah, I should have read further in the Java doc here, though, before 
> posting.  In particular, the last sentence of this paragraph: "If the 
> length of b is zero, then no bytes are read and 0 is returned; otherwise, 
> there is an attempt to read at least one byte. If no byte is available 
> because the stream is at the end of the file, the value -1 is returned; 
> otherwise, at least one byte is read and stored into b."
>
>
> https://docs.oracle.com/javase/8/docs/api/java/io/InputStream.html#read-byte:A-
>
> If implementations actually follow that documentation, then it appears 
> that a non-0 buffer size should never result in a return value of 0.
>
> Andy
>
> On Sun, Aug 25, 2019 at 9:54 PM Andy Fingerhut  > wrote:
>
>> That bit of Java-Doc says nothing about the behavior when providing a 
>> buffer b with a non-0 length.
>>
>> If it said: "If the length of b is non-zero, then the return value will 
>> never be 0", then I might agree with you.  Also, if you knew for some 
>> reason that the implementation guaranteed this.  (I have not looked at the 
>> implementation to know either way.)
>>
>> Andy
>>
>> On Sun, Aug 25, 2019 at 9:21 PM 'Dirk Wetzel' via Clojure <
>> clo...@googlegroups.com > wrote:
>>
>>> As alpeware already said, *.read* will not return [1024, 0, 1024, 201, 
>>> -1] because it will not return a zero unless *buffer-size* is zero.
>>> This bit of the Java-Doc: If the length of b is zero, then no bytes are 
>>> read and 0 is returned;
>>>
>>> So if the check is changed to (<= 0 size), passing a *buffer-size* of 0 
>>> (for whatever silly reason) would result in an infinite loop.
>>> While not my decision to make, I personally don't think that this would 
>>> be desirable behaviour.
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clo...@googlegroups.com 
>>> 
>>> Note that posts from new members are moderated - please be patient with 
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clo...@googlegroups.com 
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to clo...@googlegroups.com .
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/clojure/9deb8024-ce4b-4ef4-88d0-bbd817152ab6%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/c6674761-8e16-4cc7-8b46-ab18aa7911f2%40googlegroups.com.

Re: Strange behavior with clojure.java.io/copy

[Andy Fingerhut]

Ah, I should have read further in the Java doc here, though, before
posting.  In particular, the last sentence of this paragraph: "If the
length of b is zero, then no bytes are read and 0 is returned; otherwise,
there is an attempt to read at least one byte. If no byte is available
because the stream is at the end of the file, the value -1 is returned;
otherwise, at least one byte is read and stored into b."

https://docs.oracle.com/javase/8/docs/api/java/io/InputStream.html#read-byte:A-

If implementations actually follow that documentation, then it appears that
a non-0 buffer size should never result in a return value of 0.

Andy

On Sun, Aug 25, 2019 at 9:54 PM Andy Fingerhut 
wrote:

> That bit of Java-Doc says nothing about the behavior when providing a
> buffer b with a non-0 length.
>
> If it said: "If the length of b is non-zero, then the return value will
> never be 0", then I might agree with you.  Also, if you knew for some
> reason that the implementation guaranteed this.  (I have not looked at the
> implementation to know either way.)
>
> Andy
>
> On Sun, Aug 25, 2019 at 9:21 PM 'Dirk Wetzel' via Clojure <
> clojure@googlegroups.com> wrote:
>
>> As alpeware already said, *.read* will not return [1024, 0, 1024, 201,
>> -1] because it will not return a zero unless *buffer-size* is zero.
>> This bit of the Java-Doc: If the length of b is zero, then no bytes are
>> read and 0 is returned;
>>
>> So if the check is changed to (<= 0 size), passing a *buffer-size* of 0
>> (for whatever silly reason) would result in an infinite loop.
>> While not my decision to make, I personally don't think that this would
>> be desirable behaviour.
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/clojure/9deb8024-ce4b-4ef4-88d0-bbd817152ab6%40googlegroups.com
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/CAKvLtDYjqzJrwphmVJWachsZVwd__Y0vcpUwymQNBx64q%3DROVA%40mail.gmail.com.

Re: Strange behavior with clojure.java.io/copy

[Andy Fingerhut]

That bit of Java-Doc says nothing about the behavior when providing a
buffer b with a non-0 length.

If it said: "If the length of b is non-zero, then the return value will
never be 0", then I might agree with you.  Also, if you knew for some
reason that the implementation guaranteed this.  (I have not looked at the
implementation to know either way.)

Andy

On Sun, Aug 25, 2019 at 9:21 PM 'Dirk Wetzel' via Clojure <
clojure@googlegroups.com> wrote:

> As alpeware already said, *.read* will not return [1024, 0, 1024, 201, -1] 
> because
> it will not return a zero unless *buffer-size* is zero.
> This bit of the Java-Doc: If the length of b is zero, then no bytes are
> read and 0 is returned;
>
> So if the check is changed to (<= 0 size), passing a *buffer-size* of 0
> (for whatever silly reason) would result in an infinite loop.
> While not my decision to make, I personally don't think that this would be
> desirable behaviour.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/clojure/9deb8024-ce4b-4ef4-88d0-bbd817152ab6%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/CAKvLtDagFDJ_XDpeFTv487jcVPLdeZXuCDnPFpPHdTM4NOXmRw%40mail.gmail.com.

Re: Strange behavior with clojure.java.io/copy

['Dirk Wetzel' via Clojure]

As alpeware already said, *.read* will not return [1024, 0, 1024, 201, -1] 
because 
it will not return a zero unless *buffer-size* is zero.
This bit of the Java-Doc: If the length of b is zero, then no bytes are 
read and 0 is returned;

So if the check is changed to (<= 0 size), passing a *buffer-size* of 0 
(for whatever silly reason) would result in an infinite loop.
While not my decision to make, I personally don't think that this would be 
desirable behaviour.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/9deb8024-ce4b-4ef4-88d0-bbd817152ab6%40googlegroups.com.

Re: Strange behavior with clojure.java.io/copy

[alpeware]

The read method returns the number of bytes consumed from the input stream 
and stored in the provided buffer.

The only case the read method would return 0 is when a buffer of length 0 
is provided since there would be no place to store any bytes consumed from 
the input stream.

In most cases, a fixed length buffer is allocated and the input stream is 
consumed in chunks by reading it into the buffer, do something with the 
contents of the buffer and then iterate until the whole stream has been 
consumed.

I hope this helps.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/24d989ae-fdfd-45ed-ae21-ad0e53912564%40googlegroups.com.

Re: Strange behavior with clojure.java.io/copy

[Juha Syrjälä]

I posted this also to ask.clojure.org forum at 
https://ask.clojure.org/index.php/8450/strange-behavior-with-clojure-java-io-copy

On Sunday, August 25, 2019 at 8:05:57 PM UTC+3, Juha Syrjälä wrote:
>
> Hi,
>
> I found a strange behavior in implementation of clojure.java.io/copy 
> function
>
>
> https://github.com/clojure/clojure/blob/ee1b606ad066ac8df2efd4a6b8d0d365c206f5bf/src/clj/clojure/java/io.clj#L391
> (defn copy
>   "Copies input to output.  Returns nil or throws IOException.
>   Input may be an InputStream, Reader, File, byte[], char[], or String.
>   Output may be an OutputStream, Writer, or File.
>   Options are key/value pairs and may be one of
> :buffer-size  buffer size to use, default is 1024.
> :encoding encoding to use if converting between
>   byte and char streams.   
>   Does not close any streams except those it opens itself 
>   (on a File)."
>   {:added "1.2"}
>   [input output & opts]
>   (do-copy input output (when opts (apply hash-map opts
>
> Actual copying is implemented here when copying from an InputStream to an 
> OutputStream.
>
>
> https://github.com/clojure/clojure/blob/ee1b606ad066ac8df2efd4a6b8d0d365c206f5bf/src/clj/clojure/java/io.clj#L306
> (defmethod do-copy [InputStream OutputStream] [^InputStream input 
> ^OutputStream output opts]
>   (let [buffer (make-array Byte/TYPE (buffer-size opts))]
> (loop []
>   (let [size (.read input buffer)]  ;;; XXX point 1
> (when (pos? size)   ;;; XXX point 2
>   (do (.write output buffer 0 size)
>   (recur)))
>
> Here .read function at point 1 is 
> https://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html#read(byte[])
>
> The javadoc states following
>
> Reads some number of bytes from the input stream and stores them into the 
> buffer array b. The number of bytes actually read is returned as an 
> integer. This method blocks until input data is available, end of file is 
> detected, or an exception is thrown.
>
> If the length of b is zero, then no bytes are read and 0 is returned; 
> otherwise, there is an attempt to read at least one byte. If no byte is 
> available because the stream is at the end of the file, the value -1 is 
> returned; otherwise, at least one byte is read and stored into b.
>
> --- 
>
>
> Meaning that return value -1 means end of stream, and return value 0 
> doesn't mean end of stream. However condition at point 2 in code above, 
> stops recursion when .read returns value that is smaller than 1.
>
>
> Now consider a case where .read returns this sequence of values in 
> consecutive calls:
>
>
> 1024, 0, 1024, 201, -1
>
>
> clojure.java/io copies only the first 1024 bytes, when the whole stream 
> has 2249 bytes. Is this the intended behavior? Should the condition at 
> point 1 be "(not (neg? size))" ?
>
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/039a4a5d-75d3-4da9-8da1-d3af340989cc%40googlegroups.com.

Strange behavior with clojure.java.io/copy

[Juha Syrjälä]

Hi,

I found a strange behavior in implementation of clojure.java.io/copy 
function

https://github.com/clojure/clojure/blob/ee1b606ad066ac8df2efd4a6b8d0d365c206f5bf/src/clj/clojure/java/io.clj#L391
(defn copy
  "Copies input to output.  Returns nil or throws IOException.
  Input may be an InputStream, Reader, File, byte[], char[], or String.
  Output may be an OutputStream, Writer, or File.
  Options are key/value pairs and may be one of
:buffer-size  buffer size to use, default is 1024.
:encoding encoding to use if converting between
  byte and char streams.   
  Does not close any streams except those it opens itself 
  (on a File)."
  {:added "1.2"}
  [input output & opts]
  (do-copy input output (when opts (apply hash-map opts

Actual copying is implemented here when copying from an InputStream to an 
OutputStream.

https://github.com/clojure/clojure/blob/ee1b606ad066ac8df2efd4a6b8d0d365c206f5bf/src/clj/clojure/java/io.clj#L306
(defmethod do-copy [InputStream OutputStream] [^InputStream input 
^OutputStream output opts]
  (let [buffer (make-array Byte/TYPE (buffer-size opts))]
(loop []
  (let [size (.read input buffer)]  ;;; XXX point 1
(when (pos? size)   ;;; XXX point 2
  (do (.write output buffer 0 size)
  (recur)))

Here .read function at point 1 is 
https://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html#read(byte[])

The javadoc states following

Reads some number of bytes from the input stream and stores them into the 
buffer array b. The number of bytes actually read is returned as an 
integer. This method blocks until input data is available, end of file is 
detected, or an exception is thrown.

If the length of b is zero, then no bytes are read and 0 is returned; 
otherwise, there is an attempt to read at least one byte. If no byte is 
available because the stream is at the end of the file, the value -1 is 
returned; otherwise, at least one byte is read and stored into b.

--- 


Meaning that return value -1 means end of stream, and return value 0 
doesn't mean end of stream. However condition at point 2 in code above, 
stops recursion when .read returns value that is smaller than 1.


Now consider a case where .read returns this sequence of values in 
consecutive calls:


1024, 0, 1024, 201, -1


clojure.java/io copies only the first 1024 bytes, when the whole stream has 
2249 bytes. Is this the intended behavior? Should the condition at point 1 
be "(not (neg? size))" ?



-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/b23eb6a9-6eb9-4116-9485-5a1f805e1825%40googlegroups.com.