Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
For completeness, here is the finalized spec that we intend to submit
for approval. Just some minor rewording and grammaticality amendments
over the previous.
/**
* Reads all bytes from this input stream and writes the bytes to
the given
* output stream in the order that they are read. On return, this
input
* stream will be at end of stream.
* p
* This method may block indefinitely reading from the input stream,
* or writing to the output stream. The behavior for the case that the
* input and/or output stream is iasynchronously closed/i, or the
* thread interrupted during the transfer, is highly input and
output stream
* specific, and therefore not specified.
* p
* If an I/O error occurs reading from the input stream or writing to
* the output stream, then it may do so after some bytes have been
read or
* written. Consequently the input stream may not be at end of
stream and
* one, or both, streams may be in an inconsistent state. It is
strongly
* recommended that both streams be promptly closed if an I/O error
occurs.
*
* @param out the output stream, non-null
* @return the number of bytes transferred
* @throws IOException if an I/O error occurs when reading or writing
* @throws NullPointerException if {@code out} is {@code null}
*
* @since 1.9
*/
public long transferTo(OutputStream out) throws IOException { ... }
-Chris.
On 05/12/14 18:29, Chris Hegarty wrote:
On 5 Dec 2014, at 17:28, Patrick Reinhart patr...@reini.net wrote:
Hi Chris,
Am 05.12.2014 um 17:36 schrieb Chris Hegarty chris.hega...@oracle.com:
On 05/12/14 15:59, Alan Bateman wrote:
On 05/12/2014 15:38, Chris Hegarty wrote:
The addition of a copyTo method to java.io.InputStream is binary
compatible [1], but it may, in some cases, be source incompatible. I
believe the benefits of this approach out weigh the potential source
incompatibility, but it will be for the spec gods, the CCC, to have
final say.
Here is what I propose. Let's take just InputStream.copyTo and bring
it, by itself, to a conclusion. And then revisit each of the
additional useful/convenience methods on their own merit.
I think the approach is good. An alternative name to consider is
transferTo. For the javadoc then you can make it clear that the
InputStream is at EOF when the method completes. I don't think the
javadoc needs to say that the OutputStream should be closed promptly
(think cat a b c d).
Right. Updated spec:
/**
* Reads all remaining bytes from this input stream and writes them to
* the given output stream. On return, this input stream will be at end of
* stream.
* p
* This method may block indefinitely reading from the input stream,
* or writing to the output stream. The behavior for the case that the
* input and/or output stream is iasynchronously closed/i, or the
* thread interrupted during the copy, is highly input and output stream
* specific, and therefore not specified.
* p
* If an I/O error occurs reading from the input stream or writing to
* the output stream, then it may do so after some bytes have been read or
* written. Consequently the input stream may not be at end of stream and
* may be in an inconsistent state. It is strongly recommended that the
* input stream be promptly closed if an I/O error occurs.
*
* @param out the output stream to write to, non-null
* @return the number of bytes copied
* @throws IOException if an I/O error occurs when reading or writing
* @throws NullPointerException if {@code out} is {@code null}
*
* @since 1.9
*/
public long transferTo(OutputStream out) throws IOException { ... }
-Chris.
Looks good to me. The only thing I would change is the @return documentation to
„the number of bytes transferred“ to be in sync with the method name.
Agreed. Consider it changed.
-Chris.
-Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
On 08/12/2014 11:48, Chris Hegarty wrote:
For completeness, here is the finalized spec that we intend to submit
for approval. Just some minor rewording and grammaticality amendments
over the previous.
/**
* Reads all bytes from this input stream and writes the bytes to
the given
* output stream in the order that they are read. On return, this
input
* stream will be at end of stream.
* p
* This method may block indefinitely reading from the input stream,
* or writing to the output stream. The behavior for the case that
the
* input and/or output stream is iasynchronously closed/i, or the
* thread interrupted during the transfer, is highly input and
output stream
* specific, and therefore not specified.
* p
* If an I/O error occurs reading from the input stream or writing to
* the output stream, then it may do so after some bytes have been
read or
* written. Consequently the input stream may not be at end of
stream and
* one, or both, streams may be in an inconsistent state. It is
strongly
* recommended that both streams be promptly closed if an I/O
error occurs.
*
* @param out the output stream, non-null
* @return the number of bytes transferred
* @throws IOException if an I/O error occurs when reading or writing
* @throws NullPointerException if {@code out} is {@code null}
*
* @since 1.9
*/
public long transferTo(OutputStream out) throws IOException { ... }
I'm happy with this updated wording, it's consistent with the wording
that we already have in Files.copy, and it addresses all the points that
have come up.
-Alan
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Hi Chris,
Am 08.12.2014 um 15:17 schrieb Chris Hegarty chris.hega...@oracle.com:
Again, just for completeness...
Some feedback from Paul Sandoz, to be explicit about the openness of the
streams on return. Does not close either stream.
/**
* Reads all bytes from this input stream and writes the bytes to the given
* output stream in the order that they are read. On return, this input
* stream will be at end of stream. Does not close either stream.
* p
* This method may block indefinitely reading from the input stream, or
* writing to the output stream. The behavior for the case that the input
* and/or output stream is iasynchronously closed/i, or the thread
* interrupted during the transfer, is highly input and output stream
* specific, and therefore not specified.
* p
* If an I/O error occurs reading from the input stream or writing to
* the output stream, then it may do so after some bytes have been read or
* written. Consequently the input stream may not be at end of stream and
* one, or both, streams may be in an inconsistent state. It is strongly
* recommended that both streams be promptly closed if an I/O error occurs.
*
* @param out the output stream, non-null
* @return the number of bytes transferred
* @throws IOException if an I/O error occurs when reading or writing
* @throws NullPointerException if {@code out} is {@code null}
*
* @since 1.9
*/
public long transferTo(OutputStream out) throws IOException
-Chris.
Looks good to me,
I guess if there are no further change requests on that method we could look at
the transfer methods for Readable/Appendable. Here we could put a transferTo
method as a default method on the Readable interface or what do you think?
-Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Should those methods also be on the FilterInput- and OutputStream? I'm not sure I'm following you. j.i.FilterInputStream will get the base version of InputStream.copyTo method for free as a child though it doesn't define it itself. Every class down the hierarchy starting from the j.i.InputStream will get InputStream.copyTo version unless it overrides it. To override it, it should of course have some rationale behind it. I believe we have found it for at least two widely used subclasses. Sorry, you are absolutely right, it seems I was somewhere else in my thinking. Just ignore that point :-) I'm not quit sure about the impact on may already existing customer code that had implemented such a method without declaring a IOException for example. In this case the existing code would may break? (as a comment of Alan Bateman earlier before) In the case you've mentioned everything should be just fine (technically) as overriding method has right to neither throw nor even declare any exceptions thrown by the parent method. This point was not completely clear for me. I thought it, but I was not completely sure about that fact. I said *technically* because the sole fact of overriding doesn't guarantee the child's method has the same semantics as the parent's one. It seems to me that it’s not the only possible problematic thing here. We'll see. -Pavel I hope there is a change to get that in the JDK9 at the end. I will do as much as I can do from my side though. -Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
The addition of a copyTo method to java.io.InputStream is binary
compatible [1], but it may, in some cases, be source incompatible. I
believe the benefits of this approach out weigh the potential source
incompatibility, but it will be for the spec gods, the CCC, to have
final say.
Here is what I propose. Let's take just InputStream.copyTo and bring it,
by itself, to a conclusion. And then revisit each of the additional
useful/convenience methods on their own merit.
I think we are mostly in agreement on the latest patch Pavel sent out.
Cut'n'past here, with some grammatical updates.
/**
* Reads all remaining bytes from this input stream and writes them to
* the given output stream.
* p
* There are no guarantees on the stream's state if an error occurs.
* Even if this method returns normally you can't rely on previously
* marked positions, or the contents of any internal buffers. That
* said, this is a terminal operation. It is strongly recommended
* that both streams are promptly closed after this method returns:
* pre{@code
* try (InputStream is = ...; OutputStream os = ...;) {
* is.copyTo(os);
* } catch (IOException e) {
* ...
* }
* }/pre
* p
* This method may block indefinitely reading from the input stream,
* or writing to the output stream. The behavior for the case that the
* input and/or output stream is iasynchronously closed/i, or the
* thread interrupted during the copy, is highly input and output stream
* specific, and therefore not specified.
*
* @param out the output stream to write to, non-null
* @return the number of bytes copied
* @throws IOException if an I/O error occurs when reading or writing
*
* @since 1.9
*/
public long copyTo(OutputStream out) throws IOException { ... }
-Chris
[1] http://docs.oracle.com/javase/specs/jls/se8/html/jls-13.html
On 04/12/14 23:13, Pavel Rappo wrote:
Patrick,
Should those methods also be on the FilterInput- and OutputStream?
I'm not sure I'm following you. j.i.FilterInputStream will get the base version
of InputStream.copyTo method for free as a child though it doesn't define it
itself. Every class down the hierarchy starting from the j.i.InputStream will
get InputStream.copyTo version unless it overrides it.
To override it, it should of course have some rationale behind it. I believe we
have found it for at least two widely used subclasses.
I'm not quit sure about the impact on may already existing customer code that
had implemented such a method without declaring a IOException for example. In
this case the existing code would may break? (as a comment of Alan Bateman
earlier before)
In the case you've mentioned everything should be just fine (technically) as
overriding method has right to neither throw nor even declare any exceptions
thrown by the parent method.
I said *technically* because the sole fact of overriding doesn't guarantee the
child's method has the same semantics as the parent's one.
It seems to me that it’s not the only possible problematic thing here. We'll
see.
-Pavel
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
On 05/12/2014 15:38, Chris Hegarty wrote: The addition of a copyTo method to java.io.InputStream is binary compatible [1], but it may, in some cases, be source incompatible. I believe the benefits of this approach out weigh the potential source incompatibility, but it will be for the spec gods, the CCC, to have final say. Here is what I propose. Let's take just InputStream.copyTo and bring it, by itself, to a conclusion. And then revisit each of the additional useful/convenience methods on their own merit. I think the approach is good. An alternative name to consider is transferTo. For the javadoc then you can make it clear that the InputStream is at EOF when the method completes. I don't think the javadoc needs to say that the OutputStream should be closed promptly (think cat a b c d). -Alan.
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
On 05/12/14 15:59, Alan Bateman wrote:
On 05/12/2014 15:38, Chris Hegarty wrote:
The addition of a copyTo method to java.io.InputStream is binary
compatible [1], but it may, in some cases, be source incompatible. I
believe the benefits of this approach out weigh the potential source
incompatibility, but it will be for the spec gods, the CCC, to have
final say.
Here is what I propose. Let's take just InputStream.copyTo and bring
it, by itself, to a conclusion. And then revisit each of the
additional useful/convenience methods on their own merit.
I think the approach is good. An alternative name to consider is
transferTo. For the javadoc then you can make it clear that the
InputStream is at EOF when the method completes. I don't think the
javadoc needs to say that the OutputStream should be closed promptly
(think cat a b c d).
Right. Updated spec:
/**
* Reads all remaining bytes from this input stream and writes them to
* the given output stream. On return, this input stream will be
at end of
* stream.
* p
* This method may block indefinitely reading from the input stream,
* or writing to the output stream. The behavior for the case that the
* input and/or output stream is iasynchronously closed/i, or the
* thread interrupted during the copy, is highly input and output
stream
* specific, and therefore not specified.
* p
* If an I/O error occurs reading from the input stream or writing to
* the output stream, then it may do so after some bytes have been
read or
* written. Consequently the input stream may not be at end of
stream and
* may be in an inconsistent state. It is strongly recommended that the
* input stream be promptly closed if an I/O error occurs.
*
* @param out the output stream to write to, non-null
* @return the number of bytes copied
* @throws IOException if an I/O error occurs when reading or writing
* @throws NullPointerException if {@code out} is {@code null}
*
* @since 1.9
*/
public long transferTo(OutputStream out) throws IOException { ... }
-Chris.
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Hi Chris,
Am 05.12.2014 um 17:36 schrieb Chris Hegarty chris.hega...@oracle.com:
On 05/12/14 15:59, Alan Bateman wrote:
On 05/12/2014 15:38, Chris Hegarty wrote:
The addition of a copyTo method to java.io.InputStream is binary
compatible [1], but it may, in some cases, be source incompatible. I
believe the benefits of this approach out weigh the potential source
incompatibility, but it will be for the spec gods, the CCC, to have
final say.
Here is what I propose. Let's take just InputStream.copyTo and bring
it, by itself, to a conclusion. And then revisit each of the
additional useful/convenience methods on their own merit.
I think the approach is good. An alternative name to consider is
transferTo. For the javadoc then you can make it clear that the
InputStream is at EOF when the method completes. I don't think the
javadoc needs to say that the OutputStream should be closed promptly
(think cat a b c d).
Right. Updated spec:
/**
* Reads all remaining bytes from this input stream and writes them to
* the given output stream. On return, this input stream will be at end of
* stream.
* p
* This method may block indefinitely reading from the input stream,
* or writing to the output stream. The behavior for the case that the
* input and/or output stream is iasynchronously closed/i, or the
* thread interrupted during the copy, is highly input and output stream
* specific, and therefore not specified.
* p
* If an I/O error occurs reading from the input stream or writing to
* the output stream, then it may do so after some bytes have been read or
* written. Consequently the input stream may not be at end of stream and
* may be in an inconsistent state. It is strongly recommended that the
* input stream be promptly closed if an I/O error occurs.
*
* @param out the output stream to write to, non-null
* @return the number of bytes copied
* @throws IOException if an I/O error occurs when reading or writing
* @throws NullPointerException if {@code out} is {@code null}
*
* @since 1.9
*/
public long transferTo(OutputStream out) throws IOException { ... }
-Chris.
Looks good to me. The only thing I would change is the @return documentation to
„the number of bytes transferred“ to be in sync with the method name.
-Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
On 5 Dec 2014, at 17:28, Patrick Reinhart patr...@reini.net wrote:
Hi Chris,
Am 05.12.2014 um 17:36 schrieb Chris Hegarty chris.hega...@oracle.com:
On 05/12/14 15:59, Alan Bateman wrote:
On 05/12/2014 15:38, Chris Hegarty wrote:
The addition of a copyTo method to java.io.InputStream is binary
compatible [1], but it may, in some cases, be source incompatible. I
believe the benefits of this approach out weigh the potential source
incompatibility, but it will be for the spec gods, the CCC, to have
final say.
Here is what I propose. Let's take just InputStream.copyTo and bring
it, by itself, to a conclusion. And then revisit each of the
additional useful/convenience methods on their own merit.
I think the approach is good. An alternative name to consider is
transferTo. For the javadoc then you can make it clear that the
InputStream is at EOF when the method completes. I don't think the
javadoc needs to say that the OutputStream should be closed promptly
(think cat a b c d).
Right. Updated spec:
/**
* Reads all remaining bytes from this input stream and writes them to
* the given output stream. On return, this input stream will be at end
of
* stream.
* p
* This method may block indefinitely reading from the input stream,
* or writing to the output stream. The behavior for the case that the
* input and/or output stream is iasynchronously closed/i, or the
* thread interrupted during the copy, is highly input and output stream
* specific, and therefore not specified.
* p
* If an I/O error occurs reading from the input stream or writing to
* the output stream, then it may do so after some bytes have been read or
* written. Consequently the input stream may not be at end of stream and
* may be in an inconsistent state. It is strongly recommended that the
* input stream be promptly closed if an I/O error occurs.
*
* @param out the output stream to write to, non-null
* @return the number of bytes copied
* @throws IOException if an I/O error occurs when reading or writing
* @throws NullPointerException if {@code out} is {@code null}
*
* @since 1.9
*/
public long transferTo(OutputStream out) throws IOException { ... }
-Chris.
Looks good to me. The only thing I would change is the @return documentation
to „the number of bytes transferred“ to be in sync with the method name.
Agreed. Consider it changed.
-Chris.
-Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Patrick, thanks a lot for doing this! I've had a look at these usages, and I
think it's safe to say that in the JDK it has never been required (yet) to
provide a copy method with custom byte array. So let's skip it for now.
I think more and more about your initial suggestion of introducing a 'copyTo'
method directly into the types:
InputStream, OutputStream, Readable and Appendable
I believe it will suit us a lot more. If we go this way, we can fully utilize
polymorphic calls. Implementations could then provide their own more efficient
solutions. Have a look at this:
/**
* Reads all remaining bytes from this input stream and writes them to
* a specified output stream.
* p
* There are no guarantees on streams state in a case error occurs. Even
* if method returns normally you can't rely on previously marked positions
* or the contents of any internal buffers.
* That said, it is a terminal operation. It is strongly recommended that
* both streams are promptly closed after this method returns:
* pre{@code
* try (InputStream is = ...; OutputStream os = ...;) {
* is.read(os);
* } catch (IOException e) {
* ...
* }
* }/pre
* p
* This method may block indefinitely reading from the input stream
* (or writing to the output stream). The behavior for the case that the
* input and output streams is iasynchronously closed/i or the thread
* interrupted during the copy is highly input and output stream
* system provider specific and therefore not specified.
*
* @param target the output stream to write to
* ({@code target != null})
* @return the number of bytes copied
* @throws IOException if an I/O error occurs when reading or writing
*
* @since 1.9
*/
public long copyTo(OutputStream target) throws IOException {
Objects.requireNonNull(target, target);
int copied = 0;
byte[] buffer = new byte[8192];
for (int read; (read = this.read(buffer)) -1; copied += read) {
target.write(buffer, 0, read);
}
return copied;
}
This method will be defined in java.io.InputStream. And the following methods
will override 'copyTo' in java.io.ByteArrayInputStream and
java.io.BufferedInputStream respectively:
/**
* {@inheritDoc}
*/
@Override
public synchronized long copyTo(OutputStream target) throws IOException {
Objects.requireNonNull(target, target);
int avail = count - pos;
target.write(buf, pos, avail);
return avail;
}
/**
* {@inheritDoc}
*/
@Override
public synchronized long copyTo(OutputStream target) throws IOException {
Objects.requireNonNull(target, target);
long copied = 0;
// 1. Get whatever is left unread in the buffer
if (pos count) {
int len = count - pos;
target.write(getBufIfOpen(), pos, len);
copied += len;
}
// 2. Get everything else directly from the underlying stream
for (int read; (read = getInIfOpen().read(getBufIfOpen())) -1;
copied += read) {
target.write(getBufIfOpen(), 0, read);
}
return copied;
}
1. Please note, that this is not the actual code that will be pushed (if at
all). It's merely a sketch to illustrate your proposal.
2. The equivalent thing will apply for Readable/Appendable.
-Pavel
On 2 Dec 2014, at 09:22, Patrick Reinhart patr...@reini.net wrote:
I found around 190 usage references in total of those there are the following
28 references that could use my proposed copy feature and 5 of those use 8192
bytes sized buffers.
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Hi Pavel,
Patrick, thanks a lot for doing this! I've had a look at these usages, and I
think it's safe to say that in the JDK it has never been required (yet) to
provide a copy method with custom byte array. So let's skip it for now.
Welcome, unfortunately I did not manage it to check those on the JDK9
code base due the fact, that the NetBeans project does not work with the
modularized codebase jet.
I think more and more about your initial suggestion of introducing a 'copyTo'
method directly into the types:
InputStream, OutputStream, Readable and Appendable
Having a copyTo method on InputStream and Readable was indeed my fist
idea to have. Also to have a copyFrom method on the OutputStream and
Appendable was then a natural fit. Looking at those methods it became
clear to me that in fact we would have a the same code on either
InputStream and OutputStream or Readable and Appendable.
Of course for OutputStream we could also have something like:
public long copyFrom(InputStream source) throws IOException {
return source.copyTo(this);
}
For the Appendable could it be the same same with a Readable as the
source in that case.
I believe it will suit us a lot more. If we go this way, we can fully utilize
polymorphic calls. Implementations could then provide their own more efficient
solutions. Have a look at this:
/**
* Reads all remaining bytes from this input stream and writes them to
* a specified output stream.
* p
* There are no guarantees on streams state in a case error occurs. Even
* if method returns normally you can't rely on previously marked positions
* or the contents of any internal buffers.
* That said, it is a terminal operation. It is strongly recommended that
* both streams are promptly closed after this method returns:
* pre{@code
* try (InputStream is = ...; OutputStream os = ...;) {
* is.read(os);
* } catch (IOException e) {
* ...
* }
* }/pre
* p
* This method may block indefinitely reading from the input stream
* (or writing to the output stream). The behavior for the case that the
* input and output streams is iasynchronously closed/i or the thread
* interrupted during the copy is highly input and output stream
* system provider specific and therefore not specified.
*
* @param target the output stream to write to
* ({@code target != null})
* @return the number of bytes copied
* @throws IOException if an I/O error occurs when reading or writing
*
* @since 1.9
*/
public long copyTo(OutputStream target) throws IOException {
Objects.requireNonNull(target, target);
int copied = 0;
byte[] buffer = new byte[8192];
for (int read; (read = this.read(buffer)) -1; copied += read) {
target.write(buffer, 0, read);
}
return copied;
}
This method will be defined in java.io.InputStream. And the following methods
will override 'copyTo' in java.io.ByteArrayInputStream and
java.io.BufferedInputStream respectively:
Should those methods also be on the FilterInput- and OutputStream?
/**
* {@inheritDoc}
*/
@Override
public synchronized long copyTo(OutputStream target) throws IOException {
Objects.requireNonNull(target, target);
int avail = count - pos;
target.write(buf, pos, avail);
return avail;
}
/**
* {@inheritDoc}
*/
@Override
public synchronized long copyTo(OutputStream target) throws IOException {
Objects.requireNonNull(target, target);
long copied = 0;
// 1. Get whatever is left unread in the buffer
if (pos count) {
int len = count - pos;
target.write(getBufIfOpen(), pos, len);
copied += len;
}
// 2. Get everything else directly from the underlying stream
for (int read; (read = getInIfOpen().read(getBufIfOpen())) -1;
copied += read) {
target.write(getBufIfOpen(), 0, read);
}
return copied;
}
1. Please note, that this is not the actual code that will be pushed (if at
all). It's merely a sketch to illustrate your proposal.
2. The equivalent thing will apply for Readable/Appendable.
-Pavel
I like that solution almost more then having a separate utility class
though.
I'm not quit sure about the impact on may already existing customer code
that had implemented such a method without declaring a IOException for
example. In this case the existing code would may break? (as a comment
of Alan Bateman earlier before)
-Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Patrick, Should those methods also be on the FilterInput- and OutputStream? I'm not sure I'm following you. j.i.FilterInputStream will get the base version of InputStream.copyTo method for free as a child though it doesn't define it itself. Every class down the hierarchy starting from the j.i.InputStream will get InputStream.copyTo version unless it overrides it. To override it, it should of course have some rationale behind it. I believe we have found it for at least two widely used subclasses. I'm not quit sure about the impact on may already existing customer code that had implemented such a method without declaring a IOException for example. In this case the existing code would may break? (as a comment of Alan Bateman earlier before) In the case you've mentioned everything should be just fine (technically) as overriding method has right to neither throw nor even declare any exceptions thrown by the parent method. I said *technically* because the sole fact of overriding doesn't guarantee the child's method has the same semantics as the parent's one. It seems to me that it’s not the only possible problematic thing here. We'll see. -Pavel
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Finally got the time to search thru the JDK codebase - at the moment JDK8 due the lack of a working Netbeans Project for JDK9 (I will try to make that working again) What usage would you actually search in the OpenJDK code base? We're talking about copying with provided buffer, right? long copy(InputStream source, OutputStream target, byte[] buffer) ^ It would be very helpful to verify that this method is needed extremely rare, if at all. In our case it would be an array of very specific (other than 1024, 2048, 4096 or 8192) size, i.e. unusually large or small or heavily reused one. To copy from an input stream to an output stream you need to try to read a byte from the input stream, at least once. Therefore search for usages of these methods will be a good start: java.io.InputStream#read(byte[]) not referred within the JDK at all java.io.InputStream#read(byte[], int, int) I found around 190 usage references in total of those there are the following 28 references that could use my proposed copy feature and 5 of those use 8192 bytes sized buffers. The short list following shows the path to the resource, line where the byte buffer is initialized and the buffer size taken jdk/src/solaris/classes/sun/print/UnixPrintJob.java 589 / 1024 jdk/src/windows/classes/sun/print/Win32PrintJob.java 444 / 1024 jdk/src/share/classes/sun/security/provider/X509Factory.java 119 / 2048 jdk/src/share/classes/com/sun/media/sound/JavaSoundAudioClip.java 356 / 16384 jdk/src/share/classes/com/sun/media/sound/AudioFileSoundbankReader.java 83 / 1024 jdk/src/share/classes/javax/swing/text/rtf/AbstractFilter.java 99 / 16384 jdk/src/share/classes/sun/net/NetworkServer.java 120 / 300 jdk/src/share/classes/sun/net/ftp/impl/FtpClient.java 1352 / 15000 jdk/src/share/classes/javax/swing/plaf/basic/BasicLookAndFeel.java 2148 / 1024 jdk/src/share/classes/java/awt/Font.java 934 / 8192 jdk/src/share/classes/sun/security/provider/certpath/X509CertPath.java 259 / 8192 jdk/src/share/classes/com/sun/media/sound/AiffFileWriter.java 236 / 4096 jdk/src/share/classes/sun/net/www/MimeLauncher.java 118 / 2048 jdk/src/share/classes/com/sun/org/apache/xml/internal/security/signature/XMLSignatureInput.java 492 / 4096 jdk/src/share/classes/java/util/jar/JarInputStream.java 109 / 8192 jdk/src/share/classes/sun/tools/jar/Main.java 822 / 8192 jdk/src/share/classes/javax/management/loading/MLet.java 1170 / 4096 jdk/src/share/classes/sun/awt/datatransfer/DataTransferer.java 1311 / 8192 jdk/src/share/classes/com/sun/org/apache/xml/internal/security/utils/resolver/implementations/ResolverDirectHTTP.java 144 / 4096 jdk/src/share/classes/org/jcp/xml/dsig/internal/dom/Utils.java 52 / 1024 jdk/src/share/classes/java/nio/file/Files.java 2906 / 8192 jdk/src/share/classes/sun/security/tools/keytool/Main.java 2140 / 4096 jdk/src/share/classes/com/sun/media/sound/ModelByteBuffer.java 194 / 1024 jdk/src/share/classes/sun/swing/SwingUtilities2.java 1566 / 1024 jdk/src/share/classes/com/sun/org/apache/xml/internal/security/utils/JavaUtils.java 140 / 4096 jdk/src/share/classes/com/sun/media/sound/SoftMixingClip.java 333 / multiple of 512 jdk/src/share/classes/com/sun/media/sound/StandardMidiFileWriter.java 141 / 16384 jdk/src/share/classes/com/sun/media/sound/WaveFileWriter.java 236 / 4096
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
I'm happy with any tool which is capable of usages search. Configured IDE is a good choice. -Pavel On 27 Nov 2014, at 22:59, Patrick Reinhart patr...@reini.net wrote: I have to say, that I normally use my IDE to search for references. What technique do you use to search for such patterns?
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Am Fri, 28 Nov 2014 12:09:11 + schrieb Pavel Rappo pavel.ra...@oracle.com: I'm happy with any tool which is capable of usages search. Configured IDE is a good choice. What usage would you actually search in the OpenJDK code base? I think it is not very representative to search for usage in the JDK code base even if it would be possible. However you can use for commons IO or Guava. Both have those helpers. (Which initself might be justification enough - the question is only pro or con including it in JCL :) Gruss Bernd
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Am 28.11.2014 um 20:15 schrieb Bernd Eckenfels e...@zusammenkunft.net: Am Fri, 28 Nov 2014 12:09:11 + schrieb Pavel Rappo pavel.ra...@oracle.com: I'm happy with any tool which is capable of usages search. Configured IDE is a good choice. What usage would you actually search in the OpenJDK code base? I think it is not very representative to search for usage in the JDK code base even if it would be possible. However you can use for commons IO or Guava. Both have those helpers. (Which initself might be justification enough - the question is only pro or con including it in JCL :) Gruss Bernd When looking at the guava library there seems to be no need for copy method that is able to pass in a separate byte array or CharBuffer. It would make more sense to have also a copy channels method on the existing java.nio.channels.Channels class instead. In all the projects I have worked on, there was always the point of having to either implement my own copy methods or use a certain library (for mostly only using the copy methods). Having said that, it seems to be a natural fit. Also when looking on the java.nio.files.Files class for copying with source or target Path’s it makes even more sense. -Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
What usage would you actually search in the OpenJDK code base? We're talking about copying with provided buffer, right? long copy(InputStream source, OutputStream target, byte[] buffer) ^ It would be very helpful to verify that this method is needed extremely rare, if at all. In our case it would be an array of very specific (other than 1024, 2048, 4096 or 8192) size, i.e. unusually large or small or heavily reused one. To copy from an input stream to an output stream you need to try to read a byte from the input stream, at least once. Therefore search for usages of these methods will be a good start: java.io.InputStream#read(byte[]) java.io.InputStream#read(byte[], int, int) -Pavel
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Hi Paul, Attached a patch containing my complete proposed code including tests. Patrick Am 24.11.2014 um 18:03 schrieb Patrick Reinhart patr...@reini.net: Well I will then suggest using IOUtils with two simple methods: public static long copy(InputStream source, OutputStream target) public static long copy(Readable source, Appendable target) To speed things along for now i recommend creating a new class say IOUtils or preferably ByteStreams. Hmm, in the case of Reader/Writer ByteStreams seams to be a bit confusing for me, should then the reader copy stuff go to also a separate class like „CharStreams“. Locally I created an implementation named „IOUtils“ containing a copy method for InputStream to OutputStream. Pick the one you prefer so we don't rat hole down the naming abyss :-) Pavel, your point about the overload with j.u.stream.Stream is a reasonable one (I am not too concerned by it myself). I hope one day in the future we can have Streamint :-) Paul. Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Hi Paul,
Seems not to work. See inline patch…
Patrick
Hi Paul,
Attached a patch containing my complete proposed code including tests.
Patrick
--- /dev/null 2014-11-24 08:00:19.282694686 +0100
+++ new/src/java.base/share/classes/java/io/IOUtil.java 2014-11-27
10:33:39.955791833 +0100
@@ -0,0 +1,86 @@
+/*
+ * Copyright (c) 1996, 2014, Oracle and/or its affiliates. All rights reserved.
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This code is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 only, as
+ * published by the Free Software Foundation. Oracle designates this
+ * particular file as subject to the Classpath exception as provided
+ * by Oracle in the LICENSE file that accompanied this code.
+ *
+ * This code is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+ * version 2 for more details (a copy is included in the LICENSE file that
+ * accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License version
+ * 2 along with this work; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
+ * or visit www.oracle.com if you need additional information or have any
+ * questions.
+ */
+
+package java.io;
+
+import java.nio.CharBuffer;
+
+/**
+ * Utility methods for copying data between a {@link InputStream} and
+ * {@link OutputStream} or between a {@link Readable} and a {@link Appendable}.
+ */
+public final class IOUtil {
+
+// buffer size used for reading and writing
+private static final int BUFFER_SIZE = 8192;
+
+private IOUtil() {
+throw new Error(no instances);
+}
+
+/**
+ * Reads all bytes from an input stream and writes them to an output
stream.
+ *
+ * @param source the input stream to read from
+ * @param target the output stream to write to
+ *
+ * @return the number of bytes successfully read and written
+ *
+ * @throws IOException if an I/O error occurs when reading or writing
+ */
+public static long copy(InputStream source, OutputStream target)
+throws IOException {
+long totalRead = 0L;
+byte[] buffer = new byte[BUFFER_SIZE];
+int read;
+while ((read = source.read(buffer)) -1) {
+target.write(buffer, 0, read);
+totalRead += read;
+}
+return totalRead;
+}
+
+/**
+ * Reads all characters from an readable and writes them to an appendable.
+ *
+ * @param source the readable to read from
+ * @param target the appendable to write to
+ *
+ * @return the number of characters successfully read and written
+ *
+ * @throws IOException if an I/O error occurs when reading or writing
+ */
+public static long copy(Readable source, Appendable target) throws
IOException {
+long totalRead = 0L;
+CharBuffer buffer = CharBuffer.allocate(BUFFER_SIZE);
+int read;
+while ((read = source.read(buffer)) -1) {
+ buffer.flip();
+target.append(buffer, 0, read);
+totalRead += read;
+}
+return totalRead;
+}
+}
--- /dev/null 2014-11-24 08:00:19.282694686 +0100
+++ new/test/java/io/IOUtil/CopyInputStreamOutputStream.java2014-11-27
10:33:40.352791912 +0100
@@ -0,0 +1,53 @@
+/*
+ * Copyright (c) 1998, 2014, Oracle and/or its affiliates. All rights reserved.
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This code is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 only, as
+ * published by the Free Software Foundation.
+ *
+ * This code is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+ * version 2 for more details (a copy is included in the LICENSE file that
+ * accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License version
+ * 2 along with this work; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
+ * or visit www.oracle.com if you need additional information or have any
+ * questions.
+ */
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOUtil;
+import java.util.Arrays;
+
+/* @test
+ @summary Test copy a InputStream to a OutputStream reporting the amount
+of bytes copied in total.
+ */
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
I've created an initial webrev for this change: http://cr.openjdk.java.net/~prappo/ioutil/webrev.00/ Welcome to discussion. P.S. The only thing I changed was the license header. -Pavel On 27 Nov 2014, at 09:48, Patrick Reinhart patr...@reini.net wrote: Hi Paul, Seems not to work. See inline patch…
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Hi Pavel, Thanks very much for your help. I’ve try my best to get author state ;-) Patrick Am 27.11.2014 um 12:58 schrieb Pavel Rappo pavel.ra...@oracle.com: I've created an initial webrev for this change: http://cr.openjdk.java.net/~prappo/ioutil/webrev.00/ Welcome to discussion. P.S. The only thing I changed was the license header. -Pavel On 27 Nov 2014, at 09:48, Patrick Reinhart patr...@reini.net wrote: Hi Paul, Seems not to work. See inline patch…
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
At the bare minimum I would use runtime null/range checks and give a little bit more freedom for those who want to control buffer (or probably reuse already existing one): http://cr.openjdk.java.net/~prappo/ioutil/webrev.01/ I would also add some clarifications in the javadoc. -Pavel
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Added null checks also to Readable/Appendable copy methods
Patrick
Am 27.11.2014 um 13:26 schrieb Pavel Rappo pavel.ra...@oracle.com:
At the bare minimum I would use runtime null/range checks and give a little
bit more freedom for those who want to control buffer (or probably reuse
already existing one): http://cr.openjdk.java.net/~prappo/ioutil/webrev.01/
I would also add some clarifications in the javadoc.
-Pavel
diff -r 194458fe7339 src/java.base/share/classes/java/io/IOUtil.java
--- a/src/java.base/share/classes/java/io/IOUtil.java Thu Nov 27 13:50:36
2014 +0100
+++ b/src/java.base/share/classes/java/io/IOUtil.java Thu Nov 27 14:00:07
2014 +0100
@@ -71,7 +71,6 @@
*/
public static long copy(InputStream source, OutputStream target,
byte[] buffer) throws IOException {
-
Objects.requireNonNull(source, source);
Objects.requireNonNull(target, target);
Objects.requireNonNull(buffer, buffer);
@@ -100,15 +99,15 @@
* @throws IOException if an I/O error occurs when reading or writing
*/
public static long copy(Readable source, Appendable target) throws
IOException {
-long totalRead = 0L;
+Objects.requireNonNull(source, source);
+Objects.requireNonNull(target, target);
+long copied = 0L;
CharBuffer buffer = CharBuffer.allocate(BUFFER_SIZE);
-int read;
-while ((read = source.read(buffer)) -1) {
+for (int read; (read = source.read(buffer)) -1; copied += read) {
buffer.flip();
target.append(buffer, 0, read);
-totalRead += read;
}
-return totalRead;
+return copied;
}
}
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Patrick, as soon as I get feedback from Alan and Paul I'll merge these changes
into the current version.
1. I wonder if it makes sense to provide an additional method where client
specifies CharBuffer to be used?
Moreover if the first argument is CharSequence then we don't even need a buffer:
/**
* Reads all characters from a readable and appends them to an appendable.
*
* @param source the readable to read from ({@code source != null})
* @param target the appendable to append to ({@code target != null})
*
* @return the number of characters copied
*
* @throws IOException if an I/O error occurs when reading or appending
*/
public static long copy(Readable source, Appendable target)
throws IOException {
Objects.requireNonNull(source, source);
Objects.requireNonNull(target, target);
if (source instanceof CharSequence)
return copy1((CharSequence) source, target);
return copy0(source, target, CharBuffer.allocate(BUFFER_SIZE));
}
/**
* Reads all characters from a readable and appends them to an appendable.
*
* @param source the readable to read from ({@code source != null})
* @param target the appendable to append to ({@code target != null})
* @param buffer the intermediate buffer to use
* ({@code buffer != null buffer.hasRemaining()
* !buffer.isReadOnly()})
* @return the number of characters copied
* @throws IOException if an I/O error occurs when reading or appending
*/
public static long copy(Readable source, Appendable target,
CharBuffer buffer) throws IOException {
Objects.requireNonNull(source, source);
Objects.requireNonNull(target, target);
Objects.requireNonNull(buffer, buffer);
if (!buffer.hasRemaining())
throw new IllegalArgumentException(Insufficient buffer size);
if (buffer.isReadOnly())
throw new IllegalArgumentException(buffer is read only);
return copy0(source, target, buffer);
}
private static long copy1(CharSequence source, Appendable target)
throws IOException {
long copied = source.length();
target.append(source);
return copied;
}
private static long copy0(Readable source, Appendable target,
CharBuffer buffer) throws IOException {
2. I think guts of the copy method could be changed to something like this:
long copied = 0L;
+ int oldLim = buffer.limit();
for (int read; (read = source.read(buffer)) -1; copied += read) {
buffer.flip();
target.append(buffer, 0, read);
+ buffer.limit(oldLim);
}
return copied;
Otherwise on each iteration you're potentially shrinking available buffer
storage based on
amount of data read into it (if I'm not mistaken). So each time you should
restore the limit.
3. We should think of revisiting and including the only method from
sun.misc.IOUtils. Removing
the class and reparenting its dependencies.
-Pavel
diff -r 194458fe7339 src/java.base/share/classes/java/io/IOUtil.java
--- a/src/java.base/share/classes/java/io/IOUtil.java Thu Nov 27 13:50:36
2014 +0100
+++ b/src/java.base/share/classes/java/io/IOUtil.java Thu Nov 27 16:08:09
2014 +0100
@@ -47,10 +47,28 @@
/**
* Reads all bytes from an input stream and writes them to an output
stream.
*
- * @param source the input stream to read from
- * @param target the output stream to write to
- * @return the number of bytes successfully copied
- * @throws IOException if an I/O error occurs when reading or writing
+ * p If an I/O error occurs reading from the input stream or writing
to
+ * the output stream, then it may do so after the some bytes have been
read
+ * or written. Consequently the input stream may not be at end of stream
and
+ * may be in an inconsistent state. It is strongly recommended that the
input
+ * and output stream be promptly closed if an I/O error occurs.
+ *
+ * p This method may block indefinitely reading from the input stream
(or
+ * writing to the output stream). The behavior for the case that the
input
+ * and output stream is iasynchronously closed/i or the thread
interrupted
+ * during the copy is highly input and output stream system provider
specific
+ * and therefore not specified.
+ *
+ * @param source
+ * the input stream to read from
+ * @param target
+ * the output stream to write to
+ *
+ * @return the number of bytes successfully copied
+ * @throws IOException
+ * if an I/O error occurs when reading or writing. In
particular,
+ * an codeIOException/code may be thrown if the output
stream
+ * has been closed.
*/
public static long copy(InputStream source, OutputStream target)
throws
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Patrick, as soon as I get feedback from Alan and Paul I'll merge these
changes into the current version.
1. I wonder if it makes sense to provide an additional method where client
specifies CharBuffer to be used?
Moreover if the first argument is CharSequence then we don't even need a
buffer:
/**
* Reads all characters from a readable and appends them to an appendable.
*
* @param source the readable to read from ({@code source != null})
* @param target the appendable to append to ({@code target != null})
*
* @return the number of characters copied
*
* @throws IOException if an I/O error occurs when reading or appending
*/
public static long copy(Readable source, Appendable target)
throws IOException {
Objects.requireNonNull(source, source);
Objects.requireNonNull(target, target);
if (source instanceof CharSequence)
return copy1((CharSequence) source, target);
return copy0(source, target, CharBuffer.allocate(BUFFER_SIZE));
}
/**
* Reads all characters from a readable and appends them to an appendable.
*
* @param source the readable to read from ({@code source != null})
* @param target the appendable to append to ({@code target != null})
* @param buffer the intermediate buffer to use
* ({@code buffer != null buffer.hasRemaining()
* !buffer.isReadOnly()})
* @return the number of characters copied
* @throws IOException if an I/O error occurs when reading or appending
*/
public static long copy(Readable source, Appendable target,
CharBuffer buffer) throws IOException {
Objects.requireNonNull(source, source);
Objects.requireNonNull(target, target);
Objects.requireNonNull(buffer, buffer);
if (!buffer.hasRemaining())
throw new IllegalArgumentException(Insufficient buffer size);
if (buffer.isReadOnly())
throw new IllegalArgumentException(buffer is read only);
return copy0(source, target, buffer);
}
private static long copy1(CharSequence source, Appendable target)
throws IOException {
long copied = source.length();
target.append(source);
return copied;
}
private static long copy0(Readable source, Appendable target,
CharBuffer buffer) throws IOException {
Seems reasonable to me. Will you add those changes additionally to mine then?
2. I think guts of the copy method could be changed to something like this:
long copied = 0L;
+ int oldLim = buffer.limit();
for (int read; (read = source.read(buffer)) -1; copied += read) {
buffer.flip();
target.append(buffer, 0, read);
+ buffer.limit(oldLim);
}
return copied;
Otherwise on each iteration you're potentially shrinking available buffer
storage based on
amount of data read into it (if I'm not mistaken). So each time you should
restore the limit.
There I’m not that experienced either. I’m shamed about it, but never used
CharBuffer by myself so far… (my implementation could therefore be not
completely correct)
3. We should think of revisiting and including the only method from
sun.misc.IOUtils. Removing
the class and reparenting its dependencies.
That would be a followup task - on the way to authorship ;-)
-Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Yes I will. If I don't see any objections. -Pavel On 27 Nov 2014, at 16:10, Patrick Reinhart patr...@reini.net wrote: Will you add those changes additionally to mine then?
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
On 27/11/2014 15:56, Pavel Rappo wrote: Patrick, as soon as I get feedback from Alan and Paul I'll merge these changes into the current version. 1. I wonder if it makes sense to provide an additional method where client specifies CharBuffer to be used? Moreover if the first argument is CharSequence then we don't even need a buffer: I would suggest starting out with the simple copy(source, target) and only add additional variants if there is strong need. In this case, is a variant that allows the intermediate buffer to be specified be something that would be widely used? -Alan
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Am 27.11.2014 um 17:57 schrieb Alan Bateman alan.bate...@oracle.com: On 27/11/2014 15:56, Pavel Rappo wrote: Patrick, as soon as I get feedback from Alan and Paul I'll merge these changes into the current version. 1. I wonder if it makes sense to provide an additional method where client specifies CharBuffer to be used? Moreover if the first argument is CharSequence then we don't even need a buffer: I would suggest starting out with the simple copy(source, target) and only add additional variants if there is strong need. In this case, is a variant that allows the intermediate buffer to be specified be something that would be widely used? -Alan In our codebase here we have variants of different buffer sizes, mostly there are some with smaller sizes than the default I took from the Files class. To have the copy function within the JDK instead using a separate library or have this code written over and over again is much more important to me. -Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
On 27/11/2014 17:40, Patrick Reinhart wrote: : In our codebase here we have variants of different buffer sizes, mostly there are some with smaller sizes than the default I took from the Files class. To have the copy function within the JDK instead using a separate library or have this code written over and over again is much more important to me. I think a variant that has a size as an argument is worth considering, we've all done that. That said, Pavel's suggestion seem to be a intermediate buffer the hint as to the chunk size. -Alan
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Patrick, Alan is right in the sense that we have to estimate usefulness of each method wisely. Unfortunately your codebase (or anyone else's) might have some very specific biases and thus might not be a good sample to base conclusions upon. We need some more data to justify this proposal. The good start could be the jdk itself -- to see if those copy-with-my-buffer methods could be useful here. -Pavel On 27 Nov 2014, at 17:40, Patrick Reinhart patr...@reini.net wrote: Am 27.11.2014 um 17:57 schrieb Alan Bateman alan.bate...@oracle.com: On 27/11/2014 15:56, Pavel Rappo wrote: Patrick, as soon as I get feedback from Alan and Paul I'll merge these changes into the current version. 1. I wonder if it makes sense to provide an additional method where client specifies CharBuffer to be used? Moreover if the first argument is CharSequence then we don't even need a buffer: I would suggest starting out with the simple copy(source, target) and only add additional variants if there is strong need. In this case, is a variant that allows the intermediate buffer to be specified be something that would be widely used? -Alan In our codebase here we have variants of different buffer sizes, mostly there are some with smaller sizes than the default I took from the Files class. To have the copy function within the JDK instead using a separate library or have this code written over and over again is much more important to me. -Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Patrick, Alan is right in the sense that we have to estimate usefulness of each method wisely. Unfortunately your codebase (or anyone else’s) might have some very specific biases and thus might not be a good sample to base conclusions upon. Aggreed. We need some more data to justify this proposal. The good start could be the jdk itself -- to see if those copy-with-my-buffer methods could be useful here. -Pavel I have to say, that I normally use my IDE to search for references. What technique do you use to search for such patterns? -Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Hi Patrick, To speed things along for now i recommend creating a new class say IOUtils or preferably ByteStreams. You should keep things simple for an initial iteration and just add one static method :-) which is essentially a refined copy of the private method that Pavel pointed out. Then write some tests for that method [1]. Then consider any nio related classes for an equivalent copy method. Some other candidates to consider are on Guava's ByteStreams class: http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/io/ByteStreams.html It's very easy to get carried away and over-engineer such utility methods thinking they need to be all things to all people so we need to consider them carefully and i think the ones you included with size and reporting are not required. Simply just one static method copying input to output will go a long way (i have lost count of how many times i have reimplemented that!). Hth, Paul. [1] You can download jtreg from here: https://adopt-openjdk.ci.cloudbees.com/view/OpenJDK/job/jtreg/ for executing JDK tests. Depending on how you want to write your tests you can also make 'em run standalone too, but sometimes it is easier convenient to use testng and execute via jtreg. On Nov 21, 2014, at 3:12 PM, Patrick Reinhart patr...@reini.net wrote: Am 21.11.2014 um 09:36 schrieb Alan Bateman alan.bate...@oracle.com: On 21/11/2014 03:05, Stuart Marks wrote: Thanks to Pavel for pointing out the existing copy operations in the nio Files class. I think there's a case for the InputStream = OutputStream copy method to be placed there too, although I admit it is somewhat unusual in that it doesn't actually have anything to do with files. At my first encounter with the nio.Files class some years ago I saw the following copying methods: copy(istream, targetfile) copy(sourcefile, ostream) copy(sourcefile, targetfile) and I immediately thought, Where is copy(istream, ostream)? So to me at least, it makes more sense to be in the Files class than in some newly created IOUtils class. (I'd like to hear further from Alan on this.) As Pavel pointed out, the logic is also already there in the Files class. Probably the way to proceed would be to rename the existing (private) method to be something like copyInternal() and then create a new, public copy(istream, ostream) method that does argument checking before calling copyInternal(). The Files class works on files, it's not the right place for a general purpose copy(InputStream, OutputStream). That was the reason in my proposal to not put those methods on the Files class. I also would not try to find such methods there. Personally I tried to look for such method on „Streams“ ;-) When we prototyped a copy(InputStream, OutputStream) and many other I/O methods a few years ago then the intention was to put it java.io. One option on the table was a Collections-like utility class with static methods. Another option was to add methods to InputStream, Reader, etc. A concern with the latter was whether the new methods would cause problems for existing InputStream/etc. implementations, similar to concerns about adding default methods to the collection types in 8. I think that discussion needs to happen again and having a few prototypes to get experience with the various approaches would be good too. -Alan It would be good to have such methods in the next release and I would love to help here. Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Paul, I wouldn't name it 'ByteStreams' because of the possible subjective associations (and confusion) with java.util.stream.IntStream, java.util.stream.LongStream, etc. IOStreams? -Pavel On 24 Nov 2014, at 14:32, Paul Sandoz paul.san...@oracle.com wrote: i recommend creating a new class say IOUtils or preferably ByteStreams.
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Hi Paul, Am 24.11.2014 um 15:32 schrieb Paul Sandoz paul.san...@oracle.com: Hi Patrick, To speed things along for now i recommend creating a new class say IOUtils or preferably ByteStreams. Hmm, in the case of Reader/Writer ByteStreams seams to be a bit confusing for me, should then the reader copy stuff go to also a separate class like „CharStreams“. Locally I created an implementation named „IOUtils“ containing a copy method for InputStream to OutputStream. You should keep things simple for an initial iteration and just add one static method :-) which is essentially a refined copy of the private method that Pavel pointed out. Then write some tests for that method [1]. Then consider any nio related classes for an equivalent copy method. Some other candidates to consider are on Guava's ByteStreams class: http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/io/ByteStreams.html I will also take a look at this It's very easy to get carried away and over-engineer such utility methods thinking they need to be all things to all people so we need to consider them carefully and i think the ones you included with size and reporting are not required. Simply just one static method copying input to output will go a long way (i have lost count of how many times i have reimplemented that!). For now a single copy(from, to) for Input-/OutputStream and Readable/Appendable would be perfect for now. Hth, Paul. [1] You can download jtreg from here: https://adopt-openjdk.ci.cloudbees.com/view/OpenJDK/job/jtreg/ for executing JDK tests. Depending on how you want to write your tests you can also make ‚em run standalone too, but sometimes it is easier convenient to use testng and execute via jtreg. The environment I will update in the next. Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
On Nov 24, 2014, at 5:34 PM, Patrick Reinhart patr...@reini.net wrote: Hi Paul, Am 24.11.2014 um 15:32 schrieb Paul Sandoz paul.san...@oracle.com: Hi Patrick, To speed things along for now i recommend creating a new class say IOUtils or preferably ByteStreams. Hmm, in the case of Reader/Writer ByteStreams seams to be a bit confusing for me, should then the reader copy stuff go to also a separate class like „CharStreams“. Locally I created an implementation named „IOUtils“ containing a copy method for InputStream to OutputStream. Pick the one you prefer so we don't rat hole down the naming abyss :-) Pavel, your point about the overload with j.u.stream.Stream is a reasonable one (I am not too concerned by it myself). I hope one day in the future we can have Streamint :-) Paul.
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Well I will then suggest using IOUtils with two simple methods: public static long copy(InputStream source, OutputStream target) public static long copy(Readable source, Appendable target) To speed things along for now i recommend creating a new class say IOUtils or preferably ByteStreams. Hmm, in the case of Reader/Writer ByteStreams seams to be a bit confusing for me, should then the reader copy stuff go to also a separate class like „CharStreams“. Locally I created an implementation named „IOUtils“ containing a copy method for InputStream to OutputStream. Pick the one you prefer so we don't rat hole down the naming abyss :-) Pavel, your point about the overload with j.u.stream.Stream is a reasonable one (I am not too concerned by it myself). I hope one day in the future we can have Streamint :-) Paul. Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
On 21/11/2014 03:05, Stuart Marks wrote: Thanks to Pavel for pointing out the existing copy operations in the nio Files class. I think there's a case for the InputStream = OutputStream copy method to be placed there too, although I admit it is somewhat unusual in that it doesn't actually have anything to do with files. At my first encounter with the nio.Files class some years ago I saw the following copying methods: copy(istream, targetfile) copy(sourcefile, ostream) copy(sourcefile, targetfile) and I immediately thought, Where is copy(istream, ostream)? So to me at least, it makes more sense to be in the Files class than in some newly created IOUtils class. (I'd like to hear further from Alan on this.) As Pavel pointed out, the logic is also already there in the Files class. Probably the way to proceed would be to rename the existing (private) method to be something like copyInternal() and then create a new, public copy(istream, ostream) method that does argument checking before calling copyInternal(). The Files class works on files, it's not the right place for a general purpose copy(InputStream, OutputStream). When we prototyped a copy(InputStream, OutputStream) and many other I/O methods a few years ago then the intention was to put it java.io. One option on the table was a Collections-like utility class with static methods. Another option was to add methods to InputStream, Reader, etc. A concern with the latter was whether the new methods would cause problems for existing InputStream/etc. implementations, similar to concerns about adding default methods to the collection types in 8. I think that discussion needs to happen again and having a few prototypes to get experience with the various approaches would be good too. -Alan
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Am 21.11.2014 um 09:36 schrieb Alan Bateman alan.bate...@oracle.com: On 21/11/2014 03:05, Stuart Marks wrote: Thanks to Pavel for pointing out the existing copy operations in the nio Files class. I think there's a case for the InputStream = OutputStream copy method to be placed there too, although I admit it is somewhat unusual in that it doesn't actually have anything to do with files. At my first encounter with the nio.Files class some years ago I saw the following copying methods: copy(istream, targetfile) copy(sourcefile, ostream) copy(sourcefile, targetfile) and I immediately thought, Where is copy(istream, ostream)? So to me at least, it makes more sense to be in the Files class than in some newly created IOUtils class. (I'd like to hear further from Alan on this.) As Pavel pointed out, the logic is also already there in the Files class. Probably the way to proceed would be to rename the existing (private) method to be something like copyInternal() and then create a new, public copy(istream, ostream) method that does argument checking before calling copyInternal(). The Files class works on files, it's not the right place for a general purpose copy(InputStream, OutputStream). That was the reason in my proposal to not put those methods on the Files class. I also would not try to find such methods there. Personally I tried to look for such method on „Streams“ ;-) When we prototyped a copy(InputStream, OutputStream) and many other I/O methods a few years ago then the intention was to put it java.io. One option on the table was a Collections-like utility class with static methods. Another option was to add methods to InputStream, Reader, etc. A concern with the latter was whether the new methods would cause problems for existing InputStream/etc. implementations, similar to concerns about adding default methods to the collection types in 8. I think that discussion needs to happen again and having a few prototypes to get experience with the various approaches would be good too. -Alan It would be good to have such methods in the next release and I would love to help here. Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Hi Patrick,
There is at least one method in the JDK with similar characteristics:
java.nio.file.Files#copy(java.io.InputStream, java.io.OutputStream)
But, (1) it has a private access (2) even if we made it public I doubt
java.nio.file.Files would be a good place for it
P.S. The thing that confuses me though, is the progress consumer. I believe
this feature is rarely needed (at least in a way you described it).
If you want to do something like this, you should probably go asynchronous with
full blown solution like what they have in javax.swing.SwingWorker.
-Pavel
On 13 Nov 2014, at 19:31, Patrick Reinhart patr...@reini.net wrote:
Hi there,
In the followup of a BOF with Stephen Colebourne with his ideas of small
library changes that may could get in JDK9. As of the fact that in the
codebase of my company there are several locations where we copy from/to IO
stream over and over again using either external libraries or do it by
ourselves, I suggested to have some support for easy coping data between
Input-/OutputStream or Reader/Writer without having to use to external
libraries.
My suggestion would look something like this:
try (InputStream in = … ; OutputStream out = …) {
in.copyTo(out);
}
Or from the other end:
try (InputStream in = … ; OutputStream out = …) {
out.copyFrom(in);
}
The same for character based streams:
try (Reader in = …; Writer out = …) {
in.copyTo(out);
}
or
try (Reader in = …; Writer out = …) {
out.copyFrom(in);
}
What do you think about this idea? I could imagine, that we could also pass a
IntConsumer / LongConsumer in, that will be called with each amount of
successfully copied amount of bytes/chars in case we need a total amount of
data or displaying the progress of copy externally.
I’m looking forward to your opinions upon this proposal.
Cheers
Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Hi Pavel,
Am 20.11.2014 um 10:22 schrieb Pavel Rappo pavel.ra...@oracle.com:
There is at least one method in the JDK with similar characteristics:
java.nio.file.Files#copy(java.io.InputStream, java.io.OutputStream)
But, (1) it has a private access (2) even if we made it public I doubt
java.nio.file.Files would be a good place for it
I would suggest to introduce a separate IOUtil class holding such utility
methods. Additionally to copyTo() and copyFrom() method could be added later
for more intuitive usage. Also the copy method within the Files method could be
replaced with a static reference to the IOUtil class.
P.S. The thing that confuses me though, is the progress consumer. I believe
this feature is rarely needed (at least in a way you described it).
If you want to do something like this, you should probably go asynchronous
with full blown solution like what they have in javax.swing.SwingWorker.
-Pavel
The method having a IntConsumer I have already discussed with some other
colleagues and they suggested to better use a IntPredicate in order to have the
possibility to interrupt the copy process, without having to interrupt any
threads. Additionally there is still the possibility to use such a Predicate
for counting the total or using the information for other possibilities.
Here’s implementation example for Input-/OutputStream (similar code would apply
for Reader/Writer):
/**
* Reads all bytes from an input stream and writes them to an output stream.
*
* @param source the input stream to read from
* @param target the path to the file
*
* @return the number of bytes read and successfully written
*
* @throws IOException if an I/O error occurs when reading or writing
*/
public static long copy(InputStream source, OutputStream target)
throws IOException {
LongAdder totalread = new LongAdder();
copy(source, target, nread - {
totalread.add(nread);
return true;
});
return totalread.sum();
}
/**
* Reads all bytes from an input stream and writes them to an output stream.
* While doing so, the given predicate is called with the amount of bytes
* being read, to decide whenever they should be written to the output
* stream. If the predicate returns codefalse/code the copy operation is
* stopped and no more data is written to the output stream.
*
* @param source the input stream to read from
* @param target the path to the file
* @param predicate the predicate tests if the copy operation should proceed
*
* @throws IOException if an I/O error occurs when reading or writing
*/
public static void copy(InputStream source, OutputStream target,
IntPredicate predicate) throws IOException {
byte[] buf = new byte[BUFFER_SIZE];
int n;
while ((n = source.read(buf)) 0 predicate.test(n)) {
target.write(buf, 0, n);
}
}
Cheers
Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Hi Patrick, Good to meet you in Antwerp! On 11/20/14 10:30 AM, Patrick Reinhart wrote: Am 20.11.2014 um 10:22 schrieb Pavel Rappo pavel.ra...@oracle.com: There is at least one method in the JDK with similar characteristics: java.nio.file.Files#copy(java.io.InputStream, java.io.OutputStream) But, (1) it has a private access (2) even if we made it public I doubt java.nio.file.Files would be a good place for it I would suggest to introduce a separate IOUtil class holding such utility methods. Additionally to copyTo() and copyFrom() method could be added later for more intuitive usage. Also the copy method within the Files method could be replaced with a static reference to the IOUtil class. Thanks to Pavel for pointing out the existing copy operations in the nio Files class. I think there's a case for the InputStream = OutputStream copy method to be placed there too, although I admit it is somewhat unusual in that it doesn't actually have anything to do with files. At my first encounter with the nio.Files class some years ago I saw the following copying methods: copy(istream, targetfile) copy(sourcefile, ostream) copy(sourcefile, targetfile) and I immediately thought, Where is copy(istream, ostream)? So to me at least, it makes more sense to be in the Files class than in some newly created IOUtils class. (I'd like to hear further from Alan on this.) As Pavel pointed out, the logic is also already there in the Files class. Probably the way to proceed would be to rename the existing (private) method to be something like copyInternal() and then create a new, public copy(istream, ostream) method that does argument checking before calling copyInternal(). P.S. The thing that confuses me though, is the progress consumer. I believe this feature is rarely needed (at least in a way you described it). If you want to do something like this, you should probably go asynchronous with full blown solution like what they have in javax.swing.SwingWorker. -Pavel The method having a IntConsumer I have already discussed with some other colleagues and they suggested to better use a IntPredicate in order to have the possibility to interrupt the copy process, without having to interrupt any threads. Additionally there is still the possibility to use such a Predicate for counting the total or using the information for other possibilities. I'd suggest starting simple with a copy(istream, ostream) operation and considering some kind of interruptible, progress-reporting operation separately. It would seem quite limiting if the *only* progress-reporting operation in the system were the stream copier. We'd want a way to apply such a mechanism to other long-running operations. I think the progress update reports also need to be decoupled from the actual I/O operations. For example, the current buffer size in nio.Files is 8192 bytes. If the streams are connected to a fast network connection, this will result in thousands of calls to the predicate per second. On the other hand, if the buffer size were raised, and the streams are connected to a slow network connection -- like my home internet connection :-) -- that might result in too few callbacks per second. How to report progress from a running operation is an interesting problem and it's worthy of considering, but a copying utility doesn't seem like quite the right place for it. s'marks
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
On 11/21/2014 04:05 AM, Stuart Marks wrote: Hi Patrick, Good to meet you in Antwerp! On 11/20/14 10:30 AM, Patrick Reinhart wrote: Am 20.11.2014 um 10:22 schrieb Pavel Rappo pavel.ra...@oracle.com: There is at least one method in the JDK with similar characteristics: java.nio.file.Files#copy(java.io.InputStream, java.io.OutputStream) But, (1) it has a private access (2) even if we made it public I doubt java.nio.file.Files would be a good place for it I would suggest to introduce a separate IOUtil class holding such utility methods. Additionally to copyTo() and copyFrom() method could be added later for more intuitive usage. Also the copy method within the Files method could be replaced with a static reference to the IOUtil class. Thanks to Pavel for pointing out the existing copy operations in the nio Files class. I think there's a case for the InputStream = OutputStream copy method to be placed there too, although I admit it is somewhat unusual in that it doesn't actually have anything to do with files. At my first encounter with the nio.Files class some years ago I saw the following copying methods: copy(istream, targetfile) copy(sourcefile, ostream) copy(sourcefile, targetfile) and I immediately thought, Where is copy(istream, ostream)? So to me at least, it makes more sense to be in the Files class than in some newly created IOUtils class. (I'd like to hear further from Alan on this.) As Pavel pointed out, the logic is also already there in the Files class. Probably the way to proceed would be to rename the existing (private) method to be something like copyInternal() and then create a new, public copy(istream, ostream) method that does argument checking before calling copyInternal(). P.S. The thing that confuses me though, is the progress consumer. I believe this feature is rarely needed (at least in a way you described it). If you want to do something like this, you should probably go asynchronous with full blown solution like what they have in javax.swing.SwingWorker. -Pavel The method having a IntConsumer I have already discussed with some other colleagues and they suggested to better use a IntPredicate in order to have the possibility to interrupt the copy process, without having to interrupt any threads. Additionally there is still the possibility to use such a Predicate for counting the total or using the information for other possibilities. I'd suggest starting simple with a copy(istream, ostream) operation and considering some kind of interruptible, progress-reporting operation separately. It would seem quite limiting if the *only* progress-reporting operation in the system were the stream copier. We'd want a way to apply such a mechanism to other long-running operations. I think the progress update reports also need to be decoupled from the actual I/O operations. For example, the current buffer size in nio.Files is 8192 bytes. If the streams are connected to a fast network connection, this will result in thousands of calls to the predicate per second. On the other hand, if the buffer size were raised, and the streams are connected to a slow network connection -- like my home internet connection :-) -- that might result in too few callbacks per second. How to report progress from a running operation is an interesting problem and it's worthy of considering, but a copying utility doesn't seem like quite the right place for it. I also think you don't need a specific progress API given that you can write an InputStream that act as a proxy on top of the real inputStream and delegate the number of read bytes to a specific lambda. s'marks cheers, Rémi
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Hi Stuart, Am 21.11.2014 um 04:05 schrieb Stuart Marks stuart.ma...@oracle.com: Hi Patrick, Good to meet you in Antwerp! The pleasure was all mine :-) On 11/20/14 10:30 AM, Patrick Reinhart wrote: Am 20.11.2014 um 10:22 schrieb Pavel Rappo pavel.ra...@oracle.com: There is at least one method in the JDK with similar characteristics: java.nio.file.Files#copy(java.io.InputStream, java.io.OutputStream) But, (1) it has a private access (2) even if we made it public I doubt java.nio.file.Files would be a good place for it I would suggest to introduce a separate IOUtil class holding such utility methods. Additionally to copyTo() and copyFrom() method could be added later for more intuitive usage. Also the copy method within the Files method could be replaced with a static reference to the IOUtil class. Thanks to Pavel for pointing out the existing copy operations in the nio Files class. I think there's a case for the InputStream = OutputStream copy method to be placed there too, although I admit it is somewhat unusual in that it doesn't actually have anything to do with files. At my first encounter with the nio.Files class some years ago I saw the following copying methods: copy(istream, targetfile) copy(sourcefile, ostream) copy(sourcefile, targetfile) and I immediately thought, Where is copy(istream, ostream)? So to me at least, it makes more sense to be in the Files class than in some newly created IOUtils class. (I’d like to hear further from Alan on this.) The reason to not put those copy operations in the Files class in the first place, is the fact that this operation is not specific to files. If I would like to copy some database blob stream to the servlet output stream for example, there is no relation to a file at all. As Pavel pointed out, the logic is also already there in the Files class. Probably the way to proceed would be to rename the existing (private) method to be something like copyInternal() and then create a new, public copy(istream, ostream) method that does argument checking before calling copyInternal(). My first idea was to even in fact, to some sort of copy function on the InputStream like „copyFrom(OutputStream)“ and on the OutputStream something like „copyTo(InputStream)“. Those methods would use the copyInternal“ method on Files if the one would be made default access. The same would the also apply to Reader/Writer of course, for which there would be the need to have a similar copy method. P.S. The thing that confuses me though, is the progress consumer. I believe this feature is rarely needed (at least in a way you described it). If you want to do something like this, you should probably go asynchronous with full blown solution like what they have in javax.swing.SwingWorker. -Pavel The method having a IntConsumer I have already discussed with some other colleagues and they suggested to better use a IntPredicate in order to have the possibility to interrupt the copy process, without having to interrupt any threads. Additionally there is still the possibility to use such a Predicate for counting the total or using the information for other possibilities. I'd suggest starting simple with a copy(istream, ostream) operation and considering some kind of interruptible, progress-reporting operation separately. It would seem quite limiting if the *only* progress-reporting operation in the system were the stream copier. We’d want a way to apply such a mechanism to other long-running operations. Agreed. I could implement some Input-/OutputFilterStream that could implement such a behavior, without having to do that on the API. I think the progress update reports also need to be decoupled from the actual I/O operations. For example, the current buffer size in nio.Files is 8192 bytes. If the streams are connected to a fast network connection, this will result in thousands of calls to the predicate per second. On the other hand, if the buffer size were raised, and the streams are connected to a slow network connection -- like my home internet connection :-) -- that might result in too few callbacks per second. I see your point for decoupling there. So it would be more practical to have the actual buffer size being optionally specified? - by the way: I also don’t have a real fast internet connection ;-) How to report progress from a running operation is an interesting problem and it’s worthy of considering, but a copying utility doesn't seem like quite the right place for it. I see your point there. Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Am 13.11.2014 um 20:59 schrieb Alan Bateman alan.bate...@oracle.com: On 13/11/2014 19:31, Patrick Reinhart wrote: Hi there, In the followup of a BOF with Stephen Colebourne with his ideas of small library changes that may could get in JDK9. As of the fact that in the codebase of my company there are several locations where we copy from/to IO stream over and over again using either external libraries or do it by ourselves, I suggested to have some support for easy coping data between Input-/OutputStream or Reader/Writer without having to use to external libraries. Long overdue. I remember we prototyped methods like this (and much more) during JDK 7 but didn't do enough at the time to actually get them in. We did include a bunch of easy to use methods in the Files class at the time, including copy between input/output streams and files, but we didn't introduce an IOUtils or such class in java.io. I did some implementation of a IOUtils class for the basic copy task already. Should I post a patch with the work I already done? Best regards, Patrick
Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Hi there,
In the followup of a BOF with Stephen Colebourne with his ideas of small
library changes that may could get in JDK9. As of the fact that in the codebase
of my company there are several locations where we copy from/to IO stream over
and over again using either external libraries or do it by ourselves, I
suggested to have some support for easy coping data between Input-/OutputStream
or Reader/Writer without having to use to external libraries.
My suggestion would look something like this:
try (InputStream in = … ; OutputStream out = …) {
in.copyTo(out);
}
Or from the other end:
try (InputStream in = … ; OutputStream out = …) {
out.copyFrom(in);
}
The same for character based streams:
try (Reader in = …; Writer out = …) {
in.copyTo(out);
}
or
try (Reader in = …; Writer out = …) {
out.copyFrom(in);
}
What do you think about this idea? I could imagine, that we could also pass a
IntConsumer / LongConsumer in, that will be called with each amount of
successfully copied amount of bytes/chars in case we need a total amount of
data or displaying the progress of copy externally.
I’m looking forward to your opinions upon this proposal.
Cheers
Patrick
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
On 13/11/2014 19:31, Patrick Reinhart wrote: Hi there, In the followup of a BOF with Stephen Colebourne with his ideas of small library changes that may could get in JDK9. As of the fact that in the codebase of my company there are several locations where we copy from/to IO stream over and over again using either external libraries or do it by ourselves, I suggested to have some support for easy coping data between Input-/OutputStream or Reader/Writer without having to use to external libraries. Long overdue. I remember we prototyped methods like this (and much more) during JDK 7 but didn't do enough at the time to actually get them in. We did include a bunch of easy to use methods in the Files class at the time, including copy between input/output streams and files, but we didn't introduce an IOUtils or such class in java.io. -Alan
Re: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream
Hi Alan,
Where would you suggest to put such methods though? Should we aim for a
„IOUitls class holding such methods in the first place and eventually place
some simple convenience methods on Input-/OutputStream and Reader/Writer?
I personally would do both, because I think it would make the usage more fluid
from the usage perspective coming from a InputStream/Reader.
Having said that, this leads me to the question:
Are some other use cases where it would be good to have some sort of a callback
in form of a function reference being called to notify about the amount of
bytes/chars read and written to the target. This to have a hook for cases where
we want to show either some sort of a progress or secondly sum of the total
amount of transferred data.
Also I think there should be may some possibility to stop the copy job on a
„feedback“ event as follows:
while ((read=input.read(buffer)-1 proceedCheck.read(read)) {
output.write(buffer, 0, read);
}
Where proceedCheck would be a functional interface similar to the IntConsumer
or LongConsumer but with a boolean return type signaling to keep on copying or
terminate. (or there is a better approach using other functional interfaces, I
missed so far)
Other opinions / input? I would be glad to help improve the library in this
direction to finally get rid of lots of duplicated code copying data…
Cheers Patrick
Am 13.11.2014 um 20:59 schrieb Alan Bateman alan.bate...@oracle.com:
On 13/11/2014 19:31, Patrick Reinhart wrote:
Hi there,
In the followup of a BOF with Stephen Colebourne with his ideas of small
library changes that may could get in JDK9. As of the fact that in the
codebase of my company there are several locations where we copy from/to IO
stream over and over again using either external libraries or do it by
ourselves, I suggested to have some support for easy coping data between
Input-/OutputStream or Reader/Writer without having to use to external
libraries.
Long overdue. I remember we prototyped methods like this (and much more)
during JDK 7 but didn't do enough at the time to actually get them in. We did
include a bunch of easy to use methods in the Files class at the time,
including copy between input/output streams and files, but we didn't
introduce an IOUtils or such class in java.io.
-Alan
