Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
Hello.
I think an efforts for readability of code came to improve stability of
code.
Please read next patch and give me your opinion.
Best regards.
Knut Anders Hatlen wrote:
TomohitoNakayama [EMAIL PROTECTED] writes:
Hello.
Knut Anders Hatlen (JIRA) wrote:
[...]
if (stream == null) {
// same code as before for null
} else if (stream instanceof LayerBStreamedEXTDTAReaderInputStream) {
// new Layer B code
} else {
// same code as in the old else-clause
}
The main point is that we should have an else clause at the end
which catches all cases, instead of only having if and else if
clauses. I think that it will make the code clearer, since a reader
doesn't have to figure out whether there are cases that don't match
any of the if/else-if clauses.
See ...
I want to state explicitly that stream is instance of
LayerBStreamedEXTDTAReaderInputStream or StandardEXTDTAReaderInputStream.
Now I think what is needed is the last else clause as error processing.
I don't think it is necessary to create a new error message and throw
an exception in this case. My comment was primarily about readability,
not error checking, and if you want to keep the instanceof check for
StandardEXTDTAReaderInputStream, I think it's enough to add a comment
saying that the stream is always one of those types.
--
/*
Tomohito Nakayama
[EMAIL PROTECTED]
[EMAIL PROTECTED]
[EMAIL PROTECTED]
Naka
http://www5.ocn.ne.jp/~tomohito/TopPage.html
*/
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
Hello.
Knut Anders Hatlen (JIRA) wrote:
However, compatibility with older servers seems to have been broken. When I try to stream
a lob (small enough to fit in the client's memory) to a 10.2.1.6 server, I get a protocol
error: java.sql.SQLException: A network protocol error was encountered and the
connection has been terminated: A PROTOCOL Data Stream Syntax Error was detected.
Reason: 0x0. I think we need to check the version of the server before using layer
b streaming. This could be done in NetDatabaseMetaData.computeFeatureSet_().
I see.
I will add one more condition for executing layer B streaming from
client driver.
if (stream == null) {
// same code as before for null
} else if (stream instanceof LayerBStreamedEXTDTAReaderInputStream) {
// new Layer B code
} else {
// same code as in the old else-clause
}
The main point is that we should have an else clause at the end which catches all cases, instead
of only having if and else if clauses. I think that it will make the code clearer,
since a reader doesn't have to figure out whether there are cases that don't match any of the if/else-if
clauses.
See ...
I want to state explicitly that stream is instance of
LayerBStreamedEXTDTAReaderInputStream or StandardEXTDTAReaderInputStream.
Now I think what is needed is the last else clause as error processing.
Best regards.
Knut Anders Hatlen (JIRA) wrote:
[ http://issues.apache.org/jira/browse/DERBY-1471?page=comments#action_12458092 ]
Knut Anders Hatlen commented on DERBY-1471:
---
I have tested the lengthless versions of PreparedStatement.setClob(),
PreparedStatement.setCharacterStream(), PreparedStatement.setAsciiStream(),
PreparedStatement.setBlob(), PreparedStatement.setBinaryStream(),
ResultSet.updateClob(), ResultSet.updateCharacterStream(),
ResultSet.updateAsciiStream(), ResultSet.updateBlob() and
ResultSet.updateBinaryStream(), and they all seem to work without throwing
OutOfMemoryError for large lobs. Great! :)
However, compatibility with older servers seems to have been broken. When I try to stream
a lob (small enough to fit in the client's memory) to a 10.2.1.6 server, I get a protocol
error: java.sql.SQLException: A network protocol error was encountered and the
connection has been terminated: A PROTOCOL Data Stream Syntax Error was detected.
Reason: 0x0. I think we need to check the version of the server before using layer
b streaming. This could be done in NetDatabaseMetaData.computeFeatureSet_().
Thank you for addressing my previous comments. My comment about
readAndSetExtParam() was a bit unclear. What I meant, was that the if-clause
could be written as
if (stream == null) {
// same code as before for null
} else if (stream instanceof LayerBStreamedEXTDTAReaderInputStream) {
// new Layer B code
} else {
// same code as in the old else-clause
}
The main point is that we should have an else clause at the end which catches all cases, instead
of only having if and else if clauses. I think that it will make the code clearer,
since a reader doesn't have to figure out whether there are cases that don't match any of the if/else-if
clauses.
Implement layer B streaming for new methods defined in JDBC4.0
--
Key: DERBY-1471
URL: http://issues.apache.org/jira/browse/DERBY-1471
Project: Derby
Issue Type: New Feature
Components: Network Client
Reporter: Tomohito Nakayama
Assigned To: Tomohito Nakayama
Attachments: DERBY-1471.diff, DERBY-1471.patch, DERBY-1471.stat,
DERBY-1471_2.patch, DERBY-1471_2.stat, DERBY-1471_3.patch, DERBY-1471_3.stat,
DERBY-1471_4.patch, DERBY-1471_4.stat
JDBC 4.0 introduced new methods which take parameters for object to be sent to
sever without length information.
For those methods, Layer B streaming is best way to implement sending object to
server.
This issue is representation of DERBY-1417 in Network Client.
--
/*
Tomohito Nakayama
[EMAIL PROTECTED]
[EMAIL PROTECTED]
[EMAIL PROTECTED]
Naka
http://www5.ocn.ne.jp/~tomohito/TopPage.html
*/
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
TomohitoNakayama [EMAIL PROTECTED] writes:
Hello.
Knut Anders Hatlen (JIRA) wrote:
[...]
if (stream == null) {
// same code as before for null
} else if (stream instanceof LayerBStreamedEXTDTAReaderInputStream) {
// new Layer B code
} else {
// same code as in the old else-clause
}
The main point is that we should have an else clause at the end
which catches all cases, instead of only having if and else if
clauses. I think that it will make the code clearer, since a reader
doesn't have to figure out whether there are cases that don't match
any of the if/else-if clauses.
See ...
I want to state explicitly that stream is instance of
LayerBStreamedEXTDTAReaderInputStream or StandardEXTDTAReaderInputStream.
Now I think what is needed is the last else clause as error processing.
I don't think it is necessary to create a new error message and throw
an exception in this case. My comment was primarily about readability,
not error checking, and if you want to keep the instanceof check for
StandardEXTDTAReaderInputStream, I think it's enough to add a comment
saying that the stream is always one of those types.
--
Knut Anders
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
Thank you for your attention.
I think another patch is needed before committing ...
* BrokeredPreparedStatement now implements length-less setBinaryStream
and setCharacterStream. We could therefore remove them from
BrokeredPreparedStatement40.
* In DRDAConnThread.readAndSetExtParam(), some of the code has been
changed from
if (stream == null) {
...
} else {
...
}
to
if (stream == null) {
...
} else if (stream instanceof StandardEXTDTAReaderInputStream) {
... // basically do the same as in the old else clause
} else if (stream instanceof LayerBStreamedEXTDTAReaderInputStream) {
...
}
I would feel more comfortable if the code in the first else-if
clause were put into an else clause. If the stream is of another
type (shouldn't happen, but someone might extend the code later), it
will be silently ignored.
I see.
I will fix those in next patch.
* In NetStatementRequest, 0x8002 is used as a magic number many
places. Perhaps it could be explained in a comment, or a constant
could be used instead?
This number comes from specification of DRDA.
It told to set the Layer B 2-byte object length field to X'8004'.
I think comment is needed in the code.
* Inner class PublicBufferOutputStream in DRDAConnThread could be
private. Or even better, since ReEncodedInputStream also implements
such a class, it could be implemented as a stand-alone class that
could be shared between DRDAConnThread and ReEncodedInputStream.
I think PublicBufferOutputStream should not be shared between classes,
because accessing internal buffer is a kind of a necessary evil and
should not be used widely ...
Well ... I will make it as private.
* Blob/Clob: The variable willBeLayerBStreamed_ and the method
willBeLayerBStreamed() could be moved to the base class (Lob) to
avoid duplicated code.
I am somewhat repulsed for sharing code around instance variable
However, the value never be changed in the life of the object and
they are equal in semantics
I will try to modify the code and will judge the answer reading the result.
Best regards.
Knut Anders Hatlen (JIRA) wrote:
[ http://issues.apache.org/jira/browse/DERBY-1471?page=comments#action_12456093 ]
Knut Anders Hatlen commented on DERBY-1471:
---
I have looked at patch 3. I don't know enough about layer B streaming
to say whether all details of the implementation are correct, but I
have a couple of comments. All of the comments are about minor issues,
so it would be perfectly OK to commit the patch as it is and address
the comments in a followup patch.
* BrokeredPreparedStatement now implements length-less setBinaryStream
and setCharacterStream. We could therefore remove them from
BrokeredPreparedStatement40.
* In DRDAConnThread.readAndSetExtParam(), some of the code has been
changed from
if (stream == null) {
...
} else {
...
}
to
if (stream == null) {
...
} else if (stream instanceof StandardEXTDTAReaderInputStream) {
... // basically do the same as in the old else clause
} else if (stream instanceof LayerBStreamedEXTDTAReaderInputStream) {
...
}
I would feel more comfortable if the code in the first else-if
clause were put into an else clause. If the stream is of another
type (shouldn't happen, but someone might extend the code later), it
will be silently ignored.
* Inner class PublicBufferOutputStream in DRDAConnThread could be
private. Or even better, since ReEncodedInputStream also implements
such a class, it could be implemented as a stand-alone class that
could be shared between DRDAConnThread and ReEncodedInputStream.
* In NetStatementRequest, 0x8002 is used as a magic number many
places. Perhaps it could be explained in a comment, or a constant
could be used instead?
* Blob/Clob: The variable willBeLayerBStreamed_ and the method
willBeLayerBStreamed() could be moved to the base class (Lob) to
avoid duplicated code.
Implement layer B streaming for new methods defined in JDBC4.0
--
Key: DERBY-1471
URL: http://issues.apache.org/jira/browse/DERBY-1471
Project: Derby
Issue Type: New Feature
Components: Network Client
Reporter: Tomohito Nakayama
Assigned To: Tomohito Nakayama
Attachments: DERBY-1471.diff, DERBY-1471.patch, DERBY-1471.stat,
DERBY-1471_2.patch, DERBY-1471_2.stat, DERBY-1471_3.patch, DERBY-1471_3.stat
JDBC 4.0 introduced new methods which take parameters for object to be sent to
sever without length information.
For those methods, Layer B streaming is best way to implement sending object to
server.
This issue is representation of DERBY-1417 in Network Client.
--
/*
Tomohito Nakayama
[EMAIL PROTECTED]
[EMAIL PROTECTED]
[EMAIL PROTECTED]
Naka
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
Ooops.
* In NetStatementRequest, 0x8002 is used as a magic number many
places. Perhaps it could be explained in a comment, or a constant
could be used instead?
This number comes from specification of DRDA.
It told to set the Layer B 2-byte object length field to X'8004'.
I think comment is needed in the code.
We have both 0x8002 and 0x8004 in the patch.
Either of them need comment
Best regards.
TomohitoNakayama wrote:
Thank you for your attention.
I think another patch is needed before committing ...
* BrokeredPreparedStatement now implements length-less setBinaryStream
and setCharacterStream. We could therefore remove them from
BrokeredPreparedStatement40.
* In DRDAConnThread.readAndSetExtParam(), some of the code has been
changed from
if (stream == null) {
...
} else {
...
}
to
if (stream == null) {
...
} else if (stream instanceof StandardEXTDTAReaderInputStream) {
... // basically do the same as in the old else clause
} else if (stream instanceof LayerBStreamedEXTDTAReaderInputStream) {
...
}
I would feel more comfortable if the code in the first else-if
clause were put into an else clause. If the stream is of another
type (shouldn't happen, but someone might extend the code later), it
will be silently ignored.
I see.
I will fix those in next patch.
* In NetStatementRequest, 0x8002 is used as a magic number many
places. Perhaps it could be explained in a comment, or a constant
could be used instead?
This number comes from specification of DRDA.
It told to set the Layer B 2-byte object length field to X'8004'.
I think comment is needed in the code.
* Inner class PublicBufferOutputStream in DRDAConnThread could be
private. Or even better, since ReEncodedInputStream also implements
such a class, it could be implemented as a stand-alone class that
could be shared between DRDAConnThread and ReEncodedInputStream.
I think PublicBufferOutputStream should not be shared between classes,
because accessing internal buffer is a kind of a necessary evil and
should not be used widely ...
Well ... I will make it as private.
* Blob/Clob: The variable willBeLayerBStreamed_ and the method
willBeLayerBStreamed() could be moved to the base class (Lob) to
avoid duplicated code.
I am somewhat repulsed for sharing code around instance variable
However, the value never be changed in the life of the object and
they are equal in semantics
I will try to modify the code and will judge the answer reading the
result.
Best regards.
Knut Anders Hatlen (JIRA) wrote:
[
http://issues.apache.org/jira/browse/DERBY-1471?page=comments#action_12456093
]Knut Anders Hatlen commented on DERBY-1471:
---
I have looked at patch 3. I don't know enough about layer B streaming
to say whether all details of the implementation are correct, but I
have a couple of comments. All of the comments are about minor issues,
so it would be perfectly OK to commit the patch as it is and address
the comments in a followup patch.
* BrokeredPreparedStatement now implements length-less setBinaryStream
and setCharacterStream. We could therefore remove them from
BrokeredPreparedStatement40.
* In DRDAConnThread.readAndSetExtParam(), some of the code has been
changed from
if (stream == null) {
...
} else {
...
}
to
if (stream == null) {
...
} else if (stream instanceof StandardEXTDTAReaderInputStream) {
... // basically do the same as in the old else clause
} else if (stream instanceof LayerBStreamedEXTDTAReaderInputStream) {
...
}
I would feel more comfortable if the code in the first else-if
clause were put into an else clause. If the stream is of another
type (shouldn't happen, but someone might extend the code later), it
will be silently ignored.
* Inner class PublicBufferOutputStream in DRDAConnThread could be
private. Or even better, since ReEncodedInputStream also implements
such a class, it could be implemented as a stand-alone class that
could be shared between DRDAConnThread and ReEncodedInputStream.
* In NetStatementRequest, 0x8002 is used as a magic number many
places. Perhaps it could be explained in a comment, or a constant
could be used instead?
* Blob/Clob: The variable willBeLayerBStreamed_ and the method
willBeLayerBStreamed() could be moved to the base class (Lob) to
avoid duplicated code.
Implement layer B streaming for new methods defined in JDBC4.0
--
Key: DERBY-1471
URL: http://issues.apache.org/jira/browse/DERBY-1471
Project: Derby
Issue Type: New Feature
Components: Network Client
Reporter: Tomohito Nakayama
Assigned To: Tomohito Nakayama
Attachments: DERBY-1471.diff, DERBY-1471.patch,
DERBY-1471.stat, DERBY-1471_2.patch,
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
TomohitoNakayama [EMAIL PROTECTED] writes: Thank you for your attention. I think another patch is needed before committing ... Thank you for taking my comments into consideration. * Blob/Clob: The variable willBeLayerBStreamed_ and the method willBeLayerBStreamed() could be moved to the base class (Lob) to avoid duplicated code. I am somewhat repulsed for sharing code around instance variable However, the value never be changed in the life of the object and they are equal in semantics In that case I think it would be good to declare the variable as final as well. Seems like it is initialized in the constructor and is newer changed, so that should work. -- Knut Anders
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
Hello. Thank you. I think the problem was solved using EmbedPreparedStatement :) Best regards. Knut Anders Hatlen wrote: TomohitoNakayama [EMAIL PROTECTED] writes: Hello. Reading the code ( BaseMonitor.java, modules.properties, InternalDriver.java and so on ), it seems that possible class of engine is different between running environment ... Yes, but the implementation of PreparedStatement.setBinaryStream(int, java.io.InputStream) is in EmbedPreparedStatement (not in EmbedPreparedStatement40) and therefore available to the network server regardless of environment. I think all that is needed is to put setBinaryStream(int, java.io.InputStream) into the EnginePreparedStatement interface, and it should be possible to invoke it from the network server. Then, I think it is needed to consider when the class is not expected class. That's what the Engine* interfaces are there for. To see an example, you could look at how the EngineStatement interface is used in DRDAStatement to make the JDBC 3.0 methods getResultSetHoldability() and getMoreResults() available for a network server which is possibly running JDBC 2.0 only. -- /* Tomohito Nakayama [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] Naka http://www5.ocn.ne.jp/~tomohito/TopPage.html */
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
TomohitoNakayama [EMAIL PROTECTED] writes: Hello. Thank you for your comment :) I think you understood what the issue is. I understand your idea as using module of Engine where JDBC4.0 feature is implemented directly rather than via JDBC interface from NetworkServer. I take the idea in consideration. The network server already does this for some JDBC 3.0 methods that it needs even when the JVM only supports JDBC 2.0. The methods are defined in interfaces starting with Engine (for instance, EngineStatement and EnginePreparedStatement). When the network server needs to call one of those methods, it casts the Statement object to an EngineStatement. -- Knut Anders
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
Hello. Reading the code ( BaseMonitor.java, modules.properties, InternalDriver.java and so on ), it seems that possible class of engine is different between running environment ... Then, I think it is needed to consider when the class is not expected class. Best regards. Knut Anders Hatlen wrote: TomohitoNakayama [EMAIL PROTECTED] writes: Hello. Thank you for your comment :) I think you understood what the issue is. I understand your idea as using module of Engine where JDBC4.0 feature is implemented directly rather than via JDBC interface from NetworkServer. I take the idea in consideration. The network server already does this for some JDBC 3.0 methods that it needs even when the JVM only supports JDBC 2.0. The methods are defined in interfaces starting with Engine (for instance, EngineStatement and EnginePreparedStatement). When the network server needs to call one of those methods, it casts the Statement object to an EngineStatement. -- /* Tomohito Nakayama [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] Naka http://www5.ocn.ne.jp/~tomohito/TopPage.html */
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
TomohitoNakayama [EMAIL PROTECTED] writes: Hello. Reading the code ( BaseMonitor.java, modules.properties, InternalDriver.java and so on ), it seems that possible class of engine is different between running environment ... Yes, but the implementation of PreparedStatement.setBinaryStream(int, java.io.InputStream) is in EmbedPreparedStatement (not in EmbedPreparedStatement40) and therefore available to the network server regardless of environment. I think all that is needed is to put setBinaryStream(int, java.io.InputStream) into the EnginePreparedStatement interface, and it should be possible to invoke it from the network server. Then, I think it is needed to consider when the class is not expected class. That's what the Engine* interfaces are there for. To see an example, you could look at how the EngineStatement interface is used in DRDAStatement to make the JDBC 3.0 methods getResultSetHoldability() and getMoreResults() available for a network server which is possibly running JDBC 2.0 only. -- Knut Anders
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
TomohitoNakayama wrote: Hello. Thank you for your kindness :) What is needed is calling method which was introduced in JDBC 4.0 from NetworkServer to utilize JDBC 4.0 feature in Engine. Concretely,the method is PreparedStatement#setBinaryStream(int, java.io.InputStream). http://download.java.net/jdk6/docs/api/java/sql/PreparedStatement.html#setBinaryStream(int,%20java.io.InputStream) What is important is that JDBC 4.0 is optional and implementation to utilize the feature of it should be appropriately placed in code of NetworkServer in order to keep compatibility for previous spec in NetworkServer. If I understand you correctly the issue is: - When using Layer B streaming, you want the Network Server to be able to use the new JDBC 4 methods for lengthless streaming. - The problem is that the JDBC 4 methods is not directly available to the NetworkServer if using older run-time libraries (e.g., Java SE 5). Since the implementation of the JDBC 4 methods will be part of derby.jar regardless of which JVM you are running on, I would guess it is just a question of how to make them available to the network server. -- Øystein
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
Hello. Thank you for your comment :) I think you understood what the issue is. I understand your idea as using module of Engine where JDBC4.0 feature is implemented directly rather than via JDBC interface from NetworkServer. I take the idea in consideration. Best regards. Oystein Grovlen - Sun Norway wrote: TomohitoNakayama wrote: Hello. Thank you for your kindness :) What is needed is calling method which was introduced in JDBC 4.0 from NetworkServer to utilize JDBC 4.0 feature in Engine. Concretely,the method is PreparedStatement#setBinaryStream(int, java.io.InputStream). http://download.java.net/jdk6/docs/api/java/sql/PreparedStatement.html#setBinaryStream(int,%20java.io.InputStream) What is important is that JDBC 4.0 is optional and implementation to utilize the feature of it should be appropriately placed in code of NetworkServer in order to keep compatibility for previous spec in NetworkServer. If I understand you correctly the issue is: - When using Layer B streaming, you want the Network Server to be able to use the new JDBC 4 methods for lengthless streaming. - The problem is that the JDBC 4 methods is not directly available to the NetworkServer if using older run-time libraries (e.g., Java SE 5). Since the implementation of the JDBC 4 methods will be part of derby.jar regardless of which JVM you are running on, I would guess it is just a question of how to make them available to the network server. -- /* Tomohito Nakayama [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] Naka http://www5.ocn.ne.jp/~tomohito/TopPage.html */
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
Tomohito Nakayama (JIRA) wrote: [ http://issues.apache.org/jira/browse/DERBY-1471?page=comments#action_12448487 ] Tomohito Nakayama commented on DERBY-1471: -- I found that it is needed to make server side implementation compatible with both of JDBC 4.0 and before that ... Tomohito, Could elaborate on a bit this? In what way is this needed with respect to JDBC compatibility? -- Øystein
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
Hello. Thank you for your kindness :) What is needed is calling method which was introduced in JDBC 4.0 from NetworkServer to utilize JDBC 4.0 feature in Engine. Concretely,the method is PreparedStatement#setBinaryStream(int, java.io.InputStream). http://download.java.net/jdk6/docs/api/java/sql/PreparedStatement.html#setBinaryStream(int,%20java.io.InputStream) What is important is that JDBC 4.0 is optional and implementation to utilize the feature of it should be appropriately placed in code of NetworkServer in order to keep compatibility for previous spec in NetworkServer. Do you know the code in Derby which is in similar situation ? Best regards. Øystein Grøvlen wrote: Tomohito Nakayama (JIRA) wrote: [ http://issues.apache.org/jira/browse/DERBY-1471?page=comments#action_12448487 ] Tomohito Nakayama commented on DERBY-1471: -- I found that it is needed to make server side implementation compatible with both of JDBC 4.0 and before that ... Tomohito, Could elaborate on a bit this? In what way is this needed with respect to JDBC compatibility? -- Øystein -- /* Tomohito Nakayama [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] Naka http://www5.ocn.ne.jp/~tomohito/TopPage.html */
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
Hello. IMHO, I think java.lang.IndexOutOfBoundsException is better than java.lang.OutOfMemoryError for this situation ... Error is too much for ... Best regards. Kristian Waagan wrote: John Embretsen wrote: Bryan Pendleton wrote: Kristian Waagan commented on DERBY-1471: The approach is to exhaust the application stream and copy it into memory to determine the length. If the data is too big to fit in memory, the client will fail with an out-of-memory exception. That seems fine to me. This is already the case with the current API and implementation, I believe, so we aren't making anything worse. Hi Kristian and Bryan, I am assuming that there is no out-of-memory specific exception implemented in Derby, and that you by out-of-memory exception mean java.lang.OutOfMemoryError. Please give me a hint if my assumption is wrong... Hello John, There is no java.lang.OutOfMemoryException, but there is an out-of-memory exception - the unchecked exception class OutOfMemoryError, which is also referred to as an error class! I'll follow your advice and use the term out-of-memory error, which is much more precise :) I believe it is important to distinguish between java Errors and Exceptions, since an Error indicates serious problems that a reasonable application should not try to catch [1]. I agree. According to the The Java Language Specification, Third Edition, the reason why errors are not checked exceptions is ... they can occur at many points in the program and recovery from them is difficult or impossible. A program declaring such exceptions would be cluttered, pointlessly.*. Cheers, -- /* Tomohito Nakayama [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] Naka http://www5.ocn.ne.jp/~tomohito/TopPage.html */
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
TomohitoNakayama wrote:
Hello.
IMHO, I think java.lang.IndexOutOfBoundsException is better than
java.lang.OutOfMemoryError for this situation ...
Error is too much for ...
Hi Tomohito,
Not sure I understand what you are referring to in your comment. I'll
describe the situation I think you are talking about. Please correct me
if I'm mistaken.
The OutOfMemoryError I have been talking about when reading LOBs with
unknown length on the client side, is an exception thrown by the JVM
when the LOB data cannot fit into the heap space. I did not plan to
throw this error myself.
Further, if there is enough memory on the client to hold more than 2 GB
of data, and the user passes in a stream with more than 2 GB of data
available, I thought a SQLException would be in place:
22004.S.4=The length ({0}) exceeds the maximum length for the data type
({1}).
Not quite sure what to put into {{0}}, but maybe Integer._MAX_VALUE?
Are we on the same track here, or are you referring to another situation?
Regards,
--
Kristian
Best regards.
Kristian Waagan wrote:
John Embretsen wrote:
Bryan Pendleton wrote:
Kristian Waagan commented on DERBY-1471:
The approach is to exhaust the application stream and copy it into
memory to determine the length. If the data is too big to fit in
memory,
the client will fail with an out-of-memory exception.
That seems fine to me. This is already the case with the current
API and implementation, I believe, so we aren't making anything worse.
Hi Kristian and Bryan,
I am assuming that there is no out-of-memory specific exception
implemented in Derby, and that you by out-of-memory exception mean
java.lang.OutOfMemoryError. Please give me a hint if my assumption is
wrong...
Hello John,
There is no java.lang.OutOfMemoryException, but there is an
out-of-memory exception - the unchecked exception class
OutOfMemoryError, which is also referred to as an error class!
I'll follow your advice and use the term out-of-memory error, which is
much more precise :)
I believe it is important to distinguish between java Errors and
Exceptions, since an Error indicates serious problems that a reasonable
application should not try to catch [1].
I agree.
According to the The Java Language Specification, Third Edition, the
reason why errors are not checked exceptions is ... they can occur at
many points in the program and recovery from them is difficult or
impossible. A program declaring such exceptions would be cluttered,
pointlessly.*.
Cheers,
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
Hello Kristian.
The OutOfMemoryError I have been talking about when reading LOBs with
unknown length on the client side, is an exception thrown by the JVM
when the LOB data cannot fit into the heap space. I did not plan to
throw this error myself.
Oh, I mistaken your thought.
I thought that you limit max length of LOB in the program and throw
exception intentionally.
Please forget my previous mistaken comment.
Best regards.
Kristian Waagan wrote:
TomohitoNakayama wrote:
Hello.
IMHO, I think java.lang.IndexOutOfBoundsException is better than
java.lang.OutOfMemoryError for this situation ...
Error is too much for ...
Hi Tomohito,
Not sure I understand what you are referring to in your comment. I'll
describe the situation I think you are talking about. Please correct
me if I'm mistaken.
The OutOfMemoryError I have been talking about when reading LOBs with
unknown length on the client side, is an exception thrown by the JVM
when the LOB data cannot fit into the heap space. I did not plan to
throw this error myself.
Further, if there is enough memory on the client to hold more than 2
GB of data, and the user passes in a stream with more than 2 GB of
data available, I thought a SQLException would be in place:
22004.S.4=The length ({0}) exceeds the maximum length for the data
type ({1}).
Not quite sure what to put into {{0}}, but maybe Integer._MAX_VALUE?
Are we on the same track here, or are you referring to another situation?
Regards,
--
/*
Tomohito Nakayama
[EMAIL PROTECTED]
[EMAIL PROTECTED]
[EMAIL PROTECTED]
Naka
http://www5.ocn.ne.jp/~tomohito/TopPage.html
*/
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
Bryan Pendleton wrote: Kristian Waagan commented on DERBY-1471: The approach is to exhaust the application stream and copy it into memory to determine the length. If the data is too big to fit in memory, the client will fail with an out-of-memory exception. That seems fine to me. This is already the case with the current API and implementation, I believe, so we aren't making anything worse. Hi Kristian and Bryan, I am assuming that there is no out-of-memory specific exception implemented in Derby, and that you by out-of-memory exception mean java.lang.OutOfMemoryError. Please give me a hint if my assumption is wrong... I believe it is important to distinguish between java Errors and Exceptions, since an Error indicates serious problems that a reasonable application should not try to catch [1]. -- John [1] http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Error.html
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
John Embretsen wrote: Bryan Pendleton wrote: Kristian Waagan commented on DERBY-1471: The approach is to exhaust the application stream and copy it into memory to determine the length. If the data is too big to fit in memory, the client will fail with an out-of-memory exception. That seems fine to me. This is already the case with the current API and implementation, I believe, so we aren't making anything worse. Hi Kristian and Bryan, I am assuming that there is no out-of-memory specific exception implemented in Derby, and that you by out-of-memory exception mean java.lang.OutOfMemoryError. Please give me a hint if my assumption is wrong... Hello John, There is no java.lang.OutOfMemoryException, but there is an out-of-memory exception - the unchecked exception class OutOfMemoryError, which is also referred to as an error class! I'll follow your advice and use the term out-of-memory error, which is much more precise :) I believe it is important to distinguish between java Errors and Exceptions, since an Error indicates serious problems that a reasonable application should not try to catch [1]. I agree. According to the The Java Language Specification, Third Edition, the reason why errors are not checked exceptions is ... they can occur at many points in the program and recovery from them is difficult or impossible. A program declaring such exceptions would be cluttered, pointlessly.*. Cheers, -- Kristian * 11.2.4, page 301
Where does the Client Driver expand object into memory before sending it ? (Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0)
Hello. Concerning about use of memory at driver, I briefly surveyed client driver program to find where *whole of* object was expanded into memory before sending it, from code searched with keyword writeScalarStream as starting point of survey. The result was as next. When org.apache.derby.client.am.Blob object was constructed with parameter named binaryString typed as byte[], whole of object is expanded into memory before the call of constructor. But this seems not to be the scenario when InputStream was passed to driver, reading around setBinaryStreamX method in the org.apache.derby.client.am.PreparedStatement class. When org.apache.derby.client.am.Blob object was constructed with parameter named binaryStream typed as InputStream, it seems that whole of object was *not* expanded into continuous memory. //Here, continuous means that it is true that the object is expanded into *upper limited* memory array //segmentedly at writeScalarStream method in org.apache.derby.client.net.Request. I'm not sure . My question is what was the stackTrace found in DERBY-550 ... Does anyone have any information ? I searched received mails in my mailbox , but could not found the information of stackTrace Now I found it not clear that the driver is writting all the file into memory (RAM) before sending ... Deeper survey may be needed yet ... Best regards. Bryan Pendleton wrote: Kristian Waagan commented on DERBY-1471: The approach is to exhaust the application stream and copy it into memory to determine the length. If the data is too big to fit in memory, the client will fail with an out-of-memory exception. That seems fine to me. This is already the case with the current API and implementation, I believe, so we aren't making anything worse. In practice, do application programs end up having *two* copies of the BLOB/CLOB in client-side memory at the same time, one copy inside the Derby code and one copy in the application code? Or is there actually only a single copy of the BLOB/CLOB in memory? I'm asking only because it seems like we might want to be clear about this in our documentation. thanks, bryan -- /* Tomohito Nakayama [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] Naka http://www5.ocn.ne.jp/~tomohito/TopPage.html */
Re: Where does the Client Driver expand object into memory before sending it ? (Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0)
Hello. I think this should be told in DERBY-1472 more than DERBY-1471 .. Best regards. TomohitoNakayama wrote: Hello. Concerning about use of memory at driver, I briefly surveyed client driver program to find where *whole of* object was expanded into memory before sending it, from code searched with keyword writeScalarStream as starting point of survey. The result was as next. When org.apache.derby.client.am.Blob object was constructed with parameter named binaryString typed as byte[], whole of object is expanded into memory before the call of constructor. But this seems not to be the scenario when InputStream was passed to driver, reading around setBinaryStreamX method in the org.apache.derby.client.am.PreparedStatement class. When org.apache.derby.client.am.Blob object was constructed with parameter named binaryStream typed as InputStream, it seems that whole of object was *not* expanded into continuous memory. //Here, continuous means that it is true that the object is expanded into *upper limited* memory array //segmentedly at writeScalarStream method in org.apache.derby.client.net.Request. I'm not sure . My question is what was the stackTrace found in DERBY-550 ... Does anyone have any information ? I searched received mails in my mailbox , but could not found the information of stackTrace Now I found it not clear that the driver is writting all the file into memory (RAM) before sending ... Deeper survey may be needed yet ... Best regards. Bryan Pendleton wrote: Kristian Waagan commented on DERBY-1471: The approach is to exhaust the application stream and copy it into memory to determine the length. If the data is too big to fit in memory, the client will fail with an out-of-memory exception. That seems fine to me. This is already the case with the current API and implementation, I believe, so we aren't making anything worse. In practice, do application programs end up having *two* copies of the BLOB/CLOB in client-side memory at the same time, one copy inside the Derby code and one copy in the application code? Or is there actually only a single copy of the BLOB/CLOB in memory? I'm asking only because it seems like we might want to be clear about this in our documentation. thanks, bryan -- /* Tomohito Nakayama [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] Naka http://www5.ocn.ne.jp/~tomohito/TopPage.html */
Re: [jira] Commented: (DERBY-1471) Implement layer B streaming for new methods defined in JDBC4.0
Kristian Waagan commented on DERBY-1471: The approach is to exhaust the application stream and copy it into memory to determine the length. If the data is too big to fit in memory, the client will fail with an out-of-memory exception. That seems fine to me. This is already the case with the current API and implementation, I believe, so we aren't making anything worse. In practice, do application programs end up having *two* copies of the BLOB/CLOB in client-side memory at the same time, one copy inside the Derby code and one copy in the application code? Or is there actually only a single copy of the BLOB/CLOB in memory? I'm asking only because it seems like we might want to be clear about this in our documentation. thanks, bryan
