RE: Re: JDBCClient
Hi. Just some thoughts around JDBCClient. I have to say that I was a bit surprised when I saw the implementation of JDBCClient as it is. What I had envisioned was an implementations that used a basic mapping from methods to SQL keywords, eg SELECT, INSERT, UPDATE and DELETE (CRUD, remember?), with the rest of the information in the input representation. I think that returning the entire result set in XML will limit the size of the result returned from the database, although it probably would suffice for most database queries used in real-life applications. As network access is horrendously expensive compared to doing something locally (memory/disk), it is usually better to pin down the result as much as possible in the database before returning it to the application. At least that was what my database lecturer told me back in the happy student days. Regards, Kyrre --- Jerome Louvel [EMAIL PROTECTED] wrote: Hi John, I have some concerns about the proposed approach... (A) What about results that are quite large? I.e. that take up a lot of memory or won't fit into memory? One major use case I see for this JDBC connector is to fetch a reasonable amount of data in order to assemble a representation for a client of a Restlet application. This representation might be in XML, HTML, PDF, RTF, ... or any other format that can be generated from an XML source. As for memory, we will rely on streams. When the representation is fully consumed, the stream or channel should be closed, which will transparently release the JDBC connection. The result XML format will use the WebRowSet implementation for now: http://java.sun.com/j2se/1.5.0/docs/api/javax/sql/rowset/WebRowSet.html A useful piece, will be the addition of an XsltFilter later in 1.1: http://restlet.tigris.org/issues/show_bug.cgi?id=112 (B) What about performance? Doing that extra transformation is costly. There is always the option to use JDBC APIs directly. If you need to built business object from this JDBC data, obviously using a solution like EJB or JPA/Hibernate seems even better. (C) What about those of use who really don't want to have to bother with yet still more XML in the system? If people want to keep the current approach (XML request and JdbcResult response) we will preserve it and provide an option to switch between the two types of results (XML or JdbcResult wrapper). The main reason of this refactoring is that keeping an open connection does not really comply with a REST approach. Yes and no. The purpose, IMHO, is for the service to bridge from a purely REST to the messiness of things like databases. I think we all agree that this is a bridge more than a pure connector (JDBC doesn't define a standard protocol anyway). Hopefully the approach proposed above will coverage most use cases for this connector. Another road we could explore is the SQL/XML standardization efforts that allow an RDBMS to directly return results as an XML stream, without any intermediary transformation. See: http://restlet.tigris.org/issues/show_bug.cgi?id=7 Also, I'd like to explore the usage of XQuery to access databases in the future: http://restlet.tigris.org/issues/show_bug.cgi?id=8 Finally, we should also investigate the idea of a DatabaseHandler/Connector that will attempt to map as well as possible table rows and columns to a set of resources/representations, using a flexible URI namespace. It would support GET/PUT/DELETE/POST. See initial thoughts here: http://restlet.tigris.org/issues/show_bug.cgi?id=6 Best regards, Jerome Kyrre Kristiansen ___ Try the all-new Yahoo! Mail. The New Version is radically easier to use The Wall Street Journal http://uk.docs.yahoo.com/nowyoucan.html
RE: Re: JDBCClient
Hi Kyrre, Thanks for sharing your thoughts: I fully agree with you. We need a higher level of mapping between JDBC database and Restlet Resources. The JDBC connector is a first step at a lower-level. In 1.1 we will focus on providing this higher-level of mapping where people can manipulation table rows/columns/cells directly as resources with URIs and using GET/POST/PUT/DELETE methods (internally mapped to corresponding SQL verbs). This will be a major effort in order to get this mapping right and will bring us closer to other frameworks like Rails which introduced a similar feature in their 1.2 RC1 release: http://weblog.rubyonrails.com/2006/11/23/rails-1-2-release-candidate-1 I've also just added an XSLT Filter (called org.restlet.Transformer) that will facilitate the task of transforming the JDBC XML resultsets (or any XML document) into a representation (XML, HTML, etc.) suitable for the clients. Stay tuned! Jerome -Message d'origine- De : Kyrre Kristiansen [mailto:[EMAIL PROTECTED] Envoyé : vendredi 24 novembre 2006 10:24 À : discuss@restlet.tigris.org Objet : RE: Re: JDBCClient Hi. Just some thoughts around JDBCClient. I have to say that I was a bit surprised when I saw the implementation of JDBCClient as it is. What I had envisioned was an implementations that used a basic mapping from methods to SQL keywords, eg SELECT, INSERT, UPDATE and DELETE (CRUD, remember?), with the rest of the information in the input representation. I think that returning the entire result set in XML will limit the size of the result returned from the database, although it probably would suffice for most database queries used in real-life applications. As network access is horrendously expensive compared to doing something locally (memory/disk), it is usually better to pin down the result as much as possible in the database before returning it to the application. At least that was what my database lecturer told me back in the happy student days. Regards, Kyrre --- Jerome Louvel [EMAIL PROTECTED] wrote: Hi John, I have some concerns about the proposed approach... (A) What about results that are quite large? I.e. that take up a lot of memory or won't fit into memory? One major use case I see for this JDBC connector is to fetch a reasonable amount of data in order to assemble a representation for a client of a Restlet application. This representation might be in XML, HTML, PDF, RTF, ... or any other format that can be generated from an XML source. As for memory, we will rely on streams. When the representation is fully consumed, the stream or channel should be closed, which will transparently release the JDBC connection. The result XML format will use the WebRowSet implementation for now: http://java.sun.com/j2se/1.5.0/docs/api/javax/sql/rowset/WebRo wSet.html A useful piece, will be the addition of an XsltFilter later in 1.1: http://restlet.tigris.org/issues/show_bug.cgi?id=112 (B) What about performance? Doing that extra transformation is costly. There is always the option to use JDBC APIs directly. If you need to built business object from this JDBC data, obviously using a solution like EJB or JPA/Hibernate seems even better. (C) What about those of use who really don't want to have to bother with yet still more XML in the system? If people want to keep the current approach (XML request and JdbcResult response) we will preserve it and provide an option to switch between the two types of results (XML or JdbcResult wrapper). The main reason of this refactoring is that keeping an open connection does not really comply with a REST approach. Yes and no. The purpose, IMHO, is for the service to bridge from a purely REST to the messiness of things like databases. I think we all agree that this is a bridge more than a pure connector (JDBC doesn't define a standard protocol anyway). Hopefully the approach proposed above will coverage most use cases for this connector. Another road we could explore is the SQL/XML standardization efforts that allow an RDBMS to directly return results as an XML stream, without any intermediary transformation. See: http://restlet.tigris.org/issues/show_bug.cgi?id=7 Also, I'd like to explore the usage of XQuery to access databases in the future: http://restlet.tigris.org/issues/show_bug.cgi?id=8 Finally, we should also investigate the idea of a DatabaseHandler/Connector that will attempt to map as well as possible table rows and columns to a set of resources/representations, using a flexible URI namespace. It would support GET/PUT/DELETE/POST. See initial thoughts here: http://restlet.tigris.org/issues/show_bug.cgi?id=6 Best regards, Jerome Kyrre
RE: Re: JDBCClient
Jerome, I saw that you had checked in a fix in SVN for the issue I reported. The fix does not fix my problem, however. I believe that the finally-block should be removed completely, and the connection closed elsewhere. It seems to me that the pooling mechanism should keep a lifetime of the connection (eg number of times used), and close the connection and remove from the pool after a certain amount of time. This raises the question on what to do in the case when connections aren't pooled, however. I'm quite busy with work at the moment, but I'll look into setting up my environment for development against the restlet source, so I can submit a fix rather than just ranting on the mailing list :-) BTW, I'm using the JavaDB (derby) as the database to test for, in case that makes any difference. Regards, Kyrre --- Jerome Louvel [EMAIL PROTECTED] wrote: Hi Kyrre, Thanks for sharing your thoughts: I fully agree with you. We need a higher level of mapping between JDBC database and Restlet Resources. The JDBC connector is a first step at a lower-level. In 1.1 we will focus on providing this higher-level of mapping where people can manipulation table rows/columns/cells directly as resources with URIs and using GET/POST/PUT/DELETE methods (internally mapped to corresponding SQL verbs). This will be a major effort in order to get this mapping right and will bring us closer to other frameworks like Rails which introduced a similar feature in their 1.2 RC1 release: http://weblog.rubyonrails.com/2006/11/23/rails-1-2-release-candidate-1 I've also just added an XSLT Filter (called org.restlet.Transformer) that will facilitate the task of transforming the JDBC XML resultsets (or any XML document) into a representation (XML, HTML, etc.) suitable for the clients. Stay tuned! Jerome -Message d'origine- De : Kyrre Kristiansen [mailto:[EMAIL PROTECTED] Envoyé : vendredi 24 novembre 2006 10:24 À : discuss@restlet.tigris.org Objet : RE: Re: JDBCClient Hi. Just some thoughts around JDBCClient. I have to say that I was a bit surprised when I saw the implementation of JDBCClient as it is. What I had envisioned was an implementations that used a basic mapping from methods to SQL keywords, eg SELECT, INSERT, UPDATE and DELETE (CRUD, remember?), with the rest of the information in the input representation. I think that returning the entire result set in XML will limit the size of the result returned from the database, although it probably would suffice for most database queries used in real-life applications. As network access is horrendously expensive compared to doing something locally (memory/disk), it is usually better to pin down the result as much as possible in the database before returning it to the application. At least that was what my database lecturer told me back in the happy student days. Regards, Kyrre --- Jerome Louvel [EMAIL PROTECTED] wrote: Hi John, I have some concerns about the proposed approach... (A) What about results that are quite large? I.e. that take up a lot of memory or won't fit into memory? One major use case I see for this JDBC connector is to fetch a reasonable amount of data in order to assemble a representation for a client of a Restlet application. This representation might be in XML, HTML, PDF, RTF, ... or any other format that can be generated from an XML source. As for memory, we will rely on streams. When the representation is fully consumed, the stream or channel should be closed, which will transparently release the JDBC connection. The result XML format will use the WebRowSet implementation for now: http://java.sun.com/j2se/1.5.0/docs/api/javax/sql/rowset/WebRo wSet.html A useful piece, will be the addition of an XsltFilter later in 1.1: http://restlet.tigris.org/issues/show_bug.cgi?id=112 (B) What about performance? Doing that extra transformation is costly. There is always the option to use JDBC APIs directly. If you need to built business object from this JDBC data, obviously using a solution like EJB or JPA/Hibernate seems even better. (C) What about those of use who really don't want to have to bother with yet still more XML in the system? If people want to keep the current approach (XML request and JdbcResult response) we will preserve it and provide an option to switch between the two types of results (XML or JdbcResult wrapper). The main reason of this refactoring is that keeping an open connection does not really comply with a REST approach. Yes and no. The purpose, IMHO, is for the service to bridge from a purely REST to the messiness of things like databases. I think we all agree that this is a
RE: RE: POSTing dynamic Representations
Jerome,
I downloaded the latest from Subversion after seeing that you made the fix and
it all works.
Thanks for the quick response!
Mitch
-Original Message-
From: Jerome Louvel [mailto:[EMAIL PROTECTED]
Sent: Friday, November 24, 2006 1:50 AM
To: discuss@restlet.tigris.org
Subject: RE: RE: POSTing dynamic Representations
Hi Mitch,
My turn to have a Aha! moment ;-) It's a rather major bug that you found here.
I will fix it today and release a new snapshot.
Best regards,
Jerome
-Message d'origine-
De : Mitch Stewart [mailto:[EMAIL PROTECTED] Envoyé : vendredi
24 novembre 2006 03:44 À : discuss@restlet.tigris.org Objet : RE: RE:
POSTing dynamic Representations
Jerome,
Thank you for the response...I thought I had an Aha! moment there. :)
I've changed my Representation to set the size to
Representation.UNKNOWN_SIZE, however, I still think I am having a
problem. I think the problem lies in the
Message.isEntityAvailable() method, shown here:
public boolean isEntityAvailable()
{
return (getEntity() != null) (getEntity().getSize() 0)
getEntity().isAvailable();
}
The entity is my Representation class which returns a -1 size, however
it is still checking to see if the size of the entity is greater than
0, not != to 0. This then returns false, and the calling method,
which is
HttpClientCall.sendRequest() still determines that the Representation
entity is null and doesn't send the data.
Thanks,
Mitch
-Original Message-
From: Jerome Louvel [mailto:[EMAIL PROTECTED]
Sent: Thu 11/23/2006 3:32 AM
To: discuss@restlet.tigris.org
Subject: RE: POSTing dynamic Representations
Hi Mitch,
You got nearly everything right. The only thing that you missed is
that when the size of a Representation is unknown, you need to set it
to -1 (using the Representation.UNKNOWN_SIZE constant). Setting the
size to 0 means that there is no content in the representation which
is not true in your case.
Best regards,
Jerome
-Message d'origine-
De : Mitch Stewart [mailto:[EMAIL PROTECTED] Envoyé :
mercredi 22 novembre 2006 22:19 À : discuss@restlet.tigris.org Objet
: POSTing dynamic Representations
I'm using the Restlet HTTP client implementation to post a dynamic
Representation to my Restlet server implementation, however it
doesn't look like my data is getting written to the HTTP stream. I
might be missing something critical, but I've been able to
understand the API so far. :) My Representation class resembles the
ObjectRepresentation that already exists, except instead of
serializing a Java object to an ObjectOutputStream, it serializes a
Java object to an XML stream. But, testing with the
ObjectRepresentation produces the same result.
Here's the basics of what I am trying to do:
Client client = new Client(Protocol.HTTP);
Response response =
client.post(http://somehost.com/someurl;, new
ObjectRepresentation(TestData));
The response variable is not filled with any status or response.
Drilling down through the code I think I found the culprit:
in com.noelios.restlet.http.HttpClientCall.sendRequest()
there's a check to see if an entity exists:
Representation entity = request.isEntityAvailable() ?
request.getEntity() : null;
if(entity != null)
{
//The code to write the representation to the output stream.
}
The call to request.isEntityAvailable() goes back to the
org.restlet.data.Message class which checks that the entity is not
null and that the size of the entity is greater than 0. This is
where I'm having difficulty. In my Representation, I do not know the
size of the resulting data prior to it being written to the output
stream, so my size is 0. However, if you look at the
com.noelios.restlet.ext.net.HttpUrlConnectionCall.sendRequest(
) method, you see this:
// Adjust the streaming mode
if (entity.getSize() 0)
{
// The size of the entity is known in advance
getConnection().setFixedLengthStreamingMode((int)
entity.getSize());
}
else
{
// The size of the entity is not known in advance
if (this.clientHelper.getChunkLength() = 0)
{
// Use chunked encoding
getConnection().setChunkedStreamingMode(
this.clientHelper.getChunkLength());
}
else
{
// Use entity buffering to determine the content length
}
}
This suggests that the data I am sending can have a size of 0, but
when it does it is never sent. And I can't guess a size, because
if I guess wrong then the setFixedLengthStreamingMode will cause the
HTTP Post to fail when more bytes are written than expected.
I guess my question is: When we subclass Representation,
