On 10/12/13 18:08, Martynas Jusevičius wrote:
Thanks Andy. I'm trying 2/ and 4/ simultaneously. I think an optional
non-streaming mode would be a good idea anyway.
As a sidenote, I want to complain about extensibility of Jena a
little. Lets take the same DatasetGraphAccessorHTTP as example:
- I cannot override graphToHttpEntity() because it is static and
others depend on it
- I cannot override doPost() etc. since they're private - would
protected not be enough?
Fine - can do that but the key is interface DatasetGraphAccessor which
is public. (now "have done that")
DatasetGraphAccessorHTTP is not exactly rocket science as classes go -
it's all format-and-call HttpOp.
We are talking about only this much code:
@Override public void httpPost(Graph data)
{ doPost(targetDefault(), data) ; }
@Override public void httpPost(Node graphName, Graph data)
{ doPost(target(graphName), data) ; }
private void doPost(String url, Graph data) {
HttpEntity entity = graphToHttpEntity(data) ;
HttpOp.execHttpPost(url, entity, null, null, this.authenticator) ;
}
Both httpPost are public!
The whole of GSP isn't rocket science - it's barely rocket enginneering [*].
- say I want to change the default serialization language to
N-Triples. I cannot do that sine sendLang is static and there's no
getter method for it
Submit a patch!
So instead of extending the class and overriding a single method I now
end up copying over all of the code, because in my opinion there are
too many static and private methods. End of complaint,
Submit patches!
Jena is great otherwise :)
Thanks.
Andy
Martynas
[*] Rocket science is simple to explain - you throw things out the back
and the rocket goes forward. Rocket engineering, doing it efficiently
and safely, is much, much harder.
On Tue, Dec 10, 2013 at 6:53 PM, Andy Seaborne <[email protected]> wrote:
On 10/12/13 16:26, Martynas Jusevičius wrote:
I'm upgrading from an old version of DatasetGraphAccessorHTTP (that I
had tweaked to support authentication):
https://github.com/Graphity/graphity-ldp/blob/master/src/main/java/org/graphity/update/DatasetGraphAccessorHTTP.java
HttpClient must be doing it because of the HttpEntity that is set.
The server is Dydra and the log follows. If getting the server to
stream is not possible at the moment, what are my options?
1/ Call HttpOps.execHttpPost(String.HttpEntity) yourself.
2/ Change DatasetGraphAccessorHTTP.graphToHttpEntity to use the same sort of
HttpEntity as before (which will send a single Conent-Length set
3/ ?? Use a different HttpClient from the default
See HttpOps.setDefaultHttpClient
No idea what the right setting are I'm afraid.
4/ Get Dydra to support streaming - I guess they don't support chunk
encoding and only support simple modes with Content-Length: - but it looks
like nginx proxied to the backend so hard to be sure.
It's http-client that's doing that and as a very well used piece of
software, I'm mildly confident it is doing the right thing.
Maybe it's just a matter of tuning that off (see 3).
http://en.wikipedia.org/wiki/Chunked_transfer_encoding
"""
Chunked encoding has the benefit that it is not necessary to generate the
full content before writing the header, as it allows streaming of content as
chunks and explicitly signaling the end of the content, making the
connection available for the next HTTP request/response.
"""
I suppose we could have an option on DatasetGraphAccessorHTTP for
"inefficient" mode (several of us are rather pro streaming :-) but not the
default.
Andy
15:17:05,340 DEBUG wire:63 - >> "POST
/graphity/some-test/service?graph=http%3A%2F%2Flocalhost%3A8080%2Fdanske_aviser%2Fgraphs%2Fceb6ec92-3b71-4bc8-a438-a327ba889ecf%23this
HTTP/1.1[\r][\n]"
15:17:05,340 DEBUG wire:63 - >> "Transfer-Encoding: chunked[\r][\n]"
15:17:05,341 DEBUG wire:63 - >> "Content-Type:
application/rdf+xml[\r][\n]"
15:17:05,341 DEBUG wire:63 - >> "Host: xx.dydra.com[\r][\n]"
15:17:05,341 DEBUG wire:63 - >> "Connection: Keep-Alive[\r][\n]"
15:17:05,342 DEBUG wire:63 - >> "User-Agent: Apache-HttpClient/4.2.3
(java 1.5)[\r][\n]"
15:17:05,342 DEBUG wire:63 - >> "Authorization: Basic
.......[\r][\n]"
15:17:05,342 DEBUG wire:63 - >> "[\r][\n]"
15:17:05,342 DEBUG headers:273 - >> POST
/graphity/some-test/service?graph=http%3A%2F%2Flocalhost%3A8080%2Fdanske_aviser%2Fgraphs%2Fceb6ec92-3b71-4bc8-a438-a327ba889ecf%23this
HTTP/1.1
15:17:05,343 DEBUG headers:276 - >> Transfer-Encoding: chunked
15:17:05,343 DEBUG headers:276 - >> Content-Type: application/rdf+xml
15:17:05,343 DEBUG headers:276 - >> Host: xx.dydra.com
15:17:05,343 DEBUG headers:276 - >> Connection: Keep-Alive
15:17:05,343 DEBUG headers:276 - >> User-Agent:
Apache-HttpClient/4.2.3 (java 1.5)
...
15:17:05,406 DEBUG DefaultClientConnection:254 - Receiving response:
HTTP/1.1 411 Length Required
15:17:05,406 DEBUG headers:257 - << HTTP/1.1 411 Length Required
15:17:05,407 DEBUG headers:260 - << Server: nginx/1.2.1
15:17:05,407 DEBUG headers:260 - << Date: Tue, 10 Dec 2013 14:17:07 GMT
15:17:05,407 DEBUG headers:260 - << Content-Type: text/html
15:17:05,407 DEBUG headers:260 - << Content-Length: 180
15:17:05,408 DEBUG headers:260 - << Connection: close
15:17:05,409 DEBUG SystemDefaultHttpClient:73 - Authentication succeeded
15:17:05,409 DEBUG HttpOp:1017 - [8] 411 Length Required
15:17:05,410 DEBUG DefaultClientConnection:169 - Connection
0.0.0.0:51690<->176.9.131.227:80 closed
15:17:05,410 DEBUG PoolingClientConnectionManager:281 - Connection
released: [id: 7][route: {}->http://xx.dydra.com][total kept alive:
On Tue, Dec 10, 2013 at 5:18 PM, Andy Seaborne <[email protected]> wrote:
On 10/12/13 14:33, Martynas Jusevičius wrote:
Hey,
after upgrade to 2.11.0 I get this exception during Graph Store
Protocol request:
org.apache.jena.atlas.web.HttpException: 411 - Length Required
org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1020)
org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:633)
org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:601)
org.apache.jena.web.DatasetGraphAccessorHTTP.doPost(DatasetGraphAccessorHTTP.java:184)
org.apache.jena.web.DatasetGraphAccessorHTTP.httpPost(DatasetGraphAccessorHTTP.java:179)
org.apache.jena.web.DatasetAdapter.add(DatasetAdapter.java:98)
org.graphity.server.util.DataManager.addModel(DataManager.java:440)
Has something changed wrt to Content-Length? I guess I'll need to
extend DatasetGraphAccessorHTTP... Or is there an easier way to work
around this?
Martynas
graphityhq.com
Looks like the server not liking the client request - what did the server
say about the request? (and what is the server?)
ARQ like to stream - and that means no content length.
What does the Jena code from the version you are upgarding for under
DatasetGraphAccessorHTTP look like?
Andy
