Re: HttpException: 411 - Length Required

[Andy Seaborne]

On 12/12/13 11:08, Martynas Jusevičius wrote:
Andy,
speaking of nginx - maybe you happen to know if is possible to limit
request content size without a Content-Length header?

Martynas

Don't know - I would not be surprised if it's "can't" because in a 
stream, the length isn't known so you can only cleanly reject it (413) 
by reading up to the limit without passing it on the intended target 
function.

Otherwise, the intended target may see the data start coming in, start 
working, send 200 and start to send out it's own stream of results and 
... then have a problem because you can't now send 413.

Fuseki streams data in but it's still a transaction (see the "Fuseki 
v1.0.0 data import problems" message) because it may fail for some 
reason e.g parse error way down the stream.

        Andy


On Tue, Dec 10, 2013 at 6:53 PM, Andy Seaborne <a...@apache.org> wrote:
On 10/12/13 16:26, Martynas Jusevičius wrote:

I'm upgrading from an old version of DatasetGraphAccessorHTTP (that I
had tweaked to support authentication):

https://github.com/Graphity/graphity-ldp/blob/master/src/main/java/org/graphity/update/DatasetGraphAccessorHTTP.java


HttpClient must be doing it because of the HttpEntity that is set.


The server is Dydra and the log follows. If getting the server to
stream is not possible at the moment, what are my options?


1/ Call HttpOps.execHttpPost(String.HttpEntity) yourself.

2/ Change DatasetGraphAccessorHTTP.graphToHttpEntity to use the same sort of
HttpEntity as before (which will send a single Conent-Length set

3/ ?? Use a different HttpClient from the default
    See HttpOps.setDefaultHttpClient
    No idea what the right setting are I'm afraid.

4/ Get Dydra to support streaming - I guess they don't support chunk
encoding and only support simple modes with Content-Length: - but it looks
like nginx proxied to the backend so hard to be sure.

It's http-client that's doing that and as a very well used piece of
software, I'm mildly confident it is doing the right thing.

Maybe it's just a matter of tuning that off (see 3).

http://en.wikipedia.org/wiki/Chunked_transfer_encoding

"""
Chunked encoding has the benefit that it is not necessary to generate the
full content before writing the header, as it allows streaming of content as
chunks and explicitly signaling the end of the content, making the
connection available for the next HTTP request/response.
"""


I suppose we could have an option on DatasetGraphAccessorHTTP for
"inefficient" mode (several of us are rather pro streaming :-) but not the
default.

         Andy



15:17:05,340 DEBUG wire:63 - >> "POST

/graphity/some-test/service?graph=http%3A%2F%2Flocalhost%3A8080%2Fdanske_aviser%2Fgraphs%2Fceb6ec92-3b71-4bc8-a438-a327ba889ecf%23this
HTTP/1.1[\r][\n]"
15:17:05,340 DEBUG wire:63 - >> "Transfer-Encoding: chunked[\r][\n]"
15:17:05,341 DEBUG wire:63 - >> "Content-Type:
application/rdf+xml[\r][\n]"
15:17:05,341 DEBUG wire:63 - >> "Host: xx.dydra.com[\r][\n]"
15:17:05,341 DEBUG wire:63 - >> "Connection: Keep-Alive[\r][\n]"
15:17:05,342 DEBUG wire:63 - >> "User-Agent: Apache-HttpClient/4.2.3
(java 1.5)[\r][\n]"
15:17:05,342 DEBUG wire:63 - >> "Authorization: Basic
.......[\r][\n]"

15:17:05,342 DEBUG wire:63 - >> "[\r][\n]"
15:17:05,342 DEBUG headers:273 - >> POST

/graphity/some-test/service?graph=http%3A%2F%2Flocalhost%3A8080%2Fdanske_aviser%2Fgraphs%2Fceb6ec92-3b71-4bc8-a438-a327ba889ecf%23this
HTTP/1.1
15:17:05,343 DEBUG headers:276 - >> Transfer-Encoding: chunked
15:17:05,343 DEBUG headers:276 - >> Content-Type: application/rdf+xml
15:17:05,343 DEBUG headers:276 - >> Host: xx.dydra.com
15:17:05,343 DEBUG headers:276 - >> Connection: Keep-Alive
15:17:05,343 DEBUG headers:276 - >> User-Agent:
Apache-HttpClient/4.2.3 (java 1.5)
...
15:17:05,406 DEBUG DefaultClientConnection:254 - Receiving response:
HTTP/1.1 411 Length Required
15:17:05,406 DEBUG headers:257 - << HTTP/1.1 411 Length Required
15:17:05,407 DEBUG headers:260 - << Server: nginx/1.2.1
15:17:05,407 DEBUG headers:260 - << Date: Tue, 10 Dec 2013 14:17:07 GMT
15:17:05,407 DEBUG headers:260 - << Content-Type: text/html
15:17:05,407 DEBUG headers:260 - << Content-Length: 180
15:17:05,408 DEBUG headers:260 - << Connection: close
15:17:05,409 DEBUG SystemDefaultHttpClient:73 - Authentication succeeded
15:17:05,409 DEBUG HttpOp:1017 - [8] 411 Length Required
15:17:05,410 DEBUG DefaultClientConnection:169 - Connection
0.0.0.0:51690<->176.9.131.227:80 closed
15:17:05,410 DEBUG PoolingClientConnectionManager:281 - Connection
released: [id: 7][route: {}->http://xx.dydra.com][total kept alive:

On Tue, Dec 10, 2013 at 5:18 PM, Andy Seaborne <a...@apache.org> wrote:

On 10/12/13 14:33, Martynas Jusevičius wrote:


Hey,

after upgrade to 2.11.0 I get this exception during Graph Store
Protocol request:

org.apache.jena.atlas.web.HttpException: 411 - Length Required
org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1020)
org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:633)
org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:601)


org.apache.jena.web.DatasetGraphAccessorHTTP.doPost(DatasetGraphAccessorHTTP.java:184)


org.apache.jena.web.DatasetGraphAccessorHTTP.httpPost(DatasetGraphAccessorHTTP.java:179)
org.apache.jena.web.DatasetAdapter.add(DatasetAdapter.java:98)
org.graphity.server.util.DataManager.addModel(DataManager.java:440)

Has something changed wrt to Content-Length? I guess I'll need to
extend DatasetGraphAccessorHTTP... Or is there an easier way to work
around this?

Martynas
graphityhq.com


Looks like the server not liking the client request - what did the server
say about the request? (and what is the server?)

ARQ like to stream - and that means no content length.

What does the Jena code from the version you are upgarding for under
DatasetGraphAccessorHTTP look like?

          Andy