Hi Andy,
a follow-up to the problem: In order to successfully PUT bigger graphs
(roughly around 2k triples) I needed to enable expect-continue on the
request:
HttpOp.setRequestTransformer(req -> {
if (req instanceof HttpPut) {
((HttpPut) req).setConfig(RequestConfig.custom()
.setExpectContinueEnabled(true)
.build());
}
return req;
}
);
Otherwise I would constantly get a "broken pipe".
Hope this helps.
Regards,
Sebastian
On 01.11.19 10:57, Andy Seaborne wrote:
> Switching to putting in the length shoudl work. As this is for models
> (not sending large files), buffering might be the better choice after all.
>
> The below worked in limited testing.
>
> Sorry to rush this ...
>
> Andy
>
> RDFConnection:
>
> /** Create an HttpEntity for the graph */
> protected HttpEntity graphToHttpEntity(Graph graph, RDFFormat syntax) {
> if ( true )
> return graphToHttpEntityWithLength(graph, syntax);
>
> EntityTemplate entity = new
> EntityTemplate((out)->RDFDataMgr.write(out, graph, syntax));
> String ct = syntax.getLang().getContentType().getContentType();
> entity.setContentType(ct);
> return entity;
> }
>
> /** Create an HttpEntity for the graph */
> protected HttpEntity graphToHttpEntityWithLength(Graph graph, RDFFormat
> syntax) {
> String ct = syntax.getLang().getContentType().getContentType();
> ByteArrayOutputStream out = new ByteArrayOutputStream(128*1024);
> RDFDataMgr.write(out, graph, syntax);
> ByteArrayEntity entity = new ByteArrayEntity(out.toByteArray());
> entity.setContentType(ct);
> return entity;
> }
>
>
>
>
> On 01/11/2019 07:49, Sebastian Trueg wrote:
>> ah, interesting. I will dig a bit deeper then, maybe I will be able to
>> help. Thank you.
>>
>> On 31.10.19 23:26, Andy Seaborne wrote:
>>> Yes, Jena bug, possibly related to the -1 length.
>>>
>>> Recorded as:
>>> https://issues.apache.org/jira/browse/JENA-1776
>>>
>>> -1 is because Jena streams the content for a GSP PUT operation. It does
>>> not know the length at the start of request. To get the length you have
>>> to produce the content then send the request - which is not streaming. A
>>> "feature" of HTTP.
>>>
>>> So I'm guessing that the connection is not handled like the default (no
>>> auth) version in some way but I don't know why.
>>>
>>> RDFConnectionRemote has a builder and connectPW is just this:
>>>
>>>
>>> public static RDFConnection connectPW(String URL, String user, String
>>> password) {
>>> BasicCredentialsProvider credsProvider =
>>> new BasicCredentialsProvider();
>>> Credentials credentials =
>>> new UsernamePasswordCredentials(user, password);
>>> credsProvider.setCredentials(AuthScope.ANY, credentials);
>>> HttpClient client = HttpClients.custom()
>>> .setDefaultCredentialsProvider(credsProvider)
>>> .build();
>>>
>>> return RDFConnectionRemote.create()
>>> .destination(URL)
>>> .httpClient(client)
>>> .build();
>>> }
>>>
>>> so it is a matter for setting up the HttpClient correctly (setting it up
>>> so it is closed after each use? This is the compromise HTTP requires -
>>> stream once, or buffer and send length with connection reuse).
>>>
>>> The normal, no auth default is
>>> HttpOp.createPoolingHttpClientBuilder
>>>
>>> Andy
>>>
>>> FYI: I'm away, no development machine (if I even have internet access,
>>> and that's not certain), for a week or so.
>>>
>>>
>>> On 31/10/2019 14:45, Sebastian Trueg wrote:
>>>> So... a Jena bug then? Do you have an idea if there is something I can
>>>> configure in the HttpClient as a workaround?
>>>>
>>>>
>>>> On 31.10.19 15:18, Andy Seaborne wrote:
>>>>> I can make it happen sporadically using Fuseki main (so no shiro,
>>>>> not a
>>>>> webapp environment). It happens maybe one in four test runs.
>>>>>
>>>>> It looks like a client-side problem - it creates the HTTP connection
>>>>> each time and maybe something is cached in HttpClient. A hash-map-ism
>>>>> would explain the "sporadically". If so, the # triples effect maybe
>>>>> causing the timing to get changed a little.
>>>>>
>>>>> Andy
>>>>>
>>>>> On 31/10/2019 13:27, Sebastian Trueg wrote:
>>>>>> Interestingly it does not. Only going down to a really small
>>>>>> number of
>>>>>> triples does work.
>>>>>>
>>>>>> And to make sure I did implement manual HTTP DELETE+PUT via OkHttp
>>>>>> which
>>>>>> works without problems on the same Fuseki instance.
>>>>>>
>>>>>> Regards,
>>>>>> Sebastian
>>>>>>
>>>>>> On 31.10.19 14:14, Andy Seaborne wrote:
>>>>>>> Thanks.
>>>>>>>
>>>>>>> Presumably it works if you create the RDFConnection once and
>>>>>>> reuse the
>>>>>>> java object?
>>>>>>>
>>>>>>> Andy
>>>>>>>
>>>>>>> On 31/10/2019 12:10, Sebastian Trueg wrote:
>>>>>>>> Hi Andy,
>>>>>>>>
>>>>>>>> - Fuseki started via "fuseki-server".
>>>>>>>> - shiro config attached
>>>>>>>> - "pdm-data-model" dataset created via attached config
>>>>>>>> - Simple test app attached which takes the fuseki dataset url as
>>>>>>>> parameter.
>>>>>>>>
>>>>>>>> Hope this helps.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Sebastian
>>>>>>>>
>>>>>>>> On 31.10.19 11:55, Andy Seaborne wrote:
>>>>>>>>> Sebastian,
>>>>>>>>>
>>>>>>>>> Do you have a complete, minimal example, including the Fuseki
>>>>>>>>> setup
>>>>>>>>> for
>>>>>>>>> the user/password. Which Fuseki variant? war? full-jar? main?
>>>>>>>>>
>>>>>>>>> Andy
>>>>>>>>>
>>>>>>>>> PS Don't forget teh javadoc on connectPW : it's "basic auth"
>>>>>>>>>
>>>>>>>>> On 31/10/2019 10:30, Sebastian Trueg wrote:
>>>>>>>>>> Hi everyone,
>>>>>>>>>>
>>>>>>>>>> trying to use RDFConnection with Jena 3.13.1 to put a Model
>>>>>>>>>> into a
>>>>>>>>>> remote Fuseki instance I encountered very strange behavior. First
>>>>>>>>>> off,
>>>>>>>>>> let me show my very simple code:
>>>>>>>>>>
>>>>>>>>>> try(RDFConnection conn
>>>>>>>>>> = RDFConnectionFactory.connectPW(datasetUrl, "admin",
>>>>>>>>>> "admin")) {
>>>>>>>>>> conn.put(graphUri, model);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> This works fine on its own and for very small models in general.
>>>>>>>>>> But as
>>>>>>>>>> soon as I repeat the exact same snippet of code, ie. run the
>>>>>>>>>> same try
>>>>>>>>>> block twice I get a SocketException (Broken pipe) on the first
>>>>>>>>>> call to
>>>>>>>>>> RDFConnection::put.
>>>>>>>>>>
>>>>>>>>>> So, to sum up:
>>>>>>>>>> - Single put works fine.
>>>>>>>>>> - A subsequent call to put will result in the first one already
>>>>>>>>>> throwing
>>>>>>>>>> an exception!
>>>>>>>>>> - Using a model with less than 100 triples results in both put
>>>>>>>>>> operations to succeed.
>>>>>>>>>> - In all this the Fuseki instance keeps on working.
>>>>>>>>>>
>>>>>>>>>> Any ideas?
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Sebastian
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
--
Sebastian Trueg
Managing Director
TrueGeeX UG (haftungsbeschränkt)
[email protected]
http://www.linkedin.com/in/trueg
Mobile: 0049 1762 3244 664
