I simply wrote a wrapper around
RDFConnectionRemove and RDFConnectionRemoteBuilder:
public class RDFConnectionRemoteWithLength extends RDFConnectionRemote {
protected RDFConnectionRemoteWithLength(Transactional txnLifecycle,
HttpClient httpClient, HttpContext httpContext, String destination,
String queryURL, String updateURL, String gspURL, RDFFormat outputQuads,
RDFFormat outputTriples, String acceptDataset, String acceptGraph,
String acceptSparqlResults, String acceptSelectResult, String
acceptAskResult, boolean parseCheckQueries, boolean parseCheckUpdates) {
super(txnLifecycle, httpClient, httpContext, destination,
queryURL, updateURL, gspURL, outputQuads, outputTriples, acceptDataset,
acceptGraph, acceptSparqlResults, acceptSelectResult, acceptAskResult,
parseCheckQueries, parseCheckUpdates);
}
public static RDFConnectionRemoteBuilder create() {
return new RDFConnectioRemoteWithLengthBuilder();
}
@Override
protected HttpEntity graphToHttpEntity(Graph graph, RDFFormat syntax) {
return graphToHttpEntityWithLength(graph, syntax);
}
protected HttpEntity graphToHttpEntityWithLength(Graph graph,
RDFFormat syntax) {
String ct = syntax.getLang().getContentType().getContentType();
ByteArrayOutputStream out = new ByteArrayOutputStream(128*1024);
RDFDataMgr.write(out, graph, syntax);
ByteArrayEntity entity = new ByteArrayEntity(out.toByteArray());
entity.setContentType(ct);
return entity;
}
}
public class RDFConnectioRemoteWithLengthBuilder extends
RDFConnectionRemoteBuilder {
@Override
protected RDFConnectionRemote buildConnection() {
return new RDFConnectionRemoteWithLength(txnLifecycle,
httpClient, httpContext,
destination, queryURL, updateURL, gspURL,
outputQuads, outputTriples,
acceptDataset, acceptGraph,
acceptSparqlResults, acceptSelectResult, acceptAskResult,
parseCheckQueries, parseCheckUpdates);
}
}
Thanks a lot.
Regards,
Sebastian
On 01.11.19 10:57, Andy Seaborne wrote:
Switching to putting in the length shoudl work. As this is for models
(not sending large files), buffering might be the better choice after all.
The below worked in limited testing.
Sorry to rush this ...
Andy
RDFConnection:
/** Create an HttpEntity for the graph */
protected HttpEntity graphToHttpEntity(Graph graph, RDFFormat syntax) {
if ( true )
return graphToHttpEntityWithLength(graph, syntax);
EntityTemplate entity = new
EntityTemplate((out)->RDFDataMgr.write(out, graph, syntax));
String ct = syntax.getLang().getContentType().getContentType();
entity.setContentType(ct);
return entity;
}
/** Create an HttpEntity for the graph */
protected HttpEntity graphToHttpEntityWithLength(Graph graph, RDFFormat
syntax) {
String ct = syntax.getLang().getContentType().getContentType();
ByteArrayOutputStream out = new ByteArrayOutputStream(128*1024);
RDFDataMgr.write(out, graph, syntax);
ByteArrayEntity entity = new ByteArrayEntity(out.toByteArray());
entity.setContentType(ct);
return entity;
}
On 01/11/2019 07:49, Sebastian Trueg wrote:
ah, interesting. I will dig a bit deeper then, maybe I will be able to
help. Thank you.
On 31.10.19 23:26, Andy Seaborne wrote:
Yes, Jena bug, possibly related to the -1 length.
Recorded as:
https://issues.apache.org/jira/browse/JENA-1776
-1 is because Jena streams the content for a GSP PUT operation. It does
not know the length at the start of request. To get the length you have
to produce the content then send the request - which is not streaming. A
"feature" of HTTP.
So I'm guessing that the connection is not handled like the default (no
auth) version in some way but I don't know why.
RDFConnectionRemote has a builder and connectPW is just this:
public static RDFConnection connectPW(String URL, String user, String
password) {
BasicCredentialsProvider credsProvider =
new BasicCredentialsProvider();
Credentials credentials =
new UsernamePasswordCredentials(user, password);
credsProvider.setCredentials(AuthScope.ANY, credentials);
HttpClient client = HttpClients.custom()
.setDefaultCredentialsProvider(credsProvider)
.build();
return RDFConnectionRemote.create()
.destination(URL)
.httpClient(client)
.build();
}
so it is a matter for setting up the HttpClient correctly (setting it up
so it is closed after each use? This is the compromise HTTP requires -
stream once, or buffer and send length with connection reuse).
The normal, no auth default is
HttpOp.createPoolingHttpClientBuilder
Andy
FYI: I'm away, no development machine (if I even have internet access,
and that's not certain), for a week or so.
On 31/10/2019 14:45, Sebastian Trueg wrote:
So... a Jena bug then? Do you have an idea if there is something I can
configure in the HttpClient as a workaround?
On 31.10.19 15:18, Andy Seaborne wrote:
I can make it happen sporadically using Fuseki main (so no shiro,
not a
webapp environment). It happens maybe one in four test runs.
It looks like a client-side problem - it creates the HTTP connection
each time and maybe something is cached in HttpClient. A hash-map-ism
would explain the "sporadically". If so, the # triples effect maybe
causing the timing to get changed a little.
Andy
On 31/10/2019 13:27, Sebastian Trueg wrote:
Interestingly it does not. Only going down to a really small
number of
triples does work.
And to make sure I did implement manual HTTP DELETE+PUT via OkHttp
which
works without problems on the same Fuseki instance.
Regards,
Sebastian
On 31.10.19 14:14, Andy Seaborne wrote:
Thanks.
Presumably it works if you create the RDFConnection once and
reuse the
java object?
Andy
On 31/10/2019 12:10, Sebastian Trueg wrote:
Hi Andy,
- Fuseki started via "fuseki-server".
- shiro config attached
- "pdm-data-model" dataset created via attached config
- Simple test app attached which takes the fuseki dataset url as
parameter.
Hope this helps.
Regards,
Sebastian
On 31.10.19 11:55, Andy Seaborne wrote:
Sebastian,
Do you have a complete, minimal example, including the Fuseki
setup
for
the user/password. Which Fuseki variant? war? full-jar? main?
Andy
PS Don't forget teh javadoc on connectPW : it's "basic auth"
On 31/10/2019 10:30, Sebastian Trueg wrote:
Hi everyone,
trying to use RDFConnection with Jena 3.13.1 to put a Model
into a
remote Fuseki instance I encountered very strange behavior. First
off,
let me show my very simple code:
try(RDFConnection conn
= RDFConnectionFactory.connectPW(datasetUrl, "admin",
"admin")) {
conn.put(graphUri, model);
}
This works fine on its own and for very small models in general.
But as
soon as I repeat the exact same snippet of code, ie. run the
same try
block twice I get a SocketException (Broken pipe) on the first
call to
RDFConnection::put.
So, to sum up:
- Single put works fine.
- A subsequent call to put will result in the first one already
throwing
an exception!
- Using a model with less than 100 triples results in both put
operations to succeed.
- In all this the Fuseki instance keeps on working.
Any ideas?
Regards,
Sebastian
