[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[Smalyshev]

Smalyshev added a comment.
POST can be used with any queries, same as GET, though I'm not sure how it would be cached, so I'd recommend using POST only for big queries now, unless you don't have a choice (e.g. the tool only supports POST). I'll update about caching once I know it.TASK DETAILhttps://phabricator.wikimedia.org/T112151EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Ricordisamoa, Arbnos, Jonas, chasemp, JanZerebecki, BBlack, Andrew, Deskana, Joe, gerritbot, nichtich, Jneubert, Karima, Aklapper, Smalyshev, Th3d3v1ls, Ramalepe, Liugev6, EBjune, merbst, Avner, Lewizho99, Maathavan, debt, TerraCodes, Gehel, D3r1ck01, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, aude, Manybubbles, fbstj, santhosh, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[gerritbot]

gerritbot added a comment.
Change 243883 merged by Gehel:
Allow SPARQL endpoint to be queried via POST

https://gerrit.wikimedia.org/r/243883TASK DETAILhttps://phabricator.wikimedia.org/T112151EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, gerritbotCc: Ricordisamoa, Arbnos, Jonas, chasemp, JanZerebecki, BBlack, Andrew, Deskana, Joe, gerritbot, nichtich, Jneubert, Karima, Aklapper, Smalyshev, Th3d3v1ls, Ramalepe, Liugev6, EBjune, merbst, Avner, Lewizho99, Maathavan, debt, Gehel, D3r1ck01, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, aude, Manybubbles, Mbch331___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[Smalyshev]

Smalyshev added a comment.
Java/Blazegraph seems to have URL length limit around 8K. So we may have to support real POST on Blazegraph side anyway.TASK DETAILhttps://phabricator.wikimedia.org/T112151EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Ricordisamoa, Arbnos, Jonas, chasemp, JanZerebecki, BBlack, Andrew, Deskana, Joe, gerritbot, nichtich, Jneubert, Karima, Aklapper, Smalyshev, Th3d3v1ls, Ramalepe, Liugev6, EBjune, merbst, Avner, Lewizho99, Maathavan, debt, Gehel, D3r1ck01, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, aude, Manybubbles, Mbch331___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[Jneubert]

Jneubert added a comment.
Examples for the federated query use case see at https://github.com/zbw/sparql-queries/tree/master/wikidata#power-queries (queries with econ_pers or ebds in name). The intermediate set fed into Wikidata was about 450,000 GND IDs. Currently, that requieres setting up a custom Wikidata endpoint.TASK DETAILhttps://phabricator.wikimedia.org/T112151EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, JneubertCc: Jonas, chasemp, JanZerebecki, BBlack, Andrew, Deskana, Joe, gerritbot, nichtich, Jneubert, Karima, Aklapper, Smalyshev, Th3d3v1ls, Ramalepe, Liugev6, EBjune, mschwarzer, Avner, Lewizho99, Maathavan, debt, Gehel, D3r1ck01, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, aude, Manybubbles, Mbch331___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[JanZerebecki]

JanZerebecki added a comment.

Yes. But: 1) This ticket is only about compatibility with existing clients we 
didn't create and don't control. Those clients are conforming implementations 
of SPARQL 1.1 2) Option 2 and 3 in 
https://phabricator.wikimedia.org/T112151#1818392 are not defined in the SPARQL 
1.1 protocol. See 
http://www.w3.org/TR/2013/REC-sparql11-protocol-20130321/#query-operation . 3) 
That standard explicitly allows POST for read queries. (I'll spare you a long 
rant about W3C standards.)


TASK DETAIL
  https://phabricator.wikimedia.org/T112151

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, JanZerebecki
Cc: chasemp, JanZerebecki, BBlack, Andrew, Deskana, Joe, gerritbot, nichtich, 
Jneubert, Karima, Aklapper, Smalyshev, jkroll, Wikidata-bugs, Jdouglas, aude, 
Manybubbles, Mbch331



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[Smalyshev]

Smalyshev added a comment.

> Primarily it's that we can't cache them in Varnish like we should be able to

Caching SPARQL results via Varnish is not a good idea in most cases. For two 
reasons:

1. Usually the same query is not repeated very often (unless somebody has a 
buggy script, etc.). Every client would run its own query and once the result 
is returned, it will go away and another client would use another query.
2. When the query **is** repeated, it is usually because the client expects 
something to change - like getting update from Wikidata. Since we do not have 
any real way to invalidate Varnish when underlying data changes, caching here 
would only hinder the clients.

Also, these results can be pretty big - so we'd be storing tons of information 
which is not useful. Of course, there are exceptions and some scenarios would 
still benefit from caching, but most would not.

> but as mentioned before it's going to affect how multi-DC request routing 
> works as well,

There are no plans of multi-DC setup for WDQS, as far as I know.

> AFAICS there is no general solution to this problem.

We do not need a general solution for the whole HTTP world. We need a 
particular solution for specific practical issue, and we have everything we 
need to actually solve it - except willingness to solve it.

> But still, trying to stick to standards here isn't just a matter of theory 
> trumping user requests.

I think in this particular case it is exactly that, as all objections so far, 
when applied to what happens with WDQS, appear to be only theoretical.

All proposed solutions require creating a whole new middleware layer, and I 
don't think anybody is ready to allocate resources to develop and maintain this 
layer.

> If there's a problem with client tooling, we can submit patches to those 
> client projects

Not all of those are open source projects, and they would not spend resources 
on rewriting their tools just because one particular endpoint (ours) does not 
support POST. Also, who exactly would be spending time on learning these tools, 
developing and submitting those patches?


TASK DETAIL
  https://phabricator.wikimedia.org/T112151

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: chasemp, JanZerebecki, BBlack, Andrew, Deskana, Joe, gerritbot, nichtich, 
Jneubert, Karima, Aklapper, Smalyshev, jkroll, Wikidata-bugs, Jdouglas, aude, 
Manybubbles, Mbch331



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[BBlack]

BBlack added a comment.

In https://phabricator.wikimedia.org/T112151#1739918, @Smalyshev wrote:

> > As Andrew said above, why not support this directly in WQDS if you have to 
> > support it at all?
>
>
> Because in Blazegraph, allowing POST means allowing write requests. This is 
> not good for security.


So even in blazegraph, GET and POST have standard semantic meanings... why is 
it that clients don't honor this?

> > even though most (all?) SPARQL traffic is readonly and probably can be 
> > cached

> 

> 

> I don't think it is a good idea to cache SPARQL queries. They are big, they 
> are rarely repeated as-is, and if they are repeated this is usually because 
> the client expects new result. At least until we have some setup that 
> repeatedly requests same data over SPARQL, I don't see much point in caching 
> SPARQL responses.


It would be wiser to give them some (perhaps minimal) cacheability via 
Cache-Control, if nothing else as a buffer against simplistic DoS attacks that 
spam the same query at high rates...

> > This also conflicts with our overall strategy for multi-datacenter work,

> 

> 

> Given that there is no multi-datacenter setup for wdqs, I'm not sure how it 
> is relevant. Most SPARQL clients for which it is relevant wouldn't probably 
> have cookie storage mechanisms anyway.


There is eventually multi-datacenter for **everything** in the long term, so 
yeah that includes wdqs and is very relevant.  Every service is going to have 
to account for it in the long run with how state is managed, and one of our 
mechanisms for balancing traffic and avoiding state replication lag is the idea 
that while a normal (no special cookie) GET request is balanced between the DCs 
based on geography and load like normal, but POST requests are always directed 
to the primary DC only.  Additionally, once a POST request is seen, it sets a 
short-duration session cookie that maps all of that client's GET requests to 
the primary DC only as well (so that they don't suffer lag effects in reading 
the results of their own modifications).  If a read-(only|mostly) service uses 
POST for all of its traffic due to client deficiency, all of that traffic will 
be stuck on the primary DC only and not benefit from read load balancing.


TASK DETAIL
  https://phabricator.wikimedia.org/T112151

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, BBlack
Cc: JanZerebecki, BBlack, Andrew, Deskana, Joe, gerritbot, nichtich, Jneubert, 
Karima, Aklapper, Smalyshev, JGirault, jkroll, Wikidata-bugs, Jdouglas, aude, 
Manybubbles



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[JanZerebecki]

JanZerebecki added a comment.

In https://phabricator.wikimedia.org/T112151#1742574, @chasemp wrote:

> At first glance, it seems like this means mangling all POST to GET would 
> result in an incompatibility with the sparql protocol.


I didn't suggest that. I suggested to apply GET semantics to POST requests that 
indicate this via a part in the query of their Request-URI.

> If we are saying we don't want to leave clients (which follow a somewhat odd 
> spec to my mind) in the dust I don't think we are accomplishing that, and as 
> Brandon outlined POST in the Wikimedia multi-dc world has specific 
> consequences.


Would you describe how my suggestion doesn't accomplish that?


TASK DETAIL
  https://phabricator.wikimedia.org/T112151

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, JanZerebecki
Cc: chasemp, JanZerebecki, BBlack, Andrew, Deskana, Joe, gerritbot, nichtich, 
Jneubert, Karima, Aklapper, Smalyshev, JGirault, jkroll, Wikidata-bugs, 
Jdouglas, aude, Manybubbles



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[JanZerebecki]

JanZerebecki added a comment.

The first sentence in the task is the use case I looked at while coming up with 
my suggestion. To explain why these tools would use POST I came up with the 
Request-URI length. But another workable explanation is that the tool just 
doesn't bother to differentiate between write (update) and read-only query and 
thus to support both it always uses POST, which is a superset of what works 
with GET.


TASK DETAIL
  https://phabricator.wikimedia.org/T112151

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, JanZerebecki
Cc: chasemp, JanZerebecki, BBlack, Andrew, Deskana, Joe, gerritbot, nichtich, 
Jneubert, Karima, Aklapper, Smalyshev, jkroll, Wikidata-bugs, Jdouglas, aude, 
Manybubbles



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[chasemp]

chasemp added a subscriber: chasemp.
chasemp added a comment.

Yesterday in IRC it seemed like there was consensus we don't have any actual 
(quantifiable) concerns regarding query length at the moment.

  SMalyshev: chasemp: I'm not worried too much about query length

The 2008  
guidelines seem explicit that GET is the mechanism of choice up to the point 
that length for GET isn't viable.  The 2013 
 guidelines still indicate the 
use of POST query but also outline that it isn't the only operation for which 
POST is necessary :

  This protocol is a companion to the use of both SPARQL Update and SPARQL 
Query over the SPARQL  
  protocol via HTTP POST. Both protocols specify different operations performed 
via the HTTP POST method.

At first glance, it seems like this means mangling all POST to GET would result 
in an incompatibility with the sparql protocol.

If we are saying we don't want to leave clients (which follow a somewhat odd 
spec to my mind) in the dust I don't think we are accomplishing that, and as 
Brandon outlined POST in the Wikimedia multi-dc world has specific consequences.

Besides knowing that at some point length is a factor do we have a 
demonstratable need for this functionality (which will be very nuanced in 
implementation)?


TASK DETAIL
  https://phabricator.wikimedia.org/T112151

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, chasemp
Cc: chasemp, JanZerebecki, BBlack, Andrew, Deskana, Joe, gerritbot, nichtich, 
Jneubert, Karima, Aklapper, Smalyshev, JGirault, jkroll, Wikidata-bugs, 
Jdouglas, aude, Manybubbles



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[chasemp]

chasemp added a comment.

In https://phabricator.wikimedia.org/T112151#1742627, @JanZerebecki wrote:

> In https://phabricator.wikimedia.org/T112151#1742574, @chasemp wrote:
>
> > At first glance, it seems like this means mangling all POST to GET would 
> > result in an incompatibility with the sparql protocol.
>
>
> I didn't suggest that. I suggested to apply GET semantics to POST requests 
> that indicate this via a part in the query of their Request-URI.


Most of the discussion in this ticket is surrounding 
https://gerrit.wikimedia.org/r/#/c/243883/

> 

> 

> > If we are saying we don't want to leave clients (which follow a somewhat 
> > odd spec to my mind) in the dust I don't think we are accomplishing that, 
> > and as Brandon outlined POST in the Wikimedia multi-dc world has specific 
> > consequences.

> 

> 

> Would you describe how my suggestion doesn't accomplish that?


I wasn't meaning to respond to the outline you provided necessarily, but is 
anyone being limited by this now?  Are users submitting queries from IE at the 
moment that are limited in this way?

I am looking for a use case we are trying to solve.  The sort of 'we are trying 
to do X now and can't because of this' that we can revolve the discussion 
around.


TASK DETAIL
  https://phabricator.wikimedia.org/T112151

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, chasemp
Cc: chasemp, JanZerebecki, BBlack, Andrew, Deskana, Joe, gerritbot, nichtich, 
Jneubert, Karima, Aklapper, Smalyshev, JGirault, jkroll, Wikidata-bugs, 
Jdouglas, aude, Manybubbles



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[Smalyshev]

Smalyshev added a comment.

> I am looking for a use case we are trying to solve.


The use case is users using SPARQL query tool that only does POST. 
Unfortunately, such tools exist, and in non-nelgligible numbers

> At first glance, it seems like this means mangling all POST to GET would 
> result in an incompatibility with the sparql protocol.


@chasemp I think you're missing an important point here - lack of SPARQL Update 
support on query endpoint is **intentional**. That's the whole point of the 
exercise - how we allow to send POST without allowing SPARQL Update. We do not 
want to be compatible with SPARQL Update in query endpoint - in fact, we want 
to explicitly forbid it, since we don't want the whole internet to mess with 
our database (there's wikidata.org site for that ;) That's why we do not want 
to allow anybody from outside to POST to Blazegraph. However, we do want to 
allow tools that use POST to do SPARQL Query to do so. Unfortunately, 
distinguishing SPARQL Query from SPARQL Update and other update requests may be 
non-trivial to do from something like nginx, so it is much safer and easier to 
just never send POST to Blazegraph, thus ensuring we never produce an update.

We could, of course, take a stance that since REST dictates retrieval queries 
should go through GET, we only support GET. However, this stance sounds to me 
unpractical and not user-friendly, as most users do not care about the purity 
of our REST track record (in fact, most of them have only the vaguest idea of 
REST and their requirements about POST/GET) and just want their SPARQL tool 
(which they probably use with a dozen of other SPARQL endpoints by now) to work 
with WDQS endpoint.

Answering @BBlack's question, we do not know why these tools only use POST, but 
most importantly, it's completely beyond the point **why** they do it - 
whatever the reasons are, they do it, and we're not going to change that. So we 
can either support them or refuse to support them. I do not see any practical 
reason to refuse support given that we can do it.

As for cookies and multi-DC setup, many non-browser clients would just ignore 
cookies completely anyway. For browser-based clients, I don't think in the 
perceivable timeframe we'd get traffic numbers that would make it any problem. 
The traffic numbers now are low, and if they raise most of the traffic would be 
bots, tools and widgets feeding data from SPARQL, not browser traffic. If it 
ever does become a problem, it should be trivial to make an exception for 
requests coming to query.wikidata.org - they are easily identifiable. Also, we 
will never have all or significant part of the overall traffic produced by 
these tools - these tools are user-friendly frontends, but the bulk of the work 
will be done by automated tools, just like a typical SQL database would serve 
most traffic via programmatic connections, not via Web interface.


TASK DETAIL
  https://phabricator.wikimedia.org/T112151

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: chasemp, JanZerebecki, BBlack, Andrew, Deskana, Joe, gerritbot, nichtich, 
Jneubert, Karima, Aklapper, Smalyshev, jkroll, Wikidata-bugs, Jdouglas, aude, 
Manybubbles



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[JanZerebecki]

JanZerebecki added a comment.

Short: Because IE is broken.
Long: The problem here is that while 
https://tools.ietf.org/html/rfc2616#section-3.2.1 says "The HTTP protocol does 
not place any a priori limit on the length of a URI." this reality doesn't 
conform. On the server side our caching layer is AFAIK even in violation of 
"SHOULD be able to handle URIs of unbounded length if they provide GET-based 
forms that could generate such URIs". It probably caps at around 16k. In 
reality we can probably live with that as the limit on queries for the public 
queries.wikidata.org. But while we might manage to fix all the SPARQL tools, we 
can't fix all the browsers. Which leaves us with: "Servers ought to be cautious 
about depending on URI lengths above 255 bytes".

To best serve the listed requirements we probably need to have generic handling 
for this in the layer that differentiates between POST and GET in the described 
multi data center way. It would e.g. know by a certain query argument in the 
Request-URI that this POST request has GET semantics with the actual resource 
specified by the Request-URI together with the request body. Should we then 
transform the request to a GET request or handle this special case all the way 
in the stack? Any better ideas?


TASK DETAIL
  https://phabricator.wikimedia.org/T112151

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, JanZerebecki
Cc: JanZerebecki, BBlack, Andrew, Deskana, Joe, gerritbot, nichtich, Jneubert, 
Karima, Aklapper, Smalyshev, JGirault, jkroll, Wikidata-bugs, Jdouglas, aude, 
Manybubbles



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[Andrew]

Andrew added a subscriber: Andrew.
Andrew added a comment.

WDQS is in internal tool, so if its API doesn't conform to clients' 
expectations then this is a bug in WDQS, yes?  Would it not be better to fix 
that rather than adding mysterious middleware obfuscation?


TASK DETAIL
  https://phabricator.wikimedia.org/T112151

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, Andrew
Cc: Andrew, Deskana, Joe, gerritbot, nichtich, Jneubert, Karima, Aklapper, 
Smalyshev, JGirault, jkroll, Wikidata-bugs, Jdouglas, aude, Manybubbles, 
JanZerebecki



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[BBlack]

BBlack added a subscriber: BBlack.
BBlack added a comment.

I'm not a fan of this on a few levels:

1. As Andrew said above, why not support this directly in WQDS if you have to 
support it at all? as in, let the POSTs come through the rest of the stack 
unmolested, and deal with it inside WQDS, instead of a dubious POST-to-GET 
conversion in the midst of our inbound traffic stack.
2. We don't want more app-specific hacks in nginx|varnish|apache config than 
necessary, and this seems unnecessary.
3. Regardless of which layer handles the issue: It breaks the semantics of GET 
as readonly and POST as a potential write-op.  POSTs as a rule as not cacheable 
responses, even though most (all?) SPARQL traffic is readonly and probably can 
be cached with at least some minimal TTL.  This also conflicts with our overall 
strategy for multi-datacenter work, which involves sticky-routing via cookies 
to the primary DC at the applayer for POST requests, but allowing normal 
balancing of GET/read requests.


TASK DETAIL
  https://phabricator.wikimedia.org/T112151

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, BBlack
Cc: BBlack, Andrew, Deskana, Joe, gerritbot, nichtich, Jneubert, Karima, 
Aklapper, Smalyshev, JGirault, jkroll, Wikidata-bugs, Jdouglas, aude, 
Manybubbles, JanZerebecki



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[Smalyshev]

Smalyshev added a comment.

> As Andrew said above, why not support this directly in WQDS if you have to 
> support it at all?


Because in Blazegraph, allowing POST means allowing write requests. This is not 
good for security.

> let the POSTs come through the rest of the stack unmolested, and deal with it 
> inside WQDS


This would mean building some middleware layer to filter read requests from 
write requests, which would create both a lot on unnecessary work and a 
security issue if the filtering is not perfect. Not allowing POST on Blazegraph 
side automatically cuts off all modification requests since GET processing 
engine in Blazegraph only supports read queries. POST processing supports both, 
so we'd have to work much harder to achieve same level of security.

> even though most (all?) SPARQL traffic is readonly and probably can be cached


I don't think it is a good idea to cache SPARQL queries. They are big, they are 
rarely repeated as-is, and if they are repeated this is usually because the 
client expects new result. At least until we have some setup that repeatedly 
requests same data over SPARQL, I don't see much point in caching SPARQL 
responses.

> This also conflicts with our overall strategy for multi-datacenter work,


Given that there is no multi-datacenter setup for wdqs, I'm not sure how it is 
relevant. Most SPARQL clients for which it is relevant wouldn't probably have 
cookie storage mechanisms anyway.


TASK DETAIL
  https://phabricator.wikimedia.org/T112151

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: BBlack, Andrew, Deskana, Joe, gerritbot, nichtich, Jneubert, Karima, 
Aklapper, Smalyshev, JGirault, jkroll, Wikidata-bugs, Jdouglas, aude, 
Manybubbles, JanZerebecki



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[Smalyshev]

Smalyshev added a comment.

I've tested it in labs and it seems OK, would like somebody from ops to review 
if it's OK for production.


TASK DETAIL
  https://phabricator.wikimedia.org/T112151

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: Deskana, Joe, gerritbot, nichtich, Jneubert, Karima, Aklapper, Smalyshev, 
jkroll, Wikidata-bugs, Jdouglas, aude, Manybubbles, JanZerebecki



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[Karima]

Karima added a comment.

You can test your endpoint with this tool (View->Show endpoint bar and write 
the url of your endpoint).

http://openuplabs.tso.co.uk/demos/sparqleditor

Try to respect the standard. (there is again a problem with the content 
negociation)
http://www.w3.org/TR/sparql11-protocol/#query-operation


TASK DETAIL
  https://phabricator.wikimedia.org/T112151

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, Karima
Cc: gerritbot, nichtich, Jneubert, Karima, Aklapper, Smalyshev, jkroll, 
Wikidata-bugs, Jdouglas, aude, Deskana, Manybubbles, JanZerebecki



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[gerritbot]

gerritbot added a subscriber: gerritbot.
gerritbot added a comment.

Change 243883 had a related patch set uploaded (by Smalyshev):
Allow SPARQL endpoint to be queries via POST

https://gerrit.wikimedia.org/r/243883


TASK DETAIL
  https://phabricator.wikimedia.org/T112151

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, gerritbot
Cc: gerritbot, nichtich, Jneubert, Karima, Aklapper, Smalyshev, jkroll, 
Wikidata-bugs, Jdouglas, aude, Deskana, Manybubbles, JanZerebecki



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

[Jneubert]

Jneubert added a subscriber: Jneubert.
Jneubert added a comment.

Another argument for enabling POST requests is, that GET requests / URLs are 
limited in length. The exact limit depends (see 
http://stackoverflow.com/questions/417142/what-is-the-maximum-length-of-a-url-in-different-browsers),
 but often seems to be about 2000 characters. I hit this in the past in several 
real queries.


TASK DETAIL
  https://phabricator.wikimedia.org/T112151

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, Jneubert
Cc: Jneubert, Karima, Aklapper, Smalyshev, jkroll, Wikidata-bugs, Jdouglas, 
aude, Deskana, Manybubbles, JanZerebecki



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs