[Wikidata-bugs] [Maniphest] [Commented On] T107602: Set up a public interface to the wikidata query service

2015-08-04 Thread GWicke
GWicke added a subscriber: GWicke. GWicke added a comment. Will the query service return raw HTML or SVG content? If it's only returning other content types like JSON, then CORS might not end up mattering too much. An alternative to a separate domain could be to use `https://wikidata.org/api

[Wikidata-bugs] [Maniphest] [Edited] T84923: Reliable publish / subscribe event bus

2015-07-31 Thread GWicke
GWicke edited the task description. TASK DETAIL https://phabricator.wikimedia.org/T84923 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GWicke Cc: Aklapper, Matanya, Mattflaschen, Ottomata, mmodell, Eevans, chasemp, brion, Krenair, Halfak

[Wikidata-bugs] [Maniphest] [Edited] T84923: Reliable publish / subscribe event bus

2015-07-16 Thread GWicke
GWicke edited the task description. Herald added subscribers: Matanya, Aklapper. TASK DETAIL https://phabricator.wikimedia.org/T84923 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GWicke Cc: Aklapper, Matanya, Mattflaschen, Ottomata, mmodell

[Wikidata-bugs] [Maniphest] [Edited] T84923: Reliable publish / subscribe event bus

2015-06-22 Thread GWicke
GWicke edited the task description. TASK DETAIL https://phabricator.wikimedia.org/T84923 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GWicke Cc: Mattflaschen, Ottomata, mmodell, Eevans, chasemp, brion, Krenair, Halfak, JanZerebecki, bd808

[Wikidata-bugs] [Maniphest] [Edited] T84923: Reliable publish / subscribe event bus

2015-05-15 Thread GWicke
GWicke edited the task description. TASK DETAIL https://phabricator.wikimedia.org/T84923 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GWicke Cc: mmodell, Eevans, chasemp, brion, Krenair, Halfak, JanZerebecki, bd808, MZMcBride, mobrovac, GWicke

[Wikidata-bugs] [Maniphest] [Edited] T84923: Reliable publish / subscribe event bus

2015-05-15 Thread GWicke
GWicke edited the task description. TASK DETAIL https://phabricator.wikimedia.org/T84923 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GWicke Cc: mmodell, Eevans, chasemp, brion, Krenair, Halfak, JanZerebecki, bd808, MZMcBride, mobrovac, GWicke

[Wikidata-bugs] [Maniphest] [Updated] T93913: Investigate using RESTBase for batched entity lookups

2015-04-01 Thread GWicke
GWicke added a project: RESTBase-Usecase. TASK DETAIL https://phabricator.wikimedia.org/T93913 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T93913: Investigate using RESTBase for batched entity lookups

2015-04-01 Thread GWicke
GWicke added subscribers: mobrovac, Eevans. TASK DETAIL https://phabricator.wikimedia.org/T93913 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Changed Project Column] T84923: Reliable publish / subscribe event bus

2015-03-17 Thread GWicke
GWicke moved this task to Future on the Services workboard. TASK DETAIL https://phabricator.wikimedia.org/T84923 WORKBOARD https://phabricator.wikimedia.org/project/board/69/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username

[Wikidata-bugs] [Maniphest] [Edited] T84923: Reliable publish / subscribe event bus

2015-03-13 Thread GWicke
GWicke edited the task description. TASK DETAIL https://phabricator.wikimedia.org/T84923 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T84923: Reliable publish / subscribe event bus

2015-03-10 Thread GWicke
GWicke added a subscriber: Eevans. TASK DETAIL https://phabricator.wikimedia.org/T84923 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T90109: BlazeGraph Finalization: Zookeeper

2015-03-02 Thread GWicke
GWicke added a subscriber: GWicke. TASK DETAIL https://phabricator.wikimedia.org/T90109 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Up For Grabs] T85181: Investigate design public API, possibly using MQL

2015-02-20 Thread GWicke
GWicke placed this task up for grabs. TASK DETAIL https://phabricator.wikimedia.org/T85181 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Edited] T84923: Reliable publish / subscribe event bus

2015-02-03 Thread GWicke
GWicke edited the task description. TASK DETAIL https://phabricator.wikimedia.org/T84923 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate design public API, possibly using MQL

2015-01-25 Thread GWicke
GWicke added a comment. In https://phabricator.wikimedia.org/T85181#971763, @JanZerebecki wrote: No, just a list of things that were brought up, when people talked about querying Wikidata. I don't like Ask. Only ever used Ask and Cypher. I think the way Cypher goes about the problem is neat

[Wikidata-bugs] [Maniphest] [Edited] T85181: Investigate design public API, possibly using MQL

2015-01-25 Thread GWicke
GWicke edited the task description. TASK DETAIL https://phabricator.wikimedia.org/T85181 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Edited] T85181: Investigate design public API, possibly using MQL

2015-01-25 Thread GWicke
GWicke edited the task description. TASK DETAIL https://phabricator.wikimedia.org/T85181 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T84923: Reliable publish / subscribe event bus

2015-01-24 Thread GWicke
GWicke added a comment. In https://phabricator.wikimedia.org/T84923#968636, @JanZerebecki wrote: http://www.fedmsg.com might fit this need. It is used/developed by Fedora and Debian people and is a federated, reliable message bus with history of cryptographically authenticated json messages

[Wikidata-bugs] [Maniphest] [Updated] T85159: Deploy a Wikidata complex query service into production

2015-01-12 Thread GWicke
GWicke added a blocking task: T85181: Investigate design public API, possibly using MQL. TASK DETAIL https://phabricator.wikimedia.org/T85159 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] [Commented On] T86561: Wikidata Query Service hardware

2015-01-12 Thread GWicke
GWicke added a comment. In https://phabricator.wikimedia.org/T86561#971491, @JanZerebecki wrote: Do we want to have the Cassandra and Titan nodes be in the same rack as I assume that query performance is very latency sensitive? I don't think that there is a huge enough difference in latency

[Wikidata-bugs] [Maniphest] [Changed CC] T86016: Simple Query Language against both Titan and SQL

2015-01-12 Thread GWicke
GWicke added a subscriber: GWicke. GWicke added a comment. I would actually argue for starting with a simple subset of whatever we plan to use in the longer term (MQL?), and then expand from there. Lets not expose n different interfaces for the same thing just because we can. Doing so would

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate design public API, possibly using MQL

2015-01-12 Thread GWicke
GWicke added a comment. @janzerebecki: Are there any in that list that you would prefer over MQL? If so, for which reasons? TASK DETAIL https://phabricator.wikimedia.org/T85181 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username

[Wikidata-bugs] [Maniphest] [Changed CC] T85181: Investigate design public API, possibly using MQL

2015-01-07 Thread GWicke
GWicke added a subscriber: Manybubbles. TASK DETAIL https://phabricator.wikimedia.org/T85181 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T84923: Reliable publish / subscribe event bus

2015-01-07 Thread GWicke
GWicke added a comment. @halfak has written up very similar ideas at https://meta.wikimedia.org/wiki/Research:MediaWiki_events:_a_generalized_public_event_datasource TASK DETAIL https://phabricator.wikimedia.org/T84923 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close

[Wikidata-bugs] [Maniphest] [Edited] T84923: Reliable publish / subscribe event bus

2015-01-07 Thread GWicke
GWicke edited the task description. TASK DETAIL https://phabricator.wikimedia.org/T84923 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Edited] T84923: Reliable publish / subscribe event bus

2015-01-07 Thread GWicke
GWicke edited the task description. TASK DETAIL https://phabricator.wikimedia.org/T84923 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Updated] T84923: Reliable publish / subscribe event bus

2015-01-07 Thread GWicke
GWicke added a project: Analytics. GWicke added a comment. In https://phabricator.wikimedia.org/T84923#961622, @bd808 wrote: can support large delays (order of days) for individual consumers Do you have a strong use case to support this need? Yes. Hosts can go down for multiple days

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate design public API, possibly using MQL

2015-01-07 Thread GWicke
GWicke added a comment. In https://phabricator.wikimedia.org/T85181#958820, @JeroenDeDauw wrote: As far as I can tell, decisions are currently made based on the use case the WMF has, rather than also holding the Wkidata plans into account. I'm getting this impression because I'm not seeing

[Wikidata-bugs] [Maniphest] [Edited] T85181: Investigate design public API, possibly using MQL

2015-01-06 Thread GWicke
GWicke edited the task description. GWicke removed a subscriber: Manybubbles. TASK DETAIL https://phabricator.wikimedia.org/T85181 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] [Commented On] T84923: Reliable publish / subscribe event bus

2015-01-05 Thread GWicke
GWicke added a comment. In https://phabricator.wikimedia.org/T84923#943464, @JanZerebecki wrote: https://www.wikidata.org/w/api.php?action=helpmodules=wbgetentities and recent changes, see also https://phabricator.wikimedia.org/T85103 and https://phabricator.wikimedia.org/T85100

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate design public API, possibly using MQL

2014-12-29 Thread GWicke
GWicke added a comment. @jeroendedauw: I think it's possible to implement a simple query interface (such as the current wikibase API, as far as I understand it) on top of a more powerful one such as the one we are discussing here. The reverse is not necessarily true. What is your view

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-24 Thread GWicke
GWicke added a comment. In https://phabricator.wikimedia.org/T76373#943426, @Smalyshev wrote: Proposed storage format for dates: 1. Dates are stored as long signed integers, representing number of seconds since 1970-01-01 00:00:00 UTC. 2. This gives us range of 292 bln years http

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-24 Thread GWicke
GWicke added a comment. In https://phabricator.wikimedia.org/T76373#943454, @Smalyshev wrote: seconds(292M-12-31T23:59:59) seconds(292M+1) It's pretty likely that there were more than 356 leap years between -292M and 1970, so it's very possible for the years to be non-monotonic if we don't

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-24 Thread GWicke
GWicke added a comment. In https://phabricator.wikimedia.org/T76373#943459, @Smalyshev wrote: For dates beyond real Gregorian calendar, the values more precise than years have little meaning anyway, so I don't think it matters too much as long as comparisons and lookups (i.e. which Greek

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-23 Thread GWicke
GWicke added a comment. In https://phabricator.wikimedia.org/T76373#941917, @Manybubbles wrote: I'd spin up a new one - probably just on a single node. I think in the long run we probably can run this on the production search cluster but for now lets keep it off just in case it does

[Wikidata-bugs] [Maniphest] [Commented On] T84923: Reliable publish / subscribe event bus

2014-12-23 Thread GWicke
GWicke added a comment. In https://phabricator.wikimedia.org/T84923#940155, @JanZerebecki wrote: The nature of these event type candidates is such that they are changes with a log existing at the provider. Wikidata might be the exeception here. Most other events are not available

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate design public API, possibly using MQL

2014-12-23 Thread GWicke
GWicke added a comment. In https://phabricator.wikimedia.org/T85181#940893, @Smalyshev wrote: Are we considering supporting WDQ API mini-language as the option for the queries or it's not a viable option? The problem I see with the WDQ language is the need to perform error-prone custom

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-23 Thread GWicke
GWicke added a comment. Another fun article for dates: http://en.wikipedia.org/wiki/Timeline_of_the_far_future TASK DETAIL https://phabricator.wikimedia.org/T76373 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL

[Wikidata-bugs] [Maniphest] [Edited] T85181: Investigate design public API, possibly using MQL

2014-12-23 Thread GWicke
GWicke edited the task description. TASK DETAIL https://phabricator.wikimedia.org/T85181 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Created] T85181: Investigate design public API, possibly using MQL

2014-12-22 Thread GWicke
GWicke created this task. GWicke added a subscriber: GWicke. GWicke added a project: wikidata-query-service. TASK DESCRIPTION We need to investigate design a public query API. Goals for this API are: - Easy to build queries with a variety of clients (probably JSON) without dealing

[Wikidata-bugs] [Maniphest] [Changed CC] T85181: Investigate design public API, possibly using MQL

2014-12-22 Thread GWicke
GWicke removed a subscriber: Aklapper. GWicke added a project: Services. GWicke set Security to none. TASK DETAIL https://phabricator.wikimedia.org/T85181 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-22 Thread GWicke
GWicke added a comment. Re performance and indexing, from a mail thread: Earlier today Stas I were looking a bit into what is happening behind the scenes in some of the slower queries like https://www.mediawiki.org/wiki/Wikibase/Indexing/Benchmarks

[Wikidata-bugs] [Maniphest] [Created] T84923: Reliable publish / subscribe event bus

2014-12-18 Thread GWicke
GWicke created this task. GWicke added a subscriber: GWicke. GWicke added a project: wikidata-query-service. GWicke changed Security from none to none. TASK DESCRIPTION We need a reliable way to distribute a variety of update events emitted from MediaWiki core (and other services) to various

[Wikidata-bugs] [Maniphest] [Edited] T84923: Reliable publish / subscribe event bus

2014-12-18 Thread GWicke
GWicke edited the task description. TASK DETAIL https://phabricator.wikimedia.org/T84923 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Triaged] T84923: Reliable publish / subscribe event bus

2014-12-18 Thread GWicke
GWicke triaged this task as Normal priority. TASK DETAIL https://phabricator.wikimedia.org/T84923 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Commented On] T77897: Test host for Titan

2014-12-17 Thread GWicke
GWicke added a comment. @smalyshev started testing yesterday. Things are working fine so far. Thanks! TASK DETAIL https://phabricator.wikimedia.org/T77897 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Edited] T77897: Test host for Titan

2014-12-11 Thread GWicke
GWicke edited the task description. TASK DETAIL https://phabricator.wikimedia.org/T77897 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T76372: Evaluate TinkerPop GraphSail frontend

2014-12-11 Thread GWicke
GWicke added a comment. Might be worth starting with Tinkerpop3 https://github.com/tinkerpop/tinkerpop3/blob/master/CHANGELOG.asciidoc. TASK DETAIL https://phabricator.wikimedia.org/T76372 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign

[Wikidata-bugs] [Maniphest] [Edited] T76372: Evaluate TinkerPop GraphSail frontend

2014-12-11 Thread GWicke
GWicke edited the task description. TASK DETAIL https://phabricator.wikimedia.org/T76372 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Changed CC] T75460: Sane Lua label access in non-content languages

2014-12-09 Thread GWicke
GWicke added a subscriber: GWicke. GWicke added a comment. Please make sure that the language(s) used are static for anything in the content of a page. In particular, it should not depend on the user interface language, as that would impose a heavy performance penalty on the user. TASK DETAIL

[Wikidata-bugs] [Maniphest] [Changed CC] T77897: Test host for Titan

2014-12-08 Thread GWicke
GWicke added a subscriber: wikidata-query-service. TASK DETAIL https://phabricator.wikimedia.org/T77897 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. To: GWicke Cc: Aklapper, GWicke, mark, RobH, Smalyshev, Eloquence, aude

[Wikidata-bugs] [Maniphest] [Edited] T77897: Test host for Titan

2014-12-08 Thread GWicke
GWicke edited the task description. TASK DETAIL https://phabricator.wikimedia.org/T77897 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. To: GWicke Cc: Aklapper, GWicke, mark, RobH, Smalyshev, Eloquence, aude, Tobi_WMDE_SW

[Wikidata-bugs] [Maniphest] [Commented On] T76509: Figure out why Titan gets OOM when importing data

2014-12-02 Thread GWicke
GWicke added a comment. Is 48k transaction records all transactions committed up to that point? TASK DETAIL https://phabricator.wikimedia.org/T76509 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. To: Smalyshev, GWicke Cc

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-02 Thread GWicke
GWicke added a comment. ! In T76373#799513, @Smalyshev wrote: Technical issues: # On import, titan sometimes slows down and gets into GC loops. # On querying, for vertices with a lot of edges (such as `wd(Q5).in(P31)`, i.e. humans, titan produces a backend exception: ``` Caused

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-02 Thread GWicke
GWicke added a comment. ! In T76373#802449, @Smalyshev wrote: Note, that running Titan with Cassandra embedded requires GC tuning. While embedded Cassandra can provide lower latency query answering, its GC behavior under load is less predictable. Yeah, agreed. GC scaling limits are kind

<    1   2