> > I do not think API should support the case you described with gaplimit=1, > > because that fundamentally breaks the original API goal of "get data > about > > many pages with lots of elements on them in one request". > > Oh? I thought the goal of the API was to provide a machine-usable > interface to MediaWiki so people don't have to screen-scrape the HTML > pages, which alleviates the worry about whether changes to the user > interface are going to break screen-scrapers. I never knew it was all > about *bulk* data access *only*. >
Brad, API has a clear goal (at least that was my goal when I wrote it), to provide access to all the functionality a wiki UI offers. The point here is how *easy* and at the same time *efficient* API is. The continue in the past few years has gotten too complex for client libraries to use efficiently to combine multiple requests. The example you gave seems very uncommon, and it can easily be solved with making one extra API call to get a list of articles first - which in this case of O(N) would only make it O(N+1) -- still O(N). That's why I don't think we should even go into the "|next" ability - it is very rare it will be used and can be done easily by another call without generator. See below on iteration point. > > But even if we do find compelling reasons to include that, for the > advanced > > scenario "skip subquery and follow on with the generator" it might make > > sense to introduce appendable "|next" value keyword gapcontinue=A|next > > How do things decide whether "foocontinue=A|next" is saying "the next > foocontinue after A" or really means "A|next"? For example, > > https://en.wiktionary.org/w/api.php?action=query&titles=secundus&prop=links&pllimit=1&plcontinue=46486|0|nautical > currently returns plcontinue "46486|0|next". > > Or are you proposing every module be individually coded to recognize > this "|next"? > > Again, unless there are good usage scenarios to keep this, I don't think we ever need this "|next" feature - it was a "just in case" idea, which I doubt we will need. > > Ideally all > > "continue" values should be joined into a single "query-continue = > > magic-value" of no interesting user-passable properties. > > So clients can make absolutely no decisions about processing the data > they get back? No thanks. > When you make a SQL query to the server, you don't get to control the "continue" process. You can stop and make another query with different initial parameters. Same goes for iterating through a collection - none of the programming languages offering IEnumerable have stream control functionality - too complicated without clear benefits. API can be seen as a stream returning server - with some "continue" parameter. You don't like result - you do another query. That's how you control it. Documenting the "continue" properties is a sure way to over-complicate API usage and remove server's ability to optimize the process in the future, without adding any significant benefit. Why not propose adding something like that as an option, instead of > trying to force everyone to do things your way? Say have a parameter > dumbcontinue=1 that replaces query-continue with > > > <query-dumb-continue>prop=links|categories&plcontinue=...&clcontinue=...&wlstart=...&allmessages=...</query-dumb-continue> > > Entirely compatible. > This might be a good solution. Need community feedback on this. > > >> IMO, if a client wants to ensure it has complete results for any page > >> objects in the result, it should just process all of the prop > >> continuation parameters to completion. > > > > The result set might be huge. It wouldn't be nice to have a 12GB x64 only > > client lib requirement :) > > Then use a smaller limit on your generator. And don't do this for > prop=revisions&rvprop=content. > My bad, didn't thee the "prop" continuation, thought you meant all of them. Lastly, lets try keeping sarcasm to the minimal with a technical discussion. We have Wikipedia talk pages for that.
_______________________________________________ Mediawiki-api mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mediawiki-api
