Re: [Wikitech-l] The never-dying topic: category intersection

2009-01-31 Thread Roan Kattouw
Marcus Buck schreef: I just read the last category intersection discussion from December to see, what's the latest state of it. While doing that, I saw, that the last message in that thread was this post from Roan Kattouw, providing his extension. Oddly, nobody reacted on it. After 65 posts

Re: [Wikitech-l] The never-dying topic: category intersection

2009-01-31 Thread David Gerard
2009/1/31 Roan Kattouw roan.katt...@home.nl: Marcus Buck schreef: I just read the last category intersection discussion from December to see, what's the latest state of it. While doing that, I saw, that the last message in that thread was this post from Roan Kattouw, providing his extension.

Re: [Wikitech-l] The never-dying topic: category intersection

2009-01-31 Thread Roan Kattouw
David Gerard schreef: Win! So what's in the way of this going live on Wikimedia? (Commons first? As I said before, the extension was written especially for MixesDB, and has all kinds of features WMF wikis don't need or don't want for performance reasons. Also, the UI is pretty crude (note

Re: [Wikitech-l] The never-dying topic: category intersection

2009-01-30 Thread Marcus Buck
I just read the last category intersection discussion from December to see, what's the latest state of it. While doing that, I saw, that the last message in that thread was this post from Roan Kattouw, providing his extension. Oddly, nobody reacted on it. After 65 posts on that thread somebody

Re: [Wikitech-l] The never-dying topic: category intersection

2009-01-30 Thread Mohamed Magdy
On Fri, Jan 30, 2009 at 10:04 PM, Marcus Buck w...@marcusbuck.org wrote: I just read the last category intersection discussion fromsnip After 65 posts on that thread somebody posted a solution and nobody reacted. They got exhausted? (p m, c r) -- --alnokta

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of three)

2008-12-08 Thread Ilmari Karonen
Gregory Maxwell wrote: Because adding the parents produces non-sense results because categorization is a flawed concept except at the most fuzzy and course levels: Reality doesn't fit into neat nested boxes (not even the N-dimensional ones created by multiple parentage). The two primary

Re: [Wikitech-l] The never-dying topic: category intersection

2008-12-08 Thread Aryeh Gregor
On Mon, Dec 8, 2008 at 5:38 PM, Aerik Sylvan [EMAIL PROTECTED] wrote: Okay, it's not quite done, and it's still really crude, but starting to take shape - I've got some basic intersections functionality running on Wikidweb - I hacked skin.php and added links to the special intersections page.

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of three)

2008-12-06 Thread Platonides
David Gerard wrote: 2008/12/4 Tim Landscheidt: Add to that the maintenance costs because you would want to ensure that if someone who is not aware of the concept of atomic categories adds a [[Category:Manhattan]] to something he adds [[Category:New York]], [[Category:East Coast of the

Re: [Wikitech-l] The never-dying topic: category intersection

2008-12-06 Thread Aryeh Gregor
On Sat, Dec 6, 2008 at 4:49 PM, [EMAIL PROTECTED] wrote: Okay, that's a green light if I ever saw one, awesome. So let's create a a categorysearch myisam table, stick all the categories in it, set up hooks to maintain it, and implement the fulltext index solution. We'll use a special page

Re: [Wikitech-l] The never-dying topic: category intersection

2008-12-04 Thread Magnus Manske
On Thu, Dec 4, 2008 at 7:39 AM, Ilmari Karonen [EMAIL PROTECTED] wrote: A _minimal_ solution would be simply to present a link to the intersection of _all_ the categories (which might well have only one page on it) and let the user broaden the intersection from there. Even better if this can

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of three)

2008-12-04 Thread Tim Landscheidt
Aryeh Gregor [EMAIL PROTECTED] wrote: I'd like for you to be right. But switching from the present category system to atomic categories is not as straight forward as having a few bots run over all existing cats. Of course, humans would have to manually specify which new categories each old

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of three)

2008-12-04 Thread David Gerard
2008/12/4 Tim Landscheidt [EMAIL PROTECTED]: Add to that the maintenance costs because you would want to ensure that if someone who is not aware of the concept of atomic categories adds a [[Category:Manhattan]] to something he adds [[Category:New York]], [[Category:East Coast of the United

Re: [Wikitech-l] The never-dying topic: category intersection

2008-12-04 Thread Aerik Sylvan
Magnus - I checked out your tool, but it looks like you're using a query against the categorylinks table? Have you played with setting up a new table for categories and fulltext indexing it? Use group_concat to get all of a pages categories into one field, then create a fulltext index on that

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of three)

2008-12-04 Thread Aryeh Gregor
On Wed, Dec 3, 2008 at 7:12 PM, Daniel Schwen [EMAIL PROTECTED] wrote: Uhm, yeah.. except that intersection of atomic categories are not vaporware. We had proofs of concept for that and the interest was marginal. Vaporware with proofs of concept is still vaporware. The definition of vaporware

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of three)

2008-12-04 Thread David Gerard
2008/12/4 Aryeh Gregor [EMAIL PROTECTED]: On Wed, Dec 3, 2008 at 8:16 PM, Gregory Maxwell [EMAIL PROTECTED] wrote: With a JS hack I had my tool integrated to the site. The AJAX calls went to the toolserver, but as far as the users could see it was running on the site. No one cared: It didn't

Re: [Wikitech-l] The never-dying topic: category intersection

2008-12-04 Thread Aryeh Gregor
On Wed, Dec 3, 2008 at 12:37 PM, Aerik Sylvan [EMAIL PROTECTED] wrote: But it sounds like maybe those of us who'd like to see this happen should discuss a UI (or several) for it. No, someone should *write* a UI. It should be written and added to the software. If it's subpar, fine, it can be

Re: [Wikitech-l] The never-dying topic: category intersection

2008-12-04 Thread Aerik
Re-posting, the original seems to have been lost in cyberspace: Magnus - I checked out your tool, but it looks like you're using a query against the categorylinks table? Have you played with setting up a new table for categories and fulltext indexing it? Use group_concat to get all of a

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of three)

2008-12-03 Thread Roan Kattouw
We had a pretty lengthy discussion about this before the summer, and the consensus seemed to be that a fulltext-based approach looked most viable. I actually wrote an extension that does that, and promised to release it soon; that was quite a few months ago, and I never got around to it. I'll

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of three)

2008-12-03 Thread Daniel Schwen
We had a pretty lengthy discussion about this before the summer, and the consensus seemed to be that a fulltext-based approach looked most viable. So how does this take care of deep indexing non-atomic categories? =How will this extension be even remotely useful for let's say commons? This

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of three)

2008-12-03 Thread David Gerard
2008/12/3 Daniel Schwen [EMAIL PROTECTED]: I'm sure this thread will die out soon. Half of the participants will again be soothed by the promise of some easy solution just barely beyond the horizon, while the half that realizes that said solution _cannot possibly work_ without a radical

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of three)

2008-12-03 Thread Roan Kattouw
Daniel Schwen schreef: We had a pretty lengthy discussion about this before the summer, and the consensus seemed to be that a fulltext-based approach looked most viable. So how does this take care of deep indexing non-atomic categories? Err.. what? Please explain what you mean by

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of three)

2008-12-03 Thread Aryeh Gregor
On Wed, Dec 3, 2008 at 10:59 AM, Daniel Schwen [EMAIL PROTECTED] wrote: So how does this take care of deep indexing non-atomic categories? =How will this extension be even remotely useful for let's say commons? That's a social problem, and so of secondary importance. Once a technical mechanism

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of three)

2008-12-03 Thread Daniel Schwen
the other useful technical innovations that get introduced. All it would take is running some bots for a while to switch to the better system, not a big cost for a large wiki like Commons with plenty of bot operators. I'd like for you to be right. But switching from the present category

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that)

2008-12-03 Thread David Gerard
2008/12/3 Aerik [EMAIL PROTECTED]: I'm with you - we've shown feasibility in large datasets with a lucene based approach, and I think we need to roll it out and test it with real users on real data. We need a new lucene index and a user interface (needs to be defined) suitable for average

Re: [Wikitech-l] The never-dying topic: category intersection

2008-12-03 Thread Gregory Maxwell
On Wed, Dec 3, 2008 at 12:37 PM, Aerik Sylvan [EMAIL PROTECTED] wrote: [snip] But it sounds like maybe those of us who'd like to see this happen should discuss a UI (or several) for it. I was thinking the most intuitive interface was a sort of browse type function, where for any given group

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of three)

2008-12-03 Thread Aryeh Gregor
On Wed, Dec 3, 2008 at 11:43 AM, Daniel Schwen [EMAIL PROTECTED] wrote: I'd like for you to be right. But switching from the present category system to atomic categories is not as straight forward as having a few bots run over all existing cats. Of course, humans would have to manually specify

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of three)

2008-12-03 Thread Daniel Schwen
how things are categorized. As long as category intersections remain vaporware, there's no incentive to change. A technical fait accompli will bring about change. Uhm, yeah.. except that intersection of atomic categories are not vaporware. We had proofs of concept for that and the interest

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of three)

2008-12-03 Thread Gregory Maxwell
On Wed, Dec 3, 2008 at 8:12 PM, David Gerard [EMAIL PROTECTED] wrote: The last time will be when there's a feature end-users can use without going off to the toolserver. With a JS hack I had my tool integrated to the site. The AJAX calls went to the toolserver, but as far as the users could see

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of three)

2008-12-03 Thread Ilmari Karonen
Gregory Maxwell wrote: With a JS hack I had my tool integrated to the site. The AJAX calls went to the toolserver, but as far as the users could see it was running on the site. No one cared: It didn't produce useful results because of how categories are used, and when I suggested changing

Re: [Wikitech-l] The never-dying topic: category intersection

2008-12-03 Thread Ilmari Karonen
Gregory Maxwell wrote: So an interface I had that was really pleasing was that I asked the database to find a random subset of the results, which it could do quickly, (or I used the whole results if the initial query contained them) and I found the set of categories which maximally bisected

Re: [Wikitech-l] The never-dying topic: category intersection

2008-12-03 Thread Ilmari Karonen
Aerik Sylvan wrote: But it sounds like maybe those of us who'd like to see this happen should discuss a UI (or several) for it. I was thinking the most intuitive interface was a sort of browse type function, where for any given group of categories (could just be one category), you have two

Re: [Wikitech-l] The never-dying topic: category intersection

2008-12-02 Thread Magnus Manske
OTOH, looking for images on Commons in GFDL and Buildings in Berlin took ~2min. Might be the giant GFDL category, or the toolserver, or both. I'll try to fiddle with it some more utilising cat_pages/cat_files. Hah! By using small categories first, then restricting possible page_ids in the

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that)

2008-12-02 Thread Daniel Schwen
curse of never getting anything into the core might strike ;-) Doesn't he have commit access? True, but is gems never seem to get enabled... -- [[en:User:Dschwen]] [[de:Benutzer:Dschwen]] [[commons:User:Dschwen]] ___ Wikitech-l mailing list

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that .. to the power of two)

2008-12-02 Thread Daniel Schwen
Perhaps just category intersection isn't enough. I was thinking about a tool that would allow intersection of various article data, including the categories. For example, suppose that I am maintaining [[1991 in art]]. I would want to find all articles that link to [[1991]] and are in a

Re: [Wikitech-l] The never-dying topic: category intersection

2008-12-02 Thread Bryan Tong Minh
On Tue, Dec 2, 2008 at 5:40 PM, Magnus Manske [EMAIL PROTECTED] wrote: * Is there a machine-readable interface for this? One that will return 5K hits without screenscraping? api.php?list=search? ___ Wikitech-l mailing list

Re: [Wikitech-l] The never-dying topic: category intersection (been there done that)

2008-12-02 Thread Lars Aronsson
Daniel Schwen wrote: I find it a little frustrating that this wheel gets reinvented so often. My tool was used a couple of times after I posted it, and now as maybe one user per day (from a quick glance at the Users of the Swedish Wikipedia are increasingly starting to use Duesentrieb's