[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 Mark A. Hershberger m...@everybody.org changed: What|Removed |Added Priority|High|Highest -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #48 from Bawolff bawolff...@gmail.com 2011-03-17 15:37:30 UTC --- The other thing I wanted to mention is the order parameter (order ascending or descending). Do we really need that. Is there any valid use case for wanting an rss feed of the earliest 20 articles? -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #47 from Bawolff bawolff...@gmail.com 2011-03-17 04:51:39 UTC --- I poked at this last weekend, and a little bit this week. I think most of the issues brought up in previous reviews have been addressed, and that this might be ready for another review. Some remaining issues that I'm not sure and would really like some input on: Major ones: *Caching. Currently it caches the feed in memcache for up to a maximum of 12 hours (It also does checks to see if the feed is still valid, so the moment someone adds something to the relevant categories the cache is no longer considered valid, so everything is nice) **Is 12 hours an appropriate max time limit (Given that cache entries are checked that they're still good before being used). **Should we be sending maxage/s-maxage headers to cache on the client side and/or in squids. (Currently its sent with cache-control private i believe) ***If we did do caching on that level, the longest (s)maxage that would be acceptable would be very short (like half an hour) since we want these things to be up to date **Should cache times differ between Sitemap and RSS/ATOM formats. *It does a check to see if cl_timestamp is null, and returns the current timestamp if so. This seems very wrong. I'm not sure what circumstances a null cl_timestamp can happen (There was one but it looked accidental and is now removed). I'm not sure if checking for a null cl_timestamp and throwing an exception if that happens is the right way to go. (Is there any circumstance cl_timestamp can be null?) *The flagged revision support is flaky. This may be a valid concern. Its outside of my knowledge of flagged revs what the best approach is. It appears that DPL also detects dpl in the same way. Minor ones: *The default parameters could be tweaked. I'm not sure what the ideal defaults would be. A lot depends on which of the use-cases we want to optimize for. *[bikeshed issue] This extension does rss feeds based on category membership. That feature may be of general interest to some people totally independent from the Sitemap aspect. Perhaps a more general name should be chosen like IntersectionFeed. But thats really not important. Response to some of the review comments: *It's doing DIY category intersection That is sort of the feature request. (The intersection part. The DIY part I'm not sure what was meant) *Also, at least one of the two sort modes is likely to be very DB-inefficient. Can't tell for sure cause I can't read the query too well, the code is not very pretty and the query is complex and very dynamic The lastedit (which is really order by last_touched, which doesn't correspond to edit) sort order is rather scary (I'm not familiar with db scaling and stuff ery much, but I think it does a full table scan of the page table, and a file sort which is very very bad from my understanding). Anyways, lastedit is not very important to the use-case and could be removed. the Categoryadd sort order is not as bad (Still not great). It might filesort if the smallest category specified is not the first category specified. It is no worst than the already deployed intersection (DynamicPageList) extension (Which has many scary queries ;) Actually its the same query as intersection. *General hating on direct output/echo/$wgOut-disable(). This is done the same way as its done in core. In ideal world, this extension might be better done as an api module, but I don't think thats a critical issue. Most special pages that output an xml document call $wgOut-disable(). All the feed stuff in core calls $wgOut-disable() and outputs with echo. *is there a way to integrate multiple site maps? We are not anywhere near outputting more than 1000 articles over a 2 day period. So I don't feel this is a near or long term concern. Thanks, -bawolff -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #43 from Jason Giglio g...@gigstaggart.com 2011-03-09 16:37:18 UTC --- The code style issues raised before should be addressed now. SQL is parameterized, and the output is using an XML library. The only remaining problem is that there remains a large overlap between this and DPL. Without a clear statement of what functionality can be removed and what is required, no one can go forward with stripping out unnecessary DPL-based functions. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #44 from Reedy s...@reedyboy.net 2011-03-09 17:01:31 UTC --- It's annoying. Though, if it's not an easy task to strip out the duplication, IMHO, it's not really blocker for deployment, as long as the code is up to standard -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #45 from Reedy s...@reedyboy.net 2011-03-09 17:15:13 UTC --- Though, the echo's need tidying up... -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #46 from Jason Giglio g...@gigstaggart.com 2011-03-09 19:18:47 UTC --- I don't think we ever want to output the full skinned html here. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #42 from Amgine amgine.sae...@gmail.com 2011-03-09 05:48:39 UTC --- There is a renewed need for this functionality on en.WN. Is there any way WMF devs could take a look at this bug? -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #41 from Mark A. Hershberger m...@everybody.org 2011-02-11 22:22:47 UTC --- Jason: Sorry I didn't see your update earlier. The most recent changes I see are from you about a month ago (2010-12-31). Let me know if those are your most recent changes and I or someone else will look at what more needs to happen. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #40 from Jason Giglio g...@gigstaggart.com 2011-02-01 22:34:40 UTC --- So at this point, most of the major code style problems have been addressed. There's still a lot of overlap between this and DPL though. I don't really have a clear way forward because I don't know which DPL features we want to keep in this and which ones can be stripped out. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #37 from Reedy s...@reedyboy.net 2010-11-29 08:35:43 UTC --- For an easy changelog, please see http://www.mediawiki.org/w/index.php?path=/trunk/extensions/GoogleNewsSitemaptitle=Special:Code/MediaWiki/path There hasn't been much functional changes (bar people going through and removing stuff while doing other general cleanup along the way) I split the code files A version of Mark Hershbergers review above, is also on Code Review http://www.mediawiki.org/wiki/Special:Code/MediaWiki/76544#c11017 -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #38 from Jason Giglio g...@gigstaggart.com 2010-11-29 15:02:45 UTC --- I think the easiest thing would be to check my copy in losing Reedy's changes. Mine seems to be further along, and it should include all the whitespace fixing that Reedy did as well, since we duplicated effort there. I don't care about the naming if the SVN name makes more sense. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #39 from Reedy s...@reedyboy.net 2010-11-29 15:30:08 UTC --- Feel free to put it over the top -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #36 from Jason Giglio g...@gigstaggart.com 2010-11-29 05:12:58 UTC --- I know I merged two of the files. The renaming, I don't remember if I did it or if it's happened in SVN since my development started. As far as my changes, mostly from memory: Started to clean up the parameter handling to use wgrequest instead of the home rolled thing. This means the documentation is going to be wrong about the naked options (i.e. you need to use var=option instead of just /option) Major cleanup of the incorrect indent levels. All spaces should be tab chars now per the MW standards. Merged the XML generation into the main file. Still need to move that to XMLWriter or something similar, but the structure of what it's doing is similar now. Moved some of it out of a class where it didn't really belong. Cut out some sections of cruft, and simplified many of the deeply nested if statements to use early return logic. Broke the notcategories handling, I don't remember why but there was a good reason. I think it was part of the params conversion. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #33 from Kim Bruning k...@bruning.xs4all.nl 2010-11-25 01:27:56 UTC --- (In reply to comment #32) Kim, Could you commit that code? Sure. One small snag. Going through the steps to commit, While attempting to merge and compare with code that was already there, the location: http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/GNSM no longer appears to exist. Further, the google query: GNSM site:http://svn.wikimedia.org/ returns no hits. Has this been deleted or moved? What is the current location? -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 Reedy s...@reedyboy.net changed: What|Removed |Added CC||s...@reedyboy.net --- Comment #34 from Reedy s...@reedyboy.net 2010-11-25 01:28:56 UTC --- http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/GoogleNewsSitemap/ -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #35 from Kim Bruning k...@bruning.xs4all.nl 2010-11-25 01:43:31 UTC --- Ok, there's some pretty major discrepancies. All the files have been renamed, for starters. I'm going to have to take a better look this coming week. ^^;; -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #32 from Mark A. Hershberger m...@everybody.org 2010-11-23 18:33:33 UTC --- Kim, Could you commit that code? -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #31 from Kim Bruning k...@bruning.xs4all.nl 2010-11-15 04:53:19 UTC --- And I have now found the partially refactored version. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #30 from Jason Giglio g...@gigstaggart.com 2010-11-12 19:03:42 UTC --- Thanks Mark. That will be helpful. As an update, Kim and I are trying to find my partially refactored version. I'm sure we have a copy somewhere. :) -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #28 from Mark A. Hershberger m...@everybody.org 2010-11-12 03:22:36 UTC --- Copying this from my notes, will put on CR if it doesn't format here. * From RK: ** Detection for FlaggedRevs presence is fragile and I'm not sure it'll even work any more ** Also, there's an arbitrary, unlimited number of joins against categorylinks, that's a no-go (has potential to make the DB servers sweat) ** It's doing DIY category intersection ** Also, at least one of the two sort modes is likely to be very DB-inefficient. Can't tell for sure cause I can't read the query too well, the code is not very pretty and the query is complex and very dynamic ** Amgine: That's fine; if you can give me a general idea of what the query is gonna look like, I can poke at it (to optimize the above) ** Amgine: What I've given you just now should keep a dev busy for a few hours already. Am of course available for general help, but don't think it'd be useful to do an exhaustive review before those things have been taken care of * From me: ** Echo?!?! REALLY? Don't do it. That and direct output (e.g. closing ?) should not be done *** To read: http://www.mediawiki.org/wiki/Manual:Special_pages#OutputPage.php ** Don't try to create XML yourself, it doesn't work. Use XMLWriter or DOM to create XML. That way you don't have do xmlEncode(). ** Check http://www.google.com/support/webmasters/bin/answer.py?hl=enanswer=74288 it looks like there are some things in the GNSM format that should be handled. *** should limit number of urls (1000?) *** is there a way to integrate multiple site maps *** document robots.txt? ** Hate the use of params as member variable. *** You should just copy things from wgRequest after checking them. *** Having a member variable named params seems like it is the beginning of confusion. *** Also put wgRequest directly in the foreach, or at the very least closer to the point of use to avoid confusion with the others. *** If you don't intend for the result of $wgRequest to be used, then it shouldn't be in $params at all. ** code indention looks lacking. ** Incorrect use of wfEmptyMsg(??) not sure about this but the function def doesn't match function declaration in GlobalFunctions while examples elsewhere abound. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #29 from Mark A. Hershberger m...@everybody.org 2010-11-12 03:45:15 UTC --- See r76544 for a easier to read review. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #27 from Mark A. Hershberger m...@everybody.org 2010-11-09 18:49:09 UTC --- Kim, Jason, I'm contacting you via email to see if we can work with you to get this fixed up and deployed. Mark. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #26 from Kim Bruning k...@bruning.xs4all.nl 2010-11-07 00:43:10 UTC --- (In reply to comment #25) I will try to find some time to look at it again. I had a version that was partially rewritten. Cool, Can you show me? -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 Rob Lanphier ro...@wikimedia.org changed: What|Removed |Added CC||m...@everybody.org, ||ro...@wikimedia.org AssignedTo|ro...@wikimedia.org |wikibug...@lists.wikimedia. ||org --- Comment #19 from Rob Lanphier ro...@wikimedia.org 2010-10-26 16:17:54 UTC --- I'm going to unassign this to signal that we'd like volunteer help on this. Mark Hershberger and I will both remain on the cc list, and can help a volunteer get this through the process. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #20 from Amgine amgine.sae...@gmail.com 2010-10-26 18:25:00 UTC --- Let me see, written by volunteers, begged for feedback for months, rewritten by staff, left on hold for review for 10 months... What do you want, Rob Lanphier? -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #21 from Rob Lanphier ro...@wikimedia.org 2010-10-26 18:44:14 UTC --- Amgine, this extension only came to my attention this weekend. I think you'll do best by asking nicely for help on wikitech-l. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #22 from Amgine amgine.sae...@gmail.com 2010-10-26 18:54:32 UTC --- No thanks. I've washed my hands of it. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 Kim Bruning k...@bruning.xs4all.nl changed: What|Removed |Added Priority|Normal |High --- Comment #23 from Kim Bruning k...@bruning.xs4all.nl 2010-10-26 19:22:36 UTC --- This extension is awaiting review, approval, and roll-out to en.wikinews, with what was critical blocker priority for the en.wikinews community at the time. I believe there have been some rubber bands, sellotape and matchsticks holding things together in the mean time. I understand that we have some capacity to review and roll out? I would like to get this rolled out as quickly as possible. What would you like me to do to proceed? -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #24 from Rob Lanphier ro...@wikimedia.org 2010-10-26 19:45:41 UTC --- Hi Kim, this is being discussed on IRC on #mediawiki: http://toolserver.org/~mwbot/logs/%23mediawiki/20101026.txt Roan had a few concerns (see log) that would need to be addressed before deploying something like this. Mark Hershberger (hexmode on IRC) can also help with the review process. We still need someone to volunteer as the main developer to make the fixes. Kim, is that what you're signing up for? -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #25 from Jason Giglio g...@gigstaggart.com 2010-10-26 21:06:19 UTC --- I will try to find some time to look at it again. I had a version that was partially rewritten. I concur with Roan's observations about the code quality. It's really not up to modern PHP standards. Don't count this as volunteering to be the main developer, but count it as a I'll try to look at it -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 Priyanka Dhanda pdha...@wikimedia.org changed: What|Removed |Added AssignedTo|pdha...@wikimedia.org |ro...@wikimedia.org --- Comment #18 from Priyanka Dhanda pdha...@wikimedia.org 2010-10-23 20:40:31 UTC --- Assigning this to Rob Lanphier so that he can figure out how to proceed. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 Amgine amgine.sae...@gmail.com changed: What|Removed |Added Keywords||bugsmash -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #17 from Amgine amgine.sae...@gmail.com 2010-02-09 20:08:02 UTC --- (In reply to comment #16) What's current status on this? Has activity died down again? Told you so. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are watching all bug changes. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #13 from Priyanka Dhanda pdha...@wikimedia.org 2009-12-30 22:29:54 UTC --- Brian, Jason, I did a little cleanup. Let me know when you have verified it and whether you are satisfied with it now. -p -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919
--- Comment #14 from Brian McNeil brian.mcn...@wikinewsie.org 2009-12-31
01:23:45 UTC ---
Priyanka,
I'm looking at the changes you checked in (I'm a noob to php)
#par is used by dpl_param on line 305 thus:
$feedType = explode('/',$par,2)
What is actually in $par? It appears the result of the operation on line 305
should produce an array based on items separated by '/'.
If $par is the url path after Special:GNSM/ then line 305 will only work if
the ? and parameters after it are stripped before it gets to there. That
would also mean that Special:GNSM/feedtype/junk path and name?option=xxx
would work.
Of course, Special:GNSM/feedtype/url-escaped-feed-title?option=xxx would
be nice and allow for more descriptive URLs.
--
Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are on the CC list for the bug.
___
Wikibugs-l mailing list
Wikibugs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #15 from Jason Giglio g...@gigstaggart.com 2009-12-31 03:17:17 UTC --- I don't have access to svn right now, (on vacation), but I have already done most of the changes to migrate to wgRequest and get away from par entirely.. I have not implmented the multiple categories yet and it's still got some bugs. I'll try to get some time in on it tomorrow and see if I can integrate the new changes that were checked in. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #8 from Brian McNeil brian.mcn...@wikinewsie.org 2009-12-23 11:47:48 UTC --- I'm going to ask Josh at Google if there's someone from there can take a look at this. We're effectively developing a general-purpose addon which a main use of will be to get a MediaWiki install indexed by Google using a sitemap. The category matching tables that would be needed on-wiki to give Google appropriate categories are one thing but, I noticed on one of the other highlighted Google help pages that company stock ticker listings can be provided. If the table with mappings of [[Category:Microsoft]] to a stock ticker can be indicated by a parameter this is another very useful thing to set up on Wikimedia wikis. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919
--- Comment #9 from Roan Kattouw roan.katt...@gmail.com 2009-12-23 13:14:21
UTC ---
(In reply to comment #6)
Amgine,
I noticed that the parameters work as part of the url
http://wiki.enwn.net/index.php/Special:Google_News_SiteMap/feed=rssorder=descending
but not as request parameters
http://wiki.enwn.net/index.php/Special:Google_News_SiteMap?feed=rssorder=descending
http://www.mediawiki.org/wiki/Manual:Special_pages#The_Body_File
It may be better to use $wgRequest-getValues() instead of the $par
For stuff like this, you should really use $wgRequest-getVal('feed') instead
of $par. You should really only use $par for one parameter, like
Special:Listusers/sysop (alias for Special:Listusers?group=sysop); otherwise
you end up splitting and parsing $par yourself, which is kind of unnecessary.
If you want to use $par for something, the feed parameter makes most sense IMO.
--
Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are on the CC list for the bug.
___
Wikibugs-l mailing list
Wikibugs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #10 from Priyanka Dhanda pdha...@wikimedia.org 2009-12-23 19:50:38 UTC --- Bawolff, makes sense. The fix for the request params is pretty simple, I can make it if we don't hear back from Amgine soon. Thanks for the feedback everyone. -p -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #11 from Jason Giglio g...@gigstaggart.com 2009-12-23 19:54:21 UTC --- I was planning on doing that as the first stage in cleaning it up. I've got the test environment going and plan to take a look at it tonight. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #12 from Amgine amgine.sae...@gmail.com 2009-12-23 22:06:41 UTC --- Sorry for the delays: I'm road tripping into a winter storm here, but I should have internet access fairly regularly for the next two days (or longer if we get snowed in.) $par vs. $wgRequest: I used the first/fastest tool I found - start to finish here was about 3 weeks. After 5 weeks of trying to get exactly this kind of feedback, I gave up on the project. It's GPL, so fix/expand as you see fit. I don't own it, I just tried to fix a problem Wikinews asked me to address. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #4 from Kim Bruning k...@bruning.xs4all.nl 2009-12-22 13:20:43 UTC --- I've created a vm with the latest versions of mediawiki and gnsm from svn ( http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/GNSM ) Please send me mail with your pubkey if you would like access. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 Priyanka Dhanda pdha...@wikimedia.org changed: What|Removed |Added CC||pdha...@wikimedia.org Status|NEW |ASSIGNED --- Comment #5 from Priyanka Dhanda pdha...@wikimedia.org 2009-12-22 19:57:33 UTC --- Just as a reference it may be good to look at these: http://www.google.com/support/news_pub/bin/answer.py?hl=enanswer=74288 and http://www.google.com/support/webmasters/bin/answer.py?answer=82299 I'll also apply Bawolff's patch and test all the changes. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 Roan Kattouw roan.katt...@gmail.com changed: What|Removed |Added CC||roan.katt...@gmail.com AssignedTo|wikibugs- |pdha...@wikimedia.org |l...@lists.wikimedia.org | Status|ASSIGNED|NEW -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #6 from Priyanka Dhanda pdha...@wikimedia.org 2009-12-23 00:52:55 UTC --- Amgine, I noticed that the parameters work as part of the url http://wiki.enwn.net/index.php/Special:Google_News_SiteMap/feed=rssorder=descending but not as request parameters http://wiki.enwn.net/index.php/Special:Google_News_SiteMap?feed=rssorder=descending http://www.mediawiki.org/wiki/Manual:Special_pages#The_Body_File It may be better to use $wgRequest-getValues() instead of the $par Bawolff, Do you thing the category mapping may be better as part of the local setting? It may be easier to configure per site. -p -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #7 from Bawolff bawolff...@gmail.com 2009-12-23 04:15:21 UTC --- Note, Amgine is currently on a road trip, and might have limited internet connectivity, and might take longer to respond. (however that might have just been for today.) I am fairly certain that Amgine would not be opposed to changing the feed to use the request parameter method. As for category mapping, I think it would be easier doing it as a system message, since: *Google's categories they accept might change, or be expanded *The category scheme that we use might change *Each language would use a different scheme Since we can modify a system message without bothering/waiting for the devs, i feel that using a system message would be easier. However with that being said, if you think it'd be better to do in some other fashion, please don't hesitate. I'm more concerned that such a functionality exists rather then how it is implemented. Cheers, Bawolff -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 Reedy s...@reedyboy.net changed: What|Removed |Added Component|General/Unknown |General/Unknown Product|Wikimedia |MediaWiki extensions Version|unspecified |any -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #1 from Bawolff bawolff...@gmail.com 2009-12-21 22:37:51 UTC --- Created an attachment (id=6899) -- (https://bugzilla.wikimedia.org/attachment.cgi?id=6899) diff to add support for mapping between category schemas. I've added a diff that does the following *Fixes a syntax error in the i18n file *Provides a method to map category to keywords for the sitemap (Google uses different category names then we do: http://www.google.com/support/news_pub/bin/answer.py?hl=enanswer=116037 ). It adds a new message gnsm_categorymap which takes a list of items of the form: *mediawiki_category_name_without_leading_ns_prefix|keyword_to_output or *mediawiki_cat_name_no_ns|__MASK__ to supress outputting that category. (the extension should perhaps automatically mask hidden categoris, i wasn't sure). Cheers. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 p858snake p858sn...@yahoo.com.au changed: What|Removed |Added CC||p858sn...@yahoo.com.au Keywords||need-review -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919
--- Comment #2 from Jon Davis w...@konsoletek.com 2009-12-21 23:25:48 UTC ---
Someone munged the FR translations in SpecialGNSM.i18n.php . Among the bad
characters in almost every line, specifically 'gnsm_noincludecats' contains an
extra apostrophe (') in despace. Could someone please fix this?
Secondarily I've updated
http://wiki.enwn.net/index.php/Special:Google_News_SiteMap to SVN (As seen
http://wiki.enwn.net/index.php/Special:Version )
--
Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the assignee for the bug.
You are on the CC list for the bug.
___
Wikibugs-l mailing list
Wikibugs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 21919] Make an extension to produce Google news sitemaps to allow Wikinews to be more efficiently indexed by google
https://bugzilla.wikimedia.org/show_bug.cgi?id=21919 --- Comment #3 from Bawolff bawolff...@gmail.com 2009-12-21 23:37:00 UTC --- /me notes patch above that fixes apostraphe issue :P -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
