toolserver users dont have access to text On Wed, Mar 25, 2009 at 7:05 PM, Brian <[email protected]> wrote:
> Perhaps the toolserver can make you a current dump of current en? > > On Wed, Mar 25, 2009 at 11:08 AM, Christian Storm <[email protected] > >wrote: > > > Thanks to everyone who got the enwiki dumps going again! Should we > expect > > more regular dumps now? What was the final solution of fixing this? > > > > > > > > > > > > We are having to resort to crawling en.wikipedia.org while we await > > > for regular dumps. > > > What is the minimum crawling delay we can get away with? I figure if we > > > have 1 second delay then we'd be able to crawl the 2+ million articles > > > in a month. > > > > > > I know crawling is discouraged but it seems a lot of parties still do > > > so after looking at robots.txt > > > I have to assume that is how Google et al. is able to keep up to date. > > > > > > Are their private data feeds? I noticed a wg_enwiki dump listed. > > > > > > Christian > > > > > > On Jan 28, 2009, at 10:47 AM, Christian Storm wrote: > > > > > > > That would be great. I second this notion whole heartedly. > > > > > > > > > > > > On Jan 28, 2009, at 7:34 AM, Russell Blau wrote: > > > > > > > >> "Brion Vibber" <[email protected]> wrote in message > > > >> news:[email protected]... > > > >>> On 1/27/09 2:55 PM, Robert Rohde wrote: > > > >>>> On Tue, Jan 27, 2009 at 2:42 PM, Brion Vibber<[email protected] > > > > > >>>> wrote: > > > >>>>> On 1/27/09 2:35 PM, Thomas Dalton wrote: > > > >>>>>> The way I see it, what we need is to get a really powerful > server > > > >>>>> Nope, it's a software architecture issue. We'll restart it with > > > >>>>> the new > > > >>>>> arch when it's ready to go. > > > >>>> The simplest solution is just to kill the current dump job if you > > > >>>> have > > > >>>> faith that a new architecture can be put in place in less than a > > > >>>> year. > > > >>> > > > >>> We'll probably do that. > > > >>> > > > >>> -- brion > > > >> > > > >> FWIW, I'll add my vote for aborting the current dump *now* if we > > > >> don't > > > >> expect it ever to actually be finished, so we can at least get a > > > >> fresh dump > > > >> of the current pages. > > > >> > > > >> Russ > > > >> > > > >> > > > >> > > > >> > > > >> _______________________________________________ > > > >> Wikitech-l mailing list > > > >> [email protected] > > > >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l > > > > > > > > > > > > _______________________________________________ > > > > Wikitech-l mailing list > > > > [email protected] > > > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > > > > > > > > > _______________________________________________ > > > Wikitech-l mailing list > > > [email protected] > > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > > > > > _______________________________________________ > > Wikitech-l mailing list > > [email protected] > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > > > _______________________________________________ > Wikitech-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > _______________________________________________ Wikitech-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikitech-l
