+1 to this. Incremental dumps (even if just weekly) would be extremely useful.
On Fri, Feb 26, 2021 at 9:58 AM Chris Hokamp <[email protected]> wrote: > Hi, first time chiming in here, this topic is very relevant for our > wikidata usecase as well, having daily or weekly diffs would be very useful. > > Cheers, > CH > > On Fri, Feb 26, 2021 at 2:47 PM Kingsley Idehen via Wikidata < > [email protected]> wrote: > >> On 2/26/21 3:46 AM, Guillaume Lederrey wrote: >> >> Hello! >> >> We are working on a new update process for WDQS, based on a stream of >> changes [1]. While not exactly the solution you are looking for, this might >> be a building block for differential dumps. For example by aggregating the >> stream of changes over a period of time. >> >> Note that at this point, the stream of changes that we construct is >> published to an internal Kafka that isn't exposed to the internet. If there >> is enough interest, we might be able to expose it in some form. >> >> Have fun! >> >> Guillaume >> >> >> >> [1] https://phabricator.wikimedia.org/T244590 >> >> >> Hi Guillaume, >> >> I am very interested in exposure right now since we are trying to have an >> up-to-date mirror of Wikidata. >> >> We can discuss offline if you like. >> >> >> Kingsley >> >> >> >> On Fri, Feb 26, 2021 at 8:49 AM Federico Leva (Nemo) <[email protected]> >> wrote: >> >>> Kingsley Idehen via Wikidata, 25/02/21 19:26: >>> > Is there a mechanism in place for producing and publishing >>> delta-centric >>> > dumps for Wikidata? >>> >>> There's >>> https://phabricator.wikimedia.org/T72246 >>> >>> Magnus Manske used to maintain some biweekly dumps as part of its WDQ >>> service, IIRC. >>> >>> Federico >>> >>> _______________________________________________ >>> Wikidata mailing list >>> [email protected] >>> https://lists.wikimedia.org/mailman/listinfo/wikidata >>> >> >> >> -- >> *Guillaume Lederrey* (he/him) >> Engineering Manager >> Wikimedia Foundation <https://wikimediafoundation.org/> >> >> _______________________________________________ >> Wikidata mailing >> [email protected]https://lists.wikimedia.org/mailman/listinfo/wikidata >> >> >> -- >> Regards, >> >> Kingsley Idehen >> Founder & CEO >> OpenLink Software >> Home Page: http://www.openlinksw.com >> Community Support: https://community.openlinksw.com >> Weblogs (Blogs): >> Company Blog: https://medium.com/openlink-software-blog >> Virtuoso Blog: https://medium.com/virtuoso-blog >> Data Access Drivers Blog: >> https://medium.com/openlink-odbc-jdbc-ado-net-data-access-drivers >> >> Personal Weblogs (Blogs): >> Medium Blog: https://medium.com/@kidehen >> Legacy Blogs: http://www.openlinksw.com/blog/~kidehen/ >> http://kidehen.blogspot.com >> >> Profile Pages: >> Pinterest: https://www.pinterest.com/kidehen/ >> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >> Twitter: https://twitter.com/kidehen >> Google+: https://plus.google.com/+KingsleyIdehen/about >> LinkedIn: http://www.linkedin.com/in/kidehen >> >> Web Identities (WebID): >> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >> : >> http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >> >> _______________________________________________ >> Wikidata mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/wikidata >> > _______________________________________________ > Wikidata mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wikidata > -- Samuel Klein @metasj w:user:sj +1 617 529 4266
_______________________________________________ Wikidata mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata
