I considered it. But again, the ability to define the local graph name was important, and well as being able to do the configuration without interacting with the conductor site. Virtuoso is acting as a backend for another website, and that site defines which pages need to be imported, and the XSLTs used to translate them. I use a cron'd java program to sync virtuoso to the front end. The bulk of the content is generated on a wiki, so the pages being scrapped are created dynamically, but I can interrogate the database directly to find when their revision numbers have changed. I manually extend the expiration times in the SYS_HTTP_SPONGE table to prevent overactive cache refresh ( sponging 20,000 pages takes a lot of CPU time, so I'd like to do it less often), and then I pass a get:refresh option when I want to force an update on a particular page.
This manual setup seems to give me tighter and easier control then interacting with the sponge proxy would. Kyle On Sat, Aug 28, 2010 at 8:37 AM, Hugh Williams <[email protected]> wrote: > Hi Kyle, > > Have you considered using the Sponger RDF proxy Service (Rest style web > service) which exposes the Virtuoso sponger functionality via a web service > endpoint, as detailed at: > > > http://docs.openlinksw.com/virtuoso/rdfsparqlintegrationmiddleware.html#rdfspongerprogrammerguide > > http://virtuoso.openlinksw.com/presentations/Virtuoso_Sponger_1/Virtuoso_Sponger_1.html#%2813%29 > > Best Regards > Hugh Williams > Professional Services > OpenLink Software > Web: http://www.openlinksw.com > Support: http://support.openlinksw.com > Forums: http://boards.openlinksw.com/support > Twitter: http://twitter.com/OpenLink > > On 27 Aug 2010, at 18:24, Kyle wrote: > >> Using DB.DBA.SYS_HTTP_SPONGE_UP directly solves a few different >> problems, namely it allows me to define the local name of the graph >> URI manually. >> Also by engaging SYS_HTTP_SPONGE_UP directly, I can write replacements >> for RDF_LOAD_HTTP_RESPONSE. For example I have one called >> RDF_XSLT_LOAD_HTTP_RESPONSE, which does a XSLT transformation on the >> incoming data before loading it. >> And this can be done programatically, without having to interact with >> the conductor webtool. >> >> Kyle >> >> On Fri, Aug 27, 2010 at 6:08 AM, Hugh Williams <[email protected]> >> wrote: >>> Hi Kyle, >>> >>> The DB.DBA.SYS_HTTP_SPONGE_UP procedure is for internal use only. The >>> Virtuoso "sparql define get:soft ..." method would be the way to sponge >>> remote pages as detailed at: >>> >>> >>> http://virtuoso.openlinksw.com/presentations/SPARQL_Tutorials/SPARQL_Tutorials_Part_7/SPARQL_Tutorials_Part_7.html#%2815%29 >>> >>> Although the "get:soft" pragma does not have an authentication paramter and >>> so cannot authenticate for you currently, although a request has been made >>> for an authentication parameter to be added which should be in the next >>> release ... >>> >> >> ------------------------------------------------------------------------------ >> Sell apps to millions through the Intel(R) Atom(Tm) Developer Program >> Be part of this innovative community and reach millions of netbook users >> worldwide. Take advantage of special opportunities to increase revenue and >> speed time-to-market. Join now, and jumpstart your future. >> http://p.sf.net/sfu/intel-atom-d2d >> _______________________________________________ >> Virtuoso-users mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/virtuoso-users > >
