Thanks for all those tips Wes! I’m on leave this week but some time next week I’ll read some of those and make more of a SPARQLWrapper plan and announce that. Then I’ll ask for volunteers for plates of it...
Cheers, Nick — Dr Nicholas Car Data Systems Architect SURROUND Australia 0477 560 177 nicholas....@surroundaustralia.com > On 30 Aug 2020, at 11:51 pm, Wes Turner <wes.tur...@gmail.com> wrote: > > > There are many ways to set up each and every major SPARQL server. > > Tox envs for each database may be the way to go. > > A test grid with GitHub Actions and separate containers may be the easiest > thing; but IDK that the whole grid needs to be run every time. > > ## Tox docs > > https://tox.readthedocs.io/en/latest/config.html#generative-envlist > https://tox.readthedocs.io/en/latest/config.html#substitutions-for-virtualenv-related-sections > {envname} > https://tox.readthedocs.io/en/latest/config.html#conf-commands_pre > > Rather than writing the PID and shutting the external process down in e.g. > tox commands_post, there's pytest-xprocess: > https://github.com/pytest-dev/pytest-xprocess/issues/19#issuecomment-682185903 > > Here's how SQLAlchemy and Django test multiple databases with tox: > - https://github.com/sqlalchemy/sqlalchemy/blob/master/tox.ini > - https://github.com/django/django/blob/master/tox.ini > > > ## Containers, Docker, and Podman > Containers can contain already-configured databases (with data already > loaded). > > Podman containers run as a subprocess. > > Podman may be installed in a container on a CI system. With CentOS and > Fedora, it's just `dnf install -y podman` and then configuration of > containers so that they'll run as rootless containers. > https://podman.io/getting-started/installation.html > > With containers as subprocesses, the build and test would run once in one > container and launch each database as a [rootless] sub-container, run tests, > and then shut down the server (thus not using unnecessary resources for every > build). > > It looks like the SQLAlchemy tox.ini also accomplishes that somehow in a > different way? > > > ## GitHub Actions > > An actions.yml for GitHub Actions which installs SPARQLwrapper into > successive containers for each database (rather than trying to install every > database within the test container) may be the easiest thing. > > https://github.com/sdras/awesome-actions#database doesn't yet list much in > the way of testing a grid of versions and database versions. > > https://freek.dev/1590-how-to-use-a-mysql-database-on-github-actions > describes how to add a `service:` dependency on a MySQL image (which > presumably unnecessarily runs for the duration of the test run?) > > https://medium.com/@ldarren/number-of-ways-to-setup-database-in-github-actions-2cd48df9faae > describes 4 ways to setup a database with GitHub Actions > > https://github.com/ymyzk/tox-gh-actions#advanced-examples describes how to > use Factor-conditional settings > > On Sun, Aug 30, 2020, 7:44 AM Nicholas Car > <nicholas....@surroundaustralia.com> wrote: >> Hi all, >> >> We use SPARQL queries with EDG all the time since my company uses EDG as the >> basis for some of our products, so let me know if you have any problems >> accessing EDG via SPARQLWrapper. >> >> Also, I hope to run a small SPARQLWrapper sprint soon as the codebase is in >> need of a touch-up. A bunch of PRs are failing tests and I think some of the >> failures are false in some way. The failures make passing PRs impossible and >> prevent me from lodging more - I have one to turn XML responses from a >> SimpleXml object into a Python dict, just as JSON responses are converted - >> but need to see the exisiting, backed-up PRs dealt with first. >> >> My plans for SPARQLWrapper: >> >> 1. Check to see why all existing PRs fail some, but mostly not all, Python >> version tests >> >> 2. Check all modern triplestore SPARQL endpoints, and modern versions of >> them, are represented in test suite >> >> 3. Update to Python 3.6+ only in next release (to match main rdflib package) >> >> 4. Update documentation - not all possible configurations are documented >> well, as shown by this thread >> >> Would love any help with this! >> >> Cheers, >> >> Nick >> >> >> — >> Dr Nicholas Car >> Data Systems Architect >> SURROUND Australia >> 0477 560 177 >> nicholas....@surroundaustralia.com >> >>> On 13 Aug 2020, at 2:56 am, Richard Dijkstra >>> <richarddijkstra1...@gmail.com> wrote: >>> >>> >>> I found it by a Google search resulting in some code that had some of these >>> Sparql settings. By luck this worked. Then I started looking for default >>> behaviour by eliminating explicite settings. setOnlyConneg has a default >>> Setting False which was the 406 cause. From some extra testing no other >>> setting was causing a 406. >>> >>> We might add the possible options for accessing the Topbraid EDG server >>> Sparql endpoint in the documentation. I believe their Sparql engine >>> originates from Jena but has some extra capabilities. >>> >>>> Op 12 aug. 2020 om 16:47 heeft Wes Turner <wes.tur...@gmail.com> het >>>> volgende geschreven: >>>> >>>> >>>> How did you determine that sparql.setOnlyConneg(True) was the issue? >>>> >>>> On Wed, Aug 12, 2020, 7:40 AM Richard Dijkstra >>>> <richarddijkstra1...@gmail.com> wrote: >>>>> Sergio, >>>>> >>>>> It was luckely just a user error. Apperently content negotiation is >>>>> required: >>>>> >>>>> sparql.setOnlyConneg(True) >>>>> >>>>> did the trick. >>>>> >>>>> So this code works fine to query and construct results from a local >>>>> Topbraid EDG server: >>>>> >>>>> endpoint = "http://localhost:8083/tbl/sparql"; >>>>> query = """SELECT ?prop ?datatype >>>>> WHERE { >>>>> <http://example.org/ontologies/assetonto#Object> >>>>> sh:property ?prop . >>>>> OPTIONAL { >>>>> ?prop sh:datatype ?datatype . >>>>> } >>>>> }""" >>>>> >>>>> sparql = SPARQLWrapper(endpoint) >>>>> sparql.setQuery(query) >>>>> >>>>> sparql.addDefaultGraph("urn:x-evn-master:assetonto") >>>>> sparql.setOnlyConneg(True) >>>>> sparql.setReturnFormat(CSV) >>>>> result = sparql.query().convert() >>>>> >>>>> >>>>> kind regards, >>>>> Richard >>>>> >>>>> On Monday, July 6, 2020 at 5:44:24 PM UTC+2, Sergio Fernández wrote: >>>>>> >>>>>> Please, report the details if the issue to GitHub, and we'll try to >>>>>> troubleshoot it. >>>>>> >>>>>> On Mon., Jul. 6, 2020, 08:32 Richard Dijkstra <richarddi...@gmail.com> >>>>>> wrote: >>>>>>> I am not able to go to too much detail because my lack of knowledge of >>>>>>> proper requests etc >>>>>>> But interesting might be that the Python library sparql-client queries >>>>>>> the endpoint with no issues. >>>>>>> >>>>>>> However I want to use SPARQLwrapper because it has more features. >>>>>>> >>>>>>> >>>>>>> On Friday, July 3, 2020 at 4:00:33 PM UTC+2, Richard Dijkstra wrote: >>>>>>>> >>>>>>>> Thank you Wes for the tips for further investigations. I will give it >>>>>>>> a try next week. >>>>>>>> Regards, >>>>>>>> Ruchard >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> http://github.com/RDFLib >>>>>>> --- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "rdflib-dev" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>>> an email to rdfli...@googlegroups.com. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/rdflib-dev/98176a74-860f-4647-83a6-80bcdec4919do%40googlegroups.com. >>>>> >>>>> -- >>>>> http://github.com/RDFLib >>>>> --- >>>>> You received this message because you are subscribed to the Google Groups >>>>> "rdflib-dev" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send an >>>>> email to rdflib-dev+unsubscr...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/rdflib-dev/e807c91b-e35f-4cc2-b490-f95bbc9650ffo%40googlegroups.com. >>>> >>>> -- >>>> http://github.com/RDFLib >>>> --- >>>> You received this message because you are subscribed to the Google Groups >>>> "rdflib-dev" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to rdflib-dev+unsubscr...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/rdflib-dev/CACfEFw_MR5ueH69188Y4gWhWvL5sM-CM84xOakOa0FF2QLy%2Bjw%40mail.gmail.com. >>> -- >>> http://github.com/RDFLib >>> --- >>> You received this message because you are subscribed to the Google Groups >>> "rdflib-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to rdflib-dev+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/rdflib-dev/1FD48B54-FBE5-452E-ABA8-A3E98FDC0DD2%40gmail.com. >> >> -- >> http://github.com/RDFLib >> --- >> You received this message because you are subscribed to the Google Groups >> "rdflib-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to rdflib-dev+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/rdflib-dev/8165FA21-F64C-44EF-AD81-B36312E416A6%40surroundaustralia.com. > > -- > http://github.com/RDFLib > --- > You received this message because you are subscribed to the Google Groups > "rdflib-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to rdflib-dev+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/rdflib-dev/CACfEFw_9BNXNjhAFwLFGiUcr9U1tLRXSp9JVE23ksM56Q8fBYQ%40mail.gmail.com. -- http://github.com/RDFLib --- You received this message because you are subscribed to the Google Groups "rdflib-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to rdflib-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/rdflib-dev/F82160C4-DA2E-4CFE-9B49-AD2CC82606CA%40surroundaustralia.com.
