Re: [Wikidata-tech] Wikibase query service update question

[David Causse]

On Wed, Dec 2, 2020 at 1:30 PM Victor Agroskin  wrote:

> Thank you, David. I'd indeed resolved the problem. It was a
> desynchronisation between base  concept URI set in repo parameters and
> parameters for updater and munger calls.
>
>
Glad to hear that you have a working setup.


> It was broken because I'd tried different settings to get a
> dereferenceable "Concept URI" link at the bottom of the side menu. Looks
> like it is impossible to do it in MediaWiki, requires some configuration at
> web server level.
>
>
Indeed WMF machines run some apache URL rewrite rules[0] at the apache
level to solve this.

0:
https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/modules/mediawiki/files/apache/sites/wikidata-uris.incl

David.
___
Wikidata-tech mailing list
Wikidata-tech@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-tech

Re: [Wikidata-tech] Wikibase query service update question

[Victor Agroskin]

Thank you, David. I'd indeed resolved the problem. It was a
desynchronisation between base  concept URI set in repo parameters and
parameters for updater and munger calls.

It was broken because I'd tried different settings to get a dereferenceable
"Concept URI" link at the bottom of the side menu. Looks like it is
impossible to do it in MediaWiki, requires some configuration at web server
level.

Now all namespace parameters are the same and import goes smoothly.

Regards,
Victor

On Tue, 1 Dec 2020 at 16:20, Victor Agroskin  wrote:

> I need further help, if possible.
>
> I'm trying to get query service updater to get data from my Wikibase. It
> correctly identifies changes and accesses pages of changed items like
> https://standartopedia.ru/wiki/Special:EntityData/Q183.ttl
>
> There are subjects on this page in three distinct namespaces:
>   data: 
>   wd: 
>   s: 
>
> But the Munger reports an error saying
>
> $BadSubjectException: Unrecognized subjects:  [
>> https://standartopedia.ru/wiki/Item:Q183,
>> https://standartopedia.ru/wiki/Item:statement/
>> Q183-e349d2b6-4624-654f-b242-22497248197d].  Expected only sitelinks and
>> subjects starting with https://standartopedia.ru/wiki/Special:EntityData/
>> and [https://standartopedia.ru/entity/]
>>
>
> So the first namespace is recognised, but no triples are imported. The 2nd
> and 3rd namespaces do not correspond to the expected values at all.
>
> The runUpdate.sh script was called with the parameter
>
> --conceptUri https://standartopedia.ru/
>
> Looks like it forms one correct namespace, but not all the three which are
> actually found in the data.
>
> Can anyone advise me on how to make Munger process the data?
>
> Regards,
> Victor Agroskin
>
>
>
> On Thu, 26 Nov 2020 at 18:35, Victor Agroskin  wrote:
>
>> Нi All!
>>
>> I'm running a custom Wikibase and because of extension
>> compatibility issues I can not use docker image, so I'm installing software
>> from repositories.
>>
>> I'm installing RDF query service as described in
>> https://github.com/wikimedia/wikidata-query-rdf/blob/master/docs/getting-started.md
>>
>>
>> I have a following problem:
>>
>> When I start runUpdate.sh with the command $ ./runUpdate.sh -n wdq , it
>> starts populating my Blazegraph instance with updates from Wikidata. I can
>> not find any way to configure it to use my local Wikibase instead. No
>> parameters documented, no global or local variables found. Can anyone
>> provide any advice on this?
>>
>> Thank you for your help!
>> Victor Agroskin
>>
>
___
Wikidata-tech mailing list
Wikidata-tech@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-tech

Re: [Wikidata-tech] Wikibase query service update question

[David Causse]

Hi Victor,

please see my answers inline.

On Tue, Dec 1, 2020 at 2:29 PM Victor Agroskin  wrote:

> I need further help, if possible.
>
> I'm trying to get query service updater to get data from my Wikibase. It
> correctly identifies changes and accesses pages of changed items like
> https://standartopedia.ru/wiki/Special:EntityData/Q183.ttl
>
> There are subjects on this page in three distinct namespaces:
>   data: 
>   wd: 
>   s: 
>
>
As you noticed the munger will not be able to import such data. The
prefixes should look like:

@prefix wd:  .
@prefix data:  .
@prefix s:  .

Looking at https://standartopedia.ru/wiki/Special:EntityData/Q183.ttl I see
that it is now correct, did you already solve the problem? If not I
probably misunderstood your issue.

Regards,

David.
___
Wikidata-tech mailing list
Wikidata-tech@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-tech

Re: [Wikidata-tech] Wikibase query service update question

[Victor Agroskin]

I need further help, if possible.

I'm trying to get query service updater to get data from my Wikibase. It
correctly identifies changes and accesses pages of changed items like
https://standartopedia.ru/wiki/Special:EntityData/Q183.ttl

There are subjects on this page in three distinct namespaces:
  data: 
  wd: 
  s: 

But the Munger reports an error saying

$BadSubjectException: Unrecognized subjects:  [
> https://standartopedia.ru/wiki/Item:Q183,
> https://standartopedia.ru/wiki/Item:statement/
> Q183-e349d2b6-4624-654f-b242-22497248197d].  Expected only sitelinks and
> subjects starting with https://standartopedia.ru/wiki/Special:EntityData/
> and [https://standartopedia.ru/entity/]
>

So the first namespace is recognised, but no triples are imported. The 2nd
and 3rd namespaces do not correspond to the expected values at all.

The runUpdate.sh script was called with the parameter

--conceptUri https://standartopedia.ru/

Looks like it forms one correct namespace, but not all the three which are
actually found in the data.

Can anyone advise me on how to make Munger process the data?

Regards,
Victor Agroskin



On Thu, 26 Nov 2020 at 18:35, Victor Agroskin  wrote:

> Нi All!
>
> I'm running a custom Wikibase and because of extension
> compatibility issues I can not use docker image, so I'm installing software
> from repositories.
>
> I'm installing RDF query service as described in
> https://github.com/wikimedia/wikidata-query-rdf/blob/master/docs/getting-started.md
>
>
> I have a following problem:
>
> When I start runUpdate.sh with the command $ ./runUpdate.sh -n wdq , it
> starts populating my Blazegraph instance with updates from Wikidata. I can
> not find any way to configure it to use my local Wikibase instead. No
> parameters documented, no global or local variables found. Can anyone
> provide any advice on this?
>
> Thank you for your help!
> Victor Agroskin
>
___
Wikidata-tech mailing list
Wikidata-tech@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-tech

Re: [Wikidata-tech] Wikibase query service update question

[Victor Agroskin]

David, thank you! It is very helpful to have the list and examples there,
as this page comes first in the searches somehow.

Regards,
Victor




> Date: Fri, 27 Nov 2020 10:22:21 +0100
> From: David Causse 
> To: Wikidata technical discussion 
> Subject: Re: [Wikidata-tech] Wikibase query service update question
> Message-ID:
>  zkdr6inxkpbp8xjkeoeursskjgg4...@mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Victor,
>
> please see my response inline.
>
> On Thu, Nov 26, 2020 at 4:36 PM Victor Agroskin  wrote:
>
> >
> > When I start runUpdate.sh with the command $ ./runUpdate.sh -n wdq , it
> > starts populating my Blazegraph instance with updates from Wikidata. I
> can
> > not find any way to configure it to use my local Wikibase instead. No
> > parameters documented, no global or local variables found. Can anyone
> > provide any advice on this?
> >
>
> Most of the default values assume that you are running the service for
> wikidata.
> I've added a few notes about that in
>
> https://wikitech.wikimedia.org/wiki/Wikidata_query_service#Notes_about_running_the_service_on_non-WMF_infrastructure
>
> Regards,
>
> David.
> -- next part --
> An HTML attachment was scrubbed...
> URL: <
> https://lists.wikimedia.org/pipermail/wikidata-tech/attachments/20201127/5e60cd0a/attachment-0001.htm
> >
>
> --
>
> Subject: Digest Footer
>
> ___
> Wikidata-tech mailing list
> Wikidata-tech@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikidata-tech
>
>
> --
>
> End of Wikidata-tech Digest, Vol 91, Issue 2
> 
>
___
Wikidata-tech mailing list
Wikidata-tech@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-tech

Re: [Wikidata-tech] Wikibase query service update question

[David Causse]

Hi Victor,

please see my response inline.

On Thu, Nov 26, 2020 at 4:36 PM Victor Agroskin  wrote:

>
> When I start runUpdate.sh with the command $ ./runUpdate.sh -n wdq , it
> starts populating my Blazegraph instance with updates from Wikidata. I can
> not find any way to configure it to use my local Wikibase instead. No
> parameters documented, no global or local variables found. Can anyone
> provide any advice on this?
>

Most of the default values assume that you are running the service for
wikidata.
I've added a few notes about that in
https://wikitech.wikimedia.org/wiki/Wikidata_query_service#Notes_about_running_the_service_on_non-WMF_infrastructure

Regards,

David.
___
Wikidata-tech mailing list
Wikidata-tech@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-tech

Re: [Wikidata-tech] Wikibase query service update question

[Thomas Arrow]

Hi Victor!

The command the docker image runs is as follows:

./runUpdate.sh -h http://$WDQS_HOST:$WDQS_PORT -- --wikibaseUrl
$WIKIBASE_SCHEME://$WIKIBASE_HOST --conceptUri
$WIKIBASE_SCHEME://$WIKIBASE_HOST --entityNamespaces
$WDQS_ENTITY_NAMESPACES

That was taken from:
https://github.com/wmde/wikibase-docker/blob/master/wdqs/0.3.6/runUpdate.sh#L9

You can see what all the variables could be set to in the environment
variables section of
https://github.com/wmde/wikibase-docker/blob/master/wdqs/README.md.

This is assuming you're using something like this archive:
https://archiva.wikimedia.org/repository/snapshots/org/wikidata/query/rdf/service/0.3.6-SNAPSHOT/service-0.3.6-SNAPSHOT-dist.zip

All the best,
Tom


On Thu, 26 Nov 2020 at 15:36, Victor Agroskin  wrote:
>
> Нi All!
>
> I'm running a custom Wikibase and because of extension compatibility issues I 
> can not use docker image, so I'm installing software from repositories.
>
> I'm installing RDF query service as described in 
> https://github.com/wikimedia/wikidata-query-rdf/blob/master/docs/getting-started.md
>
> I have a following problem:
>
> When I start runUpdate.sh with the command $ ./runUpdate.sh -n wdq , it 
> starts populating my Blazegraph instance with updates from Wikidata. I can 
> not find any way to configure it to use my local Wikibase instead. No 
> parameters documented, no global or local variables found. Can anyone provide 
> any advice on this?
>
> Thank you for your help!
> Victor Agroskin
> ___
> Wikidata-tech mailing list
> Wikidata-tech@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikidata-tech

___
Wikidata-tech mailing list
Wikidata-tech@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-tech

[Wikidata-tech] Wikibase query service update question

[Victor Agroskin]

Нi All!

I'm running a custom Wikibase and because of extension compatibility issues
I can not use docker image, so I'm installing software from repositories.

I'm installing RDF query service as described in
https://github.com/wikimedia/wikidata-query-rdf/blob/master/docs/getting-started.md


I have a following problem:

When I start runUpdate.sh with the command $ ./runUpdate.sh -n wdq , it
starts populating my Blazegraph instance with updates from Wikidata. I can
not find any way to configure it to use my local Wikibase instead. No
parameters documented, no global or local variables found. Can anyone
provide any advice on this?

Thank you for your help!
Victor Agroskin
___
Wikidata-tech mailing list
Wikidata-tech@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-tech