Igorkim78 added a comment.
The configuration changes for SDC data are as follows (note that namespace
'sdc' is used to store RDF data in blazegraph journal, might be changed as
needed):
- Blazegraph journal config (RWStore.properties)
replace the similar configuration for WDQS (search for
com.bigdata.namespace.wdq prefix for the parameters to be replaced):
# Bump up the branching factor for the lexicon indices on the default kb.
com.bigdata.namespace.sdc.lex.BLOBS.com.bigdata.btree.BTree.branchingFactor=400
com.bigdata.namespace.sdc.lex.ID2TERM.com.bigdata.btree.BTree.branchingFactor=599
com.bigdata.namespace.sdc.lex.TERM2ID.com.bigdata.btree.BTree.branchingFactor=300
# Bump up the branching factor for the statement indices on the default kb.
com.bigdata.namespace.sdc.spo.JUST.com.bigdata.btree.BTree.branchingFactor=1024
com.bigdata.namespace.sdc.spo.OSP.com.bigdata.btree.BTree.branchingFactor=866
com.bigdata.namespace.sdc.spo.POS.com.bigdata.btree.BTree.branchingFactor=954
com.bigdata.namespace.sdc.spo.SPO.com.bigdata.btree.BTree.branchingFactor=934
Note, that the final configuration should be adjusted for the real production
data according to instructions in T232768
<https://phabricator.wikimedia.org/T232768>.
- Scripts to run Updater should be called with proper namespace:
On data load:
./loadRestAPI.sh -n wdq -d `pwd`/data/split
replace by
./loadRestAPI.sh -n sdc -d `pwd`/data/split
On single file load:
./loadRestAPI.sh -n wdq -d `pwd`/data/split/wikidump-000000001.ttl.gz
replace by
./loadRestAPI.sh -n sdc -d `pwd`/data/split/wikidump-000000001.ttl.gz
On run updater:
./runUpdate.sh -n wdq
replace by
./runUpdate.sh -n sdc
On any calls to Blazegraph REST, instead of
http://localhost:9999/bigdata/namespace/wdq/sparql
use
http://localhost:9999/bigdata/namespace/sdc/sparql
Categories store might need similar changes, but that has to be discussed, if
separate categories are needed for production SDC data.
TASK DETAIL
https://phabricator.wikimedia.org/T237089
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Mathew.onipe, Igorkim78
Cc: Aklapper, Igorkim78, Gehel, Liuxinyu970226, Mathew.onipe, darthmon_wmde,
Legado_Shulgin, Nandana, JKSTNK, Davinaclare77, Qtn1293, Techguru.pc, Lahi,
PDrouin-WMF, Gq86, E1presidente, Ramsey-WMF, Cparle, Anooprao, SandraF_WMF,
GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, Tramullas, Acer,
LawExplorer, Salgo60, Zppix, Silverfish, _jensen, rosalieper, Scott_WUaS,
Susannaanas, Wong128hk, Jane023, Wikidata-bugs, Base, matthiasmullie, aude,
Ricordisamoa, Wesalius, Lydia_Pintscher, Fabrice_Florin, Raymond, faidon,
Jdforrester-WMF, Steinsplitter, Mbch331, Rxy, Jay8g, fgiunchedi
_______________________________________________
Wikidata-bugs mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
