Hi David, In Arches 5.0, ontologies are loaded as part of a package. See https://github.com/archesproject/arches-her/tree/master/arches_her/pkg for an example of how ontologies are defined. These are automatically loaded as part of the package load command.
Hope this helps. -Dennis On Saturday, March 7, 2020 at 3:13:44 PM UTC-8, David Osborne wrote: > > Hi Ben > > Thanks, that was it: starting the elasticsearch and db containers first > was the solution. > > My docker-compose file asked for the default resource models and concepts. > Although the concepts loaded ok, the branches and models didn't, with a > missing ontology, although I can't see how to enable whatever's missing: > > arches | Running: python manage.py packages -o import_graphs > arches | operation: import_graphs > arches | /web_root/arches/arches/db/graphs/branches/Information > Resource Copyright.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Event Phase > Type.json > arches | Ontology matching query does not exist. > arches | > /web_root/arches/arches/db/graphs/branches/Description.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Place 1.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Resource > Type Classification.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Publication > Event.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/External > Identifier.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Heritage > Resource Group Phase Type.json > arches | Ontology matching query does not exist. > arches | > /web_root/arches/arches/db/graphs/branches/Appellation.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Modification > Event.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Heritage > Resource Phase Type.json > arches | Ontology matching query does not exist. > arches | > /web_root/arches/arches/db/graphs/branches/Component.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Settings > System Defaults.json > arches | Ontology matching query does not exist. > arches | > /web_root/arches/arches/db/graphs/branches/Evaluation.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Activity > Phase.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Place 3.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Resource > Update Event.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Keyword.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Beginning of > Existence.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Settings > Map.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Actor > Phase.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Right.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Title.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Information > Carrier.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Information > Resource Creation Event.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Settings > Saved Searches.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Settings > Time Search.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Language.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/End of > Existence.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Settings > Basic Search.json > arches | Ontology matching query does not exist. > arches | > /web_root/arches/arches/db/graphs/branches/Existence.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Temporal > Coverage.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Place 2.json > arches | Ontology matching query does not exist. > arches | > /web_root/arches/arches/db/graphs/branches/Measurement.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Name.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/branches/Condition > Assessment.json > arches | Ontology matching query does not exist. > arches | > /web_root/arches/arches/db/graphs/resource_models/Heritage Resource > Model.json > arches | Ontology matching query does not exist. > arches | > /web_root/arches/arches/db/graphs/resource_models/Activity Resource > Model.json > arches | Ontology matching query does not exist. > arches | > /web_root/arches/arches/db/graphs/resource_models/Heritage Resource Group > Resource Model.json > arches | Ontology matching query does not exist. > arches | > /web_root/arches/arches/db/graphs/resource_models/Historical Event Resource > Model.json > arches | Ontology matching query does not exist. > arches | /web_root/arches/arches/db/graphs/resource_models/Actor > Resource Model.json > arches | Ontology matching query does not exist. > arches | > /web_root/arches/arches/db/graphs/resource_models/Information Resource > Model.json > arches | Ontology matching query does not exist. > > David > > On Thursday, 5 March 2020 23:25:52 UTC, Ben O'Steen wrote: >> >> Im going to guess that the elasticsearch container is not ready and the >> migration script fails when it attempts to run the first time through and >> Arches ends up in a bad state which it generally can never recover from. >> >> docker-compose down -v >> >> This will kill and delete all the badly formed docker containers created >> by a failed initial run. >> >> docker-compose up --no-start >> >> This will initialize the containers and network for the stack. >> >> docker-compose start elasticsearch db >> >> This will start the elasticsearch instance and the database. >> >> docker logs -f elasticsearch >> >> This will let you watch what it is doing. Ctrl-c to get back to normal. >> I've found that if you run antivirus checkers and have some network >> slowness, it can take a while (30s to 1min) for elasticsearch to actually >> be ready. It sadly lies on its health check command and reports "green" >> while it is still doing its random clustering things. >> >> docker-compose up >> >> After elasticsearch has finished doing whatever it does on start up that >> takes so long, run this and watch to make sure there are no connection >> errors. >> >> >> Ben >> >> On Thu, 5 Mar 2020, 07:44 David Osborne, <[email protected]> wrote: >> >>> Thanks, Ben: your comment helped me make some progress (to another error >>> ;-) >>> >>> The problem is that the Readme.md in the docker directory of the Github >>> archesproject/arches repo (file last revised 17 months ago!) is inaccurate >>> or vague when it says "Create a new folder for your custom Arches code" >>> without specifying where. I created one alongside my cloned arches repo, >>> with the result that the path to /docker-entrypoint-initdb.d/init.sql did >>> not exist, so Docker created directories instead. >>> >>> Having located the arches/install/init-unix.sql file referred to in the >>> db service, I worked out that it's easiest to run docker-compose in the >>> cloned arches directory itself, so I edited the docker-compose.yml file, >>> and that file and the resulting output from docker-compose is in >>> https://gist.github.com/daosborne/bc83812e33d9396d679af54f31565895 >>> (with the lines from couchdb and elasticsearch edited out from the >>> docker-compose log as not relevant). >>> >>> It gets further but has some problems (partly because I want the default >>> graphs and concepts installed): >>> >>> 1. error setting up db (traceback starting line 133) >>> 2. database tables graphs, ontologies, concepts, d_value_types don't >>> exist (lines 222--648) >>> 3. db connection error during migrations (line 734) >>> 4. db column nodes.fieldname doesn't exist (lines 1123, 1185) >>> >>> I had also tried setting a project name, as suggested in the Readme but >>> abandoned that because the directory was missing from /web_root (and >>> produces a spurious error message about the project name clashing with a >>> Python module). >>> >>> I hoped that using Docker would be a quick and easy way of getting a new >>> Arches installation running but it seems it still has some rough edges. As >>> I need to get some development done to make some slides for a conference >>> paper, I think I'll go back to the tried and tested method of setting up a >>> Vagrant VM! >>> >>> Cheers >>> David >>> >>> On Wednesday, 4 March 2020 00:33:38 UTC, Ben O'Steen wrote: >>>> >>>> Hi, >>>> >>>> I think the key error is on line 63 of the gist you posted. For some >>>> reason, it is interpreting file bind mount as a directory and not as a >>>> file. If I run the default docker compose file here, and start up the db >>>> container, this is the log output I get: >>>> >>>> $ docker logs db >>>> Add rule to pg_hba: 0.0.0.0/0 >>>> Add rule to pg_hba: replication replicator >>>> Setup master database >>>> 2020-03-04 00:28:12.848 UTC [25] LOG: starting PostgreSQL 12.1 (Debian >>>> 12.1-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) >>>> 8.3.0, 64-bit >>>> 2020-03-04 00:28:12.849 UTC [25] LOG: listening on IPv4 address >>>> "127.0.0.1", port 5432 >>>> 2020-03-04 00:28:12.855 UTC [25] LOG: listening on Unix socket >>>> "/var/run/postgresql/.s.PGSQL.5432" >>>> 2020-03-04 00:28:12.928 UTC [36] LOG: database system was shut down at >>>> 2019-12-03 16:35:30 UTC >>>> 2020-03-04 00:28:12.931 UTC [37] postgres@postgres FATAL: the database >>>> system is starting up >>>> psql: error: could not connect to server: FATAL: the database system >>>> is starting up >>>> 2020-03-04 00:28:12.948 UTC [25] LOG: database system is ready to >>>> accept connections >>>> List of databases >>>> Name | Owner | Encoding | Collate | Ctype | Access >>>> privileges >>>> >>>> -----------+----------+----------+---------+---------+----------------------- >>>> postgres | postgres | UTF8 | C.UTF-8 | C.UTF-8 | >>>> template0 | postgres | UTF8 | C.UTF-8 | C.UTF-8 | =c/postgres >>>> + >>>> | | | | | >>>> postgres=CTc/postgres >>>> template1 | postgres | UTF8 | C.UTF-8 | C.UTF-8 | =c/postgres >>>> + >>>> | | | | | >>>> postgres=CTc/postgres >>>> (3 rows) >>>> >>>> postgres ready >>>> Setup postgres User:Password >>>> Creating superuser postgres >>>> ALTER ROLE >>>> Creating replication user replicator >>>> CREATE ROLE >>>> postgres db already exists >>>> List of databases >>>> Name | Owner | Encoding | Collate | Ctype | Access >>>> privileges >>>> >>>> -----------+----------+----------+---------+---------+----------------------- >>>> postgres | postgres | UTF8 | C.UTF-8 | C.UTF-8 | >>>> template0 | postgres | UTF8 | C.UTF-8 | C.UTF-8 | =c/postgres >>>> + >>>> | | | | | >>>> postgres=CTc/postgres >>>> template1 | postgres | UTF8 | C.UTF-8 | C.UTF-8 | =c/postgres >>>> + >>>> | | | | | >>>> postgres=CTc/postgres >>>> (3 rows) >>>> >>>> /docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init.sql >>>> CREATE DATABASE >>>> UPDATE 1 >>>> You are now connected to database "template_postgis" as user "postgres". >>>> CREATE EXTENSION >>>> GRANT >>>> GRANT >>>> GRANT >>>> CREATE EXTENSION >>>> >>>> 2020-03-04 00:28:16.764 UTC [25] LOG: received smart shutdown request >>>> 2020-03-04 00:28:16.783 UTC [25] LOG: background worker "logical >>>> replication launcher" (PID 44) exited with exit code 1 >>>> 2020-03-04 00:28:16.791 UTC [39] LOG: shutting down >>>> 2020-03-04 00:28:16.866 UTC [25] LOG: database system is shut down >>>> Postgres initialisation process completed .... restarting in foreground >>>> >>>> >>>> Why it is different for you I cannot tell right now. I'm using >>>> docker-compose version 1.24.1 (docker engine 19.03.5). >>>> >>>> Ben >>>> >>>> On Tue, 3 Mar 2020 at 15:13, David Osborne <[email protected]> >>>> wrote: >>>> >>>>> I need a new Arches installation to develop some stuff for a >>>>> conference presentation, so I thought I'd take the opportunity to not >>>>> only >>>>> use the latest version but also run it under Docker. >>>>> >>>>> Following the instructions at >>>>> https://github.com/archesproject/arches/blob/master/docker/Readme.md, >>>>> I got as far as Quick Start step 6, docker-compose up, but there were >>>>> problems creating the database for my *archesdemo* project. >>>>> >>>>> I've uploaded my docker-compose.yml file and the results of running >>>>> docker-compose >>>>> up into a gist at >>>>> https://gist.github.com/daosborne/cd501c56dc117e066b4f118f15af7e57 to >>>>> show the problem. Is it obvious what I've missed or is wrong? >>>>> >>>>> I'm running Docker Desktop 2.2.0.3 on MacOS 10.14.6 Mojave. >>>>> >>>>> Thanks! >>>>> David >>>>> >>>>> -- >>>>> -- To post, send email to [email protected]. To unsubscribe, >>>>> send email to [email protected]. For more information, visit >>>>> https://groups.google.com/d/forum/archesproject?hl=en >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Arches Project" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/archesproject/5f42603a-ca04-4b8f-8e90-92af93d20b40%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/archesproject/5f42603a-ca04-4b8f-8e90-92af93d20b40%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>> -- To post, send email to [email protected]. To unsubscribe, >>> send email to [email protected]. For more information, visit >>> https://groups.google.com/d/forum/archesproject?hl=en >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Arches Project" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/archesproject/7faf5315-c7ae-4aa1-881c-c21378266483%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/archesproject/7faf5315-c7ae-4aa1-881c-c21378266483%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- -- To post, send email to [email protected]. To unsubscribe, send email to [email protected]. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en --- You received this message because you are subscribed to the Google Groups "Arches Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/archesproject/d66ef518-c2a2-42b7-964e-4f1ecb6bbd77%40googlegroups.com.
