It would be better, and thanks for pointing it out. I'm surprised it's not set up that way already. I'll that as a ticket on bitbucket.
Dennis should be getting back to you on setting up a time to chat via Skype. Best, Adam -- Adam Lodge Geospatial Systems Consultant Farallon Geographics 415.317.6625 On Tuesday, April 1, 2014 at 9:16 AM, Tobias Kohr wrote: > OK that was it. Authority Files need to be encoded in ANSI/ASCII for Arches > to import them. Wouldn't it be better to use UTF-8? > > Cheers, > Tobias > > > > 2014-04-01 17:48 GMT+02:00 Tobias Kohr <[email protected] > (mailto:[email protected])>: > > Adding this line > > > > COMPONENT CERTAINTY TYPE.E55,COMPONENT CERTAINTY TYPE AUTHORITY > > DOCUMENT.csv,EH SOURCE DATA > > > > to ENTITY_TYPE_X_ADOC.csv doesn't solve the problem. What does > > authoritydocconceptschemename refer to? The DB scheme? > > > > The Authority Files should be encoded in UTF-8 (on Linux), right? Does > > with/without BOM make a difference? > > Notepad++ on Windows tells me that the Authority Files of the default cds > > package are encoded in ANSI, however, which surprises me. > > I will try to see if the error is an encoding problem. > > > > Best, > > Tobias > > > > > > > > 2014-04-01 17:16 GMT+02:00 Adam Lodge <[email protected] > > (mailto:[email protected])>: > > > > > I'm not sure, but it's the first thing I'd try. > > > Adam > > > On Apr 1, 2014 8:14 AM, "Tobias Kohr" <[email protected] > > > (mailto:[email protected])> wrote: > > > > Adam, > > > > > > > > alright, I was not aware of the existence of this file (using Arches > > > > v2). I guess we need include our newly defined entity type here. Is > > > > this the reason for the error? > > > > > > > > Thanks! > > > > -Tobias > > > > > > > > > > > > > > > > 2014-04-01 17:02 GMT+02:00 Adam Lodge <[email protected] > > > > (mailto:[email protected])>: > > > > > Tobias, > > > > > > > > > > No such thing as a stupid question. > > > > > > > > > > Assuming that you are running v2 of Arches, the file should exist in > > > > > this folder: source_data\concepts\authority_files , and its name is > > > > > actually ENTITY_TYPE_X_ADOC.csv > > > > > > > > > > If you're running an earlier version, just send me COMPONENT > > > > > CERTAINTY TYPE AUTHORITY DOCUMENT.CSV and we'll start there. > > > > > > > > > > -- > > > > > Adam Lodge > > > > > Geospatial Systems Consultant > > > > > Farallon Geographics > > > > > 415.317.6625 (tel:415.317.6625) > > > > > > > > > > > > > > > On Tuesday, April 1, 2014 at 7:57 AM, Tobias Kohr wrote: > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > I feel a little stupid asking this, but which file do you mean with > > > > > > ENTITY_TYPE_X_AUTHDOC.csv? (Perhaps already the solution to my > > > > > > problem?) > > > > > > > > > > > > Cheers, > > > > > > Tobias > > > > > > > > > > > > On Tuesday, April 1, 2014 4:46:28 PM UTC+2, Adam Lodge wrote: > > > > > > > Tobias, > > > > > > > > > > > > > > Could you send me a the COMPONENT CERTAINTY TYPE AUTHORITY > > > > > > > DOCUMENT.CSV file and the ENTITY_TYPE_X_AUTHDOC.csv file? With > > > > > > > those, I can probably tell you what the issue is. > > > > > > > > > > > > > > Adam > > > > > > > > > > > > > > -- > > > > > > > Adam Lodge > > > > > > > Geospatial Systems Consultant > > > > > > > Farallon Geographics > > > > > > > 415.317.6625 (tel:415.317.6625) > > > > > > > > > > > > > > > > > > > > > On Tuesday, April 1, 2014 at 7:43 AM, Tobias Kohr wrote: > > > > > > > > > > > > > > > Hi Dennis, Koen, et al. > > > > > > > > > > > > > > > > we will have a closer look at the CIDOC extension and try to > > > > > > > > keep in mind that people have different interpretations for > > > > > > > > uncerainty. > > > > > > > > > > > > > > > > Regarding the technical implementation we're encountering > > > > > > > > problems in step 3, running install_packages.sh > > > > > > > > (http://install_packages.sh) which throws the following error: > > > > > > > > > > > > > > > > root@srv-i3-fundstellendb:/arches-web/archesproject/build# > > > > > > > > source install_packages.sh (http://install_packages.sh) > > > > > > > > Install packages defined in settings.py > > > > > > > > operation: install > > > > > > > > ...||ABSOLUTE DATING METHOD AUTHORITY DOCUMENT.CSV > > > > > > > > ADMINISTRATIVE SUBDIVISION AUTHORITY DOCUMENT.CSV > > > > > > > > ADMINISTRATIVE SUBDIVISION AUTHORITY DOCUMENT.VALUES.CSV > > > > > > > > ADMINISTRATIVE SUBDIVISION TYPE AUTHORITY DOCUMENT.CSV > > > > > > > > ARCHAEOLOGICAL COMPONENT TYPE AUTHORITY DOCUMENT.CSV > > > > > > > > ARCHAEOLOGICAL HERITAGE (ARTIFACT) TYPE AUTHORITY DOCUMENT.CSV > > > > > > > > ARCHAEOLOGICAL HERITAGE (SITE) TYPE AUTHORITY DOCUMENT.CSV > > > > > > > > ARCHAEOLOGICAL TECHNIQUE AUTHORITY DOCUMENT.CSV > > > > > > > > ARCHES RESOURCE CROSS-REFERENCE RELATIONSHIP TYPE AUTHORITY > > > > > > > > DOCUMENT.CSV > > > > > > > > ARCHES RESOURCE CROSS-REFERENCE RELATIONSHIP TYPE AUTHORITY > > > > > > > > DOCUMENT.VALUES.CSV > > > > > > > > ARCHITECTURAL COMPONENT TYPE AUTHORITY DOCUMENT.CSV > > > > > > > > ARCHITECTURAL HERITAGE TYPE AUTHORITY DOCUMENT.CSV > > > > > > > > ARCHITECTURAL TECHNIQUE AUTHORITY DOCUMENT.CSV > > > > > > > > COMPONENT CERTAINTY TYPE AUTHORITY DOCUMENT.CSV > > > > > > > > Traceback (most recent call last): > > > > > > > > File "../manage.py", line 28, in <module> > > > > > > > > execute_from_command_line(sys.argv) > > > > > > > > File > > > > > > > > "/arches-web/archesproject/virtualenv/ENV/local/lib/python2.7/site-packages/django/core/management/__init__.py", > > > > > > > > line 399, in execute_from_command_line > > > > > > > > utility.execute() > > > > > > > > File > > > > > > > > "/arches-web/archesproject/virtualenv/ENV/local/lib/python2.7/site-packages/django/core/management/__init__.py", > > > > > > > > line 392, in execute > > > > > > > > self.fetch_command(subcommand).run_from_argv(self.argv) > > > > > > > > File > > > > > > > > "/arches-web/archesproject/virtualenv/ENV/local/lib/python2.7/site-packages/django/core/management/base.py", > > > > > > > > line 242, in run_from_argv > > > > > > > > self.execute(*args, **options.__dict__) > > > > > > > > File > > > > > > > > "/arches-web/archesproject/virtualenv/ENV/local/lib/python2.7/site-packages/django/core/management/base.py", > > > > > > > > line 285, in execute > > > > > > > > output = self.handle(*args, **options) > > > > > > > > File > > > > > > > > "/arches-web/archesproject/build/management/commands/packages.py", > > > > > > > > line 47, in handle > > > > > > > > self.load_package(package) > > > > > > > > File > > > > > > > > "/arches-web/archesproject/build/management/commands/packages.py", > > > > > > > > line 52, in load_package > > > > > > > > install(settings.ROOT_DIR) > > > > > > > > File "/arches-web/archesproject/packages/cdscert/setup.py", > > > > > > > > line 60, in install > > > > > > > > > > > > > > > > authority_files.load_authority_files(package_settings.ROOT_DIR) > > > > > > > > File > > > > > > > > "/arches-web/archesproject/packages/cdscert/install/authority_files.py", > > > > > > > > line 22, in load_authority_files > > > > > > > > load_authority_file(cursor, mapping_files_directory, > > > > > > > > file_name) > > > > > > > > File > > > > > > > > "/arches-web/archesproject/packages/cdscert/install/authority_files.py", > > > > > > > > line 63, in load_authority_file > > > > > > > > concepts.insert_concept(settings.DATA_CONCEPT_SCHEME, > > > > > > > > adoc_dict['PREFLABEL'], '', 'en-us', adoc_dict['CONCEPTID']) > > > > > > > > KeyError: 'CONCEPTID' > > > > > > > > > > > > > > > > Our provisional Authority Files look like this: > > > > > > > > > > > > > > > > - COMPONENT CERTAINTY TYPE AUTHORITY DOCUMENT.csv > > > > > > > > conceptid,PrefLabel,AltLabels,ParentConceptid,ConceptType,Provider > > > > > > > > > > > > > > > > COMPONENT_CERTAINTY_1,certain,,COMPONENT CERTAINTY TYPE > > > > > > > > AUTHORITY DOCUMENT.csv,Index,i3mainz > > > > > > > > COMPONENT_CERTAINTY_2,uncertain,,COMPONENT CERTAINTY TYPE > > > > > > > > AUTHORITY DOCUMENT.csv,Index,i3mainz > > > > > > > > COMPONENT_CERTAINTY_3,unknown,,COMPONENT CERTAINTY TYPE > > > > > > > > AUTHORITY DOCUMENT.csv,Index,i3mainz > > > > > > > > > > > > > > > > - COMPONENT CERTAINTY TYPE AUTHORITY DOCUMENT.values.csv (do we > > > > > > > > need this one?) > > > > > > > > conceptid,Value,ValueType,Provider > > > > > > > > COMPONENT_CERTAINTY_1,1,sortorder,i3mainz > > > > > > > > COMPONENT_CERTAINTY_2,2,sortorder,i3mainz > > > > > > > > COMPONENT_CERTAINTY_3,3,sortorder,i3mainz > > > > > > > > > > > > > > > > Can anybody tell us what's wrong with our conceptid? Does the > > > > > > > > authority_files.py search for the ID in any additional place, > > > > > > > > where we should reference it? > > > > > > > > > > > > > > > > Cheers, > > > > > > > > Tobias > > > > > > > > > > > > > > > > On Monday, March 31, 2014 11:22:27 PM UTC+2, Koen Van Daele > > > > > > > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > I just wanted to get back at what Dennis said at the > > > > > > > > > beginning of this thread. Im quite curious how you will get > > > > > > > > > people to agree on (un)certainty. If feels like a very > > > > > > > > > natural idea to talk and think about, but I haven't really > > > > > > > > > seen it function properly in practice. > > > > > > > > > > > > > > > > > > We once did an experiment where we had 10 people who were > > > > > > > > > used to entering data in our archaeological inventory system > > > > > > > > > enter the same site. We paired the archaeologists: one more > > > > > > > > > more experienced data entry person (a few years experience) > > > > > > > > > and one newbie (a few months), so they would be forced to > > > > > > > > > really think things through and discuss. In our database we > > > > > > > > > have a field for certain the data entry person is about the > > > > > > > > > location of the site, ie. about the polygon they might have > > > > > > > > > drawn on a map. This field only allowed 5 choices, ranging > > > > > > > > > from 1 (I'm sure it's exactly where it needs to be) to 5 (I > > > > > > > > > have no idea whatsoever where the site is). We had a very > > > > > > > > > detailed manual with examples of all these cases, what to use > > > > > > > > > when, ... > > > > > > > > > Final result of our experiment: every group had entered the > > > > > > > > > location with a different level of certainty. So, based on > > > > > > > > > the exact same information they had all drawn totally > > > > > > > > > different conclusions. And this was about something as simple > > > > > > > > > as the location of the site. > > > > > > > > > > > > > > > > > > So, I'm very curious about how you manage to prevent stuff > > > > > > > > > like this from happening. > > > > > > > > > > > > > > > > > > The other thing I wonder about: how does certainty affect > > > > > > > > > searching? Should a search for 'churches' only return sites > > > > > > > > > that have a certain "certainty" attached to the > > > > > > > > > interpretation? Are you working with sliding scale of > > > > > > > > > certainty (ie. we are 75% percent certain about this > > > > > > > > > statement) or a binary one (we're certain or uncertain)? > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > Koen > > > > > > > > > ________________________________________ > > > > > > > > > Van: [email protected] [[email protected]] > > > > > > > > > namens [email protected] [[email protected]] > > > > > > > > > Verzonden: donderdag 27 maart 2014 22:36 > > > > > > > > > Aan: [email protected] > > > > > > > > > CC: [email protected] > > > > > > > > > Onderwerp: Re: [Arches] "uncertain information" in Arches > > > > > > > > > > > > > > > > > > Thomas, > > > > > > > > > > > > > > > > > > Good question! You are quite correct that we haven’t tried > > > > > > > > > to include uncertainty in Arches. > > > > > > > > > > > > > > > > > > One reason is pretty basic: certainty is quite subjective > > > > > > > > > from person to person. For example: most people agree that > > > > > > > > > the earth is spherical. But a “flat-earther” may be very > > > > > > > > > certain that the earth is not a sphere, but is instead a > > > > > > > > > plane. His certainty does not make him correct, it merely > > > > > > > > > states the degree to which he believes in his interpretation. > > > > > > > > > Clearly, you can be very certain and very wrong at the same > > > > > > > > > time. I guess my point is that in many cases “certainty” > > > > > > > > > says more about the person making the assertion than it does > > > > > > > > > about the thing being described. > > > > > > > > > > > > > > > > > > OK, all philosophy aside, one could easily extend any Arches > > > > > > > > > graph to include a “certainty node”. Such a node could point > > > > > > > > > to a thesaurus (as many of the nodes in Arches already do), > > > > > > > > > allowing a user to select from a list of “uncertainty > > > > > > > > > levels”. Really, any Arches graph could include a “certainty > > > > > > > > > node” under any entity that you might want to qualify (for > > > > > > > > > example, one certainty node for period and another certainty > > > > > > > > > node for heritage type). > > > > > > > > > > > > > > > > > > Really, the hard part is not in getting Arches to allow you > > > > > > > > > to add an “uncertainty level” to your cultural heritage data. > > > > > > > > > Rather, the difficult thing is to decide how you’ll get > > > > > > > > > different people to agree on what constitutes certain vs. > > > > > > > > > uncertain interpretations of heritage. > > > > > > > > > > > > > > > > > > Sorry that I can’t be any more helpful… However, I’m very > > > > > > > > > interested to hear how you will model uncertainty and how you > > > > > > > > > will get people to implement it consistently. Please keep me > > > > > > > > > posted! > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > > > > > > > > Dennis > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mar 27, 2014, at 2:41, > > > > > > > > > [email protected]<mailto:[email protected]> wrote: > > > > > > > > > > > > > > > > > > I have a question about conceptual modeling in CIDOC CRM, > > > > > > > > > maybe there is someone one the list who is able to provide > > > > > > > > > some guidance. > > > > > > > > > > > > > > > > > > As posted before, we are trying to integrate research data of > > > > > > > > > neolithic sites into Arches. Now, naturally a significant > > > > > > > > > part of this data has a level of "certainty" to which the > > > > > > > > > information is correct. e.g. a site can consist of some > > > > > > > > > features for certain (in this case modeled in the > > > > > > > > > Archaeological Heritage (Site).E27 - Component.E18 > > > > > > > > > relationship) but if others exist is uncertain. We believe > > > > > > > > > this valuable information should not get lost (quite often > > > > > > > > > theory construction is based on such information). > > > > > > > > > > > > > > > > > > For example it could be uncertain if an archaeological > > > > > > > > > feature is to be named "pit" or "ditch" - or if it exists at > > > > > > > > > all. Another example could be the questionable relationship > > > > > > > > > of a findspot to a certain archaeological period. To make it > > > > > > > > > even more difficult, different authors could have different > > > > > > > > > thoughts on that. > > > > > > > > > > > > > > > > > > As far as we can see, the expression of such "uncertainty" is > > > > > > > > > not covered by Arches yet. Is there a concept for the > > > > > > > > > integration of such data in the future? We are currently > > > > > > > > > thinking into potential solutions but are struggeling to find > > > > > > > > > adequate expressions for uncertain information in CIDOC. > > > > > > > > > > > > > > > > > > thanks, Thomas > > > > > > > > > > > > > > > > > > -- > > > > > > > > > -- To post, send email to > > > > > > > > > [email protected]<mailto:[email protected]>. > > > > > > > > > To unsubscribe, send email to > > > > > > > > > [email protected]<mailto:[email protected]>. > > > > > > > > > For more information, visit > > > > > > > > > https://groups.google.com/d/forum/archesproject?hl=en > > > > > > > > > --- You received this message because you are subscribed to > > > > > > > > > the Google Groups "Arches Project" group. > > > > > > > > > To unsubscribe from this group and stop receiving emails from > > > > > > > > > it, send an email to > > > > > > > > > [email protected]<mailto:[email protected]>. > > > > > > > > > > > > > > > > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > -- To post, send email to [email protected]. To > > > > > > > > > unsubscribe, send email to [email protected]. > > > > > > > > > For more information, visit > > > > > > > > > https://groups.google.com/d/forum/archesproject?hl=en > > > > > > > > > --- > > > > > > > > > You received this message because you are subscribed to the > > > > > > > > > Google Groups "Arches Project" group. > > > > > > > > > To unsubscribe from this group and stop receiving emails from > > > > > > > > > it, send an email to > > > > > > > > > [email protected]<mailto:[email protected]>. > > > > > > > > > > > > > > > > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > > > > > > > -- > > > > > > > > -- To post, send email to [email protected]. To > > > > > > > > unsubscribe, send email to [email protected]. > > > > > > > > For more information, visit > > > > > > > > https://groups.google.com/d/forum/archesproject?hl=en > > > > > > > > --- > > > > > > > > You received this message because you are subscribed to the > > > > > > > > Google Groups "Arches Project" group. > > > > > > > > To unsubscribe from this group and stop receiving emails from > > > > > > > > it, send an email to [email protected]. > > > > > > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > > > > -- > > > > > > -- To post, send email to [email protected] > > > > > > (mailto:[email protected]). To unsubscribe, send email > > > > > > to [email protected] > > > > > > (mailto:[email protected]). For more > > > > > > information, visit > > > > > > https://groups.google.com/d/forum/archesproject?hl=en > > > > > > > > > > > > --- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "Arches Project" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > send an email to [email protected] > > > > > > (mailto:[email protected]). > > > > > > > > > > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > > > -- -- To post, send email to [email protected]. To unsubscribe, send email to [email protected]. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en --- You received this message because you are subscribed to the Google Groups "Arches Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
