OK that was it. Authority Files need to be encoded in ANSI/ASCII for Arches to import them. Wouldn't it be better to use UTF-8?
Cheers, Tobias 2014-04-01 17:48 GMT+02:00 Tobias Kohr <[email protected]>: > Adding this line > > COMPONENT CERTAINTY TYPE.E55,COMPONENT CERTAINTY TYPE AUTHORITY > DOCUMENT.csv,EH SOURCE DATA > > to ENTITY_TYPE_X_ADOC.csv doesn't solve the problem. What > does authoritydocconceptschemename refer to? The DB scheme? > > The Authority Files should be encoded in UTF-8 (on Linux), right? Does > with/without BOM make a difference? > Notepad++ on Windows tells me that the Authority Files of the default cds > package are encoded in ANSI, however, which surprises me. > I will try to see if the error is an encoding problem. > > Best, > Tobias > > > 2014-04-01 17:16 GMT+02:00 Adam Lodge <[email protected]>: > > I'm not sure, but it's the first thing I'd try. >> >> Adam >> On Apr 1, 2014 8:14 AM, "Tobias Kohr" <[email protected]> wrote: >> >>> Adam, >>> >>> alright, I was not aware of the existence of this file (using Arches >>> v2). I guess we need include our newly defined entity type here. Is this >>> the reason for the error? >>> >>> Thanks! >>> -Tobias >>> >>> >>> 2014-04-01 17:02 GMT+02:00 Adam Lodge <[email protected]>: >>> >>>> Tobias, >>>> >>>> No such thing as a stupid question. >>>> >>>> Assuming that you are running v2 of Arches, the file should exist in >>>> this folder: source_data\concepts\authority_files , and its name is >>>> actually ENTITY_TYPE_X_ADOC.csv >>>> >>>> If you're running an earlier version, just send me COMPONENT CERTAINTY >>>> TYPE AUTHORITY DOCUMENT.CSV and we'll start there. >>>> >>>> -- >>>> Adam Lodge >>>> Geospatial Systems Consultant >>>> Farallon Geographics >>>> 415.317.6625 >>>> >>>> On Tuesday, April 1, 2014 at 7:57 AM, Tobias Kohr wrote: >>>> >>>> Hi, >>>> >>>> I feel a little stupid asking this, but which file do you mean >>>> with ENTITY_TYPE_X_AUTHDOC.csv? (Perhaps already the solution to my >>>> problem?) >>>> >>>> Cheers, >>>> Tobias >>>> >>>> On Tuesday, April 1, 2014 4:46:28 PM UTC+2, Adam Lodge wrote: >>>> >>>> Tobias, >>>> >>>> Could you send me a the COMPONENT CERTAINTY TYPE AUTHORITY DOCUMENT.CSV >>>> file and the ENTITY_TYPE_X_AUTHDOC.csv file? With those, I can probably >>>> tell you what the issue is. >>>> >>>> Adam >>>> >>>> -- >>>> Adam Lodge >>>> Geospatial Systems Consultant >>>> Farallon Geographics >>>> 415.317.6625 >>>> >>>> On Tuesday, April 1, 2014 at 7:43 AM, Tobias Kohr wrote: >>>> >>>> Hi Dennis, Koen, et al. >>>> >>>> we will have a closer look at the CIDOC extension and try to keep in >>>> mind that people have different interpretations for uncerainty. >>>> >>>> Regarding the technical implementation we're encountering problems in >>>> step 3, running install_packages.sh which throws the following error: >>>> >>>> root@srv-i3-fundstellendb:/arches-web/archesproject/build# source >>>> install_packages.sh >>>> Install packages defined in settings.py >>>> operation: install >>>> ...||ABSOLUTE DATING METHOD AUTHORITY DOCUMENT.CSV >>>> ADMINISTRATIVE SUBDIVISION AUTHORITY DOCUMENT.CSV >>>> ADMINISTRATIVE SUBDIVISION AUTHORITY DOCUMENT.VALUES.CSV >>>> ADMINISTRATIVE SUBDIVISION TYPE AUTHORITY DOCUMENT.CSV >>>> ARCHAEOLOGICAL COMPONENT TYPE AUTHORITY DOCUMENT.CSV >>>> ARCHAEOLOGICAL HERITAGE (ARTIFACT) TYPE AUTHORITY DOCUMENT.CSV >>>> ARCHAEOLOGICAL HERITAGE (SITE) TYPE AUTHORITY DOCUMENT.CSV >>>> ARCHAEOLOGICAL TECHNIQUE AUTHORITY DOCUMENT.CSV >>>> ARCHES RESOURCE CROSS-REFERENCE RELATIONSHIP TYPE AUTHORITY DOCUMENT.CSV >>>> ARCHES RESOURCE CROSS-REFERENCE RELATIONSHIP TYPE AUTHORITY >>>> DOCUMENT.VALUES.CSV >>>> ARCHITECTURAL COMPONENT TYPE AUTHORITY DOCUMENT.CSV >>>> ARCHITECTURAL HERITAGE TYPE AUTHORITY DOCUMENT.CSV >>>> ARCHITECTURAL TECHNIQUE AUTHORITY DOCUMENT.CSV >>>> COMPONENT CERTAINTY TYPE AUTHORITY DOCUMENT.CSV >>>> Traceback (most recent call last): >>>> File "../manage.py", line 28, in <module> >>>> execute_from_command_line(sys.argv) >>>> File "/arches-web/archesproject/virtualenv/ENV/local/lib/ >>>> python2.7/site-packages/django/core/management/__init__.py", line 399, >>>> in execute_from_command_line >>>> utility.execute() >>>> File "/arches-web/archesproject/virtualenv/ENV/local/lib/ >>>> python2.7/site-packages/django/core/management/__init__.py", line 392, >>>> in execute >>>> self.fetch_command(subcommand).run_from_argv(self.argv) >>>> File "/arches-web/archesproject/virtualenv/ENV/local/lib/ >>>> python2.7/site-packages/django/core/management/base.py", line 242, in >>>> run_from_argv >>>> self.execute(*args, **options.__dict__) >>>> File "/arches-web/archesproject/virtualenv/ENV/local/lib/ >>>> python2.7/site-packages/django/core/management/base.py", line 285, in >>>> execute >>>> output = self.handle(*args, **options) >>>> File "/arches-web/archesproject/build/management/commands/packages.py", >>>> line 47, in handle >>>> self.load_package(package) >>>> File "/arches-web/archesproject/build/management/commands/packages.py", >>>> line 52, in load_package >>>> install(settings.ROOT_DIR) >>>> File "/arches-web/archesproject/packages/cdscert/setup.py", line 60, >>>> in install >>>> authority_files.load_authority_files(package_settings.ROOT_DIR) >>>> File >>>> "/arches-web/archesproject/packages/cdscert/install/authority_files.py", >>>> line 22, in load_authority_files >>>> load_authority_file(cursor, mapping_files_directory, file_name) >>>> File >>>> "/arches-web/archesproject/packages/cdscert/install/authority_files.py", >>>> line 63, in load_authority_file >>>> concepts.insert_concept(settings.DATA_CONCEPT_SCHEME, >>>> adoc_dict['PREFLABEL'], '', 'en-us', adoc_dict['CONCEPTID']) >>>> KeyError: 'CONCEPTID' >>>> >>>> Our provisional Authority Files look like this: >>>> >>>> - COMPONENT CERTAINTY TYPE AUTHORITY DOCUMENT.csv >>>> conceptid,PrefLabel,AltLabels,ParentConceptid,ConceptType,Provider >>>> COMPONENT_CERTAINTY_1,certain,,COMPONENT CERTAINTY TYPE AUTHORITY >>>> DOCUMENT.csv,Index,i3mainz >>>> COMPONENT_CERTAINTY_2,uncertain,,COMPONENT CERTAINTY TYPE AUTHORITY >>>> DOCUMENT.csv,Index,i3mainz >>>> COMPONENT_CERTAINTY_3,unknown,,COMPONENT CERTAINTY TYPE AUTHORITY >>>> DOCUMENT.csv,Index,i3mainz >>>> >>>> - COMPONENT CERTAINTY TYPE AUTHORITY DOCUMENT.values.csv (do we need >>>> this one?) >>>> conceptid,Value,ValueType,Provider >>>> COMPONENT_CERTAINTY_1,1,sortorder,i3mainz >>>> COMPONENT_CERTAINTY_2,2,sortorder,i3mainz >>>> COMPONENT_CERTAINTY_3,3,sortorder,i3mainz >>>> >>>> Can anybody tell us what's wrong with our conceptid? Does the >>>> authority_files.py search for the ID in any additional place, where we >>>> should reference it? >>>> >>>> Cheers, >>>> Tobias >>>> >>>> On Monday, March 31, 2014 11:22:27 PM UTC+2, Koen Van Daele wrote: >>>> >>>> Hi all, >>>> >>>> I just wanted to get back at what Dennis said at the beginning of this >>>> thread. Im quite curious how you will get people to agree on (un)certainty. >>>> If feels like a very natural idea to talk and think about, but I haven't >>>> really seen it function properly in practice. >>>> >>>> We once did an experiment where we had 10 people who were used to >>>> entering data in our archaeological inventory system enter the same site. >>>> We paired the archaeologists: one more more experienced data entry person >>>> (a few years experience) and one newbie (a few months), so they would be >>>> forced to really think things through and discuss. In our database we have >>>> a field for certain the data entry person is about the location of the >>>> site, ie. about the polygon they might have drawn on a map. This field only >>>> allowed 5 choices, ranging from 1 (I'm sure it's exactly where it needs to >>>> be) to 5 (I have no idea whatsoever where the site is). We had a very >>>> detailed manual with examples of all these cases, what to use when, ... >>>> Final result of our experiment: every group had entered the location >>>> with a different level of certainty. So, based on the exact same >>>> information they had all drawn totally different conclusions. And this was >>>> about something as simple as the location of the site. >>>> >>>> So, I'm very curious about how you manage to prevent stuff like this >>>> from happening. >>>> >>>> The other thing I wonder about: how does certainty affect searching? >>>> Should a search for 'churches' only return sites that have a certain >>>> "certainty" attached to the interpretation? Are you working with sliding >>>> scale of certainty (ie. we are 75% percent certain about this statement) or >>>> a binary one (we're certain or uncertain)? >>>> >>>> Cheers, >>>> Koen >>>> ________________________________________ >>>> Van: [email protected] [[email protected]] namens >>>> [email protected] [[email protected]] >>>> Verzonden: donderdag 27 maart 2014 22:36 >>>> Aan: [email protected] >>>> CC: [email protected] >>>> Onderwerp: Re: [Arches] "uncertain information" in Arches >>>> >>>> Thomas, >>>> >>>> Good question! You are quite correct that we haven't tried to include >>>> uncertainty in Arches. >>>> >>>> One reason is pretty basic: certainty is quite subjective from person >>>> to person. For example: most people agree that the earth is spherical. >>>> But a "flat-earther" may be very certain that the earth is not a sphere, >>>> but is instead a plane. His certainty does not make him correct, it merely >>>> states the degree to which he believes in his interpretation. Clearly, you >>>> can be very certain and very wrong at the same time. I guess my point is >>>> that in many cases "certainty" says more about the person making the >>>> assertion than it does about the thing being described. >>>> >>>> OK, all philosophy aside, one could easily extend any Arches graph to >>>> include a "certainty node". Such a node could point to a thesaurus (as >>>> many of the nodes in Arches already do), allowing a user to select from a >>>> list of "uncertainty levels". Really, any Arches graph could include a >>>> "certainty node" under any entity that you might want to qualify (for >>>> example, one certainty node for period and another certainty node for >>>> heritage type). >>>> >>>> Really, the hard part is not in getting Arches to allow you to add an >>>> "uncertainty level" to your cultural heritage data. Rather, the difficult >>>> thing is to decide how you'll get different people to agree on what >>>> constitutes certain vs. uncertain interpretations of heritage. >>>> >>>> Sorry that I can't be any more helpful... However, I'm very interested to >>>> hear how you will model uncertainty and how you will get people to >>>> implement it consistently. Please keep me posted! >>>> >>>> Cheers, >>>> >>>> Dennis >>>> >>>> >>>> On Mar 27, 2014, at 2:41, [email protected]<mailto:th >>>> [email protected]> wrote: >>>> >>>> I have a question about conceptual modeling in CIDOC CRM, maybe there >>>> is someone one the list who is able to provide some guidance. >>>> >>>> As posted before, we are trying to integrate research data of neolithic >>>> sites into Arches. Now, naturally a significant part of this data has a >>>> level of "certainty" to which the information is correct. e.g. a site can >>>> consist of some features for certain (in this case modeled in the >>>> Archaeological Heritage (Site).E27 - Component.E18 relationship) but if >>>> others exist is uncertain. We believe this valuable information should not >>>> get lost (quite often theory construction is based on such information). >>>> >>>> For example it could be uncertain if an archaeological feature is to be >>>> named "pit" or "ditch" - or if it exists at all. Another example could be >>>> the questionable relationship of a findspot to a certain archaeological >>>> period. To make it even more difficult, different authors could have >>>> different thoughts on that. >>>> >>>> As far as we can see, the expression of such "uncertainty" is not >>>> covered by Arches yet. Is there a concept for the integration of such data >>>> in the future? We are currently thinking into potential solutions but are >>>> struggeling to find adequate expressions for uncertain information in >>>> CIDOC. >>>> >>>> thanks, Thomas >>>> >>>> -- >>>> -- To post, send email to [email protected]<mailto: >>>> [email protected]>. To unsubscribe, send email to >>>> [email protected]<mailto:archesproject+ >>>> [email protected]>. For more information, visit >>>> https://groups.google.com/d/forum/archesproject?hl=en >>>> --- You received this message because you are subscribed to the Google >>>> Groups "Arches Project" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]<mailto:archesproject+ >>>> [email protected]>. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> -- >>>> -- To post, send email to [email protected]. To unsubscribe, >>>> send email to [email protected]. For more information, >>>> visit https://groups.google.com/d/forum/archesproject?hl=en >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Arches Project" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]<mailto:archesproject+ >>>> [email protected]>. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> -- >>>> -- To post, send email to [email protected]. To unsubscribe, >>>> send email to [email protected]. For more information, >>>> visit https://groups.google.com/d/forum/archesproject?hl=en >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Arches Project" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> -- >>>> -- To post, send email to [email protected]. To >>>> unsubscribe, send email to [email protected]. >>>> For more information, visit >>>> https://groups.google.com/d/forum/archesproject?hl=en >>>> >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Arches Project" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> >>> > -- -- To post, send email to [email protected]. To unsubscribe, send email to [email protected]. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en --- You received this message because you are subscribed to the Google Groups "Arches Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
