Just to wrap up this conversation. Gabriel was able to resolve the issue. It turned he had tile data spanning multiple lines within a single cell, so Arches was unable to read the data properly.
- Cyrus On Tuesday, February 2, 2021 at 8:07:48 AM UTC-8 [email protected] wrote: > > Hi, > > The permissions were solved, thanks all for the tips. Problem I have > still problems when loading and saving the csv in arches... despite it > shows that some records were saved they doesnt show in the app > > arches.app.utils.data_management.resources.formats.csvfile WARNING Grid > ID is null or not mapped on rows > 2,2,2,3,3,3,4,4,4,5,5,5,6,6,6,7,7,7,8,8,8,9,9,9,10,10,10,11,11,11,12,12,12,13,13,13,14,14,14,15,15,15,16,16,16,17,17,17 > > and participates in a display value > function. > Time to import_business_data = 0:00:01.989878 > ***** Errors occured during import. Some data may not have been imported. > For more information, check resource import error log: > /home/archesdev/arches/eamena/eamena/logs/resource_import.log > > > The log just show the same message so it doesnt display any additional > information. Someone have experienced the same issue or similar? > > Thanks in advance > > On Thursday, 28 January 2021 at 21:03:50 UTC Markus S. wrote: > >> Hi, >> >> It looks like it was successfully able to save 6384 records but was not >> able to write the log entry. Can you confirm the import itself was >> successful? >> >> In addition to file permissions, also ensure that the folder in which the >> file is located has sufficient permissions so that files inside of it can >> be accessed. >> >> Markus >> >> On Thursday, January 28, 2021 at 7:37:19 AM UTC-8 [email protected] >> wrote: >> >>> Hi, >>> >>> In the absence of a Linux expert . . . >>> >>> I would suggest changing permissions (ie chmod) temporarily to 777 and >>> if that makes the difference you will know you are looking in the right >>> place. The first number is permissions for the user associated with the >>> file, the second is for the user group associated with the file and the >>> third is for anybody. >>> >>> Each of the digits refers to permissions for different users/groups.The >>> first will be used if the user that is trying to write to the file is that >>> associated with the file. The second would be used for a different user, >>> but one that is a member of the group associated with the file. The last >>> will work for anybody. Normally the owner/group would have more rights than >>> the general public. >>> >>> Each digit is made up of 4+2+1 where the 4 is read permission, 2 is >>> write permission (which you want in this case), 1 is execute. >>> Thus 7 is all three; 6 is 4+2 so read and write but not execute; 5 is >>> read and execute but not write. >>> >>> You can use the linux command: >>> $ ls -la >>> to list the files in the current working directory and see the names of >>> the associated users and group as well as the permissions. I get: >>> -rw-rw-r-- 1 ubuntu ubuntu 0 Dec 4 19:39 resource_import.log >>> for that file, which is 664 and the user and group are both ubuntu (on >>> an AWS implementation) >>> >>> I am fairly new to Linux myself so not able to advise on finding out >>> which user is trying to do the writing, or what group they belong to; >>> perhaps someone else can. >>> >>> If this does allow writing, you can experiment with setting permissions >>> to 700, 070, 007 in turn to see whether the user is the file's owner, a >>> member of the group (which is usually the same as the owner - linux creates >>> a group with the same name and just that user when a new user is created), >>> or anybody, ie a different user. >>> >>> I suspect you might have problems beyond that. The fact that you had a >>> problem writing to the log probably means that the process had a problem >>> that it wanted to write about in the log. But sorting this first will >>> hopefully let you get a clearer look at the underlying problem. >>> >>> Good luck, >>> Pat >>> On Thursday, 28 January 2021 at 14:54:44 UTC >>> [email protected] wrote: >>> >>>> I am afraid I am not a linux expert - we use Windows. Maybe someone >>>> else with linux knowledge can help? >>>> >>>> On Thursday, January 28, 2021 at 2:32:01 PM UTC [email protected] >>>> wrote: >>>> >>>>> >>>>> Hi thanks for coming back to me, yes I tried to change the permisiions >>>>> but still receiving the same error. >>>>> >>>>> sudo chmod 775 and 755 to the file getting permission: >>>>> -rwxr-xr-x 1 >>>>> >>>>> >>>>> >>>>> On Thursday, 28 January 2021 at 13:52:18 UTC >>>>> [email protected] wrote: >>>>> >>>>>> The last line of the error is stating that it cannot write to the >>>>>> resource_import.log. Check write permissions on this. >>>>>> >>>>>> f = open(settings.RESOURCE_IMPORT_LOG, "a") >>>>>> PermissionError: [Errno 13] Permission denied: >>>>>> '/home/archesdev/arches/eamena/eamena/logs/resource_import.log' >>>>>> >>>>>> On Thursday, January 28, 2021 at 12:12:10 PM UTC >>>>>> [email protected] wrote: >>>>>> >>>>>>> Thanks it runs but it return few issues, that in local didnt. >>>>>>> >>>>>>> otal resources saved: 6384 >>>>>>> 2021-01-28 05:48:59,707 >>>>>>> arches.app.utils.data_management.resources.formats.csvfile WARNING >>>>>>> Grid ID >>>>>>> is null or not mapped on rows >>>>>>> 2,2,2,3,3,3,4,4,4,5,5,5,6,6,6,7,7,7,8,8,8,9,9,9,10,10,10,11,11,11,12,12,12,13,13,13,14,14,14,15,15,15,16,16,16,17,17,17,18,18,18,19,19,19,20,20,20,21,21,21,22,22,22,23,23,23,24,24,24,25,25,25,26,26,26,27,27,27,28,28,28,29,29,29,30,30,30,31,31,31,32,32,32,33,33,33,34,34,34,35,35,35,36,36,36,37,37,37,38,38,38,39,39,39,40,40,40,41,41,41,42,42,42,43,43,43,44,44,44,45,45,45,46,46,46,47,47,47,48,48,48,49,49,49,50,50,50,51,51,51,52,52,52,53,53,53,54,54,54,55,55,55,56,56,56,57,57,57,58,58,58,59,59,59,60,60,60,61,61,61,62,62,62,63,63,63,64,64,64,65,65,6.......... >>>>>>> >>>>>>> and participates in a display value >>>>>>> function. >>>>>>> Time to import_business_data = 0:08:29.653874 >>>>>>> ***** Errors occured during import. Some data may not have been >>>>>>> imported. For more information, check resource import error log: >>>>>>> /home/archesdev/arches/eamena/eamena/logs/resource_import.log >>>>>>> 2021-01-28 05:49:00,812 arches.app.utils.task_management ERROR A >>>>>>> celery broker is running, but a celery worker is not available >>>>>>> Traceback (most recent call last): >>>>>>> File "manage.py", line 30, in <module> >>>>>>> execute_from_command_line(sys.argv) >>>>>>> File >>>>>>> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/__init__.py", >>>>>>> >>>>>>> line 381, in execute_from_command_line >>>>>>> utility.execute() >>>>>>> File >>>>>>> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/__init__.py", >>>>>>> >>>>>>> line 375, in execute >>>>>>> self.fetch_command(subcommand).run_from_argv(self.argv) >>>>>>> File >>>>>>> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/base.py", >>>>>>> >>>>>>> line 323, in run_from_argv >>>>>>> self.execute(*args, **cmd_options) >>>>>>> File >>>>>>> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/base.py", >>>>>>> >>>>>>> line 364, in execute >>>>>>> output = self.handle(*args, **options) >>>>>>> File >>>>>>> "/home/archesdev/arches/arches/arches/management/commands/packages.py", >>>>>>> line 277, in handle >>>>>>> prevent_indexing=options["prevent_indexing"], >>>>>>> File >>>>>>> "/home/archesdev/arches/arches/arches/management/commands/packages.py", >>>>>>> line 1070, in import_business_data >>>>>>> prevent_indexing=prevent_indexing, >>>>>>> File >>>>>>> "/home/archesdev/arches/arches/arches/app/utils/data_management/resources/importer.py", >>>>>>> >>>>>>> line 217, in import_business_data >>>>>>> reader.report_errors() >>>>>>> File >>>>>>> "/home/archesdev/arches/arches/arches/app/utils/data_management/resources/formats/format.py", >>>>>>> >>>>>>> line 178, in report_errors >>>>>>> f = open(settings.RESOURCE_IMPORT_LOG, "a") >>>>>>> PermissionError: [Errno 13] Permission denied: >>>>>>> '/home/archesdev/arches/eamena/eamena/logs/resource_import.log' >>>>>>> >>>>>>> I have checked both files csv and mapping, seems to be everyything >>>>>>> as it is explained in the arches documentation >>>>>>> <https://arches.readthedocs.io/en/stable/import-export/#csv-file-requirements> >>>>>>> >>>>>>> Any idea about what is wrong? I have run the same in local and >>>>>>> everything works. >>>>>>> >>>>>>> >>>>>>> Thanks again >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Thursday, 28 January 2021 at 10:40:02 UTC >>>>>>> [email protected] wrote: >>>>>>> >>>>>>>> Yes, you need to run all the arches commands in the env, assuming >>>>>>>> you have been following the documentation. Arches 5+ requires python >>>>>>>> 3.7+ >>>>>>>> so you need to make sure that the env was built using the python 3.x >>>>>>>> interpreter. >>>>>>>> >>>>>>>> >>>>>>>> On Thursday, January 28, 2021 at 10:32:41 AM UTC >>>>>>>> [email protected] wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> My mistake currently the version we are running is 5x. After run >>>>>>>>> the command >>>>>>>>> >>>>>>>>> *-->* >>>>>>>>> >>>>>>>>> * python manage.py packages -o import_business_data -s >>>>>>>>> /path/to/the/file.csv -c /path/to/the/file.mapping -ow 'overwrite' >>>>>>>>> -bulk * >>>>>>>>> *<--* >>>>>>>>> >>>>>>>>> >>>>>>>>> in the same folder than manage.py file the retun is: >>>>>>>>> >>>>>>>>> *Traceback (most recent call last):* >>>>>>>>> * File "manage.py", line 28, in <module>* >>>>>>>>> * from django.core.management import execute_from_command_line* >>>>>>>>> *ImportError: No module named django.core.management* >>>>>>>>> >>>>>>>>> >>>>>>>>> I guess that the error came because it should be run in the env >>>>>>>>> mode, is that correct? or maybe with python2 or 3 instead of just >>>>>>>>> python >>>>>>>>> manage. py... >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks in advance for answering me. >>>>>>>>> >>>>>>>>> On Thursday, 28 January 2021 at 09:10:09 UTC >>>>>>>>> [email protected] wrote: >>>>>>>>> >>>>>>>>>> Just an observation, are you using Arches 4.x or 5.x? I see you >>>>>>>>>> are using Python 2.7 so am assuming Arches 4.x? >>>>>>>>>> >>>>>>>>>> On Wednesday, January 27, 2021 at 11:04:05 AM UTC >>>>>>>>>> [email protected] wrote: >>>>>>>>>> >>>>>>>>>>> Hello, >>>>>>>>>>> >>>>>>>>>>> I hope someone could help with this, or maybe someone had the >>>>>>>>>>> same problem. >>>>>>>>>>> >>>>>>>>>>> I am trying to import some csv by using >>>>>>>>>>> >>>>>>>>>>> python manage.py packages -o import_business_data -s >>>>>>>>>>> /path/to/the/file.csv -c /path/to/the/file.mapping -ow 'overwrite' >>>>>>>>>>> -bulk >>>>>>>>>>> >>>>>>>>>>> After run the command in my local machine it successfully load >>>>>>>>>>> all the data, but when doing the same in the instance I get this >>>>>>>>>>> messages: >>>>>>>>>>> >>>>>>>>>>> Traceback (most recent call last): >>>>>>>>>>> File "manage.py", line 28, in <module> >>>>>>>>>>> execute_from_command_line(sys.argv) >>>>>>>>>>> File >>>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", >>>>>>>>>>> >>>>>>>>>>> line 364, in execute_from_command_line >>>>>>>>>>> utility.execute() >>>>>>>>>>> File >>>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", >>>>>>>>>>> >>>>>>>>>>> line 308, in execute >>>>>>>>>>> settings.INSTALLED_APPS >>>>>>>>>>> File >>>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", >>>>>>>>>>> line 56, >>>>>>>>>>> in __getattr__ >>>>>>>>>>> self._setup(name) >>>>>>>>>>> File >>>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", >>>>>>>>>>> line 41, >>>>>>>>>>> in _setup >>>>>>>>>>> self._wrapped = Settings(settings_module) >>>>>>>>>>> File >>>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", >>>>>>>>>>> line 110, >>>>>>>>>>> in __init__ >>>>>>>>>>> mod = importlib.import_module(self.SETTINGS_MODULE) >>>>>>>>>>> File "/usr/lib/python2.7/importlib/__init__.py", line 37, in >>>>>>>>>>> import_module >>>>>>>>>>> __import__(name) >>>>>>>>>>> File >>>>>>>>>>> "/opt/maindb-app/eamena-main-app/eamena/eamena/settings.py", line >>>>>>>>>>> 225, in >>>>>>>>>>> <module> >>>>>>>>>>> ('NAME.E41', _('Resource Names')), >>>>>>>>>>> File >>>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/__init__.py", >>>>>>>>>>> >>>>>>>>>>> line 89, in ugettext >>>>>>>>>>> return _trans.ugettext(message) >>>>>>>>>>> File >>>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py", >>>>>>>>>>> >>>>>>>>>>> line 345, in ugettext >>>>>>>>>>> return do_translate(message, 'ugettext') >>>>>>>>>>> File >>>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py", >>>>>>>>>>> >>>>>>>>>>> line 321, in do_translate >>>>>>>>>>> _default = _default or translation(settings.LANGUAGE_CODE) >>>>>>>>>>> File >>>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py", >>>>>>>>>>> >>>>>>>>>>> line 228, in translation >>>>>>>>>>> _translations[language] = DjangoTranslation(language) >>>>>>>>>>> File >>>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py", >>>>>>>>>>> >>>>>>>>>>> line 129, in __init__ >>>>>>>>>>> self._add_installed_apps_translations() >>>>>>>>>>> File >>>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py", >>>>>>>>>>> >>>>>>>>>>> line 171, in _add_installed_apps_translations >>>>>>>>>>> "The translation infrastructure cannot be initialized before >>>>>>>>>>> the " >>>>>>>>>>> django.core.exceptions.AppRegistryNotReady: The translation >>>>>>>>>>> infrastructure cannot be initialized before the apps registry is >>>>>>>>>>> ready. >>>>>>>>>>> Check that you don't make non-lazy gettext calls at import time. >>>>>>>>>>> >>>>>>>>>>> I am not sure if changing settings.py from >>>>>>>>>>> >>>>>>>>>>> from django.utils.translation import gettext as _ >>>>>>>>>>> >>>>>>>>>>> to >>>>>>>>>>> >>>>>>>>>>> from django.utils.translation import ugettext_lazy as _ >>>>>>>>>>> >>>>>>>>>>> Will resolve the issue or it going to generate otehr adittional >>>>>>>>>>> errors. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- -- To post, send email to [email protected]. To unsubscribe, send email to [email protected]. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en --- You received this message because you are subscribed to the Google Groups "Arches Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/archesproject/a59552df-7177-4681-a127-63148a742620n%40googlegroups.com.
