Hi thanks for your answer, yes I resolve the problem with the permission I follow Pat guidance and clear explanation, thanks a lot.
The other records doesn't show so I guess is because the Grid Id and row. Appreciate the help. On Thursday, 28 January 2021 at 21:03:50 UTC Markus S. wrote: > Hi, > > It looks like it was successfully able to save 6384 records but was not > able to write the log entry. Can you confirm the import itself was > successful? > > In addition to file permissions, also ensure that the folder in which the > file is located has sufficient permissions so that files inside of it can > be accessed. > > Markus > > On Thursday, January 28, 2021 at 7:37:19 AM UTC-8 [email protected] > wrote: > >> Hi, >> >> In the absence of a Linux expert . . . >> >> I would suggest changing permissions (ie chmod) temporarily to 777 and if >> that makes the difference you will know you are looking in the right place. >> The first number is permissions for the user associated with the file, the >> second is for the user group associated with the file and the third is for >> anybody. >> >> Each of the digits refers to permissions for different users/groups.The >> first will be used if the user that is trying to write to the file is that >> associated with the file. The second would be used for a different user, >> but one that is a member of the group associated with the file. The last >> will work for anybody. Normally the owner/group would have more rights than >> the general public. >> >> Each digit is made up of 4+2+1 where the 4 is read permission, 2 is write >> permission (which you want in this case), 1 is execute. >> Thus 7 is all three; 6 is 4+2 so read and write but not execute; 5 is >> read and execute but not write. >> >> You can use the linux command: >> $ ls -la >> to list the files in the current working directory and see the names of >> the associated users and group as well as the permissions. I get: >> -rw-rw-r-- 1 ubuntu ubuntu 0 Dec 4 19:39 resource_import.log >> for that file, which is 664 and the user and group are both ubuntu (on an >> AWS implementation) >> >> I am fairly new to Linux myself so not able to advise on finding out >> which user is trying to do the writing, or what group they belong to; >> perhaps someone else can. >> >> If this does allow writing, you can experiment with setting permissions >> to 700, 070, 007 in turn to see whether the user is the file's owner, a >> member of the group (which is usually the same as the owner - linux creates >> a group with the same name and just that user when a new user is created), >> or anybody, ie a different user. >> >> I suspect you might have problems beyond that. The fact that you had a >> problem writing to the log probably means that the process had a problem >> that it wanted to write about in the log. But sorting this first will >> hopefully let you get a clearer look at the underlying problem. >> >> Good luck, >> Pat >> On Thursday, 28 January 2021 at 14:54:44 UTC >> [email protected] wrote: >> >>> I am afraid I am not a linux expert - we use Windows. Maybe someone else >>> with linux knowledge can help? >>> >>> On Thursday, January 28, 2021 at 2:32:01 PM UTC [email protected] >>> wrote: >>> >>>> >>>> Hi thanks for coming back to me, yes I tried to change the permisiions >>>> but still receiving the same error. >>>> >>>> sudo chmod 775 and 755 to the file getting permission: >>>> -rwxr-xr-x 1 >>>> >>>> >>>> >>>> On Thursday, 28 January 2021 at 13:52:18 UTC >>>> [email protected] wrote: >>>> >>>>> The last line of the error is stating that it cannot write to the >>>>> resource_import.log. Check write permissions on this. >>>>> >>>>> f = open(settings.RESOURCE_IMPORT_LOG, "a") >>>>> PermissionError: [Errno 13] Permission denied: >>>>> '/home/archesdev/arches/eamena/eamena/logs/resource_import.log' >>>>> >>>>> On Thursday, January 28, 2021 at 12:12:10 PM UTC >>>>> [email protected] wrote: >>>>> >>>>>> Thanks it runs but it return few issues, that in local didnt. >>>>>> >>>>>> otal resources saved: 6384 >>>>>> 2021-01-28 05:48:59,707 >>>>>> arches.app.utils.data_management.resources.formats.csvfile WARNING Grid >>>>>> ID >>>>>> is null or not mapped on rows >>>>>> 2,2,2,3,3,3,4,4,4,5,5,5,6,6,6,7,7,7,8,8,8,9,9,9,10,10,10,11,11,11,12,12,12,13,13,13,14,14,14,15,15,15,16,16,16,17,17,17,18,18,18,19,19,19,20,20,20,21,21,21,22,22,22,23,23,23,24,24,24,25,25,25,26,26,26,27,27,27,28,28,28,29,29,29,30,30,30,31,31,31,32,32,32,33,33,33,34,34,34,35,35,35,36,36,36,37,37,37,38,38,38,39,39,39,40,40,40,41,41,41,42,42,42,43,43,43,44,44,44,45,45,45,46,46,46,47,47,47,48,48,48,49,49,49,50,50,50,51,51,51,52,52,52,53,53,53,54,54,54,55,55,55,56,56,56,57,57,57,58,58,58,59,59,59,60,60,60,61,61,61,62,62,62,63,63,63,64,64,64,65,65,6.......... >>>>>> >>>>>> and participates in a display value >>>>>> function. >>>>>> Time to import_business_data = 0:08:29.653874 >>>>>> ***** Errors occured during import. Some data may not have been >>>>>> imported. For more information, check resource import error log: >>>>>> /home/archesdev/arches/eamena/eamena/logs/resource_import.log >>>>>> 2021-01-28 05:49:00,812 arches.app.utils.task_management ERROR A >>>>>> celery broker is running, but a celery worker is not available >>>>>> Traceback (most recent call last): >>>>>> File "manage.py", line 30, in <module> >>>>>> execute_from_command_line(sys.argv) >>>>>> File >>>>>> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/__init__.py", >>>>>> >>>>>> line 381, in execute_from_command_line >>>>>> utility.execute() >>>>>> File >>>>>> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/__init__.py", >>>>>> >>>>>> line 375, in execute >>>>>> self.fetch_command(subcommand).run_from_argv(self.argv) >>>>>> File >>>>>> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/base.py", >>>>>> >>>>>> line 323, in run_from_argv >>>>>> self.execute(*args, **cmd_options) >>>>>> File >>>>>> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/base.py", >>>>>> >>>>>> line 364, in execute >>>>>> output = self.handle(*args, **options) >>>>>> File >>>>>> "/home/archesdev/arches/arches/arches/management/commands/packages.py", >>>>>> line 277, in handle >>>>>> prevent_indexing=options["prevent_indexing"], >>>>>> File >>>>>> "/home/archesdev/arches/arches/arches/management/commands/packages.py", >>>>>> line 1070, in import_business_data >>>>>> prevent_indexing=prevent_indexing, >>>>>> File >>>>>> "/home/archesdev/arches/arches/arches/app/utils/data_management/resources/importer.py", >>>>>> >>>>>> line 217, in import_business_data >>>>>> reader.report_errors() >>>>>> File >>>>>> "/home/archesdev/arches/arches/arches/app/utils/data_management/resources/formats/format.py", >>>>>> >>>>>> line 178, in report_errors >>>>>> f = open(settings.RESOURCE_IMPORT_LOG, "a") >>>>>> PermissionError: [Errno 13] Permission denied: >>>>>> '/home/archesdev/arches/eamena/eamena/logs/resource_import.log' >>>>>> >>>>>> I have checked both files csv and mapping, seems to be everyything as >>>>>> it is explained in the arches documentation >>>>>> <https://arches.readthedocs.io/en/stable/import-export/#csv-file-requirements> >>>>>> >>>>>> Any idea about what is wrong? I have run the same in local and >>>>>> everything works. >>>>>> >>>>>> >>>>>> Thanks again >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Thursday, 28 January 2021 at 10:40:02 UTC >>>>>> [email protected] wrote: >>>>>> >>>>>>> Yes, you need to run all the arches commands in the env, assuming >>>>>>> you have been following the documentation. Arches 5+ requires python >>>>>>> 3.7+ >>>>>>> so you need to make sure that the env was built using the python 3.x >>>>>>> interpreter. >>>>>>> >>>>>>> >>>>>>> On Thursday, January 28, 2021 at 10:32:41 AM UTC >>>>>>> [email protected] wrote: >>>>>>> >>>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> My mistake currently the version we are running is 5x. After run >>>>>>>> the command >>>>>>>> >>>>>>>> *-->* >>>>>>>> >>>>>>>> * python manage.py packages -o import_business_data -s >>>>>>>> /path/to/the/file.csv -c /path/to/the/file.mapping -ow 'overwrite' >>>>>>>> -bulk * >>>>>>>> *<--* >>>>>>>> >>>>>>>> >>>>>>>> in the same folder than manage.py file the retun is: >>>>>>>> >>>>>>>> *Traceback (most recent call last):* >>>>>>>> * File "manage.py", line 28, in <module>* >>>>>>>> * from django.core.management import execute_from_command_line* >>>>>>>> *ImportError: No module named django.core.management* >>>>>>>> >>>>>>>> >>>>>>>> I guess that the error came because it should be run in the env >>>>>>>> mode, is that correct? or maybe with python2 or 3 instead of just >>>>>>>> python >>>>>>>> manage. py... >>>>>>>> >>>>>>>> >>>>>>>> Thanks in advance for answering me. >>>>>>>> >>>>>>>> On Thursday, 28 January 2021 at 09:10:09 UTC >>>>>>>> [email protected] wrote: >>>>>>>> >>>>>>>>> Just an observation, are you using Arches 4.x or 5.x? I see you >>>>>>>>> are using Python 2.7 so am assuming Arches 4.x? >>>>>>>>> >>>>>>>>> On Wednesday, January 27, 2021 at 11:04:05 AM UTC >>>>>>>>> [email protected] wrote: >>>>>>>>> >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> I hope someone could help with this, or maybe someone had the >>>>>>>>>> same problem. >>>>>>>>>> >>>>>>>>>> I am trying to import some csv by using >>>>>>>>>> >>>>>>>>>> python manage.py packages -o import_business_data -s >>>>>>>>>> /path/to/the/file.csv -c /path/to/the/file.mapping -ow 'overwrite' >>>>>>>>>> -bulk >>>>>>>>>> >>>>>>>>>> After run the command in my local machine it successfully load >>>>>>>>>> all the data, but when doing the same in the instance I get this >>>>>>>>>> messages: >>>>>>>>>> >>>>>>>>>> Traceback (most recent call last): >>>>>>>>>> File "manage.py", line 28, in <module> >>>>>>>>>> execute_from_command_line(sys.argv) >>>>>>>>>> File >>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", >>>>>>>>>> >>>>>>>>>> line 364, in execute_from_command_line >>>>>>>>>> utility.execute() >>>>>>>>>> File >>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", >>>>>>>>>> >>>>>>>>>> line 308, in execute >>>>>>>>>> settings.INSTALLED_APPS >>>>>>>>>> File >>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", >>>>>>>>>> line 56, >>>>>>>>>> in __getattr__ >>>>>>>>>> self._setup(name) >>>>>>>>>> File >>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", >>>>>>>>>> line 41, >>>>>>>>>> in _setup >>>>>>>>>> self._wrapped = Settings(settings_module) >>>>>>>>>> File >>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", >>>>>>>>>> line 110, >>>>>>>>>> in __init__ >>>>>>>>>> mod = importlib.import_module(self.SETTINGS_MODULE) >>>>>>>>>> File "/usr/lib/python2.7/importlib/__init__.py", line 37, in >>>>>>>>>> import_module >>>>>>>>>> __import__(name) >>>>>>>>>> File >>>>>>>>>> "/opt/maindb-app/eamena-main-app/eamena/eamena/settings.py", line >>>>>>>>>> 225, in >>>>>>>>>> <module> >>>>>>>>>> ('NAME.E41', _('Resource Names')), >>>>>>>>>> File >>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/__init__.py", >>>>>>>>>> >>>>>>>>>> line 89, in ugettext >>>>>>>>>> return _trans.ugettext(message) >>>>>>>>>> File >>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py", >>>>>>>>>> >>>>>>>>>> line 345, in ugettext >>>>>>>>>> return do_translate(message, 'ugettext') >>>>>>>>>> File >>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py", >>>>>>>>>> >>>>>>>>>> line 321, in do_translate >>>>>>>>>> _default = _default or translation(settings.LANGUAGE_CODE) >>>>>>>>>> File >>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py", >>>>>>>>>> >>>>>>>>>> line 228, in translation >>>>>>>>>> _translations[language] = DjangoTranslation(language) >>>>>>>>>> File >>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py", >>>>>>>>>> >>>>>>>>>> line 129, in __init__ >>>>>>>>>> self._add_installed_apps_translations() >>>>>>>>>> File >>>>>>>>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py", >>>>>>>>>> >>>>>>>>>> line 171, in _add_installed_apps_translations >>>>>>>>>> "The translation infrastructure cannot be initialized before >>>>>>>>>> the " >>>>>>>>>> django.core.exceptions.AppRegistryNotReady: The translation >>>>>>>>>> infrastructure cannot be initialized before the apps registry is >>>>>>>>>> ready. >>>>>>>>>> Check that you don't make non-lazy gettext calls at import time. >>>>>>>>>> >>>>>>>>>> I am not sure if changing settings.py from >>>>>>>>>> >>>>>>>>>> from django.utils.translation import gettext as _ >>>>>>>>>> >>>>>>>>>> to >>>>>>>>>> >>>>>>>>>> from django.utils.translation import ugettext_lazy as _ >>>>>>>>>> >>>>>>>>>> Will resolve the issue or it going to generate otehr adittional >>>>>>>>>> errors. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- -- To post, send email to [email protected]. To unsubscribe, send email to [email protected]. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en --- You received this message because you are subscribed to the Google Groups "Arches Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/archesproject/535e24c2-b32f-4ec7-a438-8a78a8545a44n%40googlegroups.com.
