[Arches] Re: Problems importing data from a csv with the mapping file

[Cyrus Hiatt]

Just to wrap up this conversation. Gabriel was able to resolve the issue. 
It turned he had tile data spanning multiple lines within a single cell, so 
Arches was unable to read the data properly. 

- Cyrus 
On Tuesday, February 2, 2021 at 8:07:48 AM UTC-8 gabriel.c...@gmail.com 
wrote:

>
> Hi,
>
> The permissions were solved, thanks all for the tips.  Problem I have 
> still problems when loading and saving the csv in arches... despite it 
> shows that some records were saved they doesnt show in the app 
>
> arches.app.utils.data_management.resources.formats.csvfile WARNING  Grid 
> ID is null or not mapped on rows 
> 2,2,2,3,3,3,4,4,4,5,5,5,6,6,6,7,7,7,8,8,8,9,9,9,10,10,10,11,11,11,12,12,12,13,13,13,14,14,14,15,15,15,16,16,16,17,17,17
>  
> and participates in a display value 
> function.
> Time to import_business_data = 0:00:01.989878
> * Errors occured during import. Some data may not have been imported. 
> For more information, check resource import error log: 
> /home/archesdev/arches/eamena/eamena/logs/resource_import.log
>
>
> The log just show the same message so it doesnt display any additional 
> information. Someone have experienced the same issue or similar?
>
> Thanks in advance
>
> On Thursday, 28 January 2021 at 21:03:50 UTC Markus S. wrote:
>
>> Hi,
>>
>> It looks like it was successfully able to save 6384 records but was not 
>> able to write the log entry. Can you confirm the import itself was 
>> successful? 
>>
>> In addition to file permissions, also ensure that the folder in which the 
>> file is located has sufficient permissions so that files inside of it can 
>> be accessed.
>>
>> Markus
>>
>> On Thursday, January 28, 2021 at 7:37:19 AM UTC-8 patlu...@gmail.com 
>> wrote:
>>
>>> Hi,
>>>
>>> In the absence of a Linux expert . . .
>>>
>>> I would suggest changing permissions (ie chmod) temporarily to 777 and 
>>> if that makes the difference you will know you are looking in the right 
>>> place. The first number is permissions for the user associated with the 
>>> file, the second is for the user group associated with the file and the 
>>> third is for anybody.
>>>
>>> Each of the digits refers to permissions for different users/groups.The 
>>> first will be used if the user that is trying to write to the file is that 
>>> associated with the file. The second would be used for a different user, 
>>> but one that is a member of the group associated with the file. The last 
>>> will work for anybody. Normally the owner/group would have more rights than 
>>> the general public.
>>>
>>> Each digit is made up of 4+2+1 where the 4 is read permission, 2 is 
>>> write permission (which you want in this case), 1 is execute.
>>> Thus 7 is all three; 6 is 4+2 so read and write but not execute; 5 is 
>>> read and execute but not write.
>>>
>>> You can use the linux command:
>>> $ ls -la
>>> to list the files in the current working directory and see the names of 
>>> the associated users and group as well as the permissions. I get:
>>> -rw-rw-r--  1 ubuntu ubuntu  0 Dec  4 19:39 resource_import.log
>>> for that file, which is 664 and the user and group are both ubuntu (on 
>>> an AWS implementation)
>>>
>>> I am fairly new to Linux myself so not able to advise on finding out 
>>> which user is trying to do the writing, or what group they belong to; 
>>> perhaps someone else can.
>>>
>>> If this does allow writing, you can experiment with setting permissions 
>>> to 700, 070, 007 in turn to see whether the user is the file's owner, a 
>>> member of the group (which is usually the same as the owner - linux creates 
>>> a group with the same name and just that user when a new user is created), 
>>> or anybody, ie a different user.
>>>
>>> I suspect you might have problems beyond that. The fact that you had a 
>>> problem writing to the log probably means that the process had a problem 
>>> that it wanted to write about in the log. But sorting this first will 
>>> hopefully let you get a clearer look at the underlying problem.
>>>
>>> Good luck,
>>> Pat
>>> On Thursday, 28 January 2021 at 14:54:44 UTC 
>>> andrew...@historicengland.org.uk wrote:
>>>
 I am afraid I am not a linux expert - we use Windows. Maybe someone 
 else with linux knowledge can help?

 On Thursday, January 28, 2021 at 2:32:01 PM UTC gabriel.c...@gmail.com 
 wrote:

>
> Hi thanks for coming back to me, yes I tried to change the permisiions 
> but still receiving the same error. 
>
> sudo chmod 775 and 755 to the file getting permission:
> -rwxr-xr-x 1
>
>
>
> On Thursday, 28 January 2021 at 13:52:18 UTC 
> andrew...@historicengland.org.uk wrote:
>
>> The last line of the error is stating that it cannot write to the 
>> resource_import.log. Check write permissions on this. 
>>
>> f = open(settings.RESOURCE_IMPORT_LOG, "a")
>> PermissionError: [Errno 13] Permission denied: 
>> 

[Arches] Re: Problems importing data from a csv with the mapping file

[Gabriel Contreras]

Hi,

The permissions were solved, thanks all for the tips.  Problem I have still 
problems when loading and saving the csv in arches... despite it shows that 
some records were saved they doesnt show in the app 

arches.app.utils.data_management.resources.formats.csvfile WARNING  Grid ID 
is null or not mapped on rows 
2,2,2,3,3,3,4,4,4,5,5,5,6,6,6,7,7,7,8,8,8,9,9,9,10,10,10,11,11,11,12,12,12,13,13,13,14,14,14,15,15,15,16,16,16,17,17,17
 
and participates in a display value 
function.
Time to import_business_data = 0:00:01.989878
* Errors occured during import. Some data may not have been imported. 
For more information, check resource import error log: 
/home/archesdev/arches/eamena/eamena/logs/resource_import.log


The log just show the same message so it doesnt display any additional 
information. Someone have experienced the same issue or similar?

Thanks in advance

On Thursday, 28 January 2021 at 21:03:50 UTC Markus S. wrote:

> Hi,
>
> It looks like it was successfully able to save 6384 records but was not 
> able to write the log entry. Can you confirm the import itself was 
> successful? 
>
> In addition to file permissions, also ensure that the folder in which the 
> file is located has sufficient permissions so that files inside of it can 
> be accessed.
>
> Markus
>
> On Thursday, January 28, 2021 at 7:37:19 AM UTC-8 patlu...@gmail.com 
> wrote:
>
>> Hi,
>>
>> In the absence of a Linux expert . . .
>>
>> I would suggest changing permissions (ie chmod) temporarily to 777 and if 
>> that makes the difference you will know you are looking in the right place. 
>> The first number is permissions for the user associated with the file, the 
>> second is for the user group associated with the file and the third is for 
>> anybody.
>>
>> Each of the digits refers to permissions for different users/groups.The 
>> first will be used if the user that is trying to write to the file is that 
>> associated with the file. The second would be used for a different user, 
>> but one that is a member of the group associated with the file. The last 
>> will work for anybody. Normally the owner/group would have more rights than 
>> the general public.
>>
>> Each digit is made up of 4+2+1 where the 4 is read permission, 2 is write 
>> permission (which you want in this case), 1 is execute.
>> Thus 7 is all three; 6 is 4+2 so read and write but not execute; 5 is 
>> read and execute but not write.
>>
>> You can use the linux command:
>> $ ls -la
>> to list the files in the current working directory and see the names of 
>> the associated users and group as well as the permissions. I get:
>> -rw-rw-r--  1 ubuntu ubuntu  0 Dec  4 19:39 resource_import.log
>> for that file, which is 664 and the user and group are both ubuntu (on an 
>> AWS implementation)
>>
>> I am fairly new to Linux myself so not able to advise on finding out 
>> which user is trying to do the writing, or what group they belong to; 
>> perhaps someone else can.
>>
>> If this does allow writing, you can experiment with setting permissions 
>> to 700, 070, 007 in turn to see whether the user is the file's owner, a 
>> member of the group (which is usually the same as the owner - linux creates 
>> a group with the same name and just that user when a new user is created), 
>> or anybody, ie a different user.
>>
>> I suspect you might have problems beyond that. The fact that you had a 
>> problem writing to the log probably means that the process had a problem 
>> that it wanted to write about in the log. But sorting this first will 
>> hopefully let you get a clearer look at the underlying problem.
>>
>> Good luck,
>> Pat
>> On Thursday, 28 January 2021 at 14:54:44 UTC 
>> andrew...@historicengland.org.uk wrote:
>>
>>> I am afraid I am not a linux expert - we use Windows. Maybe someone else 
>>> with linux knowledge can help?
>>>
>>> On Thursday, January 28, 2021 at 2:32:01 PM UTC gabriel.c...@gmail.com 
>>> wrote:
>>>

 Hi thanks for coming back to me, yes I tried to change the permisiions 
 but still receiving the same error. 

 sudo chmod 775 and 755 to the file getting permission:
 -rwxr-xr-x 1



 On Thursday, 28 January 2021 at 13:52:18 UTC 
 andrew...@historicengland.org.uk wrote:

> The last line of the error is stating that it cannot write to the 
> resource_import.log. Check write permissions on this. 
>
> f = open(settings.RESOURCE_IMPORT_LOG, "a")
> PermissionError: [Errno 13] Permission denied: 
> '/home/archesdev/arches/eamena/eamena/logs/resource_import.log'
>
> On Thursday, January 28, 2021 at 12:12:10 PM UTC 
> gabriel.c...@gmail.com wrote:
>
>> Thanks it runs but it return few issues, that in local didnt. 
>>
>> otal resources saved: 6384
>> 2021-01-28 05:48:59,707 
>> arches.app.utils.data_management.resources.formats.csvfile WARNING  Grid 
>> ID 
>> is null or not mapped on 

[Arches] Re: Problems importing data from a csv with the mapping file

[Gabriel Contreras]

Hi again,

Sorry for asking again but the change in the permission didnt work. 

Still returning this 
issue arches.app.utils.data_management.resources.formats.csvfile WARNING  
Grid ID is null or not mapped on rows 
2,2,2,3,3,3,4,4,4,5,5,5,6,6,6,7,7,7,8,8,8,9,9,9,10,10,10,11,11,11,12,12,12,13,13,13,14,14,14,15,15,15,16,16,16,17,17,17,18,18,18,19,19,19,20,20,20,21,21,21,22,22,22,23,23,23,24,24,24,25,25,25,26,26,26,27,27,27,28,28,28.
 
6534, 

it seems to save records but doesn't create geometries or doesnt save some 
of the data in the CSV.

I have transformed the geometries into WKT same that I did for local at it 
worked, but with the server doesnt and cannot upload these data. 

The mapping document is correct and the relation between file and mapping 
document. Some idea about why?


Thanks

On Friday, 29 January 2021 at 08:37:46 UTC Gabriel Contreras wrote:

> Hi thanks for your answer,
>
> yes I resolve the problem with the permission I follow Pat guidance and 
> clear explanation, thanks a lot.
>
> The other records doesn't show so I guess is because the Grid Id and row. 
>
> Appreciate the help.
>
>
>
>
>
> On Thursday, 28 January 2021 at 21:03:50 UTC Markus S. wrote:
>
>> Hi,
>>
>> It looks like it was successfully able to save 6384 records but was not 
>> able to write the log entry. Can you confirm the import itself was 
>> successful? 
>>
>> In addition to file permissions, also ensure that the folder in which the 
>> file is located has sufficient permissions so that files inside of it can 
>> be accessed.
>>
>> Markus
>>
>> On Thursday, January 28, 2021 at 7:37:19 AM UTC-8 patlu...@gmail.com 
>> wrote:
>>
>>> Hi,
>>>
>>> In the absence of a Linux expert . . .
>>>
>>> I would suggest changing permissions (ie chmod) temporarily to 777 and 
>>> if that makes the difference you will know you are looking in the right 
>>> place. The first number is permissions for the user associated with the 
>>> file, the second is for the user group associated with the file and the 
>>> third is for anybody.
>>>
>>> Each of the digits refers to permissions for different users/groups.The 
>>> first will be used if the user that is trying to write to the file is that 
>>> associated with the file. The second would be used for a different user, 
>>> but one that is a member of the group associated with the file. The last 
>>> will work for anybody. Normally the owner/group would have more rights than 
>>> the general public.
>>>
>>> Each digit is made up of 4+2+1 where the 4 is read permission, 2 is 
>>> write permission (which you want in this case), 1 is execute.
>>> Thus 7 is all three; 6 is 4+2 so read and write but not execute; 5 is 
>>> read and execute but not write.
>>>
>>> You can use the linux command:
>>> $ ls -la
>>> to list the files in the current working directory and see the names of 
>>> the associated users and group as well as the permissions. I get:
>>> -rw-rw-r--  1 ubuntu ubuntu  0 Dec  4 19:39 resource_import.log
>>> for that file, which is 664 and the user and group are both ubuntu (on 
>>> an AWS implementation)
>>>
>>> I am fairly new to Linux myself so not able to advise on finding out 
>>> which user is trying to do the writing, or what group they belong to; 
>>> perhaps someone else can.
>>>
>>> If this does allow writing, you can experiment with setting permissions 
>>> to 700, 070, 007 in turn to see whether the user is the file's owner, a 
>>> member of the group (which is usually the same as the owner - linux creates 
>>> a group with the same name and just that user when a new user is created), 
>>> or anybody, ie a different user.
>>>
>>> I suspect you might have problems beyond that. The fact that you had a 
>>> problem writing to the log probably means that the process had a problem 
>>> that it wanted to write about in the log. But sorting this first will 
>>> hopefully let you get a clearer look at the underlying problem.
>>>
>>> Good luck,
>>> Pat
>>> On Thursday, 28 January 2021 at 14:54:44 UTC 
>>> andrew...@historicengland.org.uk wrote:
>>>
 I am afraid I am not a linux expert - we use Windows. Maybe someone 
 else with linux knowledge can help?

 On Thursday, January 28, 2021 at 2:32:01 PM UTC gabriel.c...@gmail.com 
 wrote:

>
> Hi thanks for coming back to me, yes I tried to change the permisiions 
> but still receiving the same error. 
>
> sudo chmod 775 and 755 to the file getting permission:
> -rwxr-xr-x 1
>
>
>
> On Thursday, 28 January 2021 at 13:52:18 UTC 
> andrew...@historicengland.org.uk wrote:
>
>> The last line of the error is stating that it cannot write to the 
>> resource_import.log. Check write permissions on this. 
>>
>> f = open(settings.RESOURCE_IMPORT_LOG, "a")
>> PermissionError: [Errno 13] Permission denied: 
>> '/home/archesdev/arches/eamena/eamena/logs/resource_import.log'
>>
>> On Thursday, January 28, 2021 at 12:12:10 PM UTC 

[Arches] Re: Problems importing data from a csv with the mapping file

[Gabriel Contreras]

Hi thanks for your answer,

yes I resolve the problem with the permission I follow Pat guidance and 
clear explanation, thanks a lot.

The other records doesn't show so I guess is because the Grid Id and row. 

Appreciate the help.





On Thursday, 28 January 2021 at 21:03:50 UTC Markus S. wrote:

> Hi,
>
> It looks like it was successfully able to save 6384 records but was not 
> able to write the log entry. Can you confirm the import itself was 
> successful? 
>
> In addition to file permissions, also ensure that the folder in which the 
> file is located has sufficient permissions so that files inside of it can 
> be accessed.
>
> Markus
>
> On Thursday, January 28, 2021 at 7:37:19 AM UTC-8 patlu...@gmail.com 
> wrote:
>
>> Hi,
>>
>> In the absence of a Linux expert . . .
>>
>> I would suggest changing permissions (ie chmod) temporarily to 777 and if 
>> that makes the difference you will know you are looking in the right place. 
>> The first number is permissions for the user associated with the file, the 
>> second is for the user group associated with the file and the third is for 
>> anybody.
>>
>> Each of the digits refers to permissions for different users/groups.The 
>> first will be used if the user that is trying to write to the file is that 
>> associated with the file. The second would be used for a different user, 
>> but one that is a member of the group associated with the file. The last 
>> will work for anybody. Normally the owner/group would have more rights than 
>> the general public.
>>
>> Each digit is made up of 4+2+1 where the 4 is read permission, 2 is write 
>> permission (which you want in this case), 1 is execute.
>> Thus 7 is all three; 6 is 4+2 so read and write but not execute; 5 is 
>> read and execute but not write.
>>
>> You can use the linux command:
>> $ ls -la
>> to list the files in the current working directory and see the names of 
>> the associated users and group as well as the permissions. I get:
>> -rw-rw-r--  1 ubuntu ubuntu  0 Dec  4 19:39 resource_import.log
>> for that file, which is 664 and the user and group are both ubuntu (on an 
>> AWS implementation)
>>
>> I am fairly new to Linux myself so not able to advise on finding out 
>> which user is trying to do the writing, or what group they belong to; 
>> perhaps someone else can.
>>
>> If this does allow writing, you can experiment with setting permissions 
>> to 700, 070, 007 in turn to see whether the user is the file's owner, a 
>> member of the group (which is usually the same as the owner - linux creates 
>> a group with the same name and just that user when a new user is created), 
>> or anybody, ie a different user.
>>
>> I suspect you might have problems beyond that. The fact that you had a 
>> problem writing to the log probably means that the process had a problem 
>> that it wanted to write about in the log. But sorting this first will 
>> hopefully let you get a clearer look at the underlying problem.
>>
>> Good luck,
>> Pat
>> On Thursday, 28 January 2021 at 14:54:44 UTC 
>> andrew...@historicengland.org.uk wrote:
>>
>>> I am afraid I am not a linux expert - we use Windows. Maybe someone else 
>>> with linux knowledge can help?
>>>
>>> On Thursday, January 28, 2021 at 2:32:01 PM UTC gabriel.c...@gmail.com 
>>> wrote:
>>>

 Hi thanks for coming back to me, yes I tried to change the permisiions 
 but still receiving the same error. 

 sudo chmod 775 and 755 to the file getting permission:
 -rwxr-xr-x 1



 On Thursday, 28 January 2021 at 13:52:18 UTC 
 andrew...@historicengland.org.uk wrote:

> The last line of the error is stating that it cannot write to the 
> resource_import.log. Check write permissions on this. 
>
> f = open(settings.RESOURCE_IMPORT_LOG, "a")
> PermissionError: [Errno 13] Permission denied: 
> '/home/archesdev/arches/eamena/eamena/logs/resource_import.log'
>
> On Thursday, January 28, 2021 at 12:12:10 PM UTC 
> gabriel.c...@gmail.com wrote:
>
>> Thanks it runs but it return few issues, that in local didnt. 
>>
>> otal resources saved: 6384
>> 2021-01-28 05:48:59,707 
>> arches.app.utils.data_management.resources.formats.csvfile WARNING  Grid 
>> ID 
>> is null or not mapped on rows 
>> 2,2,2,3,3,3,4,4,4,5,5,5,6,6,6,7,7,7,8,8,8,9,9,9,10,10,10,11,11,11,12,12,12,13,13,13,14,14,14,15,15,15,16,16,16,17,17,17,18,18,18,19,19,19,20,20,20,21,21,21,22,22,22,23,23,23,24,24,24,25,25,25,26,26,26,27,27,27,28,28,28,29,29,29,30,30,30,31,31,31,32,32,32,33,33,33,34,34,34,35,35,35,36,36,36,37,37,37,38,38,38,39,39,39,40,40,40,41,41,41,42,42,42,43,43,43,44,44,44,45,45,45,46,46,46,47,47,47,48,48,48,49,49,49,50,50,50,51,51,51,52,52,52,53,53,53,54,54,54,55,55,55,56,56,56,57,57,57,58,58,58,59,59,59,60,60,60,61,61,61,62,62,62,63,63,63,64,64,64,65,65,6..
>>  
>> and participates in a display value 
>> function.
>> 

[Arches] Re: Problems importing data from a csv with the mapping file

[Markus S.]

Hi,

It looks like it was successfully able to save 6384 records but was not 
able to write the log entry. Can you confirm the import itself was 
successful? 

In addition to file permissions, also ensure that the folder in which the 
file is located has sufficient permissions so that files inside of it can 
be accessed.

Markus

On Thursday, January 28, 2021 at 7:37:19 AM UTC-8 patlu...@gmail.com wrote:

> Hi,
>
> In the absence of a Linux expert . . .
>
> I would suggest changing permissions (ie chmod) temporarily to 777 and if 
> that makes the difference you will know you are looking in the right place. 
> The first number is permissions for the user associated with the file, the 
> second is for the user group associated with the file and the third is for 
> anybody.
>
> Each of the digits refers to permissions for different users/groups.The 
> first will be used if the user that is trying to write to the file is that 
> associated with the file. The second would be used for a different user, 
> but one that is a member of the group associated with the file. The last 
> will work for anybody. Normally the owner/group would have more rights than 
> the general public.
>
> Each digit is made up of 4+2+1 where the 4 is read permission, 2 is write 
> permission (which you want in this case), 1 is execute.
> Thus 7 is all three; 6 is 4+2 so read and write but not execute; 5 is read 
> and execute but not write.
>
> You can use the linux command:
> $ ls -la
> to list the files in the current working directory and see the names of 
> the associated users and group as well as the permissions. I get:
> -rw-rw-r--  1 ubuntu ubuntu  0 Dec  4 19:39 resource_import.log
> for that file, which is 664 and the user and group are both ubuntu (on an 
> AWS implementation)
>
> I am fairly new to Linux myself so not able to advise on finding out which 
> user is trying to do the writing, or what group they belong to; perhaps 
> someone else can.
>
> If this does allow writing, you can experiment with setting permissions to 
> 700, 070, 007 in turn to see whether the user is the file's owner, a member 
> of the group (which is usually the same as the owner - linux creates a 
> group with the same name and just that user when a new user is created), or 
> anybody, ie a different user.
>
> I suspect you might have problems beyond that. The fact that you had a 
> problem writing to the log probably means that the process had a problem 
> that it wanted to write about in the log. But sorting this first will 
> hopefully let you get a clearer look at the underlying problem.
>
> Good luck,
> Pat
> On Thursday, 28 January 2021 at 14:54:44 UTC 
> andrew...@historicengland.org.uk wrote:
>
>> I am afraid I am not a linux expert - we use Windows. Maybe someone else 
>> with linux knowledge can help?
>>
>> On Thursday, January 28, 2021 at 2:32:01 PM UTC gabriel.c...@gmail.com 
>> wrote:
>>
>>>
>>> Hi thanks for coming back to me, yes I tried to change the permisiions 
>>> but still receiving the same error. 
>>>
>>> sudo chmod 775 and 755 to the file getting permission:
>>> -rwxr-xr-x 1
>>>
>>>
>>>
>>> On Thursday, 28 January 2021 at 13:52:18 UTC 
>>> andrew...@historicengland.org.uk wrote:
>>>
 The last line of the error is stating that it cannot write to the 
 resource_import.log. Check write permissions on this. 

 f = open(settings.RESOURCE_IMPORT_LOG, "a")
 PermissionError: [Errno 13] Permission denied: 
 '/home/archesdev/arches/eamena/eamena/logs/resource_import.log'

 On Thursday, January 28, 2021 at 12:12:10 PM UTC gabriel.c...@gmail.com 
 wrote:

> Thanks it runs but it return few issues, that in local didnt. 
>
> otal resources saved: 6384
> 2021-01-28 05:48:59,707 
> arches.app.utils.data_management.resources.formats.csvfile WARNING  Grid 
> ID 
> is null or not mapped on rows 
> 2,2,2,3,3,3,4,4,4,5,5,5,6,6,6,7,7,7,8,8,8,9,9,9,10,10,10,11,11,11,12,12,12,13,13,13,14,14,14,15,15,15,16,16,16,17,17,17,18,18,18,19,19,19,20,20,20,21,21,21,22,22,22,23,23,23,24,24,24,25,25,25,26,26,26,27,27,27,28,28,28,29,29,29,30,30,30,31,31,31,32,32,32,33,33,33,34,34,34,35,35,35,36,36,36,37,37,37,38,38,38,39,39,39,40,40,40,41,41,41,42,42,42,43,43,43,44,44,44,45,45,45,46,46,46,47,47,47,48,48,48,49,49,49,50,50,50,51,51,51,52,52,52,53,53,53,54,54,54,55,55,55,56,56,56,57,57,57,58,58,58,59,59,59,60,60,60,61,61,61,62,62,62,63,63,63,64,64,64,65,65,6..
>  
> and participates in a display value 
> function.
> Time to import_business_data = 0:08:29.653874
> * Errors occured during import. Some data may not have been 
> imported. For more information, check resource import error log: 
> /home/archesdev/arches/eamena/eamena/logs/resource_import.log
> 2021-01-28 05:49:00,812 arches.app.utils.task_management ERRORA 
> celery broker is running, but a celery worker is not available
> Traceback 

[Arches] Re: Problems importing data from a csv with the mapping file

[patlu...@gmail.com]

Hi,

In the absence of a Linux expert . . .

I would suggest changing permissions (ie chmod) temporarily to 777 and if 
that makes the difference you will know you are looking in the right place. 
The first number is permissions for the user associated with the file, the 
second is for the user group associated with the file and the third is for 
anybody.

Each of the digits refers to permissions for different users/groups.The 
first will be used if the user that is trying to write to the file is that 
associated with the file. The second would be used for a different user, 
but one that is a member of the group associated with the file. The last 
will work for anybody. Normally the owner/group would have more rights than 
the general public.

Each digit is made up of 4+2+1 where the 4 is read permission, 2 is write 
permission (which you want in this case), 1 is execute.
Thus 7 is all three; 6 is 4+2 so read and write but not execute; 5 is read 
and execute but not write.

You can use the linux command:
$ ls -la
to list the files in the current working directory and see the names of the 
associated users and group as well as the permissions. I get:
-rw-rw-r--  1 ubuntu ubuntu  0 Dec  4 19:39 resource_import.log
for that file, which is 664 and the user and group are both ubuntu (on an 
AWS implementation)

I am fairly new to Linux myself so not able to advise on finding out which 
user is trying to do the writing, or what group they belong to; perhaps 
someone else can.

If this does allow writing, you can experiment with setting permissions to 
700, 070, 007 in turn to see whether the user is the file's owner, a member 
of the group (which is usually the same as the owner - linux creates a 
group with the same name and just that user when a new user is created), or 
anybody, ie a different user.

I suspect you might have problems beyond that. The fact that you had a 
problem writing to the log probably means that the process had a problem 
that it wanted to write about in the log. But sorting this first will 
hopefully let you get a clearer look at the underlying problem.

Good luck,
Pat
On Thursday, 28 January 2021 at 14:54:44 UTC 
andrew...@historicengland.org.uk wrote:

> I am afraid I am not a linux expert - we use Windows. Maybe someone else 
> with linux knowledge can help?
>
> On Thursday, January 28, 2021 at 2:32:01 PM UTC gabriel.c...@gmail.com 
> wrote:
>
>>
>> Hi thanks for coming back to me, yes I tried to change the permisiions 
>> but still receiving the same error. 
>>
>> sudo chmod 775 and 755 to the file getting permission:
>> -rwxr-xr-x 1
>>
>>
>>
>> On Thursday, 28 January 2021 at 13:52:18 UTC 
>> andrew...@historicengland.org.uk wrote:
>>
>>> The last line of the error is stating that it cannot write to the 
>>> resource_import.log. Check write permissions on this. 
>>>
>>> f = open(settings.RESOURCE_IMPORT_LOG, "a")
>>> PermissionError: [Errno 13] Permission denied: 
>>> '/home/archesdev/arches/eamena/eamena/logs/resource_import.log'
>>>
>>> On Thursday, January 28, 2021 at 12:12:10 PM UTC gabriel.c...@gmail.com 
>>> wrote:
>>>
 Thanks it runs but it return few issues, that in local didnt. 

 otal resources saved: 6384
 2021-01-28 05:48:59,707 
 arches.app.utils.data_management.resources.formats.csvfile WARNING  Grid 
 ID 
 is null or not mapped on rows 
 2,2,2,3,3,3,4,4,4,5,5,5,6,6,6,7,7,7,8,8,8,9,9,9,10,10,10,11,11,11,12,12,12,13,13,13,14,14,14,15,15,15,16,16,16,17,17,17,18,18,18,19,19,19,20,20,20,21,21,21,22,22,22,23,23,23,24,24,24,25,25,25,26,26,26,27,27,27,28,28,28,29,29,29,30,30,30,31,31,31,32,32,32,33,33,33,34,34,34,35,35,35,36,36,36,37,37,37,38,38,38,39,39,39,40,40,40,41,41,41,42,42,42,43,43,43,44,44,44,45,45,45,46,46,46,47,47,47,48,48,48,49,49,49,50,50,50,51,51,51,52,52,52,53,53,53,54,54,54,55,55,55,56,56,56,57,57,57,58,58,58,59,59,59,60,60,60,61,61,61,62,62,62,63,63,63,64,64,64,65,65,6..
  
 and participates in a display value 
 function.
 Time to import_business_data = 0:08:29.653874
 * Errors occured during import. Some data may not have been 
 imported. For more information, check resource import error log: 
 /home/archesdev/arches/eamena/eamena/logs/resource_import.log
 2021-01-28 05:49:00,812 arches.app.utils.task_management ERRORA 
 celery broker is running, but a celery worker is not available
 Traceback (most recent call last):
   File "manage.py", line 30, in 
 execute_from_command_line(sys.argv)
   File 
 "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/__init__.py",
  
 line 381, in execute_from_command_line
 utility.execute()
   File 
 "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/__init__.py",
  
 line 375, in execute
 self.fetch_command(subcommand).run_from_argv(self.argv)
   File 
 

[Arches] Re: Problems importing data from a csv with the mapping file

[Gabriel Contreras]

Thank you very much for your time. I really appreciate it
On Thursday, 28 January 2021 at 14:54:44 UTC 
andrew...@historicengland.org.uk wrote:

> I am afraid I am not a linux expert - we use Windows. Maybe someone else 
> with linux knowledge can help?
>
> On Thursday, January 28, 2021 at 2:32:01 PM UTC gabriel.c...@gmail.com 
> wrote:
>
>>
>> Hi thanks for coming back to me, yes I tried to change the permisiions 
>> but still receiving the same error. 
>>
>> sudo chmod 775 and 755 to the file getting permission:
>> -rwxr-xr-x 1
>>
>>
>>
>> On Thursday, 28 January 2021 at 13:52:18 UTC 
>> andrew...@historicengland.org.uk wrote:
>>
>>> The last line of the error is stating that it cannot write to the 
>>> resource_import.log. Check write permissions on this. 
>>>
>>> f = open(settings.RESOURCE_IMPORT_LOG, "a")
>>> PermissionError: [Errno 13] Permission denied: 
>>> '/home/archesdev/arches/eamena/eamena/logs/resource_import.log'
>>>
>>> On Thursday, January 28, 2021 at 12:12:10 PM UTC gabriel.c...@gmail.com 
>>> wrote:
>>>
 Thanks it runs but it return few issues, that in local didnt. 

 otal resources saved: 6384
 2021-01-28 05:48:59,707 
 arches.app.utils.data_management.resources.formats.csvfile WARNING  Grid 
 ID 
 is null or not mapped on rows 
 2,2,2,3,3,3,4,4,4,5,5,5,6,6,6,7,7,7,8,8,8,9,9,9,10,10,10,11,11,11,12,12,12,13,13,13,14,14,14,15,15,15,16,16,16,17,17,17,18,18,18,19,19,19,20,20,20,21,21,21,22,22,22,23,23,23,24,24,24,25,25,25,26,26,26,27,27,27,28,28,28,29,29,29,30,30,30,31,31,31,32,32,32,33,33,33,34,34,34,35,35,35,36,36,36,37,37,37,38,38,38,39,39,39,40,40,40,41,41,41,42,42,42,43,43,43,44,44,44,45,45,45,46,46,46,47,47,47,48,48,48,49,49,49,50,50,50,51,51,51,52,52,52,53,53,53,54,54,54,55,55,55,56,56,56,57,57,57,58,58,58,59,59,59,60,60,60,61,61,61,62,62,62,63,63,63,64,64,64,65,65,6..
  
 and participates in a display value 
 function.
 Time to import_business_data = 0:08:29.653874
 * Errors occured during import. Some data may not have been 
 imported. For more information, check resource import error log: 
 /home/archesdev/arches/eamena/eamena/logs/resource_import.log
 2021-01-28 05:49:00,812 arches.app.utils.task_management ERRORA 
 celery broker is running, but a celery worker is not available
 Traceback (most recent call last):
   File "manage.py", line 30, in 
 execute_from_command_line(sys.argv)
   File 
 "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/__init__.py",
  
 line 381, in execute_from_command_line
 utility.execute()
   File 
 "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/__init__.py",
  
 line 375, in execute
 self.fetch_command(subcommand).run_from_argv(self.argv)
   File 
 "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/base.py",
  
 line 323, in run_from_argv
 self.execute(*args, **cmd_options)
   File 
 "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/base.py",
  
 line 364, in execute
 output = self.handle(*args, **options)
   File 
 "/home/archesdev/arches/arches/arches/management/commands/packages.py", 
 line 277, in handle
 prevent_indexing=options["prevent_indexing"],
   File 
 "/home/archesdev/arches/arches/arches/management/commands/packages.py", 
 line 1070, in import_business_data
 prevent_indexing=prevent_indexing,
   File 
 "/home/archesdev/arches/arches/arches/app/utils/data_management/resources/importer.py",
  
 line 217, in import_business_data
 reader.report_errors()
   File 
 "/home/archesdev/arches/arches/arches/app/utils/data_management/resources/formats/format.py",
  
 line 178, in report_errors
 f = open(settings.RESOURCE_IMPORT_LOG, "a")
 PermissionError: [Errno 13] Permission denied: 
 '/home/archesdev/arches/eamena/eamena/logs/resource_import.log'

 I have checked both files csv and mapping, seems to be everyything as 
 it is explained in the arches documentation 
 

 Any idea about what is wrong? I have run the same in local and 
 everything works.


 Thanks again





 On Thursday, 28 January 2021 at 10:40:02 UTC 
 andrew...@historicengland.org.uk wrote:

> Yes, you need to run all the arches commands in the env, assuming you 
> have been following the documentation. Arches 5+ requires python 3.7+ so 
> you need to make sure that the env was built using the python 3.x 
> interpreter.
>
>
> On Thursday, January 28, 2021 at 10:32:41 AM UTC 
> gabriel.c...@gmail.com wrote:
>
>>
>> Hello, 
>>
>> My mistake 

[Arches] Re: Problems importing data from a csv with the mapping file

[andrew...@historicengland.org.uk]

I am afraid I am not a linux expert - we use Windows. Maybe someone else 
with linux knowledge can help?

On Thursday, January 28, 2021 at 2:32:01 PM UTC gabriel.c...@gmail.com 
wrote:

>
> Hi thanks for coming back to me, yes I tried to change the permisiions but 
> still receiving the same error. 
>
> sudo chmod 775 and 755 to the file getting permission:
> -rwxr-xr-x 1
>
>
>
> On Thursday, 28 January 2021 at 13:52:18 UTC 
> andrew...@historicengland.org.uk wrote:
>
>> The last line of the error is stating that it cannot write to the 
>> resource_import.log. Check write permissions on this. 
>>
>> f = open(settings.RESOURCE_IMPORT_LOG, "a")
>> PermissionError: [Errno 13] Permission denied: 
>> '/home/archesdev/arches/eamena/eamena/logs/resource_import.log'
>>
>> On Thursday, January 28, 2021 at 12:12:10 PM UTC gabriel.c...@gmail.com 
>> wrote:
>>
>>> Thanks it runs but it return few issues, that in local didnt. 
>>>
>>> otal resources saved: 6384
>>> 2021-01-28 05:48:59,707 
>>> arches.app.utils.data_management.resources.formats.csvfile WARNING  Grid ID 
>>> is null or not mapped on rows 
>>> 2,2,2,3,3,3,4,4,4,5,5,5,6,6,6,7,7,7,8,8,8,9,9,9,10,10,10,11,11,11,12,12,12,13,13,13,14,14,14,15,15,15,16,16,16,17,17,17,18,18,18,19,19,19,20,20,20,21,21,21,22,22,22,23,23,23,24,24,24,25,25,25,26,26,26,27,27,27,28,28,28,29,29,29,30,30,30,31,31,31,32,32,32,33,33,33,34,34,34,35,35,35,36,36,36,37,37,37,38,38,38,39,39,39,40,40,40,41,41,41,42,42,42,43,43,43,44,44,44,45,45,45,46,46,46,47,47,47,48,48,48,49,49,49,50,50,50,51,51,51,52,52,52,53,53,53,54,54,54,55,55,55,56,56,56,57,57,57,58,58,58,59,59,59,60,60,60,61,61,61,62,62,62,63,63,63,64,64,64,65,65,6..
>>>  
>>> and participates in a display value 
>>> function.
>>> Time to import_business_data = 0:08:29.653874
>>> * Errors occured during import. Some data may not have been 
>>> imported. For more information, check resource import error log: 
>>> /home/archesdev/arches/eamena/eamena/logs/resource_import.log
>>> 2021-01-28 05:49:00,812 arches.app.utils.task_management ERRORA 
>>> celery broker is running, but a celery worker is not available
>>> Traceback (most recent call last):
>>>   File "manage.py", line 30, in 
>>> execute_from_command_line(sys.argv)
>>>   File 
>>> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/__init__.py",
>>>  
>>> line 381, in execute_from_command_line
>>> utility.execute()
>>>   File 
>>> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/__init__.py",
>>>  
>>> line 375, in execute
>>> self.fetch_command(subcommand).run_from_argv(self.argv)
>>>   File 
>>> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/base.py",
>>>  
>>> line 323, in run_from_argv
>>> self.execute(*args, **cmd_options)
>>>   File 
>>> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/base.py",
>>>  
>>> line 364, in execute
>>> output = self.handle(*args, **options)
>>>   File 
>>> "/home/archesdev/arches/arches/arches/management/commands/packages.py", 
>>> line 277, in handle
>>> prevent_indexing=options["prevent_indexing"],
>>>   File 
>>> "/home/archesdev/arches/arches/arches/management/commands/packages.py", 
>>> line 1070, in import_business_data
>>> prevent_indexing=prevent_indexing,
>>>   File 
>>> "/home/archesdev/arches/arches/arches/app/utils/data_management/resources/importer.py",
>>>  
>>> line 217, in import_business_data
>>> reader.report_errors()
>>>   File 
>>> "/home/archesdev/arches/arches/arches/app/utils/data_management/resources/formats/format.py",
>>>  
>>> line 178, in report_errors
>>> f = open(settings.RESOURCE_IMPORT_LOG, "a")
>>> PermissionError: [Errno 13] Permission denied: 
>>> '/home/archesdev/arches/eamena/eamena/logs/resource_import.log'
>>>
>>> I have checked both files csv and mapping, seems to be everyything as it 
>>> is explained in the arches documentation 
>>> 
>>>
>>> Any idea about what is wrong? I have run the same in local and 
>>> everything works.
>>>
>>>
>>> Thanks again
>>>
>>>
>>>
>>>
>>>
>>> On Thursday, 28 January 2021 at 10:40:02 UTC 
>>> andrew...@historicengland.org.uk wrote:
>>>
 Yes, you need to run all the arches commands in the env, assuming you 
 have been following the documentation. Arches 5+ requires python 3.7+ so 
 you need to make sure that the env was built using the python 3.x 
 interpreter.


 On Thursday, January 28, 2021 at 10:32:41 AM UTC gabriel.c...@gmail.com 
 wrote:

>
> Hello, 
>
> My mistake currently the version we are running is 5x. After run the 
> command
>
> *-->*
>
> *  python manage.py packages -o import_business_data -s 
> /path/to/the/file.csv -c /path/to/the/file.mapping -ow 'overwrite' -bulk *
> *<--*
>
>
> in the 

[Arches] Re: Problems importing data from a csv with the mapping file

[Gabriel Contreras]

Hi thanks for coming back to me, yes I tried to change the permisiions but 
still receiving the same error. 

sudo chmod 775 and 755 to the file getting permission:
-rwxr-xr-x 1



On Thursday, 28 January 2021 at 13:52:18 UTC 
andrew...@historicengland.org.uk wrote:

> The last line of the error is stating that it cannot write to the 
> resource_import.log. Check write permissions on this. 
>
> f = open(settings.RESOURCE_IMPORT_LOG, "a")
> PermissionError: [Errno 13] Permission denied: 
> '/home/archesdev/arches/eamena/eamena/logs/resource_import.log'
>
> On Thursday, January 28, 2021 at 12:12:10 PM UTC gabriel.c...@gmail.com 
> wrote:
>
>> Thanks it runs but it return few issues, that in local didnt. 
>>
>> otal resources saved: 6384
>> 2021-01-28 05:48:59,707 
>> arches.app.utils.data_management.resources.formats.csvfile WARNING  Grid ID 
>> is null or not mapped on rows 
>> 2,2,2,3,3,3,4,4,4,5,5,5,6,6,6,7,7,7,8,8,8,9,9,9,10,10,10,11,11,11,12,12,12,13,13,13,14,14,14,15,15,15,16,16,16,17,17,17,18,18,18,19,19,19,20,20,20,21,21,21,22,22,22,23,23,23,24,24,24,25,25,25,26,26,26,27,27,27,28,28,28,29,29,29,30,30,30,31,31,31,32,32,32,33,33,33,34,34,34,35,35,35,36,36,36,37,37,37,38,38,38,39,39,39,40,40,40,41,41,41,42,42,42,43,43,43,44,44,44,45,45,45,46,46,46,47,47,47,48,48,48,49,49,49,50,50,50,51,51,51,52,52,52,53,53,53,54,54,54,55,55,55,56,56,56,57,57,57,58,58,58,59,59,59,60,60,60,61,61,61,62,62,62,63,63,63,64,64,64,65,65,6..
>>  
>> and participates in a display value 
>> function.
>> Time to import_business_data = 0:08:29.653874
>> * Errors occured during import. Some data may not have been imported. 
>> For more information, check resource import error log: 
>> /home/archesdev/arches/eamena/eamena/logs/resource_import.log
>> 2021-01-28 05:49:00,812 arches.app.utils.task_management ERRORA 
>> celery broker is running, but a celery worker is not available
>> Traceback (most recent call last):
>>   File "manage.py", line 30, in 
>> execute_from_command_line(sys.argv)
>>   File 
>> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/__init__.py",
>>  
>> line 381, in execute_from_command_line
>> utility.execute()
>>   File 
>> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/__init__.py",
>>  
>> line 375, in execute
>> self.fetch_command(subcommand).run_from_argv(self.argv)
>>   File 
>> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/base.py",
>>  
>> line 323, in run_from_argv
>> self.execute(*args, **cmd_options)
>>   File 
>> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/base.py",
>>  
>> line 364, in execute
>> output = self.handle(*args, **options)
>>   File 
>> "/home/archesdev/arches/arches/arches/management/commands/packages.py", 
>> line 277, in handle
>> prevent_indexing=options["prevent_indexing"],
>>   File 
>> "/home/archesdev/arches/arches/arches/management/commands/packages.py", 
>> line 1070, in import_business_data
>> prevent_indexing=prevent_indexing,
>>   File 
>> "/home/archesdev/arches/arches/arches/app/utils/data_management/resources/importer.py",
>>  
>> line 217, in import_business_data
>> reader.report_errors()
>>   File 
>> "/home/archesdev/arches/arches/arches/app/utils/data_management/resources/formats/format.py",
>>  
>> line 178, in report_errors
>> f = open(settings.RESOURCE_IMPORT_LOG, "a")
>> PermissionError: [Errno 13] Permission denied: 
>> '/home/archesdev/arches/eamena/eamena/logs/resource_import.log'
>>
>> I have checked both files csv and mapping, seems to be everyything as it 
>> is explained in the arches documentation 
>> 
>>
>> Any idea about what is wrong? I have run the same in local and everything 
>> works.
>>
>>
>> Thanks again
>>
>>
>>
>>
>>
>> On Thursday, 28 January 2021 at 10:40:02 UTC 
>> andrew...@historicengland.org.uk wrote:
>>
>>> Yes, you need to run all the arches commands in the env, assuming you 
>>> have been following the documentation. Arches 5+ requires python 3.7+ so 
>>> you need to make sure that the env was built using the python 3.x 
>>> interpreter.
>>>
>>>
>>> On Thursday, January 28, 2021 at 10:32:41 AM UTC gabriel.c...@gmail.com 
>>> wrote:
>>>

 Hello, 

 My mistake currently the version we are running is 5x. After run the 
 command

 *-->*

 *  python manage.py packages -o import_business_data -s 
 /path/to/the/file.csv -c /path/to/the/file.mapping -ow 'overwrite' -bulk *
 *<--*


 in the same folder than manage.py file the retun is:

 *Traceback (most recent call last):*
 *  File "manage.py", line 28, in *
 *from django.core.management import execute_from_command_line*
 *ImportError: No module named django.core.management*


 I guess that the error came because 

[Arches] Re: Problems importing data from a csv with the mapping file

[andrew...@historicengland.org.uk]

The last line of the error is stating that it cannot write to the 
resource_import.log. Check write permissions on this. 

f = open(settings.RESOURCE_IMPORT_LOG, "a")
PermissionError: [Errno 13] Permission denied: 
'/home/archesdev/arches/eamena/eamena/logs/resource_import.log'

On Thursday, January 28, 2021 at 12:12:10 PM UTC gabriel.c...@gmail.com 
wrote:

> Thanks it runs but it return few issues, that in local didnt. 
>
> otal resources saved: 6384
> 2021-01-28 05:48:59,707 
> arches.app.utils.data_management.resources.formats.csvfile WARNING  Grid ID 
> is null or not mapped on rows 
> 2,2,2,3,3,3,4,4,4,5,5,5,6,6,6,7,7,7,8,8,8,9,9,9,10,10,10,11,11,11,12,12,12,13,13,13,14,14,14,15,15,15,16,16,16,17,17,17,18,18,18,19,19,19,20,20,20,21,21,21,22,22,22,23,23,23,24,24,24,25,25,25,26,26,26,27,27,27,28,28,28,29,29,29,30,30,30,31,31,31,32,32,32,33,33,33,34,34,34,35,35,35,36,36,36,37,37,37,38,38,38,39,39,39,40,40,40,41,41,41,42,42,42,43,43,43,44,44,44,45,45,45,46,46,46,47,47,47,48,48,48,49,49,49,50,50,50,51,51,51,52,52,52,53,53,53,54,54,54,55,55,55,56,56,56,57,57,57,58,58,58,59,59,59,60,60,60,61,61,61,62,62,62,63,63,63,64,64,64,65,65,6..
>  
> and participates in a display value 
> function.
> Time to import_business_data = 0:08:29.653874
> * Errors occured during import. Some data may not have been imported. 
> For more information, check resource import error log: 
> /home/archesdev/arches/eamena/eamena/logs/resource_import.log
> 2021-01-28 05:49:00,812 arches.app.utils.task_management ERRORA celery 
> broker is running, but a celery worker is not available
> Traceback (most recent call last):
>   File "manage.py", line 30, in 
> execute_from_command_line(sys.argv)
>   File 
> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/__init__.py",
>  
> line 381, in execute_from_command_line
> utility.execute()
>   File 
> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/__init__.py",
>  
> line 375, in execute
> self.fetch_command(subcommand).run_from_argv(self.argv)
>   File 
> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/base.py",
>  
> line 323, in run_from_argv
> self.execute(*args, **cmd_options)
>   File 
> "/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/base.py",
>  
> line 364, in execute
> output = self.handle(*args, **options)
>   File 
> "/home/archesdev/arches/arches/arches/management/commands/packages.py", 
> line 277, in handle
> prevent_indexing=options["prevent_indexing"],
>   File 
> "/home/archesdev/arches/arches/arches/management/commands/packages.py", 
> line 1070, in import_business_data
> prevent_indexing=prevent_indexing,
>   File 
> "/home/archesdev/arches/arches/arches/app/utils/data_management/resources/importer.py",
>  
> line 217, in import_business_data
> reader.report_errors()
>   File 
> "/home/archesdev/arches/arches/arches/app/utils/data_management/resources/formats/format.py",
>  
> line 178, in report_errors
> f = open(settings.RESOURCE_IMPORT_LOG, "a")
> PermissionError: [Errno 13] Permission denied: 
> '/home/archesdev/arches/eamena/eamena/logs/resource_import.log'
>
> I have checked both files csv and mapping, seems to be everyything as it 
> is explained in the arches documentation 
> 
>
> Any idea about what is wrong? I have run the same in local and everything 
> works.
>
>
> Thanks again
>
>
>
>
>
> On Thursday, 28 January 2021 at 10:40:02 UTC 
> andrew...@historicengland.org.uk wrote:
>
>> Yes, you need to run all the arches commands in the env, assuming you 
>> have been following the documentation. Arches 5+ requires python 3.7+ so 
>> you need to make sure that the env was built using the python 3.x 
>> interpreter.
>>
>>
>> On Thursday, January 28, 2021 at 10:32:41 AM UTC gabriel.c...@gmail.com 
>> wrote:
>>
>>>
>>> Hello, 
>>>
>>> My mistake currently the version we are running is 5x. After run the 
>>> command
>>>
>>> *-->*
>>>
>>> *  python manage.py packages -o import_business_data -s 
>>> /path/to/the/file.csv -c /path/to/the/file.mapping -ow 'overwrite' -bulk *
>>> *<--*
>>>
>>>
>>> in the same folder than manage.py file the retun is:
>>>
>>> *Traceback (most recent call last):*
>>> *  File "manage.py", line 28, in *
>>> *from django.core.management import execute_from_command_line*
>>> *ImportError: No module named django.core.management*
>>>
>>>
>>> I guess that the error came because it should be run in the env mode, is 
>>> that correct? or maybe with python2 or 3 instead of just python manage. 
>>> py... 
>>>
>>>
>>> Thanks in advance for answering me.
>>>
>>> On Thursday, 28 January 2021 at 09:10:09 UTC 
>>> andrew...@historicengland.org.uk wrote:
>>>
 Just an observation, are you using Arches 4.x or 5.x? I see you are 
 using Python 2.7 so am assuming Arches 

[Arches] Re: Problems importing data from a csv with the mapping file

[Gabriel Contreras]

Thanks it runs but it return few issues, that in local didnt. 

otal resources saved: 6384
2021-01-28 05:48:59,707 
arches.app.utils.data_management.resources.formats.csvfile WARNING  Grid ID 
is null or not mapped on rows 
2,2,2,3,3,3,4,4,4,5,5,5,6,6,6,7,7,7,8,8,8,9,9,9,10,10,10,11,11,11,12,12,12,13,13,13,14,14,14,15,15,15,16,16,16,17,17,17,18,18,18,19,19,19,20,20,20,21,21,21,22,22,22,23,23,23,24,24,24,25,25,25,26,26,26,27,27,27,28,28,28,29,29,29,30,30,30,31,31,31,32,32,32,33,33,33,34,34,34,35,35,35,36,36,36,37,37,37,38,38,38,39,39,39,40,40,40,41,41,41,42,42,42,43,43,43,44,44,44,45,45,45,46,46,46,47,47,47,48,48,48,49,49,49,50,50,50,51,51,51,52,52,52,53,53,53,54,54,54,55,55,55,56,56,56,57,57,57,58,58,58,59,59,59,60,60,60,61,61,61,62,62,62,63,63,63,64,64,64,65,65,6..
 
and participates in a display value 
function.
Time to import_business_data = 0:08:29.653874
* Errors occured during import. Some data may not have been imported. 
For more information, check resource import error log: 
/home/archesdev/arches/eamena/eamena/logs/resource_import.log
2021-01-28 05:49:00,812 arches.app.utils.task_management ERRORA celery 
broker is running, but a celery worker is not available
Traceback (most recent call last):
  File "manage.py", line 30, in 
execute_from_command_line(sys.argv)
  File 
"/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/__init__.py",
 
line 381, in execute_from_command_line
utility.execute()
  File 
"/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/__init__.py",
 
line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
  File 
"/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/base.py",
 
line 323, in run_from_argv
self.execute(*args, **cmd_options)
  File 
"/home/archesdev/arches/ENV/lib/python3.6/site-packages/django/core/management/base.py",
 
line 364, in execute
output = self.handle(*args, **options)
  File 
"/home/archesdev/arches/arches/arches/management/commands/packages.py", 
line 277, in handle
prevent_indexing=options["prevent_indexing"],
  File 
"/home/archesdev/arches/arches/arches/management/commands/packages.py", 
line 1070, in import_business_data
prevent_indexing=prevent_indexing,
  File 
"/home/archesdev/arches/arches/arches/app/utils/data_management/resources/importer.py",
 
line 217, in import_business_data
reader.report_errors()
  File 
"/home/archesdev/arches/arches/arches/app/utils/data_management/resources/formats/format.py",
 
line 178, in report_errors
f = open(settings.RESOURCE_IMPORT_LOG, "a")
PermissionError: [Errno 13] Permission denied: 
'/home/archesdev/arches/eamena/eamena/logs/resource_import.log'

I have checked both files csv and mapping, seems to be everyything as it is 
explained in the arches documentation 


Any idea about what is wrong? I have run the same in local and everything 
works.


Thanks again





On Thursday, 28 January 2021 at 10:40:02 UTC 
andrew...@historicengland.org.uk wrote:

> Yes, you need to run all the arches commands in the env, assuming you have 
> been following the documentation. Arches 5+ requires python 3.7+ so you 
> need to make sure that the env was built using the python 3.x interpreter.
>
>
> On Thursday, January 28, 2021 at 10:32:41 AM UTC gabriel.c...@gmail.com 
> wrote:
>
>>
>> Hello, 
>>
>> My mistake currently the version we are running is 5x. After run the 
>> command
>>
>> *-->*
>>
>> *  python manage.py packages -o import_business_data -s 
>> /path/to/the/file.csv -c /path/to/the/file.mapping -ow 'overwrite' -bulk *
>> *<--*
>>
>>
>> in the same folder than manage.py file the retun is:
>>
>> *Traceback (most recent call last):*
>> *  File "manage.py", line 28, in *
>> *from django.core.management import execute_from_command_line*
>> *ImportError: No module named django.core.management*
>>
>>
>> I guess that the error came because it should be run in the env mode, is 
>> that correct? or maybe with python2 or 3 instead of just python manage. 
>> py... 
>>
>>
>> Thanks in advance for answering me.
>>
>> On Thursday, 28 January 2021 at 09:10:09 UTC 
>> andrew...@historicengland.org.uk wrote:
>>
>>> Just an observation, are you using Arches 4.x or 5.x? I see you are 
>>> using Python 2.7 so am assuming Arches 4.x?
>>>
>>> On Wednesday, January 27, 2021 at 11:04:05 AM UTC gabriel.c...@gmail.com 
>>> wrote:
>>>
 Hello,

 I hope someone could help with this, or maybe someone had the same 
 problem.

 I am trying to import some csv by using

 python manage.py packages -o import_business_data -s 
 /path/to/the/file.csv -c /path/to/the/file.mapping -ow 'overwrite' -bulk

 After run the command in my local machine it successfully load all the 
 data, but when doing the same in the instance I get this messages:

[Arches] Re: Problems importing data from a csv with the mapping file

[andrew...@historicengland.org.uk]

Yes, you need to run all the arches commands in the env, assuming you have 
been following the documentation. Arches 5+ requires python 3.7+ so you 
need to make sure that the env was built using the python 3.x interpreter.


On Thursday, January 28, 2021 at 10:32:41 AM UTC gabriel.c...@gmail.com 
wrote:

>
> Hello, 
>
> My mistake currently the version we are running is 5x. After run the 
> command
>
> *-->*
>
> *  python manage.py packages -o import_business_data -s 
> /path/to/the/file.csv -c /path/to/the/file.mapping -ow 'overwrite' -bulk *
> *<--*
>
>
> in the same folder than manage.py file the retun is:
>
> *Traceback (most recent call last):*
> *  File "manage.py", line 28, in *
> *from django.core.management import execute_from_command_line*
> *ImportError: No module named django.core.management*
>
>
> I guess that the error came because it should be run in the env mode, is 
> that correct? or maybe with python2 or 3 instead of just python manage. 
> py... 
>
>
> Thanks in advance for answering me.
>
> On Thursday, 28 January 2021 at 09:10:09 UTC 
> andrew...@historicengland.org.uk wrote:
>
>> Just an observation, are you using Arches 4.x or 5.x? I see you are using 
>> Python 2.7 so am assuming Arches 4.x?
>>
>> On Wednesday, January 27, 2021 at 11:04:05 AM UTC gabriel.c...@gmail.com 
>> wrote:
>>
>>> Hello,
>>>
>>> I hope someone could help with this, or maybe someone had the same 
>>> problem.
>>>
>>> I am trying to import some csv by using
>>>
>>> python manage.py packages -o import_business_data -s 
>>> /path/to/the/file.csv -c /path/to/the/file.mapping -ow 'overwrite' -bulk
>>>
>>> After run the command in my local machine it successfully load all the 
>>> data, but when doing the same in the instance I get this messages:
>>>
>>> Traceback (most recent call last):
>>>   File "manage.py", line 28, in 
>>> execute_from_command_line(sys.argv)
>>>   File 
>>> "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py",
>>>  
>>> line 364, in execute_from_command_line
>>> utility.execute()
>>>   File 
>>> "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py",
>>>  
>>> line 308, in execute
>>> settings.INSTALLED_APPS
>>>   File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", 
>>> line 56, in __getattr__
>>> self._setup(name)
>>>   File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", 
>>> line 41, in _setup
>>> self._wrapped = Settings(settings_module)
>>>   File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", 
>>> line 110, in __init__
>>> mod = importlib.import_module(self.SETTINGS_MODULE)
>>>   File "/usr/lib/python2.7/importlib/__init__.py", line 37, in 
>>> import_module
>>> __import__(name)
>>>   File "/opt/maindb-app/eamena-main-app/eamena/eamena/settings.py", line 
>>> 225, in 
>>> ('NAME.E41', _('Resource Names')),
>>>   File 
>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/__init__.py",
>>>  
>>> line 89, in ugettext
>>> return _trans.ugettext(message)
>>>   File 
>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py",
>>>  
>>> line 345, in ugettext
>>> return do_translate(message, 'ugettext')
>>>   File 
>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py",
>>>  
>>> line 321, in do_translate
>>> _default = _default or translation(settings.LANGUAGE_CODE)
>>>   File 
>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py",
>>>  
>>> line 228, in translation
>>> _translations[language] = DjangoTranslation(language)
>>>   File 
>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py",
>>>  
>>> line 129, in __init__
>>> self._add_installed_apps_translations()
>>>   File 
>>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py",
>>>  
>>> line 171, in _add_installed_apps_translations
>>> "The translation infrastructure cannot be initialized before the "
>>> django.core.exceptions.AppRegistryNotReady: The translation 
>>> infrastructure cannot be initialized before the apps registry is ready. 
>>> Check that you don't make non-lazy gettext calls at import time.
>>>
>>> I am not sure if changing settings.py from 
>>>
>>> from django.utils.translation import gettext as _ 
>>>
>>> to 
>>>
>>> from django.utils.translation import ugettext_lazy as _ 
>>>
>>> Will resolve the issue or it going to generate otehr adittional errors. 
>>>
>>>
>>>
>>> Regards,
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>

-- 
-- To post, send email to archesproject@googlegroups.com. To unsubscribe, send 
email to archesproject+unsubscr...@googlegroups.com. For more information, 
visit https://groups.google.com/d/forum/archesproject?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Arches Project" group.
To unsubscribe from this group and stop receiving emails from it, send an 

[Arches] Re: Problems importing data from a csv with the mapping file

[Gabriel Contreras]

Hello, 

My mistake currently the version we are running is 5x. After run the command

*-->*

*  python manage.py packages -o import_business_data -s 
/path/to/the/file.csv -c /path/to/the/file.mapping -ow 'overwrite' -bulk *
*<--*


in the same folder than manage.py file the retun is:

*Traceback (most recent call last):*
*  File "manage.py", line 28, in *
*from django.core.management import execute_from_command_line*
*ImportError: No module named django.core.management*


I guess that the error came because it should be run in the env mode, is 
that correct? or maybe with python2 or 3 instead of just python manage. 
py... 


Thanks in advance for answering me.

On Thursday, 28 January 2021 at 09:10:09 UTC 
andrew...@historicengland.org.uk wrote:

> Just an observation, are you using Arches 4.x or 5.x? I see you are using 
> Python 2.7 so am assuming Arches 4.x?
>
> On Wednesday, January 27, 2021 at 11:04:05 AM UTC gabriel.c...@gmail.com 
> wrote:
>
>> Hello,
>>
>> I hope someone could help with this, or maybe someone had the same 
>> problem.
>>
>> I am trying to import some csv by using
>>
>> python manage.py packages -o import_business_data -s 
>> /path/to/the/file.csv -c /path/to/the/file.mapping -ow 'overwrite' -bulk
>>
>> After run the command in my local machine it successfully load all the 
>> data, but when doing the same in the instance I get this messages:
>>
>> Traceback (most recent call last):
>>   File "manage.py", line 28, in 
>> execute_from_command_line(sys.argv)
>>   File 
>> "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", 
>> line 364, in execute_from_command_line
>> utility.execute()
>>   File 
>> "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", 
>> line 308, in execute
>> settings.INSTALLED_APPS
>>   File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", 
>> line 56, in __getattr__
>> self._setup(name)
>>   File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", 
>> line 41, in _setup
>> self._wrapped = Settings(settings_module)
>>   File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", 
>> line 110, in __init__
>> mod = importlib.import_module(self.SETTINGS_MODULE)
>>   File "/usr/lib/python2.7/importlib/__init__.py", line 37, in 
>> import_module
>> __import__(name)
>>   File "/opt/maindb-app/eamena-main-app/eamena/eamena/settings.py", line 
>> 225, in 
>> ('NAME.E41', _('Resource Names')),
>>   File 
>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/__init__.py",
>>  
>> line 89, in ugettext
>> return _trans.ugettext(message)
>>   File 
>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py",
>>  
>> line 345, in ugettext
>> return do_translate(message, 'ugettext')
>>   File 
>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py",
>>  
>> line 321, in do_translate
>> _default = _default or translation(settings.LANGUAGE_CODE)
>>   File 
>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py",
>>  
>> line 228, in translation
>> _translations[language] = DjangoTranslation(language)
>>   File 
>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py",
>>  
>> line 129, in __init__
>> self._add_installed_apps_translations()
>>   File 
>> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py",
>>  
>> line 171, in _add_installed_apps_translations
>> "The translation infrastructure cannot be initialized before the "
>> django.core.exceptions.AppRegistryNotReady: The translation 
>> infrastructure cannot be initialized before the apps registry is ready. 
>> Check that you don't make non-lazy gettext calls at import time.
>>
>> I am not sure if changing settings.py from 
>>
>> from django.utils.translation import gettext as _ 
>>
>> to 
>>
>> from django.utils.translation import ugettext_lazy as _ 
>>
>> Will resolve the issue or it going to generate otehr adittional errors. 
>>
>>
>>
>> Regards,
>>
>>
>>
>>
>>
>>
>>
>>
>>

-- 
-- To post, send email to archesproject@googlegroups.com. To unsubscribe, send 
email to archesproject+unsubscr...@googlegroups.com. For more information, 
visit https://groups.google.com/d/forum/archesproject?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Arches Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to archesproject+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/archesproject/eb3c1e11-3d26-4dc8-b126-f01ebce09087n%40googlegroups.com.

[Arches] Re: Problems importing data from a csv with the mapping file

[andrew...@historicengland.org.uk]

Just an observation, are you using Arches 4.x or 5.x? I see you are using 
Python 2.7 so am assuming Arches 4.x?

On Wednesday, January 27, 2021 at 11:04:05 AM UTC gabriel.c...@gmail.com 
wrote:

> Hello,
>
> I hope someone could help with this, or maybe someone had the same problem.
>
> I am trying to import some csv by using
>
> python manage.py packages -o import_business_data -s /path/to/the/file.csv 
> -c /path/to/the/file.mapping -ow 'overwrite' -bulk
>
> After run the command in my local machine it successfully load all the 
> data, but when doing the same in the instance I get this messages:
>
> Traceback (most recent call last):
>   File "manage.py", line 28, in 
> execute_from_command_line(sys.argv)
>   File 
> "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", 
> line 364, in execute_from_command_line
> utility.execute()
>   File 
> "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", 
> line 308, in execute
> settings.INSTALLED_APPS
>   File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", 
> line 56, in __getattr__
> self._setup(name)
>   File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", 
> line 41, in _setup
> self._wrapped = Settings(settings_module)
>   File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", 
> line 110, in __init__
> mod = importlib.import_module(self.SETTINGS_MODULE)
>   File "/usr/lib/python2.7/importlib/__init__.py", line 37, in 
> import_module
> __import__(name)
>   File "/opt/maindb-app/eamena-main-app/eamena/eamena/settings.py", line 
> 225, in 
> ('NAME.E41', _('Resource Names')),
>   File 
> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/__init__.py",
>  
> line 89, in ugettext
> return _trans.ugettext(message)
>   File 
> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py",
>  
> line 345, in ugettext
> return do_translate(message, 'ugettext')
>   File 
> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py",
>  
> line 321, in do_translate
> _default = _default or translation(settings.LANGUAGE_CODE)
>   File 
> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py",
>  
> line 228, in translation
> _translations[language] = DjangoTranslation(language)
>   File 
> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py",
>  
> line 129, in __init__
> self._add_installed_apps_translations()
>   File 
> "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py",
>  
> line 171, in _add_installed_apps_translations
> "The translation infrastructure cannot be initialized before the "
> django.core.exceptions.AppRegistryNotReady: The translation infrastructure 
> cannot be initialized before the apps registry is ready. Check that you 
> don't make non-lazy gettext calls at import time.
>
> I am not sure if changing settings.py from 
>
> from django.utils.translation import gettext as _ 
>
> to 
>
> from django.utils.translation import ugettext_lazy as _ 
>
> Will resolve the issue or it going to generate otehr adittional errors. 
>
>
>
> Regards,
>
>
>
>
>
>
>
>
>

-- 
-- To post, send email to archesproject@googlegroups.com. To unsubscribe, send 
email to archesproject+unsubscr...@googlegroups.com. For more information, 
visit https://groups.google.com/d/forum/archesproject?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Arches Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to archesproject+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/archesproject/c2071fe9-bb9f-44c9-94a1-1cdb94d36cb3n%40googlegroups.com.