[Arches] Re: Error Exporting Business Data from Arches 4.4.2

[Martha S]

This is great, Bryan, thank you.

Once we repair the garbage that resulted during import, this might enable 
us to have the preferred spellings in our database without issue. 

I shall report back once we've tested. I kept wondering how non-English 
installations manage.

Martha



On Tuesday, October 8, 2019 at 4:08:01 PM UTC-7, Bryan Alvey wrote:
>
> Hi Martha - 
>
> I'm not sure this will is what you are after, but if you are having 
> problems importiing/exporting non-ascii characters to/from Arches, this may 
> help. We uploaded a cyrillic data set by CSV successfully using this.
>
> https://github.com/archesproject/arches/issues/2831
>
>
> For those who have this issue on Apache2, here is what solved my problem:
>
>
> (
> https://code.djangoproject.com/wiki/django_apache_and_mod_wsgi#AdditionalTweaking
> )
>
>
> If you're taking advantage of the great Internationalization features of 
> Django you may come across a curious problem. Namely, uploading of 
> non-ascii filenames with the Django storage system with the default apache 
> settings on most systems will trigger UnicodeEncodeError exceptions when 
> calling functions like os.path(). To avoid these issues, ensure that the 
> following lines are included in your apache envvars file (typically found 
> in /etc/apache2/envvars).
>
>
> export LANG='en_US.UTF-8'
> export LC_ALL='en_US.UTF-8'
>
>
> This error likely wont rear its head during development on the test server 
> as, when run from the command line, the ./manage.py script inherits the 
> users language and locale settings.'
>
>
>
>
> Not sure this is what you want, but I thought it may help.
>
>
> Bryan
>
>
>
> On Saturday, 28 September 2019 01:13:59 UTC+1, Martha S wrote:
>>
>> I am trying to export all the data for a particular resource model to CSV 
>> for review and modification and ran into an error during the process -- 
>> UnicodeEncodeError: 
>> 'ascii' codec can't encode character u'\xa6' in position 51: ordinal not in 
>> range(128)
>>  
>> *My command*
>> python manage.py packages -o export_business_data -d 
>> '/hpladata/Projects/Downloads/Historic District Mapping Files' -f 'csv' 
>> -c '/hpladata/Projects/Downloads/Historic District Mapping 
>> Files/Historic District.mapping' 
>>
>> *Here's the full error dump*
>> operation: export_business_data
>> Traceback (most recent call last):
>>   File "manage.py", line 29, in 
>> execute_from_command_line(sys.argv)
>>   File 
>> "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", 
>> line 364, in execute_from_command_line
>> utility.execute()
>>   File 
>> "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", 
>> line 356, in execute
>> self.fetch_command(subcommand).run_from_argv(self.argv)
>>   File 
>> "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", 
>> line 283, in run_from_argv
>> self.execute(*args, **cmd_options)
>>   File 
>> "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", 
>> line 330, in execute
>> output = self.handle(*args, **options)
>>   File "/Projects/prod/arches/arches/management/commands/packages.py", 
>> line 190, in handle
>> self.export_business_data(options['dest_dir'], options['format'], 
>> options['config_file'], options['graphs'], options['single_file'])
>>   File "/Projects/prod/arches/arches/management/commands/packages.py", 
>> line 770, in export_business_data
>> data = resource_exporter.export(graph_id=graph, 
>> resourceinstanceids=None)
>>   File 
>> "/Projects/prod/arches/arches/app/utils/data_management/resources/exporter.py",
>>  
>> line 37, in export
>> resources = self.writer.write_resources(graph_id=graph_id, 
>> resourceinstanceids=resourceinstanceids)
>>   File 
>> "/Projects/prod/arches/arches/app/utils/data_management/resources/formats/csvfile.py",
>>  
>> line 194, in write_resources
>> csvs_for_export = csvs_for_export + self.write_resource_relations(
>> file_name=self.file_name)
>>   File 
>> "/Projects/prod/arches/arches/app/utils/data_management/resources/formats/csvfile.py",
>>  
>> line 215, in write_resource_relations
>> csvwriter.writerow({k:str(v) for k,v in relation.items()})
>>   File 
>> "/Projects/prod/arches/arches/app/utils/data_management/resources/formats/csvfile.py",
>>  
>> line 215, in 
>> csvwriter.writerow({k:str(v) for k,v in relation.items()})
>> UnicodeEncodeError: 'ascii' codec can't encode character u'\xa6' in 
>> position 51: ordinal not in range(128)
>>
>> Any suggestions?
>>
>> Thanks,
>> Martha
>>
>

-- 
-- To post, send email to archesproject@googlegroups.com. To unsubscribe, send 
email to archesproject+unsubscr...@googlegroups.com. For more information, 
visit https://groups.google.com/d/forum/archesproject?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Arches Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 

[Arches] Re: Error Exporting Business Data from Arches 4.4.2

[Bryan Alvey]

Hi Martha - 

I'm not sure this will is what you are after, but if you are having 
problems importiing/exporting non-ascii characters to/from Arches, this may 
help. We uploaded a cyrillic data set by CSV successfully using this.

https://github.com/archesproject/arches/issues/2831


For those who have this issue on Apache2, here is what solved my problem:


(
https://code.djangoproject.com/wiki/django_apache_and_mod_wsgi#AdditionalTweaking
)


If you're taking advantage of the great Internationalization features of 
Django you may come across a curious problem. Namely, uploading of 
non-ascii filenames with the Django storage system with the default apache 
settings on most systems will trigger UnicodeEncodeError exceptions when 
calling functions like os.path(). To avoid these issues, ensure that the 
following lines are included in your apache envvars file (typically found 
in /etc/apache2/envvars).


export LANG='en_US.UTF-8'
export LC_ALL='en_US.UTF-8'


This error likely wont rear its head during development on the test server 
as, when run from the command line, the ./manage.py script inherits the 
users language and locale settings.'




Not sure this is what you want, but I thought it may help.


Bryan



On Saturday, 28 September 2019 01:13:59 UTC+1, Martha S wrote:
>
> I am trying to export all the data for a particular resource model to CSV 
> for review and modification and ran into an error during the process -- 
> UnicodeEncodeError: 
> 'ascii' codec can't encode character u'\xa6' in position 51: ordinal not in 
> range(128)
>  
> *My command*
> python manage.py packages -o export_business_data -d 
> '/hpladata/Projects/Downloads/Historic District Mapping Files' -f 'csv' 
> -c '/hpladata/Projects/Downloads/Historic District Mapping Files/Historic 
> District.mapping' 
>
> *Here's the full error dump*
> operation: export_business_data
> Traceback (most recent call last):
>   File "manage.py", line 29, in 
> execute_from_command_line(sys.argv)
>   File 
> "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", 
> line 364, in execute_from_command_line
> utility.execute()
>   File 
> "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", 
> line 356, in execute
> self.fetch_command(subcommand).run_from_argv(self.argv)
>   File 
> "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", 
> line 283, in run_from_argv
> self.execute(*args, **cmd_options)
>   File 
> "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", 
> line 330, in execute
> output = self.handle(*args, **options)
>   File "/Projects/prod/arches/arches/management/commands/packages.py", 
> line 190, in handle
> self.export_business_data(options['dest_dir'], options['format'], 
> options['config_file'], options['graphs'], options['single_file'])
>   File "/Projects/prod/arches/arches/management/commands/packages.py", 
> line 770, in export_business_data
> data = resource_exporter.export(graph_id=graph, 
> resourceinstanceids=None)
>   File 
> "/Projects/prod/arches/arches/app/utils/data_management/resources/exporter.py",
>  
> line 37, in export
> resources = self.writer.write_resources(graph_id=graph_id, 
> resourceinstanceids=resourceinstanceids)
>   File 
> "/Projects/prod/arches/arches/app/utils/data_management/resources/formats/csvfile.py",
>  
> line 194, in write_resources
> csvs_for_export = csvs_for_export + self.write_resource_relations(
> file_name=self.file_name)
>   File 
> "/Projects/prod/arches/arches/app/utils/data_management/resources/formats/csvfile.py",
>  
> line 215, in write_resource_relations
> csvwriter.writerow({k:str(v) for k,v in relation.items()})
>   File 
> "/Projects/prod/arches/arches/app/utils/data_management/resources/formats/csvfile.py",
>  
> line 215, in 
> csvwriter.writerow({k:str(v) for k,v in relation.items()})
> UnicodeEncodeError: 'ascii' codec can't encode character u'\xa6' in 
> position 51: ordinal not in range(128)
>
> Any suggestions?
>
> Thanks,
> Martha
>

-- 
-- To post, send email to archesproject@googlegroups.com. To unsubscribe, send 
email to archesproject+unsubscr...@googlegroups.com. For more information, 
visit https://groups.google.com/d/forum/archesproject?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Arches Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to archesproject+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/archesproject/5e35127d-b1a0-4da6-a109-be901a36ae1c%40googlegroups.com.

[Arches] Upcoming Arches Webinars: Intro to the Arches Platform

[Alina Myklebust]

Hello Arches Forum Members, 

Join us next week (Thursday, October 17th at 9:30am) via webinar to learn 
about the key capabilities of the Arches Platform and participate in a live 
Q chat.
 
Intro to the Arches Platform is a four-part webinar series.  This series is 
ideal for anyone interested in learning about and exploring the key 
capabilities, tools, concepts and software architecture of the Arches 
cultural heritage data management platform.  
 
Intro to the Arches Platform Webinar Series:
*Part 1:*  Capabilities:  Data Management Overview, Data Visualization & 
Discovery (Thursday, October 17, 2019 at 9:30am Pacific Time) 

*Part 2:*  Data Management Tools *(coming soon)*
*Part 3:*  Data Technology Concepts *(coming soon)*
*Part 4:*  Architecture and Software Stack *(coming soon)*
 
Registration for Part 1 is now open, please sign-up here:  
http://bit.ly/2AJrBys
 
We hope you can join us!
 
Cheers,


Alina
The Arches Team

-- 
-- To post, send email to archesproject@googlegroups.com. To unsubscribe, send 
email to archesproject+unsubscr...@googlegroups.com. For more information, 
visit https://groups.google.com/d/forum/archesproject?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Arches Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to archesproject+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/archesproject/e2435a28-a08e-45f0-924e-b678b4ed1fa5%40googlegroups.com.

RE: [Arches] Upgrading to a specific version of Arches - 4.4.1 (not the latest 4.4.2 version)

[Shewring, Tom]

Hi Cyrus,

Unfortunately when I tested the shortcut process detailed below I ran into 
errors in section 1 upgrade to version 4.3.1:

After running pip install arches==4.3.1 --upgrade --no-binary :all:  (success)
Trying to uninstall pycryptodome failed

pip uninstall pycryptodome, rdflib-jsonld
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. 
Please upgrade your Python as Python 2.7 won't be maintained after that date. A 
future version of pip will drop support for Python 2.7. More details about 
Python 2 support in pip, can be found at 
https://pip.pypa.io/en/latest/development/release-process/#python-2-support
ERROR: Invalid requirement: 'pycryptodome,'

Installing pycryptodome works
pip install pycryptodome django-oauth-toolkit==1.1.2 PyLD[requests]==1.0.3 
pyprind==2.11.2 (success)

but then py migrate fails

python manage.py migrate
Traceback (most recent call last):
  File "manage.py", line 29, in 
execute_from_command_line(sys.argv)
  File "c:\obap\env4.1.1\lib\site-packages\django\core\management\__init__.py", 
line 364, in execute_from_command_line
utility.execute()
  File "c:\obap\env4.1.1\lib\site-packages\django\core\management\__init__.py", 
line 338, in execute
django.setup()
  File "c:\obap\env4.1.1\lib\site-packages\django\__init__.py", line 27, in 
setup
apps.populate(settings.INSTALLED_APPS)
  File "c:\obap\env4.1.1\lib\site-packages\django\apps\registry.py", line 85, 
in populate
app_config = AppConfig.create(entry)
  File "c:\obap\env4.1.1\lib\site-packages\django\apps\config.py", line 94, in 
create
module = import_module(entry)
  File "c:\python27\Lib\importlib\__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named revproxy


I am thinking that it will be better to create a new clean 4.4.1 environment 
and recreate the application from there. Could you advise on the best process 
to follow if I take this approach.

Tom Shewring
GIS Systems Developer
IMT Development Centre
07748 182134

IMT Development and Programmes:  Together, we enable the celebration of 
England’s historic environment by empowering our customers and the public 
through technology and innovation.

From: Cyrus Hiatt [mailto:cyrusnhi...@gmail.com]
Sent: 04 October 2019 01:28
To: Adam Cox
Cc: Alexei Peters; Shewring, Tom; Arches Project
Subject: Re: [Arches] Upgrading to a specific version of Arches - 4.4.1 (not 
the latest 4.4.2 version)

Hi Tom -

One of the challenges for upgrading from 4.1.1 will be upgrading your 
project(s). This is because at version 4.2 we switched from Bower to Yarn for 
js dependencies because bower was getting deprecated. As Adam has mentioned, 
each version of Arches has the upgrade process outlined in the release notes: 
https://github.com/archesproject/arches/tree/master/releases.  You could go 
from version to version following the upgrade instructions for each version. 
Alternatively, you could try the process below which is a bit shorter. I test 
the upgrade from version to version with each release, so it should go 
smoothly. I haven't tested the shortcut process below, but I think it should be 
fine. In either case, there are quite a few steps going from 4.1 to 4.4, so I 
would recommend creating a database backup and web server snapshot (if 
possible) before getting started.

1. upgrade to version 4.3.1:


pip install arches==4.3.1 --upgrade --no-binary :all:

pip uninstall pycryptodome, rdflib-jsonld

pip install pycryptodome django-oauth-toolkit==1.1.2 PyLD[requests]==1.0.3 
pyprind==2.11.2



python manage.py migrate

python manage.py es delete_indexes

python manage.py es setup_indexes

python manage.py es index_database

2. follow the project upgrade steps in the 4.2.0 release notes starting with 
the yarn install:

https://github.com/archesproject/arches/blob/master/releases/4.2.0.md#installing-yarn

3. Then follow the project upgrade steps for version 4.3.1:

https://github.com/archesproject/arches/blob/master/releases/4.3.1.md#upgrading-an-arches-project

4. Upgrade to version 4.4.1:


pip install arches==4.4.1 --upgrade --no-binary :all:

python manage.py migrate

python manage.py es delete_indexes

python manage.py es setup_indexes

python manage.py es index_database

5. Then complete the project upgrades for 4.4.1:

https://github.com/archesproject/arches/blob/master/releases/4.4.1.md#upgrading-an-arches-project

Hope that helps and that your upgrade goes smoothly. Let me know if you have 
any questions.

- Cyrus



On Thu, Oct 3, 2019 at 9:41 AM Adam Cox 
mailto:mr.adam...@gmail.com>> wrote:
Hi Tom, one other thought, sometimes I like to just create a brand new virtual 
environment and install the new version in there, leaving the existing on as 
is. Then you can switch which environment is used by your app in your apache 
config (if that's how you're serving it), etc, and you can switch back if 
something goes wrong with the pip installation process (which, frankly, is 
unlikely).