subject:"\[jira\] \[Commented\] \(AIRAVATA\-3697\) Handle unicode characters in file names"

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-09-21 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767584#comment-17767584
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit 71bc7c3d30e34daef0b308de15092f00cfe821cb in airavata's branch 
refs/heads/master from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata.git;h=71bc7c3d30 ]

AIRAVATA-3697 Set lang and locale on WSGIDaemonProcess

Copying this change over from the template to the per gateway override files.


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
>

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-09-21 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767585#comment-17767585
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit b3d68364c323bd03af0f435c54b6039a3743f148 in airavata's branch 
refs/heads/master from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata.git;h=b3d68364c3 ]

Merge branch 'AIRAVATA-3697'


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return self.dispatch(request, *args, **kwargs)
>  File 
>

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-09-21 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767583#comment-17767583
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit 6fcefb2c476fd73cc5070d378f0730713f5ac65f in airavata's branch 
refs/heads/master from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata.git;h=6fcefb2c47 ]

AIRAVATA-3697 Switching to column level utf8 character set


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-09-19 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766801#comment-17766801
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit 71bc7c3d30e34daef0b308de15092f00cfe821cb in airavata's branch 
refs/heads/AIRAVATA-3697 from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata.git;h=71bc7c3d30 ]

AIRAVATA-3697 Set lang and locale on WSGIDaemonProcess

Copying this change over from the template to the per gateway override files.


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
>

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-07-03 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17739718#comment-17739718
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit 6fcefb2c476fd73cc5070d378f0730713f5ac65f in airavata's branch 
refs/heads/develop from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata.git;h=6fcefb2c47 ]

AIRAVATA-3697 Switching to column level utf8 character set


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-07-03 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17739719#comment-17739719
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit 3a5b8358cf295ae33d151d59ceacc71279580aa4 in airavata's branch 
refs/heads/develop from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata.git;h=3a5b8358cf ]

Merge branch 'AIRAVATA-3697' into develop


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return self.dispatch(request, *args,

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-07-03 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17739717#comment-17739717
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit 6fcefb2c476fd73cc5070d378f0730713f5ac65f in airavata's branch 
refs/heads/AIRAVATA-3697 from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata.git;h=6fcefb2c47 ]

AIRAVATA-3697 Switching to column level utf8 character set


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-07-03 Thread Marcus Christie (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17739683#comment-17739683
 ] 

Marcus Christie commented on AIRAVATA-3697:
---

Migration script for the changes mentioned in the previous comment:

{code:sql}
alter table DATA_PRODUCT modify PRODUCT_NAME text character set utf8mb4 collate 
utf8mb4_bin;
alter table DATA_PRODUCT modify PRODUCT_DESCRIPTION  varchar(255) character set 
utf8mb4 collate utf8mb4_bin;
alter table DATA_REPLICA_LOCATION modify REPLICA_NAME text character set 
utf8mb4 collate utf8mb4_bin;
alter table DATA_REPLICA_LOCATION modify REPLICA_DESCRIPTION varchar(255) 
character set utf8mb4 collate utf8mb4_bin;
alter table DATA_REPLICA_LOCATION modify FILE_PATH  varchar(1024) character set 
utf8mb4 collate utf8mb4_bin;
{code}
 

> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args,

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-06-29 Thread Marcus Christie (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17738797#comment-17738797
 ] 

Marcus Christie commented on AIRAVATA-3697:
---

So it turns out that:
* it takes a long time for this conversion to complete especially 
DATA_REPLICA_LOCATION
* when column PRODUCT_URI is utf8 on DATA_PRODUCT but still latin1 on 
DATA_REPLICA_LOCATION, the JOIN isn't able to take advantage of the index and 
hence performance really suffers. So much so that requests time out.

So the dream of doing an online conversion in production is dead, at least with 
this approach.

It occurs to me though that we don't need all of the columns to be utf8, and 
especially the primary/foreign key columns don't need to be utf8. And MariaDB 
supports having a column level specified character set, see 
https://mariadb.com/kb/en/setting-character-sets-and-collations/#column-level.

I'm going to try this out and see if this still fixes my utf8 encoding issue. 
It should. Only the following columns need to be utf8 because then store values 
that are generated externally:
* DATA_PRODUCT.PRODUCT_NAME
* DATA_PRODUCT.PRODUCT_DESCRIPTION
* DATA_REPLICA_LOCATION.REPLICA_NAME
* DATA_REPLICA_LOCATION.REPLICA_DESCRIPTION
* DATA_REPLICA_LOCATION.FILE_PATH

> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-06-22 Thread Marcus Christie (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736268#comment-17736268
 ] 

Marcus Christie commented on AIRAVATA-3697:
---

I'm using mariadb-tools to do an online conversion of a large instance of 
replica_catalog. First, I'm doing a dry run on a copy of the database:

{code}
MariaDB> create database replica_catalog_copy DEFAULT CHARACTER SET utf8 
COLLATE utf8_bin;
{code}

{code}
mysqldump replica_catalog > replica_catalog.sql
mysql replica_catalog_copy < replica_catalog.sql
{code}

Need to remove the foreign keys for this to work

{code}
ALTER TABLE DATA_PRODUCT DROP FOREIGN KEY DATA_PRODUCT_ibfk_1;
ALTER TABLE DATA_PRODUCT_METADATA DROP FOREIGN KEY DATA_PRODUCT_METADATA_ibfk_1;
ALTER TABLE DATA_REPLICA_LOCATION DROP FOREIGN KEY DATA_REPLICA_LOCATION_ibfk_1;
ALTER TABLE DATA_REPLICA_METADATA DROP FOREIGN KEY DATA_REPLICA_METADATA_ibfk_1;
{code}

Running mariadb-tools to do the incremental, online character set conversion
{code}
mariadb-schema-change --alter "CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin" 
D=replica_catalog_copy,t=DATA_PRODUCT --execute
mariadb-schema-change --alter "CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin" 
D=replica_catalog_copy,t=DATA_PRODUCT_METADATA --execute
mariadb-schema-change --alter "CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin" 
D=replica_catalog_copy,t=DATA_REPLICA_LOCATION --execute
mariadb-schema-change --alter "CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin" 
D=replica_catalog_copy,t=DATA_REPLICA_METADATA --execute
mariadb-schema-change --alter "CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin" 
D=replica_catalog_copy,t=CONFIGURATION --execute
{code}

After it's done, I'll add the FKs back
{code}
ALTER TABLE DATA_PRODUCT  ADD CONSTRAINT DATA_PRODUCT_ibfk_1 FOREIGN KEY 
(`PARENT_PRODUCT_URI`) REFERENCES `DATA_PRODUCT` (`PRODUCT_URI`) ON DELETE 
CASCADE;
ALTER TABLE DATA_PRODUCT_METADATA ADD CONSTRAINT `DATA_PRODUCT_METADATA_ibfk_1` 
FOREIGN KEY (`PRODUCT_URI`) REFERENCES `DATA_PRODUCT` (`PRODUCT_URI`) ON DELETE 
CASCADE;
ALTER TABLE DATA_REPLICA_LOCATION ADD CONSTRAINT `DATA_REPLICA_LOCATION_ibfk_1` 
FOREIGN KEY (`PRODUCT_URI`) REFERENCES `DATA_PRODUCT` (`PRODUCT_URI`) ON DELETE 
CASCADE;
ALTER TABLE DATA_REPLICA_METADATA ADD CONSTRAINT `DATA_REPLICA_METADATA_ibfk_1` 
FOREIGN KEY (`REPLICA_ID`) REFERENCES `DATA_REPLICA_LOCATION` (`REPLICA_ID`) ON 
DELETE CASCADE;
{code}



> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
>

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-06-05 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729502#comment-17729502
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit 538745201bd07a88ae3696aa59fa6d8ae4e81ceb in airavata's branch 
refs/heads/master from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata.git;h=538745201b ]

Merge branch 'AIRAVATA-3697' into develop


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return self.dispatch(request, *args,

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-06-05 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729501#comment-17729501
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit f4cc483734c6f9c674c26e5f6b9627c4d7e9fc96 in airavata's branch 
refs/heads/master from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata.git;h=f4cc483734 ]

AIRAVATA-3697 Set lang and locale on WSGIDaemonProcess

This is so that the Django portal code can handle filenames with UTF8 
characters.


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
>

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-06-05 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729504#comment-17729504
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit 8bf8ee00691c016d7357aba094fd43b093aebfe0 in airavata's branch 
refs/heads/master from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata.git;h=8bf8ee0069 ]

Merge branch 'AIRAVATA-3697' into develop


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return self.dispatch(request, *args,

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-06-05 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729503#comment-17729503
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit 33300327fc98d34ba5a62dbe75aea2678120a065 in airavata's branch 
refs/heads/master from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata.git;h=33300327fc ]

AIRAVATA-3697 Db migrations and changing PRODUCT_NAME, REPLICA_NAME to clobs


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-06-05 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729500#comment-17729500
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit a64264c10391df4bddeaa8663e18c3e508382dfc in airavata-django-portal's 
branch refs/heads/master from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata-django-portal.git;h=a64264c1 ]

AIRAVATA-3697 fix download of utf8 filename


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-06-05 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729496#comment-17729496
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit a64264c10391df4bddeaa8663e18c3e508382dfc in airavata-django-portal's 
branch refs/heads/staging from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata-django-portal.git;h=a64264c1 ]

AIRAVATA-3697 fix download of utf8 filename


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-06-05 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729495#comment-17729495
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit f29a74da5994561f6bec89df39db6a72f54128ab in airavata-django-portal's 
branch refs/heads/staging from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata-django-portal.git;h=f29a74da ]

AIRAVATA-3697 log error if filesystemencoding isn't 'utf-8'


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-06-05 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729499#comment-17729499
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit f29a74da5994561f6bec89df39db6a72f54128ab in airavata-django-portal's 
branch refs/heads/master from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata-django-portal.git;h=f29a74da ]

AIRAVATA-3697 log error if filesystemencoding isn't 'utf-8'


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-05-02 Thread Marcus Christie (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718701#comment-17718701
 ] 

Marcus Christie commented on AIRAVATA-3697:
---

This fix also fixes an issue with uploading files with unicode filenames to the 
tus upload handler that results in a UnicodeDecodeError. (the problem is that 
the tus .json file is decoded as 'ascii' instead of utf8, because that's the 
default file system encoding.)

> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-04-28 Thread Marcus Christie (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17717819#comment-17717819
 ] 

Marcus Christie commented on AIRAVATA-3697:
---

Uploaded file that was used in testing.

Also there is this experiment that reproduces the issue: 
https://django.seagrid.org/workspace/experiments/Gaussian16_on_Jun_3,_2022_4:36_PM_bbefc562-eb93-4de9-9aa7-1d0d379feed9/

Then go to *Exp Data Dir / ARCHIVE / ScrapeData / completed* in the 
experiment data directory.


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
> Attachments: Photophysical properties and photodegradation mechanism 
> of 2-(2′-furanyl)-1H-benzimidazole (Fuberidazole) - ScienceDirect.html
>
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-04-28 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17717817#comment-17717817
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit a64264c10391df4bddeaa8663e18c3e508382dfc in airavata-django-portal's 
branch refs/heads/develop from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata-django-portal.git;h=a64264c1 ]

AIRAVATA-3697 fix download of utf8 filename


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return self.dispatch(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 509, in

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-04-28 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17717816#comment-17717816
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit d0de482e134b2aac883d8c67da3a7481613ed97d in airavata-django-portal-sdk's 
branch refs/heads/master from Marcus Christie
[ 
https://gitbox.apache.org/repos/asf?p=airavata-django-portal-sdk.git;h=d0de482 ]

AIRAVATA-3697 Set filename appropriately when downloading file with utf8 
characters


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return self.dispatch(request, *args, **kwargs)
>  File 
>

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-04-28 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17717794#comment-17717794
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit f29a74da5994561f6bec89df39db6a72f54128ab in airavata-django-portal's 
branch refs/heads/develop from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata-django-portal.git;h=f29a74da ]

AIRAVATA-3697 log error if filesystemencoding isn't 'utf-8'


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return self.dispatch(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-04-27 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17717370#comment-17717370
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit 538745201bd07a88ae3696aa59fa6d8ae4e81ceb in airavata's branch 
refs/heads/develop from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata.git;h=538745201b ]

Merge branch 'AIRAVATA-3697' into develop


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return self.dispatch(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 509, in dispatch
>response =

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-04-27 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17717372#comment-17717372
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit 8bf8ee00691c016d7357aba094fd43b093aebfe0 in airavata's branch 
refs/heads/develop from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata.git;h=8bf8ee0069 ]

Merge branch 'AIRAVATA-3697' into develop


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return self.dispatch(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 509, in dispatch
>response =

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-04-27 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17717369#comment-17717369
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit f4cc483734c6f9c674c26e5f6b9627c4d7e9fc96 in airavata's branch 
refs/heads/develop from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata.git;h=f4cc483734 ]

AIRAVATA-3697 Set lang and locale on WSGIDaemonProcess

This is so that the Django portal code can handle filenames with UTF8 
characters.


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return self.dispatch(request, *args, **kwargs)
>  File 
>

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-04-27 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17717371#comment-17717371
 ] 

ASF subversion and git services commented on AIRAVATA-3697:
---

Commit 33300327fc98d34ba5a62dbe75aea2678120a065 in airavata's branch 
refs/heads/develop from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata.git;h=33300327fc ]

AIRAVATA-3697 Db migrations and changing PRODUCT_NAME, REPLICA_NAME to clobs


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return self.dispatch(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-04-27 Thread Marcus Christie (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17717363#comment-17717363
 ] 

Marcus Christie commented on AIRAVATA-3697:
---

Fixed the previous error by making PRODUCT_NAME and REPLICA_NAME CLOBs:

{code}
alter table DATA_REPLICA_LOCATION modify REPLICA_NAME text;
alter table DATA_PRODUCT modify PRODUCT_NAME text;
{code}


> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/generic/base.py",
>  line 70, in view
>return self.dispatch(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 509, in dispatch
>response = self.handle_exception(exc)
>  File 
>

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-04-27 Thread Marcus Christie (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17717360#comment-17717360
 ] 

Marcus Christie commented on AIRAVATA-3697:
---

I'm now getting this error
```
Caused by: org.apache.openjpa.persistence.PersistenceException: (conn:12787) 
Data too long for column 'REPLICA_NAME' at row 1 {prepstmnt 1166842203 
INSERT INTO DATA_REPLICA_LOCATION (REPLICA_ID, CREATION_TIME, 
FILE_PATH, LAST_MODIFIED_TIME, PRODUCT_URI, 
REPLICA_DESCRIPTION, REPLICA_LOCATION_CATEGORY, REPLICA_NAME, 
REPLICA_PERSISTENT_TYPE, STORAGE_RESOURCE_ID, VALID_UNTIL_TIME) 
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) 

```

I'll see if I can make the REPLICA_NAME column a CLOB.

> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in wrapped_view
>return view_func(*args, **kwargs)
>  File 
>

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

2023-04-27 Thread Marcus Christie (Jira)



[ 
https://issues.apache.org/jira/browse/AIRAVATA-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17717358#comment-17717358
 ] 

Marcus Christie commented on AIRAVATA-3697:
---

Ran the following in a development environment to test out converting the 
replica_catalog to utf8:

```
SET FOREIGN_KEY_CHECKS=0;
alter database replica_catalog CHARACTER SET = utf8 COLLATE = utf8_bin;
alter table DATA_PRODUCT convert to character set utf8 collate utf8_bin;
alter table DATA_REPLICA_LOCATION convert to character set utf8 collate 
utf8_bin;
alter table DATA_PRODUCT_METADATA convert to character set utf8 collate 
utf8_bin;
alter table DATA_REPLICA_METADATA convert to character set utf8 collate 
utf8_bin;
alter table CONFIGURATION convert to character set utf8 collate utf8_bin;
SET FOREIGN_KEY_CHECKS=1;
```

> Handle unicode characters in file names
> ---
>
> Key: AIRAVATA-3697
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3697
> Project: Airavata
>  Issue Type: Bug
>  Components: Airavata API, Django Portal
>Reporter: Marcus Christie
>Assignee: Marcus Christie
>Priority: Major
>
> Getting the following Django error when browsing experiment output files with 
> unicode characters in the file names:
> {code}
> Traceback (most recent call last):
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/rest_framework/views.py",
>  line 506, in dispatch
>response = handler(request, *args, **kwargs)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1467, in get
>return self._create_response(request, experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/airavata-django-portal/django_airavata/apps/api/views.py",
>  line 1471, in _create_response
>directories, files = user_storage.list_experiment_dir(request, 
> experiment_id, path)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 629, in list_experiment_dir
>backend=backend, owner=experiment.userName)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/airavata_django_portal_sdk/user_storage/api.py",
>  line 760, in _get_data_product_uri
>if user_file.exists():
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/query.py",
>  line 808, in exists
>return self.query.has_results(using=self.db)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py",
>  line 561, in has_results
>return compiler.has_results()
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1145, in has_results
>return bool(self.execute_sql(SINGLE))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>  line 1175, in execute_sql
>cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 66, in execute
>return self._execute_with_wrappers(sql, params, many=False, 
> executor=self._execute)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 75, in _execute_with_wrappers
>return executor(sql, params, many, context)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/utils.py",
>  line 84, in _execute
>return self.cursor.execute(sql, params)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/db/backends/mysql/base.py",
>  line 73, in execute
>return self.cursor.execute(query, args)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/cursors.py",
>  line 199, in execute
>args = tuple(map(db.literal, args))
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/MySQLdb/connections.py",
>  line 280, in literal
>s = self.string_literal(o.encode(self.encoding))
> During handling of the above exception ('utf-8' codec can't encode characters 
> in position 197-199: surrogates not allowed), another exception occurred:
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py",
>  line 47, in inner
>response = get_response(request)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/core/handlers/base.py",
>  line 181, in _get_response
>response = wrapped_callback(request, *callback_args, **callback_kwargs)
>  File 
> "/var/www/portals/django-seagrid/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
>  line 54, in

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

[jira] [Commented] (AIRAVATA-3697) Handle unicode characters in file names

30 matches

Site Navigation

Mail list logo

Footer information