Thanks - patch applied! On Wed, Jun 20, 2018 at 3:17 PM, Aditya Toshniwal < aditya.toshni...@enterprisedb.com> wrote:
> Hi Dave, > > Attached is the updated patch. (Playing with encodings is not at all fun > :( ) > > On Tue, Jun 19, 2018 at 2:23 AM, Dave Page <dp...@pgadmin.org> wrote: > >> Hi >> >> On Mon, Jun 18, 2018 at 2:14 PM, Aditya Toshniwal < >> aditya.toshni...@enterprisedb.com> wrote: >> >>> Hi Hackers, >>> >>> Attached is the updated patch which includes the fix for Download CSV >>> fail in SQL_ASCII database, which is RM3250 >>> <https://redmine.postgresql.org/issues/3250> >>> This should fix RM3289 and RM3250. As they interrelated, sending the >>> combined patch. >>> Kindly review. >>> >> >> This is definitely looking better - both view and save now work as >> expected. However, using the test data the I posted upthread, if I try to >> edit a value (in this case by adding a couple of chars to the end of the >> data in row 2) I get: >> > It should fix the error. > >> >> 2018-06-18 16:41:40,895: SQL pgadmin: Execute (void) for server #1 - >> DB:ascii (Query-id: 3093186): >> UPDATE public.ascii SET >> data = %(data)s::text WHERE >> id = '2'; >> 2018-06-18 16:41:41,027: INFO werkzeug: 127.0.0.1 - - [18/Jun/2018 >> 16:41:41] "POST /sqleditor/save/2805058 HTTP/1.1" 500 - >> 2018-06-18 16:41:41,042: ERROR werkzeug: Error on request: >> Traceback (most recent call last): >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py", >> line 270, in run_wsgi >> execute(self.server.app) >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py", >> line 258, in execute >> application_iter = app(environ, start_response) >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py", >> line 1997, in __call__ >> return self.wsgi_app(environ, start_response) >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py", >> line 1985, in wsgi_app >> response = self.handle_exception(e) >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py", >> line 1540, in handle_exception >> reraise(exc_type, exc_value, tb) >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py", >> line 1982, in wsgi_app >> response = self.full_dispatch_request() >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py", >> line 1614, in full_dispatch_request >> rv = self.handle_user_exception(e) >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py", >> line 1517, in handle_user_exception >> reraise(exc_type, exc_value, tb) >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py", >> line 1612, in full_dispatch_request >> rv = self.dispatch_request() >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py", >> line 1598, in dispatch_request >> return self.view_functions[rule.endpoint](**req.view_args) >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_login.py", >> line 792, in decorated_view >> return func(*args, **kwargs) >> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py", >> line 776, in save >> default_conn) >> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/sqleditor/command.py", >> line 829, in save >> item['sql'], item['data']) >> File >> "/Users/dpage/git/pgadmin4/web/pgadmin/utils/driver/psycopg2/connection.py", >> line 975, in execute_void >> self.__internal_blocking_execute(cur, query, params) >> File >> "/Users/dpage/git/pgadmin4/web/pgadmin/utils/driver/psycopg2/connection.py", >> line 629, in __internal_blocking_execute >> cur.execute(query, params) >> File >> "/Users/dpage/git/pgadmin4/web/pgadmin/utils/driver/psycopg2/cursor.py", >> line 176, in execute >> return _cursor.execute(self, query, params) >> UnicodeEncodeError: 'ascii' codec can't encode characters in position >> 19-21: ordinal not in range(128) >> >> >>> >>> On Fri, Jun 15, 2018 at 2:33 PM, Aditya Toshniwal < >>> aditya.toshni...@enterprisedb.com> wrote: >>> >>>> Hi Dave, >>>> >>>> On Fri, Jun 15, 2018 at 2:08 PM, Dave Page <dp...@pgadmin.org> wrote: >>>> >>>>> Hi >>>>> >>>>> On Thu, Jun 14, 2018 at 7:05 AM, Aditya Toshniwal < >>>>> aditya.toshni...@enterprisedb.com> wrote: >>>>> >>>>>> I am sorry I missed the attachment. :( >>>>>> PFA. >>>>>> >>>>> >>>>> It looks like the encoding names are getting munged somewhere. I see >>>>> you've accounted for that to some degree in connection.py (you have both >>>>> SQL_ASCII/MULE_INTERNAL and SQLASCII/MULEINTERNAL), however it doesn't >>>>> seem >>>>> to be enough as I'm getting the following error when trying to download >>>>> CSV >>>>> from the query tool. Can we ensure that conn.encoding contains an >>>>> un-munged >>>>> value at all times, or is that coming from psycopg2? >>>>> >>>> That is done by pyscopg2 and conn.encoding is a psycopg2 connection >>>> property. >>>> >>>> >>>>> >>>>> 2018-06-15 09:32:28,799: INFO werkzeug: 127.0.0.1 - - [15/Jun/2018 >>>>> 09:32:28] "GET /sqleditor/query_tool/download >>>>> /2732923?query=SELECT%20*%20FROM%20public.sql_ascii%0AORDER% >>>>> 20BY%20id%20ASC%20&filename=sql_ascii.csv HTTP/1.1" 500 - >>>>> 2018-06-15 09:32:28,801: ERROR werkzeug: Error on request: >>>>> Traceback (most recent call last): >>>>> File >>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py", >>>>> line 270, in run_wsgi >>>>> execute(self.server.app) >>>>> File >>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py", >>>>> line 260, in execute >>>>> for data in application_iter: >>>>> File >>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/wsgi.py", >>>>> line 870, in __next__ >>>>> return self._next() >>>>> File >>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/wrappers.py", >>>>> line 82, in _iter_encoded >>>>> for item in iterable: >>>>> File >>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/utils/driver/psycopg2/connection.py", >>>>> line 752, in gen >>>>> column_name = column_name.decode(conn_encoding) >>>>> LookupError: unknown encoding: SQLASCII >>>>> >>>> >>>> This is because there is code bug here. Below is code used to decode a >>>> column name. Connection encoding and python encoding are two different >>>> things. Python does not know what SQLASCII is. This will work with UTF-8 >>>> because python has decoder with same name. I tried to download CSV with the >>>> original code without changes and it fails there too. I will fix this and >>>> will send the updated patch. I should have checked this. >>>> conn_encoding = cur.connection.encoding >>>> column_name = column_name.decode(conn_encoding) >>>> >>>> >>>>> >>>>> -- >>>>> Dave Page >>>>> Blog: http://pgsnake.blogspot.com >>>>> Twitter: @pgsnake >>>>> >>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>> The Enterprise PostgreSQL Company >>>>> >>>> >>>> >>>> >>>> -- >>>> Thanks and Regards, >>>> Aditya Toshniwal >>>> Software Engineer | EnterpriseDB Software Solutions | Pune >>>> "Don't Complain about Heat, Plant a tree" >>>> >>> >>> >>> >>> -- >>> Thanks and Regards, >>> Aditya Toshniwal >>> Software Engineer | EnterpriseDB Software Solutions | Pune >>> "Don't Complain about Heat, Plant a tree" >>> >> >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> > > > > -- > Thanks and Regards, > Aditya Toshniwal > Software Engineer | EnterpriseDB Software Solutions | Pune > "Don't Complain about Heat, Plant a tree" > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
