Thank you Mathieu, I found the place to create issue... Actually, I wanted to better qualify the issue before open it, restarting from a clean project, and I cannot reproduce the very first issue! I encounter some other issues related to binary encoding but now I prefer triple check before speaking!
Le lundi 2 avril 2018 16:36:59 UTC+2, Mathieu Clabaut a écrit : > > Alexandre, > > just go to : https://github.com/web2py/web2py/issues/new > > And fill in the issue withe the information given in your mail. > > Le ven. 30 mars 2018 à 16:50, <[email protected] <javascript:>> a > écrit : > >> Sure! Let me see how to do that and I will. Thank you >> >> >> Le mercredi 28 mars 2018 18:49:08 UTC+2, Massimo Di Pierro a écrit : >> >>> I think this is. Can you please open an issue about it? >>> >>> On Tuesday, 27 March 2018 13:32:15 UTC-5, [email protected] wrote: >>>> >>>> Hello All, >>>> >>>> I'm an absolute newbie with web2py so I'm not sure if the issue lies in >>>> web2py/3.6 or my own mistakes... My apologies in advance. >>>> >>>> I encounter this error in a db.my_table.update_or_insert(..) >>>> >>>> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfe in position >>>> 96328: invalid start byte' >>>> >>>> >>>> What I'm trying to do: >>>> I have a text file that is the representation of a datamodel in a >>>> specific COTS format. >>>> I want to parse it, list the tables to be created and create them >>>> actually. >>>> I know that the file countains non-utf-8 characters, that's why I open >>>> it in binary: >>>> >>>> dpm_file = gzip.open(filepath,'rb') >>>> >>>> ... parse it using binary regex: (I imported the 'regex' module) >>>> regexp = regex.compile(b':columns >>>> (\(((?>[^()]+)|(?1))*\))',regex.DOTALL) >>>> header_classes = regexp.findall(one_format_text)[0] >>>> >>>> ... and insert the results in db blob fields: (the error occurs here) >>>> db.dpm_header.update_or_insert(db.dpm_header.dpm == my_dpm_id, dpm >>>> = my_dpm_id, header_classes = header_classes, header_objects = >>>> header_objects) >>>> >>>> I defined blob fields because I thought that it was the way to write >>>> directly in binary in database. >>>> I really wanted to use binary preciselly to avoid to manage the >>>> encoding/decoding issues. >>>> >>>> In addition, this project is an attempt to transport in web2py an older >>>> project that was working perfectly in pure python 3.6.3 (unix and windows) >>>> (except that instead of creating tables in a DB, I created text files in a >>>> filer...). >>>> Is it a issue on web2py/python 3.6? Or my misunderstanding on database >>>> management... >>>> >>>> Thank you for your help! >>>> >>>> Here is the trace >>>> >>>> Traceback (most recent call last): >>>> File "D:\web\zz\web2py\gluon\restricted.py", line 219, in restricted >>>> exec(ccode, environment) >>>> File "D:/web/zz/web2py/applications/dpm_manager/controllers/default.py" >>>> <http://127.0.0.1:8000/admin/default/edit/dpm_manager/controllers/default.py>, >>>> line 175, in <module> >>>> File "D:\web\zz\web2py\gluon\globals.py", line 419, in <lambda> >>>> self._caller = lambda f: f() >>>> File "D:\web\zz\web2py\gluon\tools.py", line 3982, in f >>>> return action(*a, **b) >>>> File "D:/web/zz/web2py/applications/dpm_manager/controllers/default.py" >>>> <http://127.0.0.1:8000/admin/default/edit/dpm_manager/controllers/default.py>, >>>> line 131, in analyze >>>> header_classes = header_classes) >>>> File "D:\web\zz\web2py\gluon\packages\dal\pydal\objects.py", line 811, >>>> in update_or_insert >>>> record = self(_key) >>>> File "D:\web\zz\web2py\gluon\packages\dal\pydal\objects.py", line 565, >>>> in __call__ >>>> orderby_on_limitby=False).first() >>>> File "D:\web\zz\web2py\gluon\packages\dal\pydal\objects.py", line 2250, >>>> in select >>>> return adapter.select(self.query, fields, attributes) >>>> File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\sqlite.py", >>>> line 82, in select >>>> return super(SQLite, self).select(query, fields, attributes) >>>> File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line >>>> 762, in select >>>> return self._select_aux(sql, fields, attributes, colnames) >>>> File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line >>>> 741, in _select_aux >>>> return processor(rows, fields, colnames, cacheable=cacheable) >>>> File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line >>>> 305, in parse >>>> for row in rows >>>> File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line >>>> 302, in <listcomp> >>>> self._parse( >>>> File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line >>>> 231, in _parse >>>> value = self.parse_value(value, fit, ft, blob_decode) >>>> File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line >>>> 198, in parse_value >>>> return self.parser.parse(value, field_itype, field_type) >>>> File "D:\web\zz\web2py\gluon\packages\dal\pydal\parsers\__init__.py", >>>> line 101, in parse >>>> return self.registered[field_itype](value, field_type) >>>> File "D:\web\zz\web2py\gluon\packages\dal\pydal\parsers\__init__.py", >>>> line 76, in __call__ >>>> return self.call(value, field_type) >>>> File "D:\web\zz\web2py\gluon\packages\dal\pydal\parsers\__init__.py", >>>> line 73, in _call >>>> return self.f(self.parser, value) >>>> File "D:\web\zz\web2py\gluon\packages\dal\pydal\parsers\base.py", line >>>> 37, in _blob >>>> return to_native(b64decode(to_bytes(value))) >>>> File "D:\web\zz\web2py\gluon\packages\dal\pydal\_compat.py", line 82, in >>>> to_native >>>> return obj.decode(charset, errors) >>>> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfe in position >>>> 96328: invalid start byte >>>> >>>> -- >> Resources: >> - http://web2py.com >> - http://web2py.com/book (Documentation) >> - http://github.com/web2py/web2py (Source code) >> - https://code.google.com/p/web2py/issues/list (Report Issues) >> --- >> You received this message because you are subscribed to the Google Groups >> "web2py-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
