On 10/19/10 10:33 AM, Jacek Kałucki wrote:
> Did your bizobj.Encoding is differ than dabo.defaultEncoding?
> And what encoding are you using?
> Please, change line 376 to:
>
> 376 except UnicodeError, ex:
> 377 dabo.log.error("%r" % ex.message)
>
> and give me log output.
I don't know what the customer's system encoding is set to, but let's assume
'windows-1252'. I've gotten a copy of his database and am able to reproduce the
exact
error on my Mac system, and I've put the following code in dCursorMixin (still
prefer
print statements to log output):
369 try:
370 _records = self.fetchall()
371 except Exception, e:
372 _records = dabo.db.dDataSet()
373 # Database errors need to be decoded from database encoding.
374 print sql
375 print params
376 print self.Encoding
377 print dabo.defaultEncoding
378 print e.message
379 try:
380 errMsg = ustr(e).decode(self.Encoding)
381 except UnicodeError:
382 errMsg = unicode(e)
383 dabo.log.error("Error fetching records: %s" % errMsg)
When I run I get this output:
mac:ss pmcnett$ python sbs_studio.py
Begin update 0009
Will process 87 Production Orders record(s)
select production_order_openings.id as id,
production_order_openings.order_id as order_id,
production_order_openings.material_type_id as
material_type_id,
production_order_openings.shape_id as shape_id,
production_order_openings.louver_size_id as
louver_size_id,
production_order_openings.louver_id as louver_id,
production_order_openings.color_id as color_id,
production_order_openings.line_number as line_number,
production_order_openings.room as room,
production_order_openings.fold_type_id as fold_type_id,
production_order_openings.formula_mode_id as
formula_mode_id,
production_order_openings.mount as mount,
production_order_openings.hole_strip_id as hole_strip_id,
production_order_openings.tilt_rod_id as tilt_rod_id,
production_order_openings.qty as qty,
production_order_openings.centered_tpost_mode as
centered_tpost_mode,
production_order_openings.fold_left as fold_left,
production_order_openings.fold_right as fold_right,
production_order_openings.explicit_number_panels as
explicit_number_panels,
production_order_openings.net_width as net_width,
production_order_openings.net_height as net_height,
production_order_openings.net_width_closing_tolerance as
net_width_closing_tolerance,
production_order_openings.net_height_closing_tolerance as
net_height_closing_tolerance,
production_order_openings.win_width as win_width,
production_order_openings.win_height as win_height,
production_order_openings.framed_width as framed_width,
production_order_openings.framed_height as framed_height,
production_order_openings.given_width as given_width,
production_order_openings.given_height as given_height,
production_order_openings.unit_of_measure as
unit_of_measure,
production_order_openings.price as price,
production_order_openings.override_price as
override_price,
production_order_openings.internal_notes as
internal_notes,
production_order_openings.special_instructions as
special_instructions,
production_order_openings.all_locked as all_locked,
production_order_material_types.name as material_name,
colors.name as color_name,
colors.code as color_code,
production_order_openings.calc_fold_string as
calc_fold_string,
production_order_openings.calc_number_panels as
calc_number_panels,
production_order_openings.calc_number_panels_string as
calc_number_panels_string,
production_order_openings.calc_dr_pos_string as
calc_dr_pos_string,
production_order_openings.calc_square_feet as
calc_square_feet
from production_order_openings
left join production_order_material_types on
production_order_material_types.id =
production_order_openings.material_type_id
left join colors on colors.id = production_order_openings.color_id
where production_order_openings.order_id = ?
group by production_order_openings.id
order by production_order_openings.line_number
limit 1000
(u'c098e73d-aa1c-4a47-ba8b-caadb15c54c4',)
UTF8
utf-8
/home/pmcnett/dabo-full/trunk/dabo/db/dCursorMixin.py:378: DeprecationWarning:
BaseException.message has been deprecated as of Python 2.6
print e.message
Could not decode to UTF-8 column 'special_instructions' with text 'Route for
Crank 6
?? to 12? x 2 ?? high x ?? deep, Handle 13 ?? to 15? x 1? wide x ?? deep'
/home/pmcnett/dabo-full/trunk/dabo/lib/utils.py:175: DeprecationWarning:
BaseException.message has been deprecated as of Python 2.6
if hasattr(e, "message"):
/home/pmcnett/dabo-full/trunk/dabo/lib/utils.py:176: DeprecationWarning:
BaseException.message has been deprecated as of Python 2.6
return ustr(e.message)
Traceback (most recent call last):
File "sbs_studio.py", line 166, in <module>
main()
File "sbs_studio.py", line 96, in main
open_success = app.openDatabase(db_file)
File
"/Users/pmcnett/py/sbs/shutter_studio/trunk/clients/shutter_studio/App.py",
line 1152, in openDatabase
self.applyDbUpdates()
File
"/Users/pmcnett/py/sbs/shutter_studio/trunk/clients/shutter_studio/App.py",
line 1200, in applyDbUpdates
doUpdate(needed_update)
File
"/Users/pmcnett/py/sbs/shutter_studio/trunk/clients/shutter_studio/App.py",
line 1191, in doUpdate
upd_obj.process()
File
"/Users/pmcnett/py/sbs/shutter_studio/trunk/clients/shutter_studio/db/updates/base.py",
line 27, in process
self.do_process()
File
"/Users/pmcnett/py/sbs/shutter_studio/trunk/clients/shutter_studio/db/updates/u0009.py",
line 38, in do_process
biz.moveToRowNumber(rownum)
File "/home/pmcnett/dabo-full/trunk/dabo/biz/dBizobj.py", line 1837, in
moveToRowNumber
self.RowNumber = rowNumber
File "/home/pmcnett/dabo-full/trunk/dabo/biz/dBizobj.py", line 2365, in
_setRowNumber
self.requeryAllChildren()
File "/home/pmcnett/dabo-full/trunk/dabo/biz/dBizobj.py", line 1529, in
requeryAllChildren
child.requery()
File "/home/pmcnett/dabo-full/trunk/dabo/biz/dBizobj.py", line 988, in
requery
cursor.requery(params)
File "/home/pmcnett/dabo-full/trunk/dabo/db/dCursorMixin.py", line 465, in
requery
self.execute(currSQL, params, _newQuery=newQuery)
File "/home/pmcnett/dabo-full/trunk/dabo/db/dCursorMixin.py", line 382, in
execute
errMsg = unicode(e)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xbd in position 85:
ordinal not
in range(128)
So, the relevant portion of that output seems to be the e.message:
{{{
Could not decode to UTF-8 column 'special_instructions' with text 'Route for
Crank 6
?? to 12? x 2 ?? high x ?? deep, Handle 13 ?? to 15? x 1? wide x ?? deep'
}}}
Now, I tried Jeff's suggestion of setting conn.text_factory = str (in
dabo/db/dbSQLite.py in the getConnection() method)
And, the update continued but presented this message at the console instead of
the
traceback:
{{{
2010-10-19 10:48:20 - ERROR - Field special_instructions: Incorrect unicode
encoding
set; using 'latin-1' instead
}}}
So now that I could get into the application, I went to the record number in
question
(I found it easily because the above message was reprinted with new timestamps
every
time I visited it). The special_instructions field in question is in a child
record
of the main bizobj, and on my Mac it reads:
{{{
Route for Crank 6 1/2" to 12" x 2 1/2" high x 1/2"deep, Handle 13 1/2" to 15" x
1"
wude x 1/2" deep
}}}
I don't understand why this wouldn't have decoded from UTF-8, but would decode
from
latin-1. I thought UTF-8 was a superset of latin-1.
I also don't know what the possible implications of setting text_factory are,
but
thanks Jeff because at least I can send this database back to the customer now.
I think the user scenario was probably that they used the character picker to
copy/paste single and double ticks, and fractional unicode characters, so Dabo
needs
a decode/encode cycle at that IO interface I guess before it gets saved to the
db.
Does that sound right?
Paul
_______________________________________________
Post Messages to: [email protected]
Subscription Maintenance: http://leafe.com/mailman/listinfo/dabo-dev
Searchable Archives: http://leafe.com/archives/search/dabo-dev
This message: http://leafe.com/archives/byMID/[email protected]
