Re: [SQLObject] bug in sqlobject-admin
Hi!
On Mon, Apr 26, 2010 at 09:13:01PM +0300, Imri Goldberg wrote:
> I am considering switching to your way of doing schema changes.
> I would very much appreciate more information regarding the process you use
> to do these things.
>
> For instance, given that I have a working MyThing class, to which I'd like
> to add a field my_field - would you write the upgrade script first, and then
> copy the field definitions to the MyThing class? Would you do it the other
> way around?
> Once both are updated, do you have a special order of doing things when
> moving it to production?
I use SQLObject in commercial programs that my company sells to
customers. So I have to distribute new class descriptions; so new tables
and columns are added to the code before upgrade scripts.
Also I write upgrade scripts in such a way they can be run safely many
times. When I announce new build in the internal mailing list and at the
corporation's wiki I notify support department there is a new upgrade
script. I don't expect support persons remember what upgrade scripts had
been run for a particular customer; so when installing a new version a
support person can run all upgrade scripts in order.
Because of this an upgrade script always looks into the real database to
see if it needs to do its job. For a new tables it's quite simple:
from Database.Requests import RequestIDs
if not RequestIDs.tableExists():
RequestIDs.createTable()
I.e., the script imports the class declaration from the Database modules,
tests if the table exists and creates the table.
For a new column it's a bit harder - I have to investigate the DB
deeper. Sometimes I simply create a second class declaration and connect it
to the same table:
class OldRequestIDs(Base):
class sqlmeta:
table = 'request_i_ds' # connect the table to the real table
status = StringCol(length=255, default=None)
OldRequestIDs.sqlmeta.delColumn("status", changeSchema=True)
Sometimes I draw the table declaration from the DB:
class Plate(SQLObject):
class sqlmeta:
fromDatabase = True
if 'idPl' not in Plate.sqlmeta.columns:
Plate.sqlmeta.addColumn(IntCol("IDpl", default=None, dbName="id_pl"),
changeSchema=True)
(Wow, three different ways of naming 'id_pl' column, damn!)
Oleg.
--
Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru
Programmers don't die, they just GOSUB without RETURN.
--
___
sqlobject-discuss mailing list
sqlobject-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
Re: [SQLObject] bug in sqlobject-admin
Hi Oleg!
I am considering switching to your way of doing schema changes.
I would very much appreciate more information regarding the process you use
to do these things.
For instance, given that I have a working MyThing class, to which I'd like
to add a field my_field - would you write the upgrade script first, and then
copy the field definitions to the MyThing class? Would you do it the other
way around?
Once both are updated, do you have a special order of doing things when
moving it to production?
Thanks ahead,
Imri
On Tue, Apr 20, 2010 at 2:45 PM, Oleg Broytman wrote:
> On Mon, Apr 19, 2010 at 04:11:09PM +0300, Imri Goldberg wrote:
> > Re using sqlobject-admin: I'm curious - what is your preferred way of
> > managing db migrations (i.e. versioning schema changes)?
>
>I seldom do a simple ADD/DELETE COLUMN. Usually when I change the schema
> it's more like "collect some info, ADD COLUMN, put info into the new
> column".
> Something like this (code from a real script upgradedb5.py):
>
>log.open("upgradedb5.log")
>
>if db_conn.dbName == 'sqlite':
>db_conn.use_table_info = True
>
>if 'userID' not in ExportResult.sqlmeta.columns:
>ExportResult.sqlmeta.addColumn(ForeignKey('User', name="user",
>default=None),
>changeSchema=True)
>
>for result in ExportResult.select(ExportResult.q.plate <> None):
>result.user = result.plate.user
>
>commit()
>
>log.close()
>
> This is a rather simple example. Sometime there is a lot of code to
> collect information before schema changing and even more code that puts the
> collected information into the new columns.
>
> Oleg.
> --
> Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru
> Programmers don't die, they just GOSUB without RETURN.
>
>
> --
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> ___
> sqlobject-discuss mailing list
> sqlobject-discuss@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
>
--
Imri Goldberg
--
http://plnnr.com/ - automatic trip planning
http://www.algorithm.co.il/blogs/
--
-- insert signature here
--
___
sqlobject-discuss mailing list
sqlobject-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
Re: [SQLObject] bug in sqlobject-admin
On Tue, Apr 20, 2010 at 09:35:12PM +0300, Imri Goldberg wrote: > On Tue, Apr 20, 2010 at 9:12 PM, Oleg Broytman wrote: > > BTW, DecimalCol is only recognized by SQLiteConnection. > > That's strange, as I currently use mysql. I haven't given it much thought, > but it seems to work well. If columnsFromSchema() couldn't recognize column it simply returns an instance of the generic Col class. Oleg. -- Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. -- ___ sqlobject-discuss mailing list sqlobject-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
Re: [SQLObject] bug in sqlobject-admin
On Tue, Apr 20, 2010 at 9:12 PM, Oleg Broytman wrote: > > BTW, DecimalCol is only recognized by SQLiteConnection. > > Oleg. > -- > Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru > Programmers don't die, they just GOSUB without RETURN. > That's strange, as I currently use mysql. I haven't given it much thought, but it seems to work well. -- Imri Goldberg -- http://plnnr.com/ - automatic trip planning http://www.algorithm.co.il/blogs/ -- -- insert signature here -- ___ sqlobject-discuss mailing list sqlobject-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
Re: [SQLObject] bug in sqlobject-admin
Excellent, thanks a bunch! Cheers, Imri PS, if there is some work to be done on sqlobject, I'll be happy to help :) On Tue, Apr 20, 2010 at 9:00 PM, Oleg Broytman wrote: > On Tue, Apr 20, 2010 at 08:23:39PM +0400, Oleg Broytman wrote: > >In this particular case it would be enough, I think, to extend > > columnsFromScheme to pass None's as size/precision. I will do some > > experiments before committing the hack. > >Fixed and committed in the revisions 4175-4177 (branches 0.11, 0.12 and > the trunk). Will be in the next release. > > Oleg. > -- > Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru > Programmers don't die, they just GOSUB without RETURN. > -- Imri Goldberg -- http://plnnr.com/ - automatic trip planning http://www.algorithm.co.il/blogs/ -- -- insert signature here -- ___ sqlobject-discuss mailing list sqlobject-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
Re: [SQLObject] bug in sqlobject-admin
On Tue, Apr 20, 2010 at 09:00:11PM +0300, Imri Goldberg wrote:
> When I checked it, it seemed to me to happen because the 'size' and
> 'precision' were pop()-ed from the original **kw dict passed to DecimalCol.
The problem was that the dictionary was simply empty.
SQLiteConnection.guessClass() did at line 364:
elif t.find('DECIMAL') >= 0:
return col.DecimalCol, {}
I changed that to
return col.DecimalCol, {'size': None, 'precision': None}
With the change your test works for me.
BTW, DecimalCol is only recognized by SQLiteConnection.
Oleg.
--
Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru
Programmers don't die, they just GOSUB without RETURN.
--
___
sqlobject-discuss mailing list
sqlobject-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
Re: [SQLObject] bug in sqlobject-admin
On Tue, Apr 20, 2010 at 08:23:39PM +0400, Oleg Broytman wrote: >In this particular case it would be enough, I think, to extend > columnsFromScheme to pass None's as size/precision. I will do some > experiments before committing the hack. Fixed and committed in the revisions 4175-4177 (branches 0.11, 0.12 and the trunk). Will be in the next release. Oleg. -- Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. -- ___ sqlobject-discuss mailing list sqlobject-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
Re: [SQLObject] bug in sqlobject-admin
Heya When I checked it, it seemed to me to happen because the 'size' and 'precision' were pop()-ed from the original **kw dict passed to DecimalCol. Since the this dict is later kept as a member, when used again by CommandStatus, 'size' and 'precision' are no longer there. Hope that helps, Imri On Tue, Apr 20, 2010 at 7:23 PM, Oleg Broytman wrote: > On Sat, Apr 17, 2010 at 01:25:59PM +0300, Imri Goldberg wrote: > > 1. Create a simple table with a decimal column, for example: > > > > class MyThing(sqlobject.SQLObject): > > bla1 = sqlobject.DecimalCol(size = 10, precision = 2) > > > > 2. Use sqlobject-admin create --egg=... -c sqlite:// > > 3. Use sqlobject-admin status --egg=... -c sqlite://... > [skip] > > File > > "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ > > sqlobject\manager\command.py", line 765, in command > > col = col.withClass(soClass) > > File > > "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ > > sqlobject\col.py", line 408, in withClass > > **self._kw) > > File > > "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ > > sqlobject\col.py", line 1329, in __init__ > > "You must give a size argument" > > AssertionError: You must give a size argument > > > > After researching this issue a bit, I still haven't figured out whether > this > > is the result of a bug in the command implementation, or the way that > > keyword arguments are handled in the column class hierarchy. > >Ok, found the cause. CommandStatus in sqlobject/manager/command.py list > columns using columnsFromScheme(), a kind of forced fromDatabase=True. And > most connection classes don't recognize size/precision in DecimalCol'umns. > In this particular case it would be enough, I think, to extend > columnsFromScheme to pass None's as size/precision. I will do some > experiments before committing the hack. > > Oleg. > -- > Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru > Programmers don't die, they just GOSUB without RETURN. > -- Imri Goldberg -- http://plnnr.com/ - automatic trip planning http://www.algorithm.co.il/blogs/ -- -- insert signature here -- ___ sqlobject-discuss mailing list sqlobject-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
Re: [SQLObject] bug in sqlobject-admin
On Sat, Apr 17, 2010 at 01:25:59PM +0300, Imri Goldberg wrote: > 1. Create a simple table with a decimal column, for example: > > class MyThing(sqlobject.SQLObject): > bla1 = sqlobject.DecimalCol(size = 10, precision = 2) > > 2. Use sqlobject-admin create --egg=... -c sqlite:// > 3. Use sqlobject-admin status --egg=... -c sqlite://... [skip] > File > "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ > sqlobject\manager\command.py", line 765, in command > col = col.withClass(soClass) > File > "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ > sqlobject\col.py", line 408, in withClass > **self._kw) > File > "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ > sqlobject\col.py", line 1329, in __init__ > "You must give a size argument" > AssertionError: You must give a size argument > > After researching this issue a bit, I still haven't figured out whether this > is the result of a bug in the command implementation, or the way that > keyword arguments are handled in the column class hierarchy. Ok, found the cause. CommandStatus in sqlobject/manager/command.py list columns using columnsFromScheme(), a kind of forced fromDatabase=True. And most connection classes don't recognize size/precision in DecimalCol'umns. In this particular case it would be enough, I think, to extend columnsFromScheme to pass None's as size/precision. I will do some experiments before committing the hack. Oleg. -- Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ sqlobject-discuss mailing list sqlobject-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
Re: [SQLObject] bug in sqlobject-admin
On Mon, Apr 19, 2010 at 04:11:09PM +0300, Imri Goldberg wrote:
> Re using sqlobject-admin: I'm curious - what is your preferred way of
> managing db migrations (i.e. versioning schema changes)?
I seldom do a simple ADD/DELETE COLUMN. Usually when I change the schema
it's more like "collect some info, ADD COLUMN, put info into the new column".
Something like this (code from a real script upgradedb5.py):
log.open("upgradedb5.log")
if db_conn.dbName == 'sqlite':
db_conn.use_table_info = True
if 'userID' not in ExportResult.sqlmeta.columns:
ExportResult.sqlmeta.addColumn(ForeignKey('User', name="user",
default=None),
changeSchema=True)
for result in ExportResult.select(ExportResult.q.plate <> None):
result.user = result.plate.user
commit()
log.close()
This is a rather simple example. Sometime there is a lot of code to
collect information before schema changing and even more code that puts the
collected information into the new columns.
Oleg.
--
Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru
Programmers don't die, they just GOSUB without RETURN.
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
sqlobject-discuss mailing list
sqlobject-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
Re: [SQLObject] bug in sqlobject-admin
Heya I'll be happy to help you work it, and also to solve the bug. Re using sqlobject-admin: I'm curious - what is your preferred way of managing db migrations (i.e. versioning schema changes)? Cheers, Imri On Mon, Apr 19, 2010 at 12:58 PM, Oleg Broytman wrote: > On Sat, Apr 17, 2010 at 01:25:59PM +0300, Imri Goldberg wrote: > > I've worked a bit with sqlobject-admin, and there's a bug in the > > implementation of the status command. > >Thank you for reporting. I have never used sqlobject-admin so I have to > learn how to use it to reproduce the bug. > > Oleg. > -- > Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru > Programmers don't die, they just GOSUB without RETURN. > > > -- > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > ___ > sqlobject-discuss mailing list > sqlobject-discuss@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss > -- Imri Goldberg -- http://plnnr.com/ - automatic trip planning http://www.algorithm.co.il/blogs/ -- -- insert signature here -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ sqlobject-discuss mailing list sqlobject-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
Re: [SQLObject] bug in sqlobject-admin
On Sat, Apr 17, 2010 at 01:25:59PM +0300, Imri Goldberg wrote: > I've worked a bit with sqlobject-admin, and there's a bug in the > implementation of the status command. Thank you for reporting. I have never used sqlobject-admin so I have to learn how to use it to reproduce the bug. Oleg. -- Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ sqlobject-discuss mailing list sqlobject-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
[SQLObject] bug in sqlobject-admin
Heya I've worked a bit with sqlobject-admin, and there's a bug in the implementation of the status command. Steps to reproduce: 1. Create a simple table with a decimal column, for example: class MyThing(sqlobject.SQLObject): bla1 = sqlobject.DecimalCol(size = 10, precision = 2) 2. Use sqlobject-admin create --egg=... -c sqlite:// 3. Use sqlobject-admin status --egg=... -c sqlite://... You should get the following exception: [snip] File "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ sqlobject\manager\command.py", line 765, in command col = col.withClass(soClass) File "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ sqlobject\col.py", line 408, in withClass **self._kw) File "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ sqlobject\col.py", line 1329, in __init__ "You must give a size argument" AssertionError: You must give a size argument After researching this issue a bit, I still haven't figured out whether this is the result of a bug in the command implementation, or the way that keyword arguments are handled in the column class hierarchy. Cheers, Imri -- Imri Goldberg -- http://plnnr.com/ - automatic trip planning http://www.algorithm.co.il/blogs/ -- -- insert signature here -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ sqlobject-discuss mailing list sqlobject-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
