[web2py] Old DAL bug (?) ValueError: invalid literal for int() with base 10: 'AssetTypes\\Character.png'

[Arnon Marcus]

We are using an old version of web2py in production (1.89.5), and are 
having issues with the DAL:

    if limit: return 
db().select(db[table].ALL,orderby=db[table][orderby],limitby=(0,limit))
  File "C:\wamp\www\web2py_1_89_5\gluon\sql.py", line 3307, in select
    return self.parse(db, rows, self.colnames, SetClass=Set)
  File "C:\wamp\www\web2py_1_89_5\gluon\sql.py", line 3353, in parse
    colset[fieldname] = rid = Reference(value)
ValueError: invalid literal for int() with base 10: 'AssetTypes\\Character.png'


The relevant section we pinpointed is this:

    @staticmethod
    def parse(db, rows, colnames, blob_decode=True, SetClass=None):
         .......
                    if not '.' in referee:
*                        colset[fieldname] = rid = Reference(value)*
                        (rid._table, rid._record) = (db[referee], None)
                    else: ### reference not by id
                        colset[fieldname] = value
                elif field_type == 'boolean':


We suspect that what is happening is a shift in the field-names list, that 
is causing the DAL to treat a string field as an id field.
We have some "suspected culprits" for thinking this - we suspect that there 
might be a discrepency between the order/length of the data-fields being 
returned, and that order/length in the field-names being passes,  which 
leads us back to suspect that something might be wrong with the "colnames" 
being passes in to "parse" from "select", but that is being set in "_select"
:

    def *_select*(self, **fields*, **attributes):
       ......
        # ## if not fields specified take them all from the requested tables

        if not fields:
            fields = [self._db[table].ALL for table in self._tables]
        *sql_f* = ', '.join([str(f) for f in *fields*])
        tablenames = parse_tablenames(self.sql_w + ' ' + sql_f)
        if len(tablenames) < 1:
            raise SyntaxError, 'Set: no tables selected'
        w2p_tablenames = [ t for t in tablenames if 
isinstance(self._db[t],Table) ]
        *self.colnames* = [c.strip() for c in *sql_f*.split(', ')]
        if self.sql_w:
            sql_w = ' WHERE ' + self.sql_w
       ....


    def *select*(self, **fields*, **attributes):
        """
        Always returns a Rows object, even if it may be empty
        """    
     ...
        return self.*parse*(db, rows, *self.colnames*, SetClass=Set)

    @staticmethod
    def *parse*(db, *rows*, *colnames*, blob_decode=True, SetClass=None):
        virtualtables = []
        new_rows = []
        for (i,*row*) in enumerate(*rows*):
            new_row = Row()
            for* j*,colname in enumerate(*colnames*):
                *value *= *row*[*j*]



I checked to see what has been changed regarding this in the current 
version:


    def *_select*(self, query, *fields*, attributes):
        .....
        if len(tablenames) < 1:
            raise SyntaxError('Set: no tables selected')
        *sql_f* = *', '.join(map(self.expand, fields))*
        *self._**c**olnames* = [c.strip() for c in *sql_f*.split(', ')]
        if query:
            sql_w = ' WHERE ' + self.expand(query)
....
    def *_select_aux*(self,sql,*fields*,attributes):
       ....
*        return processor(rows,fields,self._colnames,cacheable=cacheable)*

    def *select*(self, query, fields, attributes):
        """
        Always returns a Rows object, possibly empty.
       .....
            return self.*_select_aux*(sql,*fields*,attributes)

...

    def parse(self, rows, *fields*, *colnames*, blob_decode=True,
               .....
                    *value* = *self.parse_value(value,ft,blob_decode)*
                    if field.filter_out:
                        value = field.filter_out(value)
*                    colset[fieldname] = value*


It seems a much "safer" implementation.

Now, we have scheduled an upgrade of web2py in our road-map, but that will 
take some time...
In the mean-time, I was wondering weather we could use some/all of the 
new*dal.py + sql.py 
*to replace our current web2py's ones.
I was wondering if that would be safe, or if we could get into unforeseen 
trouble 
doing that...
I had seen Massimo saying that the* dal.py* is now *web-framework agnostic*
The question is weather the "old" one is also that... In a way, does this 
mean that dal.py versions are inter-changeable across web2py versions, at 
least in this case...

What do you say? 
 

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.