I have written a script to parse a large xml file and insert the contents
in to my app db. I am using lxml.
After about 10 records get inserted, the script fails with
<type 'exceptions.TypeError'> argument of type 'NoneType' is not iterable
Troubleshooting determined the following:
1. This error is associated with 3 list:reference fields. When I remove
them from the script, the script executes uneventfully. If any one of them
is included, it fails.
2. This only happens after 10 records have been successfully inserted.
3. There is no discernible difference between the records that get
successfully added and those that don't. The error happens even when I hard
code the lists for the list:reference field. It seems to be associated with
number of records, rather than which records.
4. The script executes successfully when I change the field types from
'list:reference' to 'string' and insert strings instead of lists. You
should not assume from this that there is a data issue. As I said,
hardcoded lists get rejected also. I am 99% certain valid data is not the
issue.
5. This happens in both SQLLite and Postgres
Here is the model declaration for one of the three fields. They are all
analogous:
Field('genres','string','list:reference genre', requires=IS_IN_DB(db,
'genre.id', '%(name)s [%(id)s]', multiple=True))
Here is how I update each new row in the the database:
db.movies.validate_and_insert(**movie) (movie is a dict)
Here is how I hardcoded values into the fields: movie['genre'] = {456, 368,
239]
Now, if someone doesn't have a solution, can they tell me if I can
1.Programmatically remove the list:reference from the model prior to data
updates and programmatically restore it afterwards?
2. Retain all the functionality of these fields by toggling this way?
Seriously considering going the join table route and skipping the list
reference fields. Are there any gotchas there?
--
