Re: Bulk Creating Django Models from CSV File?

[Thomas Weholt]

You might want to look at DSE as well. Currently is has some issues
with Django 1.4, but I'll look into fixing those tonight.

https://bitbucket.org/weholt/dse2

Regards,
Thomas Weholt

On Mon, Apr 2, 2012 at 7:24 AM, Victor Hooi  wrote:
> Hi,
>
> I have several large CSV files that I'm hoping to parse, and use to create
> Django objects. Each line looks might look something like this:
>
>> "Server Hostname", "Classification", "Country", "Operating System"
>>
>> "foo.bar.com", "Prod", "Australia", "Solaris"
>>
>> "alex.john.com", "Dev", "UK", "Linux"
>>
>> "bob.hope.com", "UAT, "Japan", "Windows
>
>
> For performance reasons, it would be nice to use something like 1.4's
> bulk_create:
>
> https://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.models.query.QuerySet.bulk_create
>
> However, I noticed the caveat about it not populating the auto-incrementing
> PK field. What's the recommended way to populate the PK field then? How do
> you use it?
>
> Secondly, the models will have FK relationships to other models (for
> instance, Classification, Country and Operating System would be stored in a
> lookup table). What's the best way to look these up, and assign the
> appropriate FK, whilst still preserving good performance and using bulk
> inserts?
>
> Cheers,
> Victor
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-users/-/bP0RG4NXJjEJ.
> To post to this group, send email to django-users@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.



-- 
Mvh/Best regards,
Thomas Weholt
http://www.weholt.org

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Bulk Creating Django Models from CSV File?

[Victor Hooi]

Hi,

I have several large CSV files that I'm hoping to parse, and use to create 
Django objects. Each line looks might look something like this:

"Server Hostname", "Classification", "Country", "Operating System"

"foo.bar.com", "Prod", "Australia", "Solaris"

"alex.john.com", "Dev", "UK", "Linux"

"bob.hope.com", "UAT, "Japan", "Windows


For performance reasons, it would be nice to use something like 1.4's 
bulk_create:

https://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.models.query.QuerySet.bulk_create
 

However, I noticed the caveat about it not populating the auto-incrementing 
PK field. What's the recommended way to populate the PK field then? How do 
you use it?

Secondly, the models will have FK relationships to other models (for 
instance, Classification, Country and Operating System would be stored in a 
lookup table). What's the best way to look these up, and assign the 
appropriate FK, whilst still preserving good performance and using bulk 
inserts?

Cheers,
Victor

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-users/-/bP0RG4NXJjEJ.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.