#33636: BulkProcessMixin on models.Model
-------------------------------------+-------------------------------------
Reporter: Myung Eui Yoon | Owner: Myung Eui
| Yoon
Type: New feature | Status: assigned
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: bulk, model | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Myung Eui Yoon):
* has_patch: 0 => 1
Old description:
> {{{
> """
> class BulkyModel(models.Model, ModelBulkProcessMixin):
> id = models.BigAutoField(primary_key=True)
> name = models.CharField(max_length=10, null=False)
>
> With ModelBulkProcessMixin , We could minimize memory usage.
> Without ModelBulkProcessMixin, We shold maintain bulk array size
> up to 100_000
> or manually maintain arraysize up to batch_size like 10_000
>
> if len(chunked_list)>10_000:
> Model.objects.bulk_create(chunked_list)
>
> and check remain in list again at the end.
>
> if len(chunked_list)>0:
> Model.objects.bulk_create(chunked_list)
> """
>
> names = [f"name-{num}" for num in range(100_000)]
>
> with BulkyModel.gen_bulk_create(batch_size=10_000) as bulk:
> for name in names:
> bulk.add(BulkyModel(name=name))
>
> self.assertEqual(100_000, BulkyModel.objects.all().count())
> }}}
New description:
{{{
"""
class BulkyModel(models.Model, ModelBulkProcessMixin):
id = models.BigAutoField(primary_key=True)
name = models.CharField(max_length=10, null=False)
With ModelBulkProcessMixin , We could minimize memory usage.
Without ModelBulkProcessMixin, We shold maintain bulk array size
up to 100_000
or manually maintain arraysize up to batch_size like 10_000
if len(chunked_list)>10_000:
Model.objects.bulk_create(chunked_list)
and check remain in list again at the end.
if len(chunked_list)>0:
Model.objects.bulk_create(chunked_list)
"""
names = [f"name-{num}" for num in range(100_000)]
with BulkyModel.gen_bulk_create(batch_size=10_000) as bulk:
for name in names:
bulk.add(BulkyModel(name=name))
self.assertEqual(100_000, BulkyModel.objects.all().count())
}}}
https://github.com/django/django/pull/15577
--
Comment:
Issued Pull Request
https://github.com/django/django/pull/15577
--
Ticket URL: <https://code.djangoproject.com/ticket/33636#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/0107018018f2f058-4e8357b6-6b78-4cab-825f-f45221a3e5dd-000000%40eu-central-1.amazonses.com.
