#25534: Allow using datetime lookups in QuerySets aggregate calls
-------------------------------------+-------------------------------------
Reporter: borfast | Owner: nobody
Type: New feature | Status: new
Component: Database layer | Version: 1.7
(models, ORM) |
Severity: Normal | Resolution:
Keywords: QuerySet.extra | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Description changed by borfast:
Old description:
> I've been scouring the web for an answer to this but the closest I can
> find is #25339, which is almost what I need but not quite, so I think I
> can safely conclude that it is just not possible with the currently
> available functionality and thus I'm opening this ticket to suggest
> adding it.
>
> I would like to be able to use datetime lookups in an aggregate() call on
> a QuerySet.
>
> My specific use case is this: I have a set of electricity consumption
> readings, each with a datetime field (and a few others). I need to sum
> the consumption and cost values grouped by month, day, year, week, etc.
> In other words, I need to be able to get the total energy consumption
> value and corresponding cost for each month, day, year, week, etc.
>
> I think the code I need is something along the lines of the following:
>
> {{{#!python
> result = ElectricityReading.objects\
> .filter(device__grid__building_id=1) \
> .annotate(num_readings=Count('id'))\
> .annotate(total_consumption=Sum('consumption'))\
> .annotate(total_cost=Sum('cost'))\
> .aggregate(total=Count('datetime__month'))
> }}}
>
> Right now I'm doing this with this raw SQL:
>
> {{{#!sql
> SELECT
> (EXTRACT(YEAR FROM datetime)) AS reading_date_year,
> (EXTRACT(MONTH FROM datetime)) AS reading_date_month,
> (EXTRACT(DAY FROM datetime)) AS reading_date_day,
> (EXTRACT(HOUR FROM datetime)) AS reading_date_hour,
> SUM(consumption) as total,
> COUNT(id) as num_readings,
> SUM(cost) as total_cost,
> price_id
> FROM electricity_reading
> WHERE device_id IN (1, 2, 3)
> AND datetime >= '2015-10-01'
> AND datetime <= '2015-10-10'
> GROUP BY reading_date_year, reading_date_month,
> reading_date_day,reading_date_hour, price_id
> }}}
>
> The part I can't seem to replicate with Django's ORM is the {{{GROUP
> BY}}} clause at the end, which is what I was expecting to be able to
> achieve using the {{{aggregate(total=Count('datetime__month'))}}} but
> instead I get the following error:
>
> {{{
> FieldError: Cannot resolve keyword 'datetime' into field. Choices are:
> consumption, cost, created, datetime, device, device_id, id, inserted_by,
> inserted_by_id, manual, modified, pf, price, price_id, varh, vph1, vph2,
> vph3, wh_imp, num_readings, total_consumption, total_cost
> }}}
>
> I would love that someone would tell me I am missing something, and if
> that's the case, please do! :)
>
> Otherwise, I believe it would be beneficial to add this.
New description:
I've been scouring the web for an answer to this but the closest I can
find is #25339, which is almost what I need but not quite, so I think I
can safely conclude that it is just not possible with the currently
available functionality and thus I'm opening this ticket to suggest adding
it.
I would like to be able to use datetime lookups in an aggregate() call on
a QuerySet.
My specific use case is this: I have a set of electricity consumption
readings, each with a datetime field (and a few others). I need to sum the
consumption and cost values grouped by month, day, year, week, etc. In
other words, I need to be able to get the total energy consumption value
and corresponding cost for each month, day, year, week, etc.
This is my {{{ElectricityReading}}} model and its parent {{{Reading}}}
model (separated because we also have consumption readings for water and
gas, which also derive from {{{Reading}}}):
{{{#!python
from model_utils.models import TimeStampedModel
# Other imports here...
class Reading(TimeStampedModel):
device = models.ForeignKey(Device)
datetime = models.DateTimeField() # Terrible property name, I know :)
manual = models.BooleanField(default=False)
inserted_by = models.ForeignKey(User)
class Meta:
abstract = True
class ElectricityReading(Reading):
vph1 = models.DecimalField(max_digits=18, decimal_places=3, null=True)
vph2 = models.DecimalField(max_digits=18, decimal_places=3, null=True)
vph3 = models.DecimalField(max_digits=18, decimal_places=3, null=True)
wh_imp = models.DecimalField(max_digits=18, decimal_places=3)
varh = models.DecimalField(max_digits=18, decimal_places=3, null=True)
pf = models.DecimalField(max_digits=18, decimal_places=3, null=True)
price = models.ForeignKey(ElectricityPrice)
consumption = models.DecimalField(max_digits=18, decimal_places=3,
null=True, blank=True, default=None)
cost = models.DecimalField(max_digits=18, decimal_places=3, null=True,
blank=True, default=None)
}}}
I think the code I need is something along the lines of the following:
{{{#!python
result = ElectricityReading.objects\
.filter(device__grid__building_id=1) \
.annotate(num_readings=Count('id'))\
.annotate(total_consumption=Sum('consumption'))\
.annotate(total_cost=Sum('cost'))\
.aggregate(total=Count('datetime__month'))
}}}
Right now I'm doing this with this raw SQL:
{{{#!sql
SELECT
(EXTRACT(YEAR FROM datetime)) AS reading_date_year,
(EXTRACT(MONTH FROM datetime)) AS reading_date_month,
(EXTRACT(DAY FROM datetime)) AS reading_date_day,
(EXTRACT(HOUR FROM datetime)) AS reading_date_hour,
SUM(consumption) as total,
COUNT(id) as num_readings,
SUM(cost) as total_cost,
price_id
FROM electricity_reading
WHERE device_id IN (1, 2, 3)
AND datetime >= '2015-10-01'
AND datetime <= '2015-10-10'
GROUP BY reading_date_year, reading_date_month,
reading_date_day,reading_date_hour, price_id
}}}
The part I can't seem to replicate with Django's ORM is the {{{GROUP BY}}}
clause at the end, which is what I was expecting to be able to achieve
using the {{{aggregate(total=Count('datetime__month'))}}} but instead I
get the following error:
{{{
FieldError: Cannot resolve keyword 'datetime' into field. Choices are:
consumption, cost, created, datetime, device, device_id, id, inserted_by,
inserted_by_id, manual, modified, pf, price, price_id, varh, vph1, vph2,
vph3, wh_imp, num_readings, total_consumption, total_cost
}}}
I would love that someone would tell me I am missing something, and if
that's the case, please do! :)
Otherwise, I believe it would be beneficial to add this.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/25534#comment:5>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/065.b48a5cde313cdc16ff4114ac3dee77c4%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
