Hi Aymeric,

Thank you for the answer and for the tests as well.
I understand and also agree your vision on all the line, I got the specific
purpose of parse_date.
I'd like to introduce a generalized way to parse date and datetime string
based on Django project configuration, in settings.py.
It could be linked, as reference, to DATE_INPUT_FORMATS or DATE_FORMATS or
whatever better choice.

This is the code example, it works as standalone, I hope that it's easy to
read...
````
# datetime_euristic_parser.py
import re
import datetime

DATE_FORMATS = ['%Y-%m-%d',
                '%d/%m/%Y',
                '%d/%m/%y']
DATETIME_FORMATS = ['%Y-%m-%d %H:%M:%S',
                    '%d/%m/%Y %H:%M:%S',
                    '%d/%m/%y %H:%M:%S',
                    '%Y%m%d%H%M%SZ',
                    '%Y%m%d%H%M%S.%fZ']
# to be extended with all the matching patterns.
DATETIME_ELEMENTS_REGEXP = {'%Y': '(?P<year>\d{4})',
                            '%y': '(?P<year>\d{2})',
                            '%m': '(?P<month>\d{1,2})',
                            '%d': '(?P<day>\d{1,2})',
                            '%H': '(?P<hour>\d{1,2})',
                            '%M': '(?P<minute>\d{1,2})',
                            '%S': '(?P<second>\d{1,2})',
                            '%f': '(?P<microsecond>\d{6})'} # ...

def datetime_regexp_builder(formats):
    """
    formats = DATE_FORMAT of DATETIME_FORMAT
    """
    regexp_dict = {}
    for df in formats:
        df_regexp = df
        for k,v in DATETIME_ELEMENTS_REGEXP.items():
            df_regexp = df_regexp.replace(k,v)
        regexp_dict[df] = df_regexp+'$'
    return regexp_dict

DATE_FORMATS_REGEXP = datetime_regexp_builder(DATE_FORMATS)
DATETIME_FORMATS_REGEXP = datetime_regexp_builder(DATETIME_FORMATS)

def dformat_insp(date_str, format_regexp_dict, debug=False):
    """
    Takes a date string and returns a matching date regexp.
    """
    insp_formats = []
    for f,p in format_regexp_dict.items():
        if debug: print(date_str, f, p)
        match = re.match(p, date_str)
        if match:
            res = (f, p, {k:int(v) for k,v in match.groupdict().items()})
            insp_formats.append(res)
    return insp_formats

def dateformat_insp(date_str):
    return dformat_insp(date_str, DATE_FORMATS_REGEXP)

def datetimeformat_insp(date_str):
    return dformat_insp(date_str, DATETIME_FORMATS_REGEXP)

def datetime_euristic_parser(value):
    """
    value can be a datestring or a datetimestring
    returns all the parsed date or datetime object
    """
    l = []
    res = dateformat_insp(value) or \
          datetimeformat_insp(value)
    for i in res:
        l.append(datetime.datetime(**i[-1]))
    return l

# example
if __name__ == '__main__':
    tests = ['04/12/2018',
             '04/12/2018 3:2:1',
             '2018-03-4 09:7:4',
             '2018-03-04T09:7:4.645194',
             '20180304121940.948000Z']

    for i in tests:
        res = dateformat_insp(i) or datetimeformat_insp(i)
        if res:
            print('Parsing succesfull on "{}": {}'.format(i, res))
            #print(datetime_euristic_parser(i))
        else:
            print('Parsing failed on "{}"'.format(i))
        print()
````

I also checked these tests:
python -m timeit -s "from django.utils.dateparse import parse_datetime"
"parse_datetime('2019-02-03T17:27:58.645194')"
10000 loops, best of 3: 32.7 usec per loop

python -m timeit -s "import datetime"
"datetime.datetime.strptime('2019-02-03T17:27:58.645194',
'%Y-%m-%dT%H:%M:%S.%f')"
10000 loops, best of 3: 53.5 usec per loop

python -m timeit -s "from datetime_euristic_parser. import
datetime_euristic_parser;
datetime_euristic_parser('2019-02-03T17:27:58.645194')"
10000000 loops, best of 3: 0.0241 usec per loop

In other words I'd like to have in Django a magic datetime parser, based on
the settings.py global definitions, instead of iterations on try/except  or
some other custom code.
Hope it's fun

Il giorno dom 3 feb 2019 alle ore 18:44 Aymeric Augustin <
aymeric.augus...@polytechnique.org> ha scritto:

> Hello Guiseppe,
>
> django.utils.dateparse provides helpers needed by Django to implement
> datetime, date and time fields on SQLite. (SQLite doesn't have a native
> date time type.) Their job is to parse ISO 8601 fast. That's it.
>
> A utility module should do exactly what Django needs and nothing
> more. django.utils.dateparse is documented so you can use it if it does
> what you want. If it doesn't, use something else ;-)
>
> Django forms try various formats when they receive user input because
> there's uncertainty about the format. If you're parsing user input, then
> you should use a form and you'll get the behavior you want.
>
> To clarify my point about performance, here's the function you're
> proposing, minus support for USE_L10N:
>
> def parse_datetime_alternative(value):
>     for format in settings.DATETIME_INPUT_FORMATS:
>         try:
>             return datetime.datetime.strptime(format, value)
>         except (ValueError, TypeError):
>             continue
>
> It's 10 times slower than the current implementation of parse_datetime:
>
> $ python -m timeit -s "from django.conf import settings;
> settings.configured or settings.configure(); from django.utils.dateparse
> import parse_datetime_alternative as parse_datetime"
> "parse_datetime('2019-02-03T17:27:58.645194')"
> 5000 loops, best of 5: 54.2 usec per loop
>
> $ python -m timeit -s "from django.utils.dateparse import parse_datetime"
> "parse_datetime('2019-02-03T17:27:58.645194')"
> 50000 loops, best of 5: 5.48 usec per loop
>
> I implemented parse_datetime with a regex because that's almost twice as
> fast as a single call to datetime.datetime.strptime:
>
> $ python -m timeit -s "import datetime"
> "datetime.datetime.strptime('2019-02-03T17:27:58.645194',
> '%Y-%m-%dT%H:%M:%S.%f')"
> 20000 loops, best of 5: 9.87 usec per loop
>
> Best regards,
>
> --
> Aymeric.
>
>
>
> On 3 Feb 2019, at 17:10, Giuseppe De Marco <giuseppe.dema...@unical.it>
> wrote:
>
> Hi All, it's the first time for me in this ml,
>
> I'd like to purpose a refactor of django.utils.dateparse functions.
> Currently a function in it, like parse_date for example, extract date time
> string with a static regexp...
>
> https://docs.djangoproject.com/pl/2.1/_modules/django/utils/dateparse/
>
> The first time I used It I thought that those functions parses date, time
> and date time, according to settings.py definitions like
> DATETIME_INPUT_FORMATS, but It was not this way... Then I read the code.
>
> Wouldn't It better to use settings.py definitions to manage these formats
> and django.utils.dateparse behaviour?
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CABms%2BYpGtCQ1yzVfGtRKpDv60-4oH_SyGpPQYR6j1cR%2BL5LFRA%40mail.gmail.com
> <https://groups.google.com/d/msgid/django-developers/CABms%2BYpGtCQ1yzVfGtRKpDv60-4oH_SyGpPQYR6j1cR%2BL5LFRA%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/BEAAEDB4-D58E-4CD0-8F1A-11A5F60B556B%40polytechnique.org
> <https://groups.google.com/d/msgid/django-developers/BEAAEDB4-D58E-4CD0-8F1A-11A5F60B556B%40polytechnique.org?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
____________________
Dott. Giuseppe De Marco
CENTRO ICT DI ATENEO
University of Calabria
87036 Rende (CS) - Italy
Phone: +39 0984 496945
e-mail: giuseppe.dema...@unical.it

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CABms%2BYqTLtAoBVMj9C5KYgcM0GQkV6fvK89Cg5LJWGZ3B7sP_w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to