Hi All, As usual, here are my updates:
*- Formalised pull request, looking for a formal review* Last week I published the wiki page for my meta implementation. This week I have been working on improving internal code documentation, optimising a few bits and bobs, and added the formal documentation. As usual, all code is available here: https://github.com/django/django/pull/2894. Now it's time for a real *full* *review *of the code, as maybe a few concepts and terms will change but the code is formal and most things might not change. *- Published new benchmarks, and explanations for some slowdowns* The benchmarks for my new implementation are available on the PR (https://github.com/django/django/pull/2894). These benchmarks are taken from DjangoBench (all numbers are the median of 2000 runs of each test). You might see that, while overall performance has increased compared to the current implementation, there are some very odd results. This week I have spent some time investigating the reason for some of the major slowdowns and I have come to the following conclusion: DjangoBench profiling is implemented by spinning up a separate process for each benchmark. (https://github.com/django/djangobench/blob/master/djangobench/main.py#L135). If you look at the chapter "Compute inverse relation map on first access" (https://code.djangoproject.com/wiki/new_meta_api#Computeinverserelationmaponfirstaccess) you will see that the new caching model is more expensive on startup, but is only done once per process. This means we pay a little more price on startup, but then the Options internals do much less work. Although I cannot say I am 100% correct (and I will be investigating further) I believe the slowdown on some benchmarks is related to this, and therefore it may be related more on the way it is benchmarked, rather than the benchmark itself. *- Started on my GMail Store* While I was waiting for review, I have started an implementation of my GMail store. The implementation is documented here (https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit#heading=h.wyxx4xxijubt) and will be my final deliverable. So far, I have had some interesting results. With very little code, and the new Options API, I didn't have to modify a single file inside Django for my implementation to work. All code just overrides some of the main Django classes (Manager, Query, QuerySet). Currently, I have managed to: - Browse my GMail threads through Django admin - View a single GMail thread - Read an entire email! - Use the Console to do everything said above! I have attached a picture. <https://lh3.googleusercontent.com/-br9TSK3bRXA/U8lX7kp3qeI/AAAAAAAALbw/nr0oHgRLkRk/s1600/png.png> The next step is to improve my implementation. I will be (hopefully) soon releasing a GitHub project that you will be able to use to browse, read and send your mail through Django admin and forms! *- I WILL BE ATTENDING EUROPYTHON* I'd love to meet up with anyone attending EuroPython this week. I will obviously participate at the sprints but if you want to catch up before it would be great. It will be a great idea to discuss the new API together and get to know the community more. I might also be doing a lightning talk on the new Options API, or the GMail store depending on progress. For anything, as usual, let me know Daniel Python On Friday, July 11, 2014 4:59:09 PM UTC+2, Daniel Pyrathon wrote: > > Hi All, > > Following this week's work. I have made progress in optimisation and > design of the API. > > I have opened a wiki page that contains all: > - Main concepts > - Decision process > - Benchmarks > - API designs > - How you can help! > > Please see attached: https://code.djangoproject.com/wiki/new_meta_api > > For anything, as usual, just let me know :) > Daniel > > On Saturday, July 5, 2014 12:11:07 PM UTC+2, Josh Smeaton wrote: >> >> Excellent work, well done. I have a couple of questions/ideas though. >> >> 1. The use of bit flags irks me. Would it be possible just to use >> numerous keyword arguments to the new get_fields ? >> 2. Since you've reduced the API to effectively two functions (get_fields, >> get_field), is it still necessary to contain those methods on an inner >> object? Can't the new methods live directly on the model? If they can't be >> put directly on the model for some reason, I would consider a better name >> than "meta". I can't think of a reason that the name meta should be used >> other than history. If you were to think about the purpose of the API >> without the associated history, what would you call it? >> >> log = Log.objects.get(pk=1) >> log.get_fields(concrete=True, m2m=True, related=True) >> >> log.get_field('log_level') # we're being explicit about the field name, >> no need for options >> >> >> Thoughts? >> >> I'll take a better look at the PR and see if I can offer anything with >> regards to your performance optimisations. >> >> Josh >> > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/bd45f62d-558f-4851-a1fb-f305fb075782%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
