Sure! https://github.com/PirosB3/django/compare/soc2014_meta_refactor_performance?expand=1
This contains the following: - Recently merged unit test suite for model/options - New API implementation: get_new_fields, get_new_field. - Implementation of recently merged unit test suite with new API - Small optimizations (BIG work in progress) If you have any suggestions please let me know! Dan On Sunday, June 22, 2014 2:14:56 AM UTC+2, Curtis Maloney wrote: > > Is there somewhere I can look at your work in progress code? > > > On 21 June 2014 19:57, Daniel Pyrathon <[email protected] <javascript:>> > wrote: > >> Hi All, >> >> This week I have done the following >> >> *- Openend PR and merged Options (_meta) unittests* >> https://github.com/django/django/pull/2823 >> This is a working test suite of model/options.py file. Is is currently >> testing the current Options implementation. >> >> *- Re-implemented test-suite in the new API* >> My new proposed API has been implemented on top of the Options unittests. >> This means new and old API give the same results! and therefore the new API >> is a working re-implementation. >> >> *- Performance optimizations* >> Currently, the biggest issue I have had (in the new API) regards >> performance. I am doing small optimisations and benchmarking with cProfile. >> If anyone has some good hints on how to benchmark single functions in >> Django please let me know! >> >> Regards, >> Dan >> >> >> On Friday, June 13, 2014 10:11:41 PM UTC+2, Aymeric Augustin wrote: >> >>> I like this simplification a lot. I hope you can make it work. >>> >>> Just check that you don’t "overload" fields, by storing in field objects >>> information that doesn’t belong there. >>> >>> -- >>> Aymeric. >>> >>> >>> >>> On 13 juin 2014, at 19:53, Daniel Pyrathon <[email protected]> wrote: >>> >>> Hi All, >>> >>> This week's work was a follow-up of last week's work, with some new >>> discoveries with regards to the new API: >>> >>> *1) Improved the current _meta implementation of unittests* >>> I refactored the current meta unittest branch into a more compact >>> version. I also added test cases for Generic Foreign Keys, RelatedField and >>> more improvements on virtual fields. This section will actually be our >>> first merge to master: a unit test suite for the current implementation. >>> >>> This branch is found here: https://github.com/ >>> PirosB3/django/tree/soc2014_meta_unittests >>> >>> *2) Refactored the new _meta API spec* >>> By implementing the new API, I found new redundancies in the current >>> implementation that we can avoid in the new API spec: >>> >>> *1) Only return field instances* >>> the current implementation has a common return pattern: (field_object, >>> model, direct, m2m). After a few tests I realized that model, direct, m2m >>> and field_name are all redundant and can be derived from only field_object. >>> Since there are only 3-4 places that actually use direct and m2m it makes >>> sense to remove this function from the new API spec. >>> Here I show how all the information can be derived form field_instance ( >>> https://gist.github.com/PirosB3/6cb4badbb1b8c2e41a96/revisions), I ran >>> the entire test suite and things look good. The only issue I can see now is >>> from a performance point of view. >>> >>> *2) Provide only 2 API endpoints* >>> >>> - get_fields(types, opts) -> (field_instance, field_instance, ...) >>> - get_field(field_name) -> field_instance >>> >>> The rest is all (I might be very wrong! don't trust me) redundant. By >>> continuing with this iterative approach, I will soon find out if I am >>> correct or not ;) >>> >>> >>> This branch is found here: https://github.com/ >>> PirosB3/django/tree/soc2014_meta_refactor >>> >>> For any questions, as usual, let me know >>> Dan >>> >>> >>> On Friday, June 6, 2014 7:03:36 PM UTC+2, Daniel Pyrathon wrote: >>>> >>>> Hi All, >>>> >>>> Based on the work done last week, this week I have worked on the >>>> following: >>>> >>>> *1) Covered the current _meta implementation of unittests* >>>> The current Options is not covered by unit tests at all, I have created >>>> the model_options test module containing one or more unit tests for each >>>> endpoint and usage. Each endpoint is tested with many different models and >>>> fields (data, m2m, related objects, related m2m, virtual, ..). Each unit >>>> test asserts equality in field naming and field type. Endpoints that >>>> return >>>> the model are also tested for equality. >>>> >>>> This branch is found here: https://github.com/ >>>> PirosB3/django/tree/soc2014_meta_unittests >>>> >>>> *2) Pulled in tests from soc2014_meta_unittests and tested the new >>>> implementation* >>>> The previous branch that I developed on contains the new API >>>> implementation, I have pulled in the tests from soc2014_meta_unittests and >>>> I have switched the old API calls to the new API calls (with a few >>>> adjustments). I have successfully made all tests pass even though I have >>>> found some design issues that need to be addressed in my next update call. >>>> >>>> This branch is found here: https://github.com/ >>>> PirosB3/django/tree/soc2014_meta_refactor >>>> >>>> *3) Created a new branch that maps the old implementation with the new* >>>> Today I started putting my new API in "production". This is obviously >>>> nowhere near a finalised version but it helps me spot some edge cases that >>>> are not in the unit-tests. Each issue found has been converted into a >>>> standalone unit-test and has been proved to fail. >>>> Unfortunately, this made me realise of other design issues related to >>>> the new return type of get_fields -> (field_name, field), A decision will >>>> be taken on monday. >>>> >>>> This branch is found here: https://github.com/ >>>> PirosB3/django/tree/soc2014_meta_refactor_implementation as is for >>>> personal use only >>>> >>>> For any questions, let me know! >>>> Dan >>>> >>>> >>>> On Sunday, June 1, 2014 6:10:53 PM UTC+2, Daniel Pyrathon wrote: >>>>> >>>>> Hi All, >>>>> >>>>> An update on my side, some interesting work has happened this week: me >>>>> and Russell have decided to start on the implementation early in order to >>>>> understand better the internals of Options. Currently I am working on the >>>>> following: >>>>> >>>>> *Providing one single function: get_fields(types, opts, **kwargs)* >>>>> Previously we had identified a number of functions that contained >>>>> similar data but had a different return type. We are going to provide one >>>>> function that takes a set of field types + options and returns the same >>>>> output everywhere: ((field_name, field), ...). This has the benefit of >>>>> simplicity and is more maintainable than the previous approach. >>>>> >>>>> TYPES = DATA, M2M, FK, RELATED_OBJECT, RELATED_M2M >>>>> OPTS = NONE, LOCAL_ONLY, CONCRETE, INCLUDE_HIDDEN, INCLUDE_PROXY, >>>>> VIRTUAL >>>>> >>>>> *Providing two functions for retrieving details of a specific field* >>>>> As specified in my previous document, in many parts of the code we >>>>> just want to retrieve a field object by name, other times we have a field >>>>> but need other metadata such as: owner (model_class), direct (bool), m2m >>>>> (bool). We will provide two functions: >>>>> >>>>> get_field(field_name) -> field_instance >>>>> >>>>> get_field_details(field_instance) -> direct, m2m, (still to be >>>>> defined) >>>>> >>>>> While we still have not entirely defined what *get_field_details *will >>>>> return, this will be done soon. >>>>> >>>>> >>>>> *Building a test suite for the existing API* >>>>> The new API development will be driven by a test suite that will >>>>> compare the current (legacy) API with the new implementation. While >>>>> return >>>>> types will be different, we are asserting that all the correct fields and >>>>> metadata are returned. Building a test suite means we can start >>>>> implementing before a final API spec is finalised. It also means we can >>>>> iterate faster and, from my perspective, I also understand a lot more of >>>>> the current implementation. We are testing each combination of fields and >>>>> options together. >>>>> >>>>> My current implementation is visible here: https://github.com/ >>>>> PirosB3/django/compare/soc2014_meta_refactor >>>>> >>>>> For any questions or suggestions, let me know. >>>>> >>>>> Daniel Pyrathon >>>>> >>>>> >>>>> On Monday, May 26, 2014 1:28:31 AM UTC+2, Daniel Pyrathon wrote: >>>>>> >>>>>> Hi All, >>>>>> >>>>>> Just to make you know, I have put up the current _meta API >>>>>> documentation here: >>>>>> http://162.219.6.191:8000/ref/models/meta.html?highlight=_meta >>>>>> As always, feel free to ask questions. >>>>>> >>>>>> Daniel >>>>>> >>>>>> On Monday, May 26, 2014 1:26:27 AM UTC+2, Daniel Pyrathon wrote: >>>>>>> >>>>>>> Hi Josh, >>>>>>> >>>>>>> The meta API specified in the docs (https://github.com/PirosB3/ >>>>>>> django/blob/meta_documentation/docs/ref/models/meta.txt) is the >>>>>>> current API. I have documented this in order to understand more of the >>>>>>> current implementation and it will be good to show a comparison when a >>>>>>> new >>>>>>> meta API will be approved. >>>>>>> >>>>>>> My current proposal (https://gist.github.com/ >>>>>>> PirosB3/371704ed40ed093d5a82) and it will be discussed tomorrow >>>>>>> with Russell. I will post as soon as I have updates. >>>>>>> >>>>>>> Daniel Pyrathon >>>>>>> >>>>>>> On Saturday, May 24, 2014 10:37:49 AM UTC+2, Josh Smeaton wrote: >>>>>>>> >>>>>>>> Hi Daniel, >>>>>>>> >>>>>>>> Nice work putting that document together. Is the meta document you >>>>>>>> put together the current API or is it the API you are proposing? If >>>>>>>> the >>>>>>>> latter, a few suggestions (and if others disagree, please point that >>>>>>>> out): >>>>>>>> >>>>>>>> - Remove all mention of caching. That should be an implementation >>>>>>>> detail only, and not a requirement for other implementations. >>>>>>>> - the *_with_model methods really rub me up the wrong way. I would >>>>>>>> prefer always returning the _with_model variant, and letting the >>>>>>>> caller >>>>>>>> discard the model if they don't need it. >>>>>>>> - I'm not a fan of virtual and concrete fields, though I have to >>>>>>>> admit I'm not sure how they're different, especially in the context of >>>>>>>> different implementations. >>>>>>>> - Not sure that m2m should be differentiated from related. >>>>>>>> - init_name_map should be an implementation detail. >>>>>>>> - normalize_together should be an implementation detail. >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Josh >>>>>>>> >>>>>>>> On Saturday, 24 May 2014 05:05:02 UTC+10, Daniel Pyrathon wrote: >>>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> In the last days I have built a documentation of the current state >>>>>>>>> of Options. Based on feedback and prototyping I have thought of a >>>>>>>>> potential >>>>>>>>> interface for _meta that can solve the issues currently present, such >>>>>>>>> as >>>>>>>>> redundancy (in code and in caching systems). The interface has also >>>>>>>>> been >>>>>>>>> thought to be maintainable and is a base that can be used to create >>>>>>>>> custom >>>>>>>>> meta stores. >>>>>>>>> Obviously this is far from perfect, It will need many iterations >>>>>>>>> and maybe it is too complex. I would really love to gain as much >>>>>>>>> feedback >>>>>>>>> as possible so it can be discussed with Russell and the community on >>>>>>>>> Monday. >>>>>>>>> >>>>>>>>> The documentation of _meta can be found here: https://github.com/ >>>>>>>>> PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt >>>>>>>>> I will be refining the document in the next days, I will also be >>>>>>>>> publishing the docs on a webserver and will be linking a URL soon. >>>>>>>>> >>>>>>>>> My proposal has been published here: >>>>>>>>> https://gist.github.com/PirosB3/371704ed40ed093d5a82 >>>>>>>>> In the next days I will be iterating over the feedback gained, and >>>>>>>>> based on one very interesting suggestion on IRC, I will try to see >>>>>>>>> how my >>>>>>>>> API syntax looks in modelforms.py. >>>>>>>>> >>>>>>>>> As said previously, and feedback is greatly appreciated. >>>>>>>>> >>>>>>>>> Hi from Pycon IT! >>>>>>>>> >>>>>>>>> Daniel Pyrathon >>>>>>>>> >>>>>>>>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote: >>>>>>>>>> >>>>>>>>>> Best of luck! >>>>>>>>>> >>>>>>>>>> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote: >>>>>>>>>>> >>>>>>>>>>> Hi All, >>>>>>>>>>> >>>>>>>>>>> Today I will be starting my weekly updates on my SoC project: >>>>>>>>>>> refactoring Meta to a stable API. For anyone who missed out, you >>>>>>>>>>> will be >>>>>>>>>>> able to view it here: https://docs.google.com/document/d/1yp2_ >>>>>>>>>>> skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit >>>>>>>>>>> >>>>>>>>>>> This week is the first official week of SoC. Me and my mentor >>>>>>>>>>> (Russell) are initially approaching the work in the following way: >>>>>>>>>>> >>>>>>>>>>> - *Document the existing Meta API* >>>>>>>>>>> For each endpoint, document the following: >>>>>>>>>>> - Input parameters and return type >>>>>>>>>>> - Caching pattern used >>>>>>>>>>> - Where it's called from (internally and externally to >>>>>>>>>>> Meta) >>>>>>>>>>> - Why is it being called >>>>>>>>>>> - When is it being called >>>>>>>>>>> >>>>>>>>>>> - *Propose an initial refactor plan* >>>>>>>>>>> Once the documentation has been done, I should have a better >>>>>>>>>>> idea of the current implementation. This will allow me to mock a >>>>>>>>>>> proposed >>>>>>>>>>> implementation that will be reviewed at my next update call, on >>>>>>>>>>> Monday. >>>>>>>>>>> >>>>>>>>>>> My next update will be posted on Friday, just to make sure the >>>>>>>>>>> community is informed of my progress. For any major updates that >>>>>>>>>>> require >>>>>>>>>>> community approval, I will be creating separate threads. >>>>>>>>>>> My name on the internet is pirosb3, so if you want to have a >>>>>>>>>>> chat about my progress feel free to contact me! The branch I am >>>>>>>>>>> currently >>>>>>>>>>> working on is https://github.com/PirosB3/ >>>>>>>>>>> django/tree/meta_documentation >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Daniel Pyrathon >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote: >>>>>>>>>> >>>>>>>>>> Best of luck! >>>>>>>>>> >>>>>>>>>> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote: >>>>>>>>>>> >>>>>>>>>>> Hi All, >>>>>>>>>>> >>>>>>>>>>> Today I will be starting my weekly updates on my SoC project: >>>>>>>>>>> refactoring Meta to a stable API. For anyone who missed out, you >>>>>>>>>>> will be >>>>>>>>>>> able to view it here: https://docs.google.com/document/d/1yp2_ >>>>>>>>>>> skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit >>>>>>>>>>> >>>>>>>>>>> This week is the first official week of SoC. Me and my mentor >>>>>>>>>>> (Russell) are initially approaching the work in the following way: >>>>>>>>>>> >>>>>>>>>>> - *Document the existing Meta API* >>>>>>>>>>> For each endpoint, document the following: >>>>>>>>>>> - Input parameters and return type >>>>>>>>>>> - Caching pattern used >>>>>>>>>>> - Where it's called from (internally and externally to >>>>>>>>>>> Meta) >>>>>>>>>>> - Why is it being called >>>>>>>>>>> - When is it being called >>>>>>>>>>> >>>>>>>>>>> - *Propose an initial refactor plan* >>>>>>>>>>> Once the documentation has been done, I should have a better >>>>>>>>>>> idea of the current implementation. This will allow me to mock a >>>>>>>>>>> proposed >>>>>>>>>>> implementation that will be reviewed at my next update call, on >>>>>>>>>>> Monday. >>>>>>>>>>> >>>>>>>>>>> My next update will be posted on Friday, just to make sure the >>>>>>>>>>> community is informed of my progress. For any major updates that >>>>>>>>>>> require >>>>>>>>>>> community approval, I will be creating separate threads. >>>>>>>>>>> My name on the internet is pirosb3, so if you want to have a >>>>>>>>>>> chat about my progress feel free to contact me! The branch I am >>>>>>>>>>> currently >>>>>>>>>>> working on is https://github.com/PirosB3/ >>>>>>>>>>> django/tree/meta_documentation >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Daniel Pyrathon >>>>>>>>>>> >>>>>>>>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Django developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> >>> Visit this group at http://groups.google.com/group/django-developers. >>> To view this discussion on the web visit https://groups.google. >>> com/d/msgid/django-developers/78f45c61-9626-4aa9-a854- >>> 64b11ba44572%40googlegroups.com >>> <https://groups.google.com/d/msgid/django-developers/78f45c61-9626-4aa9-a854-64b11ba44572%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Django developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected] >> <javascript:>. >> Visit this group at http://groups.google.com/group/django-developers. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-developers/445fea1d-406c-4ae9-b869-c02f189a2b86%40googlegroups.com >> >> <https://groups.google.com/d/msgid/django-developers/445fea1d-406c-4ae9-b869-c02f189a2b86%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/d7334506-3b2f-4e46-aae0-ca66d190c05a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
