Hi All, This week's work was a follow-up of last week's work, with some new discoveries with regards to the new API:
*1) Improved the current _meta implementation of unittests* I refactored the current meta unittest branch into a more compact version. I also added test cases for Generic Foreign Keys, RelatedField and more improvements on virtual fields. This section will actually be our first merge to master: a unit test suite for the current implementation. This branch is found here: https://github.com/PirosB3/django/tree/soc2014_meta_unittests *2) Refactored the new _meta API spec* By implementing the new API, I found new redundancies in the current implementation that we can avoid in the new API spec: *1) Only return field instances* the current implementation has a common return pattern: (field_object, model, direct, m2m). After a few tests I realized that model, direct, m2m and field_name are all redundant and can be derived from only field_object. Since there are only 3-4 places that actually use direct and m2m it makes sense to remove this function from the new API spec. Here I show how all the information can be derived form field_instance (https://gist.github.com/PirosB3/6cb4badbb1b8c2e41a96/revisions), I ran the entire test suite and things look good. The only issue I can see now is from a performance point of view. *2) Provide only 2 API endpoints* - get_fields(types, opts) -> (field_instance, field_instance, ...) - get_field(field_name) -> field_instance The rest is all (I might be very wrong! don't trust me) redundant. By continuing with this iterative approach, I will soon find out if I am correct or not ;) This branch is found here: https://github.com/PirosB3/django/tree/soc2014_meta_refactor For any questions, as usual, let me know Dan On Friday, June 6, 2014 7:03:36 PM UTC+2, Daniel Pyrathon wrote: > > Hi All, > > Based on the work done last week, this week I have worked on the following: > > *1) Covered the current _meta implementation of unittests* > The current Options is not covered by unit tests at all, I have created > the model_options test module containing one or more unit tests for each > endpoint and usage. Each endpoint is tested with many different models and > fields (data, m2m, related objects, related m2m, virtual, ..). Each unit > test asserts equality in field naming and field type. Endpoints that return > the model are also tested for equality. > > This branch is found here: > https://github.com/PirosB3/django/tree/soc2014_meta_unittests > > *2) Pulled in tests from soc2014_meta_unittests and tested the new > implementation* > The previous branch that I developed on contains the new API > implementation, I have pulled in the tests from soc2014_meta_unittests and > I have switched the old API calls to the new API calls (with a few > adjustments). I have successfully made all tests pass even though I have > found some design issues that need to be addressed in my next update call. > > This branch is found here: > https://github.com/PirosB3/django/tree/soc2014_meta_refactor > > *3) Created a new branch that maps the old implementation with the new* > Today I started putting my new API in "production". This is obviously > nowhere near a finalised version but it helps me spot some edge cases that > are not in the unit-tests. Each issue found has been converted into a > standalone unit-test and has been proved to fail. > Unfortunately, this made me realise of other design issues related to the > new return type of get_fields -> (field_name, field), A decision will be > taken on monday. > > This branch is found here: > https://github.com/PirosB3/django/tree/soc2014_meta_refactor_implementation > as is for personal use only > > For any questions, let me know! > Dan > > > On Sunday, June 1, 2014 6:10:53 PM UTC+2, Daniel Pyrathon wrote: >> >> Hi All, >> >> An update on my side, some interesting work has happened this week: me >> and Russell have decided to start on the implementation early in order to >> understand better the internals of Options. Currently I am working on the >> following: >> >> *Providing one single function: get_fields(types, opts, **kwargs)* >> Previously we had identified a number of functions that contained similar >> data but had a different return type. We are going to provide one function >> that takes a set of field types + options and returns the same output >> everywhere: ((field_name, field), ...). This has the benefit of simplicity >> and is more maintainable than the previous approach. >> >> TYPES = DATA, M2M, FK, RELATED_OBJECT, RELATED_M2M >> OPTS = NONE, LOCAL_ONLY, CONCRETE, INCLUDE_HIDDEN, INCLUDE_PROXY, VIRTUAL >> >> *Providing two functions for retrieving details of a specific field* >> As specified in my previous document, in many parts of the code we just >> want to retrieve a field object by name, other times we have a field but >> need other metadata such as: owner (model_class), direct (bool), m2m >> (bool). We will provide two functions: >> >> get_field(field_name) -> field_instance >> >> get_field_details(field_instance) -> direct, m2m, (still to be defined) >> >> While we still have not entirely defined what *get_field_details *will >> return, this will be done soon. >> >> >> *Building a test suite for the existing API* >> The new API development will be driven by a test suite that will compare >> the current (legacy) API with the new implementation. While return types >> will be different, we are asserting that all the correct fields and >> metadata are returned. Building a test suite means we can start >> implementing before a final API spec is finalised. It also means we can >> iterate faster and, from my perspective, I also understand a lot more of >> the current implementation. We are testing each combination of fields and >> options together. >> >> My current implementation is visible here: >> https://github.com/PirosB3/django/compare/soc2014_meta_refactor >> >> For any questions or suggestions, let me know. >> >> Daniel Pyrathon >> >> >> On Monday, May 26, 2014 1:28:31 AM UTC+2, Daniel Pyrathon wrote: >>> >>> Hi All, >>> >>> Just to make you know, I have put up the current _meta API documentation >>> here: >>> http://162.219.6.191:8000/ref/models/meta.html?highlight=_meta >>> As always, feel free to ask questions. >>> >>> Daniel >>> >>> On Monday, May 26, 2014 1:26:27 AM UTC+2, Daniel Pyrathon wrote: >>>> >>>> Hi Josh, >>>> >>>> The meta API specified in the docs ( >>>> https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt) >>>> >>>> is the current API. I have documented this in order to understand more of >>>> the current implementation and it will be good to show a comparison when a >>>> new meta API will be approved. >>>> >>>> My current proposal ( >>>> https://gist.github.com/PirosB3/371704ed40ed093d5a82) and it will be >>>> discussed tomorrow with Russell. I will post as soon as I have updates. >>>> >>>> Daniel Pyrathon >>>> >>>> On Saturday, May 24, 2014 10:37:49 AM UTC+2, Josh Smeaton wrote: >>>>> >>>>> Hi Daniel, >>>>> >>>>> Nice work putting that document together. Is the meta document you put >>>>> together the current API or is it the API you are proposing? If the >>>>> latter, >>>>> a few suggestions (and if others disagree, please point that out): >>>>> >>>>> - Remove all mention of caching. That should be an implementation >>>>> detail only, and not a requirement for other implementations. >>>>> - the *_with_model methods really rub me up the wrong way. I would >>>>> prefer always returning the _with_model variant, and letting the caller >>>>> discard the model if they don't need it. >>>>> - I'm not a fan of virtual and concrete fields, though I have to admit >>>>> I'm not sure how they're different, especially in the context of >>>>> different >>>>> implementations. >>>>> - Not sure that m2m should be differentiated from related. >>>>> - init_name_map should be an implementation detail. >>>>> - normalize_together should be an implementation detail. >>>>> >>>>> Regards, >>>>> >>>>> Josh >>>>> >>>>> On Saturday, 24 May 2014 05:05:02 UTC+10, Daniel Pyrathon wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> In the last days I have built a documentation of the current state of >>>>>> Options. Based on feedback and prototyping I have thought of a potential >>>>>> interface for _meta that can solve the issues currently present, such as >>>>>> redundancy (in code and in caching systems). The interface has also been >>>>>> thought to be maintainable and is a base that can be used to create >>>>>> custom >>>>>> meta stores. >>>>>> Obviously this is far from perfect, It will need many iterations and >>>>>> maybe it is too complex. I would really love to gain as much feedback as >>>>>> possible so it can be discussed with Russell and the community on Monday. >>>>>> >>>>>> The documentation of _meta can be found here: >>>>>> https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt >>>>>> I will be refining the document in the next days, I will also be >>>>>> publishing the docs on a webserver and will be linking a URL soon. >>>>>> >>>>>> My proposal has been published here: >>>>>> https://gist.github.com/PirosB3/371704ed40ed093d5a82 >>>>>> In the next days I will be iterating over the feedback gained, and >>>>>> based on one very interesting suggestion on IRC, I will try to see how >>>>>> my >>>>>> API syntax looks in modelforms.py. >>>>>> >>>>>> As said previously, and feedback is greatly appreciated. >>>>>> >>>>>> Hi from Pycon IT! >>>>>> >>>>>> Daniel Pyrathon >>>>>> >>>>>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote: >>>>>>> >>>>>>> Best of luck! >>>>>>> >>>>>>> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote: >>>>>>>> >>>>>>>> Hi All, >>>>>>>> >>>>>>>> Today I will be starting my weekly updates on my SoC project: >>>>>>>> refactoring Meta to a stable API. For anyone who missed out, you will >>>>>>>> be >>>>>>>> able to view it here: >>>>>>>> https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit >>>>>>>> >>>>>>>> This week is the first official week of SoC. Me and my mentor >>>>>>>> (Russell) are initially approaching the work in the following way: >>>>>>>> >>>>>>>> - *Document the existing Meta API* >>>>>>>> For each endpoint, document the following: >>>>>>>> - Input parameters and return type >>>>>>>> - Caching pattern used >>>>>>>> - Where it's called from (internally and externally to Meta) >>>>>>>> - Why is it being called >>>>>>>> - When is it being called >>>>>>>> >>>>>>>> - *Propose an initial refactor plan* >>>>>>>> Once the documentation has been done, I should have a better >>>>>>>> idea of the current implementation. This will allow me to mock a >>>>>>>> proposed >>>>>>>> implementation that will be reviewed at my next update call, on >>>>>>>> Monday. >>>>>>>> >>>>>>>> My next update will be posted on Friday, just to make sure the >>>>>>>> community is informed of my progress. For any major updates that >>>>>>>> require >>>>>>>> community approval, I will be creating separate threads. >>>>>>>> My name on the internet is pirosb3, so if you want to have a chat >>>>>>>> about my progress feel free to contact me! The branch I am currently >>>>>>>> working on is >>>>>>>> https://github.com/PirosB3/django/tree/meta_documentation >>>>>>>> >>>>>>>> Regards, >>>>>>>> Daniel Pyrathon >>>>>>>> >>>>>>> >>>>>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote: >>>>>>> >>>>>>> Best of luck! >>>>>>> >>>>>>> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote: >>>>>>>> >>>>>>>> Hi All, >>>>>>>> >>>>>>>> Today I will be starting my weekly updates on my SoC project: >>>>>>>> refactoring Meta to a stable API. For anyone who missed out, you will >>>>>>>> be >>>>>>>> able to view it here: >>>>>>>> https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit >>>>>>>> >>>>>>>> This week is the first official week of SoC. Me and my mentor >>>>>>>> (Russell) are initially approaching the work in the following way: >>>>>>>> >>>>>>>> - *Document the existing Meta API* >>>>>>>> For each endpoint, document the following: >>>>>>>> - Input parameters and return type >>>>>>>> - Caching pattern used >>>>>>>> - Where it's called from (internally and externally to Meta) >>>>>>>> - Why is it being called >>>>>>>> - When is it being called >>>>>>>> >>>>>>>> - *Propose an initial refactor plan* >>>>>>>> Once the documentation has been done, I should have a better >>>>>>>> idea of the current implementation. This will allow me to mock a >>>>>>>> proposed >>>>>>>> implementation that will be reviewed at my next update call, on >>>>>>>> Monday. >>>>>>>> >>>>>>>> My next update will be posted on Friday, just to make sure the >>>>>>>> community is informed of my progress. For any major updates that >>>>>>>> require >>>>>>>> community approval, I will be creating separate threads. >>>>>>>> My name on the internet is pirosb3, so if you want to have a chat >>>>>>>> about my progress feel free to contact me! The branch I am currently >>>>>>>> working on is >>>>>>>> https://github.com/PirosB3/django/tree/meta_documentation >>>>>>>> >>>>>>>> Regards, >>>>>>>> Daniel Pyrathon >>>>>>>> >>>>>>> -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/78f45c61-9626-4aa9-a854-64b11ba44572%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
