I like this simplification a lot. I hope you can make it work. Just check that you don't "overload" fields, by storing in field objects information that doesn't belong there.
-- Aymeric. On 13 juin 2014, at 19:53, Daniel Pyrathon <[email protected]> wrote: > Hi All, > > This week's work was a follow-up of last week's work, with some new > discoveries with regards to the new API: > > 1) Improved the current _meta implementation of unittests > I refactored the current meta unittest branch into a more compact version. I > also added test cases for Generic Foreign Keys, RelatedField and more > improvements on virtual fields. This section will actually be our first merge > to master: a unit test suite for the current implementation. > > This branch is found here: > https://github.com/PirosB3/django/tree/soc2014_meta_unittests > > 2) Refactored the new _meta API spec > By implementing the new API, I found new redundancies in the current > implementation that we can avoid in the new API spec: > > 1) Only return field instances > the current implementation has a common return pattern: (field_object, model, > direct, m2m). After a few tests I realized that model, direct, m2m and > field_name are all redundant and can be derived from only field_object. Since > there are only 3-4 places that actually use direct and m2m it makes sense to > remove this function from the new API spec. > Here I show how all the information can be derived form field_instance > (https://gist.github.com/PirosB3/6cb4badbb1b8c2e41a96/revisions), I ran the > entire test suite and things look good. The only issue I can see now is from > a performance point of view. > > 2) Provide only 2 API endpoints > > - get_fields(types, opts) -> (field_instance, field_instance, ...) > - get_field(field_name) -> field_instance > > The rest is all (I might be very wrong! don't trust me) redundant. By > continuing with this iterative approach, I will soon find out if I am correct > or not ;) > > This branch is found here: > https://github.com/PirosB3/django/tree/soc2014_meta_refactor > > For any questions, as usual, let me know > Dan > > > On Friday, June 6, 2014 7:03:36 PM UTC+2, Daniel Pyrathon wrote: > Hi All, > > Based on the work done last week, this week I have worked on the following: > > 1) Covered the current _meta implementation of unittests > The current Options is not covered by unit tests at all, I have created the > model_options test module containing one or more unit tests for each endpoint > and usage. Each endpoint is tested with many different models and fields > (data, m2m, related objects, related m2m, virtual, ..). Each unit test > asserts equality in field naming and field type. Endpoints that return the > model are also tested for equality. > > This branch is found here: > https://github.com/PirosB3/django/tree/soc2014_meta_unittests > > 2) Pulled in tests from soc2014_meta_unittests and tested the new > implementation > The previous branch that I developed on contains the new API implementation, > I have pulled in the tests from soc2014_meta_unittests and I have switched > the old API calls to the new API calls (with a few adjustments). I have > successfully made all tests pass even though I have found some design issues > that need to be addressed in my next update call. > > This branch is found here: > https://github.com/PirosB3/django/tree/soc2014_meta_refactor > > 3) Created a new branch that maps the old implementation with the new > Today I started putting my new API in "production". This is obviously nowhere > near a finalised version but it helps me spot some edge cases that are not in > the unit-tests. Each issue found has been converted into a standalone > unit-test and has been proved to fail. > Unfortunately, this made me realise of other design issues related to the new > return type of get_fields -> (field_name, field), A decision will be taken on > monday. > > This branch is found here: > https://github.com/PirosB3/django/tree/soc2014_meta_refactor_implementation > as is for personal use only > > For any questions, let me know! > Dan > > > On Sunday, June 1, 2014 6:10:53 PM UTC+2, Daniel Pyrathon wrote: > Hi All, > > An update on my side, some interesting work has happened this week: me and > Russell have decided to start on the implementation early in order to > understand better the internals of Options. Currently I am working on the > following: > > Providing one single function: get_fields(types, opts, **kwargs) > Previously we had identified a number of functions that contained similar > data but had a different return type. We are going to provide one function > that takes a set of field types + options and returns the same output > everywhere: ((field_name, field), ...). This has the benefit of simplicity > and is more maintainable than the previous approach. > > TYPES = DATA, M2M, FK, RELATED_OBJECT, RELATED_M2M > OPTS = NONE, LOCAL_ONLY, CONCRETE, INCLUDE_HIDDEN, INCLUDE_PROXY, VIRTUAL > > Providing two functions for retrieving details of a specific field > As specified in my previous document, in many parts of the code we just want > to retrieve a field object by name, other times we have a field but need > other metadata such as: owner (model_class), direct (bool), m2m (bool). We > will provide two functions: > > get_field(field_name) -> field_instance > > get_field_details(field_instance) -> direct, m2m, (still to be defined) > > While we still have not entirely defined what get_field_details will return, > this will be done soon. > > > Building a test suite for the existing API > The new API development will be driven by a test suite that will compare the > current (legacy) API with the new implementation. While return types will be > different, we are asserting that all the correct fields and metadata are > returned. Building a test suite means we can start implementing before a > final API spec is finalised. It also means we can iterate faster and, from my > perspective, I also understand a lot more of the current implementation. We > are testing each combination of fields and options together. > > My current implementation is visible here: > https://github.com/PirosB3/django/compare/soc2014_meta_refactor > > For any questions or suggestions, let me know. > > Daniel Pyrathon > > > On Monday, May 26, 2014 1:28:31 AM UTC+2, Daniel Pyrathon wrote: > Hi All, > > Just to make you know, I have put up the current _meta API documentation here: > http://162.219.6.191:8000/ref/models/meta.html?highlight=_meta > As always, feel free to ask questions. > > Daniel > > On Monday, May 26, 2014 1:26:27 AM UTC+2, Daniel Pyrathon wrote: > Hi Josh, > > The meta API specified in the docs > (https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt) > is the current API. I have documented this in order to understand more of > the current implementation and it will be good to show a comparison when a > new meta API will be approved. > > My current proposal (https://gist.github.com/PirosB3/371704ed40ed093d5a82) > and it will be discussed tomorrow with Russell. I will post as soon as I have > updates. > > Daniel Pyrathon > > On Saturday, May 24, 2014 10:37:49 AM UTC+2, Josh Smeaton wrote: > Hi Daniel, > > Nice work putting that document together. Is the meta document you put > together the current API or is it the API you are proposing? If the latter, a > few suggestions (and if others disagree, please point that out): > > - Remove all mention of caching. That should be an implementation detail > only, and not a requirement for other implementations. > - the *_with_model methods really rub me up the wrong way. I would prefer > always returning the _with_model variant, and letting the caller discard the > model if they don't need it. > - I'm not a fan of virtual and concrete fields, though I have to admit I'm > not sure how they're different, especially in the context of different > implementations. > - Not sure that m2m should be differentiated from related. > - init_name_map should be an implementation detail. > - normalize_together should be an implementation detail. > > Regards, > > Josh > > On Saturday, 24 May 2014 05:05:02 UTC+10, Daniel Pyrathon wrote: > Hi all, > > In the last days I have built a documentation of the current state of > Options. Based on feedback and prototyping I have thought of a potential > interface for _meta that can solve the issues currently present, such as > redundancy (in code and in caching systems). The interface has also been > thought to be maintainable and is a base that can be used to create custom > meta stores. > Obviously this is far from perfect, It will need many iterations and maybe it > is too complex. I would really love to gain as much feedback as possible so > it can be discussed with Russell and the community on Monday. > > The documentation of _meta can be found here: > https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt > I will be refining the document in the next days, I will also be publishing > the docs on a webserver and will be linking a URL soon. > > My proposal has been published here: > https://gist.github.com/PirosB3/371704ed40ed093d5a82 > In the next days I will be iterating over the feedback gained, and based on > one very interesting suggestion on IRC, I will try to see how my API syntax > looks in modelforms.py. > > As said previously, and feedback is greatly appreciated. > > Hi from Pycon IT! > > Daniel Pyrathon > > On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote: > Best of luck! > > On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote: > Hi All, > > Today I will be starting my weekly updates on my SoC project: refactoring > Meta to a stable API. For anyone who missed out, you will be able to view it > here: > https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit > > This week is the first official week of SoC. Me and my mentor (Russell) are > initially approaching the work in the following way: > Document the existing Meta API > For each endpoint, document the following: > - Input parameters and return type > - Caching pattern used > - Where it's called from (internally and externally to Meta) > - Why is it being called > - When is it being called > > Propose an initial refactor plan > Once the documentation has been done, I should have a better idea of the > current implementation. This will allow me to mock a proposed implementation > that will be reviewed at my next update call, on Monday. > My next update will be posted on Friday, just to make sure the community is > informed of my progress. For any major updates that require community > approval, I will be creating separate threads. > My name on the internet is pirosb3, so if you want to have a chat about my > progress feel free to contact me! The branch I am currently working on is > https://github.com/PirosB3/django/tree/meta_documentation > > Regards, > Daniel Pyrathon > > On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote: > Best of luck! > > On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote: > Hi All, > > Today I will be starting my weekly updates on my SoC project: refactoring > Meta to a stable API. For anyone who missed out, you will be able to view it > here: > https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit > > This week is the first official week of SoC. Me and my mentor (Russell) are > initially approaching the work in the following way: > Document the existing Meta API > For each endpoint, document the following: > - Input parameters and return type > - Caching pattern used > - Where it's called from (internally and externally to Meta) > - Why is it being called > - When is it being called > > Propose an initial refactor plan > Once the documentation has been done, I should have a better idea of the > current implementation. This will allow me to mock a proposed implementation > that will be reviewed at my next update call, on Monday. > My next update will be posted on Friday, just to make sure the community is > informed of my progress. For any major updates that require community > approval, I will be creating separate threads. > My name on the internet is pirosb3, so if you want to have a chat about my > progress feel free to contact me! The branch I am currently working on is > https://github.com/PirosB3/django/tree/meta_documentation > > Regards, > Daniel Pyrathon > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/django-developers. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/78f45c61-9626-4aa9-a854-64b11ba44572%40googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/4A5680B3-170D-4D31-8B70-9007F2AA18FD%40polytechnique.org. For more options, visit https://groups.google.com/d/optout.
