Re: [RFC]GSoC 2018 Test framework improvement

[Su Billy]

Dear Josh and Rotund,

Sorry for late reply, since I was surveyed for what concrete feature I want 
to implement.
According to some ticket, such as #6712 
 #17365 
 #13873 
, some developer want to 
define their own test suite instead of let test runner discover and 
construct one, and therefore the #tagging-tests 
 was 
implemented, but if we need to change the test suite, we need to add or 
edit tag in many place in the test script, therefore, I came up a new 
feature is that provide a way to let developer define their frequently used 
tests as test suite in a file.

Although it already exists ``tag`` for developer to categorize tests, 
within this feature developer could have a more convenient way to define 
the test suite for their project to let their user easier to run test 
without need to memorize which tags or test modules needs to specified in 
command line, because it already defined in test suite file. 

The* test suite file* may be written in JSON format or YAML fomat(I don't 
know which one is better)
eg:
There are two apps in this project, ``mysite`` and ``polls``.
The structure of the project may be
├── db.sqlite3
├── manage.py
├── mysite
├── test_suite.json
│   ├── __init__.py
│   ├── settings.py
│   ├── tests.py
│   ├── urls.py
│   └── wsgi.py
└── polls
├── admin.py
├── apps.py
├── __init__.py
├── migrations
│   └── __init__.py
├── models.py
├── tests.py
├── urls.py
└── views.py
The content in test_suite.json is
{
  "Fast test": {
"mysite":{
  "tag": ["fast", "quick"]
  "tests": ["urls", "views"]
},
"polls":{
  "exclude tag": ["low", "detail"]
  "tests": ["template","login"]
}
  },
  "Detail test": {
"mysite":{
  "tests": "all"
},
"polls":{
  "tests": "all"
}
  }
}
The usage of running these suite is simply use
*./manage.py run test --suite="Fast test"*

The ``mysite`` and ``poll`` are the app names in a project, tests field in 
test suite file may be module, TestCase class or even single test in 
TestCase. As for the ``tag`` and ``exclude tag`` which uses the tag 
decorator in original tag system.

This feature seems to have many behaviors similar to original tag, but I 
think this test suite feature will make developer easier to define their 
suite without add or edit many tags in script.

Sincerely 

Billy Su

Rotund於 2018年3月17日星期六 UTC+8上午4時38分16秒寫道：
>
> Django does have a testing framework that's based on the unittest module 
> from Python standard library. That stated, I've seen people use nose and 
> other frameworks in the past. (More info at: 
> https://docs.djangoproject.com/en/2.0/topics/testing/)
>
> I feel there is always a need for better helper utilities to make writing 
> tests so easy that people have no excuse NOT to write them. That stated, 
> the GSoC's that have been accepted in the past have been very specific 
> about what they are proposing and milestones as to what will be done and 
> when. There is no time in schedule to discover or define the scope. 
> Obviously, there will always be issues that come up and changes are 
> inevitable, but Django has a limited number of slots and wants to choose 
> projects that not only provide good benefits to the project but are likely 
> to succeed as well,
>
> Hopefully this is helpful. You may be on the right track, but you'll have 
> to not only sell on what you will do but that you're the correct person to 
> do it and you understand the scope.
>
> On Fri, Mar 16, 2018 at 12:56 PM Su Billy > 
> wrote:
>
>> Hello developers, 
>>
>> My name is BING-LI SU from Taiwan, you can simply call me Billy.
>>
>> *Background*
>> I'm an undergraduate student at National Chiao Tung University, major in 
>> Computer Science.
>> I have been contributed to several open source projects for one year, 
>> such as CDNJS, pyqtgraph, and I host my own project using Tornado Web to 
>> build a simple digital signage system which contain a little bit unit test, 
>> CI, and docker image for fast deployment.
>> I have know Django when I chose the framework of the digital signage, but 
>> I haven't used it util these few days, knowing that Django is a more 
>> sophisticated and handy framework compared to Tornado. In my opinion, the 
>> best way to be familiar with a project is to understand the test of the 
>> project (if there has one), therefore I choose test framework as my 
>> proposal idea.
>>
>> *Ideas*
>> I'd like to add some new feature to the test framework, it is mainly 
>> about provide some groups of common used tests for user to choose, and 
>> maybe provide hook function to add their own test before, after, during 
>> each group of tests.
>>
>> *Plan*
>> Here is my draft plan to achieve above ideas
>> 1. To make me better understand the whole 

Re: GSoC 2018

[Aymeric Augustin]

Hello,

> On 18 Mar 2018, at 09:45, Manasvi Saxena  wrote:
> 
> I simply intend to create an option for the developer to use python for 
> templating rather than DTL or jinja.
> Is Python fast compared to any other templating language? Yes.

No.

Jinja2 compiles to Python bytecode so it's as fast as Python — and probably 
faster than what you'd write in three months because a lot of effort went into 
optimizing its performance over the years.

Here's an analogy: "is assembly fast compared to any other programming 
language?" The answer to that is "No" as well.

> Why do we need to process the data using any other language when we are 
> building our app in python.

20 years of industry show that backend developers prefer template languages for 
writing templates. Even PHP, which is originally a template language, saw the 
development of Smarty.

And frontend developers want something that isn't too remote from HTML.

> Sure DTL or Jinja are python-like, but they are not python.
> I intend to develop every functionality offered by jinja or DTL in my python 
> library and to make it more flexible to use.

I'd like an examples of something useful that Jinja2 can't do and where more 
flexibility is needed. Jinja2 provides logic constructs and arbitrary function 
calls. (So does DTL but with less convenient APIs.)

> Plus the advantage over the current templating system would be speed.
> Returning an HTML page with large data is faster than rendering it using DTL.
> And templating in python would give an upper hand over templating in DTL.
> Yes, the Django templating language has been developed over years but making 
> another option for the developer to choose from does not have any harm.

A library that meets all these requirements exists already: Jinja2. I added 
first class support for Jinja2 in Django. (Even though additional integration 
work could be envisioned, I haven't heard demand for that.)

> On the Google Summer of Codes submit application page, Django is asking to 
> choose from two option.
> 1. Optimization
> 2. Add new feature
> 
> This is the new feature I wish to add.
> It has advantages over the previous one.

I haven't seen those advantages and I think that's because you don't know the 
previous one well enough.

> For now, I'm not asking Django to replace DTL, I'm simply asking to consider 
> my idea and provide me with an opportunity of developing it under the 
> guidance and experience of a mentor.
> There will be three evaluations of the project, reject the idea in any of the 
> evaluation if doesn't seem to be promising. But at least provide with an 
> opportunity to prove it at least.


Sorry if I'm being blunt here but there've been a dozen messages already 
indicating that you're going into a dead end. I'm afraid we're wasting time.

Your prejudice against DTL and Jinja2 is incorrect. I'm not taking issue with 
the solutions you propose but with the problem you'd like to address.

To discuss solutions, we'd have to agree there's a problem worth solving. Even 
after you proposed multiple angles, I'm still not seeing it.

Best regards,

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/875546D6-FE16-4806-AA44-C779E33D3699%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: GSoC 2018

[Manasvi Saxena]

Hello Sir,

I like theses idea less and less evertime.
>
> What is your usecase for a smarter rendering engine ? You can already 
> compute complex things in the view, store it in the rendering context and 
> simply display it in the template.
>
> The first idea was only slightly in the wrong direction, but could produce 
> some valuable work. Producing HTML using python is awkward. But producing 
> interface is another matter. If your library provide ban interface like "on 
> the left there is a listView widget, connected to this queryset. When an 
> item is selected, it will use that function to generate a rich widget for 
> editing the model and display the widget on the right".
>
> That kind of library would be a lot more like writing gtk or Qt 
> application. There are other webframework doing that, I'm thinking of 
> nitrogen, an Erlang framework. Maybe toga-django already try to produce 
> such library
>


I simply intend to create an option for the developer to use python for 
templating rather than DTL or jinja.
Is Python fast compared to any other templating language? Yes.
Why do we need to process the data using any other language when we are 
building our app in python.
Sure DTL or Jinja are python-like, but they are not python.
I intend to develop every functionality offered by jinja or DTL in my 
python library and to make it more flexible to use.
Plus the advantage over the current templating system would be speed.
Returning an HTML page with large data is faster than rendering it using 
DTL.
And templating in python would give an upper hand over templating in DTL.
Yes, the Django templating language has been developed over years but 
making another option for the developer to choose from does not have any 
harm.

On the Google Summer of Codes submit application page, Django is asking to 
choose from two option.
1. Optimization
2. Add new feature

This is the new feature I wish to add.
It has advantages over the previous one.

For now, I'm not asking Django to replace DTL, I'm simply asking to 
consider my idea and provide me with an opportunity of developing it under 
the guidance and experience of a mentor.
There will be three evaluations of the project, reject the idea in any of 
the evaluation if doesn't seem to be promising. But at least provide with 
an opportunity to prove it at least.

Regards,
Manasvi Saxena

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/d33cf67b-658e-4891-9da5-3e56cbcbc068%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: On adding comments to database schema

[Aymeric Augustin]

Hello,

My gut feeling is that I'd rather implement this in the migrations engine (your 
option 1) than bolt it on (the option 2).

Disclaimer — I don't know the migrations engine all that well.

Best regards,

-- 
Aymeric.



> On 18 Mar 2018, at 07:12, vanadium23  wrote:
> 
> Hi, community.
> 
> I've been stuck at realization, because Postgres and oracle have a syntax 
> like comment on {table}.{column} for storing comments, so this needs to be 
> done after table/column creation.
> 
> So there are two ways:
> 1. Add it to post migrate signal, as for content types. But I can implement 
> it as a third-party lib
> 2. Add this SQL after database creation in schema.py
> Which way is better?
> 
> воскресенье, 11 марта 2018 г., 11:34:46 UTC+3 пользователь vanadium23 написал:
> That's good. I'll prepare the patch for review.
> 
> суббота, 10 марта 2018 г., 10:27:30 UTC+3 пользователь Aymeric Augustin 
> написал:
> I think we could implement that feature.
> 
> (Yes I changed my mind from six years ago when I wontfix'd the ticket.)
> 
> -- 
> Aymeric.
> 
> 
> 
>> On 8 Mar 2018, at 11:00, Jani Tiainen > wrote:
>> 
>> Oracle supports comments as well.
>> 
>> 7.3.2018 2.52 ip. "vanadium23" > kirjoitti:
>> Hello, fellows.
>> 
>> There was once a proposal about ability to add comments to table/columns in 
>> postgres: https://code.djangoproject.com/ticket/18468 
>> 
>> I re-read discussion in ticket, and it has ended with another proposal, that 
>> now migrations is within Django, so we can add this feature.
>> 
>> I want to make it as a third-party libs, but has come to thought that need 
>> to extend SchemaEditor, but it's not good for maintainability.
>> Also, not only postgres has this feature, but also MySQL.
>> 
>> Motivation for this feature is that some sort of users can view code 
>> comments without access to VCS of the project.
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com <>.
>> To post to this group, send email to django-d...@googlegroups.com <>.
>> Visit this group at https://groups.google.com/group/django-developers 
>> .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/484264c0-b7d1-4264-b8b5-8857847f6b53%40googlegroups.com
>>  
>> .
>> For more options, visit https://groups.google.com/d/optout 
>> .
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com <>.
>> To post to this group, send email to django-d...@googlegroups.com <>.
>> Visit this group at https://groups.google.com/group/django-developers 
>> .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/CAHn91oesnBvY-mWXoHWa3sf8Vri%2B1qwRHat1yF%2B8Sdm0pU-LhQ%40mail.gmail.com
>>  
>> .
>> For more options, visit https://groups.google.com/d/optout 
>> .
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/9ff6c9f5-045e-4bc7-8331-060471b012e2%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googl

Re: GSoC 2018

[ludovic coues]

On Sat, 17 Mar 2018, 21:55 Manasvi Saxena,  wrote:

> Hello Sir,
>
> I'll give some background. The Django Templating Language was very
>> specifically designed to NOT allow putting business logic or allow calling
>> arbitrary functions in the template. Jinja2 loosened up that a bit, but it
>> still doesn't let you do whatever you want. There's a couple reasons, but
>> one is performance. Django gains a lot of optimizations because all
>> database queries can happen in a small region. It also means one cannot
>> potentially have 3 functions that each request the same data thus doing 3
>> DB queries, which is very slow.
>>
>> It is also generally "bad form" to mix your business logic and your
>> display logic. I get that there's some times where it's much simpler to
>> break that separation. That is part of why Jinja2 gained some traction
>> (though I personally really like the better whitespace control). It also
>> creates a much better chance for a security flaw and maintenance issues.
>> Obviously, there is nothing stopping you from creating such a templating
>> language. It may get really popular.
>>
>> Finally, I'll mention that there's nothing in Django that stops you from
>> just changing the template language used. It's as simple as not using the
>> render shortcut function. The joy of Python is that you just need to
>> generally match the interface.
>>
>> Not to be mean, but I wanted to point out what's being proposed. Your
>> proposal is that Django pays you to create yet another templating language
>> that they have to support. This leads to three separate paths that the code
>> can take. There then needs to be some ability to cross communicate between
>> all three languages. Then there's the "sales" end of explaining why there's
>> 3 templating languages and why one needs to choose another. One of the real
>> selling points of Django over other frameworks is that they're opinionated
>> and chose a specific set of tools. I started back in Pylons/TurboGears, and
>> I would spend a long time looking over which DB layer to chose and which
>> templating language to chose. In the end, I'll say that it was refreshing
>> to go to Django and have some of that chosen for you and to just start
>> working.
>>
>> I really do wish you the best of luck if you attempt to make this. I'm
>> sure there is a need, but I personally do not see it being a direct fit
>> into Django.
>>
>
>
> I have decided to drop my earlier idea to create a python to HTML library.
>
> But I still believe that Django templating language(DTL) or Jinja has some
> limitations and drawback. First of all, as I said earlier for large data
> set it takes more time. Secondly, it has limited scope for flexibility
> while implementing logic.
> The solution I was offering earlier for it was deeply flawed as mentioned
> above by you.
> I have decided to create an approach that (a) can cross
> communicate between jinja and DTL. (b) is easy to maintain and use.
>
> What I'm planning to do is to provide a library that can inject data into
> the template before rendering the template.
>
> For now, assume that there will be a function, only a single function, let
> us call it
>
> *populate(replace_with, new_content)*
>
> Now inside the template just like DTL ask us to write {{variable_name}} we
> now have to write let say
>
> **
>
> The above function when called will look into the file for the
> *"replace_with"  *variable and will replace it will with *"new_content".*
> This was just a simple example, simplest example I would say. The real
> functionality will be different from it.
>
> *"populate"* function can replace HTML content, plain text, or even a
> list of *new_content*. Anything that is within our custom tag will be
> replaced with the *new_content.*
>
> Benefits of this approach include:
> 1. Generation of HTML page with dynamically populated data and simply
> serving it through *HttpResponse() *[Fast maybe].
> 2. Flexibility in implementing any logic. As soon as you have the data
> call *populate() *and place it inside the template.
> 3. DTL can still be used for providing any other functionality if the need
> persists. However, I intend to cover every functionality offered by DTL in
> it.
> 4. While for some cases DTL works in a really optimized manner, it will be
> in the hands of the developer to chose which approach to follow.
> 5. It does not require the developer to learn anything extra other than
> DTL (or if successfully implemented, not even DTL).
>
> Things I'll work on is to
> 1. Find an efficient way to implement *populate() *function. While we do
> have a naive approach to finding next and replacing. I intend to find an
> optimized way to implement this function.
> 2. Making the library compatible with the existing Django features. No
> changes will be made to any existing features. It'll act as an API and will
> work independently of any other Django library.
>
> Please let me know what do you think of this new approach so tha