Re: Django ORM query syntax enhancement

[Aric Coady]

+1.  I implemented much the same thing for part of django-model-values 
, but went with F 
expressions  
as the basis instead.  Primarily because F expressions already support 
operator overloading and are a natural intermediate object;  from there one 
can create Q, OrderBy, and Func objects.

In []: from model_values import F

In []: F.user.created
Out[]: FExpr(user__created)

In []: F.user.created >= 0
Out[]: 

In []: F.user.created.min()
Out[]: Min(FExpr(user__created))

In []: -F.user.created
Out[]: OrderBy(FExpr(user__created), descending=True)

In []: F.text.iexact('...')
Out[]: 




On Thursday, October 6, 2016 at 10:04:56 AM UTC-7, Alexey Zankevich wrote:
>
> Hey all,
>
> Just want to announce recent changes in Django ORM Sugar library, which 
> might be useful in future Django query syntax enhancement (if ever happens).
> 1. Now it supports indexes and slices:
>
> >>> Q.data.owner.other_pets[0].name='Fishy'
> Q(data__owner__other_pets__0__name='Fishy')
> >>> Q.tags[0] == 'thoughts'
> Q(tags__0='thoughts')
> >>> Q.tags[0:2].contains(['thoughts']) 
> Q(tags__0_2__contains=['thoughts'])
>
>
> 2. Previously, it was only possible to call defined method (like it is 
> done for *is_not_null()* function) or registered with decorator. Now if 
> you try to call unknown function of sugar Q, it will internally pass 
> function name and append it as *__functionname *to lookup field:
>
> >>> Q.user.username.contains('Rodriguez')
> Q(user__username__contains='Rodriguez')
>
>
> There is no such function "contains" anymore in sugar Q, however, it just 
> adds *__contains* to lookup field and passes parameter to it.
>
> 3. It will pass a tuple to lookup if multiple arguments passed:
>
> >>> Q.user.create_datetime.range(d1, d2)
> Q(user__create_datetime__range=(d1, d2))
>
>
> I think the library is at the final state now and isn't going to get new 
> substantial features, but responses are highly appreciated.
>
> Regards,
> Alexey
>
>
> On Sunday, August 16, 2015 at 4:18:26 PM UTC+3, Alexey Zankevich wrote:
>>
>> Hi all,
>>
>> This topic is related to the current ORM query syntax with underscores.
>> There are lots of arguing related to it, anyway it has pros and cons.
>>
>> Let's take a concrete example of querying a model:
>>
>> >>> 
>> GameSession.objects.filter(user__profile__last_login_date__gte=yesterday)
>>
>>
>> Pros:
>>
>> 1. The syntax is easy to understand
>> 2. Can be extended with custom transforms and lookups
>>
>> However, there are several cons:
>>
>> 1. Long strings is hard to read, especially if we have fields with 
>> underscores.
>> It's really easy to make a mistake by missing one:
>>
>> >>> 
>> GameSession.objects.filter(user_profile__last_login_date__gte=yesterday)
>>
>> Not easy to catch missing underscore between user and profile, is it? Even
>> though, it's not easy to say whether it should be "user_profile" 
>> attribute or
>> user.profile foreign key.
>>
>> 2. Query strings can't be reused, thus the approach violates DRY 
>> principle.
>> For example, we need to order results by last_login_date:
>>
>> >>> 
>> GameSession.objects.filter(user__profile__last_login_date__gte=yesterday) \
>> .order_by('user__profile__last_login_date')
>>
>> We can't keep user__profile_login_date as a variable as in the first part 
>> of the
>> expression we use a keyword argument, meanwhile in the second part - just 
>> a 
>> string. And thus we just have to type query path twice.
>>
>> 3. Lookup names not natural to Python language and require to be 
>> remembered or
>> looked up in documentation. For example, "__gte" or "__lte" lookups tend 
>> to be
>> confused with "ge" and "le" due to similarity to methods "__ge__" and 
>> "__le__".
>>
>> 4. Lookup keywords limited to a single argument only, very inconvenient 
>> when
>> necessary to filter objects by range.
>>
>> I was thinking a lot trying to solve those issues, keeping in mind Django
>> approaches. Finally I came up with solution to extend Q objects with dot
>> expression syntax:
>>
>> >>> GameSession.objecs.filter(Q.user.profile.last_login_date >= yesterday)
>>
>> Q is a factory instance for old-style Q objects. Accessing attribute by 
>> dot
>> returns a child factory, calling factory will instantiate old-style Q 
>> object.
>>
>> >>> Q
>> 
>>
>> >>> Q.user.profile
>> 
>>
>> >>> Q(user__name='Bob')
>> 
>>
>> It overrides operators, so comparing factory with value returns a related 
>> Q
>> object:
>>
>> >>> Q.user.name == 'Bob'
>> 
>>
>> Factory has several helper functions for lookups which aren't related to 
>> any
>> Python operators directly:
>>
>> >>> Q.user.name.icontains('Bob')
>> 
>>
>> And helper to get query path as string, which requred by order_by or
>> select_related queryset methods: 
>>
>> >>> Q.user.profile.last_login_date.get_path()
>> 'user__profile__last_login_date'
>>
>> You can check implementation and more examples 

Re: LiveServerTestCase change in Django 1.11 means can't run functional tests against external server?

[Alex Riina]

Thanks for including a link to the TDD tutorial!

As far as I can tell, the author wants to use `--liveserver` to point his 
functional tests at an external server so he can run his tests through 
nginx after running the same tests in his normal test environment. Based on 
the shell prompts in 
http://chimera.labs.oreilly.com/books/123400754/ch08.html#_creating_the_database_with_migrate
 
I think he intends to have the database and code running on his staging 
server while he runs his tests locally. He also avoids the 
TransactionTestCase truncation operations by shorting out the 
LiveServerTestCase setUp / tearDownClass.

Simplest way?
The simplest way I can think of to resolve the issue is to read the 
external liveserver out of os.env instead of sys.argv. Then you can follow 
along exactly.
$ export EXTERNAL_SERVER='http://superlists-staging.ottg.eu'
$ python manage.py test

`allow_database_queries = False` would make me trust the setUpClass / 
tearDownClass overrides more but my main issue with this approach is that 
it does a lot of extra work: 

   1. run tests locally
   2. deploy to your code to your staging server
   3. runserver with fixed port on staging server
   4. run tests locally with LiveServerTestCases running against staging 
   server

It would be nice for the last run-through to _only_ run the functional 
tests and not have to setUp the local database or run the other tests 
against your local branch.


Make unittest run functional tests remotely

# app/test_functional_something.py
import os

if 'DJANGO_SETTINGS_MODULE' in os.env:
from django.tests import StaticLiveServerTestCase
FunctionalTest = StaticLiveServerTestCase
else:
class FunctionalTest(unittest.TestCase):
live_server_url = 'http://superlists-staging.ottg.eu'


then you can either run the tests via the DiscoveryRunner on the local 
database
$ python manage.py test

or via the normal unittest runner on your remote server
$ python -m unittest discover -p 'test_functional_*.py'

Unfortunately, you'll lose a lot of power in not being able to reuse your 
form code, assert anything about the database state on the remote server, 
access django assertion methods, ... 
At that point, I'd probably just run the tests on the remove server and do 
something like


Hardcode the port
class FixedPortServerThread(LiveServerThread):
def _create_server(self, port):
 if os.env.get('THROUGH_NGINX'):
 return super(FixedPortServerThread, self)._create_server(port=
8000)
 else:
 return super(FixedPortServerThread, self)._create_server(port=
port)


class MyLiveServerTestCase(LiveServerTestCase):
server_thread_class = FixedPortServerThread


On Saturday, October 15, 2016 at 9:32:54 AM UTC-4, Andrew Wall wrote:

> Very much appreciate the Django framework.  
>
> I noticed in the docs 
> 
>  
> for Django 1.11 that the *DJANGO_LIVE_TEST_SERVER_ADDRESS* environmental 
> variable is slated to be removed, along with the accompanying 
> python manage.py test *--liveserver* option.  I'm concerned that without 
> the ability to specify a remote IP address using --liveserver, it will no 
> longer be possible to run functional tests using selenium against an 
> external server.  I learned this technique from Harry Percival's excellent 
> TDD with Python (see 
> http://chimera.labs.oreilly.com/books/123400754/ch08.html#_configuring_domains_for_staging_and_live),
>  
> where the test command is called from the local machine but you pass in the 
> external IP.  Will there be a different way to do this in Django 1.11? 
> Perhaps the change to bind LiveserverTestCase to port zero by default can 
> be made while retaining the option to pass in --liveserver?  Realize the 
> release is a ways away but would appreciate any help as I've come to rely 
> on this method to test server deployments.  Thank you!
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/df0065f9-222f-4b22-b571-321d79d87a99%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django Channels for Client - Server asynch connection

[Andrew Godwin]

Hi Armitpal,

django-developers is for discussions about development on Django, Channels
and other Django projects themselves, not the use of them. For general
discussion we have an IRC channel at #django-channels on Freenode or
django-us...@googlegroups.com; for specific requests or bugs you can open
GitHub issues against the project.

Since I see you have cross-posted this question to both lists (please don't
do that), I will answer on django-users.

Andrew

On Sun, Oct 16, 2016 at 9:37 AM, Amirtpal  wrote:

> Hi Django folks,
>
> Just a couple of questions on using Django Channels
>
> a) Assuming I have remote and distributed clients(e.g. custom code
> deployed inside web applications) and I need that to maintain an asynch
> flow over https to our backend django server in the cloud,
> how could I use Django Channels for this above use case ?
> b) If these clients are running inside web applications behind a firewall
> and the only outbound communication is over 80 and 443 via https would
> websockets still work ?
> c) Is there a concept of a producer and consumer in django channels ?
> d) Any sample code that could be included in remote web applications in
> order to to enable them to communicate in real time via asyn flow with a
> cloud hosted django server ?
>
> Thanks and appreciate your reply.
>
> -Amirtpal
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/6292d198-8a06-446e-b682-
> cc79e75ef6f3%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1uoA9U_7xWO2nyBrSfmYixQ0-VorLbFxVp%3DxUVL5DSWfpw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Django Channels for Client - Server asynch connection

[Amirtpal]

Hi Django folks,

Just a couple of questions on using Django Channels

a) Assuming I have remote and distributed clients(e.g. custom code deployed 
inside web applications) and I need that to maintain an asynch flow over 
https to our backend django server in the cloud,
how could I use Django Channels for this above use case ? 
b) If these clients are running inside web applications behind a firewall 
and the only outbound communication is over 80 and 443 via https would 
websockets still work ?
c) Is there a concept of a producer and consumer in django channels ?
d) Any sample code that could be included in remote web applications in 
order to to enable them to communicate in real time via asyn flow with a 
cloud hosted django server ?

Thanks and appreciate your reply.

-Amirtpal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6292d198-8a06-446e-b682-cc79e75ef6f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Composite fields

[Victor Porton]

I have implemented something like this: 
https://bitbucket.org/portonv/composite-fields

(I did my implementation before reading my thread.)

Please feel free to use and modify my code.

Well, now I should read this thread and 
https://github.com/django/deps/blob/master/draft/0191-composite-fields.rst

On Wednesday, March 4, 2015 at 3:16:37 PM UTC+2, Thomas Stephenson wrote:
>
> Considering the past two proposals I've made here, I doubt I'll get more 
> than an echo chamber effect on this one.
>
> For the past week or so I've spent a bit of time on a feature I've always 
> wanted to see land in django -- composite fields. The tickets have been 
> open in the bug tracker for quite some time (and there's a few related 
> ones, such as multi-column primary keys that can all be killed with the one 
> stone).
>
> The work is available on this branch of my fork of django 
>  for the moment 
> -- I haven't opened up a PR yet because there's still some features that 
> are still to be implemented that will be explained below, but I want to 
> give everybody a chance to tell me where I can stick it before I spend 
> *too* much time on it.
>
> So, without further ado, the proposal.
>
>
> Composite Fields - Implemented
>
> A composite field is an encapsulation of the functionality of a subset of 
> fields on a model. Composite fields can be defined in one of two ways:
>
> 1. Statically declared composite fields
>
> A statically declared composite field is defined in the same way a django 
> model is defined. There are two customisable transformation functions, 
> CompositeField.value_to_dict(self, value) and 
> CompositeField.value_from_dict(self, value) which can be used to associate 
> the field with a python object.
>
> All the serialization functions are implemented via the implementations of 
> the subfields.
>
> For example,  
>
> class MoneyField(models.CompositeField):
>currency_code = models.CharacterField(max_length=3)
>amount = models.DecimalField(max_digits=16, decimal_digits=4)
>
>## Overriding __init__ can be used to pass field parameters to the 
> subfields
>
>def value_from_dict(self, value):
>if value is None:
>   return None
>return Money(**value)
>
>def value_to_dict(self, value):
>   if value is None:
>  return None
>   return {attr: getattr(value, attr) for attr in ('currency_code', 
> 'amount')}
>
> 2. Inline composite fields.
>
> An inline composite field is declared at the field definition site on the 
> body of a model, by providing the subfields as the 'fields' argument of the 
> CompositeField constructor. There are no transformation parameters 
> available to override when declaring a composite field in this fashion -- 
> the value of the field is always available as a python `dict` as an 
> attribute on the MyModel
>
> class MyModel(models.Model):
> id = models.CompositeField(fields = [
>('a', models.IntegerField()),
>('b', models.CharField(max_length=30)
> ], primary_key=True)
>
> This method for defining composite fields has a few drawbacks, but can be 
> useful if the only reason to add the composite field to the model was to 
> implement a unique_together or index_together constraint *
>
> * Although it's still possible to do that directly on class Meta.
>
> 3. Null
>
> Setting the value of a multi-column field to NULL is different than 
> setting any of the individual subfields to NULL. But there are cases (e.g. 
> Money) where we would like to be able to set `null=True` on a composite 
> field, but still retain 'NOT NULL' constraints on each of the subfield 
> columns.
>
> To solve this problem, every table which implements a CompositeField will 
> also add an implicit (semi-hidden) `isnull` subfield on the attribute, 
> which keeps track of whether it is the value of the composite field that is 
> null, or any of the particular subfields.
>
> 3. Querying.
>
> The syntax for querying over the subfields of a composite field will be 
> familiar to anyone who has queried over a relationship attribute in django.
>
> model.objects.filter(price__currency_code='USD', 
> price__amount__lt=Decimal('50.00')).all()
>
> In addition, composite fields can be queried via EXACT and IN lookups. It 
> is possible to implement custom lookups for specific statically defined 
> fields, but not recommended and not part of the official API.
>
> 4. Restrictions
>
> The following restrictions are currently imposed on the use of composite 
> fields. None of these are restrictions that can't be worked around in 
> future extensions, but they're restrictions which considerably simplify 
> both the implementation and API.
>
> - No related fields as a subfield of a composite field
> - No nested composite fields
> - No inheritance of composite fields (apart from inheriting from 
> CompositeField itself).
>
> 5. Changes to the Field API
>
> As discussed in the other thread I