Re: Propose Parallel tasks for GSOC
On Thu, Mar 25, 2010 at 11:34 PM, Rodrigo Hübner wrote: > Hey all! > > I have a project in python with implicit threads. For exemple: > > > @task > def foo(): > pass > > > In this code, every time that i call the "foo" function in my code, it > will execute in different threads... > > My proposal is to specify the some idea in view functions or other > function/methods that it can execute in parallel! > > I attended a lecture by Jacob last year and he said that Django needs > much of a project that parallels tasks. > The general purpose and have something like what I said ... > > What do you think? > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-develop...@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > I think this problem is pretty thoroughly solved by celery at this point, I'm not sure that there's a lot of value that could be added, unless you have a specific, radically different, architecture to propose. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Voltaire "The people's good is the highest law." -- Cicero "Code can always be simpler than you think, but never as simple as you want" -- Me -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: GSoC: Data importation class
On Thu, 2010-03-25 at 11:34 -0500, Malcolm Tredinnick wrote: > That's not a correct statement, since Django models can often be used to > proscribe conditions on new data that is created via the web app, yet > those conditions might not be required for the universal set of data > that already exists. For example, webapp-generated data might always > require a particular field, such as the creating user, to be filled in, > whilst machine-generated data would not require that. Don't equate > validation conditions at the model level with constraints on the data at > the storage level. In my opinion, the model validation should be the same as the storage level. Instead, I would say, "Don't equate validation conditions at the view level with constraints on the data the model level." This might be a bit off-topic for this thread, but legacy data is why I wish frameworks supported some concept of warnings in their validation code. Richard signature.asc Description: This is a digitally signed message part
Propose Parallel tasks for GSOC
Hey all! I have a project in python with implicit threads. For exemple: @task def foo(): pass In this code, every time that i call the "foo" function in my code, it will execute in different threads... My proposal is to specify the some idea in view functions or other function/methods that it can execute in parallel! I attended a lecture by Jacob last year and he said that Django needs much of a project that parallels tasks. The general purpose and have something like what I said ... What do you think? -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Propose Parallel tasks for GSOC
Hey all! I have a project in python with implicit threads. For exemple: @task def foo(): pass In this code, every time that i call the "foo" function in my code, it will execute in different threads... My proposal is to specify the some idea in view functions or other function/methods that it can execute in parallel! I attended a lecture by Jacob last year and he said that Django needs much of a project that parallels tasks. The general purpose and have something like what I said ... What do you think? -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
GSoC Project: Enhancedaut.user
Hi everyone, I removed my previous post from the group, because of bad formatting, sorry for that. I'm planning to apply for the GSoC Enhanced auth.user project. I worked on several Django based projects before (hackerspace os (http:// code.google.com/p/hackerspace-os/) is probably the biggest one (used by the Austrian hackerspace metalab (http://metalab.at)) and a lot of smaller ones) I faced some limitations of the auth framework when I tried to write a custom auth backend for Django on Google App Engine (http:// bitbucket.org/d3f3nd3r/djangoappengine-helpers/). The problem was, that the auth framework forces you to call a login() function when the user logs in and set request.session[SESSION_KEY] and request.session[BACKEND_SESSION_KEY]. When using Google Accounts for authentication no django login method is called. But django.contrib.auth.__init__ get_user() method depends one equest.session[SESSION_KEY] and request.session[BACKEND_SESSION_KEY] so a custom backend with a custom get_user() won't work without a modified middleware. (http:// groups.google.com/group/django-non-relational/browse_thread/ thread/36b760c515b75c0e) My first approach to make the auth system more flexible is to make the methods in django.contrib.auth.__init__.py more flexible, so the methods doesn't force you to call a django login method. All backends will have to extend a base backend class and implement the get_user(), login(),.. methods. The request.session stuff will be moved to the ModelBackend (other model based backends can extend ModelBackend). So it will be easy to create custom backends for Google Accounts,Twitter, Facebook... that work out of the box with the Django auth system. The second task will be to make the user model more extendable. I think the way suggested by David Danier ( ticket : http://code.djangoproject.com/ticket/3011) is a good one. Move all the login and permission stuff to extra Permission and Auth base classes. A base model class for authentication will only contain an ident field (we dont't need to store user name / password/ email if we use a remote service for authentication, but we have to link the remote service user to to Django user model). The "old" contrib.auth.models.User class will extend the two classes (Permission and Auth) and add all the extra fields (username, first-, last name) so it won't break existing apps. A list of accepted user classes can be passed to the backends. I know this is a tough issue, but I'm really interested in working on the problem. Please post your comments, ideas, suggestions,..., so we can find a good solution. (and maybe a successful proposal for GSoC) Flo -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Re: humanize.ordinal is very English-specific
On Fri, Mar 26, 2010 at 10:57 AM, Sergiy Kuzmenko wrote:
> ... just to clarify:
>> There appears to be some confusion here. The pluralize filter *already
>> exists*. It has existed since Django was open sourced. if you have
>> complex pluralization rules to support, it wont work, but in those
>> cases, you can use {% blocktrans %} in templates, or calls to
>> ungettext() in model/view code.
> I know that. I thought we talking specifically about pluralization
> that does not fit the "0-1-many" rule.
And herein lies the 80% rule.
The pluralize filter isn't perfect. However, it is sufficient to
support the needs of English and most western european languages. For
a developer that only need to support those languages, the template
snippet::
Total: {{ list|length }} object{{ list|length|pluralize:"s" }}
is simpler to use than::
{% blocktrans with list|length as counter %}
Total: {{ counter }} object.
{% plural %}
Total: {{ counter }} objects.
{% endblocktrans %}
As soon as you need to support a language that doesn't match the 0-1-N
rule. then you need to use the second form, because they don't hit the
use case where pluralize can be used. It's even arguable that you
should be using blocktrans in English if you need to say "There is/are
N object(s)", because the pluralize version starts to get as complex
as the blocktrans version.
This is all about making simple things trivial, and complex things
possible. Between the two approaches, we can hit all pluralization
requirements. We could also satisfy all pluralization requirements by
dropping the pluralize filter and forcing everyone to use blocktrans,
but then we would lose a trivial solution for a simple use case.
There's also a legacy issue - pluralize has been in place for a long
time, and removing it now would mean forcing people to update their
fully functional code for no particularly good reason. Pluralize isn't
perfect, but when it works, it works fine. It isn't a huge burden to
maintain, either, so it doesn't really hurt to keep it around.
The same rules apply to ordinal. It works fine for English. There are
a good number of sites that don't have to worry about translating
outside English, and for those sites, ordinal is a perfectly
functional solution. Unlike pluralize, ordinal doesn't work for any
language other than English, so at the moment, a blocktrans based
solution is required for those languages. There will always be
languages that won't be able to use ordinal. However, if the OP (or
anyone else) can propose a simple, low maintenance solution that will
allow ordinal to support more languages than it does currently, I'm
happy to entertain it.
Yours,
Russ Magee %-)
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Re: humanize.ordinal is very English-specific
... just to clarify:
> There appears to be some confusion here. The pluralize filter *already
> exists*. It has existed since Django was open sourced. if you have
> complex pluralization rules to support, it wont work, but in those
> cases, you can use {% blocktrans %} in templates, or calls to
> ungettext() in model/view code.
I know that. I thought we talking specifically about pluralization
that does not fit the "0-1-many" rule.
Cheers!
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Re: humanize.ordinal is very English-specific
Hm, I thought I was specific: use the same approach that GNU's
ngettext does! And apologies for spinning this conversation off the
original subject.
It appears to me that in GNU's ngettext the problem of pluralization
is solved, and what is more, ngettext has the same interface as
Django's ungettext. The trick that makes the whole difference is in
the translation files. GNU's ngettext understands special header
parameters inside those files that define pluralization rules, based
on which ngettext knows which plural form to use. The complexity of
handling plurals will thus be hidden in the translation files while
the programming interface remains unchanged.
An important disclaimer about limitation of what can be done to
improve pluralization. With the proposed approach we can successfully
solve this:
ungettext(
'There is %(count)d timbit available.',
'There are %(count)d timbits available.',
count
)
but not this:
ungettext(
'There is %(count)d %(name)s available.',
'There are %(count)d %(name)s available.',
count
) % {
'count': count,
'name': name
}
See below why.
Going back to the original subject: generating *dictionary forms* for
ordinals is only part of the problem. What we really want to do with
it is to say something like: "This is your {{ord_num}} visit of this
site."
And this is where the real problem starts: interpolation! In many
languages (including some Western European ones but excluding English)
the form of word that being interpolated may depend on the context and
what is even worse the context itself may vary depending on the word
that is being interpolated. This one is a non trivial problem because
it requires a bulk of linguistic information to generate such phrases
on the flight. So I don't think humanize.ordinal can be easily ported
into all other languages. But it could serve well analytic languages
(http://en.wikipedia.org/wiki/Isolating_language#Analytic_languages),
such as English or Chinese.
On Wed, Mar 24, 2010 at 8:49 PM, Russell Keith-Magee
wrote:
> On Thu, Mar 25, 2010 at 8:32 AM, Sergiy Kuzmenko wrote:
>>> If you have any suggestions for how we could expose ungettext better
>>> in templates, I'm all ears. However, given the complexity of
>>> pluralization rules and pluralization usage in the general case, it's
>>> not a simple task. Suggestions welcome.
>>
>> My suggestion is to make pluralization compatible with GNU's ngettext
>> which is also the default implementation of pluralization in Python. I
>> haven't looked at any possible aspect of it but at the first glance it
>> seems that it is possible to extend Django's existing pluralization
>> without breaking backward compatibility. And the complexity of
>> pluralization would be left to translators to handle. Pushing
>> pluralization to template tags is bad for a number of reasons:
>>
>> * hard to built true i18n applications (the developer would probably
>> end up using a lot of conditional statements in the template to cover
>> all languages in which the application may be displayed);
>>
>> * the developer will need to have some knowledge of languages the app
>> is localised to in order to know which language specific tags to use;
>>
>> * not all translations are done in templates; views/forms/models may
>> generate their own messages;
>>
>> * it is quirky and not compatible with the existing Python interface;
>>
>> * it's an attempt to reinvent the wheel;
>>
>> I am not sure at which stage is the option 2, but could we please give
>> this matter some additional consideration before adding this feature
>> to Django?
>
> There appears to be some confusion here. The pluralize filter *already
> exists*. It has existed since Django was open sourced. if you have
> complex pluralization rules to support, it wont work, but in those
> cases, you can use {% blocktrans %} in templates, or calls to
> ungettext() in model/view code.
>
> If you have a *specific* suggestion for how the existing pluralize
> filter can be modified in a backwards compatible way, I'm willing to
> listen. However, allow me to assure you that it isn't trivial. If you
> want to know why, read the gettext docs around the pluralization of
> "files" in Polish.
>
> If you have a *specific* suggestion for how to make pluralization
> easier to use by introducing a new template tag or filter, I'm also
> willing to listen. But you need to make a *specific* suggestion.
>
> On top of all that, the OP suggestion had nothing to do with pluralize
> - it was dealing with ordinals. I only mentioned pluralize as a
> convenient analog for the discussion.
>
> Yours,
> Russ Magee %-)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-develop...@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> ht
Re: GSoC: Data importation class
I feel the need to wade in here, since this is vaguely my area. On 25/03/10 17:47, subs...@gmail.com wrote: The last bit sounds a bit nebulous. You could optimise it by not including any empty files, or be a bit more specific about what the empty files are meant to represent. :) startapp, startproject, et al. I see where you're coming from here; in the final proposal, though, you'd want to follow Malcom's advice and have an actual use case for each file. I'd say you probably only want one, or even none - if your approach is so complicated that even simple use cases need lots of files, you're unlikely to get much traction. South has a target use case of relatively simple changes to schema, and assisting teams maintain synchronicity. However, it isn't long before you're really pushing the limits of South. Take for example one legacy model which needs to be split into two or three current models: South has no answer for this as you may only use defaults for creation of fields (in this case, foreign keys, or potentially OneToOnes, and then what? What if my defaults are based on other values in that record? Already, I'm on my own). By and large, South is an immaculate tool for tracking changes during development. South does have an answer to this - you create the columns as nullable, add in the data, and then alter them back to non-nullable. That's the only way a database is going to let you add a column; they need either a global default or NOT NULL (there are some cases you can do it, but they're really not appropriate for most people). In fact, I'd say this fits perfectly into the South model; one migration to make the two/three new tables, one that moves all the data around using the ORM (something Django developers know, and mostly love), and one to delete the old table. If you only use --auto then yes, it's only good at tracking small changes, but the rest of the power is right there, you just have to actually write code. In theory, yes. But in practice, I've found the shortest way around the mountain is to get it into "SQL-enough" format manually. As for Python, sure, but the more you write this monolithic script the more you realize you're conducting a lot of repetitive work, the mechanics of which are generally re-usable but an implementation which is completely nuanced to your current task. If data importation is something you do a lot, you've probably got a file somewhere holding piecemeal bits that are hopefully vaguely useful to the next project, all the while not being able to fight off the feeling that this general task mirrors a lot of what goes on in forms with clean_somefield() and clean(). Actually, most of my data import scripts are while loops over cursor.fetchall(), which just use the ORM to put in new data. With MultiDB, I probably won't even need the cursor part, I can just loop over the legacy model and insert into the new one. While it might be nice to make this more generic, each of those while loops has slightly different bodies - correcting coordinates here, fixing postcodes there - and the generic bits only take up one or two lines each time. I'd really like to see a more compact or succinct way of doing this, but I'm dubious as to how flexible it would be. I'm more than happy to be proven wrong, however. I'm puzzled by this conclusion. The 'system administration functionality' isn't in any way different to what you'd find in all kinds of projects--South included. I'm not even sure what to say about the 'everything else in your computing life' statement, but I will assume good faith and that you're not alleging I'm presenting this as some kind of crutch for more correct methods. As for detecting 'what is thrashing', there are only so many tasks that can be conducted in the business of moving large gobs of data, some of these tasks (often) bring CPU to 100% and (hopefully never) bring free memory to %0. Things that do one or the other are treated like bugs or avoided. I read your initial proposal here as "code things in a sensible way", not "actively monitor performance and correct on the fly". Using pagination and making sure there's no memory leaks in the code's loops is a great idea, attempting to self-optimise at runtime probably isn't. Andrew -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: GSoC Project: Enhancedaut.user
On Mar 25, 4:19 pm, Russell Keith-Magee wrote: > > First up, this is probably the most ambitious project on the proposal > list. When I warn that this is a tough issue, I'm not kidding. This is > gonna be *hard*. With a capital Huh. There are all sorts of technical > issues, and a couple of political ones too. > > Still interested? Ok. Here's a few quick comments about your proposal. > I know the authentication framework is very important and most of the apps out there on the www rely on it. Of course, this isn't the final proposal, because there are a lot of Django developers out there, with their visions about the framework. So I hope I will be able to collect enough suggestions, thoughts and ideas to draft a proposal that isn't rejected by the dev community immediately. > Firstly, Backwards compatibility. contrib.auth is a major component in > many, if not most existing Django installations. If you're proposing > *any* change, the existing code needs to be 100% backwards compatible. > > For example, I'm not entirely certain I follow why the changes to > login() et al are necessary. You're proposing to change login(), etc, > but you're a little light on details on how backwards compatibility > will be maintained. > If you say, the auth framework should only work with site based login procedures (displaying the login form, calling a login() method and redirecting to another page all inside the Django app) there is no need the refractor the base get_user() method. But when the login process is handled completely outside the Django app (for example using the Google Accounts Python API; the api forces you to redirect the user to a Google login page, Google checks username and password and redirects the user back to the app) I see no way of calling a login() method inside the Django app and setting request.session[SESSION_KEY], but a custom authentication middleware fixes the problem. But I think changing the stuff in contrib.auth.__init__ won't work without breaking a lot of custom user backends. > > Also, if you're proposing that the auth.User model be refactored, > you're going to need to demonstrate why existing auth.User installs > aren't going to be affected. > > Secondly, as far as I can make out, your proposal doesn't address the > biggest problem I can see -- Foreign Keys/ManyToMany to User. It isn't > much good proposing a way to use a different User model if you don't > provide a way for other applications that currently ship with a > ForeignKey to contrib.auth.User to change that into a ForeignKey on > your new User model. The approach used by contrib.Comments with > COMMENTS_APP may be worth investigating here. > > A related problem here: what happens if a developer changes their User > model during the development process? How do I migrate my existing > contrib.auth users to use mycustom.auth users? Is this even possible? > Is there any way to protect against this class of error? > My idea is to add an extra abstraction layer for the User class in form of a BaseUser class. The BaseUser class consists of : * the whole permission stuff (user_permissions, get_group_permissions, get_all_permissions,), * the the authentication stuff (is_anonymous and is_authenticated, is_superuser, is_staff, last_login, date_joined, groups) * a single ident field Other specific user classes extends the BaseUser class via Multi-table inheritance. The "old" User class will contain the same fields/methods as before, but it will inherit some fields from BaseUser and add the missing ones (username, password, set_password...), so it won't break existing auth.User installs. To add a custom user model you have to extend BaseUser (if you don't have database based authentication, but Facebook or Twitter) or the "new" User class (if you want to add custom information to the standard, databased user model, like extra fields or modify existing fields via overriding). I suggest linking to BaseUser instead of User if possible (to become independent from the Django database User class). Everything that has to do with permissions, authentication information or if you want to link an entry to an user (for example a blog page to a specific user) should link to BaseUser. > Another related problem - even if you make User pluggable, there will > still be a need for some common interface for user-like objects. What > should be on this interface? > For a pluggable User interface I suggest splitting into Permission and Authentication stuff. It the interface will look something like this: class BaseUserPermission: - user_permissions - groups - is_staff - is_superuser - get_group_permissions() - get_all_permissions() - has_perm() - has_module_perms() - ….. class BaseUserAuthentication: - last_login - date_joined - is_anonymous() - is_authenticated() - ….. class BaseUser(BaseUserAuthentication, BaseUserPermissions, models.Model): -
Re: GSoC: Data importation class
> > Adding system administration functionality to Django, which is what this > > monitoring is, feels like the wrong approach. I see here you probably meant its appropriate elsewhere but not in django. Gotcha. Thought I'd try anyhow. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: GSoC: Data importation class
On Mar 25, 12:34 pm, Malcolm Tredinnick wrote: > Hi, > > I see a few problems here. The gist of what follows is that it seems a > bit abstract and as one tries to nail down the specifics it either > devolves to a more-or-less already solved problem that doesn't require > Django core changes, or a problem that is so unconstrained as to not be > solvable by a framework (requiring, instead the full power of Python, > which is already in the developer's hands). In situations where inspectdb doesn't work, I've found its much less work to get data into a shape where it _will_ work, rather than resorting to scripting the entirety. > On Thu, 2010-03-25 at 09:07 -0700, subs...@gmail.com wrote: > > mix). With talk of model-level validation, the first approach is > > becoming increasingly invalid, > > That's not a correct statement, since Django models can often be used to > proscribe conditions on new data that is created via the web app, yet > those conditions might not be required for the universal set of data > that already exists. For example, webapp-generated data might always > require a particular field, such as the creating user, to be filled in, > whilst machine-generated data would not require that. Don't equate > validation conditions at the model level with constraints on the data at > the storage level. I'm hard pressed to imagine a situation where I want validation to only apply to incoming data. Out of laziness I might choose not to apply some conditions to existing data as I did to a form, knowing that the next time the user touches the record the validation will kick in. This whole point seems to rest on a 'might' which I can't recall ever encountering. Regardless, I tend to regard legacy data with the same caution as I treat incoming user data, but it is currently worlds more difficult to do the former compared to the latter. > The last bit sounds a bit nebulous. You could optimise it by not > including any empty files, or be a bit more specific about what the > empty files are meant to represent. :) startapp, startproject, et al. > It seems that you are talking about the cases where, by default, a > different schema is required. The first approach is to make the models > match the existing schema, on the grounds that the existing schema is a > reasonable representation of the data. In the case where that isn't > true, a migration is required, but the possibilities for such migrations > are endless unless the original data can already be put into natural > Django models. If inspectdb can already be run on the existing data, why > not use that as the starting point and then the dev can use something > like South to migrate to their schema of choice? It seems that we > already have all the tools in that case. South has a target use case of relatively simple changes to schema, and assisting teams maintain synchronicity. However, it isn't long before you're really pushing the limits of South. Take for example one legacy model which needs to be split into two or three current models: South has no answer for this as you may only use defaults for creation of fields (in this case, foreign keys, or potentially OneToOnes, and then what? What if my defaults are based on other values in that record? Already, I'm on my own). By and large, South is an immaculate tool for tracking changes during development. > If inspectdb cannot generate a useful schema that can be modelled by > Django, the user is going to have write a generic Python script in any > case and the possibilities there are boundless and best left to the best > tool available for the job at hand: Python itself. In theory, yes. But in practice, I've found the shortest way around the mountain is to get it into "SQL-enough" format manually. As for Python, sure, but the more you write this monolithic script the more you realize you're conducting a lot of repetitive work, the mechanics of which are generally re-usable but an implementation which is completely nuanced to your current task. If data importation is something you do a lot, you've probably got a file somewhere holding piecemeal bits that are hopefully vaguely useful to the next project, all the while not being able to fight off the feeling that this general task mirrors a lot of what goes on in forms with clean_somefield() and clean(). > Adding system administration functionality to Django, which is what this > monitoring is, feels like the wrong approach. It's not intended to > replace everything else in your computing life. What is appropriate load > usage for one case will be highly inappropriate elsewhere. How will you > detect what you are labelling as "thrashing"? I'm puzzled by this conclusion. The 'system administration functionality' isn't in any way different to what you'd find in all kinds of projects--South included. I'm not even sure what to say about the 'everything else in your computing life' statement, but I will assume good faith and that you're not alleging I'm p
Re: GSoC: Data importation class
Hi, I see a few problems here. The gist of what follows is that it seems a bit abstract and as one tries to nail down the specifics it either devolves to a more-or-less already solved problem that doesn't require Django core changes, or a problem that is so unconstrained as to not be solvable by a framework (requiring, instead the full power of Python, which is already in the developer's hands). On Thu, 2010-03-25 at 09:07 -0700, subs...@gmail.com wrote: > If we could visualize the entirety of data within django-projects, we > would probably see that this 'data economy' is growing exponentially > year-over-year. However, I know of no guided way to actually get this > data into a project that's been converted to Django. There are two > methods I generally hear about when asking people how to move between > schemas: purely SQL solutions and one-shot scripted solutions (or a > mix). With talk of model-level validation, the first approach is > becoming increasingly invalid, That's not a correct statement, since Django models can often be used to proscribe conditions on new data that is created via the web app, yet those conditions might not be required for the universal set of data that already exists. For example, webapp-generated data might always require a particular field, such as the creating user, to be filled in, whilst machine-generated data would not require that. Don't equate validation conditions at the model level with constraints on the data at the storage level. > but I wonder if we could include some > batteries for the second approach? > > My proposal is a new django class which provides a mapping for how > this data should move from its legacy schema to a django project. > I've > got a sort of proof-of-concept already working but it lacks the polish > of a refined community contribution. Moreover, with multi-database > support coming, I see this concept getting a shot in the arm, > especially in cases where the legacy db is a currently supported one. > > I imagine the usage going something like: > > 1) User creates django project > > 2) User runs a 'startconversion' app which creates a stage folder for > holding an inspectdb of the legacy data, a default router for the > legacy data, and some other empty files. The last bit sounds a bit nebulous. You could optimise it by not including any empty files, or be a bit more specific about what the empty files are meant to represent. :) > > 3) User defines the classes which defines the map between the legacy > and new schema, and defines clean functions according to their needs, > 'foreignkeys' to other conversion classes, etc. It seems that you are talking about the cases where, by default, a different schema is required. The first approach is to make the models match the existing schema, on the grounds that the existing schema is a reasonable representation of the data. In the case where that isn't true, a migration is required, but the possibilities for such migrations are endless unless the original data can already be put into natural Django models. If inspectdb can already be run on the existing data, why not use that as the starting point and then the dev can use something like South to migrate to their schema of choice? It seems that we already have all the tools in that case. If inspectdb cannot generate a useful schema that can be modelled by Django, the user is going to have write a generic Python script in any case and the possibilities there are boundless and best left to the best tool available for the job at hand: Python itself. > 4) User runs a command at the top branch of their schema (some distant > relation) and the command inspects these classes and runs them from > the ground up. As it does this measures are taken (such as use of > pagination) to avoid server CPU/memory thrashing, as well as model- > level measures such as OneToOne's being respect, etc. Adding system administration functionality to Django, which is what this monitoring is, feels like the wrong approach. It's not intended to replace everything else in your computing life. What is appropriate load usage for one case will be highly inappropriate elsewhere. How will you detect what you are labelling as "thrashing"? Regards, Malcolm -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
GSoC: Data importation class
If we could visualize the entirety of data within django-projects, we would probably see that this 'data economy' is growing exponentially year-over-year. However, I know of no guided way to actually get this data into a project that's been converted to Django. There are two methods I generally hear about when asking people how to move between schemas: purely SQL solutions and one-shot scripted solutions (or a mix). With talk of model-level validation, the first approach is becoming increasingly invalid, but I wonder if we could include some batteries for the second approach? My proposal is a new django class which provides a mapping for how this data should move from its legacy schema to a django project. I've got a sort of proof-of-concept already working but it lacks the polish of a refined community contribution. Moreover, with multi-database support coming, I see this concept getting a shot in the arm, especially in cases where the legacy db is a currently supported one. I imagine the usage going something like: 1) User creates django project 2) User runs a 'startconversion' app which creates a stage folder for holding an inspectdb of the legacy data, a default router for the legacy data, and some other empty files. 3) User defines the classes which defines the map between the legacy and new schema, and defines clean functions according to their needs, 'foreignkeys' to other conversion classes, etc. 4) User runs a command at the top branch of their schema (some distant relation) and the command inspects these classes and runs them from the ground up. As it does this measures are taken (such as use of pagination) to avoid server CPU/memory thrashing, as well as model- level measures such as OneToOne's being respect, etc. -Steve -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: GSoC Project: Enhancedaut.user
On Thu, Mar 25, 2010 at 4:27 PM, d3f3nd3r wrote: > Hello everyone, > > I'm planning to apply for the GSoC Enhanced auth.user project. I > worked on several Django based projects before. > (hackerspace os (http://code.google.com/p/hackerspace-os/) is probably > the > biggest one (used by the Austrian hackerspace metalab (http:// > metalab.at)) and > a lot of smaller ones) > > I faced some limitations of the auth framework when I tried to write a > custom > auth backend for Django on Google App Engine (http://bitbucket.org/ > d3f3nd3r/djangoappengine-helpers/). The problem was, that > the auth framework forces you to call a login() function when the > user logs in and set request.session[SESSION_KEY] and > request.session[BACKEND_SESSION_KEY]. > > When using Google Accounts for authentication no django login > method is called. But django.contrib.auth.__init__ get_user() method > depends on > request.session[SESSION_KEY] and request.session[BACKEND_SESSION_KEY] > so a > custom backend with a custom get_user() won't work without a modified > middleware. > (http://groups.google.com/group/django-non-relational/browse_thread/ > thread/36b760c515b75c0e) > > My first approach to make the auth system more flexible is to make the > methods > in django.contrib.auth.__init__.py more flexible, so the methods > doesn't force > you to call a django login method. All backends will have to extend a > base > backend class and implement the get_user(), login(),.. methods. > The request.session stuff will be moved to the ModelBackend (other > model based > backends can extend ModelBackend). So it will be easy to create custom > backends > for Google Accounts,Twitter, Facebook... that work out of the box with > the Django auth system. > > The second task will be to make the user model more extendable. I > think the > way suggested by David Danier > ( ticket : http://code.djangoproject.com/ticket/3011) is a good one. > Move > all the login and permission stuff to extra Permission and Auth base > classes. > A base model class for authentication will only contain an ident field > (we > dont't need to store user name / password/ email if we use a remote > service for > authentication, but we have to link the remote service user to to > Django user > model). The "old" contrib.auth.models.User class will extend the two > classes > (Permission and Auth) and add all the extra fields (username, first-, > last name) so it won't break existing apps. > A list of accepted user classes can be passed to the backends. > > I know this is a tough issue, but I'm really interested in working on > the > problem. Please post your comments, ideas, suggestions,..., so we can > find a > good solution. (and maybe a successful proposal for GSoC) First up, this is probably the most ambitious project on the proposal list. When I warn that this is a tough issue, I'm not kidding. This is gonna be *hard*. With a capital Huh. There are all sorts of technical issues, and a couple of political ones too. Still interested? Ok. Here's a few quick comments about your proposal. Firstly, Backwards compatibility. contrib.auth is a major component in many, if not most existing Django installations. If you're proposing *any* change, the existing code needs to be 100% backwards compatible. For example, I'm not entirely certain I follow why the changes to login() et al are necessary. You're proposing to change login(), etc, but you're a little light on details on how backwards compatibility will be maintained. (Related note - a proposal should be largely self contained. Links to mailing lists are OK if you need to show the evolution of an idea, but when it's the endpoint idea that is important, your proposal should contain a concise summary of that endpoint) Also, if you're proposing that the auth.User model be refactored, you're going to need to demonstrate why existing auth.User installs aren't going to be affected. Secondly, as far as I can make out, your proposal doesn't address the biggest problem I can see -- Foreign Keys/ManyToMany to User. It isn't much good proposing a way to use a different User model if you don't provide a way for other applications that currently ship with a ForeignKey to contrib.auth.User to change that into a ForeignKey on your new User model. The approach used by contrib.Comments with COMMENTS_APP may be worth investigating here. A related problem here: what happens if a developer changes their User model during the development process? How do I migrate my existing contrib.auth users to use mycustom.auth users? Is this even possible? Is there any way to protect against this class of error? Another related problem - even if you make User pluggable, there will still be a need for some common interface for user-like objects. What should be on this interface? Lastly, what is the impact on other areas of Django code? Backwards compatibility means that all existing code must continue to work as-is, but if we're shipping a new capab
Re: Security concern: information leakage (even under SSL)
On Thu, Mar 25, 2010 at 10:50 AM, Jeremy Dunck wrote: > There's a new-ish security paper detailing how information can be > gleaned by observing the sizes of secure responses. > > Summary here, which links to the original paper: > > http://www.freedom-to-tinker.com/blog/felten/side-channel-leaks-web-applications > > One possible mitigation is to add cover traffic in the form of padding > so that the information is obscured. > > To that end, I hacked out a middleware. I'd like to get some feedback > (testing, suggestions for improvement, etc) from the list. > > http://www.djangosnippets.org/snippets/1970/ > > Once it isn't broken in obvious ways, maybe we can have a security > expert get further feedback and add to core? Thanks for this article. It is very interesting. One thing I would like to point out this that this is only a problem on SSL connections (as anyone listening to an non encrypted connection can easily throw out the padding). Should this middleware detect if the connection is secure before adding the padding to defer someone from adding 1024 characters to each of their responses for no reason? It also seems as though the content that is compromised is relatively specific. For example, a standard HTML page with an image or 2 like Django's admin would be more secure than a site which makes considerably more calls to the server. I am no expert, but it appears as though the smaller the request, the less secure it is based on this. Perhaps there is something for the middleware to consider as well. I am absolutely no expert on this, but security-wise better safe than sorry. We should especially be looking at this as we continue to update the admin with more Ajax and dependancies. Thanks, Jeremy, Michael -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Security concern: information leakage (even under SSL)
There's a new-ish security paper detailing how information can be gleaned by observing the sizes of secure responses. Summary here, which links to the original paper: http://www.freedom-to-tinker.com/blog/felten/side-channel-leaks-web-applications One possible mitigation is to add cover traffic in the form of padding so that the information is obscured. To that end, I hacked out a middleware. I'd like to get some feedback (testing, suggestions for improvement, etc) from the list. http://www.djangosnippets.org/snippets/1970/ Once it isn't broken in obvious ways, maybe we can have a security expert get further feedback and add to core? -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Passing context variables from template to view
The topic of this list is discussion of the development of Django itself. Please ask questions about using Django on django-users. Karen -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
GSoC Project: Enhancedaut.user
Hello everyone, I'm planning to apply for the GSoC Enhanced auth.user project. I worked on several Django based projects before. (hackerspace os (http://code.google.com/p/hackerspace-os/) is probably the biggest one (used by the Austrian hackerspace metalab (http:// metalab.at)) and a lot of smaller ones) I faced some limitations of the auth framework when I tried to write a custom auth backend for Django on Google App Engine (http://bitbucket.org/ d3f3nd3r/djangoappengine-helpers/). The problem was, that the auth framework forces you to call a login() function when the user logs in and set request.session[SESSION_KEY] and request.session[BACKEND_SESSION_KEY]. When using Google Accounts for authentication no django login method is called. But django.contrib.auth.__init__ get_user() method depends on request.session[SESSION_KEY] and request.session[BACKEND_SESSION_KEY] so a custom backend with a custom get_user() won't work without a modified middleware. (http://groups.google.com/group/django-non-relational/browse_thread/ thread/36b760c515b75c0e) My first approach to make the auth system more flexible is to make the methods in django.contrib.auth.__init__.py more flexible, so the methods doesn't force you to call a django login method. All backends will have to extend a base backend class and implement the get_user(), login(),.. methods. The request.session stuff will be moved to the ModelBackend (other model based backends can extend ModelBackend). So it will be easy to create custom backends for Google Accounts,Twitter, Facebook... that work out of the box with the Django auth system. The second task will be to make the user model more extendable. I think the way suggested by David Danier ( ticket : http://code.djangoproject.com/ticket/3011) is a good one. Move all the login and permission stuff to extra Permission and Auth base classes. A base model class for authentication will only contain an ident field (we dont't need to store user name / password/ email if we use a remote service for authentication, but we have to link the remote service user to to Django user model). The "old" contrib.auth.models.User class will extend the two classes (Permission and Auth) and add all the extra fields (username, first-, last name) so it won't break existing apps. A list of accepted user classes can be passed to the backends. I know this is a tough issue, but I'm really interested in working on the problem. Please post your comments, ideas, suggestions,..., so we can find a good solution. (and maybe a successful proposal for GSoC) Flo -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Passing context variables from template to view
Hi, I am looking to implement an "Export to Excel sheet" feature. The result-set is already available and rendered in the template. I need to provide a link or button to export result to Excel sheet. Result-set is a set of list variables and cannot be passed over GET request due to length limitations. Two solutions that I can think of are: 1. Use hidden form fields and submit it back to a view that creates Excel sheet. 2. Directly pass object list to view using reverse() or URL regex. I have been unable to pass lists using URL Regex. Is it possible to pass list variables from template back to the view? If so, how? Appreciate ur help! Kasto -- -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Potential bugs / need for extra documentation in 'Customizing the comments framework'
I got myself into a situation with a project and some comments models.
Perhaps with some discussion we can uncover genuine code-bugs, but my
knowledge about this is limited and so I'm just calling for a
documentation update. A major reason I'm asking for some notice of
these things to appear somewhere is that all of the below seems to
work perfectly fine in the dev server. On deployment, its a confusing
catastrophy that, at best, gives you a vague ImproperlyConfigured?
("The COMMENTS_APP setting refers to a non-existing package."), which
is of course no help at all.
Three things:
1) Its not obvious to me why, but importing a custom comment model as
'Comment' seems, somehow, to get back upstream to
django.contrib.comments and wreck things. If this is due to the nature
of that particular frameworks' immaculate ability to be extensive and
provide conveniences all around, so be it. I'd just like a notice next
time before I attempt to save code by doing Comment =
comments.get_model() and expect everything to be honkey dorey.
2) A custom comments app should not also be called 'comments'.
Probably a very obvious one but the problems it creates are
inconsistent between dev and live and not obvious.
3) Models hoping to foreignkey to whatever comments model the app
provides should not be passed the callable comments.get_model(). I
realize passing callables into a ForeignKey probably seems strange in
and of itself, however, once again it creates inconsistent problems
between dev and live which never connect back to the FK in any obvious
way.
4) Probably a dupe of above, but passing the callable into a signal as
sender also fails, and your signal is never respected.
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
