Re: Any benefit in storing shorter strings or numbers to represent data for a choice char field?
Thanks Carl, I wasn't aware of that function, that works :) Sometimes normalization can just get you in trouble, slow things down. My question was mainly regarding the efficiency in terms of storage, and how you query that if you store the short version. I would not consider dividing the data up into more models at the time. I would use something like haystack with elasticsearch to index the human readable names if I needed them and that display function is a life saver. Thanks all. On Wednesday, 17 December 2014 11:56:08 UTC-5, Radomir Wojcik wrote: > > The official django doc uses this as an example (see below). Is there any > point to storing 'FR' instead of 'Freshman" so the choice matches what is > stored? Is it faster at all? Saves room? What about for querying the data? > Then you have to lookup the symbol 'FR' in the tuple if someone wants to > query by 'Freshman'. > > What is the best practice? Is it even more efficient if integers were > used, like 1 value to represent Freshman and so on? To me it makes sense to > store 'Freshman' as 'Freshman' but I'd like to get some insight. > > from django.db import models > > class Student(models.Model): > FRESHMAN = 'FR' > SOPHOMORE = 'SO' > JUNIOR = 'JR' > SENIOR = 'SR' > YEAR_IN_SCHOOL_CHOICES = ( > (FRESHMAN, 'Freshman'), > (SOPHOMORE, 'Sophomore'), > (JUNIOR, 'Junior'), > (SENIOR, 'Senior'), > ) > year_in_school = models.CharField(max_length=2, > choices=YEAR_IN_SCHOOL_CHOICES, > default=FRESHMAN) > > def is_upperclass(self): > return self.year_in_school in (self.JUNIOR, self.SENIOR) > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/2f8b67d6-8e98-460a-b7ee-210aef311ce6%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Any benefit in storing shorter strings or numbers to represent data for a choice char field?
> > So to answer my own question, you wouldn't use the short version in the db if > you plan to do like/contains queries on them. Correct me if I'm wrong. > > Unless I plan to use haystack later and index the human readable form later. I think I understand now, thanks -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/ca539494-3243-41c0-bd28-23587a28e966%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Any benefit in storing shorter strings or numbers to represent data for a choice char field?
> > Thanks for the insight, I'm on the same page as you. > > My only real concern is that I will have to use a function, such as this one to get the human readable version of the stored data, then you need to lookup the value from the dict. And if you're doing querying on that object using key words, you have to query based on the human readable names, so that will cost you to look them all up every time and do a like SQL query or contains every time. def school_year_to_dict(): return dict((x, y) for x, y in YEAR_IN_SCHOOL_CHOICES) -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/4aff74b9-b684-4510-aa3a-95259c13f786%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Any benefit in storing shorter strings or numbers to represent data for a choice char field?
The official django doc uses this as an example (see below). Is there any point to storing 'FR' instead of 'Freshman" so the choice matches what is stored? Is it faster at all? Saves room? What about for querying the data? Then you have to lookup the symbol 'FR' in the tuple if someone wants to query by 'Freshman'. What is the best practice? Is it even more efficient if integers were used, like 1 value to represent Freshman and so on? To me it makes sense to store 'Freshman' as 'Freshman' but I'd like to get some insight. from django.db import models class Student(models.Model): FRESHMAN = 'FR' SOPHOMORE = 'SO' JUNIOR = 'JR' SENIOR = 'SR' YEAR_IN_SCHOOL_CHOICES = ( (FRESHMAN, 'Freshman'), (SOPHOMORE, 'Sophomore'), (JUNIOR, 'Junior'), (SENIOR, 'Senior'), ) year_in_school = models.CharField(max_length=2, choices=YEAR_IN_SCHOOL_CHOICES, default=FRESHMAN) def is_upperclass(self): return self.year_in_school in (self.JUNIOR, self.SENIOR) -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/e0a71f13-1541-437b-a7b9-24c06f08c7e5%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Model QuerySet used in base.html template
I have to run some benchmarks and see. I am using SQLite and I have heard some people say that it cannot handle concurrent transactions. From what I read it can handle some 50,000 transactions/ sec depending on your disk IO. For a small deli website I take it should be fine, even if the user plans to load up the database with over 1000 items. I'll see how it copes with transaction handling as its more mature, and then maybe I will switch to a context processor and maybe a different database :) -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Model QuerySet used in base.html template
Hey Kamil, Isn't that overkill though? templatetag was an easy solution but you still do a query on the locations model on each page load. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Model QuerySet used in base.html template
I simply added an assignment tag instead as follows:
@register.assignment_tag
def get_locations():
from store.models import Location
locations = list(Location.objects.all())
return locations
And use it like so in template
{% get_locations as all_locations %}
then can iterate through it
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
For more options, visit https://groups.google.com/groups/opt_out.
Re: Model QuerySet used in base.html template
I am actually thinking using some sort of templatetag now instead of passing the query each time since its used in base.html. It can return the exact html string that can be displayed in base.html. Right now I have the " locations = Location.objects.all() " passed to the template in every single view. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Model QuerySet used in base.html template
I wanted to add location address in the menu. I have a model called locations so this is easy enough to do. But now I have to add the queryset to every view, because every view has a template that extends base.html. So do I have to add this queryset as such to every view now? locations = Location.objects.all() or is there a way to do this so I associate a view that always runs on every page to get the locations queryset? There are many ways to do this but I wanted to know what best practice suggests. Thanks -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
Yeah I figured that its python as per the message. I didn't know I had to add it to every source that has the chars but thanks for the tip. I was trying it out to see if the issue was related to the database encoding or not, a UTF-8 conversion on the data completely fixed my issue. This is the command for those with the same problem: Convert sql file from latin to utf-8: iconv -f ISO_8859-1 -t UTF-8 mandala.postgres.sql > mandala.postgres.utf8.sql -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
# Django settings for mandala project. # -*- coding: utf-8 -*- But django says its not declared. Where should I put this? -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
Michael I already have this on top of my settings.py file all along, but djanog does not pick it up.. #django project # encoding line is 2nd in settings.py -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
Thanks a bunch. Yeah when I did a full convert to UTF-8 I no longer have the issue. I still have the case sensitive login issue which came with the PostgreSQL upgrade. Do you have the same issue? Is this normal? Logging in as Rad no longer works, I have to use exact case as it is in the database (lower case), rad -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
This fixed it iconv -f ISO_8859-1 -t UTF-8 mandala.postgres.sql > mandala.postgres.utf8.sql Now it takes the accents in as utf-8. I guess it didnt' like the LATIN1 encoding mixed with UTF-8 -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
Xavier , which encoding do you use for french characters with django? UTF-8 does not like *é*, But it says it exists in utf-8:U+00E9éc3 a9LATIN SMALL LETTER E WITH ACUTE http://www.utf8-chartable.de -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
Xavier, what do you mean file is latin1 encoded? You lost me. By the way, one of the Non UTF-8 data that it complains about is the accent over the e in Neguac: * Néguac* New Brunswick See my error above given by psql db < file.sql -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
So when I changed the encoding in the sql file from LATIN1 to UTF-8 I get import errors: (1 row) ERROR: invalid byte sequence for encoding "UTF8": 0xe9 0x67 0x75 CONTEXT: COPY locations_location, line 31 setval So what encoding can I use that is safe with french accents but at the same time can be used in django no problem? LATIN1 was ok in MYSQL -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
So is it normal that without adding the u' to a string that contains french accents will give the error: ascii' codec can't decode byte 0xc3 in position 10: ordinal not in range(128) ? Are french characters UTF-8 Compliant? Would this insertion process not fail if the data contained within the postgres.sql file contain non UTF-8 characters when mandala4 in this case is UTF-8 encoded? psql mandala4 -U postgres < mandala.postgres.sql Password for user postgres: SET SET SET SET SET CREATE EXTENSION COMMENT SET SET SET CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE Seeing as it had no problems inserting the french accents into a UTF-8 encoded database, this means that french characters are UTF-8 complaint? This shows you the database is utf8: Name| Owner | Encoding | Collate |Ctype| Access privileg es ---+--+--+-+-+-- - mandala | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | mandala3 | postgres | LATIN1 | C | C | * mandala4 | mandala | UTF8 | en_US.UTF-8 | en_US.UTF-8 | * postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | template0 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres + | | | | | postgres=CTc/post gres template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres + | | | | | postgres=CTc/post gres (6 rows) Here are some snippets from the sql dump file , client encoding is LATIN1 but going into a UTF-8 database, and it did so without issue: SET statement_timeout = 0; SET client_encoding = 'LATIN1'; SET standard_conforming_strings = on; SET check_function_bodies = false; SET client_min_messages = warning; -- -- Name: plpgsql; Type: EXTENSION; Schema: -; Owner: -- CREATE EXTENSION IF NOT EXISTS plpgsql WITH SCHEMA pg_catalog; -- -- Name: EXTENSION plpgsql; Type: COMMENT; Schema: -; Owner: -- COMMENT ON EXTENSION plpgsql IS 'PL/pgSQL procedural language'; SET search_path = public, pg_catalog; SET default_tablespace = ''; SET default_with_oids = false; The auth_user creation as created by django using postgres, but now the login is case sensitive: CREATE TABLE auth_user ( id integer NOT NULL, password character varying(128) NOT NULL, last_login timestamp with time zone NOT NULL, is_superuser boolean NOT NULL, username character varying(30) NOT NULL, first_name character varying(30) NOT NULL, last_name character varying(30) NOT NULL, email character varying(75) NOT NULL, is_staff boolean NOT NULL, is_active boolean NOT NULL, date_joined timestamp with time zone NOT NULL ); example data going in: COPY auth_user (id, password, last_login, is_superuser, username, first_name, last_name, email, is_staff, is_active, date_joined) FROM stdin; 1 bcrypt$$2a$12$YW6Gxsl0z4/PgQPrbwZgm.f5tB2Bnu.wShpNg2rmsr4H3W8knbZsq 2013-08-21 22:07:57.359359-04 t admin a...@a.com t t 2013-08-21 18:16:30-04 Here is the other model with issues when the city has an accent: CREATE TABLE locations_location ( id character varying(20) NOT NULL, created timestamp with time zone, modified timestamp with time zone, address character varying(255), address2 character varying(255), city character varying(255), province character varying(255), postal_code character varying(255), county character varying(255), country character varying(255), phone character varying(255), latitude numeric(11,7), longitude numeric(11,7), open_time time without time zone, close_time time without time zone, is_24_hours boolean NOT NULL, notes text, ); And here is the insert of data and the city that has the problem: 2012-08-29 08:36:00-04 2013-07-16 14:23:00-04 t t t 74 Principale St \N * Néguac* New Brunswick Is this enough info to come up with a good conclusion of what this could be? -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
MySQL would also somehow make the built in django contrib views login case insensitive, and now its case sensitive. Since it takes the request as a parameter and request.POST is immutable I will have to write my own login without using the batteries built in solution since I cannot do a .lower() on POST.get['username'] and save it back to the request. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
This must be my problem all along: *By the way, if you're using python 2, you shouldn't be using 'string' notation for character strings and should be using the u'string' one. 'string' is a binary string while u'string' is a text string. This is misleading with python 2 as there's implicit conversion between the two and utf8 will be used as the default charset which may lead to the kind of errors you've faced*. I was not using u' at all I was always using the binary strings and had no problems. I guess MySQL would convert them automagically for me while Postgres must deal with the data raw waiting for me to add the u' to it and then it works. So what I was really looking for is an option that would convert all binary strings to unicode (text) strings for me without me having to go back and test and add it manually all over the place... -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
>
> Russ like I said before, I used COPY to get the data back in and it was
> accepted by postgres with the schema it crated using django (See step 5
> below)
1) Export each table individually from MySQL into csv format.
2) Have Django re-create the models schema from django on its own.
3) Export the django schema using pg_dump for comparison
Repeat 5, 6 for each table: 4) Compare each csv file to the postgre schema
for each table and align the data accordingly using spreadsheet software.
5) Use COPY (postgre utiity) to insert data from the csv fie to the
database.
COPY locations_location FROM '/tmp/locations_location.csv' DELIMITER ',' CSV;
6) pg_dump dbname > outfile the data in order for migration. Restore on
system using psql dbname < infile
When I was using COPY, it would complain about any issues and I fixed the
csv's as I went along, so everything inserted was validated with PostgreSQL
on the way in.
One very important question which I still don't understand. Are french
characters such as "é" UTF-8 compatible or not? Because they seem to be
sitting in the database just fine and they display just fine most of the
time, but not when passing strings from my views, such as:
messages.success(request, u'{0} edited & saved Alarm Definition
successfully.'.format(cur_alarm_rec))
I had to add the unicode u' around it because cur_alarm_rec contained an "é"
I also ended up adding the u to most of my models unicode output
def __unicode__(self):
return u'%s:%s' % (self.id, self.city)
As some cities contained special french accent characters as well.
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
> > From what I read, SQLite is production grade and its the most used > database in the world.. besides the point. So other than Postgres not picking up my encoding (UTF-8 or Latin-1) , both did not work. How can I get the login to be case insensitive again? This will really be a bummer if I have to re-write my own login instead of using the built in stuff. I think it all goes back to the encoding issue though... -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
Thanks Xavier, I had this at the top of my settings.py file, tried changing it to latin-1 and it didn't make a difference. # -*- coding: utf-8 -*- So right now I am using the UTF-8 database as shown above: mandala | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | The data contains special characters, french accents mainly and they seem to reside in the database and display just fine. By importing the csv files and letting the new django postgres schema accept it, I have had a very clean migration process, you can say I did it by hand because postgres COPY alerted me when the data was bad and I had to change it. Lets put the database aside for a second, I also have some error messages with accents in them, and they give me the error below as before it worked just fine. messages.error(request, "Sé") And now: Non-ASCII character '\xc3' in file /var/web/mandala_env/mandala/alarms/views.py on line 329, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details (views.py, line 329 But this makes no sense because I do have encoding declared in settings.py right near the top, and this must have worked before when using MySQL -> # -*- coding: utf-8 -*- -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
Also getting a bunch of these too: Non-ASCII character '\xc3' in file /var/web/mandala_env/mandala/alarms/views.py on line 329, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details (views.py, line 329) Is this normal? PostgreSQL is more strict with character types? So now I have to add # coding= to every py file in my django project? -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
Adding the unicode u to the 'string' as such u'string' fixes the issue but now I have to add this u to every string that will potentially have a character with a french accent in it? How come MySQL didn't need this? There must be a straight forward answer to this, something that will make Postgres + django handle every string as unicode. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
Also why did from django.contrib.auth.views.login become case sensitive? I have users that login with User and user, in PostgreSQL this is treated as 2 different things. Using MySQL it always saw it as one.. all very frustrating. I will try SQLite tomorrow maybe instead, been up 2 days straight trying to get this to work and I really don't want to use MySQL anymore since its now owned by Oracle and slowly going to become closed source while MariaDB is not fully supported by Django just yet. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
> > Its not a problem with the conversion process. I re-created the schema > using django syncdb and then I went table by table putting the data back in > using csv files I exported from MySQL. I took the long way to get this all > in and PostgreSQL had no problems taking my csv data as input. Any advice > on how to get PostgreSQL running properly as MySQL did would be > appreciated. It must be able to handle french accents which are not ascii. > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: Can't decode french characters after switching to PostgrSQL from MySQL
And I tried with both UTF8 and LATIN1 database encodings, both produce the same error: List of databases Name| Owner | Encoding | Collate |Ctype| Access privileg es ---+--+--+-+-+-- - mandala | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | mandala3 | postgres | LATIN1 | C | C | I have a feeling that this is related to django + postgres as I noticed too that the login username used to be case insensitive but after the conversion is case sensitive.. strange. Is Postgres really that good? If I can't fix I will have to switch back to MySQL and wait for MariaDB to be officially supported... -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Can't decode french characters after switching to PostgrSQL from MySQL
my code worked fine up until I switched to PostgreSQL from MySQL. Now I am bombarded with these errors whenever I have french accents in my strings. My database is correctly configured with LATIN1 (as UTF8 didn't like the accents when trying to import them from MySQL, the default was UTF8). My postgresql.conf has the right settings by the look of it: client_encoding = latin1# encoding # These settings are initialized by initdb, but they can be changed. lc_messages = 'en_CA.iso88591' # locale for system error message # strings lc_monetary = 'en_CA.iso88591' #en_US.UTF-8'# locale for monetary formatting lc_numeric = 'en_CA.iso88591' # locale for number formatting lc_time = 'en_CA.iso88591' How do I switch this ascii decoder in django to a decoder that can handle french accents? UnicodeDecodeError at /alarms/edit/100/ 'ascii' codec can't decode byte 0xc3 in position 10: ordinal not in range(128) Request Method:POSTRequest URL:https://mandala.cldssinc.com/alarms/edit/100/Django Version:1.5.2Exception Type:UnicodeDecodeErrorException Value: 'ascii' codec can't decode byte 0xc3 in position 10: ordinal not in range(128) Exception Location:/var/web/mandala_env/mandala/alarms/models.py in __unicode__, line 21Python Executable:/var/web/mandala_env/bin/pythonPython Version:2.6.6Python Path: ['/var/web/mandala_env/mandala/mandala', '/var/web/mandala_env/mandala', '/var/web/mandala_env/bin', '/var/web/mandala_env/lib64/python26.zip', '/var/web/mandala_env/lib64/python2.6', '/var/web/mandala_env/lib64/python2.6/plat-linux2', '/var/web/mandala_env/lib64/python2.6/lib-tk', '/var/web/mandala_env/lib64/python2.6/lib-old', '/var/web/mandala_env/lib64/python2.6/lib-dynload', '/usr/lib64/python2.6', '/usr/lib/python2.6', '/var/web/mandala_env/lib/python2.6/site-packages'] Server time: Thu, 22 Aug 2013 01:16:16 -0400 Thanks -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: export .csv file from database
>
> this might come of use, it exports any query set to csv. I use it a lot:
>
>
def queryset_export_csv(qs):
import csv
#from django.db.models.loading import get_model
response = HttpResponse(mimetype='text/csv')
# force download.
response['Content-Disposition'] =
'attachment;filename="service_export.csv"'
# the csv writer
writer = csv.writer(response)
qs_model = qs.model
headers = []
for field in qs_model._meta.fields:
headers.append(field.name)
writer.writerow(headers)
for obj in qs:
row = []
for field in headers:
val = getattr(obj, field)
if callable(val):
val = val()
if type(val) == unicode:
val = val.encode("utf-8")
row.append(val)
writer.writerow(row)
return response
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
For more options, visit https://groups.google.com/groups/opt_out.
South not picking up changes to model, app crashes
So I created a service record model like this: class ServiceRecord(models.Model): location = models.ForeignKey(Location, related_name='service_records') created = models.DateTimeField(auto_now_add=True, null=True) modified = models.DateTimeField(auto_now=True, null=True) active = models.NullBooleanField(default=True, blank=True, null=True) service_provider = models.CharField(max_length=50, blank=True, null=True) service_date = models.DateField(blank=False, null=True) emergency = models.BooleanField(default=False, blank=True) emergency_date = models.DateField(blank=True, null=True) description = models.TextField(blank=True, null=True) parts_used = models.TextField(blank=True, null=True) warranty = models.BooleanField(default=False, blank=True) verified = models.BooleanField(default=False, blank=True) I then converted my app to south Install South on server Import south from shell just to make sure you are using the same python env. python manage.py syncdb. python manage.py convert_to_south myproject.locations This went fine: + Added model locations.Location + Added model locations.Photo + Added model locations.WaterSource + Added model locations.WaterStorage + Added model locations.PumpSystem + Added model locations.PressureTank + Added model locations.CartridgeFilter + Added model locations.DisinfectionUnit + Added model locations.TreatmentEquipment + Added model locations.WasteSystem + Added model locations.WaterSample + Added model locations.Schematic + Added model locations.Contact + Added M2M table for locations on locations.Contact + Added model locations.ServiceRecord Created 0001_initial.py. You can now apply this migration with: ./manage.py migrate locations - Soft matched migration 0001 to 0001_initial. Running migrations for locations: - Nothing to migrate. - Loading initial data for locations. Installed 0 object(s) from 0 fixture(s) App 'locations' converted. Note that South assumed the application's models matched the database (i.e. you haven't changed it since last syncdb); if you have, you should delete the locations/migrations directory, revert models.py so it matches the database, and try again. I then added a user key to it so I can track who created the record, user = models.ForeignKey(User) added: class ServiceRecord(models.Model): location = models.ForeignKey(Location, related_name='service_records') user = models.ForeignKey(User) created = models.DateTimeField(auto_now_add=True, null=True) modified = models.DateTimeField(auto_now=True, null=True) active = models.NullBooleanField(default=True, blank=True, null=True) service_provider = models.CharField(max_length=50, blank=True, null=True) service_date = models.DateField(blank=False, null=True) emergency = models.BooleanField(default=False, blank=True) emergency_date = models.DateField(blank=True, null=True) description = models.TextField(blank=True, null=True) parts_used = models.TextField(blank=True, null=True) warranty = models.BooleanField(default=False, blank=True) verified = models.BooleanField(default=False, blank=True) Then i did, python manage.py migrate locations –fake 000 - Soft matched migration 0001 to 0001_initial. Running migrations for locations: - Nothing to migrate. - Loading initial data for locations. Installed 0 object(s) from 0 fixture(s) and then where I thought it would find the user_id field.. python manage.py migrate apppython manage.py syncdb Running migrations for locations: - Nothing to migrate. - Loading initial data for locations. Installed 0 object(s) from 0 fixture(s) It did not migrate it and when I checked the database I saw it was not there and the app was just crashing complaining about the user_Id field. What am I doing wrong here? Any help would be appreciated. I ended up deleting the table and doing syncdb instead and had to manually get all the data back in.. Would really love to avoid this. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Setting up Django on GoDaddy Deluxe Shared Hosting
I was looking for the answer to this today and I wrote a tutorial on how to
do this based on all the stuff I found on the net:
With the economy class Linux hosting its a bit tricky. For starters you
don't have root access to the site packages so you cannot install for
example MySQL-Python.
1. Godaddy has virtualenv installed, so first, create a virtual environment
venv: (I use $HOME/lib/ for all the installed stuff below)
cd ~/
mkdir lib
cd lib
virtualenv --no-site-packages venv
The python package folder is $HOME/lib/venv/lib/python2.7/site-packages
2. Install the latest Django through pip
pip install Django
3. Create a new project
django-admin.py startproject mysite
4. Change the database configuration in mysite/setting.py file. When
setting the path for the database file, please use the absolute path:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME':
'/var/chroot/home/content/11/10420811/lib/venv/mysite/mydatabase.db',
'USER': '',
'PASSWORD': '',
'HOST': '',
'PORT': '',
}
}
While in here there are a few other things to consider changing:
- ADMINS = can be updated with your info
- TIME_ZONE = 'America/Toronto'
- LANGUAGE_CODE = 'en-ca'
- STATIC_ROOT =
'/var/chroot/home/content/11/10420811/lib/venv/mysite/static/'
- Uncommenting admin in INSTALLED_APPS : 'django.contrib.admin',
This will save you some setup later!
5. Set the Locale information in $HOME/.bash_profile file, otherwise you
cannot set the superuser when you sync the database. You can edit the file
with vim:
export LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8
6. Now run the script using the script command:
source ~/.bash_profile
7. If everything is setup properly, you should be able to sync the database:
python2.7 manage.py syncdb
You should see the admin tables get created and it will ask you to create a
new user.
8. Now we setup the dispatch functionality so we can access the webpage
without running server through Django using flup. So download and untar
flup:
cd ~/lib/venv/lib/python2.7/site-packages
wget
http://pypi.python.org/packages/source/f/flup/flup-1.0.2.tar.gz#md5=24dad7edc5ada3149456ee8d5254
tar -xvzf flup-1.0.2.tar.gz
mv flup-1.0.2/flup/ .
9. In $HOME/html folder, create dispatch.py and add the following lines of
code:
#!/usr/local/bin/python2.7
import sys, os
sys.path += ['/your/home/path/lib/venv/lib/python2.7/site-packages']
sys.path += ['/your/home/path/lib/mysite/']
os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'
from flup.server.fcgi import WSGIServer
from django.core.handlers.wsgi import WSGIHandler
WSGIServer(WSGIHandler()).run()
Also do a chmod +x dispatch.py to make the python script executable!
10. In $HOME/html/.htaccess file, add the following codes:
AddHandler fcgid-script .fcgi
Options +FollowSymLinks
RewriteEngine On
RewriteBase /
# everything else sent to django
RewriteRule ^(dispatch\.py/.*)$ - [L]
RewriteRule ^(.*)$ dispatch.py/$1 [L]
Another good thing to note is that you can ReWrite URLS if you have current
applications isntalled, ie.e dokuwiki.
#Add this above "everything else sent to django"
#Below other folders with static content and PHP , etc..
#hosted at yourdomain/dokuwiki
RewriteRule ^(dokuwiki/.*)$ - [L]
#You can also create your "django" project at a different location rather than
your domain root by changing the 2nd RewriteRule , i.e.:
# everything else sent to django
RewriteRule ^(dispatch\.py/.*)$ - [L]
RewriteRule ^(djangoproj/.*)$ dispatch.py/$1 [L]
#Make sure to create the djangoproj folder inside your html dir, cd html; mkdir
djangproj
#This becomes a bit annoying because your urlpatterns in url.py will now always
have to include 'djangoproj' at the beginning.
11. Update your urls.py file to look like so:
from django.conf.urls import patterns, include, url
from django.conf import settings
# Uncomment the next two lines to enable the admin:
from django.contrib import admin
admin.autodiscover()
urlpatterns = patterns('',
(r'^static/(.*)$', 'django.views.static.serve', {'document_root':
settings.STATIC_ROOT}),
url(r'^admin/', include(admin.site.urls)),
)
This will ensure you can access the admin application, as well as the
static root where all your img js and css are. Also note to copy the admin
img/js/css from /django/contrib/admin/static. Mine was in
~/lib/venv/lib/python2.7/site-packages/django/contrib/admin/
I used cp -r to copy it over to my application static dir
/var/chroot/home/content/11/10420811/lib/venv/mysite/static/
cp -r ~/lib/venv/lib/python2.7/site-packages/django/contrib/admin/
~/lib/venv/mysite/static/
12. That is it, you should have access to your site now! Access to
http://your.website.com/adminshould work!
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from
Re: a simple form confirmation before commit instance to database
Thanks I'll stick to simple JavaScript here. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
a simple form confirmation before commit instance to database
I was looking at examples using 'raise ValidationError' but I don't want to
raise an error based on the value of something. I always want the user to
be prompted with a "Are you sure you want to save this?" every time they
hit "save" :
if request.method == 'POST':
form = LocationForm(request.POST, instance=l)
if form.is_valid():
#prompt user first yes_no = confrim('Are you sure you want to
save this thing into the database?')
form.save() #only if yes_no is true.
So if the form is valid the user should be prompted with a "Are you sure?"
message, and if yes was clicked, then save the form.
What would be the easiest and most correct way to do this in django?
Thanks in advance
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
