Re: How do I make a Django model from a tab-delimited data file?

[Scot Hacker]

Another approach is to use postgres' COPY command:

https://www.postgresql.org/docs/9.6/static/sql-copy.html

which knows how to parse and import tab-delimited files (and is crazy 
fast). Then, once the table exists, use Django's `inspectdb` management 
command to generate a Model corresponding to the table. It may need a 
little manual massaging afterwards, but I've done this on several projects 
and it works very well for most purposes.

./s


On Monday, January 8, 2018 at 6:38:44 PM UTC-8, Tom Tanner wrote:
>
> I have a tab-delimited data file that looks something like this:
>
>
> NAME S1903_C02_001E state county tract State-County-Tract-ID
> Census Tract 201, Autauga County, Alabama 66000 01 001 020100 01001020100
> Census Tract 202, Autauga County, Alabama 41107 01 001 020200 01001020200
> Census Tract 203, Autauga County, Alabama 51250 01 001 020300 01001020300
>
> I want to make a Django model named `MyModel` with three columns: "name", 
> "data", and "geoid", which correspond to the file's columns "NAME", 
> "S1903_C02_001E", and "State-County-Tract-ID." Can I do this via command 
> line, or with a custom Python script? I'm running my Django project locally 
> on a computer running Debian 9.3. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/708e0f1d-1cac-4321-a68e-697a4a8cd3da%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How do I make a Django model from a tab-delimited data file?

[Tom Tanner]

Thanks you two. I'll check out that parser.

On Monday, January 8, 2018 at 9:38:44 PM UTC-5, Tom Tanner wrote:
>
> I have a tab-delimited data file that looks something like this:
>
>
> NAME S1903_C02_001E state county tract State-County-Tract-ID
> Census Tract 201, Autauga County, Alabama 66000 01 001 020100 01001020100
> Census Tract 202, Autauga County, Alabama 41107 01 001 020200 01001020200
> Census Tract 203, Autauga County, Alabama 51250 01 001 020300 01001020300
>
> I want to make a Django model named `MyModel` with three columns: "name", 
> "data", and "geoid", which correspond to the file's columns "NAME", 
> "S1903_C02_001E", and "State-County-Tract-ID." Can I do this via command 
> line, or with a custom Python script? I'm running my Django project locally 
> on a computer running Debian 9.3. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/3437a3bc-6a71-4a21-a410-7ede182cb83d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How do I make a Django model from a tab-delimited data file?

[Andréas Kühne]

Kasper,

You are correct :-), I just assumed (probably incorrectly) that the data
was just like his example - in that case it would have been easiest :-)

Regards,

Andréas

2018-01-09 9:27 GMT+01:00 Kasper Laudrup :

> Hi,
>
> On 2018-01-09 08:09, Andréas Kühne wrote:
>
>> You will have to parse the CSV file manually with a custom management
>> command (at least that is what I would do). All you need to do is open the
>> file, split each row with a "," and then  import the correct columns to the
>> model.
>>
>
> Unfortunately, CSV is a really bad, non-standardized format and simply
> splitting each row with a "," will cause you all kinds of issues with
> espacing values with a "," and other things.
>
> I would suggest using the python CSV library for this:
>
> https://docs.python.org/3/library/csv.html
>
> Just though that was worth mentioning.
>
> Kind regards,
>
> Kasper Laudrup
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/ms
> gid/django-users/83268fb2-22f6-2d6e-c14c-0015d77c48f0%40stacktrace.dk.
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAK4qSCcFzovjvS8M4sgTd8euRcWKWCgqC2xJWcu_qjCvuqdpaw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: How do I make a Django model from a tab-delimited data file?

[Kasper Laudrup]

Hi,

On 2018-01-09 08:09, Andréas Kühne wrote:
You will have to parse the CSV file manually with a custom management 
command (at least that is what I would do). All you need to do is open 
the file, split each row with a "," and then  import the correct columns 
to the model.


Unfortunately, CSV is a really bad, non-standardized format and simply 
splitting each row with a "," will cause you all kinds of issues with 
espacing values with a "," and other things.


I would suggest using the python CSV library for this:

https://docs.python.org/3/library/csv.html

Just though that was worth mentioning.

Kind regards,

Kasper Laudrup

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/83268fb2-22f6-2d6e-c14c-0015d77c48f0%40stacktrace.dk.
For more options, visit https://groups.google.com/d/optout.

Re: How do I make a Django model from a tab-delimited data file?

[Andréas Kühne]

Hi,

You will have to parse the CSV file manually with a custom management
command (at least that is what I would do). All you need to do is open the
file, split each row with a "," and then  import the correct columns to the
model.

You can also use something like pandas to convert the CSV file into
something that you can create the models from. But that may be overkill in
your case.

Regards,

Andréas

2018-01-09 3:38 GMT+01:00 Tom Tanner :

> I have a tab-delimited data file that looks something like this:
>
>
> NAME S1903_C02_001E state county tract State-County-Tract-ID
> Census Tract 201, Autauga County, Alabama 66000 01 001 020100 01001020100
> Census Tract 202, Autauga County, Alabama 41107 01 001 020200 01001020200
> Census Tract 203, Autauga County, Alabama 51250 01 001 020300 01001020300
>
> I want to make a Django model named `MyModel` with three columns: "name",
> "data", and "geoid", which correspond to the file's columns "NAME",
> "S1903_C02_001E", and "State-County-Tract-ID." Can I do this via command
> line, or with a custom Python script? I'm running my Django project locally
> on a computer running Debian 9.3.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-users/389d8f38-6dbc-43a0-8a69-d20aa13ca844%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAK4qSCfv_uvC5R89ykNxHzd1yJU-GZMSCGjVrGeLudAPC%3Dfc-g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

How do I make a Django model from a tab-delimited data file?

[Tom Tanner]

I have a tab-delimited data file that looks something like this:


NAME S1903_C02_001E state county tract State-County-Tract-ID
Census Tract 201, Autauga County, Alabama 66000 01 001 020100 01001020100
Census Tract 202, Autauga County, Alabama 41107 01 001 020200 01001020200
Census Tract 203, Autauga County, Alabama 51250 01 001 020300 01001020300

I want to make a Django model named `MyModel` with three columns: "name", 
"data", and "geoid", which correspond to the file's columns "NAME", 
"S1903_C02_001E", and "State-County-Tract-ID." Can I do this via command 
line, or with a custom Python script? I'm running my Django project locally 
on a computer running Debian 9.3. 

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/389d8f38-6dbc-43a0-8a69-d20aa13ca844%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.