Re: [mezzanine-users] Syncing postgres dev and production databases...
Absolutely Ken. What I've posted is only for initial development. I'm looking into your production backup solution now. Thanks for the info! On Tuesday, January 23, 2018 at 4:40:20 PM UTC-6, Kenneth Bolton wrote: > > I want to re-emphasize that this should only be done before you are live > in production, and then only once! Your customers will hate you if you blow > away their prod database after they've started blogging. > > The automated process goes in the other direction, allowing us to take a > dump of production and migrate it back to qa, dev, and local. For that, > I've created DEV_HOSTS and QA_HOSTS in settings.FABRIC and used `fab > backup` to create the backup, a custom rsync command to move the file to > the hosts, and a custom `fab restore`-like command to restore the database > to dev and qa. > > -ken > > On Tue, Jan 23, 2018 at 5:18 PM, > wrote: > >> I'm very new to Mezzanine. Been working with it for about a week and a >> half now. And let me say, once you get your head around the structure and >> methodology, it's just an awesome framework! >> So nice to break away from the standard CMS offerings and gain full >> control. >> >> I've setup a local dev environment and a remote Linode production server. >> it took a minute but I got deployment through Fabric worked out and it's >> now working well, and repeatable. >> I'm using postgres as my database platform and the one thing that was >> giving me a bit if a fit is syncing the local and remote databases. I've >> tried the django and fab bacup and restore >> utilities but they are prone to a number of permissions issues. These >> could be resolved but it would require a change in my established project >> structure, or relatively extensive changes to >> the dev and production environments or postgres servers. I wanted >> something that required fewer changes to the default environments. I >> finally came up with a method that is working well, at least for now. >> It could no doubt be automated via a set of Fabric tasks. But i thought >> I'd share my "manual" method for those having the same issue. It may be a >> little unorthodox but it's working well, and is repeatable. >> >> Here it s... >> >> *Keeping in mind that this will delete any db based content or user data >> on the remote db.* But for initial development it's great! And I believe >> it could be easily modified to backup and restore only those tables >> required for the site structure and content. >> I suppose you could also do this in reverse to sync the dev db to the >> production db for backup and to make sure you're developing and testing >> against real world data. >> >> *On development server:* >> sudo -u postgres pg_dump > dump.sql >> >> sed -i -e 's///g' dump.sql *(NOTE: >> This is only required if the local and remote db roles are not the same.)* >> >> scp ./dump.sql @> IP>:> server>/dump.sql >> >> *On production server: NOT*E >> dropdb -h localhost -U >> sudo -u postgres createdb -O >> sudo -u postgres < > server>dump.sql >> >> I'm certain there are many other methods to get this don but this is what >> I came up with. >> I've developed this under the conditions specific to my dev and >> production environments so any comments, suggestions or questions are more >> than welcome. >> >> Cheers! :) >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Mezzanine Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to mezzanine-use...@googlegroups.com . >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Mezzanine Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[mezzanine-users] Syncing postgres dev and production databases...
I'm very new to Mezzanine. Been working with it for about a week and a half now. And let me say, once you get your head around the structure and methodology, it's just an awesome framework! So nice to break away from the standard CMS offerings and gain full control. I've setup a local dev environment and a remote Linode production server. it took a minute but I got deployment through Fabric worked out and it's now working well, and repeatable. I'm using postgres as my database platform and the one thing that was giving me a bit if a fit is syncing the local and remote databases. I've tried the django and fab bacup and restore utilities but they are prone to a number of permissions issues. These could be resolved but it would require a change in my established project structure, or relatively extensive changes to the dev and production environments or postgres servers. I wanted something that required fewer changes to the default environments. I finally came up with a method that is working well, at least for now. It could no doubt be automated via a set of Fabric tasks. But i thought I'd share my "manual" method for those having the same issue. It may be a little unorthodox but it's working well, and is repeatable. Here it s... *Keeping in mind that this will delete any db based content or user data on the remote db.* But for initial development it's great! And I believe it could be easily modified to backup and restore only those tables required for the site structure and content. I suppose you could also do this in reverse to sync the dev db to the production db for backup and to make sure you're developing and testing against real world data. *On development server:* sudo -u postgres pg_dump > dump.sql sed -i -e 's///g' dump.sql *(NOTE: This is only required if the local and remote db roles are not the same.)* scp ./dump.sql @:/dump.sql *On production server: NOT*E dropdb -h localhost -U sudo -u postgres createdb -O sudo -u postgres < dump.sql I'm certain there are many other methods to get this don but this is what I came up with. I've developed this under the conditions specific to my dev and production environments so any comments, suggestions or questions are more than welcome. Cheers! :) -- You received this message because you are subscribed to the Google Groups "Mezzanine Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [mezzanine-users] Database not populated using Fabric deploy
OK, I did manage to get logged into the admin pages by creating a django admin supersuer on the server. Just need a reliable method for syncing databases. On Thursday, January 18, 2018 at 5:50:52 PM UTC-6, Eduardo Rivas wrote: > > Hello! > > The fabfile assumes your test database contains data not suitable for > production. You will need to manually dump and restore your DB in the > production server if you want to transfer your development data. My > suggestion is you create your own Fabric task to automate the process if > this is something you'll need to do often. > > As for the migrations, they are automatically run on first deploy, so > unless you've created new migrations, nothing will be applied when running > "migrate". What you have in production is a database with all tables and > columns created but no rows. > > Hope that helps :) > > On 2018-01-18 5:40 PM, rwr@gmail.com wrote: > > Hi all > > I'm brand new to Mezzanine, but not to Django. > > I created a test mezzanine project and it runs perfectly on loclhost. It's > a default project with the provided sample data, no mods at all. > > I deployed the project to a fresh Linode Debian Stretch server using 'fab > secure' and then 'fab all'. Everything seemed to work perfectly, I can hit > the site and see the Mezzanine homepage no problem. But it doesn't seem to > have populated the database on the server at all. I'm using postgres and > can see, using psql, that the project database was created. I also can't > login into the admin page of the site. Again, I believe this is simply > because database wasn't populated. All of the project log files on the > server are empty. > > I also ran 'fab deploy' and it seemed to work but tells me there were no > migrations to apply. I also ran 'python manage.py migrate', which also > reported no migrations to apply. > > What information can I provide to help understand what's going on here. > Or, is there just something, some step maybe, that I missed? > > Thanks in advance for ny help. > -- > You received this message because you are subscribed to the Google Groups > "Mezzanine Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to mezzanine-use...@googlegroups.com . > For more options, visit https://groups.google.com/d/optout. > > > -- You received this message because you are subscribed to the Google Groups "Mezzanine Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [mezzanine-users] Database not populated using Fabric deploy
OK, time to seek help again... I have tried a number of backup/dump and restore/load mechanisms to try and sync data between my local development project db and the server production db. Again, fab deploy works great and I can access the site on my server. But no matter what db sync method I use, once the db is restored on the server side, I get an Internal Server Error (http 500) error. I've tried pg_dump/pg_restore, fab backup/restore, python manage.py dumpdata/loaddata. I also tried pgAdmin backup/restore. Every one produces the same results. I've also created another project with no sample data, same results. Again, it deploys perfectly and is accessible. I'd be OK with this except once the site is on the production server I can't login to admin. The same situation if I include sample data in the project creation. I've dug through the group posts and found lots of examples of folks using several methods for database sync but I can't get anything to work. I'm sure this shouldn't be this difficult and that I'm missing something dumb, but I can't for the life of me find it. Any suggestions? On Thursday, January 18, 2018 at 5:50:52 PM UTC-6, Eduardo Rivas wrote: > > Hello! > > The fabfile assumes your test database contains data not suitable for > production. You will need to manually dump and restore your DB in the > production server if you want to transfer your development data. My > suggestion is you create your own Fabric task to automate the process if > this is something you'll need to do often. > > As for the migrations, they are automatically run on first deploy, so > unless you've created new migrations, nothing will be applied when running > "migrate". What you have in production is a database with all tables and > columns created but no rows. > > Hope that helps :) > > On 2018-01-18 5:40 PM, rwr@gmail.com wrote: > > Hi all > > I'm brand new to Mezzanine, but not to Django. > > I created a test mezzanine project and it runs perfectly on loclhost. It's > a default project with the provided sample data, no mods at all. > > I deployed the project to a fresh Linode Debian Stretch server using 'fab > secure' and then 'fab all'. Everything seemed to work perfectly, I can hit > the site and see the Mezzanine homepage no problem. But it doesn't seem to > have populated the database on the server at all. I'm using postgres and > can see, using psql, that the project database was created. I also can't > login into the admin page of the site. Again, I believe this is simply > because database wasn't populated. All of the project log files on the > server are empty. > > I also ran 'fab deploy' and it seemed to work but tells me there were no > migrations to apply. I also ran 'python manage.py migrate', which also > reported no migrations to apply. > > What information can I provide to help understand what's going on here. > Or, is there just something, some step maybe, that I missed? > > Thanks in advance for ny help. > -- > You received this message because you are subscribed to the Google Groups > "Mezzanine Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to mezzanine-use...@googlegroups.com . > For more options, visit https://groups.google.com/d/optout. > > > -- You received this message because you are subscribed to the Google Groups "Mezzanine Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [mezzanine-users] Database not populated using Fabric deploy
Thanks for the quick reply, Eduardo . That makes perfect sense. That's why I didn't figure it out myself. lol... I thought about the migration issue right after posting. It doesn't have anything to do with data, only model structure. Thanks again for the help. On Thursday, January 18, 2018 at 5:50:52 PM UTC-6, Eduardo Rivas wrote: > > Hello! > > The fabfile assumes your test database contains data not suitable for > production. You will need to manually dump and restore your DB in the > production server if you want to transfer your development data. My > suggestion is you create your own Fabric task to automate the process if > this is something you'll need to do often. > > As for the migrations, they are automatically run on first deploy, so > unless you've created new migrations, nothing will be applied when running > "migrate". What you have in production is a database with all tables and > columns created but no rows. > > Hope that helps :) > > On 2018-01-18 5:40 PM, rwr@gmail.com wrote: > > Hi all > > I'm brand new to Mezzanine, but not to Django. > > I created a test mezzanine project and it runs perfectly on loclhost. It's > a default project with the provided sample data, no mods at all. > > I deployed the project to a fresh Linode Debian Stretch server using 'fab > secure' and then 'fab all'. Everything seemed to work perfectly, I can hit > the site and see the Mezzanine homepage no problem. But it doesn't seem to > have populated the database on the server at all. I'm using postgres and > can see, using psql, that the project database was created. I also can't > login into the admin page of the site. Again, I believe this is simply > because database wasn't populated. All of the project log files on the > server are empty. > > I also ran 'fab deploy' and it seemed to work but tells me there were no > migrations to apply. I also ran 'python manage.py migrate', which also > reported no migrations to apply. > > What information can I provide to help understand what's going on here. > Or, is there just something, some step maybe, that I missed? > > Thanks in advance for ny help. > -- > You received this message because you are subscribed to the Google Groups > "Mezzanine Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to mezzanine-use...@googlegroups.com . > For more options, visit https://groups.google.com/d/optout. > > > -- You received this message because you are subscribed to the Google Groups "Mezzanine Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[mezzanine-users] Database not populated using Fabric deploy
Hi all I'm brand new to Mezzanine, but not to Django. I created a test mezzanine project and it runs perfectly on loclhost. It's a default project with the provided sample data, no mods at all. I deployed the project to a fresh Linode Debian Stretch server using 'fab secure' and then 'fab all'. Everything seemed to work perfectly, I can hit the site and see the Mezzanine homepage no problem. But it doesn't seem to have populated the database on the server at all. I'm using postgres and can see, using psql, that the project database was created. I also can't login into the admin page of the site. Again, I believe this is simply because database wasn't populated. All of the project log files on the server are empty. I also ran 'fab deploy' and it seemed to work but tells me there were no migrations to apply. I also ran 'python manage.py migrate', which also reported no migrations to apply. What information can I provide to help understand what's going on here. Or, is there just something, some step maybe, that I missed? Thanks in advance for ny help. -- You received this message because you are subscribed to the Google Groups "Mezzanine Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
