Hi Dave, I have reopened following RM: ================================ https://redmine.postgresql.org/issues/1849
On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <dp...@pgadmin.org> wrote: > Patch applied. > > On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < > ashesh.va...@enterprisedb.com> wrote: > >> Hi Fahar, >> >> Please log the case on redmine. >> Please find the attached patch, please apply it locally, and test it. >> >> And, please update the case, and this mail chain accordingly. >> >> -- >> >> Thanks & Regards, >> >> Ashesh Vashi >> EnterpriseDB INDIA: Enterprise PostgreSQL Company >> <http://www.enterprisedb.com> >> >> >> *http://www.linkedin.com/in/asheshvashi* >> <http://www.linkedin.com/in/asheshvashi> >> >> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >> fahar.ab...@enterprisedb.com> wrote: >> >>> Here is the output of if we copy config_local.py and execute python >>> setup.py >>> pgAdmin 4 - Application Initialisation >>> ====================================== >>> >>> >>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>> not exist. >>> Entering initial setup mode... >>> NOTE: Configuring authentication for SERVER mode. >>> >>> >>> Enter the email address and password to use for the initial pgAdmin >>> user account: >>> >>> Email address: fahar.ab...@enterprisedb.com >>> Password: >>> Retype password: >>> Traceback (most recent call last): >>> File "setup.py", line 449, in <module> >>> do_setup(app) >>> File "setup.py", line 96, in do_setup >>> password = encrypt_password(p1) >>> File >>> "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>> line 150, in encrypt_password >>> signed = get_hmac(password).decode('ascii') >>> File >>> "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>> line 108, in get_hmac >>> 'set to "%s"' % _security.password_hash) >>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>> python setup.py >>> pgAdmin 4 - Application Initialisation >>> ====================================== >>> >>> User can not do any setup for web based now. >>> >>> >>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>> not exist. >>> Entering initial setup mode... >>> NOTE: Configuring authentication for SERVER mode. >>> >>> >>> Enter the email address and password to use for the initial pgAdmin >>> user account: >>> >>> Email address: fahar.ab...@enterprisedb.com >>> Password: >>> Retype password: >>> Traceback (most recent call last): >>> File "setup.py", line 449, in <module> >>> do_setup(app) >>> File "setup.py", line 96, in do_setup >>> password = encrypt_password(p1) >>> File >>> "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>> line 150, in encrypt_password >>> signed = get_hmac(password).decode('ascii') >>> File >>> "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>> line 108, in get_hmac >>> 'set to "%s"' % _security.password_hash) >>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>> >>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>> fahar.ab...@enterprisedb.com> wrote: >>> >>>> Dave, >>>> >>>> Testing Environment >>>> >>>> Ubuntu 16.04 Linux 64: >>>> -------------------------------- >>>> >>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>> >>>> >>>> 1) Install GIT >>>> >>>> sudo apt-get install git >>>> >>>> 2) Install pip3 >>>> >>>> sudo apt-get install python3-pip >>>> >>>> 3) Install virtualenv >>>> >>>> sudo pip3 install virtualenv >>>> >>>> 4) install below dependency as it is required for psycopg2 & pycrypto >>>> module >>>> >>>> sudo apt-get install libpq-dev >>>> >>>> sudo apt-get install python3-dev >>>> >>>> 5) Create virtual environment >>>> >>>> virtualenv -p python3 venv >>>> >>>> 6) Create mkdir Projects >>>> >>>> 7) Clone git repo in Projects >>>> >>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>> >>>> 8) activate virtual environment >>>> >>>> source venv/bin/activate >>>> >>>> 9) Install modules >>>> >>>> pip3 install -r requirements_py3.txt >>>> >>>> *10) Edit the config.py file to config_local.py resides in >>>> Projects\pgAdmin4\web * >>>> >>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>> python setup.py >>>> >>>> If user does not create config_local.py and do Python setup.py for new >>>> Development then SECURITY_PASSWORD_SALT message is also displayed: >>>> >>>> Here is the output: >>>> ------------------------- >>>> >>>> python setup.py >>>> pgAdmin 4 - Application Initialisation >>>> ====================================== >>>> >>>> >>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>> not exist. >>>> Entering initial setup mode... >>>> NOTE: Configuring authentication for SERVER mode. >>>> >>>> >>>> Enter the email address and password to use for the initial pgAdmin >>>> user account: >>>> >>>> Email address: fahar.ab...@enterprisedb.com >>>> Password: >>>> Retype password: >>>> Traceback (most recent call last): >>>> File "setup.py", line 449, in <module> >>>> do_setup(app) >>>> File "setup.py", line 96, in do_setup >>>> password = encrypt_password(p1) >>>> File >>>> "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 150, in encrypt_password >>>> signed = get_hmac(password).decode('ascii') >>>> File >>>> "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 108, in get_hmac >>>> 'set to "%s"' % _security.password_hash) >>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>>> be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>> "pbkdf2_sha512" >>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>> >>>> >>>> Is this expected? >>>> >>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>> fahar.ab...@enterprisedb.com> wrote: >>>> >>>>> Sure, >>>>> >>>>> Will test this thoroughly after complete investigation. >>>>> >>>>> Kind Regards, >>>>> >>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <dp...@pgadmin.org> wrote: >>>>> >>>>>> Patch applied. >>>>>> >>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>> modes, with both fresh and upgraded installations? >>>>>> >>>>>> https://redmine.postgresql.org/issues/1849 >>>>>> >>>>>> Packagers: This change means that packages are no longer forced to >>>>>> create a config_local.py file, and there is no longer any need to >>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for >>>>>> new >>>>>> installations, if you have included them in 1.0) >>>>>> >>>>>> Thanks. >>>>>> >>>>>> >>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>> ashesh.va...@enterprisedb.com> wrote: >>>>>> >>>>>>> Hi Dave, >>>>>>> >>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <dp...@pgadmin.org> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> >>>>>>>> On Friday, October 14, 2016, Dave Page <dp...@pgadmin.org> wrote: >>>>>>>> >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>> ashesh.va...@enterprisedb.com> wrote: >>>>>>>>> >>>>>>>>>> Hi Dave, >>>>>>>>>> >>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <dp...@pgadmin.org> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Ashesh, >>>>>>>>>>> >>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>> happy with it? >>>>>>>>>>> >>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen with >>>>>>>>>> 'runtime'. >>>>>>>>>> >>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>> 'web' mode. >>>>>>>>>> - Apply the patch >>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>> - Log out (if already login). >>>>>>>>>> >>>>>>>>>> And, you will see an exception. >>>>>>>>>> >>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>> the Security object. >>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Hmm. >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>> these configurations from the database. >>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>> >>>>>>>>> >>>>>>>>> OK, thanks. >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Now - I run into another issue. >>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>> >>>>>>>>>> I think - we need to think about different strategy for upgrading >>>>>>>>>> the configuration file in the 'web' mode. >>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>> >>>>>>>>> >>>>>>>>> My concern with that is that we'll likely be storing the default >>>>>>>>> config values in many cases, thus for those users, perpetuating the >>>>>>>>> problem. >>>>>>>>> >>>>>>>>> I guess what we need to do is re-encrypt the password during the >>>>>>>>> upgrade - however, that makes me think; we then have both the key and >>>>>>>>> the >>>>>>>>> encrypted passwords in the same database which is clearly not a good >>>>>>>>> idea. >>>>>>>>> Sigh... Needs more thought. >>>>>>>>> >>>>>>>> >>>>>>>> OK, so I've been thinking about this and experimenting for a couple >>>>>>>> of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>> loud >>>>>>>> in his general direction, and it looks like this isn't a major problem >>>>>>>> as >>>>>>>> from what I can see, SECURITY_PASSWORD_SALT is (aside from really >>>>>>>> being a >>>>>>>> key not a salt) not the only salting that's done. >>>>>>>> >>>>>>>> It looks like it's used system-wide as the key to generate an HMAC >>>>>>>> of the users password, which is then passed to passlib which salts and >>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>> password end up with different hashes in the database, so clearly >>>>>>>> there is >>>>>>>> also per-user salting happening. I also created two users, then >>>>>>>> dropped the >>>>>>>> database and created the same user accounts with the same passwords >>>>>>>> again, >>>>>>>> and found that the resulting hashes were different in both databases - >>>>>>>> thus >>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>> installations/databases. >>>>>>>> >>>>>>>> So, I believe we can do as you suggest and migrate existing values >>>>>>>> for SECURITY_PASSWORD_SALT, given that there's clearly some other per >>>>>>>> user >>>>>>>> and per installation/database salting going on anyway. New >>>>>>>> installations >>>>>>>> can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>> >>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT during >>>>>>> upgrade mode, which was wrong added in my addon patch. >>>>>>> >>>>>>> Please find the updated patch. >>>>>>> >>>>>>> Otherwise - looks good to me. >>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks & Regards, >>>>>>> >>>>>>> Ashesh Vashi >>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>> <http://www.enterprisedb.com/> >>>>>>> >>>>>>> >>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>> <http://www.linkedin.com/in/asheshvashi> >>>>>>> >>>>>>>> >>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>> either, as they're used for purposes that are essentially ephemeral, >>>>>>>> and >>>>>>>> thus can be changed during an upgrade. >>>>>>>> >>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>> >>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>> appreciated)! >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Dave Page >>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>> Twitter: @pgsnake >>>>>>>> >>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>> The Enterprise PostgreSQL Company >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Dave Page >>>>>> Blog: http://pgsnake.blogspot.com >>>>>> Twitter: @pgsnake >>>>>> >>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>> The Enterprise PostgreSQL Company >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Syed Fahar Abbas >>>>> Quality Management Group >>>>> >>>>> EnterpriseDB Corporation >>>>> Phone Office: +92-51-835-8874 >>>>> Phone Direct: +92-51-8466803 >>>>> Mobile: +92-333-5409707 >>>>> Skype ID: syed.fahar.abbas >>>>> Website: www.enterprisedb.com >>>>> >>>> >>>> >>>> >>>> -- >>>> Syed Fahar Abbas >>>> Quality Management Group >>>> >>>> EnterpriseDB Corporation >>>> Phone Office: +92-51-835-8874 >>>> Phone Direct: +92-51-8466803 >>>> Mobile: +92-333-5409707 >>>> Skype ID: syed.fahar.abbas >>>> Website: www.enterprisedb.com >>>> >>> >>> >>> >>> -- >>> Syed Fahar Abbas >>> Quality Management Group >>> >>> EnterpriseDB Corporation >>> Phone Office: +92-51-835-8874 >>> Phone Direct: +92-51-8466803 >>> Mobile: +92-333-5409707 >>> Skype ID: syed.fahar.abbas >>> Website: www.enterprisedb.com >>> >> >> > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com
