I assume that's an existing issue with Python 3.5? That file wasn't changed by this patch.
On Wed, Oct 19, 2016 at 1:11 PM, Neel Patel <neel.pa...@enterprisedb.com> wrote: > Hi, > > Just to update for Python 3. > It gives below error while running "pgAdmin4.py". > > ##### > > Traceback (most recent call last): > File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner > self.run() > File "/usr/lib/python3.4/threading.py", line 868, in run > self._target(*self._args, **self._kwargs) > File "/usr/lib/python3.4/socketserver.py", line 620, in > process_request_thread > self.handle_error(request, client_address) > File "/usr/lib/python3.4/socketserver.py", line 617, in > process_request_thread > self.finish_request(request, client_address) > File "/usr/lib/python3.4/socketserver.py", line 344, in finish_request > self.RequestHandlerClass(request, client_address, self) > File "/usr/lib/python3.4/socketserver.py", line 673, in __init__ > self.handle() > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/ > site-packages/werkzeug/serving.py", line 200, in handle > rv = BaseHTTPRequestHandler.handle(self) > File "/usr/lib/python3.4/http/server.py", line 398, in handle > self.handle_one_request() > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/ > site-packages/werkzeug/serving.py", line 235, in handle_one_request > return self.run_wsgi() > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/ > site-packages/werkzeug/serving.py", line 177, in run_wsgi > execute(self.server.app) > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/ > site-packages/werkzeug/serving.py", line 165, in execute > application_iter = app(environ, start_response) > File > "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 2000, in __call__ > return self.wsgi_app(environ, start_response) > File > "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 1991, in wsgi_app > response = self.make_response(self.handle_exception(e)) > File > "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 1567, in handle_exception > reraise(exc_type, exc_value, tb) > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/ > site-packages/flask/_compat.py", line 33, in reraise > raise value > File > "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 1988, in wsgi_app > response = self.full_dispatch_request() > File > "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 1643, in full_dispatch_request > response = self.process_response(response) > File > "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 1864, in process_response > self.save_session(ctx.session, response) > File > "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 926, in save_session > return self.session_interface.save_session(self, session, response) > File > "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", > line 267, in save_session > self.manager.put(session) > File > "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", > line 144, in put > self.parent.put(session) > File > "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", > line 214, in put > session.sign(self.secret) > File > "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", > line 71, in sign > self.hmac_digest = _calc_hmac('%s:%s' % (self.sid, self.randval), > secret) > File > "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", > line 44, in _calc_hmac > secret.encode(), body.encode(), hashlib.sha1 > AttributeError: 'bytes' object has no attribute 'encode' > ####### > > Thanks, > Neel Patel > > On Wed, Oct 19, 2016 at 5:12 PM, Fahar Abbas <fahar.ab...@enterprisedb.com > > wrote: > >> >> >> On Wed, Oct 19, 2016 at 4:03 PM, Fahar Abbas < >> fahar.ab...@enterprisedb.com> wrote: >> >>> >>> >>> On Wed, Oct 19, 2016 at 3:55 PM, Ashesh Vashi < >>> ashesh.va...@enterprisedb.com> wrote: >>> >>>> Hi Fahar, >>>> >>>> Please log the case on redmine. >>>> >>> https://redmine.postgresql.org/issues/1871 >>> >>>> Please find the attached patch, please apply it locally, and test it. >>>> >>>> And, please update the case, and this mail chain accordingly. >>>> >>> This is resolved now and no error message displayed when we apply the >> patch that is already shared. >> >>> >>>> Sure Will test the patch and update the status accordingly. >>> >>>> -- >>>> >>>> Thanks & Regards, >>>> >>>> Ashesh Vashi >>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>> <http://www.enterprisedb.com> >>>> >>>> >>>> *http://www.linkedin.com/in/asheshvashi* >>>> <http://www.linkedin.com/in/asheshvashi> >>>> >>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>> fahar.ab...@enterprisedb.com> wrote: >>>> >>>>> Here is the output of if we copy config_local.py and execute python >>>>> setup.py >>>>> pgAdmin 4 - Application Initialisation >>>>> ====================================== >>>>> >>>>> >>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>> not exist. >>>>> Entering initial setup mode... >>>>> NOTE: Configuring authentication for SERVER mode. >>>>> >>>>> >>>>> Enter the email address and password to use for the initial >>>>> pgAdmin user account: >>>>> >>>>> Email address: fahar.ab...@enterprisedb.com >>>>> Password: >>>>> Retype password: >>>>> Traceback (most recent call last): >>>>> File "setup.py", line 449, in <module> >>>>> do_setup(app) >>>>> File "setup.py", line 96, in do_setup >>>>> password = encrypt_password(p1) >>>>> File >>>>> "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 150, in encrypt_password >>>>> signed = get_hmac(password).decode('ascii') >>>>> File >>>>> "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 108, in get_hmac >>>>> 'set to "%s"' % _security.password_hash) >>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>> "pbkdf2_sha512" >>>>> python setup.py >>>>> pgAdmin 4 - Application Initialisation >>>>> ====================================== >>>>> >>>>> User can not do any setup for web based now. >>>>> >>>>> >>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>> not exist. >>>>> Entering initial setup mode... >>>>> NOTE: Configuring authentication for SERVER mode. >>>>> >>>>> >>>>> Enter the email address and password to use for the initial >>>>> pgAdmin user account: >>>>> >>>>> Email address: fahar.ab...@enterprisedb.com >>>>> Password: >>>>> Retype password: >>>>> Traceback (most recent call last): >>>>> File "setup.py", line 449, in <module> >>>>> do_setup(app) >>>>> File "setup.py", line 96, in do_setup >>>>> password = encrypt_password(p1) >>>>> File >>>>> "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 150, in encrypt_password >>>>> signed = get_hmac(password).decode('ascii') >>>>> File >>>>> "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 108, in get_hmac >>>>> 'set to "%s"' % _security.password_hash) >>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>> "pbkdf2_sha512" >>>>> >>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>> fahar.ab...@enterprisedb.com> wrote: >>>>> >>>>>> Dave, >>>>>> >>>>>> Testing Environment >>>>>> >>>>>> Ubuntu 16.04 Linux 64: >>>>>> -------------------------------- >>>>>> >>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>> >>>>>> >>>>>> 1) Install GIT >>>>>> >>>>>> sudo apt-get install git >>>>>> >>>>>> 2) Install pip3 >>>>>> >>>>>> sudo apt-get install python3-pip >>>>>> >>>>>> 3) Install virtualenv >>>>>> >>>>>> sudo pip3 install virtualenv >>>>>> >>>>>> 4) install below dependency as it is required for psycopg2 & pycrypto >>>>>> module >>>>>> >>>>>> sudo apt-get install libpq-dev >>>>>> >>>>>> sudo apt-get install python3-dev >>>>>> >>>>>> 5) Create virtual environment >>>>>> >>>>>> virtualenv -p python3 venv >>>>>> >>>>>> 6) Create mkdir Projects >>>>>> >>>>>> 7) Clone git repo in Projects >>>>>> >>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>> >>>>>> 8) activate virtual environment >>>>>> >>>>>> source venv/bin/activate >>>>>> >>>>>> 9) Install modules >>>>>> >>>>>> pip3 install -r requirements_py3.txt >>>>>> >>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>> Projects\pgAdmin4\web * >>>>>> >>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>> python setup.py >>>>>> >>>>>> If user does not create config_local.py and do Python setup.py for >>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>> >>>>>> Here is the output: >>>>>> ------------------------- >>>>>> >>>>>> python setup.py >>>>>> pgAdmin 4 - Application Initialisation >>>>>> ====================================== >>>>>> >>>>>> >>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>>> not exist. >>>>>> Entering initial setup mode... >>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>> >>>>>> >>>>>> Enter the email address and password to use for the initial >>>>>> pgAdmin user account: >>>>>> >>>>>> Email address: fahar.ab...@enterprisedb.com >>>>>> Password: >>>>>> Retype password: >>>>>> Traceback (most recent call last): >>>>>> File "setup.py", line 449, in <module> >>>>>> do_setup(app) >>>>>> File "setup.py", line 96, in do_setup >>>>>> password = encrypt_password(p1) >>>>>> File >>>>>> "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 150, in encrypt_password >>>>>> signed = get_hmac(password).decode('ascii') >>>>>> File >>>>>> "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 108, in get_hmac >>>>>> 'set to "%s"' % _security.password_hash) >>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>> "pbkdf2_sha512" >>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>> >>>>>> >>>>>> Is this expected? >>>>>> >>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>> fahar.ab...@enterprisedb.com> wrote: >>>>>> >>>>>>> Sure, >>>>>>> >>>>>>> Will test this thoroughly after complete investigation. >>>>>>> >>>>>>> Kind Regards, >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <dp...@pgadmin.org> >>>>>>> wrote: >>>>>>> >>>>>>>> Patch applied. >>>>>>>> >>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>> >>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>> >>>>>>>> Packagers: This change means that packages are no longer forced to >>>>>>>> create a config_local.py file, and there is no longer any need to >>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed >>>>>>>> for new >>>>>>>> installations, if you have included them in 1.0) >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>> ashesh.va...@enterprisedb.com> wrote: >>>>>>>> >>>>>>>>> Hi Dave, >>>>>>>>> >>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <dp...@pgadmin.org> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Friday, October 14, 2016, Dave Page <dp...@pgadmin.org> wrote: >>>>>>>>>> >>>>>>>>>>> Hi >>>>>>>>>>> >>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>> ashesh.va...@enterprisedb.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Dave, >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <dp...@pgadmin.org> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>> >>>>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>>>> happy with it? >>>>>>>>>>>>> >>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>> >>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>>> 'web' mode. >>>>>>>>>>>> - Apply the patch >>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>> >>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>> >>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>>>> the Security object. >>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hmm. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>>> these configurations from the database. >>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK, thanks. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>> >>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>> configurations in the database during upgrade process in 'web' >>>>>>>>>>>> mode. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> My concern with that is that we'll likely be storing the default >>>>>>>>>>> config values in many cases, thus for those users, perpetuating the >>>>>>>>>>> problem. >>>>>>>>>>> >>>>>>>>>>> I guess what we need to do is re-encrypt the password during the >>>>>>>>>>> upgrade - however, that makes me think; we then have both the key >>>>>>>>>>> and the >>>>>>>>>>> encrypted passwords in the same database which is clearly not a >>>>>>>>>>> good idea. >>>>>>>>>>> Sigh... Needs more thought. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by >>>>>>>>>> thinking out >>>>>>>>>> loud in his general direction, and it looks like this isn't a major >>>>>>>>>> problem >>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from >>>>>>>>>> really being >>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>> >>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>> HMAC of the users password, which is then passed to passlib which >>>>>>>>>> salts and >>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>> password end up with different hashes in the database, so clearly >>>>>>>>>> there is >>>>>>>>>> also per-user salting happening. I also created two users, then >>>>>>>>>> dropped the >>>>>>>>>> database and created the same user accounts with the same passwords >>>>>>>>>> again, >>>>>>>>>> and found that the resulting hashes were different in both databases >>>>>>>>>> - thus >>>>>>>>>> there is something else ensuring the hashes are unique across >>>>>>>>>> different >>>>>>>>>> installations/databases. >>>>>>>>>> >>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some >>>>>>>>>> other >>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>> >>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>> >>>>>>>>> Please find the updated patch. >>>>>>>>> >>>>>>>>> Otherwise - looks good to me. >>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> Thanks & Regards, >>>>>>>>> >>>>>>>>> Ashesh Vashi >>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>> <http://www.enterprisedb.com/> >>>>>>>>> >>>>>>>>> >>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>> <http://www.linkedin.com/in/asheshvashi> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, >>>>>>>>>> and >>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>> >>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>> >>>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>>> appreciated)! >>>>>>>>>> >>>>>>>>>> Thanks. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Dave Page >>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>> Twitter: @pgsnake >>>>>>>>>> >>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Dave Page >>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>> Twitter: @pgsnake >>>>>>>> >>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>> The Enterprise PostgreSQL Company >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Syed Fahar Abbas >>>>>>> Quality Management Group >>>>>>> >>>>>>> EnterpriseDB Corporation >>>>>>> Phone Office: +92-51-835-8874 >>>>>>> Phone Direct: +92-51-8466803 >>>>>>> Mobile: +92-333-5409707 >>>>>>> Skype ID: syed.fahar.abbas >>>>>>> Website: www.enterprisedb.com >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Syed Fahar Abbas >>>>>> Quality Management Group >>>>>> >>>>>> EnterpriseDB Corporation >>>>>> Phone Office: +92-51-835-8874 >>>>>> Phone Direct: +92-51-8466803 >>>>>> Mobile: +92-333-5409707 >>>>>> Skype ID: syed.fahar.abbas >>>>>> Website: www.enterprisedb.com >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Syed Fahar Abbas >>>>> Quality Management Group >>>>> >>>>> EnterpriseDB Corporation >>>>> Phone Office: +92-51-835-8874 >>>>> Phone Direct: +92-51-8466803 >>>>> Mobile: +92-333-5409707 >>>>> Skype ID: syed.fahar.abbas >>>>> Website: www.enterprisedb.com >>>>> >>>> >>>> >>> >>> >>> -- >>> Syed Fahar Abbas >>> Quality Management Group >>> >>> EnterpriseDB Corporation >>> Phone Office: +92-51-835-8874 >>> Phone Direct: +92-51-8466803 >>> Mobile: +92-333-5409707 >>> Skype ID: syed.fahar.abbas >>> Website: www.enterprisedb.com >>> >> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com >> > > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company