I appreciate this, Adam. I may end up uninstalling the newer version of postgreSQL anyway, but will de4finitely start here.
Thanks, Martha On Tuesday, January 7, 2020 at 2:50:27 PM UTC-8, Adam Cox wrote: > > Hm, well I hope this helps. Here's one v4 instance I currently have > running on HTTPS without any trouble. https://csh.legiongis.com. A little > examination of that server shows: > > (env)$ pip freeze > > asn1crypto==0.24.0 > backports-abc==0.5 > backports.functools-lru-cache==1.5 > bcrypt==3.1.7 > beautifulsoup4==4.7.1 > certifi==2019.6.16 > cffi==1.12.3 > chardet==3.0.4 > CouchDB==1.2 > coverage==4.5.3 > cryptography==2.7 > Django==1.11.15 > django-cors-headers==2.2.0 > django-guardian==1.4.9 > django-nose==1.4.6 > django-oauth-toolkit==1.1.2 > django-recaptcha==1.4.0 > django-revproxy==0.9.15 > django-webtest==1.9.6 > ecdsa==0.13.2 > edtf==2.7.0 > elasticsearch==5.5.3 > enum34==1.1.6 > extras==1.0.0 > fabric==2.4.0 > fixtures==3.0.0 > funcsigs==1.0.2 > future==0.17.1 > futures==3.2.0 > idna==2.8 > invoke==1.2.0 > ipaddress==1.0.22 > isodate==0.6.0 > junitxml==0.7 > linecache2==1.0.0 > livereload==2.6.1 > mapbox-vector-tile==1.2.0 > mapnik==0.1 > mock==2.0.0 > ModestMaps==1.4.7 > nose==1.3.7 > oauthlib==3.0.1 > paramiko==2.5.0 > pbr==5.3.1 > Pillow==6.0.0 > protobuf==3.8.0 > psycopg2-binary==2.7.4 > pyclipper==1.1.0.post1 > pycparser==2.19 > pycryptodome==3.8.2 > PyLD==1.0.4 > PyNaCl==1.3.0 > pyOpenSSL==19.0.0 > pyparsing==2.4.0 > PyPrind==2.11.2 > python-dateutil==2.8.0 > python-jose==2.0.2 > python-memcached==1.59 > python-mimeparse==1.6.0 > python-subunit==1.3.0 > pytz==2019.1 > PyYAML==3.12 > rdflib==4.2.2 > requests==2.22.0 > sauceclient==1.0.0 > selenium==3.141.0 > Shapely==1.6.4.post2 > simplejson==3.16.0 > singledispatch==3.4.0.3 > six==1.12.0 > soupsieve==1.9.1 > SPARQLWrapper==1.8.2 > sst==0.2.4 > testtools==2.3.0 > TileStache==1.51.13 > tornado==5.1.1 > traceback2==1.4.0 > unicodecsv==0.14.1 > unittest2==1.1.0 > urllib3==1.21.1 > waitress==1.3.0 > WebOb==1.8.5 > WebTest==2.0.33 > Werkzeug==0.15.4 > > --- > > (env)$ dpkg -l | grep wsgi > ii libapache2-mod-wsgi 4.5.17-1 > amd64 Python WSGI adapter module for Apache > > --- > > (env)$ apache2 -v > Server version: Apache/2.4.29 (Ubuntu) > Server built: 2019-09-16T12:58:48 > > --- > > (I'm not sure if you're using certbot/Let's Encrypt, but for what it's > worth) > > $ certbot --version > certbot 0.31.0 > > --- > > On that server, postgres is not installed, but psql is. > > $ psql --version > psql (PostgreSQL) 10.11 (Ubuntu 10.11-1.pgdg18.04+1) > > Hope that all helps, hope I'm not missing anything. > > Adam > > On Mon, Jan 6, 2020 at 5:07 PM Martha S <[email protected] <javascript:>> > wrote: > >> Adam, >> >> I worked my way down the psycopg2 version list with no success. I made >> sure to restart Apache after every uninstall/reinstall.. I finally had to >> quit at version 2.6.2, which could not be installed. Here's the quirk: I >> have both postgres 9.6.15 and 11.4-1 on my server. The arches database is >> 9.6.15 and I can't find anything in the forum ever suggesting that the >> database be upgraded. I imagine the server was initially provisioned with >> 11.5, but we added 9.6 during the arches install. >> >> This has not previously been an issue, but it is now that we want to >> implement SSL. While I couldn't get relief with any of the higher psycopg2 >> versions, version 2.6.2 complains Error: could not determine PostgreSQL >> version from '11.5'. That's not a typo, even though the version claims to >> be 11.4-1; psycopg2 won't install. Too bad, because it's the last version >> that worked for some folks when this issue first came to light. >> >> So, I know from our earlier conversation you thought I should perhaps >> reinstall mod_wsgi at this point, but now I'm wondering if I should >> uninstall postgres 11.4-1 instead and try again to replace psycopg2 from >> the top of the release list or if there's some other path you'd recommend. >> >> I truly appreciate you hanging in there with me, >> Martha >> >> >> On Monday, January 6, 2020 at 11:52:38 AM UTC-8, Martha S wrote: >>> >>> Thanks again, Adam, >>> >>> I'll report back when I've taken this next step, in case others have to >>> deal with this issue. >>> >>> Martha >>> >>> On Monday, January 6, 2020 at 11:48:15 AM UTC-8, Adam Cox wrote: >>>> >>>> Ok, well the key when I fixed this for myself was finding the release >>>> of psycopg2 that comes with the correct OpenSSL version: >>>> http://initd.org/psycopg/docs/news.html >>>> >>>> Hopefully if you just uninstall psycopg2 (and psycopg2-binary) from >>>> your virtual environment and then look at that release list and try >>>> installing/uninstalling releases that have different OpenSSL versions, >>>> then >>>> one of them should work. At least, that's the easiest next thing to try. >>>> >>>> (env)$ pip install psycopg2==x.x.x >>>> >>>> If none of the changes to that python library work then maybe you >>>> should go the route of reinstalling mod_wsgi. >>>> >>>> Adam >>>> >>>> On Mon, Jan 6, 2020 at 12:51 PM Martha S <[email protected]> wrote: >>>> >>>>> Yes, Adam, >>>>> >>>>> I can confirm I restarted Apache after the initial psycopg2 >>>>> uninstall/install, after uninstalling the binary, after redoing the >>>>> uninstall/install, and after adding the wsgi parameter to my config file. >>>>> Nope at every turn. >>>>> >>>>> If you don't feel wsgi is the problem, I'll proceed with installing >>>>> psycopg2 from GitHub, unless you feel that is inadvisable. I'm thinking >>>>> that there must be some difference in our server setups that made the >>>>> cure >>>>> for you not work for me. >>>>> >>>>> Thanks, >>>>> Martha >>>>> >>>>> On Monday, January 6, 2020 at 10:41:30 AM UTC-8, Adam Cox wrote: >>>>>> >>>>>> Hi Martha, these libssl libraries are the root of the problem (at >>>>>> least, when I tracked it down this is exactly where I ended up). As to >>>>>> your >>>>>> steps above, you do need to restart apache after every change to the >>>>>> virtual environment. Can you confirm that you did that right after you >>>>>> made >>>>>> the first change to the psycopg2 library? I'm not able to look at my own >>>>>> installations at the moment, but I have never had to use that >>>>>> WSGIApplicationGroup parameter. I was able to fix it by just updating >>>>>> psycopg2 (I can't remember of -binary was in use for me or not). I'll >>>>>> let >>>>>> you know when I get a chance to look. >>>>>> >>>>>> Adam >>>>>> >>>>>> On Sun, Jan 5, 2020 at 7:44 PM Martha S <[email protected]> wrote: >>>>>> >>>>>>> I've found one more item in the psycopg 2.8.4 docs. Should this be >>>>>>> my next step? >>>>>>> >>>>>>> Warning >>>>>>> >>>>>>> >>>>>>> The psycopg2 wheel package comes packaged, among the others, with >>>>>>> its own libssl binary. This may create conflicts with other >>>>>>> extension modules binding with libssl as well, for instance with >>>>>>> the Python ssl >>>>>>> <https://docs.python.org/3/library/ssl.html#module-ssl> module: in >>>>>>> some cases, under concurrency, the interaction between the two >>>>>>> libraries >>>>>>> may result in a segfault. In case of doubts you are advised to use a >>>>>>> package built from source. >>>>>>> >>>>>>> >>>>>>> On Sunday, January 5, 2020 at 1:30:56 PM UTC-8, Martha S wrote: >>>>>>>> >>>>>>>> Adam, >>>>>>>> >>>>>>>> Unfortunately, this did not repair the startup error. Once I >>>>>>>> created a new virtual environment, installed arches, >>>>>>>> uninstalled/re-installed psycopg2, corrected a directory oversight in >>>>>>>> my >>>>>>>> config file, and pointed to the new environment in apache2, I still >>>>>>>> got the >>>>>>>> following in my apache2 error.log and no coredump: >>>>>>>> >>>>>>>> [Sun Jan 05 19:58:35.485867 2020] [mpm_event:notice] [pid 37579:tid >>>>>>>> 139672454474688] AH00491: caught SIGTERM, shutting down >>>>>>>> >>>>>>>> [Sun Jan 05 19:58:35.636147 2020] [mpm_event:notice] [pid 1993:tid >>>>>>>> 140438837296064] AH00489: Apache/2.4.29 (Ubuntu) OpenSSL/1.1.1 >>>>>>>> mod_wsgi/4.5.17 Python/2.7 configured -- resuming normal operations >>>>>>>> >>>>>>>> [Sun Jan 05 19:58:35.636244 2020] [core:notice] [pid 1993:tid >>>>>>>> 140438837296064] AH00094: Command line: '/usr/sbin/apache2' >>>>>>>> >>>>>>>> [Sun Jan 05 19:58:49.346899 2020] [wsgi:error] [pid 2006:tid >>>>>>>> 140438713235200] [client 161.149.102.22:2459] wsgi:error 'arches': >>>>>>>> /Projects/ssl/hpla/hpla/wsgi.py >>>>>>>> >>>>>>>> [Sun Jan 05 19:58:49.650548 2020] [core:notice] [pid 1993:tid >>>>>>>> 140438837296064] AH00051: child pid 2002 exit signal Segmentation >>>>>>>> fault >>>>>>>> (11), possible coredump in /etc/apache2 >>>>>>>> >>>>>>>> [Sun Jan 05 19:58:50.731314 2020] [wsgi:error] [pid 2004:tid >>>>>>>> 140438585472768] [client 161.149.102.22:2460] Truncated or >>>>>>>> oversized response headers received from daemon process 'arches': >>>>>>>> /Projects/ssl/hpla/hpla/wsgi.py, referer: >>>>>>>> https://historicplacesla.lacity.org/ >>>>>>>> >>>>>>>> >>>>>>>> I went back to my new environment to double-check psycopg2 and >>>>>>>> found: >>>>>>>> >>>>>>>> (ENV)hpla:/Projects/ssl$ pip freeze | grep psycopg2 >>>>>>>> >>>>>>>> DEPRECATION: Python 2.7 nag >>>>>>>> >>>>>>>> psycopg2==2.8.3 >>>>>>>> >>>>>>>> psycopg2-binary==2.7.4 >>>>>>>> >>>>>>>> I figured the binary was the problem and uninstalled that. Nope. >>>>>>>> So I uninstalled/reinstalled psycopg2 and see that psycopg2==2.8.3 >>>>>>>> and no binary is listed. Nope. >>>>>>>> Just because, I restarted apache2. Nope. The error regarding >>>>>>>> truncated or oversized response headers remains and there is no >>>>>>>> coredump. >>>>>>>> I found a suggestion online that I should add WSGIApplicationGroup >>>>>>>> %{GLOBAL} to my config file. Nope. >>>>>>>> >>>>>>>> I see other suggestions, like switching from the system mod-wsgi to >>>>>>>> the Python package and the psycopg2-binary package, but still hope for >>>>>>>> further guidance here from anyone successfully serving arches with >>>>>>>> HTTPS on >>>>>>>> Ubuntu 18.04. Is mod-wsgi 4.5.17 a problem? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Martha >>>>>>>> >>>>>>>> >>>>>>>> On Saturday, December 28, 2019 at 6:38:02 AM UTC-8, Adam Cox wrote: >>>>>>>>> >>>>>>>>> Hi Martha, the daemon process error has to do with your config >>>>>>>>> file, you'll just need to make sure the arches process is created >>>>>>>>> before >>>>>>>>> you reference it. >>>>>>>>> >>>>>>>>> The other issue, the truncated........ Is much trickier, but has >>>>>>>>> to do specifically with the version of psycopg2 (a python library in >>>>>>>>> your >>>>>>>>> virtual environment) that you have installed. I put this comment on a >>>>>>>>> PR a >>>>>>>>> long time ago, which gives a little context: >>>>>>>>> https://github.com/archesproject/arches/pull/4965#issuecomment-521806491 >>>>>>>>> >>>>>>>>> Ultimately, I recommend creating a new virtual environment, pip >>>>>>>>> installing everything as normal, then >>>>>>>>> >>>>>>>>> Pip uninstall psycopg2 >>>>>>>>> Pip install psycopg2==2.8.3 >>>>>>>>> >>>>>>>>> You can then point the apache configs at the new environment and >>>>>>>>> it should solve this problem. >>>>>>>>> >>>>>>>>> Good luck, >>>>>>>>> Adam >>>>>>>>> >>>>>>>>> On Fri, Dec 27, 2019, 6:38 PM Martha S <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> I feel I'm sooo close to implementing secured access to Arches >>>>>>>>>> and need a little help to get over the top, please. I'm running >>>>>>>>>> Ubuntu >>>>>>>>>> 18.04. My certificate is in and working. >>>>>>>>>> >>>>>>>>>> I tried making a copy of 000-default.conf (with --preserve=all), >>>>>>>>>> changing the port to 443, and adding these lines as directed by the >>>>>>>>>> CA: >>>>>>>>>> >>>>>>>>>> ServerName historicplacesla.lacity.org >>>>>>>>>> SSLEngine on >>>>>>>>>> >>>>>>>>>> SSLCertificateFile XXX.crt >>>>>>>>>> SSLCertificateKeyFile XXX.key >>>>>>>>>> >>>>>>>>>> When I tried to restart Apache2 after enabling SSL, the complaint >>>>>>>>>> was 'Name duplicates previous WSGI daemon definition', so I >>>>>>>>>> commented it >>>>>>>>>> out in default-ssl.conf and tried again. Apache started, but I still >>>>>>>>>> can't >>>>>>>>>> launch Arches. The browser gets an error 500 and the Arches >>>>>>>>>> error.log has >>>>>>>>>> the entry: >>>>>>>>>> >>>>>>>>>> 'Daemon process called 'arches' cannot be accessed by this WSGI >>>>>>>>>> application: /Projects/prod/hpla/hpla/wsgi.py, referer: >>>>>>>>>> https://historicplacesla.lacity.org/arches.' >>>>>>>>>> >>>>>>>>>> I get the same error if I try with http. My next effort was to >>>>>>>>>> rename my 000-default.conf and uncomment the line in >>>>>>>>>> default-ssl.conf. Now, >>>>>>>>>> in addition to the error above, I get >>>>>>>>>> >>>>>>>>>> '[wsgi:error] Truncated or oversized response headers received >>>>>>>>>> from daemon process 'arches': /Projects/prod/hpla/hpla/wsgi.py >>>>>>>>>> [core:notice] AH00051: child pid 83357 exit signal Segmentation >>>>>>>>>> fault (11), possible coredump in /etc/apache2 >>>>>>>>>> >>>>>>>>>> There is no new file, dump or otherwise in /etc/apache2. >>>>>>>>>> >>>>>>>>>> For the moment, I've disabled SSL, renamed the two configuration >>>>>>>>>> files so only 000-default.conf is loaded, restarted Apache2 and >>>>>>>>>> things are >>>>>>>>>> running as usual. I get that it is at least partly an access rights >>>>>>>>>> issue. >>>>>>>>>> historicplacesla.lacity.org has been added to ALLOWED_HOSTS in >>>>>>>>>> settings.py. >>>>>>>>>> >>>>>>>>>> What do I still need to do to what to deal with these errors? >>>>>>>>>> >>>>>>>>>> Thanks so much and happy new year to all, >>>>>>>>>> Martha >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> -- To post, send email to [email protected]. To >>>>>>>>>> unsubscribe, send email to [email protected]. For more >>>>>>>>>> information, visit >>>>>>>>>> https://groups.google.com/d/forum/archesproject?hl=en >>>>>>>>>> --- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "Arches Project" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/archesproject/d5bebd17-14c3-4925-b5e7-ba5128d446d0%40googlegroups.com >>>>>>>>>> >>>>>>>>>> <https://groups.google.com/d/msgid/archesproject/d5bebd17-14c3-4925-b5e7-ba5128d446d0%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>> -- To post, send email to [email protected]. To >>>>>>> unsubscribe, send email to [email protected]. For more >>>>>>> information, visit >>>>>>> https://groups.google.com/d/forum/archesproject?hl=en >>>>>>> --- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Arches Project" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/archesproject/e14b6b6c-b617-4279-b059-bc3c941643c8%40googlegroups.com >>>>>>> >>>>>>> <https://groups.google.com/d/msgid/archesproject/e14b6b6c-b617-4279-b059-bc3c941643c8%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- >>>>> -- To post, send email to [email protected]. To unsubscribe, >>>>> send email to [email protected]. For more information, visit >>>>> https://groups.google.com/d/forum/archesproject?hl=en >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Arches Project" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/archesproject/d888b6f9-47f2-42ad-8f7b-ed6aa98724d2%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/archesproject/d888b6f9-47f2-42ad-8f7b-ed6aa98724d2%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >> -- To post, send email to [email protected] <javascript:>. To >> unsubscribe, send email to [email protected] <javascript:>. For >> more information, visit >> https://groups.google.com/d/forum/archesproject?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Arches Project" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/archesproject/4c7b9b8e-857c-48b3-8bc0-fa09f848dd42%40googlegroups.com >> >> <https://groups.google.com/d/msgid/archesproject/4c7b9b8e-857c-48b3-8bc0-fa09f848dd42%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- -- To post, send email to [email protected]. To unsubscribe, send email to [email protected]. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en --- You received this message because you are subscribed to the Google Groups "Arches Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/archesproject/d0bc2563-4628-4a45-a082-c4393f362803%40googlegroups.com.
