Joel, Glad to see you got up and running. I'm going to add the step to our installation instructions to make sure mod_python is removed. I can also add a conficts to the pulp spec file for mod_python.
Thanks. On Mon, Nov 21, 2011 at 05:00:05PM +0000, Golden, Joel wrote: > Tracked this down to a mod_python issue: > > https://bugzilla.redhat.com/show_bug.cgi?id=670967 > > Rpm -e mod_python and bingo! > > kstester-dal /etc/httpd/conf.d # pulp-admin -u admin -p admin user list > +------------------------------------------+ > Available Users > +------------------------------------------+ > > Login : admin > Name : None > Roles : super-users > > -----Original Message----- > From: Golden, Joel > Sent: Monday, November 21, 2011 8:57 AM > To: [email protected] > Subject: FW: [Pulp-list] New Pulp install - pulp-admin commands fail > > Running RHEL 5.6 with latest errata and followed the install on the URL you > listed. I started with the Pulp community repo then upgraded to the testing > repo, which produced the same error. > ________________________________________ > From: James Slagle [[email protected]] > Sent: Friday, November 18, 2011 2:43 PM > To: Golden, Joel > Cc: [email protected] > Subject: Re: [Pulp-list] New Pulp install - pulp-admin commands fail > > It looks like you're on RHEL 5 x86_64 based on the package manifest you > showed. Are you running a specific 5.X or just latest RHEL 5 with all > updates applied? > > I'm assuming you just walked through the instructions at > http://pulpproject.org/ug/UGInstallation.html. > > Let me know what version of RHEL 5 you're on, and we can try the setup and > see if we can repro the issue. > > Thanks. > > On Fri, Nov 18, 2011 at 09:19:56PM +0000, Golden, Joel wrote: > > The only logs in /var/log/pulp are the db.log and client.log. =/ > > > > I corrected hostnames in the files you specified but still get the same > > error. > > > > ##pulp-admin -u admin -p admin user list > > error: error: operation failed: <!DOCTYPE HTML PUBLIC "-//IETF//DTD > > HTML 2.0//EN"> <html><head> <title>500 Internal Server Error</title> > > </head><body> <h1>Internal Server Error</h1> <p>The server encountered > > an internal error or misconfiguration and was unable to complete your > > request.</p> <p>Please contact the server administrator, > > root@localhost and inform them of the time the error occurred, and > > anything you might have done that may have caused the error.</p> > > <p>More information about this error may be available in the server > > error log.</p> <hr> > > <address>Apache/2.2.3 (Red Hat) Server at kstester-den Port > > 443</address> > > > > ##hostname > > kstester-den > > > > ##/etc/pulp/admin/admin.conf: > > [server] > > host = kstester-den > > port = 443 > > scheme = https > > path = /pulp/api > > interval = 240 > > > > [plugins] > > plugin_dirs = /usr/lib/pulp/admin/plugins > > > > ##openssl x509 -text -in /etc/pki/tls/certs/localhost.crt: > > Certificate: > > Data: > > Version: 3 (0x2) > > Serial Number: 28289 (0x6e81) > > Signature Algorithm: sha1WithRSAEncryption > > Issuer: C=--, ST=SomeState, L=SomeCity, O=SomeOrganization, > > OU=SomeOrganizationalUnit, CN=kstester-den/emailAddress=root@kstester-den > > Validity > > Not Before: Sep 23 14:01:07 2011 GMT > > Not After : Sep 22 14:01:07 2012 GMT > > Subject: C=--, ST=SomeState, L=SomeCity, O=SomeOrganization, > > OU=SomeOrganizationalUnit, > > CN=kstester-den/emailAddress=root@kstester-den > > > > ##/etc/sysconfig/network: > > NETWORKING=yes > > NETWORKING_IPV6=no > > HOSTNAME=kstester-den > > > > ##/var/log/httpd/ssl_error_log: > > [Fri Nov 18 14:12:38 2011] [info] Connection: Client IP: 10.47.28.29, > > Protocol: TLSv1, Cipher: DHE-RSA-AES256-SHA (256/256 bits) [Fri Nov 18 > > 14:14:38 2011] [error] [client 10.47.28.29] Premature end of script > > headers: webservices.wsgi [Fri Nov 18 14:14:38 2011] [debug] > > ssl_engine_kernel.c(1819): OpenSSL: Write: SSL negotiation finished > > successfully [Fri Nov 18 14:14:38 2011] [info] [client 10.47.28.29] > > Connection closed to child 5 with standard shutdown (server > > kstester-den:443) > > > > > > -----Original Message----- > > From: James Slagle [mailto:[email protected]] > > Sent: Friday, November 18, 2011 1:48 PM > > To: Golden, Joel > > Cc: [email protected] > > Subject: Re: [Pulp-list] New Pulp install - pulp-admin commands fail > > > > Hello, > > > > I'm not sure I have anything definitive, but I did notice one thing that > > might be an issue. > > > > From ssl_error_log, I see: > > > [Fri Nov 18 12:35:35 2011] [info] [client 10.47.28.29] Connection > > > closed to child 1 with standard shutdown (server > > > kstester-den.domain.com:443) > > > > I'm wondering if apache thinks your hostname is kstester-den.domain.com, > > but pulp is configured to expect kstester-den. > > > > What's does the command 'hostname' say on the box? Also, what's configured > > in /etc/sysconfig/network? That value needs to match what's set for > > server_url in /etc/pulp/pulp.conf and for host in > > /etc/pulp/admin/admin.conf. > > > > Also, it needs to match the CN of the subject that is set in the SSL > > certificate that is referenced by the SSLCertificateFile config in > > /etc/httpd/conf.d/ssl.conf. If you haven't changed any default values in > > that file, the certificate should be /etc/pki/tls/certs/localhost.crt. > > > > You can see the certificate in text format by using: > > openssl x509 -text -in /etc/pki/tls/certs/localhost.crt > > > > You could try sending the full log files, there might be something else > > there. > > Do you have a /var/log/pulp/pulp.log? I didn't see that shown below. > > > > > > On Fri, Nov 18, 2011 at 07:59:12PM +0000, Golden, Joel wrote: > > > Spent the past 2.5 days researching on a fix but no luck... I do > > > not want to feel defeated at the end of a Friday. LOL > > > > > > I am experiencing a similar error to > > > https://www.redhat.com/archives/pulp-list/2011-January/msg00049.html > > > but does not appear to be mongodb related, which is where this thread > > > seems to stop. > > > > > > All daemons appear to come up without issue and I've searched high and > > > low for a resolution. Please let me know if you need additional logs or > > > config files. Any help will be greatly appreciated! > > > > > > pulp-admin -u admin -p admin user list > > > error: error: operation failed: <!DOCTYPE HTML PUBLIC "-//IETF//DTD > > > HTML 2.0//EN"> <html><head> <title>500 Internal Server Error</title> > > > </head><body> <h1>Internal Server Error</h1> <p>The server > > > encountered an internal error or misconfiguration and was unable to > > > complete your request.</p> <p>Please contact the server > > > administrator, root@localhost and inform them of the time the error > > > occurred, and anything you might have done that may have caused the > > > error.</p> <p>More information about this error may be available in > > > the server error log.</p> <hr> > > > <address>Apache/2.2.3 (Red Hat) Server at host.domain.com Port > > > 443</address> > > > > > > netstat: > > > tcp 0 0 0.0.0.0:5672 0.0.0.0:* > > > LISTEN 31153/qpidd > > > tcp 0 0 0.0.0.0:27017 0.0.0.0:* > > > LISTEN 31143/mongod > > > tcp 0 0 0.0.0.0:28017 0.0.0.0:* > > > LISTEN 31143/mongod > > > tcp 0 0 0.0.0.0:80 0.0.0.0:* > > > LISTEN 31192/httpd > > > tcp 0 0 0.0.0.0:443 0.0.0.0:* > > > LISTEN 31192/httpd > > > tcp 0 0 10.1.23.14:43378 10.1.18.28:636 > > > ESTABLISHED 31193/(wsgi:pulp) > > > > > > /var/log/messages: > > > Nov 18 12:33:00 kstester-den qpidd[31153]: 2011-11-18 12:33:00 > > > notice Journal "TplStore": Created Nov 18 12:33:00 kstester-den > > > qpidd[31153]: > > > 2011-11-18 12:33:00 notice Store module initialized; > > > store-dir=/var/lib/qpidd Nov 18 12:33:00 kstester-den qpidd[31153]: > > > 2011-11-18 12:33:00 notice SASL disabled: No Authentication > > > Performed Nov 18 12:33:00 kstester-den qpidd[31153]: 2011-11-18 > > > 12:33:00 notice Listening on TCP port 5672 Nov 18 12:33:00 kstester-den > > > qpidd[31153]: > > > 2011-11-18 12:33:00 notice Broker running > > > > > > /var/log/httpd/ssl_error.log: > > > [Fri Nov 18 12:33:35 2011] [debug] ssl_engine_kernel.c(1809): OpenSSL: > > > Loop: SSLv3 read finished A [Fri Nov 18 12:33:35 2011] [debug] > > > ssl_engine_kernel.c(1809): OpenSSL: Loop: SSLv3 write change cipher > > > spec A [Fri Nov 18 12:33:35 2011] [debug] ssl_engine_kernel.c(1809): > > > OpenSSL: Loop: SSLv3 write finished A [Fri Nov 18 12:33:35 2011] > > > [debug] ssl_engine_kernel.c(1809): OpenSSL: Loop: SSLv3 flush data > > > [Fri Nov 18 12:33:35 2011] [debug] ssl_scache_shmcb.c(670): inside > > > shmcb_store_session [Fri Nov 18 12:33:35 2011] [debug] > > > ssl_scache_shmcb.c(676): session_id[0]=227, masked index=3 [Fri Nov > > > 18 > > > 12:33:35 2011] [debug] ssl_scache_shmcb.c(1059): entering > > > shmcb_insert_encoded_session, *queue->pos_count = 0 [Fri Nov 18 > > > 12:33:35 2011] [debug] ssl_scache_shmcb.c(983): entering > > > shmcb_expire_division [Fri Nov 18 12:33:35 2011] [debug] > > > ssl_scache_shmcb.c(1115): we have 13853 bytes and 133 indexes free - > > > enough [Fri Nov 18 12:33:35 2011] [debug] ssl_scache_shmcb.c(1144): > > > storing in index 0, at offset 0 [Fri Nov 18 12:33:35 2011] [debug] > > > ssl_scache_shmcb.c(1159): session_id[0]=227, idx->s_id2=254 [Fri Nov > > > 18 12:33:35 2011] [debug] ssl_scache_shmcb.c(1170): leaving now with > > > 123 bytes in the cache and 1 indexes [Fri Nov 18 12:33:35 2011] > > > [debug] ssl_scache_shmcb.c(1174): leaving > > > shmcb_insert_encoded_session [Fri Nov 18 12:33:35 2011] [debug] > > > ssl_scache_shmcb.c(704): leaving shmcb_store successfully [Fri Nov > > > 18 12:33:35 2011] [debug] > > > ssl_scache_shmcb.c(418): shmcb_store successful [Fri Nov 18 12:33:35 > > > 2011] [debug] ssl_engine_kernel.c(1667): Inter-Process Session Cache: > > > request=SET status=OK > > > id=E3FE4F919F2FDADFBF497D237089584DD60494E2F9430CFB58D5651005E77C71 > > > timeout=300s (session caching) [Fri Nov 18 12:33:35 2011] [debug] > > > ssl_engine_kernel.c(1805): OpenSSL: Handshake: done [Fri Nov 18 > > > 12:33:35 2011] [info] Connection: Client IP: 10.47.28.29, Protocol: > > > TLSv1, Cipher: DHE-RSA-AES256-SHA (256/256 bits) [Fri Nov 18 > > > 12:35:35 2011] [error] [client 10.47.28.29] Premature end of script > > > headers: > > > webservices.wsgi [Fri Nov 18 12:35:35 2011] [debug] > > > ssl_engine_kernel.c(1819): OpenSSL: Write: SSL negotiation finished > > > successfully [Fri Nov 18 12:35:35 2011] [info] [client 10.47.28.29] > > > Connection closed to child 1 with standard shutdown (server > > > kstester-den.domain.com:443) > > > > > > /var/log/httpd/ssl_access_log: > > > 10.47.28.29 - - [18/Nov/2011:11:40:48 -0700] "GET /pulp/api//users/ > > > HTTP/1.1" 500 627 > > > 10.47.28.29 - - [18/Nov/2011:11:50:24 -0700] "GET /pulp/api//users/ > > > HTTP/1.1" 500 627 > > > 10.47.28.29 - - [18/Nov/2011:12:18:07 -0700] "GET /pulp/api//users/ > > > HTTP/1.1" 500 627 > > > 10.47.28.29 - - [18/Nov/2011:12:27:33 -0700] "GET /pulp/api//users/ > > > HTTP/1.1" 500 627 > > > 10.47.28.29 - - [18/Nov/2011:12:33:35 -0700] "GET /pulp/api//users/ > > > HTTP/1.1" 500 627 > > > > > > /var/log/httpd/ssl_request_log: > > > [18/Nov/2011:11:40:48 -0700] 10.47.28.29 TLSv1 DHE-RSA-AES256-SHA > > > "GET /pulp/api//users/ HTTP/1.1" 627 > > > [18/Nov/2011:11:50:24 -0700] 10.47.28.29 TLSv1 DHE-RSA-AES256-SHA > > > "GET /pulp/api//users/ HTTP/1.1" 627 > > > [18/Nov/2011:12:18:07 -0700] 10.47.28.29 TLSv1 DHE-RSA-AES256-SHA > > > "GET /pulp/api//users/ HTTP/1.1" 627 > > > [18/Nov/2011:12:27:33 -0700] 10.47.28.29 TLSv1 DHE-RSA-AES256-SHA > > > "GET /pulp/api//users/ HTTP/1.1" 627 > > > [18/Nov/2011:12:33:35 -0700] 10.47.28.29 TLSv1 DHE-RSA-AES256-SHA > > > "GET /pulp/api//users/ HTTP/1.1" 627 > > > > > > /var/log/httpd/error_log: > > > [Fri Nov 18 12:33:01 2011] [info] mod_wsgi (pid=31194): Attach > > > interpreter ''. > > > [Fri Nov 18 12:33:01 2011] [debug] proxy_util.c(1817): proxy: > > > grabbed scoreboard slot 0 in child 31202 for worker proxy:reverse > > > [Fri Nov 18 > > > 12:33:01 2011] [debug] proxy_util.c(1836): proxy: worker > > > proxy:reverse already initialized [Fri Nov 18 12:33:01 2011] [debug] > > > proxy_util.c(1930): proxy: initialized single connection worker 0 in > > > child 31202 for (*) [Fri Nov 18 12:33:01 2011] [info] mod_wsgi > > > (pid=31202): Attach interpreter ''. > > > [Fri Nov 18 12:33:01 2011] [debug] proxy_util.c(1817): proxy: > > > grabbed scoreboard slot 0 in child 31200 for worker proxy:reverse > > > [Fri Nov 18 > > > 12:33:01 2011] [debug] proxy_util.c(1836): proxy: worker > > > proxy:reverse already initialized [Fri Nov 18 12:33:01 2011] [debug] > > > proxy_util.c(1930): proxy: initialized single connection worker 0 in > > > child 31200 for (*) [Fri Nov 18 12:33:01 2011] [info] mod_wsgi > > > (pid=31200): Attach interpreter ''. > > > > > > /var/log/mongodb/mongodb.log: > > > Fri Nov 18 12:33:00 [initandlisten] Accessing: pulp_database for the > > > first time Fri Nov 18 12:33:00 [initandlisten] done repairDatabases > > > Fri Nov 18 12:33:00 [initandlisten] waiting for connections on port > > > 27017 Fri Nov 18 12:33:00 [websvr] web admin interface listening on > > > port 28017 Fri Nov 18 12:33:00 [websvr] fd limit hard:1024 soft:1024 > > > max conn: 819 Fri Nov 18 12:33:00 [initandlisten] fd limit hard:1024 > > > soft:1024 max conn: 819 Fri Nov 18 12:34:00 [DataFileSync] flushing > > > mmap took 0ms for 1 files Fri Nov 18 12:35:00 [DataFileSync] > > > flushing mmap took 0ms for 1 files > > > > > > /var/log/pulp/db.log: > > > validation on previous data model version started validation > > > complete; data model at version 25 migration to data model version > > > 26 started migration to data model version 26 complete migration to > > > data model version 27 started migration to data model version 27 > > > complete migration to data model version 28 started migration to > > > data model version 28 complete **M2Crypto<%s> lacks patch for using > > > Certificate Revocation Lists** migration to data model version 29 > > > started migration to data model version 29 complete > > > > > > /var/log/pulp/client.log: > > > 2011-11-18 12:35:35,293 [ERROR][MainThread] @ command.py:226 - error: > > > (500, '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML > > > 2.0//EN">\n<html><head>\n<title>500 Internal Server > > > Error</title>\n</head><body>\n<h1>Internal Server Error</h1>\n<p>The > > > server encountered an internal error or\nmisconfiguration and was > > > unable to complete\nyour request.</p>\n<p>Please contact the server > > > administrator,\n root@localhost and inform them of the time the > > > error occurred,\nand anything you might have done that may > > > have\ncaused the error.</p>\n<p>More information about this error > > > may be available\nin the server error > > > log.</p>\n<hr>\n<address>Apache/2.2.3 (Red Hat) Server at > > > kstester-den.twtelecom.com Port 443</address>\n</body></html>\n', > > > None) > > > > > > > > > /etc/pulp.conf: > > > [server] > > > server_name: kstester-den > > > relative_url: /pulp/repos > > > key_url: /pulp/gpg > > > ks_url: /pulp/ks > > > default_login: admin > > > default_password: admin > > > debugging_mode: true > > > > > > [messaging] > > > url: tcp://localhost:5672 > > > cacert: /etc/pki/qpid/ca/ca.crt > > > clientcert: /etc/pki/qpid/client/client.pem > > > > > > [events] > > > send_enabled: false > > > recv_enabled: false > > > > > > [security] > > > cacert: /etc/pki/pulp/ca.crt > > > cakey: /etc/pki/pulp/ca.key > > > user_cert_expiration: 7 > > > consumer_cert_expiration: 3650 > > > > > > [auditing] > > > audit_events: true > > > events_file: /var/log/pulp/events.log > > > lifetime: 90 > > > backups: 4 > > > > > > [consumer_history] > > > lifetime: 180 > > > > > > [logs] > > > config: /etc/pulp/logging/debug_sync.cfg > > > > > > [yum] > > > threads: 15 > > > remove_old_versions: false > > > num_old_pkgs_keep: 2 > > > verify_checksum: true > > > verify_size: true > > > > > > [repos] > > > content_url: https://cdn.redhat.com/ use_entitlement_certs = false > > > default_to_published: true > > > > > > [database] > > > name: pulp_database > > > seeds: localhost > > > > > > [cds] > > > sync_timeout = 10:7200 > > > > > > > > > > > > Version info: > > > pulp-client-lib-0.0.249-1 > > > pulp-admin-0.0.249-1 > > > mod_wsgi-3.2-6.pulp > > > pulp-common-0.0.249-1 > > > pulp-0.0.249-1 > > > mongodb-server-1.6.4-1.el5 > > > pymongo-1.9-8.el5 > > > mongodb-1.6.4-1.el5 > > > qpid-cpp-client-0.7.946106-34.el5 > > > qpid-cpp-server-store-0.7.946106-34.el5 > > > qpid-cpp-server-0.7.946106-34.el5 > > > python-qpid-0.7.946106-15.el5 > > > qpid-cpp-server-ssl-0.7.946106-34.el5 > > > qpid-cpp-client-ssl-0.7.946106-34.el5 > > > amqp-1.0.750054-1.el5.noarch > > > python-ssl-1.15-3.el5.x86_64 > > > xz-libs-4.999.9-0.3.beta.20091007git.el5.x86_64 > > > python-hashlib-20081119-4.el5.x86_64 > > > js-1.70-8.el5.x86_64 > > > mod_wsgi-3.2-6.pulp.x86_64 > > > python-pycurl-7.15.5.1-8.el5.x86_64 > > > gofer-0.59-1.noarch > > > grinder-0.0.129-1.noarch > > > mod_python-3.2.8-3.1 > > > python-2.4.3-44.el5 > > > httpd-2.2.3-53.el5_7.3 > > > python-httplib2-0.4.0-4.el5 > > > > > > Joel Golden > > > > > > > > > ------------- > > > > > > > > > > > > The content contained in this electronic message is not intended to > > > constitute formation of a contract binding tw telecom. tw telecom will be > > > contractually bound only upon execution, by an authorized officer, of a > > > contract including agreed terms and conditions or by express application > > > of its tariffs. This message is intended only for the use of the > > > individual or entity to which it is addressed. If the reader of this > > > message is not the intended recipient, or the employee or agent > > > responsible for delivering the message to the intended recipient, you are > > > hereby notified that any dissemination, distribution or copying of this > > > message is strictly prohibited. If you have received this communication > > > in error, please notify us immediately by replying to the sender of this > > > E-Mail or by telephone. > > > > > > > > > > > > _______________________________________________ > > > Pulp-list mailing list > > > [email protected] > > > https://www.redhat.com/mailman/listinfo/pulp-list > > -- > > -- James Slagle > > -- > > > > > -- > -- James Slagle > -- > > > > _______________________________________________ > Pulp-list mailing list > [email protected] > https://www.redhat.com/mailman/listinfo/pulp-list -- -- James Slagle -- _______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
