Hi Walter, I don’t think you need the DocumentRoot in the VirtualHost (since it’s a reverse-proxy).
-- Sean From: dspace-tech@googlegroups.com <dspace-tech@googlegroups.com> On Behalf Of wlrutherf...@alaska.edu Sent: October 26, 2018 4:24 PM To: DSpace Technical Support <dspace-tech@googlegroups.com> Subject: Re: [dspace-tech] View Statistics links not working Yes, it does make sense. It also means I must have something misconfigured because the only way I can currently access the site is if I explicitly specify http and port 8080 - http://dspace36a.library.uaf.edu:8080/ On Friday, October 26, 2018 at 12:37:42 PM UTC-8, Tim Donohue wrote: Hi Walter, No worries, we've all been newbies at some point :) Port 8080 is Tomcat. It should not be defined in Apache configuration. Think of it this way, when you use Apache in front of Tomcat... your Tomcat ports do *not* need to be publicly accessible. They are just internal ports (only accessible on localhost). So, in the setup I described above, my *only public ports* are port 80 and 443. I don't have port 8080 or port 8009 available publicly, so you cannot access my site via http://my.url:8080/ (for example). Port 8080 and 8009 are only accessible on the server (so they are private ports). So, if I login to the server and do a "wget http://localhost:8080"; then Tomcat responds. And Port 8009 is just used for proxying requests between Apache and Tomcat on that server. So, with regards to SSL, in my setup the only port where SSL is configured is port 443. As I have port 80 (HTTP) requests redirecting to port 443 (HTTPS), *all* public requests (except sometimes OAI, as mentioned) go through SSL. *After* SSL is validated, those requests *all* get proxied to Tomcat (via port 8009). Port 8009 and 8080 don't need SSL (again, cause they are not public ports and cannot be accessed unless you first login directly on the server). Does that start to make a bit more sense? Tim On Fri, Oct 26, 2018 at 3:28 PM <wlruth...@alaska.edu<javascript:>> wrote: Good to know. That should simplify things. Now for the embarrassing newb questions. I'm still not clear how my httpd secures port 8080 (which everything is going through now) since a URL that includes port 8080 isn't handled by the port 443 virtual host configuration. And how does port 8009 (AJP Tomcat) get involved; it also is only referenced in the port 80 and 443 configurations. Your example didn't show any- configuration for port 8080 which otherwise should just fall thru the other virtual hosts checks. I'm pretty sure I still have something misconfigured since our port 8080 refuses any https connection attempts, so I think SSL is being completely ignored. From your earlier post httpd should be handling all connection attempts first then passing control to Tomcat. I added some dummy rewrites to the port 80 and 443 vhost definitions that redirect to a different site if the URI matches "/catalog"; they are both definitely catching and handling URLs for those ports. Do you also have a virtual host configuration for port 8080 and does it also Proxy via 8009? Thank you, Walter R. On Friday, October 26, 2018 at 10:17:46 AM UTC-8, Tim Donohue wrote: Hi Walter, The dspace.cfg settings (any that use a URL) do not *require* a port number. The port number only needs to be specified if it's a non-standard port (e.g. 8080). So, if you are using port 80 or 443, those can be left off of any configuration as long as you appropriately use http:// or https:// where necessary. - Tim On Thu, Oct 25, 2018 at 6:04 PM <wlruth...@alaska.edu<mailto:wlruth...@alaska.edu>> wrote: Thanks, that does help clarify a lot. I already have port 80 forwarding everything to port 443 and I saw that the AJP port was already setup by default in server.xml. I just hadn't seen in the DSpace manual where I needed it and I wasn't sure what it was used for. All that I'm missing is a virtual host entry like you use to redirect (most) connections to pass thru to Tomcat AJP. We've tested OAI and it was working through port 443, but I'll keep that bypass in mind in case one of the other associated campuses has an SSL incompatible setup. The instructions/comments in the default dspace.cfg file say to include the port numbers for dspace.baseUrl and dspace.url, but your dspace.url does not include a port number. So I assume your dspace.baseUrl also doesn't include a port number and just lets rewrite rules handle that? Or does it even matter since all roads eventually lead to port 8009? I tried the new setup. httpd and Tomcat started but I ended up with the same situation. Only port 8080 is responding; ports 80 and 443 fail with an error: You don't have permission to access / on this server. Additionally, a 403 Forbidden error was encountered while trying to use an ErrorDocument to handle the request. So I tried again with port 443 defined in dspace.url and dspace.baseUrl. No change. Still getting the above error. So I'm still missing something. This is what my new simpler vhosts.conf file looks like: <VirtualHost *:80> ### Redirect insecure default port to secure port 443 ServerAdmin uaf-libra...@alaska.edu<mailto:uaf-libra...@alaska.edu> ServerName dspace36a.library.uaf.edu<http://dspace36a.library.uaf.edu> ServerAlias dspace36a # Redirect all HTTP requests to HTTPS (EXCEPT perhaps /oai/ path) RewriteEngine On #RewriteCond %{REQUEST_URI} !^/oai/ RewriteRule ^(.*) https://%{SERVER_NAME}$1<https://%25%7bSERVER_NAME%7d$1> [R=permanent,L] # Pass all requests (which weren't previously redirected to HTTPS) to Tomcat's AJP Connector (on port 8009) ProxyPass / ajp://localhost:8009/ ProxyPassReverse / ajp://localhost:8009/ </VirtualHost> <VirtualHost *:443> ### secure httpd server listening on port 443. ServerAdmin uaf-libra...@alaska.edu<mailto:uaf-libra...@alaska.edu> #DocumentRoot /dspace/webapps/xmlui DocumentRoot /dspace/webapps/ROOT ServerName dspace36a.library.uaf.edu<http://dspace36a.library.uaf.edu> ServerAlias dspace36a ErrorLog logs/dspace36a.library.uaf.edu-https-error_log CustomLog logs/dspace36a.library.uaf.edu-https-access_log combinedssl RewriteLog logs/dspace36a-rewrite Include conf.d/ssl.include Include conf.d/ssl.include.star ### ^ these point to files which specify where the ssl certificates ### live on the host. # Pass all remaining requests to Tomcat's AJP Connector ProxyPass / ajp://localhost:8009/ ProxyPassReverse / ajp://localhost:8009/ </VirtualHost> Re-verified these entries in the server.xml file: <Connector port="8080" maxThreads="150" minSpareThreads="25" maxSpareThreads="75" acceptCount="100" connectionTimeout="20000" disableUploadTimeout="true" URIEncoding="UTF-8" /> <!-- Define an AJP 1.3 Connector on port 8009 --> <!-- Connector port="8009" protocol="AJP/1.3" redirectPort="8443" / (I removed the original redirection) --> <Connector port="8009" protocol="AJP/1.3" /> I didn't see anywhere else that I needed to configure AJP. On Thursday, October 25, 2018 at 9:49:02 AM UTC-8, Tim Donohue wrote: Hi Walter, In case it helps to clarify anything.... here's how I tend to setup DSpace (in production). First, as noted, I use AJP as described here (there are other ways to do the Apache -> Tomcat communication, but I've found AJP the easiest myself): https://wiki.duraspace.org/display/DSDOC6x/Installing+DSpace#InstallingDSpace-UsingSSLonApacheHTTPDinfrontofTomcat(runningonports80and443 I always setup Apache to redirect *ALL* requests from port 80 to port 443. All of DSpace can be behind SSL. The only (minor) exception is that some sites prefer to allow the OAI-PMH interface (/oai webapp/path) to be accessible on port 80, as some older OAI harvesters don't deal well with SSL. Beyond that, put everything behind SSL / port 443 at the Apache web server level. So, for example, my <VirtualHost *:80> has these directives: <VirtualHost *:80> # This is not a real site, just a sample ServerName example.dspace.org<http://example.dspace.org> # Redirect all HTTP requests to HTTPS (EXCEPT /oai/ path) RewriteEngine On RewriteCond %{REQUEST_URI} !^/oai/ RewriteRule ^(.*) https://%{SERVER_NAME}$1<https://%25%7bSERVER_NAME%7d$1> [R=permanent,L] # Pass all requests (which were not previously redirected to HTTPS) to Tomcat's AJP Connector (on port 8009) ProxyPass / ajp://localhost:8009/ ProxyPassReverse / ajp://localhost:8009/ </VirtualHost> Then, for port 443, I just ensure those requests also all go to Tomcat's AJP Connector (again, on port 8009) <VirtualHost *:443> # This is not a real site, just a sample ServerName example.dspace.org<http://example.dspace.org> # Pass all requests to Tomcat's AJP Connector ProxyPass / ajp://localhost:8009/ ProxyPassReverse / ajp://localhost:8009/ # SSL Certificate Settings SSLEngine on SSLCertificateChainFile /etc/ssl/certs/my_chain_bundle.crt SSLCertificateFile /etc/ssl/certs/my_cert_file.crt SSLCertificateKeyFile /etc/ssl/private/my_cert_key.key </VirtualHost> At the Tomcat level, I have Tomcat setup to only respond on port 8080 (no SSL needed here, as Apache deals with that) and have its AJP Connector running on port 8009, like this: <Connector port="8009" protocol="AJP/1.3" /> This all works perfectly fine, as Apache takes the initial request, validates SSL certs, etc.... then it forwards that request down to Tomcat (via AJP) where DSpace webapps are actually running. In my setup, the DSpace webapp(s) just communicate directly to Solr on http://localhost:8080 (there is no need for them to also make requests through Apache) e.g. in dspace.cfg dspace.url = https://example.dspace.org solr.server = http://localhost:8080/solr in solr-statistics.cfg server = http://localhost:8080/solr/statistics in oai.cfg storage = solr solr.url = http://localhost:8080/solr/oai Hopefully that gives you a bit more to go on. Basically, to simply describe this... Apache is the configuration layer for the "outside world" (i.e. how you want folks to access your DSpace externally). But, Tomcat is where everything is actually running & can be used for any internal communications, so things that need Solr, can just communicate with it on http://localhost:8080 Tim On Thu, Oct 25, 2018 at 12:18 PM <wlruth...@alaska.edu<mailto:wlruth...@alaska.edu>> wrote: Just a bit of follow-up to my last post. First, I was wrong - logging in as an admin didn't make the restricted 443 pages visible. Second, I also understand why port 8080 is working now but not 443 which had worked before. Because I removed redirectPort="443" from the <Connector port="8080"> definitions in the server.xml file. All it did was move the live port. I'm not sure though why that broke all of the statistics links unless they must be non-SSL. My problems from the start might be a misunderstanding of which ports need to be secured and which need to be non-SSL. I thought that was the purpose of port 8080. But at least with everything going through port 8080 I can actually see the stats links now along with everything else so I'm getting very close. On Thursday, October 25, 2018 at 6:37:03 AM UTC-8, wlruth...@alaska.edu<mailto:wlruth...@alaska.edu> wrote: Thanks Tim, I added that stats rewrite section because, for the life of me, I couldn't get the stats links to work. I don't believe it's even necessary, especially since, as you noticed, the /solr link can't be reached from the network. But we didn't want it to be and I was still trying to figure out how to block it. Once things are configured correctly the software restricts it. I haven't tried the link while logged in as an admin but I was glad to see it said I didn't have authorization otherwise. We aren't currently using AJP but if it will simplify the setup then I'll read up on it. Thank you. Walter R. On Thursday, October 25, 2018 at 5:49:10 AM UTC-8, Tim Donohue wrote: Hi Walter, Aha, yes, I'm starting to think the issue here is in your Apache <-> Tomcat communications/redirects. The number of redirects/rewrites is quite a bit confusing in all those files. I think you could really simply the setup by just using "mod_proxy" and "mod_proxy_ajp", which is our recommended approach for running DSpace on Tomcat behind Apache. See the installation guide section at: https://wiki.duraspace.org/display/DSDOC6x/Installing+DSpace#InstallingDSpace-UsingSSLonApacheHTTPDinfrontofTomcat(runningonports80and443) If you use AJP for the connection between Tomcat & Apache, the number of rewrites/redirects is vastly reduced. Additionally, you can keep Solr running on Tomcat (port 8080) and *not even reference it from your Apache settings*. Solr does not need to be publicly accessible on the web, DSpace just needs to have it running on Tomcat. As long as DSpace can access Solr on localhost, you are fine...and all the /statistics, /statistics-home, etc paths in DSpace should just work (on whatever port your DSpace runs on). So, while you can continue to build your own redirects, it's gonna be a lot harder to get DSpace running consistently. I'd highly recommend considering using AJP, as it should make your life a lot easier (and the full setup should be in the documentation link above) - Tim On Wed, Oct 24, 2018 at 8:55 PM <wlruth...@alaska.edu<mailto:wlruth...@alaska.edu>> wrote: I made some progress and lost a little ground at the same time. First I verified that all of the statistics config files were watching the same port: /dspace/config/modules/solr-statistics.cfg server = http://localhost:8080/solr/statistics /dspace/config/modules/oai.cfg solr.url=http://localhost:8080/solr/oai /dspace/config/modules/discovery.cfg search.server = http://localhost:8080/solr/search /dspace/config/dspace.cfg solr.authority.server=https://dspace36a.library.uaf.edu:8080/solr/authority choices.plugin.dc.contributor.author = SolrAuthorAuthority choices.presentation.dc.contributor.author = authorLookup authority.controlled.dc.contributor.author = true # Still trying to get publicly viewable statistics. I was sure these had been set # in the past but I can't find them in this file now. Perhaps solr config file. 10/15/18 WLR report.public = true statistics.item.authorization.admin = false # 10/19/18 WLR // updated 10/24/18 WLR #solr.server = http://localhost:8080/solr solr.server = http://localhost:8080/solr/statistics /etc/httpd/conf.d/vhosts.conf <VirtualHost *:80> ### Redirect insecure default port to secure port 443 ServerAdmin uaf-libra...@alaska.edu<mailto:uaf-libra...@alaska.edu> RewriteEngine On RewriteCond %{HTTPS} off RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}<https://%25%7bHTTP_HOST%7d%25%7bREQUEST_URI%7d> </VirtualHost> <VirtualHost *:443> ### secure httpd server listening on port 443. ServerAdmin uaf-libra...@alaska.edu<mailto:uaf-libra...@alaska.edu> RewriteEngine On # There's probably a better way to do this RewriteCond %{REQUEST_URI} "^/solr" [NC] RewriteCond %{REQUEST_URI} "^/statistic" [NC] RewriteCond %{REQUEST_URI} "^/statistics-home" [NC] RewriteCond %{REQUEST_URI} "^/search-statistics" [NC] RewriteCond %{REQUEST_URI} "^/workflow-statistics" [NC] RewriteRule (.*) http://%{HTTP_HOST}:8080%{REQUEST_URI}<http://%25%7bHTTP_HOST%7d:8080%25%7bREQUEST_URI%7d> [R,L] DocumentRoot /dspace/webapps/xmlui ServerName dspace36a.library.uaf.edu<http://dspace36a.library.uaf.edu> ServerAlias scholarworks ErrorLog logs/dspace36a.library.uaf.edu-https-error_log CustomLog logs/dspace36a.library.uaf.edu-https-access_log combinedssl RewriteLog logs/dspace36a-rewrite Include conf.d/ssl.include Include conf.d/ssl.include.star ### ^ these point to a file which specifies where the ssl certificates ### live on the host. # ProxyPass / http://127.0.0.1:8080/ retry=10 connectiontimeout=5 ProxyPass / http://127.0.0.1:443/ retry=10 connectiontimeout=5 ### ^ this tells httpd to redirect it's / to localhost port 8080 #timeout=300 # ProxyPassReverse / http://127.0.0.1:8080/ retry=10 ProxyPassReverse / http://127.0.0.1:443/ retry=10 ### ^ this tells httpd that tomcat's url's should be rewritten to look ### like they're coming from httpd. ProxyPreserveHost On ### ^ this tells httpd to keep the Host: information from the client and ### pass it on to tomcat. #SSLUseStapling on ### Added due to error when starting apache. Dayne Ellanna </VirtualHost> <VirtualHost *:8080> # ### Redirect insecure ports to secure port 443 ServerAdmin uaf-libra...@alaska.edu<mailto:uaf-libra...@alaska.edu> RewriteEngine On RewriteCond %{HTTPS} on RewriteRule (.*) http://%{HTTP_HOST}:8080/%{REQUEST_URI}<http://%25%7bHTTP_HOST%7d:8080/%25%7bREQUEST_URI%7d> [R,L] DocumentRoot /dspace/webapps/xmlui ServerName dspace36a.library.uaf.edu<http://dspace36a.library.uaf.edu> ServerAlias scholarworks ErrorLog logs/dspace36a.library.uaf.edu-error_log CustomLog logs/dspace36a.library.uaf.edu-access_log common RewriteLog logs/dspace36a-rewrite ProxyPass / http://127.0.0.1:8080/ retry=10 connectiontimeout=5 ### ^ this tells httpd to redirect it's / to localhost port 443 #timeout=300 ProxyPassReverse / http://127.0.0.1:8080/ retry=10 ### ^ this tells httpd that tomcat's url's should be rewritten to look ### like they're coming from httpd. # ProxyPreserveHost On ### ^ this tells httpd to keep the Host: information from the client and ### pass it on to tomcat. </VirtualHost> [tomcat]/conf/server.xml <!-- Define a non-SSL HTTP/1.1 Connector on port 8080 This used to redirect to port 443 but I removed that line; that's possibly why port 443 is broken now. --> <Connector port="8080" protocol="HTTP/1.1" maxThreads="150" minSpareThreads="25" maxSpareThreads="75" acceptCount="100" connectionTimeout="20000" disableUploadTimeout="true" URIEncoding="UTF-8" /> So I managed to break both ports 80 and 443 but, amazingly port 8080 works like a charm and gives the statistics I've been trying to see for many weeks! I suspect the new problem is in the vhosts.conf file but haven't yet caught what's going wrong. http://dspace36a.library.uaf.edu:8080/ The stats links now work for the whole site (Home Page) and for individual items, although I'm not entirely sure the single item stats are working completely correctly since I haven't seen them working until now and there are often blanks within the returned information. ================================================================= On Friday, October 19, 2018 at 1:10:05 PM UTC-8, wlruth...@alaska.edu<mailto:wlruth...@alaska.edu> wrote: I set "solr.server" to use port 8080 in /dspace/config/dspace.cfg as well as "server" in solr-statistics.cfg and "solr.url" in oai.cfg. Looks like progress, it still fails but the error messages changed: 2018-10-19 12:58:58,999 ERROR org.dspace.app.xmlui.aspect.statistics.StatisticsTransformer @ Error occurred while creating statistics for home page org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html. <!DOCTYPE html><html><head><title>Apache Tomcat/9.0.0.M17 - Error report</title><style type="text/css">h1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} h2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} h3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} body {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} b {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} p {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;} a {color:black;} a.name<http://a.name> {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style> </head><body><h1>HTTP Status 404 - /solr/select</h1><div class="line"></div><p><b>type</b> Status report</p><p><b>message</b> <u>/solr/select</u></p><p><b>description</b> <u>The requested resource is not available.</u></p><hr class="line"><h3>Apache Tomcat/9.0.0.M17</h3></body></html> at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:512) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206) at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91) ... On Friday, October 19, 2018 at 11:36:45 AM UTC-8, Sean Kalynuk wrote: Hi Walter, Since there is redirect logic involved, how about updating your solr.server parameter to point directly to the Tomcat solr webapp on port 8080? solr.server = http://localhost:8080/solr Not entirely sure, but I’m guessing this type of change requires a Tomcat restart. -- Sean From: dspac...@googlegroups.com<mailto:dspac...@googlegroups.com> <dspac...@googlegroups.com<mailto:dspac...@googlegroups.com>> On Behalf Of wlruth...@alaska.edu<mailto:wlruth...@alaska.edu> Sent: October 19, 2018 2:26 PM To: DSpace Technical Support <dspac...@googlegroups.com<mailto:dspac...@googlegroups.com>> Subject: Re: [dspace-tech] View Statistics links not working Hey Sean, I started to say I saw nothing in the logs but I just saw a redirection error in dspace.log.2018-10-19 that I haven't seen before: 2018-10-19 10:51:13,345 ERROR org.dspace.app.xmlui.aspect.statistics.StatisticsTransformer @ Error occurred while creating statistics for home page org.apache.solr.client.solrj.SolrServerException: Server at http://localhost/statistics sent back a redirect (302). ... Rather than include the entire ~180 line stack trace, here is the /etc/httpd/conf.d/vhosts.conf file which handles HTTP redirection although I think we may have been having problems with the statistics links even before adding this file: [root@dspace36a conf.d]# cat vhosts.conf <VirtualHost *:80> # ### Redirect insecure ports to secure port 443 ServerAdmin uaf-library-it-d...@alaska.edu<mailto:uaf-library-it-d...@alaska.edu> RewriteEngine On RewriteCond %{HTTPS} off RewriteRule (.*) https://%{HTTP_HOST}:443%{REQUEST_URI}<https://%25%7bHTTP_HOST%7d:443%25%7bREQUEST_URI%7d> [R,L] DocumentRoot /dspace/webapps/xmlui ServerName dspace36a.library.uaf.edu<http://dspace36a.library.uaf.edu/> ServerAlias scholarworks ErrorLog logs/dspace36a.library.uaf.edu-error_log CustomLog logs/dspace36a.library.uaf.edu-access_log common RewriteLog logs/dspace36a-rewrite ProxyPass / http://127.0.0.1:443/ retry=10 connectiontimeout=5 ### ^ this tells httpd to redirect it's / to localhost port 443 #timeout=300 ProxyPassReverse / http://127.0.0.1:443/ retry=10 ### ^ this tells httpd that tomcat's url's should be rewritten to look ### like they're coming from httpd. # ProxyPreserveHost On ### ^ this tells httpd to keep the Host: information from the client and ### pass it on to tomcat. </VirtualHost> #<VirtualHost dspace36a.library.uaf.edu:443<http://dspace36a.library.uaf.edu:443/>> <VirtualHost *:443> ### ^ this creates a httpd server that listens on port 443. ServerAdmin uaf-library-it-d...@alaska.edu<mailto:uaf-library-it-d...@alaska.edu> DocumentRoot /dspace/webapps/xmlui ServerName dspace36a.library.uaf.edu<http://dspace36a.library.uaf.edu/> ServerAlias scholarworks ErrorLog logs/dspace36a.library.uaf.edu-https-error_log CustomLog logs/dspace36a.library.uaf.edu-https-access_log combinedssl RewriteLog logs/dspace36a-rewrite Include conf.d/ssl.include Include conf.d/ssl.include.star ### ^ these point to a file which specifies where the ssl certificates ### live on the host. ProxyPass / http://127.0.0.1:8080/ retry=10 connectiontimeout=5 # ProxyPass / http://127.0.0.1:443/ retry=10 connectiontimeout=5 ### ^ this tells httpd to redirect it's / to localhost port 8080 #timeout=300 ProxyPassReverse / http://127.0.0.1:8080/ retry=10 # ProxyPassReverse / http://127.0.0.1:443/ retry=10 ### ^ this tells httpd that tomcat's url's should be rewritten to look ### like they're coming from httpd. ProxyPreserveHost On ### ^ this tells httpd to keep the Host: information from the client and ### pass it on to tomcat. </VirtualHost> Ports 80, 443, 8000, and 8080 are active for TCP. Port 8000 is used by the handle-server but 8080 should also be unsecured. On Friday, October 19, 2018 at 8:46:33 AM UTC-8, Sean Kalynuk wrote: Hi Walter, Do you see errors in the logs when visiting the following page? https://dspace36a.library.uaf.edu/search-statistics At least on that page I got “There was an error while generating the search statistics, please try again later.” You can also see an error message when visiting https://dspace36a.library.uaf.edu/workflow-statistics -- Sean From: dspac...@googlegroups.com<mailto:dspac...@googlegroups.com> <dspac...@googlegroups.com<mailto:dspac...@googlegroups.com>> On Behalf Of wlruth...@alaska.edu<mailto:wlruth...@alaska.edu> Sent: October 18, 2018 2:52 PM To: DSpace Technical Support <dspac...@googlegroups.com<mailto:dspac...@googlegroups.com>> Subject: Re: [dspace-tech] View Statistics links not working This is maddening and taking far longer than it should, mostly because I can't tell where the problem is located. I've setup all of the configuration variables mentioned in the manual but I'm not seeing anything in the logfiles that says what's wrong. It could be a combination of Tomcat/DSpace, httpd, solr, ... The only other similar posts I've seen here haven't had solutions. It doesn't help that I haven't see the statistics links working so I'm not quite sure even what to expect. For example: I can see https://dspace36a.library.uaf.edu/solr but that looks more like an administrator page. But I don't need to be logged in at all to see it and I'm pretty sure that's not the statistics page. If I try this instead: https://dspace36a.library.uaf.edu/statistics I get an entirely blank page. In my experience that means it's being handled (it's not returning an error) but the output is null. I can attach config files or logs but which would be most useful to help troubleshoot this problem? Thank you. Walter R. On Wednesday, October 10, 2018 at 12:17:03 PM UTC-8, wlruth...@alaska.edu<mailto:wlruth...@alaska.edu> wrote: I am occasionally seeing errors like this in the catalina.out file: Error using query type: 2 [WARN] deprecation - The 'component-configurations' section in the sitemap is deprecated. Please check for alternatives. Error using query *:* Error using query statistics_type:workflow AND NOT(previousWorkflowStep: SUBMIT) Error using query type: 2 AND id:2695 Error using query type: 0 AND owningItem:2695 Error using query type: 2 AND id:3677 Error using query type: 0 AND owningItem:3677 The four digit numbers appear to be valid item numbers so I'm not sure what's generating the errors. So I'm slogging through the manual and sitemaps to see if I can trace what happens when the "View Usage Statistics" link is clicked. On Tuesday, October 9, 2018 at 2:48:54 PM UTC-8, wlruth...@alaska.edu<mailto:wlruth...@alaska.edu> wrote: Rats... couldn't be that easy. Finding very few non-INFO messages in the logs. I did find an entry in catalina.out that at least tells me it is/was running solr (perhaps with memory leaks). 01-Jun-2017 01:23:38.504 WARNING [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [solr] appears to have started a thread named [MultiThreadedHttpConnectionManager cleanu p] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: The rest of that log is either warnings about deprecated features or lines like these (which doesn't look fatal): 01-Jun-2017 05:57:35.306 WARNING [localhost-startStop-1] org.apache.solr.handler.component.SpellCheckComponent.inform No queryConverter defined, using default converter I made the mistake of turning on DEBUG logging. But after filtering out thousands of DEBUG messages I see: 2018-10-09 13:04:25,728 INFO org.dspace.usage.LoggerUsageEventListener @ anonymous:session_id=7D74E0A56FFA3493875B4B5436B875C6:ip_addr=127.0.0.1:view_item:handle=11122/6045 2018-10-09 13:04:25,738 ERROR org.dspace.statistics.SolrLogger @ org.apache.commons.httpclient.ProtocolException: The server scholarworks.alaska.edu<http://scholarworks.alaska.edu> failed to respond with a valid HTTP response org.apache.solr.client.solrj.SolrServerException: org.apache.commons.httpclient.ProtocolException: The server scholarworks.alaska.edu<http://scholarworks.alaska.edu> failed to respond with a valid HTTP response Perhaps Solr is trying to use HTTP while the server is configured for (mostly) HTTPS with a few exceptions. There was something a co-worker had added to the web.xml file to force HTTPS but I don't see it now. He might've removed it and put all that in the httpd configuration. If he didn't that might be the problem - the two of them fighting. On Tuesday, October 9, 2018 at 11:51:14 AM UTC-8, Tim Donohue wrote: Hi Walter, I don't think those "messages.xml" notices have anything to do with this Solr Statistics problem. That "messages.xml" file is not directly related to Solr. The XMLUI also looks for those files in several different locations, so if it doesn't find them in one place, it will look elsewhere. So, those log lines can likely be ignored. What you should be looking more closely at is whether any errors/warnings are occurring in the dspace.log files (or Tomcat logs) when you: (1) Access any Community/Collection/Item homepage on the site --- this should contact Solr and log a "view" of that Item (2) When you access the statistics page (that empty page may be throwing an error behind the scenes). Essentially look closely at the logs whenever you are performing tasks that should be logging "view" statistics to Solr, and also look closely in the Tomcat logs to be sure that the "/solr" webapp is deploying properly, without errors. - Tim On Tue, Oct 9, 2018 at 2:41 PM <wlruth...@alaska.edu<mailto:wlruth...@alaska.edu>> wrote: This might be the problem. In the latest cocoon log I saw lots of entries flash by that started with date/time like this: 2018-10-09 11:20:37,862 INFO org.apache.cocoon.i18n.XMLResourceBundle Then: - Bundle <file:///dspace/webapps/ROOT/i18n/aspects/EPerson/messages.xml> not loaded: Source URI not found - Bundle <resource://aspects/EPerson/i18n/messages.xml> not loaded: Source URI not found - Bundle <file:///dspace/webapps/ROOT/i18n/aspects/Administrative/messages.xml> not loaded: Source URI not found - Bundle <resource://aspects/Administrative/i18n/messages.xml> not loaded: Source URI not found - Bundle <file:///dspace/webapps/ROOT/i18n/aspects/SearchArtifacts/messages.xml> not loaded: Source URI not found - Bundle <resource://aspects/SearchArtifacts/i18n/messages.xml> not loaded: Source URI not found - Bundle <file:///dspace/webapps/ROOT/i18n/aspects/BrowseArtifacts/messages.xml> not loaded: Source URI not found - Bundle <resource://aspects/BrowseArtifacts/i18n/messages.xml> not loaded: Source URI not found - Bundle <file:///dspace/webapps/ROOT/i18n/aspects/ViewArtifacts/messages.xml> not loaded: Source URI not found - Bundle <resource://aspects/ViewArtifacts/i18n/messages.xml> not loaded: Source URI not found - Bundle <file:///dspace/webapps/ROOT/i18n/aspects/Versioning/messages.xml> not loaded: Source URI not found - Bundle <resource://aspects/Versioning/i18n/messages.xml> not loaded: Source URI not found The 'aspects' are something I've seen in the manual that's been added with the upgrade but I've done nothing to configure anything there. I started to say the messages.xml file exists but no, the one I've had to edit is in the i18n directory - the one in the error message doesn't exist in i18n/aspects/Versioning - the entire 'aspects' subdirectory doesn't exist at all. I could create the directories but can't populate them without knowing what should be in there. Where should it have come from? If fixing this doesn't require a scratch rebuild we might be able to quickly squash this bug. Walter R. On Tuesday, October 9, 2018 at 11:12:43 AM UTC-8, Tim Donohue wrote: Hi Walter, All I meant is that the "solr" webapp that DSpace creates needs to be available/deployed to Tomcat. So, for example, if Tomcat is running on http://localhost:8080, then Solr should be accessible on something like http://localhost:8080/solr/ In other words, the only way DSpace can properly communicate with Solr (to send/query statistics) is if it's available in Tomcat. The full path to Solr in Tomcat should also be configured in your installed [dspace]/config/modules/solr-statistics.cfg: https://github.com/DSpace/DSpace/blob/dspace-5_x/dspace/config/modules/solr-statistics.cfg#L12 If DSpace is unable to find/communicate with Solr, you likely would see errors appearing in your logs. - Tim On Tue, Oct 9, 2018 at 2:04 PM <wlruth...@alaska.edu<mailto:wlruth...@alaska.edu>> wrote: Thank you Tim. I didn't notice anything in the logs that seemed to point to a problem with Solr or the statistics. I'll look again to make sure I haven't overlooked something. Also checking the troubleshooting page you linked. I'm not sure what you mean by the DSpace Solr webapp, *must* be installed into Tomcat. Are you talking about during the build it must be configured with Tomcat or that it is configured in the [dspace]/webapps/solr directory after DSpace has been built? Walter R. On Tuesday, October 9, 2018 at 10:28:53 AM UTC-8, Tim Donohue wrote: Hi Walter, I'd recommend checking your logs for any errors or warnings. We have a guide for doing so at: https://wiki.duraspace.org/display/DSPACE/Troubleshoot+an+error DSpace does *require* using Solr for Statistics. However, the required "solr" webapp is provided out-of-the-box with DSpace (it is built into the [dspace]/webapps/solr/ directory), so you do not need to install Apache Solr separately. That DSpace Solr webapp *must* be installed into Tomcat (alongside your UI, either XMLUI or JSPUI) for Statistics to work properly. Any errors that are occurring with accessing or logging statistics should be logged in your dspace.log file (or, occasionally, in the Tomcat logs). In the XMLUI, blank pages usually mean something went wrong. Most of the time, you should have more information in your logs. However, if no information is found there, it's always possible that the problem is in your *theme* (XSLTs or similar). So, if you get stuck, you may also want to try temporarily switching to a default DSpace theme, to ensure your custom theme is not causing issues. If you have more information to share, or start to figure out what might be to blame, please let us know on this mailing list. Currently, though, it doesn't look like we have enough information to provide help (unless someone else on this mailing list has seen this exact same behavior before). Tim On Mon, Oct 8, 2018 at 5:12 PM <wlruth...@alaska.edu<mailto:wlruth...@alaska.edu>> wrote: I'm still having problems with the statistics in DSpace. I have an older server running DSpace3.1 which is a "copy" of the original system. The statistics apparently haven't worked since the system moved. The 2nd system is running DSpace5.5. On both systems there is a "View Usage Statistics" link on the Home page but following the link gives a valid, live page but with no numbers. The Search and Workflow statistic links give identical error messages on both systems. I have gone through the manual and tried enabling all of the configuration parameters mentioned but with no luck. I even tried a few ... -- All messages to this mailing list should adhere to the DuraSpace Code of Conduct: https://duraspace.org/about/policies/code-of-conduct/ --- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com<javascript:>. To post to this group, send email to dspac...@googlegroups.com<javascript:>. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout. -- Tim Donohue Technical Lead for DSpace & DSpaceDirect DuraSpace.org | DSpace.org | DSpaceDirect.org -- All messages to this mailing list should adhere to the DuraSpace Code of Conduct: https://duraspace.org/about/policies/code-of-conduct/ --- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com<mailto:dspace-tech+unsubscr...@googlegroups.com>. To post to this group, send email to dspace-tech@googlegroups.com<mailto:dspace-tech@googlegroups.com>. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout. -- All messages to this mailing list should adhere to the DuraSpace Code of Conduct: https://duraspace.org/about/policies/code-of-conduct/ --- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
