David, I am working through a number of shard-related issues within DSpace 5 and DSpace 6.
https://wiki.duraspace.org/display/~terrywbrady/Statistics+Import+Export+Issues Once these issues are resolved, it will be interesting to see if your issues persist. Could you create a DSpace Jira ticket describing the issues you are encountering? Terry On Tue, Jan 31, 2017 at 10:58 AM, David Baker <[email protected]> wrote: > The Statistics page (Administrative > Statistics) in our DSpace 5.3 > instance is only showing 4 months since our site went live (6/9/15 - > 10/19/15): > > > > <https://lh3.googleusercontent.com/-kAVwsiVK6vo/WJDZRSxr4mI/AAAAAAAAADA/7ECB8W2_c8oJLSVQLsPacc8AzOS6i5csgCLcB/s1600/2017-01-31%2B13_35_01-Statistical%2BSummary.png> > > > > I run a nightly spider deletion and optimization on the stats: > > dspace stats-util -i > dspace stats-util -o > > These commands have both been logging an error for a while. > > The spider cleaning command runs at 1:00 AM and generates this error: > > 2017-01-30 01:00:13,872 INFO org.apache.solr.core.SolrCore @ [statistics] > webapp=/solr path=/select params={q=type:2+AND+id:1&wt=javabin&version=2} > hits=0 status=0 QTime=14 > 2017-01-30 01:00:13,945 INFO org.apache.solr.handler.admin. > CoreAdminHandler @ core create command dataDir=c:/dspace\solr\ > statistics-2014\data&name=statistics-2014&action=CREATE& > instanceDir=statistics&wt=javabin&version=2 > 2017-01-30 01:00:13,952 WARN org.apache.solr.handler.admin.CoreAdminHandler > @ Creating a core with existing name is not allowed > 2017-01-30 01:00:13,953 ERROR org.apache.solr.core.SolrCore @ > org.apache.solr.common.SolrException: Core with name 'statistics-2014' > already exists. > at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction( > CoreAdminHandler.java:555) > at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal( > CoreAdminHandler.java:199) > at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody( > CoreAdminHandler.java:188) > at org.apache.solr.handler.RequestHandlerBase.handleRequest( > RequestHandlerBase.java:135) > at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest( > SolrDispatchFilter.java:729) > at org.apache.solr.servlet.SolrDispatchFilter.doFilter( > SolrDispatchFilter.java:258) > at org.apache.solr.servlet.SolrDispatchFilter.doFilter( > SolrDispatchFilter.java:207) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter( > ApplicationFilterChain.java:239) > at org.apache.catalina.core.ApplicationFilterChain.doFilter( > ApplicationFilterChain.java:206) > at org.dspace.solr.filters.LocalHostRestrictionFilter.doFilter( > LocalHostRestrictionFilter.java:50) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter( > ApplicationFilterChain.java:239) > at org.apache.catalina.core.ApplicationFilterChain.doFilter( > ApplicationFilterChain.java:206) > at org.apache.catalina.core.StandardWrapperValve.invoke( > StandardWrapperValve.java:219) > at org.apache.catalina.core.StandardContextValve.invoke( > StandardContextValve.java:106) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke( > AuthenticatorBase.java:502) > at org.apache.catalina.core.StandardHostValve.invoke( > StandardHostValve.java:142) > at org.apache.catalina.valves.ErrorReportValve.invoke( > ErrorReportValve.java:79) > at org.apache.catalina.valves.AbstractAccessLogValve.invoke( > AbstractAccessLogValve.java:617) > at org.apache.catalina.core.StandardEngineValve.invoke( > StandardEngineValve.java:88) > at org.apache.catalina.connector.CoyoteAdapter.service( > CoyoteAdapter.java:518) > at org.apache.coyote.http11.AbstractHttp11Processor.process( > AbstractHttp11Processor.java:1091) > at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler. > process(AbstractProtocol.java:668) > at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor. > doRun(AprEndpoint.java:2463) > at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor. > run(AprEndpoint.java:2452) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run( > TaskThread.java:61) > at java.lang.Thread.run(Thread.java:745) > > > 2017-01-30 01:00:13,955 INFO org.apache.solr.servlet.SolrDispatchFilter > @ [admin] webapp=null path=/admin/cores params={dataDir=c:/dspace\ > solr\statistics-2014\data&name=statistics-2014&action= > CREATE&instanceDir=statistics&wt=javabin&version=2} status=500 QTime=10 > 2017-01-30 01:00:13,955 ERROR org.apache.solr.servlet.SolrDispatchFilter > @ null:org.apache.solr.common.SolrException: Core with name > 'statistics-2014' already exists. > at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction( > CoreAdminHandler.java:555) > at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal( > CoreAdminHandler.java:199) > at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody( > CoreAdminHandler.java:188) > at org.apache.solr.handler.RequestHandlerBase.handleRequest( > RequestHandlerBase.java:135) > at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest( > SolrDispatchFilter.java:729) > at org.apache.solr.servlet.SolrDispatchFilter.doFilter( > SolrDispatchFilter.java:258) > at org.apache.solr.servlet.SolrDispatchFilter.doFilter( > SolrDispatchFilter.java:207) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter( > ApplicationFilterChain.java:239) > at org.apache.catalina.core.ApplicationFilterChain.doFilter( > ApplicationFilterChain.java:206) > at org.dspace.solr.filters.LocalHostRestrictionFilter.doFilter( > LocalHostRestrictionFilter.java:50) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter( > ApplicationFilterChain.java:239) > at org.apache.catalina.core.ApplicationFilterChain.doFilter( > ApplicationFilterChain.java:206) > at org.apache.catalina.core.StandardWrapperValve.invoke( > StandardWrapperValve.java:219) > at org.apache.catalina.core.StandardContextValve.invoke( > StandardContextValve.java:106) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke( > AuthenticatorBase.java:502) > at org.apache.catalina.core.StandardHostValve.invoke( > StandardHostValve.java:142) > at org.apache.catalina.valves.ErrorReportValve.invoke( > ErrorReportValve.java:79) > at org.apache.catalina.valves.AbstractAccessLogValve.invoke( > AbstractAccessLogValve.java:617) > at org.apache.catalina.core.StandardEngineValve.invoke( > StandardEngineValve.java:88) > at org.apache.catalina.connector.CoyoteAdapter.service( > CoyoteAdapter.java:518) > at org.apache.coyote.http11.AbstractHttp11Processor.process( > AbstractHttp11Processor.java:1091) > at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler. > process(AbstractProtocol.java:668) > at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor. > doRun(AprEndpoint.java:2463) > at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor. > run(AprEndpoint.java:2452) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run( > TaskThread.java:61) > at java.lang.Thread.run(Thread.java:745) > > > 2017-01-30 01:00:14,856 INFO org.apache.solr.core.SolrCore @ [statistics] > webapp=/solr path=/update params={wt=javabin&version=2} status=0 QTime=0 > > > The optimization command runs at 1:30 AM and generates this error: > > > > 2017-01-30 01:30:35,515 INFO org.apache.solr.core.SolrCore @ [statistics] > webapp=/solr path=/select params={q=type:2+AND+id:1&wt=javabin&version=2} > hits=0 status=0 QTime=1 > 2017-01-30 01:30:35,565 INFO org.apache.solr.handler.admin. > CoreAdminHandler @ core create command dataDir=c:/dspace\solr\ > statistics-2014\data&name=statistics-2014&action=CREATE& > instanceDir=statistics&wt=javabin&version=2 > 2017-01-30 01:30:35,565 WARN org.apache.solr.handler.admin.CoreAdminHandler > @ Creating a core with existing name is not allowed > 2017-01-30 01:30:35,566 ERROR org.apache.solr.core.SolrCore @ > org.apache.solr.common.SolrException: Core with name 'statistics-2014' > already exists. > at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction( > CoreAdminHandler.java:555) > at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal( > CoreAdminHandler.java:199) > at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody( > CoreAdminHandler.java:188) > at org.apache.solr.handler.RequestHandlerBase.handleRequest( > RequestHandlerBase.java:135) > at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest( > SolrDispatchFilter.java:729) > at org.apache.solr.servlet.SolrDispatchFilter.doFilter( > SolrDispatchFilter.java:258) > at org.apache.solr.servlet.SolrDispatchFilter.doFilter( > SolrDispatchFilter.java:207) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter( > ApplicationFilterChain.java:239) > at org.apache.catalina.core.ApplicationFilterChain.doFilter( > ApplicationFilterChain.java:206) > at org.dspace.solr.filters.LocalHostRestrictionFilter.doFilter( > LocalHostRestrictionFilter.java:50) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter( > ApplicationFilterChain.java:239) > at org.apache.catalina.core.ApplicationFilterChain.doFilter( > ApplicationFilterChain.java:206) > at org.apache.catalina.core.StandardWrapperValve.invoke( > StandardWrapperValve.java:219) > at org.apache.catalina.core.StandardContextValve.invoke( > StandardContextValve.java:106) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke( > AuthenticatorBase.java:502) > at org.apache.catalina.core.StandardHostValve.invoke( > StandardHostValve.java:142) > at org.apache.catalina.valves.ErrorReportValve.invoke( > ErrorReportValve.java:79) > at org.apache.catalina.valves.AbstractAccessLogValve.invoke( > AbstractAccessLogValve.java:617) > at org.apache.catalina.core.StandardEngineValve.invoke( > StandardEngineValve.java:88) > at org.apache.catalina.connector.CoyoteAdapter.service( > CoyoteAdapter.java:518) > at org.apache.coyote.http11.AbstractHttp11Processor.process( > AbstractHttp11Processor.java:1091) > at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler. > process(AbstractProtocol.java:668) > at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor. > doRun(AprEndpoint.java:2463) > at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor. > run(AprEndpoint.java:2452) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run( > TaskThread.java:61) > at java.lang.Thread.run(Thread.java:745) > > > 2017-01-30 01:30:35,566 INFO org.apache.solr.servlet.SolrDispatchFilter > @ [admin] webapp=null path=/admin/cores params={dataDir=c:/dspace\ > solr\statistics-2014\data&name=statistics-2014&action= > CREATE&instanceDir=statistics&wt=javabin&version=2} status=500 QTime=1 > 2017-01-30 01:30:35,566 ERROR org.apache.solr.servlet.SolrDispatchFilter > @ null:org.apache.solr.common.SolrException: Core with name > 'statistics-2014' already exists. > at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction( > CoreAdminHandler.java:555) > at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal( > CoreAdminHandler.java:199) > at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody( > CoreAdminHandler.java:188) > at org.apache.solr.handler.RequestHandlerBase.handleRequest( > RequestHandlerBase.java:135) > at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest( > SolrDispatchFilter.java:729) > at org.apache.solr.servlet.SolrDispatchFilter.doFilter( > SolrDispatchFilter.java:258) > at org.apache.solr.servlet.SolrDispatchFilter.doFilter( > SolrDispatchFilter.java:207) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter( > ApplicationFilterChain.java:239) > at org.apache.catalina.core.ApplicationFilterChain.doFilter( > ApplicationFilterChain.java:206) > at org.dspace.solr.filters.LocalHostRestrictionFilter.doFilter( > LocalHostRestrictionFilter.java:50) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter( > ApplicationFilterChain.java:239) > at org.apache.catalina.core.ApplicationFilterChain.doFilter( > ApplicationFilterChain.java:206) > at org.apache.catalina.core.StandardWrapperValve.invoke( > StandardWrapperValve.java:219) > at org.apache.catalina.core.StandardContextValve.invoke( > StandardContextValve.java:106) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke( > AuthenticatorBase.java:502) > at org.apache.catalina.core.StandardHostValve.invoke( > StandardHostValve.java:142) > at org.apache.catalina.valves.ErrorReportValve.invoke( > ErrorReportValve.java:79) > at org.apache.catalina.valves.AbstractAccessLogValve.invoke( > AbstractAccessLogValve.java:617) > at org.apache.catalina.core.StandardEngineValve.invoke( > StandardEngineValve.java:88) > at org.apache.catalina.connector.CoyoteAdapter.service( > CoyoteAdapter.java:518) > at org.apache.coyote.http11.AbstractHttp11Processor.process( > AbstractHttp11Processor.java:1091) > at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler. > process(AbstractProtocol.java:668) > at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor. > doRun(AprEndpoint.java:2463) > at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor. > run(AprEndpoint.java:2452) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run( > TaskThread.java:61) > at java.lang.Thread.run(Thread.java:745) > > > 2017-01-30 01:30:35,591 INFO org.apache.solr.update.UpdateHandler @ > start commit{,optimize=true,openSearcher=true,waitSearcher=true, > expungeDeletes=false,softCommit=false,prepareCommit=false} > 2017-01-30 01:30:38,798 INFO org.apache.solr.core.SolrCore @ > SolrDeletionPolicy.onCommit: commits: num=2 > > commit{dir=NRTCachingDirectory(MMapDirectory@C:\dspace\solr\statistics\data\index > lockFactory=NativeFSLockFactory@C:\dspace\solr\statistics\data\index; > maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_ukz,generation=39635} > > commit{dir=NRTCachingDirectory(MMapDirectory@C:\dspace\solr\statistics\data\index > lockFactory=NativeFSLockFactory@C:\dspace\solr\statistics\data\index; > maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_ul0,generation=39636} > 2017-01-30 01:30:38,798 INFO org.apache.solr.core.SolrCore @ newest > commit generation = 39636 > 2017-01-30 01:30:38,801 INFO org.apache.solr.search.SolrIndexSearcher @ > Opening Searcher@702fcd2f[statistics] main > 2017-01-30 01:30:38,801 INFO org.apache.solr.update.UpdateHandler @ > end_commit_flush > 2017-01-30 01:30:38,802 INFO org.apache.solr.core.SolrCore @ > QuerySenderListener sending requests to Searcher@702fcd2f[statistics] > main{StandardDirectoryReader(segments_ph1:90667:nrt _yhl(4.10.2):C152837)} > 2017-01-30 01:30:38,802 INFO org.apache.solr.core.SolrCore @ > QuerySenderListener done. > 2017-01-30 01:30:38,802 INFO org.apache.solr.core.SolrCore @ [statistics] > Registered new searcher Searcher@702fcd2f[statistics] > main{StandardDirectoryReader(segments_ph1:90667:nrt _yhl(4.10.2):C152837)} > 2017-01-30 01:30:38,840 INFO org.apache.solr.core.SolrCore @ [statistics] > webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher > =true&wt=javabin&version=2} status=0 QTime=3248 > > > I am also doing a yearly shard on Jan 1: > > dspace stats-util -s > > > > > My dspace\solr\ folder looks like this: > > > <https://lh3.googleusercontent.com/-119AM4vybTk/WJDckauPNYI/AAAAAAAAADY/YkOC8w40JMscjPRgBvkK2e76ngMRB3piwCLcB/s1600/solr_folder.png> > > containing statistics for 2014 and 2015 as well as what appears to be the > current statistics. > > I've also been able to query statistics from http://localhost/solr/#/ > statistics/query and I can see current statistics when I do that, so > stats appear to actually be there. Any ideas why they're not showing up on > the Statistics page in DSpace? > > Thanks for any info! > David > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- Terry Brady Applications Programmer Analyst Georgetown University Library Information Technology http://georgetown-university-libraries.github.io/ <https://www.library.georgetown.edu/lit/code> 425-298-5498 (Seattle, WA) -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
