Re: [Dspace-tech] SOLR Statistics problem during upgrade from 4.2 to 5.2 on dev server
Hi All, I think I found the problem: See: http://wiki.lib.sun.ac.za/index.php/SUNScholar/Statistics/5.X#PLEASE_NOTE: Regards hg *Hilton Gibson* Ubuntu Linux Systems Administrator Stellenbosch University Library http://staff.lib.sun.ac.za/~hgibson/docs/cv/cv.html On 3 June 2015 at 13:01, Hilton Gibson hilton.gib...@gmail.com wrote: Hi All It's that same old problem - SOLR statistics. See: http://wiki.lib.sun.ac.za/index.php/SUNScholar/SOLR_Statistics After upgrading from DSpace 4.2 to 5.2, I get the following error in catalina.out Error using query type: 2 Error using query *:* Error using query statistics_type:workflow AND NOT(previousWorkflowStep: SUBMIT) Anybody have a good fix for this? Cheers *Hilton Gibson* Ubuntu Linux Systems Administrator Stellenbosch University Library http://staff.lib.sun.ac.za/~hgibson/docs/cv/cv.html -- ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Solr Statistics problem
On Tue, Feb 17, 2015 at 7:56 PM, Nada Abo-Eita naboeit...@gmail.com wrote: *Server at http://tspace.qu.edu.qa:8080/solr/statistics http://tspace.qu.edu.qa:8080/solr/statistics returned non ok status:500, * I added the answer to your problem to FAQ: https://wiki.duraspace.org/display/DSPACE/TechnicalFaq#TechnicalFaq-ErrorsconnectingtoSolr Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server from Actuate! Instantly Supercharge Your Business Reports and Dashboards with Interactivity, Sharing, Native Excel Exports, App Integration more Get technology previously reserved for billion-dollar corporations, FREE http://pubads.g.doubleclick.net/gampad/clk?id=190641631iu=/4140/ostg.clktrk___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Solr Statistics now working DSpace 4.1
Although more testing is appropriate I believe statistics it is now. Thank you for the link posted by Hilton Gibson on rebuilding the statistics from scratch. At our site (Sydney University) when we upgrade or create a new instance of DSpace for some new purpose, we usually make a clone of a working machine. We then give it a new IP and name, enter the name and IP into the DNS alter the hosts and DSpace config files to reflect the new name then do the upgrade (or whatever). Running the unix command strings: on the stats database files in dspaceRoot/solr/statistics/data/index/data there are references to the old machine name and this suggested to me this was a possible source of the problems and we needed to rebuild statistics within the new machine identity. The link from Hilton is a little out dated the new procedure should be 1. Backup dspaceRoot/solr/statistics/data 2. Delete dspaceRoot/solr/statistics/data 4. Reboot your machine 5. Convert old log files cd dspaceRoot/log ITEM=`ls dspace.log.*` for i in $ITEM ; do dspaceRoot/bin/dspace stats-log-converter -i $i -o $i.solr done 6. Import converted log files cd dspaceRoot/log ITEM=`ls *.solr` for i in $ITEM ; do dspaceRoot/bin/dspace stats-log-importer -i $i done The statistics counts on any particular item appear correct and are updating counts of test accesses. -- Keir Vaughan-Taylor ERA DSpace Administrator Rm:123 Fisher Library University of Sydney k...@physiol.usyd.du.au ph:+61 2 9351 7408 mb:+61 434 606 885 -- ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Solr Statistics still not working DSpace 4.1
Hi Keir Perhaps this will help: http://wiki.lib.sun.ac.za/index.php/SUNScholar/Asset_Statistics/Old_Versions Cheers hg *Hilton Gibson* Ubuntu Linux Systems Administrator JS Gericke Library Room 1025C Stellenbosch University Private Bag X5036 Stellenbosch 7599 South Africa Tel: +27 21 808 4100 | Cell: +27 84 646 4758 On 21 October 2014 04:52, Keir Vaughan-Taylor k...@physiol.usyd.edu.au wrote: Okay lets try this. If I want to rebuild the solr statistics from the log files. What exactly would the procedure be? -- Keir Vaughan-Taylor ERA DSpace Administrator Rm:123 Fisher Library University of Sydney k...@physiol.usyd.du.au ph:+61 2 9351 7408 mb:+61 434 606 885 -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Solr Statistics still not working DSpace 4.1
On Mon, Oct 20, 2014 at 03:16:15PM +1100, Keir Vaughan-Taylor wrote: I took a copy of the working Solr statistics from a working machine. Then copied the database, the log files and the assetstore across. The test line wget http://127.0.0.1/solr/statistics/select?q=*:* works and also the discovery index optime etc doesn;t crash. Only one problem - no statistics! i notice in the postgres database there is a table called webapp that contains the url of each server from a previous existance. So we clone the server - recompile dspace and get a new URL in this table. On a trial clone i tried making the url all the name of the server instance and it didn't seem to do anything helpful ( or bad) What does this table do? Is it likely to be related to my problem? I think it unlikely that the webapps table is related to your problem. It only exists to allow 'bin/dspace version' to know what webapp.s are running. -- Mark H. Wood Lead Technology Analyst University Library Indiana University - Purdue University Indianapolis 755 W. Michigan Street Indianapolis, IN 46202 317-274-0749 www.ulib.iupui.edu signature.asc Description: Digital signature -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Solr Statistics still not working DSpace 4.1
Okay lets try this. If I want to rebuild the solr statistics from the log files. What exactly would the procedure be? -- Keir Vaughan-Taylor ERA DSpace Administrator Rm:123 Fisher Library University of Sydney k...@physiol.usyd.du.au ph:+61 2 9351 7408 mb:+61 434 606 885 -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Solr Statistics still not working DSpace 4.1
Hi Keir Without knowing which DSpace version, try to check: http://wiki.lib.sun.ac.za/index.php/SUNScholar/SOLR_Statistics The link is based on DSpace 3.2 and Ubuntu 12.04 LTS as per RA-02-2014 in: http://wiki.lib.sun.ac.za/index.php/SUNScholar/Reference_Architecture Cheers hg *Hilton Gibson* Ubuntu Linux Systems Administrator JS Gericke Library Room 1025C Stellenbosch University Private Bag X5036 Stellenbosch 7599 South Africa Tel: +27 21 808 4100 | Cell: +27 84 646 4758 On 16 October 2014 22:52, Keir Vaughan-Taylor k...@physiol.usyd.edu.au wrote: Here is the system and software version information. LSB_VERSION=base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch Red Hat Enterprise Linux Server release 6.6 (Santiago) psql (PostgreSQL) 8.4.20 dspace-4.1-src-release Apache Maven 3.2.1 (ea8b2b07643dbb1b84b6d16e1f08391b666bc1e9; 2014-02-15T04:37:52+10:00) Maven home: /usr/local/maven Java version: 1.7.0_71, vendor: Oracle Corporation Java home: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.71.x86_64/jre Default locale: en_US, platform encoding: UTF-8 OS name: linux, version: 2.6.32-431.29.2.el6.x86_64, arch: amd64, family: unix Apache Ant(TM) version 1.8.2 compiled on May 13 2011 Tomcat7 On Thu, 2014-10-16 at 10:16 +0200, Hilton Gibson wrote: Hi Keir Can you tell us what architecture you are using. See: http://wiki.lib.sun.ac.za/index.php/SUNScholar/Reference_Architecture This will help us to help you. If you have already sent the info to the list then my apologies. Cheers hg Hilton Gibson Ubuntu Linux Systems Administrator JS Gericke Library Room 1025C Stellenbosch University Private Bag X5036 Stellenbosch 7599 South Africa Tel: +27 21 808 4100 | Cell: +27 84 646 4758 On 16 October 2014 05:37, Keir Vaughan-Taylor k...@physiol.usyd.edu.au wrote: Still Can't get the Solr stats to work. Eliminating items without handles made no difference. One problem that might be related is something web based. A recommended test is to run: wget http://127.0.0.1/solr/statistics/select?q=*:* and I get --2014-10-16 14:30:20-- http://127.0.0.1/solr/statistics/select?q=*:* Connecting to 127.0.0.1:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2014-10-16 14:30:20 ERROR 500: Internal Server Error. This is also the error when trying to run optimise in the nightly run. I've checked the config files, the server names, I've done a rebuild from source, (restarted httpd and Tomcat of course). I've gone through the /etc/httpd/conf files and even substituted a directory from another Solr working instance of DSpace. I tried replacing the entire /dspace/config directory with that from the working instance (although changing the nameserver entries) Somebody must have been down this path - throw me a clue please. -- Keir Vaughan-Taylor ERA DSpace Administrator Rm:123 Fisher Library University of Sydney k...@physiol.usyd.du.au ph:+61 2 9351 7408 mb:+61 434 606 885 -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List +Etiquette -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Keir Vaughan-Taylor ERA DSpace Administrator Rm:123 Fisher Library University of Sydney k...@physiol.usyd.du.au ph:+61 2 9351 7408 mb:+61 434 606 885 -- Comprehensive Server Monitoring with Site24x7.
Re: [Dspace-tech] Solr Statistics still not working DSpace 4.1
Hi Keir Can you tell us what architecture you are using. See: http://wiki.lib.sun.ac.za/index.php/SUNScholar/Reference_Architecture This will help us to help you. If you have already sent the info to the list then my apologies. Cheers hg *Hilton Gibson* Ubuntu Linux Systems Administrator JS Gericke Library Room 1025C Stellenbosch University Private Bag X5036 Stellenbosch 7599 South Africa Tel: +27 21 808 4100 | Cell: +27 84 646 4758 On 16 October 2014 05:37, Keir Vaughan-Taylor k...@physiol.usyd.edu.au wrote: Still Can't get the Solr stats to work. Eliminating items without handles made no difference. One problem that might be related is something web based. A recommended test is to run: wget http://127.0.0.1/solr/statistics/select?q=*:* and I get --2014-10-16 14:30:20-- http://127.0.0.1/solr/statistics/select?q=*:* Connecting to 127.0.0.1:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2014-10-16 14:30:20 ERROR 500: Internal Server Error. This is also the error when trying to run optimise in the nightly run. I've checked the config files, the server names, I've done a rebuild from source, (restarted httpd and Tomcat of course). I've gone through the /etc/httpd/conf files and even substituted a directory from another Solr working instance of DSpace. I tried replacing the entire /dspace/config directory with that from the working instance (although changing the nameserver entries) Somebody must have been down this path - throw me a clue please. -- Keir Vaughan-Taylor ERA DSpace Administrator Rm:123 Fisher Library University of Sydney k...@physiol.usyd.du.au ph:+61 2 9351 7408 mb:+61 434 606 885 -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Solr Statistics still not working DSpace 4.1
Here is the system and software version information. LSB_VERSION=base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch Red Hat Enterprise Linux Server release 6.6 (Santiago) psql (PostgreSQL) 8.4.20 dspace-4.1-src-release Apache Maven 3.2.1 (ea8b2b07643dbb1b84b6d16e1f08391b666bc1e9; 2014-02-15T04:37:52+10:00) Maven home: /usr/local/maven Java version: 1.7.0_71, vendor: Oracle Corporation Java home: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.71.x86_64/jre Default locale: en_US, platform encoding: UTF-8 OS name: linux, version: 2.6.32-431.29.2.el6.x86_64, arch: amd64, family: unix Apache Ant(TM) version 1.8.2 compiled on May 13 2011 Tomcat7 On Thu, 2014-10-16 at 10:16 +0200, Hilton Gibson wrote: Hi Keir Can you tell us what architecture you are using. See: http://wiki.lib.sun.ac.za/index.php/SUNScholar/Reference_Architecture This will help us to help you. If you have already sent the info to the list then my apologies. Cheers hg Hilton Gibson Ubuntu Linux Systems Administrator JS Gericke Library Room 1025C Stellenbosch University Private Bag X5036 Stellenbosch 7599 South Africa Tel: +27 21 808 4100 | Cell: +27 84 646 4758 On 16 October 2014 05:37, Keir Vaughan-Taylor k...@physiol.usyd.edu.au wrote: Still Can't get the Solr stats to work. Eliminating items without handles made no difference. One problem that might be related is something web based. A recommended test is to run: wget http://127.0.0.1/solr/statistics/select?q=*:* and I get --2014-10-16 14:30:20-- http://127.0.0.1/solr/statistics/select?q=*:* Connecting to 127.0.0.1:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2014-10-16 14:30:20 ERROR 500: Internal Server Error. This is also the error when trying to run optimise in the nightly run. I've checked the config files, the server names, I've done a rebuild from source, (restarted httpd and Tomcat of course). I've gone through the /etc/httpd/conf files and even substituted a directory from another Solr working instance of DSpace. I tried replacing the entire /dspace/config directory with that from the working instance (although changing the nameserver entries) Somebody must have been down this path - throw me a clue please. -- Keir Vaughan-Taylor ERA DSpace Administrator Rm:123 Fisher Library University of Sydney k...@physiol.usyd.du.au ph:+61 2 9351 7408 mb:+61 434 606 885 -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List +Etiquette -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Keir Vaughan-Taylor ERA DSpace Administrator Rm:123 Fisher Library University of Sydney k...@physiol.usyd.du.au ph:+61 2 9351 7408 mb:+61 434 606 885 -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Solr Statistics still not working DSpace 4.1
Hi Keir, On 16/10/14 16:37, Keir Vaughan-Taylor wrote: Still Can't get the Solr stats to work. Eliminating items without handles made no difference. One problem that might be related is something web based. A recommended test is to run: wget http://127.0.0.1/solr/statistics/select?q=*:* and I get --2014-10-16 14:30:20-- http://127.0.0.1/solr/statistics/select?q=*:* Connecting to 127.0.0.1:80... connected. HTTP request sent, awaiting response... 500 Internal Server Error 2014-10-16 14:30:20 ERROR 500: Internal Server Error. Sorry if this has already been covered, I haven't had time to follow this list lately. Have you looked through your log files for a stack trace when the 500 error happens? catalina.out (in /var/log/tomcat6 on my RHEL6 box) or the solr.log file in the DSpace log directory is where I'd look. cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Solr Statistics not appearing and not appearing
Sor statistics were working in the past, then at some stage it just
stopped. We think this happened after data entries were made by a person
using a chinese keyboard. After some problems there were attempts to
delete those items using that keyboard - we don't know for sure. It
doesn't matter we need, I think to correct the problem of items with
missing handles by deleting all reference to that item in the database.
There are other posts on this forum suggesting the cause is missing
handles.
http://dspace.2283337.n4.nabble.com/Problem-attempting-to-migrate-from-DSpace-1-8-directly-to-4-1-td4673622.html
http://dspace.2283337.n4.nabble.com/problems-with-OAI-td4671531.html
that problem is not just missing handles but also dangling pointers from
one table to another. We have items with missing handles and you can't
delete them because entries in some tables are missing and standard
DSpace delete doesn't work.
On Mon, 2014-10-13 at 09:12 +0300, Edward Safari wrote:
did it uses to display the solr statistics before or wat happened?
On Mon, Oct 13, 2014 at 4:34 AM, Keir Vaughan-Taylor
k...@physiol.usyd.edu.au wrote:
I believe the error is related to items in the repository that
have some items have been corrupted and don't have a handle.
Some items have entries in some tables and not in others.
The GUI either fails to delete these items so I need to erase
all entries related to a specific item_id using SQL.
The SQL code below should erase everything in the database
related to an item code.
I am not sure what to do about the physical files .
Is it best to leave them in place or should they be removed?
What happens about the sequence id in the bitstream table - is
it a problem to just leave a hole?
I would appreciate it if anyone can see a problem with the
code and make any comments or suggestions
# Clean DSpace based on itemId
# Give IRMA id returns handle for that publication
# Run as id dspace
[ $# -eq 0 ] { echo Usage: $0 item_id; exit 1; }
itemId=$1
echo Erasing all records referencing item: $itemId
echo -n Do you wish to continue y/n
read ans
if [ $ans != y ]
then
exit 1
fi
# Delete bundles associated with item_id
# Delete bitstreams associated with item_id
# Get list of bundle ids associated with item
bundleList=$(psql -A -t -c select bundle_id from item2bundle
where item_id='$itemId';)
# For each bundle id delete the bitstream and delete the
bundle to bitstream entry
for bundle in $bundleList
do
btStmIdList=$(psql -A -t -c select bitstream_id from
bundle2bitstream where bundle_id=(select bundle_id from
item2bundle where item_id='$itemId');)
for bstrmId in $btStmIdList
do
# Do something with files
filenames=$( psql -A -t -c select name,internal_id
from bitstream where bitstream_id =$bstrmId;)
for fn in filenames
do
# do something with files maybe
done
psql -A -t -c select * from bitstream where
bitstream_id =$bstrmId;
done
psql -A -t -c select * from bundle2bitstream where
bundle_id='$bundle';
psql -A -t -c select * from bundle where
bundle_id='$bundle';
psql -A -t -c select * from bundle2bitstream where
bundle_id='$bundle';
psql -A -t -c select * from bundle where
bundle_id='$bundle';
done
for tableName in collection2item metadatavalue item
item2bundle community2item collection2item workflowitem
do
psql -A -t -c delete from $tableName where
item_id='$itemId';
done
On Fri, 2014-10-10 at 11:42 +1100, Keir Vaughan-Taylor wrote:
I have two Dspace 4.0 instances installed both using Solr
Statistics and
all appears to be working.
In both cases the generic statistics appears okay.
The academics love it when they see how many accesses there have
been by
going to the item in the repository and clicking on the nearby
statistics button.
An upgrade from 1.8.2 to 4.0 recently took place on one of the
machines.
With the 1.8.2 version all the statistics features were working. Now
however, on the upgraded machine although the
Re: [Dspace-tech] Solr Statistics not appearing and not appearing
I believe the error is related to items in the repository that have some
items have been corrupted and don't have a handle.
Some items have entries in some tables and not in others.
The GUI either fails to delete these items so I need to erase all
entries related to a specific item_id using SQL.
The SQL code below should erase everything in the database related to an
item code.
I am not sure what to do about the physical files .
Is it best to leave them in place or should they be removed?
What happens about the sequence id in the bitstream table - is it a
problem to just leave a hole?
I would appreciate it if anyone can see a problem with the code and make
any comments or suggestions
# Clean DSpace based on itemId
# Give IRMA id returns handle for that publication
# Run as id dspace
[ $# -eq 0 ] { echo Usage: $0 item_id; exit 1; }
itemId=$1
echo Erasing all records referencing item: $itemId
echo -n Do you wish to continue y/n
read ans
if [ $ans != y ]
then
exit 1
fi
# Delete bundles associated with item_id
# Delete bitstreams associated with item_id
# Get list of bundle ids associated with item
bundleList=$(psql -A -t -c select bundle_id from item2bundle where
item_id='$itemId';)
# For each bundle id delete the bitstream and delete the bundle to
bitstream entry
for bundle in $bundleList
do
btStmIdList=$(psql -A -t -c select bitstream_id from
bundle2bitstream where bundle_id=(select bundle_id from item2bundle
where item_id='$itemId');)
for bstrmId in $btStmIdList
do
# Do something with files
filenames=$( psql -A -t -c select name,internal_id from
bitstream where bitstream_id =$bstrmId;)
for fn in filenames
do
# do something with files maybe
done
psql -A -t -c select * from bitstream where bitstream_id =
$bstrmId;
done
psql -A -t -c select * from bundle2bitstream where
bundle_id='$bundle';
psql -A -t -c select * from bundle where bundle_id='$bundle';
psql -A -t -c select * from bundle2bitstream where
bundle_id='$bundle';
psql -A -t -c select * from bundle where bundle_id='$bundle';
done
for tableName in collection2item metadatavalue item item2bundle
community2item collection2item workflowitem
do
psql -A -t -c delete from $tableName where item_id='$itemId';
done
On Fri, 2014-10-10 at 11:42 +1100, Keir Vaughan-Taylor wrote:
I have two Dspace 4.0 instances installed both using Solr Statistics and
all appears to be working.
In both cases the generic statistics appears okay.
The academics love it when they see how many accesses there have been by
going to the item in the repository and clicking on the nearby
statistics button.
An upgrade from 1.8.2 to 4.0 recently took place on one of the machines.
With the 1.8.2 version all the statistics features were working. Now
however, on the upgraded machine although the overall statistics still
appear, the button associated with a publication item no longer renders
statistics information.
The statistic page appears okay but there is no data presented.
The other separate instance of DSpace 4.0 for a different database does
work. So i know its not a Dspace 4.0 bug.
I cannot find what is different about the two.
DSpace.cfg are the same, solr.cfg is the same.
The logs don't reveal anything I can identify as a problem.
The running of nightly Discovery indexing is the same.
Please can anyone suggest a list of places I should be looking at to get
this going.
--
Keir Vaughan-Taylor
ERA DSpace Administrator
Rm:123 Fisher Library University of Sydney
k...@physiol.usyd.du.au
ph:+61 2 9351 7408
mb:+61 434 606 885
--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://p.sf.net/sfu/Zoho___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Solr statistics not showing any numbers
Hi Monika,
Could you check the solr.xml to see whether there is a core for
statistics (core name=statistics instanceDir=statistics /).
If yes, check the log files for error messages. There could be some
permissions problems (i.e. Tomcat needs write access).
Regards,
Anja
--
Anja Le Blanc
Jorum Team
Mimas
On 12/06/2014 17:57, Monika Mevenkamp wrote:
I enabled solr by deploying the webapp to my server
wget 'http://localhost:8080/solr/statistics' gives a 404
wget 'http://localhost:8080/solr'gives 200
so I tried withsolr.server = http://localhost:8080/solr in
config/modules/solr-statistics.cfg
but still no numbers in the UI (I use JSPUI, 1.8) , yes I did restart
tomcat
find solr/statistics/data -type f -exec ls -l {} \;
shows that files have been touched around when I was experimenting with
solr settings
What should I try next ?
Monika
--
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Solr statistics not showing any numbers
Anja It turned out that there was no /dspace/solr directory. We used to run dspace without solr/stats for a long long time and I had simply added a link to the solr webapp to tomcat's webapps directory, but did not move the solr configs into the install dir. A simple ant overwrite_solr_configs did the trick Thanks for putting me onto the right track Monika On Fri Jun 13 03:11:26 2014, Anja Le Blanc wrote: Hi Monika, Could you check the solr.xml to see whether there is a core for statistics (core name=statistics instanceDir=statistics /). If yes, check the log files for error messages. There could be some permissions problems (i.e. Tomcat needs write access). Regards, Anja -- Monika Mevenkamp phone: 609-258-4161 Lewis Library,Washington Road and Ivy Lane, Princeton University, Princeton, NJ 08544 -- HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions Find What Matters Most in Your Big Data with HPCC Systems Open Source. Fast. Scalable. Simple. Ideal for Dirty Data. Leverages Graph Analysis for Fast Processing Easy Data Exploration http://p.sf.net/sfu/hpccsystems ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] solr statistics question
Hi Bill, these are very likely the collection/community logo's. Try comparing the ones you see occurring with the ones that come up if you query your DB as follows: SELECT * FROM bitstream WHERE bitstream_id IN (SELECT logo_bitstream_id FROM collection); SELECT * FROM bitstream WHERE bitstream_id IN (SELECT logo_bitstream_id FROM community); rgds Bram -- [image: logo] *Bram Luyten* +1 202 684 6365 *2888 Loker Avenue East, Suite 315, Carlsbad, CA. 92010* *Esperantolaan 4, Heverlee 3001, Belgium* www.atmire.comhttp://atmire.com/website/?q=servicesutm_source=emailfooterutm_medium=emailutm_campaign=braml On Thu, Apr 17, 2014 at 10:08 PM, Bill Tantzen wile...@gmail.com wrote: I'm on dspace 4.1. In my solr statistics data, I have noticed a small percentage (~5%) of type:0 (bitstream) records without any context -- no owningComm, owningColl, nor owningItem. Most of these have no bundleName either. Can anyone explain to me how these bitstream records are being added to solr? I have tried to reproduce this via the gui in every way I can think of, but have not been able to do it... Thanks in advance! Bill -- Learn Graph Databases - Download FREE O'Reilly Book Graph Databases is the definitive new guide to graph databases and their applications. Written by three acclaimed leaders in the field, this first edition is now available. Download your free book today! http://p.sf.net/sfu/NeoTech ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Start Your Social Network Today - Download eXo Platform Build your Enterprise Intranet with eXo Platform Software Java Based Open Source Intranet - Social, Extensible, Cloud Ready Get Started Now And Turn Your Intranet Into A Collaboration Platform http://p.sf.net/sfu/ExoPlatform___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Solr Statistics Error caused by lack of Java heap space
Try with this https://wiki.duraspace.org/display/DSDOC3x/Performance+Tuning+DSpace#PerformanceTuningDSpace-GiveTomcatMoreJavaHeapMemory Javier Távara 2013/9/9 Jean Paul jean.paul.1...@gmail.com Hello We're getting this error periodically: 2013-09-09 16:57:02,790 ERROR org.dspace.app.webui.servlet.DisplayStatisticsServlet @ Error occured while creating statistics for dso with ID: 596 and type 2 and handle: 1343/552 org.apache.solr.client.solrj.SolrServerException: Error executing query at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:95) at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:118) at org.dspace.statistics.SolrLogger.query(SolrLogger.java:893) ... Caused by: org.apache.solr.common.SolrException: Java heap space java.lang.OutOfMemoryError: Java heap space Java heap space java.lang.OutOfMemoryError: Java heap space I'm wondering if anyone has seen similar error and can provide some help with it. Thanks in advance. Jean-P -- Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! Discover the easy way to master current and previous Microsoft technologies and advance your career. Get an incredible 1,500+ hours of step-by-step tutorial videos with LearnDevNow. Subscribe today and save! http://pubads.g.doubleclick.net/gampad/clk?id=58041391iu=/4140/ostg.clktrk ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! Discover the easy way to master current and previous Microsoft technologies and advance your career. Get an incredible 1,500+ hours of step-by-step tutorial videos with LearnDevNow. Subscribe today and save! http://pubads.g.doubleclick.net/gampad/clk?id=58041391iu=/4140/ostg.clktrk___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] SOLR statistics problem in DSPACE 3.1
Hi helix84.Thanks four your support. In fact, solr-statistics.cfg file had URL to the repository. I changed it for localhost with port 8080 and it worked.Regards,Walter BlandónAnalista TécnicoCentro Cultural Biblioteca Luis Echavarría VillegasTel: (57) (4) 2619500, extensión 9365E-mail: cblan...@eafit.edu.coUniversidad EAFIT, Medellín, ColombiaEl 02/05/13, helix84 heli...@centrum.sk escribió:Hi Carlos,yes, it seems that for some reason DSpace can't connect to Solr inorder to write a log entry. But the message doesn't specify what theconnection was reset (I'm guessing a timeout of 100 secondssomewhere).Verify whether Solr client is listening and can be connected to from localhost:1) Look into [dspace]/config/modules/solr-statistics.cfg at the serverproperty and remember the URL and port number.2) Using that URL, try querying Solr statistics core:wget http://localhost:8080/solr/statistics/select?indent=onversion=2.2start=0rows=10fl=*%2Cscoreqt=standardwt=standardexplainOther=hl.fl=q=type:0-O response.xml3) If that fails, run netstat -tulpn and check whether there'ssomething listening on that port.4) If that succeeds, check that your firewall isn't blocking that port.5) If that dosn't resolve the problem, enable logging (in[dspace]/config/log4j.properties) in order to determine the moreimmediate cause of error:http://hc.apache.org/httpclient-3.x/logging.html#Log4j_ExamplesRegards,~~helix84Compulsory reading: DSpace Mailing List Etiquettehttps://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] SOLR statistics problem in DSPACE 3.1
Hi Carlos, yes, it seems that for some reason DSpace can't connect to Solr in order to write a log entry. But the message doesn't specify what the connection was reset (I'm guessing a timeout of 100 seconds somewhere). Verify whether Solr client is listening and can be connected to from localhost: 1) Look into [dspace]/config/modules/solr-statistics.cfg at the server property and remember the URL and port number. 2) Using that URL, try querying Solr statistics core: wget http://localhost:8080/solr/statistics/select?indent=onversion=2.2start=0rows=10fl=*%2Cscoreqt=standardwt=standardexplainOther=hl.fl=q=type:0; -O response.xml 3) If that fails, run netstat -tulpn and check whether there's something listening on that port. 4) If that succeeds, check that your firewall isn't blocking that port. 5) If that dosn't resolve the problem, enable logging (in [dspace]/config/log4j.properties) in order to determine the more immediate cause of error: http://hc.apache.org/httpclient-3.x/logging.html#Log4j_Examples Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] SOLR statistics
Guillermo, That's not my problem. I can use wget and lynx on the URL. That works. I just never have any stats show up. I would appreciate any help anyone can offer. *Jason Fowler, CA, MSLS* On Fri, Feb 3, 2012 at 5:14 PM, Guillermo Esteban Murillo Goussen mem...@gmail.com wrote: I'm having the same problem, I think is something I'm not copying into the Apache scope. I read the documentation and tried this: wget http://127.0.0.1/solr/statistics/select?q=*:* to find out which is the correct solr.server path, but I keep getting Error 404 or Error 500. Any help? -- La pureza del alma no se pierde sin consentimiento -- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] SOLR statistics
I'm having the same problem, I think is something I'm not copying into the Apache scope. I read the documentation and tried this: wget http://127.0.0.1/solr/statistics/select?q=*:* to find out which is the correct solr.server path, but I keep getting Error 404 or Error 500. Any help? -- La pureza del alma no se pierde sin consentimiento -- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Solr/Statistics on DSpace-1.7.2
Hi Bruno I have cheked logs, there was failure of DNS entry. Thanks for clearing my doubts. rergards Surinder Kumar NIC, New Delhi On 10/25/11, Bruno Oliveira brunopit...@gmail.com wrote: Hi, works only for real servers ip. using localhost will not work because the GIS is done by ip At, Bruno Carvalho 2011/10/24 Surinder Kumar Gaba surinder.g...@gmail.com I am testing solr statistics on DSpace-1.7.2. Total Visits, Total Visits per Month and File Download are working fine except Total Country Views and Total City Views are showing empty. The following configurations in dspace.cfg are defined. solr.log.server = http://127.0.0.1:8080/solr/statistics solr.dbfile = /dspace172/config/GeoLiteCity.dat solr.resolver.timeout = 200 statistics.item.authorization.admin=true #solr.statistics.logBots = true #solr.statistics.query.filter.isBot = true Please help. regards -- Surinder Kumar Gaba Senior Technical Director Digital Archiving Management Group National Informatics Centre A-Block, CGO Complex, Lodhi Road New Delhi-110 003 Tel : 011-24362359 (o) : 011-27865224 (r) -- The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Cisco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Surinder Kumar Gaba Senior Technical Director Digital Archiving Management Group National Informatics Centre A-Block, CGO Complex, Lodhi Road New Delhi-110 003 Tel : 011-24362359 (o) : 011-27865224 (r) -- The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Cisco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Solr/Statistics on DSpace-1.7.2
Hi, works only for real servers ip. using localhost will not work because the GIS is done by ip At, Bruno Carvalho 2011/10/24 Surinder Kumar Gaba surinder.g...@gmail.com I am testing solr statistics on DSpace-1.7.2. Total Visits, Total Visits per Month and File Download are working fine except Total Country Views and Total City Views are showing empty. The following configurations in dspace.cfg are defined. solr.log.server = http://127.0.0.1:8080/solr/statistics solr.dbfile = /dspace172/config/GeoLiteCity.dat solr.resolver.timeout = 200 statistics.item.authorization.admin=true #solr.statistics.logBots = true #solr.statistics.query.filter.isBot = true Please help. regards -- Surinder Kumar Gaba Senior Technical Director Digital Archiving Management Group National Informatics Centre A-Block, CGO Complex, Lodhi Road New Delhi-110 003 Tel : 011-24362359 (o) : 011-27865224 (r) -- The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Cisco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Cisco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] solr statistics
Hi Tint, Our repository is running on 1.6.2 and we have been using solr for a few months now. There seems to be some problem with solr statistics. Bitstream for some items were downloaded more than a few thousand times within a month from the same place. How can I filter out such systematic access (by bots/spiders etc)? Take a look at the following tool: /dspace/bin/dspace stats-util -h usage: StatisticsClient -b,--reindex-bitstreams Reindex the bitstreams to ensure we have the bundle name -r,--remove-deleted-bitstreams While indexing the bundle names remove the statistics about deleted bitstreams -u,--update-spider-files Update Spider IP Files from internet into /dspace/config/spiders -f,--delete-spiders-by-flag Delete Spiders in Solr By isBot Flag -i,--delete-spiders-by-ipDelete Spiders in Solr By IP Address -m,--mark-spidersUpdate isBot Flag in Solr -h,--helphelp -o,--optimizeRun maintenance on the SOLR index You might need to first register the IP address of the bots in /dspace/config/spiders/ I hope that helps, Stuart Lewis Digital Development Manager Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: +64 (0)9 373 7599 x81928 -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2d-oct ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] SOLR statistics
Hi Jeffery Try: http://wiki.lib.sun.ac.za/index.php/SUNScholar/Asset_Statistics for some tips. Cheers hg On 02/02/2011 21:18, Jeffrey Trimble wrote: We performed a dump and reload of our production DSpace instance. While this has taken about 10 days to reload all the data, we have nice clean indexes and we implemented the XMLUI interface as default.Our prior statistics where gathered and converted into a Excel spreadsheet for archival and legal purposes. So, we configures SOLR to work from the get-go, but nothing appears when you click on statistics. And if you change directories to http://localhost:8080/solr you get 403'd. We given that the Access to the specified resource has been forbidden. But I can't see any ownership issues. Is there something I've overlooked? I might be the brightest crayon in the box, but hey, this just befuddles me. Thanks, Jeff Jeffrey Trimble System LIbrarian William F. Maag Library Youngstown State University 330.941.2483 (Office) jatrim...@ysu.edu http://www.maag.ysu.edu http://digital.maag.ysu.edu For he is the Kwisatz Haderach... -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Hilton Gibson Systems Administrator JS Gericke Library Room 1025 Stellenbosch University Private Bag X5036 Stellenbosch 7599 South Africa Tel: +27 21 808 4100 | Cell: +27 84 646 4758 Simplicity is the ultimate sophistication Leonardo da Vinci -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] SOLR statistics in XMLUI
The next thing to check is your [dspace]/config/xmlui.xconf file to
see if it contains the line:
aspects
aspect name=Artifact Browser
path=resource://aspects/ArtifactBrowser/ /
aspect name=Administration
path=resource://aspects/Administrative/ /
aspect name=E-Person path=resource://aspects/EPerson/ /
aspect name=Submission and Workflow
path=resource://aspects/Submission/ /
aspect name=Statistics path=resource://aspects/Statistics/ /
/aspects
It sounds like you missing the Statistics Aspect that will add int he
Statistics functionality in the UI.
Mark
On Wed, Apr 28, 2010 at 12:48 PM, Poulter, Dale
dale.poul...@vanderbilt.edu wrote:
Mark and Marvin,
Thanks. I have everything set correctly (I think) but am still not able to
view through the xmlui. In the config file I have :
statistics.item.authorization.admin = false
I am using the delivered Reference theme and delivered jspui. The jspui
stats are available at
http://libdig2.library.vanderbilt.edu:8080/jspui/displaystats?handle=1803%2F2729submit_simple=View+Statistics
but they are not available through the xmlui
(http://libdig2.library.vanderbilt.edu:8080/xmlui/handle/1803/2729/statistics
, http://libdig2.library.vanderbilt.edu:8080/xmlui/handle/1803/2729 ). I
have checked for errors in the cocoon, dspace, and catalina logs and have
not seen any errors. The logs (see below) do indicate that the Solr
indexer is working though. Thanks in advance for any suggestions.
INFO: [statistics] webapp=null path=null params={rows=10start=0q=solr}
hits=0 status=0 QTime=1
Apr 28, 2010 2:34:22 PM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
Apr 28, 2010 2:34:22 PM org.apache.solr.core.SolrCore execute
INFO: [statistics] webapp=null path=null params={rows=10start=0q=rocks}
hits=0 status=0 QTime=0
Apr 28, 2010 2:34:22 PM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
Apr 28, 2010 2:34:22 PM org.apache.solr.core.SolrCore execute
INFO: [statistics] webapp=null path=null
params={q=static+newSearcher+warming+query+from+solrconfig.xml} hits=0
status=0 QTime=1
Apr 28, 2010 2:34:22 PM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
Apr 28, 2010 2:34:22 PM org.apache.solr.core.SolrCore registerSearcher
INFO: [statistics] Registered new searcher searc...@ff7cd8 main
Apr 28, 2010 2:34:22 PM org.apache.solr.search.SolrIndexSearcher close
INFO: Closing searc...@1243f75 main
From: mdigg...@gmail.com [mailto:mdigg...@gmail.com] On Behalf Of Mark
Diggory
Sent: Wednesday, April 28, 2010 2:14 PM
To: Poulter, Dale
Cc: dspace-tech@lists.sourceforge.net; Francis, Ronee L
Subject: Re: [Dspace-tech] SOLR statistics in XMLUI
Dale,
No changes are necessary. The link to the new statistics pages shows as a
option on the menu and is resolvable via the following URI path relative to
the Item.
http://demo.dspace.org/xmlui/handle/123456789/29/statistics
Rendering has nothing to do with DIM-Handler in this case. You may check
your dspace.cfg setting to verify if your restricting it to administrator or
not. We are working on documentation here atm.
http://fedora-commons.org/confluence/display/DSTEST/DSpace+Statistics
Mark
On Wed, Apr 28, 2010 at 11:24 AM, Poulter, Dale
dale.poul...@vanderbilt.edu wrote:
Good afternoon,
We have Dspace 1.6 working and the statistics show up as expected in the
jspui. However, the solr statistics do not appear in the xmlui. Are
changes needed in the DIM-Handler.xsl to get the statistics to appear? If
so, are these changes documented? Thanks for your assistance.
--Dale
---
Dale Poulter
Automation Coordinator
Library Information Technology Services
Vanderbilt University
Suite 700
110 21st Avenue South
Nashville, TN 37240
(615)343-5388
(615)343-8834 (fax)
(615)207-9705 (cell)
dale.poul...@vanderbilt.edu
--
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
--
Mark R. Diggory
Head of U.S. Operations - @mire
http://www.atmire.com - Institutional Repository Solutions
http://www.togather.eu - Before getting together, get t...@ther
--
Mark R. Diggory
Head of U.S. Operations - @mire
http://www.atmire.com - Institutional Repository Solutions
http://www.togather.eu - Before getting together, get t...@ther
--
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] SOLR statistics in XMLUI
Mark,
Thank you!!! I was missing the statistics piece in the xmlui.xconf. Again,
thank you for your assistance.
-Original Message-
From: mdigg...@gmail.com [mailto:mdigg...@gmail.com] On Behalf Of Mark Diggory
Sent: Thursday, April 29, 2010 1:16 AM
To: Poulter, Dale
Cc: dspace-tech@lists.sourceforge.net; Francis, Ronee L
Subject: Re: [Dspace-tech] SOLR statistics in XMLUI
The next thing to check is your [dspace]/config/xmlui.xconf file to
see if it contains the line:
aspects
aspect name=Artifact Browser
path=resource://aspects/ArtifactBrowser/ /
aspect name=Administration
path=resource://aspects/Administrative/ /
aspect name=E-Person path=resource://aspects/EPerson/ /
aspect name=Submission and Workflow
path=resource://aspects/Submission/ /
aspect name=Statistics path=resource://aspects/Statistics/ /
/aspects
It sounds like you missing the Statistics Aspect that will add int he
Statistics functionality in the UI.
Mark
On Wed, Apr 28, 2010 at 12:48 PM, Poulter, Dale
dale.poul...@vanderbilt.edu wrote:
Mark and Marvin,
Thanks. I have everything set correctly (I think) but am still not able to
view through the xmlui. In the config file I have :
statistics.item.authorization.admin = false
I am using the delivered Reference theme and delivered jspui. The jspui
stats are available at
http://libdig2.library.vanderbilt.edu:8080/jspui/displaystats?handle=1803%2F2729submit_simple=View+Statistics
but they are not available through the xmlui
(http://libdig2.library.vanderbilt.edu:8080/xmlui/handle/1803/2729/statistics
, http://libdig2.library.vanderbilt.edu:8080/xmlui/handle/1803/2729 ). I
have checked for errors in the cocoon, dspace, and catalina logs and have
not seen any errors. The logs (see below) do indicate that the Solr
indexer is working though. Thanks in advance for any suggestions.
INFO: [statistics] webapp=null path=null params={rows=10start=0q=solr}
hits=0 status=0 QTime=1
Apr 28, 2010 2:34:22 PM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
Apr 28, 2010 2:34:22 PM org.apache.solr.core.SolrCore execute
INFO: [statistics] webapp=null path=null params={rows=10start=0q=rocks}
hits=0 status=0 QTime=0
Apr 28, 2010 2:34:22 PM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
Apr 28, 2010 2:34:22 PM org.apache.solr.core.SolrCore execute
INFO: [statistics] webapp=null path=null
params={q=static+newSearcher+warming+query+from+solrconfig.xml} hits=0
status=0 QTime=1
Apr 28, 2010 2:34:22 PM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
Apr 28, 2010 2:34:22 PM org.apache.solr.core.SolrCore registerSearcher
INFO: [statistics] Registered new searcher searc...@ff7cd8 main
Apr 28, 2010 2:34:22 PM org.apache.solr.search.SolrIndexSearcher close
INFO: Closing searc...@1243f75 main
From: mdigg...@gmail.com [mailto:mdigg...@gmail.com] On Behalf Of Mark
Diggory
Sent: Wednesday, April 28, 2010 2:14 PM
To: Poulter, Dale
Cc: dspace-tech@lists.sourceforge.net; Francis, Ronee L
Subject: Re: [Dspace-tech] SOLR statistics in XMLUI
Dale,
No changes are necessary. The link to the new statistics pages shows as a
option on the menu and is resolvable via the following URI path relative to
the Item.
http://demo.dspace.org/xmlui/handle/123456789/29/statistics
Rendering has nothing to do with DIM-Handler in this case. You may check
your dspace.cfg setting to verify if your restricting it to administrator or
not. We are working on documentation here atm.
http://fedora-commons.org/confluence/display/DSTEST/DSpace+Statistics
Mark
On Wed, Apr 28, 2010 at 11:24 AM, Poulter, Dale
dale.poul...@vanderbilt.edu wrote:
Good afternoon,
We have Dspace 1.6 working and the statistics show up as expected in the
jspui. However, the solr statistics do not appear in the xmlui. Are
changes needed in the DIM-Handler.xsl to get the statistics to appear? If
so, are these changes documented? Thanks for your assistance.
--Dale
---
Dale Poulter
Automation Coordinator
Library Information Technology Services
Vanderbilt University
Suite 700
110 21st Avenue South
Nashville, TN 37240
(615)343-5388
(615)343-8834 (fax)
(615)207-9705 (cell)
dale.poul...@vanderbilt.edu
--
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
--
Mark R. Diggory
Head of U.S. Operations - @mire
http://www.atmire.com - Institutional Repository Solutions
http://www.togather.eu - Before getting together, get t...@ther
--
Mark R. Diggory
Head of U.S. Operations - @mire
http://www.atmire.com - Institutional Repository Solutions
http
Re: [Dspace-tech] SOLR statistics in XMLUI
Dale, No changes are necessary. The link to the new statistics pages shows as a option on the menu and is resolvable via the following URI path relative to the Item. http://demo.dspace.org/xmlui/handle/123456789/29/statistics Rendering has nothing to do with DIM-Handler in this case. You may check your dspace.cfg setting to verify if your restricting it to administrator or not. We are working on documentation here atm. http://fedora-commons.org/confluence/display/DSTEST/DSpace+Statistics Mark On Wed, Apr 28, 2010 at 11:24 AM, Poulter, Dale dale.poul...@vanderbilt.edu wrote: Good afternoon, We have Dspace 1.6 working and the statistics show up as expected in the jspui. However, the solr statistics do not appear in the xmlui. Are changes needed in the DIM-Handler.xsl to get the statistics to appear? If so, are these changes documented? Thanks for your assistance. --Dale --- Dale Poulter Automation Coordinator Library Information Technology Services Vanderbilt University Suite 700 110 21st Avenue South Nashville, TN 37240 (615)343-5388 (615)343-8834 (fax) (615)207-9705 (cell) dale.poul...@vanderbilt.edu -- ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Mark R. Diggory Head of U.S. Operations - @mire http://www.atmire.com - Institutional Repository Solutions http://www.togather.eu - Before getting together, get t...@ther -- ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] SOLR statistics in XMLUI
Mark and Marvin,
Thanks. I have everything set correctly (I think) but am still not able to
view through the xmlui. In the config file I have :
statistics.item.authorization.admin = false
I am using the delivered Reference theme and delivered jspui. The jspui stats
are available at
http://libdig2.library.vanderbilt.edu:8080/jspui/displaystats?handle=1803%2F2729submit_simple=View+Statistics
but they are not available through the xmlui
(http://libdig2.library.vanderbilt.edu:8080/xmlui/handle/1803/2729/statistics
, http://libdig2.library.vanderbilt.edu:8080/xmlui/handle/1803/2729 ). I have
checked for errors in the cocoon, dspace, and catalina logs and have not seen
any errors. The logs (see below) do indicate that the Solr indexer is working
though. Thanks in advance for any suggestions.
INFO: [statistics] webapp=null path=null params={rows=10start=0q=solr} hits=0
status=0 QTime=1
Apr 28, 2010 2:34:22 PM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
Apr 28, 2010 2:34:22 PM org.apache.solr.core.SolrCore execute
INFO: [statistics] webapp=null path=null params={rows=10start=0q=rocks}
hits=0 status=0 QTime=0
Apr 28, 2010 2:34:22 PM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
Apr 28, 2010 2:34:22 PM org.apache.solr.core.SolrCore execute
INFO: [statistics] webapp=null path=null
params={q=static+newSearcher+warming+query+from+solrconfig.xml} hits=0 status=0
QTime=1
Apr 28, 2010 2:34:22 PM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
Apr 28, 2010 2:34:22 PM org.apache.solr.core.SolrCore registerSearcher
INFO: [statistics] Registered new searcher searc...@ff7cd8 main
Apr 28, 2010 2:34:22 PM org.apache.solr.search.SolrIndexSearcher close
INFO: Closing searc...@1243f75 main
From: mdigg...@gmail.com [mailto:mdigg...@gmail.com] On Behalf Of Mark Diggory
Sent: Wednesday, April 28, 2010 2:14 PM
To: Poulter, Dale
Cc: dspace-tech@lists.sourceforge.net; Francis, Ronee L
Subject: Re: [Dspace-tech] SOLR statistics in XMLUI
Dale,
No changes are necessary. The link to the new statistics pages shows as a
option on the menu and is resolvable via the following URI path relative to the
Item.
http://demo.dspace.org/xmlui/handle/123456789/29/statistics
Rendering has nothing to do with DIM-Handler in this case. You may check your
dspace.cfg setting to verify if your restricting it to administrator or not. We
are working on documentation here atm.
http://fedora-commons.org/confluence/display/DSTEST/DSpace+Statistics
Mark
On Wed, Apr 28, 2010 at 11:24 AM, Poulter, Dale
dale.poul...@vanderbilt.edumailto:dale.poul...@vanderbilt.edu wrote:
Good afternoon,
We have Dspace 1.6 working and the statistics show up as expected in the jspui.
However, the solr statistics do not appear in the xmlui. Are changes needed
in the DIM-Handler.xsl to get the statistics to appear? If so, are these
changes documented? Thanks for your assistance.
--Dale
---
Dale Poulter
Automation Coordinator
Library Information Technology Services
Vanderbilt University
Suite 700
110 21st Avenue South
Nashville, TN 37240
(615)343-5388
(615)343-8834 (fax)
(615)207-9705 (cell)
dale.poul...@vanderbilt.edumailto:dale.poul...@vanderbilt.edu
--
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.netmailto:DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
--
Mark R. Diggory
Head of U.S. Operations - @mire
http://www.atmire.com - Institutional Repository Solutions
http://www.togather.eu - Before getting together, get t...@ther
--
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Solr statistics - Tomcat access denied
Hi Manu, I'm testing 1.6 version and when I try to access to http:/my-dspace-url/solr I get this message from Tomcat: Access to specified resource is forbidden. The solr statistics are displayed with the jspui / xmlui rather than via the solr webapp. The solr webapp is just there to provide the data to the xmlui / jspui, and is not normally access directly. You should see a new 'View statistics' menu item in the XMLUI when viewing collections / communities / items, or on the similar pages in the JSPUI. The /solr webapp returns 'access denied' because we have restricted access to it only to 'localhost' so that only the DSpace application can make queries to it. This can be opened up if required so that you can perform report generation functions if you wished. See the web.xml settings and mentions of the localhost filter for this. Thanks, Stuart Lewis IT Innovations Analyst and Developer Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: +64 (0)9 373 7599 x81928 http://www.library.auckland.ac.nz/ -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Solr statistics - Tomcat access denied
Thank you very much for the replies. I will try them. Regards On Thu, Apr 15, 2010 at 9:48 AM, Stuart Lewis s.le...@auckland.ac.nzwrote: Hi Manu, I'm testing 1.6 version and when I try to access to http:/my-dspace-url/solr I get this message from Tomcat: Access to specified resource is forbidden. The solr statistics are displayed with the jspui / xmlui rather than via the solr webapp. The solr webapp is just there to provide the data to the xmlui / jspui, and is not normally access directly. You should see a new 'View statistics' menu item in the XMLUI when viewing collections / communities / items, or on the similar pages in the JSPUI. The /solr webapp returns 'access denied' because we have restricted access to it only to 'localhost' so that only the DSpace application can make queries to it. This can be opened up if required so that you can perform report generation functions if you wished. See the web.xml settings and mentions of the localhost filter for this. Thanks, Stuart Lewis IT Innovations Analyst and Developer Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: +64 (0)9 373 7599 x81928 http://www.library.auckland.ac.nz/ -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Solr statistics - Tomcat access denied
I have just brought up a 1.6 instance and have also set the solr server in dspace.cfg as: solr.log.server = http://localhost:8080/solr/statistics solr.statistics.query.filter.spiderIp = true solr.statistics.query.filter.isBot = true When I click at the 'View Statistics', I got no statistics data, which should be normal? or at least it should give me some zeros ? When investigate the dspace.log file, I got the error messages as attached. What can be wrong with my setting? Panyarak Ngamsritragul Prince of Songkla University. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. 2010-04-15 22:19:17,499 ERROR org.dspace.app.webui.servlet.DisplayStatisticsServlet @ Error occured while creating statistics for dso with ID: 29 and type 3 and handle: 2553/1178 org.apache.solr.client.solrj.SolrServerException: Error executing query at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:96) at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:109) at org.dspace.statistics.SolrLogger.query(SolrLogger.java:838) at org.dspace.statistics.SolrLogger.queryFacetField(SolrLogger.java:604) at org.dspace.statistics.content.StatisticsDataVisits.queryFacetField(StatisticsDataVisits.java:561) at org.dspace.statistics.content.StatisticsDataVisits.createDataset(StatisticsDataVisits.java:202) at org.dspace.statistics.content.StatisticsDisplay.getDataset(StatisticsDisplay.java:91) at org.dspace.app.webui.servlet.DisplayStatisticsServlet.displayStatistics(DisplayStatisticsServlet.java:333) at org.dspace.app.webui.servlet.DisplayStatisticsServlet.doDSGet(DisplayStatisticsServlet.java:123) at org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.java:151) at org.dspace.app.webui.servlet.DSpaceServlet.doGet(DSpaceServlet.java:99) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.dspace.utils.servlet.DSpaceWebappServletFilter.doFilter(DSpaceWebappServletFilter.java:112) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:619) Caused by: org.apache.solr.client.solrj.SolrServerException: java.net.SocketException: Connection reset at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:391) at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:183) at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90) ... 27 more Caused by: java.net.SocketException: Connection reset at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:96) at java.net.SocketOutputStream.write(SocketOutputStream.java:136) at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105) at org.apache.commons.httpclient.HttpConnection.write(HttpConnection.java:975) at org.apache.commons.httpclient.HttpConnection.write(HttpConnection.java:943) -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Solr statistics - Tomcat access denied
Hi,
I think I got the solution for this error. There are a few steps I have
not done:
1. [dspace]/reports directory was not created. Why this is not
automatically created when we deployed the system? Just wonder.
2. In dspace.cfg, (report.dir = ${dspace.dir}/reports)
there is an extra '/' at the end of 'reports'.
When I corrected these 2 points, the statistics works like a charm.
Panyarak Ngamsritragul
Prince of Songkla University.
On Thu, 15 Apr 2010, Panyarak Ngamsritragul wrote:
I have just brought up a 1.6 instance and have also set the solr server in
dspace.cfg as:
solr.log.server = http://localhost:8080/solr/statistics
solr.statistics.query.filter.spiderIp = true
solr.statistics.query.filter.isBot = true
When I click at the 'View Statistics', I got no statistics data, which should
be normal? or at least it should give me some zeros ?
When investigate the dspace.log file, I got the error messages as attached.
What can be wrong with my setting?
Panyarak Ngamsritragul
Prince of Songkla University.
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
--
Download Intel#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Solr statistics - Tomcat access denied
The Top City Views in the Solr Statistics is still not correct. It seems
that it is able to detect only Bangkok in Thailand while it is not true.
I remember that I raised this issue during the firest testathorn, and it
was solved. But why it is not working correctly again?
Panyarak
Prince of Songkla University
On Thu, 15 Apr 2010, Panyarak Ngamsritragul wrote:
Hi,
I think I got the solution for this error. There are a few steps I have
not done:
1. [dspace]/reports directory was not created. Why this is not
automatically created when we deployed the system? Just wonder.
2. In dspace.cfg, (report.dir = ${dspace.dir}/reports)
there is an extra '/' at the end of 'reports'.
When I corrected these 2 points, the statistics works like a charm.
Panyarak Ngamsritragul
Prince of Songkla University.
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
--
Download Intel#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Solr statistics - Tomcat access denied
Thanks a lot Mark. You are right. The 'solr.statistics.query.filter.spiderIp' must be false. And it works now. But the detection of City is not quite correct. Where can I fine tune it? Panyarak Ngamsritragul Prince of Songkla University. On Thu, 15 Apr 2010, Mark Diggory wrote: Verify when you are working locally on the server that you can successfully get to solr on the port. http://localhost:8080/solr On Thu, Apr 15, 2010 at 8:36 AM, Panyarak Ngamsritragul pa...@me.psu.ac.th wrote: I have just brought up a 1.6 instance and have also set the solr server in dspace.cfg as: solr.log.server = http://localhost:8080/solr/statistics solr.statistics.query.filter.spiderIp = true Please leave this the default (false), true will cause problems with statistics queries. # control solr statistics querying to filter out spider IPs # false by default # solr.statistics.query.filter.spiderIp = false When I click at the 'View Statistics', I got no statistics data, which should be normal? or at least it should give me some zeros ? When investigate the dspace.log file, I got the error messages as attached. What can be wrong with my setting? Panyarak Ngamsritragul Prince of Songkla University. Cheers, Mark -- Mark R. Diggory Head of U.S. Operations - @mire http://www.atmire.com - Institutional Repository Solutions http://www.togather.eu - Before getting together, get t...@ther -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Solr statistics - Tomcat access denied
Panyarak, The city detection is all performed by the geoLite database, which is downloaded to your [dspace] directory when you ant update. The work is does to give location info from an IP address. To have more accurate data, they offer a monthly updated paid version, GeoIP. http://www.maxmind.com/app/geolitecity update_geolite: [echo] Downloading: http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz [get] Getting: http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz [get] To: /dspace/config/GeoLiteCity.dat.gz In the code its used: location = geoipLookup.getLocation(ip); city = location.city; country = location.countryName; So, if you wanted to make it more accurate, you can consider their other geo database, or find an alternative that works better. Also, you could encourage everyone to add their info to the user-submission IP address sites, such as hostip.info as it looks like GeoIP gets its data from those kind of sites. We obtain the known IP location pairs from sites that ask the web visitor to provide their geographic location. The geographic location along with the web visitor's IP Address is forwarded to us. Also, I fixed up the stats-log-importer with a patch that can assist you in debugging to help you find not-good-enough entries, by cleaning up verbose output. http://jira.dspace.org/jira/browse/DS-542 However, I don't think there's any manual work you can do to clean up the imported entries. On Thu, Apr 15, 2010 at 12:42 PM, Panyarak Ngamsritragul pa...@me.psu.ac.th wrote: Thanks a lot Mark. You are right. The 'solr.statistics.query.filter.spiderIp' must be false. And it works now. But the detection of City is not quite correct. Where can I fine tune it? Panyarak Ngamsritragul Prince of Songkla University. On Thu, 15 Apr 2010, Mark Diggory wrote: Verify when you are working locally on the server that you can successfully get to solr on the port. http://localhost:8080/solr On Thu, Apr 15, 2010 at 8:36 AM, Panyarak Ngamsritragul pa...@me.psu.ac.th wrote: I have just brought up a 1.6 instance and have also set the solr server in dspace.cfg as: solr.log.server = http://localhost:8080/solr/statistics solr.statistics.query.filter.spiderIp = true Please leave this the default (false), true will cause problems with statistics queries. # control solr statistics querying to filter out spider IPs # false by default # solr.statistics.query.filter.spiderIp = false When I click at the 'View Statistics', I got no statistics data, which should be normal? or at least it should give me some zeros ? When investigate the dspace.log file, I got the error messages as attached. What can be wrong with my setting? Panyarak Ngamsritragul Prince of Songkla University. Cheers, Mark -- Mark R. Diggory Head of U.S. Operations - @mire http://www.atmire.com - Institutional Repository Solutions http://www.togather.eu - Before getting together, get t...@ther -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Peter Dietz Systems Developer/Engineer Ohio State University Libraries -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Solr statistics - Tomcat access denied
Thanks a lot Peter. The information you kindly shared is very helpful. I download the patch you mention and I suspect that the file is corrupted. There is a line saying '\ No newline at end of file' and the lines follow may not be part of the patch too. Panyarak Ngamsritragul Prince of Songkla University. On Thu, 15 Apr 2010, Peter Dietz wrote: Panyarak, The city detection is all performed by the geoLite database, which is downloaded to your [dspace] directory when you ant update. The work is does to give location info from an IP address. To have more accurate data, they offer a monthly updated paid version, GeoIP. http://www.maxmind.com/app/geolitecity update_geolite: [echo] Downloading: http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz [get] Getting: http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz [get] To: /dspace/config/GeoLiteCity.dat.gz In the code its used: location = geoipLookup.getLocation(ip); city = location.city; country = location.countryName; So, if you wanted to make it more accurate, you can consider their other geo database, or find an alternative that works better. Also, you could encourage everyone to add their info to the user-submission IP address sites, such as hostip.info as it looks like GeoIP gets its data from those kind of sites. We obtain the known IP location pairs from sites that ask the web visitor to provide their geographic location. The geographic location along with the web visitor's IP Address is forwarded to us. Also, I fixed up the stats-log-importer with a patch that can assist you in debugging to help you find not-good-enough entries, by cleaning up verbose output. http://jira.dspace.org/jira/browse/DS-542 However, I don't think there's any manual work you can do to clean up the imported entries. On Thu, Apr 15, 2010 at 12:42 PM, Panyarak Ngamsritragul pa...@me.psu.ac.th wrote: Thanks a lot Mark. You are right. The 'solr.statistics.query.filter.spiderIp' must be false. And it works now. But the detection of City is not quite correct. Where can I fine tune it? Panyarak Ngamsritragul Prince of Songkla University. On Thu, 15 Apr 2010, Mark Diggory wrote: Verify when you are working locally on the server that you can successfully get to solr on the port. http://localhost:8080/solr On Thu, Apr 15, 2010 at 8:36 AM, Panyarak Ngamsritragul pa...@me.psu.ac.th wrote: I have just brought up a 1.6 instance and have also set the solr server in dspace.cfg as: solr.log.server = http://localhost:8080/solr/statistics solr.statistics.query.filter.spiderIp = true Please leave this the default (false), true will cause problems with statistics queries. # control solr statistics querying to filter out spider IPs # false by default # solr.statistics.query.filter.spiderIp = false When I click at the 'View Statistics', I got no statistics data, which should be normal? or at least it should give me some zeros ? When investigate the dspace.log file, I got the error messages as attached. What can be wrong with my setting? Panyarak Ngamsritragul Prince of Songkla University. Cheers, Mark -- Mark R. Diggory Head of U.S. Operations - @mire http://www.atmire.com - Institutional Repository Solutions http://www.togather.eu - Before getting together, get t...@ther -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. --- --- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Peter Dietz Systems Developer/Engineer Ohio State University Libraries -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Solr statistics - Tomcat access denied
Manu, you might try the following instead... we may adjust to this in the
very near future.
Instead of
solr.log.server = ${dspace.baseUrl}/solr/statistics
use
solr.log.server = http://localhost:8080/solr/statistics
On Wed, Apr 14, 2010 at 2:47 AM, ManuMohedano manumohed...@usal.es wrote:
Hi everybody.
I'm testing 1.6 version and when I try to access to
http:/my-dspace-url/solr I get this message from Tomcat: Access to
specified resource is forbidden.
Any Ideas?.
Thank's and regards.
Manu
--
Download Intel#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
--
Mark R. Diggory
Head of U.S. Operations - @mire
http://www.atmire.com - Institutional Repository Solutions
http://www.togather.eu - Before getting together, get t...@ther
--
Download Intel#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
