Re: [Dspace-tech] tomcat reporting memory leak?
On 24 Sep 2010, at 21:17, bill.ander...@library.gatech.edu wrote: We've been experiencing problems similar to some reported on this thread since our upgrade to 1.6 several months ago. We're still using the jspui, and we've wondered (among other things) if some of these problems might be alleviated by a switch to the xmlui. Has anybody had any experience comparing the memory footprint and/or resource usage issues between the two interfaces? We load-tested the XMLUI (on identical hardware) and it was even worse. It ran out of memory and crashed really quickly, so we never took it into production. But your mileage may vary. Best regards, -- Tom De Mulder td...@cam.ac.uk - Cambridge University Computing Service +44 1223 3 31843 - New Museums Site, Pembroke Street, Cambridge CB2 3QH -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] tomcat reporting memory leak?
On 29 Sep 2010, at 11:38, Hilton Gibson wrote: We started with a VM which had 2GB memory. Then added 2GB to the VM, no luck. Then luckily we had funds to buy a server. So now we have 12GB RAM and 12CPU's. No crashes so far. Using the XMLUI. Does DSpace really need this and what happens when we go to one million items ?? A lot of the back-end code of DSpace, the very core of it, is inherently inefficient. Several tasks are executed more than once, and entire objects are created when only one attribute is needed, etc. (I'd be more specific, but I'm not a specialist on this matter, and our resident DSpace developer is on leave this week.) I am really glad to hear from other people with problems similar to ours. -- Tom De Mulder td...@cam.ac.uk - Cambridge University Computing Service +44 1223 3 31843 - New Museums Site, Pembroke Street, Cambridge CB2 3QH -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] tomcat reporting memory leak?
On 29 Sep 2010, at 11:47, Mark Ehle wrote: Why was tomcat chosen as a platform for DSpace? It wasn't. You can use any Servlet engine. We used JBoss for a while but went back to Tomcat because it fitted into our infrastructure better. I believe DSpace was written in Java because Rob Tansley wanted to try writing a project in Java, but I could be wrong. :) Best, -- Tom De Mulder td...@cam.ac.uk - Cambridge University Computing Service +44 1223 3 31843 - New Museums Site, Pembroke Street, Cambridge CB2 3QH -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] tomcat reporting memory leak?
On 29 September 2010 11:38, Hilton Gibson hilton.gib...@gmail.com wrote: Using the XMLUI. Does DSpace really need this and what happens when we go to one million items ?? Does DSpace really need that? No. As I have said, I'm running 30 separate repositories - using JSPUI (circa 1.4.2 / 1.5 codebase) - all on a single server / Tomcat instance. Some of those repositories have 1000s of items, and get quite decent levels of access. The server has 8GB installed, 3GB heap turned over to Tomcat (plus 1GB for non-heap). The Tomcat instance has 2GB of *free* heap space, rarely runs above 5% cpu usage, and has plenty of capacity to run more repositories (the rate at which files are opened/closed is actually a bigger issue for Tomcat startup). Although, it's worth pointing out that the database is hosted on a separate server - I can't say how many resources that is really using, as it's shared with other services, but it is apparently 'tiny'. What happens at one million items? Well, that's an interesting issue. But is it really the right question to be asking? How far do you want/need to be able to scale a 'monolithic' instance, before you spread it over multiple servers? As long as you can spread it over multiple servers, it gives you a much higher ceiling than relying on a single box - and it is easier to scale for increasing size/usage by adding more boxes (you don't have to migrate). If you focus on scaling a single installation, then you end up increasing the overall requirements (ie. memory for caching), and make it harder to have scaling over multiple boxes at all. G -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] tomcat reporting memory leak?
On 29 September 2010 11:48, Tom De Mulder td...@cam.ac.uk wrote: A lot of the back-end code of DSpace, the very core of it, is inherently inefficient I don't entirely disagree with that statement - there are some things that can definitely be improved, particularly where you have to deal with more items in a single instance. But take a look at my numbers - at it's core, it really isn't that bad for the vast majority of DSpace users (how many have more than even 50,000 items currently)? And some of it depends on correct system setup (Postgres version/options, etc.) It's adding xmlui, solr, etc. that is putting a lot more demands on the system. G -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] tomcat reporting memory leak?
That begs the question as do you think something else should be chosen / recommended? There really isn't anything preventing you using Jetty, etc. but Tomcat is actually a pretty solid server that does a lot of things quite well - and particularly in recent versions in being defensive against bad application behaviour. And when you look at the grand scheme of things, the smaller footprint of Jetty doesn't really make a whole lot of difference. G On 29 September 2010 11:47, Mark Ehle marke...@gmail.com wrote: Why was tomcat chosen as a platform for DSpace? -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] tomcat reporting memory leak?
We're comfortably running *three* production DSpace instances in a single Tomcat 6 with these limits: JAVA_OPTS=-Xmx1024M -Xms768M JAVA_OPTS=$JAVA_OPTS -XX:MaxPermSize=128M JAVA_OPTS=$JAVA_OPTS -XX:PermSize=32M That's on a box with 3GB of physical memory. One DSpace instance is 1.6, and the other two are 1.5. Now, I do have an old weekly reminder to check PermGen on that box, but it is always around half filled these days. We had problems in the past, but newer versions of DSpace seem to do much better in that regard. I can't recall the last time we had to restart that Tomcat just to clean up memory. We have a development box with maybe two dozen DSpace instances, none of them very busy at all, various versions and states of disrepair, and we do have to restart Tomcat there from time to time if we are doing a lot of webapp. reloading. The limits there are: JAVA_OPTS=-Xmx1024M -Xms128M JAVA_OPTS=$JAVA_OPTS -XX:PermSize=192M JAVA_OPTS=$JAVA_OPTS -XX:MaxPermSize=384M on a 4GB machine. -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Balance your desire for bells and whistles with the reality that only a little more than 2 percent of world population has broadband. -- Ledford and Tyler, _Google Analytics 2.0_ pgpDu2gzdSXuS.pgp Description: PGP signature -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] tomcat reporting memory leak?
On Wed, Sep 29, 2010 at 11:48:02AM +0100, Tom De Mulder wrote: A lot of the back-end code of DSpace, the very core of it, is inherently inefficient. Several tasks are executed more than once, and entire objects are created when only one attribute is needed, etc. (I'd be more specific, but I'm not a specialist on this matter, and our resident DSpace developer is on leave this week.) When your developer has time, I think that specific JIRA tickets on these observations would be appreciated. We need all the eyes we can borrow. It needn't be a rigorous analysis (though that would be wonderful). Significant inefficiencies noted in passing are important information. -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Balance your desire for bells and whistles with the reality that only a little more than 2 percent of world population has broadband. -- Ledford and Tyler, _Google Analytics 2.0_ pgp5elq6QWOXZ.pgp Description: PGP signature -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] tomcat reporting memory leak?
On 29 Sep 2010, at 13:03, Graham Triggs wrote: Some of those repositories have 1000s of items, and get quite decent levels of access. Thousands? I don't even want to have this discussion until you're talking hundreds of thousands, and how many hits per second. I know you like to talk down the problem, but that really isn't helping. We run 5 DSpace instances, three of these are systems with hundreds of thousands of items, and it's dog slow and immensely resource-intensive. And yes, we want these to be single systems. Why shouldn't we? We have other systems here at the University that are much bigger, do similar things and require far, far less in terms of resources. -- Tom De Mulder td...@cam.ac.uk - Cambridge University Computing Service +44 1223 3 31843 - New Museums Site, Pembroke Street, Cambridge CB2 3QH -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] load_registries fails
Shouldn't be. Everything was installed the same as the earlier
version was.
But, if it could be, which files?
Marvin
On 9/29/2010 2:12 AM, Hilton Gibson wrote:
File permissions perhaps ??
On 28/09/2010 20:31, Marvin Weaver wrote:
It is set for the default: log.init.config =
${dspace.dir}/config/log4j.properties
I've made no changes in log4j.properties.
I had no trouble with 1.5.2 building on ubuntu8.04LTS/postgre8.3with
the default settings. (after build, the server kept crashing due to
segmentation faults which is why I moved to Solaris/Oracle)
Marvin
On 9/28/2010 1:30 PM, Hilton Gibson wrote:
Do you have a special setting for
log.init.config
Cheers
hg
I tried to build 1.6.2 on Solaris 10/Oracle 10 and it consistantly
failed on load_registries. So I tried building 1.6.2 on
Ubuntu10.04LTS/postgres8.4 to see what was not being loaded into the
tables. And I got exactly the same error. Would somebody tell me what
is wrong? and how to fix it? (I searched the archives and didn't find
anything other than another person with a similar problem and no answers).
load_registries:
[java] 2010-09-28 12:41:24,586 INFO
org.dspace.core.ConfigurationManager @ Loading system provided config
property (-Ddspace.configuration): config/dspace.cfg
[java] 2010-09-28 12:41:24,599 INFO
org.dspace.core.ConfigurationManager @ Using default log4j provided
log configuration,if uninitended, check your dspace.cfg for
(log.init.config)
[java] 2010-09-28 12:41:24,993 FATAL
org.dspace.administer.RegistryLoader @
anonymous::error_loading_registries:
[java] java.lang.NullPointerException
[java] at
org.dspace.storage.rdbms.TableRow.canonicalize(TableRow.java:678)
[java] at
org.dspace.storage.rdbms.TableRow.hasColumn(TableRow.java:110)
[java] at
org.dspace.storage.rdbms.TableRow.setColumn(TableRow.java:507)
[java] at
org.dspace.storage.rdbms.DatabaseManager.insert(DatabaseManager.java:713)
[java] at
org.dspace.storage.rdbms.DatabaseManager.create(DatabaseManager.java:490)
[java] at
org.dspace.content.BitstreamFormat.create(BitstreamFormat.java:419)
[java] at
org.dspace.administer.RegistryLoader.loadFormat(RegistryLoader.java:202)
[java] at
org.dspace.administer.RegistryLoader.loadBitstreamFormats(RegistryLoader.java:168)
[java] at
org.dspace.administer.RegistryLoader.main(RegistryLoader.java:105)
[java] Error:
[java] - null
BUILD FAILED
/home/dspace/dspace-1.6.2-src-release/dspace/target/dspace-1.6.2-build.dir/build.xml:663:
Java returned: 1
Marvin Weaver
--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
[Dspace-tech] PowerPoint Text Extractor
Hello, Are there plans to add a PPT text extractor to DSpace? In the meantime, can some provide information on how to implement one? Thanks, Richard Jizba Creighton University. -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] tomcat reporting memory leak?
Hi all, Interesting thread so far and keep up the good discussion. I think it'd be helpful to us all if we could all share more information about our DSpace setups (similar to Mark Wood's tip on his local JAVA_OPTS settings). The more we know about your DSpace/Java/Tomcat/Postgres (or Oracle) configurations, server setups, etc. the better chance we have at helping you out. There may be some immediate performance improvements you can achieve just by tweaking your setup/configurations slightly. I had setup a basic template for this on the Wiki at https://wiki.duraspace.org/display/DSPACE/ScalabilityIssues1.6 But, feel free to just send info along in any format you wish. The template was mostly there to give everyone an idea of what type of information can be useful to us (so that we can hopefully provide you with some helpful suggestions and find longer term fixes). Obviously, we also want to track down and fix any memory leaks or larger problems as well. So if you've already discovered specific issues, let us know about those as well, so we can add them to our Issue Tracker (http://jira.dspace.org/) and schedule them to be resolved. Thanks, Tim Donohue Technical Lead for DSpace Project DuraSpace.org On 9/29/2010 7:59 AM, Mark H. Wood wrote: We're comfortably running *three* production DSpace instances in a single Tomcat 6 with these limits: JAVA_OPTS=-Xmx1024M -Xms768M JAVA_OPTS=$JAVA_OPTS -XX:MaxPermSize=128M JAVA_OPTS=$JAVA_OPTS -XX:PermSize=32M That's on a box with 3GB of physical memory. One DSpace instance is 1.6, and the other two are 1.5. Now, I do have an old weekly reminder to check PermGen on that box, but it is always around half filled these days. We had problems in the past, but newer versions of DSpace seem to do much better in that regard. I can't recall the last time we had to restart that Tomcat just to clean up memory. We have a development box with maybe two dozen DSpace instances, none of them very busy at all, various versions and states of disrepair, and we do have to restart Tomcat there from time to time if we are doing a lot of webapp. reloading. The limits there are: JAVA_OPTS=-Xmx1024M -Xms128M JAVA_OPTS=$JAVA_OPTS -XX:PermSize=192M JAVA_OPTS=$JAVA_OPTS -XX:MaxPermSize=384M on a 4GB machine. -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] PowerPoint Text Extractor
Hi RIchard: I cannot speak to it's quality (and indeed we have had quality issues with other formats) but Apache POI library supports Powerpoint text extraction. I would study the doc at http://poi.apache.org/ for how to use the library, and look in: http://scm.dspace.org/svn/repo/dspace/trunk/dspace-api/src/main/java/org/dspace/app/mediafilter for examples of other extractor media filters. Then post any questions to the tech or dev list. Hope that is helpful, Richard Rodgers On Sep 29, 2010, at 10:14 AM, Jizba, Richard wrote: Hello, Are there plans to add a PPT text extractor to DSpace? In the meantime, can some provide information on how to implement one? Thanks, Richard Jizba Creighton University. ATT1..cATT2..c -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] tomcat reporting memory leak?
Thanks - I was just curious. On Wed, Sep 29, 2010 at 6:53 AM, Tom De Mulder td...@cam.ac.uk wrote: On 29 Sep 2010, at 11:47, Mark Ehle wrote: Why was tomcat chosen as a platform for DSpace? It wasn't. You can use any Servlet engine. We used JBoss for a while but went back to Tomcat because it fitted into our infrastructure better. I believe DSpace was written in Java because Rob Tansley wanted to try writing a project in Java, but I could be wrong. :) Best, -- Tom De Mulder td...@cam.ac.uk - Cambridge University Computing Service +44 1223 3 31843 - New Museums Site, Pembroke Street, Cambridge CB2 3QH -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
[Dspace-tech] FW: Langley Digital Repository (LDR-S) Internal Server Error
I tried to delete a collection in DSpace 1.5.1 online yesterday and got this error. I was able to finally delete it only after I went in and manually deleted all the rows in bi_7_dmap, but I'm not quite sure why I had to do this or if it's a bug...?? Does anyone have any ideas? Thanks, Sue -Original Message- From: LDR-S - Langley Digital Repository (1.5.1) [mailto:digital-reposit...@larc.nasa.gov] Sent: Tuesday, September 28, 2010 4:13 PM To: Thornton, Susan M. (LARC-B702)[Raytheon Information Services (RIS)] Subject: Langley Digital Repository (LDR-S) Internal Server Error An internal server error occurred on https://x.larc.nasa.gov: Date..: 9/28/10 4:13 PM Session ID: D884BF71A03E4AA0C04ABFC55F5A366B -- URL Was: https://x.larc.nasa.gov/tools/edit-communities -- Method: POST -- Parameters were: -- collection_id: 7 -- community_id: 4 -- action: 10 -- submit: Delete *** * Exception.: * *** org.postgresql.util.PSQLException: ERROR: update or delete on table item violates foreign key constraint bi_7_dmap_item_id_fkey on table bi_7_dmap Detail: Key (item_id)=(28) is still referenced from table bi_7_dmap. at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:1531) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1313) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:188) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:452) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:354) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:308) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:102) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:102) at org.dspace.storage.rdbms.DatabaseManager.updateQuery(DatabaseManager.java:390) at org.dspace.storage.rdbms.DatabaseManager.deleteByValue(DatabaseManager.java:535) at org.dspace.storage.rdbms.DatabaseManager.delete(DatabaseManager.java:502) at org.dspace.storage.rdbms.DatabaseManager.delete(DatabaseManager.java:737) at org.dspace.content.Item.delete(Item.java:1904) at org.dspace.content.Collection.removeItem(Collection.java:950) at org.dspace.content.Collection.delete(Collection.java:1076) at org.dspace.content.Community.removeCollection(Community.java:1073) at org.dspace.app.webui.servlet.admin.EditCommunitiesServlet.doDSPost(EditCommunitiesServlet.java:275) at org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.java:152) at org.dspace.app.webui.servlet.DSpaceServlet.doPost(DSpaceServlet.java:110) at javax.servlet.http.HttpServlet.service(HttpServlet.java:816) at javax.servlet.http.HttpServlet.service(HttpServlet.java:917) at org.apache.catalina.core.ApplicationFilterChain.servletService(ApplicationFilterChain.java:398) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:185) at org.dspace.app.webui.filter.RegisteredOnlyFilter.doFilter(RegisteredOnlyFilter.java:98) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:217) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:185) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:255) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:188) at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:586) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:556) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:187) at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:586) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:556) at com.sun.webserver.connector.nsapi.NSAPIProcessor.service(NSAPIProcessor.java:160) **Note..: This error occurs due to one or more Java errors/exceptions in LDR-S and is a catch-all for non-specific application errors. Stack traces should be analyzed to determine likely causes. In addition, DSpace, PostgreSQL, and server logs can be examined for specific error mesasges that may help with error diagnosis and resolution. LDR-S Administration
Re: [Dspace-tech] tomcat reporting memory leak?
Quick followup, in case it isn't clear (as I was asked about this off-list). The preference would be to share your DSpace setup/configuration information directly on this listserv (or you can post up on the wiki if you prefer). That way we can get more eyes on it, and hopefully come up with better suggestions. Also, this may be an area where sharing this information can help us to document some best practices, based on recommended setups and performance hints/tips that people have. So, I'm hoping that as this thread continues, we can pull out the main tips/hints and document them for future reference. At the same time, we can pull out the common memory/performance issues so that they can be investigated further, and hopefully resolved as soon as possible. Committers -- it'd also be great if you can take a few moments to send your basic setup info Dspace size to the listserv (especially noting anything that you may have tweaked above beyond the normal DSpace install docs, like JAVA_OPTS or similar settings). This can hopefully encourage others to do the same. - Tim On 9/29/2010 9:46 AM, Tim Donohue wrote: Hi all, Interesting thread so far and keep up the good discussion. I think it'd be helpful to us all if we could all share more information about our DSpace setups (similar to Mark Wood's tip on his local JAVA_OPTS settings). The more we know about your DSpace/Java/Tomcat/Postgres (or Oracle) configurations, server setups, etc. the better chance we have at helping you out. There may be some immediate performance improvements you can achieve just by tweaking your setup/configurations slightly. I had setup a basic template for this on the Wiki at https://wiki.duraspace.org/display/DSPACE/ScalabilityIssues1.6 But, feel free to just send info along in any format you wish. The template was mostly there to give everyone an idea of what type of information can be useful to us (so that we can hopefully provide you with some helpful suggestions and find longer term fixes). Obviously, we also want to track down and fix any memory leaks or larger problems as well. So if you've already discovered specific issues, let us know about those as well, so we can add them to our Issue Tracker (http://jira.dspace.org/) and schedule them to be resolved. Thanks, Tim Donohue Technical Lead for DSpace Project DuraSpace.org On 9/29/2010 7:59 AM, Mark H. Wood wrote: We're comfortably running *three* production DSpace instances in a single Tomcat 6 with these limits: JAVA_OPTS=-Xmx1024M -Xms768M JAVA_OPTS=$JAVA_OPTS -XX:MaxPermSize=128M JAVA_OPTS=$JAVA_OPTS -XX:PermSize=32M That's on a box with 3GB of physical memory. One DSpace instance is 1.6, and the other two are 1.5. Now, I do have an old weekly reminder to check PermGen on that box, but it is always around half filled these days. We had problems in the past, but newer versions of DSpace seem to do much better in that regard. I can't recall the last time we had to restart that Tomcat just to clean up memory. We have a development box with maybe two dozen DSpace instances, none of them very busy at all, various versions and states of disrepair, and we do have to restart Tomcat there from time to time if we are doing a lot of webapp. reloading. The limits there are: JAVA_OPTS=-Xmx1024M -Xms128M JAVA_OPTS=$JAVA_OPTS -XX:PermSize=192M JAVA_OPTS=$JAVA_OPTS -XX:MaxPermSize=384M on a 4GB machine. -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] PowerPoint Text Extractor
Somewhat related is the Open Office Conversion filters built by Ideals. https://services.ideals.illinois.edu/wiki/bin/view/IDEALS/Internal/OpenOfficeConvert -- Peter Dietz Systems Developer/Engineer Ohio State University Libraries On Wed, Sep 29, 2010 at 11:03 AM, Richard Rodgers rrodg...@mit.edu wrote: Hi RIchard: I cannot speak to it's quality (and indeed we have had quality issues with other formats) but Apache POI library supports Powerpoint text extraction. I would study the doc at http://poi.apache.org/ for how to use the library, and look in: http://scm.dspace.org/svn/repo/dspace/trunk/dspace-api/src/main/java/org/dspace/app/mediafilter for examples of other extractor media filters. Then post any questions to the tech or dev list. Hope that is helpful, Richard Rodgers On Sep 29, 2010, at 10:14 AM, Jizba, Richard wrote: Hello, Are there plans to add a PPT text extractor to DSpace? In the meantime, can some provide information on how to implement one? Thanks, Richard Jizba Creighton University. ATT1..cATT2..c -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] tomcat reporting memory leak?
- Tim Donohue tdono...@duraspace.org wrote: | Quick followup, in case it isn't clear (as I was asked about this | off-list). The preference would be to share your DSpace | setup/configuration information directly on this listserv Let me kick things off, then (questions truncated a bit for formatting reasons): 1) Contact Info a) Bill Anderson / Georgia Institute of Technology / bill.ander...@library.gatech.edu 2) DSpace Setup and Configuration details a) What DSpace version are you using? 1. Dspace 1.6.2 2. Currently using JSPUI, migrating to XMLUI 3. 30,498 Items 4. 610 Communities/Collections b) What Postgres/Oracle version are you using? 1. PostgreSQL 8.1.4 c) What Tomcat version are you using? 1. Tomcat/6.0.26 + mod_jk/1.2.30 + Apache/2.0.52 d) Is everything running on one server (DSpace/Tomcat/Posgres/etc)? 1. Everything is (currently) on the same server 2. PowerEdge 2850: 2x Intel Xeon CPU 2.80Ghz, 12Gb Memory, Red Hat AS 4 (Nahant Update 8), RAID5 Disk array e) How much memory are you making available to Tomcat/Java? 1. (lb worker) JAVA_OPTS=-server -Xmx462M -Xms462M -XX:+UseParallelGC -Dfile.encoding=UTF-8, webapps: jspui lni oai sword xmlui 2. (lb worker) JAVA_OPTS=-server -Xmx462M -Xms462M -XX:+UseParallelGC -Dfile.encoding=UTF-8, webapps: jspui lni oai sword xmlui 3. JAVA_OPTS=-server -Xmx600M -Xms600M -XX:+UseParallelGC -Dfile.encoding=UTF-8, webapps: solr 4. lb worker method=request, socket_keepalive=True, socket_timeout=0, ping_mode=A 5. Postgres max_connections=300 3) Performance / Scalability Issues noticed 1. We've had intermittent performance problems since upgrading to 1.6 in May. At first, the problems seemed strictly SOLR-related; SOLR was grabbing hundreds of postgres connections, and eventually generating these in dspace.log: org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error: Timeout waiting for idle object and these in catalina.out: SEVERE: org.apache.solr.common.SolrException: Error opening new searcher. exceeded limit of maxWarmingSearchers=2, try again later. ...followed by permgen errors and death. 2. We heavily revised our solrconfig.xml, and alleviated the problem, but didn't eliminate it. We also split our jspui between two load-balanced tomcat instance, and moved the SOLR webapp to another third instance, which also helped. Following OR 2010, on a suggesting from Peter Dietz, we revised the SOLR JSP code to use the auto-commit functionality rather than manually committing every transaction. All of this got us to the point where we weren't crashing routinely; but we still have major problems during times of heavy traffic. Generally, these take the form of a gradual slowdown followed by a complete failure to respond; this sometimes ends in spontaneous recovery, and sometimes in permgen errors and a crash. At the end of last week, following a bad patch caused by a LOCKSS harvest, we implemented a restart schedule, with our two jspui tomcat instances being automatically restarted every 6 hours alternating between one/two. We haven't had any crashes since; but we're not at all sure we've solved the problem. 3. On restart, we sometimes get a bunch of these: Sep 28, 2010 9:00:06 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: A web application appears to have started a thread named [FinalizableReferenceQueue] but has failed to stop it. This is very likely to create a memory leak 4. Other errors that lead to a service/application outage: Sep 23, 2010 3:47:14 PM org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run SEVERE: Caught exception (java.lang.OutOfMemoryError: PermGen space) executing org.apache.jk.common.channelsocket$socketconnect...@3aff776, terminating thread Sep 23, 2010 10:37:04 AM org.apache.catalina.connector.CoyoteAdapter service SEVERE: An exception or error occurred in the container during the request processing java.lang.OutOfMemoryError: PermGen space at java.lang.Throwable.getStackTraceElement(Native Method) at java.lang.Throwable.getOurStackTrace(Throwable.java:591) at java.lang.Throwable.getStackTrace(Throwable.java:582) at org.apache.juli.logging.DirectJDKLog.log(DirectJDKLog.java:155) at org.apache.juli.logging.DirectJDKLog.error(DirectJDKLog.java:135) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:274) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at
[Dspace-tech] DC metadata fields to show in the full item view - dspace 1.6.2.
Hello, Anybody knows how to customise the DC metadata fields to show in the full item view in dspace 1.6.2. In previous versions I could edit the file item.jsp I appreciate your help. -- Regards, Víctor Izaguirre Castro -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
[Dspace-tech] DSpace Setup
Hi... Based on Tim Donohue's suggestion to share configuration and setup information, here is Cornell University's Dspace configuration: 1. Server: Sun sun4v T5140 Memory size: 65312 Megabytes (4 CPUs) 2. Running DSpace 1.6.2 (JSPUI) with local mods for User Interface, Embargo, and Refworks. db.maxconnections = 50 db.maxwait = 5000 db.maxidle = 5 2 assetstores at 300GB each (using currently 323 GB) Number of items: 14,960 Number of Communities/Collections: 789 3. Java 1.5.0_24 4. Apache 2.29 running mod_jk to tomcat 5. Tomcat 5.5.26 JAVA_OPTS=-server -Xms1024m -Xmx2048m -Xmn64m -Dfile.encoding=UTF-8 -XX:+UseParallelGC -verbose:gc -Xloggc:/dspace/dspace/log/gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:PermSize=1024m -XX:MaxPermSize=1024m -XX:-UseGCOverheadLimit 6. PostGreSQL 8.3 max_connections = 300 shared_buffers = 32MB max_fsm_pages = 204800 Though we experienced some performance problems in the past, that seemed to disappear after we went to DSpace 1.5.2. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 501 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] need help fixing error
This error indicates you have a problem with how a field is defined in DSpace, either in your metadatafieldregisty table or in your forms definition in input-forms.xml. Look at your metadatafieldregistry at fields where the element = type. Another thing you might look at is your input-forms.xml file to see how you have type defined. Is it a required entry? I'm not exactly sure what's happening, but looking at these two things might give you an idea. Hope this helps. Sue -Original Message- From: Azim Jinha [mailto:a...@kin.ucalgary.ca] Sent: Monday, September 27, 2010 6:15 PM To: dspace-tech@lists.sourceforge.net Subject: [Dspace-tech] need help fixing error Hi list, I am running dspace version 1.5.2 on a OpenBSD box with a postgresql backend database. After upgrading to 1.5.2, creating new submissions submitting to some collection fails with an internal error. After checking dspace/log/dspace.log I find the following sql exception lines 2010-09-27 15:54:22,683 WARN org.dspace.content.Item @ a...@kin.ucalgary.ca:session_id=3A2E44FA6FE2F6BB7CE2178E6CE53775:ip_addr =xx.xx.xx.xx:bad_dc:Bad DC field. schema=dc, element: type qualifier: null value: Book 2010-09-27 15:54:22,684 ERROR org.dspace.app.webui.servlet.SubmissionController @ Error loading step class'org.dspace.submit.step.DescribeStep':java.sql.SQLException: bad_dublin_core schema=dc, type null Any help on how to fix this would be greatly appreciated Thanks Azim -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] tomcat reporting memory leak?
I'd like to point out that the discussion is broadening considerably: a system can be slow for many reasons, not just memory starvation. Step 1: what resource(s) are you short of? Something like LambdaProbe can peek inside Tomcat and show you how much of each of the various memory pools is being used. OS tools can show whether you are swapping heavily or spending a lot of time in I/O wait or are really CPU-bound (and what, besides Tomcat, may be eating CPU). DBMS tools can reveal places in the schema that don't scale well, queries that could be optimized, and additional indices that would be beneficial. It would be really helpful for large, busy sites with performance problems to share any such detailed observations. Some of those problems can probably be tuned away, and some will point to specific things for coders to investigate. Scaling experience will be valuable both in documenting good ways to tune up for DSpace and in finding design hotspots for rework. -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Balance your desire for bells and whistles with the reality that only a little more than 2 percent of world population has broadband. -- Ledford and Tyler, _Google Analytics 2.0_ pgppBjWncgb5p.pgp Description: PGP signature -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] need help fixing error
Hi Sue Thanks for your reply to my query. On Wednesday, September 29, 2010 Thornton, Susan M. wrote: This error indicates you have a problem with how a field is defined in DSpace, either in your metadatafieldregisty table or in your forms definition in input- forms.xml. Look at your metadatafieldregistry at fields where the element = type. Another thing you might look at is your input-forms.xml file to see how you have type defined. Is it a required entry? I'm not exactly sure what's happening, but looking at these two things might give you an idea. Based on your suggestion, I took a second look at the meta data field registry and input-forms.xml. Looking at input-forms.xml, I edited this file by removing the field for the dc-element type. This edit fixed the problem and the input forms worked again. Next I looked at the Meta Data Field Registry (under the administration section). I found for some reason that there was no _type_ element defined in the registry. So, the actual problem was that at some point the dc-element _type_ was removed from my meta data field registry. After adding it back removed the error and fixed my problem. Thanks for your suggestions. Azim Hope this helps. Sue -Original Message- From: Azim Jinha [mailto:a...@kin.ucalgary.ca] Sent: Monday, September 27, 2010 6:15 PM To: dspace-tech@lists.sourceforge.net Subject: [Dspace-tech] need help fixing error Hi list, I am running dspace version 1.5.2 on a OpenBSD box with a postgresql backend database. After upgrading to 1.5.2, creating new submissions submitting to some collection fails with an internal error. After checking dspace/log/dspace.log I find the following sql exception lines 2010-09-27 15:54:22,683 WARN org.dspace.content.Item @ a...@kin.ucalgary.ca:session_id=3A2E44FA6FE2F6BB7CE2178E6CE53775:ip_a ddr =xx.xx.xx.xx:bad_dc:Bad DC field. schema=dc, element: type qualifier: null value: Book 2010-09-27 15:54:22,684 ERROR org.dspace.app.webui.servlet.SubmissionController @ Error loading step class'org.dspace.submit.step.DescribeStep':java.sql.SQLException: bad_dublin_core schema=dc, type null Any help on how to fix this would be greatly appreciated Thanks Azim -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] need help fixing error
Great! I'm glad to help!! Sue -Original Message- From: Azim Jinha [mailto:a...@kin.ucalgary.ca] Sent: Wednesday, September 29, 2010 1:40 PM To: Thornton, Susan M. (LARC-B702)[Raytheon Information Services (RIS)]; dspace-tech@lists.sourceforge.net Subject: RE: need help fixing error Hi Sue Thanks for your reply to my query. On Wednesday, September 29, 2010 Thornton, Susan M. wrote: This error indicates you have a problem with how a field is defined in DSpace, either in your metadatafieldregisty table or in your forms definition in input- forms.xml. Look at your metadatafieldregistry at fields where the element = type. Another thing you might look at is your input-forms.xml file to see how you have type defined. Is it a required entry? I'm not exactly sure what's happening, but looking at these two things might give you an idea. Based on your suggestion, I took a second look at the meta data field registry and input-forms.xml. Looking at input-forms.xml, I edited this file by removing the field for the dc-element type. This edit fixed the problem and the input forms worked again. Next I looked at the Meta Data Field Registry (under the administration section). I found for some reason that there was no _type_ element defined in the registry. So, the actual problem was that at some point the dc-element _type_ was removed from my meta data field registry. After adding it back removed the error and fixed my problem. Thanks for your suggestions. Azim Hope this helps. Sue -Original Message- From: Azim Jinha [mailto:a...@kin.ucalgary.ca] Sent: Monday, September 27, 2010 6:15 PM To: dspace-tech@lists.sourceforge.net Subject: [Dspace-tech] need help fixing error Hi list, I am running dspace version 1.5.2 on a OpenBSD box with a postgresql backend database. After upgrading to 1.5.2, creating new submissions submitting to some collection fails with an internal error. After checking dspace/log/dspace.log I find the following sql exception lines 2010-09-27 15:54:22,683 WARN org.dspace.content.Item @ a...@kin.ucalgary.ca:session_id=3A2E44FA6FE2F6BB7CE2178E6CE53775:ip_a ddr =xx.xx.xx.xx:bad_dc:Bad DC field. schema=dc, element: type qualifier: null value: Book 2010-09-27 15:54:22,684 ERROR org.dspace.app.webui.servlet.SubmissionController @ Error loading step class'org.dspace.submit.step.DescribeStep':java.sql.SQLException: bad_dublin_core schema=dc, type null Any help on how to fix this would be greatly appreciated Thanks Azim -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
[Dspace-tech] dsrun - batch import
hi everyone, I'm trying to import data to dspace 1.6 using dsrun application. In dspace 1.4 I don't have problem with this function, but in dspace 1.6 appear an error when I execute the follow sentence: ./dsrun org.dspace.app.itemimport.ItemImport -a -e dsp...@alerta.cl -c 1 -s /data/carga/root -m ejemplo_8 Destination collections: Owning Collection: Prueba Carga de colecciones Adding items from directory: /data/carga/root Generating mapfile: ejemplo_8 Adding item from directory 000541595 Loading dublin core from /data/carga/root/000541595/dublin_core.xml Schema: dc Element: format Qualifier: medium Value: BK Schema: dc Element: identifier Qualifier: code Value: 000541595 Schema: dc Element: language Qualifier: none Value: spa Schema: dc Element: contributor Qualifier: author Value: BastÃas, Gabriel Schema: dc Element: title Qualifier: none Value: GarantÃas explÃcitas en salud (GES) : seis consideraciones para su implementación Schema: dc Element: publisher Qualifier: none Value: Pontificia Universidad Católica de Chile, VicerrectorÃa de Comunicaciones y Asuntos Públicos Schema: dc Element: date Qualifier: created Value: 2007 Schema: dc Element: description Qualifier: none Value: Temas de la agenda pública ; Año 2, no. 9 Schema: dc Element: subject Qualifier: other Value: Reforma en la atención de salud - Chile Schema: dc Element: subject Qualifier: other Value: PolÃtica de salud - Chile Schema: dc Element: contributor Qualifier: author Value: Andia, Marcelo. Schema: dc Element: archivos Qualifier: adjuntos Value: 12.pdf Processing contents file: /data/carga/root/000541595/contents Bitstream: 12.pdf Processing handle file: handle It appears there is no handle file -- generating one Exception in thread main java.lang.NoSuchMethodError: method java.lang.String.format with signature (Ljava.lang.String;[Ljava.lang.Object;)Ljava.lang.String; was not found. at org.dspace.content.DCDate.toStringInternal(DCDate.java:284) at org.dspace.content.DCDate.toString(DCDate.java:272) at org.dspace.content.InstallItem.installItem(InstallItem.java:112) at org.dspace.app.itemimport.ItemImport.addItem(ItemImport.java:774) at org.dspace.app.itemimport.ItemImport.addItems(ItemImport.java:608) at org.dspace.app.itemimport.ItemImport.main(ItemImport.java:500) Could someone help me? Thank for your time Miguel Pizarro Soporte Tecnológico Alerta al Conocimiento Fono 56 (2) 233 79 08 e-mail: sopo...@alerta.cl Providencia n° 2.040 of. C. CP: 7510117 -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
[Dspace-tech] Limiting Searches
I was recently asked to add type to the fields that can be searched in the Advance Search. I did do that. However, the problem is that users don't really know all of the types, so I was asked to add a drop down for all types. Along with that I was asked if I could modify the Advance Search so that people can limit their searches by Date range or by type. Before I spend a lot of time figuring this out, I was wondering if anyone has done anything similar to this that I could copy (or steal!). Thanks! George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 501 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] dsrun - batch import
Hello - Please check which version of Java you are using. DSpace 1.6 requires the Sun Java JDK 5 or later. --keith On Sep 29, 2010, at 1:58 PM, Miguel Pizarro wrote: hi everyone, I'm trying to import data to dspace 1.6 using dsrun application. In dspace 1.4 I don't have problem with this function, but in dspace 1.6 appear an error when I execute the follow sentence: ./dsrun org.dspace.app.itemimport.ItemImport -a -e dsp...@alerta.cl -c 1 -s /data/carga/root -m ejemplo_8 Destination collections: Owning Collection: Prueba Carga de colecciones Adding items from directory: /data/carga/root Generating mapfile: ejemplo_8 Adding item from directory 000541595 Loading dublin core from /data/carga/root/000541595/dublin_core.xml Schema: dc Element: format Qualifier: medium Value: BK Schema: dc Element: identifier Qualifier: code Value: 000541595 Schema: dc Element: language Qualifier: none Value: spa Schema: dc Element: contributor Qualifier: author Value: Bastà as, Gabriel Schema: dc Element: title Qualifier: none Value: Garantà as explà citas en salud (GES) : seis consideraciones para su implementación Schema: dc Element: publisher Qualifier: none Value: Pontificia Universidad Católica de Chile, Vicerrectorà a de Comunicaciones y Asuntos Públicos Schema: dc Element: date Qualifier: created Value: 2007 Schema: dc Element: description Qualifier: none Value: Temas de la agenda pública ; Año 2, no. 9 Schema: dc Element: subject Qualifier: other Value: Reforma en la atención de salud - Chile Schema: dc Element: subject Qualifier: other Value: Polà tica de salud - Chile Schema: dc Element: contributor Qualifier: author Value: Andia, Marcelo. Schema: dc Element: archivos Qualifier: adjuntos Value: 12.pdf Processing contents file: /data/carga/root/000541595/contents Bitstream: 12.pdf Processing handle file: handle It appears there is no handle file -- generating one Exception in thread main java.lang.NoSuchMethodError: method java.lang.String.format with signature (Ljava.lang.String;[Ljava.lang.Object;)Ljava.lang.String; was not found. at org.dspace.content.DCDate.toStringInternal(DCDate.java:284) at org.dspace.content.DCDate.toString(DCDate.java:272) at org.dspace.content.InstallItem.installItem(InstallItem.java:112) at org.dspace.app.itemimport.ItemImport.addItem(ItemImport.java:774) at org.dspace.app.itemimport.ItemImport.addItems(ItemImport.java:608) at org.dspace.app.itemimport.ItemImport.main(ItemImport.java:500) Could someone help me? Thank for your time… Miguel Pizarro Soporte Tecnológico Alerta al Conocimiento Fono 56 (2) 233 79 08 e-mail: sopo...@alerta.cl Providencia n° 2.040 of. C. CP: 7510117 -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] dsrun - batch import
Hello, In the Dspace 1.6, You now have to use 'dspace' batch and not 'dsrun'. Cheers Philippe Thuaud Intern IT Technical Development T +64 7 838 5223 E philippe.thu...@agresearch.co.nzmailto:philippe.thu...@agresearch.co.nz From: Keith Gilbertson [mailto:keith.gilbert...@library.gatech.edu] Sent: Thursday, 30 September 2010 9:04 a.m. To: Miguel Pizarro Cc: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] dsrun - batch import Hello - Please check which version of Java you are using. DSpace 1.6 requires the Sun Java JDK 5 or later. --keith On Sep 29, 2010, at 1:58 PM, Miguel Pizarro wrote: hi everyone, I'm trying to import data to dspace 1.6 using dsrun application. In dspace 1.4 I don't have problem with this function, but in dspace 1.6 appear an error when I execute the follow sentence: ./dsrun org.dspace.app.itemimport.ItemImport -a -e dsp...@alerta.clmailto:dsp...@alerta.cl -c 1 -s /data/carga/root -m ejemplo_8 Destination collections: Owning Collection: Prueba Carga de colecciones Adding items from directory: /data/carga/root Generating mapfile: ejemplo_8 Adding item from directory 000541595 Loading dublin core from /data/carga/root/000541595/dublin_core.xml Schema: dc Element: format Qualifier: medium Value: BK Schema: dc Element: identifier Qualifier: code Value: 000541595 Schema: dc Element: language Qualifier: none Value: spa Schema: dc Element: contributor Qualifier: author Value: Bastà as, Gabriel Schema: dc Element: title Qualifier: none Value: Garantà as explà citas en salud (GES) : seis consideraciones para su implementación Schema: dc Element: publisher Qualifier: none Value: Pontificia Universidad Católica de Chile, Vicerrectorà a de Comunicaciones y Asuntos Públicos Schema: dc Element: date Qualifier: created Value: 2007 Schema: dc Element: description Qualifier: none Value: Temas de la agenda pública ; Año 2, no. 9 Schema: dc Element: subject Qualifier: other Value: Reforma en la atención de salud - Chile Schema: dc Element: subject Qualifier: other Value: Polà tica de salud - Chile Schema: dc Element: contributor Qualifier: author Value: Andia, Marcelo. Schema: dc Element: archivos Qualifier: adjuntos Value: 12.pdf Processing contents file: /data/carga/root/000541595/contents Bitstream: 12.pdf Processing handle file: handle It appears there is no handle file -- generating one Exception in thread main java.lang.NoSuchMethodError: method java.lang.String.format with signature (Ljava.lang.String;[Ljava.lang.Object;)Ljava.lang.String; was not found. at org.dspace.content.DCDate.toStringInternal(DCDate.java:284) at org.dspace.content.DCDate.toString(DCDate.java:272) at org.dspace.content.InstallItem.installItem(InstallItem.java:112) at org.dspace.app.itemimport.ItemImport.addItem(ItemImport.java:774) at org.dspace.app.itemimport.ItemImport.addItems(ItemImport.java:608) at org.dspace.app.itemimport.ItemImport.main(ItemImport.java:500) Could someone help me? Thank for your time... Miguel Pizarro Soporte Tecnológico Alerta al Conocimiento Fono 56 (2) 233 79 08 e-mail: sopo...@alerta.clmailto:sopo...@alerta.cl Providencia n° 2.040 of. C. CP: 7510117 -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.netmailto:DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
[Dspace-tech] delete of the list please
Sofía Alicia Treviño Garza Administradora de Sistemas Estadística Internacional Dirección de Programas Internacionales. Tecnológico de Monterrey Tel. +52 (81) 8358-1400 ext.2946 Enlace Intercampus 80-689-2946 El contenido de este mensaje de datos no se considera oferta, propuesta ni aceptación del Instituto Tecnológico y de Estudios Superiores de Monterrey. El contenido de este mensaje de datos es confidencial y se entiende dirigido y para uso exclusivo del destinatario, por lo que no podrá distribuirse y/o difundirse por ningún medio sin la previa autorización del Instituto Tecnológico y de Estudios Superiores de Monterrey. Si usted no es el destinatario, se prohíbe su utilización total o parcial para cualquier fin. -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
[Dspace-tech] manikin question
I'm trying to get a better understanding of Manakin, and I've made a change in the aspect ConfirmItemForm.java And would like now to experiment with the theme that handles the display of DRIs coming from this aspect, but I can't find the template that is responsible for displaying the page that goes with this aspect. By page I mean the main body, not the header and footer display. Thank you! Jose -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
[Dspace-tech] Oracle users - 64bit or 32bit? freshly built database?
As you've all probably seen, I have been having problems getting the registry_loader part of the ant build to work on Solaris10/Oracle10 I also tried ubuntu10.04/postgres8.4 and got what appears to be the same error. Previously I had little problem building ubuntu8.04/postgres8.3. Now I am wandering if the problem might be that Solaris/Oracle and ubuntu10.04/postgres8.4 are both 64bit installations. Is anyone running on 64bit OSes? Second part: If someone has had a successful fresh build on Oracle and has saved a dump of the schema, tables, initially loaded registry data, sequence values, etc; could you please send it to me? I commented out the load_registries part of the build.xml and have been able to bring up the xmlui interface and everything there looks OK. I can't do more because the database hasn't got the registries loaded (and anything else that might get loaded at the same time.) Unless I can get dspace properly working by Thanksgiving(Nov25), the plug is going to be pulled on it. I would rather find out why the build isn't working, but doing the missing bit by hand would allow me to see if the rest of the build worked and if batch loading will work and everything else. Desperately, Marvin mwea...@sju.edu -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] manikin question
Jose, I'm not sure I fully understand the question? Manakin works much differently than JSPUI -- so there is never really a one template to one page relationship. XSL Templates (in a theme) are usually used across many different pages in the system. Also, obviously many templates are used to generate a single page. The template you are looking for should be either in your Theme's XSL file or in the main 'dri2xhml.xsl' file (which is where most of the default templates reside -- as this file just basically transforms DRI into XHTML). If you are having trouble finding the exact template, sometimes it helps to look at the DRI structure of that page (add ?XML to the end of the Manakin URL, or XML if there's other stuff on the querystring already). You also may find it useful to use a tool like FireBug (http://getfirebug.com/) to analyze the structure of the generated XHTML, so that you can search through the templates in your Theme's XSL to find where that structure is generated. Hopefully that's helpful. Let us know if any of this doesn't make sense. - Tim On 9/29/2010 3:48 PM, Blanco, Jose wrote: I'm trying to get a better understanding of Manakin, and I've made a change in the aspect ConfirmItemForm.java And would like now to experiment with the theme that handles the display of DRIs coming from this aspect, but I can't find the template that is responsible for displaying the page that goes with this aspect. By page I mean the main body, not the header and footer display. Thank you! Jose -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Exporting items from DSpace
Hi Bram, Yes, half an answer is much appreciated, thanks! You're right, the collection template system I think would work for my second point. Cheers, Gary GARY BROWNE | Development Programmer Library IT Services | University Library THE UNIVERSITY OF SYDNEY T +61 2 9351 5946 | F +61 2 9036 From: bluy...@gmail.com [mailto:bluy...@gmail.com] On Behalf Of Bram Luyten Sent: Thursday, 23 September 2010 5:53 PM To: Gary Browne Subject: Re: [Dspace-tech] Exporting items from DSpace Hi Gary, because I think you might appreciate half an answer more than no answers at all, I was just thinking for your second question, that the collection templates might be used for this, as they add metadata values by default to newly submitted items. But I must say that the whole export import functionality has changed since collections can now be bound to an OAI-PMH URL from another resource, and I'm not entirely sure how that system works. Will let you know if your technical guys can add to this. best regards, Bram @mire - http://www.atmire.com Technologielaan 9 - 3001 Heverlee - Belgium 533 2nd Street - Encinitas, CA 92024 - USA http://www.togather.eu - Before getting together, get t...@ther On Thu, Sep 23, 2010 at 3:16 AM, Gary Browne gary.bro...@sydney.edu.au wrote: Hello again, Maybe it was something in the subject that turned people off my first two attempts? So I've changed it. Any takers on any of these questions below? Thanks, Gary GARY BROWNE | Development Programmer Library IT Services | University Library THE UNIVERSITY OF SYDNEY T +61 2 9351 5946 | F +61 2 9036 -Original Message- From: Gary Browne [mailto:gary.bro...@sydney.edu.au] Sent: Thursday, 16 September 2010 1:43 PM To: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] OAI-ORE configuration Hi all, While I'd still be really interested to hear answers to my original questions (or even to be told they don't make sense), perhaps I should start with what I think is a simpler question: What is considered best practice (and options) for exporting objects (bitstreams and associated metadata) from DSpace for use by other applications? Thanks a lot, Gary GARY BROWNE | Development Programmer Library IT Services | University Library THE UNIVERSITY OF SYDNEY T +61 2 9351 5946 | F +61 2 9036 -Original Message- From: Gary Browne [mailto:gary.bro...@sydney.edu.au] Sent: Monday, 13 September 2010 12:57 PM To: dspace-tech@lists.sourceforge.net Subject: [Dspace-tech] OAI-ORE configuration Hi, I haven't tried this yet, but after perusing the documentation I have a couple of questions. Assuming I want to harvest a collection from repository A into repository B: i) can repository B accept a subset of repository A's metadata for each item, instead of the whole lot? ii) can repository B be configured for additional metadata field/value pairs to be automatically added to harvested items? iii) I see documentation for the xmlui but not jspui - is it possible in jspui (documentation pointers)? Thanks a lot, Gary GARY BROWNE | Development Programmer Library IT Services | University Library THE UNIVERSITY OF SYDNEY Level 1, Fisher Library F03 | The University of Sydney | NSW | 2006 T +61 2 9351 5946 | F +61 2 9036 E gary.bro...@sydney.edu.au | W http://sydney.edu.au Sent from my plain old desktop computer. CRICOS 00026A This email plus any attachments to it are confidential. Any unauthorised use is strictly prohibited. If you receive this email in error, please delete it and any attachments. Please think of our environment and only print this e-mail if necessary. - -- --- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech --- --- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___
Re: [Dspace-tech] Limiting Searches
Hi George, Our dspace instance has a type indexing that can be searched. I did put the type-search options in a separated page - http://hub.hku.hk/browse/browse-type.jsp In the Advanced search there is no type drop down, but there is a custom made language drop down. Welcome for stealing, or borrowing of idea. Best, Allen Lam. HKU Scholars Hub Administrator, http://hub.hku.hk On 2010-09-30 2:37 AM, George Stanley Kozak wrote: I was recently asked to add type to the fields that can be searched in the Advance Search. I did do that. However, the problem is that users don't really know all of the types, so I was asked to add a drop down for all types. Along with that I was asked if I could modify the Advance Search so that people can limit their searches by Date range or by type. Before I spend a lot of time figuring this out, I was wondering if anyone has done anything similar to this that I could copy (or steal!). Thanks! George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 501 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Handle server on 1.6.2 (Windows)
When upgrading I made a complete copy of my 1.5.1 folder and continued installing 1.6.2. While configuring (and afterward) I made sure that my configuration files are similar. When attempting to start the handle server for 1.6.2 it doesn't work. But the idea came to me last night to use the previous version's handle server as a test. So this morning I ran the start handle-server script from my 1.5.1 backup directory - and everything works fine! The server starts up fine and the hdl.handle.net/ links resolve without problems, so it doesn't seem to be a problem on the server (ports/paths/etc..) - in my mind it still points to the 1.6.2 script. For the time being I will keep it running like this, although I realize this is not at all ideal (I assume logging isn't working right now), but the downtime really makes the institution look bad. Don't know if anyone has any ideas but I'm open to suggestions. Vrywaringsklousule / Disclaimer: http://www.nwu.ac.za/it/gov-man/disclaimer.html Vrywaringsklousule / Disclaimer: http://www.nwu.ac.za/it/gov-man/disclaimer.html On 27 September 2010 at 04:52 PM, Heh, Gregory h...@carnegielibrary.org wrote: Louw, I'm not sure about any bugs within the Windows version of the 1.6.1 startup script. Posting this question on the dspace-tech list will help. Do a file search within the dspace install directory for the handle-server.log. Sands Alden Fish had a good idea about disabling any firewall momentarily to rule out a network issue. Just don't give up checking the log files and working with CNRI and your network team to find the issue. Greg From:Louw Venter [mailto:louw.ven...@nwu.ac.za] Sent: Monday, September 27, 2010 10:36 AM To: Heh, Gregory Subject: RE: [Dspace-tech] Handle server on 1.6.2 (Windows) Greg Thank you for the response, I appreciate it. It is the same machine, so nothing changed there. I changed Tomcat to use those ports temporarily as a test and could connect, so am quite sure the ports are open and nothing else is using them. The CNRI answer was that they can't connect to our server, so they thought the ports were being blocked by some other process, when in fact the handle server just doesn't start up.. I also think because the server doesn't start up successfully a log file (other than error.log) isn't created (well, couldn't find one anyway). I am scared it might be a problem with the startup script for Windows specifically as, from what I've read, the script was buggy in version 1.6.1 so it was fixed for linux systems but not Win. Vrywaringsklousule / Disclaimer: http://www.nwu.ac.za/it/gov-man/disclaimer.html On 27 September 2010 at 04:13 PM, Heh, Gregory h...@carnegielibrary.org wrote: Louw, I've had similar issues and errors when trying to configure the handle server. Granted I was installing on a RHEL 5 virtual server but I think the general installation instructions remain the same. · Make sure all necessary ports are open for the appropriate communication o port 2641 UDP (input and output) o port 2641 TCP (input and output) o port 8000 TCP (input and output) o These are the default ports but you can choose others, just make sure that there are no other processes running on these ports as the handle service needs dedicated ports. · Double check your handle configurations in dspace.cfg and /[dspace]/handle-server/config.dct · Talk with your contact from CNRI, they can help you if the configuration isn't correct. Also what is the output from handle-server.log after you run the /[dspace]/bin/ start-handle-server script? My apologies if anything is different for a Windows machine. Greg From:Louw Venter [mailto:louw.ven...@nwu.ac.za] Sent: Monday, September 27, 2010 9:36 AM To: dspace-tech@lists.sourceforge.net Subject: [Dspace-tech] Handle server on 1.6.2 (Windows) Good day all Since upgrading (1.5.1 - 1.6.2) last week I've been trying to start the Handle-server without success. Has anyone been able to successfully upgrade on the Windows platform with a working handle server? The ports are still open and I've changed all the necessary configuration files as far as I know, but the server does not start. In the odd occurrence that it does start half-way, it freezes and does not communicate. Below is the contents of the error.log after three attempts. Any help/advice would be greatly appreciated. Regards Louw Venter 2010/09/27 03:28:29 GMT+02:00 25 Started new run. 2010/09/27 03:28:29 GMT+02:00 50 unspecified max_handlers count, using default: 200 2010/09/27 03:28:29 GMT+02:00 50 unspecified max_handlers count, using default: 200 2010/09/27 03:28:29 GMT+02:00 50 unspecified max_handlers count, using default: 200 2010/09/27 03:28:29 GMT+02:00 25 Shutting down server at Mon Sep 27 15:28:29 GMT+02:00 2010 2010/09/27 03:28:29 GMT+02:00 75 class net.handle.server.HdlUdpInterface: Error handling request: java.net.SocketException: socket closed
