Re: [Dspace-tech] log4j causing slowness for DSpace

[Graham Triggs]

Jason,

It's highly unlikely that Log4J is causing the startup to be slow. Whilst
your startup takes in the region of 10 minutes, there is no indication in
the log as to when each of those log4j warnings appeared, or what was
occurring at the time.

The regular re-occurrence of org.apache.commons.digester.Digester.sax
appender warnings, and the org.apache.catalina.startup.Embedded warning,
would suggest that the application server itself (Tomcat?) is missing some
logging configuration.

Those Digester messages look like they relate to the initialization of each
webapp deployed to the server - ie. where it loads WEB-INF/web.xml (and
correlating with the 'using dspace.cfg' messages that would also appear with
each one).

Deploying multiple web applications can take some time - loading the JARs,
starting any listeners / servlets that are set to load on startup. In the
case of xmlui (and Solr?) at least that will trigger a fairly significant
initialization process.

And it appears that you are deploying the xmlui application twice (once as
xmlui, and once as root?)

Regards,
G


On 5 April 2010 21:50, Jason Fowler  wrote:

> I've been beating my brains in over this issue, and I have tried every
> possible solution I can find. I've recently migrated to 1.6.0, and since
> making the change, problems with log4j occur at startup. This makes my
> Dspace startup take ten minutes. I've narrowed down the problem (I think) to
> being an issue with log4j.
>
> I just appended a date stamp to my catalina.out file while I restarted. I
> have output it below. The first thing that shows up after the stamp is log4j
> errors. After churning for 10 minutes, everything works fine, even logging,
> solr, etc. Does anyone have any idea what might be causing this problem? If
> so, many, many thanks in advance.
>
> Mon Apr  5 15:29:51 CDT 2010
> log4j:WARN No appenders could be found for logger
> (org.apache.catalina.startup.Embedded).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN No appenders could be found for logger
> (org.apache.commons.digester.Digester.sax).
> log4j:WARN Please initialize the log4j system properly.
> INFO: Loading provided config file: /mnt/storage/dspace/config/dspace.cfg
> INFO: Using dspace provided log configuration (log.init.config)
> INFO: Loading: /mnt/storage/dspace/config/log4j.properties
> Loading catalog:
> file:/usr/local/apache-tomcat-5.5.26/work/Catalina/localhost/xmlui/cocoon_xml_resolver_entities/catalog
> log4j:WARN No appenders could be found for logger
> (org.apache.commons.digester.Digester.sax).
> log4j:WARN Please initialize the log4j system properly.
> INFO: Loading provided config file: /mnt/storage/dspace/config/dspace.cfg
> INFO: Using dspace provided log configuration (log.init.config)
> INFO: Loading: /mnt/storage/dspace/config/log4j.properties
> - Loading provided config file: /mnt/storage/dspace/config/dspace.cfg
> - Using dspace provided log configuration (log.init.config)
> - Loading: /mnt/storage/dspace/config/log4j.properties
> log4j:WARN No appenders could be found for logger
> (org.apache.commons.digester.Digester.sax).
> log4j:WARN Please initialize the log4j system properly.
> INFO: Loading provided config file: /mnt/storage/dspace/config/dspace.cfg
> INFO: Using dspace provided log configuration (log.init.config)
> INFO: Loading: /mnt/storage/dspace/config/log4j.properties
> log4j:WARN No appenders could be found for logger
> (org.apache.commons.digester.Digester.sax).
> log4j:WARN Please initialize the log4j system properly.
> INFO: Loading provided config file: /mnt/storage/dspace/config/dspace.cfg
> INFO: Using dspace provided log configuration (log.init.config)
> INFO: Loading: /mnt/storage/dspace/config/log4j.properties
> Loading catalog:
> file:/usr/local/apache-tomcat-5.5.26/work/Catalina/localhost/_/cocoon_xml_resolver_entities/catalog
> log4j:WARN No appenders could be found for logger
> (org.apache.commons.digester.Digester.sax).
> log4j:WARN Please initialize the log4j system properly.
> INFO: Loading provided config file: /mnt/storage/dspace/config/dspace.cfg
> INFO: Using dspace provided log configuration (log.init.config)
> INFO: Loading: /mnt/storage/dspace/config/log4j.properties
> Apr 5, 2010 3:39:31 PM org.apache.solr.servlet.SolrDispatchFilter init
> INFO: SolrDispatchFilter.init()
> Apr 5, 2010 3:39:31 PM org.apache.solr.core.SolrResourceLoader
> locateInstanceDir
> INFO: Using JNDI solr.home: /mnt/storage/dspace/solr
>
> Jason Fowler, CA, MSLS
> Archives and Special Collections Librarian
> The Southern Baptist Theological Seminary
> Vice President, ALABI
> jfow...@sbts.edu
>
>
> --
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> __

Re: [Dspace-tech] DSpace 1.6 Run With Tomcat Security Manager Enabled, Anybody?

[Graham Triggs]

Hi Christian,

It's good to see someone else working on this. I did look at it myself at
one stage, in a mad ambition to be all 'correct' about it. And after chasing
down the rabbit hole for hours creating a seemingly never-ending list of
permissions, I came to the conclusion that as I've got a firewall and httpd
in between my application server and the nefarious horde, it probably wasn't
worth torturing myself trying to get it completed.

But one thing in your email did strike me...

On 7 April 2010 00:07, Christian Voelker  wrote:

> I found a hint on the web, that it should be possible to put a policy file
> into the WEB-INF directory of a web application which I think would be a
> great solution, but it has to work before.
>
>
It seems really odd that an application server / security manager should
allow you to do that. The whole point is to limit the extent to which a
deployed application can interact with / harm the system, and it's quite a
large hole in the wall if an application can simply say "actually, do you
mind if I just mess around with the system a little".

One thing you might want to try to find out what is going on, is to enable
debugging of the security manager, with the CATALINA_OPTS:

export CATALINA_OPTS=-Djava.security.debug=all(Unix)
set CATALINA_OPTS=-Djava.security.debug=all   (Windows)

Good luck!
G
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Improper Install?

[Graham Triggs]

The 'work' directory is under the control of Tomcat. If the application
specific directories are not being created, then it is almost certainly
because the user that Tomcat is being run under does not have permission to
write to the work directory.

G

On 7 April 2010 19:56, bryce ray  wrote:

> I'm having some difficulties with my installation of dspace.  I've
> discovered that directories that should have been created were not.  When
> are these directories created? and what might have caused them not to be
> created?
>
> Directories in:
> /usr/share/tomcat6/work/Catalina
>
> When viewing this directory there are sub directories for other
> installations of dspace.  However, there is no directory for my most recent
> install of dspace.
>
> Let me know if you need any more information, and thank you for helping me
> diagnose this problem.
>
>
> --
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] tomcat reporting memory leak?

[Graham Triggs]

On 20 September 2010 15:59, Tom De Mulder  wrote:

> On Mon, 20 Sep 2010, Damian Marinaccio wrote:
>
> > I'm seeing the following log messages in catalina.out:
> > [...]
> > SEVERE: The web application [] appears to have started a thread named
> [FinalizableReferenceQueue] but has failed to stop it.
> > This is very likely to create a memory leak.
>
> There are quite a few memory leaks in DSpace. We have a cronjob to restart
> Tomcat nightly, because otherwise it'll break the next day.
>


Hi all,

Oh, welcome to my world!!

I'm going to start off by pointing out that the majority of DSpace code is
actually quite well behaved. Going back to the codebase circa 1.4.2 / 1.5,
and using the JSP user interface - I've got *thirty* spearate DSpace
repositories / applications running in a single Tomcat instance, which has
operated without a restart in over 90 days. And whilst be able to undeploy
and redeploy any of those applications at will - or just reload them so that
they pick up new configuration.

That does require a bit of careful setup / teardown in the context listeners
(that wasn't always part of the DSpace code), and you need to get certain
JARs - particularly the database/pooling drivers - out of the web
applications entirely and into the shared level of Tomcat. Most of that is
actually just good / recommended practise for systems administration of a
Java application server anyway.

I was careful to point out that I have achieved that with pre-1.6 code and
JSP only. Both 1.6 and XML ui (of any age) change the landscape. XML ui has
always taken a large chunk of resources, although whilst it was still based
on Cocoon 2.1, I managed to at least clean up it's startup / shutdown
behaviour by repairing it's logging handler. This behaviour has changed with
Cocoon 2.2, and I'll come back to that shortly.

So, 1.6 - I've been doing some work on the resource usage and clean
loading/unloading of both JSP and XML using 1.6.2 recently, and neither are
clean out of the box.

The first issue you run into is the FinalizableReferenceQueue noted in the
stack trace above. This is coming from a reference map in reflectutils - and
was found to be a cleanup problem in course of DSpace 2 development (the
kernel / services framework was backported from that work). I added a
LifecycleManager to reflectutils that was released as version 0.9.11 that
allows the internal structures to be shutdown cleanly, and implemented this
as part of DSpace 2, however this appears to have been ignored in the
backport.

So, with the reflectutils/Lifecycle changes, and careful placement of JARs,
etc. I did get the JSP ui to unload cleanly last week. I would note that I
didn't stress the application too heavily, so there may be some operations
that might trigger different code paths that are still a problem, but at the
baseline it was working correctly.

XML ui has proven to be a somewhat more challenging beast. I first ran into
two problems that are inside Cocoon 2.2 itself - 1) in the sitemap
processing, it's using a stack inside a ThreadLocal, but it never removes
the stack when it empties it, and 2) in one class relating to flowscript
handling, it does not clean up the Mozilla Rhino engine correctly when it's
finished using it (curiously, it's used in a number of places, and
everywhere else it appears to be structured correctly to clean up - just
this one class is screwed up).

With locally patched versions of the sitemap and flowscript JARs from Cocoon
(the ThreadLocal patch isn't really guaranteed to not leak in unexpected
circumstances - but it was sufficient to remove the problem in the scope of
this testing. Basically, ThreadLocal is really dangerous to use), I then ran
into another issue, this time with the CachingService that was backported.

With XML ui, it's using the RequestScope function of the caching service (it
didn't appear to be exercising this part with JSP - that may just be because
I only ran through limited code paths). For the RequestScope, it's tying the
cache not to the request object... but to a ThreadLocal. And that
ThreadLocal isn't being cleaned up at the end of the request. (The shutdown
code is also incapable of doing the job it's intended for, as it will only
ever execute on a single thread, and not see all the other threads that may
have processed requests).

There is a high probability of this leaking memory all over the place, and
there is also the nasty potential of leak information across requests that
is undesirable.

I made another hacked version that removes the ThreadLocal, but replicates a
lot of it's thread affinity behaviour (so, it still has the nasty side
effects of the implementation, but at least removed the hold the system had
over the application resources). XML ui was *still* not unloading correctly,
and at this point the profiler stopped giving me pointers to strong
references that were being held. So right now I'm not sure what else is up -
but there is at least one more troubling part of the c

Re: [Dspace-tech] tomcat reporting memory leak?

[Graham Triggs]

On 29 September 2010 11:38, Hilton Gibson  wrote:

> Using the XMLUI.
> Does DSpace really need this and what happens when we go to one million
> items ??
>
>
Does DSpace really need that? No. As I have said, I'm running 30 separate
repositories - using JSPUI (circa 1.4.2 / 1.5 codebase) - all on a single
server / Tomcat instance.

Some of those repositories have 1000s of items, and get quite decent levels
of access.

The server has 8GB installed, 3GB heap turned over to Tomcat (plus 1GB for
non-heap).

The Tomcat instance has 2GB of *free* heap space, rarely runs above 5% cpu
usage, and has plenty of capacity to run more repositories (the rate at
which files are opened/closed is actually a bigger issue for Tomcat
startup).

Although, it's worth pointing out that the database is hosted on a separate
server - I can't say how many resources that is really using, as it's shared
with other services, but it is apparently 'tiny'.



What happens at one million items? Well, that's an interesting issue. But is
it really the right question to be asking? How far do you want/need to be
able to scale a 'monolithic' instance, before you spread it over multiple
servers?

As long as you can spread it over multiple servers, it gives you a much
higher ceiling than relying on a single box - and it is easier to scale for
increasing size/usage by adding more boxes (you don't have to migrate).

If you focus on scaling a single installation, then you end up increasing
the overall requirements (ie. memory for caching), and make it harder to
have scaling over multiple boxes at all.

G
--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] tomcat reporting memory leak?

[Graham Triggs]

On 29 September 2010 11:48, Tom De Mulder  wrote:

> A lot of the back-end code of DSpace, the very core of it, is inherently
> inefficient


I don't entirely disagree with that statement - there are some things that
can definitely be improved, particularly where you have to deal with more
items in a single instance.

But take a look at my numbers - at it's core, it really isn't that bad for
the vast majority of DSpace users (how many have more than even 50,000 items
currently)? And some of it depends on correct system setup (Postgres
version/options, etc.)

It's adding xmlui, solr, etc. that is putting a lot more demands on the
system.


G
--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] tomcat reporting memory leak?

[Graham Triggs]

That begs the question as do you think something else should be chosen /
recommended?

There really isn't anything preventing you using Jetty, etc. but Tomcat is
actually a pretty solid server that does a lot of things quite well - and
particularly in recent versions in being defensive against bad application
behaviour.

And when you look at the grand scheme of things, the smaller footprint of
Jetty doesn't really make a whole lot of difference.

G

On 29 September 2010 11:47, Mark Ehle  wrote:

> Why was tomcat chosen as a platform for DSpace?
>
>
--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] tomcat reporting memory leak?

[Graham Triggs]

On 29 September 2010 14:17, Tom De Mulder  wrote:

> I know you like to talk down the problem, but that really isn't helping.
>

This isn't about talking down the problem - it's about finding where the
real problems are and not just patching the immediate concerns. And
considering the interests of nearly 1000 DSpace instances that are
registered on dspace.org - many of whom will probably be more worried about
rampant resource usage for small repositories from adding overhead to cover
up the problems of larger repositories.


> We run 5 DSpace instances, three of these are systems with hundreds of
> thousands of items, and it's dog slow and immensely resource-intensive. And
> yes, we want these to be single systems. Why shouldn't we?
>

Surely the more pertinent question is why wouldn't you want to be able to
run a multi-node solution? I'm sure I don't need to tell you that no matter
how good a job you do of making the system perform better with larger
datasets, there will always be a finite limit to how large the repository
can be, how many users you can service, and how quickly it will process
requests for any given hardware allocation.

Yes, DSpace can do a better job than it currently does, but it's just
postponing the inevitable. How much in technology relies on just making
things bigger/faster? Even our single system hardware is generally made of
multiple identical components - CPUs with multiple cores, memory consisting
of multiple 'sticks', each consisting of multiple storage chips, storage
combining multiple hard drives each having multiple platters.

And much of our dependencies are going the same way - Oracle database
clusters, Solr is designed to get scalability from running over multiple
shards, even Postgres has taken a major step towards clustering /
replication with it's 9.0 release.

Either way, you will always hit a hard limit with keeping things on a single
system - so at some point, something has to give, whether it's separating
out DSpace application, Solr and Postgres instances to separate machines, or
accepting this reality in the repository and building it to scale across
multiple nodes itself. This in turn would bring benefits to how easily you
can scale (in theory, a lot easier to scale at the repository level than
scaling each of it's individual components), as well as potentially better
preservation and federation capabilities.

G
--
Virtualization is moving to the mainstream and overtaking non-virtualized
environment for deploying applications. Does it make network security 
easier or more difficult to achieve? Read this whitepaper to separate the 
two and get a better understanding.
http://p.sf.net/sfu/hp-phase2-d2d___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

[Dspace-tech] A simplified version of dspace-services and Tomcat unloading

[Graham Triggs]

Hello all,

So, we've been debating the performance / resource usage of DSpace 1.6. This
has just happened to coincide with work that I needed to do to qualify
DSpace 1.6 (and/or 1.7) for our own use. So, for the past week or so, I've
been hammering away at profiling DSpace. This isn't a comprehensive analysis
of the entire codebase, but focusing on the serious issue of stability and
resource leakage.

This immediately led to finding two issues with the dspace-services code and
it's dependencies:

1) The resources in the FinalizableReferenceQueue were leaking (this was
noted before in a Tomcat stack trace posted here) - this could have been
avoided if a newer version of reflectutils had been used, along with the
LifeCycleManager it contains

2) A ThreadLocal in the CachingServiceImpl was also leaking - the service
did attempt to clean up in the shutdown, but it would only ever have been
able to clear up the resources of the thread executing the shutdown, not all
the other threads that would have processed requests.

Investigating the second issue, it turned out that this was being used to
keep track of Request scope caches - as well as potentially leaking
resources (across requests), this highlighted a problem in the service
contract, and a dangerous coupling between the implementation of the
services.

The Caching service was written to return a Request scoped cache without any
valid Request to bind it to - this was explicitly used in the listeners /
filters to acquire a cache and assign request objects into it before the
request service was called to 'start' a request and bind to the scoped
cache. There were also tests that expected this behaviour. Whilst the
request service will clear a request scoped cache if it's been bound to it,
you can't guarantee that the cache gets bound to a request.

Worse though, the Request service was depending on the Caching service
tracking request scoped caches against the current thread in order to be
able to track the current request. So, the one responsibility that the
request service should have, it was passing on and hoping that another
service might do the job for it without it being explicitly clear in the
contract.

If that wasn't bad enough, getCache doesn't actually properly respect the
CacheScope when trying to retrieve a particular cache (it could retrieve a
non-Request scoped cache if one of the same name had already been created,
regardless of the scope parameter). The Caching service also registered an
MBean that was never removed. And, to wrap up my examination of the Caching
service, it didn't close an InputStream that it opened.

So, onwards - wanting to simplify this code a bit (as we don't need to
support non-Spring IoC containers, at least not yet - and we do need to be
able to understand and maintain this code) - I quickly noticed an issue in
the SpringServiceManager where it returned multiple instances of the same
service name, because it was under the mistaken belief that it needed to
track some of the service names itself, when it was actually getting them
returned from Spring.

And the ServiceMixinManager was entirely redundant - as long as we can rely
on Spring, we can rely on the container to do everything that the Mixin
manager does, without us having to worry about the complexity of it (and it
is both relatively complex, and a source of our problems earlier as it uses
the FinalizableReferenceQueue). I've actually got a real bugbear about us
using the term 'mixin'. A mixin means something specific - a class that
provides an actual implementation, and is combined with other mixins to
produce other classes [through multiple-inheritance]. When you look at what
is implemented in dspace-services under the term mixin, well we don't
actually have mixins... we have interfaces that are implemented by service
classes. Each service class completely and wholly provide implementations of
the methods defined in all the interfaces they implement. That's a concept
that Java has had for a long time, and is (should be) well understood by all
Java programmers. That doesn't make them mixins. We don't/shouldn't want
'mixins' (multiple inheritance - there is a reason Java doesn't support it).
And we shouldn't be using a term incorrectly just because it's cool.

While I'm on a terminology rant, I'm not particularly happy about the
'interceptors' (ie. RequestInterceptor). They are looking and feeling a lot
more like listeners than interceptors - allowing other services to react to
events taking place (start request / end request), rather than affecting
it's behaviour. I've not renamed them, for the sake of keeping as much of
the existing API intact as I can (whilst clearing out [currently] redundant
or problematic code), but actually it seems that it should really be pushing
a synchronous and immediately executing event notification, so that the
listener registration can occur in a single place rather than duplicating
that functionality anywhere similar concepts may 

Re: [Dspace-tech] tomcat reporting memory leak?

[Graham Triggs]

On 5 October 2010 16:33, Simon Brown  wrote:

> Which nobody has requested, making this a massive red herring. I fail
> to see how cutting back on unnecessary and redundant database access
> constitutes "overhead to cover up the problems of larger
> repositories".


One person's "unnecessary and redundant database access" is another's very
necessary database access - well, at least it can be.

I remember the patch for reducing the updating of browse / search indexes,
and I can see why it would be useful to not do those updates during a batch
import if you have an appropriate workflow.

That won't be the case for all of the repositories - quite a few will
welcome the ability to see those items as and when they are added. There is
also the issue of how long it takes to do the one very big update at the end
of the batch run vs. incremental changes as you go - it may be less work
overall, but having one big change can be more disruptive in some cases.


> Any repository, regardless of size, will see
> improvements with this kind of optimisation, at least one example of
> which I have already highlighted (and had my arguments shouted down -
> this is also, incidentally, why I haven't bothered to open any other
> JIRA tickets on other performance issues we've seen. What would be the
> point?)
>

No, you didn't get shouted down for raising a performance issue. Where the
argument came was because you assumed that this would clearly be of benefit
to "any repository", when you did nothing to address the underlying
performance issues (which could have been helped quite dramatically with
some small SQL tweaks and some configuration work in Postgres), and instead
just bypassed them for one very specific use case.

It doesn't matter how large or small a repository is, if they don't perform
batch uploads using the ItemImporter, your change will do *nothing* for
them. But an alteration to the underlying SQL, and guidelines for getting
the best out of Postgres would benefit everyone - regardless of how large or
small the repository is, or the means by which they populate it.


> The pertinent question for me is why, whenever the issue of
> performance comes up, is one of these "theoretical future of
> repositories" screeds pulled out and slammed down in front of the
> conversation? People are reporting problems with the systems they have
> *right now*.


It's not meant to be a barrier to conversation, but a question as to what
you want to resolve. Do you want to address the *scalability* of DSpace, or
do you just want to avoid an immediate performance bottleneck? If we
conflate these, conversations are going to stall, and we're not going to
make any progress.


> Or rather, they were. And yes, it is true that there is a
> finite limit to what the hardware is capable of, but the quality of
> the software plays a significant role in how quickly that limit is
> reached. But we've had this conversation before. I don't really expect
> it to end any better this time than it did then.
>

I completely agree - but a solution that breaks the encapsulation of the
components in the system, and leaves important indexes in an inconsistent
state for an extended period of time is not an automatic win for the
majority of the community.

I offered a lot of suggestions as to how that code could be better
structured, improvements both to the SQL and the configuration of Postgres
to handle the load more efficiently, and suggestions for further tweaks that
would reduce the amount of updates that the code would have needed to do
still further. All of which would have be more beneficial to the community
(not just improving batch uploads, but interactive / singular deposits and
edits) - and not only that, would have improved the performance of your
systems further than you had so far achieved.

Any method of increasing the processing capabilities of a system,
> either through more powerful hardware or improvements in the software,
> is "postponing the inevitable" for any repository with continued
> growth. The difference is in how much cost there is to any individual
> repository in each of those methods. Our system, with the changes
> we've made to it, struggles at around 300,000 items. People are
> reporting problems (presumably running stock 1.6.2) at around 50,000,
> from what I can gather.


This is where we need to be careful about what we are reporting. Quite a few
of the issues around 1.6.x appear to be around rampant memory usage, rather
than a clear function of how many records there are in the database. There
are also different issues involved if we are talking about adding / editing
lots of records, or simply highly accessed.

Even so, regardless of what we do to the code to make it efficient, it does
not and can not absolve the system administrator of correctly maintaining
both DSpace itself, and it's dependencies. I wouldn't want to get drawn on
where that point is without any evidence, but there is a lot of scope for
altering and im

Re: [Dspace-tech] A simplified version of dspace-services and Tomcat unloading

[Graham Triggs]

On 5 October 2010 19:17, Sands Alden Fish  wrote:

> What resources were leaking here exactly?  Leaking resources across
> requests can have some serious consequences depending on the resources, and
> I'm curious to know exactly what type of errant behavior we could expect
> from a 1.6.x version running the flawed services framework.
>

I would say that the ability to leak resource across requests is more
theoretical than demonstrated in a real world scenario. By which I mean that
it's clearly demonstrated that the Caching service has the ability to store
data either at the wrong scope, or to have request scope caches that
actually get bound / re-bound to a later, different, request - however, I
haven't exercised a full DSpace application in such a way to observe this
happening.

But even if the application wouldn't normally run into an issue, that
doesn't mean we shouldn't tighten up it's behaviour to prevent it from
becoming an issue later.

G
--
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] DSpace/Maven help request - update dependency version

[Graham Triggs]

That version of tm-extractors is quite old.

There is a newer version on the Google site -
http://code.google.com/p/text-mining/ - but it will take a bit of work
wrapping things up for general use.

It has dependencies on newer versions of POI than 0.4, and some distinct
improvements to it's robustness.

G

On 6 October 2010 16:39, Tim Donohue  wrote:

> Ugh -- sounds like you've entered dependency hell.
>
> Though, I think the one shred of good news here is that it seems to only
> have a dependency conflict in one place in our codebase.
>
> It looks like (at a glance) if our WordFilter can be re-written to no
> longer need the org.textmining project, you *might* be OK (i.e.
> hopefully it wouldn't snowball on you). But, that would require finding
> a Word document text extractor that is as good as (or better than) that
> 'org.textmining' one, and then hoping it doesn't cause another
> dependency conflict.  Not sure of any alternative Word text extractors,
> off the top of my head, but maybe others know of one?
>
> - Tim
>
>
> On 10/6/2010 5:51 AM, Keith Gilbertson wrote:
> > Thanks, Tim.  That helped me to understand.  I put the version numbers of
> the dependency in the parent pom.xml ('dspace-src/pom.xml') and left the
> version numbers out of 'dspace-src/dspace-api/pom.xml'.
> >
> > So then I found another thing I didn't look at closely enough.  The
> WordFilter doesn't use poi directly, but the org.textmining project that it
> uses depends on that old version of POI.  To confuse things more, the old
> versions of poi had groupId 'poi', and the new versions have groupId
> 'org.apache.poi'.
> > I can convince Maven to forget about the old version of the POIi library
> by making this exclusion change in the parent pom:
> >   
> >  org.textmining
> >  tm-extractors
> >  0.4
> >  
> > 
> >poi
> >poi
> > 
> >  
> >   
> >   
> >
> > Then only the new version, org.apache.poi/poi/3.6 is included in the
> project.  Unfortunately, the org.textmining extractors really do need that
> version of POI.  The PowerPointFilter works, but I've broken the WordFilter:
> >
> > Exception:
> org.apache.poi.poifs.filesystem.POIFSFileSystem.getRoot()Lorg/apache/poi/poifs/filesystem/DirectoryEntry;
> > java.lang.NoSuchMethodError:
> org.apache.poi.poifs.filesystem.POIFSFileSystem.getRoot()Lorg/apache/poi/poifs/filesystem/DirectoryEntry;
> >   at
> org.textmining.text.extraction.WordExtractor.extractText(WordExtractor.java:51)
> >   at
> org.dspace.app.mediafilter.WordFilter.getDestinationStream(WordFilter.java:95)
> >
> > I have two programs that share the same classpath, but need different
> versions of the same library.
> >
> > I could rewrite the WordFilter so that it no longer uses the
> org.textmining package which needs the old library, but I keep thinking that
> the more I try to "fix" stuff, the more I'm likely to break:
> >
> >
> http://www.nypost.com/p/news/local/brooklyn/rat_bastards_f5onjzgcqxm0fu3RFz3ySL
> >
> >
> >
> > On Oct 5, 2010, at 4:09 PM, Tim Donohue wrote:
> >
> >> Hi Keith,
> >>
> >> Simply put, it's because you were accidentally looking in the wrong
> pom.xml :)  There's many of them sprinkled through the DSpace codebase, and
> they all inherit many of their settings from one main pom.xml.
> >>
> >> So, you noticed that the 'dspace-api/pom.xml' file included a dependency
> for "poi".   But, if you look closely, that dependency doesn't list
> a.  This is because, for DSpace, we manage all the versions of
> dependencies in one parent pom.xml (which is loaded via the  tag
> within the dspace-aip/pom.xml).
> >>
> >> Now, take a look at the [dspace-src]/pom.xml. This is the main Parent
> pom.xml for dspace (with an artifactid of 'dspace-parent')
> >>
> >> http://scm.dspace.org/svn/repo/dspace/trunk/pom.xml
> >>
> >> This is the pom.xml which actually lists the versions of every
> dependency used by the various APIs of DSpace.  If you search in this
> pom.xml, you'll find this entry:
> >>
> >> 
> >> poi
> >> poi
> >> 2.5.1-final-20040804
> >> 
> >>
> >> That's where the 2.5.1 version is sneaking in.  If you make your
> necessary changes to this pom.xml, everything should act as you expect it
> to. So, just undo your changes in 'dspace-src/dspace-api/pom.xml', and
> instead make those changes to 'dspace-src/pom.xml'
> >>
> >> I hope that helps!
> >>
> >> - Tim
> >>
> >> On 10/5/2010 2:36 PM, Keith Gilbertson wrote:
> >>> Hi,
> >>>
> >>> I've been experimenting with a Media Filter for text extraction from
> PowerPoint files.  It's based on the Apache POI libraries, as was suggested
> by others in a previous thread.
> >>>
> >>> It uses the poi, poi-scratchpad, and poi-ooxml artifacts, in version
> 3.6, the latest release version from Apache.  I haven't done much with
> Maven, and am not sure how to tell it which libraries I need.
> >>>
> >

Re: [Dspace-tech] A simplified version of dspace-services and Tomcat unloading

[Graham Triggs]

On 7 October 2010 06:55, Mark Diggory  wrote:

> You know I'm for (1) simplified spring only service manager. (2) jdbc
> data source delivered by servlet container (3) dropping reflectutils.
>

Well, the JDBC datasource is a change to dspace-api rather than -services,
but there is no reason why it can't go into DSpace 1.7.


> I'm not so sure about dropping the MBean registration completely, the
> whole idea was that we could have a container level service manager
> and services deployed across webapplications.  But I assume this is
> just too crazy an approach to maintain and educate others on its
> usage.  I'm just not convinced we would use a servlet container in
> this manner and simplification is no doubt more important...
>

My default position is to say that if we aren't actively using something, we
shouldn't have it. Whilst the registration technically works, we haven't got
close to properly testing and qualifying sharing resources across contexts
(/classloaders), and people will run into a lot of problems attempting to
use it that way.

Besides which, MBean isn't really the way to share resources in that way -
JNDI is. Although MBeans are available outside a container, whereas JNDI
isn't necessarily. I wouldn't class that as a good reason to use MBeans -
it's a good reason to have that part of the functionality switchable -
rather than a static manager that combines singleton and MBean registration
as conditional logic, it should be a static facade to either a singleton or
Mbean (or JNDI) implementation.

That's slightly beside the point though, as in general, I agree with you -
cross context sharing is a headache, a problem I don't think 'we' want to
try and solve, and difficult for people to understand. If you need that kind
of dynamic/component environment (and I don't think it's that big a deal to
either assemble a web application combining multiple artifacts, or deploying
it as a whole), then there are better ways of dealing with it (either
SpringDM or EJB).


>
> My only concern is that the ability I put inplace for addons to
> deliver their own spring config and have the manager load it (in a
> SpringDM style manner) be maintained, and I see its still there... :-)
>

It's possibly an area that could yet be simplified though (I've taken a
somewhat pragmatic approach so far to not rip up any existing public APIs
unless they are problematic in the current form). One question to ask is
would it only need the SpringDM style approach when you would be using
SpringDM (on the assumption that you are using that for a dynamic
environment)?


What if we put it in the dspace-services trunk, adjust the dspace
> trunk to use it and use this to get some community testing/feedback?
> If it fairs well over the next month... then we can do an official
> release of dspace-services just prior to the dspace 1.7.0 release
> candidates...
>

Yes, that sounds like a plan. Although it passes it's unit tests (with only
minor test modifications necessitated by the contract tightening), and I've
used it in a DSpace 1.6.2 application without any direct modifications
(related to the services at least), it does need more extensive
in-application testing.

G
--
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] manikin question

[Graham Triggs]

On 8 October 2010 19:06, Walker, David  wrote:

> > breaking up the xslt into separate files for each page
> > actually defeats the design goals of the XMLUI theming tier.
> > I.E. having one file within which all customizations reside to
> > manage your theme.
>
> I think there are advantages and disadvantages to both approaches.
>

I agree there are pros and cons to both approaches. And yes, if we focus on
some specific areas of the implementation, there may be too much
presentation being encoded in the DRI.

But in general, I would agree with Mark. The design of Cocoon / Manakin is
based around pipeline processing of the model, and for you to be able to
chain in components/aspects, you neither want to do it after a page based
transformation (and have to put in multiple special cases for each of the
pages), or have page based transformations after the aspect (requiring that
you modify each of those pages as well, and not just add the new component).


But the thrust of my argument here is that we should have *templates* that
> correspond to pages.  Whether each of those templates lives in it own file,
> or whether they all live together in one big file, is a minor detail, as far
> as I'm concerned.
>


I definitely agree with your point that the approach taken by Cocoon /
Manakin is a learning curve for some people, and that a number of people
would benefit from having page based templates. But if you are going to do
that, then you might as well set Cocoon aside - there isn't any point in
taking the added dependencies, xml serialization and transformation, if you
can't make [effective] use of the transformation pipeline.

Imho, if you want to use a page based approach, then you are better off
using Spring WebMVC, and writing the templates in Freemarker [disclaimer: I
did start implementing such a framework, but it doesn't have any functional
components as yet].

G
--
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] postgres/postmaster 99%

[Graham Triggs]

It's a second level browse - ie. if your 4th browse index is 'subject', then
it's someone looking at all the items that have a particular subject entry.

This shouldn't be that expensive a query (given there is no offset
involved), but you may have an issue with an index missing (although the
browse code should create all the ones that it needs when it creates the
table), or more likely you have too low a value for your shared_buffers
(required to load the indexes), or work_mem (used for the join between the
tables). Additionally, you may need to analyze and/or reindex the tables.

How many items are in the repository, and how many values do you have in the
4th browse option (subject?). And what version of Postgres are you running?

G

On 10 November 2010 22:42, Steve Swinsburg wrote:

> A followup:
>
> I restarted postgres and within minutes the same query has appeared in the
> stats and CPU is back up to 99%. Could we be missing some indexes or
> something? We only recently ran the filter-media script and generated
> thousands of thumbnails and branded previews but have since performed a
> vacuum.
>
> Any information would be much appreciated.
>
> cheers,
> Steve
>
> On 11/11/2010, at 9:12 AM, Steve Swinsburg wrote:
>
> Hi all,
>
> We are experiencing an issue on both of our dspace instances where
> postmaster spins up 99% of the CPU. Sometimes it's just one process at 99%,
> othertimes its a dozen or more processes around 7-9% each. I ran some stats
> on postgres via:
>
> select * from pg_stat_activity
>
> I found this query in the output about 15 times:
>
> SELECT bi_item.* FROM bi_item, (SELECT bi_4_dmap.item_id FROM bi_4_dmap,
> bi_4_dis WHERE bi_4_dmap.distinct_id=bi_4_dis.id AND
> bi_4_dis.sort_value=$1 ) mappings  WHERE  bi_item.item_id=mappings.item_id
> ORDER BY sort_3 ASC  LIMIT $2
>
> The earliest has a start time of about 5 hours ago.
>
> Anyone know whats up?
>
> cheers,
> Steve
>
>
>
>
> --
> Centralized Desktop Delivery: Dell and VMware Reference Architecture
> Simplifying enterprise desktop deployment and management using
> Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
> client virtualization framework. Read more!
> http://p.sf.net/sfu/dell-eql-dev2dev
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
--
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] A metadata schema for journals - Prism ?

[Graham Triggs]

Hi Robin,

Theo is quite right - I think PRISM is a good standard for repositories to
support for published material. In fact, it's probably more important for
SWORD to be able to accept PRISM metadata than it is to support SWAP - given
the majority of sources that might supply data via SWORD already support
PRISM (and it contains more useful discrete metadata - like volume / issue -
than is described by SWAP).

G

On 11 November 2010 11:05, TAYLOR Robin  wrote:

> Hi all,
>
> Periodically the subject of how to store journal volume and issue numbers
> as Dublin Core comes up on this list. Our repo admin Theo Andrew came up
> with this schema as a possible solution so I just thought I would pass it
> on. The attached diagram shows how it can be used.
>
> http://www.prismstandard.org
>
> Cheers, Robin.
>
> Robin Taylor
> Main Library
> University of Edinburgh
> Tel. 0131 6513808
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
>
> --
> Centralized Desktop Delivery: Dell and VMware Reference Architecture
> Simplifying enterprise desktop deployment and management using
> Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
> client virtualization framework. Read more!
> http://p.sf.net/sfu/dell-eql-dev2dev
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
--
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] postgres/postmaster 99%

[Graham Triggs]

Index update will only make corrections to the values written to the browse
tables and Lucene indexes - index-init recreates the table and index
definitions in Postgres.

Can you run the query with EXPLAIN ANALYZE in front of it (substitute an
actual value from sort_value in the bi_4_dis table in place of the bind
variable)? That will tell you what Postgres is actually doing.

G

On 11 November 2010 01:30, Steve Swinsburg wrote:

> Hi Graham,
>
> We have ~48000 records in the item table, and ~263,000 in the bi_4_dmap
> table. We've recreated the indexes (only via index-update though, should we
> run index-init?) and things went ok for a while, but now back up to 99% and
> the same query appears in pg_stat_activity.
>
> We are running Postgres 8.1.18.
>
> cheers,
> Steve
>
>
>
>
>
> On 11/11/2010, at 11:10 AM, Graham Triggs wrote:
>
> It's a second level browse - ie. if your 4th browse index is 'subject',
> then it's someone looking at all the items that have a particular subject
> entry.
>
> This shouldn't be that expensive a query (given there is no offset
> involved), but you may have an issue with an index missing (although the
> browse code should create all the ones that it needs when it creates the
> table), or more likely you have too low a value for your shared_buffers
> (required to load the indexes), or work_mem (used for the join between the
> tables). Additionally, you may need to analyze and/or reindex the tables.
>
> How many items are in the repository, and how many values do you have in
> the 4th browse option (subject?). And what version of Postgres are you
> running?
>
> G
>
> On 10 November 2010 22:42, Steve Swinsburg wrote:
>
>> A followup:
>>
>> I restarted postgres and within minutes the same query has appeared in the
>> stats and CPU is back up to 99%. Could we be missing some indexes or
>> something? We only recently ran the filter-media script and generated
>> thousands of thumbnails and branded previews but have since performed a
>> vacuum.
>>
>> Any information would be much appreciated.
>>
>> cheers,
>> Steve
>>
>> On 11/11/2010, at 9:12 AM, Steve Swinsburg wrote:
>>
>> Hi all,
>>
>> We are experiencing an issue on both of our dspace instances where
>> postmaster spins up 99% of the CPU. Sometimes it's just one process at 99%,
>> othertimes its a dozen or more processes around 7-9% each. I ran some stats
>> on postgres via:
>>
>> select * from pg_stat_activity
>>
>> I found this query in the output about 15 times:
>>
>> SELECT bi_item.* FROM bi_item, (SELECT bi_4_dmap.item_id FROM bi_4_dmap,
>> bi_4_dis WHERE bi_4_dmap.distinct_id=bi_4_dis.id AND
>> bi_4_dis.sort_value=$1 ) mappings  WHERE  bi_item.item_id=mappings.item_id
>> ORDER BY sort_3 ASC  LIMIT $2
>>
>> The earliest has a start time of about 5 hours ago.
>>
>> Anyone know whats up?
>>
>> cheers,
>> Steve
>>
>>
>>
>>
>> --
>> Centralized Desktop Delivery: Dell and VMware Reference Architecture
>> Simplifying enterprise desktop deployment and management using
>> Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
>> client virtualization framework. Read more!
>> http://p.sf.net/sfu/dell-eql-dev2dev
>> ___
>> DSpace-tech mailing list
>> DSpace-tech@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>
>>
>
>
--
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] postgres/postmaster 99%

[Graham Triggs]

Steve,

I've been trying again on my machine. Postgres 8.4 seemed to be quite slow /
expensive (can't recall the numbers precisely - it was the other day). I
have dumped the data and loaded it into Postgres 9.0, and the results are
interesting.

Just after restoring the database, the same query (for the first 10 records)
took about 6 seconds to execute (with a lot of CPU usage). After vacuuming
bi_4_dmap and bi_item (pgadmin recommended both be vacuumed), the query took
3 seconds.

This is with 379,900 records in the item / bi_item tables, and 4,775,227
records in bi_4_dmap.

I would recommend you vacuum and analyze the tables. And possibly consider
upgrading the version of Postgres.

G

On 11 November 2010 01:30, Steve Swinsburg wrote:

> Hi Graham,
>
> We have ~48000 records in the item table, and ~263,000 in the bi_4_dmap
> table. We've recreated the indexes (only via index-update though, should we
> run index-init?) and things went ok for a while, but now back up to 99% and
> the same query appears in pg_stat_activity.
>
> We are running Postgres 8.1.18.
>
> cheers,
> Steve
>
>
>
>
>
> On 11/11/2010, at 11:10 AM, Graham Triggs wrote:
>
> It's a second level browse - ie. if your 4th browse index is 'subject',
> then it's someone looking at all the items that have a particular subject
> entry.
>
> This shouldn't be that expensive a query (given there is no offset
> involved), but you may have an issue with an index missing (although the
> browse code should create all the ones that it needs when it creates the
> table), or more likely you have too low a value for your shared_buffers
> (required to load the indexes), or work_mem (used for the join between the
> tables). Additionally, you may need to analyze and/or reindex the tables.
>
> How many items are in the repository, and how many values do you have in
> the 4th browse option (subject?). And what version of Postgres are you
> running?
>
> G
>
> On 10 November 2010 22:42, Steve Swinsburg wrote:
>
>> A followup:
>>
>> I restarted postgres and within minutes the same query has appeared in the
>> stats and CPU is back up to 99%. Could we be missing some indexes or
>> something? We only recently ran the filter-media script and generated
>> thousands of thumbnails and branded previews but have since performed a
>> vacuum.
>>
>> Any information would be much appreciated.
>>
>> cheers,
>> Steve
>>
>> On 11/11/2010, at 9:12 AM, Steve Swinsburg wrote:
>>
>> Hi all,
>>
>> We are experiencing an issue on both of our dspace instances where
>> postmaster spins up 99% of the CPU. Sometimes it's just one process at 99%,
>> othertimes its a dozen or more processes around 7-9% each. I ran some stats
>> on postgres via:
>>
>> select * from pg_stat_activity
>>
>> I found this query in the output about 15 times:
>>
>> SELECT bi_item.* FROM bi_item, (SELECT bi_4_dmap.item_id FROM bi_4_dmap,
>> bi_4_dis WHERE bi_4_dmap.distinct_id=bi_4_dis.id AND
>> bi_4_dis.sort_value=$1 ) mappings  WHERE  bi_item.item_id=mappings.item_id
>> ORDER BY sort_3 ASC  LIMIT $2
>>
>> The earliest has a start time of about 5 hours ago.
>>
>> Anyone know whats up?
>>
>> cheers,
>> Steve
>>
>>
>>
>>
>> --
>> Centralized Desktop Delivery: Dell and VMware Reference Architecture
>> Simplifying enterprise desktop deployment and management using
>> Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
>> client virtualization framework. Read more!
>> http://p.sf.net/sfu/dell-eql-dev2dev
>> ___
>> DSpace-tech mailing list
>> DSpace-tech@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>
>>
>
>
--
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Prism Metadata for DSpace

[Graham Triggs]

That's probably because a lot of the examples of Prism in practice relate to
online publications - which don't have the same pagination requirements.

If you have a look at Nature - for example:
http://www.nature.com/nri/journal/v10/n11/full/nri2858.html - and view the
source of that document, you can see that they have embedded Prism metadata
with an ending page.

G

On 14 November 2010 20:25, Leonie Hayes  wrote:

> We would welcome the addition of a metadata schema that deals with full
> elements of Journal Article and Conference items. There are a number of us
> in the community who found working our way around the missing elements in
> Dublin Core a frustration.
>
> Some of the examples of Prism schema I looked at don't contain ending page,
> perhaps an oversight, so other examples would be helpful.
>
> We ended up adding our own mini-schema, but an out of the box DSpace
> containing all the elements based on an existing standard - very attractive!
>
> Other benefits -  take advantage of Open URL functionality, create a
> citation, export citations, and Zotero capture.
>
> When you do not store these elements separately, export to EndNote, BibTeX
> etc, and creating the various citation styles (APA, HARVARD etc), are much
> more difficult to implement.
>
>
> Leonie Hayes
> Research Repositories Manager
> http://www.library.auckland.ac.nz/contacts/?stid=124
> http://researchspace.auckland.ac.nz
>
>
>
>
>
>
> --
> Centralized Desktop Delivery: Dell and VMware Reference Architecture
> Simplifying enterprise desktop deployment and management using
> Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
> client virtualization framework. Read more!
> http://p.sf.net/sfu/dell-eql-dev2dev
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
--
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] strange effect with dspace-1.7.0

[Graham Triggs]

That doesn't entirely surprise me - I've seen odd things with 2.0.x, and it
all gets tied up with the overlay war capabilities that were introduced to
the war plugin during that release cycle.

As Maven 2.2.0 was released back in July(?) 2009, I don't think it's
entirely unreasonable to just deprecate the use of 2.0.x and focus on 2.2.0
and above.

G

On 21 December 2010 17:07, Tim Donohue  wrote:

> Hi Jochen,
>
> It looks like we have a small bug somewhere.  I've verified that
> building DSpace using Maven 2.0.x doesn't seem to work 100% properly.
>
> Essentially, everything doesn't get copied to
> [dspace-src]/dspace/target/dspace-1.7.0-build/webapps/ properly, which
> causes the fresh_install issues you've seen.
>
> The temporary fix is one of the following:
>
> (1) Either install Maven 2.2.1 (which works fine)
>
> OR
>
> (2) Manually copy the webapps from their
> [dspace-src]/dspace/modules/[webapp-name]/target/ directory.  So, for
> example, the XMLUI webapp is built properly to the path:
>
> [dspace-src]/dspace/modules/xmlui/target/xmlui-1.7.0/
>
> However, it doesn't get copied over to the "webapps" directory properly
> (which is something Maven is supposed to do for you, but it doesn't seem
> to be working).  So, if you manually perform the copy, everything should
> work fine.
>
> I'm digging around to see if I can figure out a better fix to this issue.
>
> - Tim
>
> On 12/21/2010 5:25 AM, Jochen Lienhard wrote:
> > Hi,
> >
> > I installed dspace-1.7.0 at two different Linux distribution:
> >
> > 1) openSuSE 11.3 (Apache Maven 2.2.1)
> > 2) openSuSE 11.1 (Maven version: 2.0.9)
> >
> > in case 1) all is perfect.
> > in case 2) in the webapps directory all files are in one directory ...
> > even a strange effect in the numbers of the files:
> >
> > 1)
> > fresh_install:
> > [copy] Copying 8 files to /opt/dspace-1.7.0/bin
> > [copy] Copying 71 files to /opt/dspace-1.7.0/lib
> > [copy] Copying 21 files to /opt/dspace-1.7.0/etc
> > [copy] Copying 27 files to /opt/dspace-1.7.0/solr
> >
> > copy_webapps:
> > [copy] Copying 967 files to /opt/dspace-1.7.0/webapps
> > [copy] Copied 130 empty directories to 6 empty directories under
> > /opt/dspace-1.7.0/webapps
> > [copy] Copying 6 files to /opt/dspace-1.7.0/webapps
> >
> > 2)
> > fresh_install:
> > [copy] Copying 8 files to /opt/dspace-1.7.0/bin
> > [copy] Copying 70 files to /opt/dspace-1.7.0/lib
> > [copy] Copying 21 files to /opt/dspace-1.7.0/etc
> > [copy] Copying 27 files to /opt/dspace-1.7.0/solr
> >
> > copy_webapps:
> > [copy] Copying 668 files to /opt/dspace-1.7.0/webapps
> > [copy] Copied 90 empty directories to 1 empty directory under
> > /opt/dspace-1.7.0/webapps
> > [copy] Copying 1 file to /opt/dspace-1.7.0/webapps
> >
> > Does anyone know, which could be the problem?
> >
> > Greetings
> >
> > Jochen
> >
> > P.S.
> > If I copy the xmlui folder from 1) to the webapps folder of 2) ... it
> > works . very strange
> >
> >
> >
> >
> --
> > Lotusphere 2011
> > Register now for Lotusphere 2011 and learn how
> > to connect the dots, take your collaborative environment
> > to the next level, and enter the era of Social Business.
> > http://p.sf.net/sfu/lotusphere-d2d
> >
> >
> >
> > ___
> > DSpace-tech mailing list
> > DSpace-tech@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
> --
> Forrester recently released a report on the Return on Investment (ROI) of
> Google Apps. They found a 300% ROI, 38%-56% cost savings, and break-even
> within 7 months.  Over 3 million businesses have gone Google with Google
> Apps:
> an online email calendar, and document program that's accessible from your
> browser. Read the Forrester report: http://p.sf.net/sfu/googleapps-sfnew
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
--
Forrester recently released a report on the Return on Investment (ROI) of
Google Apps. They found a 300% ROI, 38%-56% cost savings, and break-even
within 7 months.  Over 3 million businesses have gone Google with Google Apps:
an online email calendar, and document program that's accessible from your 
browser. Read the Forrester report: http://p.sf.net/sfu/googleapps-sfnew___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Search

[Graham Triggs]

That sounds plausible. Can you find the number if you phrase search the two
parts - ie, "45 858"?

On 4 January 2011 15:14, Marcelo Henrique Gomes wrote:

> Hi Graham,
>
> I run filter-media and index-update every day and I find others terms in
> the same document.
>
> I guess the problem is in the mask of number ##.### because the "." can be
> a delimiter line.
>
> What do you thing?
>
> If anyone wants to know my DSpace is
> http://www.iof.mg.gov.br/ultima-edicao.html and send comments and
> suggestions. I'll be very happy.
>
> Regards,
> Marcelo
>
> On Tue, Jan 4, 2011 at 12:49 PM, Graham Triggs wrote:
>
>> Hi,
>>
>> There are all sorts of possibilities for this. Is the term in the metadata
>> (you need to ensure the metadata field is in the search configuration), or
>> in a document file?
>>
>> Have you run filter-media?
>>
>> There is a limit to the number of terms in a single field, could this be
>> an issue? (see documentation for configuration option:
>> search.maxfieldlength).
>>
>> Finally, can you find other terms in the same document?
>>
>> If you can find other terms from the same document, but not the numbers,
>> then this would indicate it's the behaviour of BrazilianAnalyzer that is
>> causing the issue, for which you may get better responses directly from the
>> Lucene community.
>>
>> Regards,
>> G
>>
>> On 4 January 2011 11:44, Marcelo Henrique Gomes wrote:
>>
>>> Hello,
>>>
>>> In my files, I have numbers (45.858) and when I try to search for this
>>> number I don't find.
>>>
>>> I used BrazilianAnalyzer.
>>>
>>> search.analyzer = org.apache.lucene.analysis.br.BrazilianAnalyzer
>>>
>>> What can I do?
>>>
>>> Regards,
>>> Marcelo
>>>
>>>
>>> --
>>> Learn how Oracle Real Application Clusters (RAC) One Node allows
>>> customers
>>> to consolidate database storage, standardize their database environment,
>>> and,
>>> should the need arise, upgrade to a full multi-node Oracle RAC database
>>> without downtime or disruption
>>> http://p.sf.net/sfu/oracle-sfdevnl
>>> ___
>>> DSpace-tech mailing list
>>> DSpace-tech@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>>
>>>
>>
>
--
Learn how Oracle Real Application Clusters (RAC) One Node allows customers
to consolidate database storage, standardize their database environment, and, 
should the need arise, upgrade to a full multi-node Oracle RAC database 
without downtime or disruption
http://p.sf.net/sfu/oracle-sfdevnl___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] browse index config changes not reflecting in nav panel (1.6.2)

[Graham Triggs]

Which UI are you using - JSP or XML?

If it's XMLUI, then it sounds like it might be caching elements of the
Cocoon pipeline, in which case you should stop the application and remove
the cache from within the web application directory, and/or disable Cocoon
caching.


On 19 January 2011 15:03, Fred Edwards  wrote:

>
> as a test, I've recently added a browse index to our dspace and initially
> it correctly added the items to the nav. panel...
>
> however, after removing the browse index in the config file the nav panel
> is now doing weird things.
>
> at the top level it is retaining menu items for browse that is no longer
> there and clicking leads to an error page obviously...
>
> go down a level or two into communities and collections you start finding
> the nav panel in some areas do not show these links, others do...
>
> I've recompiled countless times... re-indexed countless times...
> re-initialized the index countless times...
>
> the last go I re-added the browse indexes and still the same situation...
> in some places on... in others off...
>
> we have a quasi live collection or two in the server, and no test server
> currently, so we'd rather not have to completely rebuild...
>
> I'm also loath to:  ant fresh_install   without getting some feedback from
> the community on how safe this would be...
>
>
>
>
>
>
> Fred Edwards
>
> Systems Technician
> Patrick Power Library
> Saint Mary's University
> Halifax, Nova ScotiaB3H 3C3
>
> Phone: (902) 420-5096
> Fax:   (902) 420-5561
> E-mail:fred.edwa...@smu.ca
> Website:   http://www.smu.ca/library/
>
> --
>
>
> --
> Protect Your Site and Customers from Malware Attacks
> Learn about various malware tactics and how to avoid them. Understand
> malware threats, the impact they can have on your business, and how you
> can protect your company and customers by using code signing.
> http://p.sf.net/sfu/oracle-sfdevnl
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
--
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand 
malware threats, the impact they can have on your business, and how you 
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Lots of browse errors

[Graham Triggs]

Jeffrey, I agree with Claudia that the request itself looks odd. I would
want to look at / find out what was generating that link in the first place.

But at the same time, the code ought to be able to handle with this more
gracefully (ie. know that it's an item index and ignore the value
parameter).

Please log as a bug - you can assign it to me if you like.

G

On 27 January 2011 17:19, Claudia Juergen  wrote:

> Hello Jeffrey,
>
> did you experiment with the title before and google got old stuff indexed?
> Your title index atm seems ok.
>
> Claudia
>
> > Claudia,
> >
> > The default is what came with dspace.cfg :
> >
> > webui.browse.index.1 = dateissued:item:dateissued
> > webui.browse.index.2 = author:metadata:dc.contributor.*,dc.creator:text
> > webui.browse.index.3 = title:item:title
> > webui.browse.index.4 = subject:metadata:dc.subject.*:text
> >
> > Something is just wacked if you ask me.
> >
> >
> > On Jan 27, 2011, at 10:46 AM, Claudia Jürgen wrote:
> >
> >> Hi Jeffrey,
> >>
> >> should have taken a close look,
> >>
> >>
> http://digital.maag.ysu.edu:8080/jspui/browse?type=title&order=ASC&rpp=20&value=Sigma+Gamma+clean-up+project
> .
> >>
> >> value is only used for metadata browse indices not item indices, they
> >> got an offset for paging. Title is usually an item index, seems as if
> >> you changed it to metadata, so it is not a default browse setting?
> >>
> >> Hope that helps
> >>
> >> Claudia Jürgen
> >>
> >> Am 27.01.2011 16:30, schrieb Jeffrey Trimble:
> >>> I did run it...and actually did it a second time--this morning.
> >>> Nothing, zilch.
> >>>
> >>> Quite yucky if you ask me.
> >>>
> >>> Thanks,
> >>>
> >>> On Jan 27, 2011, at 10:21 AM, Claudia Jürgen wrote:
> >>>
>  Hi Jeffrey,
> 
>  did you run index-init, when setting up this instance?
> 
>  Hope that helps
> 
>  Claudia Jürgen
> 
> 
>  Am 27.01.2011 16:07, schrieb Jeffrey Trimble:
> > I'm getting lots of browse errors.  (Like 200 within 14 minutes)  I
> > don't see but two index tables showing up
> > in my postgresql.  Also, the default browse config has been
> > implemented (nothing fancy!)  (Yeah, I know
> > its a google-bot).
> >
> >
> > Here's a sample:
> >
> > Date:   1/27/11 10:04 AM
> > Session ID: 9FB39CC6A53C23EC1A80EDCA2E76E847
> > User:   Anonymous
> > IP address: 66.249.71.110
> >
> > -- URL Was:
> >
> http://digital.maag.ysu.edu:8080/jspui/browse?type=title&order=ASC&rpp=20&value=Sigma+Gamma+clean-up+project
> .
> > -- Method: GET
> > -- Parameters were:
> > -- value: "Sigma Gamma clean-up project."
> > -- rpp: "20"
> > -- type: "title"
> > -- order: "ASC"
> >
> >
> > Exception:
> > javax.servlet.ServletException: org.dspace.browse.BrowseException:
> > org.postgresql.util.PSQLException: ERROR: relation "bi_3_dmap" does
> > not exist
> >  Position: 79
> >   at
> >
> org.dspace.app.webui.servlet.AbstractBrowserServlet.processBrowse(AbstractBrowserServlet.java:371)
> >   at
> >
> org.dspace.app.webui.servlet.BrowserServlet.doDSGet(BrowserServlet.java:112)
> >   at
> >
> org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.java:151)
> >   at
> >
> org.dspace.app.webui.servlet.DSpaceServlet.doGet(DSpaceServlet.java:99)
> >   at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> >   at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> >   at
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
> >   at
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> >   at
> >
> org.dspace.utils.servlet.DSpaceWebappServletFilter.doFilter(DSpaceWebappServletFilter.java:112)
> >   at
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> >   at
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> >   at
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> >   at
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> >   at
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
> >   at
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> >   at
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> >   at
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> >   at
> >
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
> >   at
> >
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
> >   at
> >
> org.apache.tomcat.util.n

Re: [Dspace-tech] Lost Indexes

[Graham Triggs]

Herbert,

You need to work back through the stack trace. First thing you'll notice is
that the BrowseException that is thrown in updateCommunityMappings is a
wrapping of a caught SQLException.

Following the stack trace of the SQLException ('caused by' in your output),
you'll go through the Oracle libraries, through the DatabaseManager and out
into


org.dspace.browse.BrowseCreateDAOOracle.getAllCommunityIDs(BrowseCreateDAOOracle.java:962)

The only SQL directly issued within the getAllCommunityIDs method is against
Community2Item - a view that is comprised of a join between
Community2Collection and Collection2Item.

If you look at the update script for the database in version 1.6, you'll see
that both of these tables (Com2Col and Col2Item) have been ALTERed. This
will have invalidated the Community2Item view.

Go to the database and recompile any invalidated objects - eg. ALTER VIEW
Community2Item COMPILE; - and it will fix the problem.

Regards,
G

On 1 February 2011 17:04, Herbert Lensch wrote:

> DSpace-Tech:
>
> One of the users of our DSpace 1.6.0 site noticed that the indexes are
> no longer searchable from the homepage, and worse there are no items listed
> in the entire site.
>
> When I try to run a reindex on the site, I get the following error:
>
> [root@dspaceprod run]# /dspace/run/bin/dsrun org.dspace.browse.IndexBrowse
>> -f -r -v
>
> Creating browse indexes for DSpace
>
> init complete (0 ms)
>
> Preparing browse tables
>
> Deleting old indices
>
> Checking for bi_1
>
> Checking for bi_1_dis
>
> ... doesn't exist; but will carry on as there may be something that
>> conflicts
>
> Checking for bi_2
>
> Checking for bi_2_dis
>
> ...found
>
> Deleting old index and associated resources: bi_2_dis
>
> Checking for bi_3
>
> Checking for bi_3_dis
>
> ... doesn't exist; but will carry on as there may be something that
>> conflicts
>
> Checking for bi_4
>
> Checking for bi_4_dis
>
> ...found
>
> Deleting old index and associated resources: bi_4_dis
>
> Checking for bi_5
>
> Checking for bi_5_dis
>
> ... doesn't exist; but will carry on as there may be something that
>> conflicts
>
> Checking for bi_6
>
> Checking for bi_6_dis
>
> ... doesn't exist; but will carry on as there may be something that
>> conflicts
>
> Checking for bi_7
>
> Checking for bi_7_dis
>
> ... doesn't exist; but will carry on as there may be something that
>> conflicts
>
> Checking for bi_8
>
> Checking for bi_8_dis
>
> ... doesn't exist; but will carry on as there may be something that
>> conflicts
>
> Checking for bi_9
>
> Checking for bi_9_dis
>
> ... doesn't exist; but will carry on as there may be something that
>> conflicts
>
> Checking for bi_10
>
> Checking for bi_10_dis
>
> ... doesn't exist; no more tables to delete
>
> Creating browse index dateissued: index by null sortable by:  dc.title
>>  dc.date.accessioned  dc.date.issued
>
> Creating browse index author: index by dc.contributor.*,dc.creator sortable
>> by:  dc.title  dc.date.accessioned  dc.date.issued
>
>  Creating browse index title: index by null sortable by:  dc.title
>>  dc.date.accessioned  dc.date.issued
>
> Creating browse index subject: index by dc.subject.* sortable by:  dc.title
>>  dc.date.accessioned  dc.date.issued
>
> Browse tables prepared
>
> tables prepped (1089 ms, 1089 ms)
>
> Exception: java.sql.SQLException: ORA-00942: table or view does not exist
>
>
>> org.dspace.browse.BrowseException: java.sql.SQLException: ORA-00942: table
>> or view does not exist
>
>
>>  at
> org.dspace.browse.BrowseCreateDAOOracle.updateCommunityMappings(BrowseCreateDAOOracle.java:649)
>
>>  at org.dspace.browse.IndexBrowse.indexItem(IndexBrowse.java:402)
>
>>  at org.dspace.browse.IndexBrowse.createIndex(IndexBrowse.java:1148)
>
>>  at org.dspace.browse.IndexBrowse.initBrowse(IndexBrowse.java:1098)
>
>>  at org.dspace.browse.IndexBrowse.main(IndexBrowse.java:749)
>
>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>>  at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
>>  at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
>>  at java.lang.reflect.Method.invoke(Method.java:616)
>
>>  at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:212)
>
>> Caused by: java.sql.SQLException: ORA-00942: table or view does not exist
>
>
>>  at
> oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
>
>>  at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:331)
>
>>  at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:288)
>
>>  at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:745)
>
>>  at
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:219)
>
>>  at
> oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:813)
>
>>  at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1049)
>
>>  at
> oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDesc

Re: [Dspace-tech] Re bugs in the Oracle implementation

[Graham Triggs]

Tim,

There is absolutely no requirement to have an Oracle license for a
testathon. The Oracle XE database is easy to install, works exactly the same
as the full licensed copy, and we'll get nowhere near the limits in data
sizes for the purposes of a testathon.

I'm quite happy to assist with implementing this on Duraspace provided
servers.

G

On 2 May 2011 23:05, Tim Donohue  wrote:

> Hi Hardy & Richard,
>
> I'll readily admit that our Oracle support has been more difficult
> recently. The primary reason for this is that the Committer who was
> leading DSpace's Oracle support has moved on to a different role and no
> longer works with DSpace. So, at this point in time, we have no (zero)
> Committers who actively use Oracle in their workplace (at least none
> that I'm aware of, unless someone has switched recently).
>
> Obviously, it is a concern to us that we no longer have anyone who is
> actively using Oracle. It also means we become much more dependent on
> the community members who use Oracle to report issues or problems
> (especially during Testathons & beta testing).
>
> The Committers do still attempt to do some basic Oracle testing before
> releases. But, sometimes this testing may not be as thorough as that
> provided by community members (as we may accidentally overlook testing
> of a particular feature, e.g. editing or adding e-people via XMLUI as
> Hardy reported in DS-841). Again, a large part of this is that none of
> our Committers run Oracle on a day-to-day basis, and we're all
> volunteers with a limited amount of time to devote to testing.
>
> So, to help ensure we are supporting Oracle fully, it'd be wonderful if
> those who use Oracle (and want to ensure it remains supported) can help
> out. Here's a few ways you can help:
>
> (1) Offer to install a Testathon DSpace instance backed by Oracle
> during our next Testathon. Currently, 'http://demo.dspace.org' (the
> primary Testathon server) is running on PostgreSQL as DuraSpace doesn't
> have an Oracle license, and we don't have an Oracle DB expert on staff.
>
> (2) Offer to download & test an Oracle-based DSpace install on your
> local servers during next Testathon/beta-testing & report back any
> issues you find before the final release
>
> (3) If you want, you can join the 'dspace-changelog' listserv to keep
> track of whether changes look to be Oracle specific or not.
> Alternatively, you could also just help test the latest Trunk code
> against Oracle every once in a while (and report back what you find).
>
> During development, the Committers are trying to ensure all
> patches/changes will work with Oracle (but obviously sometimes things
> slip through). So, in most cases, just testing the latest SVN Trunk code
> every once in a while should be good enough to locate any potential
> problems with the upcoming release.
>
> RE: 1.7.2 Release
>
> I'll talk to the Committers around whether we can push out a very minor
> 1.7.2 update just to fix DS-841. Are there any other known Oracle issues
> in DSpace 1.7.0/1.7.1 we should be resolving at the same time? Or is the
> only remaining Oracle issue the one described in DS-841?
>
> NOTE: At this point in time, the DSpace Committers actually were not
> planning a full 1.7.2 release to resolve any other issues.  We had
> discussed a 1.7.2. But, we had tentatively decided against it, as we
> have 1.8.0 coming up in October and most of us are actively working hard
> on features to release in 1.8.0.
>
> *However*, we should be making sure that Oracle is *fully* supported in
> a 1.7.x release series (I know there were other Oracle issues we already
> fixed between 1.7.0 and 1.7.1). So, I would be willing to recommend a
> very small 1.7.2 release *just* to fix the remaining Oracle issue(s).
>
> So, Hardy & Richard, if you have found any other issues with 1.7.1, it'd
> be good to know so that we can release a working Oracle 1.7.x version in
> the near future.
>
> - Tim
>
>
> On 5/2/2011 12:33 PM, Pottinger, Hardy J. wrote:
> > Hi, Richard, I think what needs to happen is someone from our community
> of
> > Oracle installations needs to commit an Oracle-backed installation to the
> > next Testathon, to catch items like this. I am also going to try to do
> > more testing along the way. I think what we need to do as a community is
> > to be especially vigilant for patches that touch the classes that
> interact
> > with the database in any way, as this area of DSpace is particularly
> > important to our use of DSpace.
> >
> > I'd like to reach out to the more-experienced developers on this list: is
> > there a good way, besides "paying more attention" to follow the commits
> > and watch out for changes to the database classes? Do I need to just join
> > the commit mail list and set up a a filter in my mail client? Or is there
> > some better way to keep on top of these things?
> >
> > Also, Richard, this a very minor detail, but I am the reporter for
> DS-841,
> > Peter was kind enough to add a comment

Re: [Dspace-tech] Conflict with mixing metadata schemas and batch ingest

[Graham Triggs]

On 9 May 2011 09:31, Robin Taylor  wrote:

> > 2) While you can mix metadata elements for an item from multiple
> > schemas in manual submission, for the batch ingest you can only
> > specify a single schema for the batch.
>
> I haven't used this in recent times but I thought you could mix schema.
> I know there are lots of references to the 'Dublin Core metadata' in the
> code but I thought that was just a hangover from the days when only
> Dublin Core was used, and in fact you could mix metadata schema. have
> you tried this out ?
>
>
IIRC, this isn't particularly well documented. But it is possible to create
import packages using multiple metadata schemas.

The trick is to have separate metadata files for each schema. 'dc' goes into
the standard dublin_core.xml, But for any other schema, you need to create a
file called:

metadata_.xml

The contents of which is structured just like dublin_core.xml, except on the
 element, you need to include a schema="" attribute.

G
--
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network 
management toolset available today.  Delivers lowest initial 
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Conflict with mixing metadata schemas and batch ingest

[Graham Triggs]

On 10 May 2011 16:44, Mark H. Wood  wrote:

>
> http://www.dspace.org/1_7_0Documentation/System%20Administration.html#SystemAdministration-Configuringmetadata\prefix\.xmlforDifferentSchema
>
> DS-534 addressed the documentation of this feature.  How could we
> improve it further?
>
>
I said IIRC!! I didn't actually look at the current docs :)
--
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] NPE error in searching after an upgrade to 1.7.1 (plus patches)

[Graham Triggs]

Hi,

There is an error in the 1.7.1 source. Lines 170 & 171 of

org.dspace.app.xmlui.aspect.artifactbrowser.AbstractSearch

need to be moved so that they come after the call to performSearch().
These two lines are expecting the queryResults object to be set, but
that won't happen until after the search has been executed.


Regards,

G


On 15 May 2011 20:50, Pottinger, Hardy J.  wrote:

> Hi, I'm in a pickle, our upgrade to 1.7.1 in production is not working out
> so well, we can browse OK, but searching is throwing NPE errors, I'm looking
> for advice to troubleshoot.
>
> Here's a link that is currently throwing an NPE:
> https://mospace.umsystem.edu/xmlui/search?query=test&submit=Go
>
> And of course, we didn't see anything like this in our testing. Both
> production and dev boxes are running the same toolchain on RHEL5. Very
> similar hardware.
>
> Have done the normal upgrade steps (have rebuilt the index a few times,
> restarted everything a few times, cleared the cocoon cache).
>
> Thanks for any pointers you can provide, I really appreciate it.
>
> --Hardy
>
> --
> Achieve unprecedented app performance and reliability
> What every C/C++ and Fortran developer should know.
> Learn how Intel has extended the reach of its next-generation tools
> to help boost performance applications - inlcuding clusters.
> http://p.sf.net/sfu/intel-dev2devmay
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
--
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Problem with ordering in browsing

[Graham Triggs]

Please take a look at a previous post of mine on this subject:

http://dspace.2283337.n4.nabble.com/Browse-UTF-8-and-sorting-in-1-5-tp3281449p3281450.html

Regards,
G

On 19 May 2011 15:18, Peter Dietz  wrote:

> Hi Ladislav,
>
> I've noticed that our librarians here are happier with sorting when we use
> the collate of C as opposed to utf8/en_US.
>
> postgres=# create database "dspace" with owner = dspace encoding='utf8' 
> tablespace=pg_default lc_collate = 'C' lc_ctype='en_US.UTF-8' template 
> template0;
>
>
> I've add these three authors to a test collection that had some sample data
> in it, and it has the results you were expecting:
> == Author Name ==
> Cabanová, Zuzana
> Cablová, Barbora
> creatorlast, creatorfirst
> Čabla, Michael
>
>
>
>
> Peter Dietz
>
>
>
>
> On Thu, May 19, 2011 at 4:41 AM, Ladislav Kulhanek <
> ladislav.kulha...@vsb.cz> wrote:
>
>> Hello everybody.
>>
>> We have data in our DSpace in czech language (code "cs" in accordance
>> with ISO 639-1) and we have a problem with order in browsing by
>> author, titles and subjects (order in search results is correct).
>> There are letters with diacritic in czech alphabet, for example "Č"
>> (0x010C code in unicode). This letter should be ordered between "C"
>> and "D", but in DSpace it is ordered to the same place as "C". For
>> example we have ordered list as
>>
>> Cabanová, Zuzana
>> Čabla, Michael
>> Cablová, Barbora
>>
>> and this list should be
>>
>> Cabanová, Zuzana
>> Cablová, Barbora
>> Čabla, Michael
>>
>> And czech alphabet contains letter "Ch" (it consists from two
>> characters). This letter should be ordered between "h" and "i". This
>> letter is ordered in DSpace correctly. So it looks like DSpace order
>> in accordance with czech alphabet, but ignore diacritics.
>> We have DSpace 1.7.1, Manakin, db PostgreSQL 8.4 (database has
>> Collation and Ctype set as cs_CZ.UTF-8), and in tomcat connector is
>> URIEncoding="UTF-8". Any idea how to solve it? Thanks.
>>
>> Ladislav Kulhanek
>>
>>
>> --
>> What Every C/C++ and Fortran developer Should Know!
>> Read this article and learn how Intel has extended the reach of its
>> next-generation tools to help Windows* and Linux* C/C++ and Fortran
>> developers boost performance applications - including clusters.
>> http://p.sf.net/sfu/intel-dev2devmay
>> ___
>> DSpace-tech mailing list
>> DSpace-tech@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>
>
>
>
> --
> What Every C/C++ and Fortran developer Should Know!
> Read this article and learn how Intel has extended the reach of its
> next-generation tools to help Windows* and Linux* C/C++ and Fortran
> developers boost performance applications - including clusters.
> http://p.sf.net/sfu/intel-dev2devmay
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
--
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its 
next-generation tools to help Windows* and Linux* C/C++ and Fortran 
developers boost performance applications - including clusters. 
http://p.sf.net/sfu/intel-dev2devmay___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] The file "update-discovery-index" not found...

[Graham Triggs]

Hi,

All of the batch scripts have been replaced by a launcher. Typically, you
would replace:

bin/

Re: [Dspace-tech] The file "update-discovery-index" not found...

[Graham Triggs]

Hi Sergiu,

The dspace launcher does not use other scripts within the bin directory, so
there presence or lack is not the issue here.

Commands for the launcher are configured in the config/launcher.xml - the
current version is here:

http://scm.dspace.org/svn/repo/dspace/trunk/dspace/config/launcher.xml

As you can see, it has an 'update-discovery-index' command (configured to
use the org.dspace.discovery.IndexClient class)

Regards,
G

On 9 June 2011 14:33, Sergiu Ioan Irimia  wrote:

> Mr. Graham Triggs,
>
> I know how to use the dspace launcher:
>
> -server:Windows 2003
>
> - run cmd
>
> - cd /d D:\DSpace\bin
>
> - D:\DSpace\bin>dspace update-discovery-index
>
> - error: command not found: update-discovery-index
>
> because the file "update-discovery-index" is not in D:\DSpace\bin and I
> can't seem to find it anywhere...
>
> So I need the file update-discovery-index so that the dspace launcher shall
> have what to launch...
>
> I've uploaded a jpg... maybe I failed to explain the problem...
>
> Or maybe I failed to understand your solution...
>
> Thank you for your understanding and for your time...
>
> --- On *Thu, 6/9/11, Graham Triggs * wrote:
>
>
> From: Graham Triggs 
> Subject: Re: [Dspace-tech] The file "update-discovery-index" not found...
> To: "Sergiu Ioan Irimia" 
> Cc: dspace-tech@lists.sourceforge.net
> Date: Thursday, June 9, 2011, 12:18 PM
>
>
> Hi,
>
> All of the batch scripts have been replaced by a launcher. Typically, you
> would replace:
>
> bin/

Re: [Dspace-tech] Tomcat vs Resin (vs other containers)

[Graham Triggs]

On Fri, 2007-03-09 at 11:01 +, Richard Jones wrote:
> Tomcat has always, IMO, been mildly unstable, and we've been having some 
> problems recently that have encouraged me to look at other containers. 

Any specific issues that you can / care to share?

> It was alleged on some site or other (i.e. I googled 
> for 5 minutes, and that was what I found - not a scientific approach) 
> that Tomcat and Resin are the fastest of the containers (which is a 
> significant criteria).

Similar research that I've conducted hasn't been quite so favourable for
Tomcat. Then again, it may also depend on what criteria you are using -
vanilla Tomcat doesn't tend to scale to well with multiple connections.
In such cases, you want to look at something that either has NIO (Jetty,
Grizzly - the refitted Tomcat that is part of Glassfish), or use the APR
with Tomcat (but then if you are having issues with Tomcat, I can't see
that they are going to be helped much by including native code!)

>   Any other experiences that people have had, with 
> Jetty or other, or any negative experiences with Resin would be greatly 
> appreciated.

I've tried using Jetty for some of our projects, but not really DSpace
(/ Open Repository). Unfortunately, our own in-house code has a few
dependencies on the way that we have configured Tomcat, that makes Jetty
a bit of a challenge (actually, even upgrading to Tomcat 6 looks like it
would be a challenge given the changes that have been made).

I have tried an OOTB installation of DSpace on Jetty, and that worked
without any problems. Plus the documentation looks to be pretty good,
and the configuration fairly easy - so it's definitely worth taking a
look at Jetty.

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] (What look like) Lucene errors on import andindex-all

[Graham Triggs]

Hi Mark,

If you look back at the stack trace you posted:

at _Jv_ResolvePoolEntry(java.lang.Class, int) (/usr/lib/libgcj.so.5.0.0)

there is a reference to libgcj - which means that it is running under the 
GNU JDK. You may well be using the Sun JDK to run your application server, 
and even to compile the source. But it would appear that the default java 
when running tasks from the command line is the GNU JDK.

What distribution are you running on?

G

- Original Message - 
From: "Mark Jordan" <[EMAIL PROTECTED]>
To: "Stuart Lewis [sdl]" <[EMAIL PROTECTED]>
Cc: 
Sent: Saturday, March 17, 2007 5:38 PM
Subject: Re: [Dspace-tech] (What look like) Lucene errors on import 
andindex-all


> Hi Stuart,
>
> We are using the Sun JDK. Sorry I didn't mention that.
>
> Mark
>
> Stuart Lewis [sdl] wrote:
>> Hi Mark,
>>
>> Try using a Sun JDK rather than the GNU JDK your system is using. This 
>> might
>> solve your problem.
>>
>> Thanks,
>>
>>
>> Stuart
>>
>> _
>>
>> Datblygydd Cymwysiadau'r WeWeb Applications Developer
>> Gwasanaethau Gwybodaeth  Information Services
>> Prifysgol Cymru Aberystwyth   University of Wales Aberystwyth
>>
>> E-bost / E-mail: [EMAIL PROTECTED]
>>  Ffon / Tel: (01970) 622860
>> _
>>
>>
>>
>> On 16/3/07 17:12, "Mark Jordan" <[EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>>
>>> I'm trying to import items into DSpace and am getting the following
>>> errors, just after the first record is parsed (no ID is written to the
>>> map file):
>>>
>>> Exception in thread "main" java.lang.IncompatibleClassChangeError: field
>>> org.apache.lucene.analysis.standard.StandardFilter.tokenImage was not 
>>> found.
>>> at _Jv_ResolvePoolEntry(java.lang.Class, int) 
>>> (/usr/lib/libgcj.so.5.0.0)
>>> at org.apache.lucene.analysis.standard.StandardFilter.()
>>> (Unknown Source)
>>> at java.lang.Class.initializeClass() (/usr/lib/libgcj.so.5.0.0)
>>> at org.dspace.search.DSAnalyzer.tokenStream(java.lang.String,
>>> java.io.Reader) (Unknown Source)
>>> at
>>> org.apache.lucene.index.DocumentWriter.invertDocument(org.apache.lucene.docume
>>> nt.Document)
>>> (Unknown Source)
>>> at
>>> org.apache.lucene.index.DocumentWriter.addDocument(java.lang.String,
>>> org.apache.lucene.document.Document) (Unknown Source)
>>> at
>>> org.apache.lucene.index.IndexWriter.addDocument(org.apache.lucene.document.Doc
>>> ument,
>>> org.apache.lucene.analysis.Analyzer) (Unknown Source)
>>> at
>>> org.apache.lucene.index.IndexWriter.addDocument(org.apache.lucene.document.Doc
>>> ument)
>>> (Unknown Source)
>>> at
>>> org.dspace.search.DSIndexer.writeIndexRecord(org.apache.lucene.index.IndexWrit
>>> er,
>>> int, java.lang.String, java.util.HashMap, java.lang.String) (Unknown 
>>> Source)
>>> at
>>> org.dspace.search.DSIndexer.writeItemIndex(org.dspace.core.Context,
>>> org.apache.lucene.index.IndexWriter, org.dspace.content.Item) (Unknown
>>> Source)
>>> at org.dspace.search.DSIndexer.indexContent(org.dspace.core.Context,
>>> org.dspace.content.DSpaceObject) (Unknown Source)
>>> at
>>> org.dspace.content.InstallItem.installItem(org.dspace.core.Context,
>>> org.dspace.content.InProgressSubmission, java.lang.String) (Unknown 
>>> Source)
>>> at
>>> org.dspace.app.itemimport.ItemImport.addItem(org.dspace.core.Context,
>>> org.dspace.content.Collection[], java.lang.String, java.lang.String,
>>> java.io.PrintWriter) (Unknown Source)
>>> at
>>> org.dspace.app.itemimport.ItemImport.addItems(org.dspace.core.Context,
>>> org.dspace.content.Collection[], java.lang.String, java.lang.String)
>>> (Unknown Source)
>>> at org.dspace.app.itemimport.ItemImport.main(java.lang.String[])
>>> (Unknown Source)
>>>
>>>
>>>
>>> When I ran index-all I got a similar error:
>>>
>>>
>>>
>>> Exception in thread "main" java.lang.IncompatibleClassChangeError: field
>>> org.apache.lucene.analysis.standard.StandardFilter.tokenImage was not 
>>> found.
>>> at _Jv_ResolvePoolEntry(java.lang.Class, int) 
>>> (/usr/lib/libgcj.so.5.0.0)
>>> at org.apache.lucene.analysis.standard.StandardFilter.()
>>> (Unknown Source)
>>> at java.lang.Class.initializeClass() (/usr/lib/libgcj.so.5.0.0)
>>> at org.dspace.search.DSAnalyzer.tokenStream(java.lang.String,
>>> java.io.Reader) (Unknown Source)
>>> at
>>> org.apache.lucene.index.DocumentWriter.invertDocument(org.apache.lucene.docume
>>> nt.Document)
>>> (Unknown Source)
>>> at
>>> org.apache.lucene.index.DocumentWriter.addDocument(java.lang.String,
>>> org.apache.lucene.document.Document) (Unknown Source)
>>> at
>>> org.apache.lucene.index.IndexWriter.addDocument(org.apache.lucene.document.Doc
>>> ument,
>>> org.apache.lucene.analysis.Analyzer) (Unknown Source)
>>> at
>>> org.apache.lucene.index.IndexWriter.addDocument(org.apache.lucene.document.Doc
>>> um

Re: [Dspace-tech] Lucene error

[Graham Triggs]

Jake,

You are having the same problem as Mark Jordan. Although you've compiled
you classes using JDK 1.5, when you are running index-all, it is picking
up the GNU GCJ installation when the command 'java' is executed.

How you fix it is dependent on how you installed the Sun JDK, and/or
what distribution you are running.

G

On Mon, 2007-03-26 at 08:51 -0600, Cameron, Jacob wrote:
> Hey everyone,
> 
> Anyone have any ideas or suggestions on what I can do to stop this error
> when I run index-all.  I'm running JDK1.5.0_11.
> 
> Creating browse index
> Warning: -Xmx256m not understood. Ignoring.
> Indexing all Items in DSpace ... Done
> Creating search index
> Warning: -Xmx256m not understood. Ignoring.
> Exception in thread "main" java.lang.IncompatibleClassChangeError: field
> org.apache.lucene.analysis.standard.StandardFilter.tokenImage was not
> found.   
>at _Jv_ResolvePoolEntry(java.lang.Class, int)
> (/usr/lib/libgcj.so.5.0.0)
>at org.apache.lucene.analysis.standard.StandardFilter.()
> (Unknown Source)
>at java.lang.Class.initializeClass() (/usr/lib/libgcj.so.5.0.0)
>at org.dspace.search.DSAnalyzer.tokenStream(java.lang.String,
> java.io.Reader) (Unknown Source)
>at
> org.apache.lucene.index.DocumentWriter.invertDocument(org.apache.lucene.
> document.Document) (Unknown Source)
>at
> org.apache.lucene.index.DocumentWriter.addDocument(java.lang.String,
> org.apache.lucene.document.Document) (Unknown Source)
>at
> org.apache.lucene.index.IndexWriter.addDocument(org.apache.lucene.docume
> nt.Document, org.apache.lucene.analysis.Analyzer) (Unknown Source)
>at
> org.apache.lucene.index.IndexWriter.addDocument(org.apache.lucene.docume
> nt.Document) (Unknown Source)
>at
> org.dspace.search.DSIndexer.writeIndexRecord(org.apache.lucene.index.Ind
> exWriter, int, java.lang.String, java.util.HashMap, java.lang.String)
> (Unknown Source)
>at
> org.dspace.search.DSIndexer.writeCommunityIndex(org.dspace.core.Context,
> org.apache.lucene.index.IndexWriter, org.dspace.content.Community)
> (Unknown Source)
>at
> org.dspace.search.DSIndexer.indexAllCommunities(org.dspace.core.Context,
> org.apache.lucene.index.IndexWriter) (Unknown Source)
>at org.dspace.search.DSIndexer.createIndex(org.dspace.core.Context)
> (Unknown Source)
>at org.dspace.search.DSIndexer.main(java.lang.String[]) (Unknown
> Source)
> 
> 
> 
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] sshfs assetstore and index-all

[Graham Triggs]

Monika,

Your local mount point is /var/www/dspace_root/assetstore - your asset
store is only available (locally) through that path.

However, your config file is set to have reference the datastore
as /dspace/assetstore

(your config likely reads: assetstore.dir=${dspace.dir}/assetstore,
where dspace.dir=/dspace).

Either amend your dspace.cfg to refer set:

assetstore.dir=/var/www/dspace_root/assetstore

or change the local mount point to /dspace/assetstore

G

On Mon, 2007-03-26 at 18:14 -0400, Monika Mevenkamp wrote:
> I run vanilla dspace-1.4 
> with postgres 
> on Ubuntu 
> 
> I used sshfs to mount the assetstore: 
>  sshfs -o allow_other
> [EMAIL PROTECTED]:/dspace/assetstore /var/www/dspace_root/assetstore 
> 
> I am trying to access the test assetsore on another machine, which is
> too big to be copied. 
> Is there something inherently wrong with doing sshfs ? 
> 
> As the dspace user I can list,view, and touch files in the assetstore
> but index-all still dies ungracefully as follows: 
> 
> Creating browse index
> Indexing all Items in DSpace2007-03-26 17:50:21,124 INFO
> org.dspace.core.ConfigurationManager @ DSpace logging installed using
> log4j.properties
>  ... Done
> Creating search index
> 2007-03-26 17:50:55,060 INFO  org.dspace.core.ConfigurationManager @
> DSpace logging installed using log4j.properties
> 2007-03-26 17:50:59,871 DEBUG
> org.dspace.storage.bitstore.BitstreamStorageManager @ Local filename
> for 107718164672031636339452188487841053759
> is /dspace/assetstore/10/77/18/107718164672031636339452188487841053759 
> 2007-03-26 17:50:59,871 DEBUG
> org.dspace.storage.bitstore.BitstreamStorageManager @
> IinputBitStream 
> /dspace/assetstore/10/77/18/107718164672031636339452188487841053759
> Exception in thread "main"
> java.io.FileNotFoundException : 
> /dspace/assetstore/10/77/18/107718164672031636339452188487841053759 (No such 
> file or directory)
> at java.io.FileInputStream.open(Native Method)
> at java.io.FileInputStream.(FileInputStream.java :106)
> at
> edu.sdsc.grid.io.local.LocalFileInputStream.open(LocalFileInputStream.java:171)
> at
> edu.sdsc.grid.io.GeneralFileInputStream.(GeneralFileInputStream.java:145)
> at
> edu.sdsc.grid.io.local.LocalFileInputStream 
> .(LocalFileInputStream.java:139)
> at
> edu.sdsc.grid.io.FileFactory.newFileInputStream(FileFactory.java:454)
> at
> org.dspace.storage.bitstore.BitstreamStorageManager.retrieve(BitstreamStorageManager.java
>  :553)
> at org.dspace.content.Bitstream.retrieve(Bitstream.java:512)
> at
> org.dspace.search.DSIndexer.writeItemIndex(DSIndexer.java:694)
> at
> org.dspace.search.DSIndexer.indexAllItems(DSIndexer.java :397)
> at org.dspace.search.DSIndexer.createIndex(DSIndexer.java:192)
> at org.dspace.search.DSIndexer.main(DSIndexer.java:225)
> 
> The second log message is produced by a log.debug call added in: 
> 
> package org.dspace.storage.bitstore;
> class BitstreamStorageManager.java;
> 
> public static InputStream retrieve(Context context, int id)
> throws SQLException, IOException 
> {
> TableRow bitstream = DatabaseManager.find(context,
> "bitstream", id);
> 
> GeneralFile file = getFile(bitstream);
> log.debug("InputBitStream " + file.getPath()); 
> return (file != null) ? FileFactory.newFileInputStream(file) :
> null;
> }
> 
> 
> Thanks for any tips ? 
> 
> Monika
> 
> -- 
> Monika Mevenkamp 
> Georgia Institute of Technology
> Library and Information Center 
> Phone: 404.385.0108 
> -
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> ___ DSpace-tech mailing list 
> DSpace-tech@lists.sourceforge.net 
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] OAI-PMH and Oracle

[Graham Triggs]

Hi Rafa,

The timestamps shown on:

http://rua.ua.es:8080/dspace/handle/10045/175?mode=full&submit_simple=Show+full+item+record

are 2007-02-21T10:17:00Z as you say. However, these are all metadata
values (as stored in the METADATAVALUE table).

When OAI-PMH harvesting, DSpace is using the value of the last_modified
column from the ITEM table.

Note that when an item is entered into DSpace, the accessioned,
available, and issued dates are all initially set from the same DCDate
object, so they all have exactly the same date. The last_modified date
is set in another part of the code, using a different Date (not DCDate)
object - not only does that mean it will be in local time and not UTC
(the displayed dates above are UTC), but it will be a slightly different
timestamp based on how much time has elapsed between the metadata values
being set, and the last_modified date being updated.

All of which means that your last_modified column for this item is
presumably about +0100 to the dates specified above. And you may need to
allow another second or two for where the dates are generated.

NB: I've just been testing your OAI, and although you say you have
Mike's patch applied, it doesn't appear to be working - if I adjust for
the local timezone:

<http://rua.ua.es:8080/dspace-oai/request?verb=ListRecords&from=2007-02-21T11:17:00Z&until=2007-02-21T11:17:00Z&set=&metadataPrefix=oai_dc>

Then I get the missing record listed.

Regards,
G 


On Mon, 2007-03-26 at 18:23 +0200, dspace wrote:
> Of course, it was added several days ago...
> Thxs
> 
> 
> Graham Triggs escribió:
> > Are you now running Michael's patch in addition to the one the I sent
> > you?
> >
> > This may be an error in the application of UTC vs. Local Timestamps.
> >
> > G
> >
> > On Mon, 2007-03-26 at 17:57 +0200, dspace wrote:
> >   
> >> It works!!! Now, I've just tried to pass the validation test and I get a 
> >> different error:
> >>
> >> ListRecords response gave a noRecordsMatch error when it should have 
> >> included at least the record with identifier oai:rua.ua.es:10045/175. 
> >> The from and until parameters of the request were set to the datestamp 
> >> of this record (2007-02-21T10:17:00Z). The from and until parameters are 
> >> inclusive, see protocol spec section 2.7.1. The message included in the 
> >> error response was: 'The combination of the values of the from, 
> >> until, set, and metadataPrefix arguments results in an empty list.'
> >>
> >>
> >> I think I've seen this before at dspace-tech or so...
> >>
> >> Thanks again Graham
> >>
> >>
> >> Graham Triggs escribió:
> >> 
> >>> Here you go.
> >>>
> >>> On Mon, 2007-03-26 at 17:08 +0200, Rafa wrote:
> >>>   
> >>>   
> >>>> Hi Graham,
> >>>>
> >>>> Thanks for your work. Oddly, I can´t see the patch! Can you re-send it?
> >>>>
> >>>>
> >>>> Graham Triggs escribió:
> >>>> 
> >>>> 
> >>>>> Hi Rafa,
> >>>>>
> >>>>> Please try the attached patch.
> >>>>>
> >>>>> What I think is happening is that the token is being supplied as:
> >>>>>
> >>>>> 2007-03-26T15:48:30Z
> >>>>>
> >>>>> - very specifically, as a 20 character string including the final Z. The
> >>>>> current oracleTimeStampFunction will try to use the format:
> >>>>>
> >>>>> -MM-DD"T"HH24:MI:SS.FF"Z"
> >>>>>
> >>>>> and fall over as the 'Z' doesn't match the expected '.FF'.
> >>>>>
> >>>>> This patch specifically checks for a 20 character string, and applies
> >>>>> the format:
> >>>>>
> >>>>> -MM-DD"T"HH24:MI:SS"Z"
> >>>>>
> >>>>> If this works for you, then I will submit it as a patch for inclusion in
> >>>>> the next release.
> >>>>>
> >>>>> G
> >>>>>
> >>>>> On Fri, 2007-03-23 at 15:03 +0100, dspace wrote:
> >>>>>   
> >>>>>   
> >>>>>   
> >>>>>> Hi,
> >>>>>>
> >>>>>>
> >>>>>> I'am absolutely stuck with this issue. The double hyphe

Re: [Dspace-tech] RE : MediaFilter clarification

[Graham Triggs]

On Thu, 2007-03-29 at 11:29 -0400, Tellier, Stephane wrote:
> Dspace isn't an OCR tool, because this kind of process is very
> difficult to do and I don't know if any are free and open-source
> actually...

There is Tesseract-OCR:

http://code.google.com/p/tesseract-ocr/

although it isn't Java based, so integration may be a little tricky

G


-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] [Dspace-devel] DSpace 1.4.2

[Graham Triggs]

I wasn't going to push this patch earlier as it rather evolved into a
feature enhancement - but thinking about it again, for anyone that has
non-English content/authors in their repository, it's actually a useful
bug fix:

[ 1672065 ] Configurable sort order generation for browse
http://sourceforge.net/tracker/index.php?func=detail&aid=1672065&group_id=19984&atid=319984

If you look at the example Shawna has (it's either on this page, or the
next):
https://dspace.ucalgary.ca/browse-author?top=Austad%2C+Michele+L

The author Žekulin appears in the middle of the list of authors
beginning with 'A'. It may not be possible to define a single correct
ordering, but in this case the existing behaviour just doesn't work for
anyone.

Amongst it's other capabilities, the above patch deals with this
scenario, and lets people customise the ordering further if they choose.

Whilst it allows for configuration, it does not require any
configuration changes (closely emulating the existing DSpace behaviour -
except for the addition of the character decomposition for the above
case). It also doesn't change any existing APIs, or the database schema.

The single biggest argument against it's inclusion would be that it adds
an additional dependency: ICU4J. Unfortunate, but required for
compatibility with earlier JDK releases, without resorting to
undocumented Sun classes.

The choice, as they say, is yours... ;-)

G

-- 
Graham Triggs
Technical Architect
Open Repository

Tel:   +44 (0)20 7631 9942
Skype: grahamtriggs

On Mon, 2007-04-02 at 10:19 +0100, James Rutherford wrote:
> Hi All,
> 
> I'm pleased to announce that work has started on putting together DSpace
> 1.4.2. We're aiming to get a beta release out around the 23rd, with the
> final release available shortly after (hopefully by the end of the
> month). This will be pretty much exclusively a bugfix release; there
> will be no API, configuration, or database schema changes, so upgrading
> from 1.4 or 1.4.1 should be relatively painless. If anyone is sitting on
> patches that they think would be suitable for inclusion, please upload
> them to SourceForge by the end of the week.
> 
> See http://wiki.dspace.org/index.php/RoadMap#Version_1.4.2 for more
> information.
> 
> cheers,
> 
> Jim
> 


-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Install help

[Graham Triggs]

Hi,

The default installation of Tomcat5.5 on Debian has the Java Security
Manager enabled - however it won't have any (appropriate) policies
defined.

As such, it is disallowing the ConfigurationManager from reading the
system property "dspace.configuration".

You either need to disable the security manager, or specify a policy to
allow access.

Disable security manager:

edit /etc/init.d/tomcat5.5 and set "TOMCAT5_SECURITY" to "NO".


Adding a policy should be something like:

create /etc/tomcat5.5/policy.d/05dspace.policy with the contents:

grant codebase "file:${catalina.home}/webapps/dspace/-" {
   permission java.security.AllPermission;
};


This was taken from:
http://forums.pentaho.org/showthread.php?p=157283

Someone will probably shout if it's wrong!

G

-- 
Graham Triggs
Technical Architect
Open Repository

Tel:   +44 (0)20 7631 9942
Skype: grahamtriggs


On Tue, 2007-04-03 at 14:11 -0400, Joel Gwynn wrote:
> Hi all.  Just dropping in for some quick install/config help.
> 
> I've installed dspace 1.4.1 with Tomcat 5 on Ubuntu Linux.  I think
> I'm about to be up and running, but when I fire up
> http://localhost:8180/dspace (debian runs tomcat on 8180) I get these
> errors:
> 
> java.security.AccessControlException: access denied
> (java.util.PropertyPermission dspace.configuration read)
> 
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:264)
> 
> java.security.AccessController.checkPermission(AccessController.java:427)
> java.lang.SecurityManager.checkPermission(SecurityManager.java:532)
> 
> java.lang.SecurityManager.checkPropertyAccess(SecurityManager.java:1285)
> 
> etc.  I assume that I'm missing something with permissions.  In my
> tomcat /etc/init.d/tomcat5 I set tomcat to run as the dspace user like
> so:
> 
> TOMCAT5_USER=dspace
> 
> Is there something else I need to do?
> 
> TIA
> Joel
> 
> -
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech 
 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB

-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] [Dspace-devel] DSpace 1.4.2

[Graham Triggs]

Hi,

Unfortunately, the necessary Normalizer classes (java.text.Normalizer)
only formally entered the Java platform API as part of Java 6. See:

http://java.sun.com/developer/technicalArticles/javase/i18n_enhance/


The unofficial, undocumented sun.text.Normalizer appears to exist in all
Sun Java2 platforms (I won't predict whether it exists in non-Sun
implementations), and should be usable for our purposes in at least Java
1.3. See:

http://lists.limewire.org/pipermail/codepatch/2003-March/74.html


But relying on sun.text.* classes is really bad form. Despite it
appearing to be a rather nominal distinction (according to the above
post), it would be more reliable to depend on ICU4J until such a time
that Java 6 became a requirement.

I'm not sure why you've mentioned the XMLUI at this point - although it
produces a visible change, all* the changes exist entirely inside the
Java code, and what is written to the database browse tables. So whether
you have the XMLUI or JSP, shouldn't make any difference - but if you
are thinking of some other aspect that I haven't considered, please say
so.

* In fact, the patch only makes one-line changes to Browse.java in three
places (where it inserts data into the tables). Everything else is
additional code, except for ONE change to a JSP file, to in the admin
tools to apply the same normalisation to the author name being entered
for lookup, as it uses the sort_author column in the ItemsByAuthor
table.

Note that the above does mean that it needs to be integrated with
Richard's patch when that is ready, but on the basis of what is
currently available I've already designed this to have as little impact
as possible.

G

On Wed, 2007-04-04 at 10:57 -0400, Mark Diggory wrote:
> Is ICU4J a requirement still if we switch to Java 5? In which case  
> I'd push this off to DSpace1.5 (which will require Java 5 or  
> greater). And if so, it should probably be the case that we explore  
> its implementation in the XMLUI if any at all changes are needed.
> 
> -Mark
> 
> On Apr 4, 2007, at 4:53 AM, Graham Triggs wrote:


-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Migrating data from Postgres to Oracle in DSpace 1.4.1

[Graham Triggs]

On Thu, 2007-04-05 at 11:39 +1000, Shaun Burriss wrote:
> We are about to upgrade our DSpace installation from 1.3.1 to 1.4.1 and
> also want to move our data from the Postgres database to an Oracle
> database. What things should we be aware of when doing this data
> migration? What are the steps involved?

Hi Shaun,

To be honest, migrating to an Oracle database with 1.4.1 is probably not
a good idea. At the moment, the 1.4.1 release has a number of known
issues with it's Oracle support - fixes for which are currently being
tested with a view to release as 1.4.2.

Beyond that, the existing schema definition for Oracle uses
VARCHAR2(2000) in place of Postgres' TEXT columns. This means that many
fields - importantly, all the metadata values and license text - is
limited to a maximum of 2000 characters. It is likely that this would
cause problems with any attempted conversion.

Again, one of the Oracle patches allows the use of CLOB columns, and a
schema patch exists that makes use of them in the key fields (for
example, metadata). This schema change isn't 100% perfect, as the browse
tables need to retain the use of VARCHAR2 columns for ORDERing support -
although it is highly unlikely that this would cause a problem in
practise.

> Also, should the migration to Oracle take place before or after our
> DSpace update?

Changing VARCHAR2 columns to CLOBs is a rather involved process, and my
advice here would be to only start using Oracle with a DSpace that
supports the use of CLOBs, and to start with a schema that uses CLOB
columns where necessary/possible before you migrate the data in.

In practise, I would therefore suggest that you upgrade to 1.4.1 on your
Postgres database, and then migrate to Oracle after 1.4.2 is released.

G

-- 
Graham Triggs
Technical Architect
Open Repository

Tel:   +44 (0)20 7631 9942
Skype: grahamtriggs
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] compilation warning.

[Graham Triggs]

Jose,

-Xlint:deprecation will just give you more information about the use of 
deprecated API.

There is already a patch to address the use of deprecated APIs:


however, there is no requirement to apply this patch - your DSpace installation 
will function just fine without it (for the time being - potentially a future 
JAR / Java upgrade may break the code, but the patch will probably have been 
included in a DSpace release long before then).

G



  - Original Message - 
  From: Jose Blanco 
  To: dspace-tech@lists.sourceforge.net 
  Sent: Friday, April 06, 2007 8:11 PM
  Subject: [Dspace-tech] compilation warning.


  As of my upgrade to 1.4.1, I get the following warning when I build the war 
file:

   

  [javac] 
/l1/dspace/build/prod/dspace/src/org/dspace/app/oai/DIDLCrosswalk.java:55: 
warning: sun.misc.BASE64Encoder is Sun proprietary API and may be removed in a 
future release

  [javac] import sun.misc.BASE64Encoder;

  [javac]^

  [javac] 
/l1/dspace/build/prod/dspace/src/org/dspace/app/oai/DIDLCrosswalk.java:238: 
warning: sun.misc.BASE64Encoder is Sun proprietary API and may be removed in a 
future release

  [javac] BASE64Encoder 
encoder=(BASE64Encoder) Class.forName("sun.misc.BASE64Encoder").newInstance();

  [javac] ^

  [javac] 
/l1/dspace/build/prod/dspace/src/org/dspace/app/oai/DIDLCrosswalk.java:238: 
warning: sun.misc.BASE64Encoder is Sun proprietary API and may be removed in a 
future release

  [javac] BASE64Encoder 
encoder=(BASE64Encoder) Class.forName("sun.misc.BASE64Encoder").newInstance();

  [javac]^

  [javac] Note: Some input files use or override a deprecated API.

  [javac] Note: Recompile with -Xlint:deprecation for details.

  [javac] 3 warnings

   

  Should I change my build command to include  -Xlint:deprecation, like shown 
below?

   

  /l/local/bin/ant -Xlint:deprecation -Dconfig=/a_directory/dspace.cfg 
build_wars

   

   

  Many thanks!

   

  Jose



--


  -
  Take Surveys. Earn Cash. Influence the Future of IT
  Join SourceForge.net's Techsay panel and you'll get the chance to share your
  opinions on IT & business topics through brief surveys-and earn cash
  http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV


--


  ___
  DSpace-tech mailing list
  DSpace-tech@lists.sourceforge.net
  https://lists.sourceforge.net/lists/listinfo/dspace-tech

This email has been scanned by Postini.
For more information please visit http://www.postini.com

-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Licenses Dspace

[Graham Triggs]

On Wed, 2007-04-11 at 11:19 +0200, luis jose miralls wrote:
> Hello, 
> First thing I have to say is that i am from Murcia (Spain) so my
> English is not good.

Don't worry - I'm from England, and neither is mine! (and the less said
about my Spanish the better ;-)


> - If I want to use DSpace to develop three or four
> repositories with different URL but from the same University. Can I do
> that with one License or I need three or four?
> 
>  
> 
> - If I just want to make proves. Do I need to have the
> license?


So, you are asking about the license to install and use DSpace software,
rather than licenses that may be applied to the content that you store
in it?

DSpace is issued under a BSD license, so you are free to download and
install it - even customise it - as many times as you want. You don't
need to ask permission to do so, although you do have 'accept' the terms
of the license when you do - so no giving the code to others claiming it
to be your own, or taking the code to use in a different application
without proper acknowledgement.

But other than that, go ahead and create as many repositories as you
like. There are other concerns that affect the content that you place
inside your repositories, but that is an entirely different question.

Regards,
G


-- 
Graham Triggs
Technical Architect
Open Repository

Tel:   +44 (0)20 7631 9942
Skype: grahamtriggs
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] JAVA question

[Graham Triggs]

> Private static Logger log = Logger.getLogger(SFXLinkTag.class);
>
> My question is this:  How do I make these same changes in a .jsp, such as
> display-item.jsp?   I was able to figure out the syntax for the 1st and 
> last
> lines of code, but didn't know what to do with the parameter inside Logger
> getLogger(   ), and of course got an error
> when I completely left this line out.

Sue,

Documentation for the Logger class is here:

http://logging.apache.org/log4j/docs/api/org/apache/log4j/Logger.html

As you can see, there are two variations of getLogger - one that takes a 
Class (as used above), and one that takes a String.

Ultimately, the Class based method is retrieving the name of the class as a 
string, and in both cases the logger for that name / value is returned (or 
created).

For JSPs, classes are generated automatically, and hence given obscure 
names. This makes configuration at best awkward, if not impossible. But as 
you can pass a String to getLogger(), simply pass a value you are 
comfortable with (ie. "DisplayItemJSP").

G 
 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB

-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
DSpace-tech mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Oracle and dspace

[Graham Triggs]

On Fri, 2007-04-13 at 09:37 -0500, Hulbert, Linda A. wrote:
> The University has begun a project to implement dspace using Oracle. I
> have heard differing levels of success (mostly not) using Oracle.
> While I am not an IT person, I will be passing any comments on to our
> IT and web services colleagues. Ha anyone out there successfully
> launched dspace using Oracle?

Hi Linda,

Yes, we are running a number of installations against an Oracle
database. Currently these are all based off of version 1.3.2 of DSpace
- although as I wasn't personally involved in their setup, I can't
comment to what extent we had to alter the code (I do know that there
are a number of changes related to our specific requirements, that
wouldn't affect others).

Unfortunately, the current 1.4.1 release does not support Oracle
properly, but the good news is that a number of fixes have been
accepted into the code that will be released fairly soon as 1.4.2.
There is still testing taking place on this code, but so far I am not
aware of any Oracle related problems with the upcoming release, and it
*should* just install and run successfully.

G

-- 
Graham Triggs
Technical Architect
Open Repository

Tel:   +44 (0)20 7631 9942
Skype: grahamtriggs 
 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB

-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Adding items without bitstreams

[Graham Triggs]

On Wed, 2007-04-18 at 06:52 +0100, Stuart Lewis [sdl] wrote:
> At present it is not possible - but will be in the next major release of
> DSpace (1.5) which will include what is known as the 'configurable
> submission system' (CSS).

Hi Stuart,

This is very good news for us, as we already have similar functionality
hacked in to our 1.3.2 installation(s), and I was hoping that we could
replace it with the configurable submission system in our 1.4.x upgrade.

Although, can we please agree not abbreviate it to 'CSS', otherwise
things are going to get very confusing around here! (Maybe it should be
rebranded to the 'configurable submission process').

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Cannot get a connection, pool exhausted

[Graham Triggs]

On Wed, 2007-04-18 at 13:33 +0100, Richard Jones wrote:
> I just put a page on the wiki with some thoughts and possible ways of 
> debugging this, as it happens:
> 
> http://wiki.dspace.org/index.php/Idle_In_Transaction_Problem
> 
> If you can have a look at the results of the query that this page 
> suggests and post it to the list or the wiki, perhaps we can start to 
> tighten up the database connections.

Hmmm... in theory, whilst transactions can 'fail', they should always
rollback or commit. If we assume something really obscure isn't
happening for now, it leaves us with two possibilities:

1) The Context has dropped out of scope without being cleaned up

2) A thread is still executing, and is still using it's Context / transaction.

A few obvious things here spring to mind:

1) Check the logs for Exceptions - one being thrown without a finally
to clean up the Context would be a likely (if not only) suspect for a
Context being lost.

2) Launch Tomcat under JDK 1.5 with the JMX interface enabled, and
attach JConsole to it (JMX adds little overhead to the server, and you
don't need to attach JConsole until the problem arises). From JConsole,
you can see how many Threads Tomcat is currently using, what status
they are in and often a stack trace - so you can see where in the code
that Thread is being blocked.

(As an aside, we had a problem with a different application, where
sooner or later - and often sooner - Tomcat would suddenly grind to a
halt and stop serving requests. Using JConsole showed that the Threads
were all blocked (and increasing  in number with every request), and
the point at which most of them was blocked was in trying to obtain a
database connection from a custom built pool - and it actually came
back to a background thread that was trying to close unwanted
connections at the same time the business logic was trying to close the
same connection. Both Threads got hung up on each other, and everything
else blocked as the pool couldn't give new connections whilst the clean
up was in progress.)

Also note that JConsole has a button to force garbage collection - this
may help in determining if there are dangling Contexts. If they (and
therefore the transactions) disappear after forcing the garbage
collection, then something was dangling - although that wouldn't tell you why).

G

-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Timeout and dspace.log

[Graham Triggs]

1) Someone can correct me if I'm wrong, but I think it just works on the 
standard session timeout, so:

You can specify the timeout of a session in the deployment descriptor of 
your web application (web.xml):


  
60
  

  ...


The number within the session-timout element must be expressed in minutes.


2) The dspace.log will be created as the user that your Tomcat is running 
as.

- Original Message - 
From: "Cameron, Jacob" <[EMAIL PROTECTED]>
To: 
Sent: Tuesday, April 24, 2007 9:03 PM
Subject: [Dspace-tech] Timeout and dspace.log


> Hi all,
>
> 2 quick questions.
>
> 1) Where do I change the timeout time in DSpace?  We want to adjust it
> so you don't logout for 30 minutes.
> 2) How come every time a new dspace.log file gets created, it is made
> under root instead of my dspace user, I did the whole install as my
> dspace user.
>
> Jake
>
> -- 
> Jake Cameron, BCSc(UNB)
> Technical Specialist III
> Library Systems and Web Services
> University of Lethbridge
> Phone: (403) 329-2756
> Office: L1110C
> Email: [EMAIL PROTECTED]
>
>
> -
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
> 

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Maximum File Size

[Graham Triggs]

On Wed, 2007-04-25 at 09:24 +0200, José A. Rubio wrote:
> Dear William,
> 
> In our installation, with Oracle 10g, and applying the path to its
> proper functioning, we have found
> problems with files of 1 gig or more. Perhaps, it's a problem of our
> specific system ..

What patch(es) did you apply?

And what problems are you having?

There are lots of potential issues when dealing with large files - the
amount of data you can handle in an upload / ingest, file system of the
asset store, etc.

AFAIK, the only database related issue with large files is the storing
of the file size metadata - in that regards, the changes to allow long
integers to store file size originally broke *all* file uploads with
Oracle (not just large files). Even a minimal workaround would likely
have no more issues than to misreport large file sizes, although the
latest patches do not have that limitation.

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] language problems

[Graham Triggs]

Hi,

You need to setup your application server to support UTF-8 encoded URLs.

See:



for how to do this with Tomcat.

G

On Thu, 2007-05-03 at 10:30 +0800, Anny Bridge wrote:
> Hi,
> 
> I input a word in Japanese into the search form,then submit the
> form.Then the search word in the search form be changed to some
> strange code.How can i solve this problem?
> 
> Thanks in advance.
> 
> Best Regards, 
> 
> Anny.
> -
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> ___ DSpace-tech mailing list 
> DSpace-tech@lists.sourceforge.net 
> https://lists.sourceforge.net/lists/listinfo/dspace-tech

-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] dspace.cfg -- db.statementpool = true

[Graham Triggs]

From: Damian Marinaccio
>From dspace.cfg:
>
># Determine if prepared statement should be cached. (default is true)
>db.statementpool = true
>
>
>
>I cannot find ANY documentation related to this statement. What are the 
>implications of turning it off/on.

Hi,

Well, as it says that the default is true, the first implication is that you 
can't turn it on - you can only turn it off ;-)

What it relates to is whether PreparedStatement objects are held in a pool 
(managed by DBCP).

Exact behaviour is rather dependent on the actual JDBC driver 
implementation, but pooling PreparedStatements is meant to eliminate the 
overhead of creating cursors and prevent repeated statement parsing/creation 
(I say meant - early Postgres drivers didn't hold cursors open on the 
database, and only pretended to support PreparedStatement).

So, a pool of PreparedStatement objects is generally a good thing - not 
having one will increase the execution time of frequently executed 
(parametized) SQL, and waste significant resources on the database in 
parsing, etc.

That said, maintaining open cursors on the database also takes resources. In 
particular, with some databases (ie. Oracle) the default statement pooling 
behaviour via DBCP as implemented in DSpace can overwhelm the database's 
defined ability to hold open cursors, resulting in errors.

Turning off statement pooling will reduce the open cursors on the database 
(which as seen above may be useful in some scenarios), at the cost of 
increasing the time and processing involved with executing each SQL query 
(reducing scalability).

What is missing from the current DSpace implementation is a configuration 
option to limit the number of PreparedStatement objects that get pooled 
(which is what is ideally required to strike a balance between holding open 
resources on the database and system scalability).

G 

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] dspace.cfg -- db.statementpool = true

[Graham Triggs]

Kind of hard to say.

What's happening is fairly obvious - close() has been called on the
underlying Postgres PreparedStatement object, but the
DelegatingPreparedStatement is remaining in the pool.

I would suspect that the Connection that had been used in creating the
statement has been closed at some point, which in turn has caused the
statement to be closed - but I wouldn't know where!

You could try adding:

statementFactoryConfig.testOnBorrow = true;

in DatabaseManager.initialize() inside the if(useStatementPool), before
the GenericKeyedObjectPoolFactory is created. By default statements will
not be validated, and this will have them checked prior to use.

Also, there appears to be a bug in the TableRowIterator, although this
probably isn't affecting you. In close():

results.close();
if (results != null)
results.close();

should be:

if (results != null)
results.close();
results = null;

Regards,
G


-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Persistent identifiers in DSpace -- thoughtsplease

[Graham Triggs]

Hi,

> 1) Why would an institution use more than one PI
> system?  How do you determine which PI system generates a PId (base it
> on collection, community)?

There are a lot of theoretical reasons why multiple PI schemes may be in 
use. Even if you have the simple case of an institute / repository defining 
a single PI scheme that it always uses for the contents of the repository, 
depening on what content is being added, there may already be other PIs 
associated with an item that is being deposited (for example, a published 
article may have a DOI).

Beyond that, you may have repositories that have mandated different PI 
schemes being merged, and therefore all those existing PIs need to be 
supported, as well as new ones for the final repository possibly having to 
be assigned.

And with all the issues surrounding 'ownership' and encouraging the use of 
the repository, it may well prove necessary to support (and mandate) 
different PI schemes on a community or collection level.

> 2)  It is mentioned that HTTP isn't "persistent":  Could someone explain
> why HTTP isn't as persistent as any other protocol?

Forget to pay your domain registration fee on time and see how persistent it 
is ;-)

Potentially more problematic, what happens when part (or all) of a 
repository is migrated into another? Can the domain be transferred to the 
'new' location? If not, can URL forwarding be set up on the old URLs?

HTTP can provide a unique identifier for an object at a given point in time, 
but it isn't necessarily going to be possible to rely on it always resolving 
to the same object over it's entire lifetime.

> 3) Including special characters in the URL string doesn't seem like a
> good idea.  While they are valid characters, it does take extra
> processing to encode/decode them from layer to layer.

Totally agreed - having colons, etc. in the url is going to lead to problems 
in some circumstances.

> 4) Assigning bitstreams persistent identifiers seems dangerous.  At the
> very least, version control and a history function are required by the
> application and PI system to determine if the PId is actually pointing
> to what was requested.  Also, how are multiple bitstreams handled when
> assigned to an item?  Does each bitstream get a PId?  How does a user
> look at all bitstreams associated together by the item when the PId
> references only a single bitstream?

We had a fair amount of discussion about these issues during the 
architectural review last year - which were largely centered around 
extensions to the existing mechanism in order to reference specific (or 
simply the latest) version of a bitstream as relative to the item.

Whether there is a need to assign an 'actual' PI to individual bitstreams or 
not is very much a policy decision of the repository. Assigning a PI to an 
individual bitstream does not mean that it happens in lieu of assigning one 
to the item itself - so if you want to look at other bitstreams associated 
to the same item, you should use the item PI (and if a user has only been 
given a PI for a specific bitstream, then they could potentially search for 
the item that refers to the bitstream identified by that PI).

As for versioning, again it's a bit of a policy decision, but a PI could be 
assigned to a specific revision (and therefore a new revision would get a 
new PI). You could also have a 'special' PI that would always refer to the 
latest revision.

> As far as having a default PI system out of the box for Dspace, I would
> recommend using a local identifier schema which used the existing URLs.
> Include the Handle PI system in the release as a configurable option,
> but not turned on by default.  This would remove the fake handle being
> assigned to all objects and clean up the default URLs out of the box.

Well, now to be controversial. IMHO, too much importance is being focused on 
PIs. Yes, PIs are important for preservation, but that doesn't mean that 
they have to be treated as something specific and central to DSpace.

PIs are 'just' metadata. and supporting multiple ways to resolve a piece (or 
a combination of pieces) of metadata to an asset - or simplying presenting 
them in display - isn't really that hard.

Now there are special concerns about the handling - ensuring it's presence, 
automatic generation/assignment, ensuring uniqueness (probably) - but that's 
all just a question of providing better workflows and metadata handling. In 
other words, any concerns that we have about how we handle persitent 
identifiers could be applicable to any piece (or combination) of metadata - 
and by that token, solving those issues for all metadata would resolve the 
issues for PIs, just be treating them as 'only' metadata.

This would mean that the only id we need to centrally worry about assigning 
to an asset is a unique id to be resolvable within the repository - ie. a 
UUID, which would likely be unique across all DSpace instances, and as such 
could be maintained across

Re: [Dspace-tech] Persistent identifiers in DSpace -- thoughtsplease

[Graham Triggs]

On Tue, 2007-05-29 at 10:44 +0100, James Rutherford wrote:
> > 3) Including special characters in the URL string doesn't seem like a
> > good idea.  While they are valid characters, it does take extra
> > processing to encode/decode them from layer to layer.
> 
> As I mention on the wiki, my current idea is to have URLs of the form:
> 
> http://dspace.me.ac.uk/uri/hdl:1234/56
> 
> which will resolve to the object with Handle 1234/56, etc. If the
> object also has a DOI with value 7890/12 then the following URL would
> point to the object as well:
> 
> http://dspace.me.ac.uk/uri/doi:7890/12
> 
> It is necessary to include the "hdl:" and "doi:" parts so we can
> distinguish between different persistent identifier mechanisms. The
> values allowed for the persistent identifier are dependent on the
> mechanism we are dealing with, and as far as possible this will be kept
> simple.

Whilst it is necessary to identify the persistent id scheme, that
doesn't mean that using a colon as part of the identifier is necessary
or desirable. Colons - or other 'unusual' characters - will end up
causing problems.

In fact, I don't even see that there is a reason to include 'uri' in the
url. Why not just support the existing:

http://dspace.me.ac.uk/handle/1234/56

for handles, and:

http://dspace.me.ac.uk/doi/7890/12

for DOIs, etc.?

G 
 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB

-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Persistent identifiers in DSpace -- thoughtsplease

[Graham Triggs]

On Tue, 2007-05-29 at 11:43 +0100, James Rutherford wrote:
> I don't see what's so unusual or undesirable about colons. The reasoning
> behind doing it this way was so that the value after "/uri/" is the
> canonical form of the identifier.

The colon is a reserved character, and in this example would have to be
encoded to be strictly valid according to the specifications - which
would then mean it isn't the canonical form.

Not encoding the colon will have the potential to cause problems with
proxies, firewalls, etc.

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Persistent identifiers in DSpace -- thoughtsplease

[Graham Triggs]

On Tue, 2007-05-29 at 12:52 +0100, James Rutherford wrote:
> Well if we're going to be strict, we should escape the value of the
> handle 1234/56 as 1234%2F56. Since DSpace already breaks this rule, I
> didn't deem including a colon as such a great crime ;)

Fair point, and you are probably right. But there is strict and there is
strict... and it isn't entirely clear that the handle should be treated
as a complete unit rather than the separation of prefix and suffix -
globally, that's how they need to be referred to, but then we're
discussing local urls here ;-)

Yes an unescaped slash isn't going to do anything harmful. An unescaped
colon in the middle of the url could easily trigger url parsing bugs and
security problems.

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Persistent identifiers in DSpace -- thoughtsplease

[Graham Triggs]

On Tue, 2007-05-29 at 14:58 +0100, James Rutherford wrote:
> Using UUIDs (as suggested earlier) would *work*, but would produce
> horrid URLs.

Note that I never suggested using UUIDs as part of a URL. What I said is
that UUIDs would give you a robust scheme of internal unique identifiers
- and in having that, the use of all other identifier schemes are
reduced simply to a matter of how you map to/from the UUIDs.

We could easily have an out-of-the-box mapping scheme to non-persistent
'friendly' identifiers if the concern is simply to have cleaner URLs.

But even if UUIDs where exposed in the URLs (in a default installation),
is that necessarily a problem? The ugliness of it would at least
encourage people to think about the issues of id persistence /
assignment in relation to that repository.

By assigning UUIDs as the primary / internal id of all persistent
objects in DSpace, we can use tried and tested, well understood
algorithms to generate IDs that are virtually guaranteed to be unique,
which would open up potential usage / installation scenarios that could
otherwise be impractical. It would also have some consistency with the
JCR specification, and you've got the potential to make them public,
persistent identifiers if that is deemed suitable for a given
installation.

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] DSpace with Oracle 10 Installation Problem.

[Graham Triggs]

Doubt db.name gets involved - that should only change the SQL that is
executed.

I suspect that the db.url and db.driver may be inconsistent (ie. the url
is for Oracle and the driver is for Postgres, or vice versa).

G

On Wed, 2007-05-30 at 14:29 +0100, James Rutherford wrote:
> Quick sanity-check: did you remember to set db.name = oracle in
> dspace.cfg?
> 
> Jim
> 
> On Wed, May 30, 2007 at 09:24:46AM -0400, Stéphane Collin wrote:
> > Hi,
> >  
> > I'm currently trying to install the Dspace 1.4.2 with oracle support on a
> > Red Hat machine.
> > My Oracle Server is on another machine.
> >  
> > I've been trying several times to use the Installation method provided with
> > the source, And even thoug the ojdbc14.jar is in place, I'm constantly
> > getting the following error in the setup_database part of the fresh
> > install_process
> >  
> >  
> > setup_database:
> >  [java] 2007-05-30 09:03:14,928 INFO
> > org.dspace.storage.rdbms.InitializeDatabase @ Initializing Database
> >  [java] 2007-05-30 09:03:15,037 INFO
> > org.dspace.core.ConfigurationManager @ DSpace logging installed using
> > log4j.properties
> >  [java] 2007-05-30 09:03:15,125 FATAL
> > org.dspace.storage.rdbms.InitializeDatabase @ Caught exception:
> >  [java] java.sql.SQLException: No suitable driver
> >  [java] at
> > java.sql.DriverManager.getConnection(DriverManager.java:545)
> >  [java] at
> > java.sql.DriverManager.getConnection(DriverManager.java:171)
> >  [java] at
> > org.apache.commons.dbcp.DriverManagerConnectionFactory.createConnection(Driv
> > erManagerConnectionFactory.java:48)
> >  [java] at
> > org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnect
> > ionFactory.java:290)
> >  [java] at
> > org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPoo
> > l.java:840)
> >  [java] at
> > org.apache.commons.dbcp.PoolingDriver.connect(PoolingDriver.java:175)
> >  [java] at
> > java.sql.DriverManager.getConnection(DriverManager.java:525)
> >  [java] at
> > java.sql.DriverManager.getConnection(DriverManager.java:193)
> >  [java] at
> > org.dspace.storage.rdbms.DatabaseManager.getConnection(DatabaseManager.java:
> > 717)
> >  [java] at
> > org.dspace.storage.rdbms.DatabaseManager.loadSql(DatabaseManager.java:1062)
> >  [java] at
> > org.dspace.storage.rdbms.InitializeDatabase.main(InitializeDatabase.java:71)
> > 
> >  
> >  
> > Anyone have an idea?
> >  
> >  
> >  
> > Stéphane Collin
> >  
> 
> > -
> > This SF.net email is sponsored by DB2 Express
> > Download DB2 Express C - the FREE version of DB2 express and take
> > control of your XML. No limits. Just data. Click to get it now.
> > http://sourceforge.net/powerbar/db2/
> > ___
> > DSpace-tech mailing list
> > DSpace-tech@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/dspace-tech
> 
> 
 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB

-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] handle server on windows

[Graham Triggs]

Hi Karen,

The Unix script won't work (it might be OK under Cygwin) - you can
convert to dos, but if the logged in session is exited, the console app
*will* be closed and the handle server shutdown.

The best way to run the handle server under Windows is to install it as
a service. There isn't anything provided 'out of the box' to do this,
but it's actually quite easy to achieve with the Java Service Wrapper



You should download this and install it somewhere on your server. Then
create a configuration file for it - I've attached a template for you to
get started. You will need to replace the @@@@ sections with
appropriate values at least.

IMPORTANT: This is used to start the wrapper, and won't have access to
the dspace.cfg at this point, so you MUST define the full correct path
names explicitly.

Then, from the wrapper's bin directory, run:

wrapper -i path.to.configuration.file

to install the service.

As it stands, the template will set the service to automatically start
with windows. You can manually start / stop it by running 'net start' or
'net stop'  with the value defined for wrapper.ntservice.name - ie:

net stop handleServerDSPACE

(given the name in the template configuration file).

Or, you can control the service from the services control panel.

G

On Fri, 2007-06-01 at 15:23 +0200, karen janssens wrote:
> Hello,
> 
> I've installed DSpace on my personal computer (running windowsXP) in order to
> explore its possibilities as part of an assignment for my Library and
> Information Science study.
> I really would like to install the Handle Server as well, but ran into
> problems when trying to actually start the server.
> When executing the command 'start-handle-server', i get the message "
> The name specified is not recognized as an internal or external command,
> operable program or batch file."
> 
> I tried already to convert the unix-commands into dos-commands (but have
> problems with the nohup). 
> 
> Also I tried to follow the instructions on handle.net (they give windows/dos 
> commands), but then I encouter the error:
> java.lang.ClassNotFoundException: org.dspace.handle.HandlePlugin
> 
> Does anyone have an idea what could be the best way to make Handle run in 
> DSpace and under Windows?
> 
> Thanks very much!!
> 
> Karen
> 
> 
> 
#
# Wrapper Properties
#

# Give app 5 minutes to startup
wrapper.startup.timeout=300

# This is a hack - it can't run the lock file as a command, but it will remove 
it after 5 seconds ;-)
wrapper.commandfile=@@dspace.dir@@\handle-server\txns\lock

# Java Application
wrapper.java.command=@@path.to.java.exe@@

# Java Main class.  This class must implement the WrapperListener interface
#  or guarantee that the WrapperManager class is initialized.  Helper
#  classes are provided to do this for you.  See the Integration section
#  of the documentation for details.
wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp

# Java Classpath (include wrapper.jar)  Add class path elements as
#  needed starting from 1
wrapper.java.classpath.1=@@path.to.wrapper.jar - this is where you installed 
the wrapper service@@
wrapper.java.classpath.2=@@dspace.dir@@\config
wrapper.java.classpath.3=@@dspace.dir@@\lib\*.jar

# Java Library Path (location of Wrapper.DLL or libwrapper.so)
wrapper.java.library.path.1=@@path.to.the.lib.directory.within.the.wrapper.installation@@

# Java Additional Parameters
wrapper.java.additional.1=-Dlog4j.configuration=log4j-handle-plugin.properties

# Initial Java Heap Size (in MB)
#wrapper.java.initmemory=3

# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=256

# Application parameters.  Add parameters as needed starting from 1
wrapper.app.parameter.1=net.handle.server.Main
wrapper.app.parameter.2=@@dspace.dir@@\handle-server

wrapper.working.dir=@@dspace.dir@@\bin

#
# Wrapper Logging Properties
#
# Format of output for the console.  (See docs for formats)
wrapper.console.format=PM

# Log Level for console output.  (See docs for log levels)
wrapper.console.loglevel=INFO

# Log file to use for wrapper output logging.
wrapper.logfile=@@dspace.dir@@\logs\wrapper.log

# Format of output for the log file.  (See docs for formats)
wrapper.logfile.format=LPTM

# Log Level for log file output.  (See docs for log levels)
wrapper.logfile.loglevel=INFO

# Maximum size that the log file will be allowed to grow to before
#  the log is rolled. Size is specified in bytes.  The default value
#  of 0, disables log rolling.  May abbreviate with the 'k' (kb) or
#  'm' (mb) suffix.  For example: 10m = 10 megabytes.
wrapper.logfile.maxsize=5m

# Maximum number of rolled log files which will be allowed before ol

Re: [Dspace-tech] Accented char in filename problem

[Graham Triggs]

On Wed, 2007-06-06 at 07:49 +0100, Stuart Lewis [sdl] wrote:
> > 2) Add another copy of the bitstream to the item, without the accented char 
> > in
> > the filename, using the "Add Bitstream" functionality and Remove the 
> > bitstream
> > that has the dodgy title - not sure though if the Remove function will also
> > fail because of the accented char(?)
> 
> To me, this seems the best option. The old file should delete cleanly as I
> suspect (although haven't looked) that it uses its ID rather than its
> filename to identify it in the delete request.

I'm still a little perplexed as to why this wasn't working in the first
place.

Looking at the original error message, it had output '??' in place of
the accented character - a sure sign of the UTF-8 encoding not being
handled correctly. Although by clicking through the browse lists, it
would seem that the UTF-8 encoding option has been correctly set in the
Tomcat server.xml (ie., clicking on the author in the browse list
works).

Assuming that URIEncoding has been set correctly in Tomcat,
getPathInfo() *should* return the decoded String - however there is a
bug in pre-4.0.4 where it would treat it as US-ASCII. A bit of Googling
suggests that Jetty might be similarly afflicted.

So, questions:

First, to Michael:

1) Is http://dspace.stir.ac.uk running Tomcat?
a) If so, what version?
b) If not, what is it running?

2) Can it be confirmed absolutely that the URIEncoding option is set
correctly in Tomcat's server.xml?

And to all:

3) Do we need to consider doing a manual UTF-8 decoding inside the
servlet to cope with these issues?

4) Should we simply avoid the getPathInfo() issue and make it a
parameter (ie. /bitstream?file=xxx.pdf)

5) Should we include the bitstream id as part of the url, so that we can
match that instead of the filename (although the name would still be
included as a hint to the browser)

G


-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Fwd: Problem with Data Provider Validation

[Graham Triggs]

Not really. The badArgument is valid when you don't specify a
metadataPrefix, and the noRecordsMatch error is valid when there are no
records that match the date range specified.

If you simply include a from timestamp, eg:

http://dspace.udel.edu:8080/dspace-oai/request?verb=ListIdentifiers&metadataPrefix=oai_dc&from=2001-01-01T00:00:00Z

then it works. I don't think there is anything wrong with the harvesting
code per se, it just looks like it is incorrectly interpreting a from
date when you don't specify one.

Although according to the oaicat.properties below, the earliestDatestamp
is set to 2001-01-01T00:00:00Z - exactly what I'm passing in the above
url, and which works!

G

On Tue, 2007-06-05 at 22:37 -0400, Jodi Schneider wrote:
> Forwarding this back out to the list for better answers to Keith's
> questions.
> 
> http://sourceforge.net/tracker/index.php?func=detail&aid=1688523&group_id=19984&atid=319984
> mentions the TimeStamp function of *Oracle*. Is the bug also
> applicable to Postgres?
> 
> http://dspace.udel.edu:8080/dspace-oai/request?verb=ListIdentifiers&from=2007-04-01&until=2007-05-15&metadataPrefix=oai_dc
> works beautifully.
> 
> However this request returns error code="badArgument". 
> http://dspace.udel.edu:8080/dspace-oai/request?verb=ListIdentifiers
> 
> Can anyone tell Keith how to fix this? Is upgrade to 1.4.2 the answer?
> Or...? 
> 
> -Jodi
> 
> PS-Keith, what happens if you change granularity?
> i.e. in oaicat.properties change
> AbstractCatalog.granularity=-MM-DDThh:mm:ssZ
> to
> AbstractCatalog.granularity=-MM-DD
> -- Forwarded message --
> From: Keith Jones <[EMAIL PROTECTED]>
> Date: Jun 5, 2007 3:41 PM 
> Subject: Re: [Dspace-tech] Problem with Data Provider Validation
> To: Jodi Schneider <[EMAIL PROTECTED]>
> 
> 
> Hi Jodi,
> 
> Thanks for the help. 
> 
> At my institution we are using Postgres, would this have any effect on
> the
> date granularity?
> 
> Are you saying that the harvester will not work, unless you are at
> version
> 1.4.2?
> 
> I've been able to get subsets if the listidentifiers working, atleast
> it 
> will work when I put in a from date but will not work if no date is
> specified.  I'm still getting the following error:
> 
> Code=noRecordsMatch, Description="The combination of the values of the
> from, until, set, and metadataPrefix arguments results in an empty
> list." 
> 
> With the http request of:
> 
> http://dspace.udel.edu:8080/dspace-oai/request, verb=ListIdentifiers,
> metadataPrefix=oai_dc
> 
> But I can get results with the following http request 
> http://dspace.udel.edu:8080/dspace-oai/request?verb=ListIdentifiers&from=2007-04-01&metadataPrefix=oai_dc
> 
> Thanks
> Keith
> 
> -- Forwarded message --
> From: Jodi Schneider <[EMAIL PROTECTED]>
> Date: Jun 5, 2007 9:19 AM
> Subject: [Dspace-tech] Problem with Data Provider Validation
> To: Dspace Tech 
> 
> Keith,
> 
> It looks like your repository is still having this problem. I think
> you will need to upgrade to 1.4.2 or change timestamp granularity.
> 
> It looks like 1.4.2 fixed a bug regarding  OAI timestamps: 
> "The oracleTimeStampFunction did not correctly account for 20
> character
> length timestamps (-MM-DDTHH:mm:ssZ), incorrectly treating them as
> '-MM-DDTHH:mm:ss.FFZ'."
> 
> http://sourceforge.net/tracker/index.php?func=detail&aid=1688523&group_id=19984&atid=319984
>  
> ---
> I compared the file you mention to my default oaicat.properties and it
> looks like one block is duplicated. I wouldn't expect this to harm
> anything.
> 
> AbstractCatalog.oaiCatalogClassName=org.dspace.app.oai.DSpaceOAICatalog 
> AbstractCatalog.recordFactoryClassName=org.dspace.app.oai.DSpaceRecordFactory
> AbstractCatalog.secondsToLive=3600
> 
> 
> AbstractCatalog.granularity=-MM-DDThh:mm:ssZ
> 
> Best luck, and apologies it took so long to figure this out! 
> 
> -Jodi
> Portable Citations for DSpace
> 
> On 5/29/07, Keith Jones < [EMAIL PROTECTED]> wrote:
> 
> I did not chnage anything in the oaicat.properties.
> 
> Here is my properties file:
> 
> 
> ###
> # REMEMBER: ONLY UPDATE THE VERSION IN
> dspace/config/templates 
> # AND THEN RUN dspace/bin/install-configs!
> # DO NOT EDIT THE 'LIVE' VERSION!
> 
> ###
> 
> # OAICat Configuration file - see OAICat documentation for
> details 
> 
> # Text surrounded by two '@' symbols is replaced with the
> corresponding
> # property from dspace.cfg.  For example:
> #
> # http://dspace.udel.edu:8080/dspace
> #
> # would be replaced with the dspace.url property in
> dspace.cfg.
> # When /dspace/bin/install-configs is run, this file will be
> installed in
> the
>  

Re: [Dspace-tech] Fwd: Problem with Data Provider Validation

[Graham Triggs]

Keith,

A bit of digging around the source history, and it turns out that this is 
actually bug #1123934

http://sourceforge.net/tracker/index.php?func=detail&aid=1123934&group_id=19984&atid=119984

It's fixed here:

http://dspace.svn.sourceforge.net/viewvc/dspace?view=rev&revision=1114

Thing is - that should be part of 1.4.1 already.

I would suggest that you double check firstly that you have deployed an 
update OAI war, and that the war contains at least the oaicat.jar that is 
shown in the above url.

Ultimately, this does look like it comes down to a problem with the OAICat 
code, not the DSpace plugin, and I really suspect that your application is 
using an outdated version.

Regards,
G

- Original Message - 
From: "Keith Jones" <[EMAIL PROTECTED]>
To: "Graham Triggs" <[EMAIL PROTECTED]>
Cc: "Jodi Schneider" <[EMAIL PROTECTED]>; "Dspace Tech" 

Sent: Wednesday, June 06, 2007 8:24 PM
Subject: Re: [Dspace-tech] Fwd: Problem with Data Provider Validation



Hi Graham,

I in agreement that the code is working as specified.  My problem occured
when I tried to register my Dspace instance with the official Open
Archives registry.  The registry send a listIdentifiers request with no
from date specified. As such my dspace instance has failed to pass the
registration test.  I'm trying to determine if this is a configuration
problem, or could there be bad data that is effecting the data pull.



On Wed, 6 Jun 2007, Graham Triggs wrote:

> Not really. The badArgument is valid when you don't specify a
> metadataPrefix, and the noRecordsMatch error is valid when there are no
> records that match the date range specified.
>
> If you simply include a from timestamp, eg:
>
> http://dspace.udel.edu:8080/dspace-oai/request?verb=ListIdentifiers&metadataPrefix=oai_dc&from=2001-01-01T00:00:00Z
>
> then it works. I don't think there is anything wrong with the harvesting
> code per se, it just looks like it is incorrectly interpreting a from
> date when you don't specify one.
>
> Although according to the oaicat.properties below, the earliestDatestamp
> is set to 2001-01-01T00:00:00Z - exactly what I'm passing in the above
> url, and which works!
>
> G
>
> On Tue, 2007-06-05 at 22:37 -0400, Jodi Schneider wrote:
>> Forwarding this back out to the list for better answers to Keith's
>> questions.
>>
>> http://sourceforge.net/tracker/index.php?func=detail&aid=1688523&group_id=19984&atid=319984
>> mentions the TimeStamp function of *Oracle*. Is the bug also
>> applicable to Postgres?
>>
>> http://dspace.udel.edu:8080/dspace-oai/request?verb=ListIdentifiers&from=2007-04-01&until=2007-05-15&metadataPrefix=oai_dc
>> works beautifully.
>>
>> However this request returns error code="badArgument".
>> http://dspace.udel.edu:8080/dspace-oai/request?verb=ListIdentifiers
>>
>> Can anyone tell Keith how to fix this? Is upgrade to 1.4.2 the answer?
>> Or...?
>>
>> -Jodi
>>
>> PS-Keith, what happens if you change granularity?
>> i.e. in oaicat.properties change
>> AbstractCatalog.granularity=-MM-DDThh:mm:ssZ
>> to
>> AbstractCatalog.granularity=-MM-DD
>> -- Forwarded message --
>> From: Keith Jones <[EMAIL PROTECTED]>
>> Date: Jun 5, 2007 3:41 PM
>> Subject: Re: [Dspace-tech] Problem with Data Provider Validation
>> To: Jodi Schneider <[EMAIL PROTECTED]>
>>
>>
>> Hi Jodi,
>>
>> Thanks for the help.
>>
>> At my institution we are using Postgres, would this have any effect on
>> the
>> date granularity?
>>
>> Are you saying that the harvester will not work, unless you are at
>> version
>> 1.4.2?
>>
>> I've been able to get subsets if the listidentifiers working, atleast
>> it
>> will work when I put in a from date but will not work if no date is
>> specified.  I'm still getting the following error:
>>
>> Code=noRecordsMatch, Description="The combination of the values of the
>> from, until, set, and metadataPrefix arguments results in an empty
>> list."
>>
>> With the http request of:
>>
>> http://dspace.udel.edu:8080/dspace-oai/request, verb=ListIdentifiers,
>> metadataPrefix=oai_dc
>>
>> But I can get results with the following http request
>> http://dspace.udel.edu:8080/dspace-oai/request?verb=ListIdentifiers&from=2007-04-01&metadataPrefix=oai_dc
>>
>> Thanks
>> Keith
>>
>> -- Forwarded message --
>> From: Jodi Schneider <[EMAIL PROTECTED]>
>> Date: Jun 5, 200

Re: [Dspace-tech] Accented char in filename problem

[Graham Triggs]

On Wed, 2007-06-06 at 14:16 +0100, Michael White wrote:
> > a) If so, what version?
> 
> 4.1.31

I may have misinterpreted the information I saw about Tomcat. It's
possible that the bug was only filed against 4.0.4, not actually fixed
in that release.

The bug appears to have been closed on the 16th Nov 2004 - that would be
2 months after version 4.1.31 was released.

Can you try upgrading to a new version of Tomcat? If you have to stick
with the 4.x series, you could try 4.1.36, released earlier this year:

http://www.apache.org/dist/tomcat/tomcat-4/v4.1.36/bin/

G


-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Oracle

[Graham Triggs]

Hi,

Hm... I don't think this is an SQL issue, I'm suspecting some faulty
data somewhere.

Either your start date or end date is not of the correct format - ie.
2007-01-01 - or you have a value in your metadatavalue that isn't a
valid timestamp (ie. 2007-01-01T23:01:01Z). I'm suspecting that it may
be the latter.

Check the start / end dates anyway, but can you execute this:

SELECT text_value, decode(
  REGEXP_REPLACE(TO_CHAR(text_value), '([0-9])','0'),
  '-00-00T00:00:00Z', 1
  , 0
) AS valid
FROM metadatavalue 
WHERE metadata_field_id = (
 SELECT metadata_field_id
 FROM metadatafieldregistry
 WHERE element = 'date'
 AND qualifier = 'accessioned'
) ORDER BY valid, TO_CHAR(text_value);

against your database. The first items to be listed will either have a
badly formatted timestamp (valid will be 0), or will be an invalid value
(ie. timestamp starting with ). In all other cases, the values
*should* (more or less) be valid).

G

On Mon, 2007-06-11 at 11:32 +0200, luis jose miralls wrote:
> 
> Hello !
> 
> I have a problem with oracle database.
> When i run the the "stat-initial"
> 
> i get the following errors:
> 
> Exception in thread "main" java.sql.SQLException: ORA-01841: (full)
> year must be between -4713 and +, and not be 0 
> 
> at
> oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:125)
> at
> oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:316)
> at oracle.jdbc.driver.T4CTTIoer.processError
> (T4CTTIoer.java:282)
> at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:639)
> at
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:185)
> at oracle.jdbc.driver.T4CPreparedStatement.execute_for_rows
> (T4CPreparedStatement.java:633)
> at
> oracle.jdbc.driver.OracleStatement.execute_maybe_describe(OracleStatement.java:984)
> at
> oracle.jdbc.driver.T4CPreparedStatement.execute_maybe_describe(T4CPreparedStatement.java
>  :535)
> at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1051)
> at
> oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:2984)
> at
> oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3026)
> at
> org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:92)
> at
> org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:92)
> at
> org.dspace.storage.rdbms.DatabaseManager.query(DatabaseManager.java:188)
> at org.dspace.storage.rdbms.DatabaseManager.querySingle
> (DatabaseManager.java:262)
> at
> org.dspace.app.statistics.LogAnalyser.getNumItems(LogAnalyser.java:1264)
> at
> org.dspace.app.statistics.LogAnalyser.processLogs(LogAnalyser.java:535)
> at org.dspace.app.statistics.LogAnalyser.main
> (LogAnalyser.java:327)
> 
> 
> Thanks.
> -
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> ___ DSpace-tech mailing list 
> DSpace-tech@lists.sourceforge.net 
> https://lists.sourceforge.net/lists/listinfo/dspace-tech


-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] File Storage

[Graham Triggs]

On Tue, 2007-06-12 at 22:47 -0400, Don Gourley wrote:
> On Tue, June 12, 2007 9:58 pm, Jimmy wrote:
> > Hi,
> >
> > I want to store the assets(bit stream) on another server but not the DSpace
> > application server .
> >
> > Is there any other solution?
> 
> Here's one: NFS
> Another one, what we use but not exactly 'on another server', is: SAN

More generically - anything you can mount as a general purpose file
system can potentially be used as the asset store.

So, as stated, any NFS or Samba (Windows share) export can be mounted
and used.

You could even create a distributed block device using DR:BD and spread
the asset store over a number of servers. Note that for any block device
that can potentially be mounted concurrently by a number of systems
(DR:BD, SAN, etc.), it is advisable to use a clustered file system on
them (ie. OCFS2, GFS).

Potentially, it could even be a virtual file system - although not open
source, there are solutions that will allow you to mount Amazon's S3
service as a regular filesystem.

And this is all based on using DSpace as it stands - just using the
standard filesystem access (and simply by either changing the assetstore
directory in your dspace.cfg, or using a symbolic link to the mounted
storage you wish to use). Potentially, plugins could provide other
solutions, but I'm not aware of any that are available to use (although
I believe there has been some discussion regarding an Amazon S3 plugin).

G


-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] 1.4.2 Oracle Upgrade Problem

[Graham Triggs]

It look's like it can't find the registrationdata_seq.

Try running:

SELECT registrationdata_seq.nextval FROM dual;

it probably won't work, in which case you will need to create the
sequence. First, run:

SELECT MAX(registrationdata_id) FROM registrationdata;

to see if there are any IDs already in the table - if there are, you
need to create the sequence starting with a higher number (at least 1
larger than the number returned). Then run:

CREATE SEQUENCE registrationdata_seq START WITH 1;

although change that 1 to a higher number if the previous command showed
that it was necessary.

G

On Thu, 2007-06-14 at 12:35 -0700, Brian Helstien wrote:
> Wednesday, I noted that following the directions to modify the Oracle
> database schema didn't succeed.  In trying to actually run the
> application, I am getting Oracle database errors, such as the following
> from the DEBUG dspace.log.  The Oracle database is on a different server
> than Dspace (if that makes any difference) and my question is, and how
> exactly should I go about correctly modifying the schema to support
> 1.4.2?  Help, please?



-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Lucene/Postgre and scalability

[Graham Triggs]

On Mon, 2007-06-18 at 15:10 -0400, Jose Blanco wrote:
> 2.  Does any one have any idea what the performance would be like with
> 12 million records in a lucene environment with or without an
> accompanying database?  And would a dual storage system ( Lucene and
> database ) work well when you have to handle 12 million records
> (performance) ?  

I've said it before, and I'll say it again - monolithic systems (and
that includes the way most RDBMS are set up) don't handle large datasets
particularly well.

OK, that's a sweeping, headline grabbing, statement. The reality is a
fair bit more complicated. But on a quite powerful Oracle setup, it's
possible to have performance issues querying only indexed columns on
tables with as little as 50,000 records.

It all depends on the type of queries you need to perform - how many
components, how selective, ordering requirements, etc.

> 4.  Has any one out there had to do something like this, and if so
> what have you found that works.  One solution that comes to mind is
> Zebra. It is suppose to handle large repositories quite well.  Are
> there any users of Zebra out there that might have an opinion on this?

Haven't heard of this before. But looking at the claimed performance is 
interesting.

For example, they claim to handle in the region of 50 million records, @
around 100GB data size - that's 2KB per record. How large are your
records?

Performance for 'very large databases' (doesn't specify what a very
large DB - let's just assume it's 50million records for now), is
good/acceptable - providing your queries only result in hits of around
1000 to 5000 records. Even at the upper limit, that's 0.01% of the
database. That's pretty damn specific, and I personally wouldn't be
surprised if an average user query was at least 10x less specific - how
does that impact on performance?

If you really want to look at scaling to millions of records, you will
almost certainly want to look at a divide-and-conquer solution. The most
obvious place for you to start would probably be Jargon and GridLucene.

G 
 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB

-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Non-ASCII Unicode and searching in DSpace

[Graham Triggs]

Yes, this browse problem has been discovered before, and there is a
solution to it ;)

The ordering is dependent on the 'sort_' columns of the browse table. By
altering the normalisation of the data that is stored there, you can
affect the way the ordering is done, without the LC_COLLATE side
effects.

Applying this patch:
http://sourceforge.net/tracker/index.php?func=detail&aid=1672065&group_id=19984&atid=319984

(remember to add the ICU4J jar), will decompose the non-ascii characters
into the ascii main character and a separate combining character before
writing it to the database.

It also makes it easy to customize the normalization in other ways - for
example, you could easily strip all the non-ascii combining characters
after decomposition if you wanted. Or do something more advanced - like
obey the correct ordering rules for a specified locale (although this
would be 'baked in' to the database table for everyone, not be per user
based).

The patch above has minimal impact on the existing codebase - IIRC,
there are only three lines in existing code that it alters.

As noted in the comments, there is also Richard Jones' patch:
http://sourceforge.net/tracker/index.php?func=detail&aid=1702233&group_id=19984&atid=319984

which incorporates the above ordering customisation / fixes. It's well
worth considering using, although it would take a little more effort to
switch over to.


G

On Tue, 2007-07-03 at 22:18 +0930, Steve Thomas wrote:
> I’m still investigating this problem – and I can’t imagine that others have
> not also run into this, although it is possible they have simply not
> noticed.
> 
> In our Author browse list we have things like this sequence:
> 
> Hill, Robert Stephen  
> Hill, Roderick Jeffrey  
> Hills, Brian Andrew  
> Hills, F  
> Hill, S. G.  
> Hill, S. J.  
> Hills, K.  
> Hill, S. M.  
> Hill, T.  
> 
> -- you can see that the sequence seems to be ignoring punctuation AND
> spaces, which is not good. The unix sort utility also produces the same
> sequence.
> 
> I suspect this is something to do with the Unix locale settings,
> specifically the environment variable LC_COLLATE. 
> On our RedHat system, this is LC_COLLATE="en_AU.UTF-8"
> On the old Solaris box (where the collation sequence is correct) we have
> LC_COLLATE=en_AU.ISO8859-1
> 
> However, on RedHat the collation from sort is only correct if I set
> LC_COLLATE=C
> 
> 
> So the problem appears to be two-fold: how do we get the browse sort order
> to be correct, including non-ASCII characters, AND how do we search words
> with non-ASCII characters?
> 
> 
> Any and all ideas welcomed. 
> 
> 
> 
> Stephen Thomas,
> Senior Systems Analyst,
> University of Adelaide Library
> UNIVERSITY OF ADELAIDE SA 5005 AUSTRALIA
> Phone: +61 8 830 35190
> Fax: +61 8 830 34369
> Email: [EMAIL PROTECTED]
> URL: http://www.adelaide.edu.au/directory/stephen.thomas
> CRICOS Provider Number 00123M
> 
> ---
> This email message is intended only for the addressee(s) and contains
> information that may be confidential and/or copyright. If you are not the
> intended recipient please notify the sender by reply email and immediately
> delete this email. Use, disclosure or reproduction of this email by anyone
> other than the intended recipient(s) is strictly prohibited. No
> representation is made that this email or any attachments are free of
> viruses. Virus scanning is recommended and is the responsibility of the
> recipient.
> 
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of Steve Thomas
> Sent: Monday, 2 July 2007 3:28 pm
> To: dspace-tech@lists.sourceforge.net
> Subject: [Dspace-tech] Non-ASCII Unicode and searching in DSpace
> 
> Hi.
> 
> We recently moved from a Solaris box to an HP running RHEL. Now I find that
> search and browse no longer work with words containing non-ASCII Unicode
> characters, e.g. the name Mühlhäusler. This doesn’t show up in the browse
> list, and searches fail to find it.
> 
> Was there something we missed in the installation, or is this an artefact of
> using RHEL?
> 
> We’re still on DSpace 1.3.2
> 
> Display is fine, just the searching is a problem.
> 
> 
> Cheers,
> Steve
> 
> Stephen Thomas,
> Senior Systems Analyst,
> University of Adelaide Library
> UNIVERSITY OF ADELAIDE SA 5005 AUSTRALIA
> Phone: +61 8 830 35190
> Fax: +61 8 830 34369
> Email: [EMAIL PROTECTED]
> URL: http://www.adelaide.edu.au/directory/stephen.thomas
> CRICOS Provider Number 00123M
> 
> ---
> This email message is intended only for the addressee(s) and contains
> information that may be confidential and/or copyright. If you are not the
> intended recipient please notify the sender by reply email and immediately
> delete this email. Use, disclosure or reproduction of this email by anyone
> other than the intended recipient(s) is strictly prohibited. No
> representation is made

Re: [Dspace-tech] Searching PDF-scanned documents: Adobe Capture asolution?

[Graham Triggs]

Hi,

The problem with your scanning attempts is that you are just capturing
an image of the page. To have searchable content, you need to perform
optical character recognition on the images.

According to:
http://www.adobe.com/uk/products/acrcapture/

Then yes, this will create PDFs that contain searchable words - although
with all OCR solutions, there is the question of accuracy, and for that
you would need the opinion of someone with experience of using the
product.

G

On Wed, 2007-07-04 at 12:55 +0200, Jennifer Ash wrote:
> Dear Community Members
> 
>  
> 
> The Water Research Commission (WRC, South Africa) is currently
> assessing a pilot installation of DSpace.
> 
> We want to use DSpace to store, search and retrieve all our WRC
> research reports and Water SA (a scientific publication, 4 issues pa)
> issues (this is the primary goal; other collections will most likely
> be added over time).
> 
> We are faced with a problem in that most of our older publications are
> not in electronic format and will have to be scanned.
> 
> Scanning and saving as PDF does not provide a full text searchable
> document in DSpace; I've tried it.
> 
>  
> 
> A product, Adobe Capture, is advertised as a 'tool that teams with
> your scanner to convert volumes of paper documents into searchable
> Adobe Portable Document Format (PDF) files'.
> 
> We are keen to investigate this product but there are no trial
> downloads offered by Adobe.
> 
> Do you have any knowledge of this product? Can you advise on a
> suitable tehnology solution for our problem? Our backlog is vast and
> spans many years, so there are loads of documents that need to be
> scanned.
> 
>  
> 
> I do hope someone can give me advice.
> 
>  
> 
> Kind regards
> 
>  
> 
>  
> 
> Jennifer Ash 
> ……
> Business Systems Manager
> Water Research Commission 
> Private Bag X03 
> GEZINA (Pretoria) 
> 0031 
> Tel: (012) 330-9036 / 330-0340 
> Fax: (012) 330-9010 / 331-2565 
> E-mail: [EMAIL PROTECTED] 
> 
>  
> 
> 
>  
> DISCLAIMER AND CONFIDENTIALITY NOTE: All factual and other information
> within this e-mail, including any attachments relating to the official
> business of the Water Research Commission (WRC), is the property of
> the WRC. It is confidential, legally privileged and protected against
> unauthorized use. The WRC neither owns nor endorses any other content.
> Views and opinions are those of the senders unless clearly stated as
> being that of the WRC. The addressee in the e-mail is the intended
> recipient. Please notify the sender immediately if it has
> unintentionally reached you and do not read, disclose or use the
> content in any way whatsoever. The WRC cannot assure that the
> integrity of this communication has been maintained nor that it is
> free of errors, viruses, interception or interferences.
> 
>  
> 
>  
> 
> 
> -
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> ___ DSpace-tech mailing list 
> DSpace-tech@lists.sourceforge.net 
> https://lists.sourceforge.net/lists/listinfo/dspace-tech 
 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB

-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] [Dspace-general] DSpace Bistream

[Graham Triggs]

On Thu, 2007-07-05 at 16:03 +0100, James Rutherford wrote:
> On Wed, Jul 04, 2007 at 01:13:38PM +0200, Ina Smith wrote:
> > My question relates to the preservation of the bitstream URL: In stead
> > of linking from another database (e.g. our library catalogue) to the
> > Item URL, we would like to link directly to the Bitstream URL. Is
> > there any guarantee that the Bitstream URL will also stay persistent
> > if we decide to export our items to another server one day? We also
> > use the CNRI Handle System.

Actually, that poses an interesting question on the future of the handle
system - it's not unreasonable to have urls / references that are
*relative* to the resolution of a handle, so could there in the future
be a way of passing additional information through to the URL resolved
by the handle system resolver?

(There is also another interesting theoretical question about the issues
of only exporting *some* of the items identified under a given prefix,
but I'll avoid that now).

> The short answer is "no". Bitstreams don't get handles, so if you move
> your DSpace having directly referenced a Bitstream URL, that link will
> either break, or you will have to maintain it yourself (which is a bad
> road to go down). Until this is fixed (and I should point out that not
> everyone thinks it's a problem) you shouldn't use Bitstream URLs
> anywhere unless you're prepared to deal with it when the time comes.

In general, I would agree that it isn't a problem if Bitstreams don't
routinely get handles / persistent / external identifiers.

As long as there are consistent internal identifiers, and a way of
mapping a persistent / external identifier through metadata, then (in
theory) you only need to choose to assign an identifier to the
Bitstream's metadata, and you can guarantee it's persistence. Happily
enough Jim, you've already been doing that ;)

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] DSpace Oracle statement pool

[Graham Triggs]

Stephane,

This is pretty much the best / only course of action for using Oracle
drivers with DBCP.

IMHO, a better solution for Oracle installations would be to bypass the
DBCP code, and use the Oracle provided pooling.

G

On Fri, 2007-07-06 at 09:17 -0400, Tellier, Stephane wrote:
> Hi ,
>  
> could someone confirms to me that the right thing to do for being
> able to import large amount of items in DSpace with Oracle, without
> having the open cursors limit problem, is to set the db.statementpool
> to false in dspace.cfg (if, for instance, we cannot raise the cursor
> limit to a very large number in Oracle) ?
>  
> Thanks a lot.
> -
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> ___ DSpace-tech mailing list 
> DSpace-tech@lists.sourceforge.net 
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Checksum checker not working

[Graham Triggs]

On Thu, 2007-07-05 at 17:40 +0100, Jim Downing wrote:
> Hi Rafa,
> 
> Rafa Carreres wrote:
> > Running [dspace-src]/bin/checker,  I get this error:
> >
> > "Exception in thread "main" java.lang.RuntimeException: Problem 
> > inserting missing bitstreams. ORA-00911: invalid character
> >
> > at 
> > org.dspace.checker.BitstreamInfoDAO.updateMissingBitstreams(BitstreamInfoDAO.java:300)
> > at 
> >   
> > My configuration: Dspace 1.4.1 and Oracle. Previously, I have done a 
> > cleanup of the system.
> >
> > Any idea? 
> 
> 
> The query being run there is a fairly involved chunk of SQL 
> (BitstreamInfoDAO.INSERT_MISSING_CHECKSUM_BITSTREAMS). I can't see any 
> particularly unusual characters in there, so I'd guess that the 
> timestamp parameters set  in .updateMissingBitstreams() are causing the 
> problem. If you can log the exact query that's being run at the Oracle 
> end that might give you a clue. Anyone with Oracle-fu able to shed any 
> light here?


Yep, the checker is pretty bust for Oracle - it's using boolean types,
and LIMIT keyword that don't exist in Oracle.

I've attached a patch to resolve these issues - it's a bit hacky, simply
adding additional variants of the query and switching based on the
configuration parameter (it's going to take too long to properly
Interface the DAOs).

Although, when I was debugging it, the particular invalid character
issue appeared to be the '?' for paramters... but Oracle supports that.
It may just be a quirk of trying to interactively debug the query, and
the PreparedStatement may have actually been falling over on the
comparison of a numeric field to true / false.

Anyway, patch (generated from/against the current 1.4.x branch) is
attached.

G



This email has been scanned by Postini.
For more information please visit http://www.postini.com
Index: src/org/dspace/checker/BitstreamInfoDAO.java
===
--- src/org/dspace/checker/BitstreamInfoDAO.java	(revision 2043)
+++ src/org/dspace/checker/BitstreamInfoDAO.java	(working copy)
@@ -43,6 +43,7 @@
 import java.util.List;
 
 import org.apache.log4j.Logger;
+import org.dspace.core.ConfigurationManager;
 import org.dspace.storage.rdbms.DatabaseManager;
 
 /**
@@ -98,6 +99,21 @@
 + "select 'x' from most_recent_checksum "
 + "where most_recent_checksum.bitstream_id = bitstream.bitstream_id );";
 
+private static final String INSERT_MISSING_CHECKSUM_BITSTREAMS_ORACLE = "insert into most_recent_checksum ( "
++ "bitstream_id, to_be_processed, expected_checksum, current_checksum, "
++ "last_process_start_date, last_process_end_date, "
++ "checksum_algorithm, matched_prev_checksum, result ) "
++ "select bitstream.bitstream_id, "
++ "CASE WHEN bitstream.deleted = 0 THEN 1 ELSE 0 END, "
++ "CASE WHEN bitstream.checksum IS NULL THEN '' ELSE bitstream.checksum END, "
++ "CASE WHEN bitstream.checksum IS NULL THEN '' ELSE bitstream.checksum END, "
++ "?, ?, CASE WHEN bitstream.checksum_algorithm IS NULL "
++ "THEN 'MD5' ELSE bitstream.checksum_algorithm END, 1, "
++ "CASE WHEN bitstream.deleted = 1 THEN 'BITSTREAM_MARKED_DELETED' else 'CHECKSUM_MATCH' END "
++ "from bitstream where not exists( "
++ "select 'x' from most_recent_checksum "
++ "where most_recent_checksum.bitstream_id = bitstream.bitstream_id );";
+
 /**
  * Query that updates most_recent_checksum table with checksum result for
  * specified bitstream ID.
@@ -123,6 +139,11 @@
 + "order by date_trunc('milliseconds', last_process_end_date), "
 + "bitstream_id " + "ASC LIMIT 1";
 
+public static final String GET_OLDEST_BITSTREAM_ORACLE = "SELECT bitstream_id FROM (select bitstream_id  "
++ "from most_recent_checksum " + "where to_be_processed = 1 "
++ "order by date_trunc('milliseconds', last_process_end_date), "
++ "bitstream_id " + "ASC) WHERE rownum=1";
+
 /**
  * Selects the next bitstream in order of last processing end date, ensuring
  * that no bitstream is checked more than once since the date parameter
@@ -135,6 +156,13 @@
 + "order by date_trunc('milliseconds', last_process_end_date), "
 + "bitstream_id " + "ASC LIMIT 1";
 
+public static final String GET_OLDEST_BITSTREAM_DATE_ORACLE = "SELECT bitstream_id FROM (select bitstream_id  "
++ "from most_recent_checksum "
++ "where to_be_processed = 1 "
++ "and last_process_start_date < ? "
++ "order by date_trunc('milliseconds', last_process_end_date), "
++ "bitstream_id " + "ASC) WHERE rownum=1";
+
 /** SQL query to retrieve bitstreams for a given item. */
 private static final String ITEM_BITSTREAMS = "SELECT b2b.bitstream_id "
 + "FROM bundle2bitstream b2b, item2bundle i2b WHERE "
@@ -284,7 +312,10 @@
 {
 LOG.debu

Re: [Dspace-tech] newbie question

[Graham Triggs]

You changed the user that Tomcat is being launched as, right? ;)

If so, then the '/var/lib/tomcat5/work' folder will still have the old
tomcat5:root permissions. You will need to 'chown -hR' it for Tomcat to
be able to compile the JSP files.

G

On Wed, 2007-07-11 at 10:44 +0200, Felix Mayerhofer wrote:
> Hi all!
> 
>  
> 
> I just got through the turorial “Installing Dspace on ubuntu 6.06”
> 
>  
> 
> Everything worked fine so far but when I try to start the web
> interface I get an internal server error:
> 
>  
> 
> HTTP Status 500 - 
> 
>
> __
> typeException report
> 
> message 
> 
> description The server encountered an internal error () that prevented
> it from fulfilling this request.
> 
> exception 
> 
> java.io.FileNotFoundException: 
> /var/lib/tomcat5/work/Catalina/localhost/dspace/org/apache/jsp/index_jsp.java 
> (No such file or directory)
> 
> java.io.FileOutputStream.open(Native Method)
> 
> java.io.FileOutputStream.(FileOutputStream.java:179)
> 
> java.io.FileOutputStream.(FileOutputStream.java:70)
> 
> 
> org.apache.jasper.compiler.Compiler.generateJava(Compiler.java:188)
> 
> org.apache.jasper.compiler.Compiler.compile(Compiler.java:495)
> 
> org.apache.jasper.compiler.Compiler.compile(Compiler.java:476)
> 
> org.apache.jasper.compiler.Compiler.compile(Compiler.java:464)
> 
> 
> org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:511)
> 
> 
> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:295)
> 
> 
> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:292)
> 
> 
> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:236)
> 
> javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
> 
> note The full stack trace of the root cause is available in the Apache
> Tomcat/5.0 logs.
> 
>
> __
> Apache Tomcat/5.0
> 
>  
> 
>  
> 
> The directory org/apache/jsp/ doesn’t exist on my server, what did I
> do wrong?
> 
>  
> 
>  
> 
> Greetz from Austria
> 
> Felix Mayerhofer
> 
> 
> -
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> ___ DSpace-tech mailing list 
> DSpace-tech@lists.sourceforge.net 
> https://lists.sourceforge.net/lists/listinfo/dspace-tech 
 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB

-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Another Internal System Error

[Graham Triggs]

The 'trick' is to go down the stack trace until you find what is calling the 
DatabaseManager - in this case, it's MetadataValue.create(), which means it 
is trying to insert into the MetadataValue table.

This probably means it is complaining about the text_value column, which 
is/should be a CLOB. If it is a CLOB, and it's being reported as an 'OTHER' 
type, then this may be a problem with the drivers. What version of the 
Oracle drivers are you using / what is your db.url?

G

- Original Message - 
From: "Brian Helstien" <[EMAIL PROTECTED]>
To: 
Sent: Friday, July 13, 2007 5:31 PM
Subject: [Dspace-tech] Another Internal System Error


>I too need some analysis of an Internal System Error (Dspace 1.4.2).  In
> attempting to submit something to my test instance, after filling out
> the items on the second step and selecting "next" I got the DEBUG error
> below.  I'm using Oracle and  understand the JDBC  refers to a
> problem with the Oracle tables.  Not sure which table, data type, etc I
> should be investigating.
>
> An internal server error occurred on
> http://cassandra.usc.edu:8180/dspace:
>
> Date:   7/13/07 9:25 AM
> Session ID: B9B08D52FA84D1E7B4D4EE95A2BF9EF0
>
> -- URL Was: http://cassandra.usc.edu:8180/dspace/submit
> -- Method: POST
> -- Parameters were:
> -- dc_type: "Article"
> -- dc_contributor_author_last_0: "Helstien"
> -- dc_relation_ispartofseries_1: "5678"
> -- submit_next: "Next >"
> -- dc_title: "My Test Submission"
> -- dc_contributor_author_first_0: "Brian"
> -- dc_identifier_value_0: "1234-5678"
> -- dc_identifier_qualifier_0: "issn"
> -- workspace_item_id: "82"
> -- dc_relation_ispartofseries_0: "1234"
> -- step: "2"
> -- dc_language_iso: "en_US"
>
>
> Exception:
> java.lang.IllegalArgumentException: Unsupported JDBC type: 
> at
> org.dspace.storage.rdbms.DatabaseManager.execute(DatabaseManager.java:14
> 87)
> at
> org.dspace.storage.rdbms.DatabaseManager.insert(DatabaseManager.java:815
> )
> at
> org.dspace.content.MetadataValue.create(MetadataValue.java:253)
> at org.dspace.content.Item.update(Item.java:1493)
> at
> org.dspace.content.WorkspaceItem.update(WorkspaceItem.java:473)
> at
> org.dspace.app.webui.servlet.SubmitServlet.userHasReached(SubmitServlet.
> java:2209)
> at
> org.dspace.app.webui.servlet.SubmitServlet.processEditMetadata(SubmitSer
> vlet.java:963)
> at
> org.dspace.app.webui.servlet.SubmitServlet.doDSPost(SubmitServlet.java:3
> 82)
> at
> org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.
> java:147)
> at
> org.dspace.app.webui.servlet.DSpaceServlet.doPost(DSpaceServlet.java:105
> )
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applica
> tionFilterChain.java:252)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilt
> erChain.java:173)
> at
> org.dspace.app.webui.filter.RegisteredOnlyFilter.doFilter(RegisteredOnly
> Filter.java:98)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applica
> tionFilterChain.java:202)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilt
> erChain.java:173)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValv
> e.java:213)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValv
> e.java:178)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java
> :126)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java
> :105)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.
> java:107)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:1
> 48)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:86
> 9)
> at
> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.proc
> essConnection(Http11BaseProtocol.java:664)
> at
> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint
> .java:527)
> at
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollow
> erWorkerThread.java:80)
> at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool
> .java:684)
> at java.lang.Thread.run(Thread.java:595)
>
> Brian A. Helstien, SISD, MLS,
> Director, Special Technologies Initiatives,
> IDM, University Libraries
> x06913
> University of Southern California,   (213) 740-6913
> Los Angeles, California, 90089 [EMAIL PROTECTED]
>   Information is independent of media or format
>
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of
> [EMAIL PROTECTED]
> Sent: Friday, July 13, 2007 8:59 AM
> To: dspace-tech@lists.sourceforge.net
> Subject: DSpace-tech Digest, Vol 15, Issue 28
>
> Send DSpace-tech mailing list submissions to
> dspace-tech@lists.sourceforge.net
>
> 

Re: [Dspace-tech] Another Internal System Error

[Graham Triggs]

Firstly, you don't need ocrs12 (won't cause any problems though).

Secondly, ojdbc14 doesn't say anything about the version (14 just means it 
is for JDK 1.4 and above). I suspect that you are using 9i release drivers, 
which are limited in their handling of CLOBs with the thin driver.

Upgrade the driver to the latest 10g release. You can still connect to 
previous database releases, and they improve CLOB identification / handling.

G


- Original Message - 
From: "Brian Helstien" <[EMAIL PROTECTED]>
To: "Graham Triggs" <[EMAIL PROTECTED]>; 

Sent: Sunday, July 15, 2007 10:58 PM
Subject: RE: [Dspace-tech] Another Internal System Error


Graham,

# Database name ("oracle", or "postgres")
#db.name = postgres
db.name = oracle
for connecting to database
db.url = jdbc:oracle:thin:@db-dev1.usc.edu:1521:dspacet

ojdbc14.jar
ocrs12.jar

Brian A. Helstien, SISD, MLS,
Director, Special Technologies Initiatives,
IDM, University Libraries x06913
University of Southern California, (213) 740-6913
Los Angeles, California, 90089 [EMAIL PROTECTED]
   Information is independent of media or format



From: Graham Triggs [mailto:[EMAIL PROTECTED]
Sent: Sun 7/15/2007 11:31 AM
To: Brian Helstien; dspace-tech@lists.sourceforge.net
Subject: Re: [Dspace-tech] Another Internal System Error



The 'trick' is to go down the stack trace until you find what is calling the
DatabaseManager - in this case, it's MetadataValue.create(), which means it
is trying to insert into the MetadataValue table.

This probably means it is complaining about the text_value column, which
is/should be a CLOB. If it is a CLOB, and it's being reported as an 'OTHER'
type, then this may be a problem with the drivers. What version of the
Oracle drivers are you using / what is your db.url?

G

- Original Message -
From: "Brian Helstien" <[EMAIL PROTECTED]>
To: 
Sent: Friday, July 13, 2007 5:31 PM
Subject: [Dspace-tech] Another Internal System Error


>I too need some analysis of an Internal System Error (Dspace 1.4.2).  In
> attempting to submit something to my test instance, after filling out
> the items on the second step and selecting "next" I got the DEBUG error
> below.  I'm using Oracle and  understand the JDBC  refers to a
> problem with the Oracle tables.  Not sure which table, data type, etc I
> should be investigating.
>
> An internal server error occurred on
> http://cassandra.usc.edu:8180/dspace:
>
> Date:   7/13/07 9:25 AM
> Session ID: B9B08D52FA84D1E7B4D4EE95A2BF9EF0
>
> -- URL Was: http://cassandra.usc.edu:8180/dspace/submit
> -- Method: POST
> -- Parameters were:
> -- dc_type: "Article"
> -- dc_contributor_author_last_0: "Helstien"
> -- dc_relation_ispartofseries_1: "5678"
> -- submit_next: "Next >"
> -- dc_title: "My Test Submission"
> -- dc_contributor_author_first_0: "Brian"
> -- dc_identifier_value_0: "1234-5678"
> -- dc_identifier_qualifier_0: "issn"
> -- workspace_item_id: "82"
> -- dc_relation_ispartofseries_0: "1234"
> -- step: "2"
> -- dc_language_iso: "en_US"
>
>
> Exception:
> java.lang.IllegalArgumentException: Unsupported JDBC type: 
> at
> org.dspace.storage.rdbms.DatabaseManager.execute(DatabaseManager.java:14
> 87)
> at
> org.dspace.storage.rdbms.DatabaseManager.insert(DatabaseManager.java:815
> )
> at
> org.dspace.content.MetadataValue.create(MetadataValue.java:253)
> at org.dspace.content.Item.update(Item.java:1493)
> at
> org.dspace.content.WorkspaceItem.update(WorkspaceItem.java:473)
> at
> org.dspace.app.webui.servlet.SubmitServlet.userHasReached(SubmitServlet.
> java:2209)
> at
> org.dspace.app.webui.servlet.SubmitServlet.processEditMetadata(SubmitSer
> vlet.java:963)
> at
> org.dspace.app.webui.servlet.SubmitServlet.doDSPost(SubmitServlet.java:3
> 82)
> at
> org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.
> java:147)
> at
> org.dspace.app.webui.servlet.DSpaceServlet.doPost(DSpaceServlet.java:105
> )
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applica
> tionFilterChain.java:252)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilt
> erChain.java:173)
> at
> org.dspace.app.webui.filter.RegisteredOnlyFilter.doFilter(RegisteredOnly
> Filter.java:98)
> at
> org.apache.catali

Re: [Dspace-tech] Expose DSpace as Tomcat ROOT application (was Re:First name, family name order)

[Graham Triggs]

Actually, Windows kind of supports symbolic links. But (IIRC) you need
the resource kit to be able to create them.

Even then, I'm not sure if Tomcat will follow them - btw, Tomcat won't
follow a symbolic link under Linux, unless you have it enabled in the
config (allowLinking must be set to true for the Context - it's false by
default).

G

On Thu, 2007-07-19 at 10:22 +0100, James Rutherford wrote:
> When I said "link", I meant in the file system sense. In linux, you
> would do this with a symblic link ('ln -s dspace ROOT' in the webapps
> directory). I'm not sure how you'd do it in Windows. Probably a
> shortcut.
> 
> cheers,
> 
> Jim
> 
> On Thu, Jul 19, 2007 at 05:17:13PM +0800, Jayan Chirayath Kurian wrote:
> > Hi! Jim,
> > 
> > After renaming the ROOT directory, Could you please suggest how to make the 
> > link from the dspace directory to ROOT to avoid copying contents. Is this 
> > using the context parameter.
> > 
> > In the tomcat server.XML I was trying with context but was not getting 
> > through.
> > 
> > 
> > 
> >  > crossContext="false" debug="0" reloadable="false"> 
> > 
> > 
> > 

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] [vote] Do we want to assign external identifiers (Handles) to files?

[Graham Triggs]

On Thu, 2007-07-19 at 12:03 -0400, Robert Tansley wrote:
> +1 on ability to assign arbitrary external IDs to bitstreams.
> 
> +1 on ability to assign 'hierarchical' external IDs to bitstreams (for
> graceful fallback if files are deleted etc).

Minor point, but you don't actually need a hierarchical external ID to
do this. The external id is associated with the file, not part of it,
and so deleting the file doesn't mean that the external id is or has to
be deleted. So, entirely possible for the external id to be retained by
the system, but reassigned to a 'graceful fallback' state.

> -1 on assigning Handles to bitstreams in 'out of the box' config.

Essentially, agree with the voting.

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] [vote] Do we want to assign external identifiers (Handles) to files?

[Graham Triggs]

On Thu, 2007-07-19 at 10:06 -0600, Zhiwu Xie wrote:
> Then what will happen if I remove one file from an item and add
> another file? Will the new file get the old handle or a new one or I
> can choose?
> 

How deletion of objects assigned external ids is not something that
DSpace should force on to a repository, although it would need to have
some kind of sensible default.

Although it isn't even the case that a repository can or should have a
single way of dealing with removal of an externally identified file -
the appropriate course of action would be influenced by the factors that
led to the removal / replacement.

Possibilities that should be supportable:

* reassigning the existing id (handle) to a new file
* provide a fallback mapping - for example, to the item
* reporting that the id is invalid



G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] [vote] Do we want to assign external identifiers (Handles) to files?

[Graham Triggs]

From: "James Rutherford" <[EMAIL PROTECTED]>
> This is, in my humble opinion, pure evil. How can you consider something
> to be an *identifier* if you can't actually guarantee that it identifies
> something?

I absolutely agree. But how can you guarantee that it resolves to what it is 
meant to be identifying if you completely disallow the possibility to 
reassign it?

I was tempted to say that you shouldn't be allowed to delete a file that has 
an external identifier (or at least that the default implementation 
shouldn't). As soon as I realised that wouldn't be possible, you have to 
consider the possibility of reassigning the handle.

Remember, that such a reassignment is (or rather should only be used for) 
altering the resolution of the identifier - which doesn't automatically mean 
that you are conceptually changing what it identifies.

G 

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] [vote] Do we want to assign external identifiers (Handles) to files?

[Graham Triggs]

From: "James Rutherford" <[EMAIL PROTECTED]>
> Assigning or displaying? I find it hard to believe that you actually
> have a problem with giving identifiers to files, but I can understand
> why you might not want your users to know about them.

Because I don't believe that an identifier should be assigned to something 
unless you are accepting the possibility for it to be used - there are too 
many implications to it's assignment.

G 

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] [vote] Do we want to assign external identifiers (Handles) to files?

[Graham Triggs]

From: "James Rutherford" <[EMAIL PROTECTED]>
>> I absolutely agree. But how can you guarantee that it resolves to what it
>> is meant to be identifying if you completely disallow the possibility to
>> reassign it?
>
> I'd flip this around and say how can you guarantee that it resolves to
> what it is meant to be identifying if you *do* allow the possibility to
> reassign it. Oh, what a can of worms!

You can't. But that isn't the issue. If you are going to have the situation 
where an id may not resolve correctly, then you have to have the tools to be 
able to correct - even if that can create problems through misuse.

>> I was tempted to say that you shouldn't be allowed to delete a file that
>> has an external identifier (or at least that the default implementation
>> shouldn't). As soon as I realised that wouldn't be possible, you have to
>> consider the possibility of reassigning the handle.
>
> This isn't actually strictly true. Once we have versioning, it could
> well be impossible (presumably at the discretion of the repository
> curator) to delete *anything*, only to be able to create a new "head"
> version of the container that doesn't hold any reference to the file you
> wanted to delete.

As nice as that would be in theory, and even if it is the likely 'normal 
operation', you will always have to cater for being able to completely erase 
a file or item (ie. legal issues).

> Remember that in systems with versioning, deletion is
> a very different concept to systems where versioning isn't supported.
> The points I have made so far assume we are working with a system that
> supports versioning.

Yes, the external id could refer - and continue to refer - to a 'deleted' 
but still accessible file. But bear in mind that we should make no 
assumption of what external id system(s) are used for the assignment, and 
that system may not be providing a persistent identifier. So we can't assume 
what the appropriate behaviour of handling that id is on file / item 
deletion.

>> Remember, that such a reassignment is (or rather should only be used for)
>> altering the resolution of the identifier - which doesn't automatically
>> mean that you are conceptually changing what it identifies.
>
> Danger danger! Surely we would just be giving our adopters enough rope
> with which to hang themselves by doing this. It is pretty obvious that
> people will never use things the way we've decided that they should, no
> matter how much we jump up and down and tell them that it's the wrong
> thing to do.

True - but I could argue that by even having the ability to assign external 
/ persistent identifiers to anything you are giving adopters enough rope to 
hang themselves.  But having them is also a fundamental part of 
preservation, and they are likely hanging themselves if they don't use them 
(appropriately).

There are so many issues that I don't think it's possible to ever write a 
system where it would be impossible for adopters to not hang themselves 
(with enough functionality to sustain a diverse community). The best we can 
do is minimize the potential for these problems in 'normal' operation, and 
provide extra (separate) functionality that can try to correct problems that 
do arise.

G 

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] [vote] Do we want to assign external identifiers(Handles) to files?

[Graham Triggs]

On Tue, 2007-07-24 at 09:22 +0100, James Rutherford wrote:
> As I see it there are
> (broadly) two camps: those who believe that every meaningful tier in the
> DSpace content hierarchy should get external identifiers, and those who
> don't (or at least those who can't decide and so want it to be
> configurable).

Do you mind if I take my tent and pitch a little further down the road?
Partly because I've seen images of the swollen rivers over your way, but
mostly because I can decide and that's why I want it configurable ;)

> Users (and administrators) crave consistency. If we make this assignment
> configurable, there is no guarantee of consistency of application
> between collections, or even in a single collection over extended
> periods of time.

They crave accuracy as well, and consistency isn't the same thing ;)

> The configurable parameters (if we are going to please
> everyone) would be:
> 
>  * whether or not to assign external identifiers at all
>  * which external identifier system to use by default
>  * whether or not external identifiers are re-assignable
>  * whether or not new "versions" of objects get new identifiers
>  * which tiers in the content hierarchy get identifiers (if any)
> 
> I'm sure I've missed a few, but does that sound like something that is
> reasonable to want / implement / support?

Providing the options that modify the use of an identifier system apply
on a per-system basis, that sounds like a reasonable list of what should
be possible.

But, I think we are getting a little tied up around the idea that it may
only be a single implementation that has all these possibilities
available as configuration options - and that need not be the case at
all.

ie. a pluggable 'ExternalIdentifierManager', which supports managing a
single indentifier system (configured by default to be handles), that
out-of-the-box replicates the existing behaviour (EIDs for Items, not
bitstreams) and can easily be configured to also assign EIDs to
bitstreams. Beyond that, more advanced cases can be handled not by
adding more and more configuration options, but by switching out the
implementation.

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] [vote] Do we want to assign externalidentifiers(Handles) to files?

[Graham Triggs]

On Tue, 2007-07-24 at 11:12 -0400, Robert Tansley wrote:
> The problem is I'm not sure how easy it is to disentangle the logic of
> assigning IDs (what gets an ID of what form) and minting IDs.  If the
> 'Handle' piece of the implementation could be as simple as a 'mint'
> method (i.e. if all identifiers were context-free) it would be easy to
> abstract out.  However, IDs may depend both on the object type and
> related objects -- e.g. bitstream IDs may include the item ID as a
> path component, or the version number etc -- and the ID scheme itself.

IDs will (? should!) always be free of the context of the process of
assignment.

Yes, assigning IDs will need access to the full and current state of the
data model, but there should be an API for interrogating the model as
necessary, which then wouldn't require the assignment and minting logic
to be tied together.

You would still be left with a minting implementation that would need to
understand the relationships of the model, and would only be able to
mint IDs for a set of pre-programmed circumstances - but for the most
part if your ID is going to be minted from such information, you are
going to have limitations on what it can cope with anyway.

But...

> You could create a flexible Handle system class that could sit behind
> a couple of different implementations of the above (e.g. a
> context-free one, and one that assigns contextual bitstream IDs, for
> example) but I find it hard to believe that a single interface would
> be sufficient for all different ID schemes (Handle, info:, PURL, UUID,
> ...)

...how much are we considering for implementation now, and what needs to
be left over for our idealistic system (2.0)?

My answer to the above is that it should all be a problem of accessing
the model / metadata - you interrogate the model, you write the ID
directly into the metadata attached to the object that you are creating
an ID for, etc. and a single interface would cope with all potential
schemes (at least to the extent that the schemes could be applied).

We aren't going to have that just yet, but any broadly similar API that
could at least handle the simplistic EID cases that would be thrown at
it in the short term, and could evolve along with the data model would,
imho, be the way to go.

G 
 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB

-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Problem with new submission

[Graham Triggs]

Daniel and Kasia,

What version of the Oracle JDBC drivers are you running?

The 1.4.2 code is only tested/supported with the 10.2.x series of JDBC
drivers (it doesn't matter if the database software is older, the new
drivers will still work).

Also, are you using a 'thin' connection string (jdbc:oracle:thin), or
'oci'?

G

On Mon, 2007-07-30 at 14:54 -0700, Daniel Fernÿe1ndez wrote:
> Hi:
> 
> When trying to make a new submission, after the first four steps
> (Describe, Describe, Describe, Upload) the "Verify" page shows almost
> all data as "null" values. Does somebody know how this can be possible
> or how can I solve it?
> 
> I suspect that this has something to do with "getSubmissionInfo"
> method in the "SubmitServlet" class, when trying to create the new
> "SubmissionInfo" object.
> 
> I'm using D-Space 1.4.2 and Oracle 9.2.0.4 running on RedHat
> Enterprise 3.x
> 
> All input is apreciated.
> 
> 
> 
> __
> Correo Yahoo!
> Espacio para todos tus mensajes, antivirus y antispam ¡gratis! 
> Regístrate ya - http://correo.espanol.yahoo.com/ 
> -
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >>  http://get.splunk.com/
> ___ DSpace-tech mailing list 
> DSpace-tech@lists.sourceforge.net 
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Handle problem - HTTP 403 error after upgraded to 1.4

[Graham Triggs]

Hi,

Haven't heard of that one before! If you are going to 
http://hdl.handle.net/x and getting a 403, then it sounds like there 
is either a problem with your access to the handle.net proxy (nothing to 
do with your DSpace installation), or the handle is resolving, and it is 
forwarding to your DSpace instance which is then giving the 403.

If it's the latter - and it probably is - then the url in the browser 
should have changed to that of your DSpace instance (or, with Firefox, 
you could use the LiveHTTPHeaders addon, and you would see the redirect 
from the handle proxy).

It might be that your dspace.url is wrong in your dspace.cfg file - is 
it definitely the url of your installation, and is it referencing the 
right protocol (http:// vs https://)?

G

Philip Wong wrote:
> Hi,
>  
> We encountered a strange handle problem after upgrading from 1.3.2 to 
> 1.4.1. It's not the usual non-resolving problem. When trying to 
> resolve a handle, it went into a HTTP 403 error and it displayed
>  
> "You are not authorized to view this page."
>  
> Our IP, server name, the config.dct file and the handle.prefix 
> remained the same in upgrading.
>  
> The error log looks normal:
> "2007/08/09 11:51:22 HKT" 25 Started new run.
> Saving global values to: /home/dspace/.handle/root_info
> "2007/08/09 11:51:23 HKT" 50 unspecified max_handlers count, using 
> default: 200
> and the handle process is running:
> dspace2543  0.0  0.4  4196 1024 ?S11:51   0:00 /bin/sh 
> /dspace/bin/dsrun -Dlog4j.configuration=log4j-handle-plugin.properties 
> net.handle.server.Main /dspace/handle-server
> All other areas are fine after upgrading to 1.4 except this handle 
> resolving problem.
> We tried solving by killing the processes and reboot but the problem 
> persists. Does anyone get any idea?
> Thanks!!
>  
> Philip Wong
> Library, CityU of Hong Kong
> 
>
> -
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >>  http://get.splunk.com/
> 
>
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>   

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] UTF-8 encoding in search mystery

[Graham Triggs]

On Thu, 2007-08-23 at 07:23 +0100, Stuart Lewis [sdl] wrote:
> Looking at the headers from your server, it says the server is apache 2, so
> I am guessing you are using mod_jk?
> 
> If so, have you got the following option set?
> 
> JkOptions +ForwardURIEscaped
> 
> If not, it will likely fix your problem.

Or the URIEncoding="UTF-8" attribute isn't being applied to the
Connector entry in Tomcat's server.xml.

If you are using Apache2/mod_jk, then you'll want to apply it to the JK
connector, and not just the HTTP one.

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Problem with statistics

[Graham Triggs]

Hi,

Well, the relevant part of this is the only thing that's not in
English!! But the context is clear enough - it's having trouble parsing
a valid date.

Can you confirm that the start / end dates that you are passing to the
script are of the form '-mm-dd'.

If your input is valid, then you've got a problem with one of the
metadata value fields. You'll need to check the text_value of all the
dc.date.accessioned entries (probably metadata_field_id=11). It looks
like the offending item is of the correct format (-mm-ddThh:mi:ssZ),
but contains invalid data.

G

On Mon, 2007-09-10 at 08:49 +, Andrea Garrido Fernández wrote:
> Hi:
> I have a problem when I try to execute the perl scripts for generating 
> statistics.
> When I try to run the LogAnalyser class with a start or end date, I have an 
> error like this:
> 
> java.sql.SQLException: ORA-01841: el valor (completo) del año debe estar 
> entre -4713 y +, y no debe ser igual a 0
> 
>   at 
> oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
>   at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:331)
>   at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:288)
>   at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:745)
>   at 
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:216)
>   at 
> oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:966)
>   at 
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1062)
>   at 
> oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPreparedStatement.java:850)
>   at 
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1134)
>   at 
> oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3339)
>   at 
> oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3384)
>   at 
> org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:92)
>   at 
> org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:92)
>   at 
> org.dspace.storage.rdbms.DatabaseManager.query(DatabaseManager.java:259)
>   at 
> org.dspace.storage.rdbms.DatabaseManager.querySingle(DatabaseManager.java:384)
>   at 
> org.dspace.storage.rdbms.DatabaseManager.querySingle(DatabaseManager.java:395)
>   at 
> org.dspace.app.statistics.LogAnalyser.getNumItems(LogAnalyser.java:1264)
>   at 
> org.dspace.app.statistics.LogAnalyser.processLogs(LogAnalyser.java:535)
>   at org.dspace.app.statistics.LogAnalyser.main(LogAnalyser.java:327)
> --
> 
> I have tried many things, incluiding the solution given in this forum 
> before, using  the  REGEXP_REPLACE , but I find it impossible because oracle 
> don´t recognize this expression.
> Colud yo help me, please? I am a bit lost.
> Thanks,
> Andrea
> 
> _
> Dale rienda suelta a tu tiempo libre. Mil ideas para exprimir tu ocio con 
> MSN Entretenimiento. http://entretenimiento.msn.es/
> 
> 
> -
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Problem with statistics

[Graham Triggs]

Andrea,

It looks like this is an optimizer issue in Oracle - ie. that it is
applying the TO_TIMESTAMP to all the records first, rather than just
those that match the metadata_field_id.

I've tried rewriting the query so that it uses an inline view to obtain
all the metadata records for the date accessioned field, and then apply
the TO_TIMESTAMP to that... but bizarrely that doesn't seem to be
working either.

For now, I suggest that you separate out:

SELECT metadata_field_id  FROM metadatafieldregistry  WHERE element =
'date' AND qualifier = 'accessioned'

As a distinct query, store the result in a variable, and then replace
the:

metadata_field_id = ( 
SELECT metadata_field_id  FROM metadatafieldregistry  WHERE element =
'date' 
  AND qualifier = 'accessioned')

with a "...metadata_field_id = " + variable + "..."

That should at least make it work. File a bug report, and add list a
reference to it under Bug Fixes / To Do here:

http://wiki.dspace.org/index.php/Next_Release_Status

and I'll address it properly for the 1.5 release when I check / add-in
the Oracle support.

G


On Tue, 2007-09-11 at 10:32 +, Andrea Garrido Fernández wrote:
> Sorry, I have realized that the query that doesn't work is :
> 
> SELECT COUNT(*) AS num FROM item WHERE in_archive = 1 AND withdrawn = 0  AND 
> item_id IN ( SELECT item_id FROM metadatavalue WHERE metadata_field_id = ( 
> SELECT metadata_field_id  FROM metadatafieldregistry  WHERE element = 'date' 
>   AND qualifier = 'accessioned')  AND TO_TIMESTAMP( TO_CHAR(text_value), 
> '-mm-dd"T"hh24:mi:ss"Z"' ) > TO_DATE('2007-09-01', '-MM-dd')  AND 
> TO_TIMESTAMP( TO_CHAR(text_value), '-mm-dd"T"hh24:mi:ss"Z"' ) < 
> TO_DATE('2007-10-21', '-MM-dd') )  AND item_id IN ( SELECT item_id FROM 
> metadatavalue WHERE text_value LIKE '%Thesis or Dissertation%' AND 
> metadata_field_id = ( SELECT metadata_field_id  FROM metadatafieldregistry  
> WHERE element = 'type'  AND qualifier IS NULL) )
> 
> This query gives me error in dspace and in my sql+ worksheet.
> 
> Sorry and thank yoy very much,
> Andrea Garrido
> 
> 
> 
> 
> 
> >From: Graham Triggs <[EMAIL PROTECTED]>
> >To: Andrea Garrido Fernández <[EMAIL PROTECTED]>
> >CC: dspace-tech@lists.sourceforge.net
> >Subject: Re: [Dspace-tech] Problem with statistics
> >Date: Tue, 11 Sep 2007 09:42:50 +0100
> >
> >Hi,
> >
> >Well, the relevant part of this is the only thing that's not in
> >English!! But the context is clear enough - it's having trouble parsing
> >a valid date.
> >
> >Can you confirm that the start / end dates that you are passing to the
> >script are of the form '-mm-dd'.
> >
> >If your input is valid, then you've got a problem with one of the
> >metadata value fields. You'll need to check the text_value of all the
> >dc.date.accessioned entries (probably metadata_field_id=11). It looks
> >like the offending item is of the correct format (-mm-ddThh:mi:ssZ),
> >but contains invalid data.
> >
> >G
> >
> >On Mon, 2007-09-10 at 08:49 +, Andrea Garrido Fernández wrote:
> > > Hi:
> > > I have a problem when I try to execute the perl scripts for generating
> > > statistics.
> > > When I try to run the LogAnalyser class with a start or end date, I have 
> >an
> > > error like this:
> > >
> > > java.sql.SQLException: ORA-01841: el valor (completo) del año debe 
> >estar
> > > entre -4713 y +, y no debe ser igual a 0
> > >
> > >   at
> > > 
> >oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
> > >   at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:331)
> > >   at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:288)
> > >   at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:745)
> > >   at
> > > 
> >oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:216)
> > >   at
> > > 
> >oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:966)
> > >   at
> > > 
> >oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1062)
> > >   at
> > > 
> >oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPreparedStatement.java:850)
> > >   at
> > > 
> >oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1134)
> > >   

Re: [Dspace-tech] RE : Increasing upload.max

[Graham Triggs]

Jose,

You can't just change the variable that is used to hold the return from
getIntProperty() / pass to setSizeMax(). The method getIntProperty()
returns an int, and will only ever be able to return values that fit
into an int.

You'll need to add a getLongProperty() to the ConfigurationManager that
returns a correctly parsed long value if you want to use values that
exceed the capacity of an int.

G

On Wed, 2007-09-12 at 14:55 -0400, Blanco, Jose wrote:
> That was the porblem.  I was able to load 1.8 GB but not 2GB.  I guess
> it was too close to the setting (2147483647).  I saw that the
> FileUploadRequest uses this value to pass to the method setSizeMax of
> the class FileUploadBase and that that method accepts a long.  So I
> changed the DSpace code from:
>  
>   int maxSize = ConfigurationManager.getIntProperty("upload.max");
>  
> to
>  
>   long maxSize = ConfigurationManager.getIntProperty("upload.max");
>  
> And then changed to upload.max to 3147483647, but I'm seeing the same
> error when I try to create a collection.  Not sure why that would be
> happening since the type long should be able to accomodate this
> number.
>  
> -Jose
> 
> 
> 
> __
> From: Dupriez Christophe [mailto:[EMAIL PROTECTED] 
> Sent: Wednesday, September 12, 2007 11:07 AM
> To: Blanco, Jose
> Subject: RE : [Dspace-tech] Increasing upload.max
> 
> 
> 
> Could it be that you exceed the maximum of a signed 32 bits integer?
> It is 2147483647
>  
> Have a nice day!
>  
> Christophe Dupriez
> 
> "Blanco, Jose" <[EMAIL PROTECTED]> a écrit :
> I want to set upload.max to 3536870912 and test loading
> different file
> sizes during a submittal, but I keep getting this error when I
> try to
> create a collection to test this on:
> 
> java.io.IOException: the request was rejected because its size
> (1251)
> exceeds the configured maximum (0)
> at
> org.dspace.app.webui.util.FileUploadRequest.(FileUploadRequest.jav
> a:133)
> at
> 
> org.dspace.app.webui.servlet.admin.CollectionWizardServlet.processBasicI
> nfo(CollectionWizardServlet.java:451)
> at
> 
> org.dspace.app.webui.servlet.admin.CollectionWizardServlet.doDSPost(Coll
> ectionWizardServlet.java:150)
> at
> 
> org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.
> java:167)
> at
> 
> org.dspace.app.webui.servlet.DSpaceServlet.doPost(DSpaceServlet.java:105
> )
> at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
> 
> 
> Is there a max to upload.max? 
> 
> Thank you!
> 
> Jose
> 
> 
> -
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
> 
> 
> 
> 
> 
> 
> __
> Ne gardez plus qu'une seule adresse mail ! Copiez vos mails vers
> Yahoo! Mail 
> -
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
> ___ DSpace-tech mailing list 
> DSpace-tech@lists.sourceforge.net 
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Dspace not writing to dspace.log file

[Graham Triggs]

Hi,

It's possible that filter-media had something to do with it, but under 
normal circumstances it shouldn't.

What is likely here is that something has renamed or replaced the 
dspace.log file that Tomcat was writing to. As Tomcat retains an open 
handle on the log file, it will continue writing to the 'old' file, even 
though you can only see the new one.

If you have a dspace.log.1, take a look at it - there is a distinct 
possibility that your 'missing' log entries may be in there.

G

Blanco, Jose wrote:
> For some reason the dspace.log file is not being written to by dspace.
> I ran filter-media around 12:30, and I lost track of when it ended, but
> at 1:05 is the last entry in dspace.log from filter-media.  Is there any
> reason why dspace would stop writing to the dspace.log file under these
> circumstances?  The permissions look fine:
>
> -rw-r--r--  1 dspace dspace 548742 Sep 13 13:05 dspace.log
>
> I know if I restart tomcat it would start writing again, but why would
> it stop?  Did running filter-media have something to do with it?
>
> Thanks!
> Jose 
>
>
> -
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>   

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Handle server + funky firewall

[Graham Triggs]

On Tue, 2007-09-18 at 22:16 +0100, [EMAIL PROTECTED] wrote:
> On the configuration wizard, i gave the external IP, and sent the zip
> file to handle.net
> Then, I've changed in config.dct the "bind_address" the external IP to
> the 10.x.x.x 

Yes, you should initially register the external IP address.

Note that you can simply remove the bind_address entry from the
config.dct, and the handle server will bind to all available addresses
on that machine.

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Clustering of Dspace servers

[Graham Triggs]

On Wed, 2007-09-19 at 11:51 +0200, Christian Voelker wrote:
> So in case Sequoia and Terracotta turn out not to
> be the way to go, then Slony deserves a second look.
> But I agree that all this doesnt sound to be a job
> for an afternoon.

NB: Terracotta is not part of a solution for replicating the database.
It would / could be used for replicating session data between multiple
web servers. In that case it is complimentary to either Sequoia or
Slony, not an alternative.

G

-- 
Graham Triggs
Technical Architect
Open Repository

BioMed Central Ltd
Middlesex House
34-42 Cleveland Street
London, W1T 4LB 
 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB

-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Extra ">" characters spontaneously littering pages

[Graham Triggs]

On Wed, 2007-09-19 at 10:35 -0400, Shane Beers wrote:
> Something quite unexpected greeted me this morning. I pulled up our
> repository page (http://mars.gmu.edu) to find ">" characters on our
> pages where formerly there were none. They seem to exist at the end of
> some lines, such as the left navbar text and inside picklists. We
> didn't change a thing on our production server and I'm kind of at a
> loss.

Shane,

I just viewed the source, and there are actually '>' characters inside
the  tags.

Something seems to be inserting them during generation, and I would
suspect a change to /jsp/layout/location-bar.jsp

G

-- 
Graham Triggs
Technical Architect
Open Repository

BioMed Central Ltd
Middlesex House
34-42 Cleveland Street
London, W1T 4LB

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Dspace: IllegalArgumentException

[Graham Triggs]

Hi,

Delving back through the history, it appears that you are using revision 
#1312 of Browse.java (not sure what release of DSpace that relates to!)

http://dspace.svn.sourceforge.net/viewvc/dspace/branches/dspace-1_4_x/dspace/src/org/dspace/browse/Browse.java?view=log

As you will see, the next revision (1342) specifically has a 'fix' for 
an Oracle compatibility in precisely this location.

Try applying the changes as described here:

http://dspace.svn.sourceforge.net/viewvc/dspace/branches/dspace-1_4_x/dspace/src/org/dspace/browse/Browse.java?r1=1312&r2=1342

and if that doesn't work, we'll assist you further.

G

--
Graham Triggs
Technical Architect
Open Repository

Manuel Echeverry wrote:
>
> Hi
>
>  
>
> In our university we have 3 instances of dspace
>
>  
>
> http://dspace.icesi.edu.co/dspace/
>
> http://dspace.icesi.edu.co/desarrollo/
>
> http://dspace.icesi.edu.co/academico/
>
>  
>
> The 3 of them are installed on the same server running linux and 
> sharing an oracle 10g database. On the past flew days the 3 instances 
> are experiencing son estrange behavior. If you enter every of them and 
> try to open say 2 times the same link of the menu (for example try to 
> open the search link twice) you will get an internal error (the same 
> happens if you repeat a search query).
>
>  
>
> Here I share to the list, a sample fragment of one of the dspace 
> instances Logs. As you will see the errors are always one of this 2:
>
>  
>
> * java.lang.IllegalArgumentException: Value is not an long
> * java.lang.NullPointerException
>
>  
>
> I appreciate any suggestions of what is going on.
>
>  
>
> _
>
> Manuel Echeverry
>
> Dirección de servicios y recursos de información
>
> Soporte a Biblioteca
>
>  
>
>  
>
> 
>
> -
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
> 
>
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Problems using legacy oracle database (in latin1)

[Graham Triggs]

Jose,

It's vaguely possible to run it using national character set columns
(NCHAR, NCLOB), to allow for a different character encoding (the ncs
columns could be UTF-8, whilst everything else remain Latin-1).

However, we've been down that route, and it involves some very nasty
customisations to the core DSpace classes that are quite hard to
maintain.

We have since switched to using a UTF-8 instance, and I would strongly
recommend anyone to do the same if it at all possible.

I realise you (like us), may have legacy issues that prevent the use of
UTF-8 for a shared database instance, but there are few reasons why you
couldn't have a separate instance specifically for DSpace setup as UTF-8
on the same hardware (at no additional cost).

G

On Thu, 2007-10-11 at 12:12 +0200, Jose Luis Fernandez Perez wrote:
> Hi all
> 
> Im having problems trying to use DSpace with a latin1 oracle database. I
> cannot use another charset due to production constrains. I know DSpace
> is prepared to be used in UTF8, but... Is there a workarround to such a
> case? Or, is there an easy way to intercept SQL queries and perform the
> charset conversion? I hope someone could help me.
> 
> Thanks in advance
> 
> Jose Fernández
> UOC (Universitat Oberta de Catalunya).
> 
> 
> -
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] DSpace Installation: Java-Problem

[Graham Triggs]

What is the output of:

ant -diagnostics

G

On Thu, 2007-10-25 at 13:12 +0200, Robert Roggenbuck wrote:
> No, I didn't an 'ant clean'. After doing it successful, unfortunately 
> the problem remains...
> 
> Robert
> 
> Claudia Jürgen schrieb:
> > Hi Robert,
> > 
> > did you try ant clean before you did ant fresh_install again?
> > 
> > sunny greetings
> > 
> > Claudia Jürgen
> > 
> > 
> > Robert Roggenbuck schrieb:
> >> I changed the pointer /usr/bin/javac to aim to Java 1.5. Now I get for 
> >> saying 'javac -version' this: 'javac 1.5.0_11' - but the 'ant 
> >> fresh_install' stays with its complain about a wrong version number in 
> >> a class...
> >> There must be further things wrong...
> >>
> >> Desmond Elliott schrieb:
> >>> Have you tried using the Sun javac instead of the ECJ javac?
> >>>
> >>> Robert Roggenbuck wrote:
>  Dear All,
> 
>  I'd like to install DSpace to check whether we can use it for our 
>  planned OA-Repository. But reaching the point to say 'ant 
>  fresh_install' I get the following error message where I can not 
>  figure out the reason:
> 
>  Buildfile: build.xml
> 
>  compile:
> 
>  install_code:
> 
>  build_wars:
> 
>  init_configs:
> 
>  setup_database:
>    [java] 2007-10-25 12:36:16,477 INFO 
>  org.dspace.storage.rdbms.InitializeDatabase @ Initializing Database
>    [java] 2007-10-25 12:36:16,536 INFO 
>  org.dspace.core.ConfigurationManager @ DSpace logging installed 
>  using log4j.properties
>    [java] Exception in thread "main" 
>  java.lang.UnsupportedClassVersionError: Bad version number in .class 
>  file
>    [java] at java.lang.ClassLoader.defineClass1(Native Method)
>    [java] at 
>  java.lang.ClassLoader.defineClass(ClassLoader.java:620)
>    [java] at 
>  java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
>    [java] at 
>  java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
>    [java] at 
>  java.net.URLClassLoader.access$100(URLClassLoader.java:56)
>    [java] at 
>  java.net.URLClassLoader$1.run(URLClassLoader.java:195)
>    [java] at 
>  java.security.AccessController.doPrivileged(Native Method)
>    [java] at 
>  java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>    [java] at 
>  java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>    [java] at 
>  sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
>    [java] at 
>  java.lang.ClassLoader.loadClass(ClassLoader.java:251)
>    [java] at 
>  java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
>    [java] at java.lang.Class.forName0(Native Method)
>    [java] at java.lang.Class.forName(Class.java:164)
>    [java] at 
>  org.dspace.storage.rdbms.DatabaseManager.initialize(DatabaseManager.java:1607)
>   
> 
>    [java] at 
>  org.dspace.storage.rdbms.DatabaseManager.getConnection(DatabaseManager.java:715)
>   
> 
>    [java] at 
>  org.dspace.storage.rdbms.DatabaseManager.loadSql(DatabaseManager.java:1062)
>   
> 
>    [java] at 
>  org.dspace.storage.rdbms.InitializeDatabase.main(InitializeDatabase.java:71)
>   
> 
> 
>  BUILD FAILED
>  /opt/dspace-1.4.2-source/build.xml:293: Java returned: 1
> 
>  Total time: 3 seconds
> 
> 
>  Any suggestions? I think its not directly a problem with dspace, but 
>  with my Java installation. But anyway I would be glad for any hint.
> 
>  Here some info about my environment:
> 
>   >uname -a
>  Linux OSU458 2.6.20-16-generic #2 SMP Sun Sep 23 19:50:39 UTC 2007 
>  i686 GNU/Linux
>  (in fact it's an Ubuntu 7.04)
> 
>   >javac -version
>  Eclipse Java Compiler v_686_R32x, 3.2.2 release, Copyright IBM Corp 
>  2000, 2006. All rights reserved.
> 
>   >java -version
>  java version "1.5.0_11"
>  Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_11-b03)
>  Java HotSpot(TM) Server VM (build 1.5.0_11-b03, mixed mode)
> 
>   >echo $JAVA_HOME
>  /usr/lib/jvm/java-1.5.0-sun
> 
>   >psql --version
>  psql (PostgreSQL) 8.2.5
> 
>  Further I installed Tomcat 5.5.25 successfully.
> 
> 
>  Best regards
> 
>  Robert
> 
>    
> >>>
> >>
> > 
> 
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.spl

Re: [Dspace-tech] Using Dspace with MYSQL

[Graham Triggs]

On Sat, 2007-11-03 at 18:44 -0700, pavan krishnamurthy wrote:
> Hi all,
> 
> I am trying to setup DSpace with MYSQL as database. I saw the patch
> which was given in the mailing list. Now i am confused how to use the
> file
> 
> 1) naming_patch.txt
> 2) database_schema.sql
> 
> people who have succesfully installed with MYSQL can u please throw
> some light on how to go about it.

Hi Pavan,

I should point out first of all that the MySQL patch is pretty much
unsupported and quite untested - it's provided to get feedback and maybe
encourage a few people to support it.

That said, the two files are fairly straightforward -
database_schema.sql is a replacement for the schema provided for
Postgres, and should be used to prepare your database tables.

The naming_patch.txt is a patch file that should be applied against the
1.4.x source, and corrects the casing of the tables in the SQL queries
(as MySQL is case sensitive by default on *nix), and also add a couple
of small compatibility changes to the database manager.

You will also need to add the MySQL JDBC jar to the lib directory, and
set up your db.url and db.driver entries in dspace.cfg appropriately.

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Build Failed : sun.misc cannot be resolved

[Graham Triggs]

pavan krishnamurthy wrote:
> Hi,
> I was trying to install dspace. When i try to do ant fresh_install; i
> get so many warnings plus two errors saying
>
> 1)The import sun.misc cannot be resolved
> 2)BASE64Encoder cannot be resolved to a type
>   

The sun.* packages are part of the Sun JDK. They shouldn't really be 
used directly, but currently they are.

I could point you to a way to not have that dependency, but it would 
ultimately be a rather futile gesture. Because if you are getting this 
as an error (and not a warning), then it means you aren't using and Sun 
JDK - and that most likely means that you are using the GNU Java 
compiler. And that just isn't going to work.

So the simple answer is install and use a Sun JDK. Which flavour OS are 
you using? rpm-based distros are readily provided for by Sun, and 
deb-based distros more than likely have the JDK available in the 
repository (for Debian itself, it's in the non-free repo).

G

-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Encoding URLs in Manakin

[Graham Triggs]

Dorothea Salo wrote:
> Hi, all,
>
> If I can divert a tiny bit of attention from the testathon... I'm
> trying to make author names on item-display pages clickable. The URL
> pattern for this is "browse-author-items?author=" plus the author's
> URL-encoded name -- which is the problem, as there doesn't seem to be
> any easy XSLT way to get URL encoding for a string.
>
> Ideas? Or should I not worry about the encoding and just let the browser 
> worry?
>
> Dorothea
>
>   

Hmmm... actually it appears there is an easy way if you are using xslt 2 
- but you are probably working with Xalan and won't be.

Don't know what the practicalities are of switching to use Saxon-B 9, 
that could be a possibility.

However, you appear to be using a rather young Manakin - with the browse 
code that's been merged into 1.5, those urls no longer exist. It's now 
something like "browse?type=author&value="

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] DSpace Installation Saga: Please Help!

[Graham Triggs]

Do you have the manager application installed on Tomcat? 
(http://localhost:8080/manager/html)

If so, what does it say about the status of the DSpace application?

Have you checked the Tomcat log files?

Is Tomcat running under a security manager? (I'm not sure how this is / 
would be configured under Fedora). If it is, the configuration manager 
probably won't get loaded, and the application will fail to initialise 
correctly.

G

Drrty Byl wrote:
> Hello, I have been trying to install DSpace on my Fedora Core 5 server
> to use in a school project for the past several days.  I am able to
> successfully build and install the package, but still cannot access
> the web interface.  Here are the specifications for my project along
> with the steps I have tried to get it working:
>
> Fedora Core 5 (kernel 2.6.15-1.2054_FC5, i386)
> DSpace version 1.4.2
> dspace.dir = /dspace  dspace.url = http://localhost:8080/dspace
> dspace.hostname = localhost
> Java version "1.6.0_03" Java(TM) SE Runtime Environment (build 1.6.0_03-b05)
> Tomcat 5 (installed from Redhat Package Manager/yum interface)
> Apache 2.2 (installed from Redhat Package Manager/yum interface)
> PostgreSQL 8.1.9
>
> 1 - Regular sequence of Postgres database creation is successful, "ant
> fresh_install" build is successful, software installed at "/dspace".
> 2 - Tomcat and Apache are restarted
> 3 - visiting http://localhost:8080 with Firefox displays Apache Test Page
> 4 - visiting http://localhost:8080/dspace results in a 404 error: Not Found
> 5 - visiting http://localhost:8080/dspace/ results in a 503 error:
> Service Temporarily Unavailable
>
> I am positive that both Tomcat and Apache are running; port 8080 is
> NOT firewalled.  The next step I tried was installing the mod_jk
> connector using the directions at:
> http://wiki.dspace.org/index.php/ModJk .  This did not resolve the
> problem.  Should "ProxyRequests On" and "NameVirtualHost *:80" be
> uncommented in Apache httpd.conf?  Also, do I need to define a <
> location > in this file?  Does anyone have a working Apache
> configuration file with changes made in respect to DSpace
> installation?  I get the impression this might somehow be related to
> the problem.  It looks as though Tomcat, although running, is not
> serving DSpace.  Any help to get this functioning is greatly
> appreciated!  Billy
>
> -
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>   


-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech