Hi Hardy,
We built a new UI outside of DSpace which our Jorum users will be using
for all their search and stats queries. (That is the introduction for
everyone who was not at OR13 in our DSpace API session ;-))
For that UI we built an ES index where the stats is a inner-document to
the item. See below if you are interested otherwise you can safely
ignore the next bit. Indentation is off as well.
----------
{
"_index": "jorumindex",
"_type": "oerdocument",
"_id": "19045",
"_version": 3,
"exists": true,
"_source": {
"hefe": [
"fe"
],
"author": [
"Elesa Argent",
"Radar Brookes",
"2012 Legacies for Learning"
],
"date": null,
"date_accessioned": "2012-07-25T13:54:16Z",
"date_created": null,
"date_issued": null,
"date_last_indexed": "2013-04-25T10:13:55.869Z",
"description": [
"Sports tourism can be defined as a type of tourism that
refers to trips that are undertaken specifically to attend a sporting
event. Common examples include the Olympic Games, the FIFA and Rugby
World Cups and Formula 1 Grand Prix races. An estimated average of 12m
sports tourists make international trips every year"
],
...
"resource_type": "Web Link",
"view_count": 5,
"download_count": 0,
"date_accessioned_short": "25-Jul-2012",
"id_text": "19045",
"statistics": [
{
"city": "Novo Hamburgo",
"continent": "SA",
"countryCode": "BR",
"epersonid": null,
"isBot": false,
"time": "2013-03-26T23:58:40.043Z",
"event_type": "view",
"oer_id": 19045,
"longitude_latitude": [
-51.1333,
-29.683304
],
"truncated_ip": "177.18.180.*"
},
{
"city": "Ashburn",
"continent": "NA",
"countryCode": "US",
"epersonid": null,
"isBot": false,
"time": "2013-04-16T22:31:11.889Z",
"event_type": "view",
"oer_id": 19045,
"longitude_latitude": [
-77.4875,
39.0437
],
"truncated_ip": "23.20.204.*"
},
...
----------
We got a script which creates that ES index out of Solr stats and
discovery indexes. But since we need to keep that ES up to date we added
to the statistics package an ElasticSearchLogger.java (intgrated into
UsageStatsEntity.java) and added a new Consumer to
event.dispatcher.default.consumers which gets triggered whenever
something happens to an item.
This all worked very well until we built the ES with our real data and
ES fell over with memory problems -- Footprint of 6GB in memory with a
fairly moderate index. The new version of ES solved the memory problem
but is causing library conflicts. As usual you can't have everything.
That far to the 'tell me more' part. The other question of whether I
would consider contributing - yes, if that stuff is interesting to
anyone else. Building the ES index is one bit but there is nothing
within DSpace which is using it - that part is completely external.
(And I have not forgotten your question for making the sources open to
the Rubi-UI either. We are heading towards the release of it.)
Best regards,
Anja
--
Anja Le Blanc
Jorum Team
Mimas
On 13/08/2013 15:10, Pottinger, Hardy J. wrote:
> Hi, Anja, everyone else will be chiming in on your specific question, I
> have a question for you, though: can you share more details about your
> work with Elastic Search? Is this work something that you'd consider
> submitting as an enhancement for DSpace 4.0? Shorter version: I'm
> intrigued, tell me more. Thanks!
> --
> HARDY POTTINGER <[email protected]>
> University of Missouri Library Systems
> http://lso.umsystem.edu/~pottingerhj/
> https://MOspace.umsystem.edu/
> "I am always doing that which I cannot do, in order that I may learn how
> to do it." --Pablo Picasso
>
>
>
>
>
> On 8/13/13 5:36 AM, "Anja Le Blanc" <[email protected]> wrote:
>
>> Hello,
>>
>> We are using DSpace 1.8. We modified the source to use ElasticSearch as
>> well as the build in Solr. After upgrading ElasticSearch to 0.90.3 we
>> are now forced (library conflict with lucene) to update the build in
>> Solr to 4.4.0.
>>
>> Is the version of Solr which comes with DSpace in any way modified to
>> make it work with DSpace?
>> Did anyone attempt an upgrade to Solr 4.4.0 before? Are there any known
>> pit-falls?
>>
>> Any advice appreciated!
>>
>> Best regards,
>> Anja
>>
>> --------------------------------------------------------------------------
>> ----
>> Get 100% visibility into Java/.NET code with AppDynamics Lite!
>> It's a free troubleshooting tool designed for production.
>> Get down to code-level detail for bottlenecks, with <2% overhead.
>> Download for free and get started troubleshooting in minutes.
>> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktr
>> k
>> _______________________________________________
>> DSpace-tech mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>> List Etiquette:
>> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>
>
>
------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
DSpace-tech mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
