Ryan McKinley schrieb:
found it. On page 9 of a completely unordered/unsearchable list.... you
would think google could add search/sort no?
In general looks good. We can discuss implementation details later, but
one thing to consider is taking this opportunity to build a better
example data set we all can work with:
https://issues.apache.org/jira/browse/SOLR-131
For my demo (because I want to compare technologies and need easy
generated testdata), I created a small TestDataGenerator that uses SolrJ
and that is configiurable within a spring context.
You can there define multiple fields that are randomly populated,
depending on their type. I have types for:
- A choice of String values
- Integer/Float ranges
- A hierarchic String structure
- Date fields
- UUID fields
All fields may depend on other fields' values, and can be multivalued or
not. See attached applicationContext.xml for details. I think with we
should be able to generate your Books/Movies example data set with this
tool very easily.
matthias
I clicked the "I'm willing to mentor this student" button, and we will
see what happens.
ryan
On Apr 3, 2008, at 6:17 PM, Matthias Epheser wrote:
Ryan McKinley schrieb:
Hey Matthias-
I just checked the pending Application list and don't see yours there
(the interface isn't very good -- Apache has ~200 applicants seems to
be designed for ~10) -- I'll check again tomorrow in case they
manually check them before posting.
If it helps searching, the title should read "SOLR: Create a
javascript client library for Apache Solr"
Also, would you mind posting the text to this mailing list or on the
wiki? The application is only viewable to official mentors -- but we
should get broader support/involvement before making any decisions.
As there are 2 sections (the short description describing the project
and the larger one including personal information), here is a repost
of the first one:
~~~~oooo~~~~
I want to create a javascript client library for Apache Solr, that
consists of following items:
- HTML to create the page layout
- CSS to create the looks of the widgets
- A javascript library containing different widgets, every widget
represents a (part of a) solr result. Every widget is bound to a html
node where the result is deplayed.
- Postprocessing to translate the SOLR response into the fitting html.
There will be a javascript "container" that acts as a registry for the
widgets and as connection manager to solr. The postprocessing of the
result is a matter of discussions, possible solutions would be xslt or
custom responsewriters on the server side (solr), as well as xml or
json processing on the client side (javascript in the browser). After
a user input, every widget should be automatically updated.
Possible widgets are:
- A PageableResult widget to display a solr response
- FacetWidgets that display a faceted browsing menu, including
dependencies on other widgets
- A TreeFacetWidget to support hierarchical faceted data
- A FulltextSearch widget to create "search as you type" functionality
- DateRange widgets using charts, or timelines or the likes for
selecting date ranges
- Ports to google maps or other external javascript libraries
Deliverables are:
- A basic javascript library containing the container and the widget
baseclass
- Code intelligence for the result postprocessing
- Basic widgets that support a page result, faceting and searching.
- User Documentation: Guides for integration of the javascript widgets
into html pages as well as for developing new widgets.
~~~~oooo~~~~
As already mentioned, I tried to put the description of my demo into a
more general proposal. Of course, I am still able to modify the text
if some information is missing.
matthias
Thanks
Ryan
On Apr 3, 2008, at 12:02 PM, Matthias Epheser wrote
Erik Hatcher schrieb:
Matthias,
This stuff looks great! Certainly +1 on you continuing these
efforts.
Regarding your choice of MooTools: have you considered using jQuery
instead? The reason I ask is that we now have a jQuery-based
schema browser in Solr's trunk and thus it is already there and
usable within the Solr codebase. Many of my super savvy
JavaScript/AJAX savvy pals have mentioned they prefer jQuery over
MooTools, prototype, etc. I'm a bit out of my league here though,
so certainly defer to others expertise in this area.
I just uploaded my "official" proposal to the Google summer of code
website. I tried to write a more "general" description of the tasks,
as my last mail was more technical and tied to the demo.
I think we then have to start a discussion together about a detailed
roadmap and some technical decisions that need to be made, mainly
the javascript library to use and how (and where) the postprocessing
of the solr result should happen.
matthias
Perhaps already on your roadmap, I'd love to see Solr integration
with SIMILE's Timeline, Exhibit, and their other viz stuff. I put
some basic support for both Timeline and Exhibit into Flare.
Erik
On Apr 1, 2008, at 9:31 AM, Matthias Epheser wrote:
Hello community
I am a student from Vienna currently writing his master thesis
about "Fast and userfriendly information retrieval in large
indices" (working title), that focuses on techniques like faceted
browsing and field collapsing.
The funtionalities and internals of Solr cover a large part of my
theoretical work and I also want to point out that ease of
integration into existing projects and a good self declaring user
interface are important things to consider when creating a
faceting browsing solution.
As a showcase for these things I am creating a reusable Javascript
Widget Library that directly queries solr using asynchronuous
calls to create a very fast, dynamic and easy to use data browser.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I now want to introduce my (planned) work to the community and
propose the implementation, elaboration and documentation of this
library as a Google Summer of Code 2008 project. I already
participated in gsoc and I think it's a very good way for students
to start contributing to an open soure project.
I started creating a small prototype a few weeks ago, you can test
it here: http://lovo.test.dev.indoqa.com/mepheser/moobrowser/
It covers a fictional media library created by a testdata generator.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It uses a prototype of my MooTools(http://mootools.net) based
javascript library. Mootools are chosen because of the good
inheritance mechanism, thus you can create a very "clean" widget
system using baseclasses. As mentioned before, everything is
created under the paradigm "keep it simple"
Looking at the html sourcecode, you just have to create empty divs
that act as "targets" and then create an instance of Moobrowser
using the solr url:
mooBrowser = new MooBrowser('url-of-solr');
Now you can add various widgets to this browser. Each widget
consists of its javascript class and a xslt stylesheet that is
used to create the appropriate response from solr. See
http://lovo.test.dev.indoqa.com/mepheser/moobrowser/moobrowser.js for
the js source code. Currently these widgets are supported in an
alpha version:
- MooBrowserPageableResultWidget that creates the "result view"
including paging buttons
- MooBrowserSingleFieldFacetWidget for simple facet fiels. These
fields can be dependend on other fields, see "by medium" ->
"image" as an example
- MooBrowserTreeFacetWidget: this is used to display hierarchical
facets (see "by category" -> "sports" -> "tennis" -> ..
The data is stored in layers like category_0, category_1 in the
index.
- new MooBrowserSearchFacetWidge for full text searching
All rendering is done using css, every "target" div is
self-updating and delivers its fitting html after the user changes
his selection.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Of cource I would formulate an "official" proposal containing
tasks, timeline, etc. AFAIK, the timeline is extended till next
moday.
WDYT?
Best regard,
Matthias
<beans xmlns="http://www.springframework.org/schema/beans";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd";>
<bean id="testDocument" class="dipl.testdata.TestDocument"
factory-bean="documentFactory"
factory-method="createDocument"
scope="prototype"/>
<bean id="documentFactory" class="dipl.testdata.DocumentFactory">
<property name="fieldTypes">
<list>
<bean class="dipl.testdata.UUIDFieldType">
<property name="id" value="id"/>
</bean>
<bean class="dipl.testdata.HierarchicFieldType">
<property name="id" value="category"/>
<property name="multivalue" value="false"/>
<property name="inputFile" value="classpath:categories.xml"/>
</bean>
<bean class="dipl.testdata.StringFieldType">
<property name="id" value="medium"/>
<property name="multivalue" value="false"/>
<property name="possibleValues">
<list>
<value>Book</value>
<value>Magazine Article</value>
<value>DVD</value>
<value>Newspaper Article</value>
<value>VHS</value>
<value>Image</value>
</list>
</property>
</bean>
<bean class="dipl.testdata.StringFieldType">
<property name="id" value="newspaper"/>
<property name="multivalue" value="false"/>
<property name="prerequisites">
<list>
<bean class="dipl.testdata.Field">
<property name="id" value="medium"/>
<property name="value" value="Newspaper Article"/>
</bean>
</list>
</property>
<property name="possibleValues">
<list>
<value>Die Presse</value>
<value>Der Standard</value>
<value>Herald Tribune</value>
<value>Le Monde</value>
<value>New York Times</value>
<value>The Times</value>
</list>
</property>
</bean>
<bean class="dipl.testdata.StringFieldType">
<property name="id" value="magazine"/>
<property name="multivalue" value="false"/>
<property name="prerequisites">
<list>
<bean class="dipl.testdata.Field">
<property name="id" value="medium"/>
<property name="value" value="Magazine Article"/>
</bean>
</list>
</property>
<property name="possibleValues">
<list>
<value>GEO</value>
<value>NEJM</value>
<value>Newsweek</value>
<value>Vogue</value>
</list>
</property>
</bean>
<bean class="dipl.testdata.StringFieldType">
<property name="id" value="camera"/>
<property name="multivalue" value="false"/>
<property name="prerequisites">
<list>
<bean class="dipl.testdata.Field">
<property name="id" value="medium"/>
<property name="value" value="Image"/>
</bean>
</list>
</property>
<property name="possibleValues">
<list>
<value>Canon EOS 500</value>
<value>Canon EOS 400</value>
<value>Nikon F40</value>
<value>Sony HR3</value>
</list>
</property>
</bean>
<bean class="dipl.testdata.StringFieldType">
<property name="id" value="imagetype"/>
<property name="multivalue" value="false"/>
<property name="prerequisites">
<list>
<bean class="dipl.testdata.Field">
<property name="id" value="medium"/>
<property name="value" value="Image"/>
</bean>
</list>
</property>
<property name="possibleValues">
<list>
<value>jpg</value>
<value>bmp</value>
<value>tif</value>
<value>png</value>
</list>
</property>
</bean>
<bean class="dipl.testdata.IntegerRangeFieldType">
<property name="id" value="pages"/>
<property name="min" value="100"/>
<property name="max" value="900"/>
<property name="prerequisites">
<list>
<bean class="dipl.testdata.Field">
<property name="id" value="medium"/>
<property name="value" value="Book"/>
</bean>
</list>
</property>
</bean>
<bean class="dipl.testdata.UUIDFieldType">
<property name="id" value="isbn"/>
<property name="prerequisites">
<list>
<bean class="dipl.testdata.Field">
<property name="id" value="medium"/>
<property name="value" value="Book"/>
</bean>
</list>
</property>
</bean>
<bean class="dipl.testdata.IntegerRangeFieldType">
<property name="id" value="pages"/>
<property name="min" value="2"/>
<property name="max" value="50"/>
<property name="prerequisites">
<list>
<bean class="dipl.testdata.Field">
<property name="id" value="medium"/>
<property name="value" value="Magazine"/>
</bean>
</list>
</property>
</bean>
<bean class="dipl.testdata.IntegerRangeFieldType">
<property name="id" value="pages"/>
<property name="min" value="2"/>
<property name="max" value="50"/>
<property name="prerequisites">
<list>
<bean class="dipl.testdata.Field">
<property name="id" value="medium"/>
<property name="value" value="Newspaper"/>
</bean>
</list>
</property>
</bean>
<bean class="dipl.testdata.IntegerRangeFieldType">
<property name="id" value="pages"/>
<property name="min" value="2"/>
<property name="max" value="50"/>
<property name="prerequisites">
<list>
<bean class="dipl.testdata.Field">
<property name="id" value="medium"/>
<property name="value" value="Newspaper"/>
</bean>
</list>
</property>
</bean>
<bean class="dipl.testdata.IntegerRangeFieldType">
<property name="id" value="prize"/>
<property name="min" value="0"/>
<property name="max" value="1000"/>
</bean>
<bean class="dipl.testdata.IntegerRangeFieldType">
<property name="id" value="publicationYear"/>
<property name="min" value="1995"/>
<property name="max" value="2005"/>
</bean>
<bean class="dipl.testdata.MultiStringFieldType">
<property name="id" value="tags"/>
<property name="possibleValues">
<list>
<value>spring</value>
<value>fall</value>
<value>autumn</value>
<value>summer</value>
<value>sports</value>
<value>nature</value>
<value>mauntains</value>
<value>holiday</value>
<value>peter</value>
<value>birthday</value>
</list>
</property>
</bean>
<bean class="dipl.testdata.DateFieldType">
<property name="id" value="publicationDate"/>
</bean>
</list>
</property>
</bean>
</beans><items>
<item value="Sports">
<item value="Football">
<item value="Austria">
<item value="Austria Vienna"/>
<item value="Rapid Vienna"/>
<item value="Salzburg"/>
<item value="WackerInnsbruck"/>
</item>
<item value="Spain">
<item value="FC Barcelona"/>
<item value="Real Madrid"/>
</item>
<item value="Italy">
<item value="Inter Milan"/>
<item value="AC Milan"/>
<item value="AS Roma"/>
<item value="Juventus"/>
</item>
</item>
<item value="Tennis">
<item value="Wimbledon"/>
<item value="French Open"/>
</item>
<item value="Polo"/>
<item value="Skiing">
<item value="Apline Skiing"/>
<item value="Ski Jumping"/>
<item value="Crosscountry"/>
</item>
<item value="Tabletennis"/>
<item value="Volleyball"/>
</item>
<item value="Nature"/>
<item value="Travel">
<item value="Europe">
<item value="Austria">
<item value="Vienna"/>
<item value="Salzburg"/>
<item value="inssbruck"/>
</item>
<item value="Sweden"/>
<item value="Germany"/>
<item value="Spain">
<item value="Andalusia"/>
<item value="Bracelona"/>
<item value="Madrid"/>
</item>
<item value="France"/>
<item value="Portugal"/>
</item>
<item value="Asia">
<item value="India"/>
<item value="China"/>
<item value="Indonesia"/>
</item>
</item>
<item value="Cooking">
<item value="Austrian Kitchen"/>
<item value="French Kitchen"/>
<item value="Italian Kitchen"/>
</item>
</items>
