Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for 
change notification.

The following page has been changed by udanax:
http://wiki.apache.org/lucene-hadoop/Hbase/RDF

------------------------------------------------------------------------------
  
  We have started to think about storing and querying RDF data in Hbase. But 
we'll jump into its implementation after prudence investigation. 
  
- We introduce an Hbase subsystem for RDF, called HbaseRDF, which uses Hbase + 
MapReduce to store RDF data and execute queries (e.g., SPARQL) on them.
+ We introduce an Hbase subsystem for RDF, called HbaseRDF, which uses Hbase + 
!MapReduce to store RDF data and execute queries (e.g., SPARQL) on them.
  We can store very sparse RDF data in a single table in Hbase, with as many 
columns as 
  they need. For example, we might make a row for each RDF subject in a table 
and store all the properties and their values as columns in the table. 
  This reduces costly self-joins in answering queries asking questions on the 
same subject, which results in efficient processing of queries, although we 
still need self-joins to answer RDF path queries.
  
- We can further accelerate query performance by using MapReduce for 
+ We can further accelerate query performance by using !MapReduce for 
  parallel, distributed query processing. 
  
  === Related projects ===
  
-  * [:Hbase/HbaseShell: HbaseShell] provides a command line tool in which we 
can manipulate tables in Hbase. We are also planning to use HbaseShell to 
manipulate and query RDF data stored in Hbase.
+  * [:Hbase/HbaseShell: Hbase Shell] provides a command line tool in which we 
can manipulate tables in Hbase. We are also planning to use !HbaseShell to 
manipulate and query RDF data stored in Hbase.
   * [http://www.openrdf.org/forum/mvnforum/viewthread?thread=1423 A forum at 
Aduna/Sesame] would be interested in working with this group.
   
  === Initial Contributors ===
@@ -29, +29 @@

  When we store RDF data in a single Hbase table and process queries on them, 
an important issue we have to consider is how to efficiently perform costly 
self-joins needed to process RDF path queries. 
  
  To speed up these costly self-joins, it is natural to think about using 
- the MapReduce framework we already have. However, in the Sawzall paper from 
Google, the authors say that the MapReduce framework is 
+ the !MapReduce framework we already have. However, in the Sawzall paper from 
Google, the authors say that the !MapReduce framework is 
  not good, or inappropriate for performing table joins. 
  It is possible, but while we are reading one table in map 
  or reduce functions, we have to read other tables on the fly, which
  results in less parallelized join processing.
  
  There is a paper on this subject written by Yang et al., from Yahoo (SIGMOD 
07). 
- The paper provides Map-Reduce-Merge, which is an extended version of the 
MapReduce framework, 
+ The paper provides Map-Reduce-Merge, which is an extended version of the 
!MapReduce framework, 
  that implements several relational operators, including joins. They have 
extended the 
- MapReduce framework with an additional Merge phase to implement efficient 
data relationship processing.
+ !MapReduce framework with an additional Merge phase to implement efficient 
data relationship processing.
  See the Paper section below for more information. -- Thanks stack.
- (Edward is now implementing join operators using the MapReduce framework.)
+ (Edward is now implementing join operators using the !MapReduce framework.)
  
- But the problem is that there is an initial delay in executing MapReduce jobs 
due to 
+ But the problem is that there is an initial delay in executing !MapReduce 
jobs due to 
  the time spent in assigning the computations to multiple machines. This 
- might take far more time than necessary, thus hurt query response time. So, 
parallelism obtained by using MapReduce is best enjoyable for queries over huge 
amount of RDF data, where it takes much time to process them. 
+ might take far more time than necessary, thus hurt query response time. So, 
parallelism obtained by using !MapReduce is best enjoyable for queries over 
huge amount of RDF data, where it takes much time to process them. 
  We might consider a selective parallelism where 
- people can decide whether to use MapReduce or not to process their queries, 
as in 
+ people can decide whether to use !MapReduce or not to process their queries, 
as in 
  "select ... '''in parallel'''".
  
- Now that we have two sets of join algorithms, non-parallel versions and 
parallel versions with MapReduceMerge,
+ Now that we have two sets of join algorithms, non-parallel versions and 
parallel versions with !MapReduceMerge,
  we are ready to do some massive parallel query processing on tremendous 
amount of RDF data.
  Currently, C-Store shows the best query performance on RDF data.
- However, we, armed with Hbase and MapReduceMerge, can do even better.
+ However, we, armed with Hbase and !MapReduceMerge, can do even better.
  ----
  == Resources ==
   * http://www.w3.org/TR/rdf-sparql-query/ - The SPARQL RDF Query Language, a 
candidate recommendation of W3C as of 14 June 2007.
@@ -72, +72 @@

  
  === HbaseRDF Query Processor ===
  HbaseRDF Query Processor (HQP) executes RDF queries on RDF data stored in a 
Hbase table. 
- It translates RDF queries into API calls to Hbase, or MapReduce jobs, gathers 
and returns the results
+ It translates RDF queries into API calls to Hbase, or !MapReduce jobs, 
gathers and returns the results
  to the user. 
  
  Query processing steps are as follows:
@@ -117, +117 @@

  ----
  == Papers ==
  
-  * OSDI 2004, ''MapReduce: Simplified Data Processing on Large Clusters'', 
proposes a very simple, but powerfull, and highly parallelized data processing 
technique.
+  * OSDI 2004, ''!MapReduce: Simplified Data Processing on Large Clusters'', 
proposes a very simple, but powerfull, and highly parallelized data processing 
technique.
   * CIDR 2007, ''[http://db.lcs.mit.edu/projects/cstore/abadicidr07.pdf 
Column-Stores For Wide and Sparse Data]'', discusses the benefits of using 
C-Store to store RDF and XML data.
   * VLDB 2007, ''[http://db.lcs.mit.edu/projects/cstore/abadirdf.pdf Scalable 
Semantic Web Data Management Using Vertical Partitoning]'', proposes an 
efficient method to store RDF data in table projections (i.e., columns) and 
executes queries on them.
-  * SIGMOD 2007, ''Map-Reduce-Merge: Simplified Relational Data Processing on 
Large Clusters'', MapReduce implementation of several relational operators.
+  * SIGMOD 2007, ''Map-Reduce-Merge: Simplified Relational Data Processing on 
Large Clusters'', !MapReduce implementation of several relational operators.
  

Reply via email to