[
https://issues.apache.org/jira/browse/CHUKWA-734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14332963#comment-14332963
]
Lewis John McGibbney commented on CHUKWA-734:
---------------------------------------------
Hi [~eyang]
bq. I got an error for running TestHBaseWriter unit test:
My tests hang on {code}Running
org.apache.hadoop.chukwa.datacollection.sender.TestAcksOnFailure{code}, however
when I run just the HBaseWriter test, I also get an error
{code}
5 testWriters(org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter)
Time elapsed: 0.026 sec <<< ERROR!
6 java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/Stoppable
7 at java.lang.ClassLoader.defineClass1(Native Method)
8 at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
9 at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
10 at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
11 at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
12 at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
13 at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
14 at java.security.AccessController.doPrivileged(Native Method)
15 at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
16 at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
17 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
18 at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
19 at java.lang.ClassLoader.defineClass1(Native Method)
20 at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
21 at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
22 at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
23 at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
24 at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
25 at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
26 at java.security.AccessController.doPrivileged(Native Method)
27 at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
28 at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
29 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
30 at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
31 at java.lang.ClassLoader.defineClass1(Native Method)
32 at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
33 at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
34 at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
35 at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
36 at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
37 at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
38 at java.security.AccessController.doPrivileged(Native Method)
39 at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
40 at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
41 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
42 at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
43 at
org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.setUp(TestHBaseWriter.java:65)
44 at junit.framework.TestCase.runBare(TestCase.java:132)
45 at junit.framework.TestResult$1.protect(TestResult.java:110)
46 at junit.framework.TestResult.runProtected(TestResult.java:128)
47 at junit.framework.TestResult.run(TestResult.java:113)
48 at junit.framework.TestCase.run(TestCase.java:124)
49 at junit.framework.TestSuite.runTest(TestSuite.java:243)
50 at junit.framework.TestSuite.run(TestSuite.java:238)
51 at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
52 at
org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
53 at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
54 at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
55 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{code}
bq. Does Gora support Hadoop1?
Yes we have shim layer support for Hadoop 1.2.1 and 2.5.2. We just need to
configure it properly.
bq. We probably need to setup another profile for enabling Hadoop 1 vs Hadoop 2.
Most likely Eric. I will be looking in to this. I see that there are existing
profiles for 0.96.2-hadoop1 and 0.94.9
I am therefore thinking that we could potentially set up profiles for
Gora-supported backends for HBase 0.98.8-hadoop2, Cassandra 2.0.2, Solr 4.10.3,
Mongodb 2.6 and Accumulo 1.5.1. Before I do this right now however I need
clarification on
* what version of HBase is currently activated by default in Gora?
* how and where is this defined as default within pom.xml?
bq. Can Gora map sequence ID value to column name in HBase?
Mmmmmm.... I am not sure about this. Reasoning is as follows: currently we
define AHEAD OF MAPPING
* a table name
* columns within this table (columns can also have [optional params like
compression, bloom filters,
etc|http://gora.apache.org/current/gora-hbase.html#gora-hbase-mappings])
* we then map out data beans to these definitions
It would appear to me that for us to be able to map sequenceID to a column
name, we would 1) want to dynamically create many many columns over time
directly dependent on the number of data chunks we get, is this correct? 2)
once we get a new incoming data chunk we would wish to dynamically generate a
new column within the existing table with the sequenceID as the column name, is
this correct?
Thanks
> Gora Storage System for Chuckwa Logs
> ------------------------------------
>
> Key: CHUKWA-734
> URL: https://issues.apache.org/jira/browse/CHUKWA-734
> Project: Chukwa
> Issue Type: New Feature
> Components: Data Collection
> Affects Versions: 0.6.0
> Reporter: Lewis John McGibbney
> Fix For: 0.6.0
>
> Attachments: CHUKWA-734.patch
>
> Original Estimate: 5h
> Remaining Estimate: 5h
>
> I would like to build a Gora-backed log-to-datastore module for Chuckwa. I am
> going to work on this today.
> Gora is an in-memory data modeling and storage abstraction
> http://gora.apache.org
> Gora powers the Apache Nutch 2.X software which generates a bunch of log
> data. Having a Chuckwa monitoring tool for Nutch would be grand.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
