Oops, I left some ivy crud in the patch which shouldn't be there. I'll upload
a new patch, but you can just delete these lines out of hbase-handler/ivy.xml:
<dependency org="hadoop" name="hbase" rev="${hbase.version}">
<artifact name="hbase" type="source" ext="tar.gz"/>
</dependency>
JVS
________________________________________
From: Ray Duong [[email protected]]
Sent: Friday, March 05, 2010 9:42 AM
To: John Sichi
Cc: [email protected]
Subject: Re: storage handlers and HBase integration
Hi John,
I removed the .ant directory and rebuilt the ant package but got the following
error messages. I also made sure I downloaded the hbase and zookeeper jar
files into the hive-trunk/lib directory.
Thanks again for the help.
Ray du...@sharethis
ivy-init:
ivy-retrieve-hadoop-source:
[ivy:retrieve] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ ::
[ivy:retrieve] :: loading settings :: file = /tmp/hive-trunk/ivy/ivysettings.xml
[ivy:retrieve] :: resolving dependencies ::
org.apache.hadoop.hive#contrib;work...@domu-12-31-38-02-21-81
[ivy:retrieve] confs: [default]
[ivy:retrieve] found hadoop#core;0.20.0 in hadoop-source
[ivy:retrieve] :: resolution report :: resolve 2493ms :: artifacts dl 2ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 2 | 0 | 0 | 0 || 1 | 0 |
---------------------------------------------------------------------
[ivy:retrieve]
[ivy:retrieve] :: problems summary ::
[ivy:retrieve] :::: WARNINGS
[ivy:retrieve] module not found: hadoop#hbase;${hbase.version}
[ivy:retrieve] ==== hadoop-source: tried
[ivy:retrieve] -- artifact
hadoop#hbase;${hbase.version}!hbase.tar.gz(source):
[ivy:retrieve]
http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hbase-${hbase.version}/hbase-${hbase.version}.tar.gz
[ivy:retrieve] ==== apache-snapshot: tried
[ivy:retrieve]
https://repository.apache.org/content/repositories/snapshots/hadoop/hbase/${hbase.version}/hbase-${hbase.version}.pom
[ivy:retrieve] -- artifact
hadoop#hbase;${hbase.version}!hbase.tar.gz(source):
[ivy:retrieve]
https://repository.apache.org/content/repositories/snapshots/hadoop/hbase/${hbase.version}/hbase-${hbase.version}.tar.gz
[ivy:retrieve] ==== maven2: tried
[ivy:retrieve]
http://repo1.maven.org/maven2/hadoop/hbase/${hbase.version}/hbase-${hbase.version}.pom
[ivy:retrieve] -- artifact
hadoop#hbase;${hbase.version}!hbase.tar.gz(source):
[ivy:retrieve]
http://repo1.maven.org/maven2/hadoop/hbase/${hbase.version}/hbase-${hbase.version}.tar.gz
[ivy:retrieve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:retrieve] :: UNRESOLVED DEPENDENCIES ::
[ivy:retrieve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:retrieve] :: hadoop#hbase;${hbase.version}: not found
[ivy:retrieve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:retrieve]
[ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
BUILD FAILED
/tmp/hive-trunk/build.xml:158: The following error occurred while executing
this line:
/tmp/hive-trunk/build.xml:103: The following error occurred while executing
this line:
/tmp/hive-trunk/build-common.xml:200: The following error occurred while
executing this line:
/tmp/hive-trunk/build-common.xml:191: The following error occurred while
executing this line:
/tmp/hive-trunk/build-common.xml:177: impossible to resolve dependencies:
resolve failed - see output for details
Total time: 34 seconds
On Fri, Mar 5, 2010 at 9:10 AM, John Sichi
<[email protected]<mailto:[email protected]>> wrote:
You need to delete your ~/.ant directory to get rid of the old ivy version
stored there. Nothing to do with the HBase integration; it is due to a recent
ivy upgrade on trunk.
JVS
On Mar 5, 2010, at 8:39 AM, "Ray Duong"
<[email protected]<mailto:[email protected]>> wrote:
Hi John,
Nice work on the HBase integration adapator.
I downloaded the Hive trunk and applied the
HIVE-705<http://issues.apache.org/jira/browse/HIVE-705>.3.patch, but when I
tried to 'ant package', I get the following error messages below. Is there any
other settings that is required to build the hive_hbase_handler.jar file?
Thanks,
Ray du...@sharethis
>> ant package
ivy-retrieve-hadoop-source:
[ivy:retrieve] :: Ivy 2.0.0-rc2 - 20081028224207 ::
<http://ant.apache.org/ivy/> http://ant.apache.org/ivy/ ::
:: loading settings :: file = /tmp/hive-trunk/ivy/ivysettings.xml
BUILD FAILED
/tmp/hive-trunk/build.xml:158: The following error occurred while executing
this line:
/tmp/hive-trunk/build.xml:103: The following error occurred while executing
this line:
/tmp/hive-trunk/shims/build.xml:55: The following error occurred while
executing this line:
/tmp/hive-trunk/build-common.xml:177: impossible to configure ivy:settings with
given file: /tmp/hive-trunk/ivy/ivysettings.xml : java.text.ParseException:
failed to load settings from file:/tmp/hive-trunk/ivy/ivysettings.xml:
impossible to set defaultTTL to eternal on class
org.apache.ivy.core.cache.DefaultRepositoryCacheManager
Total time: 0 seconds
>> ant -v package
Caused by: java.lang.IllegalArgumentException: impossible to set defaultTTL to
eternal on class org.apache.ivy.core.cache.DefaultRepositoryCacheManager
at org.apache.ivy.util.Configurator.setAttribute(Configurator.java:660)
at
org.apache.ivy.core.settings.XmlSettingsParser.inConfiguratorStarted(XmlSettingsParser.java:561)
at
org.apache.ivy.core.settings.XmlSettingsParser.startElement(XmlSettingsParser.java:200)
at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown
Source)
at
org.apache.xerces.parsers.AbstractXMLDocumentParser.emptyElement(Unknown Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown
Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown
Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown
Source)
at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:277)
at
org.apache.ivy.core.settings.XmlSettingsParser.doParse(XmlSettingsParser.java:159)
... 65 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.ivy.util.Configurator.setAttribute(Configurator.java:658)
... 80 more
Caused by: java.lang.IllegalArgumentException: invalid duration 'eternal': it
must match (?:(\d+)d)? ?(?:(\d+)h)? ?(?:(\d+)m)? ?(?:(\d+)s)? ?(?:(\d+)ms)?
at
org.apache.ivy.core.cache.DefaultRepositoryCacheManager.parseDuration(DefaultRepositoryCacheManager.java:275)
at
org.apache.ivy.core.cache.DefaultRepositoryCacheManager.setDefaultTTL(DefaultRepositoryCacheManager.java:178)
... 85 more
Total time: 0 seconds
On Thu, Mar 4, 2010 at 6:51 PM, John Sichi
<<mailto:[email protected]>[email protected]<mailto:[email protected]>>
wrote:
Hey folks,
In case you're not following the action over at HIVE-705, we're getting close
to having HBase integration committed into Hive. I've written up docs here:
<http://wiki.apache.org/hadoop/Hive/HBaseIntegration>http://wiki.apache.org/hadoop/Hive/HBaseIntegration
<http://wiki.apache.org/hadoop/Hive/StorageHandlers>http://wiki.apache.org/hadoop/Hive/StorageHandlers
(If you happened to read the first draft of the HBaseIntegration doc a few days
ago, I've made a lot of updates today to fill out the details on column
mapping.)
As part of commit, we'll be doing some code reviews within Facebook next week
and logging a bunch of followup tasks; if you have any comments on the approach
or implementation, please pile on in JIRA.
I'll be giving this a quick mention at the Hive user's group later this month,
and then a more detailed presentation at the HBase User Group meeting in April.
JVS
