[
https://issues.apache.org/jira/browse/HIVE-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603987#comment-14603987
]
Nishant Kelkar commented on HIVE-9557:
--------------------------------------
Hi [~apivovarov],
I followed your instructions, and everything went fine till the step where I
run the TestCliDriver with 'mvn test'. I get the following exception in
./itests/qtest/tmp/log/hive.log:
{code}
2015-06-26 22:25:47,656 DEBUG [main]: util.Shell
(Shell.java:checkHadoopHome(320)) - Failed to detect a valid hadoop home
directory
java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:302)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:327)
at
org.apache.hadoop.hive.conf.HiveConf$ConfVars.findHadoopBinary(HiveConf.java:2371)
at
org.apache.hadoop.hive.conf.HiveConf$ConfVars.<clinit>(HiveConf.java:366)
at org.apache.hadoop.hive.conf.HiveConf.<clinit>(HiveConf.java:105)
at org.apache.hadoop.hive.ql.QTestUtil.<init>(QTestUtil.java:354)
at
org.apache.hadoop.hive.cli.TestCliDriver.<clinit>(TestCliDriver.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.internal.runners.SuiteMethod.testFromSuiteMethod(SuiteMethod.java:35)
at org.junit.internal.runners.SuiteMethod.<init>(SuiteMethod.java:24)
at
org.junit.internal.builders.SuiteMethodBuilder.runnerForClass(SuiteMethodBuilder.java:11)
at
org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
at
org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
at
org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
at
org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:26)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:262)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
2015-06-26 22:25:47,669 DEBUG [main]: util.Shell
(Shell.java:isSetsidSupported(392)) - setsid is not available on this machine.
So not using it.
2015-06-26 22:25:47,669 DEBUG [main]: util.Shell
(Shell.java:isSetsidSupported(396)) - setsid exited with exit code 0
2015-06-26 22:25:48,408 WARN [main]: conf.HiveConf
(HiveConf.java:initialize(2802)) - HiveConf of name
hive.dummyparam.test.server.specific.config.metastoresite does not exist
2015-06-26 22:25:48,409 WARN [main]: conf.HiveConf
(HiveConf.java:initialize(2802)) - HiveConf of name
hive.ql.log.PerfLogger.level does not exist
2015-06-26 22:25:48,409 WARN [main]: conf.HiveConf
(HiveConf.java:initialize(2802)) - HiveConf of name
hive.dummyparam.test.server.specific.config.hivesite does not exist
2015-06-26 22:25:48,409 WARN [main]: conf.HiveConf
(HiveConf.java:initialize(2802)) - HiveConf of name
hive.dummyparam.test.server.specific.config.override does not exist
2015-06-26 22:25:48,410 WARN [main]: conf.HiveConf
(HiveConf.java:initialize(2802)) - HiveConf of name hive.metastore.metadb.dir
does not exist
2015-06-26 22:25:48,477 INFO [main]: server.ZooKeeperServer
(Environment.java:logEnv(100)) - Server
environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2015-06-26 22:25:48,477 INFO [main]: server.ZooKeeperServer
(Environment.java:logEnv(100)) - Server
environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2015-06-26 22:25:48,477 INFO [main]: server.ZooKeeperServer
(Environment.java:logEnv(100)) - Server environment:host.name=localhost
2015-06-26 22:25:48,477 INFO [main]: server.ZooKeeperServer
(Environment.java:logEnv(100)) - Server environment:host.name=localhost
2015-06-26 22:25:48,477 INFO [main]: server.ZooKeeperServer
(Environment.java:logEnv(100)) - Server environment:java.version=1.7.0_67
2015-06-26 22:25:48,477 INFO [main]: server.ZooKeeperServer
(Environment.java:logEnv(100)) - Server environment:java.version=1.7.0_67
2015-06-26 22:25:48,477 INFO [main]: server.ZooKeeperServer
(Environment.java:logEnv(100)) - Server environment:java.vendor=Oracle
Corporation
2015-06-26 22:25:48,477 INFO [main]: server.ZooKeeperServer
(Environment.java:logEnv(100)) - Server environment:java.vendor=Oracle
Corporation
2015-06-26 22:25:48,477 INFO [main]: server.ZooKeeperServer
(Environment.java:logEnv(100)) - Server
environment:java.home=/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre
2015-06-26 22:25:48,477 INFO [main]: server.ZooKeeperServer
(Environment.java:logEnv(100)) - Server
environment:java.home=/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre
{code}
I'm on a Mac OS X 10.9.5 by the way, and don't have Hadoop set up. I thought
however, that Hive spins up in memory and runs in isolation?
> create UDF to measure strings similarity using Cosine Similarity algo
> ---------------------------------------------------------------------
>
> Key: HIVE-9557
> URL: https://issues.apache.org/jira/browse/HIVE-9557
> Project: Hive
> Issue Type: Improvement
> Components: UDF
> Reporter: Alexander Pivovarov
> Assignee: Nishant Kelkar
> Labels: CosineSimilarity, SimilarityMetric, UDF
> Attachments: udf_cosine_similarity-v01.patch
>
>
> algo description http://en.wikipedia.org/wiki/Cosine_similarity
> {code}
> --one word different, total 2 words
> str_sim_cosine('Test String1', 'Test String2') = (2 - 1) / 2 = 0.5f
> {code}
> reference implementation:
> https://github.com/Simmetrics/simmetrics/blob/master/src/uk/ac/shef/wit/simmetrics/similaritymetrics/CosineSimilarity.java
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
