Re: Query result cache size

2019-06-24 Thread Jason Dere
hive.query.results.cache.max.size is the limit per HS2 instance. hive.query.results.cache.max.entry.size is the limit for a single query result. On Mon, Jun 3, 2019 at 8:24 AM Kristopher Kane wrote: > 'hive.query.results.cache.max.size' - Is this limit per query result, > total for all users

Re: Hive Custom UDF evaluate behavior when @UDFType is set

2018-05-14 Thread Jason Dere
in parameters based on the table's column values (then no constant propagation). From: PradeepKumar Yadav <pradeep.ya...@protegrity.com> Sent: Sunday, May 13, 2018 11:44 PM To: Jason Dere; user@hive.apache.org Subject: RE: Hive Custom UDF evaluate behavio

Re: Hive Custom UDF evaluate behavior when @UDFType is set

2018-04-16 Thread Jason Dere
quot;initialize"); From: PradeepKumar Yadav <pradeep.ya...@protegrity.com> Sent: Monday, April 16, 2018 4:53 AM To: user@hive.apache.org Cc: Jason Dere Subject: FW: Hive Custom UDF evaluate behavior when @UDFType is set Hi, Regarding the previous mail sent,

Re: Hive Custom UDF evaluate behavior when @UDFType is set

2018-04-10 Thread Jason Dere
Might have to do with constant propagation because the function was listed as deterministic. You can try logging the stack trace during execution and pasting both stack traces here, may help give more clues as to what is going on. From: PradeepKumar Yadav

Re: compilation errors on hive for hadoop-3.0.0

2018-03-20 Thread Jason Dere
It looks like hadoop-3.0.0-beta1 may no longer be available, my copy in ~/.m2 looks like it's from last November. The hadoop version should be changed in the hive build. In the meantime you can try building with hadoop 3.0.0, either by changing hadoop.version in your pom.xml or by adding

Re: Using JAR files located on HDFS for SerDe

2017-04-18 Thread Jason Dere
There is an existing Jira to be able to specify the necessary JARs in the CREATE TABLE command, but this is not in yet - HIVE-9252 From: Jörn Franke Sent: Wednesday, April 12, 2017 8:04 AM

Re: Seems I found a bug (Partitioned Field)

2017-02-06 Thread Jason Dere
I think this may be https://issues.apache.org/jira/browse/HIVE-14995? From: balajee venkatesh Sent: Sunday, February 05, 2017 8:14 AM To: user@hive.apache.org Subject: Seems I found a bug (Partitioned Field) Hi there, In case

Re: Usage of complex types

2017-02-03 Thread Jason Dere
> SELECT array(1, 2) a UNION ALL SELECT NULL; Error: Error while compiling statement: FAILED: SemanticException Schema of both sides of union should match: Column a is of type array on first table and type void on second table. Cannot tell the position of null AST. (state=42000,code=4)

Re: beeline null pointer exception

2016-12-16 Thread Jason Dere
You probably need to take a look at the HiveServer2 logs which might have a stack trace for that error. From: Brotanek, Jan Sent: Friday, December 16, 2016 1:05 PM To: user@hive.apache.org Subject: beeline null pointer exception

Re: hive2.1.0 timestamp

2016-11-28 Thread Jason Dere
Looks like this might be HIVE-15291. From: ? ? Sent: Wednesday, November 23, 2016 6:59 PM To: user@hive.apache.org Subject: hive2.1.0 timestamp Hi: I am using hive2.1 and hava a problem.I have a table 'xp' which contains a timestamp

Re: load data Failed with exception java.lang.IndexOutOfBoundsException

2016-09-08 Thread Jason Dere
Is there a related stack trace in the hive.log? From: ? Sent: Tuesday, September 06, 2016 8:38 PM To: user@hive.apache.org Subject: load data Failed with exception java.lang.IndexOutOfBoundsException hi, Can anybody reproduce

Re: Re:CREATE FUNCTION ERROR: does not have following privileges for operation CREATEFUNCTION

2016-09-06 Thread Jason Dere
Can you create the functions as permanent functions once, so you don't have to create temp functions every time? https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/Drop/ReloadFunction? From: Maria

Re: Where are jars stored for permanent functions

2016-06-09 Thread Jason Dere
Hive doesn't currently handle storing of the JARs. Doing ADD JAR only adds the jars to the current session, but won't help for other sessions. The permanent functions syntax allows you to specify JAR paths when you create the function. These should be on HDFS or other non-local path. create

Re: hive with ua paser (external) library gives IOException

2016-04-15 Thread Jason Dere
3431 Sent from mi iPhone On Apr 15, 2016, at 1:44 PM, Jason Dere <jd...@hortonworks.com<mailto:jd...@hortonworks.com>> wrote: Maybe check the Hive logs or the Hadoop logs of the tasks that were started by the query. Looks like that error happens when an exception is hit while tryin

Re: hive with ua paser (external) library gives IOException

2016-04-15 Thread Jason Dere
Maybe check the Hive logs or the Hadoop logs of the tasks that were started by the query. Looks like that error happens when an exception is hit while trying to invoke the UDF, it might be logged somewhere. From: dhruv kapatel Sent:

Re: [VOTE] Bylaws change to allow some commits without review

2016-04-15 Thread Jason Dere
?+1 From: Lefty Leverenz Sent: Friday, April 15, 2016 1:36 AM To: user@hive.apache.org Subject: Re: [VOTE] Bylaws change to allow some commits without review +1 Navis, you've just reactivated your PMC membership. ;-) A PMC member is

Re: Hive UDF to fetch value from distributed cache not working with outer queries

2016-04-05 Thread Jason Dere
?You might be using the wrong path to reference the distributed cache - I was under the impression that the distributed cache files would accessible using a local path not something starting with '/'. I suspect query 1 is working because fetch task conversion is running the select in a local

Re: Hive 0.12 MAPJOIN hangs sometimes

2016-03-11 Thread Jason Dere
A join between bigint and string might actually be evaluated by converting both values to a double .. try doing an EXPLAIN of the query, that might show what conversion is being done for the join keys. If that is the case, you could try explicitly casting the join keys to either string or

Re: Hive Permanent functions not working after a cluster restart.

2016-02-12 Thread Jason Dere
Yeah it looks like 1.1.0+cdh5.4.3+151 backported HIVE-2573, which has the bug you are seeing (fixed in HIVE-10288). From: Chagarlamudi, Prasanth Sent: Thursday, February 11,

Re: Hive Permanent functions not working after a cluster restart.

2016-02-08 Thread Jason Dere
com> Sent: Monday, February 08, 2016 11:42 AM To: user@hive.apache.org Subject: RE: Hive Permanent functions not working after a cluster restart. Hey Jason, Its 1.1.0. Thank you for your time. Prasanth Chagarlamudi From: Jason Dere [mailto:jd...@hortonworks.com] Sent: Monday, February 08, 2016 1:04

Re: Hive Permanent functions not working after a cluster restart.

2016-02-08 Thread Jason Dere
What version of Hive? From: Chagarlamudi, Prasanth Sent: Monday, February 08, 2016 7:38 AM To: user@hive.apache.org Subject: Hive Permanent functions not working after a cluster restart. I created permanent functions(rather

Re: Hive Permanent functions not working after a cluster restart.

2016-02-08 Thread Jason Dere
after I exit the cli and try calling the function from a new cli. It only return the attached error when the hive server or cluster is restarted. Also it’s a cloudera distribution and version is 1.1.0+cdh5.4.3+151. Not sure if this will make any difference. Thanks Prasanth Chagarlamudi From: Jason Der

Re: GenericUDF

2016-02-03 Thread Jason Dere
when I use the temporary function in the query. Can you put some more light on the call flow to the initialize method.? Regards, Anirudh Paramshetti On Wed, Feb 3, 2016 at 6:08 AM, Jason Dere <jd...@hortonworks.com<mailto:jd...@hortonworks.com>> wrote: - Created once when regist

Re: GenericUDF

2016-02-02 Thread Jason Dere
- Created once when registering the function to the FunctionRegistry. - The UDF is copied from the version in the registry during query compilation - The query plan is serialized, then deserialized by the tasks during query execution, which constructs another instance of the UDF.

Re: Metastore and SerDe jars

2015-12-09 Thread Jason Dere
I think this is still an outstanding issue. Couple of (still open) Jiras relating to solutions for this: https://issues.apache.org/jira/browse/HIVE-9252? https://issues.apache.org/jira/browse/HIVE-6343 From: Andy Srine Sent: Wednesday,

Re: Disabling local mode optimization

2015-11-02 Thread Jason Dere
Take a look at fetch.task.conversion in https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties? From: Daniel Haviv Sent: Monday, November 02, 2015 1:16 AM To: user@hive.apache.org Subject: Re: Disabling local

Re: timestamp to date conversion

2015-11-02 Thread Jason Dere
Can you get the full stack trace for this error? If this is HiveCLI you can get this from hive.log, if hiveserver2 you might be able to find this in the hiveserver2 log. ?What version of Hive? From: murali parimi Sent:

Re: Hive Generic UDF invoking Hbase

2015-09-30 Thread Jason Dere
Take a look at hive.fetch.task.conversion in https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties?, try setting to "none" or "minimal" From: Ryan Harris Sent: Wednesday, September 30, 2015 9:19 AM To:

Re: Hive Generic UDF invoking Hbase

2015-09-30 Thread Jason Dere
So your custom UDF is using org.apache.hadoop.hbase.client.HTable​? How do you resolve your UDF JAR (and this class) on the Hive client - are you doing ADD JAR, or are your UDF JARs and HBase JARs in your Hive class path? ​ From: Ryan Harris

Re: Hive Generic UDF invoking Hbase

2015-09-30 Thread Jason Dere
Not totally familiar with the aux jars property .. does that make sure that the JAR is shipped as part of the MR job? If it does not, you could try adding the necessary jars using ADD JAR to see if that is the issue From: Yogesh Keshetty

Re: Starting Hive Thrift Server

2015-09-10 Thread Jason Dere
Pretty sure those are just warning messages: http://stackoverflow.com/questions/22896243/maven-slf4j-class-path-contains-multiple-slf4j-bindings Try to connect to your server and see if it's working. ? From: Paulo Moniz Sent: Wednesday,

Re: UDF Configure method not getting called

2015-08-26 Thread Jason Dere
but without using UDF's at all? I looked at transformation scripts a while back but I don't think those would work for our use case either. On Tue, Aug 25, 2015 at 5:05 PM, Jason Dere jd...@hortonworks.commailto:jd...@hortonworks.com wrote: For getting the configuration without configure(), this may

Re: UDF Configure method not getting called

2015-08-25 Thread Jason Dere
by running it as a map reduce job via hive, as the processing is fairly compute extensive. Is there a way to force map-reduce jobs? I think hive.fetch.task.conversion to minimal might help, is there anything that can be done? Thanks a ton. On Tue, Aug 25, 2015 at 2:36 PM, Jason Dere jd

Re: UDF Configure method not getting called

2015-08-25 Thread Jason Dere
?There might be a few cases where a UDF is executed locally and not as part of a Map/Reduce job?: - Hive might choose not to run a M/R task for your query (see hive.fetch.task.conversion) - If the UDF is deterministic and has deterministic inputs, Hive might decide to run the UDF once to

Re: Running hive on tez locally

2015-08-10 Thread Jason Dere
, 2015 at 3:16 PM, Jason Dere jd...@hortonworks.commailto:jd...@hortonworks.com wrote: If you add these settings (paths are up to you), you can run Hive with Tez in local mode. You'll also need to make sure your classpath has the Tez libs etc. set hive.user.install.directory=file:///tmp; set

Re: Running hive on tez locally

2015-08-07 Thread Jason Dere
If you add these settings (paths are up to you), you can run Hive with Tez in local mode. You'll also need to make sure your classpath has the Tez libs etc. set hive.user.install.directory=file:///tmp; set fs.default.name=file:///; set fs.defaultFS=file:///; set tez.staging-dir=/tmp; set

Re: NPE error during file sink stage when inserting into bucketed table

2015-08-06 Thread Jason Dere
Got a stack trace? Might help someone identify the issue. From: Muni Chada mch...@pivotal.io Sent: Thursday, August 06, 2015 4:12 PM To: user@hive.apache.org Subject: NPE error during file sink stage when inserting into bucketed table Hi, We are on hive 0.14

Re: UDTF fails with java.lang.ClassCastException

2015-08-06 Thread Jason Dere
Would have to look at the UDTF to know for sure what is going on - it might not be using the object inspectors properly here. Is it using the ObjectInspectors that were passed in during initialize(), or is it creating a WritableStringObjectInspector and assuming this will work with the value

Re: Hive metastore using Oracle

2015-07-13 Thread Jason Dere
Looks like it's having trouble with the hive.exec.local.scratchdir setting (which defaults to ${system:java.io.tmpdir}/${system:user.name}). Is variable substitution disabled (hive.variable.substitute)? You might want to try explicitly setting hive.exec.local.scratchdir when running on the

Re: Alter table statement for create table like-tables

2015-06-16 Thread Jason Dere
Probably has to do with the fact that it is an Avro table. I don't have any experience using Avro, but maybe take a look at https://issues.apache.org/jira/browse/HIVE-7446 for some of the issues described there, or maybe look at the test that was added for that Jira. On Jun 16, 2015, at 2:42

Re: Hive : plan serialization format option

2015-05-07 Thread Jason Dere
if you see any issues. is this issue resolved in hive 0.14 ? On Tue, May 5, 2015 at 11:36 PM, Jason Dere jd...@hortonworks.commailto:jd...@hortonworks.com wrote: Looks like you are running into https://issues.apache.org/jira/browse/HIVE-8321, fixed in Hive-0.14. You might be stuck having to use

Re: Can't access file in Distributed Cache in Hive 1.1.0

2015-05-07 Thread Jason Dere
Is this on Hive CLI, or using HiveServer2? Can you run explain select in_file('a', './testfile') from a; from both Hive 1.0.0 and hive 1.1.0 and see if they look different? One possibile thing that might be happening here is that in Hive-1.1.0, this query is being executed without the need for

Re: Hive : plan serialization format option

2015-05-05 Thread Jason Dere
code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask On Tue, May 5, 2015 at 3:10 PM, Jason Dere jd...@hortonworks.commailto:jd...@hortonworks.com wrote: kryo/javaXML are the only available options. What are the errors you see with each setting? On May 1, 2015, at 9:41 AM, Bhagwan S. Soni

Re: Hive : plan serialization format option

2015-05-05 Thread Jason Dere
kryo/javaXML are the only available options. What are the errors you see with each setting? On May 1, 2015, at 9:41 AM, Bhagwan S. Soni bhgwnsson...@gmail.commailto:bhgwnsson...@gmail.com wrote: Hi Hive Users, I'm using cloudera's hive 0.13 version which by default provide Kryo plan

Re: UDF cannot be found when the query is submitted via templeton

2015-04-17 Thread Jason Dere
Does fully qualifying the function name (HiveUDF.FindPattern()) in the query help here? On Apr 17, 2015, at 6:44 AM, Xiaoyong Zhu xiaoy...@microsoft.commailto:xiaoy...@microsoft.com wrote: Hi experts I am trying to use an UDF (I have already put that in the metastore using CREATE FUNCTION)

Re: Any getting-started with UDAF development

2015-03-12 Thread Jason Dere
I think the Java code is likely still fine for the examples (though someone who actually knows about UDAFs might want to correct me here). If anything is out of date, it would be the build/test commands, which have switched from using ant to maven. Jason On Mar 11, 2015, at 4:01 AM, shahab

Re: Create Function using JDBC Client ( All Databases)

2015-02-18 Thread Jason Dere
You can call a function created in another database by using the fully qualified function name - select db_name.function_name(…) from … This is similar to referencing tables from other databases in your query. On Feb 17, 2015, at 8:03 PM, vengatesh.babu

Re: casting of hour and minute addition issue

2015-02-18 Thread Jason Dere
I believe this is due to https://issues.apache.org/jira/browse/HIVE-9278, which has been fixed in trunk (also 1.0). This affects UDFs using the old-style UDF classes (which include hour()/minute()/second()) A workaround is to do set hive.cache.expr.evaluation=false; before running the query.

Re: Unable to retrieve date

2015-01-29 Thread Jason Dere
to_date() expects a string like '2014-12-07 23:59:04' rather than '07-Dec-2014 23:59:04' On Jan 29, 2015, at 1:42 AM, Nagarjuna Vissarapu nagarjuna.v...@gmail.com wrote: Hi All, I wrote hive UDF to convert 07/Dec/2014:23:59:04 to 07-Dec-2014 23:59:04 I attached UDF. PFA ... I

Re: EXIST vs EXISTS keyword?

2015-01-22 Thread Jason Dere
Doing a quick search in the lexer/parser files (*.g), I only see KW_EXISTS and not KW_EXIST. On Jan 22, 2015, at 1:25 AM, Xiaoyong Zhu xiaoy...@microsoft.com wrote: Hello experts I am wondering when the keyword “EXIST” will be used in Hive – and if it is used, then what’s the difference

Re: show create table issue

2015-01-19 Thread Jason Dere
Looks like this is https://issues.apache.org/jira/browse/HIVE-7270, which has only recently been committed. On Jan 18, 2015, at 10:32 PM, AnilKumar B akumarb2...@gmail.com wrote: Hi All, We have observed that show create table is not providing the information on serde's. Is it a bug or

Re: date_add statement problem

2015-01-16 Thread Jason Dere
Devopam is right, your date format looks wrong (should be -mm-dd rather than /mm/dd. Casting should work, trying this with Hive trunk: hive select c1, date_add('1985-01-01', c1) from short1; FAILED: SemanticException [Error 10014]: Line 1:11 Wrong arguments 'c1': DATE_ADD() only takes

Re: Hive Query o/p to HDFS as CSV file

2015-01-09 Thread Jason Dere
A workaround might be to create an external table with the correct format, insert overwrite into the external table, then drop the external table (which I think shouldn't delete the directory) On Jan 9, 2015, at 5:46 AM, vengatesh.babu vengatesh.b...@zohocorp.com wrote: Hi, How to write

Re: CREATE FUNCTION: How to automatically load extra jar file?

2015-01-06 Thread Jason Dere
descriptionLocal scratch space for Hive jobs/description /property Will this related to configuration issue or a bug? Please help! Regards Arthur On 6 Jan, 2015, at 3:45 am, Jason Dere jd...@hortonworks.com wrote: During query compilation Hive needs to instantiate the UDF

Re: CREATE FUNCTION: How to automatically load extra jar file?

2015-01-05 Thread Jason Dere
folder for HIVE? or any suggestions to fix this issue? Thanks !! Arthur On 3 Jan, 2015, at 4:12 am, Jason Dere jd...@hortonworks.com wrote: The point of USING JAR as part of the CREATE FUNCTION statement to try to avoid having to do ADD JAR/aux path stuff to get the UDF to work

Re: CREATE FUNCTION: How to automatically load extra jar file?

2015-01-02 Thread Jason Dere
The point of USING JAR as part of the CREATE FUNCTION statement to try to avoid having to do ADD JAR/aux path stuff to get the UDF to work. Are all of these commands (Step 1-5) from the same Hive CLI prompt? hive CREATE FUNCTION sysdate AS 'com.nexr.platform.hive.udf.UDFSysDate' using JAR

Re: Custom UDF Not Working Properly

2015-01-02 Thread Jason Dere
The steps you've described sound correct. Do you have a small example (tables, data, query) to demonstrate the problem? On Dec 25, 2014, at 12:03 AM, vengatesh.babu vengatesh.b...@zohocorp.com wrote: Hi, Please tell me steps to be followed to add Custom UDF in Hive 0.14 Source Code.

Re: Hive 0.14 SQL Inserts Complex Data Types

2014-12-19 Thread Jason Dere
There are UDFs to create complex types, would something like this work? INSERT .. VALUES select map('key1', 'val1', 'key2', 'val2'); On Dec 19, 2014, at 8:50 AM, Skyler Beckman skyler.beck...@livetv.net wrote: Thanks From: Alan Gates [mailto:ga...@hortonworks.com] Sent: Friday,

Re: Hive UDFs?

2014-12-18 Thread Jason Dere
Another option is https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-PermanentFunctions, as another user mentioned on this list a few days ago. On Dec 18, 2014, at 5:54 AM, Stéphane Verlet kaweahsoluti...@gmail.com wrote: You only need the jar on the

Re: Question on varchar in 0.12.0 version of nive

2014-12-15 Thread Jason Dere
Varchar was only added to Hive in 0.12, before that there was only string if you wanted to deal with string types. Varchar will enforce the max character length, truncating the string value if necessary. We've tried to make it as compatible with string as possible. One thing about varchars, if

Re: Issue with windowing function UDAF registeration

2014-12-10 Thread Jason Dere
Created HIVE-9073 (with patch) for this. On Dec 4, 2014, at 3:40 PM, Jason Dere jd...@hortonworks.com wrote: Hi Wesley, Sorry for the late response on this .. is it true you saw this work on a different version (later) of Hive? Based on what I think should be the problem, this would

Re: Issue with windowing function UDAF registeration

2014-12-04 Thread Jason Dere
Hi Wesley, Sorry for the late response on this .. is it true you saw this work on a different version (later) of Hive? Based on what I think should be the problem, this would still be an issue in later versions of Hive. Looks like the error may be occurring because the Hive is trying to look

Re: from_unixtime() and epoch definition

2014-11-05 Thread Jason Dere
in hive or extra parameter for the from_unixtime() Hive function? On Mon, Nov 3, 2014 at 10:33 PM, Jason Dere jd...@hortonworks.com wrote: As Nitin mentions, the behavior is to a string representing the timestamp of that moment in the current system time zone. What are the timezone

Re: from_unixtime() and epoch definition

2014-11-05 Thread Jason Dere
That would be great! On Nov 5, 2014, at 10:49 PM, Nitin Pawar nitinpawar...@gmail.com wrote: May be a JIRA ? I remember having my own UDF for doing this. If possible I will share the code On Thu, Nov 6, 2014 at 6:22 AM, Jason Dere jd...@hortonworks.com wrote: Hive should probably

Re: from_unixtime() and epoch definition

2014-11-03 Thread Jason Dere
As Nitin mentions, the behavior is to a string representing the timestamp of that moment in the current system time zone. What are the timezone settings on your machine? $ TZ=GMT date -r 0 Thu Jan 1 00:00:00 GMT 1970 $ TZ=UTC date -r 0 Thu Jan 1 00:00:00 UTC 1970 $ TZ=Europe/London date

Re: [ANNOUNCE] New Hive PMC Member - Alan Gates

2014-10-27 Thread Jason Dere
Congrats Alan! On Oct 27, 2014, at 3:45 PM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Congrats! - Prasanth On Mon, Oct 27, 2014 at 3:44 PM, Matthew McCline mmccl...@hortonworks.com wrote: Congratulations! On Mon, Oct 27, 2014 at 3:38 PM, Carl Steinbach

Re: String to Timestamp conversion bug

2014-10-21 Thread Jason Dere
. On Wed, Sep 10, 2014 at 10:12 PM, Jason Dere jd...@hortonworks.com wrote: Hmm that's odd .. it looks like this works for me: hive select cast('2999-12-31 23:59:59' as timestamp); OK 2999-12-31 23:59:59 Time taken: 0.212 seconds, Fetched: 1 row(s) For string to timestamp conversion

Re: Unxpected exception: Unxpected exception: org.apache.hadoop.io.LongWritable cannot be cast to java.lang.Long

2014-09-23 Thread Jason Dere
What does your GenericUDF look like? What version of Hive? Does the query work without the UDF? On Sep 22, 2014, at 3:28 PM, Dan Fan d...@appnexus.com wrote: Dear hive users: Quick question about hive longwritable convert to long. I have a generic udf called protected_column, which works

Re: varchar

2014-08-26 Thread Jason Dere
What version of Hive? Do you have some sample SQL? On Aug 26, 2014, at 1:20 PM, upd r updbigd...@gmail.com wrote: Hi, I have a created a table with fields defined as varchar(length). Is it correct to insert data in to the table casting the fields as VARCHAR(length). I am getting this

Re: UDF with dependent JARs

2014-08-04 Thread Jason Dere
Does the order that you add the JARs in Hive matter - say adding b.jar first, then a.jar? On Aug 2, 2014, at 4:47 AM, Brian Jeltema brian.jelt...@digitalenvoy.net wrote: I've written a small UDF and placed it in a JAR (a.jar). The UDF has a dependency on a class in another JAR (b.jar).

Re: FAILED: ClassCastException org.apache.hadoop.io.Text cannot be cast to java.lang.String

2014-08-01 Thread Jason Dere
Would probably have to see the whole code of your evaluate() function. Is this while trying to use treat the arguments to the UDF as a String object? It was probably passed into the GenericUDF as a Text object (Hadoop Writable version of string type), not a String object. It would have to be

Re: hive udf cannot recognize generic method

2014-07-30 Thread Jason Dere
Sounds like you are using the older style UDF class. In that case, yes you would have to override evaluate() for each type of input. You could also try overriding the GenericUDF class - that would allow you to do a single method, though it may be a bit more complicated (can look at the Hive

Re: UDTF

2014-07-28 Thread Jason Dere
Looking at call hierarchy in Eclipse: initiaize(): org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genUDTFPlan(GenericUDTF, String, ArrayListString, QB, Operator, boolean) org.apache.hadoop.hive.ql.exec.UDTFOperator.initializeOp(Configuration) process():

Re: Issue while running Hive 0.13

2014-07-11 Thread Jason Dere
Looking at that error online, I see http://slf4j.org/faq.html#compatibility Maybe try to find what version of the slf libraries you have installed (in hadoop? hive?), and try updating to later version. On Jul 10, 2014, at 9:57 PM, Sarath Chandra sarathchandra.jos...@algofusiontech.com wrote:

Re: [ANNOUNCE] New Hive Committers - Gopal Vijayaraghavan and Szehon Ho

2014-06-23 Thread Jason Dere
Congrats! On Jun 23, 2014, at 10:28 AM, Hari Subramaniyan hsubramani...@hortonworks.com wrote: congrats to Gopal and Szehon! Thanks Hari On Mon, Jun 23, 2014 at 9:59 AM, Xiaobing Zhou xz...@hortonworks.com wrote: Congrats! On Mon, Jun 23, 2014 at 9:52 AM, Vaibhav Gumashta

Re: Hive UDF error

2014-05-16 Thread Jason Dere
What version of Hive are you running? It looks like the error you're seeing might be from Hive trying to retrieve the error message from the logs and might not be related to the actual error. Might want to check the logs for the Hadoop task that was run as part of this query, to see if that

Are these SerDe's/InputFormats still being used?

2014-05-05 Thread Jason Dere
Looking into some code cleanup and question of these SerDes/InputFormats came up. Are these still ones that are being used by folks? DynamicSerDe SymlinkTextInputFormat If there is not much usage then we could try to set them as deprecated, so they could be removed in a future release.

Re: [ANNOUNCE] New Hive PMC Member - Xuefu Zhang

2014-02-28 Thread Jason Dere
Congrats Xuefu! On Feb 28, 2014, at 1:43 AM, Biswajit Nayak biswajit.na...@inmobi.com wrote: Congrats Xuefu.. With Best Regards Biswajit ~Biswa -oThe important thing is not to stop questioning o- On Fri, Feb 28, 2014 at 2:50 PM, Carl Steinbach c...@apache.org wrote: I am

Re: Class not found exception from serde

2014-02-24 Thread Jason Dere
I think TableDesc may not be using the right class loader here when it's trying to get the class - other parts of the code specify the class loader to use but in this context it does not. I've opened https://issues.apache.org/jira/browse/HIVE-6495. On Feb 23, 2014, at 4:05 AM, Andreas Koch

Re: Class not found exception from serde

2014-02-24 Thread Jason Dere
I'm assuming the jar was loaded using ADD JAR as opposed to the jar being on the classpath? On Feb 24, 2014, at 2:40 PM, Jason Dere jd...@hortonworks.com wrote: I think TableDesc may not be using the right class loader here when it's trying to get the class - other parts of the code specify

Re: Can data be passed to the final mode init call in a UDAF?

2014-02-12 Thread Jason Dere
I've only used Hive 0.12+ and I'm not really familiar with UDAFs, but had to pass some info to a GenericUDF so this might give you some ideas: - During query planning you might be able to set the hive conf using SessionState.get().getConf(). When the query is executing you might be able to

Re: GenericUDF Testing in Hive

2014-02-05 Thread Jason Dere
Tried your example with Hive trunk. Didn't quite work out of the box, you'll need to replace ListString with ListText. Otherwise, this seemed to work: hive select ComplexUDFExample(array('a', 'b', 'c'), 'a') from src limit 3; …. OK true true true Time taken: 6.271 seconds, Fetched: 3 row(s) On

Re: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere

2014-01-27 Thread Jason Dere
and Jason! Thanks Vikram. On Jan 27, 2014, at 8:36 AM, Carl Steinbach wrote: The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere committers on the Apache Hive Project. Please join me in congratulating Sergey and Jason! Thanks. Carl

Re: Usage of TIMESTAMP

2014-01-24 Thread Jason Dere
See HIVE-2558 - the comparison between timestamp and string was done by converting both values to a number. i think this behavior should be changed, as of Hive-0.12 Jason On Jan 23, 2014, at 6:20 AM, Petter von Dolwitz (Hem) petter.von.dolw...@gmail.com wrote: Hi, I have a table where one

Re: [ANNOUNCE] New Hive Committer - Vikram Dixit

2014-01-06 Thread Jason Dere
Congrats Vikram! On Jan 6, 2014, at 11:10 AM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Congratulations Vikram!! Thanks Prasanth Jayachandran On Jan 6, 2014, at 11:50 PM, Eugene Koifman ekoif...@hortonworks.com wrote: Congratulations! On Mon, Jan 6, 2014 at 9:44

Re: [ANNOUNCE] New Hive PMC Member - Harish Butani

2013-11-14 Thread Jason Dere
Congrats Harish! On Nov 14, 2013, at 5:17 PM, Carl Steinbach c...@apache.org wrote: I am pleased to announce that Harish Butani has been elected to the Hive Project Management Committee. Please join me in congratulating Harish! Thanks. Carl -- CONFIDENTIALITY NOTICE NOTICE: This

Re: [ANNOUNCE] New Hive Committer - Xuefu Zhang

2013-11-04 Thread Jason Dere
Xuefu, congrats! On Nov 4, 2013, at 9:53 AM, Vaibhav Gumashta vgumas...@hortonworks.com wrote: Congrats Xuefu! On Mon, Nov 4, 2013 at 9:17 AM, Prasad Mujumdar pras...@cloudera.comwrote: Congratulations Xuefu! thanks Prasad On Sun, Nov 3, 2013 at 8:06 PM, Carl Steinbach

Re: Problème with min function in HiveQL

2013-08-29 Thread Jason Dere
Looks like the issue is the use of min() within the WHERE clause - the place where the exception is being thrown has the following comment: // UDAF in filter condition, group-by caluse, param of funtion, etc. On Aug 29, 2013, at 3:01 AM, Jérôme Verdier verdier.jerom...@gmail.com wrote: