HIVE LLAP question

[Oleksiy S]

Guys please help. I am using Hive-2.3.6 and I want to configure LLAP with
Apache slider. I ran

hive --service llap --instances 1 --size 2g --executors 2 --xmx 1024m
--cache 1024m --name llap0  --auxhbase=false --startImmediately --directory
/tmp


After that I can not get llap status

curl node2.cluster.com:15002/status
curl: (7) Failed connect to node2.cluster.com:15002; Connection refused


And I also have in the container's log

2022-01-11 13:15:41,970 - Error while executing command 'status':
Traceback (most recent call last):
  File
"/tmp/hadoop-mapr/nm-local-dir/usercache/mapr/appcache/application_1641822405396_0008/filecache/12/slider-agent.tar.gz/slider-agent/resource_management/libraries/script/script.py",
line 116, in execute
method(env)
  File
"/tmp/hadoop-mapr/nm-local-dir/usercache/mapr/appcache/application_1641822405396_0008/filecache/13/llap-11Jan2022.zip/package/scripts/llap.py",
line 71, in status
check_process_status(params.pid_file)
  File
"/tmp/hadoop-mapr/nm-local-dir/usercache/mapr/appcache/application_1641822405396_0008/filecache/12/slider-agent.tar.gz/slider-agent/resource_management/libraries/functions/check_process_status.py",
line 83, in check_process_status
raise ComponentIsNotRunning()
ComponentIsNotRunning


-- 
Oleksiy

CVE-2019-10172,CVE-2019-10202 vulnerabilities in jackson-mapper-asl-1.9.13.jar

[Oleksiy S]

Guys please take a look here.
https://issues.apache.org/jira/browse/HIVE-24904

Any ideas of how to fix this?

-- 
Oleksiy

If Hive Metastore is compatibility with MariaDB version 10.x.?

[Oleksiy S]

Hi all.

Could you please help? Customer asked if Hive Metastore is compatible with
MariaDB version 10.x. He is going to use 10.4.10-MariaDB MariaDB Server.

-- 
Oleksiy

Re: HIVE-2.4 release plans

[Oleksiy S]

Thanks for answering.

It would be nice to have Hive-2.4.0, but the versioning is up to you.
Waiting for new Hive!

On Fri, Jan 3, 2020 at 11:31 AM Mass Dosage  wrote:

> +1 for this, or for a Hive 2.3.7 release. We are blocked from releasing
> some of our projects which use Hive 2.3.x on Java >8 due to
> https://issues.apache.org/jira/browse/HIVE-21508 which we helped get
> merged
> but it hasn't been released yet. Similarly we'd like to be able to use some
> Parquet related functionality which didn't work but is now fixed via
> https://issues.apache.org/jira/browse/HIVE-22249 and also merged and ready
> to be released.
>
> Thanks,
>
> Adrian
>
> On Wed, 11 Dec 2019 at 15:25, Oleksiy S 
> wrote:
>
> > Hi all.
> >
> > Are there any plans for Hive-2.4 release?
> >
> > --
> > Oleksiy
> >
>


-- 
Oleksiy

HIVE-2.4 release plans

[Oleksiy S]

Hi all.

Are there any plans for Hive-2.4 release?

-- 
Oleksiy

Re: HIVE-20420 Provide a fallback authorizer when no other authorizer is in use

[Oleksiy S]

thank you.

On Wed, Nov 14, 2018 at 7:44 PM Thejas Nair  wrote:

> This was for CVE-2018-11777.
> You can find more details in description of CVE-2018-11777
>
> On Wed, Nov 14, 2018 at 3:40 AM Oleksiy S 
> wrote:
> >
> > Guys, could you help with this new feature? HIVE-20420
> >
> > I see no docs, no use cases, just nothing. Thanks.
> >
> > --
> > Oleksiy
>


-- 
Oleksiy

HIVE-20420 Provide a fallback authorizer when no other authorizer is in use

[Oleksiy S]

Guys, could you help with this new feature? HIVE-20420


I see no docs, no use cases, just nothing. Thanks.

-- 
Oleksiy

Re: Incorrect Release Notes for Hive-2.3.3

[Oleksiy S]

Guys any updates?

On Thu, Oct 4, 2018 at 11:15 AM Oleksiy S 
wrote:

> Current release notes for Hive-2.3.3 are: RNs
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342162=Text=12310843>
>
> Release Notes - Hive - Version 2.3.3
>
> ** Bug
> * [HIVE-16939] - metastore error: 'export: -Dproc_metastore : not a
> valid identifier'
> * [HIVE-18767] - Some alterPartitions invocations throw
> 'NumberFormatException: null'
> * [HIVE-18788] - Clean up inputs in JDBC PreparedStatement
> * [HIVE-18815] - Remove unused feature in HPL/SQL
> * [HIVE-18879] - Disallow embedded element in UDFXPathUtil needs to
> work if xercesImpl.jar in classpath
> * [HIVE-18885] - DbNotificationListener has a deadlock between Java
> and DB locks (2.x line)
> * [HIVE-20304] - When hive.optimize.skewjoin and
> hive.auto.convert.join are both set to true, and the execution engine is
> mr, same stage may launch twice due to the wrong generated plan
> * [HIVE-20441] - NPE in ExprNodeGenericFuncDesc  when
> hive.allow.udf.load.on.demand is set to true
>
> ** Improvement
> * [HIVE-19900] - HiveCLI HoS Performs Invalid Impersonation If User
> Name Truncated
> * [HIVE-20284] - In strict mode, if constant propagation is enable,
> the partition filter may be folded before partition pruner lead to error
> "No partition predicate for Alias"
>
>
> See table for short summary.
> Issue Comment
> HIVE-18767 Issue is not in Hive-2.3.3
> HIVE-20304 Issue is unresolved in Apache Hive JIRA
> HIVE-20441 Issue is unresolved in Apache Hive JIRA
> HIVE-19900 Resolved: Workaround. No commit to see
> HIVE-20284 Issue is unresolved in Apache Hive JIRA
>
> Please remove incorrect JIRAs from RNs.
>
> --
> Oleksiy
>


-- 
Oleksiy

Incorrect Release Notes for Hive-2.3.3

[Oleksiy S]

Current release notes for Hive-2.3.3 are: RNs


Release Notes - Hive - Version 2.3.3

** Bug
* [HIVE-16939] - metastore error: 'export: -Dproc_metastore : not a
valid identifier'
* [HIVE-18767] - Some alterPartitions invocations throw
'NumberFormatException: null'
* [HIVE-18788] - Clean up inputs in JDBC PreparedStatement
* [HIVE-18815] - Remove unused feature in HPL/SQL
* [HIVE-18879] - Disallow embedded element in UDFXPathUtil needs to
work if xercesImpl.jar in classpath
* [HIVE-18885] - DbNotificationListener has a deadlock between Java and
DB locks (2.x line)
* [HIVE-20304] - When hive.optimize.skewjoin and hive.auto.convert.join
are both set to true, and the execution engine is mr, same stage may launch
twice due to the wrong generated plan
* [HIVE-20441] - NPE in ExprNodeGenericFuncDesc  when
hive.allow.udf.load.on.demand is set to true

** Improvement
* [HIVE-19900] - HiveCLI HoS Performs Invalid Impersonation If User
Name Truncated
* [HIVE-20284] - In strict mode, if constant propagation is enable, the
partition filter may be folded before partition pruner lead to error "No
partition predicate for Alias"


See table for short summary.
Issue Comment
HIVE-18767 Issue is not in Hive-2.3.3
HIVE-20304 Issue is unresolved in Apache Hive JIRA
HIVE-20441 Issue is unresolved in Apache Hive JIRA
HIVE-19900 Resolved: Workaround. No commit to see
HIVE-20284 Issue is unresolved in Apache Hive JIRA

Please remove incorrect JIRAs from RNs.

-- 
Oleksiy

Re: Does MERGE feature works only on TEZ in Hive-2.3?

[Oleksiy S]

Eugene, please take a look https://issues.apache.org/jira/browse/HIVE-19286

On Thu, Apr 19, 2018 at 9:41 AM, Oleksiy S <osayankin.superu...@gmail.com>
wrote:

> Thanks!
>
> On Wed, Apr 18, 2018 at 7:45 PM, Eugene Koifman <ekoif...@hortonworks.com>
> wrote:
>
>> It works on MR as well.
>>
>> From: Oleksiy S <osayankin.superu...@gmail.com>
>> Reply-To: "u...@hive.apache.org" <u...@hive.apache.org>
>> Date: Wednesday, April 18, 2018 at 5:24 AM
>> To: "u...@hive.apache.org" <u...@hive.apache.org>, "dev@hive.apache.org"
>> <dev@hive.apache.org>
>> Subject: Does MERGE feature works only on TEZ in Hive-2.3?
>>
>> Hi all!
>>
>> Does MERGE feature  works only on TEZ in Hive-2.3?
>>
>> see
>>
>>   *   https://issues.apache.org/jira/browse/HIVE-10924
>>   *   https://community.hortonworks.com/articles/
>> 97113/hive-acid-merge-by-example.html
>>
>> --
>> Oleksiy
>>
>
>
>
> --
> Oleksiy
>



-- 
Oleksiy

Re: Does Hive support Hbase-synced partitioned tables?

[Oleksiy S]

Any updates?

On Fri, Apr 20, 2018 at 10:54 AM, Oleksiy S <osayankin.superu...@gmail.com>
wrote:

> Hi all.
>
> I can create following table
>
> create table hbase_partitioned(doc_id STRING, EmployeeID Int, FirstName
> String, Designation  String, Salary Int) PARTITIONED BY (Department String)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH
> SERDEPROPERTIES ("hbase.columns.mapping" = ":key,boolsCF:EmployeeID,
> intsCF:FirstName,intsCF:Designation,intsCF:Salary") TBLPROPERTIES("
> hbase.table.name" = "hbase_partitioned");
>
>
> But when I want to insert data, I have an exception. Is it expected
> behavior?
>
> INSERT INTO TABLE hbase_partitioned PARTITION(department='A') values
> ('1', 1, 'John Connor', 'New York', 2300),
> ('2', 2, 'Max Plank', 'Las Vegas', 1300),
> ('3', 3, 'Arni Shwarz', 'Los Angelos', 7700),
> ('4', 4, 'Sarah Connor', 'Oakland', 9700);
>
>
>
> WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in
> the future versions. Consider using a different execution engine (i.e.
> spark, tez) or using Hive 1.X releases.
> Query ID = mapr_20180420074356_b13d8652-1ff6-4fe1-975c-7318db6037de
> Total jobs = 3
> Launching Job 1 out of 3
> Number of reduce tasks is set to 0 since there's no reduce operator
> java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.IllegalArgumentException: Must specify table name
> at org.apache.hadoop.hive.ql.exec.FileSinkOperator.checkOutputSpecs(
> FileSinkOperator.java:1136)
> at org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl.checkOutputSpecs(
> HiveOutputFormatImpl.java:67)
> at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(
> JobSubmitter.java:271)
> at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(
> JobSubmitter.java:142)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1595)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1595)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(
> JobClient.java:570)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561)
> at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(
> ExecDriver.java:434)
> at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(
> MapRedTask.java:138)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(
> TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2074)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1745)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1454)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1172)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1162)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(
> CliDriver.java:238)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:186)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:405)
> at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:791)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:729)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:652)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:647)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
> java.lang.IllegalArgumentException:
> Must specify table name
> at org.apache.hadoop.hive.ql.exec.FileSinkOperator.createHiveOutputFormat(
> FileSinkOperator.java:1158)
> at org.apache.hadoop.hive.ql.exec.FileSinkOperator.checkOutputSpecs(
> FileSinkOperator.java:1133)
> ... 38 more
> Caused by: java.lang.IllegalArgumentException: Must specify table name
> at org.apa

Does Hive support Hbase-synced partitioned tables?

[Oleksiy S]

Hi all.

I can create following table

create table hbase_partitioned(doc_id STRING, EmployeeID Int, FirstName
String, Designation  String, Salary Int) PARTITIONED BY (Department String)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH
SERDEPROPERTIES ("hbase.columns.mapping" =
":key,boolsCF:EmployeeID,intsCF:FirstName,intsCF:Designation,intsCF:Salary")
TBLPROPERTIES("hbase.table.name" = "hbase_partitioned");


But when I want to insert data, I have an exception. Is it expected
behavior?

INSERT INTO TABLE hbase_partitioned PARTITION(department='A') values
('1', 1, 'John Connor', 'New York', 2300),
('2', 2, 'Max Plank', 'Las Vegas', 1300),
('3', 3, 'Arni Shwarz', 'Los Angelos', 7700),
('4', 4, 'Sarah Connor', 'Oakland', 9700);



WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the
future versions. Consider using a different execution engine (i.e. spark,
tez) or using Hive 1.X releases.
Query ID = mapr_20180420074356_b13d8652-1ff6-4fe1-975c-7318db6037de
Total jobs = 3
Launching Job 1 out of 3
Number of reduce tasks is set to 0 since there's no reduce operator
java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.IllegalArgumentException: Must specify table name
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.checkOutputSpecs(FileSinkOperator.java:1136)
at
org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl.checkOutputSpecs(HiveOutputFormatImpl.java:67)
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:271)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:142)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:570)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561)
at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:434)
at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:138)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2074)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1745)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1454)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1172)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1162)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:238)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:186)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:405)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:791)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:729)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:652)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:647)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.IllegalArgumentException: Must specify table name
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.createHiveOutputFormat(FileSinkOperator.java:1158)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.checkOutputSpecs(FileSinkOperator.java:1133)
... 38 more
Caused by: java.lang.IllegalArgumentException: Must specify table name
at
org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:191)
at
org.apache.hive.common.util.ReflectionUtil.setConf(ReflectionUtil.java:101)
at
org.apache.hive.common.util.ReflectionUtil.newInstance(ReflectionUtil.java:87)
at
org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:314)
at
org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:292)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.createHiveOutputFormat(FileSinkOperator.java:1156)
... 39 more
Job Submission failed with exception

Re: Does MERGE feature works only on TEZ in Hive-2.3?

[Oleksiy S]

Thanks!

On Wed, Apr 18, 2018 at 7:45 PM, Eugene Koifman <ekoif...@hortonworks.com>
wrote:

> It works on MR as well.
>
> From: Oleksiy S <osayankin.superu...@gmail.com>
> Reply-To: "u...@hive.apache.org" <u...@hive.apache.org>
> Date: Wednesday, April 18, 2018 at 5:24 AM
> To: "u...@hive.apache.org" <u...@hive.apache.org>, "dev@hive.apache.org" <
> dev@hive.apache.org>
> Subject: Does MERGE feature works only on TEZ in Hive-2.3?
>
> Hi all!
>
> Does MERGE feature  works only on TEZ in Hive-2.3?
>
> see
>
>   *   https://issues.apache.org/jira/browse/HIVE-10924
>   *   https://community.hortonworks.com/articles/97113/hive-acid-
> merge-by-example.html
>
> --
> Oleksiy
>



-- 
Oleksiy

Does MERGE feature works only on TEZ in Hive-2.3?

[Oleksiy S]

Hi all!

Does MERGE feature  works only on TEZ in Hive-2.3?

see

   - https://issues.apache.org/jira/browse/HIVE-10924
   -
   
https://community.hortonworks.com/articles/97113/hive-acid-merge-by-example.html


-- 
Oleksiy

Subscribe me

[Oleksiy S]

-- 
Oleksiy

Hive-2.0.0 and Oozie Integration

[Oleksiy S]

Hi friends!

Could you help me, how can I integrate Hive-2.0.0 and Oozie?

-- 
Oleksiy