Re: Docker Hive using tez without hdfs

2024-01-09 Thread Ayush Saxena 
Hive on MR3 isn’t an official Apache Hive thing, not even an Apache OS thing, so, it is a vendor product just being tried to advertised in the ‘Apache’ Hive spaceSo, it can be all mess, filled with security issues or bugs & we Apache Hive for the record aren’t responsible for that neither do we endorse usage of that or anything outside the scope of Apache-AyushOn 10-Jan-2024, at 1:09 PM, Sungwoo Park  wrote:As far as I know, Hive-Tez supports local mode, but does not standalone mode (like Spark). Hive-MR3 supports standalone mode, so you can run it in any type of cluster.--- SungwooOn Wed, Jan 10, 2024 at 4:22 PM Sanjay Gupta  wrote:I can run hive with mr engine in local mode. Does Hive + Tez also
works in standalone mode ?

On Tue, Jan 9, 2024 at 11:08 PM Sungwoo Park  wrote:
>
> Hello,
>
> I don't have an answer to your problem, but if your goal is to quickly test Hive 3 using Docker, there is an alternative way which uses Hive on MR3.
>
> https://mr3docs.datamonad.com/docs/quick/docker/
>
> You can also run Hive on MR3 on Kubernetes.
>
> Thanks,
>
> --- Sungwoo
>
>
>
> On Wed, Jan 10, 2024 at 3:25 PM Sanjay Gupta  wrote:
>>
>> Hi,
>> Using following docker container to run meta , hiveserver2
>>
>> https://hub.docker.com/r/apache/hive
>> https://github.com/apache/hive/blob/master/packaging/src/docker/
>>
>> I have configured hive-site.xml to se S3
>> When I set in hive.execution.engine to mr hive-site.xml, hive is
>> running fine and I can perform queries but setting to tez fails with
>> error.
>> There is no hdfs but it is running in local mode.
>>
>>     
>>         hive.execution.engine
>>         tez
>>     
>>
>> Any idea how to fix this issue ?
>>
>> hive
>> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
>> Hive Session ID = 03368207-1904-4c4c-b63e-b29dd28e0a71
>>
>> Logging initialized using configuration in
>> jar:file:/opt/hive/lib/hive-common-3.1.3.jar!/hive-log4j2.properties
>> Async: true
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/tez/dag/api/TezConfiguration
>> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:661)
>> at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:591)
>> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:747)
>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at org.apache.hadoop.util.RunJar.run(RunJar.java:308)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:222)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.tez.dag.api.TezConfiguration
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
>>
>>
>> --
>>
>> Thanks
>> Sanjay Gupta



-- 

Thanks
Sanjay Gupta

Re: Docker Hive using tez without hdfs

2024-01-09 Thread Sungwoo Park 
As far as I know, Hive-Tez supports local mode, but does not standalone
mode (like Spark). Hive-MR3 supports standalone mode, so you can run it in
any type of cluster.

--- Sungwoo

On Wed, Jan 10, 2024 at 4:22 PM Sanjay Gupta  wrote:

> I can run hive with mr engine in local mode. Does Hive + Tez also
> works in standalone mode ?
>
> On Tue, Jan 9, 2024 at 11:08 PM Sungwoo Park  wrote:
> >
> > Hello,
> >
> > I don't have an answer to your problem, but if your goal is to quickly
> test Hive 3 using Docker, there is an alternative way which uses Hive on
> MR3.
> >
> > https://mr3docs.datamonad.com/docs/quick/docker/
> >
> > You can also run Hive on MR3 on Kubernetes.
> >
> > Thanks,
> >
> > --- Sungwoo
> >
> >
> >
> > On Wed, Jan 10, 2024 at 3:25 PM Sanjay Gupta  wrote:
> >>
> >> Hi,
> >> Using following docker container to run meta , hiveserver2
> >>
> >> https://hub.docker.com/r/apache/hive
> >> https://github.com/apache/hive/blob/master/packaging/src/docker/
> >>
> >> I have configured hive-site.xml to se S3
> >> When I set in hive.execution.engine to mr hive-site.xml, hive is
> >> running fine and I can perform queries but setting to tez fails with
> >> error.
> >> There is no hdfs but it is running in local mode.
> >>
> >> 
> >> hive.execution.engine
> >> tez
> >> 
> >>
> >> Any idea how to fix this issue ?
> >>
> >> hive
> >> SLF4J: Actual binding is of type
> [org.apache.logging.slf4j.Log4jLoggerFactory]
> >> Hive Session ID = 03368207-1904-4c4c-b63e-b29dd28e0a71
> >>
> >> Logging initialized using configuration in
> >> jar:file:/opt/hive/lib/hive-common-3.1.3.jar!/hive-log4j2.properties
> >> Async: true
> >> Exception in thread "main" java.lang.NoClassDefFoundError:
> >> org/apache/tez/dag/api/TezConfiguration
> >> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:661)
> >> at
> org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:591)
> >> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:747)
> >> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> at java.lang.reflect.Method.invoke(Method.java:498)
> >> at org.apache.hadoop.util.RunJar.run(RunJar.java:308)
> >> at org.apache.hadoop.util.RunJar.main(RunJar.java:222)
> >> Caused by: java.lang.ClassNotFoundException:
> >> org.apache.tez.dag.api.TezConfiguration
> >> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> >>
> >>
> >> --
> >>
> >> Thanks
> >> Sanjay Gupta
>
>
>
> --
>
> Thanks
> Sanjay Gupta
>

Re: Docker Hive using tez without hdfs

2024-01-09 Thread Sanjay Gupta 
I can run hive with mr engine in local mode. Does Hive + Tez also
works in standalone mode ?

On Tue, Jan 9, 2024 at 11:08 PM Sungwoo Park  wrote:
>
> Hello,
>
> I don't have an answer to your problem, but if your goal is to quickly test 
> Hive 3 using Docker, there is an alternative way which uses Hive on MR3.
>
> https://mr3docs.datamonad.com/docs/quick/docker/
>
> You can also run Hive on MR3 on Kubernetes.
>
> Thanks,
>
> --- Sungwoo
>
>
>
> On Wed, Jan 10, 2024 at 3:25 PM Sanjay Gupta  wrote:
>>
>> Hi,
>> Using following docker container to run meta , hiveserver2
>>
>> https://hub.docker.com/r/apache/hive
>> https://github.com/apache/hive/blob/master/packaging/src/docker/
>>
>> I have configured hive-site.xml to se S3
>> When I set in hive.execution.engine to mr hive-site.xml, hive is
>> running fine and I can perform queries but setting to tez fails with
>> error.
>> There is no hdfs but it is running in local mode.
>>
>> 
>> hive.execution.engine
>> tez
>> 
>>
>> Any idea how to fix this issue ?
>>
>> hive
>> SLF4J: Actual binding is of type 
>> [org.apache.logging.slf4j.Log4jLoggerFactory]
>> Hive Session ID = 03368207-1904-4c4c-b63e-b29dd28e0a71
>>
>> Logging initialized using configuration in
>> jar:file:/opt/hive/lib/hive-common-3.1.3.jar!/hive-log4j2.properties
>> Async: true
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/tez/dag/api/TezConfiguration
>> at 
>> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:661)
>> at 
>> org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:591)
>> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:747)
>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at 
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> at 
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at org.apache.hadoop.util.RunJar.run(RunJar.java:308)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:222)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.tez.dag.api.TezConfiguration
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
>>
>>
>> --
>>
>> Thanks
>> Sanjay Gupta



-- 

Thanks
Sanjay Gupta

Re: Docker Hive using tez without hdfs

2024-01-09 Thread Sungwoo Park 
Hello,

I don't have an answer to your problem, but if your goal is to quickly test
Hive 3 using Docker, there is an alternative way which uses Hive on MR3.

https://mr3docs.datamonad.com/docs/quick/docker/

You can also run Hive on MR3 on Kubernetes.

Thanks,

--- Sungwoo



On Wed, Jan 10, 2024 at 3:25 PM Sanjay Gupta  wrote:

> Hi,
> Using following docker container to run meta , hiveserver2
>
> https://hub.docker.com/r/apache/hive
> https://github.com/apache/hive/blob/master/packaging/src/docker/
>
> I have configured hive-site.xml to se S3
> When I set in hive.execution.engine to mr hive-site.xml, hive is
> running fine and I can perform queries but setting to tez fails with
> error.
> There is no hdfs but it is running in local mode.
>
> 
> hive.execution.engine
> tez
> 
>
> Any idea how to fix this issue ?
>
> hive
> SLF4J: Actual binding is of type
> [org.apache.logging.slf4j.Log4jLoggerFactory]
> Hive Session ID = 03368207-1904-4c4c-b63e-b29dd28e0a71
>
> Logging initialized using configuration in
> jar:file:/opt/hive/lib/hive-common-3.1.3.jar!/hive-log4j2.properties
> Async: true
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/tez/dag/api/TezConfiguration
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:661)
> at
> org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:591)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:747)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:308)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:222)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.tez.dag.api.TezConfiguration
> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
>
>
> --
>
> Thanks
> Sanjay Gupta
>

Docker Hive using tez without hdfs

2024-01-09 Thread Sanjay Gupta 
Hi,
Using following docker container to run meta , hiveserver2

https://hub.docker.com/r/apache/hive
https://github.com/apache/hive/blob/master/packaging/src/docker/

I have configured hive-site.xml to se S3
When I set in hive.execution.engine to mr hive-site.xml, hive is
running fine and I can perform queries but setting to tez fails with
error.
There is no hdfs but it is running in local mode.


hive.execution.engine
tez


Any idea how to fix this issue ?

hive
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 03368207-1904-4c4c-b63e-b29dd28e0a71

Logging initialized using configuration in
jar:file:/opt/hive/lib/hive-common-3.1.3.jar!/hive-log4j2.properties
Async: true
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/tez/dag/api/TezConfiguration
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:661)
at 
org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:591)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:747)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:308)
at org.apache.hadoop.util.RunJar.main(RunJar.java:222)
Caused by: java.lang.ClassNotFoundException:
org.apache.tez.dag.api.TezConfiguration
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)


-- 

Thanks
Sanjay Gupta