Re: Configure hiveserver2 logs

[Chetna C]

Hi Kishore,
Setting above 3 parameters enables query/operations logs. If you are
looking for HiveServer2 process log, you'll need to configure it while
launching HiveServer2 Process  with following jvm params
"-Dhive.log.dir= -Dhive.log.file=hive-server2.log"

You can refer to "Hive Logging" section on GettingStarted

page for more details.

Thanks,
Chetna Chaudhari

On 26 September 2016 at 21:26, kishore kumar  wrote:

> Hi Hive Users,
>
> Using hive 1.2 version,
>
> I am connecting hiveserver2 via jdbc connection, could any one suggest me
> how to configure log file
>
> using this link https://cwiki.apache.org/confluence/display/Hive/
> HiveServer2+Clients
>
> I set these 3 parameters values
>
> - hive.server2.logging.operation.enabled
> 
>  =
> true
> - hive.server2.logging.operation.log.location
> =
> /${username}/logs
> - hive.server2.logging.operation.level
> 
> =VERBOSE
>
> Am i missing something ? kindly help me. I couldn't see any file
> generating logs in defined location.
>
> --
> Thanks,
> KK.
>

Re: exporting and importing a schema/database from Hive Test to Hive DEV

[Chetna C]

Hi Ashok,
You can achieve this by either sharing same metastore between test and
dev instance. With isolated access to databases , say test-user and
dev-user, where dev-user can access all test databases.Or you can use
Hive/LanguageManual+ImportExport

for
migration.

Thanks,
Chetna Chaudhari

On 11 November 2015 at 13:55, Ashok Kumar  wrote:

> Hi gurus,
>
> What is the easiest way of exporting one database in Hive in Test and
> importing it to say DEV database in another instance.
>
> How metastore at target will handle this please?
>
> Thanks
>

Re: Small files under SequenceFile table partition directories

[Chetna C]

You can get rid of processing overhead by using splittable compression for
your hive table data, something like 4mc .
Or you can use hadoop's getmerge utility to merge small files periodically.

Thanks,
Chetna Chaudhari

On 11 November 2015 at 10:56, reveen joe  wrote:

> Hi,
>
> Most of our Hive tables are SequenceFile tables and there are currently
> many small file ranging from *1-4 MB* under the Partition directories
> (created by insert-overwrite). I am assuming this is due to 2 reasons
>
> 1. Some of our tables are Bucketed and so individual files are created for
> each bucket of data for a given partition.
>
> 2. The places where we have set number of Reducers, produce 1 file per
> Reducer.
>
> So, the dir structure looks like below.
>
> /PATH/TO/TABLE/DIR/partition_column=2015-11-01//00_0
> /PATH/TO/TABLE/DIR/partition_column=2015-11-01//01_0
> /PATH/TO/TABLE/DIR/partition_column=2015-11-01//02_0
> /PATH/TO/TABLE/DIR/partition_column=2015-11-01//03_0
>
> 
>
> 
>
> 
> /PATH/TO/TABLE/DIR/partition_column=2015-11-01//000379_0
>
> The Block size of the cluster is 128 MB. I know that sequence file can
> store FileName as Key and FileContent as Value in sequence Files but in
> this case they are independent files.
>
> Am I right that - this would add overhead to the further processing of
> this data as each file would need to spin up a JVM to start a Map Task
> against that file and also because of the Disk IO overhead?
>
> If so, what could be the best remedy to combine the small files under a
> partition directory? Thank you.
>
>

Re: [ANNOUNCE] New Hive PMC Chair - Ashutosh Chauhan

[Chetna C]

Congrats Ashutosh !

Thanks,
Chetna Chaudhari

On 17 September 2015 at 06:53, Navis Ryu  wrote:

> Congratulations!
>
> 2015-09-17 9:35 GMT+09:00 Xu, Cheng A :
> > Congratulations, Ashutosh!
> >
> > -Original Message-
> > From: Mohammad Islam [mailto:misla...@yahoo.com.INVALID]
> > Sent: Thursday, September 17, 2015 8:23 AM
> > To: user@hive.apache.org; Hive
> > Subject: Re: [ANNOUNCE] New Hive PMC Chair - Ashutosh Chauhan
> >
> > Congratulations Asutosh!
> >
> >
> >  On Wednesday, September 16, 2015 4:51 PM, Bright Ling <
> brig...@hostworks.com.au> wrote:
> >
> >
> >  #yiv7221259285 #yiv7221259285 -- _filtered #yiv7221259285
> {font-family:SimSun;panose-1:2 1 6 0 3 1 1 1 1 1;} _filtered #yiv7221259285
> {font-family:PMingLiU;panose-1:2 2 5 0 0 0 0 0 0 0;} _filtered
> #yiv7221259285 {panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv7221259285
> {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered
> #yiv7221259285 {font-family:Tahoma;panose-1:2 11 6 4 3 5 4 4 2 4;}
> _filtered #yiv7221259285 {panose-1:2 2 5 0 0 0 0 0 0 0;} _filtered
> #yiv7221259285 {panose-1:2 1 6 0 3 1 1 1 1 1;}#yiv7221259285 #yiv7221259285
> p.yiv7221259285MsoNormal, #yiv7221259285 li.yiv7221259285MsoNormal,
> #yiv7221259285 div.yiv7221259285MsoNormal
> {margin:0cm;margin-bottom:.0001pt;font-size:12.0pt;}#yiv7221259285 a:link,
> #yiv7221259285 span.yiv7221259285MsoHyperlink
> {color:blue;text-decoration:underline;}#yiv7221259285 a:visited,
> #yiv7221259285 span.yiv7221259285MsoHyperlinkFollowed
> {color:purple;text-decoration:underline;}#yiv7221259285
> p.yiv7221259285MsoAcetate, #yiv7221259285 li.yiv7221259285MsoAcetate,
> #yiv7221259285 div.yiv7221259285MsoAcetate
> {margin:0cm;margin-bottom:.0001pt;font-size:8.0pt;}#yiv7221259285
> span.yiv7221259285EmailStyle17 {color:#1F497D;}#yiv7221259285
> span.yiv7221259285BalloonTextChar {}#yiv7221259285
> .yiv7221259285MsoChpDefault {font-size:10.0pt;} _filtered #yiv7221259285
> {margin:72.0pt 72.0pt 72.0pt 72.0pt;}#yiv7221259285
> div.yiv7221259285WordSection1 {}#yiv7221259285 Congratulations Asutosh!
>From: Sathi Chowdhury [mailto:sathi.chowdh...@lithium.com]
> > Sent: Thursday, 17 September 2015 8:04 AM
> > To: user@hive.apache.org
> > Subject: Re: [ANNOUNCE] New Hive PMC Chair - Ashutosh Chauhan
> Congrats Asutosh!From:Sergey Shelukhin
> > Reply-To: "user@hive.apache.org"
> > Date: Wednesday, September 16, 2015 at 2:31 PM
> > To: "user@hive.apache.org"
> > Subject: Re: [ANNOUNCE] New Hive PMC Chair - Ashutosh Chauhan
> Congrats!From:Alpesh Patel 
> > Reply-To: "user@hive.apache.org" 
> > Date: Wednesday, September 16, 2015 at 13:24
> > To: "user@hive.apache.org" 
> > Subject: Re: [ANNOUNCE] New Hive PMC Chair - Ashutosh Chauhan
> Congratulations AshutoshOn Wed, Sep 16, 2015 at 1:23 PM, Pengcheng
> Xiong  wrote: Congratulations Ashutosh!On Wed, Sep
> 16, 2015 at 1:17 PM, John Pullokkaran 
> wrote: Congrats Ashutosh!From:Vaibhav Gumashta <
> vgumas...@hortonworks.com>
> > Reply-To: "user@hive.apache.org" 
> > Date: Wednesday, September 16, 2015 at 1:01 PM
> > To: "user@hive.apache.org" , "d...@hive.apache.org"
> 
> > Cc: Ashutosh Chauhan 
> > Subject: Re: [ANNOUNCE] New Hive PMC Chair - Ashutosh Chauhan
> Congrats Ashutosh! —VaibhavFrom:Prasanth Jayachandran <
> pjayachand...@hortonworks.com>
> > Reply-To: "user@hive.apache.org" 
> > Date: Wednesday, September 16, 2015 at 12:50 PM
> > To: "d...@hive.apache.org" , "user@hive.apache.org"
> 
> > Cc: "d...@hive.apache.org" , Ashutosh Chauhan <
> hashut...@apache.org>
> > Subject: Re: [ANNOUNCE] New Hive PMC Chair - Ashutosh Chauhan
> Congratulations Ashutosh!
> >
> >  On Wed, Sep 16, 2015 at 12:48 PM -0700, "Xuefu Zhang" <
> xzh...@cloudera.com> wrote: Congratulations, Ashutosh!. Well-deserved.
> >
> > Thanks to Carl also for the hard work in the past few years!
> >
> > --Xuefu
> >
> > On Wed, Sep 16, 2015 at 12:39 PM, Carl Steinbach  wrote:
> >
> >> I am very happy to announce that Ashutosh Chauhan is taking over as
> >> the new VP of the Apache Hive project. Ashutosh has been a longtime
> >> contributor to Hive and has played a pivotal role in many of the major
> >> advances that have been made over the past couple of years. Please
> >> join me in congratulating Ashutosh on his new role!
> >>
> >
> >
>

Re: Tables missing on the file system

[Chetna C]

Hi Ravi,
Please make sure the location mentioned while create table exists at
the time of *'MSCK REPAIR'*. This error occurs, if location does not exists
on fs.

Thanks,
Chetna Chaudhari

On 15 September 2015 at 12:03, ravi teja  wrote:

> Hi,
> I am getting this exception when I repair a table.
> Not sure what this means, didnt get any info while searching also.
>
> Can someone guide , what this means?
>
>
>  CREATE EXTERNAL TABLE IF NOT EXISTS  test_table 
>  OK
>  Time taken: 0.124 seconds
>
>  MSCK REPAIR TABLE test_table
>  OK
>  Tables missing on filesystem:test_table
>
>  Time taken: 0.691 seconds, Fetched: 1 row(s)
>
>
> Thanks,
> Ravi
>
>