Re: how to extract metadata of hive tables in speed

Thank you very much for your reply, I am wondering whether I use the webhcat rightly, I don’t think it is normal to create all the directories and objects to get a table describ and take 8 seconds. The webhcat should not be so slow, Or it is because I forget to start some server which can

Re: how to extract metadata of hive tables in speed

Hi, Alexander Kolbasov has a project which might interest you (keeping in mind, that this is not production ready - more like a proof of concept): https://github.com/akolb1/gometastore/blob/master/hmstool/doc/hmstool.md Also you can use HMS thrift API directly to access the MetaStore, or if you

Re: how to extract metadata of hive tables in speed

Hi, Disclaimer: I am not too familiar with the webhcat yet. From the logs, I see, that: - the first 3 seconds spent on starting a new session, and maybe a driver - this can be reduced, if the session is already there, and the HiveServer2 is started (but do not know if webhcat could use HS2, or

Re: how to extract metadata of hive tables in speed

Hi, This really helps me a lot, and I will try it soon, Thank you very much! Hou > 在 2018年4月23日，下午3:57，Peter Vary 写道： > > Hi, > > Disclaimer: I am not too familiar with the webhcat yet. > From the logs, I see, that: > - the first 3 seconds spent on starting a new session,

Re: Review Request 66663: HIVE-19171 Persist runtime statistics in metastore

> On April 21, 2018, 5:59 a.m., Ashutosh Chauhan wrote: > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java > > Lines 4273 (patched) > > > > > > Better name : hive.runtime.stats.batch.size > > Also

Re: Review Request 66290: HIVE-14388 : Add number of rows inserted message after insert command in Beeline

> On March 27, 2018, 12:53 p.m., Peter Vary wrote: > > jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java > > Line 712 (original), 712 (patched) > > > > > > Is it possible to behave differently, when we have

Re: Review Request 64632: HIVE-18247: Use DB auto-increment for indexes

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64632/#review201724 --- Hi Sasha, I have a few questions: - Is this changing the table

[jira] [Created] (HIVE-19272) There should be a feature to control the .hivehistory (Hive CLI) command history count limit on user's wish

vaibhav created HIVE-19272: -- Summary: There should be a feature to control the .hivehistory (Hive CLI) command history count limit on user's wish Key: HIVE-19272 URL: https://issues.apache.org/jira/browse/HIVE-19272

Re: Review Request 66663: HIVE-19171 Persist runtime statistics in metastore

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3/ --- (Updated April 23, 2018, 11:25 a.m.) Review request for hive and Ashutosh

[jira] [Created] (HIVE-19273) Fix TestBeeLineWithArgs.testQueryProgressParallel

Zoltan Haindrich created HIVE-19273: --- Summary: Fix TestBeeLineWithArgs.testQueryProgressParallel Key: HIVE-19273 URL: https://issues.apache.org/jira/browse/HIVE-19273 Project: Hive Issue

Re: Review Request 66567: Migrate to Murmur hash for shuffle and bucketing

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66567/ --- (Updated April 23, 2018, 4:36 p.m.) Review request for hive, Ashutosh Chauhan,

[jira] [Created] (HIVE-19274) Add an OpTreeSignature persistence checker hook

Zoltan Haindrich created HIVE-19274: --- Summary: Add an OpTreeSignature persistence checker hook Key: HIVE-19274 URL: https://issues.apache.org/jira/browse/HIVE-19274 Project: Hive Issue

Re: org.apache.hadoop.hive.ql.metadata.HiveMetaStoreClientFactory

Your code searching skills are intact (or, at least this doesn't prove they aren't). That class doesn't exist in Hive and I assume was invented by Amazon. I assume they placed it in o.a.h.hive so they could access package specific classes etc. Apache doesn't prohibit using its name in package

[jira] [Created] (HIVE-19275) Vectorization: Wrong Results / Execution Failures when Vectorization turned on in Spark

Matt McCline created HIVE-19275: --- Summary: Vectorization: Wrong Results / Execution Failures when Vectorization turned on in Spark Key: HIVE-19275 URL: https://issues.apache.org/jira/browse/HIVE-19275

Re: Review Request 66567: Migrate to Murmur hash for shuffle and bucketing

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66567/ --- (Updated April 23, 2018, 5:26 p.m.) Review request for hive, Ashutosh Chauhan,

[jira] [Created] (HIVE-19276) Metastore delegation tokens should return non-empty service field in delegation tokens

Yuxiang Chen created HIVE-19276: --- Summary: Metastore delegation tokens should return non-empty service field in delegation tokens Key: HIVE-19276 URL: https://issues.apache.org/jira/browse/HIVE-19276

[jira] [Created] (HIVE-19281) incorrect protocol name for LLAP AM plugin

Sergey Shelukhin created HIVE-19281: --- Summary: incorrect protocol name for LLAP AM plugin Key: HIVE-19281 URL: https://issues.apache.org/jira/browse/HIVE-19281 Project: Hive Issue Type:

Re: Review Request 66485: HIVE-19124 implement a basic major compactor for MM tables

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66485/#review201756 ---

Re: Review Request 66290: HIVE-14388 : Add number of rows inserted message after insert command in Beeline

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66290/ --- (Updated April 23, 2018, 9:56 p.m.) Review request for hive, Sahil Takiar and

[jira] [Created] (HIVE-19277) Active/Passive HA web endpoints does not allow cross origin requests

Prasanth Jayachandran created HIVE-19277: Summary: Active/Passive HA web endpoints does not allow cross origin requests Key: HIVE-19277 URL: https://issues.apache.org/jira/browse/HIVE-19277

Re: Review Request 66290: HIVE-14388 : Add number of rows inserted message after insert command in Beeline

> On April 20, 2018, 4:34 p.m., Sahil Takiar wrote: > > jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java > > Line 711 (original), 711 (patched) > > > > > > why change the method call? don't both methods return

Re: Review Request 66290: HIVE-14388 : Add number of rows inserted message after insert command in Beeline

> On March 27, 2018, 12:53 p.m., Peter Vary wrote: > > jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java > > Line 712 (original), 712 (patched) > > > > > > Is it possible to behave differently, when we have

Re: Review Request 66290: HIVE-14388 : Add number of rows inserted message after insert command in Beeline

- Bharathkrishna --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66290/#review201637 --- On April 18, 2018, 11:53 p.m., Bharathkrishna

Re: Review Request 66290: HIVE-14388 : Add number of rows inserted message after insert command in Beeline

- Bharathkrishna --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66290/#review200042 --- On April 18, 2018, 11:53 p.m., Bharathkrishna

Re: Review Request 66485: HIVE-19124 implement a basic major compactor for MM tables

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66485/ --- (Updated April 23, 2018, 7:03 p.m.) Review request for hive and Eugene

[jira] [Created] (HIVE-19279) remove magic directory skipping from CopyTask

Sergey Shelukhin created HIVE-19279: --- Summary: remove magic directory skipping from CopyTask Key: HIVE-19279 URL: https://issues.apache.org/jira/browse/HIVE-19279 Project: Hive Issue Type:

Re: Review Request 66485: HIVE-19124 implement a basic major compactor for MM tables

> On April 23, 2018, 7:24 p.m., Gopal V wrote: > > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java > > Lines 356 (patched) > > > > > > Is that supposed to be a "," or a +? , - Sergey

[jira] [Created] (HIVE-19278) hive build warnings while compiling parser code

Vineet Garg created HIVE-19278: -- Summary: hive build warnings while compiling parser code Key: HIVE-19278 URL: https://issues.apache.org/jira/browse/HIVE-19278 Project: Hive Issue Type: Task

Re: Review Request 66720: HIVE-17657 export/import for MM tables is broken

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66720/ --- (Updated April 23, 2018, 9:18 p.m.) Review request for hive and Eugene

Re: Review Request 63972: [HIVE-18037] Migrate Slider LLAP package to YARN Service framework for Hadoop 3.x

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63972/ --- (Updated April 24, 2018, 1:16 a.m.) Review request for hive and Sergey

[jira] [Created] (HIVE-19280) Invalid error messages for UPDATE/DELETE on insert-only transactional tables

Steve Yeom created HIVE-19280: - Summary: Invalid error messages for UPDATE/DELETE on insert-only transactional tables Key: HIVE-19280 URL: https://issues.apache.org/jira/browse/HIVE-19280 Project: Hive

Re: Review Request 66188: HIVE-18986 Table rename will run java.lang.StackOverflowError in dataNucleus if the table contains large number of columns

> On March 22, 2018, 11:03 a.m., Peter Vary wrote: > > standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java > > Lines 154-158 (original), 154-158 (patched) > > > > > > It might

Re: Review Request 66188: HIVE-18986 Table rename will run java.lang.StackOverflowError in dataNucleus if the table contains large number of columns

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66188/ --- (Updated April 23, 2018, 10:51 p.m.) Review request for hive, Alexander

org.apache.hadoop.hive.ql.metadata.HiveMetaStoreClientFactory

Hello, I'm looking for an abstraction to use for integrating with different (non-Thrift) metadata catalog implementations. I know that AWS Glue manages this and so have explored in EMR (Hive 2.3.2) a little. I see that it uses the "org.apache.hadoop.hive.ql.metadata.HiveMetaStoreClientFactory"

[jira] [Created] (HIVE-19282) don't nest delta directories inside LB directories for ACID tables

Sergey Shelukhin created HIVE-19282: --- Summary: don't nest delta directories inside LB directories for ACID tables Key: HIVE-19282 URL: https://issues.apache.org/jira/browse/HIVE-19282 Project: Hive

[jira] [Created] (HIVE-19283) Select count(distinct()) a couple of times stuck in last reducer

Goun Na created HIVE-19283: -- Summary: Select count(distinct()) a couple of times stuck in last reducer Key: HIVE-19283 URL: https://issues.apache.org/jira/browse/HIVE-19283 Project: Hive Issue

Re: Review Request 66485: HIVE-19124 implement a basic major compactor for MM tables

> On April 23, 2018, 5:04 a.m., Gopal V wrote: > > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java > > Lines 353 (patched) > > > > > > Add a timestamp to the tmp-table and fail-retry if it