Unsubscribe

Unsubscribe - Original Message - From: "Kevin Risden" To: user@hive.apache.org Sent: Tuesday, October 16, 2018 9:44:17 AM Subject: Re: HiveServer2 performance references? Thanks for the responses. I'll wait to test with HIVE-20621 to see how much that helps. For HIVE-17194, doesn't

Re: Unable to read hive external table data which is linked to s3 after upgradation from 2.1.1 to 2.3.3

because i think if these variables are by default part of hidden.list then there will be some other proper workaround for this. Regards, Anup Tiwari On Thu, Apr 12, 2018 at 7:44 PM, Richard A. Bross < r...@oaktreepeak.com > wrote: I hear you, but given the exception log, it d

Re: Unable to read hive external table data which is linked to s3 after upgradation from 2.1.1 to 2.3.3

* fs.s3a.access.key * fs.s3a.secret.key * fs.s3a.connection.maximum * fs.s3a.impl Regards, Anup Tiwari On Thu, Apr 12, 2018 at 7:19 PM, Richard A. Bross < r...@oaktreepeak.com > wrote: Based on the exception, it looks more like an AWS credentials issue than a Hive i

Re: Unable to read hive external table data which is linked to s3 after upgradation from 2.1.1 to 2.3.3

Based on the exception, it looks more like an AWS credentials issue than a Hive issue. Are you running in AWS EMR, on-prem? In AWS the resource accessing the S3 bucket would have to have an IAM that gave permission. If you are running somewhere else whatever AWS login you use would have to

Re: ALTER TABLE DROP PARTITION not working on S3

MSCK REPAIR TABLE tablename" to make sure that the partitions are correctly loaded and then try again dropping that particular partition? Or look at your s3 folder if you see any such "partition folder file" and check if it is missing for this particular partition? On 5 April 2018 a

Re: ALTER TABLE DROP PARTITION not working on S3

ROP PARTITION not working on S3 Hi Richard, could you please check if your table is EXTERNAL? You can see it with a "DESCRIBE FORMATTED table_name ;" That's what external tables are for, they don't delete underlying data when you drop them. On 5 April 2018 at 15:18, Richard

Re: ALTER TABLE DROP PARTITION not working on S3

you please check if your table is EXTERNAL? You can see it with a "DESCRIBE FORMATTED table_name ;" That's what external tables are for, they don't delete underlying data when you drop them. On 5 April 2018 at 15:18, Richard A. Bross < r...@oaktreepeak.com > wrote:

Re: ALTER TABLE DROP PARTITION not working on S3

I think that someone put a file in there manually. Would that prevent Hive from dropping the partition. I also did a "drop table" and the s3 object keys persisted. - Original Message ----- From: "Richard A. Bross" <r...@oaktreepeak.com> To: user@hive.apache.o

ALTER TABLE DROP PARTITION not working on S3

Hi, I have a Hive managed table on S3, "api_measurements". I've tried dropping a partition like so: hive> alter table api_measurements drop if exists partition(daydate='2018-04-04', epoch=1522876500); Dropped the partition daydate=2018-04-04/epoch=1522876500 OK Time taken: 2.109 seconds Yet

Re: Hive, Tez, clustering, buckets, and Presto

1_Partitioning_and_Bucketing_in_Hive-Based_Big_Data_Warehouses Cheers, Gopal On 4/3/18, 1:42 PM, "Richard A. Bross" <r...@oaktreepeak.com> wrote: Gopal, The Presto devs say they are willing to make the changes to adhere to the Hive bucket spec. I quoted

Re: Building Datwarehouse Application in Spark

Mahender, To really address your question I think that you'd have to supply a bit more information, such as the kind of data that you want to save; RBDMS type look ups, key/value/index type look ups, insert velocity, etc. These wide choices of technologies are suited to different use cases,

Re: Hive, Tez, clustering, buckets, and Presto

Gopal, The Presto devs say they are willing to make the changes to adhere to the Hive bucket spec. I quoted "Presto could fix their fail-safe for bucketing implementation to actually trust the Hive bucketing spec & get you out of this mess - the bucketing contract for Hive is actual file

Re: Hive, Tez, clustering, buckets, and Presto

Gopal, Thanks for this. Great information and something to look at more closely to better understand the internals. Rick - Original Message - From: "Gopal Vijayaraghavan" To: user@hive.apache.org Sent: Tuesday, April 3, 2018 3:15:46 AM Subject: Re: Hive, Tez,

Re: Hive, Tez, clustering, buckets, and Presto

I'm really confused and could use help understanding. The Hive documentation here https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL+BucketedTables says: "Bucketed tables are fantastic in that they allow much more efficient sampling than do non-bucketed tables, and they may

Re: Hive, Tez, clustering, buckets, and Presto

Gopal, Thanks for taking the time to try and help. A few things in relation to your response: * Yes, the 'epoch' column is an hourly timestamp. Clustering by a column with high cardinality would make little sense. * I'm interested in your statement that CLUSTERED BY does not CLUSTER BY. My

Hive, Tez, clustering, buckets, and Presto

Hello there, I've done a ton of reading and testing between Hive, Presto, Tez, etc. In our test environment I have 8 tables being loaded with Hive that works fine under certain circumstances. Most of our data is taken from JSON files in from S3, creates a temporary external table, and then