Re: Error with flush_length File in Orc, in hive 2.1.0 and mr execution engine.

Hi Eugene, One more observation, in namenode logs when i run select count(*) on individual tables i still see the same error as before, org.apache.hadoop.ipc.Server: IPC Server handler 1 on 9000, call org.apache.hadoop.hdfs.protocol.ClientProtocol.getBlockLocations from 192.168.120.133:47114

Re: Error with flush_length File in Orc, in hive 2.1.0 and mr execution engine.

Hi Eugene, select count(*) from mls_public_record_association_snapshot_orc pra left outer join mls_listing_snapshot_orc ml on pra.primary_listing_id = ml.id left outer join attribute a on a.id = ml.standard_status ran till end and threw the below exception. MapReduce Total cumulative CPU

Re: Error with flush_length File in Orc, in hive 2.1.0 and mr execution engine.

If you can run this, then it’s safe to ignore “00094_flush_length” messages and the issue is somewhere else select count(*) from mls_public_record_association_snapshot_orc pra left outer join mls_listing_snapshot_orc ml on pra.primary_listing_id = ml.id left outer join

Re: Error with flush_length File in Orc, in hive 2.1.0 and mr execution engine.

PFB EXPLAIN PLAN from hive, and query has issues at stage-14, i have also run another query which also fail in stage, *Map Reduce Local Work.* STAGE DEPENDENCIES: Stage-13 is a root stage , consists of Stage-1 Stage-1 Stage-10 depends on stages: Stage-1 , consists of Stage-15, Stage-2

Re: Error with flush_length File in Orc, in hive 2.1.0 and mr execution engine.

Hi Eugene, PFB Transaction table in green and parquet tables in yellow, INSERT INTO access_logs.crawlstats_dpp PARTITION(day="2016-10-23") select pra.url as prUrl,pra.url_type as urlType,CAST(pra.created_at AS timestamp) as prCreated, CAST(pra.updated_at AS timestamp) as prUpdated,

Re: hive transactional table compaction fails

On Thu, Oct 27, 2016 at 12:00 AM, Eugene Koifman wrote: > could you provide output of ³SHOW COMPACTIONS¹ command I have dropped the earlier table ...but i created a simple table create table a (x int , y int) clustered by (x) into 2 buckets stored as orc

Re: hive transactional table compaction fails

could you provide output of ³SHOW COMPACTIONS¹ command On 10/25/16, 11:21 PM, "aft" wrote: >Well the auto compactor fails every timeand it has been going on >for couple of days. As "day" is a partition, I have to assume, it >happens with all of them. > >Both manual/auto

Re: Can I specify database name in hive metastore service?

I am not familiar with MySQL. However, as long as login/user used in Hive-site.xml is bound to given Hive database (metastore) on RDBMS it should work. In general you can have multiple databases/schema for multiple instances of Hive each having its own metadata on the data server. In Sybase or

Re: Can I specify database name in hive metastore service?

Hi Huang, According to the picture you want to start 2 HiveServer2 instances both with embedded metastore, where the metastore db is stored in the same RDBMS in a different database. This is certainly possible. You have to set the database options according to this:

Re: Hive metadata on Hbase

Hi Mich, No, I am not using HBase as a metastore now, but I am eager for it to become production ready and released in CDH and HDP. Concerning locks, I think HBase would do fine because it is ACID at the row level. It only appends data on HDFS, but it works by keeping regions in RAM, plus a

Re: hive transactional table compaction fails

Well the auto compactor fails every timeand it has been going on for couple of days. As "day" is a partition, I have to assume, it happens with all of them. Both manual/auto compactor fails. On Wed, Oct 26, 2016 at 12:52 AM, Eugene Koifman wrote: > does this happen