[
https://issues.apache.org/jira/browse/TRAFODION-2748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suresh Subbiah updated TRAFODION-2748:
--------------------------------------
Summary: Slower upsert performance when source is a Hive scan (was: Upsert
performance when source is a Hive scan)
> Slower upsert performance when source is a Hive scan
> ----------------------------------------------------
>
> Key: TRAFODION-2748
> URL: https://issues.apache.org/jira/browse/TRAFODION-2748
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-cmp
> Affects Versions: 1.3-incubating
> Reporter: Suresh Subbiah
> Assignee: Suresh Subbiah
> Priority: Minor
> Fix For: 1.3-incubating
>
>
> From Eric Owhadi.
> In the following set of steps, the cqd GEN_DP2I_SIZE_DOWN should not have
> been necessary. The quesize should be determined by rowset size. However we
> were not flagging Hive scans as returning multiple rows in OltOptInfo class
> in the generator. This cause the initial queue size to be set too low. The
> fix could help improve performance for other query types with Hive scan too.
> cqd GEN_DP2I_SIZE_DOWN '1024';
> cqd UPD_ORDERED 'OFF';
> cqd HBASE_ROWSET_VSBB_SIZE '20480';
> cqd allow_nullable_unique_key_constraint 'ON';
> CREATE TABLE STORE_SALES5(
> ss_sold_date_sk int,
> ss_sold_time_sk int,
> ss_item_sk int,
> ss_customer_sk int,
> ss_cdemo_sk int,
> ss_hdemo_sk int,
> ss_addr_sk int,
> ss_store_sk int,
> ss_promo_sk int,
> ss_ticket_number int,
> ss_quantity int,
> ss_wholesale_cost real,
> ss_list_price real,
> ss_sales_price real,
> ss_ext_discount_amt real,
> ss_ext_sales_price real,
> ss_ext_wholesale_cost real,
> ss_ext_list_price real,
> ss_ext_tax real,
> ss_coupon_amt real,
> ss_net_paid real,
> ss_net_paid_inc_tax real,
> ss_net_profit real)
> store by (ss_sold_date_sk, ss_sold_time_sk)
> Salt using 96 partitions on (ss_sold_date_sk, ss_sold_time_sk)
> HBASE_OPTIONS
> (
> DATA_BLOCK_ENCODING = 'FAST_DIFF',
> COMPRESSION = 'SNAPPY',
> MEMSTORE_FLUSH_SIZE = '1400000000',
> MAX_FILESIZE = '107374182400',
> BLOOMFILTER = 'NONE',
> BLOCKSIZE = '524280'
> )
> ;
> alter 'TRAF_1500000:TRAFODION.ERIC.STORE_SALES5', CONFIGURATION =>
> {'hbase.hstore.compaction.max.size' => '4000000000'}
> prepare s from
> upsert using load into store_sales5 select * from
> hive.tpcds_sf10000.store_sales_ext;
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
