[
https://issues.apache.org/jira/browse/TRAFODION-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on TRAFODION-3324 started by David Wayne Birdsall.
-------------------------------------------------------
> Make compiler a bit more robust when Hive DDL changes are done in Hive
> ----------------------------------------------------------------------
>
> Key: TRAFODION-3324
> URL: https://issues.apache.org/jira/browse/TRAFODION-3324
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-cmp
> Reporter: David Wayne Birdsall
> Assignee: David Wayne Birdsall
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The scenario: Create a Hive table, populate it with some rows, then in
> Trafodion perform UPDATE STATISTICS on it. Then, in Hive, drop the table and
> create it again with fewer columns. Then in Trafodion, execute "select
> count(*)" against the table. Trafodion will then core, because the old
> histograms are still there and are not consistent with the new Hive DDL.
> The following scripts will reproduce it:
> In Hive, do:
> {quote}create table hive1
> ( a int,
> b int,
> c int )
> stored as textfile;
> {quote}
> In Trafodion sqlci do:
> {quote}insert into hive.hive.hive1 values
> (1,2,3),(2,2,3),(2,3,4),(3,3,5),(3,4,6),(3,6,
> 8),(4,1,1);
> update statistics for table hive.hive.hive1 on every column;
> {quote}
> Then in Hive do:
> drop table if exists hive1;
> {quote}create table hive1
> ( a int,
> b int )
> stored as textfile;
> {quote}
> Then in Trafodion sqlci do:
> {quote}select count(*) from hive.hive.hive1;
> {quote}
> You will now get a sqlci core, with a stack trace like:
> {quote}(gdb) bt
> #0 0x00007fe5b6976495 in raise () from /lib64/libc.so.6
> #1 0x00007fe5b6977bfd in abort () from /lib64/libc.so.6
> #2 0x00007fe5b63a93ea in assert_botch_abend (f=
> 0x7fe5b06459e8 "../common/Collections.cpp", l=903, m=
> 0x7fe5b06459c8 "List index exceeds # of entries", c=0x0)
> at ../export/NAAbort.cpp:285
> #3 0x00007fe5b63a9107 in NAAbort (filename=
> 0x7fe5b06459e8 "../common/Collections.cpp", lineno=903, msg=
> 0x7fe5b06459c8 "List index exceeds # of entries")
> at ../export/NAAbort.cpp:207
> #4 0x00007fe5b0455371 in NAList<NAColumn*>::operator[] (this=0x7fe5a3a45e88,
> i=2) at ../common/Collections.cpp:903
> #5 0x00007fe5afd95ca6 in HSHistogrmCursor::fetch (this=0x7fff381773a0, cs=
> ..., cursor2=..., colmap=0x7fe594704428, fakeHistogram=0x7fe5947043a8,
> emptyHistogram=0x7fe5947043c8, smallSampleHistogram=0x7fe5947043e8,
> smallSampleSize=0x7fe594704408, fakeRowCount=@0x7fff38177f78, statsTime=
> @0x7fe5a3a45fd8, allFakeStats=@0x7fff38178794, preFetch=1, offset=0,
> tabDef=0x7fe594705238, cmpContextSwitched=1) at ../ustat/hs_read.cpp:1537
> #6 0x00007fe5afd954ad in readHistograms (tabDef=0x7fe594705238, fullQualName=
> ..., histogramTableName=..., histintsTableName=..., specialTable=0, type=
> ExtendedQualName::NORMAL_TABLE, colArray=..., statsTime=@0x7fe5a3a45fd8,
> allFakeStat=@0x7fff38178794, preFetch=1, fakeHistogram=0x7fe5947043a8,
> emptyHistogram=0x7fe5947043c8, smallSampleHistogram=0x7fe5947043e8,
> ---Type <return> to continue, or q <return> to quit---
> smallSampleSize=0x7fe594704408, colmap=0x7fe594704428, histogramRowCount=
> @0x7fff38177f78, cs=0x7fff38177d10, offset=0) at ../ustat/hs_read.cpp:1330
> #7 0x00007fe5afd93d3f in FetchHistograms (qualifiedName=..., type=
> ExtendedQualName::NORMAL_TABLE, colArray=..., colStatsList=...,
> isSQLMPTable=0, heap=0x7fe5946e3228, statsTime=@0x7fe5a3a45fd8,
> allFakeStat=@0x7fff38178794, preFetch=1, createStatsSize=0)
> at ../ustat/hs_read.cpp:962
> #8 0x00007fe5ae795d23 in HistogramCache::createColStatsList (this=
> 0x7fe5a3a6cac8, table=..., cachedHistograms=0x0)
> at ../optimizer/NATable.cpp:497
> #9 0x00007fe5ae7958ec in HistogramCache::getHistograms (this=0x7fe5a3a6cac8,
> table=...) at ../optimizer/NATable.cpp:327
> #10 0x00007fe5ae7a8d6d in NATable::getStatistics (this=0x7fe5a3a45be0)
> at ../optimizer/NATable.cpp:5980
> #11 0x00007fe5aea95bae in TableDesc::getTableColStats (this=0x7fe5946eb4d8)
> at ../optimizer/TableDesc.cpp:373
> #12 0x00007fe5b0631eda in TableDesc::tableColStats (this=0x7fe5946eb4d8)
> at ../optimizer/TableDesc.h:134
> #13 0x00007fe5ae892f6e in Scan::synthLogProp (this=0x7fe5946c86b0, normWAPtr=
> 0x7fff3817b270) at ../optimizer/OptLogRelExpr.cpp:5193
> #14 0x00007fe5ae880279 in RelExpr::synthLogProp (this=0x7fe5946f70f0,
> normWAPtr=0x7fff3817b270) at ../optimizer/OptLogRelExpr.cpp:622
> #15 0x00007fe5ae88ff09 in GroupByAgg::synthLogProp (this=0x7fe5946f70f0,
> ---Type <return> to continue, or q <return> to quit---
> {quote}
> Note: Trafodion has the feature that it can execute Hive DDL. So, the CREATE
> TABLE and DROP TABLE statements could be done through sqlci directly. (The
> table name has to be qualified as hive.hive.hive1 however.) When done through
> sqlci, any Trafodion histograms are cleaned up and this issue does not occur.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
