[
https://issues.apache.org/jira/browse/TRAFODION-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826409#comment-15826409
]
ASF GitHub Bot commented on TRAFODION-2440:
-------------------------------------------
GitHub user DaveBirdsall opened a pull request:
https://github.com/apache/incubator-trafodion/pull/915
[TRAFODION-2440] Replace fixed length buffers with strings in LOB code
The LOB code has two uses:
1. To read and write LOB column values in Trafodion tables
2. To read Hive files into the Trafodion executor
The LOB code made use of fixed length buffers to store LOB file names.
Unfortunately these are too short in general for use with Hive files. Rather
than simply make them longer, I have replaced these fixed length buffers with
string data types in most places in the code.
Some design notes:
The LOB code is in the middle of a slow transition from using STL classes
and the global heap to using the Trafodion NAHeap classes and heaps for memory
management. Where a LOB class uses the STL heap, I replaced its fixed length
buffer members with an STL string. Where a LOB class uses NAHeap, I used a
Trafodion NAString instead.
In one place I left the fixed length buffer alone: I did this where the LOB
code calls the CLI function SQL_EXEC_LOBcliInterface. It appears that this code
path is traversed only for LOB column values. (If anyone knows otherwise,
please comment.)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/DaveBirdsall/incubator-trafodion Trafodion2440
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafodion/pull/915.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #915
----
commit 49498dfdf6b629bf349f9630fdb5ceb197e93bf9
Author: Dave Birdsall <[email protected]>
Date: 2017-01-17T17:07:11Z
[TRAFODION-2440] Replace fixed length buffers with strings in LOB code
----
> Accessing a Hive table with long names results in error 8436
> ------------------------------------------------------------
>
> Key: TRAFODION-2440
> URL: https://issues.apache.org/jira/browse/TRAFODION-2440
> Project: Apache Trafodion
> Issue Type: Bug
> Components: -exe, sql-exe
> Affects Versions: 2.1-incubating
> Reporter: David Wayne Birdsall
> Assignee: David Wayne Birdsall
>
> If we use a long Hive database name and a long Hive table name, attempts to
> select from that table from Trafodion fail with SQL error 8436.
> Example: Use the following hive script to create a table via the Hive shell:
> drop database if exists
> Trafodion2434abcdabcdefghijabcdefghij01234567890123456789012345678901234567890123456789012345678901234567890123456789
> cascade;
> create database
> Trafodion2434abcdabcdefghijabcdefghij01234567890123456789012345678901234567890123456789012345678901234567890123456789
> ;
> use
> Trafodion2434abcdabcdefghijabcdefghij01234567890123456789012345678901234567890123456789012345678901234567890123456789
> ;
> drop table if exists
> ProblemTableSTR1abcdefabcdefghijabcdefghij01234567890123456789012345678901234567890123456789012345678901234567890123456789;
> create table
> ProblemTableSTR1abcdefabcdefghijabcdefghij01234567890123456789012345678901234567890123456789012345678901234567890123456789
> (
> Column01 int,
> Column02 string
> ) clustered by (Column01) sorted by (Column01) into 4 buckets row format
> delimited fields terminated by '|';
> insert into
> ProblemTableSTR1abcdefabcdefghijabcdefghij01234567890123456789012345678901234567890123456789012345678901234567890123456789
> values
> (1,'one');
>
> insert into
> ProblemTableSTR1abcdefabcdefghijabcdefghij01234567890123456789012345678901234567890123456789012345678901234567890123456789
> values
> (2,'two');
> insert into
> ProblemTableSTR1abcdefabcdefghijabcdefghij01234567890123456789012345678901234567890123456789012345678901234567890123456789
> values
> (3,'three');
> insert into
> ProblemTableSTR1abcdefabcdefghijabcdefghij01234567890123456789012345678901234567890123456789012345678901234567890123456789
> values
> (4,'four');
> select * from
> ProblemTableSTR1abcdefabcdefghijabcdefghij01234567890123456789012345678901234567890123456789012345678901234567890123456789
> ;
> exit;
> Next, do the following in sqlci:
> control query default HIVE_MAX_STRING_LENGTH '10';
> set schema
> hive.Trafodion2434abcdabcdefghijabcdefghij01234567890123456789012345678901234567890123456789012345678901234567890123456789
> ;
> showddl
> ProblemTableSTR1abcdefabcdefghijabcdefghij01234567890123456789012345678901234567890123456789012345678901234567890123456789
> ;
> select * from
> ProblemTableSTR1abcdefabcdefghijabcdefghij01234567890123456789012345678901234567890123456789012345678901234567890123456789
> ;
> This will fail as follows:
> >>control query default HIVE_MAX_STRING_LENGTH '10';
> --- SQL operation complete.
> >>
> >>set schema
> >>hive.Trafodion2434abcdabcdefghijabcdefghij01234567890123456789012345678901234567890123456789012345678901234567890123456789
> +>;
> --- SQL operation complete.
> >>
> >>showddl
> >>ProblemTableSTR1abcdefabcdefghijabcdefghij01234567890123456789012345678901234567890123456789012345678901234567890123456789
> +>;
> /* Hive DDL */
> CREATE TABLE
> PROBLEMTABLESTR1ABCDEFABCDEFGHIJABCDEFGHIJ01234567890123456789012345678901234567890123456789012345678901234567890123456789
> (
> COLUMN01 int
> , COLUMN02 string
> )
> stored as textfile
> ;
> --- SQL operation complete.
> >>
> >>select * from
> >>ProblemTableSTR1abcdefabcdefghijabcdefghij01234567890123456789012345678901234567890123456789012345678901234567890123456789
> +>;
> *** ERROR[8436] Mismatch detected between compiletime and runtime hive table
> definitions. DataModMismatchDetails:
> --- 0 row(s) selected.
> >>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
