[
https://issues.apache.org/jira/browse/HIVE-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Satyajit varma updated HIVE-13203:
----------------------------------
Description:
I am trying to create a table with multiple STRUCT column types, one of which
having more than 300 fields.
I get the table created with no problem, but when i load i face few errors in
loading up the data.when i check SHOW CREATE TABLE TABLENAME, i get fields only
till 266th column and rest of them do not appear in the table DDL.(check the
issue at StateOrProvinc, search for StateOrProvinc in the second DDL).
Would like to know, if there is any restriction on the number of fields to be
using in STRUCT datatype.
EXAMPLE:
BELOW IS THE DDL i run in hive and the second below DLL is the TABLE DDL i get
from SHOW CREATE TABLE.
CREATE TABLE property_feature.property_feature_new(
`_id` STRUCT<`mls_sysid`:String,`type`:String>,
all_attrs STRUCT<`1`: String ,`10`: String ,`100`: String ,`105`: String
,`113`: String ,`118`: String ,`126`: String ,`1299`: String ,`1301`: String
,`1305`: String ,`1307`: String ,`1319`: String ,`1325`: String ,`1327`: String
,`1329`: String ,`1339`: String ,`1341`: String ,`1345`: String ,`1351`: String
,`1353`: String ,`1359`: String ,`1363`: String ,`1365`: String ,`1367`: String
,`1373`: String ,`1377`: String ,`1397`: String ,`1401`: String ,`1403`: String
,`1405`: String ,`1409`: String ,`1411`: String ,`1413`: String ,`1417`: String
,`1419`: String ,`1421`: String ,`1423`: String ,`1431`: String ,`1437`: String
,`1439`: String ,`144`: String ,`1441`: String ,`145`: String ,`1451`: String
,`1453`: String ,`146`: String ,`1467`: String ,`1469`: String ,`1483`: String
,`1485`: String ,`1491`: String ,`1493`: String ,`15`: String ,`1505`: String
,`1519`: String ,`1521`: String ,`1523`: String ,`155`: String ,`156`: String
,`158`: String ,`1586`: String ,`1678`: String ,`169`: String ,`17`: String
,`177`: String ,`1783`: String ,`1785`: String ,`1787`: String ,`1789`: String
,`1793`: String ,`1795`: String ,`1797`: String ,`1799`: String ,`1801`: String
,`1803`: String ,`1805`: String ,`1807`: String ,`1809`: String ,`1811`: String
,`1815`: String ,`1818`: String ,`1826`: String ,`1829`: String ,`1874`: String
,`1876`: String ,`1878`: String ,`1880`: String ,`1882`: String ,`207`: String
,`215`: String ,`226`: String ,`2293`: String ,`232`: String ,`239`: String
,`241`: String ,`245`: String ,`2456`: String ,`248`: String ,`2482`: String
,`2490`: String ,`2491`: String ,`2492`: String ,`2493`: String ,`2495`: String
,`25`: String ,`2502`: String ,`2503`: String ,`2517`: String ,`2522`: String
,`255`: String ,`256`: String ,`2573`: String ,`2574`: String ,`260`: String
,`263`: String ,`265`: String ,`276`: String ,`278`: String ,`286`: String
,`293`: String ,`296`: String ,`299`: String ,`3`: String ,`30`: String ,`307`:
String ,`312`: String ,`320`: String ,`328`: String ,`340`: String ,`348`:
String ,`354`: String ,`363`: String ,`364`: String ,`369`: String ,`375`:
String ,`376`: String ,`383`: String ,`401`: String ,`405`: String ,`409`:
String ,`412`: String ,`42`: String ,`423`: String ,`444`: String ,`445`:
String ,`446`: String ,`447`: String ,`450`: String ,`451`: String ,`452`:
String ,`453`: String ,`455`: String ,`458`: String ,`461`: String ,`463`:
String ,`464`: String ,`51`: String ,`543472980`: String ,`57`: String ,`75`:
String ,`90`: String ,`92`: String ,`AVMYN`: String ,`AccessibilityAmenities`:
String ,`AddressPublic`: String ,`Amenities`: String ,`AmountofWaterFront`:
String ,`AppliancesIncluded`: String ,`AppraiserInitials`: String
,`AppraiserName`: String ,`Basement`: String ,`BathsFull`: String ,`BathsHalf`:
String ,`BathsTotal`: String ,`BedsTotal`: String ,`Block`: String ,`Book`:
String ,`BuildingNumber`: String ,`City`: String ,`CloseDate`: String
,`ClosePrice`: String ,
`CoListAgentDirectWorkPhone`: String ,`CoListAgentEmail`: String
,`CoListAgentFullName`: String ,`CoListAgentMLSID`: String ,`CoListAgent_MUI`:
String ,`CoListOfficeMLSID`: String ,`CoListOfficeName`: String
,`CoListOfficePhone`: String ,`CoListOffice_MUI`: String ,`Color`: String
,`CommentsYN`: String ,`Construction`: String ,`Cooling`: String
,`CountyOrParish`: String ,`DOM`: String ,`Deed`: String ,`Directions`: String
,`Driveway`: String ,`Electric`: String ,`Equipment`: String ,`Exterior`:
String ,`Fireplace`: String ,`FloorNumber`: String ,`Floors`: String
,`FoundationMaterials`: String ,`FullTaxAmount`: String ,`Gas`: String
,`HeatFuel`: String ,`HeatSystem`: String ,`Internet`: String
,`ListAgentDirectWorkPhone`: String ,`ListAgentEmail`: String
,`ListAgentFullName`: String ,`ListAgentMLSID`: String ,`ListAgent_MUI`: String
,`ListOfficeMLSID`: String ,`ListOfficeName`: String ,`ListOfficePhone`: String
,`ListOffice_MUI`: String ,`ListPrice`: String ,`ListingContractDate`: String
,`Location`: String ,`Lot`: String ,`LotSizeArea`: String ,`MLSNumber`: String
,`Make`: String ,`MapNumber`: String ,`MatrixModifiedDT`: String
,`MobileHomeIncludes`: String ,`Model`: String ,`MonthlyAssociationFee`: String
,`NeighborhoodAssoc`: String ,`NumberOfRooms`: String ,`OneBedroomUnits`:
String ,`OriginalListPrice`: String ,`Page`: String ,`Parking`: String
,`PhotoCount`: String ,`PhotoModificationTimestamp`: String ,`PostalCode`:
String ,`PropertySubType`: String ,`PropertyType`: String ,`PublicAddress`:
String ,`RecreationalWater`: String ,`Remarks`: String ,`Restrictions`: String
,`RoadFrontage`: String ,`Roads`: String ,`Roof`: String ,`SchoolDistrict`:
String ,`Seasonal`: String ,`SellingAgentDirectWorkPhone`: String
,`SellingAgentEmail`: String ,`SellingAgentFullName`: String
,`SellingAgentMLSID`: String ,`SellingAgent_MUI`: String ,`SellingOfficeMLSID`:
String ,`SellingOfficeName`: String ,`SellingOfficePhone`: String
,`SellingOffice_MUI`: String ,`SerialNumber`: String ,`Site`: String
,`SoldTerms`: String ,`SourceofSquareFootage`: String ,`SqFtFinishedTotal`:
String ,`StateOrProvince`: String ,`Status`: String ,`Style`: String
,`SupplementPublicCount`: String ,`Surveyed`: String ,`TaxYear`: String
,`ThreeBedroomUnits`: String ,`Transportation`: String ,`TwoBedroomUnits`:
String ,`UnitType`: String ,`UnitsInBuilding`: String ,`VehicleStorage`: String
,`View`: String ,`VirtualTour`: String ,`WasteWaterDisposal`: String ,`Water`:
String ,`WaterBodyName`: String ,`WaterBodyType`: String ,`WaterFrontage`:
String ,`WaterFrontageOwned`: String ,`WaterFrontageShared`: String
,`WaterHeater`: String ,`YearBuilt`: String ,`Zoning`: String
,`matrix_unique_id`: String ,`removed_created_at`: String ,`sysid`: String>
basement` string
) STORED AS parquet;
CREATE TABLE `property_feature`(
`_id` struct<mls_sysid:string,type:string>,
`all_attrs`
struct<1:string,10:string,100:string,105:string,113:string,118:string,126:string,1299:string,
1301:string,1305:string,1307:string,1319:string,1325:string,1327:string,1329:string,
1339:string,1341:string,1345:string,1351:string,1353:string,1359:string,1363:string,
1365:string,1367:string,1373:string,1377:string,1397:string,1401:string,1403:string,
1405:string,1409:string,1411:string,1413:string,1417:string,1419:string,1421:string,
1423:string,1431:string,1437:string,1439:string,144:string,1441:string,145:string,1451:string,
1453:string,146:string,1467:string,1469:string,1483:string,1485:string,1491:string,1493:string,
15:string,1505:string,1519:string,1521:string,1523:string,155:string,156:string,158:string,
1586:string,1678:string,169:string,17:string,177:string,1783:string,1785:string,1787:string,
1789:string,1793:string,1795:string,1797:string,1799:string,1801:string,1803:string,1805:string,
1807:string,1809:string,1811:string,1815:string,1818:string,1826:string,1829:string,1874:string,
1876:string,1878:string,1880:string,1882:string,207:string,215:string,226:string,2293:string,
232:string,239:string,241:string,245:string,2456:string,248:string,2482:string,2490:string,
2491:string,2492:string,2493:string,2495:string,25:string,2502:string,2503:string,2517:string,
2522:string,255:string,256:string,2573:string,2574:string,260:string,263:string,265:string,
276:string,278:string,286:string,293:string,296:string,299:string,3:string,30:string,307:string,
312:string,320:string,328:string,340:string,348:string,354:string,363:string,364:string,369:string,
375:string,376:string,383:string,401:string,405:string,409:string,412:string,42:string,423:string,444:string,445:string,446:string,447:string,450:string,451:string,452:string,453:string,455:string,458:string,461:string,463:string,464:string,51:string,543472980:string,57:string,75:string,90:string,92:string,AVMYN:string,AccessibilityAmenities:string,
AddressPublic:string,Amenities:string,AmountofWaterFront:string,AppliancesIncluded:string,AppraiserInitials:string,AppraiserName:string,Basement:string,BathsFull:string,BathsHalf:string,BathsTotal:string,BedsTotal:string,Block:string,Book:string,BuildingNumber:string,City:string,CloseDate:string,ClosePrice:string,CoListAgentDirectWorkPhone:string,CoListAgentEmail:string,CoListAgentFullName:string,CoListAgentMLSID:string,CoListAgent_MUI:string,CoListOfficeMLSID:string,CoListOfficeName:string,CoListOfficePhone:string,
CoListOffice_MUI:string,Color:string,CommentsYN:string,Construction:string,Cooling:string,CountyOrParish:string,DOM:string,Deed:string,Directions:string,Driveway:string,Electric:string,Equipment:string,Exterior:string,Fireplace:string,FloorNumber:string,Floors:string,FoundationMaterials:string,FullTaxAmount:string,Gas:string,HeatFuel:string,HeatSystem:string,Internet:string,ListAgentDirectWorkPhone:string,ListAgentEmail:string,ListAgentFullName:string,ListAgentMLSID:string,ListAgent_MUI:string,ListOfficeMLSID:string,ListOfficeName:string,ListOfficePhone:string,ListOffice_MUI:string,ListPrice:string,ListingContractDate:string,Location:string,Lot:string,LotSizeArea:string,MLSNumber:string,Make:string,MapNumber:string,MatrixModifiedDT:string,MobileHomeIncludes:string,Model:string,MonthlyAssociationFee:string,NeighborhoodAssoc:string,NumberOfRooms:string,OneBedroomUnits:string,OriginalListPrice:string,Page:string,Parking:string,PhotoCount:string,PhotoModificationTimestamp:string,PostalCode:string,PropertySubType:string,PropertyType:string,PublicAddress:string,RecreationalWater:string,Remarks:string,Restrictions:string,RoadFrontage:string,Roads:string,Roof:string,SchoolDistrict:string,Seasonal:string,SellingAgentDirectWorkPhone:string,SellingAgentEmail:string,SellingAgentFullName:string,SellingAgentMLSID:string,SellingAgent_MUI:string,SellingOfficeMLSID:string,SellingOfficeName:string,SellingOfficePhone:string,SellingOffice_MUI:string,SerialNumber:string,Site:string,SoldTerms:string,
SourceofSquareFootage:string,SqFtFinishedTotal:string,StateOrProvinc,
`basement` string
) STORED AS parquet;
Please help out in case if there is any workaround for it or any other
restriction details.
was:
I am trying to create a table with multiple STRUCT column types, one of which
having more than 300 fields.
I get the table created with no problem, but when i load i face few errors in
loading up the data.when i check SHOW CREATE TABLE TABLENAME, i get fields only
till 266th column and rest of them do not appear in the table DDL.
Would like to know, if there is any restriction on the number of fields to be
using in STRUCT datatype.
EXAMPLE:
BELOW IS THE DDL i run in hive and the below in red is the TABLE DDL i get from
SHOW CREATE TABLE.
CREATE TABLE property_feature.property_feature_new(
`_id` STRUCT<`mls_sysid`:String,`type`:String>,
all_attrs STRUCT<`1`: String ,`10`: String ,`100`: String ,`105`: String
,`113`: String ,`118`: String ,`126`: String ,`1299`: String ,`1301`: String
,`1305`: String ,`1307`: String ,`1319`: String ,`1325`: String ,`1327`: String
,`1329`: String ,`1339`: String ,`1341`: String ,`1345`: String ,`1351`: String
,`1353`: String ,`1359`: String ,`1363`: String ,`1365`: String ,`1367`: String
,`1373`: String ,`1377`: String ,`1397`: String ,`1401`: String ,`1403`: String
,`1405`: String ,`1409`: String ,`1411`: String ,`1413`: String ,`1417`: String
,`1419`: String ,`1421`: String ,`1423`: String ,`1431`: String ,`1437`: String
,`1439`: String ,`144`: String ,`1441`: String ,`145`: String ,`1451`: String
,`1453`: String ,`146`: String ,`1467`: String ,`1469`: String ,`1483`: String
,`1485`: String ,`1491`: String ,`1493`: String ,`15`: String ,`1505`: String
,`1519`: String ,`1521`: String ,`1523`: String ,`155`: String ,`156`: String
,`158`: String ,`1586`: String ,`1678`: String ,`169`: String ,`17`: String
,`177`: String ,`1783`: String ,`1785`: String ,`1787`: String ,`1789`: String
,`1793`: String ,`1795`: String ,`1797`: String ,`1799`: String ,`1801`: String
,`1803`: String ,`1805`: String ,`1807`: String ,`1809`: String ,`1811`: String
,`1815`: String ,`1818`: String ,`1826`: String ,`1829`: String ,`1874`: String
,`1876`: String ,`1878`: String ,`1880`: String ,`1882`: String ,`207`: String
,`215`: String ,`226`: String ,`2293`: String ,`232`: String ,`239`: String
,`241`: String ,`245`: String ,`2456`: String ,`248`: String ,`2482`: String
,`2490`: String ,`2491`: String ,`2492`: String ,`2493`: String ,`2495`: String
,`25`: String ,`2502`: String ,`2503`: String ,`2517`: String ,`2522`: String
,`255`: String ,`256`: String ,`2573`: String ,`2574`: String ,`260`: String
,`263`: String ,`265`: String ,`276`: String ,`278`: String ,`286`: String
,`293`: String ,`296`: String ,`299`: String ,`3`: String ,`30`: String ,`307`:
String ,`312`: String ,`320`: String ,`328`: String ,`340`: String ,`348`:
String ,`354`: String ,`363`: String ,`364`: String ,`369`: String ,`375`:
String ,`376`: String ,`383`: String ,`401`: String ,`405`: String ,`409`:
String ,`412`: String ,`42`: String ,`423`: String ,`444`: String ,`445`:
String ,`446`: String ,`447`: String ,`450`: String ,`451`: String ,`452`:
String ,`453`: String ,`455`: String ,`458`: String ,`461`: String ,`463`:
String ,`464`: String ,`51`: String ,`543472980`: String ,`57`: String ,`75`:
String ,`90`: String ,`92`: String ,`AVMYN`: String ,`AccessibilityAmenities`:
String ,`AddressPublic`: String ,`Amenities`: String ,`AmountofWaterFront`:
String ,`AppliancesIncluded`: String ,`AppraiserInitials`: String
,`AppraiserName`: String ,`Basement`: String ,`BathsFull`: String ,`BathsHalf`:
String ,`BathsTotal`: String ,`BedsTotal`: String ,`Block`: String ,`Book`:
String ,`BuildingNumber`: String ,`City`: String ,`CloseDate`: String
,`ClosePrice`: String ,
`CoListAgentDirectWorkPhone`: String ,`CoListAgentEmail`: String
,`CoListAgentFullName`: String ,`CoListAgentMLSID`: String ,`CoListAgent_MUI`:
String ,`CoListOfficeMLSID`: String ,`CoListOfficeName`: String
,`CoListOfficePhone`: String ,`CoListOffice_MUI`: String ,`Color`: String
,`CommentsYN`: String ,`Construction`: String ,`Cooling`: String
,`CountyOrParish`: String ,`DOM`: String ,`Deed`: String ,`Directions`: String
,`Driveway`: String ,`Electric`: String ,`Equipment`: String ,`Exterior`:
String ,`Fireplace`: String ,`FloorNumber`: String ,`Floors`: String
,`FoundationMaterials`: String ,`FullTaxAmount`: String ,`Gas`: String
,`HeatFuel`: String ,`HeatSystem`: String ,`Internet`: String
,`ListAgentDirectWorkPhone`: String ,`ListAgentEmail`: String
,`ListAgentFullName`: String ,`ListAgentMLSID`: String ,`ListAgent_MUI`: String
,`ListOfficeMLSID`: String ,`ListOfficeName`: String ,`ListOfficePhone`: String
,`ListOffice_MUI`: String ,`ListPrice`: String ,`ListingContractDate`: String
,`Location`: String ,`Lot`: String ,`LotSizeArea`: String ,`MLSNumber`: String
,`Make`: String ,`MapNumber`: String ,`MatrixModifiedDT`: String
,`MobileHomeIncludes`: String ,`Model`: String ,`MonthlyAssociationFee`: String
,`NeighborhoodAssoc`: String ,`NumberOfRooms`: String ,`OneBedroomUnits`:
String ,`OriginalListPrice`: String ,`Page`: String ,`Parking`: String
,`PhotoCount`: String ,`PhotoModificationTimestamp`: String ,`PostalCode`:
String ,`PropertySubType`: String ,`PropertyType`: String ,`PublicAddress`:
String ,`RecreationalWater`: String ,`Remarks`: String ,`Restrictions`: String
,`RoadFrontage`: String ,`Roads`: String ,`Roof`: String ,`SchoolDistrict`:
String ,`Seasonal`: String ,`SellingAgentDirectWorkPhone`: String
,`SellingAgentEmail`: String ,`SellingAgentFullName`: String
,`SellingAgentMLSID`: String ,`SellingAgent_MUI`: String ,`SellingOfficeMLSID`:
String ,`SellingOfficeName`: String ,`SellingOfficePhone`: String
,`SellingOffice_MUI`: String ,`SerialNumber`: String ,`Site`: String
,`SoldTerms`: String ,`SourceofSquareFootage`: String ,`SqFtFinishedTotal`:
String ,`StateOrProvince`: String ,`Status`: String ,`Style`: String
,`SupplementPublicCount`: String ,`Surveyed`: String ,`TaxYear`: String
,`ThreeBedroomUnits`: String ,`Transportation`: String ,`TwoBedroomUnits`:
String ,`UnitType`: String ,`UnitsInBuilding`: String ,`VehicleStorage`: String
,`View`: String ,`VirtualTour`: String ,`WasteWaterDisposal`: String ,`Water`:
String ,`WaterBodyName`: String ,`WaterBodyType`: String ,`WaterFrontage`:
String ,`WaterFrontageOwned`: String ,`WaterFrontageShared`: String
,`WaterHeater`: String ,`YearBuilt`: String ,`Zoning`: String
,`matrix_unique_id`: String ,`removed_created_at`: String ,`sysid`: String>
basement` string
) STORED AS parquet;
CREATE TABLE `property_feature`(
`_id` struct<mls_sysid:string,type:string>,
`all_attrs`
struct<1:string,10:string,100:string,105:string,113:string,118:string,126:string,1299:string,
1301:string,1305:string,1307:string,1319:string,1325:string,1327:string,1329:string,
1339:string,1341:string,1345:string,1351:string,1353:string,1359:string,1363:string,
1365:string,1367:string,1373:string,1377:string,1397:string,1401:string,1403:string,
1405:string,1409:string,1411:string,1413:string,1417:string,1419:string,1421:string,
1423:string,1431:string,1437:string,1439:string,144:string,1441:string,145:string,1451:string,
1453:string,146:string,1467:string,1469:string,1483:string,1485:string,1491:string,1493:string,
15:string,1505:string,1519:string,1521:string,1523:string,155:string,156:string,158:string,
1586:string,1678:string,169:string,17:string,177:string,1783:string,1785:string,1787:string,
1789:string,1793:string,1795:string,1797:string,1799:string,1801:string,1803:string,1805:string,
1807:string,1809:string,1811:string,1815:string,1818:string,1826:string,1829:string,1874:string,
1876:string,1878:string,1880:string,1882:string,207:string,215:string,226:string,2293:string,
232:string,239:string,241:string,245:string,2456:string,248:string,2482:string,2490:string,
2491:string,2492:string,2493:string,2495:string,25:string,2502:string,2503:string,2517:string,
2522:string,255:string,256:string,2573:string,2574:string,260:string,263:string,265:string,
276:string,278:string,286:string,293:string,296:string,299:string,3:string,30:string,307:string,
312:string,320:string,328:string,340:string,348:string,354:string,363:string,364:string,369:string,
375:string,376:string,383:string,401:string,405:string,409:string,412:string,42:string,423:string,444:string,445:string,446:string,447:string,450:string,451:string,452:string,453:string,455:string,458:string,461:string,463:string,464:string,51:string,543472980:string,57:string,75:string,90:string,92:string,AVMYN:string,AccessibilityAmenities:string,
AddressPublic:string,Amenities:string,AmountofWaterFront:string,AppliancesIncluded:string,AppraiserInitials:string,AppraiserName:string,Basement:string,BathsFull:string,BathsHalf:string,BathsTotal:string,BedsTotal:string,Block:string,Book:string,BuildingNumber:string,City:string,CloseDate:string,ClosePrice:string,CoListAgentDirectWorkPhone:string,CoListAgentEmail:string,CoListAgentFullName:string,CoListAgentMLSID:string,CoListAgent_MUI:string,CoListOfficeMLSID:string,CoListOfficeName:string,CoListOfficePhone:string,
CoListOffice_MUI:string,Color:string,CommentsYN:string,Construction:string,Cooling:string,CountyOrParish:string,DOM:string,Deed:string,Directions:string,Driveway:string,Electric:string,Equipment:string,Exterior:string,Fireplace:string,FloorNumber:string,Floors:string,FoundationMaterials:string,FullTaxAmount:string,Gas:string,HeatFuel:string,HeatSystem:string,Internet:string,ListAgentDirectWorkPhone:string,ListAgentEmail:string,ListAgentFullName:string,ListAgentMLSID:string,ListAgent_MUI:string,ListOfficeMLSID:string,ListOfficeName:string,ListOfficePhone:string,ListOffice_MUI:string,ListPrice:string,ListingContractDate:string,Location:string,Lot:string,LotSizeArea:string,MLSNumber:string,Make:string,MapNumber:string,MatrixModifiedDT:string,MobileHomeIncludes:string,Model:string,MonthlyAssociationFee:string,NeighborhoodAssoc:string,NumberOfRooms:string,OneBedroomUnits:string,OriginalListPrice:string,Page:string,Parking:string,PhotoCount:string,PhotoModificationTimestamp:string,PostalCode:string,PropertySubType:string,PropertyType:string,PublicAddress:string,RecreationalWater:string,Remarks:string,Restrictions:string,RoadFrontage:string,Roads:string,Roof:string,SchoolDistrict:string,Seasonal:string,SellingAgentDirectWorkPhone:string,SellingAgentEmail:string,SellingAgentFullName:string,SellingAgentMLSID:string,SellingAgent_MUI:string,SellingOfficeMLSID:string,SellingOfficeName:string,SellingOfficePhone:string,SellingOffice_MUI:string,SerialNumber:string,Site:string,SoldTerms:string,
SourceofSquareFootage:string,SqFtFinishedTotal:string,StateOrProvinc,
`basement` string
) STORED AS parquet;
Please help out in case if there is any workaround for it or any other
restriction details.
> STRUCT type not allowing more than 266 fields. Table get created with more
> fields but throws error while querying against the table. SHOW CREATE TABLE
> skips having some fields in STRUCT.
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-13203
> URL: https://issues.apache.org/jira/browse/HIVE-13203
> Project: Hive
> Issue Type: Bug
> Components: Database/Schema
> Affects Versions: 1.2.0
> Reporter: Satyajit varma
>
> I am trying to create a table with multiple STRUCT column types, one of which
> having more than 300 fields.
> I get the table created with no problem, but when i load i face few errors in
> loading up the data.when i check SHOW CREATE TABLE TABLENAME, i get fields
> only till 266th column and rest of them do not appear in the table DDL.(check
> the issue at StateOrProvinc, search for StateOrProvinc in the second DDL).
> Would like to know, if there is any restriction on the number of fields to be
> using in STRUCT datatype.
> EXAMPLE:
> BELOW IS THE DDL i run in hive and the second below DLL is the TABLE DDL i
> get from SHOW CREATE TABLE.
> CREATE TABLE property_feature.property_feature_new(
> `_id` STRUCT<`mls_sysid`:String,`type`:String>,
> all_attrs STRUCT<`1`: String ,`10`: String ,`100`: String ,`105`: String
> ,`113`: String ,`118`: String ,`126`: String ,`1299`: String ,`1301`: String
> ,`1305`: String ,`1307`: String ,`1319`: String ,`1325`: String ,`1327`:
> String ,`1329`: String ,`1339`: String ,`1341`: String ,`1345`: String
> ,`1351`: String ,`1353`: String ,`1359`: String ,`1363`: String ,`1365`:
> String ,`1367`: String ,`1373`: String ,`1377`: String ,`1397`: String
> ,`1401`: String ,`1403`: String ,`1405`: String ,`1409`: String ,`1411`:
> String ,`1413`: String ,`1417`: String ,`1419`: String ,`1421`: String
> ,`1423`: String ,`1431`: String ,`1437`: String ,`1439`: String ,`144`:
> String ,`1441`: String ,`145`: String ,`1451`: String ,`1453`: String ,`146`:
> String ,`1467`: String ,`1469`: String ,`1483`: String ,`1485`: String
> ,`1491`: String ,`1493`: String ,`15`: String ,`1505`: String ,`1519`: String
> ,`1521`: String ,`1523`: String ,`155`: String ,`156`: String ,`158`: String
> ,`1586`: String ,`1678`: String ,`169`: String ,`17`: String ,`177`: String
> ,`1783`: String ,`1785`: String ,`1787`: String ,`1789`: String ,`1793`:
> String ,`1795`: String ,`1797`: String ,`1799`: String ,`1801`: String
> ,`1803`: String ,`1805`: String ,`1807`: String ,`1809`: String ,`1811`:
> String ,`1815`: String ,`1818`: String ,`1826`: String ,`1829`: String
> ,`1874`: String ,`1876`: String ,`1878`: String ,`1880`: String ,`1882`:
> String ,`207`: String ,`215`: String ,`226`: String ,`2293`: String ,`232`:
> String ,`239`: String ,`241`: String ,`245`: String ,`2456`: String ,`248`:
> String ,`2482`: String ,`2490`: String ,`2491`: String ,`2492`: String
> ,`2493`: String ,`2495`: String ,`25`: String ,`2502`: String ,`2503`: String
> ,`2517`: String ,`2522`: String ,`255`: String ,`256`: String ,`2573`: String
> ,`2574`: String ,`260`: String ,`263`: String ,`265`: String ,`276`: String
> ,`278`: String ,`286`: String ,`293`: String ,`296`: String ,`299`: String
> ,`3`: String ,`30`: String ,`307`: String ,`312`: String ,`320`: String
> ,`328`: String ,`340`: String ,`348`: String ,`354`: String ,`363`: String
> ,`364`: String ,`369`: String ,`375`: String ,`376`: String ,`383`: String
> ,`401`: String ,`405`: String ,`409`: String ,`412`: String ,`42`: String
> ,`423`: String ,`444`: String ,`445`: String ,`446`: String ,`447`: String
> ,`450`: String ,`451`: String ,`452`: String ,`453`: String ,`455`: String
> ,`458`: String ,`461`: String ,`463`: String ,`464`: String ,`51`: String
> ,`543472980`: String ,`57`: String ,`75`: String ,`90`: String ,`92`: String
> ,`AVMYN`: String ,`AccessibilityAmenities`: String ,`AddressPublic`: String
> ,`Amenities`: String ,`AmountofWaterFront`: String ,`AppliancesIncluded`:
> String ,`AppraiserInitials`: String ,`AppraiserName`: String ,`Basement`:
> String ,`BathsFull`: String ,`BathsHalf`: String ,`BathsTotal`: String
> ,`BedsTotal`: String ,`Block`: String ,`Book`: String ,`BuildingNumber`:
> String ,`City`: String ,`CloseDate`: String ,`ClosePrice`: String ,
> `CoListAgentDirectWorkPhone`: String ,`CoListAgentEmail`: String
> ,`CoListAgentFullName`: String ,`CoListAgentMLSID`: String
> ,`CoListAgent_MUI`: String ,`CoListOfficeMLSID`: String ,`CoListOfficeName`:
> String ,`CoListOfficePhone`: String ,`CoListOffice_MUI`: String ,`Color`:
> String ,`CommentsYN`: String ,`Construction`: String ,`Cooling`: String
> ,`CountyOrParish`: String ,`DOM`: String ,`Deed`: String ,`Directions`:
> String ,`Driveway`: String ,`Electric`: String ,`Equipment`: String
> ,`Exterior`: String ,`Fireplace`: String ,`FloorNumber`: String ,`Floors`:
> String ,`FoundationMaterials`: String ,`FullTaxAmount`: String ,`Gas`: String
> ,`HeatFuel`: String ,`HeatSystem`: String ,`Internet`: String
> ,`ListAgentDirectWorkPhone`: String ,`ListAgentEmail`: String
> ,`ListAgentFullName`: String ,`ListAgentMLSID`: String ,`ListAgent_MUI`:
> String ,`ListOfficeMLSID`: String ,`ListOfficeName`: String
> ,`ListOfficePhone`: String ,`ListOffice_MUI`: String ,`ListPrice`: String
> ,`ListingContractDate`: String ,`Location`: String ,`Lot`: String
> ,`LotSizeArea`: String ,`MLSNumber`: String ,`Make`: String ,`MapNumber`:
> String ,`MatrixModifiedDT`: String ,`MobileHomeIncludes`: String ,`Model`:
> String ,`MonthlyAssociationFee`: String ,`NeighborhoodAssoc`: String
> ,`NumberOfRooms`: String ,`OneBedroomUnits`: String ,`OriginalListPrice`:
> String ,`Page`: String ,`Parking`: String ,`PhotoCount`: String
> ,`PhotoModificationTimestamp`: String ,`PostalCode`: String
> ,`PropertySubType`: String ,`PropertyType`: String ,`PublicAddress`: String
> ,`RecreationalWater`: String ,`Remarks`: String ,`Restrictions`: String
> ,`RoadFrontage`: String ,`Roads`: String ,`Roof`: String ,`SchoolDistrict`:
> String ,`Seasonal`: String ,`SellingAgentDirectWorkPhone`: String
> ,`SellingAgentEmail`: String ,`SellingAgentFullName`: String
> ,`SellingAgentMLSID`: String ,`SellingAgent_MUI`: String
> ,`SellingOfficeMLSID`: String ,`SellingOfficeName`: String
> ,`SellingOfficePhone`: String ,`SellingOffice_MUI`: String ,`SerialNumber`:
> String ,`Site`: String ,`SoldTerms`: String ,`SourceofSquareFootage`: String
> ,`SqFtFinishedTotal`: String ,`StateOrProvince`: String ,`Status`: String
> ,`Style`: String ,`SupplementPublicCount`: String ,`Surveyed`: String
> ,`TaxYear`: String ,`ThreeBedroomUnits`: String ,`Transportation`: String
> ,`TwoBedroomUnits`: String ,`UnitType`: String ,`UnitsInBuilding`: String
> ,`VehicleStorage`: String ,`View`: String ,`VirtualTour`: String
> ,`WasteWaterDisposal`: String ,`Water`: String ,`WaterBodyName`: String
> ,`WaterBodyType`: String ,`WaterFrontage`: String ,`WaterFrontageOwned`:
> String ,`WaterFrontageShared`: String ,`WaterHeater`: String ,`YearBuilt`:
> String ,`Zoning`: String ,`matrix_unique_id`: String ,`removed_created_at`:
> String ,`sysid`: String>
> basement` string
> ) STORED AS parquet;
> CREATE TABLE `property_feature`(
> `_id` struct<mls_sysid:string,type:string>,
> `all_attrs`
> struct<1:string,10:string,100:string,105:string,113:string,118:string,126:string,1299:string,
> 1301:string,1305:string,1307:string,1319:string,1325:string,1327:string,1329:string,
> 1339:string,1341:string,1345:string,1351:string,1353:string,1359:string,1363:string,
> 1365:string,1367:string,1373:string,1377:string,1397:string,1401:string,1403:string,
> 1405:string,1409:string,1411:string,1413:string,1417:string,1419:string,1421:string,
> 1423:string,1431:string,1437:string,1439:string,144:string,1441:string,145:string,1451:string,
> 1453:string,146:string,1467:string,1469:string,1483:string,1485:string,1491:string,1493:string,
> 15:string,1505:string,1519:string,1521:string,1523:string,155:string,156:string,158:string,
> 1586:string,1678:string,169:string,17:string,177:string,1783:string,1785:string,1787:string,
> 1789:string,1793:string,1795:string,1797:string,1799:string,1801:string,1803:string,1805:string,
> 1807:string,1809:string,1811:string,1815:string,1818:string,1826:string,1829:string,1874:string,
> 1876:string,1878:string,1880:string,1882:string,207:string,215:string,226:string,2293:string,
> 232:string,239:string,241:string,245:string,2456:string,248:string,2482:string,2490:string,
> 2491:string,2492:string,2493:string,2495:string,25:string,2502:string,2503:string,2517:string,
> 2522:string,255:string,256:string,2573:string,2574:string,260:string,263:string,265:string,
> 276:string,278:string,286:string,293:string,296:string,299:string,3:string,30:string,307:string,
> 312:string,320:string,328:string,340:string,348:string,354:string,363:string,364:string,369:string,
> 375:string,376:string,383:string,401:string,405:string,409:string,412:string,42:string,423:string,444:string,445:string,446:string,447:string,450:string,451:string,452:string,453:string,455:string,458:string,461:string,463:string,464:string,51:string,543472980:string,57:string,75:string,90:string,92:string,AVMYN:string,AccessibilityAmenities:string,
> AddressPublic:string,Amenities:string,AmountofWaterFront:string,AppliancesIncluded:string,AppraiserInitials:string,AppraiserName:string,Basement:string,BathsFull:string,BathsHalf:string,BathsTotal:string,BedsTotal:string,Block:string,Book:string,BuildingNumber:string,City:string,CloseDate:string,ClosePrice:string,CoListAgentDirectWorkPhone:string,CoListAgentEmail:string,CoListAgentFullName:string,CoListAgentMLSID:string,CoListAgent_MUI:string,CoListOfficeMLSID:string,CoListOfficeName:string,CoListOfficePhone:string,
> CoListOffice_MUI:string,Color:string,CommentsYN:string,Construction:string,Cooling:string,CountyOrParish:string,DOM:string,Deed:string,Directions:string,Driveway:string,Electric:string,Equipment:string,Exterior:string,Fireplace:string,FloorNumber:string,Floors:string,FoundationMaterials:string,FullTaxAmount:string,Gas:string,HeatFuel:string,HeatSystem:string,Internet:string,ListAgentDirectWorkPhone:string,ListAgentEmail:string,ListAgentFullName:string,ListAgentMLSID:string,ListAgent_MUI:string,ListOfficeMLSID:string,ListOfficeName:string,ListOfficePhone:string,ListOffice_MUI:string,ListPrice:string,ListingContractDate:string,Location:string,Lot:string,LotSizeArea:string,MLSNumber:string,Make:string,MapNumber:string,MatrixModifiedDT:string,MobileHomeIncludes:string,Model:string,MonthlyAssociationFee:string,NeighborhoodAssoc:string,NumberOfRooms:string,OneBedroomUnits:string,OriginalListPrice:string,Page:string,Parking:string,PhotoCount:string,PhotoModificationTimestamp:string,PostalCode:string,PropertySubType:string,PropertyType:string,PublicAddress:string,RecreationalWater:string,Remarks:string,Restrictions:string,RoadFrontage:string,Roads:string,Roof:string,SchoolDistrict:string,Seasonal:string,SellingAgentDirectWorkPhone:string,SellingAgentEmail:string,SellingAgentFullName:string,SellingAgentMLSID:string,SellingAgent_MUI:string,SellingOfficeMLSID:string,SellingOfficeName:string,SellingOfficePhone:string,SellingOffice_MUI:string,SerialNumber:string,Site:string,SoldTerms:string,
> SourceofSquareFootage:string,SqFtFinishedTotal:string,StateOrProvinc,
> `basement` string
> ) STORED AS parquet;
> Please help out in case if there is any workaround for it or any other
> restriction details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
