[jira] [Resolved] (CARBONDATA-4170) Support dropping of parent complex columns(array/struct/map)
[ https://issues.apache.org/jira/browse/CARBONDATA-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthu Murugesh resolved CARBONDATA-4170. --- Fix Version/s: 2.2.0 Resolution: Fixed > Support dropping of parent complex columns(array/struct/map) > > > Key: CARBONDATA-4170 > URL: https://issues.apache.org/jira/browse/CARBONDATA-4170 > Project: CarbonData > Issue Type: Sub-task > Components: spark-integration >Reporter: Akshay >Priority: Major > Fix For: 2.2.0 > > Time Spent: 10h 40m > Remaining Estimate: 0h > > Drop complex columns(array/struct/map) from carbon table. For example - > arr1 array, struct1 struct, map1 map > Command - > ALTER TABLE DROP COLUMNS(arr1, struct1, map1) > Design document - > [https://docs.google.com/document/d/1DhhkVXM8rMvOuKDZeccJpFEfO3VkA9C0c7JHCV88NXI/edit] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CARBONDATA-4167) Case sensitive issues in Geospatial index support
[
https://issues.apache.org/jira/browse/CARBONDATA-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Indhumathi Muthu Murugesh resolved CARBONDATA-4167.
---
Fix Version/s: 2.2.0
Resolution: Fixed
> Case sensitive issues in Geospatial index support
> -
>
> Key: CARBONDATA-4167
> URL: https://issues.apache.org/jira/browse/CARBONDATA-4167
> Project: CarbonData
> Issue Type: Bug
>Reporter: SHREELEKHYA GAMPA
>Priority: Minor
> Fix For: 2.2.0
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> 1) create table source_index(TIMEVALUE BIGINT,LONGITUDE long,LATITUDE long)
> STORED AS carbondata TBLPROPERTIES
> ('SPATIAL_INDEX.MYGEOHASH.type'='geohash','SPATIAL_INDEX.MYGEOHASH.sourcecolumns'='longitude,
>
> latitude','SPATIAL_INDEX.MYGEOHASH.originLatitude'='39.930753','SPATIAL_INDEX.MYGEOHASH.gridSize'='50','SPATIAL_INDEX'='MYGEOHASH','SPATIAL_INDEX.MYGEOHASH.conversionRatio'='100');
> properties are being case sensitive:
> [!https://clouddevops.huawei.com/vision-file-storage/api/file/download/upload-v2/2021/3/5/c71035/9d1b20f836a048909679864f0c0fb4d8/image.png!|https://clouddevops.huawei.com/vision-file-storage/api/file/download/upload-v2/2021/3/5/c71035/9d1b20f836a048909679864f0c0fb4d8/image.png]
>
> 2) select query with lower case in Query UDFs fails
> [!https://clouddevops.huawei.com/vision-file-storage/api/file/download/upload-v2/2021/3/9/c71035/d28803c9cbc54db997b9d28015d59bee/image.png!|https://clouddevops.huawei.com/vision-file-storage/api/file/download/upload-v2/2021/3/9/c71035/d28803c9cbc54db997b9d28015d59bee/image.png]
>
> select query with with lower case linestring in Polyline UDf does not return
> any value.
> [!https://clouddevops.huawei.com/vision-file-storage/api/file/download/upload-v2/2021/3/9/c71035/0b1fe6bdd8454079bd52047f2c72bda4/image.png!|https://clouddevops.huawei.com/vision-file-storage/api/file/download/upload-v2/2021/3/9/c71035/0b1fe6bdd8454079bd52047f2c72bda4/image.png]
>
> Select query with lower case rangelist in IN_POLYGON_RANGE_LIST UDF returns
> no value.
> [!https://clouddevops.huawei.com/vision-file-storage/api/file/download/upload-v2/2021/3/9/c71035/d4d19f80d4d740b99bc5a0b6859f/image.png!|https://clouddevops.huawei.com/vision-file-storage/api/file/download/upload-v2/2021/3/9/c71035/d4d19f80d4d740b99bc5a0b6859f/image.png]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CARBONDATA-4168) UDF validation Issues related to Geospatial support
[
https://issues.apache.org/jira/browse/CARBONDATA-4168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Indhumathi Muthu Murugesh resolved CARBONDATA-4168.
---
Fix Version/s: 2.2.0
Resolution: Fixed
> UDF validation Issues related to Geospatial support
>
>
> Key: CARBONDATA-4168
> URL: https://issues.apache.org/jira/browse/CARBONDATA-4168
> Project: CarbonData
> Issue Type: Bug
>Reporter: SHREELEKHYA GAMPA
>Priority: Minor
> Fix For: 2.2.0
>
>
> --Gives a wrong error message for size less than equal to 0.
> drop table if exists source_index;
> create table source_index(TIMEVALUE BIGINT,LONGITUDE long,LATITUDE long)
> STORED AS carbondata TBLPROPERTIES
> ('SPATIAL_INDEX'='mygeohash','SPATIAL_INDEX.mygeohash.type'='geohash','SPATIAL_INDEX.mygeohash.sourcecolumns'='longitude,
>
> latitude','SPATIAL_INDEX.mygeohash.originLatitude'='39.832277','SPATIAL_INDEX.mygeohash.gridSize'='50','SPATIAL_INDEX.mygeohash.conversionRatio'='100');
> LOAD DATA inpath '/geodata/geodata2.csv' INTO TABLE source_index OPTIONS
> ('DELIMITER'= ',');
> select longitude, latitude from source_index where
> IN_POLYLINE_LIST('LINESTRING (120.184179 30.327465, 120.191603 30.328946,
> 120.199242 30.324464, 120.190359 30.315388)', -65);
> select longitude, latitude from source_index where
> IN_POLYLINE_LIST('LINESTRING (120.184179 30.327465, 120.191603 30.328946,
> 120.199242 30.324464, 120.190359 30.315388)', 0);
> Scenario 2:
> --Accepts Invalid Buffer Size
> select longitude, latitude from source_index where
> IN_POLYLINE_LIST('LINESTRING (120.184179 30.327465, 120.191603 30.328946,
> 120.199242 30.324464), LINESTRING (120.199242 30.324464, 120.190359
> 30.315388)', 'X');
> Scenario 3:
> --Accepts negative and 0 gridSize
> select LatLngToGeoId(39930753, 116302895, 39.832277, -50) as geoId;
> select GeoIdToLatLng(855279270226, 39.832277, -50) as LatitudeAndLongitude;
> select ToRangeList('116.321011 40.123503, 116.320311 40.122503,116.32
> 40.121503, 116.321011 40.123503', 39.832277, -50) as rangeList;
> select LatLngToGeoId(39930753, 116302895, 39.832277, 0) as geoId;
> select GeoIdToLatLng(855279270226, 39.832277, 0) as LatitudeAndLongitude;
> --Gives Wrong error message fro gridSize 0
> select ToRangeList('116.321011 40.123503, 116.320311 40.122503,116.32
> 40.121503, 116.321011 40.123503', 39.832277, 0) as rangeList;
> Scenario 4:
> --Accepting Double values for GeoId
> select GeoIdToLatLng(8.55279270226, 39.832277, -50) as LatitudeAndLongitude;
> select ToUpperLayerGeoId(8.55279270226) as upperLayerGeoId;
> select GeoIdToGridXy(8.55279270226) as GridXY;
> Scanerio 5:
> --Accepting Invalid Values in All UDFs
> select GeoIdToGridXy('X') as GridXY;
> select LatLngToGeoId('X', 'X', 'X', 'X') as geoId;
> select GeoIdToLatLng('X', 'X', 'X') as LatitudeAndLongitude;
> select ToUpperLayerGeoId('X') as upperLayerGeoId;
> select ToRangeList('116.321011 40.123503, 116.320311 40.122503,116.32
> 40.121503, 116.321011 40.123503', 39.832277, 'X') as rangeList;
> select ToRangeList('116.321011 40.123503, 116.320311 40.122503,116.32
> 40.121503, 116.321011 40.123503', 'X', 50) as rangeList;
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CARBONDATA-4169) Secondary Index coarse grain datamap to work for existing Secondary Index tables
[ https://issues.apache.org/jira/browse/CARBONDATA-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venugopal Reddy K updated CARBONDATA-4169: -- Attachment: Leverage Secondary Indexes with Presto Queries.pdf > Secondary Index coarse grain datamap to work for existing Secondary Index > tables > > > Key: CARBONDATA-4169 > URL: https://issues.apache.org/jira/browse/CARBONDATA-4169 > Project: CarbonData > Issue Type: Sub-task > Components: core, spark-integration >Reporter: Venugopal Reddy K >Priority: Minor > Attachments: Leverage Secondary Indexes with Presto Queries.pdf > > > With [[https://github.com/apache/carbondata/pull/4110]] user has to delete > all segments from the older version of existing secondary index table and > reload/sync all segments again to use them in the seconday index coarse grain > datamap. This restriction is due to the fact that PR expects > CarbonCommonConstants.INDEX_STATUS property for the index to be set to > IndexStatus.ENABLED in the index metadata when main table and respective > secondary index table segments are in sync. And IndexChooser picks only > IndexStatus.ENABLED indexes in the datamap pruning. It applies to bloom, > lucene as well. > But this property was not set for the secondary indexes in the older versions > as they were not part of datamap pruning. > > To support the secondary index coarse grain datamap pruning for existing > secondary indexes, need to support a way it happens automatically. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-4166) GeoSpatial Query Enhancements
[ https://issues.apache.org/jira/browse/CARBONDATA-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated CARBONDATA-4166: - Description: Design document link: [https://docs.google.com/document/d/1YmTbHa0P39P8iURb_IZErIdZzx7jFgZ5fw6E2BdHjKQ/edit?usp=sharing] [^Geo-spatial Enhancements_v1.pdf] ^Version 2:^ [https://docs.google.com/document/d/19AQj90Rll9iXBVcCMpcuKanwgDE6wuiGMYYIJzUuktk/edit?usp=sharing] [^Geo-spatial Enhancements_v2.pdf] was: Design document link: [https://docs.google.com/document/d/1YmTbHa0P39P8iURb_IZErIdZzx7jFgZ5fw6E2BdHjKQ/edit?usp=sharing] [^Geo-spatial Enhancements_v1.pdf] > GeoSpatial Query Enhancements > - > > Key: CARBONDATA-4166 > URL: https://issues.apache.org/jira/browse/CARBONDATA-4166 > Project: CarbonData > Issue Type: Improvement >Reporter: Indhumathi Muthumurugesh >Priority: Major > Attachments: Geo-spatial Enhancements_v1.pdf, Geo-spatial > Enhancements_v2.pdf > > > Design document link: > [https://docs.google.com/document/d/1YmTbHa0P39P8iURb_IZErIdZzx7jFgZ5fw6E2BdHjKQ/edit?usp=sharing] > > [^Geo-spatial Enhancements_v1.pdf] > > ^Version 2:^ > [https://docs.google.com/document/d/19AQj90Rll9iXBVcCMpcuKanwgDE6wuiGMYYIJzUuktk/edit?usp=sharing] > [^Geo-spatial Enhancements_v2.pdf] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-4166) GeoSpatial Query Enhancements
[ https://issues.apache.org/jira/browse/CARBONDATA-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated CARBONDATA-4166: - Attachment: Geo-spatial Enhancements_v2.pdf > GeoSpatial Query Enhancements > - > > Key: CARBONDATA-4166 > URL: https://issues.apache.org/jira/browse/CARBONDATA-4166 > Project: CarbonData > Issue Type: Improvement >Reporter: Indhumathi Muthumurugesh >Priority: Major > Attachments: Geo-spatial Enhancements_v1.pdf, Geo-spatial > Enhancements_v2.pdf > > > Design document link: > [https://docs.google.com/document/d/1YmTbHa0P39P8iURb_IZErIdZzx7jFgZ5fw6E2BdHjKQ/edit?usp=sharing] > > [^Geo-spatial Enhancements_v1.pdf] -- This message was sent by Atlassian Jira (v8.3.4#803005)
