subject:"\[jira\] \[Updated\] \(CARBONDATA\-3815\) Insert into table select from another table throws exception for spatial tables"

[jira] [Updated] (CARBONDATA-3815) Insert into table select from another table throws exception for spatial tables

2020-05-11 Thread Venugopal Reddy K (Jira)



 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Venugopal Reddy K updated CARBONDATA-3815:
--
Description: 
*Issue:*

Insert into table select from another table throws exception for spatial 
tables. NoSuchElementException exception is thrown with 'mygeohash' column.

{color:#ff}Exception in thread "main" java.util.NoSuchElementException: key 
not found: mygeohashException in thread "main" 
java.util.NoSuchElementException: key not found: mygeohash at 
scala.collection.MapLike$class.default(MapLike.scala:228) at 
scala.collection.AbstractMap.default(Map.scala:59) at 
scala.collection.mutable.HashMap.apply(HashMap.scala:65) at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand$$anonfun$getReArrangedIndexAndSelectedSchema$5.apply(CarbonInsertIntoCommand.scala:504)
 at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand$$anonfun$getReArrangedIndexAndSelectedSchema$5.apply(CarbonInsertIntoCommand.scala:497)
 at 
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) 
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.getReArrangedIndexAndSelectedSchema(CarbonInsertIntoCommand.scala:496)
 at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.processData(CarbonInsertIntoCommand.scala:164){color}

*Step to reproduce:*
 # Create source table and target table spatial tables.
 # Load data to source table.
 # Insert into target table select from source table.

*TestCase:*

spark.sql(s"""
CREATE TABLE source(
timevalue BIGINT,
longitude LONG,
latitude LONG) COMMENT "This is a GeoTable"
STORED AS carbondata
TBLPROPERTIES ('INDEX_HANDLER'='mygeohash',
'INDEX_HANDLER.mygeohash.type'='geohash',
'INDEX_HANDLER.mygeohash.sourcecolumns'='longitude, latitude',
'INDEX_HANDLER.mygeohash.originLatitude'='39.832277',
'INDEX_HANDLER.mygeohash.gridSize'='50',
'INDEX_HANDLER.mygeohash.minLongitude'='115.811865',
'INDEX_HANDLER.mygeohash.maxLongitude'='116.782233',
'INDEX_HANDLER.mygeohash.minLatitude'='39.832277',
'INDEX_HANDLER.mygeohash.maxLatitude'='40.225281',
'INDEX_HANDLER.mygeohash.conversionRatio'='100')
""".stripMargin)

 

val path = s"$rootPath/examples/spark/src/main/resources/geodata.csv"
spark.sql(s"""
LOAD DATA LOCAL INPATH '$path'
INTO TABLE source
OPTIONS('COMPLEX_DELIMITER_LEVEL_1'='#')
""".stripMargin)

 

spark.sql(s"""
CREATE TABLE target(
timevalue BIGINT,
longitude LONG,
latitude LONG) COMMENT "This is a GeoTable"
STORED AS carbondata
TBLPROPERTIES ('INDEX_HANDLER'='mygeohash',
'INDEX_HANDLER.mygeohash.type'='geohash',
'INDEX_HANDLER.mygeohash.sourcecolumns'='longitude, latitude',
'INDEX_HANDLER.mygeohash.originLatitude'='39.832277',
'INDEX_HANDLER.mygeohash.gridSize'='50',
'INDEX_HANDLER.mygeohash.minLongitude'='115.811865',
'INDEX_HANDLER.mygeohash.maxLongitude'='116.782233',
'INDEX_HANDLER.mygeohash.minLatitude'='39.832277',
'INDEX_HANDLER.mygeohash.maxLatitude'='40.225281',
'INDEX_HANDLER.mygeohash.conversionRatio'='100')
""".stripMargin)
 
 
spark.sql("insert into target select * from source")

  was:
*Issue:*

Insert into table select from another table throws exception for spatial 
tables. NoSuchElementException exception is thrown with 'mygeohash' column.

{color:#ff}Exception in thread "main" java.util.NoSuchElementException: key 
not found: mygeohashException in thread "main" 
java.util.NoSuchElementException: key not found: mygeohash at 
scala.collection.MapLike$class.default(MapLike.scala:228) at 
scala.collection.AbstractMap.default(Map.scala:59) at 
scala.collection.mutable.HashMap.apply(HashMap.scala:65) at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand$$anonfun$getReArrangedIndexAndSelectedSchema$5.apply(CarbonInsertIntoCommand.scala:504)
 at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand$$anonfun$getReArrangedIndexAndSelectedSchema$5.apply(CarbonInsertIntoCommand.scala:497)
 at 
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) 
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.getReArrangedIndexAndSelectedSchema(CarbonInsertIntoCommand.scala:496)
 at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.processData(CarbonInsertIntoCommand.scala:164){color}

*Step to reproduce:*
 # Create source table and target table spatial tables.
 # Load data to source table.
 # Insert into target table select from source table.

*TestCase:*
|spark.sql(s"""|
|CREATE TABLE source(|
|timevalue BIGINT,|
|longitude LONG,|
|latitude LONG) COMMENT "This is a GeoTable"|
|STORED AS carbondata|
|TBLPROPERTIES ('INDEX_HANDLER'='mygeohash',|
|'INDEX_HANDLER.mygeohash.type'='geohash',|

[jira] [Updated] (CARBONDATA-3815) Insert into table select from another table throws exception for spatial tables

2020-05-11 Thread Venugopal Reddy K (Jira)



 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Venugopal Reddy K updated CARBONDATA-3815:
--
Description: 
*Issue:*

Insert into table select from another table throws exception for spatial 
tables. NoSuchElementException exception is thrown with 'mygeohash' column.

{color:#ff}Exception in thread "main" java.util.NoSuchElementException: key 
not found: mygeohashException in thread "main" 
java.util.NoSuchElementException: key not found: mygeohash at 
scala.collection.MapLike$class.default(MapLike.scala:228) at 
scala.collection.AbstractMap.default(Map.scala:59) at 
scala.collection.mutable.HashMap.apply(HashMap.scala:65) at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand$$anonfun$getReArrangedIndexAndSelectedSchema$5.apply(CarbonInsertIntoCommand.scala:504)
 at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand$$anonfun$getReArrangedIndexAndSelectedSchema$5.apply(CarbonInsertIntoCommand.scala:497)
 at 
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) 
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.getReArrangedIndexAndSelectedSchema(CarbonInsertIntoCommand.scala:496)
 at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.processData(CarbonInsertIntoCommand.scala:164){color}

*Step to reproduce:*
 # Create source table and target table spatial tables.
 # Load data to source table.
 # Insert into target table select from source table.

*TestCase:*
|spark.sql(s"""|
|CREATE TABLE source(|
|timevalue BIGINT,|
|longitude LONG,|
|latitude LONG) COMMENT "This is a GeoTable"|
|STORED AS carbondata|
|TBLPROPERTIES ('INDEX_HANDLER'='mygeohash',|
|'INDEX_HANDLER.mygeohash.type'='geohash',|
|'INDEX_HANDLER.mygeohash.sourcecolumns'='longitude, latitude',|
|'INDEX_HANDLER.mygeohash.originLatitude'='39.832277',|
|'INDEX_HANDLER.mygeohash.gridSize'='50',|
|'INDEX_HANDLER.mygeohash.minLongitude'='115.811865',|
|'INDEX_HANDLER.mygeohash.maxLongitude'='116.782233',|
|'INDEX_HANDLER.mygeohash.minLatitude'='39.832277',|
|'INDEX_HANDLER.mygeohash.maxLatitude'='40.225281',|
|'INDEX_HANDLER.mygeohash.conversionRatio'='100')
 """.stripMargin)
 val path = s"$rootPath/examples/spark/src/main/resources/geodata.csv"|

| spark.sql(s"""
|LOAD DATA LOCAL INPATH '$path'|
|INTO TABLE source|
|OPTIONS('COMPLEX_DELIMITER_LEVEL_1'='#')
 """.stripMargin)|

| spark.sql(s"""
|CREATE TABLE target(|
|timevalue BIGINT,|
|longitude LONG,|
|latitude LONG) COMMENT "This is a GeoTable"|
|STORED AS carbondata|
|TBLPROPERTIES ('INDEX_HANDLER'='mygeohash',|
|'INDEX_HANDLER.mygeohash.type'='geohash',|
|'INDEX_HANDLER.mygeohash.sourcecolumns'='longitude, latitude',|
|'INDEX_HANDLER.mygeohash.originLatitude'='39.832277',|
|'INDEX_HANDLER.mygeohash.gridSize'='50',|
|'INDEX_HANDLER.mygeohash.minLongitude'='115.811865',|
|'INDEX_HANDLER.mygeohash.maxLongitude'='116.782233',|
|'INDEX_HANDLER.mygeohash.minLatitude'='39.832277',|
|'INDEX_HANDLER.mygeohash.maxLatitude'='40.225281',|
|'INDEX_HANDLER.mygeohash.conversionRatio'='100')
 """.stripMargin)|

| spark.sql("insert into target select * from source")

  was:
*Issue:*

Insert into table select from another table throws exception for spatial 
tables. NoSuchElementException exception is thrown with 'mygeohash' column.

{color:#ff}Exception in thread "main" java.util.NoSuchElementException: key 
not found: mygeohashException in thread "main" 
java.util.NoSuchElementException: key not found: mygeohash at 
scala.collection.MapLike$class.default(MapLike.scala:228) at 
scala.collection.AbstractMap.default(Map.scala:59) at 
scala.collection.mutable.HashMap.apply(HashMap.scala:65) at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand$$anonfun$getReArrangedIndexAndSelectedSchema$5.apply(CarbonInsertIntoCommand.scala:504)
 at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand$$anonfun$getReArrangedIndexAndSelectedSchema$5.apply(CarbonInsertIntoCommand.scala:497)
 at 
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) 
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.getReArrangedIndexAndSelectedSchema(CarbonInsertIntoCommand.scala:496)
 at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.processData(CarbonInsertIntoCommand.scala:164){color}

*Step to reproduce:*
 # Create source table and target table spatial tables.
 # Load data to source table.
 # Insert into target table select from source table.

*TestCase:*

| spark.sql(s"""
|CREATE TABLE source(|
|timevalue BIGINT,|
|longitude LONG,|
|latitude LONG) COMMENT "This is a GeoTable"|
|STORED AS carbondata|
|TBLPROPERTIES

[jira] [Updated] (CARBONDATA-3815) Insert into table select from another table throws exception for spatial tables

2020-05-11 Thread Venugopal Reddy K (Jira)



 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Venugopal Reddy K updated CARBONDATA-3815:
--
Description: 
*Issue:*

Insert into table select from another table throws exception for spatial 
tables. NoSuchElementException exception is thrown with 'mygeohash' column.

{color:#ff}Exception in thread "main" java.util.NoSuchElementException: key 
not found: mygeohashException in thread "main" 
java.util.NoSuchElementException: key not found: mygeohash at 
scala.collection.MapLike$class.default(MapLike.scala:228) at 
scala.collection.AbstractMap.default(Map.scala:59) at 
scala.collection.mutable.HashMap.apply(HashMap.scala:65) at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand$$anonfun$getReArrangedIndexAndSelectedSchema$5.apply(CarbonInsertIntoCommand.scala:504)
 at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand$$anonfun$getReArrangedIndexAndSelectedSchema$5.apply(CarbonInsertIntoCommand.scala:497)
 at 
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) 
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.getReArrangedIndexAndSelectedSchema(CarbonInsertIntoCommand.scala:496)
 at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.processData(CarbonInsertIntoCommand.scala:164){color}

*Step to reproduce:*
 # Create source table and target table spatial tables.
 # Load data to source table.
 # Insert into target table select from source table.

*TestCase:*

| spark.sql(s"""
|CREATE TABLE source(|
|timevalue BIGINT,|
|longitude LONG,|
|latitude LONG) COMMENT "This is a GeoTable"|
|STORED AS carbondata|
|TBLPROPERTIES ('INDEX_HANDLER'='mygeohash',|
|'INDEX_HANDLER.mygeohash.type'='geohash',|
|'INDEX_HANDLER.mygeohash.sourcecolumns'='longitude, latitude',|
|'INDEX_HANDLER.mygeohash.originLatitude'='39.832277',|
|'INDEX_HANDLER.mygeohash.gridSize'='50',|
|'INDEX_HANDLER.mygeohash.minLongitude'='115.811865',|
|'INDEX_HANDLER.mygeohash.maxLongitude'='116.782233',|
|'INDEX_HANDLER.mygeohash.minLatitude'='39.832277',|
|'INDEX_HANDLER.mygeohash.maxLatitude'='40.225281',|
|'INDEX_HANDLER.mygeohash.conversionRatio'='100')
 """.stripMargin)
 val path = s"$rootPath/examples/spark/src/main/resources/geodata.csv"|


 spark.sql(
 s"""
|LOAD DATA LOCAL INPATH '$path'|
|INTO TABLE source|
|OPTIONS('COMPLEX_DELIMITER_LEVEL_1'='#')
 """.stripMargin)|

spark.sql(s"""
|CREATE TABLE target(|
|timevalue BIGINT,|
|longitude LONG,|
|latitude LONG) COMMENT "This is a GeoTable"|
|STORED AS carbondata|
|TBLPROPERTIES ('INDEX_HANDLER'='mygeohash',|
|'INDEX_HANDLER.mygeohash.type'='geohash',|
|'INDEX_HANDLER.mygeohash.sourcecolumns'='longitude, latitude',|
|'INDEX_HANDLER.mygeohash.originLatitude'='39.832277',|
|'INDEX_HANDLER.mygeohash.gridSize'='50',|
|'INDEX_HANDLER.mygeohash.minLongitude'='115.811865',|
|'INDEX_HANDLER.mygeohash.maxLongitude'='116.782233',|
|'INDEX_HANDLER.mygeohash.minLatitude'='39.832277',|
|'INDEX_HANDLER.mygeohash.maxLatitude'='40.225281',|
|'INDEX_HANDLER.mygeohash.conversionRatio'='100')
 """.stripMargin)
 spark.sql("insert into target select * from source")|

  was:
*Issue:*

Insert into table select from another table throws exception for spatial 
tables. NoSuchElementException exception is thrown with 'mygeohash' column.

{color:#FF}Exception in thread "main" java.util.NoSuchElementException: key 
not found: mygeohashException in thread "main" 
java.util.NoSuchElementException: key not found: mygeohash at 
scala.collection.MapLike$class.default(MapLike.scala:228) at 
scala.collection.AbstractMap.default(Map.scala:59) at 
scala.collection.mutable.HashMap.apply(HashMap.scala:65) at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand$$anonfun$getReArrangedIndexAndSelectedSchema$5.apply(CarbonInsertIntoCommand.scala:504)
 at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand$$anonfun$getReArrangedIndexAndSelectedSchema$5.apply(CarbonInsertIntoCommand.scala:497)
 at 
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) 
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.getReArrangedIndexAndSelectedSchema(CarbonInsertIntoCommand.scala:496)
 at 
org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.processData(CarbonInsertIntoCommand.scala:164){color}

*Step to reproduce:*
 # Create source table and target table spatial tables.
 # Load data to source table.
 # Insert into target table select from source table.

*TestCase:*

spark.sql(s"""
 | CREATE TABLE source(
 | timevalue BIGINT,
 | longitude LONG,
 | latitude LONG) COMMENT "This is a GeoTable"
 | STORED AS carbondata
 | TBLPROPERTIES

[jira] [Updated] (CARBONDATA-3815) Insert into table select from another table throws exception for spatial tables

2020-05-11 Thread Venugopal Reddy K (Jira)



 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Venugopal Reddy K updated CARBONDATA-3815:
--
Summary: Insert into table select from another table throws exception for 
spatial tables  (was: Insert into table select from another table throws 
exception)

> Insert into table select from another table throws exception for spatial 
> tables
> ---
>
> Key: CARBONDATA-3815
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3815
> Project: CarbonData
>  Issue Type: Bug
>  Components: core, spark-integration
>Affects Versions: 2.0.0
>Reporter: Venugopal Reddy K
>Priority: Major
>
> *Issue:*
> Insert into table select from another table throws exception for spatial 
> tables. NoSuchElementException exception is thrown with 'mygeohash' column.
> {color:#FF}Exception in thread "main" java.util.NoSuchElementException: 
> key not found: mygeohashException in thread "main" 
> java.util.NoSuchElementException: key not found: mygeohash at 
> scala.collection.MapLike$class.default(MapLike.scala:228) at 
> scala.collection.AbstractMap.default(Map.scala:59) at 
> scala.collection.mutable.HashMap.apply(HashMap.scala:65) at 
> org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand$$anonfun$getReArrangedIndexAndSelectedSchema$5.apply(CarbonInsertIntoCommand.scala:504)
>  at 
> org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand$$anonfun$getReArrangedIndexAndSelectedSchema$5.apply(CarbonInsertIntoCommand.scala:497)
>  at 
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
>  at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at 
> org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.getReArrangedIndexAndSelectedSchema(CarbonInsertIntoCommand.scala:496)
>  at 
> org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.processData(CarbonInsertIntoCommand.scala:164){color}
> *Step to reproduce:*
>  # Create source table and target table spatial tables.
>  # Load data to source table.
>  # Insert into target table select from source table.
> *TestCase:*
> spark.sql(s"""
>  | CREATE TABLE source(
>  | timevalue BIGINT,
>  | longitude LONG,
>  | latitude LONG) COMMENT "This is a GeoTable"
>  | STORED AS carbondata
>  | TBLPROPERTIES ('INDEX_HANDLER'='mygeohash',
>  | 'INDEX_HANDLER.mygeohash.type'='geohash',
>  | 'INDEX_HANDLER.mygeohash.sourcecolumns'='longitude, latitude',
>  | 'INDEX_HANDLER.mygeohash.originLatitude'='39.832277',
>  | 'INDEX_HANDLER.mygeohash.gridSize'='50',
>  | 'INDEX_HANDLER.mygeohash.minLongitude'='115.811865',
>  | 'INDEX_HANDLER.mygeohash.maxLongitude'='116.782233',
>  | 'INDEX_HANDLER.mygeohash.minLatitude'='39.832277',
>  | 'INDEX_HANDLER.mygeohash.maxLatitude'='40.225281',
>  | 'INDEX_HANDLER.mygeohash.conversionRatio'='100')
>  """.stripMargin)
> val path = s"$rootPath/examples/spark/src/main/resources/geodata.csv"
> // scalastyle:off
> spark.sql(
>  s"""
>  | LOAD DATA LOCAL INPATH '$path'
>  | INTO TABLE source
>  | OPTIONS('COMPLEX_DELIMITER_LEVEL_1'='#')
>  """.stripMargin)
> spark.sql(s"""
>  | CREATE TABLE target(
>  | timevalue BIGINT,
>  | longitude LONG,
>  | latitude LONG) COMMENT "This is a GeoTable"
>  | STORED AS carbondata
>  | TBLPROPERTIES ('INDEX_HANDLER'='mygeohash',
>  | 'INDEX_HANDLER.mygeohash.type'='geohash',
>  | 'INDEX_HANDLER.mygeohash.sourcecolumns'='longitude, latitude',
>  | 'INDEX_HANDLER.mygeohash.originLatitude'='39.832277',
>  | 'INDEX_HANDLER.mygeohash.gridSize'='50',
>  | 'INDEX_HANDLER.mygeohash.minLongitude'='115.811865',
>  | 'INDEX_HANDLER.mygeohash.maxLongitude'='116.782233',
>  | 'INDEX_HANDLER.mygeohash.minLatitude'='39.832277',
>  | 'INDEX_HANDLER.mygeohash.maxLatitude'='40.225281',
>  | 'INDEX_HANDLER.mygeohash.conversionRatio'='100')
>  """.stripMargin)
> spark.sql("insert into target select * from source")



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (CARBONDATA-3815) Insert into table select from another table throws exception for spatial tables

[jira] [Updated] (CARBONDATA-3815) Insert into table select from another table throws exception for spatial tables

[jira] [Updated] (CARBONDATA-3815) Insert into table select from another table throws exception for spatial tables

[jira] [Updated] (CARBONDATA-3815) Insert into table select from another table throws exception for spatial tables

4 matches

Site Navigation

Mail list logo

Footer information