[
https://issues.apache.org/jira/browse/CARBONDATA-404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacky Li resolved CARBONDATA-404.
---------------------------------
Resolution: Fixed
Assignee: Ravindra Pesala
Fix Version/s: 0.3.0-incubating
> Data loading from DataFrame to carbon table is FAILED
> -----------------------------------------------------
>
> Key: CARBONDATA-404
> URL: https://issues.apache.org/jira/browse/CARBONDATA-404
> Project: CarbonData
> Issue Type: Bug
> Components: data-load
> Affects Versions: 0.1.0-incubating
> Reporter: Babulal
> Assignee: Ravindra Pesala
> Fix For: 0.3.0-incubating
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Data loading FAILED when Loading data from DataFrame with tempCSV option
> =true (Default option ) in 3 Node cluster .
> Steps
> val customSchema = StructType(Array( StructField("imei", StringType,
> true), StructField("deviceInformationId", IntegerType, true),
> StructField("mac", StringType, true), StructField("productdate",
> TimestampType , true), StructField("updatetime", TimestampType, true),
> StructField("gamePointId", DoubleType, true),
> StructField("contractNumber", DoubleType, true) ));
> val df = cc.read.format("com.databricks.spark.csv").option("header",
> "false").schema(customSchema).load("/opt/data/xyz/100_default_date_11_header.csv");
> Start data loading
> scala> df.write.format("carbondata").option("tableName","mycarbon2").save();
> INFO 10-11 23:24:35,970 - main Query [
> CREATE TABLE IF NOT EXISTS DEFAULT.MYCARBON2
> (IMEI STRING, DEVICEINFORMATIONID INT, MAC STRING, PRODUCTDATE
> TIMESTAMP, UPDATETIME TIMESTAMP, GAMEPOINTID DOUBLE, CONTRACTNUMBER DOUBLE)
> STORED BY 'ORG.APACHE.CARBONDATA.FORMAT'
> ]
> INFO 10-11 23:24:35,977 - Parsing command:
> CREATE TABLE IF NOT EXISTS default.mycarbon2
> (imei STRING, deviceInformationId INT, mac STRING, productdate
> TIMESTAMP, updatetime TIMESTAMP, gamePointId DOUBLE, contractNumber DOUBLE)
> STORED BY 'org.apache.carbondata.format'
> INFO 10-11 23:24:35,978 - Parse Completed
> INFO 10-11 23:24:36,227 - main Query [
> LOAD DATA INPATH './TEMPCSV'
> INTO TABLE DEFAULT.MYCARBON2
> OPTIONS ('FILEHEADER' =
> 'IMEI,DEVICEINFORMATIONID,MAC,PRODUCTDATE,UPDATETIME,GAMEPOINTID,CONTRACTNUMBER')
> ]
> INFO 10-11 23:24:36,233 - Successfully able to get the table metadata file
> lock
> AUDIT 10-11 23:24:36,234 - [BLR1000007781][root][Thread-1]Dataload failed for
> default.mycarbon2. The input file does not exist: ./tempCSV
> INFO 10-11 23:24:36,234 - main Successfully deleted the lock file
> /tmp/default/mycarbon2/meta.lock
> INFO 10-11 23:24:36,234 - Table MetaData Unlocked Successfully after data
> load
> org.apache.carbondata.processing.etl.DataLoadingException: The input file
> does not exist: ./tempCSV
> at
> org.apache.spark.util.FileUtils$$anonfun$getPaths$1.apply$mcVI$sp(FileUtils.scala:66)
> CSV DATA
> 1AA1,1,Mikaa1,2015-01-01 11:00:00,2015-01-01 13:00:00,198,260
> 1AA2,3,Mikaa2,2015-01-02 12:00:00,2015-01-01 14:00:00,278,230
> 1AA3,1,Mikaa1,2015-01-03 13:00:00,2015-01-01 15:00:00,2556,1
> 1AA4,10,Mikaa2,2015-01-04 14:00:00,2015-01-01 16:00:00,640,254
> 1AA5,10,Mikaa,2015-01-05 15:00:00,2015-01-01 17:00:00,980,256
> 1AA6,10,Mikaa,2015-01-06 16:00:00,2015-01-01 18:00:00,1,2378
> 1AA7,10,Mikaa,2015-01-07 17:00:00,2015-01-01 19:00:00,96,234
> 1AA8,9,max,2015-01-08 18:00:00,2015-01-01 20:00:00,89,236
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
