Re: after load data using SaveMode.Overwrite, query through beeline return all null field
Hi, I have checked with the current version and the issue is not reproducing, when I checked the code, there are code changes happened for the savemode from 1.3 to 1.4 version. You can check the PR #2186 for the changes done for that part and you can check your issue again with that PR . Regards, Akash -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: after load data using SaveMode.Overwrite, query through beeline return all null field
Hi Thank you reported this issue. Let us check it and response to you asap. Regards Liang 喜之郎 wrote > hi dev. > carbon version :1.3.1 > spark version:2.2.1 > 1) First I create a carbon table through beeline. > 2) Then I use spark-submit and dataframe load data to carbon. Query is OK。 > 3) Then I use spark-submit and dataframe load data to carbon again, but > query through thriftserver return all NULL field. > 4) I restart thriftserver, and query through thriftserver return correct > result. > > > This problem happens everytime. I think this problem may is about > CarbonThriftServer. > is there anybody encountered this problem? can anybody give some > advice,thanks. > > > you can use this code to reproduce problem. > === > > > > val carbonTableName = args(1) > > val warehouse = new File("./warehouse").getCanonicalPath > val metastore = new File("./metastore").getCanonicalPath > val spark = SparkSession > .builder() > .appName("StreamExample") > .config("spark.sql.warehouse.dir", warehouse) > .getOrCreateCarbonSession(warehouse, metastore) > val df2 = spark.read.parquet(path) > df2.write > .format("carbondata") > .option("tableName", carbonTableName) > .option("compress", "true") > .option("tempCSV", "false") > .mode(SaveMode.Overwrite) > .save()= -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
after load data using SaveMode.Overwrite, query through beeline return all null field
hi dev. carbon version :1.3.1 spark version:2.2.1 1) First I create a carbon table through beeline. 2) Then I use spark-submit and dataframe load data to carbon. Query is OK?? 3) Then I use spark-submit and dataframe load data to carbon again, but query through thriftserver return all NULL field. 4) I restart thriftserver, and query through thriftserver return correct result. This problem happens everytime. I think this problem may is about CarbonThriftServer. is there anybody encountered this problem? can anybody give some advice,thanks. you can use this code to reproduce problem. === val carbonTableName = args(1) val warehouse = new File("./warehouse").getCanonicalPath val metastore = new File("./metastore").getCanonicalPath val spark = SparkSession .builder() .appName("StreamExample") .config("spark.sql.warehouse.dir", warehouse) .getOrCreateCarbonSession(warehouse, metastore) val df2 = spark.read.parquet(path) df2.write .format("carbondata") .option("tableName", carbonTableName) .option("compress", "true") .option("tempCSV", "false") .mode(SaveMode.Overwrite) .save()=