Re: Problem on carbondata quering performance tuning

Hi Thanks for using Carbondata. Based on Information you provided , Please try below solutions /Points. *A. Tune Resource Allocation * You have 55 core/NM , and given spark.executor.cores= 54 which means one NM will have only one Executor and total you will have only 4 Executor even

Re: Problem on carbondata quering performance tuning

HI Which carbondata+spark version? and can you provide the full configuration inside "carbondata.properties" Mick Yuan wrote > Hi,all > I have a quering performane tuning case on carbondata. > > *Enviroment is as below:*: > spark on yarn > 4 nodemanagers > 102G,55 cores each

Problem on carbondata quering performance tuning

Hi,all I have a quering performane tuning case on carbondata. *Enviroment is as below:*: spark on yarn 4 nodemanagers 102G,55 cores each nodemanager *Spark properties:* spark.master yarn spark.submit.deployMode client spark.serializer org.apache.spark.serializer.KryoSerializer

Re: Storing Data Frame as CarbonData Table

Hi Michael Yes, it is very easy to save any spark data to carbondata. Just need to do small change based on your script, as below : myDF.write .format("carbondata") .option("tableName" "MyTable") .mode(SaveMode.Overwrite) .save() For more detail, you can refer to examples: