Re: refresh table [CARBON-1.3.1]

hello mohdshahidkhan, sorry for the writing mistake and step3: xx is database and i did val cc = SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession("hdfs://ns1/user") ) step4:two table data directory like: /user/xx/prod_inst_cab_backup /user/xx/prod_inst_cab -

Re: Does index be used when doing "join" operation between a big table and a small table?

Except the number of tasks, can we confirm whether to use index from the log file? Such as something like keyword for us to search in the log file directly. - FFCS研究院 -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: can't apply mappartitions to dataframe generated from carboncontext

modify carboncontext.scala file adding @transient to two member variables and problem solved @transient val sc: SparkContext, @transient val hiveClientInterface = metadataHive - FFCS研究院 -- View this message in context:

Re: classnot found exception

i'm using hdp spark and have both spark1.6+spark2.1 installed and use spark2.1 by default. after add export SPARK_MAJOR_VERSION=1,problem solved. - FFCS研究院 -- View this message in context:

Re: problem with branch-1.1

delete aliyun mirror from maven by editing settings.xml file and remove scala-2.11 from idea's global library even you assigned scala-2.10. - FFCS研究院 -- View this message in context:

Re: can't apply mappartitions to dataframe generated from carboncontext

org.apache.spark.SparkException: Task not serializable at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:304) at org.apache.spark.util.ClosureCleaner$.org$apache$spark$util$ClosureCleaner$$clean(ClosureCleaner.scala:294) at

Re: classnot found exception

by reviewing source code,the refer order is :CatalystConf-->CodeGenerateFactory-->CarbonContext,but i'm using spark2.1,which is carbonsession,i don't know why.and the spark-catalyst.jar in jars direcotry don't have CatalystConf.class because carbonsession doesn't refer it. -- View this message

Re: when plan to implemnt merge operation

merge example like this: small: id updatatime 1 9:00 2 8:00 6 9:00 big: id updatetime 1 10:00 2 7:00 3 9:00 4 9:00 5 9:00 as for record in small: id=1,small.updatebigdate.update,update big; id=6,big doesn't

Re: SPARK update error

the path is solid,never change carbon store path. -- View this message in context: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/SPARK-update-error-tp13144p13225.html Sent from the Apache CarbonData Dev Mailing List archive mailing list archive at Nabble.com.