Re: [DISCUSS] Resolve ambiguous parser rule between two "create table"s

On Thu, Mar 19, 2020 at 3:46 AM Wenchen Fan wrote: > 2. PARTITIONED BY colTypeList: I think we can support it in the unified > syntax. Just make sure it doesn't appear together with PARTITIONED BY > transformList. > Another side note: Perhaps as part of (or after) unifying the CREATE TABLE

Re: Scala vs PySpark Inconsistency: SQLContext/SparkSession access from DataFrame/DataSet

Ah, yes, I clearly didn't read closely enough! It was right there in front of me... Thanks! On Wed, Mar 18, 2020, 12:36 PM Maciej Szymkiewicz wrote: > Hi Ben, > > Please note that `_sc` is not a SQLContext. It is a SparkContext, which > is used primarily for internal calls. > > SQLContext is

Error in custom encoder

Hello everyone, I'm trying to implement new encoder, which will be able to encode Kotlin data classes. They can be generics, so I'm calculating their types in compiletime. My trouble is I have following code generated: /* 197 */ final InternalRow deserializetoobject_value_9 = false ? /*

Data Getting Inconsistent while running a Persist Job multiple time

Hi Team, I am getting an issue where I am running a join on my persisted data and storing it to multiple different locations. And getting Duplicates Record in one set of data and no duplicate in another set of data. I have verified my source also and not getting any kind of duplicate joining