Re: thank you for reviewing our patches

On Saturday, September 27, 2014 at 4:50 AM, Nicholas Chammas wrote: > > Spark is the first (and currently only) open source project I contribute > regularly to. My first several PRs against the project, as simple as they > were, were definitely patches that I “grew up on” Yeah, me too. I th

Re: SparkSQL: map type MatchError when inserting into Hive table

Would you mind to provide the DDL of this partitioned table together with the query you tried? The stacktrace suggests that the query was trying to cast a map into something else, which is not supported in Spark SQL. And I doubt whether Hive support casting a complex type to some other type.

Re: SparkSQL: map type MatchError when inserting into Hive table

Would you mind to provide the DDL of this partitioned table together with the query you tried? The stacktrace suggests that the query was trying to cast a map into something else, which is not supported in Spark SQL. And I doubt whether Hive support casting a complex type to some other type.

Re: SparkSQL: map type MatchError when inserting into Hive table

It might be a problem when inserting into a partitioned table. It worked fine to when the target table was unpartitioned. Can you confirm this? Thanks, Du On 9/26/14, 4:48 PM, "Du Li" wrote: >Hi, > >I was loading data into a partitioned table on Spark 1.1.0 >beeline-thriftserver. The table

SparkSQL: map type MatchError when inserting into Hive table

Hi, I was loading data into a partitioned table on Spark 1.1.0 beeline-thriftserver. The table has complex data types such as map and array>. The query is like ³insert overwrite table a partition (Š) select Š² and the select clause worked if run separately. However, when running the insert query,

Re: thank you for reviewing our patches

Keep the patches coming :) On Fri, Sep 26, 2014 at 1:50 PM, Nicholas Chammas < nicholas.cham...@gmail.com> wrote: > I recently came across this mailing list post by Linus Torvalds > about the value of reviewing even > “trivial” patches. The following passag

thank you for reviewing our patches

I recently came across this mailing list post by Linus Torvalds about the value of reviewing even “trivial” patches. The following passages stood out to me: I think that much more important than the patch is the fact that people get used to the notion that th

Re: FYI: jenkins systems patched to fix bash exploit

> > > we're not running bash.x86_64 0:4.1.2-15.el6_5.2 on all of our systems. > > s/not/now :)

FYI: jenkins systems patched to fix bash exploit

all of our systems were affected by the shellshock bug, and i've just patched everything w/the latest fix from redhat: https://access.redhat.com/articles/1200223 we're not running bash.x86_64 0:4.1.2-15.el6_5.2 on all of our systems. shane

Re: PARSING_ERROR from kryo

I am seeing the same error as well since upgrading to Spark1.1: 14/09/26 15:35:05 ERROR executor.Executor: Exception in task 1032.0 in stage 5.1 (TID 22449) com.esotericsoftware.kryo.KryoException: java.io.IOException: failed to uncompress the chunk: PARSING_ERROR(2) at com.esotericsoftwar

Re: executorAdded event to DAGScheduler

In Yarn, we can easily have multiple containers allocated in the same node. On Fri, Sep 26, 2014 at 6:05 PM, Nan Zhu wrote: > just a quick reply, we cannot start two executors in the same host for a > single application in the standard deployment (one worker per machine) > > I’m not sure if it

Re: executorAdded event to DAGScheduler

just a quick reply, we cannot start two executors in the same host for a single application in the standard deployment (one worker per machine) I’m not sure if it will create an issue when you have multiple workers in the same host, as submitWaitingStages is called everywhere and I never try s

Re: executorAdded event to DAGScheduler

Some corrections. On Fri, Sep 26, 2014 at 5:32 PM, praveen seluka wrote: > Can someone explain the motivation behind passing executorAdded event to > DAGScheduler ? *DAGScheduler *does *submitWaitingStages *when *executorAdded > *method is called by *TaskSchedulerImpl*. I see some issue in the b

executorAdded event to DAGScheduler

Can someone explain the motivation behind passing executorAdded event to DAGScheduler ? *DAGScheduler *does *submitWaitingStages *when *executorAdded *method is called by *TaskSchedulerImpl*. I see some issue in the below code, *TaskSchedulerImpl.scala code* if (!executorsByHost.contains(o.host))

Re: A Spark Compilation Question

Hi Hansu, I have encountered the same problem. Maven compiled avro file and generated corresponding Java file in new directory which is not source file directory of the project. I have modified pom.xml file and it can be work. The line marked as red is added, you can add them to your spark-*.*.*/