Re: Writing to vertica from spark

Fixed this by setting fileformat -> "parquet" On Mon, Oct 22, 2018 at 11:48 AM Nikhil Goyal wrote: > Hi guys, > > My code is failing with this error > > java.lang.Exception: S2V: FATAL ERROR for job S2V_job9197956021769393773. > Job status information is available in the Vertica table >

Re: Spark 2.3.2 : No of active tasks vastly exceeds total no of executor cores

In my log, I have found mylog.2:2018-10-19 20:00:50,455 WARN [dag-scheduler-event-loop] (Logging.scala:66) - Dropped 3498 events from appStatus since Fri Oct 19 19:25:05 UTC 2018.mylog.2:2018-10-19 20:02:07,053 WARN [dispatcher-event-loop-1] (Logging.scala:66) - Dropped 123385 events from

Writing to vertica from spark

Hi guys, My code is failing with this error java.lang.Exception: S2V: FATAL ERROR for job S2V_job9197956021769393773. Job status information is available in the Vertica table test.S2V_JOB_STATUS_USER_NGOYAL. Unable to save intermediate orc files to HDFS

Re: [External Sender] Writing dataframe to vertica

Got the issue fixed. Had to set 'dbschema' param in connection options. By default it is set to public Thanks On Tue, Oct 16, 2018 at 9:49 PM Femi Anthony wrote: > How are you trying to write to Vertica ? Can you provide some snippets of > code ? > > > Femi > > On Tue, Oct 16, 2018 at 7:24 PM

Re: [Spark UI] Spark 2.3.1 UI no longer respects spark.ui.retainedJobs

Just tried on 2.3.2 and worked fine for me. UI had a single job and a single stage (+ the tasks related to that single stage), same thing in memory (checked with jvisualvm). On Sat, Oct 20, 2018 at 6:45 PM Marcelo Vanzin wrote: > > On Tue, Oct 16, 2018 at 9:34 AM Patrick Brown > wrote: > > I

Structured Streaming: stream-stream join with several equality conditions in a disjunction

Hi, I've just tried to do a stream-stream join with several equality conditions in a disjunction and got the following error: "terminated with exception: Stream stream joins without equality predicate is not supported;;" The query was in this sort of form: