Hello there,
I am trying to check to see CDB is available for Apache Spark. I could
currently use CDB using Postgres driver. But I would like to check to see
if there are any specialized drivers that I can use which optimizes for
predicate-push-down and other optimizations pertaining to
The files are processed in the order the file last modified timestamp. The
path and partitioning scheme are not used for ordering.
On Thu, Jun 14, 2018 at 6:59 AM, karthikjay wrote:
> My parquet files are first partitioned by environment and then by date
> like:
>
> env=testing/
>
Akash,
Are you able to run your code in pyspark shell with no issues?
Best Regards,
Vamshi T
From: Hyukjin Kwon
Sent: Friday, June 15, 2018 10:18 AM
To: Marcelo Vanzin
Cc: aakash.spark@gmail.com; user @spark
Subject: Re: Issue upgrading to Spark 2.3.1
unsubscribe
--
Wealth is not money. Wealth is relationships with people.
unsubscribe
I use PyCharm. Mind if I ask to elaborate what you did step by step?
2018년 6월 16일 (토) 오전 12:11, Marcelo Vanzin 님이
작성:
> I'm not familiar with PyCharm. But if you can run "pyspark" from the
> command line and not hit this, then this might be an issue with
> PyCharm or your environment - e.g.
I use PyCharm. Mind if I ask to elaborate what you did step by step?
2018년 6월 16일 (토) 오전 12:11, Marcelo Vanzin 님이
작성:
> I'm not familiar with PyCharm. But if you can run "pyspark" from the
> command line and not hit this, then this might be an issue with
> PyCharm or your environment - e.g.
I'm not familiar with PyCharm. But if you can run "pyspark" from the
command line and not hit this, then this might be an issue with
PyCharm or your environment - e.g. having an old version of the
pyspark code around, or maybe PyCharm itself might need to be updated.
On Thu, Jun 14, 2018 at 10:01
Hi,
I've a job running which shows the Event Timeline as follows, I am trying
to guess the gaps between these single lines, they seem to be parallel but
not immediately sequential with other stages.
Any other insight from this, and what is the cluster doing during these
gaps?
Thanks,
Aakash.
