Re: Installing Distributed apache spark cluster with Cluster mode on Docker

Hi Khalid Mammadov, Thank you for your response, Yes, I did, I built standalone apache spark cluster on docker containers. But I am looking for distributed spark cluster, Where spark workers are scalable and spark "deployment mode = cluster". Source url I used to built standalone ap

Re: Installing Distributed apache spark cluster with Cluster mode on Docker

1. I have a DEV-POC task to do, > 2. Need to Installing Distributed apache-spark cluster with Cluster mode > on Docker containers. > 3. with Scalable spark-worker containers. > 4. we have a 9 node cluster with some other services or tools. > > Thanks, > Dinakar >

Installing Distributed apache spark cluster with Cluster mode on Docker

Hi All, I am Dinakar, Hadoop admin, could someone help me here, 1. I have a DEV-POC task to do, 2. Need to Installing Distributed apache-spark cluster with Cluster mode on Docker containers. 3. with Scalable spark-worker containers. 4. we have a 9 node cluster with some other services or tools

Re: [ANNOUNCE] Apache Spark 3.0.3 released

Thank you, Yi! On Thu, Jun 24, 2021 at 10:52 PM Yi Wu wrote: > We are happy to announce the availability of Spark 3.0.3! > > Spark 3.0.3 is a maintenance release containing stability fixes. This > release is based on the branch-3.0 maintenance branch of Spark. We strongly > recommend all 3.0 u

[ANNOUNCE] Apache Spark 3.0.3 released

We are happy to announce the availability of Spark 3.0.3! Spark 3.0.3 is a maintenance release containing stability fixes. This release is based on the branch-3.0 maintenance branch of Spark. We strongly recommend all 3.0 users to upgrade to this stable release. To download Spark 3.0.3, head over

Re: [apache spark] Does Spark 2.4.8 have issues with ServletContextHandler

Did you include Apache Spark dependencies in your build? if you did, you should remove it. If you are using sbt, all spark dependencies should be as "provided". On Wed, Jun 2, 2021 at 10:11 AM Kanchan Kauthale < kanchankauthal...@gmail.com> wrote: > Hello Sean, > > Pl

Re: [apache spark] Does Spark 2.4.8 have issues with ServletContextHandler

Hello Sean, Please find below the stack trace- java.lang.NoclassDefFoundError: Could not initialize class org.spark.project.jetty.servlet.ServletContextHandler at org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:143) at org.apache.spark.ui.JettyUtils$.createServletHandler(J

Re: [ANNOUNCE] Apache Spark 3.1.2 released

Big shout-out to you, Dongjoon! Thank you. Pozdrawiam, Jacek Laskowski https://about.me/JacekLaskowski "The Internals Of" Online Books Follow me on https://twitter.com/jaceklaskowski On Wed, Jun 2, 2021 at 2:59 AM Dongjoon Hyu

Re: [ANNOUNCE] Apache Spark 3.1.2 released

Thank you, Dongjoon! On Wed, Jun 2, 2021 at 2:29 PM Xiao Li wrote: > Thank you! > > Xiao > > On Tue, Jun 1, 2021 at 9:29 PM Hyukjin Kwon wrote: > >> awesome! >> >> 2021년 6월 2일 (수) 오전 9:59, Dongjoon Hyun 님이 작성: >> >>> We are happy to announce the availability of Spark 3.1.2! >>> >>> Spark 3.1.2

Re: [ANNOUNCE] Apache Spark 3.1.2 released

Thank you! Xiao On Tue, Jun 1, 2021 at 9:29 PM Hyukjin Kwon wrote: > awesome! > > 2021년 6월 2일 (수) 오전 9:59, Dongjoon Hyun 님이 작성: > >> We are happy to announce the availability of Spark 3.1.2! >> >> Spark 3.1.2 is a maintenance release containing stability fixes. This >> release is based on the b

Re: [ANNOUNCE] Apache Spark 3.1.2 released

awesome! 2021년 6월 2일 (수) 오전 9:59, Dongjoon Hyun 님이 작성: > We are happy to announce the availability of Spark 3.1.2! > > Spark 3.1.2 is a maintenance release containing stability fixes. This > release is based on the branch-3.1 maintenance branch of Spark. We strongly > recommend all 3.1 users to u

回复：[ANNOUNCE] Apache Spark 3.1.2 released

Great work! Dongjoon! -- 原始邮件 -- 发件人: "Dongjoon Hyun" http

[ANNOUNCE] Apache Spark 3.1.2 released

We are happy to announce the availability of Spark 3.1.2! Spark 3.1.2 is a maintenance release containing stability fixes. This release is based on the branch-3.1 maintenance branch of Spark. We strongly recommend all 3.1 users to upgrade to this stable release. To download Spark 3.1.2, head over

Re: [apache spark] Does Spark 2.4.8 have issues with ServletContextHandler

Despite the name, the error doesn't mean the class isn't found but could not be initialized. What's the rest of the error? I don't believe any testing has ever encountered this error, so it's likely something to do with your environment, but I don't know what. On Thu, May 27, 2021 at 7:32 AM Kanch

Re: [apache spark] Does Spark 2.4.8 have issues with ServletContextHandler

Hello, I could see Jetty version has been updated to 9.4.35, from 9.4.28 in JIra- https://issues.apache.org/jira/browse/SPARK-33831 Does it have something to do with it? Thank you Kanchan On Thu, May 27, 2021 at 5:16 PM Kanchan Kauthale < kanchankauthal...@gmail.com> wrote: > Hello, > > We have

[apache spark] Does Spark 2.4.8 have issues with ServletContextHandler

Hello, We have an existing project which works fine with Spark 2.4.7. We want to upgrade the spark version to 2.4.8. Scala version we are using is- 2.11 After building with upgraded pom, we are getting error below for test cases- java.lang.NoClassDefFoundError: Could not initialize class org.spar

[PySpark][apache-spark]: Pass Pyspark SparkSession instance to Scala

SparkSession does not have such a method, the problem must be that I have to somehow wrap SparkSession. - I did find references (https://www.py4j.org/advanced_topics.html#accessing-java-collections-and-arrays-from-python) on how to wrap/unwrap Python objects to and from Java. But SparSession isn'

Re: Automated setup of a multi-node cluster for Apache Spark

> 1. Writing scripts for automated setup of a multi-node cluster for Apache Spark with Hadoop File System (HDFS). This is required since I don’t have a fixed set of machines to run my Spark experiments and hence, need an easy, quick and automated way to do the entire Spark setup. Where will

Automated setup of a multi-node cluster for Apache Spark

Hello I am new to Apache Spark and am looking for some close guidance or collaboration for my Spark Project which has the following main components: 1. Writing scripts for automated setup of a multi-node cluster for Apache Spark with Hadoop File System (HDFS). This is required since I don’t

Evaluating Apache Spark with Data Orchestration using TPC-DS

Dear Spark Users, I am sharing a whitepaper on “Evaluating Apache Spark and Alluxio for Data Analytics <https://bit.ly/2Pg2jms>” which talks about how to benchmark Spark on Alluxio to accelerate TPCDS benchmark results with details. Hope this helps. If you have any questions, feel free to

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

Congrats and thanks! From: Hyukjin Kwon Sent: Wednesday, March 3, 2021 4:09:23 PM To: Dongjoon Hyun Cc: Gabor Somogyi ; Jungtaek Lim ; angers zhu ; Wenchen Fan ; Kent Yao ; Takeshi Yamamuro ; dev ; user @spark Subject: Re: [ANNOUNCE] Announcing Apache Spark

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

3日周三 下午5:02写道： >>>> >>>>> Great work and congrats! >>>>> >>>>> On Wed, Mar 3, 2021 at 3:51 PM Kent Yao wrote: >>>>> >>>>>> Congrats, all! >>>>>> >>>>>> Bests, >>>>>>

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

zhou Research Institute, NetEase Corp. >>>>> *a spark enthusiast* >>>>> *kyuubi <https://github.com/yaooqinn/kyuubi>is a >>>>> unified multi-tenant JDBC interface for large-scale data processing and >>>>> analytics, built on top of Apac

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

tute, NetEase Corp. >> *a spark enthusiast* >> *kyuubi <https://github.com/yaooqinn/kyuubi>is a >> unified multi-tenant JDBC interface for large-scale data processing and >> analytics, built on top of Apache Spark <http://spark.apache.org/>.* >> *spark-aut

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

>>> *Kent Yao * >>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp. >>>> *a spark enthusiast* >>>> *kyuubi <https://github.com/yaooqinn/kyuubi>is a >>>> unified multi-tenant JDBC interface for large-scale data processi

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

s://github.com/yaooqinn/kyuubi>is a >>> unified multi-tenant JDBC interface for large-scale data processing and >>> analytics, built on top of Apache Spark <http://spark.apache.org/>.* >>> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spar

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

lti-tenant JDBC > interface for large-scale data processing and analytics, built on top > of Apache Spark <http://spark.apache.org/>.* > *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark > SQL extension which provides SQL Standard Authorization for **A

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.spark-func-extrasA library that brings excellent and useful functions from various modern database management systems to Apache Spark. On 03/3/2021 15:11，Takeshi Yamamuro wrote： Great work and

AW: [ANNOUNCE] Announcing Apache Spark 3.1.1

Congrats! Von: Hyukjin Kwon Gesendet: Mittwoch, 3. März 2021 02:41 An: user @spark ; dev Betreff: [ANNOUNCE] Announcing Apache Spark 3.1.1 We are excited to announce Spark 3.1.1 today. Apache Spark 3.1.1 is the second release of the 3.x line. This release adds Python type annotations and

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

On Tue, Mar 2, 2021 at 8:54 PM Yuming Wang wrote: >> >>> Great work, Hyukjin! >>> >>> On Wed, Mar 3, 2021 at 9:50 AM Hyukjin Kwon wrote: >>> >>>> We are excited to announce Spark 3.1.1 today. >>>> >>>> Apache Spark 3.1.1

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

t; >> On Wed, Mar 3, 2021 at 9:50 AM Hyukjin Kwon wrote: >> >>> We are excited to announce Spark 3.1.1 today. >>> >>> Apache Spark 3.1.1 is the second release of the 3.x line. This release >>> adds >>> Python type annotations and Python depen

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

Thanks Hyukjin and congratulations everyone on the release ! Regards, Mridul On Tue, Mar 2, 2021 at 8:54 PM Yuming Wang wrote: > Great work, Hyukjin! > > On Wed, Mar 3, 2021 at 9:50 AM Hyukjin Kwon wrote: > >> We are excited to announce Spark 3.1.1 today. >> >

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

Great work, Hyukjin! On Wed, Mar 3, 2021 at 9:50 AM Hyukjin Kwon wrote: > We are excited to announce Spark 3.1.1 today. > > Apache Spark 3.1.1 is the second release of the 3.x line. This release adds > Python type annotations and Python dependency management support as part >

[ANNOUNCE] Announcing Apache Spark 3.1.1

We are excited to announce Spark 3.1.1 today. Apache Spark 3.1.1 is the second release of the 3.x line. This release adds Python type annotations and Python dependency management support as part of Project Zen. Other major updates include improved ANSI SQL compliance support, history server

[ANNOUNCE] Announcing Apache Spark 3.0.2

We are happy to announce the availability of Spark 3.0.2! Spark 3.0.2 is a maintenance release containing stability fixes. This release is based on the branch-3.0 maintenance branch of Spark. We strongly recommend all 3.0 users to upgrade to this stable release. To download Spark 3.0.2, head over

Announcing Hyperspace v0.4.0 - an indexing subsystem for Apache Spark™

Hi, We are happy to announce that Hyperspace v0.4.0 - an indexing subsystem for Apache Spark™ - has been released <https://github.com/microsoft/hyperspace/releases/tag/v0.4.0>! Here are the some of the highlights: - Delta Lake support: Hyperspace v0.4.0 supports creating indexes on

RE: Apache Spark

: +79103801534 e-mail: avs...@mts.ru<mailto:avs...@mts.ru> г. Нижний Новгород, пр. Гагарина, д. 168А, пом. П8, 3, 310 [cid:image003.png@01D6F473.EBD3A1E0] From: Maxim Gekk Sent: Tuesday, January 26, 2021 10:08 PM To: Lalwani, Jayesh Cc: Синий Андрей ; user@spark.apache.org Subject: Re: Apache

Re: Apache Spark

spark.apache.org" > *Subject: *[EXTERNAL] Apache Spark > > > > *CAUTION*: This email originated from outside of the organization. Do not > click links or open attachments unless you can confirm the sender and know > the content is safe. > > > > Hello! > > >

Re: Apache Spark

All of the major cloud vendors have some sort of Spark offering. They provide support if you build in their cloud. From: Синий Андрей Date: Tuesday, January 26, 2021 at 7:52 AM To: "user@spark.apache.org" Subject: [EXTERNAL] Apache Spark CAUTION: This email originated from outs

Re: Apache Spark

Hello Andrey, you can try to reach Beeline beeline.ru, they use Databricks as far as I know. вт, 26 янв. 2021 г. в 15:01, Sean Owen : > To clarify: Apache projects and the ASF do not provide paid support. > However there are many vendors who provide distributions of Apache Spark >

Re: Apache Spark

To clarify: Apache projects and the ASF do not provide paid support. However there are many vendors who provide distributions of Apache Spark who will provide technical support - not nearly just Databricks but Cloudera, etc. There are also plenty of consultancies and individuals who can provide

Re: Apache Spark

you if your > name would be written in English. > > On 21/01/26 12:41PM, Синий Андрей wrote: > > Hello! > > > > We plan to use Apache Spark software in our organization, can I purchase > paid technical support for this software? > > > > > > > &

Re: Apache Spark

Hi Andrey, It looks like you may contact Databricks for that. Also it would be easier for non-russian spaekers to respond you if your name would be written in English. On 21/01/26 12:41PM, Синий Андрей wrote: > Hello! > > We plan to use Apache Spark software in our organization, can I

Apache Spark

Hello! We plan to use Apache Spark software in our organization, can I purchase paid technical support for this software? С уважением, Андрей Синий Руководитель направления Центр управления программным обеспечением Филиал ПАО «МТС» в Нижегородской области Публичное акционерное общество

Apache Spark is left out by Airbus

Looking at the Big Picture  https://backbutton.co.uk/about.html   This guy gives his reasons for choosing Flink over Spark. https://youtu.be/sYlbD_OoHhs   Airbus makes more of the sky with Flink - Jesse Anderson & Hassene Ben Salem   Is he leading people up the wrong garden path by making a

S3-SQS vs Auto Loader With Apache Spark Structured Streaming

source available in Apache Spark? Do we need to use apache Bahir's SQS implementation  https://github.com/apache/bahir/tree/master/sql-streaming-sqs - Auto Loader: This article recommends that we should use Auto Loader. Is Auto Loader available from Apache Spark  https://docs.databricks.com/s

RE: [EXTERNAL] Announcing Hyperspace v0.3.0 - an indexing subsystem for Apache Spark™

ry Kim Sent: Tuesday, November 17, 2020 4:46 PM To: User Subject: [EXTERNAL] Announcing Hyperspace v0.3.0 - an indexing subsystem for Apache Spark™ Hi, We are happy to announce that Hyperspace v0.3.0 - an indexing subsystem for Apache Spark™ - has been just released<http

Announcing Hyperspace v0.3.0 - an indexing subsystem for Apache Spark™

Hi, We are happy to announce that Hyperspace v0.3.0 - an indexing subsystem for Apache Spark™ - has been just released <https://github.com/microsoft/hyperspace/releases/tag/v0.3.0>! Here are the some of the highlights: - Mutable dataset support: Hyperspace v0.3.0 supports mutable d

Announcing .NET for Apache Spark™ 1.0

Hi, We are happy to announce that .NET for Apache Spark™ v1.0 has been released <https://github.com/dotnet/spark/releases/tag/v1.0.0>! Please check out the official blog <https://cloudblogs.microsoft.com/opensource/2020/10/30/announcing-net-apache-spark-1/>. The release note <htt

Re: Apache Spark Connector for SQL Server and Azure SQL

The best option certainly would be to recompile the Spark Connector for MS SQL server using the Spark 3.0.1/Scala 2.12 dependencies, and just fix the compiler errors as you go. The code is open source on github (https://github.com/microsoft/sql-spark-connector).  Looks like this connector is us

Re: Apache Spark Connector for SQL Server and Azure SQL

I would suggest to ask microsoft and databricks, this forum is for apache spark. if you are interested please drop me a note separately as I m keen to understand the issue as we use same setup Ayan On Mon, 26 Oct 2020 at 11:53 pm, wrote: > Hi, > > > > In a project where I work

Apache Spark Connector for SQL Server and Azure SQL

Hi, In a project where I work with Databricks, we use this connector to read / write data to Azure SQL Database. Currently with Spark 2.4.5 and Scala 2.11. But those setups are getting old. What happens if we update Spark to 3.0.1 or higher and Scala 2.12. This connector does not work according

Organize an Meetup of Apache Spark

Hi, I would like to organize a meetup of Apache Spark in Lima, Peru. I love share with all the community. Regards, Raúl Saráchaga

Re: [UPDATE] Apache Spark 3.1.0 Release Window

I think you put Jan 2020 instead of 2021 :-) Le mar. 13 oct. 2020 à 00:51, Xiao Li a écrit : > Thank you, Dongjoon > > Xiao > > On Mon, Oct 12, 2020 at 4:19 PM Dongjoon Hyun > wrote: > >> Hi, All. >> >> Apache Spark 3.1.0 Release Window is adjusted like t

Re: [UPDATE] Apache Spark 3.1.0 Release Window

Thank you, Dongjoon Xiao On Mon, Oct 12, 2020 at 4:19 PM Dongjoon Hyun wrote: > Hi, All. > > Apache Spark 3.1.0 Release Window is adjusted like the following today. > Please check the latest information on the official website. > > - > https://github.com/apache

[UPDATE] Apache Spark 3.1.0 Release Window

Hi, All. Apache Spark 3.1.0 Release Window is adjusted like the following today. Please check the latest information on the official website. - https://github.com/apache/spark-website/commit/0cd0bdc80503882b4737db7e77cc8f9d17ec12ca - https://spark.apache.org/versioning-policy.html Bests

Re: Apache Spark 3.1 Preparation Status (Oct. 2020)

Thank you so much for your feedback, Koert. Yes, SPARK-20202 was created in April 2017 and targeted for 3.1.0 since Nov 2019. However, I believe Apache Spark 3.1.0 (Hadoop 3.2/Hive 2.3 distribution) will work with old Hadoop 2.x clusters if you isolated the classpath via SPARK-31960. SPARK

Re: Apache Spark 3.1 Preparation Status (Oct. 2020)

: > Hi, All. > > As of today, master branch (Apache Spark 3.1.0) resolved > 852+ JIRA issues and 606+ issues are 3.1.0-only patches. > According to the 3.1.0 release window, branch-3.1 will be > created on November 1st and enters QA period. > > Here are some notable upda

[apache-spark] [spark-r] 503 Error - Cannot Connect to S3

0(FileSystem.java:91) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2632) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2614) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)

Re: Apache Spark 3.1 Preparation Status (Oct. 2020)

gt; to the changed release cadence the code freeze should happen in > mid-November. > > On Sun, Oct 4, 2020 at 6:26 PM Xiao Li wrote: > >> Apache Spark 3.1.0 should be compared with Apache Spark 2.1.0. >> >> >> I think we made a change in release cadence si

Re: Apache Spark 3.1 Preparation Status (Oct. 2020)

> > Apache Spark 3.1.0 should be compared with Apache Spark 2.1.0. I think we made a change in release cadence since Spark 2.3. See the commit: https://github.com/apache/spark-website/commit/88990968962e5cc47db8bc2c11a50742d2438daa Thus, Spark 3.1 might just follow the release cadence of

Re: Apache Spark 3.1 Preparation Status (Oct. 2020)

For Xiao's comment, I want to point out that Apache Spark 3.1.0 is different from 2.3 or 2.4. Apache Spark 3.1.0 should be compared with Apache Spark 2.1.0. - Apache Spark 2.0.0 was released on July 26, 2016. - Apache Spark 2.1.0 was released on December 28, 2016. Bests, Dongjoon. On Sun

Re: Apache Spark 3.1 Preparation Status (Oct. 2020)

Thank you all. BTW, Xiao and Mridul, I'm wondering what date you have in your mind specifically. Usually, `Christmas and New Year season` doesn't give us much additional time. If you think so, could you make a PR for Apache Spark website according to your expectation? https://spark.

Re: Apache Spark 3.1 Preparation Status (Oct. 2020)

Hyukjin Kwon 于2020年10月3日周六 下午5:41写道： > >> Nice summary. Thanks Dongjoon. One minor correction -> I believe we >> dropped R 3.5 and below at branch 2.4 as well. >> >> On Sun, 4 Oct 2020, 09:17 Dongjoon Hyun, wrote: >> >>> Hi, All. >>> >

Re: Apache Spark 3.1 Preparation Status (Oct. 2020)

, All. >> >> As of today, master branch (Apache Spark 3.1.0) resolved >> 852+ JIRA issues and 606+ issues are 3.1.0-only patches. >> According to the 3.1.0 release window, branch-3.1 will be >> created on November 1st and enters QA period. >> >> Here are some n

Re: Apache Spark 3.1 Preparation Status (Oct. 2020)

Nice summary. Thanks Dongjoon. One minor correction -> I believe we dropped R 3.5 and below at branch 2.4 as well. On Sun, 4 Oct 2020, 09:17 Dongjoon Hyun, wrote: > Hi, All. > > As of today, master branch (Apache Spark 3.1.0) resolved > 852+ JIRA issues and 606+ issues are 3.1

Apache Spark 3.1 Preparation Status (Oct. 2020)

Hi, All. As of today, master branch (Apache Spark 3.1.0) resolved 852+ JIRA issues and 606+ issues are 3.1.0-only patches. According to the 3.1.0 release window, branch-3.1 will be created on November 1st and enters QA period. Here are some notable updates I've been monitoring. *Languag

Re: Apache Spark Bogotá Meetup

Cool here is my PR 🤞 https://github.com/apache/spark-website/pull/291 On Wed, 30 Sep 2020 at 07:34, Sean Owen wrote: > Sure, we just ask people to open a pull request against > https://github.com/apache/spark-website to update the page and we can > merge it. > > On Wed, Sep 3

Re: Apache Spark Bogotá Meetup

Sure, we just ask people to open a pull request against https://github.com/apache/spark-website to update the page and we can merge it. On Wed, Sep 30, 2020 at 7:30 AM Miguel Angel Díaz Rodríguez < madiaz...@gmail.com> wrote: > Hello > > I am Co-organizer of Apache Spark Bog

Apache Spark Bogotá Meetup

Hello I am Co-organizer of Apache Spark Bogotá Meetup from Colombia https://www.meetup.com/es/Apache-Spark-Bogota/ And would like to include the community on the following web page. https://spark.apache.org/community.html Looking forward to meeting you Miguel.

Apache Spark Error.

I am using Apache Prediction. Prediction addicted to spark. I have never worked with Java or Scala. I'm getting an error and don't know how to fix it. Do you have any opinion on this topic? Full log: https://paste.ubuntu.com/p/zyzP9BGRXt/ My Java version: openjdk version "1.8.0_265" OpenJDK Runt

Re: [ANNOUNCE] Announcing Apache Spark 3.0.1

Great work, thanks, Ruifeng! On Fri, Sep 11, 2020 at 11:09 PM Gengliang Wang < gengliang.w...@databricks.com> wrote: > Congrats! > Thanks for the work, Ruifeng! > > > On Fri, Sep 11, 2020 at 9:51 PM Takeshi Yamamuro > wrote: > >> Congrats and thanks, Ruifeng! >> >> >> On Fri, Sep 11, 2020 at 9:5

Re: [ANNOUNCE] Announcing Apache Spark 3.0.1

Congrats! Thanks for the work, Ruifeng! On Fri, Sep 11, 2020 at 9:51 PM Takeshi Yamamuro wrote: > Congrats and thanks, Ruifeng! > > > On Fri, Sep 11, 2020 at 9:50 PM Dongjoon Hyun > wrote: > >> It's great. Thank you, Ruifeng! >> >> Bests, >> Dongjoon. >> >> On Fri, Sep 11, 2020 at 1:54 AM 郑瑞峰

Re: [ANNOUNCE] Announcing Apache Spark 3.0.1

Congrats and thanks, Ruifeng! On Fri, Sep 11, 2020 at 9:50 PM Dongjoon Hyun wrote: > It's great. Thank you, Ruifeng! > > Bests, > Dongjoon. > > On Fri, Sep 11, 2020 at 1:54 AM 郑瑞峰 wrote: > >> Hi all, >> >> We are happy to announce the availability of Spark 3.0.1! >> Spark 3.0.1 is a maintenanc

Re: [ANNOUNCE] Announcing Apache Spark 3.0.1

It's great. Thank you, Ruifeng! Bests, Dongjoon. On Fri, Sep 11, 2020 at 1:54 AM 郑瑞峰 wrote: > Hi all, > > We are happy to announce the availability of Spark 3.0.1! > Spark 3.0.1 is a maintenance release containing stability fixes. This > release is based on the branch-3.0 maintenance branch of

[ANNOUNCE] Announcing Apache Spark 3.0.1

Hi all, We are happy to announce the availability of Spark 3.0.1! Spark 3.0.1 is a maintenance release containing stability fixes. This release is based on the branch-3.0 maintenance branch of Spark. We strongly recommend all 3.0 users to upgrade to this stable release. To download Spark 3.0.

Kotlin for Apache Spark 1.0.0-preview released

Hi folks! Happy to share with you that we have released the first preview of Kotlin for Apache Spark. You can learn about the features, supported versions, and roadmap in the blog post: https://blog.jetbrains.com/kotlin/2020/08/introducing-kotlin-for-apache-spark-preview/ Have a great day!

Re: CVE-2020-9480: Apache Spark RCE vulnerability in auth-enabled standalone master

3.0.0+. For those using vendor distros, you may want to check with your vendor about whether the relevant patch has been applied. Sean On Mon, Jun 22, 2020 at 4:49 PM Sean Owen wrote: > > Severity: Important > > Vendor: The Apache Software Foundation > > Versions Affected: &

Re: Apache Spark- Help with email library

the way I am > <https://edu.51cto.com/sd/aca72> ><https://github.com/TianLangStudio> > > ------ > 发件人：sn.noufal > 发送时间：2020年7月27日(星期一) 08:01 > 收件人：user > 主 题：Apache Spark- Help with email library > > Hi, > > I am lookin

Apache Spark + Python + Pyspark + Kaola

gets 3 alerts then show me the result) 5. All the coding should be in python The ideas is something like this, my question is how should I proceed to this task. What are the technologies that I should use? *Apache Spark + Python + Pyspark + Kaola *can handle this ? Best regards, *Suat

回复：Apache Spark- Help with email library

I am -- 发件人：sn.noufal 发送时间：2020年7月27日(星期一) 08:01 收件人：user 主　题：Apache Spark- Help with email library Hi, I am looking to send a dataframe as email.How do I do that? Do you have any library with sample.Appreciate your response Re

Apache Spark- Help with email library

Hi, I am looking to send a dataframe as email.How do I do that? Do you have any library with sample.Appreciate your response Regards, Mohamed - To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Blog : Apache Spark Window Functions

ng tons of money for companies" then the same three weeks spent on that course will yield greater value comparatively spent on writing Apache spark examples of the type you are currently engaged in. I have gone past week 3 😎🤐so I have the insight. It is against my personal values to use a pr

Re: Blog : Apache Spark Window Functions

wish somebody had pointed me there as the starting point. On Thu, 25 Jun 2020, 18:58 neeraj bhadani, wrote: > Hi Team, > I would like to share with the community that my blog on "Apache > Spark Window Functions" got published. PFB link if anyone interested. > > L

Announcing .NET for Apache Spark™ 0.12

We are happy to announce that .NET for Apache Spark™ v0.12 has been released <https://github.com/dotnet/spark/releases>! Thanks to the community for the great feedback. The release note <https://github.com/dotnet/spark/blob/master/docs/release-notes/0.12/release-0.12.md> includes the

Re: Running Apache Spark Streaming on the GraalVM Native Image

On 20/07/01 09:56AM, ivo.kn...@t-online.de wrote: > Hi guys, > > so I want to get Apache Spark to run on the GraalVM Native Image in a > simple single-node streaming application, but I get the following error, > when trying to build the native image: (check attached file) >

Running Apache Spark Streaming on the GraalVM Native Image

Hi guys, so I want to get Apache Spark to run on the GraalVM Native Image in a simple single-node streaming application, but I get the following error, when trying to build the native image: (check attached file) And as I researched online, there seems to be no successful combination of

Apache Spark Meetup - Wednesday 1st July

Good morning, I hope this email finds you well. I am the host for an on-going series of live webinars/virtual meetups and the next 2 weeks are focused on Apache Spark, I was wondering if you could share within your group? It’s free to sign up and there will be live Q&A throughout

Re: apache-spark mongodb dataframe issue

mentioned. I am doing RnD on another approach, will post here if there is a breakthrough. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Blog : Apache Spark Window Functions

Hi Team, I would like to share with the community that my blog on "Apache Spark Window Functions" got published. PFB link if anyone interested. Link: https://medium.com/expedia-group-tech/deep-dive-into-apache-spark-window-functions-7b4e39ad3c86 Please share your thoughts an

Re: apache-spark mongodb dataframe issue

As far as I know, in general, there isn't a way to distinguish explicit null values from missing ones. (Someone please correct me if I'm wrong, since I would love to be able to do this for my own reasons). If you really must do it, and don't care about performance at all (since it will be horribl

apache-spark mongodb dataframe issue

Hi Please look at my issue from the link below. https://stackoverflow.com/questions/62526118/how-to-differentiate-between-null-and-missing-mongogdb-values-in-a-spark-datafra Kindly Help Best Mannat

CVE-2020-9480: Apache Spark RCE vulnerability in auth-enabled standalone master

Severity: Important Vendor: The Apache Software Foundation Versions Affected: Apache Spark 2.4.5 and earlier Description: In Apache Spark 2.4.5 and earlier, a standalone resource manager's master may be configured to require authentication (spark.authenticate) via a shared secret. When en

Reg - Why Apache Hadoop need to be Installed separately for Running Apache Spark…?

I'm learning Apache Spark, where I'm trying to run a basic Spark Program written in Java. I've installed Apache Spark *(spark-2.4.3-bin-without-hadoop)* downloaded from https://spark.apache.org/ . I've created a maven project in eclipse and added the

Re: [ANNOUNCE] Apache Spark 3.0.0

; r...@databricks.com>; dev ; user < > user@spark.apache.org> > *Subject:* Re: [ANNOUNCE] Apache Spark 3.0.0 > > Great, thanks all for your efforts on the huge step forward! > > On Fri, Jun 19, 2020 at 12:13 PM Hyukjin Kwon wrote: > > Yay! > > 2020년 6월

Re: [ANNOUNCE] Apache Spark 3.0.0

Congrats From: Jungtaek Lim Sent: Thursday, June 18, 2020 8:18:54 PM To: Hyukjin Kwon Cc: Mridul Muralidharan ; Reynold Xin ; dev ; user Subject: Re: [ANNOUNCE] Apache Spark 3.0.0 Great, thanks all for your efforts on the huge step forward! On Fri, Jun 19

Re: [ANNOUNCE] Apache Spark 3.0.0

gt;> On Thu, Jun 18, 2020 at 10:21 AM Reynold Xin wrote: >> >>> Hi all, >>> >>> Apache Spark 3.0.0 is the first release of the 3.x line. It builds on >>> many of the innovations from Spark 2.x, bringing new ideas as well as >>> continuing

Re: [ANNOUNCE] Apache Spark 3.0.0

Yay! 2020년 6월 19일 (금) 오전 4:46, Mridul Muralidharan 님이 작성: > Great job everyone ! Congratulations :-) > > Regards, > Mridul > > On Thu, Jun 18, 2020 at 10:21 AM Reynold Xin wrote: > >> Hi all, >> >> Apache Spark 3.0.0 is the first release of the 3.x line.

Re: [ANNOUNCE] Apache Spark 3.0.0

Great job everyone ! Congratulations :-) Regards, Mridul On Thu, Jun 18, 2020 at 10:21 AM Reynold Xin wrote: > Hi all, > > Apache Spark 3.0.0 is the first release of the 3.x line. It builds on many > of the innovations from Spark 2.x, bringing new ideas as well as continuing

Re: [ANNOUNCE] Apache Spark 3.0.0

Congratulations 🥳 Celebrating 🥳 Sent from my iPhone > On 18 Jun 2020, at 20:38, Gourav Sengupta wrote: > >  > CELEBRATIONS!!! > >> On Thu, Jun 18, 2020 at 6:21 PM Reynold Xin wrote: >> Hi all, >> >> Apache Spark 3.0.0 is the first rel

<    1   2   3   4   5   6   7   8   9   10   >