pache.org/docs/3.1.1/...;
Regards,
Kazuaki Ishizaki,
From: Hyukjin Kwon
To: dev
Cc: Dongjoon Hyun , Jungtaek Lim
, Tom Graves
Date: 2021/03/02 11:20
Subject:Re: Please take a look at the draft of the Spark 3.1.1
release notes
Thanks guys for suggestions and fixes. Now I feel pr
Java 16 will also includes Vector API (incubator), which is a part of
Project Panama, as shown in
https://mail.openjdk.java.net/pipermail/panama-dev/2020-October/011149.html
When the next LTS will be available, we could exploit it in Spark.
Kazuaki Ishizaki
From: Dongjoon Hyun
+1 if we add them to Alternative config.
Kazuaki Ishizaki
From: Takeshi Yamamuro
To: Wenchen Fan
Cc: Spark dev list
Date: 2020/02/13 16:02
Subject:[EXTERNAL] Re: [DISCUSS] naming policy of Spark configs
+1; the idea sounds reasonable.
Bests,
Takeshi
On Thu, Feb 13
.
Kazuaki Ishizaki
Congrats! Well deserved.
Kazuaki Ishizaki,
From: Matei Zaharia
To: dev
Date: 2019/09/10 09:32
Subject:[EXTERNAL] Welcoming some new committers and PMC members
Hi all,
The Spark PMC recently voted to add several new committers and one PMC
member. Join me in welcoming them
*
+0: None
-1: None
Regards,
Kazuaki Ishizaki
+1
Built and tested with `mvn -Pyarn -Phadoop-2.7 -Pkubernetes -Pkinesis-asl
-Phive -Phive-thriftserver test` on OpenJDK 1.8.0_211 on Ubuntu 16.04
x86_64
Regards,
Kazuaki Ishizaki
From: Dongjoon Hyun
To: dev
Date: 2019/08/28 12:14
Subject:[EXTERNAL] Re: [VOTE] Release
Thank you for pointing out the problem.
The characters and hyperlink point different URLs.
Could you please access
https://repository.apache.org/content/repositories/orgapachespark-1331/ as
you see characters?
Sorry for your inconvenience.
Kazuaki Ishizaki,
From: Takeshi Yamamuro
Please vote on releasing the following candidate as Apache Spark version
2.3.4.
The vote is open until August 29th 2PM PST and passes if a majority +1 PMC
votes are cast, with
a minimum of 3 +1 votes.
[ ] +1 Release this package as Apache Spark 2.3.4
[ ] -1 Do not release this package because
ext
Monday.
Regards,
Kazuaki Ishizaki
From: "Kazuaki Ishizaki"
To: "Kazuaki Ishizaki"
Cc: Dilip Biswal , dev ,
Hyukjin Kwon , jzh...@apache.org, Takeshi Yamamuro
, Xiao Li
Date: 2019/08/20 13:12
Subject:[EXTERNAL] RE: Release Spark 2.3.4
Due to the r
Due to the recent correctness issue at SPARK-28699, I will delay the
release for Spark 2.3.4 RC1 for a while.
https://issues.apache.org/jira/browse/SPARK-28699?focusedCommentId=16910859=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16910859
Regards,
Kazuaki Ishizaki
Hi all,
Thank you. I will prepare RC for 2.3.4 this week in parallel. It will be
in parallel with RC for 2.4.4 managed by Dongjoon.
Regards,
Kazuaki Ishizaki
From: Hyukjin Kwon
To: Dilip Biswal
Cc: jzh...@apache.org, dev , Kazuaki Ishizaki
, Takeshi Yamamuro , Xiao Li
Date
)%20ORDER%20BY%20priority%20DESC%2C%20key%20ASC
Best Regards,
Kazuaki Ishizaki
Sure, I will launch a separate e-mail thread for discussing 2.3.4 later.
Regards,
Kazuaki Ishizaki, Ph.D.
From: Dongjoon Hyun
To: Sean Owen , Kazuaki Ishizaki
Cc: dev
Date: 2019/08/16 05:10
Subject:[EXTERNAL] Re: Release Apache Spark 2.4.4
+1 for that.
Kazuaki
Thanks, Dongjoon!
+1
Kazuaki Ishizaki,
From: Hyukjin Kwon
To: Takeshi Yamamuro
Cc: Dongjoon Hyun , dev
, User
Date: 2019/08/14 09:21
Subject:[EXTERNAL] Re: Release Apache Spark 2.4.4
+1
2019년 8월 14일 (수) 오전 9:13, Takeshi Yamamuro 님
이 작성:
Hi,
Thanks for your
Thank you Dongjoon for being a release manager.
If the assumed dates are ok, I would like to volunteer for an 2.3.4
release manager.
Best Regards,
Kazuaki Ishizaki,
From: Dongjoon Hyun
To: dev , "user @spark" ,
Apache Spark PMC
Date: 2019/07/13 07:18
Subject:[EX
+1 (non-binding)
Kazuaki Ishizaki
From: Bryan Cutler
To: Bobby Evans
Cc: Thomas graves , Spark dev list
Date: 2019/05/09 03:20
Subject:Re: [VOTE][SPARK-27396] SPIP: Public APIs for extended
Columnar Processing Support
+1 (non-binding)
On Tue, May 7, 2019 at 12:04
that it uses Python UDF support with Arrow in Spark.
P.S. I will give a presentation about in-memory data storages for SPark at
SAIS 2019
https://databricks.com/sparkaisummit/north-america/sessions-single-2019?id=40
:)
Kazuaki Ishizaki
From: Wenchen Fan
To: Bobby Evans
Cc: Spark dev
Congratulations, Jose!
Kazuaki Ishizaki
From: Gengliang Wang
To: dev
Date: 2019/01/31 18:32
Subject:Re: Welcome Jose Torres as a Spark committer
Congrats Jose!
在 2019年1月31日，上午6:51，Bryan Cutler 写道：
Congrats Jose!
On Tue, Jan 29, 2019, 10:48 AM Shixiong Zhu
forward.
Kazuaki Ishizaki,
From: "Kazuaki Ishizaki"
To: Reynold Xin
Cc: dev , Takeshi Yamamuro
, Xiao Li
Date: 2018/10/31 00:56
Subject:Re: SPIP: SPARK-25728 Structured Intermediate
Representation (Tungsten IR) for generating Java code
Hi Reynold,
Thank yo
This entry includes a good figure for support lifecycle.
https://www.azul.com/products/zulu-and-zulu-enterprise/zulu-enterprise-java-support-options/
Kazuaki Ishizaki,
From: Marcelo Vanzin
To: Felix Cheung
Cc: Ryan Blue , sn...@snazy.de, dev
, Cesar Delgado
Date: 2018/11/07 08
" / " + "c").
It is important for us to learn from failures than learn from successes.
We would appreciate it if you could list up failures that you have seen.
Best Regards,
Kazuaki Ishizaki
From: Reynold Xin
To: Kazuaki Ishizaki
Cc: Xiao Li , dev ,
Takeshi Yam
Hi Xiao,
Thank you very much for becoming a shepherd.
If you feel the discussion settles, we would appreciate it if you would
start a voting.
Regards,
Kazuaki Ishizaki
From: Xiao Li
To: Kazuaki Ishizaki
Cc: dev , Takeshi Yamamuro
Date: 2018/10/22 16:31
Subject:Re: SPIP
on this SPIP procedure
http://spark.apache.org/improvement-proposals.html, can I ask one or more
PMCs to become a shepherd of this SPIP?
I would appreciate your kindness and cooperation.
Best Regards,
Kazuaki Ishizaki
From: Takeshi Yamamuro
To: Spark dev list
Cc: ishiz...@jp.ibm.com
Date
/1Jzf56bxpMpSwsGV_hSzl9wQG22hyI731McQcjognqxY/edit?usp=sharing
Looking forward to hear your feedback
Best Regards,
Kazuaki Ishizaki
Of course, we would like to eliminate all of the following tags
"flanky" or "flankytest"
Kazuaki Ishizaki
From: Hyukjin Kwon
To: dev
Cc: Xiao Li , Wenchen Fan
Date: 2018/09/04 14:20
Subject:Re: Spark JIRA tags clarification and management
Thank
value generated by Spark
with it generated by other implementations?
Regards,
Kazuaki Ishizaki
From: Sean Owen
To: jiayuanm
Cc: dev@spark.apache.org
Date: 2018/07/07 15:46
Subject:Re: [SPARK ML] Minhash integer overflow
I think it probably still does its.job; the hash
Thank for you reporting this issue. I think this is a bug regarding
integer overflow. IMHO, it would be good to compute hashes with Long.
Would it be possible to create a JIRA entry? Do you want to submit a pull
request, too?
Regards,
Kazuaki Ishizaki
From: jiayuanm
To: dev
are appreciated.
Regards,
Kazuaki Ishizaki
From: Joseph Bradley <jos...@databricks.com>
To: dev <dev@spark.apache.org>
Cc: Hossein Falaki <hoss...@databricks.com>
Date: 2018/05/03 07:31
Subject:SparkR test failures in PR builder
Hi all,
Does anyo
Congratulations to Zhenhua!
Kazuaki Ishizaki
From: sujith chacko <sujithchacko.2...@gmail.com>
To: Denny Lee <denny.g@gmail.com>
Cc: Spark dev list <dev@spark.apache.org>, Wenchen Fan
<cloud0...@gmail.com>, "叶先进" <advance...@gmail.com>
Dat
Congratulations to everyone!
Kazuaki Ishizaki
From: Takeshi Yamamuro <linguin@gmail.com>
To: Spark dev list <dev@spark.apache.org>
Date: 2018/03/03 10:45
Subject:Re: Welcoming some new committers
Congrats, all!
On Sat, Mar 3, 2018 at 10:34 AM, Takuya
updated the JIRA ticket.
I am still working for this.
Kazuaki Ishizaki
From: Herman van Hövell tot Westerflier <hvanhov...@databricks.com>
To: Kazuaki Ishizaki <ishiz...@jp.ibm.com>
Cc: Jacek Laskowski <ja...@japila.pl>, dev <dev@spark.apache.org>
Date:
produces
incorrect results. I created a JIRA entry
https://issues.apache.org/jira/browse/SPARK-22934.
Best Regards,
Kazuaki Ishizaki
From: Herman van Hövell tot Westerflier <hvanhov...@databricks.com>
To: Jacek Laskowski <ja...@japila.pl>
Cc: dev <dev@spark.apache.or
+1 for cutting a branch earlier.
In some Asian countries, 1st, 2nd, and 3rd January are off.
https://www.timeanddate.com/holidays/
How about 4th or 5th?
Regards,
Kazuaki Ishizaki
From: Felix Cheung <felixcheun...@hotmail.com>
To: Michael Armbrust <mich...@databricks.com>,
04 h
[INFO] Finished at: 2017-11-30T01:48:15+09:00
[INFO] Final Memory: 128M/329M
[INFO]
[WARNING] The requested profile "hive" could not be activated because it
does not exist.
Kazuaki Ishizaki
From: Dongj
min
[INFO] Finished at: 2017-10-03T23:53:33+09:00
[INFO] Final Memory: 112M/322M
[INFO]
[WARNING] The requested profile "hive" could not be activated because it
does not exist.
Kazuaki Ishizaki
From: Dongj
Congratulation Tejas!
Kazuaki Ishizaki
From: Matei Zaharia <matei.zaha...@gmail.com>
To: "dev@spark.apache.org" <dev@spark.apache.org>
Date: 2017/09/30 04:58
Subject:Welcoming Tejas Patil as a Spark committer
Hi all,
The Spark PMC recently added Teja
min
[INFO] Finished at: 2017-09-28T12:16:46+09:00
[INFO] Final Memory: 103M/309M
[INFO]
[WARNING] The requested profile "hive" could not be activated because it
does not exist.
Kazuaki Ishizaki
From: Dongj
Congratulations, Jerry!
Kazuaki Ishizaki
From: Hyukjin Kwon <gurwls...@gmail.com>
To: dev <dev@spark.apache.org>
Date: 2017/08/29 12:24
Subject:Re: Welcoming Saisai (Jerry) Shao as a committer
Congratulations! Very well deserved.
2017-08-29 11:41 GMT+09:00 Li
Congratulation, Hyukjin and Sameer, well deserved!!
Kazuaki Ishizaki
From: Matei Zaharia <matei.zaha...@gmail.com>
To: dev <dev@spark.apache.org>
Date: 2017/08/08 00:53
Subject:Welcoming Hyukjin Kwon and Sameer Agarwal as committers
Hi everyone,
The Spark PMC re
06 h
[INFO] Finished at: 2017-07-01T15:20:04+09:00
[INFO] Final Memory: 56M/591M
[INFO]
[WARNING] The requested profile "hive" could not be activated because it
does not exist.
Kazuaki Ishizaki
From: Mi
6T13:44:48+09:00
[INFO] Final Memory: 53M/510M
[INFO]
[WARNING] The requested profile "hive" could not be activated because it
does not exist.
Kazuaki Ishizaki
From: Michael Armbrust <mich...@datab
It looked well these days. However, it seems to go down slowly again...
When I tried to see console log (e.g.
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77149/consoleFull
), a server returns "proxy error."
Regards,
Kazuaki Ishizaki
From: shane
9T17:51:04+09:00
[INFO] Final Memory: 53M/514M
[INFO]
[WARNING] The requested profile "hive" could not be activated because it
does not exist.
Kazuaki Ishizaki,
From: Michael Armbrust <mich...@datab
9T02:23:08+09:00
[INFO] Final Memory: 53M/491M
[INFO]
----
Kazuaki Ishizaki,
From: Michael Armbrust <mich...@databricks.com>
To: "dev@spark.apache.org" <dev@spark.apache.org>
Date: 2017/04/28 03:
576M
[INFO]
Regards,
Kazuaki Ishizaki,
From: Michael Armbrust <mich...@databricks.com>
To: "dev@spark.apache.org" <dev@spark.apache.org>
Date: 2017/04/27 09:30
Subject:[VOTE] Apache
672M
[INFO]
Regards,
Kazuaki Ishizaki,
From: Michael Armbrust <mich...@databricks.com>
To: "dev@spark.apache.org" <dev@spark.apache.org>
Date: 2017/04/19 04:00
Subject:[VOTE] Apache
Thank you. Yes, it is not a regression. 2.1.0 would have this failure,
too.
Regards,
Kazuaki Ishizaki
From: Sean Owen <so...@cloudera.com>
To: Kazuaki Ishizaki/Japan/IBM@IBMJP, Michael Armbrust
<mich...@databricks.com>
Cc: "dev@spark.apache.org" <dev@spark
0.002 sec <<< ERROR!
java.lang.IllegalArgumentException: requirement failed: No support for
unaligned Unsafe. Set spark.memory.offHeap.enabled to false.
...
Tests run: 207, Failures: 7, Errors: 16, Skipped: 0
Kazuaki Ishizaki
From: Michael Armbrust <mich...@databricks.com
if null exists in non-null column.
Any comments are appreciated.
Kazuaki Ishizaki
From: Jason White <jason.wh...@shopify.com>
To: dev@spark.apache.org
Date: 2017/03/21 06:31
Subject:Why are DataFrames always read with nullable=True?
If I create a dataframe in Spark with non-nu
t;id>10") return correct result.
f(df1).count // output 89 which is incorrect
Regards,
Kazuaki Ishizaki
From: gen tang <gen.tan...@gmail.com>
To: dev@spark.apache.org
Date: 2017/02/22 15:02
Subject:Re: A DataFrame cache bug
Hi All,
I might find a related issue
Congrats!
Kazuaki Ishizaki
From: Reynold Xin <r...@databricks.com>
To: "dev@spark.apache.org" <dev@spark.apache.org>
Date: 2017/02/14 04:18
Subject:welcoming Takuya Ueshin as a new Apache Spark committer
Hi all,
Takuya-san has recently been elected an
Hi,
You may find several micro-benchmarks under
https://github.com/apache/spark/tree/master/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark
.
Regards,
Kazuaki Ishizaki
From: Prasun Ratn <prasun.r...@gmail.com>
To: Apache Spark Dev <dev@spark.apache.org>
Dat
is that both applications supports Apache Arrow
APIs.
Other approaches could be.
What approach would be good for all of applications?
Regards,
Kazuaki Ishizaki
The line where I pointed out would work correctly. This is because a type
of this division is double. d2i correctly handles overflow cases.
Kazuaki Ishizaki
From: Nicholas Chammas <nicholas.cham...@gmail.com>
To: Kazuaki Ishizaki/Japan/IBM@IBMJP, Reynold Xin
<r...@databrick
/spark/util/collection/unsafe/sort/UnsafeInMemorySorter.java#L156
Regards,
Kazuaki Ishizaki
From: Reynold Xin <r...@databricks.com>
To: Nicholas Chammas <nicholas.cham...@gmail.com>
Cc: Spark dev list <dev@spark.apache.org>
Date: 2016/12/07 14:27
Subject:Re: R
pull/11956. This PR shows room to
performance improvement for float/double values that are not compressed.
Kazuaki Ishizaki
From: linguin@gmail.com
To: Maciej Bry��ski <mac...@brynski.pl>
Cc: Spark dev list <dev@spark.apache.org>
Date: 2016/08/28 11:30
Subject
these pull requests.
Best Regards,
Kazuaki Ishizaki
From: Maciej Bryński <mac...@brynski.pl>
To: Spark dev list <dev@spark.apache.org>
Date: 2016/08/28 05:40
Subject:Cache'ing performance
Hi,
I did some benchmark of cache function today.
RDD
sc.parallelize(0 unt
equality of two Rows that have an Array or complex types (e.g. Map)?
Best Regards,
Kazuaki Ishizaki, @kiszk
sues.apache.org/jira/browse/SPARK-14689
https://issues.apache.org/jira/browse/SPARK-14690
Best regards,
Kazuaki Ishizaki
exploit GPUs such as BIDMat in Spark.
If we would run BIDMat with our columnar strorage, the performance boost
would be good as others reported.
Best Regards,
Kazuaki Ishizaki,
From: "Allen Zhang" <allenzhang...@126.com>
To: Kazuaki Ishizaki/Japan/IBM@IBMJP
Cc: "
with current Spark. I expect that this
talk would help us.
http://conferences.oreilly.com/strata/hadoop-big-data-ca/public/schedule/detail/47565
We appreciate your great feedback.
Best Regards,
Kazuaki Ishizaki, Ph.D., Senior research staff member, IBM Research -
Tokyo
From: "Ulanov, Alex
iding
binary column storage for data partition. We really appreciate it if you
would give us comments, suggestions, or feedback.
Best Regards
Kazuaki Ishizaki
From: "Ulanov, Alexander" <alexander.ula...@hpe.com>
To: Sam Halliday <sam.halli...@gmail.com>, John Canny
&
pu/blob/dev/examples/src/main/scala/org/apache/spark/examples/SparkGPULR.scala#L89
(Sorry for no documentation now).
We are glad to support requested features or to looking forward to getting
pull requests.
Best Regard,
Kazuaki Ishizaki
From: "Allen Zhang" <allenzhang...@126.co
I created a new JIRA entry
https://issues.apache.org/jira/browse/SPARK-12620 for this instead of
reopening the existing JIRA based on the suggestion.
Best Regards,
Kazuaki Ishizaki
From: Kazuaki Ishizaki/Japan/IBM@IBMJP
To: dev@spark.apache.org
Date: 2016/01/04 12:54
Subject
great
discussions to exploit GPUs in Spark.
Best Regards,
Kazuaki Ishizaki
This is because to build Spark requires maven 3.3.3 or later.
http://spark.apache.org/docs/latest/building-spark.html
Regards,
Kazuaki Ishizaki
From: salexln <sale...@gmail.com>
To: dev@spark.apache.org
Date: 2015/12/25 15:52
Subject:latest Spark build error
Hi all
Is this JIRA entry related to what you want?
https://issues.apache.org/jira/browse/SPARK-10399
Regards,
Kazuaki Ishizaki
From: Jia <jacqueline...@gmail.com>
To: Dewful <dew...@gmail.com>
Cc: "user @spark" <u...@spark.apache.org>, dev@spark.apac
68 matches
Mail list logo
