Github user dongjinleekr commented on the issue:
https://github.com/apache/spark/pull/16830
Updated with the problem description. Please have a look when you are free.
I will review the other documentation descriptions ...
---
If your project is set up for it, you can reply
GitHub user dongjinleekr opened a pull request:
https://github.com/apache/spark/pull/16830
[MINOR][CORE] Fix incorrect documentation of WritableConverter
## What changes were proposed in this pull request?
`WritableConverter` and `WritableFactory` work in opposite
Github user dongjinleekr commented on the issue:
https://github.com/apache/spark/pull/16830
Is this exactly corresponds to your intention?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user dongjinleekr opened a pull request:
https://github.com/apache/spark/pull/17303
[SPARK-19112][CORE] add codec for ZStandard
## What changes were proposed in this pull request?
Hadoop[^1] & HBase[^2] started to support ZStandard Compression from their
re
Github user dongjinleekr commented on the issue:
https://github.com/apache/spark/pull/17225
@HyukjinKwon Thanks for the information. It seems like both of Jira issue
and my PR are messed up - it will re-create PR with the Jira issue.
---
If your project is set up for it, you can
Github user dongjinleekr closed the pull request at:
https://github.com/apache/spark/pull/17225
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user dongjinleekr commented on the issue:
https://github.com/apache/spark/pull/17564
@srowen I just completed the review. I could not find any other typos from
this document. :)
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user dongjinleekr commented on the issue:
https://github.com/apache/spark/pull/17564
No problem! I will notify you after the review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user dongjinleekr opened a pull request:
https://github.com/apache/spark/pull/17564
[MINOR][DOCS] Fix spacings in Structured Streaming Programming Guide
## What changes were proposed in this pull request?
1. Omitted space between the sentences: `... on static
GitHub user dongjinleekr opened a pull request:
https://github.com/apache/spark/pull/17225
[CORE] Support ZStandard Compression
## What changes were proposed in this pull request?
Hadoop will support ZStandard Compression from 2.9.0. This update enables
saving a file
Github user dongjinleekr commented on a diff in the pull request:
https://github.com/apache/spark/pull/17225#discussion_r105342714
--- Diff: core/src/main/scala/org/apache/spark/io/CompressionCodec.scala ---
@@ -49,13 +50,14 @@ private[spark] object CompressionCodec
Github user dongjinleekr commented on a diff in the pull request:
https://github.com/apache/spark/pull/21501#discussion_r194623958
--- Diff: python/pyspark/ml/feature.py ---
@@ -2582,25 +2582,31 @@ class StopWordsRemover(JavaTransformer,
HasInputCol, HasOutputCol, JavaMLReadabl
Github user dongjinleekr commented on the issue:
https://github.com/apache/spark/pull/21525
@jkbradley Excuse me. Could have a look when you are free? Thanks.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user dongjinleekr commented on the issue:
https://github.com/apache/spark/pull/21501
@mengxr Thank you very much for your kind guidance. If you have some time,
please have a look at my another PR, #21525. I found this issue while I was
reading [an issue on
HashingTF](https
Github user dongjinleekr commented on a diff in the pull request:
https://github.com/apache/spark/pull/21501#discussion_r194222392
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/StopWordsRemover.scala ---
@@ -84,7 +86,28 @@ class StopWordsRemover @Since("
Github user dongjinleekr commented on a diff in the pull request:
https://github.com/apache/spark/pull/21501#discussion_r194222407
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/StopWordsRemover.scala ---
@@ -84,7 +86,28 @@ class StopWordsRemover @Since("
Github user dongjinleekr commented on a diff in the pull request:
https://github.com/apache/spark/pull/21501#discussion_r194296251
--- Diff: python/pyspark/ml/feature.py ---
@@ -2610,6 +2610,9 @@ def setParams(self, inputCol=None, outputCol=None,
stopWords=None, caseSensitive
GitHub user dongjinleekr opened a pull request:
https://github.com/apache/spark/pull/21525
[SPARK-24513][ML] Attribute support in UnaryTransformer
## What changes were proposed in this pull request?
This PR adds Metadata support in `UnaryTransformer`, as a preliminary work
Github user dongjinleekr commented on the issue:
https://github.com/apache/spark/pull/21501
Finally, I made it work by adding `StopWordsRemover.getDefaultLocale`! How
about this approach? @mengxr @viirya
Github user dongjinleekr commented on a diff in the pull request:
https://github.com/apache/spark/pull/21501#discussion_r194342986
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/StopWordsRemover.scala ---
@@ -84,7 +86,28 @@ class StopWordsRemover @Since("
Github user dongjinleekr commented on the issue:
https://github.com/apache/spark/pull/21525
If needed, I can propose a draft version of
[SPARK-13998](https://issues.apache.org/jira/browse/SPARK-13998) implemented on
top of this work
GitHub user dongjinleekr opened a pull request:
https://github.com/apache/spark/pull/21501
[SPARK-15064][ML] Locale support in StopWordsRemover
## What changes were proposed in this pull request?
Add locale support for `StopWordsRemover`.
## How was this patch
Github user dongjinleekr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22282#discussion_r214198620
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriteTask.scala
---
@@ -131,9 +158,25 @@ private[kafka010
Github user dongjinleekr commented on the issue:
https://github.com/apache/spark/pull/22282
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user dongjinleekr commented on the issue:
https://github.com/apache/spark/pull/22282
Retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user dongjinleekr commented on a diff in the pull request:
https://github.com/apache/spark/pull/21525#discussion_r216333687
--- Diff: mllib/src/main/scala/org/apache/spark/ml/Transformer.scala ---
@@ -116,10 +116,17 @@ abstract class UnaryTransformer[IN, OUT, T
Github user dongjinleekr commented on the issue:
https://github.com/apache/spark/pull/22282
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user dongjinleekr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22282#discussion_r214345173
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriteTask.scala
---
@@ -131,9 +158,25 @@ private[kafka010
Github user dongjinleekr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22282#discussion_r214345393
--- Diff:
sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java
---
@@ -473,17 +474,6 @@ public static
GitHub user dongjinleekr opened a pull request:
https://github.com/apache/spark/pull/22342
Feature/fix kafka sql trivials
## What changes were proposed in this pull request?
Fix unused imports & outdated comments on `kafka-0-10-sql` module. (Found
while I was wor
Github user dongjinleekr commented on the issue:
https://github.com/apache/spark/pull/22282
cc/ @zsxwing @tdas @dongjoon-hyun @srowen Rebased onto the latest master.
Please have a look when you are free. Thanks in advance
GitHub user dongjinleekr opened a pull request:
https://github.com/apache/spark/pull/22282
[SPARK-23539][SS] Add support for Kafka headers in Structured Streaming
## What changes were proposed in this pull request?
This update adds support for Kafka Headers functionality
Github user dongjinleekr commented on the issue:
https://github.com/apache/spark/pull/22282
As you can see, this PR consists of 3 parts:
1. Extend `UnsafeArrayData`, `UnsafeMapData` (commit 1~6)
2. Implement Kafka Headers functionality (commit 7, 10)
3. Update unit
Github user dongjinleekr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22282#discussion_r214856258
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriteTask.scala
---
@@ -88,7 +92,30 @@ private[kafka010] abstract
34 matches
Mail list logo
