Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/35
Merged
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/37
Updated the benchmark; the size of the left table is ~140MB and the size of
the right table is ~70MB.
```
TestUtils.benchmark("codegen top-k join") {
/**
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/37
A codegen'd top-K join is as follows;
```
Found 1 WholeStageCodegen subtrees.
== Subtree 1 / 1 ==
*ShuffledHashJoinTopK -1, [group#10], [group#27]
:- Exchange
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/38
LGTM cc: @myui
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/36
okay, thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/41
yea, I'll update just after this merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/37
okay, merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/23
yea, could you add `[WIP]` in this title?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/24
Merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/29
[HIVEMALL-39] Put the use of HiveUDFs in one place
## What changes were proposed in this pull request?
This is a refactoring issue; in the master, we directly use the logical
plan
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/29
Merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/34
[HIVEMALL-45][SPARK] Upgrade spark v2.0.0 to v2.0.2 (latest)
## What changes were proposed in this pull request?
This pr updated pom.xml for the upgrade.
## What type of PR
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/26
[HIVEMALL-35] Remove unnecessary implicit conversions in HivemallUtils
## What changes were proposed in this pull request?
This pr removed entries for implicit conversion
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/25
[HIVEMALL-34] Fix a bug to wrongly use mllib vectors in some functions
## What changes were proposed in this pull request?
In `to_hivemall_features` and `append_bias` in `HivemallUtils
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/25
Merged!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/28
[HIVEMALL-30] Temporarily ignore a streaming test
## What changes were proposed in this pull request?
This test below fails sometimes (too flaky), so we temporarily ignore
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/26
I made a pr for this flaky test failure in #28, so I'll merge this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/27
okay, I'll merge this, then I'll check the OOM issue in follow-up
activities. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/29
@myui could you check this before merging it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/49
Updated
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/49
[HIVEMALL-26][SPARK] Make docs for regression and binary classification
## What changes were proposed in this pull request?
This pr added docs for hivemall-on-spark.
## What
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/49
Many thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/44
LGTM. I'll merged later
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/41
Merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/20
@wangyum Thanks for your work! What does this pr solve? Any issue in the
current script?
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/20
@wangyum oh, I found you've already described that in the JIRA ticket.
Could you write "what does this pr solve?" in this description? If you update
that , LGTM. cc: @myui
-
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/54
[HIVEMALL-76][SPARK] Fix worng ranks in top-K funcs
## What changes were proposed in this pull request?
This pr fixed the Spark `each_top_k`/`top_k_join` behaviour along with Hive
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/54
@myui passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/42
It's okay to merge
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/62
Merged to master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/59
@wangyum Thanks for your continuous contributions!
@myui do we have any reason to have a dependency with hadoop-core
`0.20.2-cdh3u6`? I just used this dependency along with the other
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/61#discussion_r105089894
--- Diff:
spark/spark-2.1/src/main/scala/org/apache/spark/sql/hive/HivemallOps.scala ---
@@ -805,6 +805,47 @@ final class HivemallOps(df
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/61#discussion_r105090944
--- Diff:
spark/spark-2.1/src/main/scala/org/apache/spark/sql/hive/HivemallOps.scala ---
@@ -805,6 +805,47 @@ final class HivemallOps(df
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/61#discussion_r105093086
--- Diff:
spark/spark-2.1/src/main/scala/org/apache/spark/sql/hive/HivemallOps.scala ---
@@ -805,6 +805,47 @@ final class HivemallOps(df
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/61#discussion_r105088535
--- Diff:
spark/spark-2.1/src/main/scala/org/apache/spark/sql/hive/HivemallOps.scala ---
@@ -805,6 +805,47 @@ final class HivemallOps(df
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/62#discussion_r105090294
--- Diff:
spark/spark-2.1/src/main/scala/org/apache/spark/sql/execution/datasources/csv/csvExpressions.scala
---
@@ -0,0 +1,153
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/61#discussion_r105099394
--- Diff:
spark/spark-2.1/src/main/scala/org/apache/spark/sql/hive/HivemallOps.scala ---
@@ -805,6 +805,47 @@ final class HivemallOps(df
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/61#discussion_r105090100
--- Diff:
spark/spark-2.1/src/main/scala/org/apache/spark/sql/hive/HivemallOps.scala ---
@@ -805,6 +805,47 @@ final class HivemallOps(df
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/62
Updated descriptions for the two funcs.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/59
@wangyum Why you select `2.6.5` in this pr? Any reason?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/100
[HOTFIX] Update documents for DataFrame in Spark
## What changes were proposed in this pull request?
This pr updated documents for `DataFrame` in Spark.
## What type of PR
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/100
Merged to master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/99
[HIVEMALL-116][SQL][DOC] Add docs for SQL cases in hivemall-spark
## What changes were proposed in this pull request?
This pr added docs for SQL cases in `hivemall-spark
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/106
[HIVEMALL-136][SPARK] Support train_classifier and train_regressor for Spark
## What changes were proposed in this pull request?
This pr added functions `train_classifier
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/99#discussion_r126836953
--- Diff: docs/gitbook/spark/regression/e2006_sql.md ---
@@ -0,0 +1,151 @@
+
+
+E2006
+===
+http://www.csie.ntu.edu.tw/~cjlin
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/99
Merged to master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/95
@amaya382 Can you check this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/95#discussion_r125845972
--- Diff: xgboost/src/main/java/hivemall/xgboost/XGBoostUDTF.java ---
@@ -326,7 +320,7 @@ public void close() throws HiveException
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/95#discussion_r125930040
--- Diff: xgboost/src/main/java/hivemall/xgboost/XGBoostUDTF.java ---
@@ -269,44 +270,35 @@ public void checkTargetValue(double target) throws
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/95
ok, I hit the same error. I'll check again. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/95
Without `HadoopUtils.getTaskId()` in
[here](https://github.com/maropu/incubator-hivemall/blob/e9fc6cfabd295c4c49faf43c4a44fe9eca2c9025/xgboost/src/main/java/hivemall/xgboost
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/95
@amaya382 check again? I checked it worked well in my local env.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/75
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/103
ok
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/78
[HIVEMALL-103][Spark] Upgrade spark-v2.1.0 to v2.1.1
## What changes were proposed in this pull request?
This pr upgraded spark-v2.1.0 to v2.1.1.
## What type of PR
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/78
ok
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/78
merged to master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/80#discussion_r117665868
--- Diff: bin/build_xgboost.sh ---
@@ -1,87 +0,0 @@
-#!/bin/bash
-#
-# Licensed to the Apache Software Foundation (ASF) under one
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/80#discussion_r117657316
--- Diff: xgboost/src/main/java/hivemall/xgboost/NativeLibLoader.java ---
@@ -54,15 +55,47 @@ private static boolean hasResource(String path
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/80
Yea, I also think we need to use `qemu` to test them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/80#discussion_r117690004
--- Diff: bin/build_xgboost.sh ---
@@ -1,87 +0,0 @@
-#!/bin/bash
-#
-# Licensed to the Apache Software Foundation (ASF) under one
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/80#discussion_r117657090
--- Diff: bin/build_xgboost.sh ---
@@ -1,87 +0,0 @@
-#!/bin/bash
-#
-# Licensed to the Apache Software Foundation (ASF) under one
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/80
yea, I think so. I just mean I can't reproduce in my laptop and I can't
look into this issue...
---
If your project is set up for it, you can reply to this email and have your
reply
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/80
@amaya382 Aha, I've not seen that exception. Actually, I didn't check
behaviours in Hive. Could you look into this issue?
---
If your project is set up for it, you can reply to this email
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/80
@amaya382 Could you file a jira for that?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/122
[HIVEMALL-147][Spark] Support all Hivemall functions of v0.5-rc.1 in Spark
Dataframe
## What changes were proposed in this pull request?
This pr added more Hivemall functions
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/122#discussion_r144753835
--- Diff: core/src/main/java/hivemall/evaluation/HitRateUDAF.java ---
@@ -71,9 +71,6 @@
+ " - Returns HitRate")
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/122#discussion_r144753777
--- Diff: core/src/main/java/hivemall/evaluation/AUCUDAF.java ---
@@ -110,7 +110,7 @@ public ClassificationEvaluator
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/112
[HIVEMALL-133][SPARK] Support spark-v2.2 in the hivemalls-spark module
## What changes were proposed in this pull request?
This pr added supports for spark-2.2 in Hivemall
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/103
See #112
---
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/103
Thanks, I'll check later
---
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/113
[HIVEMALL-136][SPARK] Support train_classifier and train_regressor for
Spark
## What changes were proposed in this pull request?
This pr added functions `train_classifier
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/106
See #113
---
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/130
[HIVEMALL][SPARK][WIP] Fix Spark-related artifact issues
## What changes were proposed in this pull request?
A objective of this pr is to fix artifact to release hivemall-v0.5.0
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/130
yea, NVM. A main target of this pr is to solve all the issue for the
upcoming releases.
---
Github user maropu closed the pull request at:
https://github.com/apache/incubator-hivemall/pull/130
---
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/130
See #131
---
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/131
[HIVEMALL][SPARK] Update release-guide.md for spark releases
## What changes were proposed in this pull request?
This pr updated `release-guide.md` for spark releases
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/139
Sorry for my slow work. I'm checking the feasibility on my separate repo
(because there are some issues to solve):
https://github.com/maropu/spark-catalyst-rule-rewiter/tree/master
So
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/141
Finished: https://spark-packages.org/package/apache-hivemall/apache-hivemall
@myui check again?
---
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/135
@myui Spark already has these functions:
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/functions.scala#L3118
---
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/137
merged to master
---
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/138
[HIVEMALL-180][SPARK] Drop the Spark-2.0 support
## What changes were proposed in this pull request?
This pr dropped the module for Spark-2.0.
## What type of PR
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/139
I'll fix later.
---
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/141
[HIVEMALL-117][SPARK] Update the installation guide for Spark
## What changes were proposed in this pull request?
This pr updated the installation guide for Spark.
## What
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/141
I'll create a new github account for this purpose and then move the repo
there.
So, pending until the move finished.
---
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/139
[HIVEMALL-182][SPARK][WIP] Add an optimizer rule to filter out columns with
low variances
## What changes were proposed in this pull request?
This pr added a new optimizer rule
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/171#discussion_r233262036
--- Diff:
spark/spark-2.3/src/main/scala/org/apache/spark/sql/hive/HivemallOps.scala ---
@@ -1935,18 +1935,6 @@ object HivemallOps
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/171
Not finished yet (I'm still working).
---
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/171#discussion_r233262624
--- Diff:
spark/spark-2.3/src/test/scala/org/apache/spark/sql/hive/XGBoostSuite.scala ---
@@ -77,6 +77,7 @@ final class XGBoostSuite extends
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/171#discussion_r233288446
--- Diff:
spark/spark-2.3/src/main/scala/org/apache/spark/sql/hive/HivemallOps.scala ---
@@ -1935,18 +1935,6 @@ object HivemallOps
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/171
[SPARK][HOTFIX][WIP] Fix existing test failures in spark-2.3
## What changes were proposed in this pull request?
This pr is to fix the test failures for spark-2.3.
## How
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/171#discussion_r233261637
--- Diff: spark/pom.xml ---
@@ -52,6 +52,12 @@
hivemall-core
${project.version
Github user maropu commented on a diff in the pull request:
https://github.com/apache/incubator-hivemall/pull/171#discussion_r233399519
--- Diff:
spark/spark-2.3/src/test/scala/org/apache/spark/sql/hive/XGBoostSuite.scala ---
@@ -77,6 +77,7 @@ final class XGBoostSuite extends
94 matches
Mail list logo