Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19254
@srowen rebased against `master` to get the test ignore
https://github.com/apache/spark/commit/894a7561de2c2ff01fe7fcc5268378161e9e5643
, should be good to retest now
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19254
Sure, added a JIRA for the failure here
https://issues.apache.org/jira/browse/SPARK-22047 :)
---
-
To unsubscribe, e
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/17819
@viirya re `HiveExternalCatalogVersionsSuite`, jup it is
https://github.com/apache/spark/commit/dbb824125d4d31166d9a47c330f8d51f5d159515#commitcomment-24354358
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19254
Seems to be broken on Jenkins `master` too
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test/job/spark-master-test-sbt-hadoop-2.7/3490
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19254
Test failure looks unrelated (even though it happened twice in a row now):
```sh
[info] org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite ***
ABORTED *** (1 second
Github user original-brownbear commented on a diff in the pull request:
https://github.com/apache/spark/pull/19254#discussion_r139307959
--- Diff:
core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java ---
@@ -403,6 +391,16 @@ public long spill(long numBytes) throws
Github user original-brownbear commented on a diff in the pull request:
https://github.com/apache/spark/pull/19254#discussion_r139305653
--- Diff:
core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java ---
@@ -403,6 +391,16 @@ public long spill(long numBytes) throws
Github user original-brownbear commented on a diff in the pull request:
https://github.com/apache/spark/pull/19254#discussion_r139305647
--- Diff: core/src/main/java/org/apache/spark/memory/MemoryConsumer.java ---
@@ -116,13 +110,7 @@ public void freeArray(LongArray array
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19254
Test failure in PySpark appears unrelated, various tests OOMed like e.g.
```sh
FAILED (errors=2)
[Running ]
ERROR
GitHub user original-brownbear opened a pull request:
https://github.com/apache/spark/pull/19254
[MINOR][CORE] Cleanup dead code and duplication in Mem. Management
## What changes were proposed in this pull request?
* Removed the method
Github user original-brownbear closed the pull request at:
https://github.com/apache/spark/pull/19179
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19180
@srowen alright then, switched to the JDK comparison left the rest as is :)
---
-
To unsubscribe, e-mail: reviews
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19180
@srowen benchmarked this some more on a quiet workstation to make sure and:
this version still wins all categories and even beats the current version
in `master` if at least 2
Github user original-brownbear commented on a diff in the pull request:
https://github.com/apache/spark/pull/19180#discussion_r139112161
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -1097,10 +1100,23 @@ public UTF8String copy
Github user original-brownbear commented on a diff in the pull request:
https://github.com/apache/spark/pull/19180#discussion_r138991030
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -1097,10 +1100,23 @@ public UTF8String copy
Github user original-brownbear commented on a diff in the pull request:
https://github.com/apache/spark/pull/19180#discussion_r138990727
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -1097,10 +1100,23 @@ public UTF8String copy
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19180
@srowen updated, we don't even need to break out via `wordMax > 0`, we can
just get the offsets from `other` and save one level of indirect in the second
loop by using `(Platform
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19180
@srowen played a little with
https://github.com/original-brownbear/spark/blob/string-compareto-benchmark-both/core/src/test/scala/org/apache/spark/benchmarks/UTF8StringBenchmark.scala
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19180
@srowen I think we have that already here
https://github.com/apache/spark/blob/master/common/unsafe/src/test/java/org/apache/spark/unsafe/types/UTF8StringSuite.java#L103
`("
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19180
@srowen thanks, all comments addressed I think :)
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user original-brownbear commented on a diff in the pull request:
https://github.com/apache/spark/pull/19180#discussion_r138466080
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -1097,8 +1101,21 @@ public UTF8String copy
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19180
@srowen got a sec to take a look at this one? :) (should be a quick one)
---
-
To unsubscribe, e-mail: reviews
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19182
@srowen done, all changes to `org.apache.hive.*` reverted :)
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19182
@srowen looks like we're all green :)
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.or
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19182
@srowen makes perfect sense => rolled back all changes to tests + publicly
exposed methods (those package private ones adjusted are on non-public clas
GitHub user original-brownbear opened a pull request:
https://github.com/apache/spark/pull/19182
[SPARK-21970][Core] Fix Redundant Throws Declarations in Java Codebase
## What changes were proposed in this pull request?
1. Removing all redundant throws declarations from
Github user original-brownbear commented on a diff in the pull request:
https://github.com/apache/spark/pull/19180#discussion_r137956036
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -1097,8 +1101,21 @@ public UTF8String copy
Github user original-brownbear commented on a diff in the pull request:
https://github.com/apache/spark/pull/19180#discussion_r137951583
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -1097,8 +1101,21 @@ public UTF8String copy
GitHub user original-brownbear opened a pull request:
https://github.com/apache/spark/pull/19180
[SPARK-21967][CORE] org.apache.spark.unsafe.types.UTF8String#compareTo
Should Compare 8 Bytes at a Time for Better Performance
## What changes were proposed in this pull request
GitHub user original-brownbear opened a pull request:
https://github.com/apache/spark/pull/19179
TRIVIAL: Cleanup Todo for removal of
org.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter#alignToWords
Hope this qualifies as a trivial fix without a JIRA.
## What
GitHub user original-brownbear opened a pull request:
https://github.com/apache/spark/pull/17756
[SPARK-20455][DOCS] Fix Broken Docker IT Docs
## What changes were proposed in this pull request?
Just added the Maven `test`goal.
## How was this patch tested
GitHub user original-brownbear opened a pull request:
https://github.com/apache/spark/pull/16904
[SPARK-19562][BUILD] Added exclude for dev/pr-deps to gitignore
## What changes were proposed in this pull request?
Just adding a missing .gitignore entry.
## How was
32 matches
Mail list logo
