[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13680 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61660/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13680 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13680 **[Test build #61660 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61660/consoleFull)** for PR 13680 at commit [`7576c19`](https://github.com/apache/spark/commit/7576c19dfc872221d10abf7851e0782a76822ab0). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13532: [SPARK-15204][SQL] improve nullability inference for Agg...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13532 **[Test build #61667 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61667/consoleFull)** for PR 13532 at commit [`46ced5c`](https://github.com/apache/spark/commit/46ced5c5022bc930241724c6cc6e118293321dd3). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14030: [SPARK-16350][SQL] Fix `foreach` for streaming Dataset
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14030 **[Test build #61666 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61666/consoleFull)** for PR 14030 at commit [`2f8ba28`](https://github.com/apache/spark/commit/2f8ba2859c521979deacae87fa03460fec5c8191). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14030: [WIP][SPARK-16350][SQL] Fix `foreach` for streaming Data...
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/14030 Jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14030: [WIP][SPARK-16350][SQL] Fix `foreach` for streaming Data...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14030 **[Test build #61665 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61665/consoleFull)** for PR 14030 at commit [`02cb6b5`](https://github.com/apache/spark/commit/02cb6b5fd8f6877d86c3307654060316ea14f815). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14031: [SPARK-16353][BUILD][DOC] Missing javadoc options for ja...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14031 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14031: [SPARK-16353][BUILD][DOC] Missing javadoc options...
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/14031#discussion_r69382212
--- Diff: project/SparkBuild.scala ---
@@ -723,8 +723,8 @@ object Unidoc {
.map(_.filterNot(_.getCanonicalPath.contains("org/apache/hadoop")))
},
-// Javadoc options: create a window title, and group key packages on
index page
--- End diff --
BTW, I removed the mention of package groupings because none are defined.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14031: [SPARK-16353][BUILD][DOC] Missing javadoc options...
GitHub user mallman opened a pull request: https://github.com/apache/spark/pull/14031 [SPARK-16353][BUILD][DOC] Missing javadoc options for java unidoc ## What changes were proposed in this pull request? The javadoc options for the java unidoc generation are ignored when generating the java unidoc. For example, the generated `index.html` has the wrong HTML page title. This can be seen at http://spark.apache.org/docs/latest/api/java/index.html. I changed the relevant setting scope from `doc` to `(JavaUnidoc, unidoc)`. ## How was this patch tested? I ran `docs/jekyll build` and verified that the java unidoc `index.html` has the correct HTML page title. You can merge this pull request into a Git repository by running: $ git pull https://github.com/VideoAmp/spark-public spark-16353 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/14031.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #14031 commit 939e8b5d3a3b502f3a7870d437cb38ee9564e6c4 Author: Michael Allman Date: 2016-07-02T19:55:39Z [SPARK-16353][BUILD][DOC] The javadoc options for the java unidoc generation are not honored. The scope of the relevant javacOptions key should be `(JavaUnidoc, unidoc)` not `doc` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14004: [SPARK-16285][SQL] Implement sentences SQL functions
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14004 **[Test build #61664 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61664/consoleFull)** for PR 14004 at commit [`c9e235a`](https://github.com/apache/spark/commit/c9e235a3ea35bbd2cdf08503bce7156f8f3a4d1d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14004: [SPARK-16285][SQL] Implement sentences SQL functions
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14004 Just rebased. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer to rem...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13876 **[Test build #61662 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61662/consoleFull)** for PR 13876 at commit [`63b3ecd`](https://github.com/apache/spark/commit/63b3ecd98eafa6363d3c07835cb06909ea1a23e8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13967: [SPARK-16278][SPARK-16279][SQL] Implement map_keys/map_v...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13967 **[Test build #61661 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61661/consoleFull)** for PR 13967 at commit [`8db1e65`](https://github.com/apache/spark/commit/8db1e656f27aa1647fca7c86405959262c3365fd). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13765: [SPARK-16052][SQL] Improve `CollapseRepartition` optimiz...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13765 **[Test build #61663 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61663/consoleFull)** for PR 13765 at commit [`e26e956`](https://github.com/apache/spark/commit/e26e956c89593bbae52c2cdc32b788ed7eea29c7). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13967: [SPARK-16278][SPARK-16279][SQL] Implement map_keys/map_v...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13967 Rebased to the master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer to rem...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13876 Rebased to the master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13765: [SPARK-16052][SQL] Improve `CollapseRepartition` optimiz...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13765 Rebased to the master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14017: [MINOR][BUILD] Fix Java linter errors
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14017 Thank you for merging, @srowen . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/13680#discussion_r69381782
--- Diff:
sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java
---
@@ -341,63 +324,113 @@ public UnsafeArrayData copy() {
return arrayCopy;
}
- public static UnsafeArrayData fromPrimitiveArray(int[] arr) {
-if (arr.length > (Integer.MAX_VALUE - 4) / 8) {
- throw new UnsupportedOperationException("Cannot convert this array
to unsafe format as " +
-"it's too big.");
-}
+ @Override
+ public boolean[] toBooleanArray() {
+int size = numElements();
+boolean[] values = new boolean[size];
+Platform.copyMemory(
+ baseObject, baseOffset + headerInBytes, values,
Platform.BYTE_ARRAY_OFFSET, size);
+return values;
+ }
+
+ @Override
+ public byte[] toByteArray() {
+int size = numElements();
+byte[] values = new byte[size];
+Platform.copyMemory(
+ baseObject, baseOffset + headerInBytes, values,
Platform.BYTE_ARRAY_OFFSET, size);
+return values;
+ }
+
+ @Override
+ public short[] toShortArray() {
+int size = numElements();
+short[] values = new short[size];
+Platform.copyMemory(
+ baseObject, baseOffset + headerInBytes, values,
Platform.SHORT_ARRAY_OFFSET, size * 2);
+return values;
+ }
-final int offsetRegionSize = 4 * arr.length;
-final int valueRegionSize = 4 * arr.length;
-final int totalSize = 4 + offsetRegionSize + valueRegionSize;
-final byte[] data = new byte[totalSize];
+ @Override
+ public int[] toIntArray() {
+int size = numElements();
+int[] values = new int[size];
+Platform.copyMemory(
+ baseObject, baseOffset + headerInBytes, values,
Platform.INT_ARRAY_OFFSET, size * 4);
+return values;
+ }
-Platform.putInt(data, Platform.BYTE_ARRAY_OFFSET, arr.length);
+ @Override
+ public long[] toLongArray() {
+int size = numElements();
+long[] values = new long[size];
+Platform.copyMemory(
+ baseObject, baseOffset + headerInBytes, values,
Platform.LONG_ARRAY_OFFSET, size * 8);
+return values;
+ }
-int offsetPosition = Platform.BYTE_ARRAY_OFFSET + 4;
-int valueOffset = 4 + offsetRegionSize;
-for (int i = 0; i < arr.length; i++) {
- Platform.putInt(data, offsetPosition, valueOffset);
- offsetPosition += 4;
- valueOffset += 4;
+ @Override
+ public float[] toFloatArray() {
+int size = numElements();
+float[] values = new float[size];
+Platform.copyMemory(
+ baseObject, baseOffset + headerInBytes, values,
Platform.FLOAT_ARRAY_OFFSET, size * 4);
+return values;
+ }
+
+ @Override
+ public double[] toDoubleArray() {
+int size = numElements();
+double[] values = new double[size];
+Platform.copyMemory(
+ baseObject, baseOffset + headerInBytes, values,
Platform.DOUBLE_ARRAY_OFFSET, size * 8);
+return values;
+ }
+
+ private static UnsafeArrayData fromPrimitiveArray(Object arr, int
length, final int elementSize) {
+final int headerSize = calculateHeaderPortionInBytes(length);
+if (length > (Integer.MAX_VALUE - headerSize) / elementSize) {
+ throw new UnsupportedOperationException("Cannot convert this array
to unsafe format as " +
+"it's too big.");
}
+final int valueRegionSize = elementSize * length;
+final byte[] data = new byte[valueRegionSize + headerSize];
--- End diff --
Yes, when tests on Jenkins pass, I will expand fields for ```long[]```
(e.g. 4 bytes -> 8bytes for ```numElements```)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13680 **[Test build #61660 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61660/consoleFull)** for PR 13680 at commit [`7576c19`](https://github.com/apache/spark/commit/7576c19dfc872221d10abf7851e0782a76822ab0). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/13680#discussion_r69381568
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/UnsafeArrayDataBenchmark.scala
---
@@ -0,0 +1,298 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.benchmark
+
+import org.apache.spark.SparkConf
+import org.apache.spark.sql.catalyst.expressions.{UnsafeArrayData,
UnsafeRow}
+import org.apache.spark.sql.catalyst.expressions.codegen.{BufferHolder,
UnsafeArrayWriter}
+import org.apache.spark.unsafe.Platform
+import org.apache.spark.util.Benchmark
+
+/**
+ * Benchmark [[UnsafeArrayDataBenchmark]] for UnsafeArrayData
+ * To run this:
+ * build/sbt "sql/test-only *benchmark.UnsafeArrayDataBenchmark"
+ *
+ * Benchmarks in this file are skipped in normal builds.
+ */
+class UnsafeArrayDataBenchmark extends BenchmarkBase {
+
+ new SparkConf()
+.setMaster("local[1]")
+.setAppName("microbenchmark")
+.set("spark.driver.memory", "3g")
+
+ def calculateHeaderPortionInBytes(count: Int) : Int = {
+// Use this assignment for SPARK-15962
+// val size = 4 + 4 * count
+val size = UnsafeArrayData.calculateHeaderPortionInBytes(count)
+size
+ }
+
+ def readUnsafeArray(iters: Int): Unit = {
--- End diff --
I will update an allocation method of ```UnsafeArrayData``` for "normal
read". For "normal write" I think that it is not possible to turn it into
```UnsafeArray``` for write. This is because ```UnsafeArrayData``` does not
have ```write(int)``` or ```putInt()``` method. This is why we use
```UnsafeArrayWriter```.
We have already done for "from primitive array" and "to primitive array".
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14030: [WIP][SPARK-16350][SQL] Fix `foreach` for streaming Data...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14030 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14030: [WIP][SPARK-16350][SQL] Fix `foreach` for streaming Data...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14030 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61659/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14030: [WIP][SPARK-16350][SQL] Fix `foreach` for streaming Data...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14030 **[Test build #61659 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61659/consoleFull)** for PR 14030 at commit [`f3f60f9`](https://github.com/apache/spark/commit/f3f60f919a2070a6946d0d908b54225d3c2263fc). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #13517: [SPARK-14839][SQL] Support for other types as opt...
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/13517#discussion_r69380919
--- Diff:
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -45,11 +45,11 @@ statement
| ALTER DATABASE identifier SET DBPROPERTIES tablePropertyList
#setDatabaseProperties
| DROP DATABASE (IF EXISTS)? identifier (RESTRICT | CASCADE)?
#dropDatabase
| createTableHeader ('(' colTypeList ')')? tableProvider
-(OPTIONS tablePropertyList)?
+(OPTIONS optionParameterList)?
(PARTITIONED BY partitionColumnNames=identifierList)?
bucketSpec?
#createTableUsing
| createTableHeader tableProvider
-(OPTIONS tablePropertyList)?
--- End diff --
@HyukjinKwon I am on holiday - so I am bit slow with my responses.
Yo have understood me correctly. What I am suggesting will affect the
DBPROPERTIES and TBLPROPERTIES; it will also allow for boolean and numeric
options. I don't think this is a bad thing, it is better to have a lenient
parser and to constrain behavior in the `AstBuilder` (this allows us to throw
much better error messages).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14029: [MINOR] [DOCS] Remove unused images; crush PNGs that cou...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14029 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61658/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14029: [MINOR] [DOCS] Remove unused images; crush PNGs that cou...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14029 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14029: [MINOR] [DOCS] Remove unused images; crush PNGs that cou...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14029 **[Test build #61658 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61658/consoleFull)** for PR 14029 at commit [`c54e560`](https://github.com/apache/spark/commit/c54e5602c39e107680e681786287ab723586ad80). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14030: [WIP][SPARK-16350][SQL] Fix `foreach` for streami...
GitHub user lw-lin opened a pull request: https://github.com/apache/spark/pull/14030 [WIP][SPARK-16350][SQL] Fix `foreach` for streaming Dataset ## What changes were proposed in this pull request? - [x] add tests - [ ] fix `foreach` ## How was this patch tested? (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests) (If this patch involves UI changes, please attach a screenshot; otherwise, remove this) You can merge this pull request into a Git repository by running: $ git pull https://github.com/lw-lin/spark fix-foreach-complete Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/14030.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #14030 commit f3f60f919a2070a6946d0d908b54225d3c2263fc Author: Liwei Lin Date: 2016-07-02T14:56:06Z Add test(`complete`) & expand test(`append`) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14030: [WIP][SPARK-16350][SQL] Fix `foreach` for streaming Data...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14030 **[Test build #61659 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61659/consoleFull)** for PR 14030 at commit [`f3f60f9`](https://github.com/apache/spark/commit/f3f60f919a2070a6946d0d908b54225d3c2263fc). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14028 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61657/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14028 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14028 **[Test build #61657 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61657/consoleFull)** for PR 14028 at commit [`e8edd38`](https://github.com/apache/spark/commit/e8edd38a1b98f5f886b9e8f29ab181fefe2e7202). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14028 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14028 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61656/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14028 **[Test build #61656 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61656/consoleFull)** for PR 14028 at commit [`50b6c4c`](https://github.com/apache/spark/commit/50b6c4cdfae12151262a83b9283acbfaf2b05938). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14027: [SPARK-16352] Spark daemon run.sh
Github user zasdfgbnm commented on the issue: https://github.com/apache/spark/pull/14027 OK, but will the status of the original JIRA keep being "won't fix", if someone decide to reopen the discussion on it? Can anyone give a comment on whether the running on foreground feature are welcome in spark upstream? Also, if there has already been a good solution to run in foreground easily enough (so that my PR will be useless), could anyone send me a link to the document describing this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14017: [MINOR][BUILD] Fix Java linter errors
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14017 Merged to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14029: [MINOR] [DOCS] Remove unused images; crush PNGs that cou...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14029 **[Test build #61658 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61658/consoleFull)** for PR 14029 at commit [`c54e560`](https://github.com/apache/spark/commit/c54e5602c39e107680e681786287ab723586ad80). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14017: [MINOR][BUILD] Fix Java linter errors
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14017 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14015: [SPARK-16345][Documentation][Examples][GraphX] Ex...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14015 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14015: [SPARK-16345][Documentation][Examples][GraphX] Extract g...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14015 Merged to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14029: [MINOR] [DOCS] Remove unused images; crush PNGs t...
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/14029 [MINOR] [DOCS] Remove unused images; crush PNGs that could use it for good measure ## What changes were proposed in this pull request? Coincidentally, I discovered that a couple images were unused in `docs/`, and then searched and found more, and then realized some PNGs were pretty big and could be crushed, and before I knew it, had done the same for the ASF site (not committed yet). No functional change at all, just less superfluous image data. ## How was this patch tested? `jekyll serve` You can merge this pull request into a Git repository by running: $ git pull https://github.com/srowen/spark RemoveCompressImages Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/14029.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #14029 commit c54e5602c39e107680e681786287ab723586ad80 Author: Sean Owen Date: 2016-07-02T15:26:04Z Remove unused images; crush PNGs that could use it for good measure --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14027: [SPARK-16352] Spark daemon run.sh
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14027 I'm suggesting you keep the conversation together with the original JIRA, at least. I don't know if it was going to be merged or not, but it was not updated by the author in any event. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #3881: [SPARK-5964] Allow spark-daemon.sh to support foreground ...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/3881 It's right there at https://github.com/apache/spark/pull/3881#issuecomment-113860838 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #3881: [SPARK-5964] Allow spark-daemon.sh to support foreground ...
Github user zasdfgbnm commented on the issue: https://github.com/apache/spark/pull/3881 Can anyone explain what happened to this PR? Why people close this PR without adding any support for running in foreground? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14028 **[Test build #61657 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61657/consoleFull)** for PR 14028 at commit [`e8edd38`](https://github.com/apache/spark/commit/e8edd38a1b98f5f886b9e8f29ab181fefe2e7202). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14027: [SPARK-16352] Spark daemon run.sh
Github user zasdfgbnm commented on the issue: https://github.com/apache/spark/pull/14027 I followed that PR (https://github.com/apache/spark/pull/3881) "I was under the impression all the shell scripts were getting refactored and that this patch had become obsolete. I agree it's best to close this out." by @hellertime I'm not sure what happened to that. It seems that the previous PR was closed because the scripts had changed a lot before that PR was merged. I didn't see anything that support running in foreground and nobody mentioned in that issue that running in foreground is useless and will never be supported. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14028 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14028 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61654/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14028 **[Test build #61654 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61654/consoleFull)** for PR 14028 at commit [`93aea42`](https://github.com/apache/spark/commit/93aea42fc86e69eedf4f35f5682b35744e12a480). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14028 **[Test build #61656 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61656/consoleFull)** for PR 14028 at commit [`50b6c4c`](https://github.com/apache/spark/commit/50b6c4cdfae12151262a83b9283acbfaf2b05938). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14028 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61653/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14028 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14028 **[Test build #61653 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61653/consoleFull)** for PR 14028 at commit [`34ec476`](https://github.com/apache/spark/commit/34ec476b5afad926db89a03ff64e6cda8263ee86). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14027: [SPARK-16352] Spark daemon run.sh
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14027 Please don't open a duplicate JIRA. You can see the discussion on the previous issue and JIRA, which is WontFix, mostly because it ultimately wasn't followed up on. It's not out of the quesiton but you should comment on the JIRA with details about why you think it's important first. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13984: [SPARK-16310][SPARKR] R na.string-like default for csv s...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13984 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13984: [SPARK-16310][SPARKR] R na.string-like default for csv s...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13984 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61655/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13984: [SPARK-16310][SPARKR] R na.string-like default for csv s...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13984 **[Test build #61655 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61655/consoleFull)** for PR 13984 at commit [`aaa6707`](https://github.com/apache/spark/commit/aaa67075f58f707c8dd47a98702689c1cc26bbc4). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14027: [SPARK-16352] Spark daemon run.sh
Github user zasdfgbnm commented on the issue: https://github.com/apache/spark/pull/14027 I see similar pull request here: https://github.com/apache/spark/pull/3881 But I didn't get what happened to that and why it was closed without adding any support for running in foreground. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13984: [SPARK-16310][SPARKR] R na.string-like default for csv s...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13984 **[Test build #61655 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61655/consoleFull)** for PR 13984 at commit [`aaa6707`](https://github.com/apache/spark/commit/aaa67075f58f707c8dd47a98702689c1cc26bbc4). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #13984: [SPARK-16310][SPARKR] R na.string-like default fo...
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/13984#discussion_r69377881
--- Diff: R/pkg/R/SQLContext.R ---
@@ -744,6 +746,9 @@ read.df.default <- function(path = NULL, source = NULL,
schema = NULL, ...) {
if (is.null(source)) {
source <- getDefaultSqlSource()
}
+ if (source == "csv" && is.null(options[["nullValue"]])) {
+options[["nullValue"]] <- "NA"
--- End diff --
great point. updated
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #13517: [SPARK-14839][SQL] Support for other types as opt...
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/13517#discussion_r69377675
--- Diff:
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -45,11 +45,11 @@ statement
| ALTER DATABASE identifier SET DBPROPERTIES tablePropertyList
#setDatabaseProperties
| DROP DATABASE (IF EXISTS)? identifier (RESTRICT | CASCADE)?
#dropDatabase
| createTableHeader ('(' colTypeList ')')? tableProvider
-(OPTIONS tablePropertyList)?
+(OPTIONS optionParameterList)?
(PARTITIONED BY partitionColumnNames=identifierList)?
bucketSpec?
#createTableUsing
| createTableHeader tableProvider
-(OPTIONS tablePropertyList)?
--- End diff --
(Sorry for pining @hvanhovell)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14028 BTW, this approach can be applied to reading in JSON and CSV as well if this change is appropriate. I will work on them if this change is sensible and can be merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14028 **[Test build #61654 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61654/consoleFull)** for PR 14028 at commit [`93aea42`](https://github.com/apache/spark/commit/93aea42fc86e69eedf4f35f5682b35744e12a480). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14027: Spark daemon run.sh
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14027 (I think It might be nicer if this contribution follows https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Validation ...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/13894 @holdenk @MLnick is the current update okay? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14025: [WIP][DOC] update out-of-date code snippets using SQLCon...
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/14025 cc @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14015: [SPARK-16345][Documentation][Examples][GraphX] Extract g...
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/14015 Merge conflicts have been solved. cc @srowen Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14015: [SPARK-16345][Documentation][Examples][GraphX] Extract g...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14015 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14015: [SPARK-16345][Documentation][Examples][GraphX] Extract g...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14015 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61652/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14015: [SPARK-16345][Documentation][Examples][GraphX] Extract g...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14015 **[Test build #61652 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61652/consoleFull)** for PR 14015 at commit [`70fcbb0`](https://github.com/apache/spark/commit/70fcbb028ed402da8f986f2cfed2e081c67712b5). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14028: [SPARK-16351][SQL] Avoid record-per type dispatch in JSO...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14028 **[Test build #61653 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61653/consoleFull)** for PR 14028 at commit [`34ec476`](https://github.com/apache/spark/commit/34ec476b5afad926db89a03ff64e6cda8263ee86). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14028: [SPARK-16351][SQL] Avoid record-per type dispatch...
GitHub user HyukjinKwon opened a pull request:
https://github.com/apache/spark/pull/14028
[SPARK-16351][SQL] Avoid record-per type dispatch in JSON when writing
## What changes were proposed in this pull request?
Currently, `JacksonGenerator.apply` is doing type-based dispatch for each
row to write appropriate values.
It might not have to be done like this because the schema is already kept.
So, appropriate writers can be created first according to the schema, and
then apply them to each row. This approach is similar with
`CatalystWriteSupport`.
This PR corrects `JacksonGenerator` so that it creates some writes for the
schema and then applies them to each row rather than every time.
Benchmark was proceeded with the codes below:
```
test("Benchmark for JSON writer") {
val N = 500 << 8
val row =
"""{"struct":{"field1": true, "field2": 92233720368547758070},
"structWithArrayFields":{"field1":[4, 5, 6], "field2":["str1",
"str2"]},
"arrayOfString":["str1", "str2"],
"arrayOfInteger":[1, 2147483647, -2147483648],
"arrayOfLong":[21474836470, 9223372036854775807,
-9223372036854775808],
"arrayOfBigInteger":[922337203685477580700, -922337203685477580800],
"arrayOfDouble":[1.2, 1.7976931348623157E308, 4.9E-324,
2.2250738585072014E-308],
"arrayOfBoolean":[true, false, true],
"arrayOfNull":[null, null, null, null],
"arrayOfStruct":[{"field1": true, "field2": "str1"}, {"field1":
false}, {"field3": null}],
"arrayOfArray1":[[1, 2, 3], ["str1", "str2"]],
"arrayOfArray2":[[1, 2, 3], [1.1, 2.1, 3.1]]
}"""
val df =
spark.sqlContext.read.json(spark.sparkContext.parallelize(List.fill(N)(row)))
(0 to 10).foreach { _ =>
val benchmark = new Benchmark("JSON writer", N)
benchmark.addCase("writing JSON file", 10) { iter =>
withTempPath { path =>
df.write.format("json").save(path.getCanonicalPath)
}
}
benchmark.run()
}
}
```
This produced the results below
- **Before**
```
JSON writer: Best/Avg Time(ms)Rate(M/s)
Per Row(ns) Relative
writing JSON file 1675 / 1767 0.1
13087.5 1.0X
```
- **After**
```
JSON writer: Best/Avg Time(ms)Rate(M/s)
Per Row(ns) Relative
writing JSON file 1597 / 1686 0.1
12477.1 1.0X
```
In addition, I ran this benchmarks 10 times for each and calculated the
average times as below:
| **Before** | **After**|
|---||
|17478ms |16669ms |
It seems roughly ~5% is improved.
## How was this patch tested?
Existing tests should cover this.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HyukjinKwon/spark SPARK-16351
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14028.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14028
commit 76ca05a383b1b3f6deea266276af872ab3a18f36
Author: hyukjinkwon
Date: 2016-07-02T12:16:25Z
Avoid record-per type dispatch in JSON when writing
commit 34ec476b5afad926db89a03ff64e6cda8263ee86
Author: hyukjinkwon
Date: 2016-07-02T12:18:03Z
Keep the comment
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14015: [SPARK-16345][Documentation][Examples][GraphX] Extract g...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14015 **[Test build #61652 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61652/consoleFull)** for PR 14015 at commit [`70fcbb0`](https://github.com/apache/spark/commit/70fcbb028ed402da8f986f2cfed2e081c67712b5). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14027: Spark daemon run.sh
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14027 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14027: Spark daemon run.sh
GitHub user zasdfgbnm opened a pull request: https://github.com/apache/spark/pull/14027 Spark daemon run.sh ## What changes were proposed in this pull request? I add some shell scripts to support running master and slave foreground, which makes it more convenient to write systemd service and run on HPC. ## How was this patch tested? I test it manually and it works well on my computer You can merge this pull request into a Git repository by running: $ git pull https://github.com/zasdfgbnm/spark spark-daemon-run.sh Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/14027.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #14027 commit 7b99b15656daa430081b1952bc090d3824d916ce Author: Xiang Gao Date: 2016-07-01T14:22:05Z split spark-daemon.sh to allow foreground commit 700247e5cae5bfd33d24a54c133eef2b5a28df96 Author: Xiang Gao Date: 2016-07-01T15:09:46Z fix bugs to make it work commit 6a80b7986f831abaf5cdf23c42528055f4999b86 Author: Xiang Gao Date: 2016-07-01T15:26:13Z fixes commit 3095187162a821292eabedcd44c18932dbf73ca6 Author: Xiang Gao Date: 2016-07-01T15:27:11Z starting->running commit 1954640ccca1a1da7903455d67f1abdc2db0fd73 Author: Xiang Gao Date: 2016-07-01T15:29:21Z add run-master.sh commit b96f4e1c4a28ffc893779221100080c5d31a2508 Author: Xiang Gao Date: 2016-07-01T15:36:26Z add support run-slave.sh --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13911: [SPARK-16215][SQL] Reduce runtime overhead of a program ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13911 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61651/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13911: [SPARK-16215][SQL] Reduce runtime overhead of a program ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13911 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13911: [SPARK-16215][SQL] Reduce runtime overhead of a program ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13911 **[Test build #61651 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61651/consoleFull)** for PR 13911 at commit [`2e8fb0e`](https://github.com/apache/spark/commit/2e8fb0e83152672183ea29f963d8f0f4d1a45d71). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14010: [GRAPHX][EXAMPLES] move graphx test data director...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14010 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14010: [GRAPHX][EXAMPLES] move graphx test data directory and u...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14010 Merged to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14026: [SPARK-13569][STREAMING][KAFKA] pattern based topic subs...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14026 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61649/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14026: [SPARK-13569][STREAMING][KAFKA] pattern based topic subs...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14026 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14026: [SPARK-13569][STREAMING][KAFKA] pattern based topic subs...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14026 **[Test build #61649 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61649/consoleFull)** for PR 14026 at commit [`796045f`](https://github.com/apache/spark/commit/796045ff3ad53afeb56cdddb69c4770090f7c168). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
