[jira] [Created] (SPARK-17650) Adding a malformed URL to sc.addJar and/or sc.addFile bricks Executors

2016-09-23 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-17650: --- Summary: Adding a malformed URL to sc.addJar and/or sc.addFile bricks Executors Key: SPARK-17650 URL: https://issues.apache.org/jira/browse/SPARK-17650 Project: Spark

[jira] [Created] (SPARK-17613) PartitioningAwareFileCatalog.allFiles doesn't handle URI specified path at parent

2016-09-20 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-17613: --- Summary: PartitioningAwareFileCatalog.allFiles doesn't handle URI specified path at parent Key: SPARK-17613 URL: https://issues.apache.org/jira/browse/SPARK-17613

[jira] [Created] (SPARK-17599) Folder deletion after globbing may fail StructuredStreaming jobs

2016-09-19 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-17599: --- Summary: Folder deletion after globbing may fail StructuredStreaming jobs Key: SPARK-17599 URL: https://issues.apache.org/jira/browse/SPARK-17599 Project: Spark

[jira] [Created] (SPARK-17569) Don't recheck existence of files when generating File Relation resolution in StructuredStreaming

2016-09-16 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-17569: --- Summary: Don't recheck existence of files when generating File Relation resolution in StructuredStreaming Key: SPARK-17569 URL: https://issues.apache.org/jira/browse/SPARK-17569

[jira] [Updated] (SPARK-17531) Don't initialize Hive Listeners for the Execution Client

2016-09-13 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-17531: Description: If a user provides listeners inside the Hive Conf, the configuration for these

[jira] [Created] (SPARK-17531) Don't initialize Hive Listeners for the Execution Client

2016-09-13 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-17531: --- Summary: Don't initialize Hive Listeners for the Execution Client Key: SPARK-17531 URL: https://issues.apache.org/jira/browse/SPARK-17531 Project: Spark Issue

[jira] [Created] (SPARK-16531) Remove TimeZone from DataFrameTimeWindowingSuite

2016-07-13 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-16531: --- Summary: Remove TimeZone from DataFrameTimeWindowingSuite Key: SPARK-16531 URL: https://issues.apache.org/jira/browse/SPARK-16531 Project: Spark Issue Type:

[jira] [Created] (SPARK-16227) Json schema inference fails when `:` exists in file path

2016-06-27 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-16227: --- Summary: Json schema inference fails when `:` exists in file path Key: SPARK-16227 URL: https://issues.apache.org/jira/browse/SPARK-16227 Project: Spark Issue

[jira] [Created] (SPARK-16050) Flaky Test: Complete aggregation with Console sink

2016-06-18 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-16050: --- Summary: Flaky Test: Complete aggregation with Console sink Key: SPARK-16050 URL: https://issues.apache.org/jira/browse/SPARK-16050 Project: Spark Issue

[jira] [Commented] (SPARK-15835) The read path of json doesn't support write path when schema contains Options

2016-06-08 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321729#comment-15321729 ] Burak Yavuz commented on SPARK-15835: - cc [~cloud_fan] > The read path of json doesn't support write

[jira] [Created] (SPARK-15835) The read path of json doesn't support write path when schema contains Options

2016-06-08 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-15835: --- Summary: The read path of json doesn't support write path when schema contains Options Key: SPARK-15835 URL: https://issues.apache.org/jira/browse/SPARK-15835 Project:

[jira] [Comment Edited] (SPARK-14767) Codegen "no constructor found" errors with Maps inside case classes in Datasets

2016-06-06 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316904#comment-15316904 ] Burak Yavuz edited comment on SPARK-14767 at 6/6/16 6:11 PM: - I still run

[jira] [Commented] (SPARK-14767) Codegen "no constructor found" errors with Maps inside case classes in Datasets

2016-06-06 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316904#comment-15316904 ] Burak Yavuz commented on SPARK-14767: - I still run into this > Codegen "no constructor found" errors

[jira] [Commented] (SPARK-14767) Codegen "no constructor found" errors with Maps inside case classes in Datasets

2016-04-20 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250540#comment-15250540 ] Burak Yavuz commented on SPARK-14767: - cc [~cloud_fan] [~marmbrus] > Codegen "no constructor found"

[jira] [Updated] (SPARK-14767) Codegen "no constructor found" errors with Maps inside case classes in Datasets

2016-04-20 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-14767: Affects Version/s: 2.0.0 > Codegen "no constructor found" errors with Maps inside case classes in

[jira] [Created] (SPARK-14767) Codegen "no constructor found" errors with Maps inside case classes in Datasets

2016-04-20 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-14767: --- Summary: Codegen "no constructor found" errors with Maps inside case classes in Datasets Key: SPARK-14767 URL: https://issues.apache.org/jira/browse/SPARK-14767

[jira] [Created] (SPARK-14766) Attribute reference mismatch with Dataset filter + mapPartitions

2016-04-20 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-14766: --- Summary: Attribute reference mismatch with Dataset filter + mapPartitions Key: SPARK-14766 URL: https://issues.apache.org/jira/browse/SPARK-14766 Project: Spark

[jira] [Commented] (SPARK-14766) Attribute reference mismatch with Dataset filter + mapPartitions

2016-04-20 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250525#comment-15250525 ] Burak Yavuz commented on SPARK-14766: - cc [~cloud_fan] [~marmbrus] > Attribute reference mismatch

[jira] [Created] (SPARK-14555) Python API for methods introduced for Structured Streaming

2016-04-11 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-14555: --- Summary: Python API for methods introduced for Structured Streaming Key: SPARK-14555 URL: https://issues.apache.org/jira/browse/SPARK-14555 Project: Spark

[jira] [Created] (SPARK-14391) Flaky Test org.apache.spark.launcher.LauncherServerSuite.testCommunication

2016-04-04 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-14391: --- Summary: Flaky Test org.apache.spark.launcher.LauncherServerSuite.testCommunication Key: SPARK-14391 URL: https://issues.apache.org/jira/browse/SPARK-14391 Project:

[jira] [Created] (SPARK-14353) Dateset Time Windowing API for Python, R, and SQL

2016-04-03 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-14353: --- Summary: Dateset Time Windowing API for Python, R, and SQL Key: SPARK-14353 URL: https://issues.apache.org/jira/browse/SPARK-14353 Project: Spark Issue Type:

[jira] [Updated] (SPARK-14287) Method to determine if Dataset is bounded or not

2016-03-30 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-14287: Summary: Method to determine if Dataset is bounded or not (was: isStreaming method for Dataset)

[jira] [Created] (SPARK-14287) isStreaming method for Dataset

2016-03-30 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-14287: --- Summary: isStreaming method for Dataset Key: SPARK-14287 URL: https://issues.apache.org/jira/browse/SPARK-14287 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-14160) Windowing for structured streaming

2016-03-28 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-14160: Description: This JIRA is to track the status regarding event time windowing operations for

[jira] [Updated] (SPARK-14160) Windowing for structured streaming

2016-03-28 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-14160: Description: This JIRA is to track the status regarding event time windowing operations for

[jira] [Updated] (SPARK-14160) Windowing for structured streaming

2016-03-25 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-14160: Description: This JIRA is to track the status regarding event time windowing operations for

[jira] [Updated] (SPARK-14160) Windowing for structured streaming

2016-03-25 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-14160: Description: This JIRA is to track the status regarding event time windowing operations for

[jira] [Updated] (SPARK-14160) Windowing for structured streaming

2016-03-25 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-14160: Description: This JIRA is to track the status regarding event time windowing operations for

[jira] [Updated] (SPARK-14160) Windowing for structured streaming

2016-03-25 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-14160: Description: This JIRA is to track the status regarding event time windowing operations for

[jira] [Created] (SPARK-14160) Windowing for structured streaming

2016-03-25 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-14160: --- Summary: Windowing for structured streaming Key: SPARK-14160 URL: https://issues.apache.org/jira/browse/SPARK-14160 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-12106) Flaky Test: BatchedWriteAheadLog - name log with aggregated entries with the timestamp of last entry

2015-12-02 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-12106: --- Summary: Flaky Test: BatchedWriteAheadLog - name log with aggregated entries with the timestamp of last entry Key: SPARK-12106 URL:

[jira] [Updated] (SPARK-12106) Flaky Test: BatchedWriteAheadLog - name log with aggregated entries with the timestamp of last entry

2015-12-02 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-12106: Description: This test is still transiently flaky, because async methods can finish out of order,

[jira] [Created] (SPARK-11985) Update Spark Streaming - Kinesis Library Documentation regarding data de-aggregation and message handler

2015-11-25 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-11985: --- Summary: Update Spark Streaming - Kinesis Library Documentation regarding data de-aggregation and message handler Key: SPARK-11985 URL:

[jira] [Updated] (SPARK-11985) Update Spark Streaming - Kinesis Library Documentation regarding data de-aggregation and message handler

2015-11-25 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-11985: Description: Update documentation and provide how-to example in guide. > Update Spark Streaming -

[jira] [Created] (SPARK-11731) Enable batching on Driver WriteAheadLog by default

2015-11-13 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-11731: --- Summary: Enable batching on Driver WriteAheadLog by default Key: SPARK-11731 URL: https://issues.apache.org/jira/browse/SPARK-11731 Project: Spark Issue Type:

[jira] [Created] (SPARK-11639) Flaky test: BatchedWriteAheadLog - name log with aggregated entries with the timestamp of last entry

2015-11-10 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-11639: --- Summary: Flaky test: BatchedWriteAheadLog - name log with aggregated entries with the timestamp of last entry Key: SPARK-11639 URL:

[jira] [Commented] (SPARK-11198) Support record de-aggregation in KinesisReceiver

2015-11-02 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14985701#comment-14985701 ] Burak Yavuz commented on SPARK-11198: - Just tested this. It works during regular operation, but

[jira] [Created] (SPARK-11419) WriteAheadLog recovery improvements for when closeFileAfterWrite is enabled

2015-10-30 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-11419: --- Summary: WriteAheadLog recovery improvements for when closeFileAfterWrite is enabled Key: SPARK-11419 URL: https://issues.apache.org/jira/browse/SPARK-11419 Project:

[jira] [Commented] (SPARK-11198) Support record de-aggregation in KinesisReceiver

2015-10-30 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14982926#comment-14982926 ] Burak Yavuz commented on SPARK-11198: - [~boneill42], did you need to do anything special for

[jira] [Created] (SPARK-11324) Flag to close Write Ahead Log after writing

2015-10-26 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-11324: --- Summary: Flag to close Write Ahead Log after writing Key: SPARK-11324 URL: https://issues.apache.org/jira/browse/SPARK-11324 Project: Spark Issue Type:

[jira] [Created] (SPARK-11141) Batching of ReceivedBlockTrackerLogEvents for efficient WAL writes

2015-10-15 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-11141: --- Summary: Batching of ReceivedBlockTrackerLogEvents for efficient WAL writes Key: SPARK-11141 URL: https://issues.apache.org/jira/browse/SPARK-11141 Project: Spark

[jira] [Created] (SPARK-10891) Add MessageHandler to KinesisUtils.createStream similar to Direct Kafka

2015-09-30 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-10891: --- Summary: Add MessageHandler to KinesisUtils.createStream similar to Direct Kafka Key: SPARK-10891 URL: https://issues.apache.org/jira/browse/SPARK-10891 Project: Spark

[jira] [Commented] (SPARK-10889) Upgrade Kinesis Client Library

2015-09-30 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939023#comment-14939023 ] Burak Yavuz commented on SPARK-10889: - In addition, KCL 1.4.0 supports de-aggregation of records. >

[jira] [Updated] (SPARK-10599) Decrease communication in BlockMatrix multiply and increase performance

2015-09-14 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-10599: Description: The BlockMatrix multiply sends each block to all the corresponding columns of the

[jira] [Created] (SPARK-10599) Decrease communication in BlockMatrix multiply and increase performance

2015-09-14 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-10599: --- Summary: Decrease communication in BlockMatrix multiply and increase performance Key: SPARK-10599 URL: https://issues.apache.org/jira/browse/SPARK-10599 Project: Spark

[jira] [Updated] (SPARK-10353) MLlib BLAS gemm outputs wrong result when beta = 0.0 for transpose transpose matrix multiplication

2015-08-29 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-10353: Affects Version/s: 1.5.0 MLlib BLAS gemm outputs wrong result when beta = 0.0 for transpose

[jira] [Created] (SPARK-10353) MLlib BLAS gemm outputs wrong result when beta = 0.0 for transpose transpose matrix multiplication

2015-08-29 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-10353: --- Summary: MLlib BLAS gemm outputs wrong result when beta = 0.0 for transpose transpose matrix multiplication Key: SPARK-10353 URL: https://issues.apache.org/jira/browse/SPARK-10353

[jira] [Created] (SPARK-9916) Clear leftover sparkr.zip copies and creations (e.g. make-distribution.sh)

2015-08-12 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-9916: -- Summary: Clear leftover sparkr.zip copies and creations (e.g. make-distribution.sh) Key: SPARK-9916 URL: https://issues.apache.org/jira/browse/SPARK-9916 Project: Spark

[jira] [Commented] (SPARK-9742) NullPointerException when using --packages

2015-08-07 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662260#comment-14662260 ] Burak Yavuz commented on SPARK-9742: Did the behavior of Option's change for some

[jira] [Commented] (SPARK-9614) InternalRow representation during executionPlan.toRdd.aggregete possibly problematic

2015-08-06 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14661309#comment-14661309 ] Burak Yavuz commented on SPARK-9614: It used to work in Spark 1.4, without Tungsten. I

[jira] [Created] (SPARK-9615) Use rdd.aggregate in FrequentItems

2015-08-04 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-9615: -- Summary: Use rdd.aggregate in FrequentItems Key: SPARK-9615 URL: https://issues.apache.org/jira/browse/SPARK-9615 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-9614) InternalRow representation during executionPlan.toRdd.aggregete possibly problematic

2015-08-04 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-9614: -- Summary: InternalRow representation during executionPlan.toRdd.aggregete possibly problematic Key: SPARK-9614 URL: https://issues.apache.org/jira/browse/SPARK-9614

[jira] [Commented] (SPARK-9614) InternalRow representation during executionPlan.toRdd.aggregete possibly problematic

2015-08-04 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654402#comment-14654402 ] Burak Yavuz commented on SPARK-9614: cc [~joshrosen] InternalRow representation

[jira] [Created] (SPARK-9616) Erroneous result in Frequent Items (SQL) when merging FrequentItemCounters

2015-08-04 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-9616: -- Summary: Erroneous result in Frequent Items (SQL) when merging FrequentItemCounters Key: SPARK-9616 URL: https://issues.apache.org/jira/browse/SPARK-9616 Project: Spark

[jira] [Created] (SPARK-9603) Re-enable complex R package test in SparkSubmitSuite

2015-08-04 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-9603: -- Summary: Re-enable complex R package test in SparkSubmitSuite Key: SPARK-9603 URL: https://issues.apache.org/jira/browse/SPARK-9603 Project: Spark Issue Type:

[jira] [Created] (SPARK-9263) Add Spark Submit flag to exclude dependencies when using --packages

2015-07-22 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-9263: -- Summary: Add Spark Submit flag to exclude dependencies when using --packages Key: SPARK-9263 URL: https://issues.apache.org/jira/browse/SPARK-9263 Project: Spark

[jira] [Updated] (SPARK-6442) MLlib 1.4 Local Linear Algebra Package

2015-07-04 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-6442: --- Description: MLlib's local linear algebra package doesn't have any support for any type of matrix

[jira] [Created] (SPARK-8803) Crosstab element's can't contain null's and back ticks

2015-07-02 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-8803: -- Summary: Crosstab element's can't contain null's and back ticks Key: SPARK-8803 URL: https://issues.apache.org/jira/browse/SPARK-8803 Project: Spark Issue Type:

[jira] [Commented] (SPARK-8599) Use a Random operator to handle Random distribution generating expressions

2015-06-29 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605777#comment-14605777 ] Burak Yavuz commented on SPARK-8599: It would be great if it works for this case as

[jira] [Commented] (SPARK-8410) Hive VersionsSuite RuntimeException

2015-06-29 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605977#comment-14605977 ] Burak Yavuz commented on SPARK-8410: Hi Joe, Is it possible to delete those files

[jira] [Commented] (SPARK-8475) SparkSubmit with Ivy jars is very slow to load with no internet access

2015-06-29 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605968#comment-14605968 ] Burak Yavuz commented on SPARK-8475: ping. I think you can go ahead with a PR for

[jira] [Commented] (SPARK-8410) Hive VersionsSuite RuntimeException

2015-06-29 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14606080#comment-14606080 ] Burak Yavuz commented on SPARK-8410: Hi Joe, Could you please check whether

[jira] [Created] (SPARK-8715) ArrayOutOfBoundsException for DataFrameStatSuite.crosstab

2015-06-29 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-8715: -- Summary: ArrayOutOfBoundsException for DataFrameStatSuite.crosstab Key: SPARK-8715 URL: https://issues.apache.org/jira/browse/SPARK-8715 Project: Spark Issue

[jira] [Created] (SPARK-8681) crosstab column names in wrong order

2015-06-27 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-8681: -- Summary: crosstab column names in wrong order Key: SPARK-8681 URL: https://issues.apache.org/jira/browse/SPARK-8681 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-8608) After initializing a DataFrame with random columns and a seed, df.show should return same value

2015-06-24 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-8608: -- Summary: After initializing a DataFrame with random columns and a seed, df.show should return same value Key: SPARK-8608 URL: https://issues.apache.org/jira/browse/SPARK-8608

[jira] [Created] (SPARK-8609) After initializing a DataFrame with random columns and a seed, ordering by that random column should return same sorted order

2015-06-24 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-8609: -- Summary: After initializing a DataFrame with random columns and a seed, ordering by that random column should return same sorted order Key: SPARK-8609 URL:

[jira] [Commented] (SPARK-8599) Use a Random operator to handle Random distribution generating expressions

2015-06-24 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600315#comment-14600315 ] Burak Yavuz commented on SPARK-8599: cc [~marmbrus] [~rxin] Use a Random operator to

[jira] [Resolved] (SPARK-8095) Spark package dependencies not resolved when package is in local-ivy-cache

2015-06-24 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz resolved SPARK-8095. Resolution: Fixed Spark package dependencies not resolved when package is in local-ivy-cache

[jira] [Commented] (SPARK-8475) SparkSubmit with Ivy jars is very slow to load with no internet access

2015-06-21 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595363#comment-14595363 ] Burak Yavuz commented on SPARK-8475: Me too. I prefer option 1 as well. SparkSubmit

[jira] [Updated] (SPARK-8475) SparkSubmit with Ivy jars is very slow to load with no internet access

2015-06-18 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-8475: --- Issue Type: Improvement (was: Bug) SparkSubmit with Ivy jars is very slow to load with no internet

[jira] [Created] (SPARK-8313) Support Spark Packages containing R code with --packages

2015-06-11 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-8313: -- Summary: Support Spark Packages containing R code with --packages Key: SPARK-8313 URL: https://issues.apache.org/jira/browse/SPARK-8313 Project: Spark Issue

[jira] [Commented] (SPARK-8095) Spark package dependencies not resolved when package is in local-ivy-cache

2015-06-03 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572128#comment-14572128 ] Burak Yavuz commented on SPARK-8095: In the local ivy cache, it should use the

[jira] [Commented] (SPARK-8023) Random Number Generation inconsistent in projections in DataFrame

2015-06-01 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568265#comment-14568265 ] Burak Yavuz commented on SPARK-8023: cc [~yhuai] Random Number Generation

[jira] [Created] (SPARK-8023) Random Number Generation inconsistent in projections in DataFrame

2015-06-01 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-8023: -- Summary: Random Number Generation inconsistent in projections in DataFrame Key: SPARK-8023 URL: https://issues.apache.org/jira/browse/SPARK-8023 Project: Spark

[jira] [Commented] (SPARK-7944) Spark-Shell 2.11 1.4.0-RC-03 does not add jars to class path

2015-05-31 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14566700#comment-14566700 ] Burak Yavuz commented on SPARK-7944: I saw this issue with Yarn when using Scala 2.11

[jira] [Comment Edited] (SPARK-7944) Spark-Shell 2.11 1.4.0-RC-03 does not add jars to class path

2015-05-31 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14566700#comment-14566700 ] Burak Yavuz edited comment on SPARK-7944 at 5/31/15 7:46 PM: -

[jira] [Commented] (SPARK-7982) crosstab should use 0 instead of null for pairs that don't appear

2015-05-31 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14566680#comment-14566680 ] Burak Yavuz commented on SPARK-7982: The reason we used null's instead of 0L was to

[jira] [Created] (SPARK-7957) Preserve partitioning in randomSplit in RDD.scala

2015-05-29 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-7957: -- Summary: Preserve partitioning in randomSplit in RDD.scala Key: SPARK-7957 URL: https://issues.apache.org/jira/browse/SPARK-7957 Project: Spark Issue Type:

[jira] [Commented] (SPARK-7287) Flaky test: o.a.s.deploy.SparkSubmitSuite --packages

2015-05-23 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14557631#comment-14557631 ] Burak Yavuz commented on SPARK-7287: I don't understand why that's failing. It's not

[jira] [Commented] (SPARK-7785) Add missing items to pyspark.mllib.linalg.Matrices

2015-05-21 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14555313#comment-14555313 ] Burak Yavuz commented on SPARK-7785: My belief on the Python linalg api so far has

[jira] [Commented] (SPARK-7785) Add pretty printing to pyspark.mllib.linalg.Matrices

2015-05-21 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14555440#comment-14555440 ] Burak Yavuz commented on SPARK-7785: For operations with BlockMatrix, you will need

[jira] [Created] (SPARK-7745) Replace assertions with requires (IllegalArgumentException) and modify other state checks

2015-05-19 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-7745: -- Summary: Replace assertions with requires (IllegalArgumentException) and modify other state checks Key: SPARK-7745 URL: https://issues.apache.org/jira/browse/SPARK-7745

[jira] [Updated] (SPARK-7381) Missing Python API for o.a.s.ml

2015-05-08 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-7381: --- Summary: Missing Python API for o.a.s.ml (was: Python API for Transformers) Missing Python API for

[jira] [Created] (SPARK-7488) Python API for ml.recommendation

2015-05-08 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-7488: -- Summary: Python API for ml.recommendation Key: SPARK-7488 URL: https://issues.apache.org/jira/browse/SPARK-7488 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-7487) Python API for ml.regression

2015-05-08 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-7487: -- Summary: Python API for ml.regression Key: SPARK-7487 URL: https://issues.apache.org/jira/browse/SPARK-7487 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-7492) Convert LocalDataFrame to LocalMatrix

2015-05-08 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-7492: --- Description: Having a method like, {code: java} Matrices.fromDataFrame(df) {code} would provide

[jira] [Created] (SPARK-7492) Convert LocalDataFrame to LocalMatrix

2015-05-08 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-7492: -- Summary: Convert LocalDataFrame to LocalMatrix Key: SPARK-7492 URL: https://issues.apache.org/jira/browse/SPARK-7492 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-7492) Convert LocalDataFrame to LocalMatrix

2015-05-08 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-7492: --- Description: Having a method like, {code:java} Matrices.fromDataFrame(df) {code} would provide users

[jira] [Reopened] (SPARK-7245) Spearman correlation for DataFrames

2015-05-08 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz reopened SPARK-7245: Sorry, mixed this with Pearson correlation Spearman correlation for DataFrames

[jira] [Resolved] (SPARK-7245) Spearman correlation for DataFrames

2015-05-08 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz resolved SPARK-7245. Resolution: Done Fix Version/s: 1.4.0 Spearman correlation for DataFrames

[jira] [Commented] (SPARK-7486) Add the streaming implementation for estimating quantiles and median

2015-05-08 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535616#comment-14535616 ] Burak Yavuz commented on SPARK-7486: Yes, this is a clone of SPARK-6760 and SPARK-7246

[jira] [Created] (SPARK-7388) Python Api for Param[Array[T]]

2015-05-05 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-7388: -- Summary: Python Api for Param[Array[T]] Key: SPARK-7388 URL: https://issues.apache.org/jira/browse/SPARK-7388 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-7381) Python API for Transformers

2015-05-05 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-7381: -- Summary: Python API for Transformers Key: SPARK-7381 URL: https://issues.apache.org/jira/browse/SPARK-7381 Project: Spark Issue Type: Umbrella

[jira] [Created] (SPARK-7382) Python API for ml.classification

2015-05-05 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-7382: -- Summary: Python API for ml.classification Key: SPARK-7382 URL: https://issues.apache.org/jira/browse/SPARK-7382 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-7383) Python API for ml.feature

2015-05-05 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-7383: -- Summary: Python API for ml.feature Key: SPARK-7383 URL: https://issues.apache.org/jira/browse/SPARK-7383 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-7306) SPARK-7224 broke build with jdk6

2015-05-01 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14523592#comment-14523592 ] Burak Yavuz commented on SPARK-7306: I'll submit a patch using Guava within an hour.

[jira] [Created] (SPARK-7224) Mock repositories for testing with --packages

2015-04-29 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-7224: -- Summary: Mock repositories for testing with --packages Key: SPARK-7224 URL: https://issues.apache.org/jira/browse/SPARK-7224 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-7215) Make repartition and coalesce a part of the query plan

2015-04-28 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-7215: -- Summary: Make repartition and coalesce a part of the query plan Key: SPARK-7215 URL: https://issues.apache.org/jira/browse/SPARK-7215 Project: Spark Issue Type:

[jira] [Created] (SPARK-7205) Support local ivy cache in --packages

2015-04-28 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-7205: -- Summary: Support local ivy cache in --packages Key: SPARK-7205 URL: https://issues.apache.org/jira/browse/SPARK-7205 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-7185) Python API for math functions in DataFrames

2015-04-28 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz resolved SPARK-7185. Resolution: Duplicate Python API for math functions in DataFrames

<    1   2   3   4   >