[jira] [Updated] (SPARK-9550) Configuration renaming, defaults changes, and deprecation for 1.5.0 (master ticket)

[ https://issues.apache.org/jira/browse/SPARK-9550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-9550: -- Description: This ticket tracks configurations which need to be renamed, deprecated, or have their

[jira] [Updated] (SPARK-9550) Configuration renaming, defaults changes, and deprecation for 1.5.0 (master ticket)

[ https://issues.apache.org/jira/browse/SPARK-9550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-9550: -- Description: This ticket tracks configurations which need to be renamed, deprecated, or have their

[jira] [Commented] (SPARK-9550) Configuration renaming, defaults changes, and deprecation for 1.5.0 (master ticket)

[ https://issues.apache.org/jira/browse/SPARK-9550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652001#comment-14652001 ] Josh Rosen commented on SPARK-9550: --- Memory defaults changed (will find JIRA links

[jira] [Commented] (SPARK-9559) Worker redundancy/failover in spark stand-alone mode

[ https://issues.apache.org/jira/browse/SPARK-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14651984#comment-14651984 ] partha bishnu commented on SPARK-9559: -- Thanks. If I understand correctly

[jira] [Comment Edited] (SPARK-9559) Worker redundancy/failover in spark stand-alone mode

[ https://issues.apache.org/jira/browse/SPARK-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14651907#comment-14651907 ] partha bishnu edited comment on SPARK-9559 at 8/3/15 2:24 PM: --

[jira] [Commented] (SPARK-9484) Word2Vec import/export for original binary format

[ https://issues.apache.org/jira/browse/SPARK-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14651899#comment-14651899 ] Manoj Kumar commented on SPARK-9484: I just went through the C code that does the .bin

[jira] [Commented] (SPARK-9559) Worker redundancy/failover in spark stand-alone mode

[ https://issues.apache.org/jira/browse/SPARK-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14651907#comment-14651907 ] partha bishnu commented on SPARK-9559: -- The expected behavior should be that the

[jira] [Commented] (SPARK-9559) Worker redundancy/failover in spark stand-alone mode

[ https://issues.apache.org/jira/browse/SPARK-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14651923#comment-14651923 ] Sean Owen commented on SPARK-9559: -- OK so you have requested 1 total executor. Did the

[jira] [Commented] (SPARK-9559) Worker redundancy/failover in spark stand-alone mode

[ https://issues.apache.org/jira/browse/SPARK-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14651924#comment-14651924 ] Sean Owen commented on SPARK-9559: -- PS you should try reproducing this on master rather

[jira] [Commented] (SPARK-9499) Possible file handle leak in spilling/sort code

[ https://issues.apache.org/jira/browse/SPARK-9499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14651932#comment-14651932 ] Herman van Hovell commented on SPARK-9499: -- I have also tried

[jira] [Comment Edited] (SPARK-9499) Possible file handle leak in spilling/sort code

[ https://issues.apache.org/jira/browse/SPARK-9499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14651932#comment-14651932 ] Herman van Hovell edited comment on SPARK-9499 at 8/3/15 2:46 PM:

[jira] [Updated] (SPARK-9499) Possible file handle leak in spilling/sort code

[ https://issues.apache.org/jira/browse/SPARK-9499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-9499: - Attachment: open.files.II.txt {{lsof}} with {{spark.shuffle.sort.bypassMergeThreshold=0}}

[jira] [Created] (SPARK-9560) Add LDA data generator

yuhao yang created SPARK-9560: - Summary: Add LDA data generator Key: SPARK-9560 URL: https://issues.apache.org/jira/browse/SPARK-9560 Project: Spark Issue Type: New Feature Components:

[jira] [Assigned] (SPARK-9560) Add LDA data generator

[ https://issues.apache.org/jira/browse/SPARK-9560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9560: --- Assignee: Apache Spark Add LDA data generator --

[jira] [Assigned] (SPARK-9560) Add LDA data generator

[ https://issues.apache.org/jira/browse/SPARK-9560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9560: --- Assignee: (was: Apache Spark) Add LDA data generator --

[jira] [Commented] (SPARK-9560) Add LDA data generator

[ https://issues.apache.org/jira/browse/SPARK-9560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652061#comment-14652061 ] Apache Spark commented on SPARK-9560: - User 'hhbyyh' has created a pull request for

[jira] [Commented] (SPARK-9512) RemoveEvaluationFromSort reorders sort order

[ https://issues.apache.org/jira/browse/SPARK-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652620#comment-14652620 ] Apache Spark commented on SPARK-9512: - User 'marmbrus' has created a pull request for

[jira] [Assigned] (SPARK-925) Allow ec2 scripts to load default options from a json file

[ https://issues.apache.org/jira/browse/SPARK-925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-925: -- Assignee: Apache Spark Allow ec2 scripts to load default options from a json file

[jira] [Commented] (SPARK-925) Allow ec2 scripts to load default options from a json file

[ https://issues.apache.org/jira/browse/SPARK-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652622#comment-14652622 ] Apache Spark commented on SPARK-925: User 'marmbrus' has created a pull request for

[jira] [Updated] (SPARK-7165) Sort Merge Join for outer joins

[ https://issues.apache.org/jira/browse/SPARK-7165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7165: --- Sprint: Week 32 Sort Merge Join for outer joins ---

[jira] [Updated] (SPARK-7165) Sort Merge Join for outer joins

[ https://issues.apache.org/jira/browse/SPARK-7165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7165: --- Assignee: Josh Rosen (was: Reynold Xin) Sort Merge Join for outer joins

[jira] [Updated] (SPARK-7799) Move StreamingContext.actorStream to a separate project and deprecate it in StreamingContext

[ https://issues.apache.org/jira/browse/SPARK-7799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-7799: - Target Version/s: 1.6.0 (was: 1.5.0) Move StreamingContext.actorStream to a separate project

[jira] [Updated] (SPARK-4246) Add testsuite with end-to-end testing of driver failure

[ https://issues.apache.org/jira/browse/SPARK-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-4246: - Target Version/s: (was: 1.5.0) Add testsuite with end-to-end testing of driver failure

[jira] [Commented] (SPARK-9131) Python UDFs change data values

[ https://issues.apache.org/jira/browse/SPARK-9131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652694#comment-14652694 ] Davies Liu commented on SPARK-9131: --- I think this maybe fixed by

[jira] [Updated] (SPARK-7441) Implement microbatch functionality so that Spark Streaming can process a large backlog of existing files discovered in batch in smaller batches

[ https://issues.apache.org/jira/browse/SPARK-7441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-7441: - Target Version/s: 1.6.0 (was: 1.5.0) Implement microbatch functionality so that Spark Streaming

[jira] [Updated] (SPARK-6116) DataFrame API improvement umbrella ticket (Spark 1.5)

[ https://issues.apache.org/jira/browse/SPARK-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6116: --- Target Version/s: 1.5.0 (was: 1.6.0) DataFrame API improvement umbrella ticket (Spark 1.5)

[jira] [Updated] (SPARK-6116) DataFrame API improvement umbrella ticket (Spark 1.5)

[ https://issues.apache.org/jira/browse/SPARK-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6116: --- Priority: Critical (was: Blocker) DataFrame API improvement umbrella ticket (Spark 1.5)

[jira] [Updated] (SPARK-9572) Add StreamingContext.getActiveOrCreate() to python API

[ https://issues.apache.org/jira/browse/SPARK-9572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-9572: - Target Version/s: 1.4.2, 1.5.0 (was: 1.5.0) Add StreamingContext.getActiveOrCreate() to python

[jira] [Created] (SPARK-9579) Improve Word2Vec unit tests

Joseph K. Bradley created SPARK-9579: Summary: Improve Word2Vec unit tests Key: SPARK-9579 URL: https://issues.apache.org/jira/browse/SPARK-9579 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-9323) DataFrame.orderBy gives confusing analysis errors when ordering based on nested columns

[ https://issues.apache.org/jira/browse/SPARK-9323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9323: --- Target Version/s: 1.6.0 (was: 1.5.0) DataFrame.orderBy gives confusing analysis errors when

[jira] [Updated] (SPARK-7659) Sort by attributes that are not present in the SELECT clause when there is windowfunction analysis error

[ https://issues.apache.org/jira/browse/SPARK-7659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7659: --- Target Version/s: 1.6.0 (was: 1.5.0) Sort by attributes that are not present in the SELECT clause

[jira] [Assigned] (SPARK-7821) Hide private SQL JDBC classes from Javadoc

[ https://issues.apache.org/jira/browse/SPARK-7821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-7821: -- Assignee: Reynold Xin Hide private SQL JDBC classes from Javadoc

[jira] [Resolved] (SPARK-9263) Add Spark Submit flag to exclude dependencies when using --packages

[ https://issues.apache.org/jira/browse/SPARK-9263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-9263. --- Resolution: Fixed Assignee: Burak Yavuz Fix Version/s: 1.5.0 Add Spark

[jira] [Assigned] (SPARK-9583) build/mvn script should not print debug messages to stdout

[ https://issues.apache.org/jira/browse/SPARK-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9583: --- Assignee: Apache Spark build/mvn script should not print debug messages to stdout

[jira] [Assigned] (SPARK-9583) build/mvn script should not print debug messages to stdout

[ https://issues.apache.org/jira/browse/SPARK-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9583: --- Assignee: (was: Apache Spark) build/mvn script should not print debug messages to

[jira] [Updated] (SPARK-7542) Support off-heap sort buffer in UnsafeExternalSorter

[ https://issues.apache.org/jira/browse/SPARK-7542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7542: --- Issue Type: New Feature (was: Sub-task) Parent: (was: SPARK-9457) Support off-heap sort

[jira] [Commented] (SPARK-9583) build/mvn script should not print debug messages to stdout

[ https://issues.apache.org/jira/browse/SPARK-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652880#comment-14652880 ] Apache Spark commented on SPARK-9583: - User 'vanzin' has created a pull request for

[jira] [Resolved] (SPARK-9457) Sorting improvements

[ https://issues.apache.org/jira/browse/SPARK-9457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-9457. Resolution: Fixed Assignee: Reynold Xin Fix Version/s: 1.5.0 Sorting improvements

[jira] [Assigned] (SPARK-9585) HiveHBaseTableInputFormat can'be cached

[ https://issues.apache.org/jira/browse/SPARK-9585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9585: --- Assignee: Apache Spark HiveHBaseTableInputFormat can'be cached

[jira] [Assigned] (SPARK-9585) HiveHBaseTableInputFormat can'be cached

[ https://issues.apache.org/jira/browse/SPARK-9585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9585: --- Assignee: (was: Apache Spark) HiveHBaseTableInputFormat can'be cached

[jira] [Resolved] (SPARK-8064) Upgrade Hive to 1.2

[ https://issues.apache.org/jira/browse/SPARK-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-8064. - Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7191

[jira] [Commented] (SPARK-9516) Improve Thread Dump page

[ https://issues.apache.org/jira/browse/SPARK-9516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652701#comment-14652701 ] Apache Spark commented on SPARK-9516: - User 'CodingCat' has created a pull request for

[jira] [Updated] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2870: --- Parent Issue: SPARK-9576 (was: SPARK-6116) Thorough schema inference directly on RDDs of Python

[jira] [Updated] (SPARK-9392) Dataframe drop should work on unresolved columns

[ https://issues.apache.org/jira/browse/SPARK-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9392: --- Parent Issue: SPARK-9576 (was: SPARK-6116) Dataframe drop should work on unresolved columns

[jira] [Updated] (SPARK-8000) SQLContext.read.load() should be able to auto-detect input data

[ https://issues.apache.org/jira/browse/SPARK-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8000: --- Parent Issue: SPARK-9576 (was: SPARK-6116) SQLContext.read.load() should be able to auto-detect

[jira] [Commented] (SPARK-7160) Support converting DataFrames to typed RDDs.

[ https://issues.apache.org/jira/browse/SPARK-7160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652851#comment-14652851 ] Michael Armbrust commented on SPARK-7160: - I spent about and hour trying to fix

[jira] [Created] (SPARK-9580) Refactor TestSQLContext to make it non-singleton

Andrew Or created SPARK-9580: Summary: Refactor TestSQLContext to make it non-singleton Key: SPARK-9580 URL: https://issues.apache.org/jira/browse/SPARK-9580 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-9582) Improve clarity of LocalLDAModel log likelihood methods

Joseph K. Bradley created SPARK-9582: Summary: Improve clarity of LocalLDAModel log likelihood methods Key: SPARK-9582 URL: https://issues.apache.org/jira/browse/SPARK-9582 Project: Spark

[jira] [Reopened] (SPARK-9372) For a join operator, rows with null equal join key expression can be filtered out early

[ https://issues.apache.org/jira/browse/SPARK-9372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reopened SPARK-9372: I reverted the merged patch since it had a few problems. For a join operator, rows with null equal

[jira] [Updated] (SPARK-9372) For a join operator, rows with null equal join key expression can be filtered out early

[ https://issues.apache.org/jira/browse/SPARK-9372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9372: --- Target Version/s: 1.6.0 For a join operator, rows with null equal join key expression can be

[jira] [Updated] (SPARK-9372) For a join operator, rows with null equal join key expression can be filtered out early

[ https://issues.apache.org/jira/browse/SPARK-9372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9372: --- Fix Version/s: (was: 1.5.0) For a join operator, rows with null equal join key expression can be

[jira] [Assigned] (SPARK-9575) Add documentation around Mesos shuffle service and dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-9575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9575: --- Assignee: Apache Spark Add documentation around Mesos shuffle service and dynamic

[jira] [Commented] (SPARK-9575) Add documentation around Mesos shuffle service and dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-9575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652634#comment-14652634 ] Apache Spark commented on SPARK-9575: - User 'tnachen' has created a pull request for

[jira] [Assigned] (SPARK-9575) Add documentation around Mesos shuffle service and dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-9575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9575: --- Assignee: (was: Apache Spark) Add documentation around Mesos shuffle service and

[jira] [Created] (SPARK-9575) Add documentation around Mesos shuffle service and dynamic allocation

Timothy Chen created SPARK-9575: --- Summary: Add documentation around Mesos shuffle service and dynamic allocation Key: SPARK-9575 URL: https://issues.apache.org/jira/browse/SPARK-9575 Project: Spark

[jira] [Commented] (SPARK-7791) Set user for executors in standalone-mode

[ https://issues.apache.org/jira/browse/SPARK-7791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652660#comment-14652660 ] Niels Becker commented on SPARK-7791: - We endet up using your workaround. But since

[jira] [Assigned] (SPARK-9228) Combine unsafe and codegen into a single option

[ https://issues.apache.org/jira/browse/SPARK-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9228: --- Assignee: Michael Armbrust (was: Apache Spark) Combine unsafe and codegen into a single

[jira] [Commented] (SPARK-9482) flaky test: org.apache.spark.sql.hive.execution.HiveCompatibilitySuite.semijoin

[ https://issues.apache.org/jira/browse/SPARK-9482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652659#comment-14652659 ] Davies Liu commented on SPARK-9482: --- The Physical plan looks very strange, it use unsafe

[jira] [Commented] (SPARK-9228) Combine unsafe and codegen into a single option

[ https://issues.apache.org/jira/browse/SPARK-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652661#comment-14652661 ] Apache Spark commented on SPARK-9228: - User 'marmbrus' has created a pull request for

[jira] [Assigned] (SPARK-9516) Improve Thread Dump page

[ https://issues.apache.org/jira/browse/SPARK-9516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9516: --- Assignee: (was: Apache Spark) Improve Thread Dump page

[jira] [Assigned] (SPARK-9516) Improve Thread Dump page

[ https://issues.apache.org/jira/browse/SPARK-9516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9516: --- Assignee: Apache Spark Improve Thread Dump page

[jira] [Closed] (SPARK-9054) Rename RowOrdering to InterpretedOrdering and use newOrdering to build orderings

[ https://issues.apache.org/jira/browse/SPARK-9054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-9054. -- Resolution: Won't Fix [~joshrosen] closing this as won't fix for now. We can reopen later if needed.

[jira] [Updated] (SPARK-9482) flaky test: org.apache.spark.sql.hive.execution.HiveCompatibilitySuite.semijoin

[ https://issues.apache.org/jira/browse/SPARK-9482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9482: --- Sprint: Week 32 flaky test: org.apache.spark.sql.hive.execution.HiveCompatibilitySuite.semijoin

[jira] [Commented] (SPARK-8064) Upgrade Hive to 1.2

[ https://issues.apache.org/jira/browse/SPARK-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652730#comment-14652730 ] Steve Loughran commented on SPARK-8064: --- Also: we had to produce a custom release of

[jira] [Created] (SPARK-9578) Stemmer feature transformer

Joseph K. Bradley created SPARK-9578: Summary: Stemmer feature transformer Key: SPARK-9578 URL: https://issues.apache.org/jira/browse/SPARK-9578 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-5571) LDA should handle text as well

[ https://issues.apache.org/jira/browse/SPARK-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652787#comment-14652787 ] Joseph K. Bradley commented on SPARK-5571: -- The stopwords transformer made it for

[jira] [Updated] (SPARK-8887) Explicitly define which data types can be used as dynamic partition columns

[ https://issues.apache.org/jira/browse/SPARK-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-8887: -- Target Version/s: 1.5.0 (was: 1.6.0) Explicitly define which data types can be used as dynamic

[jira] [Assigned] (SPARK-8887) Explicitly define which data types can be used as dynamic partition columns

[ https://issues.apache.org/jira/browse/SPARK-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-8887: - Assignee: Cheng Lian Explicitly define which data types can be used as dynamic partition

[jira] [Updated] (SPARK-9257) Fix the false negative of Aggregate2Sort and FinalAndCompleteAggregate2Sort's missingInput

[ https://issues.apache.org/jira/browse/SPARK-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9257: --- Assignee: Yin Huai Fix the false negative of Aggregate2Sort and FinalAndCompleteAggregate2Sort's

[jira] [Commented] (SPARK-9251) do not order by expressions which still need evaluation

[ https://issues.apache.org/jira/browse/SPARK-9251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652675#comment-14652675 ] Apache Spark commented on SPARK-9251: - User 'marmbrus' has created a pull request for

[jira] [Assigned] (SPARK-9513) Create Python API for all SQL functions

[ https://issues.apache.org/jira/browse/SPARK-9513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-9513: - Assignee: Davies Liu Create Python API for all SQL functions

[jira] [Updated] (SPARK-6116) DataFrame API improvement umbrella ticket (Spark 1.5)

[ https://issues.apache.org/jira/browse/SPARK-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6116: --- Summary: DataFrame API improvement umbrella ticket (Spark 1.5) (was: DataFrame API improvement

[jira] [Resolved] (SPARK-8416) Thread dump page should highlight Spark executor threads

[ https://issues.apache.org/jira/browse/SPARK-8416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-8416. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7808

[jira] [Updated] (SPARK-8466) Bug in SQL Optimizer: Unresolved Attribute after pushing Filter below Project

[ https://issues.apache.org/jira/browse/SPARK-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8466: Description: Input Data: a parquet file stored in hdfs:///data with two columns

[jira] [Updated] (SPARK-9582) LDA cleanups

[ https://issues.apache.org/jira/browse/SPARK-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9582: - Description: Small cleanups to LDA code and recent additions CC: [~fliang] was:

[jira] [Commented] (SPARK-9559) Worker redundancy/failover in spark stand-alone mode

[ https://issues.apache.org/jira/browse/SPARK-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652933#comment-14652933 ] partha bishnu commented on SPARK-9559: -- We tested on 1.4.1 and got same results i.e.

[jira] [Updated] (SPARK-9582) LDA cleanups

[ https://issues.apache.org/jira/browse/SPARK-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9582: - Priority: Minor (was: Major) LDA cleanups Key:

[jira] [Updated] (SPARK-9582) LDA cleanups

[ https://issues.apache.org/jira/browse/SPARK-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9582: - Summary: LDA cleanups (was: Improve clarity of LocalLDAModel log likelihood methods)

[jira] [Created] (SPARK-9584) HiveHBaseTableInputFormat can'be cached

meiyoula created SPARK-9584: --- Summary: HiveHBaseTableInputFormat can'be cached Key: SPARK-9584 URL: https://issues.apache.org/jira/browse/SPARK-9584 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-9585) HiveHBaseTableInputFormat can'be cached

[ https://issues.apache.org/jira/browse/SPARK-9585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652941#comment-14652941 ] Apache Spark commented on SPARK-9585: - User 'XuTingjun' has created a pull request for

[jira] [Created] (SPARK-9585) HiveHBaseTableInputFormat can'be cached

meiyoula created SPARK-9585: --- Summary: HiveHBaseTableInputFormat can'be cached Key: SPARK-9585 URL: https://issues.apache.org/jira/browse/SPARK-9585 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-9228) Combine unsafe and codegen into a single option

[ https://issues.apache.org/jira/browse/SPARK-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9228: --- Assignee: Apache Spark (was: Michael Armbrust) Combine unsafe and codegen into a single

[jira] [Updated] (SPARK-7505) Update PySpark DataFrame docs: encourage __getitem__, mark as experimental, etc.

[ https://issues.apache.org/jira/browse/SPARK-7505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7505: --- Target Version/s: 1.5.0 (was: 1.6.0) Update PySpark DataFrame docs: encourage __getitem__, mark as

[jira] [Updated] (SPARK-7544) pyspark.sql.types.Row should implement __getitem__

[ https://issues.apache.org/jira/browse/SPARK-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7544: --- Parent Issue: SPARK-9576 (was: SPARK-6116) pyspark.sql.types.Row should implement __getitem__

[jira] [Updated] (SPARK-5517) Add input types for Java UDFs

[ https://issues.apache.org/jira/browse/SPARK-5517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5517: --- Parent Issue: SPARK-9576 (was: SPARK-6116) Add input types for Java UDFs

[jira] [Updated] (SPARK-7400) PortableDataStream UDT

[ https://issues.apache.org/jira/browse/SPARK-7400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7400: --- Parent Issue: SPARK-9576 (was: SPARK-6116) PortableDataStream UDT --

[jira] [Updated] (SPARK-8802) Decimal.apply(BigDecimal).toBigDecimal may throw NumberFormatException

[ https://issues.apache.org/jira/browse/SPARK-8802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8802: --- Target Version/s: 1.6.0 (was: 1.5.0) Decimal.apply(BigDecimal).toBigDecimal may throw

[jira] [Created] (SPARK-9577) Surface concrete iterator types in various sort classes

Reynold Xin created SPARK-9577: -- Summary: Surface concrete iterator types in various sort classes Key: SPARK-9577 URL: https://issues.apache.org/jira/browse/SPARK-9577 Project: Spark Issue

[jira] [Commented] (SPARK-9577) Surface concrete iterator types in various sort classes

[ https://issues.apache.org/jira/browse/SPARK-9577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652768#comment-14652768 ] Apache Spark commented on SPARK-9577: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-9577) Surface concrete iterator types in various sort classes

[ https://issues.apache.org/jira/browse/SPARK-9577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9577: --- Assignee: Reynold Xin (was: Apache Spark) Surface concrete iterator types in various sort

[jira] [Updated] (SPARK-8874) Add missing methods in Word2Vec ML

[ https://issues.apache.org/jira/browse/SPARK-8874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8874: - Shepherd: Joseph K. Bradley Target Version/s: 1.5.0 Add missing methods in

[jira] [Resolved] (SPARK-9483) UTF8String.getPrefix only works in little-endian order

[ https://issues.apache.org/jira/browse/SPARK-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9483. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7902

[jira] [Closed] (SPARK-8891) Calling aggregation expressions on null literals fails at runtime

[ https://issues.apache.org/jira/browse/SPARK-8891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-8891. -- Resolution: Fixed Assignee: Yin Huai (was: Josh Rosen) Fix Version/s: 1.5.0 Fixed by

[jira] [Updated] (SPARK-9526) Utilize randomized tests to reveal potential bugs in sql expressions

[ https://issues.apache.org/jira/browse/SPARK-9526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9526: --- Shepherd: Josh Rosen Assignee: Yijie Shen Utilize randomized tests to reveal potential bugs in

[jira] [Updated] (SPARK-9403) Implement code generation for In / InSet

[ https://issues.apache.org/jira/browse/SPARK-9403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9403: --- Shepherd: Davies Liu Implement code generation for In / InSet

[jira] [Assigned] (SPARK-9581) Add test for JSON UDTs

[ https://issues.apache.org/jira/browse/SPARK-9581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9581: --- Assignee: Apache Spark (was: Reynold Xin) Add test for JSON UDTs --

[jira] [Assigned] (SPARK-9581) Add test for JSON UDTs

[ https://issues.apache.org/jira/browse/SPARK-9581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9581: --- Assignee: Reynold Xin (was: Apache Spark) Add test for JSON UDTs --

[jira] [Commented] (SPARK-9581) Add test for JSON UDTs

[ https://issues.apache.org/jira/browse/SPARK-9581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652899#comment-14652899 ] Apache Spark commented on SPARK-9581: - User 'rxin' has created a pull request for this

[jira] [Updated] (SPARK-7119) ScriptTransform doesn't consider the output data type

[ https://issues.apache.org/jira/browse/SPARK-7119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-7119: Target Version/s: 1.5.0 (was: 1.6.0) ScriptTransform doesn't consider the output data

[jira] [Issue Comment Deleted] (SPARK-7148) Configure Parquet block size (row group size) for ML model import/export

[ https://issues.apache.org/jira/browse/SPARK-7148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-7148: --- Comment: was deleted (was: [~josephkb] If you are busy with other issues, please don't hesitate to

  1   2   3   4   5   >