[jira] [Created] (SPARK-27320) Converting seq to array in AggregationIterator to improve its access performance

eaton created SPARK-27320: - Summary: Converting seq to array in AggregationIterator to improve its access performance Key: SPARK-27320 URL: https://issues.apache.org/jira/browse/SPARK-27320 Project: Spark

[jira] [Created] (SPARK-26746) Adaptive causes non-action operations to trigger computation

eaton created SPARK-26746: - Summary: Adaptive causes non-action operations to trigger computation Key: SPARK-26746 URL: https://issues.apache.org/jira/browse/SPARK-26746 Project: Spark Issue Type:

[jira] [Created] (SPARK-26599) BroardCast hint can not work with PruneFileSourcePartitions

eaton created SPARK-26599: - Summary: BroardCast hint can not work with PruneFileSourcePartitions Key: SPARK-26599 URL: https://issues.apache.org/jira/browse/SPARK-26599 Project: Spark Issue Type:

[jira] [Commented] (SPARK-26567) Should we align CSV query results with hive text query results: an int field, if the input value is 1.0, hive text query results is 1, CSV query results is null

[ https://issues.apache.org/jira/browse/SPARK-26567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736685#comment-16736685 ] eaton commented on SPARK-26567: --- Ok. [~hyukjin.kwon] > Should we align CSV query results with hive text

[jira] [Updated] (SPARK-26567) Should we align CSV query results with hive text query results: an int field, if the input value is 1.0, hive text query results is 1, CSV query results is null

[ https://issues.apache.org/jira/browse/SPARK-26567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eaton updated SPARK-26567: -- Description: If we want to be consistent, we can modify the makeConverter function in UnivocityParser, but

[jira] [Updated] (SPARK-26567) Should we align CSV query results with hive text query results: an int field, if the input value is 1.0, hive text query results is 1, CSV query results is null

[ https://issues.apache.org/jira/browse/SPARK-26567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eaton updated SPARK-26567: -- Description: If we want to be consistent, we can modify the makeConverter function in UnivocityParser, but

[jira] [Updated] (SPARK-26567) Should we align CSV query results with hive text query results: an int field, if the input value is 1.0, hive text query results is 1, CSV query results is null

[ https://issues.apache.org/jira/browse/SPARK-26567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eaton updated SPARK-26567: -- Description: If we want to be consistent, we can modify the makeConverter function in UnivocityParser, but

[jira] [Updated] (SPARK-26567) Should we align CSV query results with hive text query results: an int field, if the input value is 1.0, hive text query results is 1, CSV query results is null

[ https://issues.apache.org/jira/browse/SPARK-26567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eaton updated SPARK-26567: -- Description: If we want to be consistent, we can modify the makeConverter function in UnivocityParser, but

[jira] [Created] (SPARK-26567) Should we align CSV query results with hive text query results: an int field, if the input value is 1.0, hive text query results is 1, CSV query results is null

eaton created SPARK-26567: - Summary: Should we align CSV query results with hive text query results: an int field, if the input value is 1.0, hive text query results is 1, CSV query results is null Key: SPARK-26567 URL:

[jira] [Created] (SPARK-26312) Converting converters in RDDConversions into arrays to improve their access performance

eaton created SPARK-26312: - Summary: Converting converters in RDDConversions into arrays to improve their access performance Key: SPARK-26312 URL: https://issues.apache.org/jira/browse/SPARK-26312 Project:

[jira] [Updated] (SPARK-26012) Dynamic partition will fail when both '' and null values are taken as dynamic partition values simultaneously.

[ https://issues.apache.org/jira/browse/SPARK-26012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eaton updated SPARK-26012: -- Issue Type: Bug (was: Improvement) > Dynamic partition will fail when both '' and null values are taken as

[jira] [Updated] (SPARK-26012) Dynamic partition will fail when both '' and null values are taken as dynamic partition values simultaneously.

[ https://issues.apache.org/jira/browse/SPARK-26012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eaton updated SPARK-26012: -- Description: Dynamic partition will fail when both '' and null values are taken as dynamic partition values

[jira] [Created] (SPARK-26012) Dynamic partition will fail when both '' and null values are taken as dynamic partition values simultaneously.

eaton created SPARK-26012: - Summary: Dynamic partition will fail when both '' and null values are taken as dynamic partition values simultaneously. Key: SPARK-26012 URL: https://issues.apache.org/jira/browse/SPARK-26012

[jira] [Created] (SPARK-25548) In the PruneFileSourcePartitions optimizer, replace the nonPartitionOps field with true in the And(partitionOps, nonPartitionOps) to make the partition can be pruned

eaton created SPARK-25548: - Summary: In the PruneFileSourcePartitions optimizer, replace the nonPartitionOps field with true in the And(partitionOps, nonPartitionOps) to make the partition can be pruned Key: SPARK-25548

[jira] [Created] (SPARK-25069) Using UnsafeAlignedOffset to make the entire record of 8 byte Items aligned like which is used in UnsafeExternalSorter

eaton created SPARK-25069: - Summary: Using UnsafeAlignedOffset to make the entire record of 8 byte Items aligned like which is used in UnsafeExternalSorter Key: SPARK-25069 URL:

[jira] [Created] (SPARK-24870) Cache can't work normally if there are case letters in SQL

eaton created SPARK-24870: - Summary: Cache can't work normally if there are case letters in SQL Key: SPARK-24870 URL: https://issues.apache.org/jira/browse/SPARK-24870 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-23998) It may be better to add @transient to field 'taskMemoryManager' in class Task, for it is only be set and used in executor side

[ https://issues.apache.org/jira/browse/SPARK-23998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eaton resolved SPARK-23998. --- Resolution: Won't Do > It may be better to add @transient to field 'taskMemoryManager' in class > Task,

[jira] [Created] (SPARK-23998) It may be better to add @transient to field 'taskMemoryManager' in class Task, for it is only be set and used in executor side

eaton created SPARK-23998: - Summary: It may be better to add @transient to field 'taskMemoryManager' in class Task, for it is only be set and used in executor side Key: SPARK-23998 URL:

[jira] [Created] (SPARK-22606) There may be two or more tasks in one executor will use the same kafka consumer at the same time, then it will throw an exception: "KafkaConsumer is not safe for multi-t

eaton created SPARK-22606: - Summary: There may be two or more tasks in one executor will use the same kafka consumer at the same time, then it will throw an exception: "KafkaConsumer is not safe for multi-threaded access" Key:

[jira] [Comment Edited] (SPARK-21807) The getAliasedConstraints function in LogicalPlan will take a long time when number of expressions is greater than 100

[ https://issues.apache.org/jira/browse/SPARK-21807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137920#comment-16137920 ] eaton edited comment on SPARK-21807 at 8/23/17 5:35 AM: Yes, I have got it,

[jira] [Commented] (SPARK-21807) The getAliasedConstraints function in LogicalPlan will take a long time when number of expressions is greater than 100

[ https://issues.apache.org/jira/browse/SPARK-21807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137920#comment-16137920 ] eaton commented on SPARK-21807: --- Yes, I have got it, thanks, but maybe we can improve it by reducing the

[jira] [Updated] (SPARK-21807) The getAliasedConstraints function in LogicalPlan will take a long time when number of expressions is greater than 100

[ https://issues.apache.org/jira/browse/SPARK-21807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eaton updated SPARK-21807: -- Description: The getAliasedConstraints fuction in LogicalPlan.scala will clone the expression set when an

[jira] [Created] (SPARK-21807) The getAliasedConstraints function in LogicalPlan will take a long time when number of expressions is greater than 100

eaton created SPARK-21807: - Summary: The getAliasedConstraints function in LogicalPlan will take a long time when number of expressions is greater than 100 Key: SPARK-21807 URL:

[jira] [Created] (SPARK-21603) The wholestage codegen will be much slower then wholestage codegen is closed when the function is too long

eaton created SPARK-21603: - Summary: The wholestage codegen will be much slower then wholestage codegen is closed when the function is too long Key: SPARK-21603 URL: https://issues.apache.org/jira/browse/SPARK-21603

[jira] [Created] (SPARK-21115) If the cores left is less than the coresPerExecutor,the cores left will not be allocated, so it should not to check in every schedule

eaton created SPARK-21115: - Summary: If the cores left is less than the coresPerExecutor,the cores left will not be allocated, so it should not to check in every schedule Key: SPARK-21115 URL:

[jira] [Created] (SPARK-21017) Move the length getter before the while to improve performance

eaton created SPARK-21017: - Summary: Move the length getter before the while to improve performance Key: SPARK-21017 URL: https://issues.apache.org/jira/browse/SPARK-21017 Project: Spark Issue

[jira] [Created] (SPARK-20835) It should exit directly when the --total-executor-cores parameter is setted less than 0 when submit a application

eaton created SPARK-20835: - Summary: It should exit directly when the --total-executor-cores parameter is setted less than 0 when submit a application Key: SPARK-20835 URL:

[jira] [Created] (SPARK-20386) The log info "Added %s in memory on %s (size: %s, free: %s)" in function "org.apache.spark.storage.BlockManagerInfo.updateBlockInfo" is not accurate if the block exists

eaton created SPARK-20386: - Summary: The log info "Added %s in memory on %s (size: %s, free: %s)" in function "org.apache.spark.storage.BlockManagerInfo.updateBlockInfo" is not accurate if the block exists on the slave already Key: