[jira] [Created] (SPARK-19484) continue work to create a table with an empty schema

2017-02-06 Thread Song Jun (JIRA)
Song Jun created SPARK-19484: Summary: continue work to create a table with an empty schema Key: SPARK-19484 URL: https://issues.apache.org/jira/browse/SPARK-19484 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19477) [SQL] Datasets created from a Dataframe with extra columns retain the extra columns

2017-02-07 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15855677#comment-15855677 ] Song Jun commented on SPARK-19477: -- thanks, I got it~ > [SQL] Datasets created from a Dataframe with

[jira] [Created] (SPARK-19491) add a config for tableRelation cache size in SessionCatalog

2017-02-07 Thread Song Jun (JIRA)
Song Jun created SPARK-19491: Summary: add a config for tableRelation cache size in SessionCatalog Key: SPARK-19491 URL: https://issues.apache.org/jira/browse/SPARK-19491 Project: Spark Issue

[jira] [Commented] (SPARK-19496) to_date with format has weird behavior

2017-02-07 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856205#comment-15856205 ] Song Jun commented on SPARK-19496: -- I am working on this~ > to_date with format has weird behavior >

[jira] [Created] (SPARK-19463) refresh the table cache after InsertIntoHadoopFsRelation

2017-02-05 Thread Song Jun (JIRA)
Song Jun created SPARK-19463: Summary: refresh the table cache after InsertIntoHadoopFsRelation Key: SPARK-19463 URL: https://issues.apache.org/jira/browse/SPARK-19463 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-19496) to_date with format has weird behavior

2017-02-07 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857541#comment-15857541 ] Song Jun edited comment on SPARK-19496 at 2/8/17 7:11 AM: -- mysql: select

[jira] [Commented] (SPARK-19496) to_date with format has weird behavior

2017-02-07 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857570#comment-15857570 ] Song Jun commented on SPARK-19496: -- [~hyukjin.kwon]  > to_date with format has weird behavior >

[jira] [Comment Edited] (SPARK-19496) to_date with format has weird behavior

2017-02-07 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857541#comment-15857541 ] Song Jun edited comment on SPARK-19496 at 2/8/17 7:18 AM: -- mysql: select

[jira] [Commented] (SPARK-19496) to_date with format has weird behavior

2017-02-07 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857541#comment-15857541 ] Song Jun commented on SPARK-19496: -- mysql: select str_to_date('2014-12-31','%Y-%d-%m') also return

[jira] [Comment Edited] (SPARK-19496) to_date with format has weird behavior

2017-02-07 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857541#comment-15857541 ] Song Jun edited comment on SPARK-19496 at 2/8/17 7:09 AM: -- mysql: select

[jira] [Created] (SPARK-19458) loading hive jars from the local repo which has already downloaded

2017-02-04 Thread Song Jun (JIRA)
Song Jun created SPARK-19458: Summary: loading hive jars from the local repo which has already downloaded Key: SPARK-19458 URL: https://issues.apache.org/jira/browse/SPARK-19458 Project: Spark

[jira] [Commented] (SPARK-19430) Cannot read external tables with VARCHAR columns if they're backed by ORC files written by Hive 1.2.1

2017-02-05 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15853466#comment-15853466 ] Song Jun commented on SPARK-19430: -- I think this is not a bug. If you want to access the hive table ,you

[jira] [Commented] (SPARK-19447) Fix input metrics for range operator

2017-02-05 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15853462#comment-15853462 ] Song Jun commented on SPARK-19447: -- spark.range(1,100).show there is some information in the SQL UI

[jira] [Created] (SPARK-19448) unify some duplication function in MetaStoreRelation

2017-02-03 Thread Song Jun (JIRA)
Song Jun created SPARK-19448: Summary: unify some duplication function in MetaStoreRelation Key: SPARK-19448 URL: https://issues.apache.org/jira/browse/SPARK-19448 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-19257) The type of CatalogStorageFormat.locationUri should be java.net.URI instead of String

2017-01-21 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832935#comment-15832935 ] Song Jun edited comment on SPARK-19257 at 1/21/17 11:09 AM: I found it that

[jira] [Commented] (SPARK-19257) The type of CatalogStorageFormat.locationUri should be java.net.URI instead of String

2017-01-21 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832935#comment-15832935 ] Song Jun commented on SPARK-19257: -- I found it than `CatalogTablePartition` and `CatalogTable` use the

[jira] [Created] (SPARK-19359) partition path created by Hive should be deleted after rename a partition with upper-case

2017-01-24 Thread Song Jun (JIRA)
Song Jun created SPARK-19359: Summary: partition path created by Hive should be deleted after rename a partition with upper-case Key: SPARK-19359 URL: https://issues.apache.org/jira/browse/SPARK-19359

[jira] [Updated] (SPARK-19359) partition path created by Hive should be deleted after rename a partition with upper-case

2017-01-24 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-19359: - Issue Type: Improvement (was: Bug) > partition path created by Hive should be deleted after rename a

[jira] [Commented] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

2017-01-25 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837407#comment-15837407 ] Song Jun commented on SPARK-19340: -- the reason is that spark sql treat the test{00-1}.txt as a globpath.

[jira] [Comment Edited] (SPARK-19257) The type of CatalogStorageFormat.locationUri should be java.net.URI instead of String

2017-01-21 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832935#comment-15832935 ] Song Jun edited comment on SPARK-19257 at 1/22/17 6:41 AM: --- I found it that

[jira] [Comment Edited] (SPARK-19257) The type of CatalogStorageFormat.locationUri should be java.net.URI instead of String

2017-01-21 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832935#comment-15832935 ] Song Jun edited comment on SPARK-19257 at 1/22/17 6:00 AM: --- I found it that

[jira] [Comment Edited] (SPARK-19257) The type of CatalogStorageFormat.locationUri should be java.net.URI instead of String

2017-01-21 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832935#comment-15832935 ] Song Jun edited comment on SPARK-19257 at 1/22/17 6:42 AM: --- I found it that

[jira] [Updated] (SPARK-19332) table's location should check if a URI is legal

2017-01-22 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-19332: - Description: SPARK-19257 ‘s work is to change the type of `CatalogStorageFormat` 's locationUri to

[jira] [Created] (SPARK-19332) table's location should check if a URI is legal

2017-01-22 Thread Song Jun (JIRA)
Song Jun created SPARK-19332: Summary: table's location should check if a URI is legal Key: SPARK-19332 URL: https://issues.apache.org/jira/browse/SPARK-19332 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-19329) after alter a datasource table's location to a not exist location and then insert data throw Exception

2017-01-22 Thread Song Jun (JIRA)
Song Jun created SPARK-19329: Summary: after alter a datasource table's location to a not exist location and then insert data throw Exception Key: SPARK-19329 URL: https://issues.apache.org/jira/browse/SPARK-19329

[jira] [Comment Edited] (SPARK-19257) The type of CatalogStorageFormat.locationUri should be java.net.URI instead of String

2017-01-22 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832935#comment-15832935 ] Song Jun edited comment on SPARK-19257 at 1/22/17 11:22 AM: I found it that

[jira] [Created] (SPARK-19667) Create table with HiveEnabled in default database use warehouse path instead of the location of default database

2017-02-20 Thread Song Jun (JIRA)
Song Jun created SPARK-19667: Summary: Create table with HiveEnabled in default database use warehouse path instead of the location of default database Key: SPARK-19667 URL:

[jira] [Created] (SPARK-19723) create table for data source tables should work with an non-existent location

2017-02-23 Thread Song Jun (JIRA)
Song Jun created SPARK-19723: Summary: create table for data source tables should work with an non-existent location Key: SPARK-19723 URL: https://issues.apache.org/jira/browse/SPARK-19723 Project: Spark

[jira] [Created] (SPARK-19724) create table for hive tables with an existed default location should throw an exception

2017-02-24 Thread Song Jun (JIRA)
Song Jun created SPARK-19724: Summary: create table for hive tables with an existed default location should throw an exception Key: SPARK-19724 URL: https://issues.apache.org/jira/browse/SPARK-19724

[jira] [Updated] (SPARK-19724) create managed table for hive tables with an existed default location should throw an exception

2017-02-24 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-19724: - Summary: create managed table for hive tables with an existed default location should throw an exception

[jira] [Updated] (SPARK-19724) create managed table for hive tables with an existed default location should throw an exception

2017-02-24 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-19724: - Description: This JIRA is a follow up work after

[jira] [Updated] (SPARK-19724) create a managed table with an existed default location should throw an exception

2017-02-24 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-19724: - Summary: create a managed table with an existed default location should throw an exception (was: create

[jira] [Updated] (SPARK-19664) put 'hive.metastore.warehouse.dir' in hadoopConf place

2017-02-19 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-19664: - Description: In SPARK-15959, we bring back the 'hive.metastore.warehouse.dir' , while in the logic,

[jira] [Created] (SPARK-19664) put 'hive.metastore.warehouse.dir' in hadoopConf place

2017-02-19 Thread Song Jun (JIRA)
Song Jun created SPARK-19664: Summary: put 'hive.metastore.warehouse.dir' in hadoopConf place Key: SPARK-19664 URL: https://issues.apache.org/jira/browse/SPARK-19664 Project: Spark Issue Type:

[jira] [Closed] (SPARK-19484) continue work to create a table with an empty schema

2017-02-14 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun closed SPARK-19484. Resolution: Won't Fix this has been contained in https://github.com/apache/spark/pull/16787 > continue

[jira] [Closed] (SPARK-19491) add a config for tableRelation cache size in SessionCatalog

2017-02-14 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun closed SPARK-19491. Resolution: Duplicate duplicate with https://github.com/apache/spark/pull/16736 > add a config for

[jira] [Commented] (SPARK-19558) Provide a config option to attach QueryExecutionListener to SparkSession

2017-02-12 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862768#comment-15862768 ] Song Jun commented on SPARK-19558: -- sparkSession.listenerManager.register is not enough? > Provide a

[jira] [Commented] (SPARK-19583) CTAS for data source tables with an created location does not work

2017-02-13 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865111#comment-15865111 ] Song Jun commented on SPARK-19583: -- ok, I'd like to take this one, thanks a lot! > CTAS for data source

[jira] [Closed] (SPARK-19166) change method name from InsertIntoHadoopFsRelationCommand.deleteMatchingPartitions to InsertIntoHadoopFsRelationCommand.deleteMatchingPrefix

2017-02-14 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun closed SPARK-19166. Resolution: Not A Bug minor issue > change method name from >

[jira] [Comment Edited] (SPARK-19598) Remove the alias parameter in UnresolvedRelation

2017-02-15 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867548#comment-15867548 ] Song Jun edited comment on SPARK-19598 at 2/15/17 9:46 AM: --- [~rxin] When I do

[jira] [Commented] (SPARK-19598) Remove the alias parameter in UnresolvedRelation

2017-02-14 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867398#comment-15867398 ] Song Jun commented on SPARK-19598: -- OK~ I'd like to do this. Thank you very much! > Remove the alias

[jira] [Commented] (SPARK-19598) Remove the alias parameter in UnresolvedRelation

2017-02-15 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867548#comment-15867548 ] Song Jun commented on SPARK-19598: -- [~rxin] When I do this jira, I found it that it is not proper to

[jira] [Created] (SPARK-19577) insert into a partition datasource table with InMemoryCatalog after the partition location alter by alter command failed

2017-02-13 Thread Song Jun (JIRA)
Song Jun created SPARK-19577: Summary: insert into a partition datasource table with InMemoryCatalog after the partition location alter by alter command failed Key: SPARK-19577 URL:

[jira] [Commented] (SPARK-19598) Remove the alias parameter in UnresolvedRelation

2017-02-16 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869623#comment-15869623 ] Song Jun commented on SPARK-19598: -- Thanks~ let me investigate more~ > Remove the alias parameter in

[jira] [Created] (SPARK-19575) Reading from or writing to a hive serde table with a non pre-existing location should succeed

2017-02-13 Thread Song Jun (JIRA)
Song Jun created SPARK-19575: Summary: Reading from or writing to a hive serde table with a non pre-existing location should succeed Key: SPARK-19575 URL: https://issues.apache.org/jira/browse/SPARK-19575

[jira] [Commented] (SPARK-19577) insert into a partition datasource table with InMemoryCatalog after the partition location alter by alter command failed

2017-02-13 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15863511#comment-15863511 ] Song Jun commented on SPARK-19577: -- I am working on this~ > insert into a partition datasource table

[jira] [Created] (SPARK-19284) append to a existed partitioned datasource table should have no CustomPartitionLocations

2017-01-19 Thread Song Jun (JIRA)
Song Jun created SPARK-19284: Summary: append to a existed partitioned datasource table should have no CustomPartitionLocations Key: SPARK-19284 URL: https://issues.apache.org/jira/browse/SPARK-19284

[jira] [Commented] (SPARK-19241) remove hive generated table properties if they are not useful in Spark

2017-01-16 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15823654#comment-15823654 ] Song Jun commented on SPARK-19241: -- I am working on this~ > remove hive generated table properties if

[jira] [Commented] (SPARK-19153) DataFrameWriter.saveAsTable should work with hive format to create partitioned table

2017-01-16 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15823657#comment-15823657 ] Song Jun commented on SPARK-19153: -- I am sorry, It is my fault,I forget to comment >

[jira] [Created] (SPARK-19246) CataLogTable's partitionSchema should check if each column name in partitionColumnNames must match one and only one field in schema

2017-01-16 Thread Song Jun (JIRA)
Song Jun created SPARK-19246: Summary: CataLogTable's partitionSchema should check if each column name in partitionColumnNames must match one and only one field in schema Key: SPARK-19246 URL:

[jira] [Updated] (SPARK-19246) CataLogTable's partitionSchema should check if each column name in partitionColumnNames must match one and only one field in schema, and keep order with partitionColumnN

2017-01-16 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-19246: - Description: get CataLogTable's partitionSchema should check if each column name in

[jira] [Updated] (SPARK-19246) CataLogTable's partitionSchema should check if each column name in partitionColumnNames must match one and only one field in schema, and keep order with partitionColumnN

2017-01-16 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-19246: - Summary: CataLogTable's partitionSchema should check if each column name in partitionColumnNames must

[jira] [Updated] (SPARK-19246) CataLogTable's partitionSchema should check order

2017-01-16 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-19246: - Summary: CataLogTable's partitionSchema should check order (was: CataLogTable's partitionSchema should

[jira] [Commented] (SPARK-19257) The type of CatalogStorageFormat.locationUri should be java.net.URI instead of String

2017-01-17 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15825645#comment-15825645 ] Song Jun commented on SPARK-19257: -- I am working on this~ thanks > The type of

[jira] [Created] (SPARK-19761) create InMemoryFileIndex with empty rootPaths when set PARALLEL_PARTITION_DISCOVERY_THRESHOLD to zero

2017-02-27 Thread Song Jun (JIRA)
Song Jun created SPARK-19761: Summary: create InMemoryFileIndex with empty rootPaths when set PARALLEL_PARTITION_DISCOVERY_THRESHOLD to zero Key: SPARK-19761 URL: https://issues.apache.org/jira/browse/SPARK-19761

[jira] [Created] (SPARK-19763) qualified external datasource table location stored in catalog

2017-02-27 Thread Song Jun (JIRA)
Song Jun created SPARK-19763: Summary: qualified external datasource table location stored in catalog Key: SPARK-19763 URL: https://issues.apache.org/jira/browse/SPARK-19763 Project: Spark

[jira] [Created] (SPARK-19748) refresh for InMemoryFileIndex with FileStatusCache does not work correctly

2017-02-26 Thread Song Jun (JIRA)
Song Jun created SPARK-19748: Summary: refresh for InMemoryFileIndex with FileStatusCache does not work correctly Key: SPARK-19748 URL: https://issues.apache.org/jira/browse/SPARK-19748 Project: Spark

[jira] [Commented] (SPARK-19742) When using SparkSession to write a dataset to Hive the schema is ignored

2017-02-27 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885399#comment-15885399 ] Song Jun commented on SPARK-19742: -- this is expected, see the comment. {code} /** * Inserts the

[jira] [Created] (SPARK-19784) refresh datasource table after alter the location

2017-03-01 Thread Song Jun (JIRA)
Song Jun created SPARK-19784: Summary: refresh datasource table after alter the location Key: SPARK-19784 URL: https://issues.apache.org/jira/browse/SPARK-19784 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-18137) RewriteDistinctAggregates UnresolvedException because of UDAF TypeCheckFailure

2016-10-27 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-18137: - Description: when run a sql with distinct(on spark github master branch), it throw UnresolvedException.

[jira] [Created] (SPARK-18137) RewriteDistinctAggregates UnresolvedException because of UDAF TypeCheckFailure

2016-10-27 Thread Song Jun (JIRA)
Song Jun created SPARK-18137: Summary: RewriteDistinctAggregates UnresolvedException because of UDAF TypeCheckFailure Key: SPARK-18137 URL: https://issues.apache.org/jira/browse/SPARK-18137 Project:

[jira] [Updated] (SPARK-18137) RewriteDistinctAggregates UnresolvedException when a UDAF has a foldable TypeCheck

2016-10-27 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-18137: - Summary: RewriteDistinctAggregates UnresolvedException when a UDAF has a foldable TypeCheck (was:

[jira] [Commented] (SPARK-18298) HistoryServer use GMT time all time

2016-11-07 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646333#comment-15646333 ] Song Jun commented on SPARK-18298: -- [~WangTao] I test it,and the ui should show the user's local time

[jira] [Comment Edited] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

2016-11-07 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646477#comment-15646477 ] Song Jun edited comment on SPARK-18055 at 11/8/16 5:04 AM: --- [~davies] I can't

[jira] [Comment Edited] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

2016-11-07 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646477#comment-15646477 ] Song Jun edited comment on SPARK-18055 at 11/8/16 4:51 AM: --- [~davies] I can't

[jira] [Commented] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

2016-11-07 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646477#comment-15646477 ] Song Jun commented on SPARK-18055: -- [~davies] I can't reproduce it on the master branch or on

[jira] [Commented] (SPARK-18490) duplicate nodename extrainfo of ShuffleExchange

2016-11-17 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15673918#comment-15673918 ] Song Jun commented on SPARK-18490: -- it is not a bug, just simplify this code > duplicate nodename

[jira] [Created] (SPARK-18490) duplicate nodename extrainfo of ShuffleExchange

2016-11-17 Thread Song Jun (JIRA)
Song Jun created SPARK-18490: Summary: duplicate nodename extrainfo of ShuffleExchange Key: SPARK-18490 URL: https://issues.apache.org/jira/browse/SPARK-18490 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-18490) duplicate nodename extrainfo of ShuffleExchange

2016-11-17 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-18490: - Component/s: (was: Spark Core) SQL > duplicate nodename extrainfo of

[jira] [Commented] (SPARK-14171) UDAF aggregates argument object inspector not parsed correctly

2016-10-27 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614192#comment-15614192 ] Song Jun commented on SPARK-14171: -- This issue does not reproduce on the recently spark branch master

[jira] [Created] (SPARK-18271) hash udf in HiveSessionCatalog.hiveFunctions seq is redundant

2016-11-04 Thread Song Jun (JIRA)
Song Jun created SPARK-18271: Summary: hash udf in HiveSessionCatalog.hiveFunctions seq is redundant Key: SPARK-18271 URL: https://issues.apache.org/jira/browse/SPARK-18271 Project: Spark Issue

[jira] [Commented] (SPARK-18609) [SQL] column mixup with CROSS JOIN

2016-12-13 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745309#comment-15745309 ] Song Jun commented on SPARK-18609: -- open another jira SPARK-18841 > [SQL] column mixup with CROSS JOIN

[jira] [Created] (SPARK-18841) PushProjectionThroughUnion exception when there are same column

2016-12-13 Thread Song Jun (JIRA)
Song Jun created SPARK-18841: Summary: PushProjectionThroughUnion exception when there are same column Key: SPARK-18841 URL: https://issues.apache.org/jira/browse/SPARK-18841 Project: Spark

[jira] [Commented] (SPARK-18609) [SQL] column mixup with CROSS JOIN

2016-12-06 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15728042#comment-15728042 ] Song Jun commented on SPARK-18609: -- I'm working on this~ > [SQL] column mixup with CROSS JOIN >

[jira] [Created] (SPARK-18742) readd spark.broadcast.factory for user-defined BroadcastFactory

2016-12-06 Thread Song Jun (JIRA)
Song Jun created SPARK-18742: Summary: readd spark.broadcast.factory for user-defined BroadcastFactory Key: SPARK-18742 URL: https://issues.apache.org/jira/browse/SPARK-18742 Project: Spark

[jira] [Commented] (SPARK-19154) support read and overwrite a same table

2017-01-10 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816942#comment-15816942 ] Song Jun commented on SPARK-19154: -- I am working on this~ > support read and overwrite a same table >

[jira] [Created] (SPARK-19166) change method name from InsertIntoHadoopFsRelationCommand.deleteMatchingPartitions to InsertIntoHadoopFsRelationCommand.deleteMatchingPrefix

2017-01-10 Thread Song Jun (JIRA)
Song Jun created SPARK-19166: Summary: change method name from InsertIntoHadoopFsRelationCommand.deleteMatchingPartitions to InsertIntoHadoopFsRelationCommand.deleteMatchingPrefix Key: SPARK-19166 URL:

[jira] [Comment Edited] (SPARK-18930) Inserting in partitioned table - partitioned field should be last in select statement.

2016-12-28 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784678#comment-15784678 ] Song Jun edited comment on SPARK-18930 at 12/29/16 6:44 AM: from hive

[jira] [Commented] (SPARK-18930) Inserting in partitioned table - partitioned field should be last in select statement.

2016-12-28 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784678#comment-15784678 ] Song Jun commented on SPARK-18930: -- from hive document,

[jira] [Updated] (SPARK-18742) Clarify that user-defined BroadcastFactory is not supported

2016-12-16 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-18742: - Description: After SPARK-12588 Remove HTTPBroadcast [1], the one and only implementation of

[jira] [Updated] (SPARK-18742) Clarify that user-defined BroadcastFactory is not supported

2016-12-16 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-18742: - Description: After SPARK-12588 Remove HTTPBroadcast [1], the one and only implementation of

[jira] [Updated] (SPARK-20013) merge renameTable to alterTable in ExternalCatalog

2017-03-18 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-20013: - Description: Currently when we create / rename a managed table, we should get the defaultTablePath for

[jira] [Created] (SPARK-19961) unify a exception erro msg for dropdatabase

2017-03-15 Thread Song Jun (JIRA)
Song Jun created SPARK-19961: Summary: unify a exception erro msg for dropdatabase Key: SPARK-19961 URL: https://issues.apache.org/jira/browse/SPARK-19961 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-20013) merge renameTable to alterTable in ExternalCatalog

2017-03-18 Thread Song Jun (JIRA)
Song Jun created SPARK-20013: Summary: merge renameTable to alterTable in ExternalCatalog Key: SPARK-20013 URL: https://issues.apache.org/jira/browse/SPARK-20013 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-19990) Flaky test: org.apache.spark.sql.hive.execution.HiveCatalogedDDLSuite: create temporary view using

2017-03-16 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929409#comment-15929409 ] Song Jun edited comment on SPARK-19990 at 3/17/17 4:36 AM: --- the root cause is

[jira] [Comment Edited] (SPARK-19990) Flaky test: org.apache.spark.sql.hive.execution.HiveCatalogedDDLSuite: create temporary view using

2017-03-16 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929409#comment-15929409 ] Song Jun edited comment on SPARK-19990 at 3/17/17 4:35 AM: --- the root cause is

[jira] [Commented] (SPARK-19990) Flaky test: org.apache.spark.sql.hive.execution.HiveCatalogedDDLSuite: create temporary view using

2017-03-16 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929409#comment-15929409 ] Song Jun commented on SPARK-19990: -- the root cause is [the csvfile path in this test

[jira] [Updated] (SPARK-19945) Add test suite for SessionCatalog with HiveExternalCatalog

2017-03-14 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-19945: - Summary: Add test suite for SessionCatalog with HiveExternalCatalog (was: Add test case for

[jira] [Created] (SPARK-19945) Add test case for SessionCatalog with HiveExternalCatalog

2017-03-14 Thread Song Jun (JIRA)
Song Jun created SPARK-19945: Summary: Add test case for SessionCatalog with HiveExternalCatalog Key: SPARK-19945 URL: https://issues.apache.org/jira/browse/SPARK-19945 Project: Spark Issue

[jira] [Created] (SPARK-19917) qualified partition location stored in catalog

2017-03-10 Thread Song Jun (JIRA)
Song Jun created SPARK-19917: Summary: qualified partition location stored in catalog Key: SPARK-19917 URL: https://issues.apache.org/jira/browse/SPARK-19917 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-19864) add makeQualifiedPath in SQLTestUtils to optimize some code

2017-03-08 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-19864: - Summary: add makeQualifiedPath in SQLTestUtils to optimize some code (was: add makeQualifiedPath in

[jira] [Created] (SPARK-19869) move table related ddl from ddl.scala to tables.scala

2017-03-08 Thread Song Jun (JIRA)
Song Jun created SPARK-19869: Summary: move table related ddl from ddl.scala to tables.scala Key: SPARK-19869 URL: https://issues.apache.org/jira/browse/SPARK-19869 Project: Spark Issue Type:

[jira] [Updated] (SPARK-19869) move table related ddl from ddl.scala to tables.scala

2017-03-08 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-19869: - Issue Type: Improvement (was: Bug) > move table related ddl from ddl.scala to tables.scala >

[jira] [Created] (SPARK-19845) failed to uncache datasource table after the table location altered

2017-03-06 Thread Song Jun (JIRA)
Song Jun created SPARK-19845: Summary: failed to uncache datasource table after the table location altered Key: SPARK-19845 URL: https://issues.apache.org/jira/browse/SPARK-19845 Project: Spark

[jira] [Commented] (SPARK-19845) failed to uncache datasource table after the table location altered

2017-03-06 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15898626#comment-15898626 ] Song Jun commented on SPARK-19845: -- yes, this jira is doing this

[jira] [Comment Edited] (SPARK-19845) failed to uncache datasource table after the table location altered

2017-03-06 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15898626#comment-15898626 ] Song Jun edited comment on SPARK-19845 at 3/7/17 2:33 AM: -- yes, this jira is

[jira] [Commented] (SPARK-19836) Customizable remote repository url for hive versions unit test

2017-03-07 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15899292#comment-15899292 ] Song Jun commented on SPARK-19836: -- I have do this similar https://github.com/apache/spark/pull/16803 >

[jira] [Created] (SPARK-19832) DynamicPartitionWriteTask should escape the partition name

2017-03-05 Thread Song Jun (JIRA)
Song Jun created SPARK-19832: Summary: DynamicPartitionWriteTask should escape the partition name Key: SPARK-19832 URL: https://issues.apache.org/jira/browse/SPARK-19832 Project: Spark Issue

[jira] [Created] (SPARK-19833) remove SQLConf.HIVE_VERIFY_PARTITION_PATH, we always return empty when the path does not exists

2017-03-06 Thread Song Jun (JIRA)
Song Jun created SPARK-19833: Summary: remove SQLConf.HIVE_VERIFY_PARTITION_PATH, we always return empty when the path does not exists Key: SPARK-19833 URL: https://issues.apache.org/jira/browse/SPARK-19833

[jira] [Created] (SPARK-19864) add makeQualifiedPath in CatalogUtils to optimize some code

2017-03-07 Thread Song Jun (JIRA)
Song Jun created SPARK-19864: Summary: add makeQualifiedPath in CatalogUtils to optimize some code Key: SPARK-19864 URL: https://issues.apache.org/jira/browse/SPARK-19864 Project: Spark Issue

  1   2   >