[GitHub] spark pull request: [SPARK-5966]

2015-10-22 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/9220 [SPARK-5966] You can merge this pull request into a Git repository by running: $ git pull https://github.com/kevinyu98/spark working_on_spark-5966 Alternatively you can review and apply

[GitHub] spark pull request: [SPARK-5966][WIP]

2015-10-22 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/9220#discussion_r42811834 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/impatient.sc --- @@ -0,0 +1 @@ +1+1; --- End diff -- Hello

[GitHub] spark pull request: [SPARK-5966][WIP]

2015-10-22 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/9220#discussion_r42813223 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -266,6 +266,11 @@ object SparkSubmit

[GitHub] spark pull request: [SPARK-11447][SQL] change NullType to StringTy...

2015-11-15 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/9720 [SPARK-11447][SQL] change NullType to StringType during binaryComparison between NullType and StringType During executing PromoteStrings rule, if one side of binaryComparison is StringType

[GitHub] spark pull request: [SPARK-11447][SQL] change NullType to StringTy...

2015-11-17 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/9720#discussion_r45069001 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala --- @@ -280,6 +280,12 @@ object HiveTypeCoercion

[GitHub] spark pull request: [SPARK-11447][SQL] change NullType to StringTy...

2015-11-16 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/9720#issuecomment-157287070 @cloud-fan and @marmbrus @yhuai @nongli @liancheng : thanks for reviewing the fix. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-11447][SQL] change NullType to StringTy...

2015-11-16 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/9720#discussion_r45028090 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala --- @@ -280,6 +280,12 @@ object HiveTypeCoercion

[GitHub] spark pull request: Working on spark 11827

2015-12-03 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/10125 Working on spark 11827 Hello : Can you help check this PR? I am adding support for the java.math.BigInteger for java bean code path. I saw internally spark is converting the BigInteger

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2015-12-03 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10125#issuecomment-161707755 Hello Sean: I am sorry, I forgot to update the title and description. I have made the changes, please let me know if anything needs to be changed. Thanks. Kevin

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurable value i...

2015-12-15 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/10314 [SPARK-12317][SQL]Support configurable value in SQLConf file Hello: adding the configure value for AUTO_BROADCASTJOIN_THRESHOLD and DEFAULT_SIZE_IN_BYTES. You can merge this pull request

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2015-12-16 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10125#discussion_r47871490 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/Decimal.scala --- @@ -69,6 +68,16 @@ final class Decimal extends Ordered[Decimal

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2015-12-16 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10125#discussion_r47871566 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/Decimal.scala --- @@ -362,6 +371,8 @@ object Decimal { def apply(value

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2015-12-16 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10125#discussion_r47866879 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystTypeConverters.scala --- @@ -326,6 +326,7 @@ object CatalystTypeConverters

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2015-12-16 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10125#discussion_r47866947 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala --- @@ -75,6 +75,7 @@ object JavaTypeInference

[GitHub] spark pull request: [SPARK-12231][SQL]create a combineFilters' pro...

2015-12-14 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10299#issuecomment-164603371 Hello Michael: I fixed the scala style issue, can you help re-run the test? Thanks. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-12231][SQL]create a combineFilters' pro...

2015-12-18 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10299#issuecomment-165710284 The failure is because of the changed project, will submit an updated patch tomorrow. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurable value i...

2015-12-15 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10314#discussion_r47713626 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala --- @@ -93,7 +94,7 @@ private[spark] object SQLConf { isPublic: Boolean

[GitHub] spark pull request: [SPARK-12231][SQL]create a combineFilters' pro...

2015-12-14 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/10299 [SPARK-12231][SQL]create a combineFilters' projection when we call buildPartitionedTableScan Hello Michael & All: Here I am submitting another approach to solve this problem. Can you ve

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurable value i...

2015-12-15 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10314#issuecomment-165010421 @watermen thanks for your input. It is good idea if we decide to go with approach 2. create a new method of intConf for AUTO_BROADCASTJOIN_THRESHOLD. If we

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurable value i...

2015-12-15 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10314#issuecomment-165023639 @srowen Thanks Sean, I will create a new method of intConf for AUTO_BROADCASTJOIN_THRESHOLD. Will update the PR soon. --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurable value i...

2016-01-04 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10314#issuecomment-168854235 @srowen Hello Sean: Sorry for taking so long. Can you review the code? Thanks. Kevin --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurate value fo...

2016-01-05 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10314#issuecomment-169238048 @viirya @yhuai @srowen @marmbrus @concretevitamin: SHUFFLE_TARGET_POSTSHUFFLE_INPUT_SIZE was added by spark-9850, PR#9276, I am including Yin , Michael set

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurate value fo...

2016-01-05 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10314#issuecomment-169220626 Hello @viirya : Good Point. I just test it, -1 and -1g will have different behavior. It will take -1, and throw IllegalArgumentException for -1g. Thanks

[GitHub] spark pull request: [SPARK-12317][SQL] Support units (m,k,g) in SQ...

2016-01-06 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/10629 [SPARK-12317][SQL] Support units (m,k,g) in SQLConf This PR is continue from previous closed PR 10314. In this PR, SHUFFLE_TARGET_POSTSHUFFLE_INPUT_SIZE will be taken memory string

[GitHub] spark pull request: [SPARK-12317][SQL]Support units (m,k,g) in SQL...

2016-01-06 Thread kevinyu98
Github user kevinyu98 closed the pull request at: https://github.com/apache/spark/pull/10314 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-12317][SQL] Support units (m,k,g) in SQ...

2016-01-07 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10629#issuecomment-169902816 @rxin I am so sorry that I didn't reply earlier. The code passed the style check, I copied the code from the existing codes, and I thought the indentation 2 is fine

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurate value fo...

2016-01-05 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10314#issuecomment-169173265 Hello @srowen @marmbrus @viirya : I have made the code changes, and change the title based on the comments. Can you help review the codes? Thanks. --- If your

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurable value i...

2016-01-01 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10314#issuecomment-168317362 Hello Sean: Sorry for the delay. Yes, I have made most code changes, and I will try to finish it up soon and do more testing. Will keep you updated. Thanks, Happy

[GitHub] spark pull request: [SPARK-12317][SQL]Support units (m,k,g) in SQL...

2016-01-06 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10314#issuecomment-169438242 Hello @marmbrus, thanks. So your mean I can remove the code change for intMemConf, and keep the code for longMemConf for this jira? I will make the PR title

[GitHub] spark pull request: [SPARK-12231][SQL]create a combineFilters' pro...

2015-12-21 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10388#issuecomment-166407905 @marmbrus : Can you help take a look at this PR? Thanks for your review. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurable value i...

2015-12-22 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10314#issuecomment-166747545 @srowen Hello Sean: I have submit the new code, can you help review? Thanks a lot. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-12231][SQL]create a combineFilters' pro...

2015-12-24 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10388#issuecomment-167179672 I delete the test cases from DataFrameNaFunctionsSuite.scala. I checked the previous failure, not sure why it is failed. I worked when I run the local test on my

[GitHub] spark pull request: [SPARK-12231][SQL]create a combineFilters' pro...

2015-12-18 Thread kevinyu98
Github user kevinyu98 closed the pull request at: https://github.com/apache/spark/pull/10299 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-12231][SQL]create a combineFilters' pro...

2015-12-18 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10299#issuecomment-165919581 I will create a new PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-12231][SQL]create a combineFilters' pro...

2015-12-18 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/10388 [SPARK-12231][SQL]create a combineFilters' projection when we call buildPartitionedTableScan Hello Michael & All: We have some issues to submit the new codes in the other PR(#1

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurable value i...

2015-12-26 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10314#discussion_r48451909 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala --- @@ -100,6 +101,33 @@ private[spark] object SQLConf

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurable value i...

2015-12-26 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10314#discussion_r48451916 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala --- @@ -100,6 +101,33 @@ private[spark] object SQLConf

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurable value i...

2015-12-26 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10314#discussion_r48451915 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala --- @@ -100,6 +101,33 @@ private[spark] object SQLConf

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurable value i...

2015-12-26 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10314#discussion_r48451911 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala --- @@ -100,6 +101,33 @@ private[spark] object SQLConf

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurable value i...

2015-12-26 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10314#discussion_r48451918 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala --- @@ -100,6 +101,33 @@ private[spark] object SQLConf

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurable value i...

2015-12-26 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10314#discussion_r48451947 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala --- @@ -107,7 +135,7 @@ private[spark] object SQLConf { isPublic: Boolean

[GitHub] spark pull request: [SPARK-12231][SQL]create a combineFilters' pro...

2015-12-22 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10388#discussion_r48231314 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameNaFunctionsSuite.scala --- @@ -194,4 +194,45 @@ class DataFrameNaFunctionsSuite extends

[GitHub] spark pull request: [SPARK-12231][SQL]create a combineFilters' pro...

2015-12-22 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10388#discussion_r48230827 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -88,16 +88,27 @@ private[sql] object

[GitHub] spark pull request: [SPARK-12231][SQL]create a combineFilters' pro...

2015-12-22 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10388#discussion_r48230905 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -88,16 +88,27 @@ private[sql] object

[GitHub] spark pull request: [SPARK-12231][SQL]create a combineFilters' pro...

2015-12-22 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10388#discussion_r48230807 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameNaFunctionsSuite.scala --- @@ -194,4 +194,45 @@ class DataFrameNaFunctionsSuite extends

[GitHub] spark pull request: [SPARK-12231][SQL]create a combineFilters' pro...

2015-12-22 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10388#discussion_r48230771 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -88,16 +88,27 @@ private[sql] object

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurable value i...

2015-12-22 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10314#issuecomment-166560467 @srowen Hello Sean: I am sorry that I have't been able to submit the PR yet. I will conti. work on it tomorrow. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurable value i...

2015-12-21 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10314#issuecomment-166326525 @srowen Hello Sean, yes, sorry for the delay. I will submit the updated PR today. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #13506: [SPARK-15763][SQL] Support DELETE FILE command na...

2016-06-04 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/13506#discussion_r65805306 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1441,6 +1441,32 @@ class SparkContext(config: SparkConf) extends Logging

[GitHub] spark pull request #13555: [SPARK-15804][SQL]Include metadata in the toStruc...

2016-06-08 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/13555#discussion_r66366855 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala --- @@ -625,6 +625,21 @@ class

[GitHub] spark pull request #13555: [SPARK-15804][SQL]Include metadata in the toStruc...

2016-06-08 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/13555#discussion_r66366487 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala --- @@ -625,6 +625,21 @@ class

[GitHub] spark pull request #13555: [SPARK-15804][SQL]Include metadata in the toStruc...

2016-06-08 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/13555#discussion_r66366470 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala --- @@ -625,6 +625,21 @@ class

[GitHub] spark pull request #13555: [SPARK-15804][SQL]Include metadata in the toStruc...

2016-06-08 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/13555 [SPARK-15804][SQL]Include metadata in the toStructType ## What changes were proposed in this pull request? The help function 'toStructType' in the AttributeSeq class doesn't include

[GitHub] spark pull request #13506: [SPARK-15763][SQL] Support DELETE FILE command na...

2016-06-03 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/13506 [SPARK-15763][SQL] Support DELETE FILE command natively ## What changes were proposed in this pull request? Hive supports these cli commands to manage the resource [Hive Doc](https

[GitHub] spark pull request #13506: [SPARK-15763][SQL] Support DELETE FILE command na...

2016-06-04 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/13506#discussion_r65799162 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1441,6 +1441,32 @@ class SparkContext(config: SparkConf) extends Logging

[GitHub] spark pull request #13506: [SPARK-15763][SQL] Support DELETE FILE command na...

2016-06-06 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/13506#discussion_r65981961 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1441,6 +1441,32 @@ class SparkContext(config: SparkConf) extends Logging

[GitHub] spark pull request #13555: [SPARK-15804][SQL]Include metadata in the toStruc...

2016-06-08 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/13555#discussion_r66264754 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala --- @@ -625,6 +625,22 @@ class

[GitHub] spark pull request #13555: [SPARK-15804][SQL]Include metadata in the toStruc...

2016-06-08 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/13555#discussion_r66264631 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala --- @@ -625,6 +625,22 @@ class

[GitHub] spark pull request #13555: [SPARK-15804][SQL]Include metadata in the toStruc...

2016-06-08 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/13555#discussion_r66265069 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala --- @@ -625,6 +625,22 @@ class

[GitHub] spark pull request #13555: [SPARK-15804][SQL]Include metadata in the toStruc...

2016-06-08 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/13555#discussion_r66383008 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala --- @@ -625,6 +625,21 @@ class

[GitHub] spark pull request: [SPARK-10777] [SQL]avoid checking nullability ...

2016-02-12 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/11184 [SPARK-10777] [SQL]avoid checking nullability for complex data type for typeSuffix code path Hello: When we call the typeSuffix method, it will call the dataType and get the LongType

[GitHub] spark pull request: [SPARK-10777] [SQL] Resolve Aliases in the Gro...

2016-01-27 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/10967 [SPARK-10777] [SQL] Resolve Aliases in the Group By clause @gatorsmile @yhuai @marmbrus @cloud-fan : Hello All, I tried to run the failing query with PR 10678 from Spark-12705, still got

[GitHub] spark pull request: [SPARK-12987][SQL]Fixing the name resolution i...

2016-02-02 Thread kevinyu98
GitHub user kevinyu98 reopened a pull request: https://github.com/apache/spark/pull/11009 [SPARK-12987][SQL]Fixing the name resolution in drop column @marmbrus @cloud-fan @thomas @jayadevan : Hello All: Can you help review this code fix? This problem is coming from drop

[GitHub] spark pull request: [SPARK-12987][SQL]Fixing the name resolution i...

2016-02-01 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/11009 [SPARK-12987][SQL]Fixing the name resolution in drop column @marmbrus @cloud-fan @thomas @jayadevan : Hello All: Can you help review this code fix? This problem is coming from drop

[GitHub] spark pull request: [SPARK-12987][SQL]Fixing the name resolution i...

2016-02-03 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/11009#issuecomment-179478292 @marmbrus @cloud-fan @dilipbiswal @yzhou2001 : I have change the code based on Michael's comments, can you help review it again? Not sure why the first test

[GitHub] spark pull request: [SPARK-12987][SQL]Fixing the name resolution i...

2016-02-02 Thread kevinyu98
Github user kevinyu98 closed the pull request at: https://github.com/apache/spark/pull/11009 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-12987][SQL]Fixing the name resolution i...

2016-02-02 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/11009#issuecomment-178712237 @marmbrus @cloud-fan @dilipbiswal @yzhou2001 : it seems this is the duplicate of [SPARK-12988][SQL] Can't drop columns that contain dots #10943, I will close this PR

[GitHub] spark pull request: [SPARK-14878][SQL] Trim characters string func...

2016-04-24 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/12646#issuecomment-214038288 Hello : I removed some invalid unit test cases and correct the error messages in the unit test cases. It passed the local tests. Can you retest it ? Thanks

[GitHub] spark pull request: [SPARK-14878][SQL] Trim characters string func...

2016-04-25 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/12646#issuecomment-214465479 Hello Dongjoon: Thanks for your comments, I will make changes. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-14878][SQL] Trim characters string func...

2016-04-25 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/12646#issuecomment-214521095 @dongjoon-hyun Hello Dongjoon: I have fix the comments, let me know if you see anything else I need to change. Also I did git fetch upstream, git merge upstream

[GitHub] spark pull request: [SPARK-14878][SQL] Trim characters string func...

2016-04-26 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/12646#issuecomment-214921975 retest please, I just did rebase to resolve the conflicts. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-14878][SQL] Trim characters string func...

2016-04-28 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/12646#issuecomment-215444282 retest it please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-14878][SQL] Trim characters string func...

2016-04-23 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/12646#issuecomment-213853674 @hvanhovell @yhuai @chenghao-intel @gatorsmile @dilipbiswal @viirya @xwu0226 can you help take a look at this PR? Thanks. --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-14878][SQL] Trim characters string func...

2016-04-23 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/12646 [SPARK-14878][SQL] Trim characters string function support What changes were proposed in this pull request? This PR enhances the TRIM function support in Spark SQL

[GitHub] spark pull request: [SPARK-14878][SQL] Trim characters string func...

2016-04-30 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/12646#issuecomment-216018276 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-18 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10125#issuecomment-219945586 I just did rebase to solve the conflict. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-18 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10125#issuecomment-219945597 retest it please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-19 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10125#issuecomment-220346292 retest it please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-19 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10125#discussion_r63880607 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/DecimalType.scala --- @@ -109,6 +109,7 @@ object DecimalType extends AbstractDataType

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-19 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10125#discussion_r63900108 --- Diff: sql/core/src/test/java/test/org/apache/spark/sql/JavaDataFrameSuite.java --- @@ -163,7 +168,9 @@ void validateDataFrameWithBeans(Bean bean

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-19 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10125#discussion_r63900146 --- Diff: sql/core/src/test/java/test/org/apache/spark/sql/JavaDataFrameSuite.java --- @@ -182,6 +189,8 @@ void validateDataFrameWithBeans(Bean bean

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-19 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10125#issuecomment-220361041 I will push the latest one after jenkins finish. Thanks very much ! --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-19 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10125#issuecomment-220237290 @cloud-fan I tried, and it still fail. It didn't go through the createDataFrame you added in SparkSession. It went with this createDataFrame(data

[GitHub] spark pull request: [HOTFIX][SPARK-15445] Build fails for java 1.7...

2016-05-20 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/13223#issuecomment-220639711 @techaddict @srowen @cloud-fan @gatorsmile : Hi Sandeep , thanks for fixing this. I didn't realize the method is java 1.8 only. The code looks good to me

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-17 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10125#discussion_r63608338 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystTypeConverters.scala --- @@ -321,11 +323,13 @@ object CatalystTypeConverters

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-14 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10125#discussion_r63284859 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/Decimal.scala --- @@ -129,6 +129,23 @@ final class Decimal extends Ordered[Decimal

[GitHub] spark pull request: [SPARK-14878][SQL] Trim characters string func...

2016-05-13 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/12646#issuecomment-219164845 retest it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-18 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10125#discussion_r63655581 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystTypeConverters.scala --- @@ -321,11 +323,13 @@ object CatalystTypeConverters

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-18 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10125#issuecomment-220037011 @cloud-fan can you help take a look? I have made changes based on your comments. Thanks. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-18 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10125#issuecomment-220078687 sure, I will do that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-13 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/10125#discussion_r63228007 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/ScalaReflectionRelationSuite.scala --- @@ -34,7 +34,13 @@ case class ReflectData

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-13 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10125#issuecomment-219120180 @srowen @davies @cloud-fan I updated the code, can you help review? Sorry for the delay. Thanks. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-13 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10125#issuecomment-219136812 retest it please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-13 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10125#issuecomment-219136788 I just run the ./dev/mima locally, it works, [info] Done packaging. [info] spark-examples: previous-artifact not set, not analyzing binary compatibility

[GitHub] spark pull request: [Spark-15051] [SQL] Create a TypedColumn alias...

2016-05-04 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/12893 [Spark-15051] [SQL] Create a TypedColumn alias ## What changes were proposed in this pull request? Currently when we try to create an alias against an aggregator TypedColumn

[GitHub] spark pull request: [SPARK-11827] [SQL] Adding java.math.BigIntege...

2016-05-06 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10125#issuecomment-217526107 @srowen: sorry for the long delay. I will work on it now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [Spark-15051] [SQL] Create a TypedColumn alias...

2016-05-06 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/12893#issuecomment-217443570 test please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark-15051] [SQL] Create a TypedColumn alias...

2016-05-04 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/12893#issuecomment-216911919 @cloud-fan can you help take a look at this pr? Thanks very much ! --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [Spark-15051] [SQL] Create a TypedColumn alias...

2016-05-04 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12893#discussion_r62150990 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Column.scala --- @@ -68,6 +68,25 @@ class TypedColumn[-T, U]( } new TypedColumn[T

[GitHub] spark pull request: [Spark-15051] [SQL] Create a TypedColumn alias...

2016-05-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12893#discussion_r62151072 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Column.scala --- @@ -68,6 +68,25 @@ class TypedColumn[-T, U]( } new TypedColumn[T

  1   2   3   4   >