GitHub user OopsOutOfMemory opened a pull request:
https://github.com/apache/spark/pull/2998
[SQL] Add string operation function trim, ltrim, rtrim, length to support
SparkSql (HiveQL)
@marmbrus @chenghao-intel
Add three string operation functions to support spark sql and hiveql.
eg:
sql("select trim(' a b ') from src ").collect() --> 'a b'
sql("select ltrim(' a b ') from src ").collect() --> 'a b '
sql("select rtrim(' a b ') from src ").collect() --> ' a b'
sql("select length('ab') from src ").collect() --> 2
And Rename the trait of stringOperations.scala.
I prefer to rename trait CaseConversionExpression to
StringTransformationExpression, it is more make sence than before so that this
trait can support more string transformation but not only caseconversion.
And also add a trait StringCalculationExpression that do string computation
like length, indexof etc....
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/OopsOutOfMemory/spark sparksql
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/2998.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2998
----
commit 58094965a090f9dfd9271b2183f57c1d03e31d62
Author: OopsOutOfMemory <[email protected]>
Date: 2014-10-28T08:43:50Z
Add description of SQL CLI and HiveServer.
commit c577589e93441e2fa3577e92007b7def546c9d9e
Author: OopsOutOfMemory <[email protected]>
Date: 2014-10-28T08:48:12Z
Merge branch 'master' of https://github.com/apache/spark
commit e654704cf13c3081bc01d2f1a00599c07c5a9a2e
Author: OopsOutOfMemory <[email protected]>
Date: 2014-10-29T02:06:37Z
Merge branch 'master' of https://github.com/apache/spark
commit 3269eab23ca8adfc746ced8ddef391f51af43677
Author: OopsOutOfMemory <[email protected]>
Date: 2014-10-29T06:55:15Z
Add system function trim
commit ce6899abe365ade3b776bd65e110f0a7e679df3e
Author: OopsOutOfMemory <[email protected]>
Date: 2014-10-29T07:31:51Z
HiveQL support trim
commit e2781ee7ff0a32fba7854cdd618a37e20c829e7d
Author: OopsOutOfMemory <[email protected]>
Date: 2014-10-29T07:49:41Z
modify keyword of Trim
commit 2166c7765abea43d86ccf3df4b891f5949998241
Author: OopsOutOfMemory <[email protected]>
Date: 2014-10-29T08:23:03Z
correct spelling mistake
commit a4f4e4b5d119a4479c7594987e364de23e89ee16
Author: OopsOutOfMemory <[email protected]>
Date: 2014-10-29T08:27:31Z
correct spelling mistake
commit 2137e28f5e56c7f2f203dacfd1b9eca2b8083c17
Author: OopsOutOfMemory <[email protected]>
Date: 2014-10-29T08:59:25Z
add test suit for trim
commit 10d8ace1633dd52305fe0fcab9fe6fcb1ddd55d8
Author: OopsOutOfMemory <[email protected]>
Date: 2014-10-29T09:36:01Z
support 3 functions : ltrim rtrim length in SparkQL and HiveQl
commit 0a0f4e0aa922453f1a8f7a0cbc5b3349de798075
Author: OopsOutOfMemory <[email protected]>
Date: 2014-10-29T09:58:39Z
change the name of the trait CaseConversionExpression to
StringTransformationExpression and add a StringCalculationExpression for string
calculation in stringOperation.scala , eg: trim is for transformation and
length is for calculation
commit 0fa2cd6e986bc0c56abbf6f0e1e86491db6aecf0
Author: OopsOutOfMemory <[email protected]>
Date: 2014-10-29T10:05:54Z
change return type
commit b358048d6e0598238c866cd6849145da4610e2b7
Author: OopsOutOfMemory <[email protected]>
Date: 2014-10-29T11:47:27Z
deleted: sql/README.md
commit 558d7bf37a154c17ea7d5af1daddd11255938afd
Author: OopsOutOfMemory <[email protected]>
Date: 2014-10-29T11:48:48Z
new file: sql/README.md
commit ab29a7e892b09e01171fcfd217f2cfb80d66d651
Author: OopsOutOfMemory <[email protected]>
Date: 2014-10-29T11:50:39Z
Merge branch 'sparksql' of https://github.com/OopsOutOfMemory/spark into
sparksql
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
