GitHub user culler opened a pull request:
https://github.com/apache/spark/pull/3649
[SPARK-4804] StringContext method to allow String references to column
names in Catalyst DSL
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/culler/spark dslstringcontext
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3649.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3649
----
commit d6460f6396e1a5cb3f356236cf22a1ef1aad3147
Author: Marc Culler <[email protected]>
Date: 2014-11-07T16:11:09Z
Adds RichDate and RichTimestamp classes with comparison operators,
allowing them to be used in DSL expressions. These classes provide
initializers which accept string representations of dates or times.
They are renamed as Date and Timestamp when the members of an
SQLContext are in scope.
commit fa073246a392a1234141bceb61a6a71780fd3b23
Author: Marc Culler <[email protected]>
Date: 2014-11-07T16:17:33Z
Adds new implicit conversions which allow DSL expressions to start
with a literal, e.g. 0 < 'x .
These conversions expose a conflict with the scalatest === operator
if assert(X === Y) is used when the conversions are in scope. To
fix this, several tests are modified, as recommended in the scalatest
documentation, by making the change:
assert(X === Y) --> assert(convertToEqualizer(X).===(Y))
commit 492feefaea6531b3374bf3e753e61234b79e1184
Author: Marc Culler <[email protected]>
Date: 2014-11-07T16:38:18Z
Clarification of one comment.
commit 53aced3bd6f7dba5d124a7070283e95e05b61914
Author: Marc Culler <[email protected]>
Date: 2014-11-08T19:46:10Z
Fixed implicit conversions for RichDate and RichTimestamp and added
a test which would have detected the problem.
commit bd539b6ce8a45e9a6d32b77fe55296a7a960d4bf
Author: Marc Culler <[email protected]>
Date: 2014-11-08T21:00:14Z
Rebased and fixed the assert commands in ParquetQuerySuite.scala.
commit 353d12ca82eea8b097076adcf5b315641bbb81b9
Author: Marc Culler <[email protected]>
Date: 2014-11-09T16:54:10Z
Add unapply methods to RichDate and RichTimestamp and make their types
available after importing the members of an SQLContext.
commit eff68fe04ac699ad180fd7a96b71d6b14159aba0
Author: Marc Culler <[email protected]>
Date: 2014-11-10T03:28:25Z
Modified RichTimestamp.toString to use the Hive format.
commit 759e883ed1eab286ef6b437ec8c194bc8cced50b
Author: Marc Culler <[email protected]>
Date: 2014-11-10T13:56:16Z
Rebasing and making comments consistent with ScalaDocs.
commit 0c86b93e350bb3927f07907b256418036789c038
Author: Marc Culler <[email protected]>
Date: 2014-11-10T15:05:41Z
Removing all changes to SpecificMutableRow.scala. Mutable Dates and
Timestamps
are not useful.
commit 1414f302fe5587dba5c2dec4f33f7001223584ce
Author: Marc Culler <[email protected]>
Date: 2014-11-10T23:31:19Z
New DSL equals operator -=-. Restored all test files as this does not
collide
with scalatests === operator.
commit dabcaf262d68565ecae0092e70fe81f0be6307bd
Author: Marc Culler <[email protected]>
Date: 2014-11-10T23:44:42Z
Cleaning up more comments and one more test.
commit ad2940dad053089182dd65c2d9d0db6c4fedc0cf
Author: Marc Culler <[email protected]>
Date: 2014-11-11T04:57:08Z
Removed LhsLiterals for another PR. Cleaned up.
commit f917e06db9925dc02bc98df3e25ba60642d007a3
Author: Marc Culler <[email protected]>
Date: 2014-11-11T05:02:22Z
Removed test which is no longer part of this PR.
commit 4c32c0416093fedcb70acdcbfbeb112ded4406eb
Author: Marc Culler <[email protected]>
Date: 2014-11-11T05:25:55Z
Restored lost import of scalatest.Assertions.convertToEqualizer.
commit 4cfb8640f3b185de1e2ffacaf30f6d72a8c1d9dc
Author: Marc Culler <[email protected]>
Date: 2014-12-09T20:51:32Z
Merge branch 'master' of https://github.com/apache/spark
commit b3a432d94bfd6856653c4019e0da4f9949582619
Author: Marc Culler <[email protected]>
Date: 2014-12-09T21:31:23Z
Clean copy of upstream master
commit a6e7a307b97cb951dcbb20aa52e08ac1c24d43da
Author: Marc Culler <[email protected]>
Date: 2014-12-09T21:47:19Z
Merge branch 'master' of https://github.com/apache/spark into clean
commit 26aa58f951df85c279da844775dbee9d7a0953ff
Author: Marc Culler <[email protected]>
Date: 2014-12-09T22:34:59Z
Added StringContext method to convert String to UnresolvedAttribute.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
