GitHub user rxin opened a pull request:
https://github.com/apache/spark/pull/6690
[SPARK-8117] [SQL] Push codegen implementation into each Expression
This PR move codegen implementation of expressions into Expression class
itself, make it easy to manage.
It introduces two APIs in Expression:
```
def gen(ctx: CodeGenContext): GeneratedExpressionCode
def genCode(ctx: CodeGenContext, ev: GeneratedExpressionCode): Code
```
gen(ctx) will call genSource(ctx, ev) to generate Java source code for the
current expression. A expression needs to override genSource().
Here are the types:
```
type Term String
type Code String
/**
* Java source for evaluating an [[Expression]] given a [[Row]] of input.
*/
case class GeneratedExpressionCode(var code: Code,
nullTerm: Term,
primitiveTerm: Term,
objectTerm: Term)
/**
* A context for codegen, which is used to bookkeeping the expressions
those are not supported
* by codegen, then they are evaluated directly. The unsupported expression
is appended at the
* end of `references`, the position of it is kept in the code, used to
access and evaluate it.
*/
class CodeGenContext {
/**
* Holding all the expressions those do not support codegen, will be
evaluated directly.
*/
val references: Seq[Expression] = new mutable.ArrayBuffer[Expression]()
}
```
This is basically #6660, but fixed style violation and compilation failure.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rxin/spark codegen
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/6690.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6690
----
commit 593d61704844989e5d2e61d39efe4d7b7a0e1d1a
Author: Davies Liu <[email protected]>
Date: 2015-06-03T22:27:44Z
pushing codegen into Expression
commit 3ff25f81a8fc6840b5c6dc75377fc89e41454586
Author: Davies Liu <[email protected]>
Date: 2015-06-04T00:45:19Z
refactor
commit e57959d60bb841851623898790a5cb1cba314cdd
Author: Davies Liu <[email protected]>
Date: 2015-06-05T00:00:17Z
add type alias
commit b1450476fb355699326aaedcc5f9bac2806e4cd5
Author: Davies Liu <[email protected]>
Date: 2015-06-05T00:05:42Z
fix style
commit 8c6d82d61fdc81755b8971fdaa73729617d6df2f
Author: Davies Liu <[email protected]>
Date: 2015-06-05T00:09:05Z
update docs
commit c5fb5146ee476b4f4b70ac34dbca0cafd9e249dd
Author: Davies Liu <[email protected]>
Date: 2015-06-05T00:18:40Z
rename
commit 12ff88a4297871ec8a39bddedf0c377a46361c1b
Author: Davies Liu <[email protected]>
Date: 2015-06-05T00:35:12Z
fix build
commit 2344bc0d48fc2a3ec91de69a6233665a0ae3635e
Author: Davies Liu <[email protected]>
Date: 2015-06-05T02:43:12Z
fix test
commit 48c454ff529f045e6c24b6f73d85d09dd4b4279a
Author: Reynold Xin <[email protected]>
Date: 2015-06-05T08:08:56Z
Some code gen update.
commit b5d3617af9742593571c61c9a34736c8781f92b2
Author: Davies Liu <[email protected]>
Date: 2015-06-05T18:16:56Z
Merge pull request #5 from rxin/codegen
Code gen code review.
commit 02262c91747bcd12579ebc7784d5047f6b73a268
Author: Davies Liu <[email protected]>
Date: 2015-06-05T20:04:01Z
address comments
commit 86fac2c6dc7add53b694ccaab6cf676d312a1da8
Author: Davies Liu <[email protected]>
Date: 2015-06-05T20:42:37Z
fix style
commit e03edaaf9c235c2acd86e97e8f6c4efb21487437
Author: Davies Liu <[email protected]>
Date: 2015-06-05T21:30:41Z
consts fold
commit bad682819a2013eff8c9e3c9a8c106dc6f8b924f
Author: Davies Liu <[email protected]>
Date: 2015-06-05T22:01:26Z
address comments
commit f42c732febd5fa720ab72dfa0fc442c847012b8c
Author: Davies Liu <[email protected]>
Date: 2015-06-06T07:23:31Z
improve coverage and tests
commit 9adaeaf9963afa42c0c9aa2ddf562df5d6a8d9ba
Author: Davies Liu <[email protected]>
Date: 2015-06-06T07:38:28Z
address comments
commit 19d643580ba45211ea21248d86b9c59d0ffa4c7f
Author: Reynold Xin <[email protected]>
Date: 2015-06-07T06:13:30Z
Fixed style violation.
commit 73db80ee0cc4f7f5ab8aaaceeb527e2ea81a2d04
Author: Reynold Xin <[email protected]>
Date: 2015-06-07T08:13:56Z
Fixed compilation failure.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
