[
https://issues.apache.org/jira/browse/PARQUET-218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14363810#comment-14363810
]
Neville Li commented on PARQUET-218:
------------------------------------
True in most cases it doesn't and we only use a tiny subset.
In Projection[T], only avroObj.getField1.getField2... is allowed. And in
Predicate[T], only basic boolean comparison & logical expression are allowed.
Value expressions/literals from the closure will be pulled in as is.
Also it probably only works for data representation formats from schema, e.g.
Avro, Proto, Thrift?
> Macro based Scala APIs for Predicate and Projection push down
> -------------------------------------------------------------
>
> Key: PARQUET-218
> URL: https://issues.apache.org/jira/browse/PARQUET-218
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-mr
> Reporter: Julien Le Dem
>
> There has been some experiments with Scala macros to make Predicate and
> Projection APIs in Scala.
> In particular by [~neville]:
> https://github.com/nevillelyh/parquet-avro-extra/blob/master/parquet-avro-examples/src/main/scala/me/lyh/parquet/avro/examples/ExampleApp.scala
> https://github.com/nevillelyh/parquet-avro-extra/commit/090f8d1d1d7d44489bb71e04465d4a12a46432c9#diff-89dca6adcdf105c6063661de43931c29R15
> There have also been some discussion at Twitter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
