[
https://issues.apache.org/jira/browse/HADOOP-15364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16726445#comment-16726445
]
Sameer Choudhary commented on HADOOP-15364:
-------------------------------------------
Took a quick initial pass through the diffs on
[https://github.com/apache/hadoop/compare/trunk...steveloughran:s3/select/HADOOP-15364-S3-select-trunk.]
It would be better to a have a PR where we can track all the feedback for the
release. Not sure if we are already tracking cr feedback through some other
mechanism. If so, then please do share with me.
At a high level, I have following feedback (primarily on
[https://github.com/steveloughran/hadoop/commit/875062e43d6144a66eac12e911902fa7ba6befde]):
# We need to improve error handling and resource closer.
# Documentation can be improved.
# Need integration and load tests with real S3 service.
I will perform a more thorough review and will get back to you.
> Add support for S3 Select to S3A
> --------------------------------
>
> Key: HADOOP-15364
> URL: https://issues.apache.org/jira/browse/HADOOP-15364
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Attachments: HADOOP-15364-001.patch, HADOOP-15364-002.patch,
> HADOOP-15364-004.patch
>
>
> Expect a PoC patch for this in a couple of days;
> * it'll depend on an SDK update to work, plus a couple of of other minor
> changes
> * Adds command line option too
> {code}
> hadoop s3guard select -header use -compression gzip -limit 100
> s3a://landsat-pds/scene_list.gz" \
> "SELECT s.entityId FROM S3OBJECT s WHERE s.cloudCover = '0.0' "
> {code}
> For wider use we'll need to implement the HADOOP-15229 so that callers can
> pass down the expression along with any other parameters
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
