[
https://issues.apache.org/jira/browse/KUDU-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17170998#comment-17170998
]
ASF subversion and git services commented on KUDU-3177:
-------------------------------------------------------
Commit 40289e2a2faa021826b9424864ab2935507bef33 in kudu's branch
refs/heads/master from kevinmccarthy
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=40289e2 ]
[KUDU-3177] Added kudu.snapshotTimestampMicros to kudu spark readOptions
as optional property
Added property snapshotTimestampMs to spark read options which will
allow consistant scanswhen timestamp is set before the first
dataFrame read.
Change-Id: I00862c0e174a964efc6cab0b8141b1ac5a1bebc0
Reviewed-on: http://gerrit.cloudera.org:8080/16276
Tested-by: Kudu Jenkins
Reviewed-by: Grant Henke <[email protected]>
> Expose snapshotTimestampMicros to Spark Read Options
> ----------------------------------------------------
>
> Key: KUDU-3177
> URL: https://issues.apache.org/jira/browse/KUDU-3177
> Project: Kudu
> Issue Type: Improvement
> Components: spark
> Reporter: Kevin J McCarthy
> Priority: Major
> Labels: beginner
>
> If a spark application needs to read from the same table multiple times and
> that table has new records that may come in during the life of the
> application, you may get inconsistent scan results unless you persist the
> DataFrame. I'd like to expose snapshotTimestampMicros to the spark read
> options so I can set a timestamp before the first scan and use that for
> READ_AT_SNAPSHOT to keep all scans on the same table consistent throughout
> the run of the application.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
