[
https://issues.apache.org/jira/browse/BEAM-13159?focusedWorklogId=695230&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-695230
]
ASF GitHub Bot logged work on BEAM-13159:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Dec/21 17:31
Start Date: 13/Dec/21 17:31
Worklog Time Spent: 10m
Work Description: n-oden commented on a change in pull request #15858:
URL: https://github.com/apache/beam/pull/15858#discussion_r767973423
##########
File path:
sdks/java/io/redis/src/test/java/org/apache/beam/sdk/io/redis/RedisIOTest.java
##########
@@ -250,6 +260,102 @@ public void testWriteUsingDECRBY() {
assertEquals(-1, count);
}
+ @Test
+ public void testWriteStreams() {
+ ArrayList<String> streams = new ArrayList<String>();
+ for (int i = 0; i <= 10; i++) {
+ UUID uuid = UUID.randomUUID();
+ /* stream keys are uuids to ensure that test runs are idempotent */
+ streams.add(uuid.toString());
+ }
+ Map<String, String> fooValues = ImmutableMap.of("sensor-id", "1234",
"temperature", "19.8");
+ Map<String, String> barValues = ImmutableMap.of("sensor-id", "9999",
"temperature", "18.2");
+
+ List<KV<String, Map<String, String>>> fooData =
+ streams.stream().map(key -> KV.of(key,
fooValues)).collect(Collectors.toList());
+
+ List<KV<String, Map<String, String>>> barData =
+ streams.stream().map(key -> KV.of(key,
barValues)).collect(Collectors.toList());
+
+ List<KV<String, Map<String, String>>> allData =
+ Stream.of(fooData,
barData).flatMap(Collection::stream).collect(Collectors.toList());
Review comment:
Aha, thank you, I knew there had to be a simpler way to do this. I've
made the slight adjustment of preserving the randomized key names (otherwise
test runs end up not necessarily being idempotent) but this is much more
readable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 695230)
Time Spent: 10h 40m (was: 10.5h)
> Support for Redis 5.0 streams (XADD/XREAD/XRANGE) in RedisIO
> ------------------------------------------------------------
>
> Key: BEAM-13159
> URL: https://issues.apache.org/jira/browse/BEAM-13159
> Project: Beam
> Issue Type: New Feature
> Components: io-java-redis
> Reporter: N. M.
> Assignee: N. M.
> Priority: P2
> Time Spent: 10h 40m
> Remaining Estimate: 0h
>
> Redis 5.0 and later support an append-only log data structure called a
> "stream" -- each element has a unique incrementing ID and contains one or
> more string:string key-value pairs.
> This seems like a gimme for a stream processing platform to support, and in
> fact Jedis already has support for XADD/XREAD/XRANGE in it, but the current
> redisIO Read and Write classes only support pcollections of single KVs and
> thus will need some additional work to support streams.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
