allthingssecurity commented on code in PR #27696:
URL: https://github.com/apache/camel/pull/27696#discussion_r4245780790
##########
components/camel-atom/src/main/java/org/apache/camel/component/atom/AtomEntryPollingConsumer.java:
##########
@@ -67,6 +69,12 @@ private List<Item> readItems() throws IOException {
if (items == null) {
items = AtomUtils.readItems(getEndpoint().getCamelContext(),
getEndpoint().getFeedUri(), rssReader,
getEndpoint().isSortEntries());
+ if (getEndpoint().isSortEntries()) {
+ // sorted entries have the oldest entry first, but the entries
are routed from the end of the list
+ // (without sortEntries the newest entry is first): reverse
them, so the oldest entry is routed first
Review Comment:
You are right. The natural order of `Item` is
`ItemComparator.newestPublishedItemFirst()` (rssreader 3.12.0), so
the reversed list was the order of the default and `sortEntries` did nothing
for the split consumer.
Changed in edbcc7bf9a19 as you suggest: with `sortEntries=true` the consumer
sorts with
`ItemComparator.newestUpdatedItemFirst()` (AtomEntryPollingConsumer.java:77)
and the loop routes from the end, so
the oldest updated entry is routed first. That comparator already falls back
to the published date for an entry
without an updated date, which is the same date
`ItemUpdatedIdempotentStrategy` compares. I checked the two
references: 3.22.x has `feed.sortEntriesByUpdated(true)` and its
`AtomEntrySortTest` expects ascending updated
dates, and camel-rss sorts with `RssDateComparator` (updated, else
published) and routes the oldest first.
New test data `edited-feed.atom`: four entries published in the order first
to fourth, the first one updated after
all the others, the third without an updated date. Expected: `second, third,
fourth, first`.
- main: `[fourth, first]` is routed, second and third are lost.
- previous head (bb9f3fad5c4c): `[first]` is routed, the three newer entries
are skipped.
- now: all four, in that order, also with `idempotent=false`.
Not changed: `AtomUtils.readItems`, so the list of a consumer with
`splitEntries=false` is as before (published
date, oldest first), and camel-rss. The description of the option
(`FeedEndpoint`, shared with camel-rss) said "by
published date"; it now names the updated date with the published date as
fallback.
_Claude Code on behalf of allthingssecurity_
##########
components/camel-atom/src/test/java/org/apache/camel/component/atom/AtomEntrySortTest.java:
##########
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.atom;
+
+import java.time.ZonedDateTime;
+import java.util.ArrayList;
+import java.util.List;
+
+import com.apptasticsoftware.rssreader.Item;
+import org.apache.camel.Consumer;
+import org.apache.camel.test.junit6.CamelTestSupport;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.condition.DisabledOnOs;
+import org.junit.jupiter.api.condition.OS;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+/**
+ * With sortEntries the entries of a feed must be routed from the oldest to
the newest one, as camel-rss does. The
+ * default idempotent strategy skips an entry that is older than the one
routed before it, so any other order loses
+ * entries.
+ */
+@DisabledOnOs(OS.AIX)
+class AtomEntrySortTest extends CamelTestSupport {
+
+ // the feed has 10 entries, which are not in the order of their dates
+ private static final String FEED
+ =
"atom:file:src/test/data/unsortedfeed.atom?splitEntries=true&throttleEntries=false&startScheduler=false";
+
+ @Test
+ void sortedEntriesAreAllRoutedOldestFirst() throws Exception {
+ List<ZonedDateTime> routed = pollOnce(FEED + "&sortEntries=true");
+
+ assertEquals(10, routed.size(), "All the entries of the feed must be
routed");
+ assertEquals(sorted(routed), routed);
+ }
+
+ @Test
+ void sortedEntriesAreRoutedOldestFirstWithoutIdempotency() throws
Exception {
+ List<ZonedDateTime> routed = pollOnce(FEED +
"&sortEntries=true&idempotent=false");
+
+ assertEquals(10, routed.size());
+ assertEquals(sorted(routed), routed);
+ }
+
+ @Test
+ void entriesAreAllRoutedWithoutSortEntries() throws Exception {
+ List<ZonedDateTime> routed = pollOnce(FEED);
+
+ assertEquals(10, routed.size());
+ assertEquals(sorted(routed), routed);
+ }
+
+ /**
+ * Runs one poll of a consumer of the given endpoint (its scheduler is not
started) and returns the dates of the
+ * routed entries.
+ */
+ private List<ZonedDateTime> pollOnce(String uri) throws Exception {
+ List<ZonedDateTime> routed = new ArrayList<>();
+ Consumer consumer = context.getEndpoint(uri).createConsumer(
+ exchange ->
routed.add(exchange.getIn().getBody(Item.class).getUpdatedZonedDateTime().orElseThrow()));
Review Comment:
Done in edbcc7bf9a19: the test uses `getUpdatedAsZonedDateTime()` (it is in
rssreader 3.12.0, the version of the
build).
_Claude Code on behalf of allthingssecurity_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
