Github user robdouglas commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/51#discussion_r15954062
--- Diff:
streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/provider/TwitterTimelineProvider.java
---
@@ -272,12 +210,11 @@ void shutdownAndAwaitTermination(ExecutorService
pool) {
}
}
-
@Override
public void prepare(Object o) {
--- End diff --
I'm not sure I agree that using the numericIdsOnly and screenNamesOnly are
the best call in the prepare method. Is the goal here to ensure that our "id"
list contains nothing but numeric IDs, given the possibility that the list of
identifiers that we start with can have both screenNames as well as numeric IDs?
If that's the case then I would suggest getting rid of the numericIdsOnly
and screenNamesOnly methods and replace them with something like this:
''''
@Override
public void prepare(Object o) {
executor = getExecutor();
try {
lock.writeLock().lock();
providerQueue = constructQueue();
} finally {
lock.writeLock().unlock();
}
Preconditions.checkNotNull(providerQueue);
Preconditions.checkNotNull(this.klass);
Preconditions.checkNotNull(config.getOauth().getConsumerKey());
Preconditions.checkNotNull(config.getOauth().getConsumerSecret());
Preconditions.checkNotNull(config.getOauth().getAccessToken());
Preconditions.checkNotNull(config.getOauth().getAccessTokenSecret());
Preconditions.checkNotNull(config.getInfo());
consolidateToIDs();
}
/**
* Using the "info" list that is contained in the configuration, ensure
that all
* account identifiers are converted to IDs (Longs) instead of
screenNames (Strings)
*/
private void consolidateToIDs() {
List<String> screenNames = Lists.newArrayList();
ids = Lists.newArrayList();
for(Object account : config.getInfo()) {
if(account instanceof String) {
screenNames.add((String)account);
} else if (account instanceof Long) {
ids.add(Long.parseLong(Objects.toString(account, null)));
}
}
// Twitter allows for batches up to 100 per request, but you cannot
mix types
screenNameBatches = new ArrayList<String[]>();
while(screenNames.size() >= 100) {
screenNameBatches.add(screenNames.subList(0, 100).toArray(new
String[0]));
screenNames = screenNames.subList(100, screenNames.size());
}
if(screenNames.size() > 0)
screenNameBatches.add(screenNames.toArray(new
String[ids.size()]));
Iterator<String[]> screenNameBatchIterator =
screenNameBatches.iterator();
while(screenNameBatchIterator.hasNext()) {
Collection<Long> batchIds =
retrieveIds(screenNameBatchIterator.next());
ids.addAll(batchIds);
}
}
''''
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
