[GitHub] incubator-streams issue #370: partly resolves STREAMS-513
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/370 closing - will rebase including the STREAMS-517 branch which fixed the compile error in master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #370: partly resolves STREAMS-513
Github user steveblackmon closed the pull request at: https://github.com/apache/incubator-streams/pull/370 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #371: upgrade instagram to work with latest j...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/371 upgrade instagram to work with latest juneau You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-517 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/371.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #371 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #371: upgrade instagram to work with latest juneau
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/371 merging now to resolve build issue --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #370: partly resolves STREAMS-513
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/370 partly resolves STREAMS-513 the new ITs arenât passing yet... but the code in master fails to compile this module so IMO this is an improvement worth merging although we need to get the tests passing before doing a release You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-513 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/370.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #370 commit bcbf8408f72538e7b6a40a2ed6169d57c635d6a6 Author: Steve Blackmon <sblack...@apache.org> Date: 2017-06-10T19:30:06Z resolves STREAMS-513 - except the ITs arenât passing yet --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #369: resolves STREAMS-516
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/369 trivial change, not breaking. merging now in the interest of restoring jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #369: resolves STREAMS-516
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/369 resolves STREAMS-516 You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-516 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/369.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #369 commit b757539256a8f96c33ad076784528a601043a8e2 Author: Steve Blackmon <sblack...@apache.org> Date: 2017-06-21T03:38:54Z resolves STREAMS-516 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #368: resolves STREAMS-508
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/368#discussion_r123142711
--- Diff:
streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/TwitterOAuthRequestInterceptor.java
---
@@ -84,16 +85,20 @@ public void process(HttpRequest httpRequest,
HttpContext httpContext) throws Htt
oauthParamMap.put("oauth_version", oauth_version);
String request_host =
((HttpRequestWrapper)httpRequest).getTarget().toString().replace(":443","");
-String request_path =
httpRequest.getRequestLine().getUri().substring(0,
httpRequest.getRequestLine().getUri().indexOf('?'));
-String request_param_line =
httpRequest.getRequestLine().getUri().substring(httpRequest.getRequestLine().getUri().indexOf('?')+1);
-String[] request_params =
URLDecoder.decode(request_param_line).split("&");
+String request_path = httpRequest.getRequestLine().getUri();
Map<String,String> allParamsMap = new HashMap<>(oauthParamMap);
-for( String request_param : request_params ) {
- String key = request_param.substring(0, request_param.indexOf('='));
- String value = request_param.substring(request_param.indexOf('=')+1,
request_param.length());
- allParamsMap.put(key, value);
+if( request_path.contains("?")) {
--- End diff --
having now looked into that class, i agree that using it could reduce
number of lines of code. on the other hand, I just tried to rewrite the class
to use it (changed ~100 lines) and the unit tests are failing without a clear
remedy. so i'm going to note this on a new ticket and attach a patch file. it
took a LOT of tweaking and testing to get the as-is implementation working...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #368: resolves STREAMS-508
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/368#discussion_r123141972
--- Diff:
streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/TwitterRetryHandler.java
---
@@ -22,141 +22,68 @@
import org.apache.streams.util.api.requests.backoff.BackOffException;
import
org.apache.streams.util.api.requests.backoff.impl.LinearTimeBackOffStrategy;
+import org.apache.http.HttpResponse;
import org.apache.http.client.HttpRequestRetryHandler;
import org.apache.http.protocol.HttpContext;
+import org.apache.http.util.EntityUtils;
+import org.apache.juneau.json.JsonParser;
+import org.apache.juneau.parser.ParseException;
import org.apache.juneau.rest.client.RetryOn;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
+import javax.ws.rs.core.Response;
/**
* Handle expected and unexpected exceptions.
*/
-public class TwitterRetryHandler implements RetryOn {
+public class TwitterRetryHandler extends RetryOn {
private static final Logger LOGGER =
LoggerFactory.getLogger(TwitterRetryHandler.class);
- private static AbstractBackOffStrategy backoff_strategy;
-
-// TODO: once request context is available, we can maintain multiple
BackoffStrategy one per request path / params
-// private static Map<String, AbstractBackOffStrategy> backoffs = new
ConcurrentHashMap<>();
-
-// This is everything we used to check via twitter4j to decide whether to
retry.
-//
-// @Deprecated
-// public static int handleTwitterError(Twitter twitter, Exception
exception) {
-//return handleTwitterError( twitter, null, exception);
-// }
-//
-//
-// public static int handleTwitterError(Twitter twitter, Long id,
Exception exception) {
-//
-//if (exception instanceof TwitterException) {
-// TwitterException twitterException = (TwitterException)exception;
-//
-// if (twitterException.exceededRateLimitation()) {
-//
-//long millisUntilReset = retry;
-//
-//final RateLimitStatus rateLimitStatus =
twitterException.getRateLimitStatus();
-//if (rateLimitStatus != null) {
-// millisUntilReset = rateLimitStatus.getSecondsUntilReset() *
1000;
-//}
-//
-//LOGGER.warn("Rate Limit Exceeded. Will retry in {} seconds...",
millisUntilReset / 1000);
-//
-//try {
-// Thread.sleep(millisUntilReset);
-//} catch (InterruptedException e1) {
-// Thread.currentThread().interrupt();
-//}
-//
-//return 1;
-// } else if (twitterException.isCausedByNetworkIssue()) {
-//LOGGER.info("Twitter Network Issues Detected. Backing off...");
-//LOGGER.info("{} - {}", twitterException.getExceptionCode(),
twitterException.getLocalizedMessage());
-//try {
-// Thread.sleep(retry);
-//} catch (InterruptedException e1) {
-// Thread.currentThread().interrupt();
-//}
-//return 1;
-// } else if (twitterException.isErrorMessageAvailable()) {
-//if (twitterException.getMessage().toLowerCase().contains("does
not exist")) {
-// if ( id != null ) {
-//LOGGER.warn("User does not exist: {}", id);
-// } else {
-//LOGGER.warn("User does not exist");
-// }
-// return (int)retryMax;
-//} else {
-// return (int)retryMax / 3;
-//}
-// } else {
-//if
(twitterException.getExceptionCode().equals("ced778ef-0c669ac0")) {
-// // This is a known weird issue, not exactly sure the cause,
but you'll never be able to get the data.
-// return (int)retryMax / 3;
-//} else if
(twitterException.getExceptionCode().equals("4be80492-0a7bf7c7")) {
-// // This is a 401 reflecting credentials don't have access to
the requested resource.
-// if ( id != null ) {
-//LOGGER.warn("Authentication Exception accessing id: {}", id);
-// } else {
-//LOGGER.warn("Authentication Exception");
-// }
-// return (int)retryMax;
-//} else {
-// LOGGER.warn("Unknown Twitter Exception...");
-// LOGGER.warn(" Access: {}",
twitterException.getAccessLevel());
-// LOGGER.warn(" Code: {}",
twitterException.getExcept
[GitHub] incubator-streams pull request #368: resolves STREAMS-508
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/368#discussion_r123141835
--- Diff:
streams-contrib/streams-provider-twitter/src/main/jsonschema/org/apache/streams/twitter/api/AccountSettingsResponse.json
---
@@ -0,0 +1,111 @@
+{
+ "$schema": "http://json-schema.org/draft-03/schema;,
+ "$license": [
+"http://www.apache.org/licenses/LICENSE-2.0;
+ ],
+ "id": "#",
+ "javaType": "org.apache.streams.twitter.api.AccountSettingsResponse",
+ "javaInterfaces": [
+"java.io.Serializable"
+ ],
+ "description":
"https://dev.twitter.com/rest/reference/get/account/settings;,
+ "properties": {
+"always_use_https": {
+ "type": "boolean"
+},
+"discoverable_by_email": {
+ "type": "boolean"
+},
+"geo_enabled": {
+ "type": "boolean"
+},
+"language": {
+ "type": "string"
+},
+"language": {
--- End diff --
no. fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #368: resolves STREAMS-508
Github user steveblackmon commented on a diff in the pull request: https://github.com/apache/incubator-streams/pull/368#discussion_r123141789 --- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Users.java --- @@ -46,6 +56,7 @@ * @see https://dev.twitter.com/rest/reference/get/users/show;>https://dev.twitter.com/rest/reference/get/users/show --- End diff -- fixed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #368: resolves STREAMS-508
Github user steveblackmon commented on a diff in the pull request: https://github.com/apache/incubator-streams/pull/368#discussion_r123141780 --- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Users.java --- @@ -36,7 +45,8 @@ * @see https://dev.twitter.com/rest/reference/get/users/lookup;>https://dev.twitter.com/rest/reference/get/users/lookup --- End diff -- fixed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #368: resolves STREAMS-508
Github user steveblackmon commented on a diff in the pull request: https://github.com/apache/incubator-streams/pull/368#discussion_r12314 --- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Statuses.java --- @@ -47,7 +83,8 @@ * @see https://dev.twitter.com/rest/reference/get/statuses/show/id;>https://dev.twitter.com/rest/reference/get/statuses/show/id --- End diff -- fixed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #368: resolves STREAMS-508
Github user steveblackmon commented on a diff in the pull request: https://github.com/apache/incubator-streams/pull/368#discussion_r123111042 --- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Friends.java --- @@ -47,6 +53,7 @@ * @see https://dev.twitter.com/rest/reference/get/friends/list;>https://dev.twitter.com/rest/reference/get/friends/list --- End diff -- fixed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #368: resolves STREAMS-508
Github user steveblackmon commented on a diff in the pull request: https://github.com/apache/incubator-streams/pull/368#discussion_r123111031 --- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Friends.java --- @@ -37,7 +42,8 @@ * @see https://dev.twitter.com/rest/reference/get/friends/ids;>https://dev.twitter.com/rest/reference/get/friends/ids --- End diff -- fixed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #368: resolves STREAMS-508
Github user steveblackmon commented on a diff in the pull request: https://github.com/apache/incubator-streams/pull/368#discussion_r123110925 --- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Followers.java --- @@ -37,7 +42,8 @@ * @see https://dev.twitter.com/rest/reference/get/followers/ids;>https://dev.twitter.com/rest/reference/get/followers/ids --- End diff -- fixed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #368: resolves STREAMS-508
Github user steveblackmon commented on a diff in the pull request: https://github.com/apache/incubator-streams/pull/368#discussion_r123110941 --- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Followers.java --- @@ -47,6 +53,7 @@ * @see https://dev.twitter.com/rest/reference/get/followers/list;>https://dev.twitter.com/rest/reference/get/followers/list --- End diff -- fixed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #368: resolves STREAMS-508
Github user steveblackmon commented on a diff in the pull request: https://github.com/apache/incubator-streams/pull/368#discussion_r123110842 --- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Account.java --- @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.streams.twitter.api; + +import org.apache.streams.twitter.pojo.User; + +import org.apache.juneau.remoteable.RemoteMethod; +import org.apache.juneau.remoteable.Remoteable; + +import java.util.List; + +/** --- End diff -- fixed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #368: resolves STREAMS-508
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/368#discussion_r123110801
--- Diff:
streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Account.java
---
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.twitter.pojo.User;
+
+import org.apache.juneau.remoteable.RemoteMethod;
+import org.apache.juneau.remoteable.Remoteable;
+
+import java.util.List;
+
+/**
+ */
+@Remoteable(path = "https://api.twitter.com/1.1/account;)
+public interface Account {
+
+ /**
+ * Returns settings (including current trend, geo and sleep time
information) for the authenticating user.
+ *
+ * @return AccountSettingsResponse
+ * @see https://dev.twitter.com/rest/reference/get/account/settings;>https://dev.twitter.com/rest/reference/get/account/settings
+ *
+ */
+ @RemoteMethod(httpMethod = "GET", path = "/settings.json")
+ public AccountSettings settings();
+
+ /**
+ * Returns settings (including current trend, geo and sleep time
information) for the authenticating user.
--- End diff --
fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams issue #368: resolves STREAMS-508
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/368 this changeset only works against the snapshot, but fortunately juneau 6.3.0 rc1 vote just started. i will incorporate your suggested changes, update the poms to use the release, and confirm that all of the integration tests are passing before merging. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #368: resolves STREAMS-508
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/368 resolves STREAMS-508 You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-508 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/368.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #368 commit af6368249963440bbfb3cdcec159f9fd14106e23 Author: Steve Blackmon <sblack...@apache.org> Date: 2017-06-10T19:55:22Z resolves STREAMS-508 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #358: STREAMS-499: fix behavior of site plugin
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/358 this was merged, just didn't close the PR --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #358: STREAMS-499: fix behavior of site plugi...
Github user steveblackmon closed the pull request at: https://github.com/apache/incubator-streams/pull/358 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #367: STREAMS-512: add juneau annotations to all gen...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/367 There are a lot of juneau related updates in steveblackmon/STREAMS-504 which i intend to rebase on this, and will remove the hardcoded pom versions at that point (and bump to 6.2.1-incubating once that release completes) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #367: STREAMS-512: add juneau annotations to ...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/367 STREAMS-512: add juneau annotations to all generated pojos also add juneau-core to streams-pojo and streams-config You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-512 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/367.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #367 commit c49624a17ece024dc9f4567535f47904a4884232 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-05-23T07:31:57Z STREAMS-512: add juneau annotations to all generated pojos also add juneau-core to streams-pojo and streams-config --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #366: STREAMS-506 Add ASFv2 License to InstagramOAut...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/366 :+1: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #360: STREAMS-496: Remove twitter4j dependency from ...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/360 addressed all PR feedback and double-checked that build and tests pass. merging --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #362: STREAMS-502: rewrite instagram provider...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/362 STREAMS-502: rewrite instagram provider using juneau You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-502 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/362.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #362 commit f582374b5145479e2d3a623fed24dca732523b18 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-13T03:35:38Z pom commit 2c09a730491c29fa7009e938019471802fef90f9 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-13T03:36:20Z configs change commit 67ee5f9ad516cfdcad0eb3dde1f9ac828c5de550 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-13T03:36:30Z base pojos commit 5747fd87b2191355324c8c0c145b75c8115c8a10 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-13T03:36:37Z api pojos commit 9a409917bc6c12af0deebdcd70c3858c769e7498 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-13T03:37:11Z interfaces commit 923f483cef3296f08e67228712877aed16111d9f Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-13T03:37:38Z primary class commit 6ff4d3fe649323e345c7f0cf104f08a5198501f4 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-13T04:07:51Z need version b/c STREAMS-496 hasnât merged yet commit 5b9b51cbf265a4b5f99beeb6f5d1f01c9e2c2106 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-13T04:08:11Z request signing commit ad6c63159d77b077bd2d8798cdea1fe3b26019e3 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-13T04:08:20Z retry handling commit 341bfc63515246ed1eee0197451502b79e37dfdb Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-13T04:08:48Z updated providers commit c8d12837bd48d556d83dd274442836077b5e0195 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-13T04:08:57Z updated converters commit 51e63caff45e6ae56a87e40259f2b45502a9d1b2 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-13T04:09:08Z updated tests commit 41aa562a96e2f83a2691f177365c62ec5a2e3f22 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-13T04:09:24Z unnecessary class commit 6e3a2e65b03a8eb5e9a545865904838584583477 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-13T04:29:54Z primary implement Media commit 93f03e6088bc17881a9ca66368f5d0e0018b27a9 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-15T19:52:10Z licenses / headers commit 802385bc501f599121cd5134202bea5412fe1cb5 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-15T20:23:04Z checkstyle fixes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111310493
--- Diff:
streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Twitter.java
---
@@ -0,0 +1,521 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.jackson.StreamsJacksonMapper;
+import org.apache.streams.twitter.TwitterConfiguration;
+import org.apache.streams.twitter.pojo.Tweet;
+import org.apache.streams.twitter.pojo.User;
+import org.apache.streams.twitter.provider.TwitterProviderUtil;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.HttpRequestInterceptor;
+import org.apache.http.client.HttpRequestRetryHandler;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.params.ClientPNames;
+import org.apache.http.client.params.CookiePolicy;
+import org.apache.http.client.utils.URIBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.juneau.json.JsonParser;
+import org.apache.juneau.parser.ParseException;
+import org.apache.juneau.plaintext.PlainTextSerializer;
+import org.apache.juneau.rest.client.RestCall;
+import org.apache.juneau.rest.client.RestCallException;
+import org.apache.juneau.rest.client.RestClient;
+//import org.apache.juneau.rest.client.RestClientBuilder;
+import org.apache.juneau.rest.client.RetryOn;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URISyntaxException;
+import java.util.ArrayList;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.ConcurrentHashMap;
+
+/**
+ * Implementation of all twitter interfaces using juneau.
+ */
+public class Twitter implements Followers, Friends, Statuses, Users {
+
+ private static final Logger LOGGER =
LoggerFactory.getLogger(Twitter.class);
+
+ private static Map<TwitterConfiguration, Twitter> INSTANCE_MAP = new
ConcurrentHashMap<>();
+
+ private TwitterConfiguration configuration;
+
+ private ObjectMapper mapper;
+
+ private String rootUrl;
+
+ private CloseableHttpClient httpclient;
+
+ private HttpRequestInterceptor oauthInterceptor;
+
+ RestClient restClient;
+
+ private Twitter(TwitterConfiguration configuration) throws
InstantiationException {
+this.configuration = configuration;
+this.rootUrl = TwitterProviderUtil.baseUrl(configuration);
+this.oauthInterceptor = new
TwitterOAuthRequestInterceptor(configuration.getOauth());
+this.httpclient = HttpClientBuilder.create()
+.addInterceptorFirst(oauthInterceptor)
+.setDefaultRequestConfig(RequestConfig.custom()
+ .setConnectionRequestTimeout(5000)
+ .setConnectTimeout(5000)
+ .setSocketTimeout(5000)
+ .setCookieSpec("easy")
+ .build()
+)
+.setMaxConnPerRoute(20)
+.setMaxConnTotal(100)
+.build();
+
+// TODO: juneau-6.2.0-incubating
+// this.restClient = new RestClientBuilder()
+//.httpClient(httpclient, true)
+//.parser(JsonParser.class)
+//.rootUrl(rootUrl)
+//.retryable(
+//configuration.getRetryMax().intValue(),
+//configuration.getRetrySleepMs(),
+//new TwitterRetryHandler())
+//.build();
+this.restClient = new RestClient()
+.setHttpClient(httpclient)
+.setParser(JsonParser.class)
+.setRootUrl(rootUrl);
+
+this.mapper = StreamsJacksonMapper.getInsta
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111310308
--- Diff:
streams-contrib/streams-provider-twitter/src/main/jsonschema/org/apache/streams/twitter/api/FollowingIdsRequest.json
---
@@ -0,0 +1,36 @@
+{
+ "$schema": "http://json-schema.org/draft-03/schema;,
+ "$license": [
+"http://www.apache.org/licenses/LICENSE-2.0;
+ ],
+ "id": "#",
+ "javaType" : "org.apache.streams.twitter.api.FollowingIdsRequest",
+ "javaInterfaces": ["java.io.Serializable"],
+ "properties": {
+"id": {
+ "description": "The ID of the user for whom to return results for.",
+ "required": false,
+ "type": "integer"
+},
+"screen_name": {
+ "description": "The screen name of the user for whom to return
results for.",
+ "required": false,
+ "type": "string"
+},
+"curser": {
--- End diff --
nice catch. that was wrong a bunch of places. all fixed now and i watch
the providers page as expected.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111310313
--- Diff:
streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Twitter.java
---
@@ -0,0 +1,521 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.jackson.StreamsJacksonMapper;
+import org.apache.streams.twitter.TwitterConfiguration;
+import org.apache.streams.twitter.pojo.Tweet;
+import org.apache.streams.twitter.pojo.User;
+import org.apache.streams.twitter.provider.TwitterProviderUtil;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.HttpRequestInterceptor;
+import org.apache.http.client.HttpRequestRetryHandler;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.params.ClientPNames;
+import org.apache.http.client.params.CookiePolicy;
+import org.apache.http.client.utils.URIBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.juneau.json.JsonParser;
+import org.apache.juneau.parser.ParseException;
+import org.apache.juneau.plaintext.PlainTextSerializer;
+import org.apache.juneau.rest.client.RestCall;
+import org.apache.juneau.rest.client.RestCallException;
+import org.apache.juneau.rest.client.RestClient;
+//import org.apache.juneau.rest.client.RestClientBuilder;
+import org.apache.juneau.rest.client.RetryOn;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URISyntaxException;
+import java.util.ArrayList;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.ConcurrentHashMap;
+
+/**
+ * Implementation of all twitter interfaces using juneau.
+ */
+public class Twitter implements Followers, Friends, Statuses, Users {
+
+ private static final Logger LOGGER =
LoggerFactory.getLogger(Twitter.class);
+
+ private static Map<TwitterConfiguration, Twitter> INSTANCE_MAP = new
ConcurrentHashMap<>();
+
+ private TwitterConfiguration configuration;
+
+ private ObjectMapper mapper;
+
+ private String rootUrl;
+
+ private CloseableHttpClient httpclient;
+
+ private HttpRequestInterceptor oauthInterceptor;
+
+ RestClient restClient;
+
+ private Twitter(TwitterConfiguration configuration) throws
InstantiationException {
+this.configuration = configuration;
+this.rootUrl = TwitterProviderUtil.baseUrl(configuration);
+this.oauthInterceptor = new
TwitterOAuthRequestInterceptor(configuration.getOauth());
+this.httpclient = HttpClientBuilder.create()
+.addInterceptorFirst(oauthInterceptor)
+.setDefaultRequestConfig(RequestConfig.custom()
+ .setConnectionRequestTimeout(5000)
+ .setConnectTimeout(5000)
+ .setSocketTimeout(5000)
+ .setCookieSpec("easy")
+ .build()
+)
+.setMaxConnPerRoute(20)
+.setMaxConnTotal(100)
+.build();
+
+// TODO: juneau-6.2.0-incubating
+// this.restClient = new RestClientBuilder()
+//.httpClient(httpclient, true)
+//.parser(JsonParser.class)
+//.rootUrl(rootUrl)
+//.retryable(
+//configuration.getRetryMax().intValue(),
+//configuration.getRetrySleepMs(),
+//new TwitterRetryHandler())
+//.build();
+this.restClient = new RestClient()
+.setHttpClient(httpclient)
+.setParser(JsonParser.class)
+.setRootUrl(rootUrl);
+
+this.mapper = StreamsJacksonMapper.getInsta
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111309069
--- Diff:
streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Twitter.java
---
@@ -0,0 +1,521 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.jackson.StreamsJacksonMapper;
+import org.apache.streams.twitter.TwitterConfiguration;
+import org.apache.streams.twitter.pojo.Tweet;
+import org.apache.streams.twitter.pojo.User;
+import org.apache.streams.twitter.provider.TwitterProviderUtil;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.HttpRequestInterceptor;
+import org.apache.http.client.HttpRequestRetryHandler;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.params.ClientPNames;
+import org.apache.http.client.params.CookiePolicy;
+import org.apache.http.client.utils.URIBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.juneau.json.JsonParser;
+import org.apache.juneau.parser.ParseException;
+import org.apache.juneau.plaintext.PlainTextSerializer;
+import org.apache.juneau.rest.client.RestCall;
+import org.apache.juneau.rest.client.RestCallException;
+import org.apache.juneau.rest.client.RestClient;
+//import org.apache.juneau.rest.client.RestClientBuilder;
+import org.apache.juneau.rest.client.RetryOn;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URISyntaxException;
+import java.util.ArrayList;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.ConcurrentHashMap;
+
+/**
+ * Implementation of all twitter interfaces using juneau.
+ */
+public class Twitter implements Followers, Friends, Statuses, Users {
+
+ private static final Logger LOGGER =
LoggerFactory.getLogger(Twitter.class);
+
+ private static Map<TwitterConfiguration, Twitter> INSTANCE_MAP = new
ConcurrentHashMap<>();
+
+ private TwitterConfiguration configuration;
+
+ private ObjectMapper mapper;
+
+ private String rootUrl;
+
+ private CloseableHttpClient httpclient;
+
+ private HttpRequestInterceptor oauthInterceptor;
+
+ RestClient restClient;
+
+ private Twitter(TwitterConfiguration configuration) throws
InstantiationException {
+this.configuration = configuration;
+this.rootUrl = TwitterProviderUtil.baseUrl(configuration);
+this.oauthInterceptor = new
TwitterOAuthRequestInterceptor(configuration.getOauth());
+this.httpclient = HttpClientBuilder.create()
+.addInterceptorFirst(oauthInterceptor)
+.setDefaultRequestConfig(RequestConfig.custom()
+ .setConnectionRequestTimeout(5000)
+ .setConnectTimeout(5000)
+ .setSocketTimeout(5000)
+ .setCookieSpec("easy")
+ .build()
+)
+.setMaxConnPerRoute(20)
+.setMaxConnTotal(100)
+.build();
+
+// TODO: juneau-6.2.0-incubating
+// this.restClient = new RestClientBuilder()
+//.httpClient(httpclient, true)
+//.parser(JsonParser.class)
+//.rootUrl(rootUrl)
+//.retryable(
+//configuration.getRetryMax().intValue(),
+//configuration.getRetrySleepMs(),
+//new TwitterRetryHandler())
+//.build();
+this.restClient = new RestClient()
+.setHttpClient(httpclient)
+.setParser(JsonParser.class)
+.setRootUrl(rootUrl);
+
+this.mapper = StreamsJacksonMapper.getInsta
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Github user steveblackmon commented on a diff in the pull request: https://github.com/apache/incubator-streams/pull/360#discussion_r111306857 --- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Followers.java --- @@ -0,0 +1,52 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.streams.twitter.api; + +import org.apache.streams.twitter.pojo.Tweet; + +import java.util.List; + +/** + * Returns a collection of the most recent Tweets posted by the user indicated by the screen_name or user_id parameters. --- End diff -- edited all interface headers for accuracy --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Github user steveblackmon commented on a diff in the pull request: https://github.com/apache/incubator-streams/pull/360#discussion_r111306855 --- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Friends.java --- @@ -0,0 +1,52 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.streams.twitter.api; + +import org.apache.streams.twitter.pojo.Tweet; + +import java.util.List; + +/** + * Returns a collection of the most recent Tweets posted by the user indicated by the screen_name or user_id parameters. --- End diff -- edited all interface headers for accuracy --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111306821
--- Diff:
streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Followers.java
---
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.twitter.pojo.Tweet;
+
+import java.util.List;
+
+/**
+ * Returns a collection of the most recent Tweets posted by the user
indicated by the screen_name or user_id parameters.
+ *
+ * @see https://dev.twitter.com/rest/reference/get/statuses/user_timeline;>https://api.twitter.com/1.1/statuses/user_timeline.json
+ */
+public interface Followers {
+
+ /**
+ * Returns a cursored collection of user IDs for every user following
the specified user.
+ *
+ * @param parameters StatusesLookupRequest
+ * @return List < Tweet >
+ * @see https://api.twitter.com/1.1/followers/ids.json;>https://api.twitter.com/1.1/followers/ids.json
--- End diff --
good point, just made this change across all interfaces
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #361: STREAMS-500: Remove jackson-datatype-js...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/361 STREAMS-500: Remove jackson-datatype-json-org from project Remove jackson-datatype-json-org from project You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-500 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/361.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #361 commit a460c8e4b0bce3d3acea38829593301108f586a8 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-03T21:03:53Z STREAMS-500: Remove jackson-datatype-json-org from project Remove jackson-datatype-json-org from project --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/360 STREAMS-496: Remove twitter4j dependency from streams-provider-twitter You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-496 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/360.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #360 commit 7dedbb42251e1c5214690e32f12a5efb5032dc92 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-03-20T06:35:33Z STREAMS-496: WIP commit a8fd796096296bb2e8af61b722c4a261acbba987 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-03-20T18:07:27Z more WIP commit 68d7976f928173dbd6de636c58d1a23dbe95eb15 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-03-24T18:39:52Z STREAMS-496: additional pojos and interfaces needed to maintain features commit c95bccd41b4f5f51d7e614e5927210458d1dbfbd Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-03-24T19:21:34Z implementations using RestClient commit 78f30284caefb09e06818a9f435c3f380ee986a3 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-03-25T16:38:06Z STREAMS-496: WIP - twitter4j gone, src/main/java compiles commit 0d172e76232e3d0bd48af8d6616716c444eb94f0 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-03-26T17:56:11Z STREAMS-496: consolidate httpcomponents version selection commit 434657439abc0dde2998a8cb213d8e8398f43ad6 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-03-26T17:56:34Z first implementation of oauth signing commit 1885090c6d683c8a9a6df7c64f933529378ee965 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-03-26T18:09:24Z cleanup authentication. debugging âBad Authentication data.â commit edc2662fd68b9e1e04c20b8721a44a6ee9fb36e6 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-03-26T21:20:10Z adding unit testing to debug authentication commit 5a07c57b553f8c87af1d42e0bbef4f0795f17bf7 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-03-26T22:44:37Z STREAMS-496: more unit tests, all passing, ITs still fail commit c623f7139421049d5b536576c739b60e959cf11e Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-03-26T22:55:09Z STREAMS-496: going through guide word-by-word and debugging line-by-line looking for mistakes commit eddf003ba5b5555b10c3d51267c00225b6da4ef1 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-03-27T03:46:19Z TwitterOAuthRequestInterceptorTest.testProcess TwitterOAuthRequestInterceptorTest.testProcess is pretty much directly off the twitter dev guidelines and tests the entire auth process at once. passes. ITs still fail commit 06845f2dd848c91971c10f194f0d5503d5f4a964 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-03-28T18:20:03Z return empty array not null on error commit 517b85f02e93c51d5fa36a704f7f0e0e2cdaed4e Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-03T16:01:25Z oauth signing working commit 069be4da745b9cd262159cca1982a9b52cb5cefc Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-03T19:56:01Z all integration tests passing using juneau 6.1.0-incubating commit d7cd07110d602d24a07aad8e8f390ff0d98f1e19 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-04-03T20:11:32Z checkstyle clean up --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #358: STREAMS-499: fix behavior of site plugi...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/358 STREAMS-499: fix behavior of site plugin You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-499 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/358.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #358 commit c8638ab0808b25403b1a87f8db03c84c1c09ee03 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-03-19T23:03:08Z STREAMS-499: fix behavior of site plugin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #357: STREAMS-495: delete unneeded, sometimes...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/357 STREAMS-495: delete unneeded, sometimes wrong streams-plugin-pojo bindings You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-495 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/357.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #357 commit 80070a9c967260b758f43d1d0fa071cf07d06f26 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-03-02T16:04:06Z STREAMS-495: delete unneeded, sometimes wrong streams-plugin-pojo bindings --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #356: STREAMS-463: Move every class in all repos und...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/356 ð this should improve checkstyle score considerably --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #355: STREAMS-438: Create coding guidelines document...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/355 :+1: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #354: STREAMS-448: TwitterTimelineProvider bu...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/354 STREAMS-448: TwitterTimelineProvider bug fix resolves STREAMS-448 You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-448 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/354.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #354 commit 5649ff6ff0fe7253f9a3e7ffb076b6b3be51c1c1 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-01-27T22:58:48Z STREAMS-448: TwitterTimelineProvider bug fix resolves STREAMS-448 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #353: STREAMS-483: add support for SSL connec...
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/353#discussion_r98104694
--- Diff:
streams-contrib/streams-persist-cassandra/src/main/java/org/apache/streams/cassandra/CassandraClient.java
---
@@ -0,0 +1,160 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.cassandra;
+
+import com.datastax.driver.core.Cluster;
+import com.datastax.driver.core.Host;
+import com.datastax.driver.core.JdkSSLOptions;
+import com.datastax.driver.core.Metadata;
+import com.datastax.driver.core.SSLOptions;
+import com.datastax.driver.core.Session;
+import com.datastax.driver.core.SocketOptions;
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.Lists;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileInputStream;
+import java.io.InputStream;
+import java.net.InetSocketAddress;
+import java.security.KeyStore;
+import java.util.Collection;
+import javax.net.ssl.KeyManagerFactory;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.TrustManagerFactory;
+
+import static
com.datastax.driver.core.SocketOptions.DEFAULT_CONNECT_TIMEOUT_MILLIS;
+import static
com.datastax.driver.core.SocketOptions.DEFAULT_READ_TIMEOUT_MILLIS;
+
+public class CassandraClient {
+
+ private static final Logger LOGGER = LoggerFactory
+ .getLogger(CassandraClient.class);
+
+ private Cluster cluster;
+ private Session session;
+
+ public CassandraConfiguration config;
+
+ public CassandraClient(CassandraConfiguration config) throws Exception {
+this.config = config;
+org.apache.cassandra.config.Config.setClientMode(true);
+ }
+
+ public void start() throws Exception {
+
+Preconditions.checkNotNull(config);
+
+LOGGER.info("CassandraClient.start {}", config);
+
+Cluster.Builder builder = Cluster.builder()
+.withPort(config.getPort().intValue())
+.withoutJMXReporting()
+.withoutMetrics()
+.withSocketOptions(
+new SocketOptions()
+.setConnectTimeoutMillis(DEFAULT_CONNECT_TIMEOUT_MILLIS*10)
+.setReadTimeoutMillis(DEFAULT_READ_TIMEOUT_MILLIS*10)
+);
+
+if( config.getSsl() != null && config.getSsl().getEnabled() == true) {
+
+ Ssl ssl = config.getSsl();
+
+ KeyStore ks = KeyStore.getInstance("JKS");
+
+ InputStream trustStore = new FileInputStream(ssl.getTrustStore());
+ ks.load(trustStore, ssl.getTrustStorePassword().toCharArray());
+ InputStream keyStore = new FileInputStream(ssl.getKeyStore());
+ ks.load(keyStore, ssl.getKeyStorePassword().toCharArray());
+
+ TrustManagerFactory tmf =
TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
+ tmf.init(ks);
+
+ KeyManagerFactory kmf =
KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
+ kmf.init(ks, ssl.getKeyStorePassword().toCharArray());
+
+ SSLContext sslContext = SSLContext.getInstance("SSLv3");
+ sslContext.init(kmf.getKeyManagers(), tmf.getTrustManagers(), null);
+
+ SSLOptions sslOptions = JdkSSLOptions.builder()
+ .withSSLContext(sslContext)
+ .build();
+
+ builder = builder.withSSL(sslOptions);
+}
+
+Collection addresses = Lists.newArrayList();
+for (String h : config.getHosts()) {
+ LOGGER.info("Adding Host: {}", h);
+ InetSocketAddress socketAddress = new InetSocketAddress(h,
config.getPort().intValue());
+ addresses.add(socketAddress);
+}
+builder.addContactPointsWithPorts(addresses);
+
+if( !Strings.isNullOrEmpty(config.getUser()) &&
+!Strings.isNullOrEmpt
[GitHub] incubator-streams pull request #353: STREAMS-483: add support for SSL connec...
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/353#discussion_r98104675
--- Diff:
streams-contrib/streams-persist-cassandra/src/main/java/org/apache/streams/cassandra/CassandraClient.java
---
@@ -0,0 +1,160 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.cassandra;
+
+import com.datastax.driver.core.Cluster;
+import com.datastax.driver.core.Host;
+import com.datastax.driver.core.JdkSSLOptions;
+import com.datastax.driver.core.Metadata;
+import com.datastax.driver.core.SSLOptions;
+import com.datastax.driver.core.Session;
+import com.datastax.driver.core.SocketOptions;
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.Lists;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileInputStream;
+import java.io.InputStream;
+import java.net.InetSocketAddress;
+import java.security.KeyStore;
+import java.util.Collection;
+import javax.net.ssl.KeyManagerFactory;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.TrustManagerFactory;
+
+import static
com.datastax.driver.core.SocketOptions.DEFAULT_CONNECT_TIMEOUT_MILLIS;
+import static
com.datastax.driver.core.SocketOptions.DEFAULT_READ_TIMEOUT_MILLIS;
+
+public class CassandraClient {
+
+ private static final Logger LOGGER = LoggerFactory
+ .getLogger(CassandraClient.class);
+
+ private Cluster cluster;
+ private Session session;
+
+ public CassandraConfiguration config;
+
+ public CassandraClient(CassandraConfiguration config) throws Exception {
+this.config = config;
+org.apache.cassandra.config.Config.setClientMode(true);
+ }
+
+ public void start() throws Exception {
+
+Preconditions.checkNotNull(config);
+
+LOGGER.info("CassandraClient.start {}", config);
+
+Cluster.Builder builder = Cluster.builder()
+.withPort(config.getPort().intValue())
+.withoutJMXReporting()
+.withoutMetrics()
+.withSocketOptions(
+new SocketOptions()
+.setConnectTimeoutMillis(DEFAULT_CONNECT_TIMEOUT_MILLIS*10)
+.setReadTimeoutMillis(DEFAULT_READ_TIMEOUT_MILLIS*10)
+);
+
+if( config.getSsl() != null && config.getSsl().getEnabled() == true) {
+
+ Ssl ssl = config.getSsl();
+
+ KeyStore ks = KeyStore.getInstance("JKS");
+
+ InputStream trustStore = new FileInputStream(ssl.getTrustStore());
+ ks.load(trustStore, ssl.getTrustStorePassword().toCharArray());
+ InputStream keyStore = new FileInputStream(ssl.getKeyStore());
+ ks.load(keyStore, ssl.getKeyStorePassword().toCharArray());
+
+ TrustManagerFactory tmf =
TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
+ tmf.init(ks);
+
+ KeyManagerFactory kmf =
KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
+ kmf.init(ks, ssl.getKeyStorePassword().toCharArray());
+
+ SSLContext sslContext = SSLContext.getInstance("SSLv3");
+ sslContext.init(kmf.getKeyManagers(), tmf.getTrustManagers(), null);
+
+ SSLOptions sslOptions = JdkSSLOptions.builder()
+ .withSSLContext(sslContext)
+ .build();
+
+ builder = builder.withSSL(sslOptions);
+}
+
+Collection addresses = Lists.newArrayList();
--- End diff --
i'll change it, to make it easier to remove once we have a substitute for
Queues
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #353: STREAMS-483: add support for SSL connec...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/353 STREAMS-483: add support for SSL connections to streams-persist-cassandra refactor client object to be shared by reader and writer also ensures the ITs can run & pass repeatedly You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-483 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/353.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #353 commit cc80a98947bbde19183924c3b98686cd291cdc1b Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-01-26T00:12:42Z STREAMS-483: add support for SSL connections to streams-persist-cassandra also ensures the ITs can run & pass repeatedly --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #352: STREAMS-482: streams-persist-riak
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/352 STREAMS-482: streams-persist-riak You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-482 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/352.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #352 commit 6c87bef3beb17b276d4b8455d7c4010338edd7be Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-01-24T16:50:10Z basic implementation of streams-persist-riak commit 59f20bfd9f2bb88c51e86c02adec669f61fcddbb Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-01-24T21:35:25Z license headers commit 73b4a17eab6cecd96ec9bd63e80cac7b39058c28 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-01-24T21:40:13Z headers and indentation commit add5bfad2db50b5fd6054421348daf4e1ac120ad Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2017-01-24T21:41:49Z imports --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #351: STREAMS-481 Updated .travis.yml to oraclejdk8,...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/351 :+1: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #348: STREAMS-344: streams-persist-neo4j
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/348#discussion_r95085434
--- Diff:
streams-contrib/streams-persist-neo4j/src/main/java/org/apache/streams/neo4j/http/Neo4jHttpClient.java
---
@@ -0,0 +1,84 @@
+package org.apache.streams.neo4j.http;
+
+import org.apache.streams.neo4j.Neo4jConfiguration;
+
+import com.google.common.base.Preconditions;
+
+import org.apache.http.client.HttpClient;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+public class Neo4jHttpClient {
+
+private static final Logger LOGGER = LoggerFactory
+.getLogger(Neo4jHttpClient.class);
+
+public Neo4jConfiguration config;
+
+private HttpClient client;
+
+private Neo4jHttpClient(Neo4jConfiguration neo4jConfiguration) {
+this.config = neo4jConfiguration;
+try {
+this.start();
+} catch (Exception e) {
+e.printStackTrace();
+this.client = null;
+}
+}
+
+private static Map<Neo4jConfiguration, Neo4jHttpClient> INSTANCE_MAP =
new ConcurrentHashMap<Neo4jConfiguration, Neo4jHttpClient>();
+
+public static Neo4jHttpClient getInstance(Neo4jConfiguration
neo4jConfiguration) {
+if (INSTANCE_MAP != null &&
+INSTANCE_MAP.size() > 0 &&
+INSTANCE_MAP.containsKey(neo4jConfiguration)
+)
+return INSTANCE_MAP.get(neo4jConfiguration);
+else {
+Neo4jHttpClient instance = new
Neo4jHttpClient(neo4jConfiguration);
+if( instance != null && instance.client != null ) {
+INSTANCE_MAP.put(neo4jConfiguration, instance);
+return instance;
+} else {
+return null;
+}
+}
+}
+
+public void start() throws Exception {
+
+Preconditions.checkNotNull(config);
+Preconditions.checkArgument(
+config.getScheme().startsWith("http")
+);
+
+LOGGER.info("Neo4jConfiguration.start {}", config);
+
+Preconditions.checkNotNull(client);
--- End diff --
âï¸
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #348: STREAMS-344: streams-persist-neo4j
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/348#discussion_r95085431
--- Diff:
streams-contrib/streams-persist-neo4j/src/main/java/org/apache/streams/neo4j/bolt/Neo4jBoltClient.java
---
@@ -0,0 +1,100 @@
+package org.apache.streams.neo4j.bolt;
+
+import org.apache.streams.neo4j.Neo4jConfiguration;
+
+import com.google.common.base.Preconditions;
+
+import org.apache.commons.lang3.StringUtils;
+import org.neo4j.driver.v1.AuthToken;
+import org.neo4j.driver.v1.AuthTokens;
+import org.neo4j.driver.v1.Driver;
+import org.neo4j.driver.v1.GraphDatabase;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+public class Neo4jBoltClient {
+
+private static final Logger LOGGER = LoggerFactory
+.getLogger(Neo4jBoltClient.class);
+
+private Driver client;
+
+public Neo4jConfiguration config;
+
+private Neo4jBoltClient(Neo4jConfiguration neo4jConfiguration) {
+this.config = neo4jConfiguration;
+try {
+this.start();
+} catch (Exception e) {
+e.printStackTrace();
+this.client = null;
+}
+}
+
+private static Map<Neo4jConfiguration, Neo4jBoltClient> INSTANCE_MAP =
new ConcurrentHashMap<Neo4jConfiguration, Neo4jBoltClient>();
+
+public static Neo4jBoltClient getInstance(Neo4jConfiguration
neo4jConfiguration) {
+if (INSTANCE_MAP != null &&
--- End diff --
âï¸
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #348: STREAMS-344: streams-persist-neo4j
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/348#discussion_r95085437
--- Diff:
streams-util/src/main/java/org/apache/streams/util/PropertyUtil.java ---
@@ -98,22 +102,24 @@ public static ObjectNode
unflattenObjectNode(ObjectNode flatObject, char seperat
while (iter.hasNext()) {
Map.Entry<String, JsonNode> item = iter.next();
String fullKey = item.getKey();
- if ( !fullKey.contains(Character.valueOf(seperator).toString())) {
+ if( !fullKey.contains(Character.valueOf(seperator).toString())) {
root.put(item.getKey(), item.getValue());
} else {
ObjectNode currentNode = root;
-List keyParts = new ArrayList<>();
+List keyParts = Lists.newArrayList();
--- End diff --
âï¸ removed guava from this class
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #348: STREAMS-344: streams-persist-neo4j
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/348#discussion_r95085386
--- Diff:
streams-contrib/streams-persist-neo4j/src/main/java/org/apache/streams/neo4j/Neo4jPersistUtil.java
---
@@ -0,0 +1,151 @@
+package org.apache.streams.neo4j;
+
+import org.apache.streams.core.StreamsDatum;
+import org.apache.streams.jackson.StreamsJacksonMapper;
+import org.apache.streams.neo4j.bolt.Neo4jBoltPersistWriter;
+import org.apache.streams.pojo.json.Activity;
+import org.apache.streams.pojo.json.ActivityObject;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+
+import org.apache.commons.lang3.StringUtils;
+import org.javatuples.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * Created by steve on 1/2/17.
+ */
+public class Neo4jPersistUtil {
+
+ private static final Logger LOGGER =
LoggerFactory.getLogger(Neo4jBoltPersistWriter.class);
+
+ private static ObjectMapper mapper = StreamsJacksonMapper.getInstance();
+
+ private static CypherQueryGraphHelper helper = new
CypherQueryGraphHelper();
+
+ public static List<Pair<String, Map<String, Object>>>
prepareStatements(StreamsDatum entry) throws Exception {
+
+List<Pair<String, Map<String, Object>>> statements = new ArrayList<>();
+
+String id = entry.getId();
+Activity activity = null;
+ActivityObject activityObject = null;
+Object document = entry.getDocument();
+
+if (document instanceof Activity) {
+ activity = (Activity) document;
+} else if (document instanceof ActivityObject) {
+ activityObject = (ActivityObject) document;
+} else {
+ ObjectNode objectNode;
+ if (document instanceof ObjectNode) {
+objectNode = (ObjectNode) document;
+ } else if ( document instanceof String) {
+try {
+ objectNode = mapper.readValue((String) document,
ObjectNode.class);
+} catch (IOException ex) {
+ LOGGER.error("Can't handle input: ", entry);
+ throw ex;
+}
+ } else {
+LOGGER.error("Can't handle input: ", entry);
+throw new Exception("Can't create statements from datum.");
+ }
+
+ if ( objectNode.get("verb") != null ) {
+try {
+ activity = mapper.convertValue(objectNode, Activity.class);
+ activityObject = activity.getObject();
+} catch (Exception ex) {
+ activityObject = mapper.convertValue(objectNode,
ActivityObject.class);
+}
+ } else {
+activityObject = mapper.convertValue(objectNode,
ActivityObject.class);
+ }
+
+}
+
+Preconditions.checkArgument(activity != null ^ activityObject != null);
+
+if ( activityObject != null &&
!Strings.isNullOrEmpty(activityObject.getId())) {
+
+ statements.add(vertexStatement(activityObject));
+
+} else if ( activity != null &&
!Strings.isNullOrEmpty(activity.getId())) {
+
+ statements.addAll(vertexStatements(activity));
+
+ statements.addAll(edgeStatements(activity));
+
+}
+
+return statements;
+ }
+
+ public static List<Pair<String, Map<String, Object>>>
vertexStatements(Activity activity) {
+List<Pair<String, Map<String, Object>>> statements = new
ArrayList<>();;
+ActivityObject actor = activity.getActor();
+ActivityObject object = activity.getObject();
+ActivityObject target = activity.getTarget();
+
+if (actor != null && StringUtils.isNotBlank(actor.getId())) {
+ Pair<String, Map<String, Object>> actorStatement =
vertexStatement(actor);
+ statements.add(actorStatement);
+}
+
+if (object != null && StringUtils.isNotBlank(object.getId())) {
+ Pair<String, Map<String, Object>> objectStatement =
vertexStatement(object);
+ statements.add(objectStatement);
+}
+
+if (target != null && StringUtils.isNotBlank(target.getId())) {
+ Pair<String, Map<String, Object>> targetStatement =
vertexStatement(target);
+ statements.add(targetStatement);
+}
+
+return s
[GitHub] incubator-streams pull request #348: STREAMS-344: streams-persist-neo4j
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/348#discussion_r95085389
--- Diff:
streams-contrib/streams-persist-neo4j/src/main/java/org/apache/streams/neo4j/Neo4jPersistUtil.java
---
@@ -0,0 +1,151 @@
+package org.apache.streams.neo4j;
+
+import org.apache.streams.core.StreamsDatum;
+import org.apache.streams.jackson.StreamsJacksonMapper;
+import org.apache.streams.neo4j.bolt.Neo4jBoltPersistWriter;
+import org.apache.streams.pojo.json.Activity;
+import org.apache.streams.pojo.json.ActivityObject;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+
+import org.apache.commons.lang3.StringUtils;
+import org.javatuples.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * Created by steve on 1/2/17.
+ */
+public class Neo4jPersistUtil {
+
+ private static final Logger LOGGER =
LoggerFactory.getLogger(Neo4jBoltPersistWriter.class);
+
+ private static ObjectMapper mapper = StreamsJacksonMapper.getInstance();
+
+ private static CypherQueryGraphHelper helper = new
CypherQueryGraphHelper();
+
+ public static List<Pair<String, Map<String, Object>>>
prepareStatements(StreamsDatum entry) throws Exception {
+
+List<Pair<String, Map<String, Object>>> statements = new ArrayList<>();
+
+String id = entry.getId();
+Activity activity = null;
+ActivityObject activityObject = null;
+Object document = entry.getDocument();
+
+if (document instanceof Activity) {
+ activity = (Activity) document;
+} else if (document instanceof ActivityObject) {
+ activityObject = (ActivityObject) document;
+} else {
+ ObjectNode objectNode;
+ if (document instanceof ObjectNode) {
+objectNode = (ObjectNode) document;
+ } else if ( document instanceof String) {
+try {
+ objectNode = mapper.readValue((String) document,
ObjectNode.class);
+} catch (IOException ex) {
+ LOGGER.error("Can't handle input: ", entry);
+ throw ex;
+}
+ } else {
+LOGGER.error("Can't handle input: ", entry);
+throw new Exception("Can't create statements from datum.");
+ }
+
+ if ( objectNode.get("verb") != null ) {
+try {
+ activity = mapper.convertValue(objectNode, Activity.class);
+ activityObject = activity.getObject();
+} catch (Exception ex) {
+ activityObject = mapper.convertValue(objectNode,
ActivityObject.class);
+}
+ } else {
+activityObject = mapper.convertValue(objectNode,
ActivityObject.class);
+ }
+
+}
+
+Preconditions.checkArgument(activity != null ^ activityObject != null);
+
+if ( activityObject != null &&
!Strings.isNullOrEmpty(activityObject.getId())) {
+
+ statements.add(vertexStatement(activityObject));
+
+} else if ( activity != null &&
!Strings.isNullOrEmpty(activity.getId())) {
+
+ statements.addAll(vertexStatements(activity));
+
+ statements.addAll(edgeStatements(activity));
+
+}
+
+return statements;
+ }
+
+ public static List<Pair<String, Map<String, Object>>>
vertexStatements(Activity activity) {
+List<Pair<String, Map<String, Object>>> statements = new
ArrayList<>();;
+ActivityObject actor = activity.getActor();
+ActivityObject object = activity.getObject();
+ActivityObject target = activity.getTarget();
+
+if (actor != null && StringUtils.isNotBlank(actor.getId())) {
+ Pair<String, Map<String, Object>> actorStatement =
vertexStatement(actor);
+ statements.add(actorStatement);
+}
+
+if (object != null && StringUtils.isNotBlank(object.getId())) {
+ Pair<String, Map<String, Object>> objectStatement =
vertexStatement(object);
+ statements.add(objectStatement);
+}
+
+if (target != null && StringUtils.isNotBlank(target.getId())) {
+ Pair<String, Map<String, Object>> targetStatement =
vertexStatement(target);
+ statements.add(targetStatement);
+}
+
+return s
[GitHub] incubator-streams pull request #350: STREAMS-255: remove incubator-streams d...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/350 STREAMS-255: remove incubator-streams dependency on incubator-streams-master also reorganizes website hierarchy some-what You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-255 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/350.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #350 commit b3604a6ed808dc04490f098c6b69c31369a95ca3 Author: Steve Blackmon <sblack...@apache.org> Date: 2017-01-07T22:55:38Z STREAMS-255: remove incubator-streams dependency on incubator-streams-master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #349: STREAMS-472: fixes root cause
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/349 STREAMS-472: fixes root cause You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-472.2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/349.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #349 commit 16edd38dfec66566969409c0ad9fc00a9bfeccd8 Author: Steve Blackmon <sblack...@apache.org> Date: 2017-01-07T20:00:50Z STREAMS-472: fixes root cause --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #348: STREAMS-344: streams-persist-neo4j
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/348 STREAMS-344: streams-persist-neo4j now that neo4j-java-driver is apache licensed, we can support binding to the binary bolt endpoint to write and read streams data. everything related to neo4j is now in streams-persist-neo4j, leaving only interfaces in streams-persist-graph for now. unlike most other persisters, this module expects to find a valid activity or activityobject in the datum, not just any document, but then it is able to insert activityobjects as vertices and activities as edges connecting activityobjects. You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-344 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/348.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #348 commit 1f4e175cf84a208252d488c2858ea420af0642f9 Author: Steve Blackmon <sblack...@apache.org> Date: 2017-01-03T00:11:01Z new neo4j module with bolt:// and http:// support, and tests commit ee700fd16e8631bdb0fb453d686beef4167af13b Author: Steve Blackmon <sblack...@apache.org> Date: 2017-01-03T01:42:33Z add constructor --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #346: Replace Guava API calls with Java 8 API
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/346#discussion_r94513265
--- Diff:
streams-contrib/streams-provider-facebook/src/main/java/org/apache/streams/facebook/provider/FacebookUserstreamProvider.java
---
@@ -241,7 +240,7 @@ void shutdownAndAwaitTermination(ExecutorService pool) {
@Override
public void prepare(Object configurationObject) {
-executor =
MoreExecutors.listeningDecorator(newFixedThreadPoolWithQueueSize(5, 20));
+executor = newFixedThreadPoolWithQueueSize(5, 20);
--- End diff --
I don't think we're going to be able to do this everywhere, but apparently
it's ok in this provider if there's no compile error
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #347: new neo4j module with bolt:// and http:...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/347 new neo4j module with bolt:// and http:// support, and tests now that neo4j-java-driver is apache licensed, we can support binding to the binary bolt endpoint to write and read streams. everything related to neo4j is now in streams-persist-neo4j, leaving only interfaces in streams-persist-graph for now. unlike most other persisters, this module expects to find a valid activity or activityobject in the datum, not just any document, but then it is able to insert activityobjects as vertices and activities as edges connecting activityobjects. for this reason the counts expected in the tests aren't exactly what might be expected. all in all an improvement in capability, flexibility, and performance over the prior streams-persist-graph which had no integration tests. You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-344 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/347.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #347 commit 1f4e175cf84a208252d488c2858ea420af0642f9 Author: Steve Blackmon <sblack...@apache.org> Date: 2017-01-03T00:11:01Z new neo4j module with bolt:// and http:// support, and tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #345: [STREAMS-478] streams persist reader/writer fo...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/345 ð --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #345: [STREAMS-478] streams persist reader/wr...
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/345#discussion_r94047751
--- Diff:
streams-contrib/streams-persist-cassandra/src/main/jsonschema/org/apache/streams/cassandra/CassandraConfiguration.json
---
@@ -0,0 +1,44 @@
+{
+ "$schema": "http://json-schema.org/draft-03/schema;,
+ "$license": [
+"http://www.apache.org/licenses/LICENSE-2.0;
+ ],
+ "id": "#",
+ "type": "object",
+ "javaType": "org.apache.streams.cassandra.CassandraConfiguration",
+ "javaInterfaces": ["java.io.Serializable"],
+ "properties": {
+"host": {
+ "type": "string",
--- End diff --
perhaps this should be an array of strings, so module can start-up with
more than one seed host.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #345: [STREAMS-478] streams persist reader/wr...
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/345#discussion_r94047563
--- Diff:
streams-contrib/streams-persist-cassandra/src/main/java/org/apache/streams/cassandra/CassandraPersistWriter.java
---
@@ -0,0 +1,312 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.cassandra;
+
+import org.apache.streams.config.ComponentConfigurator;
+import org.apache.streams.config.StreamsConfigurator;
+import org.apache.streams.core.StreamsDatum;
+import org.apache.streams.core.StreamsPersistWriter;
+import org.apache.streams.jackson.StreamsJacksonMapper;
+import org.apache.streams.util.GuidUtils;
+
+import com.datastax.driver.core.BatchStatement;
+import com.datastax.driver.core.BoundStatement;
+import com.datastax.driver.core.Cluster;
+import com.datastax.driver.core.DataType;
+import com.datastax.driver.core.KeyspaceMetadata;
+import com.datastax.driver.core.Metadata;
+import com.datastax.driver.core.PreparedStatement;
+import com.datastax.driver.core.Session;
+import com.datastax.driver.core.TableMetadata;
+import com.datastax.driver.core.querybuilder.Insert;
+import com.datastax.driver.core.querybuilder.QueryBuilder;
+import com.datastax.driver.core.schemabuilder.SchemaBuilder;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import com.google.common.base.Strings;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.Closeable;
+import java.io.Flushable;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Queue;
+import java.util.Random;
+import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.locks.ReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+public class CassandraPersistWriter implements StreamsPersistWriter,
Runnable, Flushable, Closeable {
+
+ public static final String STREAMS_ID = "CassandraPersistWriter";
+
+ private static final Logger LOGGER =
LoggerFactory.getLogger(CassandraPersistWriter.class);
+
+ private static final long MAX_WRITE_LATENCY = 1000;
+
+ protected volatile Queue persistQueue;
+
+ private ObjectMapper mapper = StreamsJacksonMapper.getInstance();
+ private volatile AtomicLong lastWrite = new
AtomicLong(System.currentTimeMillis());
+ private ScheduledExecutorService backgroundFlushTask =
Executors.newSingleThreadScheduledExecutor();
+
+ private CassandraConfiguration config;
+
+ protected Cluster cluster;
+ protected Session session;
+
+ protected String keyspace;
+ protected String table;
+ protected PreparedStatement insertStatement;
+
+ protected List insertBatch = new ArrayList<>();
+
+ protected final ReadWriteLock lock = new ReentrantReadWriteLock();
+
+ public CassandraPersistWriter() {
+this(new ComponentConfigurator<>(CassandraConfiguration.class)
+
.detectConfiguration(StreamsConfigurator.getConfig().getConfig("cassandra")));
+ }
+
+ public CassandraPersistWriter(CassandraConfiguration config) {
+this.config = config;
+ }
+
+ public void setPersistQueue(Queue persistQueue) {
+this.persistQueue = persistQueue;
+ }
+
+ public Queue getPersistQueue() {
+return persistQueue;
+ }
+
+ @Override
+ public String getId() {
+return STREAMS_ID;
+ }
+
+ @Override
+ public void write(StreamsDatum streamsDatum) {
+
+ObjectNode node;
[GitHub] incubator-streams pull request #340: streams-provider-twitter bug fixes & re...
Github user steveblackmon closed the pull request at: https://github.com/apache/incubator-streams/pull/340 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #341: Replace Guava API calls with Java 8 API
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/341 ð --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #335: refactor to use modern apache hbase ins...
Github user steveblackmon closed the pull request at: https://github.com/apache/incubator-streams/pull/335 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #335: refactor to use modern apache hbase ins...
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/335#discussion_r92702137
--- Diff:
streams-contrib/streams-persist-hbase/src/test/java/org/apache/streams/hbase/test/HbasePersistWriterIT.java
---
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.hbase.test;
+
+import org.apache.streams.config.ComponentConfigurator;
+import org.apache.streams.core.StreamsDatum;
+import org.apache.streams.hbase.HbaseConfiguration;
+import org.apache.streams.hbase.HbasePersistWriter;
+import org.apache.streams.jackson.StreamsJacksonMapper;
+import org.apache.streams.pojo.json.Activity;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.typesafe.config.Config;
+import com.typesafe.config.ConfigFactory;
+import com.typesafe.config.ConfigParseOptions;
+import org.apache.commons.io.Charsets;
+import org.apache.commons.io.IOUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+import java.io.File;
+import java.io.InputStream;
+import java.util.List;
+
+/**
+ * Test HbasePersistWriterIT.
+ */
+@Test
+(
+groups={"HbasePersistWriterIT"}
+)
+public class HbasePersistWriterIT {
+
+ private final static Logger LOGGER =
LoggerFactory.getLogger(HbasePersistWriterIT.class);
+
+ private static ObjectMapper MAPPER = StreamsJacksonMapper.getInstance();
+
+ protected HbaseConfiguration testConfiguration;
+
+ @BeforeClass
+ public void prepareTest() throws Exception {
+
+Config reference = ConfigFactory.load();
+File conf_file = new
File("target/test-classes/HbasePersistWriterIT.conf");
+assert(conf_file.exists());
+Config testResourceConfig =
ConfigFactory.parseFileAnySyntax(conf_file,
ConfigParseOptions.defaults().setAllowMissing(false));
+Config typesafe =
testResourceConfig.withFallback(reference).resolve();
+testConfiguration = new
ComponentConfigurator<>(HbaseConfiguration.class).detectConfiguration(typesafe,
"hbase");
+
+ }
+
+ @Test(enabled = false)
+ public void testPersistWriter() throws Exception {
+
+HbasePersistWriter testPersistWriter = new
HbasePersistWriter(testConfiguration);
+testPersistWriter.prepare(testConfiguration);
+
+InputStream testActivityFolderStream =
HbasePersistWriterIT.class.getClassLoader()
+.getResourceAsStream("activities");
+List files = IOUtils.readLines(testActivityFolderStream,
Charsets.UTF_8);
--- End diff --
changed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams issue #343: STREAMS-475: LocalRuntimeConfiguration does no...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/343 trivial fix, merging --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #343: STREAMS-475: LocalRuntimeConfiguration ...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/343 STREAMS-475: LocalRuntimeConfiguration does not properly extend StreamsRuntimeConfiguration You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-475 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/343.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #343 commit 0c5975941b19cc3886d4f4b03eea17fc721cae9c Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-12-15T20:11:19Z STREAMS-475: LocalRuntimeConfiguration does not properly extend StreamsRuntimeConfiguration --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #342: STREAMS-472: Fix failing IT in streams-plugins
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/342 trivial fix, merging --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #342: STREAMS-472: Fix failing IT in streams-...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/342 STREAMS-472: Fix failing IT in streams-plugins You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-472 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/342.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #342 commit dc0adfbbb53cfd7075123b64afc73accafd357a2 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-12-15T17:03:22Z STREAMS-472: Fix failing IT in streams-plugins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #340: streams-provider-twitter bug fixes & re...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/340 streams-provider-twitter bug fixes & related improvements. STREAMS-465 STREAMS-466 STREAMS-467 You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-465 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/340.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #340 commit fea030e27afb39257b55f8d9a750818ac341a8e0 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-12-13T21:29:28Z streams-provider-twitter bug fixes & related improvements. STREAMS-465 STREAMS-466 STREAMS-467 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #339: STREAMS-470: Permit use of streams-persist-hdf...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/339 merging directly as there is no risk of side-effects --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #339: STREAMS-470: Permit use of streams-pers...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/339 STREAMS-470: Permit use of streams-persist-hdfs to read/write s3 You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-470 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/339.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #339 commit 48e618713dd411bead032cc5018bc9f4462779bc Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-12-13T20:07:38Z STREAMS-470: Permit use of streams-persist-hdfs to read/write s3 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #335: refactor to use modern apache hbase ins...
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/335#discussion_r92259073
--- Diff: streams-contrib/streams-persist-hbase/pom.xml ---
@@ -56,6 +56,10 @@
${project.version}
+org.slf4j
+jcl-over-slf4j
--- End diff --
theres no preference for jcl or for log4j - whats going on here is jcl is
being neutered and routed into slf4j.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #338: STREAMS-469: fit failing ITs in streams...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/338 STREAMS-469: fit failing ITs in streams-provider-rss As best I can tell, junit forking was allowing the tests to pass - they started failing under TestNG. Wound up removing the static modifier from some state where it didnât really make sense. Along the way, upgraded the underlying library to one under active maintenance. You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-469 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/338.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #338 commit 21086e9a84721ef0d777a65a9cbd9e1f28040b18 Author: Steve Blackmon <sblack...@apache.org> Date: 2016-12-12T03:35:05Z STREAMS-469: fit failing ITs in streams-provider-rss As best I can tell, junit forking was allowing the tests to pass - they started failing under TestNG. Wound up removing the static modifier from some state where it didnât really make sense. Along the way, upgraded the underlying library to one under active maintenance. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #337: STREAMS-468: Fix failing IT in streams-...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/337 STREAMS-468: Fix failing IT in streams-persist-elasticsearch As best I can tell, the test began failing under testNG because it tries to open too many clients. The intent was to support multiple clients while letting components and testing share those clients. After refactoring to use a map<config,client> pattern with a singleton accessor, all tests began passing. You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-468 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/337.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #337 commit 553662b7ed61309b3074dd28f3b2f9171774450f Author: Steve Blackmon <sblack...@apache.org> Date: 2016-12-12T03:31:33Z STREAMS-468: Fix failing IT in streams-persist-elasticsearch As best I can tell, the test began failing under testNG because it tries to open too many clients. The intent was to support multiple clients while letting components and testing share those clients. After refactoring to use a map<config,client> pattern with a singleton accessor, all tests began passing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #335: refactor to use modern apache hbase ins...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/335 refactor to use modern apache hbase instead of cdh4 the tests donât work yet - but STREAMS-48 is open and tagged 0.5 You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-155 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/335.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #335 commit dab3c89f45398f56aa87446b361bce4ec897cfcd Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-12-03T19:24:02Z refactor to use modern apache hbase instead of cdh4 the tests donât work yet --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #334: STREAMS-464: switch ITs from junit to testng
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/334 smarthi [10:06 PM] +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #334: STREAMS-464: switch ITs from junit to t...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/334 STREAMS-464: switch ITs from junit to testng testing with: mvn clean verify -DargLine="-Dconfig.file=`pwd`/application.conf -Djava.util.Arrays.useLegacyMergeSort=true" -DskipTests=true -DskipITs=false problems remain in several modules incuding: streams-persist-elasticsearch streams-provider-rss You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-464 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/334.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #334 commit 899bb84f3c9b411137f37d18114bfda76fc77c48 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-30T03:43:00Z STREAMS-464: switch ITs from junit to testng testing with: mvn clean verify -DargLine="-Dconfig.file=`pwd`/application.conf -Djava.util.Arrays.useLegacyMergeSort=true" -DskipTests=true -DskipITs=false problems remain in several modules incuding: streams-persist-elasticsearch streams-provider-rss --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #333: Streams 458
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/333 Streams 458 rebased, same code as https://github.com/apache/incubator-streams/pull/331 You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-458 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/333.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #333 commit 6e5afce354c7d4e3535771ecdc49b0b604b22187 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-26T02:18:42Z STREAMS-458: GPlusTypeConverter fails processing recent plus#person without a gender Run type converter on live-collected documents during integration test cycle. Collect both a person and an organization. Check that nodes are present before accessing them. commit ac60c5fae7c5a01100ed999f9df5e55899844de4 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-26T14:47:59Z STREAMS-458: GPlusTypeConverter fix + gplus converter integration testing - peer review update --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #331: STREAMS-458: GPlusTypeConverter fix + gplus co...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/331 rebasing, will --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #331: STREAMS-458: GPlusTypeConverter fix + gplus co...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/331 rebasing, will reopen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #331: STREAMS-458: GPlusTypeConverter fix + g...
Github user steveblackmon closed the pull request at: https://github.com/apache/incubator-streams/pull/331 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #331: STREAMS-458: GPlusTypeConverter fix + gplus co...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/331 reopen... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #332: Streams-441: Minimize use of Guava Apis
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/332 ð --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #320: STREAMS-441: Remove compile dependency on guav...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/320 :+1: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #320: STREAMS-441: Remove compile dependency on guav...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/320 I was able to confirm - 08bd8d6ad6c1cb1ea390f7cc0b99da86d972fd19 compiles unit tests pass wc -l all checkstyle results went from 4999 to 4758 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #320: STREAMS-441: Remove compile dependency on guav...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/320 i don't see any pom.xml changes - don't we want to take them out of the reactor wherever we can now? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #331: STREAMS-458: GPlusTypeConverter fix + g...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/331 STREAMS-458: GPlusTypeConverter fix + gplus converter integration testing GPlusTypeConverter fails processing recent plus#person wâ¦ithout a gender Run type converter on live-collected documents during integration test cycle. Collect both a person and an organization. Check that nodes are present before accessing them. You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-458 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/331.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #331 commit f15ca8897806281c7e60f97142e7f151bb0291e4 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-26T02:18:42Z STREAMS-458: GPlusTypeConverter fails processing recent plus#person without a gender Run type converter on live-collected documents during integration test cycle. Collect both a person and an organization. Check that nodes are present before accessing them. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #330: Adding KEYS file
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/330 ð --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #328: STREAMS-440: custom checkstyle.xml, address co...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/328 this merged w --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #328: STREAMS-440: custom checkstyle.xml, address co...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/328 this merged with 5dffd5c32d0d150727a39104d428b21b52c911d4 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #328: STREAMS-440: custom checkstyle.xml, add...
Github user steveblackmon closed the pull request at: https://github.com/apache/incubator-streams/pull/328 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #328: STREAMS-440: custom checkstyle.xml, add...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/328 STREAMS-440: custom checkstyle.xml, address compliance one pass through entire project to increase compliance with checkstyle. certain modules (streams-runtime-*) held to lower standard because discussions on whether to delete them are on-going. You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-440 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/328.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #328 commit 2048f43dcff52621e16a1969efce92ee1bb7545f Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:14:06Z STREAMS-440: streams-converters decrease wc -l checkstyle-result.xml from 928 to 122 commit fe649be82ab3d81cae61e0451858d6372d3a8780 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:22:25Z STREAMS-440: streams-filters decrease wc -l checkstyle-result.xml from 84 to 13 commit cfa4f706f9d184aaf5e26b359b35068e2b57fc0f Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:23:16Z STREAMS-440: streams-http decrease wc -l checkstyle-result.xml from 742 to 65 commit b3b75d4e7f84b7753309652d59fa46f134977c8b Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:25:12Z STREAMS-440: streams-config reduce wc -l target/checkstyle-result.xml from 85 to 60 commit db47e801b2901d7b1720cf6dc45646bcf5373dae Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:26:49Z STREAMS-440: streams-persist-kinesis decrease wc -l checkstyle-result.xml from 222 to 64 commit f8210dddbd3de79065e1ed494a69df21a2e0ff13 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:28:04Z STREAMS-440: streams-persist-s3 decrease wc -l checkstyle-result.xml from 580 to 61 commit fa0d73e7569e02742f0be0bdcd4871d7c0f30931 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:28:55Z STREAMS-440: streams-persist-console decrease wc -l checkstyle-result.xml from 114 to 12 commit 953f4cbb609b9c6399691989b84690cb79afc43a Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:30:28Z STREAMS-440: streams-persist-elasticsearch decrease wc -l checkstyle-result.xml from 1572 to 131 commit 9afe2db595736f8f4497a36870f2dd757e946835 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:31:00Z STREAMS-440: streams-persist-filebuffer commit 676be795634a5ef712362c35df915468d6732d8d Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:33:51Z STREAMS-440: streams-persist-graph decrease wc -l checkstyle-result.xml from 664 to 65 commit 7e2b49f32ff11d0b6b5f06b1e22385a8c4fedf22 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:34:41Z STREAMS-440: streams-persist-hbase reduce wc -l target/checkstyle-result.xml from 204 to 20 commit b4e59d2f68742679f3ccbb65dcda5de2feb03db7 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:36:26Z STREAMS-440: streams-persist-hdfs decrease wc -l target/checkstyle-result.xml from 508 to 58 commit 50e1390047edfa4daaa2089325cf408bf3d3872a Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:37:07Z STREAMS-440: streams-persist-kafka reduces wc -l target/checkstyle-result.xml from 233 to 31 commit 43a9a7dcb660488b07a19090afedfffd5f529416 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:37:39Z STREAMS-440: streams-persist-mongo reduce wc -l target/checkstyle-result.xml to 37 commit 5adb1495ea54ccd69e24ffbe5cbc191ac60d75a3 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:38:30Z STREAMS-440: streams-processor-jackson reduce wc -l target/checkstyle-result.xml from 127 to 17 commit 5e96ff4e686a4e206d31976b626d1d35cd88e251 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:39:08Z STREAMS-440: streams-processor-json reduce wc -l target/checkstyle-result.xml from 242 to 31 commit 29ee86a7dba47adb2dfba5e9bc6ffaa4d0c1 Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-19T17:40:19Z STREAMS-440: streams-processor-peoplepattern reduce wc -l target/checkstyle-result.xml from 67 to 9 commit 8d861005797b13093ce4b39eb94d
[GitHub] incubator-streams issue #327: STREAMS-457: update page schema and converter ...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/327 I could not replicate the build failure running mvn -o clean install -DargLine="-Djava.util.Arrays.useLegacyMergeSort=true" I suppose we'll see if jenkins runs into any problems. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #327: STREAMS-457: update page schema and converter ...
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/327 I'm trying to replicate. On Nov 17, 2016 9:58 PM, "Matt Franklin" <notificati...@github.com> wrote: > The build is failing to find the plugin, is the build order wrong for > modules? > > [ERROR] Plugin org.apache.streams.plugins:streams-plugin-pojo:0.5-incubating-SNAPSHOT > or one of its dependencies could not be resolved: Could not find artifact > org.apache.streams.plugins:streams-plugin-pojo:jar:0.5-incubating-SNAPSHOT > -> [Help 1] > > â > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <https://github.com/apache/incubator-streams/pull/327#issuecomment-261367461>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAq5zMnWkSWyQIajDe1Y7AULnkafS9KKks5q_L_1gaJpZM4K0PT2> > . > --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #327: STREAMS-457: update page schema and con...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/327 STREAMS-457: update page schema and converter to match api v2.8 You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-457 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/327.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #327 commit 9e6ed3e389fa55f3714391a5ef3d570ab06e870f Author: Steve Blackmon @steveblackmon <sblack...@apache.org> Date: 2016-11-16T18:07:44Z STREAMS-457: update page schema and converter to match api v2.8 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #326: STREAMS-456: Facebook page provider doe...
GitHub user steveblackmon opened a pull request: https://github.com/apache/incubator-streams/pull/326 STREAMS-456: Facebook page provider does not include many fields You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveblackmon/incubator-streams STREAMS-456 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-streams/pull/326.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #326 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams-master pull request #12: STREAMS-439: add and enable check...
Github user steveblackmon closed the pull request at: https://github.com/apache/incubator-streams-master/pull/12 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #324: STREAMS-443 streams-project
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/324 merged in 4f100f3a2e97ddc512a9d745b999625602ad274e Squashed commit of STREAMS-443: use org.apache.streams.plugins:streams-plugin-pojo instead of org.jsonschema2pojo --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams pull request #324: STREAMS-443 streams-project
Github user steveblackmon closed the pull request at: https://github.com/apache/incubator-streams/pull/324 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-streams issue #324: STREAMS-443 streams-project
Github user steveblackmon commented on the issue: https://github.com/apache/incubator-streams/pull/324 taking this as +1 and beginning merge --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
