[GitHub] incubator-metron pull request #300: METRON-489: RemoveSubdomains Stellar Fun...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-metron/pull/300 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Re: [GitHub] incubator-metron pull request #300: METRON-489: RemoveSubdomains Stellar Fun...
My assumption was that he was expecting to get "com.com" back instead of "
subdomain.com". www.subdomain being the subdomain of com.com, which is a
valid website.
Jon
On Fri, Oct 7, 2016 at 2:10 PM james-sirota wrote:
> Github user james-sirota commented on a diff in the pull request:
>
>
> https://github.com/apache/incubator-metron/pull/300#discussion_r82441035
>
> --- Diff:
> metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/NetworkFunctions.java
> ---
> @@ -78,13 +78,16 @@ public Object apply(List list) {
>
> @Override
> public Object apply(List objects) {
> + if(objects.isEmpty()) {
> +return null;
> + }
>Object dnObj = objects.get(0);
>InternetDomainName idn = toDomainName(dnObj);
>if(idn != null) {
> String dn = dnObj.toString();
> String tld = idn.publicSuffix().toString();
> -String suffix =
> Iterables.getFirst(Splitter.on(tld).split(dn), null);
> -if(suffix != null)
> +String suffix = dn.substring(0, dn.length() - tld.length());
> +if(suffix != null )
> --- End diff --
>
> www.subdomain.com.com is not a valid TLD. for a list of valid
> domains see here: https://publicsuffix.org/list/effective_tld_names.dat
>
> If the system sees this kind of domain this should immediately be
> flagged as alert and triaged with a very high score
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
> with INFRA.
> ---
>
--
Jon
[GitHub] incubator-metron pull request #300: METRON-489: RemoveSubdomains Stellar Fun...
Github user james-sirota commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/300#discussion_r82441035
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/NetworkFunctions.java
---
@@ -78,13 +78,16 @@ public Object apply(List list) {
@Override
public Object apply(List objects) {
+ if(objects.isEmpty()) {
+return null;
+ }
Object dnObj = objects.get(0);
InternetDomainName idn = toDomainName(dnObj);
if(idn != null) {
String dn = dnObj.toString();
String tld = idn.publicSuffix().toString();
-String suffix = Iterables.getFirst(Splitter.on(tld).split(dn),
null);
-if(suffix != null)
+String suffix = dn.substring(0, dn.length() - tld.length());
+if(suffix != null )
--- End diff --
www.subdomain.com.com is not a valid TLD. for a list of valid domains see
here: https://publicsuffix.org/list/effective_tld_names.dat
If the system sees this kind of domain this should immediately be flagged
as alert and triaged with a very high score
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #300: METRON-489: RemoveSubdomains Stellar Fun...
Github user justinleet commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/300#discussion_r82378741
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/NetworkFunctions.java
---
@@ -78,13 +78,16 @@ public Object apply(List list) {
@Override
public Object apply(List objects) {
+ if(objects.isEmpty()) {
+return null;
+ }
Object dnObj = objects.get(0);
InternetDomainName idn = toDomainName(dnObj);
if(idn != null) {
String dn = dnObj.toString();
String tld = idn.publicSuffix().toString();
-String suffix = Iterables.getFirst(Splitter.on(tld).split(dn),
null);
-if(suffix != null)
+String suffix = dn.substring(0, dn.length() - tld.length());
+if(suffix != null )
--- End diff --
Could you just drop this if-else? Substring never returns null, so it can
be dropped entirely.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #300: METRON-489: RemoveSubdomains Stellar Fun...
GitHub user cestella opened a pull request: https://github.com/apache/incubator-metron/pull/300 METRON-489: RemoveSubdomains Stellar Function behaves incorrectly for some domains com.com throws an exception www.subdomain.com.com returns subdomain.com Unsure if other standard weirdness with TLDs get handled like this (e.g. net.net, co.uk.co.uk) You can merge this pull request into a Git repository by running: $ git pull https://github.com/cestella/incubator-metron remove_subdomains_edge_cases Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-metron/pull/300.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #300 commit 17c386a56aa9963e5c3947e53f20292518747891 Author: cstella Date: 2016-10-07T08:02:48Z METRON-489: RemoveSubdomains Stellar Function behaves incorrectly for some domains --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
