[jira] [Commented] (KNOX-924) X-forwarded-host, X-forwared-port header behavior when there are multiple balancers/proxys causing redirection problems
[ https://issues.apache.org/jira/browse/KNOX-924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15972763#comment-15972763 ] Sumit Gupta commented on KNOX-924: -- [~jeffreyr97]. Sorry for jumping in late. There has been a lot of discussion here so I may be missing something, but essentially you just want a way to access the X-Forwarded-Context in the rewrite rules? Does it boil down to that or is there more to it? > X-forwarded-host, X-forwared-port header behavior when there are multiple > balancers/proxys causing redirection problems > > > Key: KNOX-924 > URL: https://issues.apache.org/jira/browse/KNOX-924 > Project: Apache Knox > Issue Type: Bug > Components: Server >Reporter: Jeffrey E Rodriguez >Assignee: Jeffrey E Rodriguez >Priority: Critical > > When there are other intermediaries between client and Knox it is possible to > cause Knox rewrites to not go to the gateway host but in fact the > gateway.host reflects X-forwar-host. > e.g. > "curl -v -k -u guest:guest-password -H X-Forwarded-Proto: http -H > X-Forwarded-For: jeff -H X-Forwarded-Context:/ordexecvalid-prod/data/ -H > X-Forwarded-Port: 843 -H X-Forwarded-Host: myfoo.com -i > https://knox1.fyre.ibm.com:8443/gateway/default/webhdfs/v1/apps/hbase/data/hbase.id?op=OPEN > * Trying 9.30.57.135... > * TCP_NODELAY set > * Connected to knox1.fyre.ibm.com (9.30.57.135) port 8443 (#0) > * TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 > * Server certificate: localhost > * Server auth using Basic with user 'guest' > > GET > > /gateway/default/webhdfs/v1//iop/apps/4.3.0.0-/mapreduce/mapreduce.tar.gz?op=OPEN > > HTTP/1.1 > > Host: knox1.fyre.ibm.com:8443 > > Authorization: Basic Z3Vlc3Q6Z3Vlc3QtcGFzc3dvcmQ= > > User-Agent: curl/7.51.0 > > Accept: */* > > X-Forwarded-Proto: http > > X-Forwarded-Server: myfooserver.com > > X-Forwarded-For: jeff > > X-Forwarded-Context: /ordexecvalid-prod/data/ > > X-Forwarded-Port: 843 > > X-Forwarded-Host: myfoo.com > < HTTP/1.1 307 Temporary Redirect > HTTP/1.1 307 Temporary Redirect > < Date: Sun, 16 Apr 2017 00:18:44 GMT > Date: Sun, 16 Apr 2017 00:18:44 GMT > < Set-Cookie: > JSESSIONID=1ry9x2q6sxuju1r5vr5tfkcifu;Path=/gateway/default;Secure;HttpOnly > Set-Cookie: > JSESSIONID=1ry9x2q6sxuju1r5vr5tfkcifu;Path=/gateway/default;Secure;HttpOnly > < Expires: Thu, 01 Jan 1970 00:00:00 GMT > Expires: Thu, 01 Jan 1970 00:00:00 GMT > < Set-Cookie: rememberMe=deleteMe; Path=/gateway/default; Max-Age=0; > Expires=Sat, 15-Apr-2017 00:18:44 GMT > Set-Cookie: rememberMe=deleteMe; Path=/gateway/default; Max-Age=0; > Expires=Sat, 15-Apr-2017 00:18:44 GMT > < Cache-Control: no-cache > Cache-Control: no-cache > < Expires: Sun, 16 Apr 2017 00:18:59 GMT > Expires: Sun, 16 Apr 2017 00:18:59 GMT > < Date: Sun, 16 Apr 2017 00:18:59 GMT > Date: Sun, 16 Apr 2017 00:18:59 GMT > < Pragma: no-cache > Pragma: no-cache > < Expires: Sun, 16 Apr 2017 00:18:59 GMT > Expires: Sun, 16 Apr 2017 00:18:59 GMT > < Date: Sun, 16 Apr 2017 00:18:59 GMT > Date: Sun, 16 Apr 2017 00:18:59 GMT > < Pragma: no-cache > Pragma: no-cache > < Location: > http://myfoo.com:843/gateway/default/webhdfs/data/v1/webhdfs/v1/iop/apps/4.3.0.0-/mapreduce/mapreduce.tar.gz?_=CBCA3IsAYTfMdBdl3-s2CYz7RWXKMvT_S9OT2LSLanl61bElhqjA3hCyqdCHhVJ1MLvn_NDL6JEBX5xdZ_N8ALSTINW1er2-7qih_nE4riDc17LQFLs9YYkBt2P3dZz_sFKOsPzCPmyNrcaHijxkloX2m4Jy4hkqEqlCakpVFEL-ZVBqiNGlBvFk1O6fbpX_UJn8qbm67uq4M6I > Location: > http://myfoo.com:843/gateway/default/webhdfs/data/v1/webhdfs/v1/iop/apps/4.3.0.0-/mapreduce/mapreduce.tar.gz?_=CBCA3IsAYTfMdBdl3-s2CYz7RWXKMvT_S9OT2LSLanl61bElhqjA3hCyqdCHhVJ1MLvn_NDL6JEBX5xdZ_N8ALSTINW1er2-7qih_nE4riDc17LQFLs9YYkBt2P3dZz_sFKOsPzCPmyNrcaHijxkloX2m4Jy4hkqEqlCakpVFEL-ZVBqiNGlBvFk1O6fbpX_UJn8qbm67uq4M6I > < Content-Type: application/octet-stream > Content-Type: application/octet-stream > < Server: Jetty(6.1.26-ibm) > Server: Jetty(6.1.26-ibm) > < Content-Length: 0 > Content-Length: 0 > See here the redirection header. > http://myfoo.com:843/gateway/default/webhdfs/data/v1/webhdfs/v1/iop/apps/4.3.0.0-/mapreduce/mapreduce.tar.gz?_=CBCA3IsAYTfMdBdl3-s2CYz7RWXKMvT_S9OT2LSLanl61bElhqjA3hCyqdCHhVJ1MLvn_NDL6JEBX5xdZ_N8ALSTINW1er2-7qih_nE4riDc17LQFLs9YYkBt2P3dZz_sFKOsPzCPmyNrcaHijxkloX2m4Jy4hkqEqlCakpVFEL-ZVBqiNGlBvFk1O6fbpX_UJn8qbm67uq4M6I > This could cause unexpected results. > The UrlRequestResponse would use the X-Forward headers (even if > gateway.xforwarded.enabled is false). > The documentation is very succinct. It seems like the xforwarded.enabled only > accomplish passing the "Knox" x-Forward-* headers when it is not set to false. > But there is no documentation on the effect of other servers setting up >
[jira] [Commented] (KNOX-911) Ability to scope cookies to a given Path
[
https://issues.apache.org/jira/browse/KNOX-911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936436#comment-15936436
]
Sumit Gupta commented on KNOX-911:
--
[~akanto], it does seem a bit of special case if the two shared clusters as
you mention it cannot share anything other than the reverse proxy. Can they not
simply have their own domain and reverse proxy? It would be nice to understand
if there is a workaround for this use case or if you are blocked and really
need cookie rewriting to move forward.
> Ability to scope cookies to a given Path
>
>
> Key: KNOX-911
> URL: https://issues.apache.org/jira/browse/KNOX-911
> Project: Apache Knox
> Issue Type: Wish
>Reporter: Attila Kanto
>
> If there are multiple individual Knox instances behind of a reverse proxy,
> then it would be very useful if the Cookies could be scoped to a given Path.
> If a reverse proxy is put at the font of multiple Knox instances then scoping
> the Cookies to domain is not sufficient since the /gateway1/... and
> /gateway2/... cookies will overwrite each other.
> {code}
> +-+
> | |
> |Reverse Proxy|
> | |
> +-+
> ||
>/gateway1/topology || /gateway2/topology
> ||
> +v+
> +--v--+
> | ||
> |
> | Knox 1 (/gateway1/topology)|| Knox 2 (/gateway2/topology)
> |
> | ||
> |
> +-+
> +-+
> {code}
> Proposal:
> Cookies can be scoped with Set-Cookie: Path=/somepath header field.
> It would be very convenient if this scope path could be set in
> gateway-site.xml and Knox would return it in Set-Cookie header field to
> clients.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (KNOX-907) Knox Proxy - Ambari HIVE2.0 view doesn't showup due to Servicecheck issues
[
https://issues.apache.org/jira/browse/KNOX-907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Gupta resolved KNOX-907.
--
Resolution: Fixed
> Knox Proxy - Ambari HIVE2.0 view doesn't showup due to Servicecheck issues
> --
>
> Key: KNOX-907
> URL: https://issues.apache.org/jira/browse/KNOX-907
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
>Reporter: SuryaKranthi Koneru
>Assignee: Larry McCay
>Priority: Critical
> Labels: kip-4
> Fix For: 0.12.0
>
> Attachments: KNOX-907-001.patch
>
>
> Ambari URL:-
> http://hostname:8080/#/main/views/HIVE/2.0.0/AUTO_HIVE20_INSTANCE
> Knox proxy enabled for Ambari:-
> https://hostname:8443/gateway/ui/ambari/#/main/views/HIVE/2.0.0/AUTO_HIVE20_INSTANCE
> steps to reproduce:-
> 1- Configure Knox Proxy for Ambari.
> create ui.xml in /usr/hdp/current/knox-server/conf/topologies
> Note:- I have already deployed ui.xml topology in this path. There is no Knox
> restart required.
> below is the sample ui.xml
> 2- Invoke Ambari through knox proxy
> https://hostname:8443/gateway/ui/ambari/#/login
> 3- Click on hive2.0 view
> Notice all the service checks failed messages. Attached is screenshot
> Note:- If you go through Ambari URL then you will be able to access HIVE2.0
> view.
> http://hostname:8080/#/main/views/HIVE/2.0.0/AUTO_HIVE20_INSTANCE
> ui.xml:-
>
>
>
> authentication
> ShiroProvider
> true
>
> sessionTimeout
> 30
>
>
> main.ldapRealm
>
> org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm
>
>
> main.ldapRealm.userDnTemplate
>
> uid={0},ou=people,dc=hadoop,dc=apache,dc=org
>
>
> main.ldapRealm.contextFactory.url
> ldap://:
>
>
>
> main.ldapRealm.contextFactory.authenticationMechanism
> simple
>
>
> urls./**
> authcBasic
>
>
>
> identity-assertion
> Default
> true
>
>
>
> AMBARI
> http://hostname:8080
>
>
> AMBARIUI
> http://hostname:8080
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (KNOX-907) Knox Proxy - Ambari HIVE2.0 view doesn't showup due to Servicecheck issues
[
https://issues.apache.org/jira/browse/KNOX-907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Gupta updated KNOX-907:
-
Fix Version/s: (was: 0.13.0)
0.12.0
> Knox Proxy - Ambari HIVE2.0 view doesn't showup due to Servicecheck issues
> --
>
> Key: KNOX-907
> URL: https://issues.apache.org/jira/browse/KNOX-907
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
>Reporter: SuryaKranthi Koneru
>Assignee: Larry McCay
>Priority: Critical
> Labels: kip-4
> Fix For: 0.12.0
>
> Attachments: KNOX-907-001.patch
>
>
> Ambari URL:-
> http://hostname:8080/#/main/views/HIVE/2.0.0/AUTO_HIVE20_INSTANCE
> Knox proxy enabled for Ambari:-
> https://hostname:8443/gateway/ui/ambari/#/main/views/HIVE/2.0.0/AUTO_HIVE20_INSTANCE
> steps to reproduce:-
> 1- Configure Knox Proxy for Ambari.
> create ui.xml in /usr/hdp/current/knox-server/conf/topologies
> Note:- I have already deployed ui.xml topology in this path. There is no Knox
> restart required.
> below is the sample ui.xml
> 2- Invoke Ambari through knox proxy
> https://hostname:8443/gateway/ui/ambari/#/login
> 3- Click on hive2.0 view
> Notice all the service checks failed messages. Attached is screenshot
> Note:- If you go through Ambari URL then you will be able to access HIVE2.0
> view.
> http://hostname:8080/#/main/views/HIVE/2.0.0/AUTO_HIVE20_INSTANCE
> ui.xml:-
>
>
>
> authentication
> ShiroProvider
> true
>
> sessionTimeout
> 30
>
>
> main.ldapRealm
>
> org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm
>
>
> main.ldapRealm.userDnTemplate
>
> uid={0},ou=people,dc=hadoop,dc=apache,dc=org
>
>
> main.ldapRealm.contextFactory.url
> ldap://:
>
>
>
> main.ldapRealm.contextFactory.authenticationMechanism
> simple
>
>
> urls./**
> authcBasic
>
>
>
> identity-assertion
> Default
> true
>
>
>
> AMBARI
> http://hostname:8080
>
>
> AMBARIUI
> http://hostname:8080
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KNOX-907) Knox Proxy - Ambari HIVE2.0 view doesn't showup due to Servicecheck issues
[
https://issues.apache.org/jira/browse/KNOX-907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924188#comment-15924188
]
Sumit Gupta commented on KNOX-907:
--
Looks good to me, I'll get the patch in.
> Knox Proxy - Ambari HIVE2.0 view doesn't showup due to Servicecheck issues
> --
>
> Key: KNOX-907
> URL: https://issues.apache.org/jira/browse/KNOX-907
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
>Reporter: SuryaKranthi Koneru
>Assignee: Larry McCay
>Priority: Critical
> Labels: kip-4
> Fix For: 0.13.0
>
> Attachments: KNOX-907-001.patch
>
>
> Ambari URL:-
> http://hostname:8080/#/main/views/HIVE/2.0.0/AUTO_HIVE20_INSTANCE
> Knox proxy enabled for Ambari:-
> https://hostname:8443/gateway/ui/ambari/#/main/views/HIVE/2.0.0/AUTO_HIVE20_INSTANCE
> steps to reproduce:-
> 1- Configure Knox Proxy for Ambari.
> create ui.xml in /usr/hdp/current/knox-server/conf/topologies
> Note:- I have already deployed ui.xml topology in this path. There is no Knox
> restart required.
> below is the sample ui.xml
> 2- Invoke Ambari through knox proxy
> https://hostname:8443/gateway/ui/ambari/#/login
> 3- Click on hive2.0 view
> Notice all the service checks failed messages. Attached is screenshot
> Note:- If you go through Ambari URL then you will be able to access HIVE2.0
> view.
> http://hostname:8080/#/main/views/HIVE/2.0.0/AUTO_HIVE20_INSTANCE
> ui.xml:-
>
>
>
> authentication
> ShiroProvider
> true
>
> sessionTimeout
> 30
>
>
> main.ldapRealm
>
> org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm
>
>
> main.ldapRealm.userDnTemplate
>
> uid={0},ou=people,dc=hadoop,dc=apache,dc=org
>
>
> main.ldapRealm.contextFactory.url
> ldap://:
>
>
>
> main.ldapRealm.contextFactory.authenticationMechanism
> simple
>
>
> urls./**
> authcBasic
>
>
>
> identity-assertion
> Default
> true
>
>
>
> AMBARI
> http://hostname:8080
>
>
> AMBARIUI
> http://hostname:8080
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KNOX-897) X-Forwarded-Port is incorrectly determined
[
https://issues.apache.org/jira/browse/KNOX-897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905467#comment-15905467
]
Sumit Gupta commented on KNOX-897:
--
Ah I missed that! Thanks for pointing that out [~smore]. Sorry for the extra
investigation here but I thought we were potentially missing out on a use case.
I think this JIRA can go back to being resolved!
> X-Forwarded-Port is incorrectly determined
> --
>
> Key: KNOX-897
> URL: https://issues.apache.org/jira/browse/KNOX-897
> Project: Apache Knox
> Issue Type: Bug
>Affects Versions: 0.11.0
>Reporter: Attila Kanto
> Fix For: 0.12.0
>
> Attachments: gateway.log, knox-897.patch, Screen Shot 2017-03-01 at
> 14.44.03.png
>
>
> If the client fills out the following headers:
> {code}
> Header[X-Forwarded-Host]=local.somehost.com
> Header[X-Forwarded-Proto]=https
> {code}
> And does not specify the port number in X-Forwarded-Host since it uses the
> the default port, then Knox automatically populates X-Forwarded-Port header
> field with its own local port e.g. 8443 instead of the default 443 which
> results in page not founds (see screenshot and log).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KNOX-897) X-Forwarded-Port is incorrectly determined
[
https://issues.apache.org/jira/browse/KNOX-897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905387#comment-15905387
]
Sumit Gupta commented on KNOX-897:
--
Hmm, not sure [~smore]. I just tried with the 0.12.0 RC and I see this with a
plain old client request as the outgoing from Knox (I'm looking at the gateway
log with http wire level debug turned on). There is a config flag in
gateway-site to turn the x-forwarded population off, but I can't imagine that
you have set that.
2017-03-10 11:46:16,899 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0
>> "GET /webhdfs/v1/?op=LISTSTATUS=guest HTTP/1.1[\r][\n]"
2017-03-10 11:46:16,900 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0
>> "X-Forwarded-For: 127.0.0.1[\r][\n]"
2017-03-10 11:46:16,900 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0
>> "X-Forwarded-Proto: https[\r][\n]"
2017-03-10 11:46:16,900 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0
>> "X-Forwarded-Port: 8443[\r][\n]"
2017-03-10 11:46:16,901 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0
>> "X-Forwarded-Host: localhost:8443[\r][\n]"
2017-03-10 11:46:16,901 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0
>> "X-Forwarded-Server: localhost[\r][\n]"
2017-03-10 11:46:16,901 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0
>> "X-Forwarded-Context: /gateway/sandbox[\r][\n]"
2017-03-10 11:46:16,902 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0
>> "Connection: keep-alive[\r][\n]"
2017-03-10 11:46:16,902 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0
>> "User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_66)[\r][\n]"
2017-03-10 11:46:16,902 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0
>> "Accept-Encoding: gzip,deflate[\r][\n]"
2017-03-10 11:46:16,903 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0
>> "Host: c6401.ambari.apache.org:50070[\r][\n]"
2017-03-10 11:46:16,903 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0
>> "[\r][\n]"
> X-Forwarded-Port is incorrectly determined
> --
>
> Key: KNOX-897
> URL: https://issues.apache.org/jira/browse/KNOX-897
> Project: Apache Knox
> Issue Type: Bug
>Affects Versions: 0.11.0
>Reporter: Attila Kanto
> Fix For: 0.12.0
>
> Attachments: gateway.log, knox-897.patch, Screen Shot 2017-03-01 at
> 14.44.03.png
>
>
> If the client fills out the following headers:
> {code}
> Header[X-Forwarded-Host]=local.somehost.com
> Header[X-Forwarded-Proto]=https
> {code}
> And does not specify the port number in X-Forwarded-Host since it uses the
> the default port, then Knox automatically populates X-Forwarded-Port header
> field with its own local port e.g. 8443 instead of the default 443 which
> results in page not founds (see screenshot and log).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KNOX-897) X-Forwarded-Port is incorrectly determined
[
https://issues.apache.org/jira/browse/KNOX-897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905347#comment-15905347
]
Sumit Gupta commented on KNOX-897:
--
[~akanto], the missing variation in the scenario you describe is :
- User connects directly to Knox specifying no X-Forwarded-* headers at all.
In such a scenario Knox will populate the X-Forwarded-* headers to match its
own configuration so that those values can be consumed downstream correctly.
> X-Forwarded-Port is incorrectly determined
> --
>
> Key: KNOX-897
> URL: https://issues.apache.org/jira/browse/KNOX-897
> Project: Apache Knox
> Issue Type: Bug
>Affects Versions: 0.11.0
>Reporter: Attila Kanto
> Fix For: 0.12.0
>
> Attachments: gateway.log, knox-897.patch, Screen Shot 2017-03-01 at
> 14.44.03.png
>
>
> If the client fills out the following headers:
> {code}
> Header[X-Forwarded-Host]=local.somehost.com
> Header[X-Forwarded-Proto]=https
> {code}
> And does not specify the port number in X-Forwarded-Host since it uses the
> the default port, then Knox automatically populates X-Forwarded-Port header
> field with its own local port e.g. 8443 instead of the default 443 which
> results in page not founds (see screenshot and log).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KNOX-897) X-Forwarded-Port is incorrectly determined
[
https://issues.apache.org/jira/browse/KNOX-897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903824#comment-15903824
]
Sumit Gupta commented on KNOX-897:
--
I'm hoping we don't have to choose [~smore] but I haven't really looked at that
code in a while. Ideally we can figure out if we have received any
X-Forwarded-* headers and then default to 443 or 80, otherwise default to our
own port.
> X-Forwarded-Port is incorrectly determined
> --
>
> Key: KNOX-897
> URL: https://issues.apache.org/jira/browse/KNOX-897
> Project: Apache Knox
> Issue Type: Bug
>Affects Versions: 0.11.0
>Reporter: Attila Kanto
> Fix For: 0.12.0
>
> Attachments: gateway.log, knox-897.patch, Screen Shot 2017-03-01 at
> 14.44.03.png
>
>
> If the client fills out the following headers:
> {code}
> Header[X-Forwarded-Host]=local.somehost.com
> Header[X-Forwarded-Proto]=https
> {code}
> And does not specify the port number in X-Forwarded-Host since it uses the
> the default port, then Knox automatically populates X-Forwarded-Port header
> field with its own local port e.g. 8443 instead of the default 443 which
> results in page not founds (see screenshot and log).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (KNOX-905) Add httpclient properties to knoxshell log4j properties file
Sumit Gupta created KNOX-905: Summary: Add httpclient properties to knoxshell log4j properties file Key: KNOX-905 URL: https://issues.apache.org/jira/browse/KNOX-905 Project: Apache Knox Issue Type: Bug Components: KnoxShell Affects Versions: 0.12.0 Reporter: Sumit Gupta Assignee: Sumit Gupta Fix For: 0.12.0 The file knoxshell-log4j.properties file does not contain any helpful properties for httpclient debugging. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (KNOX-904) 4.5.2 httpclient dependency re-introduced
[ https://issues.apache.org/jira/browse/KNOX-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta resolved KNOX-904. -- Resolution: Fixed > 4.5.2 httpclient dependency re-introduced > - > > Key: KNOX-904 > URL: https://issues.apache.org/jira/browse/KNOX-904 > Project: Apache Knox > Issue Type: Bug >Affects Versions: 0.12.0 >Reporter: Sumit Gupta >Assignee: Sumit Gupta > Fix For: 0.12.0 > > > As per this bug [KNOX-762] there is an issue with this specific version of > httpclient. We need to explore the latest version or fall back to 4.5.1 again. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KNOX-904) 4.5.2 httpclient dependency re-introduced
[ https://issues.apache.org/jira/browse/KNOX-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903288#comment-15903288 ] Sumit Gupta commented on KNOX-904: -- Going to 4.5.3 seemed risky since we are trying to release 0.12.0. Mostly because 4.5.3 yanked away certain annotations like org.apache.http.annotation.NotThreadSafe; This can be traced back to this https://issues.apache.org/jira/browse/HTTPCLIENT-1743 As a note for the future, it looks like we need to uptake the new annotation: https://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/annotation/ThreadingBehavior.html > 4.5.2 httpclient dependency re-introduced > - > > Key: KNOX-904 > URL: https://issues.apache.org/jira/browse/KNOX-904 > Project: Apache Knox > Issue Type: Bug >Affects Versions: 0.12.0 >Reporter: Sumit Gupta >Assignee: Sumit Gupta > Fix For: 0.12.0 > > > As per this bug [KNOX-762] there is an issue with this specific version of > httpclient. We need to explore the latest version or fall back to 4.5.1 again. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (KNOX-904) 4.5.2 httpclient dependency re-introduced
Sumit Gupta created KNOX-904: Summary: 4.5.2 httpclient dependency re-introduced Key: KNOX-904 URL: https://issues.apache.org/jira/browse/KNOX-904 Project: Apache Knox Issue Type: Bug Affects Versions: 0.12.0 Reporter: Sumit Gupta Fix For: 0.12.0 As per this bug [KNOX-762] there is an issue with this specific version of httpclient. We need to explore the latest version or fall back to 4.5.1 again. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (KNOX-903) KnoxShell allows self signed certs to be used without any checks
Sumit Gupta created KNOX-903: Summary: KnoxShell allows self signed certs to be used without any checks Key: KNOX-903 URL: https://issues.apache.org/jira/browse/KNOX-903 Project: Apache Knox Issue Type: Bug Components: KnoxShell Affects Versions: 0.12.0 Reporter: Sumit Gupta Priority: Critical Fix For: 0.12.0 A TrustStrategy of TrustSelfSignedStrategy is being used while setting up http clients to communicate with Knox over SSL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (KNOX-901) Fix Knox-master-daily jenkins build
Sumit Gupta created KNOX-901: Summary: Fix Knox-master-daily jenkins build Key: KNOX-901 URL: https://issues.apache.org/jira/browse/KNOX-901 Project: Apache Knox Issue Type: Bug Reporter: Sumit Gupta The https://builds.apache.org/job/Knox-master-daily/ is failing at this point due to port conflicts. Though the test passes locally we need to put in a fix so that our builds work. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KNOX-841) Proxy support for Solr UI
[ https://issues.apache.org/jira/browse/KNOX-841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893032#comment-15893032 ] Sumit Gupta commented on KNOX-841: -- Thanks for testing it [~risdenk]! The trailing slash issue with UI proxying seems to be a recurring thing. At this point we should file a bug for that. Since you discovered it, it would be great if you could file it. The 6.1.0 version is not a requirement. That version number on the service definition was really meant for the service def but it immediately became a way to do some book-keeping for what backend service version it represents. So essentially it doesn't really matter much but it is important for us to document what the minimum version the backend service is for support. However, If you are confident it works well with 6.0.0 I can update the version number to reflect that. > Proxy support for Solr UI > - > > Key: KNOX-841 > URL: https://issues.apache.org/jira/browse/KNOX-841 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Affects Versions: 0.10.0 >Reporter: Richard Ding >Assignee: Richard Ding > Fix For: 0.12.0 > > Attachments: KNOX_841_1.patch, KNOX-841_2.patch, KNOX-841-3.patch, > KNOX_841.patch, Screen Shot 2017-02-05 at 3.01.20 PM.png > > > Provide proxy UI support for the Solr UI. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-841) Proxy support for Solr UI
[ https://issues.apache.org/jira/browse/KNOX-841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-841: - Resolution: Fixed Status: Resolved (was: Patch Available) > Proxy support for Solr UI > - > > Key: KNOX-841 > URL: https://issues.apache.org/jira/browse/KNOX-841 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Affects Versions: 0.10.0 >Reporter: Richard Ding >Assignee: Richard Ding > Fix For: 0.12.0 > > Attachments: KNOX_841_1.patch, KNOX-841_2.patch, KNOX-841-3.patch, > KNOX_841.patch, Screen Shot 2017-02-05 at 3.01.20 PM.png > > > Provide proxy UI support for the Solr UI. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-841) Proxy support for Solr UI
[ https://issues.apache.org/jira/browse/KNOX-841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-841: - Attachment: KNOX-841-3.patch Based on the discussion I'm going to push in a slightly modified version of the patch that was provided by [~rding]. The only changes I am making are: 1. Renaming the role for the service to "SOLR" since it is both the UI and API. 2. Moving it to the 'solr' folder in the service definitions. Thanks for the contribution and I do hope you can help test the RC coming up soon [~rding]! > Proxy support for Solr UI > - > > Key: KNOX-841 > URL: https://issues.apache.org/jira/browse/KNOX-841 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Affects Versions: 0.10.0 >Reporter: Richard Ding >Assignee: Richard Ding > Fix For: 0.12.0 > > Attachments: KNOX_841_1.patch, KNOX-841_2.patch, KNOX-841-3.patch, > KNOX_841.patch, Screen Shot 2017-02-05 at 3.01.20 PM.png > > > Provide proxy UI support for the Solr UI. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KNOX-841) Proxy support for Solr UI
[ https://issues.apache.org/jira/browse/KNOX-841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890259#comment-15890259 ] Sumit Gupta commented on KNOX-841: -- +1 on [~lmccay]'s comments above. > Proxy support for Solr UI > - > > Key: KNOX-841 > URL: https://issues.apache.org/jira/browse/KNOX-841 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Affects Versions: 0.10.0 >Reporter: Richard Ding >Assignee: Richard Ding > Fix For: 0.12.0 > > Attachments: KNOX_841_1.patch, KNOX-841_2.patch, KNOX_841.patch, > Screen Shot 2017-02-05 at 3.01.20 PM.png > > > Provide proxy UI support for the Solr UI. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-719) Knox support for Yarn Resource Manager HA
[ https://issues.apache.org/jira/browse/KNOX-719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-719: - Resolution: Fixed Status: Resolved (was: Patch Available) > Knox support for Yarn Resource Manager HA > - > > Key: KNOX-719 > URL: https://issues.apache.org/jira/browse/KNOX-719 > Project: Apache Knox > Issue Type: Bug >Affects Versions: 0.6.0, 0.7.0, 0.8.0, 0.9.0, 0.10.0, 0.11.0 >Reporter: Jeffrey E Rodriguez >Assignee: Jeffrey E Rodriguez > Fix For: 0.12.0 > > Attachments: KNOX-719-4.patch, KNOX_719_5.patch > > > This would support both REST/UI YARN Resource Manager HA. Based on other HA > providers added in Knox. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KNOX-719) Knox support for Yarn Resource Manager HA
[ https://issues.apache.org/jira/browse/KNOX-719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889422#comment-15889422 ] Sumit Gupta commented on KNOX-719: -- Thanks [~jeffreyr97]! I think the changes are good enough for now. We'll file follow up JIRAs for improvement. I did make some additional changes to the one of the tests and then associated changes for a fix. I've attached the final patch (number 5) which is what I pushed in. The crux of the changes I made in addition to your changes were related to book-keeping that the ha provider does for what the 'active' URL is. I changed your test to test out 3 URLs instead of two for HA and that is where I discovered the bug. I added a way to set the active url on the HA provider where previously you could only mark a failed URL and an algorithm would figure out the next active URL. > Knox support for Yarn Resource Manager HA > - > > Key: KNOX-719 > URL: https://issues.apache.org/jira/browse/KNOX-719 > Project: Apache Knox > Issue Type: Bug >Affects Versions: 0.6.0, 0.7.0, 0.8.0, 0.9.0, 0.10.0, 0.11.0 >Reporter: Jeffrey E Rodriguez >Assignee: Jeffrey E Rodriguez > Fix For: 0.12.0 > > Attachments: KNOX-719-4.patch, KNOX_719_5.patch > > > This would support both REST/UI YARN Resource Manager HA. Based on other HA > providers added in Knox. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-719) Knox support for Yarn Resource Manager HA
[ https://issues.apache.org/jira/browse/KNOX-719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-719: - Attachment: KNOX_719_5.patch > Knox support for Yarn Resource Manager HA > - > > Key: KNOX-719 > URL: https://issues.apache.org/jira/browse/KNOX-719 > Project: Apache Knox > Issue Type: Bug >Affects Versions: 0.6.0, 0.7.0, 0.8.0, 0.9.0, 0.10.0, 0.11.0 >Reporter: Jeffrey E Rodriguez >Assignee: Jeffrey E Rodriguez > Fix For: 0.12.0 > > Attachments: KNOX-719-4.patch, KNOX_719_5.patch > > > This would support both REST/UI YARN Resource Manager HA. Based on other HA > providers added in Knox. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KNOX-719) Knox support for Yarn Resource Manager HA
[ https://issues.apache.org/jira/browse/KNOX-719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885963#comment-15885963 ] Sumit Gupta commented on KNOX-719: -- Thanks for the update [~jeffreyr97]! The patch looks much better and I tested it against a HA setup and it seems to work well. Just a couple of last nitpicks before I can push in the code: 1. There is an extraneous class RMUIHaHttpClientDispatch. Can this be removed? 2. There should be an opportunity to share some code between the various HA dispatches. There is a a lot of duplicate code that generally makes me uncomfortable. 3. There is an extra executeRequest method in the new Dispatch classes that is commented as a test method. Is this needed? I'm also uncomfortable with test only methods in code. > Knox support for Yarn Resource Manager HA > - > > Key: KNOX-719 > URL: https://issues.apache.org/jira/browse/KNOX-719 > Project: Apache Knox > Issue Type: Bug >Affects Versions: 0.6.0, 0.7.0, 0.8.0, 0.9.0, 0.10.0, 0.11.0 >Reporter: Jeffrey E Rodriguez >Assignee: Jeffrey E Rodriguez > Fix For: 0.12.0 > > Attachments: KNOX-719-3.patch > > > This would support both REST/UI YARN Resource Manager HA. Based on other HA > providers added in Knox. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (KNOX-839) Add Admin UI development documentation
[ https://issues.apache.org/jira/browse/KNOX-839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta resolved KNOX-839. -- Resolution: Fixed > Add Admin UI development documentation > -- > > Key: KNOX-839 > URL: https://issues.apache.org/jira/browse/KNOX-839 > Project: Apache Knox > Issue Type: Improvement >Affects Versions: 0.11.0 >Reporter: Sumit Gupta >Assignee: Sumit Gupta > Fix For: 0.12.0 > > > Add documentation to the wiki and/or README information to allow for > developers to continue improving the Admin UI. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (KNOX-845) Add some tests for the Shell scripts
[ https://issues.apache.org/jira/browse/KNOX-845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta resolved KNOX-845. -- Resolution: Fixed > Add some tests for the Shell scripts > > > Key: KNOX-845 > URL: https://issues.apache.org/jira/browse/KNOX-845 > Project: Apache Knox > Issue Type: Bug > Components: ClientDSL >Reporter: Sumit Gupta >Assignee: Sumit Gupta > Labels: KIP-4 > Fix For: 0.12.0 > > > The KnoxShell code and scripts are missing any unit/functional tests. Some > functional testing based on the MiniDFSCluster methodology (used in the > release tests for secure cluster testing) could be a good starting point. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KNOX-728) Don't encode Jobhistory URLs
[
https://issues.apache.org/jira/browse/KNOX-728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881048#comment-15881048
]
Sumit Gupta commented on KNOX-728:
--
Thanks [~Wancy] for the update. I'm moving this out to 0.13.0 since it is
likely that we don't need Knox changes anyway.
> Don't encode Jobhistory URLs
>
>
> Key: KNOX-728
> URL: https://issues.apache.org/jira/browse/KNOX-728
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
>Affects Versions: 0.9.0
> Environment: Knox 0.9.0, Hadoop 2.7.2
>Reporter: Andrey Kulikov
>Assignee: Shi Wang
> Fix For: 0.13.0
>
>
> I think this problem is similar to KNOX-709. Going to the Jobhistory log page:
> https://hdp-node1:8443/gateway/hadoop-staging/jobhistory/joblogs/hdp-node2:45454/container_e696_1469407938027_0072_01_11/attempt_1469407938027_0072_m_09_0/hadoop
> {noformat}
> 16/07/26 10:21:37
> ||f4b6aeec-289f-4f96-9a81-fcf6df6cf762|audit|JOBHISTORYUI|akulikov|||dispatch|uri|http://hdp-node1:19888/jobhistory/logs/hdp-node2%3A45454/container_e696_1469407938027_0072_01_11/attempt_1469407938027_0072_m_09_0/hadoop/?user.name=akulikov|unavailable|Request
> method: GET
> 16/07/26 10:21:37
> ||f4b6aeec-289f-4f96-9a81-fcf6df6cf762|audit|JOBHISTORYUI|akulikov|||dispatch|uri|http://hdp-node1:19888/jobhistory/logs/hdp-node2%3A45454/container_e696_1469407938027_0072_01_11/attempt_1469407938027_0072_m_09_0/hadoop/?user.name=akulikov|success|Response
> status: 200
> 16/07/26 10:21:37
> ||f4b6aeec-289f-4f96-9a81-fcf6df6cf762|audit|JOBHISTORYUI|akulikov|||access|uri|/gateway/hadoop-staging/jobhistory/joblogs/hdp-node2:45454/container_e696_1469407938027_0072_01_11/attempt_1469407938027_0072_m_09_0/hadoop|success|Response
> status: 200
> {noformat}
> results in the error:
> {noformat}
> Cannot get container logs. Invalid nodeId: hdp-node2%3A45454
> {noformat}
> Jobhistory can't handle encoded paths.
> Knox 0.8.0 doesn't have this problem.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KNOX-751) Need rewrite function to capture href information from single page host to rewrite output of node hosts
[
https://issues.apache.org/jira/browse/KNOX-751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880981#comment-15880981
]
Sumit Gupta commented on KNOX-751:
--
[~jeffreyr97], as we are trying to close down 0.12.0 I wanted to know if you
are going to provide a patch for this or should this be moved out to 0.13.0?
> Need rewrite function to capture href information from single page host to
> rewrite output of node hosts
> ---
>
> Key: KNOX-751
> URL: https://issues.apache.org/jira/browse/KNOX-751
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
>Affects Versions: 0.7.0, 0.8.0, 0.9.0, 0.9.1
> Environment: All environment support Knox
>Reporter: Jeffrey E Rodriguez
>Assignee: Jeffrey E Rodriguez
> Fix For: 0.12.0
>
>
> User Interface page rewriting through Knox presents many challenges. One of
> the challenges is that a Web page may have more than one link to other hosts
> of a given type (e.g. Yarn may have a link to more than one host) which may
> be different from the originating one. The main issue here is that Knox maps
> a single host to a rewrite and service. If the page has multiple hosts of a
> single type the rewrite rules may not be enough without some extra
> information. For example, Yarn UI may have links to Node URLs (more than one
> node) and even through the Yarn page may be rewritten to include the schema,
> and hostname as a query parameters so the dispatcher is able to fetch the
> correct host like:
> https://localhost:8443/gateway/nodeui/node?scheme=http?host=sandbox1?port=8042
> https://localhost:8443/gateway/nodeui/node?scheme=http?host=sandbox2?port=8042
> https://localhost:8443/gateway/nodeui/node?scheme=http?host=sandbox3?port=8042
> the response pages from each node need to have the right scheme, host, and
> port attached to their query parameters. We need a function that given a
> request to one of the links above can save the information needed to rewrite
> their request output with the query parameters needed for every response
> links.
> e.g.
> if we click link
> http://localhost:8443/gateway/nodeui/node?scheme=http?host=sandbox2?port=8042
> this can be rewritten as:
> “/node/node” to
> https://localhost:8443/gateway/nodeui/node/node?scheme=http?host=sandbox2?port=8042
> These functions are needed:
> $hrefUrl.ur
> $hrefUrl.host
> $hrefUrl.scheme
> $hrefUrl.port
> $hrefUrl.path
> $hrefUrl.query
> The rewrite would then be something like this:
> {code:xml}
>
>
> template="{$gateway[url]}/node/node?scheme={hrefUrl.scheme}?host={hrefUrl.host}?port={hrefUrl.port}?{**}”
>
> {code}
> This way then we are not only able to request the page as we do today but the
> output response would be rewritten to have the information needed to be
> accessed.
> There is a KNOX-618 ( patch) but after evaluation we think that relying on
> the “Host” header still doesn’t provide all the information needed such as
> the scheme. We have access to the request object so it may be easier to get
> the information directly from the href request than expecting a header. The
> “Host” header see https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
> section 14.23, only defines host and port. There is no reliable way to get
> the scheme from the response headers. Thus we can't really make use of the
> patch since we would still need the function to provide us the scheme
> function. Also there is still more information we may want such the path and
> query but those as not as critical to our effort to map all links from Yarn
> UI at this point.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KNOX-789) Apache Atlas REST API support
[ https://issues.apache.org/jira/browse/KNOX-789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880979#comment-15880979 ] Sumit Gupta commented on KNOX-789: -- [~jeffreyr97], as we are trying to close down 0.12.0 I wanted to know if you are going to provide a patch for this or should this be moved out to 0.13.0? > Apache Atlas REST API support > - > > Key: KNOX-789 > URL: https://issues.apache.org/jira/browse/KNOX-789 > Project: Apache Knox > Issue Type: New Feature > Environment: all >Reporter: Jeffrey E Rodriguez >Assignee: Jeffrey E Rodriguez > Fix For: 0.12.0 > > > Apache REST API support through Knox > https://atlas.incubator.apache.org/api/rest.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KNOX-788) Apache Atlas Admin UI Support through Knox
[ https://issues.apache.org/jira/browse/KNOX-788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880978#comment-15880978 ] Sumit Gupta commented on KNOX-788: -- [~jeffreyr97], as we are trying to close down 0.12.0 I wanted to know if you are going to provide a patch for this or should this be moved out to 0.13.0? > Apache Atlas Admin UI Support through Knox > -- > > Key: KNOX-788 > URL: https://issues.apache.org/jira/browse/KNOX-788 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeffrey E Rodriguez >Assignee: Jeffrey E Rodriguez > Fix For: 0.12.0 > > > Apache Atlas Admin UI Support through Knox. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-882) Bind KnoxTokens to the Requesting Clients
[ https://issues.apache.org/jira/browse/KNOX-882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-882: - Fix Version/s: (was: 0.12.0) 0.13.0 > Bind KnoxTokens to the Requesting Clients > - > > Key: KNOX-882 > URL: https://issues.apache.org/jira/browse/KNOX-882 > Project: Apache Knox > Issue Type: Bug > Components: Server >Reporter: Larry McCay >Assignee: Larry McCay > Labels: kip-4 > Fix For: 0.13.0 > > > When issuing the KnoxToken, the requesting client IP address should be added > to the resulting token. This IP address will then need to be validated > against the IP address of any incoming request that presents the bearer token > as proof of identity. > This will prevent the misappropriation of a token from allowing access from > any other machine. > We will also want to make this binding requirement configurable and provide > appropriate warning messages when not in use. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-793) fail to put when the filesize equal 8g
[ https://issues.apache.org/jira/browse/KNOX-793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-793: - Fix Version/s: (was: 0.12.0) 0.13.0 > fail to put when the filesize equal 8g > --- > > Key: KNOX-793 > URL: https://issues.apache.org/jira/browse/KNOX-793 > Project: Apache Knox > Issue Type: Bug > Components: Server >Affects Versions: 0.9.1 >Reporter: linwaterbin > Fix For: 0.13.0 > > > when I put a file that size over 8g must fail,but when I test other item as > following: > 1、the filesize is 7.9g，will success > 2、the filesize is 8.1g，will success > 3、the filesize is 8g，will fail > my command is: > curl -i -k -u root:123 -L -T l_3306_20161126_041435_xtra.split.0 -X PUT > "https://1.1.5.3:8080/db/st/webhdfs/v1/IED/l_3306_20161126_041435_xtra.split.0?op=CREATE; > the 8g is a magic size，anyone can help me，thanks before :) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-761) KnoxSSO Needs to Support Multi-tenant Usecases
[ https://issues.apache.org/jira/browse/KNOX-761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-761: - Fix Version/s: (was: 0.12.0) 0.13.0 > KnoxSSO Needs to Support Multi-tenant Usecases > -- > > Key: KNOX-761 > URL: https://issues.apache.org/jira/browse/KNOX-761 > Project: Apache Knox > Issue Type: Bug > Components: Server >Reporter: Larry McCay >Assignee: Larry McCay > Fix For: 0.13.0 > > > In a deployment that separates tenant access to Hadoop resources through > dedicated topologies with tenant specific authentication, there are a couple > issues: > * pac4j provider seems to be caching config settings in a singleton which > makes the redirect url nondeterministic. > * knoxsso cookie would be trusted across tenant specific topologies which > could lead to unauthorized access to resources that belongs to another tenant > The use of tenant specific audience claims within the JWT token could be used > to mitigate the cross tenant trust issue. > We need to investigate the pac4j provider issue with the singleton config. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-811) Skip parsing of request/response body if no filters/rules apply.
[ https://issues.apache.org/jira/browse/KNOX-811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-811: - Fix Version/s: (was: 0.12.0) 0.13.0 > Skip parsing of request/response body if no filters/rules apply. > > > Key: KNOX-811 > URL: https://issues.apache.org/jira/browse/KNOX-811 > Project: Apache Knox > Issue Type: Bug >Reporter: Sumit Gupta > Fix For: 0.13.0 > > > This is related to [KNOX-767] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-797) Default Topology Feature Not Working
[
https://issues.apache.org/jira/browse/KNOX-797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Gupta updated KNOX-797:
-
Fix Version/s: (was: 0.12.0)
0.13.0
> Default Topology Feature Not Working
>
>
> Key: KNOX-797
> URL: https://issues.apache.org/jira/browse/KNOX-797
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
>Reporter: Larry McCay
>Assignee: Larry McCay
> Fix For: 0.13.0
>
>
> From Mohammad Islam:
> I set "default.app.topology.name" in gateway-site.xml to "uber" (my default
> topology name).
> It worked fine if I gave the full URL. The command looks like this "curl
> http:///gateway/uber/webhdfs/v1/?op=GETHOMEDIRECTORY'".
>
> However, when I tried with command "curl
> http:///webhdfs/v1/?op=GETHOMEDIRECTORY'". I got the HTTP error
> code 500. I looked into gateway.log file and found quite a few error related
> to rewrite. The exact error messages are shown below:
> Error message
> {noformat}
> 2016-11-30 00:39:51,565 ERROR hadoop.gateway
> (UrlRewriteProcessor.java:rewrite(169)) - Failed to rewrite URL:
> http:///webhdfs/v1/?op=GETHOMEDIRECTORY, direction: IN via rule:
> WEBHDFS/webhdfs/inbound/namenode/root, status: FAILURE
> 2016-11-30 00:39:51,565 ERROR hadoop.gateway
> (UrlRewriteProcessor.java:rewrite(169)) - Failed to rewrite URL:
> http:///webhdfs/v1/?op=GETHOMEDIRECTORY, direction: IN via rule:
> WEBHDFS/webhdfs/inbound/namenode/root, status: FAILURE
> {noformat}
> After that, I modified the webhdfs/2.4.0/rewrite.xml by rewriting the
> following pattern and it worked for short URL but long URL faces the same
> issue.
> Original:
> {noformat}
> pattern="*://*:*/**/webhdfs/{version}/?{**}">
>
>
> {noformat}
> Modified :
> {noformat}
> pattern="*://*:*/webhdfs/{version}/?{**}">
>
>
> {noformat}
>
> Overall, the rewrite pattern may be the issue. We will need to support for
> both short and long URL. May be, we can add multiple rewrite rules for each
> route in service.xml.
> Is there any other cleaner way which may work for all cases such as webhdfs,
> yarn, hive, UIs etc?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (KNOX-834) Admin UI Fails to Redirect to Knox SSO for API Calls
[ https://issues.apache.org/jira/browse/KNOX-834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-834: - Fix Version/s: (was: 0.12.0) 0.13.0 > Admin UI Fails to Redirect to Knox SSO for API Calls > > > Key: KNOX-834 > URL: https://issues.apache.org/jira/browse/KNOX-834 > Project: Apache Knox > Issue Type: Bug > Components: Server >Reporter: Larry McCay > Fix For: 0.13.0 > > > When using KnoxSSO the Admin UI will redirect appropriately to KnoxSSO when > attempting to load the main page either the first time or when the > cookie/token expires. > However, when the cookie/token expires or is removed and a different topology > is selected or a refresh is initiated the KnoxSSO redirect for the API call > is returned into the editor. > We need to detect that a redirect is being requested for the topology call > and force a refresh of the page or do the redirect from js. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-832) Admin UI Should Display Unauthorized Message when 403 is Received via API
[ https://issues.apache.org/jira/browse/KNOX-832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-832: - Fix Version/s: (was: 0.12.0) 0.13.0 > Admin UI Should Display Unauthorized Message when 403 is Received via API > - > > Key: KNOX-832 > URL: https://issues.apache.org/jira/browse/KNOX-832 > Project: Apache Knox > Issue Type: Bug > Components: Server >Reporter: Larry McCay > Fix For: 0.13.0 > > > Logging into the Admin UI as a non-admin user such as guest in the DEMO LDAP > server results in the page being displayed with no topologies listed. > This should include an Unauthorized error message. > Additionally, we need to see whether authorization is being enforced on the > application itself and not just the API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-840) Admin UI add ability to create a new topology based on a template
[ https://issues.apache.org/jira/browse/KNOX-840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-840: - Fix Version/s: (was: 0.12.0) 0.13.0 > Admin UI add ability to create a new topology based on a template > - > > Key: KNOX-840 > URL: https://issues.apache.org/jira/browse/KNOX-840 > Project: Apache Knox > Issue Type: Improvement > Components: AdminUI >Reporter: Sumit Gupta >Assignee: Sumit Gupta > Fix For: 0.13.0 > > > Admin user should be able to use the Admin UI to create new topologies based > on one of the template files that we ship with Knox. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-727) Authorization Support for Knox Hosted Applications
[ https://issues.apache.org/jira/browse/KNOX-727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-727: - Fix Version/s: (was: 0.12.0) 0.13.0 > Authorization Support for Knox Hosted Applications > -- > > Key: KNOX-727 > URL: https://issues.apache.org/jira/browse/KNOX-727 > Project: Apache Knox > Issue Type: Bug > Components: Server >Affects Versions: 0.9.0 >Reporter: Zachary Blanco >Assignee: Zachary Blanco > Fix For: 0.13.0 > > Attachments: admin.xml, knoxsso.xml > > > In the process of making an Administrator UI for the Knox, I've encountered > an issue where we can log into the app as an unauthorized user, but then fail > to make any AJAX requests. The Ajax requests return a 403 - which is probably > what should happen when logging into the app with an unauthorized user. > Steps to reproduce: > 1. Set up the Knox admin UI app using the instructions here: > https://github.com/ZacBlanco/knox-admin-ui/blob/master/README.md > 2. Place attached knoxsso and admin topology files under conf/topologies > 3. Navigate to https://www.local.com:8443/gateway/admin/knox-manager > 4. Attempt to login with guest:guest-password > The knox-manager page should render but in the dev console you should see > 403-Forbidden on the Ajax requests. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-784) java.lang.IllegalArgumentException: A metric named org.apache.http.conn.HttpClientConnectionManager.available-connections already exists
[ https://issues.apache.org/jira/browse/KNOX-784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-784: - Fix Version/s: (was: 0.12.0) 0.13.0 > java.lang.IllegalArgumentException: A metric named > org.apache.http.conn.HttpClientConnectionManager.available-connections > already exists > - > > Key: KNOX-784 > URL: https://issues.apache.org/jira/browse/KNOX-784 > Project: Apache Knox > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Sumit Gupta > Fix For: 0.13.0 > > > Facing this error on latest trunk version - > Caused by: java.lang.IllegalArgumentException: A metric named > org.apache.http.conn.HttpClientConnectionManager.available-connections > already exists > at > com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:91) > at > com.codahale.metrics.httpclient.InstrumentedHttpClientConnectionManager.(InstrumentedHttpClientConnectionManager.java:63) > at > com.codahale.metrics.httpclient.InstrumentedHttpClientConnectionManager.(InstrumentedHttpClientConnectionManager.java:49) > at > com.codahale.metrics.httpclient.InstrumentedHttpClientConnectionManager.(InstrumentedHttpClientConnectionManager.java:41) > at > com.codahale.metrics.httpclient.InstrumentedHttpClientConnectionManager.(InstrumentedHttpClientConnectionManager.java:36) > at > org.apache.hadoop.gateway.services.metrics.impl.instr.InstrHttpClientBuilderProvider.getInstrumented(InstrHttpClientBuilderProvider.java:41) > at > org.apache.hadoop.gateway.services.metrics.impl.instr.InstrHttpClientBuilderProvider.getInstrumented(InstrHttpClientBuilderProvider.java:36) > at > org.apache.hadoop.gateway.services.metrics.impl.DefaultMetricsService.getInstrumented(DefaultMetricsService.java:128) > at > org.apache.hadoop.gateway.dispatch.DefaultHttpClientFactory.createHttpClient(DefaultHttpClientFactory.java:67) > at > org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter.init(GatewayDispatchFilter.java:75) > at > org.apache.hadoop.gateway.GatewayFilter$Holder.getInstance(GatewayFilter.java:362) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (KNOX-874) Update hadoop dependency in root pom
[ https://issues.apache.org/jira/browse/KNOX-874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta resolved KNOX-874. -- Resolution: Fixed > Update hadoop dependency in root pom > > > Key: KNOX-874 > URL: https://issues.apache.org/jira/browse/KNOX-874 > Project: Apache Knox > Issue Type: Bug > Components: Build >Reporter: Sumit Gupta >Assignee: Sumit Gupta > Fix For: 0.12.0 > > > We are a bit behind in our hadoop dependency. Now that we have added hadoop > group lookup we should stay closer to the latest release. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KNOX-719) Knox support for Yarn Resource Manager HA
[ https://issues.apache.org/jira/browse/KNOX-719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868414#comment-15868414 ] Sumit Gupta commented on KNOX-719: -- Thanks [~jeffreyr97], The new patch seems to be the same as the old one. I'll move around the classes manually for now, but before I do a deep dive into the review I have a few high level observations: 1. I see some new classes that are marked as @Deprecated. What is the purpose of these? 2. Does this also require some changes to the service.xml file so that the correct classes are used? 3. The main pom.xml needs to change to be able to add in the new module. > Knox support for Yarn Resource Manager HA > - > > Key: KNOX-719 > URL: https://issues.apache.org/jira/browse/KNOX-719 > Project: Apache Knox > Issue Type: Bug >Affects Versions: 0.6.0, 0.7.0, 0.8.0, 0.9.0, 0.10.0, 0.11.0 >Reporter: Jeffrey E Rodriguez >Assignee: Jeffrey E Rodriguez > Fix For: 0.12.0 > > Attachments: KNOX-719-1.patch > > > This would support both REST/UI YARN Resource Manager HA. Based on other HA > providers added in Knox. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KNOX-719) Knox support for Yarn Resource Manager HA
[ https://issues.apache.org/jira/browse/KNOX-719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868094#comment-15868094 ] Sumit Gupta commented on KNOX-719: -- [~jeffreyr97], Also a question about the HA functionality. This looks almost exactly the same as webhdfs HA. My understanding is that in RM HA, there is a redirect issued from a standby node to the active node if the standby node is up and accessed. Are you also handling this scenario? > Knox support for Yarn Resource Manager HA > - > > Key: KNOX-719 > URL: https://issues.apache.org/jira/browse/KNOX-719 > Project: Apache Knox > Issue Type: Bug >Affects Versions: 0.6.0, 0.7.0, 0.8.0, 0.9.0, 0.10.0, 0.11.0 >Reporter: Jeffrey E Rodriguez >Assignee: Jeffrey E Rodriguez > Fix For: 0.12.0 > > Attachments: KNOX-719.patch > > > This would support both REST/UI YARN Resource Manager HA. Based on other HA > providers added in Knox. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KNOX-719) Knox support for Yarn Resource Manager HA
[ https://issues.apache.org/jira/browse/KNOX-719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868039#comment-15868039 ] Sumit Gupta commented on KNOX-719: -- [~jeffreyr97], The patch is not applying correctly. You may have not created it in the right spot? The new module you created is fine but the files land up somewhere else. Probably because of this : gateway-service-rm/pom.xml | 81 + .../hadoop/gateway/rm/dispatch/RMDispatch.java | 35 .../hadoop/gateway/rm/dispatch/RMHaDispatch.java | 185 + .../rm/dispatch/RMHaHttpClientDispatch.java| 36 .../gateway/rm/dispatch/RMHttpClientDispatch.java | 45 + .../hadoop/gateway/rm/dispatch/RMUIHaDispatch.java | 185 + .../rm/dispatch/RMUIHaHttpClientDispatch.java | 33 .../rm/dispatch/RMUIHttpClientDispatch.java| 45 + .../gateway/rm/dispatch/SafeModeException.java | 21 +++ .../gateway/rm/dispatch/StandbyException.java | 21 +++ .../apache/hadoop/gateway/rm/i18n/RMMessages.java | 43 + .../gateway/rm/dispatch/RMHaDispatchTest.java | 113 + > Knox support for Yarn Resource Manager HA > - > > Key: KNOX-719 > URL: https://issues.apache.org/jira/browse/KNOX-719 > Project: Apache Knox > Issue Type: Bug >Affects Versions: 0.6.0, 0.7.0, 0.8.0, 0.9.0, 0.10.0, 0.11.0 >Reporter: Jeffrey E Rodriguez >Assignee: Jeffrey E Rodriguez > Fix For: 0.12.0 > > Attachments: KNOX-719.patch > > > This would support both REST/UI YARN Resource Manager HA. Based on other HA > providers added in Knox. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KNOX-719) Knox support for Yarn Resource Manager HA
[ https://issues.apache.org/jira/browse/KNOX-719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867990#comment-15867990 ] Sumit Gupta commented on KNOX-719: -- This would be a great addition [~jeffreyr97]. I'll take a look at the patch. > Knox support for Yarn Resource Manager HA > - > > Key: KNOX-719 > URL: https://issues.apache.org/jira/browse/KNOX-719 > Project: Apache Knox > Issue Type: Bug >Affects Versions: 0.6.0, 0.7.0, 0.8.0, 0.9.0, 0.10.0, 0.11.0 >Reporter: Jeffrey E Rodriguez >Assignee: Jeffrey E Rodriguez > Fix For: 0.12.0 > > Attachments: KNOX-719.patch > > > This would support both REST/UI YARN Resource Manager HA. Based on other HA > providers added in Knox. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KNOX-728) Don't encode Jobhistory URLs
[
https://issues.apache.org/jira/browse/KNOX-728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867937#comment-15867937
]
Sumit Gupta commented on KNOX-728:
--
[~Wancy]. we did switch to always encoding URLs. We had a lot of issues with
special characters in URLs for some services. Specifically when we did it we
were solving for various URL issues in Ambari. So when you are looking at this,
please make sure you don't break that functionality and also I would double
check why jobhistory can't decode URLs.
> Don't encode Jobhistory URLs
>
>
> Key: KNOX-728
> URL: https://issues.apache.org/jira/browse/KNOX-728
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
>Affects Versions: 0.9.0
> Environment: Knox 0.9.0, Hadoop 2.7.2
>Reporter: Andrey Kulikov
>Assignee: Shi Wang
>
> I think this problem is similar to KNOX-709. Going to the Jobhistory log page:
> https://hdp-node1:8443/gateway/hadoop-staging/jobhistory/joblogs/hdp-node2:45454/container_e696_1469407938027_0072_01_11/attempt_1469407938027_0072_m_09_0/hadoop
> {noformat}
> 16/07/26 10:21:37
> ||f4b6aeec-289f-4f96-9a81-fcf6df6cf762|audit|JOBHISTORYUI|akulikov|||dispatch|uri|http://hdp-node1:19888/jobhistory/logs/hdp-node2%3A45454/container_e696_1469407938027_0072_01_11/attempt_1469407938027_0072_m_09_0/hadoop/?user.name=akulikov|unavailable|Request
> method: GET
> 16/07/26 10:21:37
> ||f4b6aeec-289f-4f96-9a81-fcf6df6cf762|audit|JOBHISTORYUI|akulikov|||dispatch|uri|http://hdp-node1:19888/jobhistory/logs/hdp-node2%3A45454/container_e696_1469407938027_0072_01_11/attempt_1469407938027_0072_m_09_0/hadoop/?user.name=akulikov|success|Response
> status: 200
> 16/07/26 10:21:37
> ||f4b6aeec-289f-4f96-9a81-fcf6df6cf762|audit|JOBHISTORYUI|akulikov|||access|uri|/gateway/hadoop-staging/jobhistory/joblogs/hdp-node2:45454/container_e696_1469407938027_0072_01_11/attempt_1469407938027_0072_m_09_0/hadoop|success|Response
> status: 200
> {noformat}
> results in the error:
> {noformat}
> Cannot get container logs. Invalid nodeId: hdp-node2%3A45454
> {noformat}
> Jobhistory can't handle encoded paths.
> Knox 0.8.0 doesn't have this problem.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (KNOX-848) Support for Gremlin Server REST
[ https://issues.apache.org/jira/browse/KNOX-848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-848: - Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the contribution [~Wancy]! Please note that I changed the service role to 'GREMLIN' from 'GREMLINREST'. > Support for Gremlin Server REST > --- > > Key: KNOX-848 > URL: https://issues.apache.org/jira/browse/KNOX-848 > Project: Apache Knox > Issue Type: New Feature >Affects Versions: 0.11.0 >Reporter: Shi Wang >Assignee: Shi Wang > Fix For: 0.12.0 > > Attachments: 0001-KNOX-848-Support-for-Apache-Titan-REST.patch, > 0001-KNOX-848-Support-for-Gremlin-Server-REST.patch, > 0001-KNOX-848-Support-for-Titan-REST.patch > > > Support Apache Titan 1.0.0 REST go through knox. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (KNOX-865) Add a release module for KnoxShell
[ https://issues.apache.org/jira/browse/KNOX-865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta resolved KNOX-865. -- Resolution: Fixed > Add a release module for KnoxShell > -- > > Key: KNOX-865 > URL: https://issues.apache.org/jira/browse/KNOX-865 > Project: Apache Knox > Issue Type: Improvement >Reporter: Sumit Gupta >Assignee: Sumit Gupta > Fix For: 0.12.0 > > > Create a release module for KnoxShell so that it can be downloaded and run > independently of the gateway installation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KNOX-848) Support for Titan REST
[ https://issues.apache.org/jira/browse/KNOX-848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15864359#comment-15864359 ] Sumit Gupta commented on KNOX-848: -- [~Wancy], I tried out the docker environment you provided and the followed the curl commands listed. They seem to work fine. Please let me know if you will be providing an updated patch with the new names or if you want me to change the names. Thanks again for the contribution! > Support for Titan REST > -- > > Key: KNOX-848 > URL: https://issues.apache.org/jira/browse/KNOX-848 > Project: Apache Knox > Issue Type: New Feature >Affects Versions: 0.11.0 >Reporter: Shi Wang >Assignee: Shi Wang > Fix For: 0.12.0 > > Attachments: 0001-KNOX-848-Support-for-Apache-Titan-REST.patch, > 0001-KNOX-848-Support-for-Titan-REST.patch > > > Support Apache Titan 1.0.0 REST go through knox. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (KNOX-874) Update hadoop dependency in root pom
Sumit Gupta created KNOX-874: Summary: Update hadoop dependency in root pom Key: KNOX-874 URL: https://issues.apache.org/jira/browse/KNOX-874 Project: Apache Knox Issue Type: Bug Components: Build Reporter: Sumit Gupta Assignee: Sumit Gupta Fix For: 0.12.0 We are a bit behind in our hadoop dependency. Now that we have added hadoop group lookup we should stay closer to the latest release. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (KNOX-865) Add a release module for KnoxShell
[ https://issues.apache.org/jira/browse/KNOX-865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta reassigned KNOX-865: Assignee: Sumit Gupta > Add a release module for KnoxShell > -- > > Key: KNOX-865 > URL: https://issues.apache.org/jira/browse/KNOX-865 > Project: Apache Knox > Issue Type: Improvement >Reporter: Sumit Gupta >Assignee: Sumit Gupta > Fix For: 0.12.0 > > > Create a release module for KnoxShell so that it can be downloaded and run > independently of the gateway installation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-864) Knox init scripts are not Upstart compatible
[
https://issues.apache.org/jira/browse/KNOX-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Gupta updated KNOX-864:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Knox init scripts are not Upstart compatible
>
>
> Key: KNOX-864
> URL: https://issues.apache.org/jira/browse/KNOX-864
> Project: Apache Knox
> Issue Type: Improvement
> Components: Server
>Affects Versions: 0.11.0
>Reporter: Attila Kanto
> Fix For: 0.12.0
>
> Attachments: knox-864.patch
>
>
> It is critical that we have a service that can auto-restart during crashes
> and reboots. On Amazon Linux this tasks are done by Upstart.
> By default Upstart will track the life cycle of the first PID that it
> executes in the exec or script stanzas (defined in the Upstart config file),
> however, most Unix services will "daemonize", meaning that they will create a
> new process (using fork(2)) which is a child of the initial process. This is
> what also happens when when gateway.sh or ldap.sh is invoked.
> In order to track the right PID, Upstart must determine the final process ID
> for a job, and in case of daemonized processes it needs to know how many
> times that process will call fork(2).
> Upstart supports the followings:
> * *expect fork*: Upstart will expect the process executed to call fork(2)
> exactly once.
> * *expect daemon*: Upstart will expect the process executed to call fork(2)
> exactly twice
> Unfortunately none of the above cases fits to gateway.sh and ldap.sh, since
> they are calling fork many times and Upstart always tracks the wrong PID.
> According to Upstart doc
> http://upstart.ubuntu.com/cookbook/#how-to-establish-fork-count if the
> application you are attempting to create a Job Configuration File does not
> document how many times it forks, you can run it with a tool such as
> strace(1) which will allow you to count the number of forks:
> {code}
> [root@ip-10-0-4-107 ~]# strace -o /tmp/strace.log -fFv su -c
> "/usr/hdp/current/knox-server/bin/gateway.sh start" knox
> Starting Gateway succeeded with PID 25528.
> [root@ip-10-0-4-107 ~]# sudo egrep "\<(fork|clone)\>\(" /tmp/strace.log | wc
> | awk '{print $1}'
> 86
> {code}
> Ambari had similar issues in the past:
> https://issues.apache.org/jira/browse/AMBARI-14842
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KNOX-864) Knox init scripts are not Upstart compatible
[
https://issues.apache.org/jira/browse/KNOX-864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15850009#comment-15850009
]
Sumit Gupta commented on KNOX-864:
--
Thanks for the patch [~akanto]! We will review this and get the fix pushed in
the 0.12.0 timeframe.
> Knox init scripts are not Upstart compatible
>
>
> Key: KNOX-864
> URL: https://issues.apache.org/jira/browse/KNOX-864
> Project: Apache Knox
> Issue Type: Improvement
> Components: Server
>Affects Versions: 0.11.0
>Reporter: Attila Kanto
> Fix For: 0.12.0
>
> Attachments: knox-864.patch
>
>
> It is critical that we have a service that can auto-restart during crashes
> and reboots. On Amazon Linux this tasks are done by Upstart.
> By default Upstart will track the life cycle of the first PID that it
> executes in the exec or script stanzas (defined in the Upstart config file),
> however, most Unix services will "daemonize", meaning that they will create a
> new process (using fork(2)) which is a child of the initial process. This is
> what also happens when when gateway.sh or ldap.sh is invoked.
> In order to track the right PID, Upstart must determine the final process ID
> for a job, and in case of daemonized processes it needs to know how many
> times that process will call fork(2).
> Upstart supports the followings:
> * *expect fork*: Upstart will expect the process executed to call fork(2)
> exactly once.
> * *expect daemon*: Upstart will expect the process executed to call fork(2)
> exactly twice
> Unfortunately none of the above cases fits to gateway.sh and ldap.sh, since
> they are calling fork many times and Upstart always tracks the wrong PID.
> According to Upstart doc
> http://upstart.ubuntu.com/cookbook/#how-to-establish-fork-count if the
> application you are attempting to create a Job Configuration File does not
> document how many times it forks, you can run it with a tool such as
> strace(1) which will allow you to count the number of forks:
> {code}
> [root@ip-10-0-4-107 ~]# strace -o /tmp/strace.log -fFv su -c
> "/usr/hdp/current/knox-server/bin/gateway.sh start" knox
> Starting Gateway succeeded with PID 25528.
> [root@ip-10-0-4-107 ~]# sudo egrep "\<(fork|clone)\>\(" /tmp/strace.log | wc
> | awk '{print $1}'
> 86
> {code}
> Ambari had similar issues in the past:
> https://issues.apache.org/jira/browse/AMBARI-14842
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (KNOX-864) Knox init scripts are not Upstart compatible
[
https://issues.apache.org/jira/browse/KNOX-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Gupta updated KNOX-864:
-
Fix Version/s: 0.12.0
> Knox init scripts are not Upstart compatible
>
>
> Key: KNOX-864
> URL: https://issues.apache.org/jira/browse/KNOX-864
> Project: Apache Knox
> Issue Type: Improvement
> Components: Server
>Affects Versions: 0.11.0
>Reporter: Attila Kanto
> Fix For: 0.12.0
>
> Attachments: knox-864.patch
>
>
> It is critical that we have a service that can auto-restart during crashes
> and reboots. On Amazon Linux this tasks are done by Upstart.
> By default Upstart will track the life cycle of the first PID that it
> executes in the exec or script stanzas (defined in the Upstart config file),
> however, most Unix services will "daemonize", meaning that they will create a
> new process (using fork(2)) which is a child of the initial process. This is
> what also happens when when gateway.sh or ldap.sh is invoked.
> In order to track the right PID, Upstart must determine the final process ID
> for a job, and in case of daemonized processes it needs to know how many
> times that process will call fork(2).
> Upstart supports the followings:
> * *expect fork*: Upstart will expect the process executed to call fork(2)
> exactly once.
> * *expect daemon*: Upstart will expect the process executed to call fork(2)
> exactly twice
> Unfortunately none of the above cases fits to gateway.sh and ldap.sh, since
> they are calling fork many times and Upstart always tracks the wrong PID.
> According to Upstart doc
> http://upstart.ubuntu.com/cookbook/#how-to-establish-fork-count if the
> application you are attempting to create a Job Configuration File does not
> document how many times it forks, you can run it with a tool such as
> strace(1) which will allow you to count the number of forks:
> {code}
> [root@ip-10-0-4-107 ~]# strace -o /tmp/strace.log -fFv su -c
> "/usr/hdp/current/knox-server/bin/gateway.sh start" knox
> Starting Gateway succeeded with PID 25528.
> [root@ip-10-0-4-107 ~]# sudo egrep "\<(fork|clone)\>\(" /tmp/strace.log | wc
> | awk '{print $1}'
> 86
> {code}
> Ambari had similar issues in the past:
> https://issues.apache.org/jira/browse/AMBARI-14842
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] (KNOX-838) Admin UI Titles Change
[ https://issues.apache.org/jira/browse/KNOX-838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta resolved KNOX-838. -- Resolution: Fixed > Admin UI Titles Change > -- > > Key: KNOX-838 > URL: https://issues.apache.org/jira/browse/KNOX-838 > Project: Apache Knox > Issue Type: Improvement > Components: AdminUI >Reporter: Larry McCay >Assignee: Sumit Gupta >Priority: Trivial > Fix For: 0.12.0 > > > Admin UI page titles: > After some thought, I think that I have titles that might work better: > Topologies > Resource Access Management > Naming.ugh... :) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (KNOX-820) Knox query processing: If the originalUrl value has "ampersand"(&) in it, the value after ampersand is ignored.
[ https://issues.apache.org/jira/browse/KNOX-820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta resolved KNOX-820. -- Resolution: Fixed Fix Version/s: (was: 0.11.0) 0.12.0 > Knox query processing: If the originalUrl value has "ampersand"(&) in it, the > value after ampersand is ignored. > --- > > Key: KNOX-820 > URL: https://issues.apache.org/jira/browse/KNOX-820 > Project: Apache Knox > Issue Type: Bug > Components: Server >Reporter: Larry McCay >Assignee: Sumit Gupta > Fix For: 0.12.0 > > > From Sharmadha Sainath offline: > Example URL is: > https://ctr-e64-1480733150053-8567-01-02.hwx.site:8443/gateway/knoxsso/knoxauth/login.html?originalUrl=http://ctr-e64-1480733150053-8567-01-02.hwx.site:21000/api/atlas/discovery/search/dsl?limit=25=hive_table > After providing the username and password, the above page gets redirected to > "ctr-e64-1480733150053-8567-01-02.hwx.site:21000/api/atlas/discovery/search/dsl?limit=25", > leaving "query=hive_table". This seems to be an issue with knox query > processing. > Original query: > http://ctr-e64-1480733150053-8567-01-02.hwx.site:21000/api/atlas/discovery/search/dsl?limit=25=hive_table > Redirects to > https://ctr-e64-1480733150053-8567-01-02.hwx.site:8443/gateway/knoxsso/knoxauth/login.html?originalUrl=http://ctr-e64-1480733150053-8567-01-02.hwx.site:21000/api/atlas/discovery/search/dsl?limit=25=hive_table > Enclosed the URL > http://ctr-e64-1480733150053-8567-01-02.hwx.site:21000/api/atlas/discovery/search/dsl?limit=25=hive_table > with double quotes and encoded it and gave it as original URL . Still it > doesn't work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (KNOX-820) Knox query processing: If the originalUrl value has "ampersand"(&) in it, the value after ampersand is ignored.
[ https://issues.apache.org/jira/browse/KNOX-820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta reopened KNOX-820: -- Assignee: Sumit Gupta (was: Larry McCay) Found an additional minor issue, so reopening this bug > Knox query processing: If the originalUrl value has "ampersand"(&) in it, the > value after ampersand is ignored. > --- > > Key: KNOX-820 > URL: https://issues.apache.org/jira/browse/KNOX-820 > Project: Apache Knox > Issue Type: Bug > Components: Server >Reporter: Larry McCay >Assignee: Sumit Gupta > Fix For: 0.11.0 > > > From Sharmadha Sainath offline: > Example URL is: > https://ctr-e64-1480733150053-8567-01-02.hwx.site:8443/gateway/knoxsso/knoxauth/login.html?originalUrl=http://ctr-e64-1480733150053-8567-01-02.hwx.site:21000/api/atlas/discovery/search/dsl?limit=25=hive_table > After providing the username and password, the above page gets redirected to > "ctr-e64-1480733150053-8567-01-02.hwx.site:21000/api/atlas/discovery/search/dsl?limit=25", > leaving "query=hive_table". This seems to be an issue with knox query > processing. > Original query: > http://ctr-e64-1480733150053-8567-01-02.hwx.site:21000/api/atlas/discovery/search/dsl?limit=25=hive_table > Redirects to > https://ctr-e64-1480733150053-8567-01-02.hwx.site:8443/gateway/knoxsso/knoxauth/login.html?originalUrl=http://ctr-e64-1480733150053-8567-01-02.hwx.site:21000/api/atlas/discovery/search/dsl?limit=25=hive_table > Enclosed the URL > http://ctr-e64-1480733150053-8567-01-02.hwx.site:21000/api/atlas/discovery/search/dsl?limit=25=hive_table > with double quotes and encoded it and gave it as original URL . Still it > doesn't work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KNOX-848) Support for Apache Titan REST
[ https://issues.apache.org/jira/browse/KNOX-848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839897#comment-15839897 ] Sumit Gupta commented on KNOX-848: -- [~Wancy], I got an error trying to pull your docker image. Here is what I did: $ docker pull wancy/knox_titanrest_testing Using default tag: latest Error response from daemon: manifest for wancy/knox_titanrest_testing:latest not found Is that the right one? > Support for Apache Titan REST > - > > Key: KNOX-848 > URL: https://issues.apache.org/jira/browse/KNOX-848 > Project: Apache Knox > Issue Type: New Feature >Affects Versions: 0.11.0 >Reporter: Shi Wang >Assignee: Shi Wang > Fix For: 0.12.0 > > Attachments: 0001-KNOX-848-Support-for-Apache-Titan-REST.patch > > > Support Apache Titan 1.0.0 REST go through knox. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KNOX-854) Consolidate DocumentBuilderFactory calls + enable SECURE_PROCESSING
[ https://issues.apache.org/jira/browse/KNOX-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-854: - Resolution: Fixed Status: Resolved (was: Patch Available) Thanks [~coheig]! This was some good cleanup and much needed refactoring. > Consolidate DocumentBuilderFactory calls + enable SECURE_PROCESSING > --- > > Key: KNOX-854 > URL: https://issues.apache.org/jira/browse/KNOX-854 > Project: Apache Knox > Issue Type: Improvement >Affects Versions: 0.11.0 >Reporter: Colm O hEigeartaigh >Assignee: Colm O hEigeartaigh >Priority: Minor > Fix For: 0.12.0 > > Attachments: > 0001-KNOX-854-Consolidate-DocumentBuilderFactory-calls-en.patch > > > This task is to consolidate DocumentBuilderFactory calls into a single class > (XmlUtils), in order to remove duplicate code. This task also encompasses > enabling the SECURE_PROCESSING feature when parsing XML, which is generally a > "good thing to do". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KNOX-848) Support for Apache Titan REST
[ https://issues.apache.org/jira/browse/KNOX-848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832572#comment-15832572 ] Sumit Gupta commented on KNOX-848: -- Thanks for the notice [~Wancy], please let me know when I can try your new repo. > Support for Apache Titan REST > - > > Key: KNOX-848 > URL: https://issues.apache.org/jira/browse/KNOX-848 > Project: Apache Knox > Issue Type: New Feature >Affects Versions: 0.11.0 >Reporter: Shi Wang >Assignee: Shi Wang > Fix For: 0.12.0 > > Attachments: 0001-KNOX-848-Support-for-Apache-Titan-REST.patch > > > Support Apache Titan 1.0.0 REST go through knox. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KNOX-848) Support for Apache Titan REST
[ https://issues.apache.org/jira/browse/KNOX-848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832291#comment-15832291 ] Sumit Gupta commented on KNOX-848: -- Thanks [~Wancy], I'll try it out and let you know. That may be enough... > Support for Apache Titan REST > - > > Key: KNOX-848 > URL: https://issues.apache.org/jira/browse/KNOX-848 > Project: Apache Knox > Issue Type: New Feature >Affects Versions: 0.11.0 >Reporter: Shi Wang >Assignee: Shi Wang > Fix For: 0.12.0 > > Attachments: 0001-KNOX-848-Support-for-Apache-Titan-REST.patch > > > Support Apache Titan 1.0.0 REST go through knox. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KNOX-858) Simplify Hamcrest dependencies
[ https://issues.apache.org/jira/browse/KNOX-858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-858: - Resolution: Fixed Status: Resolved (was: Patch Available) > Simplify Hamcrest dependencies > -- > > Key: KNOX-858 > URL: https://issues.apache.org/jira/browse/KNOX-858 > Project: Apache Knox > Issue Type: Improvement >Reporter: Colm O hEigeartaigh >Assignee: Colm O hEigeartaigh >Priority: Trivial > Fix For: 0.12.0 > > Attachments: 0001-KNOX-858-Simplify-Hamcrest-dependencies.patch > > > The Hamcrest dependencies could be simplified: > a) The Hamcrest "api" module in the root pom is not needed > b) There is no need to specify the "core" module when the library module is > included as a "test" dependency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KNOX-848) Support for Apache Titan REST
[ https://issues.apache.org/jira/browse/KNOX-848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831973#comment-15831973 ] Sumit Gupta commented on KNOX-848: -- [~Wancy], any update on this. I would love to test this out and get it committed. What kind of KnoxShell JIRA are you looking for? There have been a bunch recently. For example [KNOX-828]. Does that help or do you want something specific? > Support for Apache Titan REST > - > > Key: KNOX-848 > URL: https://issues.apache.org/jira/browse/KNOX-848 > Project: Apache Knox > Issue Type: New Feature >Affects Versions: 0.11.0 >Reporter: Shi Wang >Assignee: Shi Wang > Fix For: 0.12.0 > > Attachments: 0001-KNOX-848-Support-for-Apache-Titan-REST.patch > > > Support Apache Titan 1.0.0 REST go through knox. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KNOX-857) Incorrect formation of causes oozie mapreduce action failure
[
https://issues.apache.org/jira/browse/KNOX-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15830377#comment-15830377
]
Sumit Gupta commented on KNOX-857:
--
[~kuldeepkulkarn...@gmail.com], my understanding is that the ${outputDir}
property is meant to be used for an hdfs directory. Knox provides this as a
convenience and rewrites the path to work as an hdfs path. As you have
determined, you don't have to use this in your workflow.xml file.
> Incorrect formation of causes oozie mapreduce action failure
> --
>
> Key: KNOX-857
> URL: https://issues.apache.org/jira/browse/KNOX-857
> Project: Apache Knox
> Issue Type: Bug
> Components: KnoxCLI
>Affects Versions: 0.6.0
> Environment: HDP-2.4.0.0
>Reporter: Kuldeep Kulkarni
>Priority: Critical
> Attachments: job.xml.bug, knox.conf.tar.gz, knox.log.tz
>
>
> When Oozie workflow is submitted via Knox containing mapreduce-action. it
> always gets failed with below error:
> {code}
> Heart beat
> {"properties":[{"key":"oozie.launcher.job.id","value":"job_1484748086731_0022","isFinal":false,"resource":"programatically"},{"key":"oozie.job.id","value":"017-170118140025343-oozie-oozi-W","isFinal":false,"resource":"programatically"},{"key":"oozie.action.id","value":"017-170118140025343-oozie-oozi-W@mr-node","isFinal":false,"resource":"programatically"},{"key":"mapreduce.job.tags","value":"oozie-20b9b1ecd11f08843b078e701ce5c1ad","isFinal":false,"resource":"programatically"}]}Starting
> the execution of prepare actions
> Prepare execution in the Launcher Mapper has failed
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.MapReduceMain], exception invoking main(),
> java.net.URISyntaxException: Expected scheme-specific part at index 5: hdfs:
> org.apache.oozie.action.hadoop.LauncherException:
> java.net.URISyntaxException: Expected scheme-specific part at index 5: hdfs:
> at
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:193)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> Caused by: org.apache.oozie.action.hadoop.LauncherException:
> java.net.URISyntaxException: Expected scheme-specific part at index 5: hdfs:
> at
> org.apache.oozie.action.hadoop.PrepareActionsDriver.doOperations(PrepareActionsDriver.java:77)
> at
> org.apache.oozie.action.hadoop.LauncherMapper.executePrepare(LauncherMapper.java:494)
> at
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:189)
> ... 8 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Expected scheme-specific part at index 5: hdfs:
> at org.apache.hadoop.fs.Path.initialize(Path.java:205)
> at org.apache.hadoop.fs.Path.(Path.java:171)
> at org.apache.hadoop.fs.Path.(Path.java:93)
> at org.apache.hadoop.fs.Globber.glob(Globber.java:211)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1655)
> at
> org.apache.oozie.action.hadoop.FSLauncherURIHandler.delete(FSLauncherURIHandler.java:59)
> at
> org.apache.oozie.action.hadoop.PrepareActionsDriver.execute(PrepareActionsDriver.java:90)
> at
> org.apache.oozie.action.hadoop.PrepareActionsDriver.doOperations(PrepareActionsDriver.java:68)
> ... 10 more
> Caused by: java.net.URISyntaxException: Expected scheme-specific part at
> index 5: hdfs:
> at java.net.URI$Parser.fail(URI.java:2848)
> at java.net.URI$Parser.failExpecting(URI.java:2854)
> at java.net.URI$Parser.parse(URI.java:3057)
> at java.net.URI.(URI.java:746)
> at org.apache.hadoop.fs.Path.initialize(Path.java:202)
> ... 17 more
> Oozie Launcher failed, finishing Hadoop job gracefully
> {code}
> This happens because, in the action configuration gets
> generated incorrectly.
> {code}
> path="hdfs://oozieknox2.openstacklocal:8020/user/ambari-qa/examples/output-data/hdfs://oozieknox2.openstacklocal:8020/map-reduce"
> />
> {code}
> Expected value is:
> {code}
> path="hdfs://oozieknox2.openstacklocal:8020/user/ambari-qa/examples/output-data/map-reduce"
> />
> {code}
> Command used to submit oozie job is:
> {code}
> [ambari-qa@oozieknox1 ~]$ curl -i -k -u ambari-qa:hadoop -H
> Content-Type:application/xml
[jira] [Updated] (KNOX-806) Implement Closeable for deallocable resources
[
https://issues.apache.org/jira/browse/KNOX-806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Gupta updated KNOX-806:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Implement Closeable for deallocable resources
> -
>
> Key: KNOX-806
> URL: https://issues.apache.org/jira/browse/KNOX-806
> Project: Apache Knox
> Issue Type: Improvement
> Components: ClientDSL
>Reporter: Vincent Devillers
>Assignee: Vincent Devillers
>Priority: Minor
> Labels: KIP-4
> Fix For: 0.12.0
>
> Attachments: KNOX-806.patch
>
>
> org.apache.hadoop.gateway.shell.BasicResponse and
> org.apache.hadoop.gateway.shell.Hadoop are not java.io.Closeable, which means
> that we have to do this:
> {code:java}
> BasicResponse response = null;
> String jobId;
> try {
> response = Workflow.submit(getHadoop()).text(xmlConfiguration).now();
> jobId = JsonPath.read(response.getString(), "$.id");
> } finally {
> if(response != null) {
> response.close();
> }
> }
> {code}
> instead of this:
> {code:java}
> String jobId;
> try (BasicResponse response =
> Workflow.submit(getHadoop()).text(xmlConfiguration).now()){
> jobId = JsonPath.read(response.getString(), "$.id");
> }
> {code}
> [Source|https://blog.layer4.fr/2016/12/06/knox-production-pitfalls-and-common-mistakes/]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KNOX-805) Allow defining a custom HttpClient when creating a new session
[ https://issues.apache.org/jira/browse/KNOX-805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-805: - Resolution: Fixed Status: Resolved (was: Patch Available) Made a minor change to get the insecureLogin to work. > Allow defining a custom HttpClient when creating a new session > -- > > Key: KNOX-805 > URL: https://issues.apache.org/jira/browse/KNOX-805 > Project: Apache Knox > Issue Type: Bug > Components: ClientDSL >Reporter: Vincent Devillers >Assignee: Vincent Devillers > Labels: KIP-4 > Fix For: 0.12.0 > > Attachments: KNOX-805.patch > > > When creating a new session, I think it should be possible to specify a > custom CloseableHttpClient in parameter of the static method > Hadoop.login(url, user,password). This will allow to prepare a well defined > CloaseableHttpClient if you want to add headers, limit the length of the > message send, define specific strategy on dns resolution or hostname > verification, size the connection pool… -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KNOX-805) Allow defining a custom HttpClient when creating a new session
[ https://issues.apache.org/jira/browse/KNOX-805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15818988#comment-15818988 ] Sumit Gupta commented on KNOX-805: -- On trying out the patch I found that loginInsecure does not work anymore. I'll look into a fix and try and commit the changes all together. > Allow defining a custom HttpClient when creating a new session > -- > > Key: KNOX-805 > URL: https://issues.apache.org/jira/browse/KNOX-805 > Project: Apache Knox > Issue Type: Bug > Components: ClientDSL >Reporter: Vincent Devillers >Assignee: Vincent Devillers > Labels: KIP-4 > Fix For: 0.12.0 > > Attachments: KNOX-805.patch > > > When creating a new session, I think it should be possible to specify a > custom CloseableHttpClient in parameter of the static method > Hadoop.login(url, user,password). This will allow to prepare a well defined > CloaseableHttpClient if you want to add headers, limit the length of the > message send, define specific strategy on dns resolution or hostname > verification, size the connection pool… -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KNOX-829) HDFS commands for client: append,checksum,concat,homedir,chown,chmod,touch,symlink,truncate
[
https://issues.apache.org/jira/browse/KNOX-829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812685#comment-15812685
]
Sumit Gupta commented on KNOX-829:
--
[~treydone], I am incurring a bit of a struggle testing this patch. Some
operations do not work for me.
For example, for checksum I get 400 bad request
"{"RemoteException":{"exception":"IllegalArgumentException","javaClassName":"java.lang.IllegalArgumentException","message":"Invalid
value for webhdfs parameter \"op\": No enum constant
org.apache.hadoop.hdfs.web.resources.PostOpParam.Op.GETFILECHECKSUM"}}[\r][\n]"
I think that needs to be a GET and not a POST.
Also for symlink I get,
{"RemoteException":{"exception":"UnsupportedOperationException","javaClassName":"java.lang.UnsupportedOperationException","message":"Symlinks
not supported”}}
Do I need a special setting for this in hdfs?
Append and touch don't seem to do anything for me either. Homedir works great.
Can you help out by providing settings and samples so that I can verify I am
testing this correctly?
> HDFS commands for client:
> append,checksum,concat,homedir,chown,chmod,touch,symlink,truncate
> ---
>
> Key: KNOX-829
> URL: https://issues.apache.org/jira/browse/KNOX-829
> Project: Apache Knox
> Issue Type: Improvement
> Components: ClientDSL
>Reporter: Vincent Devillers
>Assignee: Vincent Devillers
>Priority: Minor
> Labels: KIP-4
> Fix For: 0.12.0
>
> Attachments: KNOX-829.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KNOX-845) Add some tests for the Shell scripts
[ https://issues.apache.org/jira/browse/KNOX-845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-845: - Labels: KIP-4 (was: ) > Add some tests for the Shell scripts > > > Key: KNOX-845 > URL: https://issues.apache.org/jira/browse/KNOX-845 > Project: Apache Knox > Issue Type: Bug > Components: ClientDSL >Reporter: Sumit Gupta >Assignee: Sumit Gupta > Labels: KIP-4 > Fix For: 0.12.0 > > > The KnoxShell code and scripts are missing any unit/functional tests. Some > functional testing based on the MiniDFSCluster methodology (used in the > release tests for secure cluster testing) could be a good starting point. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KNOX-845) Add some tests for the Shell scripts
Sumit Gupta created KNOX-845: Summary: Add some tests for the Shell scripts Key: KNOX-845 URL: https://issues.apache.org/jira/browse/KNOX-845 Project: Apache Knox Issue Type: Bug Components: ClientDSL Reporter: Sumit Gupta Assignee: Sumit Gupta Fix For: 0.12.0 The KnoxShell code and scripts are missing any unit/functional tests. Some functional testing based on the MiniDFSCluster methodology (used in the release tests for secure cluster testing) could be a good starting point. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KNOX-809) Add copyFromLocal along with the put in HDFS for ClientDSL
[ https://issues.apache.org/jira/browse/KNOX-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-809: - Resolution: Fixed Status: Resolved (was: Patch Available) > Add copyFromLocal along with the put in HDFS for ClientDSL > -- > > Key: KNOX-809 > URL: https://issues.apache.org/jira/browse/KNOX-809 > Project: Apache Knox > Issue Type: Improvement > Components: ClientDSL >Reporter: Vincent Devillers >Assignee: Vincent Devillers >Priority: Minor > Labels: KIP-4 > Fix For: 0.12.0 > > Attachments: KNOX-809.patch > > > By default, Knox has the “put” endpoint for HDFS. It copies one or multiple > sources from local file system to the destination file system. The > copyFromLocal is very similar to the put command, except that you have an > option to overwrite the destination if it already exists. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KNOX-810) Add exists in HDFS for ClientDSL
[ https://issues.apache.org/jira/browse/KNOX-810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-810: - Resolution: Fixed Status: Resolved (was: Patch Available) > Add exists in HDFS for ClientDSL > > > Key: KNOX-810 > URL: https://issues.apache.org/jira/browse/KNOX-810 > Project: Apache Knox > Issue Type: Improvement > Components: ClientDSL >Reporter: Vincent Devillers >Assignee: Vincent Devillers >Priority: Minor > Labels: KIP-4 > Fix For: 0.12.0 > > Attachments: KNOX-810.patch > > > The ClientDSL for HDFS provides various methods like "list" or "get", but the > "exists" may be very useful too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KNOX-807) Missing resource deallocation
[
https://issues.apache.org/jira/browse/KNOX-807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802469#comment-15802469
]
Sumit Gupta commented on KNOX-807:
--
[~treydone], adding a close method along with shutdown seemed a bit confusing
to me. The client code will need to know what to call when. Can shutdown not
simply release all resources? The argument I can see is if the user wants to
cancel requests but keep using the session but that seems weird to me too. What
are your thoughts on this?
Any input on this [~lmccay]?
> Missing resource deallocation
> -
>
> Key: KNOX-807
> URL: https://issues.apache.org/jira/browse/KNOX-807
> Project: Apache Knox
> Issue Type: Bug
> Components: ClientDSL
>Reporter: Vincent Devillers
>Assignee: Vincent Devillers
> Labels: KIP-4
> Fix For: 0.12.0
>
> Attachments: KNOX-807.patch
>
>
> h2. Release connection to pool after HTTP call
> The close() method in BasicResponse does only
> EntityUtils.consumeQuietly(response.getEntity()):
> {code}
> public void close() {
> this.consume();
> }
> public void consume() {
> if(!this.consumed) {
> EntityUtils.consumeQuietly(this.response.getEntity());
> this.consumed = true;
> }
> }
> {code}
> The underlying HTTP connection is still held. In order to ensure correct
> deallocation of OS resources we need to call CloseableHttpResponse#close()
> like this:
> {code}
> public void close() {
> try {
> this.consume();
> } finally {
> if (response instanceof CloseableHttpResponse) {
> try {
> ((CloseableHttpResponse) response).close();
> } catch (IOException e) {
> throw Throwables.propagate(e);
> }
> }
> }
> }
> {code}
> h2. Shutdown connection pool
> Also when the Hadoop session is no more used, the only method present by to
> be call is shutdown(), but it close only the ExecutorService. When a
> CloseableHttpClient instance is no longer needed, we have to shut down the
> connection manager to ensure immediate deallocation of all OS resources.
> {code}
> public void close() {
> try {
> executor.shutdownNow();
> } catch(Exception e) {
> // log something here
> }
> try {
> client.close();
> // client.close() should call getConnectionManager().shutdown();
> } catch(Exception e) {
> // log something here
> }
> }
> {code}
> [Source|https://blog.layer4.fr/2016/12/06/knox-production-pitfalls-and-common-mistakes/]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KNOX-813) Add rename in HDFS for ClientDSL
[
https://issues.apache.org/jira/browse/KNOX-813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Gupta updated KNOX-813:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Add rename in HDFS for ClientDSL
>
>
> Key: KNOX-813
> URL: https://issues.apache.org/jira/browse/KNOX-813
> Project: Apache Knox
> Issue Type: Improvement
> Components: ClientDSL
>Affects Versions: 0.7.0
>Reporter: Khanh Maudoux
>Assignee: Vincent Devillers
>Priority: Trivial
> Labels: KIP-4
> Fix For: 0.12.0
>
> Attachments: KNOX-813.patch
>
>
> The ClientDSL for HDFS shoud provide the rename method.
> It could be easily done with something like that :
> {code:java}
> protected Callable callable() {
> return new Callable() {
> @Override
> public Rename.Response call() throws Exception {
> URIBuilder uri = uri( Hdfs.SERVICE_PATH, file );
> addQueryParam( uri, "op", "RENAME" );
> addQueryParam( uri, "destination", to );
> HttpPut request = new HttpPut( uri.build() );
> return new Rename.Response( execute( request ) );
> }
> };
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KNOX-813) Add rename in HDFS for ClientDSL
[
https://issues.apache.org/jira/browse/KNOX-813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802431#comment-15802431
]
Sumit Gupta commented on KNOX-813:
--
Added copyright and removed author tags.
> Add rename in HDFS for ClientDSL
>
>
> Key: KNOX-813
> URL: https://issues.apache.org/jira/browse/KNOX-813
> Project: Apache Knox
> Issue Type: Improvement
> Components: ClientDSL
>Affects Versions: 0.7.0
>Reporter: Khanh Maudoux
>Assignee: Vincent Devillers
>Priority: Trivial
> Labels: KIP-4
> Fix For: 0.12.0
>
> Attachments: KNOX-813.patch
>
>
> The ClientDSL for HDFS shoud provide the rename method.
> It could be easily done with something like that :
> {code:java}
> protected Callable callable() {
> return new Callable() {
> @Override
> public Rename.Response call() throws Exception {
> URIBuilder uri = uri( Hdfs.SERVICE_PATH, file );
> addQueryParam( uri, "op", "RENAME" );
> addQueryParam( uri, "destination", to );
> HttpPut request = new HttpPut( uri.build() );
> return new Rename.Response( execute( request ) );
> }
> };
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KNOX-835) Improvements for Oozie in the ClientDSL
[ https://issues.apache.org/jira/browse/KNOX-835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802352#comment-15802352 ] Sumit Gupta commented on KNOX-835: -- I also get a build and test failure. If possible can you take a look to see if the build passes with this patch in master? > Improvements for Oozie in the ClientDSL > --- > > Key: KNOX-835 > URL: https://issues.apache.org/jira/browse/KNOX-835 > Project: Apache Knox > Issue Type: Improvement > Components: ClientDSL >Reporter: Vincent Devillers >Assignee: Vincent Devillers >Priority: Minor > Labels: KIP-4 > Fix For: 0.12.0 > > Attachments: KNOX-835.patch > > > Implements: list, kill, change, info, definition, rerun, resume, log, start, > dryrun -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KNOX-835) Improvements for Oozie in the ClientDSL
[ https://issues.apache.org/jira/browse/KNOX-835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802323#comment-15802323 ] Sumit Gupta commented on KNOX-835: -- Hey [~treydone], I've started reviewing this patch. The upfront comment I have for future patches is that we don't use the @author tag on classes and we absolutely require the ASF license header on each file. I'll make the changes to your patch if there is nothing else that I find. > Improvements for Oozie in the ClientDSL > --- > > Key: KNOX-835 > URL: https://issues.apache.org/jira/browse/KNOX-835 > Project: Apache Knox > Issue Type: Improvement > Components: ClientDSL >Reporter: Vincent Devillers >Assignee: Vincent Devillers >Priority: Minor > Labels: KIP-4 > Fix For: 0.12.0 > > Attachments: KNOX-835.patch > > > Implements: list, kill, change, info, definition, rerun, resume, log, start, > dryrun -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KNOX-836) Websockets connection is terminated when message size is larger than 65536
[ https://issues.apache.org/jira/browse/KNOX-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-836: - Resolution: Fixed Status: Resolved (was: Patch Available) > Websockets connection is terminated when message size is larger than 65536 > -- > > Key: KNOX-836 > URL: https://issues.apache.org/jira/browse/KNOX-836 > Project: Apache Knox > Issue Type: Bug > Components: Server >Affects Versions: 0.10.0, 0.11.0 >Reporter: Sandeep More >Assignee: Sandeep More > Attachments: KNOX-836.001.patch > > > In some cases, or e.g. in Zeppelin with large demo Notebooks, Knox will > abruptly close connection with error: > [exec] org.eclipse.jetty.websocket.api.MessageTooLargeException: Text message > size [1313822] exceeds maximum size [65536] > [exec] at > org.eclipse.jetty.websocket.api.WebSocketPolicy.assertValidTextMessageSize(WebSocketPolicy.java:140) > [exec] at > org.eclipse.jetty.websocket.common.Parser.assertSanePayloadLength(Parser.java:127) > [exec] at > org.eclipse.jetty.websocket.common.Parser.parseFrame(Parser.java:482) > [exec] at > org.eclipse.jetty.websocket.common.Parser.parse(Parser.java:254) > [exec] at > org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.readParse(AbstractWebSocketConnection.java:632) > [exec] at > org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:480) > [exec] at > org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) > [exec] at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) > [exec] at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) > [exec] at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KNOX-840) Admin UI add ability to create a new topology based on a template
Sumit Gupta created KNOX-840: Summary: Admin UI add ability to create a new topology based on a template Key: KNOX-840 URL: https://issues.apache.org/jira/browse/KNOX-840 Project: Apache Knox Issue Type: Improvement Components: AdminUI Reporter: Sumit Gupta Fix For: 0.12.0 Admin user should be able to use the Admin UI to create new topologies based on one of the template files that we ship with Knox. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KNOX-839) Add Admin UI development documentation
Sumit Gupta created KNOX-839: Summary: Add Admin UI development documentation Key: KNOX-839 URL: https://issues.apache.org/jira/browse/KNOX-839 Project: Apache Knox Issue Type: Improvement Affects Versions: 0.11.0 Reporter: Sumit Gupta Assignee: Sumit Gupta Fix For: 0.12.0 Add documentation to the wiki and/or README information to allow for developers to continue improving the Admin UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KNOX-827) Admin UI needs a way to configure API URL
[ https://issues.apache.org/jira/browse/KNOX-827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15770332#comment-15770332 ] Sumit Gupta commented on KNOX-827: -- For completeness : This change essentially changes the hardcoding to /gateway/manager/api/v1 instead. That makes the UI fixed to a topology but allows for configurability outside of the Admin API used by other services. > Admin UI needs a way to configure API URL > - > > Key: KNOX-827 > URL: https://issues.apache.org/jira/browse/KNOX-827 > Project: Apache Knox > Issue Type: Bug >Reporter: Sumit Gupta >Assignee: Sumit Gupta > Fix For: 0.11.0 > > > The Admin UI is hard coded to use the colocated Admin API that is essentially > /gateway/admin/api/v1. We need a mechanism to be able to configure this URL > so that other topologies can be used to host the API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KNOX-827) Admin UI needs a way to configure API URL
[ https://issues.apache.org/jira/browse/KNOX-827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta reassigned KNOX-827: Assignee: Sumit Gupta > Admin UI needs a way to configure API URL > - > > Key: KNOX-827 > URL: https://issues.apache.org/jira/browse/KNOX-827 > Project: Apache Knox > Issue Type: Bug >Reporter: Sumit Gupta >Assignee: Sumit Gupta > Fix For: 0.11.0 > > > The Admin UI is hard coded to use the colocated Admin API that is essentially > /gateway/admin/api/v1. We need a mechanism to be able to configure this URL > so that other topologies can be used to host the API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KNOX-827) Admin UI needs a way to configure API URL
[ https://issues.apache.org/jira/browse/KNOX-827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15770324#comment-15770324 ] Sumit Gupta commented on KNOX-827: -- I think a separate topology has merits anyway as discussed in [KNOX-749]. Pushing that solution in... > Admin UI needs a way to configure API URL > - > > Key: KNOX-827 > URL: https://issues.apache.org/jira/browse/KNOX-827 > Project: Apache Knox > Issue Type: Bug >Reporter: Sumit Gupta > Fix For: 0.11.0 > > > The Admin UI is hard coded to use the colocated Admin API that is essentially > /gateway/admin/api/v1. We need a mechanism to be able to configure this URL > so that other topologies can be used to host the API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KNOX-749) Simple Admin UI
[ https://issues.apache.org/jira/browse/KNOX-749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15770318#comment-15770318 ] Sumit Gupta commented on KNOX-749: -- Some issues here with admin-ui in admin topology has made us rethink things and add a manager topology after all. See [KNOX-827] > Simple Admin UI > --- > > Key: KNOX-749 > URL: https://issues.apache.org/jira/browse/KNOX-749 > Project: Apache Knox > Issue Type: Bug > Components: Server >Reporter: Larry McCay >Assignee: Sumit Gupta > Fix For: 0.11.0 > > Attachments: knox-admin-ui-001.patch > > > We need to a simple UI for management/admin insights based on the Knox Admin > API. > We can start with a read-only iteration and extend it in a future release. > [~zacblanco] has expressed interest in this effort on the dev list and > possibly other JIRAs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KNOX-825) Turn off directory browsing that Admin UI exposes
[ https://issues.apache.org/jira/browse/KNOX-825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta resolved KNOX-825. -- Resolution: Fixed > Turn off directory browsing that Admin UI exposes > - > > Key: KNOX-825 > URL: https://issues.apache.org/jira/browse/KNOX-825 > Project: Apache Knox > Issue Type: Bug >Reporter: Sumit Gupta >Assignee: Sumit Gupta > Fix For: 0.11.0 > > > While testing the admin ui from [KNOX-749] it is apparent that you can browse > the directory for example: > https://localhost:8443/gateway/admin/admin-ui/assets/ > The directory browsing capability should be turned off. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KNOX-825) Turn off directory browsing that Admin UI exposes
Sumit Gupta created KNOX-825: Summary: Turn off directory browsing that Admin UI exposes Key: KNOX-825 URL: https://issues.apache.org/jira/browse/KNOX-825 Project: Apache Knox Issue Type: Bug Reporter: Sumit Gupta Assignee: Sumit Gupta Fix For: 0.11.0 While testing the admin ui from [KNOX-749] it is apparent that you can browse the directory for example: https://localhost:8443/gateway/admin/admin-ui/assets/ The directory browsing capability should be turned off. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KNOX-749) Simple Admin UI
[ https://issues.apache.org/jira/browse/KNOX-749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta resolved KNOX-749. -- Resolution: Fixed > Simple Admin UI > --- > > Key: KNOX-749 > URL: https://issues.apache.org/jira/browse/KNOX-749 > Project: Apache Knox > Issue Type: Bug > Components: Server >Reporter: Larry McCay >Assignee: Sumit Gupta > Fix For: 0.11.0 > > Attachments: knox-admin-ui-001.patch > > > We need to a simple UI for management/admin insights based on the Knox Admin > API. > We can start with a read-only iteration and extend it in a future release. > [~zacblanco] has expressed interest in this effort on the dev list and > possibly other JIRAs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KNOX-824) WebAppSec providers don't work with Jersey based services
[
https://issues.apache.org/jira/browse/KNOX-824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Gupta resolved KNOX-824.
--
Resolution: Fixed
> WebAppSec providers don't work with Jersey based services
> -
>
> Key: KNOX-824
> URL: https://issues.apache.org/jira/browse/KNOX-824
> Project: Apache Knox
> Issue Type: Bug
>Reporter: Sumit Gupta
>Assignee: Sumit Gupta
> Fix For: 0.11.0
>
>
> The following configuration doesn't get applied in deployment (thus doesn't
> affect the topology) for a Jersey based service in a topology file.
> {code}
>
> webappsec
> WebAppSec
> true
> csrf.enabledtrue
>
> csrf.customHeaderX-XSRF-Header
>
> csrf.methodsToIgnoreGET,OPTIONS,HEAD
> cors.enabledtrue
>
> xframe-options.enabledtrue
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KNOX-824) WebAppSec providers don't work with Jersey based services
[
https://issues.apache.org/jira/browse/KNOX-824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Gupta updated KNOX-824:
-
Fix Version/s: 0.11.0
> WebAppSec providers don't work with Jersey based services
> -
>
> Key: KNOX-824
> URL: https://issues.apache.org/jira/browse/KNOX-824
> Project: Apache Knox
> Issue Type: Bug
>Reporter: Sumit Gupta
> Fix For: 0.11.0
>
>
> The following configuration doesn't get applied in deployment (thus doesn't
> affect the topology) for a Jersey based service in a topology file.
> {code}
>
> webappsec
> WebAppSec
> true
> csrf.enabledtrue
>
> csrf.customHeaderX-XSRF-Header
>
> csrf.methodsToIgnoreGET,OPTIONS,HEAD
> cors.enabledtrue
>
> xframe-options.enabledtrue
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (KNOX-824) WebAppSec providers don't work with Jersey based services
[
https://issues.apache.org/jira/browse/KNOX-824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Gupta reassigned KNOX-824:
Assignee: Sumit Gupta
> WebAppSec providers don't work with Jersey based services
> -
>
> Key: KNOX-824
> URL: https://issues.apache.org/jira/browse/KNOX-824
> Project: Apache Knox
> Issue Type: Bug
>Reporter: Sumit Gupta
>Assignee: Sumit Gupta
> Fix For: 0.11.0
>
>
> The following configuration doesn't get applied in deployment (thus doesn't
> affect the topology) for a Jersey based service in a topology file.
> {code}
>
> webappsec
> WebAppSec
> true
> csrf.enabledtrue
>
> csrf.customHeaderX-XSRF-Header
>
> csrf.methodsToIgnoreGET,OPTIONS,HEAD
> cors.enabledtrue
>
> xframe-options.enabledtrue
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (KNOX-802) Inconsistent gateway.xml deployment descriptors when topology contains application
[ https://issues.apache.org/jira/browse/KNOX-802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta resolved KNOX-802. -- Resolution: Fixed Closing this issue as the root cause is because of [KNOX-824] > Inconsistent gateway.xml deployment descriptors when topology contains > application > -- > > Key: KNOX-802 > URL: https://issues.apache.org/jira/browse/KNOX-802 > Project: Apache Knox > Issue Type: Bug >Reporter: Sumit Gupta > Fix For: 0.12.0 > > > Topologies with applications don't get gateway.xml updates on redeployment. > To reproduce, add an application to the admin.xml topology. For example > > admin-ui > > Then add a new provider, for example: > > webappsec > WebAppSec > true > > xframe.options.enabled > true > > > cors.enabled > true > > > The deployment now contains two gateway.xml files and only the gateway.xml > file contains the WebAppSec related filters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KNOX-824) WebAppSec providers don't work with Jersey based services
Sumit Gupta created KNOX-824:
Summary: WebAppSec providers don't work with Jersey based services
Key: KNOX-824
URL: https://issues.apache.org/jira/browse/KNOX-824
Project: Apache Knox
Issue Type: Bug
Reporter: Sumit Gupta
The following configuration doesn't get applied in deployment (thus doesn't
affect the topology) for a Jersey based service in a topology file.
{code}
webappsec
WebAppSec
true
csrf.enabledtrue
csrf.customHeaderX-XSRF-Header
csrf.methodsToIgnoreGET,OPTIONS,HEAD
cors.enabledtrue
xframe-options.enabledtrue
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KNOX-749) Simple Admin UI
[ https://issues.apache.org/jira/browse/KNOX-749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765257#comment-15765257 ] Sumit Gupta commented on KNOX-749: -- Thanks [~lmccay], this makes life easier for sure to integrate the UI in. There is an issue with WebAppSec getting included into a topology file's deployment that just contains the 'KNOX' service. This is what triggered the filing of the bug [KNOX-802], but now that I understand the issue more I might need to modify this bug or file another one. Anyhow, aside from the related implementation issue the topologies as described make sense. > Simple Admin UI > --- > > Key: KNOX-749 > URL: https://issues.apache.org/jira/browse/KNOX-749 > Project: Apache Knox > Issue Type: Bug > Components: Server >Reporter: Larry McCay >Assignee: Sumit Gupta > Fix For: 0.11.0 > > Attachments: knox-admin-ui-001.patch > > > We need to a simple UI for management/admin insights based on the Knox Admin > API. > We can start with a read-only iteration and extend it in a future release. > [~zacblanco] has expressed interest in this effort on the dev list and > possibly other JIRAs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KNOX-804) Knox ehcache usage has intermittent CacheManger exception
[ https://issues.apache.org/jira/browse/KNOX-804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta resolved KNOX-804. -- Resolution: Fixed > Knox ehcache usage has intermittent CacheManger exception > - > > Key: KNOX-804 > URL: https://issues.apache.org/jira/browse/KNOX-804 > Project: Apache Knox > Issue Type: Bug > Components: Server, Site >Affects Versions: 0.10.0 >Reporter: Sumit Gupta >Assignee: Sumit Gupta > Fix For: 0.11.0 > > > There is an intermittent issue when using ehcache in the shiro config. It > doesn't seem to happen in a new topology file. > 2016-10-10 08:22:09,171 ERROR env.EnvironmentLoader > (EnvironmentLoader.java:initEnvironment(146)) - Shiro environment > initialization failed > org.apache.shiro.cache.CacheException: net.sf.ehcache.CacheException: Another > unnamed CacheManager already exists in the same VM. Please provide unique > names for each CacheManager in the config or do one of following: > 1. Use one of the CacheManager.create() static factory methods to reuse same > CacheManager with same name or create one if necessary > 2. Shutdown the earlier cacheManager before creating new one with same name. > The source of the existing CacheManager is: InputStreamConfigurationSource > stream=sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@42caa857 > at > org.apache.shiro.cache.ehcache.EhCacheManager.ensureCacheManager(EhCacheManager.java:224) > at org.apache.shiro.cache.ehcache.EhCacheManager.init(EhCacheManager.java:199) > at org.apache.shiro.util.LifecycleUtils.init(LifecycleUtils.java:45) > at org.apache.shiro.util.LifecycleUtils.init(LifecycleUtils.java:40) > at org.apache.shiro.util.LifecycleUtils.init(LifecycleUtils.java:61) > at > org.apache.shiro.config.ReflectionBuilder.buildObjects(ReflectionBuilder.java:129) > at > org.apache.shiro.config.IniSecurityManagerFactory.buildInstances(IniSecurityManagerFactory.java:161) > at > org.apache.shiro.config.IniSecurityManagerFactory.createSecurityManager(IniSecurityManagerFactory.java:124) > at > org.apache.shiro.config.IniSecurityManagerFactory.createSecurityManager(IniSecurityManagerFactory.java:102) > at > org.apache.shiro.config.IniSecurityManagerFactory.createInstance(IniSecurityManagerFactory.java:88) > at > org.apache.shiro.config.IniSecurityManagerFactory.createInstance(IniSecurityManagerFactory.java:46) > at > org.apache.shiro.config.IniFactorySupport.createInstance(IniFactorySupport.java:123) > at org.apache.shiro.util.AbstractFactory.getInstance(AbstractFactory.java:47) > at > org.apache.shiro.web.env.IniWebEnvironment.createWebSecurityManager(IniWebEnvironment.java:203) > at > org.apache.shiro.web.env.IniWebEnvironment.configure(IniWebEnvironment.java:99) > at org.apache.shiro.web.env.IniWebEnvironment.init(IniWebEnvironment.java:92) > at org.apache.shiro.util.LifecycleUtils.init(LifecycleUtils.java:45) > at org.apache.shiro.util.LifecycleUtils.init(LifecycleUtils.java:40) > at > org.apache.shiro.web.env.EnvironmentLoader.createEnvironment(EnvironmentLoader.java:221) > at > org.apache.shiro.web.env.EnvironmentLoader.initEnvironment(EnvironmentLoader.java:133) > at > org.apache.shiro.web.env.EnvironmentLoaderListener.contextInitialized(EnvironmentLoaderListener.java:58) > at > org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:782) > at > org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:424) > at > org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:774) > at > org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:249) > at > org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1242) > at > org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:717) > at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:494) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64) > at > org.apache.hadoop.gateway.GatewayServer.internalDeploy(GatewayServer.java:364) > at org.apache.hadoop.gateway.GatewayServer.access$700(GatewayServer.java:72) > at > org.apache.hadoop.gateway.GatewayServer$InternalTopologyListener.handleCreateDeployment(GatewayServer.java:441) > at > org.apache.hadoop.gateway.GatewayServer$InternalTopologyListener.handleTopologyEvent(GatewayServer.java:402) > at > org.apache.hadoop.gateway.services.topology.impl.DefaultTopologyService.notifyChangeListeners(DefaultTopologyService.java:347) > at > org.apache.hadoop.gateway.services.topology.impl.DefaultTopologyService.reloadTopologies(DefaultTopologyService.java:321) > at >
[jira] [Resolved] (KNOX-818) Admin API PUT corrupts the topology file
[
https://issues.apache.org/jira/browse/KNOX-818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Gupta resolved KNOX-818.
--
Resolution: Fixed
> Admin API PUT corrupts the topology file
>
>
> Key: KNOX-818
> URL: https://issues.apache.org/jira/browse/KNOX-818
> Project: Apache Knox
> Issue Type: Bug
>Reporter: Sumit Gupta
>Assignee: Sumit Gupta
> Fix For: 0.11.0
>
>
> When using the Admin API to PUT/save a topology file, the content doesn't
> always wind up being correct.
> For example when saving the sandbox topology:
> curl -iku admin:admin-password -H "Accept: application/xml" -H "Content-Type:
> application/xml" -T sandbox.xml -X PUT
> http://localhost:8443/gateway/admin/api/v1/topologies/sandbox
> The following is saved:
> {code}
>
>
>sandbox2
>
>
> authentication
> ShiroProvider
> true
>
>
> identity-assertion
> Default
> true
>
>
> hostmap
> static
> true
>
>
>
> NAMENODE
> hdfs://localhost:8020
>
>
> JOBTRACKER
> rpc://localhost:8050
>
>
> WEBHDFS
> http://localhost:50070/webhdfs
>
>
> WEBHCAT
> http://localhost:50111/templeton
>
>
> OOZIE
> http://localhost:11000/oozie
>
>
> WEBHBASE
> http://localhost:60080
>
>
> HIVE
> http://localhost:10001/cliservice
>
>
> RESOURCEMANAGER
> http://localhost:8088/ws
>
>
> DRUID-COORDINATOR-UI
> http://localhost:8081
>
>
> DRUID-COORDINATOR
> http://localhost:8081
>
>
> DRUID-BROKER
> http://localhost:8082
>
>
> DRUID-ROUTER
> http://localhost:8082
>
>
> DRUID-OVERLORD
> http://localhost:8090
>
>
> DRUID-OVERLORD-UI
> http://localhost:8090
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (KNOX-801) proper escaping of html attributes while rewriting
[ https://issues.apache.org/jira/browse/KNOX-801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta resolved KNOX-801. -- Resolution: Fixed Resolved by [KNOX-791] > proper escaping of html attributes while rewriting > -- > > Key: KNOX-801 > URL: https://issues.apache.org/jira/browse/KNOX-801 > Project: Apache Knox > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Sumit Gupta > Fix For: 0.11.0 > > Attachments: KNOX-801.patch > > > while parsing and rewriting html attributes, If the attribute contains > special characters e.g quotes, they are not escaped properly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KNOX-784) java.lang.IllegalArgumentException: A metric named org.apache.http.conn.HttpClientConnectionManager.available-connections already exists
[ https://issues.apache.org/jira/browse/KNOX-784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-784: - Fix Version/s: (was: 0.11.0) 0.12.0 > java.lang.IllegalArgumentException: A metric named > org.apache.http.conn.HttpClientConnectionManager.available-connections > already exists > - > > Key: KNOX-784 > URL: https://issues.apache.org/jira/browse/KNOX-784 > Project: Apache Knox > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Sumit Gupta > Fix For: 0.12.0 > > > Facing this error on latest trunk version - > Caused by: java.lang.IllegalArgumentException: A metric named > org.apache.http.conn.HttpClientConnectionManager.available-connections > already exists > at > com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:91) > at > com.codahale.metrics.httpclient.InstrumentedHttpClientConnectionManager.(InstrumentedHttpClientConnectionManager.java:63) > at > com.codahale.metrics.httpclient.InstrumentedHttpClientConnectionManager.(InstrumentedHttpClientConnectionManager.java:49) > at > com.codahale.metrics.httpclient.InstrumentedHttpClientConnectionManager.(InstrumentedHttpClientConnectionManager.java:41) > at > com.codahale.metrics.httpclient.InstrumentedHttpClientConnectionManager.(InstrumentedHttpClientConnectionManager.java:36) > at > org.apache.hadoop.gateway.services.metrics.impl.instr.InstrHttpClientBuilderProvider.getInstrumented(InstrHttpClientBuilderProvider.java:41) > at > org.apache.hadoop.gateway.services.metrics.impl.instr.InstrHttpClientBuilderProvider.getInstrumented(InstrHttpClientBuilderProvider.java:36) > at > org.apache.hadoop.gateway.services.metrics.impl.DefaultMetricsService.getInstrumented(DefaultMetricsService.java:128) > at > org.apache.hadoop.gateway.dispatch.DefaultHttpClientFactory.createHttpClient(DefaultHttpClientFactory.java:67) > at > org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter.init(GatewayDispatchFilter.java:75) > at > org.apache.hadoop.gateway.GatewayFilter$Holder.getInstance(GatewayFilter.java:362) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KNOX-818) Admin API PUT corrupts the topology file
[
https://issues.apache.org/jira/browse/KNOX-818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Gupta reassigned KNOX-818:
Assignee: Sumit Gupta
> Admin API PUT corrupts the topology file
>
>
> Key: KNOX-818
> URL: https://issues.apache.org/jira/browse/KNOX-818
> Project: Apache Knox
> Issue Type: Bug
>Reporter: Sumit Gupta
>Assignee: Sumit Gupta
> Fix For: 0.11.0
>
>
> When using the Admin API to PUT/save a topology file, the content doesn't
> always wind up being correct.
> For example when saving the sandbox topology:
> curl -iku admin:admin-password -H "Accept: application/xml" -H "Content-Type:
> application/xml" -T sandbox.xml -X PUT
> http://localhost:8443/gateway/admin/api/v1/topologies/sandbox
> The following is saved:
> {code}
>
>
>sandbox2
>
>
> authentication
> ShiroProvider
> true
>
>
> identity-assertion
> Default
> true
>
>
> hostmap
> static
> true
>
>
>
> NAMENODE
> hdfs://localhost:8020
>
>
> JOBTRACKER
> rpc://localhost:8050
>
>
> WEBHDFS
> http://localhost:50070/webhdfs
>
>
> WEBHCAT
> http://localhost:50111/templeton
>
>
> OOZIE
> http://localhost:11000/oozie
>
>
> WEBHBASE
> http://localhost:60080
>
>
> HIVE
> http://localhost:10001/cliservice
>
>
> RESOURCEMANAGER
> http://localhost:8088/ws
>
>
> DRUID-COORDINATOR-UI
> http://localhost:8081
>
>
> DRUID-COORDINATOR
> http://localhost:8081
>
>
> DRUID-BROKER
> http://localhost:8082
>
>
> DRUID-ROUTER
> http://localhost:8082
>
>
> DRUID-OVERLORD
> http://localhost:8090
>
>
> DRUID-OVERLORD-UI
> http://localhost:8090
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KNOX-818) Admin API PUT corrupts the topology file
Sumit Gupta created KNOX-818:
Summary: Admin API PUT corrupts the topology file
Key: KNOX-818
URL: https://issues.apache.org/jira/browse/KNOX-818
Project: Apache Knox
Issue Type: Bug
Reporter: Sumit Gupta
Fix For: 0.11.0
When using the Admin API to PUT/save a topology file, the content doesn't
always wind up being correct.
For example when saving the sandbox topology:
curl -iku admin:admin-password -H "Accept: application/xml" -H "Content-Type:
application/xml" -T sandbox.xml -X PUT
http://localhost:8443/gateway/admin/api/v1/topologies/sandbox
The following is saved:
{code}
sandbox2
authentication
ShiroProvider
true
identity-assertion
Default
true
hostmap
static
true
NAMENODE
hdfs://localhost:8020
JOBTRACKER
rpc://localhost:8050
WEBHDFS
http://localhost:50070/webhdfs
WEBHCAT
http://localhost:50111/templeton
OOZIE
http://localhost:11000/oozie
WEBHBASE
http://localhost:60080
HIVE
http://localhost:10001/cliservice
RESOURCEMANAGER
http://localhost:8088/ws
DRUID-COORDINATOR-UI
http://localhost:8081
DRUID-COORDINATOR
http://localhost:8081
DRUID-BROKER
http://localhost:8082
DRUID-ROUTER
http://localhost:8082
DRUID-OVERLORD
http://localhost:8090
DRUID-OVERLORD-UI
http://localhost:8090
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KNOX-802) Inconsistent gateway.xml deployment descriptors when topology contains application
[ https://issues.apache.org/jira/browse/KNOX-802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Gupta updated KNOX-802: - Fix Version/s: 0.12.0 > Inconsistent gateway.xml deployment descriptors when topology contains > application > -- > > Key: KNOX-802 > URL: https://issues.apache.org/jira/browse/KNOX-802 > Project: Apache Knox > Issue Type: Bug >Reporter: Sumit Gupta > Fix For: 0.12.0 > > > Topologies with applications don't get gateway.xml updates on redeployment. > To reproduce, add an application to the admin.xml topology. For example > > admin-ui > > Then add a new provider, for example: > > webappsec > WebAppSec > true > > xframe.options.enabled > true > > > cors.enabled > true > > > The deployment now contains two gateway.xml files and only the gateway.xml > file contains the WebAppSec related filters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KNOX-791) "
[ https://issues.apache.org/jira/browse/KNOX-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15743272#comment-15743272 ] Sumit Gupta commented on KNOX-791: -- Thanks [~smore]! I'd love to take a look but it looks like you attached the patch for KNOX-237 instead by accident. Also, can you see if there is overlap with the patch/bug for [KNOX-801]? > "
