[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap authentication, based on ldapurl, basedn, domain
mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap
authentication, based on ldapurl, basedn, domain
URL: https://github.com/apache/incubator-livy/pull/236#discussion_r329613707
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/auth/LdapAuthenticationProviderImpl.scala
##
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.livy.thriftserver.auth
+
+import javax.security.sasl.AuthenticationException
+
+import org.apache.commons.lang.StringUtils
+import org.apache.hive.service.auth.PasswdAuthenticationProvider
+
+import org.apache.livy.thriftserver.auth.ldap._
+import org.apache.livy.LivyConf
+
+object LdapAuthenticationProviderImpl {
+
+ // Initialize the Chain Filter List. Now GroupFilter is not supported.
+ // If needed, GroupFilter can be added in this list.
+
+ private def createFilters(conf: LivyConf): Filter = {
+val chainFilters: List[Filter] = List[Filter](new UserFilter(conf))
Review comment:
simply `new ChainFilter(List(new UserFilter(conf)))`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap authentication, based on ldapurl, basedn, domain
mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap
authentication, based on ldapurl, basedn, domain
URL: https://github.com/apache/incubator-livy/pull/236#discussion_r329616180
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/auth/ldap/LdapUtils.scala
##
@@ -0,0 +1,134 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.livy.thriftserver.auth.ldap
+
+import org.apache.livy.{LivyConf, Logging}
+
+/**
+ * Static utility methods related to LDAP authentication module.
+ */
+object LdapUtils extends Logging{
Review comment:
```suggestion
object LdapUtils extends Logging {
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap authentication, based on ldapurl, basedn, domain
mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap
authentication, based on ldapurl, basedn, domain
URL: https://github.com/apache/incubator-livy/pull/236#discussion_r329616871
##
File path: server/src/main/scala/org/apache/livy/LivyConf.scala
##
@@ -167,7 +167,13 @@ object LivyConf {
Entry("livy.server.thrift.delegation.token.max-lifetime", "7d")
val THRIFT_DELEGATION_TOKEN_RENEW_INTERVAL =
Entry("livy.server.thrift.delegation.token.renew-interval", "1d")
-
+ val THRIFT_LDAP_AUTHENTICATION_URL =
Entry("livy.server.thrift.ldap.authentication.url", null)
+ val THRIFT_LDAP_AUTHENTICATION_BASEDN =
Review comment:
I wondering about merging these with the ones from the server PR. @jerryshao
WDYT?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap authentication, based on ldapurl, basedn, domain
mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap
authentication, based on ldapurl, basedn, domain
URL: https://github.com/apache/incubator-livy/pull/236#discussion_r329614280
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/auth/LdapAuthenticationProviderImpl.scala
##
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.livy.thriftserver.auth
+
+import javax.security.sasl.AuthenticationException
+
+import org.apache.commons.lang.StringUtils
+import org.apache.hive.service.auth.PasswdAuthenticationProvider
+
+import org.apache.livy.thriftserver.auth.ldap._
+import org.apache.livy.LivyConf
+
+object LdapAuthenticationProviderImpl {
+
+ // Initialize the Chain Filter List. Now GroupFilter is not supported.
+ // If needed, GroupFilter can be added in this list.
+
+ private def createFilters(conf: LivyConf): Filter = {
+val chainFilters: List[Filter] = List[Filter](new UserFilter(conf))
+val filter: Filter = new ChainFilter(chainFilters)
+filter
+ }
+}
+
+class LdapAuthenticationProviderImpl(val conf: LivyConf) extends
PasswdAuthenticationProvider {
+ final private val filter: Filter =
LdapAuthenticationProviderImpl.createFilters(conf)
+ final private val searchFactory: DirSearchFactory = new LdapSearchFactory()
+
+ @throws[AuthenticationException]
+ def Authenticate(user: String, password: String): Unit = {
+createDirSearch(user, password)
+applyFilter(user)
+ }
+
+ @throws[AuthenticationException]
+ private def createDirSearch(user: String, password: String): Unit = {
+if (StringUtils.isBlank(user) || StringUtils.isEmpty(user)) {
+ throw new AuthenticationException("Error validating LDAP:" +
+" a null or blank user name has been provided")
+}
+if (StringUtils.isBlank(password) || StringUtils.isEmpty(password)) {
+ throw new AuthenticationException("Error validating LDAP:" +
Review comment:
ditto
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap authentication, based on ldapurl, basedn, domain
mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap
authentication, based on ldapurl, basedn, domain
URL: https://github.com/apache/incubator-livy/pull/236#discussion_r329616421
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/auth/ldap/LdapUtils.scala
##
@@ -0,0 +1,134 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.livy.thriftserver.auth.ldap
+
+import org.apache.livy.{LivyConf, Logging}
+
+/**
+ * Static utility methods related to LDAP authentication module.
+ */
+object LdapUtils extends Logging{
+
+ /**
+ * Extracts username from user DN.
+ *
+ * Examples:
+ *
+ * LdapUtils.extractUserName("UserName")= "UserName"
+ * LdapUtils.extractUserName("usern...@mycorp.com") = "UserName"
+ * LdapUtils.extractUserName("cn=UserName,dc=mycompany,dc=com") = "UserName"
+ *
+ *
+ * @param userDn
+ * @return
+ */
+ def extractUserName(userDn: String): String = {
+var userName = userDn
+
+if (!isDn(userDn) && !hasDomain(userDn)) {
+ userName = userDn
+} else {
+ val domainIdx = indexOfDomainMatch(userDn)
+ if (domainIdx > 0) {
+userName = userDn.substring(0, domainIdx)
+ } else if (userDn.contains("=")) {
+userName = userDn.substring(userDn.indexOf("=") + 1,
userDn.indexOf(","))
+ }
+}
+userName
+ }
+
+ /**
+ * Get the index separating the user name from domain name (the user's name
up
+ * to the first '/' or '@').
+ *
+ * @param userName full user name.
+ * @return index of domain match or -1 if not found
+ */
+ def indexOfDomainMatch(userName: String): Int = {
+var endIdx = -1
+if (userName == null) {
+ endIdx = -1
Review comment:
useless..
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap authentication, based on ldapurl, basedn, domain
mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap
authentication, based on ldapurl, basedn, domain
URL: https://github.com/apache/incubator-livy/pull/236#discussion_r329614222
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/auth/LdapAuthenticationProviderImpl.scala
##
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.livy.thriftserver.auth
+
+import javax.security.sasl.AuthenticationException
+
+import org.apache.commons.lang.StringUtils
+import org.apache.hive.service.auth.PasswdAuthenticationProvider
+
+import org.apache.livy.thriftserver.auth.ldap._
+import org.apache.livy.LivyConf
+
+object LdapAuthenticationProviderImpl {
+
+ // Initialize the Chain Filter List. Now GroupFilter is not supported.
+ // If needed, GroupFilter can be added in this list.
+
+ private def createFilters(conf: LivyConf): Filter = {
+val chainFilters: List[Filter] = List[Filter](new UserFilter(conf))
+val filter: Filter = new ChainFilter(chainFilters)
+filter
+ }
+}
+
+class LdapAuthenticationProviderImpl(val conf: LivyConf) extends
PasswdAuthenticationProvider {
+ final private val filter: Filter =
LdapAuthenticationProviderImpl.createFilters(conf)
+ final private val searchFactory: DirSearchFactory = new LdapSearchFactory()
+
+ @throws[AuthenticationException]
+ def Authenticate(user: String, password: String): Unit = {
+createDirSearch(user, password)
+applyFilter(user)
+ }
+
+ @throws[AuthenticationException]
+ private def createDirSearch(user: String, password: String): Unit = {
+if (StringUtils.isBlank(user) || StringUtils.isEmpty(user)) {
+ throw new AuthenticationException("Error validating LDAP:" +
Review comment:
all the string on next line?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap authentication, based on ldapurl, basedn, domain
mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap
authentication, based on ldapurl, basedn, domain
URL: https://github.com/apache/incubator-livy/pull/236#discussion_r329614672
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/auth/LdapAuthenticationProviderImpl.scala
##
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.livy.thriftserver.auth
+
+import javax.security.sasl.AuthenticationException
+
+import org.apache.commons.lang.StringUtils
+import org.apache.hive.service.auth.PasswdAuthenticationProvider
+
+import org.apache.livy.thriftserver.auth.ldap._
+import org.apache.livy.LivyConf
+
+object LdapAuthenticationProviderImpl {
+
+ // Initialize the Chain Filter List. Now GroupFilter is not supported.
+ // If needed, GroupFilter can be added in this list.
+
+ private def createFilters(conf: LivyConf): Filter = {
+val chainFilters: List[Filter] = List[Filter](new UserFilter(conf))
+val filter: Filter = new ChainFilter(chainFilters)
+filter
+ }
+}
+
+class LdapAuthenticationProviderImpl(val conf: LivyConf) extends
PasswdAuthenticationProvider {
+ final private val filter: Filter =
LdapAuthenticationProviderImpl.createFilters(conf)
+ final private val searchFactory: DirSearchFactory = new LdapSearchFactory()
+
+ @throws[AuthenticationException]
+ def Authenticate(user: String, password: String): Unit = {
+createDirSearch(user, password)
+applyFilter(user)
+ }
+
+ @throws[AuthenticationException]
+ private def createDirSearch(user: String, password: String): Unit = {
+if (StringUtils.isBlank(user) || StringUtils.isEmpty(user)) {
+ throw new AuthenticationException("Error validating LDAP:" +
+" a null or blank user name has been provided")
+}
+if (StringUtils.isBlank(password) || StringUtils.isEmpty(password)) {
+ throw new AuthenticationException("Error validating LDAP:" +
+" a null or blank password has been provided")
+}
+val principal = LdapUtils.createCandidatePrincipal(conf, user)
+try {
+ searchFactory.getInstance(conf, principal, password)
+} catch {
+ case e: AuthenticationException =>
+throw new AuthenticationException(s"Error validating " +
+ s"LDAP user: $user, password: $password", e)
+}
+ }
+
+ @throws[AuthenticationException]
+ private def applyFilter(user: String): Unit = {
+if (filter != null) {
+ if (LdapUtils.hasDomain(user)) {
+filter.apply(LdapUtils.extractUserName(user))
+ }
Review comment:
no newline here please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap authentication, based on ldapurl, basedn, domain
mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap
authentication, based on ldapurl, basedn, domain
URL: https://github.com/apache/incubator-livy/pull/236#discussion_r329614333
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/auth/LdapAuthenticationProviderImpl.scala
##
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.livy.thriftserver.auth
+
+import javax.security.sasl.AuthenticationException
+
+import org.apache.commons.lang.StringUtils
+import org.apache.hive.service.auth.PasswdAuthenticationProvider
+
+import org.apache.livy.thriftserver.auth.ldap._
+import org.apache.livy.LivyConf
+
+object LdapAuthenticationProviderImpl {
+
+ // Initialize the Chain Filter List. Now GroupFilter is not supported.
+ // If needed, GroupFilter can be added in this list.
+
+ private def createFilters(conf: LivyConf): Filter = {
+val chainFilters: List[Filter] = List[Filter](new UserFilter(conf))
+val filter: Filter = new ChainFilter(chainFilters)
+filter
+ }
+}
+
+class LdapAuthenticationProviderImpl(val conf: LivyConf) extends
PasswdAuthenticationProvider {
+ final private val filter: Filter =
LdapAuthenticationProviderImpl.createFilters(conf)
+ final private val searchFactory: DirSearchFactory = new LdapSearchFactory()
+
+ @throws[AuthenticationException]
+ def Authenticate(user: String, password: String): Unit = {
+createDirSearch(user, password)
+applyFilter(user)
+ }
+
+ @throws[AuthenticationException]
+ private def createDirSearch(user: String, password: String): Unit = {
+if (StringUtils.isBlank(user) || StringUtils.isEmpty(user)) {
+ throw new AuthenticationException("Error validating LDAP:" +
+" a null or blank user name has been provided")
+}
+if (StringUtils.isBlank(password) || StringUtils.isEmpty(password)) {
+ throw new AuthenticationException("Error validating LDAP:" +
+" a null or blank password has been provided")
+}
+val principal = LdapUtils.createCandidatePrincipal(conf, user)
+try {
+ searchFactory.getInstance(conf, principal, password)
+} catch {
+ case e: AuthenticationException =>
+throw new AuthenticationException(s"Error validating " +
Review comment:
ditto
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap authentication, based on ldapurl, basedn, domain
mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap
authentication, based on ldapurl, basedn, domain
URL: https://github.com/apache/incubator-livy/pull/236#discussion_r329613194
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/auth/LdapAuthenticationProviderImpl.scala
##
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.livy.thriftserver.auth
+
+import javax.security.sasl.AuthenticationException
+
+import org.apache.commons.lang.StringUtils
+import org.apache.hive.service.auth.PasswdAuthenticationProvider
+
+import org.apache.livy.thriftserver.auth.ldap._
+import org.apache.livy.LivyConf
+
+object LdapAuthenticationProviderImpl {
+
+ // Initialize the Chain Filter List. Now GroupFilter is not supported.
Review comment:
please javadoc comment
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap authentication, based on ldapurl, basedn, domain
mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap
authentication, based on ldapurl, basedn, domain
URL: https://github.com/apache/incubator-livy/pull/236#discussion_r329614549
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/auth/LdapAuthenticationProviderImpl.scala
##
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.livy.thriftserver.auth
+
+import javax.security.sasl.AuthenticationException
+
+import org.apache.commons.lang.StringUtils
+import org.apache.hive.service.auth.PasswdAuthenticationProvider
+
+import org.apache.livy.thriftserver.auth.ldap._
+import org.apache.livy.LivyConf
+
+object LdapAuthenticationProviderImpl {
+
+ // Initialize the Chain Filter List. Now GroupFilter is not supported.
+ // If needed, GroupFilter can be added in this list.
+
+ private def createFilters(conf: LivyConf): Filter = {
+val chainFilters: List[Filter] = List[Filter](new UserFilter(conf))
+val filter: Filter = new ChainFilter(chainFilters)
+filter
+ }
+}
+
+class LdapAuthenticationProviderImpl(val conf: LivyConf) extends
PasswdAuthenticationProvider {
+ final private val filter: Filter =
LdapAuthenticationProviderImpl.createFilters(conf)
+ final private val searchFactory: DirSearchFactory = new LdapSearchFactory()
+
+ @throws[AuthenticationException]
+ def Authenticate(user: String, password: String): Unit = {
+createDirSearch(user, password)
+applyFilter(user)
+ }
+
+ @throws[AuthenticationException]
+ private def createDirSearch(user: String, password: String): Unit = {
+if (StringUtils.isBlank(user) || StringUtils.isEmpty(user)) {
+ throw new AuthenticationException("Error validating LDAP:" +
+" a null or blank user name has been provided")
+}
+if (StringUtils.isBlank(password) || StringUtils.isEmpty(password)) {
+ throw new AuthenticationException("Error validating LDAP:" +
+" a null or blank password has been provided")
+}
+val principal = LdapUtils.createCandidatePrincipal(conf, user)
+try {
+ searchFactory.getInstance(conf, principal, password)
+} catch {
+ case e: AuthenticationException =>
+throw new AuthenticationException(s"Error validating " +
+ s"LDAP user: $user, password: $password", e)
+}
+ }
+
+ @throws[AuthenticationException]
+ private def applyFilter(user: String): Unit = {
+if (filter != null) {
Review comment:
why do we need this check?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap authentication, based on ldapurl, basedn, domain
mgaido91 commented on a change in pull request #236: [LIVY-678] Thrift ldap
authentication, based on ldapurl, basedn, domain
URL: https://github.com/apache/incubator-livy/pull/236#discussion_r329615436
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/auth/ldap/ChainFilter.scala
##
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.livy.thriftserver.auth.ldap
+
+import javax.security.sasl.AuthenticationException
+
+/**
+ * A factory that produces a Filter that is implemented as a chain of other
filters.
+ * The chain of filters are created as a result of
+ * getInstance(org.apache.livy.LivyConf)
+ */
+
+class ChainFilter(val chainedFilters: List[Filter]) extends Filter {
+ @throws[AuthenticationException]
+ def apply(user: String): Unit = {
+chainedFilters.foreach { f => f.apply(user) }
Review comment:
```suggestion
chainedFilters.foreach { f => f(user) }
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io commented on issue #240: [LIVY-541] Generate the unqiue session id when multiple livy servers work at the same time
codecov-io commented on issue #240: [LIVY-541] Generate the unqiue session id when multiple livy servers work at the same time URL: https://github.com/apache/incubator-livy/pull/240#issuecomment-536592263 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/240?src=pr=h1) Report > Merging [#240](https://codecov.io/gh/apache/incubator-livy/pull/240?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/0804c8ea8ece67d01ababec616c9ad8e3b15dc9f?src=pr=desc) will **decrease** coverage by `0.02%`. > The diff coverage is `74.48%`. [](https://codecov.io/gh/apache/incubator-livy/pull/240?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #240 +/- ## - Coverage 68.45% 68.43% -0.03% - Complexity 927 935 +8 Files 100 101 +1 Lines 5729 5765 +36 Branches870 872 +2 + Hits 3922 3945 +23 - Misses 1247 1258 +11 - Partials560 562 +2 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/240?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...he/livy/server/recovery/FileSystemStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/240/diff?src=pr=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvRmlsZVN5c3RlbVN0YXRlU3RvcmUuc2NhbGE=) | `63.04% <100%> (ø)` | `11 <4> (ø)` | :arrow_down: | | [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/240/diff?src=pr=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `95.91% <100%> (+0.04%)` | `21 <0> (ø)` | :arrow_down: | | [...org/apache/livy/server/recovery/SessionStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/240/diff?src=pr=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU2Vzc2lvblN0b3JlLnNjYWxh) | `77.77% <100%> (-2.23%)` | `9 <1> (-1)` | | | [...cala/org/apache/livy/sessions/SessionManager.scala](https://codecov.io/gh/apache/incubator-livy/pull/240/diff?src=pr=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXNzaW9ucy9TZXNzaW9uTWFuYWdlci5zY2FsYQ==) | `81.72% <100%> (-0.1%)` | `26 <1> (-1)` | | | [...che/livy/server/recovery/ZooKeeperStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/240/diff?src=pr=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvWm9vS2VlcGVyU3RhdGVTdG9yZS5zY2FsYQ==) | `85.71% <100%> (+6.64%)` | `5 <5> (-12)` | :arrow_down: | | [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/240/diff?src=pr=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `34.78% <20%> (-0.69%)` | `11 <0> (ø)` | | | [...a/org/apache/livy/server/recovery/StateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/240/diff?src=pr=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvU3RhdGVTdG9yZS5zY2FsYQ==) | `75.67% <62.5%> (-7.66%)` | `3 <3> (+2)` | | | [...che/livy/server/recovery/BlackholeStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/240/diff?src=pr=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvQmxhY2tob2xlU3RhdGVTdG9yZS5zY2FsYQ==) | `68.42% <66.66%> (-31.58%)` | `8 <7> (+3)` | | | [...apache/livy/server/recovery/ZooKeeperManager.scala](https://codecov.io/gh/apache/incubator-livy/pull/240/diff?src=pr=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvWm9vS2VlcGVyTWFuYWdlci5zY2FsYQ==) | `80.85% <80.85%> (ø)` | `18 <18> (?)` | | | ... and [1 more](https://codecov.io/gh/apache/incubator-livy/pull/240/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/240?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/240?src=pr=footer). Last update [0804c8e...9320dfa](https://codecov.io/gh/apache/incubator-livy/pull/240?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #231: [LIVY-356][SERVER]Add LDAP authentication for livy-server.
mgaido91 commented on a change in pull request #231: [LIVY-356][SERVER]Add LDAP
authentication for livy-server.
URL: https://github.com/apache/incubator-livy/pull/231#discussion_r329610645
##
File path:
server/src/main/scala/org/apache/livy/server/auth/LdapAuthenticationHandlerImpl.scala
##
@@ -0,0 +1,231 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.server.auth
+
+import java.io.IOException
+import java.nio.charset.StandardCharsets
+import java.util
+import java.util.Properties
+import javax.naming.NamingException
+import javax.naming.directory.InitialDirContext
+import javax.naming.ldap.{InitialLdapContext, StartTlsRequest,
StartTlsResponse}
+import javax.net.ssl.{HostnameVerifier, SSLSession}
+import javax.servlet.ServletException
+import javax.servlet.http.{HttpServletRequest, HttpServletResponse}
+
+import org.apache.commons.codec.binary.Base64
+import org.apache.hadoop.security.authentication.client.AuthenticationException
+import
org.apache.hadoop.security.authentication.server.{AuthenticationHandler,
AuthenticationToken}
+
+import org.apache.livy._
+
+object LdapAuthenticationHandlerImpl {
+
+ val AUTHORIZATION_SCHEME = "Basic"
+ val TYPE = "ldap"
+ val SECURITY_AUTHENTICATION = "simple"
+ val PROVIDER_URL = "ldap.providerurl"
+ val BASE_DN = "ldap.basedn"
+ val LDAP_BIND_DOMAIN = "ldap.binddomain"
+ val ENABLE_START_TLS = "ldap.enablestarttls"
+
+ private def hasDomain(userName: String): Boolean = {
+indexOfDomainMatch(userName) > 0
+ }
+
+ /**
+ * Get the index separating the user name from domain name (the user's name
up
+ * to the first '/' or '@').
+ */
+ private def indexOfDomainMatch(userName: String): Int = {
+if (userName == null) {
+ -1
+} else {
+ val idx = userName.indexOf('/')
+ val idx2 = userName.indexOf('@')
+ // Use the earlier match.
+ var endIdx = Math.min(idx, idx2)
+
+ // Unless at least one of '/' or '@' was not found, in
+ // which case, user the latter match.
+ if (endIdx == -1) Math.max(idx, idx2) else endIdx
+}
+ }
+}
+
+class LdapAuthenticationHandlerImpl extends AuthenticationHandler with Logging
{
+ private var ldapDomain = "null"
+ private var baseDN = "null"
+ private var providerUrl = "null"
+ private var enableStartTls = false
+ private var disableHostNameVerification = false
+
+ def getType: String = LdapAuthenticationHandlerImpl.TYPE
+
+ @throws[ServletException]
+ def init(config: Properties): Unit = {
+this.baseDN = config.getProperty(LdapAuthenticationHandlerImpl.BASE_DN)
+this.providerUrl =
config.getProperty(LdapAuthenticationHandlerImpl.PROVIDER_URL)
+this.ldapDomain =
config.getProperty(LdapAuthenticationHandlerImpl.LDAP_BIND_DOMAIN)
+this.enableStartTls =
config.getProperty(LdapAuthenticationHandlerImpl.ENABLE_START_TLS,
+ "false").toBoolean
+require(this.providerUrl != null, "The LDAP URI can not be null")
+
+if (this.enableStartTls.booleanValue) {
+ require(!this.providerUrl.toLowerCase.startsWith("ldaps"),
+"Can not use ldaps and StartTLS option at the same time")
+}
+ }
+
+ def destroy(): Unit = { }
+
+ @throws[IOException]
+ @throws[AuthenticationException]
+ def managementOperation(token: AuthenticationToken, request:
HttpServletRequest,
+response: HttpServletResponse) : Boolean = true
+
+ @throws[IOException]
+ @throws[AuthenticationException]
+ def authenticate(
+request: HttpServletRequest,
+response: HttpServletResponse): AuthenticationToken = {
+var token: AuthenticationToken = null
+var authorization = request.getHeader("Authorization")
+var regionMatch = false
+if (authorization != null) regionMatch = authorization.regionMatches(
+ true,
+ 0,
+ LdapAuthenticationHandlerImpl.AUTHORIZATION_SCHEME,
+ 0,
+ LdapAuthenticationHandlerImpl.AUTHORIZATION_SCHEME.length
+)
+
+if (authorization != null && regionMatch) {
+ authorization = authorization.substring("Basic".length).trim
+ val base64 = new Base64(0)
+ val credentials = new String(base64.decode(authorization),
+StandardCharsets.UTF_8).split(":", 2)
+
+
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #231: [LIVY-356][SERVER]Add LDAP authentication for livy-server.
mgaido91 commented on a change in pull request #231: [LIVY-356][SERVER]Add LDAP
authentication for livy-server.
URL: https://github.com/apache/incubator-livy/pull/231#discussion_r329609100
##
File path:
server/src/main/scala/org/apache/livy/server/auth/LdapAuthenticationHandlerImpl.scala
##
@@ -0,0 +1,231 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.server.auth
+
+import java.io.IOException
+import java.nio.charset.StandardCharsets
+import java.util
+import java.util.Properties
+import javax.naming.NamingException
+import javax.naming.directory.InitialDirContext
+import javax.naming.ldap.{InitialLdapContext, StartTlsRequest,
StartTlsResponse}
+import javax.net.ssl.{HostnameVerifier, SSLSession}
+import javax.servlet.ServletException
+import javax.servlet.http.{HttpServletRequest, HttpServletResponse}
+
+import org.apache.commons.codec.binary.Base64
+import org.apache.hadoop.security.authentication.client.AuthenticationException
+import
org.apache.hadoop.security.authentication.server.{AuthenticationHandler,
AuthenticationToken}
+
+import org.apache.livy._
+
+object LdapAuthenticationHandlerImpl {
+
+ val AUTHORIZATION_SCHEME = "Basic"
+ val TYPE = "ldap"
+ val SECURITY_AUTHENTICATION = "simple"
+ val PROVIDER_URL = "ldap.providerurl"
+ val BASE_DN = "ldap.basedn"
+ val LDAP_BIND_DOMAIN = "ldap.binddomain"
+ val ENABLE_START_TLS = "ldap.enablestarttls"
+
+ private def hasDomain(userName: String): Boolean = {
+indexOfDomainMatch(userName) > 0
+ }
+
+ /**
+ * Get the index separating the user name from domain name (the user's name
up
+ * to the first '/' or '@').
+ */
+ private def indexOfDomainMatch(userName: String): Int = {
+if (userName == null) {
Review comment:
how can this happen?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] runzhiwang opened a new pull request #240: [LIVY-541] Generate the unqiue session id when multiple livy servers work at the same time
runzhiwang opened a new pull request #240: [LIVY-541] Generate the unqiue session id when multiple livy servers work at the same time URL: https://github.com/apache/incubator-livy/pull/240 ## What changes were proposed in this pull request? [LIVY-541] Generate the unqiue session id when multiple livy servers work at the same time 1. When generate unique session id with multiple livy servers working. First, get the distributed lock, Second, get the session id from the filesystem or zookeeper. Third, increase the sessionid and save it in the filesystem or zookeeper. Forth, release the distributed lock. 2. ZooKeeperManager provides the distributed lock to generate the unique session id. 3. If set recovery.mode = off, i.e. generate session id in BlackholeStateStore, and set multi-active.enabled = true, store the session id in zookeeper. If set recovery.mode = off, and set multi-active.enabled = false, store the session id in memory, i.e. stateMap ## How was this patch tested? 1. Start multiple livy server and set multi-active.enabled = true. 2. Set zookeeper.url or state-store.url to save session id. 3. Create session in different livy servers at the same time. 4. Check the session id without duplicating. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] huianyi commented on a change in pull request #238: [LIVY-689] Deliver stage process message to the end user using thriftserver
huianyi commented on a change in pull request #238: [LIVY-689] Deliver stage
process message to the end user using thriftserver
URL: https://github.com/apache/incubator-livy/pull/238#discussion_r329540012
##
File path: rsc/src/main/java/org/apache/livy/rsc/RSCClient.java
##
@@ -413,6 +426,28 @@ private void handle(ChannelHandlerContext ctx, JobResult
msg) {
}
}
+private void handle(ChannelHandlerContext ctx, JobProcessMessage msg){
Review comment:
gonna change this format soon.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] huianyi closed pull request #238: [LIVY-689] Deliver stage process message to the end user using thriftserver
huianyi closed pull request #238: [LIVY-689] Deliver stage process message to the end user using thriftserver URL: https://github.com/apache/incubator-livy/pull/238 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] huianyi opened a new pull request #238: [LIVY-689] Deliver stage process message to the end user using thriftserver
huianyi opened a new pull request #238: [LIVY-689] Deliver stage process message to the end user using thriftserver URL: https://github.com/apache/incubator-livy/pull/238 ## What changes were proposed in this pull request? When we use beeline to connect with thriftserver, we must open the spark history server to get the stage process, which is unconvenient, we can deliver process message to the end user. ## How was this patch tested? In our environment, the log to the end user is below, ```log 0: jdbc:hive2://localhost:10090/> select count(*) from test1; Livy session has not yet started. Please wait for it to be ready... RSC client is executing SQL query: select count(*) from test1, statementId = a0a8223d-0900-45dd-a1da-9e392e566844, session = SessionHandle [3c2a6dc6-34c0-43db-a888-df8bdbb6302c] The query is submitted to remote server successfully. [Stage 0:> (0 + 0) / 25] [Stage 0:> (0 + 1) / 25] [Stage 0:> (0 + 1) / 25] [Stage 0:>(0 + 18) / 25] [Stage 0:>(0 + 20) / 25] [Stage 0:>(0 + 21) / 25] [Stage 0:>(0 + 25) / 25] [Stage 0:>(0 + 25) / 25] [Stage 0:>(0 + 25) / 25] [Stage 0:>(0 + 25) / 25] [Stage 0:>(0 + 25) / 25] [Stage 0:>(0 + 25) / 25] [Stage 0:>(0 + 25) / 25] [Stage 0:>(0 + 25) / 25] [Stage 0:>(0 + 25) / 25] [Stage 0:>(0 + 25) / 25] [Stage 0:>(0 + 25) / 25] [Stage 0:==> (1 + 24) / 25] [Stage 0:>(2 + 23) / 25] [Stage 0:=> (4 + 21) / 25] [Stage 0:==> (8 + 17) / 25] [Stage 0:> (11 + 14) / 25] [Stage 0:> (11 + 14) / 25] [Stage 0:=> (13 + 12) / 25] [Stage 0:==> (17 + 8) / 25] [Stage 0:==> (22 + 3) / 25] RSC client is fetching result schema for statementId = a0a8223d-0900-45dd-a1da-9e392e566844 +-+ | count(1) | +-+ | 1828008656 | +-+ 1 row selected (68.554 seconds) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] huianyi edited a comment on issue #238: [LIVY-689] Deliver stage process message to the end user using thriftserver
huianyi edited a comment on issue #238: [LIVY-689] Deliver stage process
message to the end user using thriftserver
URL: https://github.com/apache/incubator-livy/pull/238#issuecomment-536483452
> Hi @huianyi ，I have saw the ERROR in your uts. The root cause of your UT
failure is:
> Can't find class ConcurrentBoundedLinkedQueue in thriftserver .
>
> You can avoid this error by adding the following code in
thriftserver/session/pom.xml：
> ` org.apache.livy
livy-core_${scala.binary.version}
${project.version} provided `
Thanks for your very kind support, gonna push a new commit.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] huianyi commented on issue #238: [LIVY-689] Deliver stage process message to the end user using thriftserver
huianyi commented on issue #238: [LIVY-689] Deliver stage process message to
the end user using thriftserver
URL: https://github.com/apache/incubator-livy/pull/238#issuecomment-536483452
> Hi @huianyi ，I have saw the ERROR in your uts. The root cause of your UT
failure is:
> Can't find class ConcurrentBoundedLinkedQueue in thriftserver .
>
> You can avoid this error by adding the following code in
thriftserver/session/pom.xml：
> ` org.apache.livy
livy-core_${scala.binary.version}
${project.version} provided `
Thanks for your very kind support, gonna push a new commit.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] captainzmc commented on a change in pull request #238: [LIVY-689] Deliver stage process message to the end user using thriftserver
captainzmc commented on a change in pull request #238: [LIVY-689] Deliver stage
process message to the end user using thriftserver
URL: https://github.com/apache/incubator-livy/pull/238#discussion_r329485621
##
File path: rsc/src/main/java/org/apache/livy/rsc/RSCClient.java
##
@@ -413,6 +426,28 @@ private void handle(ChannelHandlerContext ctx, JobResult
msg) {
}
}
+private void handle(ChannelHandlerContext ctx, JobProcessMessage msg){
Review comment:
It feels better to use text descriptions. It is also easier to parse if we
need to collect logs:
Stage:0 Finished:8 Total:25; Running job progress: 32%
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] captainzmc commented on issue #238: [LIVY-689] Deliver stage process message to the end user using thriftserver
captainzmc commented on issue #238: [LIVY-689] Deliver stage process message to
the end user using thriftserver
URL: https://github.com/apache/incubator-livy/pull/238#issuecomment-536476557
Hi @huianyi ，I have saw the ERROR in your uts. The root cause of your UT
failure is:
Can't find class ConcurrentBoundedLinkedQueue in thriftserver .
You can avoid this error by adding the following code in
thriftserver/session/pom.xml：
org.apache.livy
livy-core_${scala.binary.version}
${project.version}
provided
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] captainzmc commented on issue #238: [LIVY-689] Deliver stage process message to the end user using thriftserver
captainzmc commented on issue #238: [LIVY-689] Deliver stage process message to
the end user using thriftserver
URL: https://github.com/apache/incubator-livy/pull/238#issuecomment-536476832
Hi @huianyi ，I have saw the ERROR in your uts. The root cause of your UT
failure is:
Can't find class ConcurrentBoundedLinkedQueue in thriftserver .
You can avoid this error by adding the following code in
thriftserver/session/pom.xml：
`
org.apache.livy
livy-core_${scala.binary.version}
${project.version}
provided
`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] captainzmc removed a comment on issue #238: [LIVY-689] Deliver stage process message to the end user using thriftserver
captainzmc removed a comment on issue #238: [LIVY-689] Deliver stage process
message to the end user using thriftserver
URL: https://github.com/apache/incubator-livy/pull/238#issuecomment-536476557
Hi @huianyi ，I have saw the ERROR in your uts. The root cause of your UT
failure is:
Can't find class ConcurrentBoundedLinkedQueue in thriftserver .
You can avoid this error by adding the following code in
thriftserver/session/pom.xml：
org.apache.livy
livy-core_${scala.binary.version}
${project.version}
provided
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] captainzmc commented on a change in pull request #231: [LIVY-356][SERVER]Add LDAP authentication for livy-server.
captainzmc commented on a change in pull request #231: [LIVY-356][SERVER]Add
LDAP authentication for livy-server.
URL: https://github.com/apache/incubator-livy/pull/231#discussion_r329444807
##
File path:
server/src/main/scala/org/apache/livy/server/auth/LdapAuthenticationHandlerImpl.scala
##
@@ -0,0 +1,231 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.server.auth
+
+import java.io.IOException
+import java.nio.charset.StandardCharsets
+import java.util
+import java.util.Properties
+import javax.naming.NamingException
+import javax.naming.directory.InitialDirContext
+import javax.naming.ldap.{InitialLdapContext, StartTlsRequest,
StartTlsResponse}
+import javax.net.ssl.{HostnameVerifier, SSLSession}
+import javax.servlet.ServletException
+import javax.servlet.http.{HttpServletRequest, HttpServletResponse}
+
+import org.apache.commons.codec.binary.Base64
+import org.apache.hadoop.security.authentication.client.AuthenticationException
+import
org.apache.hadoop.security.authentication.server.{AuthenticationHandler,
AuthenticationToken}
+
+import org.apache.livy._
+
+object LdapAuthenticationHandlerImpl {
+
+ val AUTHORIZATION_SCHEME = "Basic"
+ val TYPE = "ldap"
+ val SECURITY_AUTHENTICATION = "simple"
+ val PROVIDER_URL = "ldap.providerurl"
+ val BASE_DN = "ldap.basedn"
+ val LDAP_BIND_DOMAIN = "ldap.binddomain"
+ val ENABLE_START_TLS = "ldap.enablestarttls"
+
+ private def hasDomain(userName: String): Boolean = {
+indexOfDomainMatch(userName) > 0
+ }
+
+ /**
+ * Get the index separating the user name from domain name (the user's name
up
+ * to the first '/' or '@').
+ */
+ private def indexOfDomainMatch(userName: String): Int = {
+if (userName == null) {
+ -1
+} else {
+ val idx = userName.indexOf('/')
+ val idx2 = userName.indexOf('@')
+ // Use the earlier match.
+ var endIdx = Math.min(idx, idx2)
+
+ // Unless at least one of '/' or '@' was not found, in
+ // which case, user the latter match.
+ if (endIdx == -1) Math.max(idx, idx2) else endIdx
+}
+ }
+}
+
+class LdapAuthenticationHandlerImpl extends AuthenticationHandler with Logging
{
+ private var ldapDomain = "null"
+ private var baseDN = "null"
+ private var providerUrl = "null"
+ private var enableStartTls = false
+ private var disableHostNameVerification = false
+
+ def getType: String = LdapAuthenticationHandlerImpl.TYPE
+
+ @throws[ServletException]
+ def init(config: Properties): Unit = {
+this.baseDN = config.getProperty(LdapAuthenticationHandlerImpl.BASE_DN)
+this.providerUrl =
config.getProperty(LdapAuthenticationHandlerImpl.PROVIDER_URL)
+this.ldapDomain =
config.getProperty(LdapAuthenticationHandlerImpl.LDAP_BIND_DOMAIN)
+this.enableStartTls =
config.getProperty(LdapAuthenticationHandlerImpl.ENABLE_START_TLS,
+ "false").toBoolean
+require(this.providerUrl != null, "The LDAP URI can not be null")
+
+if (this.enableStartTls.booleanValue) {
+ require(!this.providerUrl.toLowerCase.startsWith("ldaps"),
+"Can not use ldaps and StartTLS option at the same time")
+}
+ }
+
+ def destroy(): Unit = { }
+
+ @throws[IOException]
+ @throws[AuthenticationException]
+ def managementOperation(token: AuthenticationToken, request:
HttpServletRequest,
+response: HttpServletResponse) : Boolean = true
+
+ @throws[IOException]
+ @throws[AuthenticationException]
+ def authenticate(
+request: HttpServletRequest,
+response: HttpServletResponse): AuthenticationToken = {
+var token: AuthenticationToken = null
+var authorization = request.getHeader("Authorization")
+var regionMatch = false
+if (authorization != null) regionMatch = authorization.regionMatches(
Review comment:
I'm going to reorganize this part.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on a change in pull request #231: [LIVY-356][SERVER]Add LDAP authentication for livy-server.
jerryshao commented on a change in pull request #231: [LIVY-356][SERVER]Add
LDAP authentication for livy-server.
URL: https://github.com/apache/incubator-livy/pull/231#discussion_r329432750
##
File path:
server/src/main/scala/org/apache/livy/server/auth/LdapAuthenticationHandlerImpl.scala
##
@@ -0,0 +1,231 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.server.auth
+
+import java.io.IOException
+import java.nio.charset.StandardCharsets
+import java.util
+import java.util.Properties
+import javax.naming.NamingException
+import javax.naming.directory.InitialDirContext
+import javax.naming.ldap.{InitialLdapContext, StartTlsRequest,
StartTlsResponse}
+import javax.net.ssl.{HostnameVerifier, SSLSession}
+import javax.servlet.ServletException
+import javax.servlet.http.{HttpServletRequest, HttpServletResponse}
+
+import org.apache.commons.codec.binary.Base64
+import org.apache.hadoop.security.authentication.client.AuthenticationException
+import
org.apache.hadoop.security.authentication.server.{AuthenticationHandler,
AuthenticationToken}
+
+import org.apache.livy._
+
+object LdapAuthenticationHandlerImpl {
+
+ val AUTHORIZATION_SCHEME = "Basic"
+ val TYPE = "ldap"
+ val SECURITY_AUTHENTICATION = "simple"
+ val PROVIDER_URL = "ldap.providerurl"
+ val BASE_DN = "ldap.basedn"
+ val LDAP_BIND_DOMAIN = "ldap.binddomain"
+ val ENABLE_START_TLS = "ldap.enablestarttls"
+
+ private def hasDomain(userName: String): Boolean = {
+indexOfDomainMatch(userName) > 0
+ }
+
+ /**
+ * Get the index separating the user name from domain name (the user's name
up
+ * to the first '/' or '@').
+ */
+ private def indexOfDomainMatch(userName: String): Int = {
+if (userName == null) {
+ -1
+} else {
+ val idx = userName.indexOf('/')
+ val idx2 = userName.indexOf('@')
+ // Use the earlier match.
+ var endIdx = Math.min(idx, idx2)
+
+ // Unless at least one of '/' or '@' was not found, in
+ // which case, user the latter match.
+ if (endIdx == -1) Math.max(idx, idx2) else endIdx
+}
+ }
+}
+
+class LdapAuthenticationHandlerImpl extends AuthenticationHandler with Logging
{
+ private var ldapDomain = "null"
+ private var baseDN = "null"
+ private var providerUrl = "null"
+ private var enableStartTls = false
+ private var disableHostNameVerification = false
+
+ def getType: String = LdapAuthenticationHandlerImpl.TYPE
+
+ @throws[ServletException]
+ def init(config: Properties): Unit = {
+this.baseDN = config.getProperty(LdapAuthenticationHandlerImpl.BASE_DN)
+this.providerUrl =
config.getProperty(LdapAuthenticationHandlerImpl.PROVIDER_URL)
+this.ldapDomain =
config.getProperty(LdapAuthenticationHandlerImpl.LDAP_BIND_DOMAIN)
+this.enableStartTls =
config.getProperty(LdapAuthenticationHandlerImpl.ENABLE_START_TLS,
+ "false").toBoolean
+require(this.providerUrl != null, "The LDAP URI can not be null")
+
+if (this.enableStartTls.booleanValue) {
+ require(!this.providerUrl.toLowerCase.startsWith("ldaps"),
+"Can not use ldaps and StartTLS option at the same time")
+}
+ }
+
+ def destroy(): Unit = { }
+
+ @throws[IOException]
+ @throws[AuthenticationException]
+ def managementOperation(token: AuthenticationToken, request:
HttpServletRequest,
+response: HttpServletResponse) : Boolean = true
Review comment:
nit: one parameter per line.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on a change in pull request #231: [LIVY-356][SERVER]Add LDAP authentication for livy-server.
jerryshao commented on a change in pull request #231: [LIVY-356][SERVER]Add
LDAP authentication for livy-server.
URL: https://github.com/apache/incubator-livy/pull/231#discussion_r329434810
##
File path:
server/src/main/scala/org/apache/livy/server/auth/LdapAuthenticationHandlerImpl.scala
##
@@ -0,0 +1,231 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.server.auth
+
+import java.io.IOException
+import java.nio.charset.StandardCharsets
+import java.util
+import java.util.Properties
+import javax.naming.NamingException
+import javax.naming.directory.InitialDirContext
+import javax.naming.ldap.{InitialLdapContext, StartTlsRequest,
StartTlsResponse}
+import javax.net.ssl.{HostnameVerifier, SSLSession}
+import javax.servlet.ServletException
+import javax.servlet.http.{HttpServletRequest, HttpServletResponse}
+
+import org.apache.commons.codec.binary.Base64
+import org.apache.hadoop.security.authentication.client.AuthenticationException
+import
org.apache.hadoop.security.authentication.server.{AuthenticationHandler,
AuthenticationToken}
+
+import org.apache.livy._
+
+object LdapAuthenticationHandlerImpl {
+
+ val AUTHORIZATION_SCHEME = "Basic"
+ val TYPE = "ldap"
+ val SECURITY_AUTHENTICATION = "simple"
+ val PROVIDER_URL = "ldap.providerurl"
+ val BASE_DN = "ldap.basedn"
+ val LDAP_BIND_DOMAIN = "ldap.binddomain"
+ val ENABLE_START_TLS = "ldap.enablestarttls"
+
+ private def hasDomain(userName: String): Boolean = {
+indexOfDomainMatch(userName) > 0
+ }
+
+ /**
+ * Get the index separating the user name from domain name (the user's name
up
+ * to the first '/' or '@').
+ */
+ private def indexOfDomainMatch(userName: String): Int = {
+if (userName == null) {
+ -1
+} else {
+ val idx = userName.indexOf('/')
+ val idx2 = userName.indexOf('@')
+ // Use the earlier match.
+ var endIdx = Math.min(idx, idx2)
+
+ // Unless at least one of '/' or '@' was not found, in
+ // which case, user the latter match.
+ if (endIdx == -1) Math.max(idx, idx2) else endIdx
+}
+ }
+}
+
+class LdapAuthenticationHandlerImpl extends AuthenticationHandler with Logging
{
+ private var ldapDomain = "null"
+ private var baseDN = "null"
+ private var providerUrl = "null"
+ private var enableStartTls = false
+ private var disableHostNameVerification = false
+
+ def getType: String = LdapAuthenticationHandlerImpl.TYPE
+
+ @throws[ServletException]
+ def init(config: Properties): Unit = {
+this.baseDN = config.getProperty(LdapAuthenticationHandlerImpl.BASE_DN)
+this.providerUrl =
config.getProperty(LdapAuthenticationHandlerImpl.PROVIDER_URL)
+this.ldapDomain =
config.getProperty(LdapAuthenticationHandlerImpl.LDAP_BIND_DOMAIN)
+this.enableStartTls =
config.getProperty(LdapAuthenticationHandlerImpl.ENABLE_START_TLS,
+ "false").toBoolean
+require(this.providerUrl != null, "The LDAP URI can not be null")
+
+if (this.enableStartTls.booleanValue) {
+ require(!this.providerUrl.toLowerCase.startsWith("ldaps"),
+"Can not use ldaps and StartTLS option at the same time")
+}
+ }
+
+ def destroy(): Unit = { }
+
+ @throws[IOException]
+ @throws[AuthenticationException]
+ def managementOperation(token: AuthenticationToken, request:
HttpServletRequest,
+response: HttpServletResponse) : Boolean = true
+
+ @throws[IOException]
+ @throws[AuthenticationException]
+ def authenticate(
+request: HttpServletRequest,
+response: HttpServletResponse): AuthenticationToken = {
+var token: AuthenticationToken = null
+var authorization = request.getHeader("Authorization")
+var regionMatch = false
+if (authorization != null) regionMatch = authorization.regionMatches(
+ true,
+ 0,
+ LdapAuthenticationHandlerImpl.AUTHORIZATION_SCHEME,
+ 0,
+ LdapAuthenticationHandlerImpl.AUTHORIZATION_SCHEME.length
+)
+
+if (authorization != null && regionMatch) {
+ authorization = authorization.substring("Basic".length).trim
+ val base64 = new Base64(0)
+ val credentials = new String(base64.decode(authorization),
+StandardCharsets.UTF_8).split(":", 2)
+
+
[GitHub] [incubator-livy] jerryshao commented on a change in pull request #231: [LIVY-356][SERVER]Add LDAP authentication for livy-server.
jerryshao commented on a change in pull request #231: [LIVY-356][SERVER]Add
LDAP authentication for livy-server.
URL: https://github.com/apache/incubator-livy/pull/231#discussion_r329432160
##
File path:
server/src/main/scala/org/apache/livy/server/auth/LdapAuthenticationHandlerImpl.scala
##
@@ -0,0 +1,206 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.server.auth
+
+import java.io.IOException
+import java.nio.charset.StandardCharsets
+import java.util
+import java.util.Properties
+import javax.naming.NamingException
+import javax.naming.directory.InitialDirContext
+import javax.naming.ldap.{Control, InitialLdapContext, StartTlsRequest,
StartTlsResponse}
+import javax.net.ssl.{HostnameVerifier, SSLSession}
+import javax.servlet.ServletException
+import javax.servlet.http.{HttpServletRequest, HttpServletResponse}
+
+import org.apache.commons.codec.binary.Base64
+import org.apache.hadoop.security.authentication.client.AuthenticationException
+import
org.apache.hadoop.security.authentication.server.{AuthenticationHandler,
AuthenticationToken}
+
+import org.apache.livy._
+
+object LdapAuthenticationHandlerImpl {
+ val AUTHORIZATION_SCHEME = "Basic"
+ val TYPE = "ldap"
+ val SECURITY_AUTHENTICATION = "simple"
+ val PROVIDER_URL = "ldap.providerurl"
+ val BASE_DN = "ldap.basedn"
+ val LDAP_BIND_DOMAIN = "ldap.binddomain"
+ val ENABLE_START_TLS = "ldap.enablestarttls"
+
+ private def hasDomain(userName: String): Boolean = {
+indexOfDomainMatch(userName) > 0
+ }
+
+ private def indexOfDomainMatch(userName: String): Integer = {
+if (userName == null) {
+ -1
+} else {
+ val idx = userName.indexOf(47)
+ val idx2 = userName.indexOf(64)
+ var endIdx = Math.min(idx, idx2)
+ if (endIdx == -1) endIdx = Math.max(idx, idx2)
+ endIdx
+}
+ }
+}
+
+class LdapAuthenticationHandlerImpl extends AuthenticationHandler with Logging
{
+ private var ldapDomain = "null"
+ private var baseDN = "null"
+ private var providerUrl = "null"
+ private var enableStartTls = false
+ private var disableHostNameVerification = false
+
+ def getType: String = LdapAuthenticationHandlerImpl.TYPE
+
+ @throws[ServletException]
+ def init(config: Properties): Unit = {
+this.baseDN = config.getProperty(LdapAuthenticationHandlerImpl.BASE_DN)
+this.providerUrl =
config.getProperty(LdapAuthenticationHandlerImpl.PROVIDER_URL)
+this.ldapDomain =
config.getProperty(LdapAuthenticationHandlerImpl.LDAP_BIND_DOMAIN)
+this.enableStartTls =
config.getProperty(LdapAuthenticationHandlerImpl.ENABLE_START_TLS,
+ "false").toBoolean
+require(this.providerUrl != null, "The LDAP URI can not be null")
+if (this.enableStartTls.booleanValue) {
Review comment:
Why do you call `booleanValue` on a boolean value?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on a change in pull request #231: [LIVY-356][SERVER]Add LDAP authentication for livy-server.
jerryshao commented on a change in pull request #231: [LIVY-356][SERVER]Add
LDAP authentication for livy-server.
URL: https://github.com/apache/incubator-livy/pull/231#discussion_r329435696
##
File path:
server/src/main/scala/org/apache/livy/server/auth/LdapAuthenticationHandlerImpl.scala
##
@@ -0,0 +1,231 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.server.auth
+
+import java.io.IOException
+import java.nio.charset.StandardCharsets
+import java.util
+import java.util.Properties
+import javax.naming.NamingException
+import javax.naming.directory.InitialDirContext
+import javax.naming.ldap.{InitialLdapContext, StartTlsRequest,
StartTlsResponse}
+import javax.net.ssl.{HostnameVerifier, SSLSession}
+import javax.servlet.ServletException
+import javax.servlet.http.{HttpServletRequest, HttpServletResponse}
+
+import org.apache.commons.codec.binary.Base64
+import org.apache.hadoop.security.authentication.client.AuthenticationException
+import
org.apache.hadoop.security.authentication.server.{AuthenticationHandler,
AuthenticationToken}
+
+import org.apache.livy._
+
+object LdapAuthenticationHandlerImpl {
+
+ val AUTHORIZATION_SCHEME = "Basic"
+ val TYPE = "ldap"
+ val SECURITY_AUTHENTICATION = "simple"
+ val PROVIDER_URL = "ldap.providerurl"
+ val BASE_DN = "ldap.basedn"
+ val LDAP_BIND_DOMAIN = "ldap.binddomain"
+ val ENABLE_START_TLS = "ldap.enablestarttls"
+
+ private def hasDomain(userName: String): Boolean = {
+indexOfDomainMatch(userName) > 0
+ }
+
+ /**
+ * Get the index separating the user name from domain name (the user's name
up
+ * to the first '/' or '@').
+ */
+ private def indexOfDomainMatch(userName: String): Int = {
+if (userName == null) {
+ -1
+} else {
+ val idx = userName.indexOf('/')
+ val idx2 = userName.indexOf('@')
+ // Use the earlier match.
+ var endIdx = Math.min(idx, idx2)
+
+ // Unless at least one of '/' or '@' was not found, in
+ // which case, user the latter match.
+ if (endIdx == -1) Math.max(idx, idx2) else endIdx
+}
+ }
+}
+
+class LdapAuthenticationHandlerImpl extends AuthenticationHandler with Logging
{
+ private var ldapDomain = "null"
+ private var baseDN = "null"
+ private var providerUrl = "null"
+ private var enableStartTls = false
+ private var disableHostNameVerification = false
+
+ def getType: String = LdapAuthenticationHandlerImpl.TYPE
+
+ @throws[ServletException]
+ def init(config: Properties): Unit = {
+this.baseDN = config.getProperty(LdapAuthenticationHandlerImpl.BASE_DN)
+this.providerUrl =
config.getProperty(LdapAuthenticationHandlerImpl.PROVIDER_URL)
+this.ldapDomain =
config.getProperty(LdapAuthenticationHandlerImpl.LDAP_BIND_DOMAIN)
+this.enableStartTls =
config.getProperty(LdapAuthenticationHandlerImpl.ENABLE_START_TLS,
+ "false").toBoolean
+require(this.providerUrl != null, "The LDAP URI can not be null")
+
+if (this.enableStartTls.booleanValue) {
+ require(!this.providerUrl.toLowerCase.startsWith("ldaps"),
+"Can not use ldaps and StartTLS option at the same time")
+}
+ }
+
+ def destroy(): Unit = { }
+
+ @throws[IOException]
+ @throws[AuthenticationException]
+ def managementOperation(token: AuthenticationToken, request:
HttpServletRequest,
+response: HttpServletResponse) : Boolean = true
+
+ @throws[IOException]
+ @throws[AuthenticationException]
+ def authenticate(
+request: HttpServletRequest,
+response: HttpServletResponse): AuthenticationToken = {
+var token: AuthenticationToken = null
+var authorization = request.getHeader("Authorization")
+var regionMatch = false
+if (authorization != null) regionMatch = authorization.regionMatches(
+ true,
+ 0,
+ LdapAuthenticationHandlerImpl.AUTHORIZATION_SCHEME,
+ 0,
+ LdapAuthenticationHandlerImpl.AUTHORIZATION_SCHEME.length
+)
+
+if (authorization != null && regionMatch) {
+ authorization = authorization.substring("Basic".length).trim
+ val base64 = new Base64(0)
+ val credentials = new String(base64.decode(authorization),
+StandardCharsets.UTF_8).split(":", 2)
+
+
[GitHub] [incubator-livy] jerryshao commented on a change in pull request #231: [LIVY-356][SERVER]Add LDAP authentication for livy-server.
jerryshao commented on a change in pull request #231: [LIVY-356][SERVER]Add
LDAP authentication for livy-server.
URL: https://github.com/apache/incubator-livy/pull/231#discussion_r329433134
##
File path:
server/src/main/scala/org/apache/livy/server/auth/LdapAuthenticationHandlerImpl.scala
##
@@ -0,0 +1,231 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.server.auth
+
+import java.io.IOException
+import java.nio.charset.StandardCharsets
+import java.util
+import java.util.Properties
+import javax.naming.NamingException
+import javax.naming.directory.InitialDirContext
+import javax.naming.ldap.{InitialLdapContext, StartTlsRequest,
StartTlsResponse}
+import javax.net.ssl.{HostnameVerifier, SSLSession}
+import javax.servlet.ServletException
+import javax.servlet.http.{HttpServletRequest, HttpServletResponse}
+
+import org.apache.commons.codec.binary.Base64
+import org.apache.hadoop.security.authentication.client.AuthenticationException
+import
org.apache.hadoop.security.authentication.server.{AuthenticationHandler,
AuthenticationToken}
+
+import org.apache.livy._
+
+object LdapAuthenticationHandlerImpl {
+
+ val AUTHORIZATION_SCHEME = "Basic"
+ val TYPE = "ldap"
+ val SECURITY_AUTHENTICATION = "simple"
+ val PROVIDER_URL = "ldap.providerurl"
+ val BASE_DN = "ldap.basedn"
+ val LDAP_BIND_DOMAIN = "ldap.binddomain"
+ val ENABLE_START_TLS = "ldap.enablestarttls"
+
+ private def hasDomain(userName: String): Boolean = {
+indexOfDomainMatch(userName) > 0
+ }
+
+ /**
+ * Get the index separating the user name from domain name (the user's name
up
+ * to the first '/' or '@').
+ */
+ private def indexOfDomainMatch(userName: String): Int = {
+if (userName == null) {
+ -1
+} else {
+ val idx = userName.indexOf('/')
+ val idx2 = userName.indexOf('@')
+ // Use the earlier match.
+ var endIdx = Math.min(idx, idx2)
+
+ // Unless at least one of '/' or '@' was not found, in
+ // which case, user the latter match.
+ if (endIdx == -1) Math.max(idx, idx2) else endIdx
+}
+ }
+}
+
+class LdapAuthenticationHandlerImpl extends AuthenticationHandler with Logging
{
+ private var ldapDomain = "null"
+ private var baseDN = "null"
+ private var providerUrl = "null"
+ private var enableStartTls = false
+ private var disableHostNameVerification = false
+
+ def getType: String = LdapAuthenticationHandlerImpl.TYPE
+
+ @throws[ServletException]
+ def init(config: Properties): Unit = {
+this.baseDN = config.getProperty(LdapAuthenticationHandlerImpl.BASE_DN)
+this.providerUrl =
config.getProperty(LdapAuthenticationHandlerImpl.PROVIDER_URL)
+this.ldapDomain =
config.getProperty(LdapAuthenticationHandlerImpl.LDAP_BIND_DOMAIN)
+this.enableStartTls =
config.getProperty(LdapAuthenticationHandlerImpl.ENABLE_START_TLS,
+ "false").toBoolean
+require(this.providerUrl != null, "The LDAP URI can not be null")
+
+if (this.enableStartTls.booleanValue) {
+ require(!this.providerUrl.toLowerCase.startsWith("ldaps"),
+"Can not use ldaps and StartTLS option at the same time")
+}
+ }
+
+ def destroy(): Unit = { }
+
+ @throws[IOException]
+ @throws[AuthenticationException]
+ def managementOperation(token: AuthenticationToken, request:
HttpServletRequest,
+response: HttpServletResponse) : Boolean = true
+
+ @throws[IOException]
+ @throws[AuthenticationException]
+ def authenticate(
+request: HttpServletRequest,
+response: HttpServletResponse): AuthenticationToken = {
+var token: AuthenticationToken = null
+var authorization = request.getHeader("Authorization")
+var regionMatch = false
+if (authorization != null) regionMatch = authorization.regionMatches(
Review comment:
weird code style here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on issue #230: [WIP][LIVY-664][Server] Spark application still running when Livy session creating was rejected
jerryshao commented on issue #230: [WIP][LIVY-664][Server] Spark application still running when Livy session creating was rejected URL: https://github.com/apache/incubator-livy/pull/230#issuecomment-536426983 Is this PR still WIP? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
