[jira] [Commented] (FLUME-2912) thrift Sources/Sinks can only authenticate with kerberos principal in format with hostname
[ https://issues.apache.org/jira/browse/FLUME-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15360125#comment-15360125 ] Lior Zeno commented on FLUME-2912: -- [~jrufus], what's your status on this? > thrift Sources/Sinks can only authenticate with kerberos principal in format > with hostname > --- > > Key: FLUME-2912 > URL: https://issues.apache.org/jira/browse/FLUME-2912 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources >Affects Versions: v1.6.0 >Reporter: Ping Wang >Assignee: Johny Rufus > Fix For: v1.7.0 > > > Using Thrift Sources/Sinks in Kerberos environment, the Flume agents > only work with principle in format "name/_h...@your-realm.com". > If using other valid principle in the format "n...@your-realm.com" it will > hit ERROR of "GSS initiate failed". > Here's the configuration file: > g1.sources.source1.type = spooldir > g1.sources.source1.spoolDir = /test > g1.sources.source1.fileHeader = false > g1.sinks.sink1.type = thrift > g1.sinks.sink1.hostname = localhost > g1.sinks.sink1.port = 5 > g1.channels.channel1.type = memory > g1.channels.channel1.capacity = 1000 > g1.channels.channel1.transactionCapacity = 100 > g1.sources.source1.channels = channel1 > g1.sinks.sink1.channel = channel1 > g2.sources = source2 > g2.sinks = sink2 > g2.channels = channel2 > g2.sources.source2.type = thrift > g2.sources.source2.bind = localhost > g2.sources.source2.port = 5 > g2.sinks.sink2.type = hdfs > g2.sinks.sink2.hdfs.path = /tmp > g2.sinks.sink2.hdfs.filePrefix = thriftData > g2.sinks.sink2.hdfs.writeFormat = Text > g2.sinks.sink2.hdfs.fileType = DataStream > g2.channels.channel2.type = memory > g2.channels.channel2.capacity = 1000 > g2.channels.channel2.transactionCapacity = 100 > g2.sources.source2.channels = channel2 > g2.sinks.sink2.channel = channel2 > g1.sinks.sink1.kerberos = true > g1.sinks.sink1.client-principal = flume/hostn...@xxx.com > g1.sinks.sink1.client-keytab > = /etc/security/keytabs/flume-1563.server.keytab > g1.sinks.sink1.server-principal = flume/hostn...@xxx.com > g2.sources.source2.kerberos = true > g2.sources.source2.agent-principal = flume/hostn...@xxx.com > g2.sources.source2.agent-keytab > = /etc/security/keytabs/flume-1563.server.keytab > If using other valid principle like "t...@ibm.com" as below, will hit error: > g1.sinks.sink1.kerberos = true > g1.sinks.sink1.client-principal = t...@ibm.com > g1.sinks.sink1.client-keytab = /home/test/test.keytab > g1.sinks.sink1.server-principal = t...@ibm.com > g2.sources.source2.kerberos = true > g2.sources.source2.agent-principal = t...@ibm.com > g2.sources.source2.agent-keytab = /home/test/test.keytab > Agent g1: > ERROR server.TThreadPoolServer: Error occurred during processing of > message. > java.lang.RuntimeException: > org.apache.thrift.transport.TTransportException: Peer indicated failure: > GSS initiate failed > at org.apache.thrift.transport.TSaslServerTransport > $Factory.getTransport(TSaslServerTransport.java:219) > at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run > (TThreadPoolServer.java:189) > at java.util.concurrent.ThreadPoolExecutor.runWorker > (ThreadPoolExecutor.java:1142) > Agent g2: > ERROR transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed [Caused by > GSSException: No valid credentials provided (Mechanism level: Server not > found in Kerberos database (7) - UNKNOWN_SERVER)] > at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge > (GssKrb5Client.java:211) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-2912) thrift Sources/Sinks can only authenticate with kerberos principal in format with hostname
[ https://issues.apache.org/jira/browse/FLUME-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15307931#comment-15307931 ] Johny Rufus commented on FLUME-2912: [~wpwang], I will take a look at this. > thrift Sources/Sinks can only authenticate with kerberos principal in format > with hostname > --- > > Key: FLUME-2912 > URL: https://issues.apache.org/jira/browse/FLUME-2912 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources >Affects Versions: v1.6.0 >Reporter: Ping Wang >Assignee: Johny Rufus > Fix For: v1.7.0 > > > Using Thrift Sources/Sinks in Kerberos environment, the Flume agents > only work with principle in format "name/_h...@your-realm.com". > If using other valid principle in the format "n...@your-realm.com" it will > hit ERROR of "GSS initiate failed". > Here's the configuration file: > g1.sources.source1.type = spooldir > g1.sources.source1.spoolDir = /test > g1.sources.source1.fileHeader = false > g1.sinks.sink1.type = thrift > g1.sinks.sink1.hostname = localhost > g1.sinks.sink1.port = 5 > g1.channels.channel1.type = memory > g1.channels.channel1.capacity = 1000 > g1.channels.channel1.transactionCapacity = 100 > g1.sources.source1.channels = channel1 > g1.sinks.sink1.channel = channel1 > g2.sources = source2 > g2.sinks = sink2 > g2.channels = channel2 > g2.sources.source2.type = thrift > g2.sources.source2.bind = localhost > g2.sources.source2.port = 5 > g2.sinks.sink2.type = hdfs > g2.sinks.sink2.hdfs.path = /tmp > g2.sinks.sink2.hdfs.filePrefix = thriftData > g2.sinks.sink2.hdfs.writeFormat = Text > g2.sinks.sink2.hdfs.fileType = DataStream > g2.channels.channel2.type = memory > g2.channels.channel2.capacity = 1000 > g2.channels.channel2.transactionCapacity = 100 > g2.sources.source2.channels = channel2 > g2.sinks.sink2.channel = channel2 > g1.sinks.sink1.kerberos = true > g1.sinks.sink1.client-principal = flume/hostn...@xxx.com > g1.sinks.sink1.client-keytab > = /etc/security/keytabs/flume-1563.server.keytab > g1.sinks.sink1.server-principal = flume/hostn...@xxx.com > g2.sources.source2.kerberos = true > g2.sources.source2.agent-principal = flume/hostn...@xxx.com > g2.sources.source2.agent-keytab > = /etc/security/keytabs/flume-1563.server.keytab > If using other valid principle like "t...@ibm.com" as below, will hit error: > g1.sinks.sink1.kerberos = true > g1.sinks.sink1.client-principal = t...@ibm.com > g1.sinks.sink1.client-keytab = /home/test/test.keytab > g1.sinks.sink1.server-principal = t...@ibm.com > g2.sources.source2.kerberos = true > g2.sources.source2.agent-principal = t...@ibm.com > g2.sources.source2.agent-keytab = /home/test/test.keytab > Agent g1: > ERROR server.TThreadPoolServer: Error occurred during processing of > message. > java.lang.RuntimeException: > org.apache.thrift.transport.TTransportException: Peer indicated failure: > GSS initiate failed > at org.apache.thrift.transport.TSaslServerTransport > $Factory.getTransport(TSaslServerTransport.java:219) > at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run > (TThreadPoolServer.java:189) > at java.util.concurrent.ThreadPoolExecutor.runWorker > (ThreadPoolExecutor.java:1142) > Agent g2: > ERROR transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed [Caused by > GSSException: No valid credentials provided (Mechanism level: Server not > found in Kerberos database (7) - UNKNOWN_SERVER)] > at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge > (GssKrb5Client.java:211) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-2912) thrift Sources/Sinks can only authenticate with kerberos principal in format with hostname
[ https://issues.apache.org/jira/browse/FLUME-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297753#comment-15297753 ] Ping Wang commented on FLUME-2912: -- The Thrift Src/Sink kerberos authentication were enabled via FLUME-2631. In our test, only the host-based service principal can do authentication, the simple principal like s...@example.com can not. This is not flexible and it's better to fix the strict limits. > thrift Sources/Sinks can only authenticate with kerberos principal in format > with hostname > --- > > Key: FLUME-2912 > URL: https://issues.apache.org/jira/browse/FLUME-2912 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources >Affects Versions: v1.6.0 >Reporter: Ping Wang > Fix For: v1.7.0 > > > Using Thrift Sources/Sinks in Kerberos environment, the Flume agents > only work with principle in format "name/_h...@your-realm.com". > If using other valid principle in the format "n...@your-realm.com" it will > hit ERROR of "GSS initiate failed". > Here's the configuration file: > g1.sources.source1.type = spooldir > g1.sources.source1.spoolDir = /test > g1.sources.source1.fileHeader = false > g1.sinks.sink1.type = thrift > g1.sinks.sink1.hostname = localhost > g1.sinks.sink1.port = 5 > g1.channels.channel1.type = memory > g1.channels.channel1.capacity = 1000 > g1.channels.channel1.transactionCapacity = 100 > g1.sources.source1.channels = channel1 > g1.sinks.sink1.channel = channel1 > g2.sources = source2 > g2.sinks = sink2 > g2.channels = channel2 > g2.sources.source2.type = thrift > g2.sources.source2.bind = localhost > g2.sources.source2.port = 5 > g2.sinks.sink2.type = hdfs > g2.sinks.sink2.hdfs.path = /tmp > g2.sinks.sink2.hdfs.filePrefix = thriftData > g2.sinks.sink2.hdfs.writeFormat = Text > g2.sinks.sink2.hdfs.fileType = DataStream > g2.channels.channel2.type = memory > g2.channels.channel2.capacity = 1000 > g2.channels.channel2.transactionCapacity = 100 > g2.sources.source2.channels = channel2 > g2.sinks.sink2.channel = channel2 > g1.sinks.sink1.kerberos = true > g1.sinks.sink1.client-principal = flume/hostn...@xxx.com > g1.sinks.sink1.client-keytab > = /etc/security/keytabs/flume-1563.server.keytab > g1.sinks.sink1.server-principal = flume/hostn...@xxx.com > g2.sources.source2.kerberos = true > g2.sources.source2.agent-principal = flume/hostn...@xxx.com > g2.sources.source2.agent-keytab > = /etc/security/keytabs/flume-1563.server.keytab > If using other valid principle like "t...@ibm.com" as below, will hit error: > g1.sinks.sink1.kerberos = true > g1.sinks.sink1.client-principal = t...@ibm.com > g1.sinks.sink1.client-keytab = /home/test/test.keytab > g1.sinks.sink1.server-principal = t...@ibm.com > g2.sources.source2.kerberos = true > g2.sources.source2.agent-principal = t...@ibm.com > g2.sources.source2.agent-keytab = /home/test/test.keytab > Agent g1: > ERROR server.TThreadPoolServer: Error occurred during processing of > message. > java.lang.RuntimeException: > org.apache.thrift.transport.TTransportException: Peer indicated failure: > GSS initiate failed > at org.apache.thrift.transport.TSaslServerTransport > $Factory.getTransport(TSaslServerTransport.java:219) > at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run > (TThreadPoolServer.java:189) > at java.util.concurrent.ThreadPoolExecutor.runWorker > (ThreadPoolExecutor.java:1142) > Agent g2: > ERROR transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed [Caused by > GSSException: No valid credentials provided (Mechanism level: Server not > found in Kerberos database (7) - UNKNOWN_SERVER)] > at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge > (GssKrb5Client.java:211) -- This message was sent by Atlassian JIRA (v6.3.4#6332)