[jira] [Created] (HDFS-16644) java.io.IOException Invalid token in javax.security.sasl.qop

2022-06-29 Thread Walter Su (Jira) 
Walter Su created HDFS-16644:


 Summary: java.io.IOException Invalid token in 
javax.security.sasl.qop
 Key: HDFS-16644
 URL: https://issues.apache.org/jira/browse/HDFS-16644
 Project: Hadoop HDFS
  Issue Type: Bug
Affects Versions: 3.2.1
Reporter: Walter Su


deployment:

server side: kerberos enabled cluster with jdk 1.8 and hdfs-server 3.2.1

client side:
I run command hadoop fs -put a test file, with kerberos ticket inited first, 
and use identical core-site.xml & hdfs-site.xml configuration.

 using client ver 3.2.1, it succeeds.

 using client ver 2.8.5, it succeeds.

 using client ver 2.10.1, it fails. The client side error info is:

org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient: SASL 
encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2022-06-27 01:06:15,781 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: 
DataNode{data=FSDataset{dirpath='[/mnt/disk1/hdfs, /mnt/***/hdfs, 
/mnt/***/hdfs, /mnt/***/hdfs]'}, localName='emr-worker-***.***:9866', 
datanodeUuid='b1c7f64a-6389-4739-bddf-***', xmitsInProgress=0}:Exception 
transfering block BP-1187699012-10.-***:blk_1119803380_46080919 to mirror 
10.*:9866
java.io.IOException: Invalid token in javax.security.sasl.qop: D
at 
org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil.readSaslMessage(DataTransferSaslUtil.java:220)

Once any client ver 2.10.1 connect to hdfs server, the DataNode no longer 
accepts any client connection, even client ver 3.2.1 cannot connects to hdfs 
server. The DataNode rejects any client connection. For a short time, all 
DataNodes rejects client connections. 

The problem exists even if I replace DataNode with ver 3.3.0 or replace java 
with jdk 11.
The problem is fixed if I replace DataNode with ver 3.2.0. I guess the problem 
is related to HDFS-13541




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Work logged] (HDFS-16642) [SBN read] Moving selecting inputstream from JN in EditlogTailer out of FSNLock

2022-06-29 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/HDFS-16642?focusedWorklogId=786215=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-786215
 ]

ASF GitHub Bot logged work on HDFS-16642:
-

Author: ASF GitHub Bot
Created on: 29/Jun/22 23:19
Start Date: 29/Jun/22 23:19
Worklog Time Spent: 10m 
  Work Description: jojochuang commented on PR #4497:
URL: https://github.com/apache/hadoop/pull/4497#issuecomment-1170586655

   Looks thread-safe to me. Has this patch been tested in a production 
environment?




Issue Time Tracking
---

Worklog Id: (was: 786215)
Time Spent: 0.5h  (was: 20m)

> [SBN read] Moving selecting inputstream from JN in EditlogTailer out of 
> FSNLock
> ---
>
> Key: HDFS-16642
> URL: https://issues.apache.org/jira/browse/HDFS-16642
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> EditlogTailer cost a long time for selecting InputStreams from Journalnode 
> while holding the write lock of FSNLock.
> During this period, 8020 handlers of Observer NameNode will be blocked by the 
> FSN Lock.
> In theory, selecting inputstreams from JournalNode does not involve changing 
> memory information in NameNode, so we can move the selecting out of the FSN 
> Lock, and it can improve the throughput of Observer NameNode.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-16642) [SBN read] Moving selecting inputstream from JN in EditlogTailer out of FSNLock

2022-06-29 Thread Wei-Chiu Chuang (Jira) 


 [ 
https://issues.apache.org/jira/browse/HDFS-16642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei-Chiu Chuang updated HDFS-16642:
---
Summary: [SBN read] Moving selecting inputstream from JN in EditlogTailer 
out of FSNLock  (was: [HDFS] Moving selecting inputstream from JN in 
EditlogTailer out of FSNLock)

> [SBN read] Moving selecting inputstream from JN in EditlogTailer out of 
> FSNLock
> ---
>
> Key: HDFS-16642
> URL: https://issues.apache.org/jira/browse/HDFS-16642
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> EditlogTailer cost a long time for selecting InputStreams from Journalnode 
> while holding the write lock of FSNLock.
> During this period, 8020 handlers of Observer NameNode will be blocked by the 
> FSN Lock.
> In theory, selecting inputstreams from JournalNode does not involve changing 
> memory information in NameNode, so we can move the selecting out of the FSN 
> Lock, and it can improve the throughput of Observer NameNode.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org