[jira] [Commented] (NIFI-10143) ListFTP/GetFTP has issues with danish special characters (UTF-8)

[Paul Grey (Jira)]

Title: Message Title

Paul Grey commented on NIFI-10143   Re: ListFTP/GetFTP has issues with danish special characters (UTF-8) Thanks for those details. I guess what we've found is a disconnect between the native character encoding of Windows and its UTF-8 support for the FTP protocol.  When you create a folder or a file in Windows Explorer, the name is stored using the native encoding.  This name is only compatible with UTF-8 for characters up to code point 127 (which covers the English alphabet). (1) Since the representation of "æøå/æøå.txt" is different in code page 437 versus UTF-8, the MS FTP server does not "see" the folder/file requested by ListFTP.  Since Linux uses UTF-8 as its native encoding, these issues don't typically exist. (2) This might be the basis of a solution that will work for you.  Since this PutFTP has UTF-8 enabled, and MS FTP has UTF-8 enabled, the folder / filename is encoded as UTF-8 in the Windows filesystem, and it is thus visible to ListFTP.  When you do this, the Windows Explorer view of the filenames is garbled, as you noted, because Explorer doesn't understand the UTF-8 characters.  MS FTP understands UTF-8 (if enabled), but the filesystem does not. If PutFTP and ListFTP are used with UTF-8 disabled, then they communicate using the native encoding of the machine on which they are run.  If this native encoding matches that of the MS FTP server, this should work for any character representable in that encoding.  So this is why Danish works but Arabic doesn't; Danish characters can be represented in code page 437, but Arabic characters cannot. FileZilla must have been developed to have an awareness of both UTF-8 and the native machine encoding, and to try both when servicing a request. Since you earlier indicated a need for MS-AD and NTFS rights, you should be able to use MS FTP (with UTF-8 enabled) to do what you need (Put, List, Fetch).  You will want to avoid file / folder naming through Windows Explorer in your designated FTP folder. One other wrinkle.  It may be possible to open an MS command prompt on your FTP server and to switch its code page to be UTF-8.  You would then be able to create files / folders in that command prompt window that MS FTP would be able to serve to UTF-8 clients.  (These names would appear garbled in Windows Explorer.)  I recommend extreme caution if you consider any of the options discussed in this link. https://stackoverflow.com/questions/57131654/using-utf-8-encoding-chcp-65001-in-command-prompt-windows-powershell-window   Add Comment   This message was sent by Atlassian Jira (v8.20.10#820010-sha1:ace47f9)