symious commented on pull request #2424:
URL: https://github.com/apache/ozone/pull/2424#issuecomment-882200589
@adoroszlai Thanks for the view.
Seems the config of "OZONE_OM_KEYNAME_CHARACTER_CHECK_ENABLED_KEY" will deny
```
* "\", "{", "}", "^", "<", ">", "#", "|", "%", "`", "[", "]", "~", "?"
* and Non-printable ASCII characters (128–255 decimal characters).
```
, but if the user does create a name contains something like "U-00000000",
it won't help with the issue.
I think it is the catastrophic outcome that is not acceptable to users. In
our scenario, SRE provides the basic env and we install the applications, we
are not quite sure if SRE has set the correcting encoding. So we start the
Ozone cluster, then it runs well for several months, suddenly the OM crashes
because users are creating some invalid keys.
Besides, the keys are created by the scheduling system, the invalid key was
created several hours after we told them to correct them, which means if we
don't check the "Paths.get", the OM cluster isn't available for several hours,
and it's not even decided by cluster maintainers, but the users.
It took some time before I found out it's related to the encoding. When the
outage happens, and the error message is so obvious, I won't even check the
env, so the recovery still relies on the users.
IMHO, the encoding of JVM should affect Ozone so much. If Ozone runs on
"ANSI_X3.4-1968", it should provide the functions on this encoding, and if this
encoding can not resolve a keyName, it should just return an INVALID_KEY_NAME
and reject. If runs on "UTF-8", it should reject the user requests as well if
it can not resolve.
Attached the original error log and keyName.
The original error is as follows:
```
java.nio.file.InvalidPathException: Malformed input or input contains
unmappable characters:
/vol_name/bucket_name/task-exec/di_scheduler_test.monhly_dudutestshell063002??prod_2021071519_HOUR_1_0_prod/main.sh
at sun.nio.fs.UnixPath.encode(UnixPath.java:147)
at sun.nio.fs.UnixPath.<init>(UnixPath.java:71)
at sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:281)
at java.nio.file.Paths.get(Paths.java:84)
at
org.apache.hadoop.ozone.util.RadixTree.getLongestPrefix(RadixTree.java:190)
at
org.apache.hadoop.ozone.om.PrefixManagerImpl.getLongestPrefixPath(PrefixManagerImpl.java:260)
at
org.apache.hadoop.ozone.om.request.key.OMKeyRequest.getAclsForKey(OMKeyRequest.java:284)
at
org.apache.hadoop.ozone.om.request.key.OMKeyRequest.createKeyInfo(OMKeyRequest.java:267)
at
org.apache.hadoop.ozone.om.request.key.OMKeyRequest.prepareKeyInfo(OMKeyRequest.java:352)
at
org.apache.hadoop.ozone.om.request.key.OMKeyCreateRequest.validateAndUpdateCache(OMKeyCreateRequest.java:278)
at
org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleWriteRequest(OzoneManagerRequestHandler.java:227)
at
org.apache.hadoop.ozone.om.ratis.OzoneManagerStateMachine.runCommand(OzoneManagerStateMachine.java:417)
at
org.apache.hadoop.ozone.om.ratis.OzoneManagerStateMachine.lambda$applyTransaction$1(OzoneManagerStateMachine.java:242)
at
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
The keyName user wants to create is
`task-exec/di_scheduler_test.monhly_dudutestshell063002\342\200\224\342\200\224prod_2021071519_HOUR_1_0_prod/main.sh`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
