[
https://issues.apache.org/jira/browse/NIFI-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361536#comment-17361536
]
Arun Prakash edited comment on NIFI-8669 at 6/11/21, 9:17 AM:
--------------------------------------------------------------
[~jfrazee]
Thanks for taking a look at this issue.
I am running NiFi in a docker environment (Docker Desktop for Windows 10
version 21H1) with the latest image (apache/nifi:1.13.2)
{code:bash}
# which java
/usr/local/openjdk-8/bin/java
{code}
A bit more background on when the issue arises.
I am using the nifi-api to create ProcessGroups. I use, the following REST POST
request:
{code:java}
curl --location --request POST
'http://localhost:9090/nifi-api/process-groups/58d527f9-0178-1000-21a9-cf75fa6606c0/process-groups'
\
--header 'Content-Type: application/json' \
--data-raw '{
"revision": {
"version": 0
},
"component": {
"name": "test"
}
}'
{code}
which returns
{code:json}
{
"revision": {
"version": 1,
"lastModifier": "anonymous"
},
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"uri":
"http://localhost:9090/nifi-api/process-groups/fa4829ef-0179-1000-5e44-7fc558cbb6d3";,
"position": {
"x": 0.0,
"y": 0.0
},
"permissions": {
"canRead": true,
"canWrite": true
},
"bulletins": [],
"component": {
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"parentGroupId": "58d527f9-0178-1000-21a9-cf75fa6606c0",
"position": {
"x": 0.0,
"y": 0.0
},
"name": "test",
"comments": "",
"variables": {},
"flowfileConcurrency": "UNBOUNDED",
"flowfileOutboundPolicy": "STREAM_WHEN_AVAILABLE",
"runningCount": 0,
"stoppedCount": 0,
"invalidCount": 0,
"disabledCount": 0,
"activeRemotePortCount": 0,
"inactiveRemotePortCount": 0,
"upToDateCount": 0,
"locallyModifiedCount": 0,
"staleCount": 0,
"locallyModifiedAndStaleCount": 0,
"syncFailureCount": 0,
"localInputPortCount": 0,
"localOutputPortCount": 0,
"publicInputPortCount": 0,
"publicOutputPortCount": 0,
"contents": {
"processGroups": [],
"remoteProcessGroups": [],
"processors": [],
"inputPorts": [],
"outputPorts": [],
"connections": [],
"labels": [],
"funnels": [],
"controllerServices": []
},
"inputPortCount": 0,
"outputPortCount": 0
},
"status": {
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"name": "test",
"statsLastRefreshed": "08:55:04 UTC",
"aggregateSnapshot": {
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"name": "test",
"flowFilesIn": 0,
"bytesIn": 0,
"input": "0 (0 bytes)",
"flowFilesQueued": 0,
"bytesQueued": 0,
"queued": "0 (0 bytes)",
"queuedCount": "0",
"queuedSize": "0 bytes",
"bytesRead": 0,
"read": "0 bytes",
"bytesWritten": 0,
"written": "0 bytes",
"flowFilesOut": 0,
"bytesOut": 0,
"output": "0 (0 bytes)",
"flowFilesTransferred": 0,
"bytesTransferred": 0,
"transferred": "0 (0 bytes)",
"bytesReceived": 0,
"flowFilesReceived": 0,
"received": "0 (0 bytes)",
"bytesSent": 0,
"flowFilesSent": 0,
"sent": "0 (0 bytes)",
"activeThreadCount": 0,
"terminatedThreadCount": 0
}
},
"runningCount": 0,
"stoppedCount": 0,
"invalidCount": 0,
"disabledCount": 0,
"activeRemotePortCount": 0,
"inactiveRemotePortCount": 0,
"upToDateCount": 0,
"locallyModifiedCount": 0,
"staleCount": 0,
"locallyModifiedAndStaleCount": 0,
"syncFailureCount": 0,
"localInputPortCount": 0,
"localOutputPortCount": 0,
"publicInputPortCount": 0,
"publicOutputPortCount": 0,
"inputPortCount": 0,
"outputPortCount": 0
}
{code}
Decoding using the [https://www.uuidtools.com/decode] we can see that the *id*
{{"id": "fa4829ef-0179-*1*000-*5*e44-7fc558cbb6d3"}}
has
{code:java}
Version 1 (time and node based)
Variant reserved (NCS backward compatible)
{code}
which makes it unfortunately *INVALID*, due to the use of reserved characters
for the _Variant_
Assuming that the *POST* request is handled by:
{code:java}
org.apache.nifi.web.api.ProcessGroupResource#createProcessGroup()
{code}
the *UUID* is generated in line *2036*
{code:java}
// set the processor id as appropriate
processGroup.setId(generateUuid());
{code}
which in turn is a call to the inherited method in the abstract class
{{ApplicationResource}}
{code:java}
protected String generateUuid() {
final Optional<String> seed = getIdGenerationSeed();
UUID uuid;
if (seed.isPresent()) {
try {
UUID seedId = UUID.fromString(seed.get());
uuid = new UUID(seedId.getMostSignificantBits(),
seed.get().hashCode());
} catch (Exception e) {
logger.warn("Provided 'seed' does not represent UUID. Will not
be able to extract most significant bits for ID generation.");
uuid =
UUID.nameUUIDFromBytes(seed.get().getBytes(StandardCharsets.UTF_8));
}
} else {
uuid = ComponentIdGenerator.generateId();
}
return uuid.toString();
}
protected Optional<String> getIdGenerationSeed() {
final String idGenerationSeed =
httpServletRequest.getHeader(RequestReplicator.CLUSTER_ID_GENERATION_SEED_HEADER);
if (StringUtils.isBlank(idGenerationSeed)) {
return Optional.empty();
}
return Optional.of(idGenerationSeed);
}
{code}
was (Author: aprakash):
[~jfrazee]
Thanks for taking a look at this issue.
I am running NiFi in a docker environment (Docker Desktop for Windows 10
version 21H1) with the latest image (apache/nifi:1.13.2)
{code:bash}
# which java
/usr/local/openjdk-8/bin/java
{code}
A bit more background on when the issue arises.
I am using the nifi-api to create ProcessGroups. I use, the following REST POST
request:
{code:java}
curl --location --request POST
'http://localhost:9090/nifi-api/process-groups/58d527f9-0178-1000-21a9-cf75fa6606c0/process-groups'
\
--header 'Content-Type: application/json' \
--data-raw '{
"revision": {
"version": 0
},
"component": {
"name": "test"
}
}'
{code}
which returns
{code:json}
{
"revision": {
"version": 1,
"lastModifier": "anonymous"
},
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"uri":
"http://localhost:9090/nifi-api/process-groups/fa4829ef-0179-1000-5e44-7fc558cbb6d3";,
"position": {
"x": 0.0,
"y": 0.0
},
"permissions": {
"canRead": true,
"canWrite": true
},
"bulletins": [],
"component": {
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"parentGroupId": "58d527f9-0178-1000-21a9-cf75fa6606c0",
"position": {
"x": 0.0,
"y": 0.0
},
"name": "test",
"comments": "",
"variables": {},
"flowfileConcurrency": "UNBOUNDED",
"flowfileOutboundPolicy": "STREAM_WHEN_AVAILABLE",
"runningCount": 0,
"stoppedCount": 0,
"invalidCount": 0,
"disabledCount": 0,
"activeRemotePortCount": 0,
"inactiveRemotePortCount": 0,
"upToDateCount": 0,
"locallyModifiedCount": 0,
"staleCount": 0,
"locallyModifiedAndStaleCount": 0,
"syncFailureCount": 0,
"localInputPortCount": 0,
"localOutputPortCount": 0,
"publicInputPortCount": 0,
"publicOutputPortCount": 0,
"contents": {
"processGroups": [],
"remoteProcessGroups": [],
"processors": [],
"inputPorts": [],
"outputPorts": [],
"connections": [],
"labels": [],
"funnels": [],
"controllerServices": []
},
"inputPortCount": 0,
"outputPortCount": 0
},
"status": {
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"name": "test",
"statsLastRefreshed": "08:55:04 UTC",
"aggregateSnapshot": {
"id": "fa4829ef-0179-1000-5e44-7fc558cbb6d3",
"name": "test",
"flowFilesIn": 0,
"bytesIn": 0,
"input": "0 (0 bytes)",
"flowFilesQueued": 0,
"bytesQueued": 0,
"queued": "0 (0 bytes)",
"queuedCount": "0",
"queuedSize": "0 bytes",
"bytesRead": 0,
"read": "0 bytes",
"bytesWritten": 0,
"written": "0 bytes",
"flowFilesOut": 0,
"bytesOut": 0,
"output": "0 (0 bytes)",
"flowFilesTransferred": 0,
"bytesTransferred": 0,
"transferred": "0 (0 bytes)",
"bytesReceived": 0,
"flowFilesReceived": 0,
"received": "0 (0 bytes)",
"bytesSent": 0,
"flowFilesSent": 0,
"sent": "0 (0 bytes)",
"activeThreadCount": 0,
"terminatedThreadCount": 0
}
},
"runningCount": 0,
"stoppedCount": 0,
"invalidCount": 0,
"disabledCount": 0,
"activeRemotePortCount": 0,
"inactiveRemotePortCount": 0,
"upToDateCount": 0,
"locallyModifiedCount": 0,
"staleCount": 0,
"locallyModifiedAndStaleCount": 0,
"syncFailureCount": 0,
"localInputPortCount": 0,
"localOutputPortCount": 0,
"publicInputPortCount": 0,
"publicOutputPortCount": 0,
"inputPortCount": 0,
"outputPortCount": 0
}
{code}
Decoding using the [https://www.uuidtools.com/decode] we can see that the *id*
{{"id": "fa4829ef-0179-*1*000-*5*e44-7fc558cbb6d3"}}
has
{code:java}
Version 1 (time and node based)
Variant reserved (NCS backward compatible)
{code}
which makes it unfortunately *INVALID*, due to the use of reserved characters
for the _Variant_
Assuming that the *POST *request is handled by:
{code:java}
org.apache.nifi.web.api.ProcessGroupResource#createProcessGroup()
{code}
the UUID is generated in line 2036
{code:java}
// set the processor id as appropriate
processGroup.setId(generateUuid());
{code}
which in turn is a call to the inherited method in the abstract class
ApplicationResource
{code:java}
protected String generateUuid() {
final Optional<String> seed = getIdGenerationSeed();
UUID uuid;
if (seed.isPresent()) {
try {
UUID seedId = UUID.fromString(seed.get());
uuid = new UUID(seedId.getMostSignificantBits(),
seed.get().hashCode());
} catch (Exception e) {
logger.warn("Provided 'seed' does not represent UUID. Will not
be able to extract most significant bits for ID generation.");
uuid =
UUID.nameUUIDFromBytes(seed.get().getBytes(StandardCharsets.UTF_8));
}
} else {
uuid = ComponentIdGenerator.generateId();
}
return uuid.toString();
}
protected Optional<String> getIdGenerationSeed() {
final String idGenerationSeed =
httpServletRequest.getHeader(RequestReplicator.CLUSTER_ID_GENERATION_SEED_HEADER);
if (StringUtils.isBlank(idGenerationSeed)) {
return Optional.empty();
}
return Optional.of(idGenerationSeed);
}
{code}
> Invalid UUIDs generated
> -----------------------
>
> Key: NIFI-8669
> URL: https://issues.apache.org/jira/browse/NIFI-8669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.13.2
> Reporter: Arun Prakash
> Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> The UUIDs generated (type 3) are not fully compliant to the RFC.
> For instance:
> {code:java}
> 9adabada-3b4b-32eb-fd7e-c41191eefa2a
> 642ff728-2fb3-3e6b-f48f-0cfb94b772e2 {code}
> {{generated by NiFi as _id_ for components are based on type 3. For type 3,
> 4, and 5, the *UUID variant* has to be one of (8, 9, a, b)}}
> *Example A:*
> 9adabada-3b4b-*3*2eb-*f*d7e-c41191eefa2a
> This is a *uuid v3*, and variant character is *{{f}}*, and *{{f}}* is not a
> valid uuid variant (just 8, 9, a and b are valid). *NOT VALID*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
