[
https://issues.apache.org/jira/browse/NIFI-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seokwon Yang updated NIFI-8652:
-------------------------------
Description:
When we generate and insert bulk records into Azure Cosmos DB with
PutAzureCosmosDBRecord, we don't see update of the 'In' process rate in the
processor and see the warning, while we see the corresponding records in Azure
Cosmos DB side. I captured the screenshot in the below attachment.
For now, 2 workarounds I found to overcome this issue:
# Change the consistency property to 'Eventual'.
# If you keep consistency as 'Session', add a success relationship/connection
to dummy log processor.
The related warning in the log file:
2021-06-02 16:41:39,395 WARN [Long Running Task Monitor Thread-1]
o.a.n.c.monitor.LongRunningTaskMonitor Long running task detected on processor
[id=256082ee-341e-3c42-e379-5c5d365e9e18, name=PutAzureCosmosDBRecord,
type=PutAzureCosmosDBRecord]. Task time: 324 seconds. Stack trace:
"Timer-Driven Process Thread-5" Id=74 WAITING on
java.util.concurrent.CountDownLatch$Sync@307e6f8b
at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
at
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
at
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
at
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
at
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
at
[email protected]/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
at
reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:87)
at reactor.core.publisher.Mono.block(Mono.java:1678)
at com.azure.cosmos.CosmosContainer.blockItemResponse(CosmosContainer.java:210)
at com.azure.cosmos.CosmosContainer.createItem(CosmosContainer.java:144)
at
org.apache.nifi.processors.azure.cosmos.document.PutAzureCosmosDBRecord.bulkInsert(PutAzureCosmosDBRecord.java:134)
at
org.apache.nifi.processors.azure.cosmos.document.PutAzureCosmosDBRecord.onTrigger(PutAzureCosmosDBRecord.java:192)
at
app//org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1180)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:103)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at
[email protected]/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at
[email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at [email protected]/java.lang.Thread.run(Thread.java:829)
Number of Locked Synchronizers: 1
- java.util.concurrent.ThreadPoolExecutor$Worker@52bef30b
was:
When we generate and insert bulk records into Azure Cosmos DB with
PutAzureCosmosDBRecord, we don't see the 'In' process rate in the processor and
see the warning, while we see the corresponding records in Azure Cosmos DB
side. I captured the screenshot in the below attachment.
For now, 2 workarounds I found to overcome this issue:
# Change the consistency property to 'Eventual'.
# If you keep consistency as 'Session', add a success relationship/connection
to dummy log processor.
The related warning in the log file:
2021-06-02 16:41:39,395 WARN [Long Running Task Monitor Thread-1]
o.a.n.c.monitor.LongRunningTaskMonitor Long running task detected on processor
[id=256082ee-341e-3c42-e379-5c5d365e9e18, name=PutAzureCosmosDBRecord,
type=PutAzureCosmosDBRecord]. Task time: 324 seconds. Stack trace:
"Timer-Driven Process Thread-5" Id=74 WAITING on
java.util.concurrent.CountDownLatch$Sync@307e6f8b
at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
at
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
at
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
at
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
at
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
at
[email protected]/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
at
reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:87)
at reactor.core.publisher.Mono.block(Mono.java:1678)
at com.azure.cosmos.CosmosContainer.blockItemResponse(CosmosContainer.java:210)
at com.azure.cosmos.CosmosContainer.createItem(CosmosContainer.java:144)
at
org.apache.nifi.processors.azure.cosmos.document.PutAzureCosmosDBRecord.bulkInsert(PutAzureCosmosDBRecord.java:134)
at
org.apache.nifi.processors.azure.cosmos.document.PutAzureCosmosDBRecord.onTrigger(PutAzureCosmosDBRecord.java:192)
at
app//org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1180)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:103)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at
[email protected]/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at
[email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at [email protected]/java.lang.Thread.run(Thread.java:829)
Number of Locked Synchronizers: 1
- java.util.concurrent.ThreadPoolExecutor$Worker@52bef30b
> PutAzureCosmosDBRecord generates Long running task detected on processor
> warning and does not update 'In' process rate.
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-8652
> URL: https://issues.apache.org/jira/browse/NIFI-8652
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.13.2
> Reporter: Seokwon Yang
> Priority: Minor
> Labels: Azure
> Attachments: PutAzureCosmosDBRecordWarning.jpg
>
>
> When we generate and insert bulk records into Azure Cosmos DB with
> PutAzureCosmosDBRecord, we don't see update of the 'In' process rate in the
> processor and see the warning, while we see the corresponding records in
> Azure Cosmos DB side. I captured the screenshot in the below attachment.
>
> For now, 2 workarounds I found to overcome this issue:
> # Change the consistency property to 'Eventual'.
> # If you keep consistency as 'Session', add a success
> relationship/connection to dummy log processor.
>
> The related warning in the log file:
>
> 2021-06-02 16:41:39,395 WARN [Long Running Task Monitor Thread-1]
> o.a.n.c.monitor.LongRunningTaskMonitor Long running task detected on
> processor [id=256082ee-341e-3c42-e379-5c5d365e9e18,
> name=PutAzureCosmosDBRecord, type=PutAzureCosmosDBRecord]. Task time: 324
> seconds. Stack trace:
> "Timer-Driven Process Thread-5" Id=74 WAITING on
> java.util.concurrent.CountDownLatch$Sync@307e6f8b
> at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> at
> [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> at
> [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
> at
> [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
> at
> [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
> at
> [email protected]/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
> at
> reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:87)
> at reactor.core.publisher.Mono.block(Mono.java:1678)
> at
> com.azure.cosmos.CosmosContainer.blockItemResponse(CosmosContainer.java:210)
> at com.azure.cosmos.CosmosContainer.createItem(CosmosContainer.java:144)
> at
> org.apache.nifi.processors.azure.cosmos.document.PutAzureCosmosDBRecord.bulkInsert(PutAzureCosmosDBRecord.java:134)
> at
> org.apache.nifi.processors.azure.cosmos.document.PutAzureCosmosDBRecord.onTrigger(PutAzureCosmosDBRecord.java:192)
> at
> app//org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1180)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:103)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> [email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at
> [email protected]/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
> at
> [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
> at
> [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at [email protected]/java.lang.Thread.run(Thread.java:829)
> Number of Locked Synchronizers: 1
> - java.util.concurrent.ThreadPoolExecutor$Worker@52bef30b
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
