I have tried to perform the below operation.
My Input File(.dat)-->3,00,000 rows.
*Objective:* Move the data from '.dat' file into SQLServer.
I can able to Store the data in SQL Server by using combination of above
processors.But it takes almost 4-5 hrs to move complete data into SQLServer.
Combination of SplitText's perform data read quickly.But Extract Text takes
long time to pass given data matches with user defined expression.If input
comes 107 MB but it send outputs in KB size only even ReplaceText processor
also processing data in KB Size only.
In accordance with above slow processing leads the more time taken for data
Extract Text,ReplaceText,ConvertJsonToSQL processors send's outgoing flow
file in Kilobytes only.
If i have specify concurrent tasks for those
ExtractText,ReplaceText,ConvertJsonToSQL then it occupy the 100% cpu and
It just 30 MB data ,But processors takes 6 hrs for data movement into
Faced Problem is..,
1. Almost 6 hrs taken for move the 3lakhs data into SQL Server.
2. ExtractText,ReplaceText take long time for processing data(it
send output flowfile kb size only).
Can anyone help me to solve below *requirement*?
Need to reduce the number of time taken by the processors for move the
lakhs of data into SQL Server.
If anything i'm done wrong,please help me to done it right.