[
https://issues.apache.org/jira/browse/NIFI-9224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17417280#comment-17417280
]
Every edited comment on NIFI-9224 at 9/19/21, 3:13 AM:
-------------------------------------------------------
I have a 40G csv file, but my hardware is configured as 4C 8G, and I need to
convert one field of each to call a service at the far end into multiple
information values and write to another csv, regardless of sharding, how to
implement it, thank you
我的CSV文件有40G大小，但是我的硬件配置为4C
8G，我需要将每一条的一个字段调用远端的一个服务转换成多个信息值然后写入另一个csv，不分片的情况下如何实现呢，谢谢
was (Author: every888):
My CSV file is 40G in size, but my hardware is configured as 4C 8G, and I need
to convert one field of each to call a service at the far end into multiple
information values and write to another csv, regardless of sharding, how to
implement it, thank you
我的CSV文件有40G大小，但是我的硬件配置为4C
8G，我需要将每一条的一个字段调用远端的一个服务转换成多个信息值然后写入另一个csv，不分片的情况下如何实现呢，谢谢
> 按文件分片读取或写文件
> -----------
>
> Key: NIFI-9224
> URL: https://issues.apache.org/jira/browse/NIFI-9224
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Affects Versions: 1.15.0
> Reporter: Every
> Priority: Major
> Labels: fetchfile, fragment
>
> For very large files, fetchfile cannot be read to nifi at once, and no
> suitable processor shard read files are found, so expand the service and
> processor that reads files by line shards, writes files by shards, and makes
> it easy to process very large files with smaller resources.
> 针对超大文件，无法使用fetchfile一次性读取到nifi中，也没有找到合适的处理器分片读取文件，因此扩展按行分片读取文件，按分片写入文件的服务及处理器，便于使用较小的资源处理超大文件。
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
