[
https://issues.apache.org/jira/browse/DAFFODIL-1799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Lawrence resolved DAFFODIL-1799.
--------------------------------------
Resolution: Duplicate
Duplicate of alot of streaming related issues. Resolving.
> Enable data streaming in the CLI
> --------------------------------
>
> Key: DAFFODIL-1799
> URL: https://issues.apache.org/jira/browse/DAFFODIL-1799
> Project: Daffodil
> Issue Type: Bug
> Components: CLI, Performance
> Reporter: Steve Lawrence
> Priority: Major
> Fix For: 2.2.0
>
>
> This CLI currently reads the input data as a byte array. This is simple and
> allows for ensuring all data is read into a memory, reducing disk overhead
> during the preformance command. However, this means the CLI is limited to the
> maximum size of an array, which is INT_MAX. In order to support the CLI
> parsing/unparsing larger files, we should instead work on InputStreams rather
> than array buffers. For the performance subcommand, this will mean requiring
> something like a SplittalbeInputStream that will allow multiple consumers of
> a single InputStream.
> Some SplittableInputStream implementations do exist, for example in JMRTD and
> on stack overflow, but licensing issues make it so these aren't an option.
> Either need to find a solution compatible with our license or implement our
> own.
> This work should be done concurrently with changes to improve the efficiency
> of the I/O layer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
