[
https://issues.apache.org/jira/browse/APEXMALHAR-2263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15563965#comment-15563965
]
Munagala V. Ramanath commented on APEXMALHAR-2263:
--------------------------------------------------
To provide a bit more detail, imagine a file with small lines, maybe 20 chars
each; that's
adequate, for example a timestamp and id of some sort.
At Integer.MAX_VALUE lines, we have 85_899_345_900 bytes or ~79 GB. By today's
standards
where TB size files are not uncommon, we can easily overflow ints.
> Offsets in AbstractFileInputOperator should be long rather than int
> -------------------------------------------------------------------
>
> Key: APEXMALHAR-2263
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2263
> Project: Apache Apex Malhar
> Issue Type: Bug
> Components: adapters other
> Reporter: Munagala V. Ramanath
> Assignee: Matt Zhang
>
> Offsets in AbstractFileInputOperator use the int type which means files with
> more that (2**31 -1) records will cause overflows and mysterious failures.
> Should be changed to use long.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
