[ https://issues.apache.org/jira/browse/KYLIN-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nichunen reassigned KYLIN-3398: ------------------------------- Assignee: nichunen > Inaccurate arithmetic operation in LookupTableToHFileJob#calculateShardNum > -------------------------------------------------------------------------- > > Key: KYLIN-3398 > URL: https://issues.apache.org/jira/browse/KYLIN-3398 > Project: Kylin > Issue Type: Bug > Reporter: Ted Yu > Assignee: nichunen > Priority: Major > > There're two issues with the following code: > {code} > private int calculateShardNum(KylinConfig kylinConfig, long dataSize) { > long shardSize = kylinConfig.getExtTableSnapshotShardingMB() * 1024 * > 1024; > return dataSize < shardSize ? 1 : (int) (Math.ceil(dataSize / > shardSize)); > {code} > getExtTableSnapshotShardingMB returns an int. The multiplication is done > using 32-bit arithmetic, and then used in a context that expects an > expression of type "long". > Second, Math.ceil expects a double. The integer division would lose some > precision. -- This message was sent by Atlassian JIRA (v7.6.3#76005)