[ https://issues.apache.org/jira/browse/BEAM-1592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ismaël Mejía resolved BEAM-1592. -------------------------------- Resolution: Invalid Fix Version/s: Not applicable I am closing this one since this is not longer valid after the arrival of BeamFileSystem. > Unify HdfsIO and HadoopInputFormatIO > ------------------------------------ > > Key: BEAM-1592 > URL: https://issues.apache.org/jira/browse/BEAM-1592 > Project: Beam > Issue Type: Bug > Components: io-java-hadoop > Reporter: Stephen Sisk > Assignee: Jean-Baptiste Onofré > Priority: Major > Fix For: Not applicable > > > HIFIO is currently in PR (https://github.com/apache/beam/pull/1994) and as > per discussion in > https://lists.apache.org/thread.html/803857877804165e798cf31edf079e6603eb9682b7690d52124c31e7@%3Cdev.beam.apache.org%3E, > we'd like to check HIFIO in as-is, then unify the two since they share a lot > of code. > [~dhalp...@google.com] has mentioned: "the FileInputFormat reader gets to > call some special APIs that the > generic InputFormat reader cannot -- so they are not completely redundant. > Specifically, FileInputFormat reader can do size-based splitting." > Dan recommended: "See if we can "inline" the FileInputFormat specific parts > of HdfsIO inside of HadoopInputFormatIO via reflection. If so, we can get the > best of both worlds with shared code." > This seems reasonable to me. -- This message was sent by Atlassian JIRA (v7.6.3#76005)