How are you modifying the file manually ? Are you copying to local file system, make changes and copying back to HDFS ?
Thanks Padma On Tue, Oct 30, 2018 at 5:45 PM Jean-Claude Cote <[email protected]> wrote: > I'm writing a msgpack reader which supports schema validation. The msgpack > reader is able to discover the schema and store the result in a file > named .schema.proto along side the data files. There is also an additional > ..schema.proto.crc file created by the hadoop file system I believe. > > However even if the reader can discover the schema I would like to be able > to edit the file manually. However when I do the checksum file does not > match anymore and my reader fails to load the file. > > My question is how can I read a file ignoring the checksum file. Or how > difficult is it to produce these checksum files. > > I save the file like so > try (FSDataOutputStream out = fileSystem.create(schemaLocation, true)) > { > > This call fails if I modify the schema file manually. > try (FSDataInputStream in = fileSystem.open(schemaLocation)) { > > Thank you > jc >
