Shane Evans
Mon, 16 Nov 2009 08:50:00 -0800
Hi,I am writing a storage converter (implementing ReversibleLoadStoreFunc) for pig 0.4. The schema for this data is stored in an external file and I need this schema to correctly serialize and deserialize the format I am working with. I can't see any obvious way to load this external file within my storage converter.
I can see that I have a reference to the DataStorage and ExecType in the determineSchema method and I can call FileLocalizer.open(..), but determineSchema isn't called when storing data so I don't have a reference to a DataStorage or ExecType at any point. I could call the FileLocalizer.openDFS(String) method, but I would like my StorageConverter to work in local mode. I am new to pig and I feel that I must be missing something. Any suggestions are appreciated.
Thanks, Shane