Hey Paul, In my pull request you mentioned handling splits.. I put a comment in the pull request but essentially msgpack files are a list of records so technically they can be split. However I'm not sure if that's beneficial because I'm not sure how the splitting process works..
The other thing you mentioned is using TupleSchema. I have not looked at it yet but in my initial design I had tried to use it but ran into limitations. I will try it again and let you know what the issue is (if I run into it again). I did a major refactoring of my msgpack format reader and added more documentation/explanations. Hopefully it will be easier to follow. Could you have a look when you have a moment? Thanks jc
