t829702 edited a comment on pull request #2035: URL: https://github.com/apache/arrow/pull/2035#issuecomment-695004215
Consider the other implementations have CSV Reader and JSON Reader, to read from arbitrary JSON and can infer schema, 1. https://arrow.apache.org/docs/python/json.html 2. https://docs.rs/crate/arrow/1.0.1/source/src/json/reader.rs 3. https://arrow.apache.org/docs/java/reference/org/apache/arrow/vector/ipc/JsonFileReader.html 4. https://pkg.go.dev/github.com/apache/arrow/go/arrow@v0.0.0-20200918023255-1f30466ac704/csv Why not keep that `csv-to-arrow.js` in the tree, and deliver into NPM package's bin folder? it would be a great helper util, and also include a CLI script `json-to-arrow` as another? for many cases csv/json files size below some hundreds MB it's not slow, but of great help > It is not an optimized or ergonomic way to interact with Arrow So, what is the better way to generate Arrow files? better way to convert many existing datasets in csv/json or other formats? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
