[
https://issues.apache.org/jira/browse/ARROW-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16258227#comment-16258227
]
ASF GitHub Bot commented on ARROW-1693:
---------------------------------------
wesm commented on issue #1294: ARROW-1693: [JS] Fix reading C++
dictionary-encoded vectors
URL: https://github.com/apache/arrow/pull/1294#issuecomment-345472682
Here's what I'm seeing in the diff in the test directory:
```
js/test/Arrow.ts | 57 +-
js/test/__snapshots__/reader-tests.ts.snap | 497 -
js/test/__snapshots__/table-tests.ts.snap | 1815 ---
js/test/arrows/cpp/file/datetime.arrow | Bin 0 -> 6490 bytes
js/test/arrows/cpp/file/decimal.arrow | Bin 0 -> 259090 bytes
js/test/arrows/cpp/file/dictionary.arrow | Bin 0 -> 2562 bytes
js/test/arrows/cpp/file/nested.arrow | Bin 0 -> 2218 bytes
js/test/arrows/cpp/file/primitive-empty.arrow | Bin 0 -> 9498 bytes
js/test/arrows/cpp/file/primitive.arrow | Bin 0 -> 9442 bytes
js/test/arrows/cpp/file/simple.arrow | Bin 0 -> 1154 bytes
js/test/arrows/cpp/file/struct_example.arrow | Bin 0 -> 1538 bytes
js/test/arrows/cpp/stream/datetime.arrow | Bin 0 -> 5076 bytes
js/test/arrows/cpp/stream/decimal.arrow | Bin 0 -> 255228 bytes
js/test/arrows/cpp/stream/dictionary.arrow | Bin 0 -> 2004 bytes
js/test/arrows/cpp/stream/nested.arrow | Bin 0 -> 1636 bytes
js/test/arrows/cpp/stream/primitive-empty.arrow | Bin 0 -> 6852 bytes
js/test/arrows/cpp/stream/primitive.arrow | Bin 0 -> 7020 bytes
js/test/arrows/cpp/stream/simple.arrow | Bin 0 -> 748 bytes
js/test/arrows/cpp/stream/struct_example.arrow | Bin 0 -> 1124 bytes
js/test/arrows/file/dictionary.arrow | Bin 2522 -> 0 bytes
js/test/arrows/file/dictionary2.arrow | Bin 2762 -> 0 bytes
js/test/arrows/file/multi_dictionary.arrow | Bin 3482 -> 0 bytes
js/test/arrows/file/simple.arrow | Bin 1642 -> 0 bytes
js/test/arrows/file/struct.arrow | Bin 2354 -> 0 bytes
js/test/arrows/java/file/datetime.arrow | Bin 0 -> 6746 bytes
js/test/arrows/java/file/decimal.arrow | Bin 0 -> 259730 bytes
js/test/arrows/java/file/dictionary.arrow | Bin 0 -> 2666 bytes
js/test/arrows/java/file/nested.arrow | Bin 0 -> 2314 bytes
js/test/arrows/java/file/primitive-empty.arrow | Bin 0 -> 9778 bytes
js/test/arrows/java/file/primitive.arrow | Bin 0 -> 10034 bytes
js/test/arrows/java/file/simple.arrow | Bin 0 -> 1210 bytes
js/test/arrows/java/file/struct_example.arrow | Bin 0 -> 1602 bytes
js/test/arrows/java/stream/datetime.arrow | Bin 0 -> 5196 bytes
js/test/arrows/java/stream/decimal.arrow | Bin 0 -> 255564 bytes
js/test/arrows/java/stream/dictionary.arrow | Bin 0 -> 2036 bytes
js/test/arrows/java/stream/nested.arrow | Bin 0 -> 1676 bytes
js/test/arrows/java/stream/primitive-empty.arrow | Bin 0 -> 6916 bytes
js/test/arrows/java/stream/primitive.arrow | Bin 0 -> 7404 bytes
js/test/arrows/java/stream/simple.arrow | Bin 0 -> 772 bytes
js/test/arrows/java/stream/struct_example.arrow | Bin 0 -> 1148 bytes
js/test/arrows/json/datetime.json | 1091 ++
js/test/arrows/json/decimal.json | 33380
+++++++++++++++++++++++++++++++++++++++++++++++++++
js/test/arrows/json/dictionary.json | 424 +
js/test/arrows/json/nested.json | 384 +
js/test/arrows/json/primitive-empty.json | 1099 ++
js/test/arrows/json/primitive.json | 1788 +++
js/test/arrows/json/simple.json | 66 +
js/test/arrows/json/struct_example.json | 237 +
js/test/arrows/multi/count/records.arrow | Bin 224 -> 0 bytes
js/test/arrows/multi/count/schema.arrow | Bin 184 -> 0 bytes
js/test/arrows/multi/latlong/records.arrow | Bin 352 -> 0 bytes
js/test/arrows/multi/latlong/schema.arrow | Bin 264 -> 0 bytes
js/test/arrows/multi/origins/records.arrow | Bin 224 -> 0 bytes
js/test/arrows/multi/origins/schema.arrow | Bin 1604 -> 0 bytes
js/test/arrows/stream/dictionary.arrow | Bin 1776 -> 0 bytes
js/test/arrows/stream/simple.arrow | Bin 1188 -> 0 bytes
js/test/arrows/stream/struct.arrow | Bin 1884 -> 0 bytes
js/test/integration-tests.ts | 114 +
js/test/reader-tests.ts | 69 +-
js/test/table-tests.ts | 175 +-
js/test/test-config.ts | 61 +-
js/test/tsconfig.json | 2 +-
js/test/vector-tests.ts | 109 +-
63 files changed, 38864 insertions(+), 2504 deletions(-)
```
Can we remove the JSON files and the large decimal arrow files? I'm sorry to
be OCD about the git history
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> [JS] Error reading dictionary-encoded integration test files
> ------------------------------------------------------------
>
> Key: ARROW-1693
> URL: https://issues.apache.org/jira/browse/ARROW-1693
> Project: Apache Arrow
> Issue Type: Bug
> Components: JavaScript
> Reporter: Brian Hulette
> Assignee: Brian Hulette
> Labels: pull-request-available
> Fix For: 0.8.0
>
> Attachments: dictionary-cpp.arrow, dictionary-java.arrow,
> dictionary.json
>
>
> The JS implementation crashes when reading the dictionary test case from the
> integration tests.
> To replicate, first generate the test files with java and cpp impls:
> {code}
> $ cd ${ARROW_HOME}/integration/
> $ python -c 'from integration_test import generate_dictionary_case;
> generate_dictionary_case().write("dictionary.json")'
> $ ../cpp/debug/debug/json-integration-test --integration
> --json=dictionary.json --arrow=dictionary-cpp.arrow --mode=JSON_TO_ARROW
> $ java -cp
> ../java/tools/target/arrow-tools-0.8.0-SNAPSHOT-jar-with-dependencies.jar
> org.apache.arrow.tools.Integration -c JSON_TO_ARROW -a dictionary-java.arrow
> -j dictionary.json
> {code}
> Attempt to read the files with the JS impl:
> {code}
> $ cd ${ARROW_HOME}/js/
> $ ./bin/arrow2csv.js -s dict1_0 -f ../integration/dictionary-{java,cpp}.arrow
> {code}
> Both files result in an error for me on
> [a8f51858|https://github.com/apache/arrow/commit/a8f518588fda471b2e3cc8e0f0064e7c4bb99899]:
> {{TypeError: Cannot read property 'buffer' of undefined}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
