[
https://issues.apache.org/jira/browse/ARROW-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16222832#comment-16222832
]
ASF GitHub Bot commented on ARROW-1727:
---------------------------------------
TheNeuralBit commented on a change in pull request #1257: ARROW-1727: [Format]
Expand Arrow streaming format to permit deltas / additions to existing
dictionaries
URL: https://github.com/apache/arrow/pull/1257#discussion_r147517384
##########
File path: format/IPC.md
##########
@@ -67,15 +67,18 @@ We provide a streaming format for record batches. It is
presented as a sequence
of encapsulated messages, each of which follows the format above. The schema
comes first in the stream, and it is the same for all of the record batches
that follow. If any fields in the schema are dictionary-encoded, one or more
-`DictionaryBatch` messages will follow the schema.
+`DictionaryBatch` messages will be included. `DictionaryBatch` and
+`RecordBatch` messages may be interleaved, but before any dictionary key is
used
+in a `RecordBatch` it should be defined in a `DictionaryBatch`.
```
<SCHEMA>
<DICTIONARY 0>
...
-<DICTIONARY k - 1>
<RECORD BATCH 0>
...
+<DICTIONARY k - 1>
+...
Review comment:
Yeah thats fair, I can tweak it to make it clear that any dictionaries after
the first record batch should be modifying the originals.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> [Format] Expand Arrow streaming format to permit new dictionaries and deltas
> / additions to existing dictionaries
> -----------------------------------------------------------------------------------------------------------------
>
> Key: ARROW-1727
> URL: https://issues.apache.org/jira/browse/ARROW-1727
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Format
> Reporter: Wes McKinney
> Assignee: Brian Hulette
> Labels: pull-request-available
> Fix For: 0.8.0
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
