[jira] [Commented] (ARROW-2397) Document changes in Tensor encoding in IPC.md.
[ https://issues.apache.org/jira/browse/ARROW-2397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438501#comment-16438501 ] ASF GitHub Bot commented on ARROW-2397: --- robertnishihara closed pull request #1837: ARROW-2397: [Documentation] Update format documentation to describe tensor alignment. URL: https://github.com/apache/arrow/pull/1837 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/format/IPC.md b/format/IPC.md index 5a5d3aef6..1c8532377 100644 --- a/format/IPC.md +++ b/format/IPC.md @@ -240,8 +240,9 @@ tools. Arrow implementations in general are not required to implement this data format, though we provide a reference implementation in C++. When writing a standalone encapsulated tensor message, we use the format as -indicated above, but additionally align the starting offset (if writing to a -shared memory region) to be a multiple of 8: +indicated above, but additionally align the starting offset of the metadata as +well as the starting offset of the tensor body (if writing to a shared memory +region) to be multiples of 64 bytes: ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Document changes in Tensor encoding in IPC.md. > -- > > Key: ARROW-2397 > URL: https://issues.apache.org/jira/browse/ARROW-2397 > Project: Apache Arrow > Issue Type: Improvement > Components: Documentation >Reporter: Robert Nishihara >Priority: Major > Labels: pull-request-available > Fix For: JS-0.4.0 > > > Update IPC.md to reflect the changes in > https://github.com/apache/arrow/pull/1802. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-2397) Document changes in Tensor encoding in IPC.md.
[ https://issues.apache.org/jira/browse/ARROW-2397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437351#comment-16437351 ] ASF GitHub Bot commented on ARROW-2397: --- xhochy commented on issue #1837: ARROW-2397: [Documentation] Update format documentation to describe tensor alignment. URL: https://github.com/apache/arrow/pull/1837#issuecomment-381149693 @robertnishihara I think we can go ahead and merge this. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Document changes in Tensor encoding in IPC.md. > -- > > Key: ARROW-2397 > URL: https://issues.apache.org/jira/browse/ARROW-2397 > Project: Apache Arrow > Issue Type: Improvement > Components: Documentation >Reporter: Robert Nishihara >Priority: Major > Labels: pull-request-available > > Update IPC.md to reflect the changes in > https://github.com/apache/arrow/pull/1802. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-2397) Document changes in Tensor encoding in IPC.md.
[ https://issues.apache.org/jira/browse/ARROW-2397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436428#comment-16436428 ] ASF GitHub Bot commented on ARROW-2397: --- robertnishihara commented on issue #1837: ARROW-2397: [Documentation] Update format documentation to describe tensor alignment. URL: https://github.com/apache/arrow/pull/1837#issuecomment-380966533 @xhochy should we merge this? Or should I change the number back to 8? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Document changes in Tensor encoding in IPC.md. > -- > > Key: ARROW-2397 > URL: https://issues.apache.org/jira/browse/ARROW-2397 > Project: Apache Arrow > Issue Type: Improvement > Components: Documentation >Reporter: Robert Nishihara >Priority: Major > Labels: pull-request-available > > Update IPC.md to reflect the changes in > https://github.com/apache/arrow/pull/1802. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-2397) Document changes in Tensor encoding in IPC.md.
[ https://issues.apache.org/jira/browse/ARROW-2397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426573#comment-16426573 ] ASF GitHub Bot commented on ARROW-2397: --- robertnishihara commented on a change in pull request #1837: ARROW-2397: [Documentation] Update format documentation to describe tensor alignment. URL: https://github.com/apache/arrow/pull/1837#discussion_r179371622 ## File path: format/IPC.md ## @@ -240,8 +240,9 @@ tools. Arrow implementations in general are not required to implement this data format, though we provide a reference implementation in C++. When writing a standalone encapsulated tensor message, we use the format as -indicated above, but additionally align the starting offset (if writing to a -shared memory region) to be a multiple of 8: +indicated above, but additionally align the starting offset of the metadata as +well as the starting offset of the tensor body (if writing to a shared memory +region) to be multiples of 64 bytes: Review comment: I see, I was just looking at https://github.com/apache/arrow/blob/045470c3879fd05f528dbe68f2c2299b2ebba105/cpp/src/arrow/ipc/writer.cc#L520-L528 and https://github.com/apache/arrow/blob/045470c3879fd05f528dbe68f2c2299b2ebba105/cpp/src/arrow/ipc/util.h#L31-L36 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Document changes in Tensor encoding in IPC.md. > -- > > Key: ARROW-2397 > URL: https://issues.apache.org/jira/browse/ARROW-2397 > Project: Apache Arrow > Issue Type: Improvement > Components: Documentation >Reporter: Robert Nishihara >Priority: Major > Labels: pull-request-available > > Update IPC.md to reflect the changes in > https://github.com/apache/arrow/pull/1802. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-2397) Document changes in Tensor encoding in IPC.md.
[ https://issues.apache.org/jira/browse/ARROW-2397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426568#comment-16426568 ] ASF GitHub Bot commented on ARROW-2397: --- xhochy commented on a change in pull request #1837: ARROW-2397: [Documentation] Update format documentation to describe tensor alignment. URL: https://github.com/apache/arrow/pull/1837#discussion_r179371132 ## File path: format/IPC.md ## @@ -240,8 +240,9 @@ tools. Arrow implementations in general are not required to implement this data format, though we provide a reference implementation in C++. When writing a standalone encapsulated tensor message, we use the format as -indicated above, but additionally align the starting offset (if writing to a -shared memory region) to be a multiple of 8: +indicated above, but additionally align the starting offset of the metadata as +well as the starting offset of the tensor body (if writing to a shared memory +region) to be multiples of 64 bytes: Review comment: From what we want we want as alignment in the Arrow standard, this is correct. I don't know if all our implementations that emit tensors conform to this. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Document changes in Tensor encoding in IPC.md. > -- > > Key: ARROW-2397 > URL: https://issues.apache.org/jira/browse/ARROW-2397 > Project: Apache Arrow > Issue Type: Improvement > Components: Documentation >Reporter: Robert Nishihara >Priority: Major > Labels: pull-request-available > > Update IPC.md to reflect the changes in > https://github.com/apache/arrow/pull/1802. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-2397) Document changes in Tensor encoding in IPC.md.
[ https://issues.apache.org/jira/browse/ARROW-2397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426565#comment-16426565 ] ASF GitHub Bot commented on ARROW-2397: --- robertnishihara commented on a change in pull request #1837: ARROW-2397: [Documentation] Update format documentation to describe tensor alignment. URL: https://github.com/apache/arrow/pull/1837#discussion_r179370619 ## File path: format/IPC.md ## @@ -240,8 +240,8 @@ tools. Arrow implementations in general are not required to implement this data format, though we provide a reference implementation in C++. When writing a standalone encapsulated tensor message, we use the format as -indicated above, but additionally align the starting offset (if writing to a -shared memory region) to be a multiple of 8: +indicated above, but additionally align the starting offset of the metadata as well as the starting offset of the tensor body (if writing to a shared Review comment: fixed This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Document changes in Tensor encoding in IPC.md. > -- > > Key: ARROW-2397 > URL: https://issues.apache.org/jira/browse/ARROW-2397 > Project: Apache Arrow > Issue Type: Improvement > Components: Documentation >Reporter: Robert Nishihara >Priority: Major > Labels: pull-request-available > > Update IPC.md to reflect the changes in > https://github.com/apache/arrow/pull/1802. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-2397) Document changes in Tensor encoding in IPC.md.
[ https://issues.apache.org/jira/browse/ARROW-2397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426564#comment-16426564 ] ASF GitHub Bot commented on ARROW-2397: --- robertnishihara commented on a change in pull request #1837: ARROW-2397: [Documentation] Update format documentation to describe tensor alignment. URL: https://github.com/apache/arrow/pull/1837#discussion_r179370592 ## File path: format/IPC.md ## @@ -240,8 +240,9 @@ tools. Arrow implementations in general are not required to implement this data format, though we provide a reference implementation in C++. When writing a standalone encapsulated tensor message, we use the format as -indicated above, but additionally align the starting offset (if writing to a -shared memory region) to be a multiple of 8: +indicated above, but additionally align the starting offset of the metadata as +well as the starting offset of the tensor body (if writing to a shared memory +region) to be multiples of 64 bytes: Review comment: Note that this used to be 8 and I replaced it with 64. Please check that this is correct. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Document changes in Tensor encoding in IPC.md. > -- > > Key: ARROW-2397 > URL: https://issues.apache.org/jira/browse/ARROW-2397 > Project: Apache Arrow > Issue Type: Improvement > Components: Documentation >Reporter: Robert Nishihara >Priority: Major > Labels: pull-request-available > > Update IPC.md to reflect the changes in > https://github.com/apache/arrow/pull/1802. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-2397) Document changes in Tensor encoding in IPC.md.
[ https://issues.apache.org/jira/browse/ARROW-2397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426544#comment-16426544 ] ASF GitHub Bot commented on ARROW-2397: --- xhochy commented on a change in pull request #1837: ARROW-2397: [Documentation] Update format documentation to describe tensor alignment. URL: https://github.com/apache/arrow/pull/1837#discussion_r179367108 ## File path: format/IPC.md ## @@ -240,8 +240,8 @@ tools. Arrow implementations in general are not required to implement this data format, though we provide a reference implementation in C++. When writing a standalone encapsulated tensor message, we use the format as -indicated above, but additionally align the starting offset (if writing to a -shared memory region) to be a multiple of 8: +indicated above, but additionally align the starting offset of the metadata as well as the starting offset of the tensor body (if writing to a shared Review comment: Can you wrap the text at the same line width as above? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Document changes in Tensor encoding in IPC.md. > -- > > Key: ARROW-2397 > URL: https://issues.apache.org/jira/browse/ARROW-2397 > Project: Apache Arrow > Issue Type: Improvement > Components: Documentation >Reporter: Robert Nishihara >Priority: Major > Labels: pull-request-available > > Update IPC.md to reflect the changes in > https://github.com/apache/arrow/pull/1802. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-2397) Document changes in Tensor encoding in IPC.md.
[ https://issues.apache.org/jira/browse/ARROW-2397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426420#comment-16426420 ] ASF GitHub Bot commented on ARROW-2397: --- robertnishihara opened a new pull request #1837: ARROW-2397: [Documentation] Update format documentation to describe tensor alignment. URL: https://github.com/apache/arrow/pull/1837 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Document changes in Tensor encoding in IPC.md. > -- > > Key: ARROW-2397 > URL: https://issues.apache.org/jira/browse/ARROW-2397 > Project: Apache Arrow > Issue Type: Improvement > Components: Documentation >Reporter: Robert Nishihara >Priority: Major > Labels: pull-request-available > > Update IPC.md to reflect the changes in > https://github.com/apache/arrow/pull/1802. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
