[jira] [Commented] (ARROW-1678) [Python] Incorrect serialization of numpy.float16

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208589#comment-16208589
 ] 

ASF GitHub Bot commented on ARROW-1678:
---

Github user wesm commented on the issue:

https://github.com/apache/arrow/pull/1205
  
Thanks @Licht-T for your contribution!


> [Python] Incorrect serialization of numpy.float16
> -
>
> Key: ARROW-1678
> URL: https://issues.apache.org/jira/browse/ARROW-1678
> Project: Apache Arrow
>  Issue Type: Bug
>  Components: Python
>Affects Versions: 0.7.1
>Reporter: Licht Takeuchi
>Assignee: Licht Takeuchi
>  Labels: pull-request-available
> Fix For: 0.8.0
>
>
> The issue can be reproduced as follows.
> {code:python}
> import pyarrow as pa
> import numpy as np
> t = np.float16(-1.1)
> print(type(t), t) #  -1.0996
> p = pa.serialize(t)
> t_new = pa.deserialize(p.to_buffer())
> print(type(t_new), t_new) #   0
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ARROW-1678) [Python] Incorrect serialization of numpy.float16

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208584#comment-16208584
 ] 

ASF GitHub Bot commented on ARROW-1678:
---

Github user asfgit closed the pull request at:

https://github.com/apache/arrow/pull/1205


> [Python] Incorrect serialization of numpy.float16
> -
>
> Key: ARROW-1678
> URL: https://issues.apache.org/jira/browse/ARROW-1678
> Project: Apache Arrow
>  Issue Type: Bug
>  Components: Python
>Affects Versions: 0.7.1
>Reporter: Licht Takeuchi
>  Labels: pull-request-available
> Fix For: 0.8.0
>
>
> The issue can be reproduced as follows.
> {code:python}
> import pyarrow as pa
> import numpy as np
> t = np.float16(-1.1)
> print(type(t), t) #  -1.0996
> p = pa.serialize(t)
> t_new = pa.deserialize(p.to_buffer())
> print(type(t_new), t_new) #   0
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ARROW-1678) [Python] Incorrect serialization of numpy.float16

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208348#comment-16208348
 ] 

ASF GitHub Bot commented on ARROW-1678:
---

Github user wesm commented on the issue:

https://github.com/apache/arrow/pull/1205
  
I will give this a last look, then merge later today


> [Python] Incorrect serialization of numpy.float16
> -
>
> Key: ARROW-1678
> URL: https://issues.apache.org/jira/browse/ARROW-1678
> Project: Apache Arrow
>  Issue Type: Bug
>  Components: Python
>Affects Versions: 0.7.1
>Reporter: Licht Takeuchi
>  Labels: pull-request-available
> Fix For: 0.8.0
>
>
> The issue can be reproduced as follows.
> {code:python}
> import pyarrow as pa
> import numpy as np
> t = np.float16(-1.1)
> print(type(t), t) #  -1.0996
> p = pa.serialize(t)
> t_new = pa.deserialize(p.to_buffer())
> print(type(t_new), t_new) #   0
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ARROW-1678) [Python] Incorrect serialization of numpy.float16

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207547#comment-16207547
 ] 

ASF GitHub Bot commented on ARROW-1678:
---

Github user Licht-T commented on the issue:

https://github.com/apache/arrow/pull/1205
  
@wesm @xhochy @pcmoritz Thanks for your review! Fixed!


> [Python] Incorrect serialization of numpy.float16
> -
>
> Key: ARROW-1678
> URL: https://issues.apache.org/jira/browse/ARROW-1678
> Project: Apache Arrow
>  Issue Type: Bug
>  Components: Python
>Affects Versions: 0.7.1
>Reporter: Licht Takeuchi
>  Labels: pull-request-available
> Fix For: 0.8.0
>
>
> The issue can be reproduced as follows.
> {code:python}
> import pyarrow as pa
> import numpy as np
> t = np.float16(-1.1)
> print(type(t), t) #  -1.0996
> p = pa.serialize(t)
> t_new = pa.deserialize(p.to_buffer())
> print(type(t_new), t_new) #   0
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ARROW-1678) [Python] Incorrect serialization of numpy.float16

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16206835#comment-16206835
 ] 

ASF GitHub Bot commented on ARROW-1678:
---

Github user Licht-T commented on the issue:

https://github.com/apache/arrow/pull/1205
  
HalfFloat is the IEEE defined floating point, but there are some variety in 
its type.
AFAIK, `unint16_t` or `unsigned short` in C are used.



> [Python] Incorrect serialization of numpy.float16
> -
>
> Key: ARROW-1678
> URL: https://issues.apache.org/jira/browse/ARROW-1678
> Project: Apache Arrow
>  Issue Type: Bug
>  Components: Python
>Affects Versions: 0.7.1
>Reporter: Licht Takeuchi
>  Labels: pull-request-available
> Fix For: 0.8.0
>
>
> The issue can be reproduced as follows.
> {code:python}
> import pyarrow as pa
> import numpy as np
> t = np.float16(-1.1)
> print(type(t), t) #  -1.0996
> p = pa.serialize(t)
> t_new = pa.deserialize(p.to_buffer())
> print(type(t_new), t_new) #   0
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ARROW-1678) [Python] Incorrect serialization of numpy.float16

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16206326#comment-16206326
 ] 

ASF GitHub Bot commented on ARROW-1678:
---

Github user pcmoritz commented on the issue:

https://github.com/apache/arrow/pull/1205
  
Thanks for the contribution! You will also need to add a case to 
https://github.com/Licht-T/arrow/blob/5a985b8b5fef6e3781de056bdc13f754a673376d/cpp/src/arrow/python/python_to_arrow.cc#L540
 and can you also add a test to 
https://github.com/apache/arrow/blob/master/python/pyarrow/tests/test_tensor.py?


> [Python] Incorrect serialization of numpy.float16
> -
>
> Key: ARROW-1678
> URL: https://issues.apache.org/jira/browse/ARROW-1678
> Project: Apache Arrow
>  Issue Type: Bug
>  Components: Python
>Affects Versions: 0.7.1
>Reporter: Licht Takeuchi
>  Labels: pull-request-available
> Fix For: 0.8.0
>
>
> The issue can be reproduced as follows.
> {code:python}
> import pyarrow as pa
> import numpy as np
> t = np.float16(-1.1)
> print(type(t), t) #  -1.0996
> p = pa.serialize(t)
> t_new = pa.deserialize(p.to_buffer())
> print(type(t_new), t_new) #   0
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ARROW-1678) [Python] Incorrect serialization of numpy.float16

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16206123#comment-16206123
 ] 

ASF GitHub Bot commented on ARROW-1678:
---

GitHub user Licht-T opened a pull request:

https://github.com/apache/arrow/pull/1205

ARROW-1678: [Python] Implement numpy.float16 SerDe

This is the patch for 
[ARROW-1678](https://issues.apache.org/jira/projects/ARROW/issues/ARROW-1678).

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Licht-T/arrow feature-halffloat-python-serde

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/arrow/pull/1205.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1205


commit 7348cc6761a9d94da029123365e0dd24d86f69de
Author: Licht-T 
Date:   2017-10-16T15:55:39Z

ENH: Add HalfFloat BUILDER_CASE

commit 0be305a3443b897dcc2d99e72a5d808e02bb768a
Author: Licht-T 
Date:   2017-10-16T15:56:39Z

ENH: Add the IPC internal metadata for HalfFloat

commit dda98b9e013947a7172c35b71e7a0d79027cd99d
Author: Licht-T 
Date:   2017-10-16T15:59:43Z

ENH: Add HalfFloat type traits

commit 9d19e0866da1feecc82f1ac4bda8bcfa57f6ca41
Author: Licht-T 
Date:   2017-10-16T16:01:28Z

ENH: Add HalfFloat type case for numpy

commit 02dc77b7e8a6439efd3dedad461191c47eb15627
Author: Licht-T 
Date:   2017-10-16T16:04:55Z

ENH: Add numpy.float16 serializer

commit 834357e7d3bb1e1cf8dc0a0383854c7e3818d38f
Author: Licht-T 
Date:   2017-10-16T16:05:50Z

ENH: Add numpy.float16 deserializer

commit 66d5f15abb439c3ac63f87c44cc97e517b5d6ec5
Author: Licht-T 
Date:   2017-10-16T16:06:41Z

ENH: Set VISIT_INLINE for HalfFloat

commit f19e543129c49145ec6ac058b73a60c943f7413a
Author: Licht-T 
Date:   2017-10-16T16:07:47Z

TST: Add tests for numpy.float16 SerDe




> [Python] Incorrect serialization of numpy.float16
> -
>
> Key: ARROW-1678
> URL: https://issues.apache.org/jira/browse/ARROW-1678
> Project: Apache Arrow
>  Issue Type: Bug
>  Components: Python
>Affects Versions: 0.7.1
>Reporter: Licht Takeuchi
>  Labels: pull-request-available
> Fix For: 0.8.0
>
>
> The issue can be reproduced as follows.
> {code:python}
> import pyarrow as pa
> import numpy as np
> t = np.float16(-1.1)
> print(type(t), t) #  -1.0996
> p = pa.serialize(t)
> t_new = pa.deserialize(p.to_buffer())
> print(type(t_new), t_new) #   0
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ARROW-1678) [Python] Incorrect serialization of numpy.float16

[Licht Takeuchi (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205681#comment-16205681
 ] 

Licht Takeuchi commented on ARROW-1678:
---

I also made the patch for this issue.

> [Python] Incorrect serialization of numpy.float16
> -
>
> Key: ARROW-1678
> URL: https://issues.apache.org/jira/browse/ARROW-1678
> Project: Apache Arrow
>  Issue Type: Bug
>  Components: Python
>Reporter: Licht Takeuchi
>
> The issue can be reproduced as follows.
> {code:python}
> import pyarrow as pa
> import numpy as np
> t = np.float16(-1.1)
> print(type(t), t) #  -1.0996
> p = pa.serialize(t)
> t_new = pa.deserialize(p.to_buffer())
> print(type(t_new), t_new) #   0
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)