[jira] [Updated] (ARROW-4452) [Python] Serializing sparse torch tensors
[ https://issues.apache.org/jira/browse/ARROW-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antoine Pitrou updated ARROW-4452: -- Component/s: Python > [Python] Serializing sparse torch tensors > - > > Key: ARROW-4452 > URL: https://issues.apache.org/jira/browse/ARROW-4452 > Project: Apache Arrow > Issue Type: Improvement > Components: Python >Reporter: Philipp Moritz >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Using the pytorch serialization handler on sparse Tensors: > {code:java} > import torch > i = torch.LongTensor([[0, 2], [1, 0], [1, 2]]) > v = torch.FloatTensor([3, 4, 5 ]) > tensor = torch.sparse.FloatTensor(i.t(), v, torch.Size([2,3])) > pyarrow.serialization.register_torch_serialization_handlers(pyarrow.serialization._default_serialization_context) > s = pyarrow.serialize(tensor, > context=pyarrow.serialization._default_serialization_context) {code} > Produces this result: > {code:java} > TypeError: can't convert sparse tensor to numpy. Use Tensor.to_dense() to > convert to a dense tensor first.{code} > We should provide a way to serialize sparse torch tensors, especially now > that we are getting support for sparse Tensors. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4452) [Python] Serializing sparse torch tensors
[ https://issues.apache.org/jira/browse/ARROW-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4452: -- Labels: pull-request-available (was: ) > [Python] Serializing sparse torch tensors > - > > Key: ARROW-4452 > URL: https://issues.apache.org/jira/browse/ARROW-4452 > Project: Apache Arrow > Issue Type: Improvement >Reporter: Philipp Moritz >Priority: Major > Labels: pull-request-available > > Using the pytorch serialization handler on sparse Tensors: > {code:java} > import torch > i = torch.LongTensor([[0, 2], [1, 0], [1, 2]]) > v = torch.FloatTensor([3, 4, 5 ]) > tensor = torch.sparse.FloatTensor(i.t(), v, torch.Size([2,3])) > pyarrow.serialization.register_torch_serialization_handlers(pyarrow.serialization._default_serialization_context) > s = pyarrow.serialize(tensor, > context=pyarrow.serialization._default_serialization_context) {code} > Produces this result: > {code:java} > TypeError: can't convert sparse tensor to numpy. Use Tensor.to_dense() to > convert to a dense tensor first.{code} > We should provide a way to serialize sparse torch tensors, especially now > that we are getting support for sparse Tensors. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4452) [Python] Serializing sparse torch tensors
[ https://issues.apache.org/jira/browse/ARROW-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philipp Moritz updated ARROW-4452: -- Description: Using the pytorch serialization handler on sparse Tensors: {code:java} import torch i = torch.LongTensor([[0, 2], [1, 0], [1, 2]]) v = torch.FloatTensor([3, 4, 5 ]) tensor = torch.sparse.FloatTensor(i.t(), v, torch.Size([2,3])) pyarrow.serialization.register_torch_serialization_handlers(pyarrow.serialization._default_serialization_context) s = pyarrow.serialize(tensor, context=pyarrow.serialization._default_serialization_context) {code} Produces this result: {code:java} TypeError: can't convert sparse tensor to numpy. Use Tensor.to_dense() to convert to a dense tensor first.{code} We should provide a way to serialize sparse torch tensors, especially now that we are getting support for sparse Tensors. was: Using the pytorch serialization handler on sparse Tensors: {code:java} import torch i = torch.LongTensor([[0, 2], [1, 0], [1, 2]]) v = torch.FloatTensor([3, 4, 5 ]) tensor = torch.sparse.FloatTensor(i.t(), v, torch.Size([2,3])) register_torch_serialization_handlers(pyarrow.serialization._default_serialization_context) s = pyarrow.serialize(tensor, context=pyarrow.serialization._default_serialization_context) {code} Produces this result: {code:java} TypeError: can't convert sparse tensor to numpy. Use Tensor.to_dense() to convert to a dense tensor first.{code} We should provide a way to serialize sparse torch tensors, especially now that we are getting support for sparse Tensors. > [Python] Serializing sparse torch tensors > - > > Key: ARROW-4452 > URL: https://issues.apache.org/jira/browse/ARROW-4452 > Project: Apache Arrow > Issue Type: Improvement >Reporter: Philipp Moritz >Priority: Major > > Using the pytorch serialization handler on sparse Tensors: > {code:java} > import torch > i = torch.LongTensor([[0, 2], [1, 0], [1, 2]]) > v = torch.FloatTensor([3, 4, 5 ]) > tensor = torch.sparse.FloatTensor(i.t(), v, torch.Size([2,3])) > pyarrow.serialization.register_torch_serialization_handlers(pyarrow.serialization._default_serialization_context) > s = pyarrow.serialize(tensor, > context=pyarrow.serialization._default_serialization_context) {code} > Produces this result: > {code:java} > TypeError: can't convert sparse tensor to numpy. Use Tensor.to_dense() to > convert to a dense tensor first.{code} > We should provide a way to serialize sparse torch tensors, especially now > that we are getting support for sparse Tensors. -- This message was sent by Atlassian JIRA (v7.6.3#76005)