Bo Xu created CARBONDATA-3695:
---------------------------------
Summary: Apache CarbonData should provides python interface to
support deep learning framework PyTorch to ready and write data from/to
CarbonData
Key: CARBONDATA-3695
URL: https://issues.apache.org/jira/browse/CARBONDATA-3695
Project: CarbonData
Issue Type: Sub-task
Reporter: Bo Xu
Assignee: Bo Xu
Nowadays AI model training is getting more and more popular. Currently many AI
framework uses raw data files or row format data files for model training, it
could not provide projection, filtering, and fast scan capability like in
columnar store. So, if CarbonData supports AI framework, it can speed up model
training by increase IO throughput, and provide more flexible training set
selection ability to AI developers
AI compute engine integration:
PyTorch integration: New python API in pycarbon to support PyTorch to read data
from CarbonData files for training model
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
