[ https://issues.apache.org/jira/browse/CARBONDATA-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ajantha Bhat resolved CARBONDATA-3695. -------------------------------------- Fix Version/s: 2.0.0 Resolution: Fixed > Apache CarbonData should provides python interface to support deep learning > framework PyTorch to ready and write data from/to CarbonData > ---------------------------------------------------------------------------------------------------------------------------------------- > > Key: CARBONDATA-3695 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3695 > Project: CarbonData > Issue Type: Sub-task > Reporter: Bo Xu > Assignee: Bo Xu > Priority: Major > Fix For: 2.0.0 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > Nowadays AI model training is getting more and more popular. Currently many > AI framework uses raw data files or row format data files for model training, > it could not provide projection, filtering, and fast scan capability like in > columnar store. So, if CarbonData supports AI framework, it can speed up > model training by increase IO throughput, and provide more flexible training > set selection ability to AI developers > AI compute engine integration: > PyTorch integration: New python API in pycarbon to support PyTorch to read > data from CarbonData files for training model -- This message was sent by Atlassian Jira (v8.3.4#803005)