Hello Romain, Thanks for the feedback. It sounds like a good idea! How could I get the rest IO from JB?
Thank you. 2018-04-11 10:50 GMT-03:00 Romain Manni-Bucau <rmannibu...@gmail.com>: > Hi Daniel, > > I know JB has a started a RestIO, it is not yet complete and needs > some love but probably some opportunity for converge and collaboration > here > > Romain Manni-Bucau > @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book > > > 2018-04-11 15:42 GMT+02:00 Daniel Salerno <salerno....@gmail.com>: > > Good morning, > > > > In my Big Data Google project we need to read batch data from the VTEX > > platform every 15 minutes and record json's return to our cloud storage > > datalake. > > It makes the data available through HTTP GET requests for its API: > > (https://documenter.getpostman.com/view/487146/vtex-oms-api/6tjSKqi) > > > > I have not found a good way to do this reading using standard Apache Beam > > components, my idea is to develop an HttpIO component using Source Sink > to > > read the request and write a JSON file to Cloud Storage. After that > another > > pipeline will process the file and write to BigQuery. > > > > Can you please help me with this matter? Is that the best way to go? Do > you > > have any idea how to develop this new component? > > > > Thank you. > > > > Hugs, > > > > Daniel Salerno de Arruda >
