Re: Beam and Python API: Pandas/Numpy?

2017-10-03 Thread Lukasz Cwik
This is neat. On Fri, Sep 29, 2017 at 1:26 PM, Vilhelm von Ehrenheim < vonehrenh...@gmail.com> wrote: > Hi Steve! > I have several pipelines that successfully use both numpy and scikit > models without any problems. I don't think I use Pandas atm but I'm sure > that is fine too. > > However, you

Re: Beam and Python API: Pandas/Numpy?

2017-09-29 Thread Vilhelm von Ehrenheim
Hi Steve! I have several pipelines that successfully use both numpy and scikit models without any problems. I don't think I use Pandas atm but I'm sure that is fine too. However, you might have to do some special stuff if you encounter serializabillity problems. I also have tensorflow models in

Beam and Python API: Pandas/Numpy?

2017-09-29 Thread Steven DeLaurentis
Hi everyone, Came across this interesting project recently. Read through some of the docs and still had a question: is it possible to use NumPy/Pandas in the DoFn of a Beam? Or does the requirement of a serializable function preclude this possibility? Thanks, Steve