Hi Eila,
You can turn your DataFrame into a list via *df.values.tolist()* and pass
that into *beam.Create(...)* directly:
****************************************
import apache_beam as beam
import pandas as pd
from apache_beam.options.pipeline_options import PipelineOptions
def print_item(item):
print(item)
df = pd.DataFrame({
'letters' : ['a', 'b', 'c', 'd', 'e'],
'numbers' : [1, 2, 3, 4, 5],
})
with beam.Pipeline() as p:
(
p
## You can also use df.values for a numpy.ndarray
| 'Read From Data Frame' >> beam.Create(df.values.tolist())
| 'Print Results' >> beam.Map(print_item)
)
****************************************
On Aug 8, 2018, at 4:01 AM, Eila Arich-Landkof <[email protected]>
wrote:
Hello all,
Is it possible to create a pcollection from a pandas dataframe
Many thanks,
Eila
———
Eila
www.orielresearch.org
www.meetup.com/deep-learning-in-production
