subject:"Live coding \& reviewing adventures"

Re: Live coding & reviewing adventures

2018-08-02 Thread Holden Karau

Ok Gris has an even more delayed laptop so I'm going to push it out a week
and hope it shows up for then. Sorry about that one and thanks for everyone
who tuned in for the Go SDK one :)

On Mon, Jul 30, 2018 at 1:54 PM, Holden Karau  wrote:

> So small schedule changes.
> I’ll be doing some poking at the Go SDK at 2pm today -
> https://www.youtube.com/watch?v=9UAu1DOZJhM and the one with Gris setting
> up Beam on a new machine will be moved to Friday because her laptop got
> delayed - https://www.youtube.com/watch?v=x8Wg7qCDA5k
>
> On Tue, Jul 24, 2018 at 8:41 PM Holden Karau  wrote:
>
>> I'll be doing this again this week & next looking at a few different
>> topics.
>>
>> Tomorrow (July 25th @ 10am pacific) Gris & I will be updating the PR from
>> my last live stream (adding Python dependency handling) -
>> https://www.twitch.tv/events/P92irbgYR9Sx6nMQ-lGY3g /
>> https://www.youtube.com/watch?v=4xDsY5QL2zM
>>
>> In the afternoon @ 3 pm pacific I'll be looking at the dev tools we've
>> had some discussions around with respect to reviews -
>> https://www.twitch.tv/events/vNzcZ7DdSuGFNYURW_9WEQ /
>> https://www.youtube.com/watch?v=6cTmC_fP9B0
>>
>> Next week on Thursday August 1st @ 2pm pacific Gris & I will be setting
>> up Beam on her new laptop together, so for any new users looking to see how
>> to install Beam from source this one is for you (or for devs looking to see
>> how painful set up is) - https://www.twitch.tv/events
>> /YAYvNp3tT0COkcpNBxnp6A / https://www.youtube.com/watch?v=x8Wg7qCDA5k
>>
>> P.S.
>>
>> As always I'll be doing my regular Friday code reviews in Spark -
>> https://www.youtube.com/watch?v=O4rRx-3PTiM . You can see the other ones
>> I have planned on my twitch  events
>>  and youtube
>> .
>>
>> On Fri, Jul 13, 2018 at 11:54 AM, Holden Karau 
>> wrote:
>>
>>> Hi folks! I've been doing some live coding in my other projects and I
>>> figured I'd do some with Apache Beam as well.
>>>
>>> Today @ 3pm pacific I'm going be doing some impromptu exploration better
>>> review tooling possibilities (looking at forking spark-pr-dashboard for
>>> other projects like beam and setting up mentionbot to work with ASF infra)
>>> - https://www.youtube.com/watch?v=ff8_jbzC8JI
>>>
>>> Next week (Thursday the 19th at 2pm pacific) I'm going to be working on
>>> trying to get easier dependency management for the Python portable runner
>>> in place - https://www.youtube.com/watch?v=Sv0XhS2pYqA
>>>
>>> If your interested in seeing more of the development process I hope you
>>> will join me :)
>>>
>>> P.S.
>>>
>>> You can also follow on twitch which does a better job of notifications
>>> https://www.twitch.tv/holdenkarau
>>>
>>> Also one of the other thing I do is "live reviews" of PRs but they are
>>> generally opt-in and I don't have enough opt-ins from the Beam community to
>>> do live reviews in Beam, if you work on Beam and would be OK with me doing
>>> a live streamed review of your PRs let me know (if your curious to what
>>> they look like you can see some of them here in Spark land
>>> 
>>> ).
>>>
>>> --
>>> Twitter: https://twitter.com/holdenkarau
>>>
>>
>>
>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>>
>


-- 
Twitter: https://twitter.com/holdenkarau

Re: Live coding & reviewing adventures

2018-07-30 Thread Holden Karau

So small schedule changes.
I’ll be doing some poking at the Go SDK at 2pm today -
https://www.youtube.com/watch?v=9UAu1DOZJhM and the one with Gris setting
up Beam on a new machine will be moved to Friday because her laptop got
delayed - https://www.youtube.com/watch?v=x8Wg7qCDA5k

On Tue, Jul 24, 2018 at 8:41 PM Holden Karau  wrote:

> I'll be doing this again this week & next looking at a few different
> topics.
>
> Tomorrow (July 25th @ 10am pacific) Gris & I will be updating the PR from
> my last live stream (adding Python dependency handling) -
> https://www.twitch.tv/events/P92irbgYR9Sx6nMQ-lGY3g /
> https://www.youtube.com/watch?v=4xDsY5QL2zM
>
> In the afternoon @ 3 pm pacific I'll be looking at the dev tools we've had
> some discussions around with respect to reviews - https://www.twitch.tv/
> events/vNzcZ7DdSuGFNYURW_9WEQ / https://www.youtube.com/
> watch?v=6cTmC_fP9B0
>
> Next week on Thursday August 1st @ 2pm pacific Gris & I will be setting up
> Beam on her new laptop together, so for any new users looking to see how to
> install Beam from source this one is for you (or for devs looking to see
> how painful set up is) - https://www.twitch.tv/
> events/YAYvNp3tT0COkcpNBxnp6A / https://www.youtube.com/watch?
> v=x8Wg7qCDA5k
>
> P.S.
>
> As always I'll be doing my regular Friday code reviews in Spark -
> https://www.youtube.com/watch?v=O4rRx-3PTiM . You can see the other ones
> I have planned on my twitch  events
>  and youtube
> .
>
> On Fri, Jul 13, 2018 at 11:54 AM, Holden Karau 
> wrote:
>
>> Hi folks! I've been doing some live coding in my other projects and I
>> figured I'd do some with Apache Beam as well.
>>
>> Today @ 3pm pacific I'm going be doing some impromptu exploration better
>> review tooling possibilities (looking at forking spark-pr-dashboard for
>> other projects like beam and setting up mentionbot to work with ASF infra)
>> - https://www.youtube.com/watch?v=ff8_jbzC8JI
>>
>> Next week (Thursday the 19th at 2pm pacific) I'm going to be working on
>> trying to get easier dependency management for the Python portable runner
>> in place - https://www.youtube.com/watch?v=Sv0XhS2pYqA
>>
>> If your interested in seeing more of the development process I hope you
>> will join me :)
>>
>> P.S.
>>
>> You can also follow on twitch which does a better job of notifications
>> https://www.twitch.tv/holdenkarau
>>
>> Also one of the other thing I do is "live reviews" of PRs but they are
>> generally opt-in and I don't have enough opt-ins from the Beam community to
>> do live reviews in Beam, if you work on Beam and would be OK with me doing
>> a live streamed review of your PRs let me know (if your curious to what
>> they look like you can see some of them here in Spark land
>> 
>> ).
>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>>
>
>
>
> --
> Twitter: https://twitter.com/holdenkarau
>

Re: Live coding & reviewing adventures

2018-07-24 Thread Holden Karau

I'll be doing this again this week & next looking at a few different topics.

Tomorrow (July 25th @ 10am pacific) Gris & I will be updating the PR from
my last live stream (adding Python dependency handling) -
https://www.twitch.tv/events/P92irbgYR9Sx6nMQ-lGY3g /
https://www.youtube.com/watch?v=4xDsY5QL2zM

In the afternoon @ 3 pm pacific I'll be looking at the dev tools we've had
some discussions around with respect to reviews - https://www.twitch.tv/
events/vNzcZ7DdSuGFNYURW_9WEQ / https://www.youtube.com/watch?v=6cTmC_fP9B0

Next week on Thursday August 1st @ 2pm pacific Gris & I will be setting up
Beam on her new laptop together, so for any new users looking to see how to
install Beam from source this one is for you (or for devs looking to see
how painful set up is) - https://www.twitch.tv/events/YAYvNp3tT0COkcpNBxnp6A
/ https://www.youtube.com/watch?v=x8Wg7qCDA5k

P.S.

As always I'll be doing my regular Friday code reviews in Spark -
https://www.youtube.com/watch?v=O4rRx-3PTiM . You can see the other ones I
have planned on my twitch  events
 and youtube
.

On Fri, Jul 13, 2018 at 11:54 AM, Holden Karau  wrote:

> Hi folks! I've been doing some live coding in my other projects and I
> figured I'd do some with Apache Beam as well.
>
> Today @ 3pm pacific I'm going be doing some impromptu exploration better
> review tooling possibilities (looking at forking spark-pr-dashboard for
> other projects like beam and setting up mentionbot to work with ASF infra)
> - https://www.youtube.com/watch?v=ff8_jbzC8JI
>
> Next week (Thursday the 19th at 2pm pacific) I'm going to be working on
> trying to get easier dependency management for the Python portable runner
> in place - https://www.youtube.com/watch?v=Sv0XhS2pYqA
>
> If your interested in seeing more of the development process I hope you
> will join me :)
>
> P.S.
>
> You can also follow on twitch which does a better job of notifications
> https://www.twitch.tv/holdenkarau
>
> Also one of the other thing I do is "live reviews" of PRs but they are
> generally opt-in and I don't have enough opt-ins from the Beam community to
> do live reviews in Beam, if you work on Beam and would be OK with me doing
> a live streamed review of your PRs let me know (if your curious to what
> they look like you can see some of them here in Spark land
> 
> ).
>
> --
> Twitter: https://twitter.com/holdenkarau
>



-- 
Twitter: https://twitter.com/holdenkarau

Re: Live coding & reviewing adventures

2018-07-18 Thread Holden Karau

Ok so follow up I'll be doing part 2 today at noon pacific today -
https://www.youtube.com/watch?v=6krU3YWsgYQ . If your @oscon come see the
talk w/the demo (and other thins) at 2:30 pm pacific in Portland 251 -
https://conferences.oreilly.com/oscon/oscon-or/public/schedule/speaker/128567

As for the venv reqs:

absl-py==0.2.2
apache-beam==2.6.0.dev0
astor==0.7.1
avro==1.8.2
backports-abc==0.5
backports.shutil-get-terminal-size==1.0.0
backports.weakref==1.0.post1
bleach==2.1.3
cachetools==2.1.0
certifi==2018.4.16
chardet==3.0.4
-e git+
https://github.com/holdenk/model-analysis.git@2cee83428d4db58fbba987f3e268114f3ac0e694#egg=chicago_taxi_setup=examples/chicago_taxi
configparser==3.5.0
crcmod==1.7
decorator==4.3.0
dill==0.2.6
docopt==0.6.2
entrypoints==0.2.3
enum34==1.1.6
fastavro==0.19.7
fasteners==0.14.1
funcsigs==1.0.2
functools32==3.2.3.post2
future==0.16.0
futures==3.2.0
gapic-google-cloud-pubsub-v1==0.15.4
gast==0.2.0
google-api-core==1.2.1
google-apitools==0.5.20
google-auth==1.5.0
google-auth-httplib2==0.0.3
google-cloud-bigquery==0.25.0
google-cloud-core==0.25.0
google-cloud-pubsub==0.26.0
google-gax==0.15.16
google-resumable-media==0.3.1
googleapis-common-protos==1.5.3
googledatastore==7.0.1
grpc-google-iam-v1==0.11.1
grpcio==1.13.0
hdfs==2.1.0
html5lib==0.999
httplib2==0.9.2
idna==2.7
ipykernel==4.8.2
ipython==5.7.0
ipython-genutils==0.2.0
ipywidgets==7.2.1
Jinja2==2.10
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.2.3
jupyter-console==5.2.0
jupyter-core==4.4.0
Markdown==2.6.11
MarkupSafe==1.0
mistune==0.8.3
mock==2.0.0
monotonic==1.5
nbconvert==5.3.1
nbformat==4.4.0
notebook==5.5.0
numpy==1.14.5
oauth2client==4.1.2
pandocfilters==1.4.2
pathlib2==2.3.2
pbr==4.1.0
pexpect==4.6.0
pickleshare==0.7.4
pkg-resources==0.0.0
ply==3.8
prompt-toolkit==1.0.15
proto-google-cloud-datastore-v1==0.90.4
proto-google-cloud-pubsub-v1==0.15.4
protobuf==3.6.0
ptyprocess==0.6.0
pyasn1==0.4.3
pyasn1-modules==0.2.2
Pygments==2.2.0
python-dateutil==2.7.3
pytz==2018.4
PyVCF==0.6.8
PyYAML==3.13
pyzmq==17.0.0
qtconsole==4.3.1
requests==2.19.1
rsa==3.4.2
scandir==1.7
Send2Trash==1.5.0
simplegeneric==0.8.1
singledispatch==3.4.0.3
six==1.11.0
tensorboard==1.6.0
tensorflow==1.6.0
tensorflow-model-analysis==0.6.0
tensorflow-serving-api==1.6.0
tensorflow-transform==0.6.0
termcolor==1.1.0
terminado==0.8.1
testpath==0.3.1
tornado==5.0.2
traitlets==4.3.2
typing==3.6.4
urllib3==1.23
wcwidth==0.1.7
Werkzeug==0.14.1
widgetsnbextension==3.2.1



On Wed, Jul 18, 2018 at 8:19 AM, Holden Karau  wrote:

> That’s a thing I’ve been thinking about but haven’t had the time to do
> yet. It’s a bit tricky because I don’t always know what I’m doing before I
> start and remembering to go back and tag things after a long stream is hard.
>
> On Tue, Jul 17, 2018 at 11:11 PM Ismaël Mejía  wrote:
>
>> Have you thought about creating some sort of index page for your past
>> live streams?
>> At least for the non-review ones it can provide great value given that
>> searching videos is not the easiest thing to do.
>> On Wed, Jul 18, 2018 at 12:51 AM Holden Karau 
>> wrote:
>> >
>> > Sure! I’ll respond with a pip freeze when I land.
>
> also ooos forgot to do that, I’ll do it today.
>
>>
>> >
>> > On Tue, Jul 17, 2018 at 2:28 PM Suneel Marthi 
>> wrote:
>> >>
>> >> Could u publish the python transitive deps some place that have the
>> Beam-Flink runner working ?
>> >>
>> >> On Tue, Jul 17, 2018 at 5:26 PM, Holden Karau 
>> wrote:
>> >>>
>> >>> And I've got an hour to kill @ SFO today so at some of the
>> suggestions from folks I'm going to do a more user focused one trying
>> getting the TFT demo to work with the portable flink runner (hopefully) -
>> https://www.youtube.com/watch?v=wL9mvQeN36E
>> >>>
>> >>> On Fri, Jul 13, 2018 at 11:54 AM, Holden Karau 
>> wrote:
>> 
>>  Hi folks! I've been doing some live coding in my other projects and
>> I figured I'd do some with Apache Beam as well.
>> 
>>  Today @ 3pm pacific I'm going be doing some impromptu exploration
>> better review tooling possibilities (looking at forking spark-pr-dashboard
>> for other projects like beam and setting up mentionbot to work with ASF
>> infra) - https://www.youtube.com/watch?v=ff8_jbzC8JI
>> 
>>  Next week (Thursday the 19th at 2pm pacific) I'm going to be working
>> on trying to get easier dependency management for the Python portable
>> runner in place - https://www.youtube.com/watch?v=Sv0XhS2pYqA
>> 
>>  If your interested in seeing more of the development process I hope
>> you will join me :)
>> 
>>  P.S.
>> 
>>  You can also follow on twitch which does a better job of
>> notifications https://www.twitch.tv/holdenkarau
>> 
>>  Also one of the other thing I do is "live reviews" of PRs but they
>> are generally opt-in and I don't have enough opt-ins from the Beam
>> community to do live reviews in Beam, if you work on Beam and would be OK
>> with me doing a live streamed review of your PRs let me know (if your

Re: Live coding & reviewing adventures

2018-07-18 Thread Holden Karau

That’s a thing I’ve been thinking about but haven’t had the time to do yet.
It’s a bit tricky because I don’t always know what I’m doing before I start
and remembering to go back and tag things after a long stream is hard.

On Tue, Jul 17, 2018 at 11:11 PM Ismaël Mejía  wrote:

> Have you thought about creating some sort of index page for your past
> live streams?
> At least for the non-review ones it can provide great value given that
> searching videos is not the easiest thing to do.
> On Wed, Jul 18, 2018 at 12:51 AM Holden Karau 
> wrote:
> >
> > Sure! I’ll respond with a pip freeze when I land.

also ooos forgot to do that, I’ll do it today.

>
> >
> > On Tue, Jul 17, 2018 at 2:28 PM Suneel Marthi 
> wrote:
> >>
> >> Could u publish the python transitive deps some place that have the
> Beam-Flink runner working ?
> >>
> >> On Tue, Jul 17, 2018 at 5:26 PM, Holden Karau 
> wrote:
> >>>
> >>> And I've got an hour to kill @ SFO today so at some of the suggestions
> from folks I'm going to do a more user focused one trying getting the TFT
> demo to work with the portable flink runner (hopefully) -
> https://www.youtube.com/watch?v=wL9mvQeN36E
> >>>
> >>> On Fri, Jul 13, 2018 at 11:54 AM, Holden Karau 
> wrote:
> 
>  Hi folks! I've been doing some live coding in my other projects and I
> figured I'd do some with Apache Beam as well.
> 
>  Today @ 3pm pacific I'm going be doing some impromptu exploration
> better review tooling possibilities (looking at forking spark-pr-dashboard
> for other projects like beam and setting up mentionbot to work with ASF
> infra) - https://www.youtube.com/watch?v=ff8_jbzC8JI
> 
>  Next week (Thursday the 19th at 2pm pacific) I'm going to be working
> on trying to get easier dependency management for the Python portable
> runner in place - https://www.youtube.com/watch?v=Sv0XhS2pYqA
> 
>  If your interested in seeing more of the development process I hope
> you will join me :)
> 
>  P.S.
> 
>  You can also follow on twitch which does a better job of
> notifications https://www.twitch.tv/holdenkarau
> 
>  Also one of the other thing I do is "live reviews" of PRs but they
> are generally opt-in and I don't have enough opt-ins from the Beam
> community to do live reviews in Beam, if you work on Beam and would be OK
> with me doing a live streamed review of your PRs let me know (if your
> curious to what they look like you can see some of them here in Spark land).
> 
>  --
>  Twitter: https://twitter.com/holdenkarau
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> Twitter: https://twitter.com/holdenkarau
> >>
> >>
> > --
> > Twitter: https://twitter.com/holdenkarau
>
-- 
Twitter: https://twitter.com/holdenkarau

Re: Live coding & reviewing adventures

2018-07-18 Thread Ismaël Mejía

Have you thought about creating some sort of index page for your past
live streams?
At least for the non-review ones it can provide great value given that
searching videos is not the easiest thing to do.
On Wed, Jul 18, 2018 at 12:51 AM Holden Karau  wrote:
>
> Sure! I’ll respond with a pip freeze when I land.
>
> On Tue, Jul 17, 2018 at 2:28 PM Suneel Marthi  wrote:
>>
>> Could u publish the python transitive deps some place that have the 
>> Beam-Flink runner working ?
>>
>> On Tue, Jul 17, 2018 at 5:26 PM, Holden Karau  wrote:
>>>
>>> And I've got an hour to kill @ SFO today so at some of the suggestions from 
>>> folks I'm going to do a more user focused one trying getting the TFT demo 
>>> to work with the portable flink runner (hopefully) - 
>>> https://www.youtube.com/watch?v=wL9mvQeN36E
>>>
>>> On Fri, Jul 13, 2018 at 11:54 AM, Holden Karau  wrote:

 Hi folks! I've been doing some live coding in my other projects and I 
 figured I'd do some with Apache Beam as well.

 Today @ 3pm pacific I'm going be doing some impromptu exploration better 
 review tooling possibilities (looking at forking spark-pr-dashboard for 
 other projects like beam and setting up mentionbot to work with ASF infra) 
 - https://www.youtube.com/watch?v=ff8_jbzC8JI

 Next week (Thursday the 19th at 2pm pacific) I'm going to be working on 
 trying to get easier dependency management for the Python portable runner 
 in place - https://www.youtube.com/watch?v=Sv0XhS2pYqA

 If your interested in seeing more of the development process I hope you 
 will join me :)

 P.S.

 You can also follow on twitch which does a better job of notifications 
 https://www.twitch.tv/holdenkarau

 Also one of the other thing I do is "live reviews" of PRs but they are 
 generally opt-in and I don't have enough opt-ins from the Beam community 
 to do live reviews in Beam, if you work on Beam and would be OK with me 
 doing a live streamed review of your PRs let me know (if your curious to 
 what they look like you can see some of them here in Spark land).

 --
 Twitter: https://twitter.com/holdenkarau
>>>
>>>
>>>
>>>
>>> --
>>> Twitter: https://twitter.com/holdenkarau
>>
>>
> --
> Twitter: https://twitter.com/holdenkarau

Re: Live coding & reviewing adventures

2018-07-17 Thread Holden Karau

Sure! I’ll respond with a pip freeze when I land.

On Tue, Jul 17, 2018 at 2:28 PM Suneel Marthi  wrote:

> Could u publish the python transitive deps some place that have the
> Beam-Flink runner working ?
>
> On Tue, Jul 17, 2018 at 5:26 PM, Holden Karau 
> wrote:
>
>> And I've got an hour to kill @ SFO today so at some of the suggestions
>> from folks I'm going to do a more user focused one trying getting the TFT
>> demo to work with the portable flink runner (hopefully) -
>> https://www.youtube.com/watch?v=wL9mvQeN36E
>>
>> On Fri, Jul 13, 2018 at 11:54 AM, Holden Karau 
>> wrote:
>>
>>> Hi folks! I've been doing some live coding in my other projects and I
>>> figured I'd do some with Apache Beam as well.
>>>
>>> Today @ 3pm pacific I'm going be doing some impromptu exploration better
>>> review tooling possibilities (looking at forking spark-pr-dashboard for
>>> other projects like beam and setting up mentionbot to work with ASF infra)
>>> - https://www.youtube.com/watch?v=ff8_jbzC8JI
>>>
>>> Next week (Thursday the 19th at 2pm pacific) I'm going to be working on
>>> trying to get easier dependency management for the Python portable runner
>>> in place - https://www.youtube.com/watch?v=Sv0XhS2pYqA
>>>
>>> If your interested in seeing more of the development process I hope you
>>> will join me :)
>>>
>>> P.S.
>>>
>>> You can also follow on twitch which does a better job of notifications
>>> https://www.twitch.tv/holdenkarau
>>>
>>> Also one of the other thing I do is "live reviews" of PRs but they are
>>> generally opt-in and I don't have enough opt-ins from the Beam community to
>>> do live reviews in Beam, if you work on Beam and would be OK with me doing
>>> a live streamed review of your PRs let me know (if your curious to what
>>> they look like you can see some of them here in Spark land
>>> 
>>> ).
>>>
>>> --
>>> Twitter: https://twitter.com/holdenkarau
>>>
>>
>>
>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>>
>
> --
Twitter: https://twitter.com/holdenkarau

Re: Live coding & reviewing adventures

2018-07-17 Thread Holden Karau

Yup I’ve got a fork of that (I skip re-uploading the input data files since
conference WiFi) and store to gcs so I can see the output.

Didn’t finish everything in today’s stream because the WiFi was a bit
flakey during the first container build and stalled apt, but I’ll do a
follow up to finish it up.

On Tue, Jul 17, 2018 at 2:34 PM Ankur Goenka  wrote:

> +1
> For reference here is a sample job
> https://github.com/axelmagn/model-analysis/blob/axelmagn-hacks/examples/chicago_taxi/preprocess_flink.sh
>
> Also 1 quick heads up, output file will be created in docker container if
> you use local file system.
>
>
>
>
> On Tue, Jul 17, 2018 at 2:27 PM Holden Karau  wrote:
>
>> And I've got an hour to kill @ SFO today so at some of the suggestions
>> from folks I'm going to do a more user focused one trying getting the TFT
>> demo to work with the portable flink runner (hopefully) -
>> https://www.youtube.com/watch?v=wL9mvQeN36E
>>
>> On Fri, Jul 13, 2018 at 11:54 AM, Holden Karau 
>> wrote:
>>
>>> Hi folks! I've been doing some live coding in my other projects and I
>>> figured I'd do some with Apache Beam as well.
>>>
>>> Today @ 3pm pacific I'm going be doing some impromptu exploration better
>>> review tooling possibilities (looking at forking spark-pr-dashboard for
>>> other projects like beam and setting up mentionbot to work with ASF infra)
>>> - https://www.youtube.com/watch?v=ff8_jbzC8JI
>>>
>>> Next week (Thursday the 19th at 2pm pacific) I'm going to be working on
>>> trying to get easier dependency management for the Python portable runner
>>> in place - https://www.youtube.com/watch?v=Sv0XhS2pYqA
>>>
>>> If your interested in seeing more of the development process I hope you
>>> will join me :)
>>>
>>> P.S.
>>>
>>> You can also follow on twitch which does a better job of notifications
>>> https://www.twitch.tv/holdenkarau
>>>
>>> Also one of the other thing I do is "live reviews" of PRs but they are
>>> generally opt-in and I don't have enough opt-ins from the Beam community to
>>> do live reviews in Beam, if you work on Beam and would be OK with me doing
>>> a live streamed review of your PRs let me know (if your curious to what
>>> they look like you can see some of them here in Spark land
>>> 
>>> ).
>>>
>>> --
>>> Twitter: https://twitter.com/holdenkarau
>>>
>>
>>
>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>>
> --
Twitter: https://twitter.com/holdenkarau

Re: Live coding & reviewing adventures

2018-07-17 Thread Ankur Goenka

+1
For reference here is a sample job
https://github.com/axelmagn/model-analysis/blob/axelmagn-hacks/examples/chicago_taxi/preprocess_flink.sh

Also 1 quick heads up, output file will be created in docker container if
you use local file system.




On Tue, Jul 17, 2018 at 2:27 PM Holden Karau  wrote:

> And I've got an hour to kill @ SFO today so at some of the suggestions
> from folks I'm going to do a more user focused one trying getting the TFT
> demo to work with the portable flink runner (hopefully) -
> https://www.youtube.com/watch?v=wL9mvQeN36E
>
> On Fri, Jul 13, 2018 at 11:54 AM, Holden Karau 
> wrote:
>
>> Hi folks! I've been doing some live coding in my other projects and I
>> figured I'd do some with Apache Beam as well.
>>
>> Today @ 3pm pacific I'm going be doing some impromptu exploration better
>> review tooling possibilities (looking at forking spark-pr-dashboard for
>> other projects like beam and setting up mentionbot to work with ASF infra)
>> - https://www.youtube.com/watch?v=ff8_jbzC8JI
>>
>> Next week (Thursday the 19th at 2pm pacific) I'm going to be working on
>> trying to get easier dependency management for the Python portable runner
>> in place - https://www.youtube.com/watch?v=Sv0XhS2pYqA
>>
>> If your interested in seeing more of the development process I hope you
>> will join me :)
>>
>> P.S.
>>
>> You can also follow on twitch which does a better job of notifications
>> https://www.twitch.tv/holdenkarau
>>
>> Also one of the other thing I do is "live reviews" of PRs but they are
>> generally opt-in and I don't have enough opt-ins from the Beam community to
>> do live reviews in Beam, if you work on Beam and would be OK with me doing
>> a live streamed review of your PRs let me know (if your curious to what
>> they look like you can see some of them here in Spark land
>> 
>> ).
>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>>
>
>
>
> --
> Twitter: https://twitter.com/holdenkarau
>

Re: Live coding & reviewing adventures

2018-07-17 Thread Suneel Marthi

Could u publish the python transitive deps some place that have the
Beam-Flink runner working ?

On Tue, Jul 17, 2018 at 5:26 PM, Holden Karau  wrote:

> And I've got an hour to kill @ SFO today so at some of the suggestions
> from folks I'm going to do a more user focused one trying getting the TFT
> demo to work with the portable flink runner (hopefully) -
> https://www.youtube.com/watch?v=wL9mvQeN36E
>
> On Fri, Jul 13, 2018 at 11:54 AM, Holden Karau 
> wrote:
>
>> Hi folks! I've been doing some live coding in my other projects and I
>> figured I'd do some with Apache Beam as well.
>>
>> Today @ 3pm pacific I'm going be doing some impromptu exploration better
>> review tooling possibilities (looking at forking spark-pr-dashboard for
>> other projects like beam and setting up mentionbot to work with ASF infra)
>> - https://www.youtube.com/watch?v=ff8_jbzC8JI
>>
>> Next week (Thursday the 19th at 2pm pacific) I'm going to be working on
>> trying to get easier dependency management for the Python portable runner
>> in place - https://www.youtube.com/watch?v=Sv0XhS2pYqA
>>
>> If your interested in seeing more of the development process I hope you
>> will join me :)
>>
>> P.S.
>>
>> You can also follow on twitch which does a better job of notifications
>> https://www.twitch.tv/holdenkarau
>>
>> Also one of the other thing I do is "live reviews" of PRs but they are
>> generally opt-in and I don't have enough opt-ins from the Beam community to
>> do live reviews in Beam, if you work on Beam and would be OK with me doing
>> a live streamed review of your PRs let me know (if your curious to what
>> they look like you can see some of them here in Spark land
>> 
>> ).
>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>>
>
>
>
> --
> Twitter: https://twitter.com/holdenkarau
>

Re: Live coding & reviewing adventures

2018-07-17 Thread Holden Karau

And I've got an hour to kill @ SFO today so at some of the suggestions from
folks I'm going to do a more user focused one trying getting the TFT demo
to work with the portable flink runner (hopefully) -
https://www.youtube.com/watch?v=wL9mvQeN36E

On Fri, Jul 13, 2018 at 11:54 AM, Holden Karau  wrote:

> Hi folks! I've been doing some live coding in my other projects and I
> figured I'd do some with Apache Beam as well.
>
> Today @ 3pm pacific I'm going be doing some impromptu exploration better
> review tooling possibilities (looking at forking spark-pr-dashboard for
> other projects like beam and setting up mentionbot to work with ASF infra)
> - https://www.youtube.com/watch?v=ff8_jbzC8JI
>
> Next week (Thursday the 19th at 2pm pacific) I'm going to be working on
> trying to get easier dependency management for the Python portable runner
> in place - https://www.youtube.com/watch?v=Sv0XhS2pYqA
>
> If your interested in seeing more of the development process I hope you
> will join me :)
>
> P.S.
>
> You can also follow on twitch which does a better job of notifications
> https://www.twitch.tv/holdenkarau
>
> Also one of the other thing I do is "live reviews" of PRs but they are
> generally opt-in and I don't have enough opt-ins from the Beam community to
> do live reviews in Beam, if you work on Beam and would be OK with me doing
> a live streamed review of your PRs let me know (if your curious to what
> they look like you can see some of them here in Spark land
> 
> ).
>
> --
> Twitter: https://twitter.com/holdenkarau
>



-- 
Twitter: https://twitter.com/holdenkarau

Re: Live coding & reviewing adventures

2018-07-13 Thread Holden Karau

Thats a great idea! I did something like that earlier focused on the Go SDK
only (see - https://www.youtube.com/watch?v=g0Iq4np-WVk &
https://www.youtube.com/watch?v=P4jIfhPTKQo ), and I'll try and do some
more general ones later on as we get more stuff working on the portability
framework with Flink :)

On Fri, Jul 13, 2018 at 12:39 PM, Ankur Goenka  wrote:

> Thanks Holden for doing this.
> Looking forward to attend the live session.
> Suggestion: It will super useful to do a live session for beam setup for
> users and another for contributors.
>
>
> On Fri, Jul 13, 2018 at 12:33 PM Innocent Djiofack 
> wrote:
>
>> Thanks I think this will be super useful. I will tune in.
>>
>> On Fri, Jul 13, 2018 at 2:54 PM Holden Karau 
>> wrote:
>>
>>> Hi folks! I've been doing some live coding in my other projects and I
>>> figured I'd do some with Apache Beam as well.
>>>
>>> Today @ 3pm pacific I'm going be doing some impromptu exploration better
>>> review tooling possibilities (looking at forking spark-pr-dashboard for
>>> other projects like beam and setting up mentionbot to work with ASF infra)
>>> - https://www.youtube.com/watch?v=ff8_jbzC8JI
>>>
>>> Next week (Thursday the 19th at 2pm pacific) I'm going to be working on
>>> trying to get easier dependency management for the Python portable runner
>>> in place - https://www.youtube.com/watch?v=Sv0XhS2pYqA
>>>
>>> If your interested in seeing more of the development process I hope you
>>> will join me :)
>>>
>>> P.S.
>>>
>>> You can also follow on twitch which does a better job of notifications
>>> https://www.twitch.tv/holdenkarau
>>>
>>> Also one of the other thing I do is "live reviews" of PRs but they are
>>> generally opt-in and I don't have enough opt-ins from the Beam community to
>>> do live reviews in Beam, if you work on Beam and would be OK with me doing
>>> a live streamed review of your PRs let me know (if your curious to what
>>> they look like you can see some of them here in Spark land
>>> 
>>> ).
>>>
>>>
>>> --
>>> Twitter: https://twitter.com/holdenkarau
>>>
>> --
>>
>> *DJIOFACK INNOCENT*
>> *"Be better than the day before!" -*
>> *+1 404 751 8024*
>>
>


-- 
Twitter: https://twitter.com/holdenkarau

Re: Live coding & reviewing adventures

2018-07-13 Thread Ankur Goenka

Thanks Holden for doing this.
Looking forward to attend the live session.
Suggestion: It will super useful to do a live session for beam setup for
users and another for contributors.


On Fri, Jul 13, 2018 at 12:33 PM Innocent Djiofack 
wrote:

> Thanks I think this will be super useful. I will tune in.
>
> On Fri, Jul 13, 2018 at 2:54 PM Holden Karau  wrote:
>
>> Hi folks! I've been doing some live coding in my other projects and I
>> figured I'd do some with Apache Beam as well.
>>
>> Today @ 3pm pacific I'm going be doing some impromptu exploration better
>> review tooling possibilities (looking at forking spark-pr-dashboard for
>> other projects like beam and setting up mentionbot to work with ASF infra)
>> - https://www.youtube.com/watch?v=ff8_jbzC8JI
>>
>> Next week (Thursday the 19th at 2pm pacific) I'm going to be working on
>> trying to get easier dependency management for the Python portable runner
>> in place - https://www.youtube.com/watch?v=Sv0XhS2pYqA
>>
>> If your interested in seeing more of the development process I hope you
>> will join me :)
>>
>> P.S.
>>
>> You can also follow on twitch which does a better job of notifications
>> https://www.twitch.tv/holdenkarau
>>
>> Also one of the other thing I do is "live reviews" of PRs but they are
>> generally opt-in and I don't have enough opt-ins from the Beam community to
>> do live reviews in Beam, if you work on Beam and would be OK with me doing
>> a live streamed review of your PRs let me know (if your curious to what
>> they look like you can see some of them here in Spark land
>> 
>> ).
>>
>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>>
> --
>
> *DJIOFACK INNOCENT*
> *"Be better than the day before!" -*
> *+1 404 751 8024*
>

Re: Live coding & reviewing adventures

2018-07-13 Thread Innocent Djiofack

Thanks I think this will be super useful. I will tune in.

On Fri, Jul 13, 2018 at 2:54 PM Holden Karau  wrote:

> Hi folks! I've been doing some live coding in my other projects and I
> figured I'd do some with Apache Beam as well.
>
> Today @ 3pm pacific I'm going be doing some impromptu exploration better
> review tooling possibilities (looking at forking spark-pr-dashboard for
> other projects like beam and setting up mentionbot to work with ASF infra)
> - https://www.youtube.com/watch?v=ff8_jbzC8JI
>
> Next week (Thursday the 19th at 2pm pacific) I'm going to be working on
> trying to get easier dependency management for the Python portable runner
> in place - https://www.youtube.com/watch?v=Sv0XhS2pYqA
>
> If your interested in seeing more of the development process I hope you
> will join me :)
>
> P.S.
>
> You can also follow on twitch which does a better job of notifications
> https://www.twitch.tv/holdenkarau
>
> Also one of the other thing I do is "live reviews" of PRs but they are
> generally opt-in and I don't have enough opt-ins from the Beam community to
> do live reviews in Beam, if you work on Beam and would be OK with me doing
> a live streamed review of your PRs let me know (if your curious to what
> they look like you can see some of them here in Spark land
> 
> ).
>
>
> --
> Twitter: https://twitter.com/holdenkarau
>
-- 

*DJIOFACK INNOCENT*
*"Be better than the day before!" -*
*+1 404 751 8024*

Live coding & reviewing adventures

2018-07-13 Thread Holden Karau

Hi folks! I've been doing some live coding in my other projects and I
figured I'd do some with Apache Beam as well.

Today @ 3pm pacific I'm going be doing some impromptu exploration better
review tooling possibilities (looking at forking spark-pr-dashboard for
other projects like beam and setting up mentionbot to work with ASF infra)
- https://www.youtube.com/watch?v=ff8_jbzC8JI

Next week (Thursday the 19th at 2pm pacific) I'm going to be working on
trying to get easier dependency management for the Python portable runner
in place - https://www.youtube.com/watch?v=Sv0XhS2pYqA

If your interested in seeing more of the development process I hope you
will join me :)

P.S.

You can also follow on twitch which does a better job of notifications
https://www.twitch.tv/holdenkarau

Also one of the other thing I do is "live reviews" of PRs but they are
generally opt-in and I don't have enough opt-ins from the Beam community to
do live reviews in Beam, if you work on Beam and would be OK with me doing
a live streamed review of your PRs let me know (if your curious to what
they look like you can see some of them here in Spark land

).

-- 
Twitter: https://twitter.com/holdenkarau

Re: Live coding & reviewing adventures

Re: Live coding & reviewing adventures

Re: Live coding & reviewing adventures

Re: Live coding & reviewing adventures

Re: Live coding & reviewing adventures

Re: Live coding & reviewing adventures

Re: Live coding & reviewing adventures

Re: Live coding & reviewing adventures

Re: Live coding & reviewing adventures

Re: Live coding & reviewing adventures

Re: Live coding & reviewing adventures

Re: Live coding & reviewing adventures

Re: Live coding & reviewing adventures

Re: Live coding & reviewing adventures

Live coding & reviewing adventures

15 matches

Site Navigation

Mail list logo

Footer information