Re: Need help running beam word count example on apex/hdfs

2018-01-09 Thread Shashank Prabhakara 
For anyone who faces the same issue, I was not able to make work with "mvn
compile exec:java ...". Instead, I ran with "hadoop jar ..." command which
magically fixed this. Best guess is that maven is picking up incompatible
version of commons-io from the wrong side of dependency tree.

Regards,
Shashank

On Mon, Jan 8, 2018 at 4:07 PM, Shashank Prabhakara <shash...@infoworks.io>
wrote:

> Forgot to mention:
>
> Execution works in embedded mode and counts are created on the local fs. I
> need this to run on hdfs/yarn with --embeddedExecution=false.
>
> Regards,
> Shashank
>
> On Mon, Jan 8, 2018 at 3:06 PM, Shashank Prabhakara <shash...@infoworks.io
> > wrote:
>
>> Hi All,
>>
>> I want to test beam on apex using the word count example provided in the
>> beam repository, but I'm facing some difficulties while executing word
>> count as described in the documentation.
>>
>> I'm running hadoop version 2.8.2 on debian in a multi-node environment.
>> I cloned the beam github repository - master branch and executed:
>>
>> cd examples/java
>> mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount
>> -Dexec.args="--inputFile=/tmp/input/pom.xml --output=/tmp/output/counts
>> --runner=ApexRunner --embeddedExecution=false" -Papex-runner
>>
>> However the driver hangs (waited for > 1hr) after printing the classpath
>> on the console. I have attached the stdout and stacktrace to this (Pls let
>> me know if not visible in ML).
>>
>> Thanks in advance for any help.
>>
>> Regards,
>> Shashank
>>
>
>

Re: Regarding Beam Slack Channel

2018-01-08 Thread Shashank Prabhakara 
Got it, thanks.

On 9 Jan 2018 11:53 am, "Jean-Baptiste Onofré" <j...@nanthrax.net> wrote:

> Done, you should have received the invite.
>
> Welcome !
>
> Regards
> JB
>
> On 01/09/2018 06:31 AM, Shashank Prabhakara wrote:
>
>> I'd also like to be added, please.
>>
>> Thanks.
>>
>> On 2018-01-04 11:58, Jean-Baptiste Onofr� <j...@nanthrax.net > j...@nanthrax.net>> wrote:
>>  > Hi,>
>>  >
>>  > you should have received the invite.>
>>  >
>>  > Welcome aboard !>
>>  >
>>  > Regards>
>>  > JB>
>>  >
>>  > On 01/04/2018 01:38 AM, David Sabater Dinter wrote:>
>>  > > Hi,>
>>  > > Can I join also, please?>
>>  > > >
>>  > > Thanks!>
>>  > > >
>>  > > On Thu, Jan 4, 2018 at 12:54 AM Maria Tjahjadi <
>> maria.tjahj...@tokopedia.com <mailto:maria.tjahj...@tokopedia.com> >
>>  > > <ma...@tokopedia.com <mailto:ma...@tokopedia.com>>> wrote:>
>>  > > >
>>  > > Hi,>
>>  > > >
>>  > > Could I join also?>
>>  > > >
>>  > > Thanks,>
>>  > > Maria Tjahjadi>
>>  > > >
>>  > > On 4 Jan 2018, at 06.15, Lukasz Cwik <lc...@google.com > lc...@google.com>>
>>  > > <ma...@google.com <mailto:ma...@google.com>>> wrote:>
>>  > > >
>>  > >> Invite sent, welcome.>
>>  > >>>
>>  > >> On Wed, Jan 3, 2018 at 3:01 PM, Carlos Alonso <
>> car...@mrcalonso.com <mailto:car...@mrcalonso.com>>
>>  > >> <ma...@mrcalonso.com <mailto:ma...@mrcalonso.com>>> wrote:>
>>  > >>>
>>  > >> Hi, I'd like to be added too, please!>
>>  > >>>
>>  > >> Thanks!>
>>  > >>>
>>  > >> On Tue, Dec 19, 2017 at 3:41 PM Jean-Baptiste Onofr� <
>> j...@nanthrax.net <mailto:j...@nanthrax.net>>
>>  > >> <ma...@nanthrax.net <mailto:ma...@nanthrax.net>>> wrote:>
>>  > >>>
>>  > >> Done,>
>>  > >>>
>>  > >> you should have received an invite.>
>>  > >>>
>>  > >> Regards>
>>  > >> JB>
>>  > >>>
>>  > >> On 12/19/2017 03:20 PM, Unais T wrote:>
>>  > >> > Hello>
>>  > >> >>
>>  > >> > Can someone please add me to the Beam slack channel?>
>>  > >> >>
>>  > >> > Thanks.>
>>  > >> >>
>>  > >> >>
>>  > >>>
>>  > >> -->
>>  > >> Jean-Baptiste Onofr�>
>>  > >> jbono...@apache.org <mailto:jbono...@apache.org> <ma...@apache.org
>> <mailto:ma...@apache.org>>>
>>  > >> http://blog.nanthrax.net>
>>  > >> Talend - http://www.talend.com>
>>  > >>>
>>  > >>>
>>  >
>>  > -- >
>>  > Jean-Baptiste Onofr�>
>>  > jbono...@apache.org <mailto:jbono...@apache.org>>
>>  > http://blog.nanthrax.net>
>>  > Talend - http://www.talend.com>
>>  >
>>
>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Re: Regarding Beam Slack Channel

2018-01-08 Thread Shashank Prabhakara 
Hi Austin,

Not sure what you meant by follow along there, but I can't sign up on the
slack page due to email domain restrictions and it does not allow me to
request entry there.

Regards,
Shashank

On Tue, Jan 9, 2018 at 11:11 AM, Austin Bennett <whatwouldausti...@gmail.com
> wrote:

> It'd be easier to follow along there
>
> On Jan 8, 2018 9:32 PM, "Shashank Prabhakara" <shash...@infoworks.io>
> wrote:
>
>> I'd also like to be added, please.
>>
>> Thanks.
>>
>> On 2018-01-04 11:58, Jean-Baptiste Onofr� <j...@nanthrax.net> wrote:
>> > Hi,>
>> >
>> > you should have received the invite.>
>> >
>> > Welcome aboard !>
>> >
>> > Regards>
>> > JB>
>> >
>> > On 01/04/2018 01:38 AM, David Sabater Dinter wrote:>
>> > > Hi,>
>> > > Can I join also, please?>
>> > > >
>> > > Thanks!>
>> > > >
>> > > On Thu, Jan 4, 2018 at 12:54 AM Maria Tjahjadi <
>> maria.tjahj...@tokopedia.com >
>> > > <ma...@tokopedia.com>> wrote:>
>> > > >
>> > > Hi,>
>> > > >
>> > > Could I join also?>
>> > > >
>> > > Thanks,>
>> > > Maria Tjahjadi>
>> > > >
>> > > On 4 Jan 2018, at 06.15, Lukasz Cwik <lc...@google.com>
>> > > <ma...@google.com>> wrote:>
>> > > >
>> > >> Invite sent, welcome.>
>> > >>>
>> > >> On Wed, Jan 3, 2018 at 3:01 PM, Carlos Alonso <
>> car...@mrcalonso.com>
>> > >> <ma...@mrcalonso.com>> wrote:>
>> > >>>
>> > >> Hi, I'd like to be added too, please!>
>> > >>>
>> > >> Thanks!>
>> > >>>
>> > >> On Tue, Dec 19, 2017 at 3:41 PM Jean-Baptiste Onofr� <
>> j...@nanthrax.net>
>> > >> <ma...@nanthrax.net>> wrote:>
>> > >>>
>> > >> Done,>
>> > >>>
>> > >> you should have received an invite.>
>> > >>>
>> > >> Regards>
>> > >> JB>
>> > >>>
>> > >> On 12/19/2017 03:20 PM, Unais T wrote:>
>> > >> > Hello>
>> > >> >>
>> > >> > Can someone please add me to the Beam slack channel?>
>> > >> >>
>> > >> > Thanks.>
>> > >> >>
>> > >> >>
>> > >>>
>> > >> -->
>> > >> Jean-Baptiste Onofr�>
>> > >> jbono...@apache.org <ma...@apache.org>>
>> > >> http://blog.nanthrax.net>
>> > >> Talend - http://www.talend.com>
>> > >>>
>> > >>>
>> >
>> > -- >
>> > Jean-Baptiste Onofr�>
>> > jbono...@apache.org>
>> > http://blog.nanthrax.net>
>> > Talend - http://www.talend.com>
>> >
>>
>

Re: Regarding Beam Slack Channel

2018-01-08 Thread Shashank Prabhakara 
I'd also like to be added, please.

Thanks.

On 2018-01-04 11:58, Jean-Baptiste Onofr�  wrote:
> Hi,>
>
> you should have received the invite.>
>
> Welcome aboard !>
>
> Regards>
> JB>
>
> On 01/04/2018 01:38 AM, David Sabater Dinter wrote:>
> > Hi,>
> > Can I join also, please?>
> > >
> > Thanks!>
> > >
> > On Thu, Jan 4, 2018 at 12:54 AM Maria Tjahjadi <
maria.tjahj...@tokopedia.com >
> > > wrote:>
> > >
> > Hi,>
> > >
> > Could I join also?>
> > >
> > Thanks,>
> > Maria Tjahjadi>
> > >
> > On 4 Jan 2018, at 06.15, Lukasz Cwik 
> > > wrote:>
> > >
> >> Invite sent, welcome.>
> >>>
> >> On Wed, Jan 3, 2018 at 3:01 PM, Carlos Alonso 

> >> > wrote:>
> >>>
> >> Hi, I'd like to be added too, please!>
> >>>
> >> Thanks!>
> >>>
> >> On Tue, Dec 19, 2017 at 3:41 PM Jean-Baptiste Onofr� <
j...@nanthrax.net>
> >> > wrote:>
> >>>
> >> Done,>
> >>>
> >> you should have received an invite.>
> >>>
> >> Regards>
> >> JB>
> >>>
> >> On 12/19/2017 03:20 PM, Unais T wrote:>
> >> > Hello>
> >> >>
> >> > Can someone please add me to the Beam slack channel?>
> >> >>
> >> > Thanks.>
> >> >>
> >> >>
> >>>
> >> -->
> >> Jean-Baptiste Onofr�>
> >> jbono...@apache.org >
> >> http://blog.nanthrax.net>
> >> Talend - http://www.talend.com>
> >>>
> >>>
>
> -- >
> Jean-Baptiste Onofr�>
> jbono...@apache.org>
> http://blog.nanthrax.net>
> Talend - http://www.talend.com>
>

Re: Need help running beam word count example on apex/hdfs

2018-01-08 Thread Shashank Prabhakara 
Forgot to mention:

Execution works in embedded mode and counts are created on the local fs. I
need this to run on hdfs/yarn with --embeddedExecution=false.

Regards,
Shashank

On Mon, Jan 8, 2018 at 3:06 PM, Shashank Prabhakara <shash...@infoworks.io>
wrote:

> Hi All,
>
> I want to test beam on apex using the word count example provided in the
> beam repository, but I'm facing some difficulties while executing word
> count as described in the documentation.
>
> I'm running hadoop version 2.8.2 on debian in a multi-node environment.
> I cloned the beam github repository - master branch and executed:
>
> cd examples/java
> mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount
> -Dexec.args="--inputFile=/tmp/input/pom.xml --output=/tmp/output/counts
> --runner=ApexRunner --embeddedExecution=false" -Papex-runner
>
> However the driver hangs (waited for > 1hr) after printing the classpath
> on the console. I have attached the stdout and stacktrace to this (Pls let
> me know if not visible in ML).
>
> Thanks in advance for any help.
>
> Regards,
> Shashank
>

Need help running beam word count example on apex/hdfs

2018-01-08 Thread Shashank Prabhakara 
Hi All,

I want to test beam on apex using the word count example provided in the
beam repository, but I'm facing some difficulties while executing word
count as described in the documentation.

I'm running hadoop version 2.8.2 on debian in a multi-node environment.
I cloned the beam github repository - master branch and executed:

cd examples/java
mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount
-Dexec.args="--inputFile=/tmp/input/pom.xml --output=/tmp/output/counts
--runner=ApexRunner --embeddedExecution=false" -Papex-runner

However the driver hangs (waited for > 1hr) after printing the classpath on
the console. I have attached the stdout and stacktrace to this (Pls let me
know if not visible in ML).

Thanks in advance for any help.

Regards,
Shashank


stacktrace.out
Description: Binary data


std.out
Description: Binary data