[ 
https://issues.apache.org/jira/browse/BEAM-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359268#comment-16359268
 ] 

Thomas Weise commented on BEAM-2267:
------------------------------------

Found working during 2.2.0 release verification, possibly due to BEAM-2575. 
Please re-open if still an issue.

> Final files for WordCount not appearing with Apex on YARN
> ---------------------------------------------------------
>
>                 Key: BEAM-2267
>                 URL: https://issues.apache.org/jira/browse/BEAM-2267
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-apex
>            Reporter: Kenneth Knowles
>            Priority: Major
>             Fix For: Not applicable
>
>
> When I run WordCount with the Apex runner on a YARN cluster - specifically 
> Dataproc, reading/writing GCS - the word counts are all written to temporary 
> files but they are never moved to their final destination.
> Hadoop version 2.7.3
> Beam RC 2.0.0
> Steps to repro:
> 1. Instantiate archetype (see below)
> 2. Build uber jar {{mvn --settings ../beamrc-settings.xml clean package -P 
> apex-runner}}
> 3. SCP to master (or wherever you'd like to launch from)
> 4. {{java -cp word-count-beam-0.1.jar beamrc.WordCount --runner=ApexRunner 
> --embeddedExecution=false 
> --inputfile=gs://apache-beam-samples/shakespeare/winterstale-personae 
> --output=SOMEWHERE}}
> Appendix: steps to instantiate RC archetype:
> Build an RC-specific {{beamrc-settings.xml}}
> {code}
> <settings>
>   <profiles>
>     <profile>
>       <id>beam-2.0.0</id>
>       <repositories>
>         <repository>
>           <!-- This id _must_ be "archetype" -->
>           <id>archetype</id>
>           <url>RC_REPO</url>
>         </repository>
>       </repositories>
>     </profile>
>   </profiles>
>  
>   <activeProfiles>
>     <activeProfile>beam-2.0.0</activeProfile>
>   </activeProfiles>
> </settings>
> {code}
> And then instantiate like so
> {code}
> mvn archetype:generate \
>       --settings beam-rc-settings.xml \
>       -D archetypeCatalog=internal \
>       -D archetypeGroupId=org.apache.beam \
>       -D archetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
>       -D archetypeVersion=2.0.0 \
>       -D groupId=beamrc \
>       -D artifactId=word-count-beam \
>       -D version="0.1" \
>       -D package=beamrc \
>       -D interactiveMode=false
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to