timyuer opened a new pull request, #948:
URL: https://github.com/apache/bigtop/pull/948
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines:
https://cwiki.apache.org/confluence/display/BIGTOP/How+to+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g.,
'BIGTOP-3638: Your PR title ...'.
-->
### Description of PR
BIGTOP-3739: Add Tez support for Bigtop 3.1.0 Mpack
Please use the following snapshot repository to run smoke test:
https://sekikn-repo.s3.ap-southeast-1.amazonaws.com/centos-7/
### How was this patch tested?
Tested along with #945 and #927 in centos7, Looks good to me.
hadoop jar /usr/lib/tez/tez-examples-0.10.1.jar orderedwordcount
/tmp/tezsmokeinput/sample-tez-test /tmp/tezsmokeoutput/
```
[hdfs@ambari-server tez]$ hadoop jar /usr/lib/tez/tez-examples-0.10.1.jar
orderedwordcount /tmp/tezsmokeinput/sample-tez-test /tmp/tezsmokeoutput/
22/07/15 03:22:31 INFO shim.HadoopShimsLoader: Trying to locate
HadoopShimProvider for hadoopVersion=3.2.3, majorVersion=3, minorVersion=2
22/07/15 03:22:31 INFO shim.HadoopShimsLoader: Picked HadoopShim
org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null,
overrideProviderViaConfig=null, hadoopVersion=3.2.3, majorVersion=3,
minorVersion=2
22/07/15 03:22:31 INFO counters.Limits: Counter limits initialized with
parameters: GROUP_NAME_MAX=256, MAX_GROUPS=3000, COUNTER_NAME_MAX=64,
MAX_COUNTERS=10000
22/07/15 03:22:31 INFO counters.Limits: Counter limits initialized with
parameters: GROUP_NAME_MAX=256, MAX_GROUPS=3000, COUNTER_NAME_MAX=64,
MAX_COUNTERS=10000
22/07/15 03:22:31 INFO client.TezClient: Tez Client Version: [
component=tez-api, version=0.10.1,
revision=ff1ba5f1442094bca7ed53aba1faa991499dc1aa,
SCM-URL=scm:git:https://gitbox.apache.org/repos/asf/tez.git,
buildTime=2022-05-07T04:27:17Z, buildUser=jenkins, buildJavaVersion=1.8.0_322 ]
22/07/15 03:22:31 INFO client.RMProxy: Connecting to ResourceManager at
ambari-agent-02/172.30.0.4:8050
22/07/15 03:22:32 INFO examples.OrderedWordCount: Running OrderedWordCount
22/07/15 03:22:33 INFO client.TezClient: Submitting DAG application with id:
application_1657843102751_0006
22/07/15 03:22:33 INFO client.TezClientUtils: Using tez.lib.uris value from
configuration: /apps/tez/tez.tar.gz
22/07/15 03:22:33 INFO client.TezClientUtils: Using tez.lib.uris.classpath
value from configuration:
/etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*,/usr/lib/tez/*,/usr/lib/tez/lib/*
22/07/15 03:22:33 INFO client.TezClient: Stage directory /tmp/hdfs/staging
doesn't exist and is created
22/07/15 03:22:33 INFO client.TezClient: Tez system stage directory
hdfs://ambari-agent-01:8020/tmp/hdfs/staging/.tez/application_1657843102751_0006
doesn't exist and is created
22/07/15 03:22:33 INFO conf.Configuration: resource-types.xml not found
22/07/15 03:22:33 INFO resource.ResourceUtils: Unable to find
'resource-types.xml'.
22/07/15 03:22:34 INFO client.TezClient: Submitting DAG to YARN,
applicationId=application_1657843102751_0006, dagName=OrderedWordCount,
callerContext={ context=TezExamples, callerType=null, callerId=null }
22/07/15 03:22:34 INFO impl.YarnClientImpl: Submitted application
application_1657843102751_0006
22/07/15 03:22:34 INFO client.TezClient: The url to track the Tez AM:
http://ambari-agent-02:8088/proxy/application_1657843102751_0006/
22/07/15 03:22:42 INFO client.DAGClientImpl: DAG initialized:
CurrentState=Running
22/07/15 03:22:42 INFO client.DAGClientImpl: DAG: State: RUNNING Progress:
0% TotalTasks: 3 Succeeded: 0 Running: 0 Failed: 0 Killed: 0
22/07/15 03:22:42 INFO client.DAGClientImpl: VertexStatus:
VertexName: Tokenizer Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0
Failed: 0 Killed: 0
22/07/15 03:22:42 INFO client.DAGClientImpl: VertexStatus:
VertexName: Summation Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0
Failed: 0 Killed: 0
22/07/15 03:22:42 INFO client.DAGClientImpl: VertexStatus:
VertexName: Sorter Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0 Failed: 0
Killed: 0
22/07/15 03:22:46 INFO client.DAGClientImpl: DAG: State: RUNNING Progress:
33.33% TotalTasks: 3 Succeeded: 1 Running: 0 Failed: 0 Killed: 0
22/07/15 03:22:46 INFO client.DAGClientImpl: VertexStatus:
VertexName: Tokenizer Progress: 100% TotalTasks: 1 Succeeded: 1 Running: 0
Failed: 0 Killed: 0
22/07/15 03:22:46 INFO client.DAGClientImpl: VertexStatus:
VertexName: Summation Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0
Failed: 0 Killed: 0
22/07/15 03:22:46 INFO client.DAGClientImpl: VertexStatus:
VertexName: Sorter Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0 Failed: 0
Killed: 0
22/07/15 03:22:49 INFO client.DAGClientImpl: DAG: State: RUNNING Progress:
66.67% TotalTasks: 3 Succeeded: 2 Running: 1 Failed: 0 Killed: 0
22/07/15 03:22:49 INFO client.DAGClientImpl: VertexStatus:
VertexName: Tokenizer Progress: 100% TotalTasks: 1 Succeeded: 1 Running: 0
Failed: 0 Killed: 0
22/07/15 03:22:49 INFO client.DAGClientImpl: VertexStatus:
VertexName: Summation Progress: 100% TotalTasks: 1 Succeeded: 1 Running: 0
Failed: 0 Killed: 0
22/07/15 03:22:49 INFO client.DAGClientImpl: VertexStatus:
VertexName: Sorter Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 1 Failed: 0
Killed: 0
22/07/15 03:22:49 INFO client.DAGClientImpl: DAG: State: SUCCEEDED Progress:
100% TotalTasks: 3 Succeeded: 3 Running: 0 Failed: 0 Killed: 0
22/07/15 03:22:49 INFO client.DAGClientImpl: VertexStatus:
VertexName: Tokenizer Progress: 100% TotalTasks: 1 Succeeded: 1 Running: 0
Failed: 0 Killed: 0
22/07/15 03:22:49 INFO client.DAGClientImpl: VertexStatus:
VertexName: Summation Progress: 100% TotalTasks: 1 Succeeded: 1 Running: 0
Failed: 0 Killed: 0
22/07/15 03:22:49 INFO client.DAGClientImpl: VertexStatus:
VertexName: Sorter Progress: 100% TotalTasks: 1 Succeeded: 1 Running: 0 Failed:
0 Killed: 0
22/07/15 03:22:49 INFO client.DAGClientImpl: DAG completed.
FinalState=SUCCEEDED
```
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'BIGTOP-3638. Your PR title ...')?
- [ ] Make sure that newly added files do not have any licensing issues.
When in doubt refer to https://www.apache.org/licenses/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
