[jira] [Commented] (HBASE-16591) Add a docker file only contains java 8 for running pre commit on master
[ https://issues.apache.org/jira/browse/HBASE-16591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15482976#comment-15482976 ] Hudson commented on HBASE-16591: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #1585 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1585/]) HBASE-16591 Add a docker file only contains java 8 for running pre (zhangduo: rev 7bda5151eee2febc03a8e0434705e0aa2d6a8c34) * (add) dev-support/docker/Dockerfile > Add a docker file only contains java 8 for running pre commit on master > --- > > Key: HBASE-16591 > URL: https://issues.apache.org/jira/browse/HBASE-16591 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-16591.patch > > > As described in YETUS-369, this is a workaround before YETUS-369 done. > Hadoop's pre-commit has already been java 8 only, so I think we could just > copy their docker file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16591) Add a docker file only contains java 8 for running pre commit on master
[ https://issues.apache.org/jira/browse/HBASE-16591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15482496#comment-15482496 ] Duo Zhang commented on HBASE-16591: --- No. We could add docker file for other branches in HBASE-16373 to add protoc dependency I think. > Add a docker file only contains java 8 for running pre commit on master > --- > > Key: HBASE-16591 > URL: https://issues.apache.org/jira/browse/HBASE-16591 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-16591.patch > > > As described in YETUS-369, this is a workaround before YETUS-369 done. > Hadoop's pre-commit has already been java 8 only, so I think we could just > copy their docker file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16591) Add a docker file only contains java 8 for running pre commit on master
[ https://issues.apache.org/jira/browse/HBASE-16591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15481846#comment-15481846 ] Sean Busbey commented on HBASE-16591: - +1 sounds good to me. Should this close out HBASE-16373, or not since we'll only have protoc support on master? > Add a docker file only contains java 8 for running pre commit on master > --- > > Key: HBASE-16591 > URL: https://issues.apache.org/jira/browse/HBASE-16591 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-16591.patch > > > As described in YETUS-369, this is a workaround before YETUS-369 done. > Hadoop's pre-commit has already been java 8 only, so I think we could just > copy their docker file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16591) Add a docker file only contains java 8 for running pre commit on master
[
https://issues.apache.org/jira/browse/HBASE-16591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15479866#comment-15479866
]
Duo Zhang commented on HBASE-16591:
---
In the docker file added in the patch above, we have installed
protobuf-compiler.
{noformat}
RUN apt-get -q update && apt-get -q install --no-install-recommends -y \
build-essential \
bzip2 \
cmake \
curl \
doxygen \
fuse \
g++ \
gcc \
git \
gnupg-agent \
make \
libbz2-dev \
libcurl4-openssl-dev \
libfuse-dev \
libprotobuf-dev \
libprotoc-dev \
libsnappy-dev \
libssl-dev \
libtool \
pinentry-curses \
pkg-config \
protobuf-compiler \
protobuf-c-compiler \
python \
python2.7 \
python-pip \
rsync \
snappy \
zlib1g-dev
{noformat}
And for ubuntu trusty(14.04), the default version is '2.5.0-9ubuntu1', which is
the right version for us.
https://launchpad.net/ubuntu/trusty/+package/protobuf-compiler
So let's commit this and try a pre-commit build of HBASE-15624 again? [~busbey]
Thanks.
> Add a docker file only contains java 8 for running pre commit on master
> ---
>
> Key: HBASE-16591
> URL: https://issues.apache.org/jira/browse/HBASE-16591
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-16591.patch
>
>
> As described in YETUS-369, this is a workaround before YETUS-369 done.
> Hadoop's pre-commit has already been java 8 only, so I think we could just
> copy their docker file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16591) Add a docker file only contains java 8 for running pre commit on master
[
https://issues.apache.org/jira/browse/HBASE-16591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15479806#comment-15479806
]
Duo Zhang commented on HBASE-16591:
---
[~busbey] I think the problem is the default docker image of yetus does not
have a protoc?
{noformat}
[WARNING] [protoc, --version] failed: java.io.IOException: Cannot run program
"protoc": error=2, No such file or directory
{noformat}
> Add a docker file only contains java 8 for running pre commit on master
> ---
>
> Key: HBASE-16591
> URL: https://issues.apache.org/jira/browse/HBASE-16591
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-16591.patch
>
>
> As described in YETUS-369, this is a workaround before YETUS-369 done.
> Hadoop's pre-commit has already been java 8 only, so I think we could just
> copy their docker file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16591) Add a docker file only contains java 8 for running pre commit on master
[
https://issues.apache.org/jira/browse/HBASE-16591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15478774#comment-15478774
]
Hadoop QA commented on HBASE-16591:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 4s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
5s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 56s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 11s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 51s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-09-09 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12827864/HBASE-16591.patch |
| JIRA Issue | HBASE-16591 |
| Optional Tests | asflicense shellcheck shelldocs |
| uname | Linux de26e984f878 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / e1e0637 |
| shellcheck | v0.4.4 |
| hbaseprotoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/3490/artifact/patchprocess/patch-hbaseprotoc-root.txt
|
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3490/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Add a docker file only contains java 8 for running pre commit on master
> ---
>
> Key: HBASE-16591
> URL: https://issues.apache.org/jira/browse/HBASE-16591
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-16591.patch
>
>
> As described in YETUS-369, this is a workaround before YETUS-369 done.
> Hadoop's pre-commit has already been java 8 only, so I think we could just
> copy their docker file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16591) Add a docker file only contains java 8 for running pre commit on master
[ https://issues.apache.org/jira/browse/HBASE-16591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15478697#comment-15478697 ] Duo Zhang commented on HBASE-16591: --- They do not change git branch in the pre-commit jenkins scripts. After reading and testing yetus, I found that the 'confirm git environment' is happened before we setup the docker image. The only thing is that I need to add a '--resetrepo' options otherwise yetus will not try to checkout the correct branch. Seems '--jenkins' could also work so it is fine. So I think we could left the docker file empty for branch other than master to let them use default docker image of yetus? Let me prepare the docker file patch. Thanks. > Add a docker file only contains java 8 for running pre commit on master > --- > > Key: HBASE-16591 > URL: https://issues.apache.org/jira/browse/HBASE-16591 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > > As described in YETUS-369, this is a workaround before YETUS-369 done. > Hadoop's pre-commit has already been java 8 only, so I think we could just > copy their docker file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16591) Add a docker file only contains java 8 for running pre commit on master
[
https://issues.apache.org/jira/browse/HBASE-16591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15478546#comment-15478546
]
Duo Zhang commented on HBASE-16591:
---
Seems fine. The default java is 1.8.0_101 and there is only one javac and
compile in the report.
{noformat}
| Vote | Subsystem | Runtime | Comment
| 0 | reexec | 0m 3s | Docker mode activated.
| +1 |@author | 0m 0s | The patch does not contain any @author
| | || tags.
| -1 | test4tests | 0m 0s | The patch doesn't appear to include any
| | || new or modified tests. Please justify
| | || why no new tests are needed for this
| | || patch. Also please list what manual
| | || steps were performed to verify this
| | || patch.
| +1 | mvninstall | 77m 13s | master passed
| +1 |compile | 0m 16s| master passed
| +1 | checkstyle | 2m 12s| master passed
| +1 | mvneclipse | 2m 29s| master passed
| +1 | findbugs | 0m 52s| master passed
| +1 |javadoc | 1m 7s | master passed
| +1 | mvninstall | 0m 18s| the patch passed
| +1 |compile | 0m 15s| the patch passed
| +1 | javac | 0m 15s| the patch passed
| +1 | checkstyle | 0m 20s| the patch passed
| +1 | mvneclipse | 0m 10s| the patch passed
| +1 | whitespace | 0m 0s | The patch has no whitespace issues.
| -1 |hadoopcheck | 44m 32s | The patch causes 10 errors with Hadoop
| | || v2.4.0.
| +1 |hbaseprotoc | 0m 8s | the patch passed
| +1 | findbugs | 0m 58s| the patch passed
| +1 |javadoc | 0m 14s| the patch passed
| +1 | asflicense | 0m 19s| The patch does not generate ASF License
| | || warnings.
| | | 189m 53s |
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.1 Server=1.12.1 Image:yetus/hbase:9d55c22 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux ac76faf581d9 3.13.0-54-generic #91-Ubuntu SMP Tue May 26
19:15:08 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/zhangduo/hbase/code/dev-support/hbase-personality.sh |
| git revision | master / 9d55c22 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| modules | C: hbase-client U: hbase-client |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
Finished build.
{noformat}
[~busbey] Let me check the script of hadoop pre-commit to see how they deal
with the branch selection.
Thanks.
> Add a docker file only contains java 8 for running pre commit on master
> ---
>
> Key: HBASE-16591
> URL: https://issues.apache.org/jira/browse/HBASE-16591
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 2.0.0
>
>
> As described in YETUS-369, this is a workaround before YETUS-369 done.
> Hadoop's pre-commit has already been java 8 only, so I think we could just
> copy their docker file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16591) Add a docker file only contains java 8 for running pre commit on master
[ https://issues.apache.org/jira/browse/HBASE-16591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477869#comment-15477869 ] Sean Busbey commented on HBASE-16591: - I'm not sure, but hadoop is doing it now. It's possibly the branch change in yetus happens before the docker run, which would allow using per-branch images so long as they were all at the same path. > Add a docker file only contains java 8 for running pre commit on master > --- > > Key: HBASE-16591 > URL: https://issues.apache.org/jira/browse/HBASE-16591 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > > As described in YETUS-369, this is a workaround before YETUS-369 done. > Hadoop's pre-commit has already been java 8 only, so I think we could just > copy their docker file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16591) Add a docker file only contains java 8 for running pre commit on master
[ https://issues.apache.org/jira/browse/HBASE-16591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477294#comment-15477294 ] Duo Zhang commented on HBASE-16591: --- I mean, how can we select different docker files if we do not know what branch we are going to test before running the test-patch script of yetus? > Add a docker file only contains java 8 for running pre commit on master > --- > > Key: HBASE-16591 > URL: https://issues.apache.org/jira/browse/HBASE-16591 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > > As described in YETUS-369, this is a workaround before YETUS-369 done. > Hadoop's pre-commit has already been java 8 only, so I think we could just > copy their docker file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16591) Add a docker file only contains java 8 for running pre commit on master
[ https://issues.apache.org/jira/browse/HBASE-16591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477223#comment-15477223 ] Duo Zhang commented on HBASE-16591: --- Ah when will we know what branch we are going to test? > Add a docker file only contains java 8 for running pre commit on master > --- > > Key: HBASE-16591 > URL: https://issues.apache.org/jira/browse/HBASE-16591 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > > As described in YETUS-369, this is a workaround before YETUS-369 done. > Hadoop's pre-commit has already been java 8 only, so I think we could just > copy their docker file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16591) Add a docker file only contains java 8 for running pre commit on master
[ https://issues.apache.org/jira/browse/HBASE-16591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477215#comment-15477215 ] Sean Busbey commented on HBASE-16591: - Won't the git working directory be at master when that check happens though? So yetus will look for that path all the time regardless of branch? > Add a docker file only contains java 8 for running pre commit on master > --- > > Key: HBASE-16591 > URL: https://issues.apache.org/jira/browse/HBASE-16591 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > > As described in YETUS-369, this is a workaround before YETUS-369 done. > Hadoop's pre-commit has already been java 8 only, so I think we could just > copy their docker file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16591) Add a docker file only contains java 8 for running pre commit on master
[ https://issues.apache.org/jira/browse/HBASE-16591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477210#comment-15477210 ] Sean Busbey commented on HBASE-16591: - That's true, yes. I was thinking of once we get things in place for protoc support. HBASE-16373 I think? > Add a docker file only contains java 8 for running pre commit on master > --- > > Key: HBASE-16591 > URL: https://issues.apache.org/jira/browse/HBASE-16591 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > > As described in YETUS-369, this is a workaround before YETUS-369 done. > Hadoop's pre-commit has already been java 8 only, so I think we could just > copy their docker file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16591) Add a docker file only contains java 8 for running pre commit on master
[
https://issues.apache.org/jira/browse/HBASE-16591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477202#comment-15477202
]
Duo Zhang commented on HBASE-16591:
---
[~busbey] Let's move the discussion here
{quote}
We'll also need docker files on the older branches with the right JVMs.
{quote}
Do we really need to do this?
{code}
if [ -r "${COMPONENT}/dev-support/docker/Dockerfile" ]; then
YETUS_ARGS=(--dockerfile="${COMPONENT}/dev-support/docker/Dockerfile"
${YETUS_ARGS[@]})
fi
{code}
Currently we do not specific a docker file when running yetus since
'dev-support/docker/Dockerfile' does not exist, so I think we will just use the
default docker file when running pre commit for HBase? Then I think we only
need to overwrite the default docker file for master and just let the other
branches use the default docker file which means just left the docker file
unexists?
Thanks.
> Add a docker file only contains java 8 for running pre commit on master
> ---
>
> Key: HBASE-16591
> URL: https://issues.apache.org/jira/browse/HBASE-16591
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 2.0.0
>
>
> As described in YETUS-369, this is a workaround before YETUS-369 done.
> Hadoop's pre-commit has already been java 8 only, so I think we could just
> copy their docker file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
