[jira] [Commented] (GIRAPH-183) Add Claudio's FOSDEM presentation (slides and video) to the site
[
https://issues.apache.org/jira/browse/GIRAPH-183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13255845#comment-13255845
]
Eugene Koontz commented on GIRAPH-183:
--
I found that {{mvn clean compile site}} works on trunk (you may also shorten
build time with: {{mvn -DskipTests clean compile site}}).
The key is that the 'compile' phase prevents the default 'compiler:compile'
('compiler' plugin and 'compile' goal) from executing.
{{mvn -PX clean compile site}} also works for {{X}} in {{{hadoop_trunk,
hadoop_facebook*, hadoop_0.23, hadoop_1.0, hadoop_0.20.203,
hadoop_non_secure}}}.
*with {{-Dhadoop.jar.path=$HOME/hadoop-20/build/hadoop-0.20.1-dev-core.jar}}
I think it would be good to get 'mvn site' working on its own, though, which
will work *if* we can get maven to always run the 'compile' phase prior to the
'site' phase, whenever a user runs 'mvn site'.
Add Claudio's FOSDEM presentation (slides and video) to the site
Key: GIRAPH-183
URL: https://issues.apache.org/jira/browse/GIRAPH-183
Project: Giraph
Issue Type: Improvement
Components: site
Reporter: Claudio Martella
Assignee: Claudio Martella
Priority: Trivial
Labels: newbie
Attachments: GIRAPH-183.diff
Presentation:
http://prezi.com/9ake_klzwrga/apache-giraph-distributed-graph-processing-in-the-cloud/
Video: http://www.youtube.com/watch?v=3ZrqPEIPRe4,
http://www.youtube.com/watch?v=BmRaejKGeDM
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-184) Upgrade to junit4
[ https://issues.apache.org/jira/browse/GIRAPH-184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254838#comment-13254838 ] Eugene Koontz commented on GIRAPH-184: -- (sorry for the double post) Upgrade to junit4 - Key: GIRAPH-184 URL: https://issues.apache.org/jira/browse/GIRAPH-184 Project: Giraph Issue Type: Bug Reporter: Devaraj K Attachments: GIRAPH-184.patch Presently Giraph uses JUnit 3.8.1. We can upgrade to JUnit 4 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-183) Add Claudio's FOSDEM presentation (slides and video) to the site
[
https://issues.apache.org/jira/browse/GIRAPH-183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13253819#comment-13253819
]
Eugene Koontz commented on GIRAPH-183:
--
GIRAPH-168 is definitely related. I get the same kind of errors with mvn
compiler:compile, which is being invoked by mvn site:site:
{code}
[ec2-user@ip-10-161-115-188 giraph]$ mvn clean compiler:compile
[INFO] Scanning for projects...
[INFO]
[INFO]
[INFO] Building Apache Incubator Giraph 0.2-SNAPSHOT
[INFO]
[INFO]
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ giraph ---
[INFO] Deleting /home/ec2-user/giraph/target
[INFO]
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-cli) @ giraph ---
[INFO] Compiling 142 source files to /home/ec2-user/giraph/target/classes
[INFO] -
[ERROR] COMPILATION ERROR :
[INFO] -
[ERROR]
/home/ec2-user/giraph/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java:[66,28]
can\
not find symbol
symbol : class ProtocolSignature
location: package org.apache.hadoop.ipc
{code}
Add Claudio's FOSDEM presentation (slides and video) to the site
Key: GIRAPH-183
URL: https://issues.apache.org/jira/browse/GIRAPH-183
Project: Giraph
Issue Type: Improvement
Components: site
Reporter: Claudio Martella
Assignee: Claudio Martella
Priority: Trivial
Labels: newbie
Attachments: GIRAPH-183.diff
Presentation:
http://prezi.com/9ake_klzwrga/apache-giraph-distributed-graph-processing-in-the-cloud/
Video: http://www.youtube.com/watch?v=3ZrqPEIPRe4,
http://www.youtube.com/watch?v=BmRaejKGeDM
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-183) Add Claudio's FOSDEM presentation (slides and video) to the site
[ https://issues.apache.org/jira/browse/GIRAPH-183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13253899#comment-13253899 ] Eugene Koontz commented on GIRAPH-183: -- Requested help for this on the maven mailing list: http://www.mail-archive.com/users@maven.apache.org/msg124398.html Add Claudio's FOSDEM presentation (slides and video) to the site Key: GIRAPH-183 URL: https://issues.apache.org/jira/browse/GIRAPH-183 Project: Giraph Issue Type: Improvement Components: site Reporter: Claudio Martella Assignee: Claudio Martella Priority: Trivial Labels: newbie Attachments: GIRAPH-183.diff Presentation: http://prezi.com/9ake_klzwrga/apache-giraph-distributed-graph-processing-in-the-cloud/ Video: http://www.youtube.com/watch?v=3ZrqPEIPRe4, http://www.youtube.com/watch?v=BmRaejKGeDM -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-168) Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than HADOOP_FACEBOOK) and remove usage of HADOOP
[
https://issues.apache.org/jira/browse/GIRAPH-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250860#comment-13250860
]
Eugene Koontz commented on GIRAPH-168:
--
Avery, thanks a lot for figuring it out. What you said makes sense now. Hudson
must be building the default profile, hadoop_0.20.203, which is using the munge
plugin now. This is causing the reports to be placed in
trunk/target/munged/surefire-reports.
It would be nice, though, to be able to have Hudson run multiple profiles, as
Jakob mentioned above.
Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than
HADOOP_FACEBOOK) and remove usage of HADOOP
-
Key: GIRAPH-168
URL: https://issues.apache.org/jira/browse/GIRAPH-168
Project: Giraph
Issue Type: Improvement
Affects Versions: 0.2.0
Reporter: Eugene Koontz
Assignee: Eugene Koontz
Attachments: GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch,
GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch
This JIRA relates to the mail thread here:
http://mail-archives.apache.org/mod_mbox/incubator-giraph-dev/201203.mbox/browser
Currently we check for the munge flags HADOOP, HADOOP_FACEBOOK and
HADOOP_NON_SECURE when using munge in a few places. Hopefully we can
eliminate usage of munge in the future, but until then, we can mitigate the
complexity by consolidating the number of flags checked. This JIRA renames
HADOOP_FACEBOOK to HADOOP_SECURE, and removes usages of HADOOP, to handle the
same conditional compilation requirements. It also makes it easier to add
more maven profiles so that we can easily increase our hadoop version
coverage.
This patch modifies the existing hadoop_facebook profile to use the new
HADOOP_SECURE munge flag, rather than HADOOP_FACEBOOK.
It also adds a new hadoop maven profile, hadoop_trunk, which also sets
HADOOP_SECURE.
Finally, it adds a default profile, hadoop_0.20.203. This is needed so that
we can specify its dependencies separately from hadoop_trunk, because the
hadoop dependencies have changed between trunk and 0.205.0 - the former
requires hadoop-common, hadoop-mapreduce-client-core, and
hadoop-mapreduce-client-common, whereas the latter requires hadoop-core.
With this patch, the following passes:
{code}
mvn clean verify mvn -Phadoop_trunk clean verify mvn -Phadoop_0.20.203
clean verify
{code}
Current problems:
* I left in place the usage of HADOOP_NON_SECURE, but note that the profile
that uses this is hadoop_non_secure, which fails to compile on trunk:
https://issues.apache.org/jira/browse/GIRAPH-167 .
* I couldn't get -Phadoop_facebook to work; does this work outside of
Facebook?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-181) Add Hadoop 1.0 profile to pom.xml
[
https://issues.apache.org/jira/browse/GIRAPH-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250884#comment-13250884
]
Eugene Koontz commented on GIRAPH-181:
--
{{mvn -Phadoop_1.0 clean verify}} succeeds.
Add Hadoop 1.0 profile to pom.xml
-
Key: GIRAPH-181
URL: https://issues.apache.org/jira/browse/GIRAPH-181
Project: Giraph
Issue Type: Improvement
Components: build
Affects Versions: 0.2.0
Reporter: Eugene Koontz
Assignee: Eugene Koontz
Fix For: 0.2.0
Attachments: GIRAPH-181.patch
Hadoop 1.0.x is now considered the current stable version of Hadoop,
according to http://hadoop.apache.org/common/releases.html#Download .
This JIRA is to add support within Giraph's maven profile for the 1.0.x
Hadoop release.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-168) Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than HADOOP_FACEBOOK) and remove usage of HADOOP
[
https://issues.apache.org/jira/browse/GIRAPH-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250932#comment-13250932
]
Eugene Koontz commented on GIRAPH-168:
--
+1, it sounds great to me if we can run all the profiles through Hudson for
every commit and patch submission. I'm sure it will catch some Hadoop
inter-version incompatibilities that would not be seen if we only build the
default profile.
Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than
HADOOP_FACEBOOK) and remove usage of HADOOP
-
Key: GIRAPH-168
URL: https://issues.apache.org/jira/browse/GIRAPH-168
Project: Giraph
Issue Type: Improvement
Affects Versions: 0.2.0
Reporter: Eugene Koontz
Assignee: Eugene Koontz
Attachments: GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch,
GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch
This JIRA relates to the mail thread here:
http://mail-archives.apache.org/mod_mbox/incubator-giraph-dev/201203.mbox/browser
Currently we check for the munge flags HADOOP, HADOOP_FACEBOOK and
HADOOP_NON_SECURE when using munge in a few places. Hopefully we can
eliminate usage of munge in the future, but until then, we can mitigate the
complexity by consolidating the number of flags checked. This JIRA renames
HADOOP_FACEBOOK to HADOOP_SECURE, and removes usages of HADOOP, to handle the
same conditional compilation requirements. It also makes it easier to add
more maven profiles so that we can easily increase our hadoop version
coverage.
This patch modifies the existing hadoop_facebook profile to use the new
HADOOP_SECURE munge flag, rather than HADOOP_FACEBOOK.
It also adds a new hadoop maven profile, hadoop_trunk, which also sets
HADOOP_SECURE.
Finally, it adds a default profile, hadoop_0.20.203. This is needed so that
we can specify its dependencies separately from hadoop_trunk, because the
hadoop dependencies have changed between trunk and 0.205.0 - the former
requires hadoop-common, hadoop-mapreduce-client-core, and
hadoop-mapreduce-client-common, whereas the latter requires hadoop-core.
With this patch, the following passes:
{code}
mvn clean verify mvn -Phadoop_trunk clean verify mvn -Phadoop_0.20.203
clean verify
{code}
Current problems:
* I left in place the usage of HADOOP_NON_SECURE, but note that the profile
that uses this is hadoop_non_secure, which fails to compile on trunk:
https://issues.apache.org/jira/browse/GIRAPH-167 .
* I couldn't get -Phadoop_facebook to work; does this work outside of
Facebook?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-168) Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than HADOOP_FACEBOOK) and remove usage of HADOOP
[
https://issues.apache.org/jira/browse/GIRAPH-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250938#comment-13250938
]
Eugene Koontz commented on GIRAPH-168:
--
Although I wonder if Hudson will be able to find the Facebook Hadoop jar? Would
we need to add some repo information to the pom.xml to tell Hudson where it can
find this jar?
Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than
HADOOP_FACEBOOK) and remove usage of HADOOP
-
Key: GIRAPH-168
URL: https://issues.apache.org/jira/browse/GIRAPH-168
Project: Giraph
Issue Type: Improvement
Affects Versions: 0.2.0
Reporter: Eugene Koontz
Assignee: Eugene Koontz
Attachments: GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch,
GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch
This JIRA relates to the mail thread here:
http://mail-archives.apache.org/mod_mbox/incubator-giraph-dev/201203.mbox/browser
Currently we check for the munge flags HADOOP, HADOOP_FACEBOOK and
HADOOP_NON_SECURE when using munge in a few places. Hopefully we can
eliminate usage of munge in the future, but until then, we can mitigate the
complexity by consolidating the number of flags checked. This JIRA renames
HADOOP_FACEBOOK to HADOOP_SECURE, and removes usages of HADOOP, to handle the
same conditional compilation requirements. It also makes it easier to add
more maven profiles so that we can easily increase our hadoop version
coverage.
This patch modifies the existing hadoop_facebook profile to use the new
HADOOP_SECURE munge flag, rather than HADOOP_FACEBOOK.
It also adds a new hadoop maven profile, hadoop_trunk, which also sets
HADOOP_SECURE.
Finally, it adds a default profile, hadoop_0.20.203. This is needed so that
we can specify its dependencies separately from hadoop_trunk, because the
hadoop dependencies have changed between trunk and 0.205.0 - the former
requires hadoop-common, hadoop-mapreduce-client-core, and
hadoop-mapreduce-client-common, whereas the latter requires hadoop-core.
With this patch, the following passes:
{code}
mvn clean verify mvn -Phadoop_trunk clean verify mvn -Phadoop_0.20.203
clean verify
{code}
Current problems:
* I left in place the usage of HADOOP_NON_SECURE, but note that the profile
that uses this is hadoop_non_secure, which fails to compile on trunk:
https://issues.apache.org/jira/browse/GIRAPH-167 .
* I couldn't get -Phadoop_facebook to work; does this work outside of
Facebook?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-168) Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than HADOOP_FACEBOOK) and remove usage of HADOOP
[
https://issues.apache.org/jira/browse/GIRAPH-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250124#comment-13250124
]
Eugene Koontz commented on GIRAPH-168:
--
Avery, thanks for spotting that line. I may have been testing something and
left it in accidentally. I've removed it and {{mvn -Phadoop_facebook
-Dhadoop.jar.path=/Users/ekoontz/hadoop-20/build/hadoop-0.20.1-dev-core.jar
clean verify}} still works. I'm attaching a new patch now.
Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than
HADOOP_FACEBOOK) and remove usage of HADOOP
-
Key: GIRAPH-168
URL: https://issues.apache.org/jira/browse/GIRAPH-168
Project: Giraph
Issue Type: Improvement
Affects Versions: 0.2.0
Reporter: Eugene Koontz
Assignee: Eugene Koontz
Attachments: GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch,
GIRAPH-168.patch, GIRAPH-168.patch
This JIRA relates to the mail thread here:
http://mail-archives.apache.org/mod_mbox/incubator-giraph-dev/201203.mbox/browser
Currently we check for the munge flags HADOOP, HADOOP_FACEBOOK and
HADOOP_NON_SECURE when using munge in a few places. Hopefully we can
eliminate usage of munge in the future, but until then, we can mitigate the
complexity by consolidating the number of flags checked. This JIRA renames
HADOOP_FACEBOOK to HADOOP_SECURE, and removes usages of HADOOP, to handle the
same conditional compilation requirements. It also makes it easier to add
more maven profiles so that we can easily increase our hadoop version
coverage.
This patch modifies the existing hadoop_facebook profile to use the new
HADOOP_SECURE munge flag, rather than HADOOP_FACEBOOK.
It also adds a new hadoop maven profile, hadoop_trunk, which also sets
HADOOP_SECURE.
Finally, it adds a default profile, hadoop_0.20.203. This is needed so that
we can specify its dependencies separately from hadoop_trunk, because the
hadoop dependencies have changed between trunk and 0.205.0 - the former
requires hadoop-common, hadoop-mapreduce-client-core, and
hadoop-mapreduce-client-common, whereas the latter requires hadoop-core.
With this patch, the following passes:
{code}
mvn clean verify mvn -Phadoop_trunk clean verify mvn -Phadoop_0.20.203
clean verify
{code}
Current problems:
* I left in place the usage of HADOOP_NON_SECURE, but note that the profile
that uses this is hadoop_non_secure, which fails to compile on trunk:
https://issues.apache.org/jira/browse/GIRAPH-167 .
* I couldn't get -Phadoop_facebook to work; does this work outside of
Facebook?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-168) Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than HADOOP_FACEBOOK) and remove usage of HADOOP
[
https://issues.apache.org/jira/browse/GIRAPH-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250385#comment-13250385
]
Eugene Koontz commented on GIRAPH-168:
--
Jakob, that would be nice to have. I found this:
http://stackoverflow.com/questions/4932944/maven-build-multiple-profiles-in-one-go
Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than
HADOOP_FACEBOOK) and remove usage of HADOOP
-
Key: GIRAPH-168
URL: https://issues.apache.org/jira/browse/GIRAPH-168
Project: Giraph
Issue Type: Improvement
Affects Versions: 0.2.0
Reporter: Eugene Koontz
Assignee: Eugene Koontz
Attachments: GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch,
GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch
This JIRA relates to the mail thread here:
http://mail-archives.apache.org/mod_mbox/incubator-giraph-dev/201203.mbox/browser
Currently we check for the munge flags HADOOP, HADOOP_FACEBOOK and
HADOOP_NON_SECURE when using munge in a few places. Hopefully we can
eliminate usage of munge in the future, but until then, we can mitigate the
complexity by consolidating the number of flags checked. This JIRA renames
HADOOP_FACEBOOK to HADOOP_SECURE, and removes usages of HADOOP, to handle the
same conditional compilation requirements. It also makes it easier to add
more maven profiles so that we can easily increase our hadoop version
coverage.
This patch modifies the existing hadoop_facebook profile to use the new
HADOOP_SECURE munge flag, rather than HADOOP_FACEBOOK.
It also adds a new hadoop maven profile, hadoop_trunk, which also sets
HADOOP_SECURE.
Finally, it adds a default profile, hadoop_0.20.203. This is needed so that
we can specify its dependencies separately from hadoop_trunk, because the
hadoop dependencies have changed between trunk and 0.205.0 - the former
requires hadoop-common, hadoop-mapreduce-client-core, and
hadoop-mapreduce-client-common, whereas the latter requires hadoop-core.
With this patch, the following passes:
{code}
mvn clean verify mvn -Phadoop_trunk clean verify mvn -Phadoop_0.20.203
clean verify
{code}
Current problems:
* I left in place the usage of HADOOP_NON_SECURE, but note that the profile
that uses this is hadoop_non_secure, which fails to compile on trunk:
https://issues.apache.org/jira/browse/GIRAPH-167 .
* I couldn't get -Phadoop_facebook to work; does this work outside of
Facebook?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-168) Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than HADOOP_FACEBOOK) and remove usage of HADOOP
[
https://issues.apache.org/jira/browse/GIRAPH-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247396#comment-13247396
]
Eugene Koontz commented on GIRAPH-168:
--
Hi Jakob, I wonder if HADOOP_NO_SASL might be better than HADOOP_OLDRPC (since
the divergence in RPC has to do with HADOOP-6419 (Change RPC layer to support
SASL based mutual authentication))?
Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than
HADOOP_FACEBOOK) and remove usage of HADOOP
-
Key: GIRAPH-168
URL: https://issues.apache.org/jira/browse/GIRAPH-168
Project: Giraph
Issue Type: Improvement
Affects Versions: 0.2.0
Reporter: Eugene Koontz
Assignee: Eugene Koontz
Attachments: GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch
This JIRA relates to the mail thread here:
http://mail-archives.apache.org/mod_mbox/incubator-giraph-dev/201203.mbox/browser
Currently we check for the munge flags HADOOP, HADOOP_FACEBOOK and
HADOOP_NON_SECURE when using munge in a few places. Hopefully we can
eliminate usage of munge in the future, but until then, we can mitigate the
complexity by consolidating the number of flags checked. This JIRA renames
HADOOP_FACEBOOK to HADOOP_SECURE, and removes usages of HADOOP, to handle the
same conditional compilation requirements. It also makes it easier to add
more maven profiles so that we can easily increase our hadoop version
coverage.
This patch modifies the existing hadoop_facebook profile to use the new
HADOOP_SECURE munge flag, rather than HADOOP_FACEBOOK.
It also adds a new hadoop maven profile, hadoop_trunk, which also sets
HADOOP_SECURE.
Finally, it adds a default profile, hadoop_0.20.203. This is needed so that
we can specify its dependencies separately from hadoop_trunk, because the
hadoop dependencies have changed between trunk and 0.205.0 - the former
requires hadoop-common, hadoop-mapreduce-client-core, and
hadoop-mapreduce-client-common, whereas the latter requires hadoop-core.
With this patch, the following passes:
{code}
mvn clean verify mvn -Phadoop_trunk clean verify mvn -Phadoop_0.20.203
clean verify
{code}
Current problems:
* I left in place the usage of HADOOP_NON_SECURE, but note that the profile
that uses this is hadoop_non_secure, which fails to compile on trunk:
https://issues.apache.org/jira/browse/GIRAPH-167 .
* I couldn't get -Phadoop_facebook to work; does this work outside of
Facebook?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-164) fix 5 Line is longer than 80 characters style errors in GiraphRunner
[
https://issues.apache.org/jira/browse/GIRAPH-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13234575#comment-13234575
]
Eugene Koontz commented on GIRAPH-164:
--
Thanks Brian!
-Eugene
fix 5 Line is longer than 80 characters style errors in GiraphRunner
--
Key: GIRAPH-164
URL: https://issues.apache.org/jira/browse/GIRAPH-164
Project: Giraph
Issue Type: Bug
Affects Versions: 0.2.0
Reporter: Eugene Koontz
Priority: Trivial
Fix For: 0.2.0
Attachments: GIRAPH-164.patch
{code}
file
name=/Users/ekoontz/giraph/src/main/java/org/apache/giraph/GiraphRunner.java
error line=155 severity=error message=Line is longer than 80
characters.
source=com.puppycrawl.tools.checkstyle.checks.sizes.LineLengthCheck/
error line=156 severity=error message=Line is longer than 80
characters.
source=com.puppycrawl.tools.checkstyle.checks.sizes.LineLengthCheck/
error line=158 severity=error message=Line is longer than 80
characters.
source=com.puppycrawl.tools.checkstyle.checks.sizes.LineLengthCheck/
error line=161 severity=error message=Line is longer than 80
characters.
source=com.puppycrawl.tools.checkstyle.checks.sizes.LineLengthCheck/
/file
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-163) bin/giraph script overwrites CLASSPATH if dev environment detected (this also removes USER_JAR from CLASSPATH)
[ https://issues.apache.org/jira/browse/GIRAPH-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13234590#comment-13234590 ] Eugene Koontz commented on GIRAPH-163: -- Looks good to me; +1. bin/giraph script overwrites CLASSPATH if dev environment detected (this also removes USER_JAR from CLASSPATH) Key: GIRAPH-163 URL: https://issues.apache.org/jira/browse/GIRAPH-163 Project: Giraph Issue Type: Improvement Components: conf and scripts Affects Versions: 0.1.0, 0.2.0 Environment: current trunk of giraph, after running mvn compile (as advised in the quick start guide). Also Hadoop 1.0.1 was used. Reporter: Benjamin Heitmann Labels: newbie Attachments: GIRAPH-163.patch Original Estimate: 1h Remaining Estimate: 1h If no ./lib dir is present, then the bin/giraph script assumes it is running in a dev environment. This chooses an execution path through the bin/giraph script, which overwrites the CLASSPATH variable instead of appending to it. Incidentally, this also removes the name of the jar submitted by the user, which got appended to CLASSPATH earlier in the script. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-158) Support YARN (next generation MapReduce)
[ https://issues.apache.org/jira/browse/GIRAPH-158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13232176#comment-13232176 ] Eugene Koontz commented on GIRAPH-158: -- Not ready for Submit Patch yet: needs to be made into a profile, rather than modifying hadoop.version directly. Support YARN (next generation MapReduce) Key: GIRAPH-158 URL: https://issues.apache.org/jira/browse/GIRAPH-158 Project: Giraph Issue Type: New Feature Reporter: Eugene Koontz Labels: features, hadoop Attachments: GIRAPH-158.patch YARN is a re-architecturing of the Hadoop MapReduce framework, described here: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/YARN.html It would be good to offer support within Giraph for this framework. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
