[jira] [Updated] (GIRAPH-182) Provide SequenceFileVertexOutputFormat as an available OutputFormat
[ https://issues.apache.org/jira/browse/GIRAPH-182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Gollakota updated GIRAPH-182: - Attachment: GIRAPH-182-1.patch Implemented an abstract SequenceFileVertexOutputFormat. Provided an example implementation. > Provide SequenceFileVertexOutputFormat as an available OutputFormat > --- > > Key: GIRAPH-182 > URL: https://issues.apache.org/jira/browse/GIRAPH-182 > Project: Giraph > Issue Type: New Feature > Components: lib >Reporter: Pradeep Gollakota >Assignee: Pradeep Gollakota >Priority: Minor > Attachments: GIRAPH-182-1.patch > > > SequenceFile's are heavily used in Hadoop. We should provide > SequenceFileVertexOutputFormat. Since SequenceFileVertexInputFormat is > already provided, it makes sense to also provide a mirroring OutputFormat -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (GIRAPH-182) Provide SequenceFileVertexOutputFormat as an available OutputFormat
[ https://issues.apache.org/jira/browse/GIRAPH-182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Avery Ching reassigned GIRAPH-182: -- Assignee: Pradeep Gollakota Great! > Provide SequenceFileVertexOutputFormat as an available OutputFormat > --- > > Key: GIRAPH-182 > URL: https://issues.apache.org/jira/browse/GIRAPH-182 > Project: Giraph > Issue Type: New Feature > Components: lib >Reporter: Pradeep Gollakota >Assignee: Pradeep Gollakota >Priority: Minor > > SequenceFile's are heavily used in Hadoop. We should provide > SequenceFileVertexOutputFormat. Since SequenceFileVertexInputFormat is > already provided, it makes sense to also provide a mirroring OutputFormat -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-182) Provide SequenceFileVertexOutputFormat as an available OutputFormat
[ https://issues.apache.org/jira/browse/GIRAPH-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251183#comment-13251183 ] Pradeep Gollakota commented on GIRAPH-182: -- Would be glad to. > Provide SequenceFileVertexOutputFormat as an available OutputFormat > --- > > Key: GIRAPH-182 > URL: https://issues.apache.org/jira/browse/GIRAPH-182 > Project: Giraph > Issue Type: New Feature > Components: lib >Reporter: Pradeep Gollakota >Priority: Minor > > SequenceFile's are heavily used in Hadoop. We should provide > SequenceFileVertexOutputFormat. Since SequenceFileVertexInputFormat is > already provided, it makes sense to also provide a mirroring OutputFormat -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-182) Provide SequenceFileVertexOutputFormat as an available OutputFormat
[ https://issues.apache.org/jira/browse/GIRAPH-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251180#comment-13251180 ] Avery Ching commented on GIRAPH-182: Agreed, would you like to work on it Pradeep? > Provide SequenceFileVertexOutputFormat as an available OutputFormat > --- > > Key: GIRAPH-182 > URL: https://issues.apache.org/jira/browse/GIRAPH-182 > Project: Giraph > Issue Type: New Feature > Components: lib >Reporter: Pradeep Gollakota >Priority: Minor > > SequenceFile's are heavily used in Hadoop. We should provide > SequenceFileVertexOutputFormat. Since SequenceFileVertexInputFormat is > already provided, it makes sense to also provide a mirroring OutputFormat -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (GIRAPH-182) Provide SequenceFileVertexOutputFormat as an available OutputFormat
Provide SequenceFileVertexOutputFormat as an available OutputFormat --- Key: GIRAPH-182 URL: https://issues.apache.org/jira/browse/GIRAPH-182 Project: Giraph Issue Type: New Feature Components: lib Reporter: Pradeep Gollakota Priority: Minor SequenceFile's are heavily used in Hadoop. We should provide SequenceFileVertexOutputFormat. Since SequenceFileVertexInputFormat is already provided, it makes sense to also provide a mirroring OutputFormat -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-168) Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than HADOOP_FACEBOOK) and remove usage of HADOOP
[
https://issues.apache.org/jira/browse/GIRAPH-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250947#comment-13250947
]
Avery Ching commented on GIRAPH-168:
I would ignore the facebook one for now (we can add it later), but I can try
mvn -Phadoop_non_secure clean verify &&
mvn -Phadoop_0.20.203 clean verify &&
mvn clean verify &&
mvn -Phadoop_0.23 clean verify &&
mvn -Phadoop_trunk clean verify
> Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than
> HADOOP_FACEBOOK) and remove usage of HADOOP
> -
>
> Key: GIRAPH-168
> URL: https://issues.apache.org/jira/browse/GIRAPH-168
> Project: Giraph
> Issue Type: Improvement
>Affects Versions: 0.2.0
>Reporter: Eugene Koontz
>Assignee: Eugene Koontz
> Attachments: GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch,
> GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch
>
>
> This JIRA relates to the mail thread here:
> http://mail-archives.apache.org/mod_mbox/incubator-giraph-dev/201203.mbox/browser
> Currently we check for the munge flags HADOOP, HADOOP_FACEBOOK and
> HADOOP_NON_SECURE when using munge in a few places. Hopefully we can
> eliminate usage of munge in the future, but until then, we can mitigate the
> complexity by consolidating the number of flags checked. This JIRA renames
> HADOOP_FACEBOOK to HADOOP_SECURE, and removes usages of HADOOP, to handle the
> same conditional compilation requirements. It also makes it easier to add
> more maven profiles so that we can easily increase our hadoop version
> coverage.
> This patch modifies the existing hadoop_facebook profile to use the new
> HADOOP_SECURE munge flag, rather than HADOOP_FACEBOOK.
> It also adds a new hadoop maven profile, hadoop_trunk, which also sets
> HADOOP_SECURE.
> Finally, it adds a default profile, hadoop_0.20.203. This is needed so that
> we can specify its dependencies separately from hadoop_trunk, because the
> hadoop dependencies have changed between trunk and 0.205.0 - the former
> requires hadoop-common, hadoop-mapreduce-client-core, and
> hadoop-mapreduce-client-common, whereas the latter requires hadoop-core.
> With this patch, the following passes:
> {code}
> mvn clean verify && mvn -Phadoop_trunk clean verify && mvn -Phadoop_0.20.203
> clean verify
> {code}
> Current problems:
> * I left in place the usage of HADOOP_NON_SECURE, but note that the profile
> that uses this is hadoop_non_secure, which fails to compile on trunk:
> https://issues.apache.org/jira/browse/GIRAPH-167 .
> * I couldn't get -Phadoop_facebook to work; does this work outside of
> Facebook?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-168) Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than HADOOP_FACEBOOK) and remove usage of HADOOP
[
https://issues.apache.org/jira/browse/GIRAPH-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250942#comment-13250942
]
Jakob Homan commented on GIRAPH-168:
Can we add the bash script above, where ever Maven decrees such things should
go so it's quick and reproducible for Hudson or contributors. It's not as good
as having the script itself test each profile, but it'll make things easier.
> Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than
> HADOOP_FACEBOOK) and remove usage of HADOOP
> -
>
> Key: GIRAPH-168
> URL: https://issues.apache.org/jira/browse/GIRAPH-168
> Project: Giraph
> Issue Type: Improvement
>Affects Versions: 0.2.0
>Reporter: Eugene Koontz
>Assignee: Eugene Koontz
> Attachments: GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch,
> GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch
>
>
> This JIRA relates to the mail thread here:
> http://mail-archives.apache.org/mod_mbox/incubator-giraph-dev/201203.mbox/browser
> Currently we check for the munge flags HADOOP, HADOOP_FACEBOOK and
> HADOOP_NON_SECURE when using munge in a few places. Hopefully we can
> eliminate usage of munge in the future, but until then, we can mitigate the
> complexity by consolidating the number of flags checked. This JIRA renames
> HADOOP_FACEBOOK to HADOOP_SECURE, and removes usages of HADOOP, to handle the
> same conditional compilation requirements. It also makes it easier to add
> more maven profiles so that we can easily increase our hadoop version
> coverage.
> This patch modifies the existing hadoop_facebook profile to use the new
> HADOOP_SECURE munge flag, rather than HADOOP_FACEBOOK.
> It also adds a new hadoop maven profile, hadoop_trunk, which also sets
> HADOOP_SECURE.
> Finally, it adds a default profile, hadoop_0.20.203. This is needed so that
> we can specify its dependencies separately from hadoop_trunk, because the
> hadoop dependencies have changed between trunk and 0.205.0 - the former
> requires hadoop-common, hadoop-mapreduce-client-core, and
> hadoop-mapreduce-client-common, whereas the latter requires hadoop-core.
> With this patch, the following passes:
> {code}
> mvn clean verify && mvn -Phadoop_trunk clean verify && mvn -Phadoop_0.20.203
> clean verify
> {code}
> Current problems:
> * I left in place the usage of HADOOP_NON_SECURE, but note that the profile
> that uses this is hadoop_non_secure, which fails to compile on trunk:
> https://issues.apache.org/jira/browse/GIRAPH-167 .
> * I couldn't get -Phadoop_facebook to work; does this work outside of
> Facebook?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-168) Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than HADOOP_FACEBOOK) and remove usage of HADOOP
[
https://issues.apache.org/jira/browse/GIRAPH-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250938#comment-13250938
]
Eugene Koontz commented on GIRAPH-168:
--
Although I wonder if Hudson will be able to find the Facebook Hadoop jar? Would
we need to add some repo information to the pom.xml to tell Hudson where it can
find this jar?
> Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than
> HADOOP_FACEBOOK) and remove usage of HADOOP
> -
>
> Key: GIRAPH-168
> URL: https://issues.apache.org/jira/browse/GIRAPH-168
> Project: Giraph
> Issue Type: Improvement
>Affects Versions: 0.2.0
>Reporter: Eugene Koontz
>Assignee: Eugene Koontz
> Attachments: GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch,
> GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch
>
>
> This JIRA relates to the mail thread here:
> http://mail-archives.apache.org/mod_mbox/incubator-giraph-dev/201203.mbox/browser
> Currently we check for the munge flags HADOOP, HADOOP_FACEBOOK and
> HADOOP_NON_SECURE when using munge in a few places. Hopefully we can
> eliminate usage of munge in the future, but until then, we can mitigate the
> complexity by consolidating the number of flags checked. This JIRA renames
> HADOOP_FACEBOOK to HADOOP_SECURE, and removes usages of HADOOP, to handle the
> same conditional compilation requirements. It also makes it easier to add
> more maven profiles so that we can easily increase our hadoop version
> coverage.
> This patch modifies the existing hadoop_facebook profile to use the new
> HADOOP_SECURE munge flag, rather than HADOOP_FACEBOOK.
> It also adds a new hadoop maven profile, hadoop_trunk, which also sets
> HADOOP_SECURE.
> Finally, it adds a default profile, hadoop_0.20.203. This is needed so that
> we can specify its dependencies separately from hadoop_trunk, because the
> hadoop dependencies have changed between trunk and 0.205.0 - the former
> requires hadoop-common, hadoop-mapreduce-client-core, and
> hadoop-mapreduce-client-common, whereas the latter requires hadoop-core.
> With this patch, the following passes:
> {code}
> mvn clean verify && mvn -Phadoop_trunk clean verify && mvn -Phadoop_0.20.203
> clean verify
> {code}
> Current problems:
> * I left in place the usage of HADOOP_NON_SECURE, but note that the profile
> that uses this is hadoop_non_secure, which fails to compile on trunk:
> https://issues.apache.org/jira/browse/GIRAPH-167 .
> * I couldn't get -Phadoop_facebook to work; does this work outside of
> Facebook?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-168) Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than HADOOP_FACEBOOK) and remove usage of HADOOP
[
https://issues.apache.org/jira/browse/GIRAPH-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250932#comment-13250932
]
Eugene Koontz commented on GIRAPH-168:
--
+1, it sounds great to me if we can run all the profiles through Hudson for
every commit and patch submission. I'm sure it will catch some Hadoop
inter-version incompatibilities that would not be seen if we only build the
default profile.
> Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than
> HADOOP_FACEBOOK) and remove usage of HADOOP
> -
>
> Key: GIRAPH-168
> URL: https://issues.apache.org/jira/browse/GIRAPH-168
> Project: Giraph
> Issue Type: Improvement
>Affects Versions: 0.2.0
>Reporter: Eugene Koontz
>Assignee: Eugene Koontz
> Attachments: GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch,
> GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch
>
>
> This JIRA relates to the mail thread here:
> http://mail-archives.apache.org/mod_mbox/incubator-giraph-dev/201203.mbox/browser
> Currently we check for the munge flags HADOOP, HADOOP_FACEBOOK and
> HADOOP_NON_SECURE when using munge in a few places. Hopefully we can
> eliminate usage of munge in the future, but until then, we can mitigate the
> complexity by consolidating the number of flags checked. This JIRA renames
> HADOOP_FACEBOOK to HADOOP_SECURE, and removes usages of HADOOP, to handle the
> same conditional compilation requirements. It also makes it easier to add
> more maven profiles so that we can easily increase our hadoop version
> coverage.
> This patch modifies the existing hadoop_facebook profile to use the new
> HADOOP_SECURE munge flag, rather than HADOOP_FACEBOOK.
> It also adds a new hadoop maven profile, hadoop_trunk, which also sets
> HADOOP_SECURE.
> Finally, it adds a default profile, hadoop_0.20.203. This is needed so that
> we can specify its dependencies separately from hadoop_trunk, because the
> hadoop dependencies have changed between trunk and 0.205.0 - the former
> requires hadoop-common, hadoop-mapreduce-client-core, and
> hadoop-mapreduce-client-common, whereas the latter requires hadoop-core.
> With this patch, the following passes:
> {code}
> mvn clean verify && mvn -Phadoop_trunk clean verify && mvn -Phadoop_0.20.203
> clean verify
> {code}
> Current problems:
> * I left in place the usage of HADOOP_NON_SECURE, but note that the profile
> that uses this is hadoop_non_secure, which fails to compile on trunk:
> https://issues.apache.org/jira/browse/GIRAPH-167 .
> * I couldn't get -Phadoop_facebook to work; does this work outside of
> Facebook?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-181) Add Hadoop 1.0 profile to pom.xml
[ https://issues.apache.org/jira/browse/GIRAPH-181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koontz updated GIRAPH-181: - Attachment: GIRAPH-181.patch Add support for Hadoop 1.0.2 to README. Thanks for the reminder, Avery. Also added some whitespace formatting for consistency. > Add Hadoop 1.0 profile to pom.xml > - > > Key: GIRAPH-181 > URL: https://issues.apache.org/jira/browse/GIRAPH-181 > Project: Giraph > Issue Type: Improvement > Components: build >Affects Versions: 0.2.0 >Reporter: Eugene Koontz >Assignee: Eugene Koontz > Fix For: 0.2.0 > > Attachments: GIRAPH-181.patch, GIRAPH-181.patch > > > Hadoop 1.0.x is now considered the "current stable version" of Hadoop, > according to http://hadoop.apache.org/common/releases.html#Download . > This JIRA is to add support within Giraph's maven profile for the 1.0.x > Hadoop release. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-181) Add Hadoop 1.0 profile to pom.xml
[ https://issues.apache.org/jira/browse/GIRAPH-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250891#comment-13250891 ] Avery Ching commented on GIRAPH-181: Please add documentation for the README too. > Add Hadoop 1.0 profile to pom.xml > - > > Key: GIRAPH-181 > URL: https://issues.apache.org/jira/browse/GIRAPH-181 > Project: Giraph > Issue Type: Improvement > Components: build >Affects Versions: 0.2.0 >Reporter: Eugene Koontz >Assignee: Eugene Koontz > Fix For: 0.2.0 > > Attachments: GIRAPH-181.patch > > > Hadoop 1.0.x is now considered the "current stable version" of Hadoop, > according to http://hadoop.apache.org/common/releases.html#Download . > This JIRA is to add support within Giraph's maven profile for the 1.0.x > Hadoop release. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-181) Add Hadoop 1.0 profile to pom.xml
[ https://issues.apache.org/jira/browse/GIRAPH-181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koontz updated GIRAPH-181: - Attachment: GIRAPH-181.patch > Add Hadoop 1.0 profile to pom.xml > - > > Key: GIRAPH-181 > URL: https://issues.apache.org/jira/browse/GIRAPH-181 > Project: Giraph > Issue Type: Improvement > Components: build >Affects Versions: 0.2.0 >Reporter: Eugene Koontz >Assignee: Eugene Koontz > Fix For: 0.2.0 > > Attachments: GIRAPH-181.patch > > > Hadoop 1.0.x is now considered the "current stable version" of Hadoop, > according to http://hadoop.apache.org/common/releases.html#Download . > This JIRA is to add support within Giraph's maven profile for the 1.0.x > Hadoop release. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-181) Add Hadoop 1.0 profile to pom.xml
[
https://issues.apache.org/jira/browse/GIRAPH-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250884#comment-13250884
]
Eugene Koontz commented on GIRAPH-181:
--
{{mvn -Phadoop_1.0 clean verify}} succeeds.
> Add Hadoop 1.0 profile to pom.xml
> -
>
> Key: GIRAPH-181
> URL: https://issues.apache.org/jira/browse/GIRAPH-181
> Project: Giraph
> Issue Type: Improvement
> Components: build
>Affects Versions: 0.2.0
>Reporter: Eugene Koontz
>Assignee: Eugene Koontz
> Fix For: 0.2.0
>
> Attachments: GIRAPH-181.patch
>
>
> Hadoop 1.0.x is now considered the "current stable version" of Hadoop,
> according to http://hadoop.apache.org/common/releases.html#Download .
> This JIRA is to add support within Giraph's maven profile for the 1.0.x
> Hadoop release.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (GIRAPH-181) Add Hadoop 1.0 profile to pom.xml
Add Hadoop 1.0 profile to pom.xml - Key: GIRAPH-181 URL: https://issues.apache.org/jira/browse/GIRAPH-181 Project: Giraph Issue Type: Improvement Components: build Affects Versions: 0.2.0 Reporter: Eugene Koontz Assignee: Eugene Koontz Fix For: 0.2.0 Hadoop 1.0.x is now considered the "current stable version" of Hadoop, according to http://hadoop.apache.org/common/releases.html#Download . This JIRA is to add support within Giraph's maven profile for the 1.0.x Hadoop release. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-168) Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than HADOOP_FACEBOOK) and remove usage of HADOOP
[
https://issues.apache.org/jira/browse/GIRAPH-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250874#comment-13250874
]
Avery Ching commented on GIRAPH-168:
I can modify Hudson to do execute the commands you used above. Any
thoughts/comments?
> Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than
> HADOOP_FACEBOOK) and remove usage of HADOOP
> -
>
> Key: GIRAPH-168
> URL: https://issues.apache.org/jira/browse/GIRAPH-168
> Project: Giraph
> Issue Type: Improvement
>Affects Versions: 0.2.0
>Reporter: Eugene Koontz
>Assignee: Eugene Koontz
> Attachments: GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch,
> GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch
>
>
> This JIRA relates to the mail thread here:
> http://mail-archives.apache.org/mod_mbox/incubator-giraph-dev/201203.mbox/browser
> Currently we check for the munge flags HADOOP, HADOOP_FACEBOOK and
> HADOOP_NON_SECURE when using munge in a few places. Hopefully we can
> eliminate usage of munge in the future, but until then, we can mitigate the
> complexity by consolidating the number of flags checked. This JIRA renames
> HADOOP_FACEBOOK to HADOOP_SECURE, and removes usages of HADOOP, to handle the
> same conditional compilation requirements. It also makes it easier to add
> more maven profiles so that we can easily increase our hadoop version
> coverage.
> This patch modifies the existing hadoop_facebook profile to use the new
> HADOOP_SECURE munge flag, rather than HADOOP_FACEBOOK.
> It also adds a new hadoop maven profile, hadoop_trunk, which also sets
> HADOOP_SECURE.
> Finally, it adds a default profile, hadoop_0.20.203. This is needed so that
> we can specify its dependencies separately from hadoop_trunk, because the
> hadoop dependencies have changed between trunk and 0.205.0 - the former
> requires hadoop-common, hadoop-mapreduce-client-core, and
> hadoop-mapreduce-client-common, whereas the latter requires hadoop-core.
> With this patch, the following passes:
> {code}
> mvn clean verify && mvn -Phadoop_trunk clean verify && mvn -Phadoop_0.20.203
> clean verify
> {code}
> Current problems:
> * I left in place the usage of HADOOP_NON_SECURE, but note that the profile
> that uses this is hadoop_non_secure, which fails to compile on trunk:
> https://issues.apache.org/jira/browse/GIRAPH-167 .
> * I couldn't get -Phadoop_facebook to work; does this work outside of
> Facebook?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-168) Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than HADOOP_FACEBOOK) and remove usage of HADOOP
[
https://issues.apache.org/jira/browse/GIRAPH-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250860#comment-13250860
]
Eugene Koontz commented on GIRAPH-168:
--
Avery, thanks a lot for figuring it out. What you said makes sense now. Hudson
must be building the default profile, hadoop_0.20.203, which is using the munge
plugin now. This is causing the reports to be placed in
trunk/target/munged/surefire-reports.
It would be nice, though, to be able to have Hudson run multiple profiles, as
Jakob mentioned above.
> Simplify munge directive usage with new munge flag HADOOP_SECURE (rather than
> HADOOP_FACEBOOK) and remove usage of HADOOP
> -
>
> Key: GIRAPH-168
> URL: https://issues.apache.org/jira/browse/GIRAPH-168
> Project: Giraph
> Issue Type: Improvement
>Affects Versions: 0.2.0
>Reporter: Eugene Koontz
>Assignee: Eugene Koontz
> Attachments: GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch,
> GIRAPH-168.patch, GIRAPH-168.patch, GIRAPH-168.patch
>
>
> This JIRA relates to the mail thread here:
> http://mail-archives.apache.org/mod_mbox/incubator-giraph-dev/201203.mbox/browser
> Currently we check for the munge flags HADOOP, HADOOP_FACEBOOK and
> HADOOP_NON_SECURE when using munge in a few places. Hopefully we can
> eliminate usage of munge in the future, but until then, we can mitigate the
> complexity by consolidating the number of flags checked. This JIRA renames
> HADOOP_FACEBOOK to HADOOP_SECURE, and removes usages of HADOOP, to handle the
> same conditional compilation requirements. It also makes it easier to add
> more maven profiles so that we can easily increase our hadoop version
> coverage.
> This patch modifies the existing hadoop_facebook profile to use the new
> HADOOP_SECURE munge flag, rather than HADOOP_FACEBOOK.
> It also adds a new hadoop maven profile, hadoop_trunk, which also sets
> HADOOP_SECURE.
> Finally, it adds a default profile, hadoop_0.20.203. This is needed so that
> we can specify its dependencies separately from hadoop_trunk, because the
> hadoop dependencies have changed between trunk and 0.205.0 - the former
> requires hadoop-common, hadoop-mapreduce-client-core, and
> hadoop-mapreduce-client-common, whereas the latter requires hadoop-core.
> With this patch, the following passes:
> {code}
> mvn clean verify && mvn -Phadoop_trunk clean verify && mvn -Phadoop_0.20.203
> clean verify
> {code}
> Current problems:
> * I left in place the usage of HADOOP_NON_SECURE, but note that the profile
> that uses this is hadoop_non_secure, which fails to compile on trunk:
> https://issues.apache.org/jira/browse/GIRAPH-167 .
> * I couldn't get -Phadoop_facebook to work; does this work outside of
> Facebook?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Incubator PMC/Board report for Apr 2012 ([ppmc])
Dear podling, This email was sent by an automated system on behalf of the Apache Incubator PMC. It is an initial reminder to give you plenty of time to prepare your quarterly board report. The board meeting is scheduled for Wed, 18 April 2012, 10:00:00 PST. The report for your podling will form a part of the Incubator PMC report. The Incubator PMC requires your report to be submitted 2 weeks before the board meeting, to allow sufficient time for review and submission (Wed, Apr 4th). Please submit your report with sufficient time to allow the incubator PMC, and subsequently board members to review and digest. Again, the very latest you should submit your report is 2 weeks prior to the board meeting. Thanks, The Apache Incubator PMC Submitting your Report -- Your report should contain the following: * Your project name * A brief description of your project, which assumes no knowledge of the project or necessarily of its field * A list of the three most important issues to address in the move towards graduation. * Any issues that the Incubator PMC or ASF Board might wish/need to be aware of * How has the community developed since the last report * How has the project developed since the last report. This should be appended to the Incubator Wiki page at: http://wiki.apache.org/incubator/April2012 Note: This manually populated. You may need to wait a little before this page is created from a template. Mentors --- Mentors should review reports for their project(s) and sign them off on the Incubator wiki page. Signing off reports shows that you are following the project - projects that are not signed may raise alarms for the Incubator PMC. Incubator PMC
[jira] [Updated] (GIRAPH-177) SimplePageRankVertex has two redundant casts
[
https://issues.apache.org/jira/browse/GIRAPH-177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated GIRAPH-177:
-
Attachment: GIRAPH-177.patch
> SimplePageRankVertex has two redundant casts
>
>
> Key: GIRAPH-177
> URL: https://issues.apache.org/jira/browse/GIRAPH-177
> Project: Giraph
> Issue Type: Improvement
>Affects Versions: 0.2.0
>Reporter: Jakob Homan
>Priority: Trivial
> Labels: newbie
> Attachments: GIRAPH-177.patch
>
>
> {code}DoubleWritable maxPagerank =
> (DoubleWritable) maxAggreg.getAggregatedValue();
> LOG.info("aggregatedMaxPageRank=" + maxPagerank.get());
> DoubleWritable minPagerank =
> (DoubleWritable) minAggreg.getAggregatedValue();
> LOG.info("aggregatedMinPageRank=" + minPagerank.get());{code}
> Both MinAggregator and MaxAggregator are already parameterized on
> DoubleWritable, so it's not necessary to cast their functions' results.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-179) BspServiceMaster's PathFilter can be simplified
[
https://issues.apache.org/jira/browse/GIRAPH-179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated GIRAPH-179:
-
Attachment: GIRAPH-179.patch
> BspServiceMaster's PathFilter can be simplified
> ---
>
> Key: GIRAPH-179
> URL: https://issues.apache.org/jira/browse/GIRAPH-179
> Project: Giraph
> Issue Type: Improvement
>Affects Versions: 0.2.0
>Reporter: Jakob Homan
>Priority: Trivial
> Labels: newbie
> Attachments: GIRAPH-179.patch
>
>
> {code} /**
>* Only get the finalized checkpoint files
>*/
> public static class FinalizedCheckpointPathFilter implements PathFilter {
> @Override
> public boolean accept(Path path) {
> if (path.getName().endsWith(
> BspService.CHECKPOINT_FINALIZED_POSTFIX)) {
> return true;
> }
> return false;
> }
> }{code}
> we can simplify this, eliminating the if statement and just returning the
> result of {{endsWith()}}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-180) Publish SNAPSHOTs and released artifacts in the Maven repository
[ https://issues.apache.org/jira/browse/GIRAPH-180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paolo Castagna updated GIRAPH-180: -- Description: Currently Giraph uses Maven to drive its build. However, no Maven artifacts nor SNAPSHOTs are published in the Apache Maven repository or Maven central. It would be useful to have Apache Giraph artifacts and SNAPSHOTs published and enable people to use Giraph without recompiling themselves. Right now users can checkout Giraph, mvn install it and use this for their dependency: org.apache.giraph giraph 0.2-SNAPSHOT So, it's not that bad, but it can be better. :-) was: Currently Giraph uses Maven to drive its build. However, no Maven artifacts nor SNAPSHOTs are published in the Apache Maven repository or Maven central. It would be useful to have Apache Giraph artifacts and SNAPSHOTs published and enable people to use Giraph without recompiling themselves. > Publish SNAPSHOTs and released artifacts in the Maven repository > > > Key: GIRAPH-180 > URL: https://issues.apache.org/jira/browse/GIRAPH-180 > Project: Giraph > Issue Type: Improvement > Components: build >Affects Versions: 0.1.0 >Reporter: Paolo Castagna >Priority: Minor > Original Estimate: 4h > Remaining Estimate: 4h > > Currently Giraph uses Maven to drive its build. > However, no Maven artifacts nor SNAPSHOTs are published in the Apache Maven > repository or Maven central. > It would be useful to have Apache Giraph artifacts and SNAPSHOTs published > and enable people to use Giraph without recompiling themselves. > Right now users can checkout Giraph, mvn install it and use this for their > dependency: > > org.apache.giraph > giraph > 0.2-SNAPSHOT > > So, it's not that bad, but it can be better. :-) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (GIRAPH-180) Publish SNAPSHOTs and released artifacts in the Maven repository
Publish SNAPSHOTs and released artifacts in the Maven repository Key: GIRAPH-180 URL: https://issues.apache.org/jira/browse/GIRAPH-180 Project: Giraph Issue Type: Improvement Components: build Affects Versions: 0.1.0 Reporter: Paolo Castagna Priority: Minor Currently Giraph uses Maven to drive its build. However, no Maven artifacts nor SNAPSHOTs are published in the Apache Maven repository or Maven central. It would be useful to have Apache Giraph artifacts and SNAPSHOTs published and enable people to use Giraph without recompiling themselves. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
