According to CHANGE log, I've added input/output system to BSP
framework. But, I don't know exactly why we need to check
(conf.get("bsp.input (or output).dir") != null) conditions when
initialize record reader/writer objects?
If there's no objection or opinion, I'd like to change like this:
Index: core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java
===================================================================
--- core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java (revision
1329523)
+++ core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java (working copy)
@@ -185,8 +185,7 @@
initInput();
- // just output something when the user configured it
- if (conf.get("bsp.output.dir") != null) {
+ if (!bspJob.getInputFormat().getClass().equals(NullOutputFormat.class)) {
Path outdir = new Path(conf.get("bsp.output.dir"),
Task.getOutputName(partition));
outWriter = bspJob.getOutputFormat().getRecordWriter(fs, bspJob,
@@ -204,8 +203,7 @@
@SuppressWarnings("unchecked")
public final void initInput() throws IOException {
- // just read input if the user defined one
- if (conf.get("bsp.input.dir") != null) {
+ if (!bspJob.getInputFormat().getClass().equals(NullInputFormat.class)) {
InputSplit inputSplit = null;
// reinstantiate the split
try {
On Tue, Apr 24, 2012 at 6:39 AM, Edward J. Yoon <[email protected]> wrote:
> FYI,
>
> "Let me know if you see any issues with this. It could probably use some
> more testing. I was able to get the unit tests working (even the part
> commented out in the patch) but I had to set fake input and output paths. It
> seems that BSP doesn't initialize the RecordReader and RecordWriter unless
> the configuration options "bsp.input.dir" and "bsp.output.dir" are set."
>
> Sent from my iPad
>
> Begin forwarded message:
>
>> From: "Billie Rinaldi (JIRA)" <[email protected]>
>> Date: April 24, 2012 2:54:35 AM GMT+09:00
>> To: [email protected]
>> Subject: [jira] [Assigned] (ACCUMULO-532) Add BSP input/output formats to
>> client package
>> Reply-To: [email protected]
>>
>> Let me know if you see any issues with this. It could probably use some
>> more testing. I was able to get the unit tests working (even the part
>> commented out in the patch) but I had to set fake input and output paths.
>> It seems that BSP doesn't initialize the RecordReader and RecordWriter
>> unless the configuration options "bsp.input.dir" and "bsp.output.dir" are
>> set.
--
Best Regards, Edward J. Yoon
@eddieyoon
