[jira] [Commented] (GORA-76) Upgrade to Hadoop 1.0.0

2012-04-05 Thread Lewis John McGibbney (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/GORA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247183#comment-13247183
 ] 

Lewis John McGibbney commented on GORA-76:
--

Hi Ferdy, I'm with you on this one. I can replicate absolutely everything 
you've posted above. If you look at your log output, the code fails at line 148 
on HadoopTestCase [0]. This is because 
javax.management.InstanceAlreadyExistsException: MXBean already registered with 
name Hadoop:service=NameNode,name=MetricsSystem,sub=Control

I'll look into it, but right now I need help from elsewhere to find out why 
this is the case.

[0] 
http://svn.apache.org/viewvc/hadoop/common/tags/release-1.0.1/src/test/org/apache/hadoop/mapred/HadoopTestCase.java?view=markup

 Upgrade to Hadoop 1.0.0
 ---

 Key: GORA-76
 URL: https://issues.apache.org/jira/browse/GORA-76
 Project: Apache Gora
  Issue Type: Improvement
  Components: build process, storage
Affects Versions: 0.2
Reporter: Lewis John McGibbney
 Fix For: 0.2

 Attachments: GORA-76.patch


 This was driven entirely by Juliens recent issue over in Nutchland.
 Somewhere down the line this will be useful.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GORA-76) Upgrade to Hadoop 1.0.0

2012-04-05 Thread Ferdy Galema (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/GORA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247248#comment-13247248
 ] 

Ferdy Galema commented on GORA-76:
--

Hi Lewis,

Although it's true that there is the MXBean warning, I think the test fails 
because of this:
12/03/07 17:33:21 WARN datanode.DataNode: Invalid directory in dfs.data.dir: 
Incorrect permission for 
/home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data1, expected: 
rwxr-xr-x, while actual: rwxrwxr-x 
12/03/07 17:33:21 WARN datanode.DataNode: Invalid directory in dfs.data.dir: 
Incorrect permission for 
/home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data2, expected: 
rwxr-xr-x, while actual: rwxrwxr-x 
12/03/07 17:33:21 ERROR datanode.DataNode: All directories in dfs.data.dir are 
invalid.

Of course the MXBean exception might suffer from the same root cause.

 Upgrade to Hadoop 1.0.0
 ---

 Key: GORA-76
 URL: https://issues.apache.org/jira/browse/GORA-76
 Project: Apache Gora
  Issue Type: Improvement
  Components: build process, storage
Affects Versions: 0.2
Reporter: Lewis John McGibbney
 Fix For: 0.2

 Attachments: GORA-76.patch


 This was driven entirely by Juliens recent issue over in Nutchland.
 Somewhere down the line this will be useful.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GORA-76) Upgrade to Hadoop 1.0.0

2012-03-07 Thread Lewis John McGibbney (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/GORA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13224615#comment-13224615
 ] 

Lewis John McGibbney commented on GORA-76:
--

Well o.a.g.mapreduce.DataStoreMapReduceTestBase[0] extends HadoopTestCase [1] 
which is now deprecated. We need to look at refactoring this extension, and 
reimplementing the constructor configurations using the new API.

One other thing Ferdy, I still am not quite sure why we don't rely upon 
hadoop-tests 1.0.0! I did not have any justification other than copying the 
consistency from Nutch trunk over to Nutchgora for this. It's entirely up to 
you if you want to utilise hadoop-tests 1.0.0... they are available on maven 
central. 

[0] 
http://svn.apache.org/viewvc/gora/trunk/gora-core/src/test/java/org/apache/gora/mapreduce/DataStoreMapReduceTestBase.java?view=markup
[1] 
http://javasourcecode.org/html/open-source/hadoop/hadoop-0.20.2/org/apache/hadoop/mapred/HadoopTestCase.html

 Upgrade to Hadoop 1.0.0
 ---

 Key: GORA-76
 URL: https://issues.apache.org/jira/browse/GORA-76
 Project: Apache Gora
  Issue Type: Improvement
  Components: build process, storage
Affects Versions: 0.2
Reporter: Lewis John McGibbney
 Fix For: 0.2

 Attachments: GORA-76.patch


 This was driven entirely by Juliens recent issue over in Nutchland.
 Somewhere down the line this will be useful.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GORA-76) Upgrade to Hadoop 1.0.0

2012-03-07 Thread Lewis John McGibbney (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/GORA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13224617#comment-13224617
 ] 

Lewis John McGibbney commented on GORA-76:
--

Another thing is, that if you look at your first (hadoop-test-0.20.2) output 
you'll see that we never formatted any storage directories. I can only assume 
that the author of the test suit wanted to keep everything as simple as 
possible when utilising the HadoopTestCase. 

 Upgrade to Hadoop 1.0.0
 ---

 Key: GORA-76
 URL: https://issues.apache.org/jira/browse/GORA-76
 Project: Apache Gora
  Issue Type: Improvement
  Components: build process, storage
Affects Versions: 0.2
Reporter: Lewis John McGibbney
 Fix For: 0.2

 Attachments: GORA-76.patch


 This was driven entirely by Juliens recent issue over in Nutchland.
 Somewhere down the line this will be useful.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GORA-76) Upgrade to Hadoop 1.0.0

2012-03-07 Thread Ferdy Galema (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/GORA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13224701#comment-13224701
 ] 

Ferdy Galema commented on GORA-76:
--

Well o.a.g.mapreduce.DataStoreMapReduceTestBase[0] extends HadoopTestCase [1] 
which is now deprecated. We need to look at refactoring this extension, and 
reimplementing the constructor configurations using the new API.
Is it? I could not find any reference that HadoopTestCase is deprecated. It is 
true that some classes were replaced with the *.mapred.* to *.mapreduce.* API 
migration, but I'm not sure if that the case for HadoopTestCase.

One other thing Ferdy, I still am not quite sure why we don't rely upon 
hadoop-tests 1.0.0! I did not have any justification other than copying the 
consistency from Nutch trunk over to Nutchgora for this. It's entirely up to 
you if you want to utilise hadoop-tests 1.0.0... they are available on maven 
central.
Yeah I took that into consideration. I already tried hadoop-test-1.0.0 too, but 
it did not make a difference. But indeed I think it's best to stick to 1.0.0 
anyway.

Another thing is, that if you look at your first (hadoop-test-0.20.2) output 
you'll see that we never formatted any storage directories. I can only assume 
that the author of the test suit wanted to keep everything as simple as 
possible when utilising the HadoopTestCase.
I would expect that this is normally all managed by the mini cluster utilities. 
But I will have another look at it.

Thanks.

 Upgrade to Hadoop 1.0.0
 ---

 Key: GORA-76
 URL: https://issues.apache.org/jira/browse/GORA-76
 Project: Apache Gora
  Issue Type: Improvement
  Components: build process, storage
Affects Versions: 0.2
Reporter: Lewis John McGibbney
 Fix For: 0.2

 Attachments: GORA-76.patch


 This was driven entirely by Juliens recent issue over in Nutchland.
 Somewhere down the line this will be useful.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GORA-76) Upgrade to Hadoop 1.0.0

2012-03-07 Thread Lewis John McGibbney (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/GORA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13224718#comment-13224718
 ] 

Lewis John McGibbney commented on GORA-76:
--

{bq}
Is it? I could not find any reference that HadoopTestCase is deprecated. It is 
true that some classes were replaced with the *.mapred.* to *.mapreduce.* API 
migration, but I'm not sure if that the case for HadoopTestCase. 
{bq}
Sh*t your right. When I was browsing the 1.0.0 code I couldn't see it initially 
but here it is here [1]

So the problem lies somewhere else here then.

[1]http://svn.apache.org/repos/asf/hadoop/common/tags/release-1.0.0/src/test/org/apache/hadoop/mapred/HadoopTestCase.java

 Upgrade to Hadoop 1.0.0
 ---

 Key: GORA-76
 URL: https://issues.apache.org/jira/browse/GORA-76
 Project: Apache Gora
  Issue Type: Improvement
  Components: build process, storage
Affects Versions: 0.2
Reporter: Lewis John McGibbney
 Fix For: 0.2

 Attachments: GORA-76.patch


 This was driven entirely by Juliens recent issue over in Nutchland.
 Somewhere down the line this will be useful.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GORA-76) Upgrade to Hadoop 1.0.0

2012-02-28 Thread Ferdy Galema (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/GORA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13218139#comment-13218139
 ] 

Ferdy Galema commented on GORA-76:
--

Ok I will try to use the same packages for Gora. Will get back at this.

 Upgrade to Hadoop 1.0.0
 ---

 Key: GORA-76
 URL: https://issues.apache.org/jira/browse/GORA-76
 Project: Apache Gora
  Issue Type: Improvement
  Components: build process, storage
Affects Versions: 0.2
Reporter: Lewis John McGibbney
 Fix For: 0.2


 This was driven entirely by Juliens recent issue over in Nutchland.
 Somewhere down the line this will be useful.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira