[
https://issues.apache.org/jira/browse/GORA-227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164096#comment-14164096
]
Lewis John McGibbney edited comment on GORA-227 at 10/8/14 8:21 PM:
--------------------------------------------------------------------
[~talat]
bq. DataStoreFactory should return same instance.
+1
bq. But We should be careful about concurrency issues.
Concurrency can I believe be handled by us using a LinkedHashMap (1) as follows
{code}
Map m = Collections.synchronizedMap(new LinkedHashMap(...));
{code}
(1)
http://docs.oracle.com/javase/7/docs/api/index.html?java/util/LinkedHashMap.html
was (Author: lewismc):
[~talat]
.bq DataStoreFactory should return same instance.
+1
.bq But We should be careful about concurrency issues.
Concurrency can I believe be handled by us using a LinkedHashMap (1) as follows
{code}
Map m = Collections.synchronizedMap(new LinkedHashMap(...));
{code}
(1)
http://docs.oracle.com/javase/7/docs/api/index.html?java/util/LinkedHashMap.html
> Failing assertions when putting and getting Values using MemStore#execute
> -------------------------------------------------------------------------
>
> Key: GORA-227
> URL: https://issues.apache.org/jira/browse/GORA-227
> Project: Apache Gora
> Issue Type: Sub-task
> Components: gora-core
> Affects Versions: 0.3
> Environment: gora-core 0.3, Nutch 2.x HEAD
> Reporter: Lewis John McGibbney
> Fix For: 0.6
>
>
> Test [0] fails with the following useless logging... I need to DEBUG this
> much more throughly
> {code}
> Testcase: testGenerateHighest took 1.845 sec
> FAILED
> expected:<2> but was:<0>
> junit.framework.AssertionFailedError: expected:<2> but was:<0>
> at
> org.apache.nutch.crawl.TestGenerator.testGenerateHighest(TestGenerator.java:78)
> Testcase: testGenerateHostLimit took 1.207 sec
> FAILED
> expected:<1> but was:<0>
> junit.framework.AssertionFailedError: expected:<1> but was:<0>
> at
> org.apache.nutch.crawl.TestGenerator.testGenerateHostLimit(TestGenerator.java:134)
> Testcase: testGenerateDomainLimit took 1.175 sec
> FAILED
> expected:<1> but was:<0>
> junit.framework.AssertionFailedError: expected:<1> but was:<0>
> at
> org.apache.nutch.crawl.TestGenerator.testGenerateDomainLimit(TestGenerator.java:185)
> Testcase: testFilter took 2.31 sec
> FAILED
> expected:<3> but was:<0>
> junit.framework.AssertionFailedError: expected:<3> but was:<0>
> at
> org.apache.nutch.crawl.TestGenerator.testFilter(TestGenerator.java:239)
> {code}
> However so far I have found commonality in the fact that the tests all use
> the following code:
> {code}
> public static ArrayList<URLWebPage> readContents(DataStore<String,WebPage>
> store,
> Mark requiredMark, String... fields) throws Exception {
> ArrayList<URLWebPage> l = new ArrayList<URLWebPage>();
> Query<String, WebPage> query = store.newQuery();
> if (fields != null) {
> query.setFields(fields);
> }
> Result<String, WebPage> results = store.execute(query);
> while (results.next()) {
> try {
> WebPage page = results.get();
> String url = results.getKey();
> if (page == null)
> continue;
> if (requiredMark != null && requiredMark.checkMark(page) == null)
> continue;
> l.add(new URLWebPage(TableUtil.unreverseUrl(url),
> (WebPage)page.clone()));
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> return l;
> }
> {code}
> and also that the assertions are all of the type
> {code}
> ArrayList<URLWebPage> fetchList =
> CrawlTestUtil.readContents(webPageStore, Mark.GENERATE_MARK, FIELDS);
> // verify we got right amount of records
> assertEquals(1, fetchList.size());
> {code}
> [0]
> http://svn.apache.org/viewvc/nutch/branches/2.x/src/test/org/apache/nutch/crawl/TestGenerator.java?view=markup
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
