Thanks for the reply, Okay, I understand...The hama messenger que warning still persists though...I deleted the 'hama.messenger.compression,class' property and re-started hama...
On Sun, Jul 14, 2013 at 2:58 PM, Edward J. Yoon <[email protected]>wrote: > Disabling message compression by deleting > 'hama.messenger.compression.class' property in hama-default.xml might > be helpful. > > However, the fundamental problems won't fixed by client-side > configuration. You need to wait next releases. > > On Sun, Jul 14, 2013 at 8:24 PM, Kostas Xirog <[email protected]> > wrote: > > I installed 0.6.2 and ran my program, and in 0.6.2 it actually uses more > > memory than it did before...plus the program needs more time to > execute... > > > > What could I be doing wrong?...The input file that hama takes in is only > > 172.233.579 Bytes ...Any ideas anyone? > > > > Also I'm getting this warning in 0.6.2 : " WARN > > message.MessageTransferQueueFactory: Message queue is configured on > > deprecated parameter:hama.messenger.queue.class" > > > > Does anyone know how I can fix it and if it might be conflicting with my > > program's execution in some way? > > > > Thank you in advance, > > Kostas X. > > > > > > On Sun, Jul 14, 2013 at 11:51 AM, Edward J. Yoon <[email protected] > >wrote: > > > >> Please use the latest version. > >> > >> On Sun, Jul 14, 2013 at 4:28 PM, Kostas Xirog <[email protected]> > >> wrote: > >> > Thanks for your reply, > >> > > >> > I don't know what I can actually show you that will be of any > >> help(except > >> > from my code which is about 1000 lines), but I'll try to give you guys > >> the > >> > basic idea. > >> > Of course I'm using the hama's graph (implementation of Pregel) for > this. > >> > > >> > My program creates a graph with nodes and edges that both have big > sets > >> of > >> > data (such as recordIds and edge values in each record) , as values. > The > >> > basic idea is that I'm running a query on this graph in the form of a > >> path > >> > (or subgraph), and the program returns the records that contain this > >> path, > >> > as well as the values of each of the records that contain this path. > >> > > >> > The compute function executes and only the nodes that are part of the > >> query > >> > wake up at first, all others halt. As this happens, I collect the > >> recordIds > >> > from the node Values and the edge values from the edges, and when the > end > >> > nodes have been reached, the program terminates, I collect the result > >> from > >> > the end nodes and write it to the result file... > >> > > >> > Is there some way I can access a memory mapping or something?... After > >> > execution with 400.000 records, the log is: > >> > > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: The total number of > supersteps: > >> 48 > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: Counters: 12 > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: > >> > org.apache.hama.graph.GraphJobRunner$GraphJobCounter > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: ITERATIONS=42 > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: MULTISTEP_PARTITIONING=4 > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: INPUT_VERTICES=1001 > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: > >> > org.apache.hama.bsp.JobInProgress$JobCounter > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: SUPERSTEPS=48 > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: LAUNCHED_TASKS=6 > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: > >> > org.apache.hama.bsp.BSPPeerImpl$PeerCounter > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: SUPERSTEP_SUM=294 > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: IO_BYTES_READ=344290795 > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: TIME_IN_SYNC_MS=411231 > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: TOTAL_MESSAGES_SENT=1592 > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: TASK_INPUT_RECORDS=1001 > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: > TOTAL_MESSAGES_RECEIVED=1580 > >> >>13/07/14 10:17:57 INFO bsp.BSPJobClient: TASK_OUTPUT_RECORDS=1001 > >> > Job 1 Finished in 3559.706 seconds > >> > > >> > > >> > Any ideas? > >> > Thanks in advance, > >> > Kostas X. > >> > > >> > > >> > On Sun, Jul 14, 2013 at 10:08 AM, Chia-Hung Lin < > [email protected] > >> >wrote: > >> > > >> >> Any chance to show how the code, logic, log, etc. is executed? Others > >> >> might be able to help spot the issue in underlying infrastructure or > >> >> somewhere else. > >> >> > >> >> On 14 July 2013 15:00, Kostas Xirog <[email protected]> wrote: > >> >> > Hello, > >> >> > > >> >> > I'm running my program with 400.000 records as data and the > execution > >> >> takes > >> >> > 50 minutes whereas the execution of the same query on 200.000 > records > >> >> > takes 70 seconds. Any idea why that might be? I've been monitoring > my > >> >> > system with the 'top' command, and I see that for these 50 minutes > the > >> >> > memory usage is 75.5% and the CPU as at 100 almost constantly... > >> >> > > >> >> > I'm running hama in local mode on one machine with 8GB of RAM and 8 > >> CPUs. > >> >> > Any idea why that might be? Any ideas of how I can fix it? > >> >> > > >> >> > Thanks in advance, > >> >> > Kostas X. > >> >> > >> > >> > >> > >> -- > >> Best Regards, Edward J. Yoon > >> @eddieyoon > >> > > > > -- > Best Regards, Edward J. Yoon > @eddieyoon >
