[jira] [Created] (HAMA-856) Bundle Serialization

2014-01-20 Thread Edward J. Yoon (JIRA) 
Edward J. Yoon created HAMA-856:
---

 Summary: Bundle Serialization
 Key: HAMA-856
 URL: https://issues.apache.org/jira/browse/HAMA-856
 Project: Hama
  Issue Type: Bug
Reporter: Edward J. Yoon
Assignee: Edward J. Yoon
 Fix For: 0.7.0


For memory efficiency.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Resolved] (HAMA-845) The size() of Spilling Queue returns always numMessagesWritten

2014-01-20 Thread Edward J. Yoon (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HAMA-845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward J. Yoon resolved HAMA-845.
-

Resolution: Fixed

Fixed.

> The size() of Spilling Queue returns always numMessagesWritten
> --
>
> Key: HAMA-845
> URL: https://issues.apache.org/jira/browse/HAMA-845
> Project: Hama
>  Issue Type: Bug
>  Components: bsp core
>Affects Versions: 0.6.3
>Reporter: Edward J. Yoon
>Assignee: Edward J. Yoon
> Fix For: 0.7.0
>
>
> It should be 
> {code}
>   public int size() {
> return numMessagesWritten - numMessagesRead;
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Resolved] (HAMA-837) Add sort behaviour to runtime partitioner

2014-01-20 Thread Edward J. Yoon (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HAMA-837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward J. Yoon resolved HAMA-837.
-

Resolution: Fixed

Fixed.

> Add sort behaviour to runtime partitioner
> -
>
> Key: HAMA-837
> URL: https://issues.apache.org/jira/browse/HAMA-837
> Project: Hama
>  Issue Type: Bug
>  Components: bsp core
>Reporter: Edward J. Yoon
>Assignee: Edward J. Yoon
> Fix For: 0.7.0
>
> Attachments: HAMA-837_v01.patch, HAMA-837_v02.patch
>
>
> If key is comparable and it need to be sorted by key of record, 
> {code}
>   if (outputPair.getKey() instanceof WritableComparable
>   && conf.getBoolean(Constants.PARTITION_SORT_BY_KEY, false)) {
> {code}
> each patitions should be sorted by key. In here, we can use external merge 
> sort.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (HAMA-853) Outgoing message manager

2014-01-20 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/HAMA-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877105#comment-13877105
 ] 

Hudson commented on HAMA-853:
-

SUCCESS: Integrated in Hama-trunk #246 (See 
[https://builds.apache.org/job/Hama-trunk/246/])
HAMA-853: Refactor Outgoing message manager (edwardyoon) (edwardyoon: rev 
1559885)
* /hama/trunk/CHANGES.txt
* /hama/trunk/c++/src/main/native/examples/conf/matrixmultiplication.xml
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java
* 
/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/AbstractMessageManager.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/MessageManager.java
* 
/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/MessageTransferQueueFactory.java
* 
/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/OutgoingMessageManager.java
* 
/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/OutgoingPOJOMessageBundle.java
* 
/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/DiskQueueTransfer.java
* 
/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/MemoryQueue.java
* 
/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/MemoryQueueTransfer.java
* 
/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/MessageTransferProtocol.java
* 
/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/SortedMemoryQueue.java
* 
/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/SortedMemoryQueueTransfer.java
* 
/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/SpillingQueue.java
* 
/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/SpillingQueueTransfer.java
* /hama/trunk/core/src/test/java/org/apache/hama/bsp/TestCheckpoint.java
* /hama/trunk/core/src/test/java/org/apache/hama/bsp/TestPersistQueue.java
* 
/hama/trunk/core/src/test/java/org/apache/hama/bsp/message/TestHamaMessageManager.java
* /hama/trunk/core/src/test/java/org/apache/hama/pipes/TestPipes.java
* /hama/trunk/graph/src/main/java/org/apache/hama/graph/GraphJob.java


> Outgoing message manager
> 
>
> Key: HAMA-853
> URL: https://issues.apache.org/jira/browse/HAMA-853
> Project: Hama
>  Issue Type: Improvement
>  Components: bsp core
>Affects Versions: 0.6.3
>Reporter: Edward J. Yoon
>Assignee: Edward J. Yoon
> Fix For: 0.7.0
>
> Attachments: HAMA-853_v02.patch, outgoingmanager.patch
>
>
> As we discussed in mailing list, http://markmail.org/message/xgtyuqtdb6vkvbiv
> ...
> We have to think about the streamlined messaging system.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Jenkins build is back to normal : Hama-trunk #246

2014-01-20 Thread Apache Jenkins Server 
See

[jira] [Created] (HAMA-855) Handling Address binding exceptions

2014-01-20 Thread Edward J. Yoon (JIRA) 
Edward J. Yoon created HAMA-855:
---

 Summary: Handling Address binding exceptions
 Key: HAMA-855
 URL: https://issues.apache.org/jira/browse/HAMA-855
 Project: Hama
  Issue Type: Bug
  Components: bsp core
Affects Versions: 0.6.3
Reporter: Edward J. Yoon
 Fix For: 0.7.0


Often, build fails with Address binding exceptions. We can add some re-attempt 
mechanism to avoid this situations.

https://builds.apache.org/job/Hama-trunk/245/testReport/org.apache.hama.bsp/TestBSPMasterGroomServer/testSubmitJob/

{code}
14/01/21 01:15:23 ERROR bsp.GroomServer: Got fatal exception while 
reinitializing GroomServer: java.net.BindException: Problem binding to 
vesta.apache.org/67.195.138.9:53307 : Address already in use
at org.apache.hama.ipc.Server.bind(Server.java:250)
at org.apache.hama.ipc.Server$Listener.(Server.java:316)
at org.apache.hama.ipc.Server.(Server.java:1313)
at org.apache.hama.ipc.RPC$Server.(RPC.java:595)
at org.apache.hama.ipc.RPC.getServer(RPC.java:551)
at org.apache.hama.ipc.RPC.getServer(RPC.java:538)
at org.apache.hama.ipc.RPC.getServer(RPC.java:528)
at org.apache.hama.bsp.GroomServer.initialize(GroomServer.java:326)
at org.apache.hama.bsp.GroomServer.run(GroomServer.java:832)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind(Native Method)
at 
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:124)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59)
at org.apache.hama.ipc.Server.bind(Server.java:248)
... 9 more
{code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Build failed in Jenkins: Hama-trunk #245

2014-01-20 Thread Apache Jenkins Server 
See 

Changes:

[edwardyoon] stylesheet change

--
[...truncated 20746 lines...]

14/01/21 01:18:02 INFO bsp.BSPJobClient: TOTAL_MESSAGES_RECEIVED=135

14/01/21 01:18:02 INFO bsp.BSPJobClient: TASK_OUTPUT_RECORDS=135

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec
Running org.apache.hama.zookeeper.TestZKTools
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.009 sec
Running org.apache.hama.bsp.TestZooKeeper
14/01/21 01:18:03 ERROR quorum.QuorumPeerConfig: Invalid configuration, only 
one server specified (ignoring)

14/01/21 01:18:03 INFO server.ZooKeeperServerMain: Starting server

14/01/21 01:18:03 INFO server.ZooKeeperServer: Server 
environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT

14/01/21 01:18:03 INFO server.ZooKeeperServer: Server 
environment:host.name=vesta.apache.org

14/01/21 01:18:03 INFO server.ZooKeeperServer: Server 
environment:java.version=1.6.0_45

14/01/21 01:18:03 INFO server.ZooKeeperServer: Server 
environment:java.vendor=Sun Microsystems Inc.

14/01/21 01:18:03 INFO server.ZooKeeperServer: Server 
environment:java.home=/home/hudson/tools/java/jdk1.6.0_45-32/jre

14/01/21 01:18:03 INFO server.ZooKeeperServer: Server 
environment:java.class.path=/tmp/surefirebooter5359103140981852488.jar

14/01/21 01:18:03 INFO server.ZooKeeperServer: Server 
environment:java.library.path=/home/hudson/tools/java/jdk1.6.0_45-32/jre/lib/i386/server:/home/hudson/tools/java/jdk1.6.0_45-32/jre/lib/i386:/home/hudson/tools/java/jdk1.6.0_45-32/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib

14/01/21 01:18:03 INFO server.ZooKeeperServer: Server 
environment:java.io.tmpdir=/tmp

14/01/21 01:18:03 INFO server.ZooKeeperServer: Server 
environment:java.compiler=

14/01/21 01:18:03 INFO server.ZooKeeperServer: Server environment:os.name=Linux

14/01/21 01:18:03 INFO server.ZooKeeperServer: Server environment:os.arch=i386

14/01/21 01:18:03 INFO server.ZooKeeperServer: Server 
environment:os.version=3.2.0-51-generic

14/01/21 01:18:03 INFO server.ZooKeeperServer: Server 
environment:user.name=jenkins

14/01/21 01:18:03 INFO server.ZooKeeperServer: Server 
environment:user.home=/home/jenkins

14/01/21 01:18:03 INFO server.ZooKeeperServer: Server environment:user.dir=/tmp

14/01/21 01:18:03 INFO server.ZooKeeperServer: tickTime set to 3000

14/01/21 01:18:03 INFO server.ZooKeeperServer: minSessionTimeout set to -1

14/01/21 01:18:03 INFO server.ZooKeeperServer: maxSessionTimeout set to 120

14/01/21 01:18:03 INFO server.NIOServerCnxnFactory: binding to port 
0.0.0.0/0.0.0.0:2

14/01/21 01:18:03 INFO persistence.FileSnap: Reading snapshot 
/tmp/hama-testzookeeper/version-2/snapshot.262

14/01/21 01:18:03 INFO persistence.FileTxnSnapLog: Snapshotting: 0x267 to 
/tmp/hama-testzookeeper/version-2/snapshot.267

14/01/21 01:18:13 INFO zookeeper.ZooKeeper: Client 
environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT

14/01/21 01:18:13 INFO zookeeper.ZooKeeper: Client 
environment:host.name=vesta.apache.org

14/01/21 01:18:13 INFO zookeeper.ZooKeeper: Client 
environment:java.version=1.6.0_45

14/01/21 01:18:13 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Sun 
Microsystems Inc.

14/01/21 01:18:13 INFO zookeeper.ZooKeeper: Client 
environment:java.home=/home/hudson/tools/java/jdk1.6.0_45-32/jre

14/01/21 01:18:13 INFO zookeeper.ZooKeeper: Client 
environment:java.class.path=/tmp/surefirebooter5359103140981852488.jar

14/01/21 01:18:13 INFO zookeeper.ZooKeeper: Client 
environment:java.library.path=/home/hudson/tools/java/jdk1.6.0_45-32/jre/lib/i386/server:/home/hudson/tools/java/jdk1.6.0_45-32/jre/lib/i386:/home/hudson/tools/java/jdk1.6.0_45-32/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib

14/01/21 01:18:13 INFO zookeeper.ZooKeeper: Client 
environment:java.io.tmpdir=/tmp

14/01/21 01:18:13 INFO zookeeper.ZooKeeper: Client 
environment:java.compiler=

14/01/21 01:18:13 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux

14/01/21 01:18:13 INFO zookeeper.ZooKeeper: Client environment:os.arch=i386

14/01/21 01:18:13 INFO zookeeper.ZooKeeper: Client 
environment:os.version=3.2.0-51-generic

14/01/21 01:18:13 INFO zookeeper.ZooKeeper: Client environment:user.name=jenkins

14/01/21 01:18:13 INFO zookeeper.ZooKeeper: Client 
environment:user.home=/home/jenkins

14/01/21 01:18:13 INFO zookeeper.ZooKeeper: Client environment:user.dir=/tmp

14/01/21 01:18:13 INFO zookeeper.ZooKeeper: Initiating client connection, 
connectString=localhost:2 sessionTimeout=120 
watcher=org.apache.hama.bsp.sync.ZKSyncBSPMasterClient@859a68

14/01/21 01:18:13 INFO sync.ZKSyncBSPMasterClient: Initialized ZK false

14/01/21 01:18:13 INFO sync.ZKSyncClient: Initializing ZK Sync Client

14/01/21 01:18:13 INFO zookeeper.ClientCnxn: Opening socket connection to 
server localhost/0:0:0:0:0:0:0:1:2. Will not atte

Re: New logo on website

2014-01-20 Thread Anastasis Andronidis 
+1 

nice

On 20 Ιαν 2014, at 2:47 μ.μ., Yexi Jiang  wrote:

> The new logo seems really nice :)
> 
> It seems that the main color most of the projects in hadoop ecosystem is
> yellow, e.g. hadoop, mahout, hive, giraph, oozie, and the original hama
> logo. Only pig and hbase have different style.
> 
> In my humble opinion, the original logo has been well recognized by the
> community. I'm not sure whether it is good to replace the original one.
> 
> 
> 2014/1/20 Tommaso Teofili 
> 
>> +1
>> 
>> just remember we have to include also the ASF logo on the website, maybe on
>> the right.
>> 
>> Regards,
>> Tommaso
>> 
>> 
>> 2014/1/20 Chia-Hung Lin 
>> 
>>> + one hama
>>> 
>>> On 20 January 2014 20:18, Martin Illecker  wrote:
 Nice ;-)
 
 
 2014/1/20 Edward J. Yoon 
 
> Hi,
> 
> http://people.apache.org/~edwardyoon/site/index.html
> 
> Do you like this new logo?
> 
> If no objections arise, I'd like to commit this!
> 
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
> 
>>> 
>> 
> 
> 
> 
> -- 
> --
> Yexi Jiang,
> ECS 251,  yjian...@cs.fiu.edu
> School of Computer and Information Science,
> Florida International University
> Homepage: http://users.cis.fiu.edu/~yjian004/

Re: New logo on website

2014-01-20 Thread Yexi Jiang 
The new logo seems really nice :)

It seems that the main color most of the projects in hadoop ecosystem is
yellow, e.g. hadoop, mahout, hive, giraph, oozie, and the original hama
logo. Only pig and hbase have different style.

In my humble opinion, the original logo has been well recognized by the
community. I'm not sure whether it is good to replace the original one.


2014/1/20 Tommaso Teofili 

> +1
>
> just remember we have to include also the ASF logo on the website, maybe on
> the right.
>
> Regards,
> Tommaso
>
>
> 2014/1/20 Chia-Hung Lin 
>
> > + one hama
> >
> > On 20 January 2014 20:18, Martin Illecker  wrote:
> > > Nice ;-)
> > >
> > >
> > > 2014/1/20 Edward J. Yoon 
> > >
> > >> Hi,
> > >>
> > >> http://people.apache.org/~edwardyoon/site/index.html
> > >>
> > >> Do you like this new logo?
> > >>
> > >> If no objections arise, I'd like to commit this!
> > >>
> > >> --
> > >> Best Regards, Edward J. Yoon
> > >> @eddieyoon
> > >>
> >
>



-- 
--
Yexi Jiang,
ECS 251,  yjian...@cs.fiu.edu
School of Computer and Information Science,
Florida International University
Homepage: http://users.cis.fiu.edu/~yjian004/

Re: New logo on website

2014-01-20 Thread Tommaso Teofili 
+1

just remember we have to include also the ASF logo on the website, maybe on
the right.

Regards,
Tommaso


2014/1/20 Chia-Hung Lin 

> + one hama
>
> On 20 January 2014 20:18, Martin Illecker  wrote:
> > Nice ;-)
> >
> >
> > 2014/1/20 Edward J. Yoon 
> >
> >> Hi,
> >>
> >> http://people.apache.org/~edwardyoon/site/index.html
> >>
> >> Do you like this new logo?
> >>
> >> If no objections arise, I'd like to commit this!
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
>

[jira] [Commented] (HAMA-854) BSP peers need messaging mechanism.

2014-01-20 Thread ChiaHung Lin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HAMA-854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13876420#comment-13876420
 ] 

ChiaHung Lin commented on HAMA-854:
---

That looks nice by removing native dependency. 

> BSP peers need messaging mechanism. 
> 
>
> Key: HAMA-854
> URL: https://issues.apache.org/jira/browse/HAMA-854
> Project: Hama
>  Issue Type: Improvement
>  Components: bsp core
>Reporter: ChiaHung Lin
>Priority: Minor
>
> This may be a mid/ long term solution. During communication, BSP peers will 
> exchange messages, it might be good to have a layer such as 0mq or [1], which 
> removes native dep, helping communication between peers. 
> See also HAMA-853
> [1]. Making Storm fly with Netty. 
> http://yahooeng.tumblr.com/post/64758709722/making-storm-fly-with-netty



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (HAMA-854) BSP peers need messaging mechanism.

2014-01-20 Thread ChiaHung Lin (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HAMA-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ChiaHung Lin updated HAMA-854:
--

Description: 
This may be a mid/ long term solution. During communication, BSP peers will 
exchange messages, it might be good to have a layer such as 0mq or [1], which 
removes native dep, helping communication between peers. 

See also HAMA-853
[1]. Making Storm fly with Netty. 
http://yahooeng.tumblr.com/post/64758709722/making-storm-fly-with-netty

  was:
This may be a mid/ long term solution. During communication, BSP peers will 
exchange messages, it might be good to have a layer such as 0mq helping 
communication between peers. 

See also HAMA-853


> BSP peers need messaging mechanism. 
> 
>
> Key: HAMA-854
> URL: https://issues.apache.org/jira/browse/HAMA-854
> Project: Hama
>  Issue Type: Improvement
>  Components: bsp core
>Reporter: ChiaHung Lin
>Priority: Minor
>
> This may be a mid/ long term solution. During communication, BSP peers will 
> exchange messages, it might be good to have a layer such as 0mq or [1], which 
> removes native dep, helping communication between peers. 
> See also HAMA-853
> [1]. Making Storm fly with Netty. 
> http://yahooeng.tumblr.com/post/64758709722/making-storm-fly-with-netty



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Re: New logo on website

2014-01-20 Thread Chia-Hung Lin 
+ one hama

On 20 January 2014 20:18, Martin Illecker  wrote:
> Nice ;-)
>
>
> 2014/1/20 Edward J. Yoon 
>
>> Hi,
>>
>> http://people.apache.org/~edwardyoon/site/index.html
>>
>> Do you like this new logo?
>>
>> If no objections arise, I'd like to commit this!
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>

Re: New logo on website

2014-01-20 Thread Martin Illecker 
Nice ;-)


2014/1/20 Edward J. Yoon 

> Hi,
>
> http://people.apache.org/~edwardyoon/site/index.html
>
> Do you like this new logo?
>
> If no objections arise, I'd like to commit this!
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>

New logo on website

2014-01-20 Thread Edward J. Yoon 
Hi,

http://people.apache.org/~edwardyoon/site/index.html

Do you like this new logo?

If no objections arise, I'd like to commit this!

-- 
Best Regards, Edward J. Yoon
@eddieyoon