+1 for hama-io or hama-commons But we should discuss the parts that can be combined in a common module.
I would suggest the following: *hama-commons module* -- io -- CombineFileInputFormat.java CombineFileSplit.java FileInputFormat.java FileOutputFormat.java FileSplit.java HashPartitioner.java InputFormat.java InputSplit.java LineRecordReader.java Messagable.java MonitorManager.java NullInputFormat.java NullOutputFormat.java OutputCollector.java OutputFormat.java Partitioner.java PartitioningRunner.java RecordReader.java RecordWriter.java SequenceFileInputFormat.java SequenceFileOutputFormat.java SequenceFileRecordReader.java SequenceFileRecordWriter.java TrackedRecordReader.java TextInputFormat.java TextOutputFormat.java -- io.writable -- MatrixWritable.java TextArrayWritable.java VectorWritable.java -- math -- CrossEntropy.java DenseDoubleMatrix.java DenseDoubleVector.java DoubleDoubleFunction.java DoubleDoubleVectorFunction.java DoubleFunction.java DoubleMatrix.java DoubleVector.java DoubleVectorFunction.java Function.java FunctionFactory.java IdentityFunction.java Sigmoid.java SquaredError.java Tanh.java Tuple.java *hama-core module submitted by Suraj* -- bsp -- BSP.java BSPInterface.java BSPJob.java BSPJobClient.java BSPJobContext.java BSPJobID.java BSPPeer.java Counters.java Combiner.java ID.java LocalBSPRunner.java -- bps.master -- BSPMaster.java -- bsp.peer.ipc -- -- bsp.peer.ipc.message -- BSPMessageBundle.java -- bsp.peer -- BSPPeerImpl.java -- bsp.task -- BSPTask.java BSPTaskRunner.java Task.java TaskAttemptContext.java TaskAttemptID.java TaskCompletionEvent.java TaskID.java TaskInProgress.java TaskLog.java TaskLogAppender.java TaskLogServlet.java TaskRunner.java TaskStatus.java -- bsp.ipc -- Directive.java DirectiveException.java DirectiveHandler.java DispatchTasksDirective.java -- bsp.ipc.action -- CommitTaskAction.java GroomServerAction.java JobChangeEvent.java KillJobAction.java KillTaskAction.java LaunchTaskAction.java RecoverTaskAction.java ReinitGroomAction.java ReportGroomStatusDirective.java UpdatePeerAction.java ClusterStatus.java -- bsp.groom -- GroomServer.java GroomServerManager.java GroomServerStatus.java -- bsp.job -- JobInProgress.java JobInProgressListener.java JobProfile.java JobStatus.java RunningJob.java -- bsp.scheduler -- Schedulable.java SimpleTaskScheduler.java TaskScheduler.java FCFSQueue.java Queue.java QueueManager.java -- bsp.superstep -- Superstep.java SuperstepBSP.java Thanks! 2013/10/21 Chia-Hung Lin <[email protected]> > +1 for hama-io or hama-commons > > > On 21 October 2013 21:35, Tommaso Teofili <[email protected]> > wrote: > > what about creating a module for that (Writables and InputFormats for > now) > > hama-io / hama-commons that can be used by both (containing math stuff as > > well) ? > > > > Tommaso > > > > > > 2013/10/21 Martin Illecker <[email protected]> > > > >> VectorWritable and MatrixWritable have both some dependencies > >> to org.apache.hama.ml.math. > >> (DenseDoubleVector, DoubleVector and DenseDoubleMatrix, DoubleMatrix) > >> > >> If we move VectorWritable and MatrixWritable to core (e.g., > >> org.apache.hama.io.writable), we have to move org.apache.hama.ml.math as > >> well. > >> I think that's not possible because of other classes in hama-ml > depending > >> on ml.math. > >> > >> Temporary I will have to copy VectorWritable to the core to use it in a > >> test case. > >> > >> > >> 2013/10/21 Tommaso Teofili <[email protected]> > >> > >> > 2013/10/21 Martin Illecker <[email protected]> > >> > > >> > > Hello, > >> > > > >> > > regarding to my Hama Pipes test case [1], I want to use > VectorWritable > >> > > inside the hama-core module. > >> > > Therefore I would need a dependency to hama-ml but this will cause a > >> > cyclic > >> > > dependency. > >> > > > >> > > So is it possible to move both writables, VectorWritable and > >> > > MatrixWritable, from org.apache.hama.ml.writable into a new package? > >> > > e.g., org.apache.hama.io.writable based on [2] > >> > > > >> > > >> > I think this really makes sense. > >> > > >> > > >> > > > >> > > Regarding to [3] we can also move TextArrayWritable > >> > > from org.apache.hama.bsp into this new package. > >> > > > >> > > Do you think we can move the writables of > org.apache.hama.ml.writable > >> to > >> > > the core module? > >> > > > >> > > >> > +1 > >> > > >> > > >> > > And can we do the package refactoring [2] of org.apache.hama.bsp > >> > submitted > >> > > by Suraj? > >> > > > >> > > >> > +1 here too. > >> > > >> > Tommaso > >> > > >> > > >> > > > >> > > Thanks! > >> > > > >> > > Martin > >> > > > >> > > [1] https://issues.apache.org/jira/browse/HAMA-808 > >> > > [2] > >> > https://issues.apache.org/jira/secure/attachment/12609417/bsplist.txt > >> > > [3] https://issues.apache.org/jira/browse/HAMA-727 > >> > > > >> > > >> >
