Just a thought, have you looked into anything that the Jabber Software
Foundation has done as far as both synchronization and data transfer?
On 4/19/06, Krishna <[EMAIL PROTECTED]> wrote:
>
> Hi Ansink,
>
> Yes, this is a known problem with Alchemi.
>
> The issue is, currently Alchemi uses only remoting to transfer the files
> as method "arguments". This causes these files to be in memory, and
> eventually there are scalability problems.
> We are trying to fix the problem by having a seperate data transfer
> service, so that large-ish input files dont cause a problem.
>
> Till then, I guess this is known "bug".
>
> Regards
> Krishna.
>
> >Krishna,
> >
> >I have a problem using Alchemi and i try to explain my problem. Look at the
> >following code
> >
> > GConnection gc = GConnection.FromConsole(hostname, "9000", "admin",
> > "admin");
> >
> > // create a new grid application
> > _gridApplicatie = new GApplication(gc);
> >
> > // subcribe to events
> > _gridApplicatie.ThreadFinish += new GThreadFinish(JobFinished);
> > _gridApplicatie.ThreadFailed += new GThreadFailed(JobFailed);
> >
> > _gridApplicatie.ApplicationFinish += new
> > GApplicationFinish(ApplicationFinished);
> >
> > Initialize(programProfile);
> >
> > // Create jobs
> > if (Directory.Exists(_map))
> > {
> > string [] calcs = Directory.GetFiles(_map);
> >
> > // Loop over the jobs
> > for (int jobNr = 0; jobNr < calcs.Length; jobNr++)
> > {
> > // Create a new Alchemi job
> > GJob alchemiJob = new GJob();
> >
> > // Add Input files to the alchemiJob
> > for (int i = 0; i < _inputFiles.Length; i++)
> > {
> > alchemiJob.InputFiles.Add(new
> > EmbeddedFileDependency(Path.GetFileName(_inputFiles[i]), _inputFiles[i]));
> > }
> >
> > // Add Output files to the alchemiJob
> > for (int i = 0; i < _outputFiles.Length; i++)
> > {
> > alchemiJob.OutputFiles.Add(new
> > EmbeddedFileDependency(_outputFiles[i]));
> > }
> >
> > // Run command
> > alchemiJob.RunCommand = string.Format("{0} {1} ",
> > Path.GetFileName(_executable),
> > Path.GetFileName(programProfile.Name));
> >
> > // Add job
> > _gridApplicatie.Threads.Add(alchemiJob);
> > }
> >
> > // start the grid application
> > _gridApplicatie.Start();
> >
> > Console.WriteLine("\nStarted Sobek grid application .. Waiting for
> > jobs to finish ..\n");
> >
> >
> >The input files that i add for my job as a embeddedfiledependency is about
> >20 mb of data.
> >So when i want to run a calculation of more than a dozen of jobs the memory
> >allocated is too much, because all of the files are in memory until the
> >start of the grid application.
> >Is this a known problem or am i doing someting wrong. I can run my grid
> >application for only +- 12 jobs fine. Is it possible to read & encrypt the
> >input files at a later moment??
> >
> >Johan Ansink
> >senior programmer
> >HKV consultants.
> >
> >
> >
> >
>
>
>
> -------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> alchemi-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/alchemi-users
>
-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid0709&bid&3057&dat1642
_______________________________________________
alchemi-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/alchemi-users
