On 10/07/2012 10:38 AM, Kirk, Benjamin (JSC-EG311) wrote:
> Just a couple other questions:
>
> (1) th coefficients are distributed in the parallel case, right? So they do
> need to be gathered for serialization?
Yeah, we just get the coefficients by doing a solve and then copying
from system.solution to another NumericVector to generate the
std::vector< NumericVector<Number>* > that I mentioned earlier.
> If this is the case and they are all distributed then it'll be easy to
> reduce the number of mpi calls from nproc*nvec to just nproc, already a huge
> win,
OK, it sounds like that'll work, since the NumericVectors in the
std::vector< NumericVector<Number>* > are indeed all distributed.
> (2) what are realistic vector sizes, number of vectors for real applications,
> and
For real applications, we'd typically have vector sizes between 10,000
and 500,000, though the upper end may get larger soon. Also, the number
of vectors would be in the range 20 to 500, but again the upper end
could get larger.
> (3) which would be the best example to test this with?
reduced_basis_ex1 provides a simple test case, though it's very small
scale --- though presumably that's helpful for testing. Also, to crank
up the mesh size you can just change n_elem in reduced_basis_ex1.in.
The Makefile runs the executable twice, first in the Offline mode and
then in the Online mode.
In the Offline mode ("-online_mode 0"), the call
rb_con.get_rb_evaluation().write_out_basis_functions(rb_con); writes out
20 .xdr into the offline_data directory.
In the Online stage those .xdr files are read in by the call
rb_eval.read_in_basis_functions(rb_con);
David
On Oct 7, 2012, at 8:36 AM, "David Knezevic"
<dkneze...@seas.harvard.edu> wrote:
>> That'd be awesome, thanks!
>>
>>
>>
>> On 10/07/2012 09:35 AM, Kirk, Benjamin (JSC-EG311) wrote:
>>> Well if they truly have the same size it should be really easy - Ill have
>>> some time on a flight Tuesday if you want me to take a shot at it.
>>>
>>> -Ben
>>>
>>>
>>>
>>> On Oct 7, 2012, at 8:11 AM, "David Knezevic" <dkneze...@seas.harvard.edu>
>>> wrote:
>>>
>>>> On 10/07/2012 09:05 AM, Kirk, Benjamin (JSC-EG311) wrote:
>>>>> The file format os secondary at this point - Xdr is already there and
>>>>> gets you platform independent binary files, (as does netcdf for that
>>>>> matter). HDF5 can do the same, and there are other good reasons to
>>>>> support linking to it, but I don't want to get the cart before the
>>>>> horse...
>>>>>
>>>>> Probably the place to start the discussion is capabilities - will the
>>>>> vectors all have the same metadata? Size, partitioning, etc... just
>>>>> different data? Or are they all different sizes?
>>>> Yep, they all have the same metadata. Each vector is the coefficients of
>>>> a solution based on the same mesh/approximation space.
>>>>
>>>>
>>>>> Also, upon read, do you immediately need access to all the vectors, or is
>>>>> it a more random access database type pattern?
>>>> I would say we need immediate access to all vectors. We currently store
>>>> the data that is read in in a std::vector< NumericVector<Number>* > and
>>>> we need to be able to access any of those NumericVectors immediately
>>>> after the read.
>>>>
>>>>
>>>>> If we get the desired behavior defined the implementation will be pretty
>>>>> obvious I think. So I'm asking this not just to understand how you do
>>>>> things now with the current setup but more how would you like things to
>>>>> work if we are going to refactor this.
>>>> Sounds good.
>>>>
>>>>
>>>> David
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Oct 6, 2012, at 8:51 PM, "David Knezevic"
>>>> <dkneze...@seas.harvard.edu> wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> I wanted to pick up on this thread again. I gather that a change was
>>>>>> made to the block size in read_serialized_data that led to a significant
>>>>>> speedup for the use case that Jens was looking at.
>>>>>>
>>>>>> But this is still a bottleneck for a lot of the reduced basis code, so
>>>>>> I'd like to improve it further. Ben wrote:
>>>>>>
>>>>>>
>>>>>> On 08/17/2012 02:13 PM, Kirk, Benjamin (JSC-EG311) wrote:
>>>>>>> But beyond that there is likely a lot of libMesh overhead too - string
>>>>>>> comparisons, memory allocation, etc... I'm wondering if you need a
>>>>>>> specialized I/O implementation where all the vectors are strung
>>>>>>> together consecutively and then streamed to/from disk as one big
>>>>>>> super-vector? That will be much, much faster...
>>>>>> I agree, I think we do need a specialized I/O implementation that
>>>>>> concatenates vectors. The key parts of the RB code related to this are
>>>>>> at line 873 and 940 of rb_evaluation.C. We currently write (or read) a
>>>>>> bunch of vectors one after the other, but it'd obviously be much better
>>>>>> to string these all together into one big file.
>>>>>>
>>>>>> Any thoughts on an API?
>>>>>>
>>>>>> System::write_list_of_vectors(Xdr& io, std::vector<
>>>>>> NumericVector<Number>* > list_of_vectors)
>>>>>>
>>>>>> and
>>>>>>
>>>>>> System::read_list_of_vectors(Xdr& io, std::vector<
>>>>>> NumericVector<Number>* > list_of_vectors)
>>>>>>
>>>>>> ?
>>>>>>
>>>>>> Also, any thoughts on how to get started on the implementation would be
>>>>>> very helpful. I guess a lot would carry over from
>>>>>> write_serialized_vector/read_serialized_vector.
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Don't let slow site performance ruin your business. Deploy New Relic APM
>>>>>> Deploy New Relic app performance management and know exactly
>>>>>> what is happening inside your Ruby, Python, PHP, Java, and .NET app
>>>>>> Try New Relic at no cost today and get our sweet Data Nerd shirt too!
>>>>>> http://p.sf.net/sfu/newrelic-dev2dev
>>>>>> _______________________________________________
>>>>>> Libmesh-devel mailing list
>>>>>> Libmesh-devel@lists.sourceforge.net
>>>>>> https://lists.sourceforge.net/lists/listinfo/libmesh-devel
------------------------------------------------------------------------------
Don't let slow site performance ruin your business. Deploy New Relic APM
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
http://p.sf.net/sfu/newrelic-dev2dev
_______________________________________________
Libmesh-devel mailing list
Libmesh-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libmesh-devel
