Atri, thanks for the note on LOs. http://www.postgresql.org/docs/current/static/lo-intro.html mentions LO facility supporting values up to 4TB in size. Internal aggregate states much larger than 1 GB are very attractive to MADlib.
Do you have a code example where a LO is used as an internal aggregate state? That would give use an idea of how to implement. Our main questions are: 1) Performance * Does use of the LO facility result in acceptable performance? Related to Caleb's question on memory management. * If use of LO means disk I/O operations where in the past we used in-memory operations, then performance will degrade. 2) Ease of implementation * Need to learn more about this one 3) GPBD and HAWQ * Ideally we would want both to have the LO capability, but if it is just GPDB initially, we could put the equivalent if #ifdef's in the code. Rgds, Frank On Wed, Dec 23, 2015 at 1:59 PM, Roman Shaposhnik <[email protected]> wrote: > On Wed, Dec 23, 2015 at 1:49 PM, Ivan Novick <[email protected]> wrote: > > Its currently functioning on PostgreSQL so maybe thats the place to try > it > > first before worry about porting to GPDB and HAWQ which should be doable. > > That's a great point! > > Thanks, > Roman. >
