reductionista commented on issue #401: DL: fix default_buffer_size calculation URL: https://github.com/apache/madlib/pull/401#issuecomment-497474308 I just want to comment that we should make sure this is tested for 16-bit and 8-bit data types. If the logic is just based on the size being < 1GB then it will crash whenever there are more than 134217728 elements in the batched array. (134217728 is the hardcoded maximum number of elements that can be in any array for all of postgres/greenplum). For 32-bit data types, that ends up being the same as the < 1GB requirement, but for 16-bit it's < 512MB, and for 8-bit it's < 256MB. I don't think we can close this JIRA (https://issues.apache.org/jira/browse/MADLIB-1340) until we're sure both of these issues are handled properly.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
