Massive matrix pattern?

I have a matrix that is too large for my current machines to load directly into memory. Is there a preferred patten in the Chapel community for something like incremental loads + distribution? Spark uses the RDD notion to persist to disk strategically. Does Chapel support a pattern like

Serialization and data storage formats

Can someone share a list of currently supported serialization formats, like HDF5 (I know not current), Parquet, etc. Second, we data storage is supported. I see CSV and HDFS, but no SQL or others. Thanks! b ~ May All Your Sequences Converge

Re: Serialization and data storage formats

Thanks Michael, That helped me find this page: http://chapel.cray.com/docs/master/modules/standard/IO.html#i-o-overview (Google is still not that friendly to Chapel, getting better) In this example: var myFile =

Re: Serialization and data storage formats

Oh, yeah! I loved this example. It helped inspire this issue: https://github.com/chapel-lang/chapel/issues/6840 I looked right past the JSON part... We are extensively using graph structures and analytics, so I’d be pretty excited to see a

Re: Starting Gitter for discussion

I agree that IRC is harder to use and does not retain history. I think Slack might be a better choice than Gitter, but both are better than IRC. To be honest, I don’t know if any project with an active Slack or Gitter. ~ May All Your Sequences Converge > On Jan 21, 2018, at 11:16 PM,

Re: Starting Gitter for discussion

This article convinced me away from Slack and towards Gitter. We love Slack at Deep 6, but it really is “entreprisey” https://www.quora.com/What-are-the-pros-and-cons-of-Gitter-vs-Slack ~ May All Your Sequences

Re: On Nomenclature

FWIW we are holding off on Mason until it can handle external dependencies. We are big fans of Ben’s LinearAlgebra package and Marcos’ CDO package, they require LBLAS and Postgres, respectively. Right now, Mason cannot build with those requirements. Instead, I have been using Make. An

Re: Mason and testing

The pressure is on! I’ll add some docs & examples today. Feel free to open issues in Github if you like. ~ May All Your Sequences Converge > On Mar 22, 2018, at 9:45 AM, Russel Winder wrote: > > On Thu, 2018-03-22 at 14:13 +, Ben Albrecht wrote: >> Hi Russel,