Re. something like C#’s LINQ, I think we will see a massive transformation of how people write Julia code once 0.5 arrives, since then we will have fast anonymous functions. This in turn will make functional-style idioms like collection pipelines <http://martinfowler.com/articles/collection-pipeline/> (an example of which are LINQ’s method syntax) much more common in various Julia code bases. For example, I think usage of the chaining operator |> will explode, compared to today.
When this happens, I think there will be numerous efforts to provide various utility methods that do similar things to e.g. C# LINQ extension methods. For example, someone will implement take(2) to return a function which, when applied in a method chain, returns the two first elements of a collection (perhaps even lazily). (This might even already exist, but the performance implications of using anonymous functions has so far hindered its widespread use.) Standard functions like map, collect etc will be extended to new data types, and new (or existing but little-used) functions will crop up to fill in the gaps in order to create this high-level API you are longing for. This will in turn make it easier for implementors of database adapters etc to hook into the emerging framework - and data consumers will have an API to work with that will (at least mostly) work regardless of whether the underlying data source is a DataFrame or a connection to an SQL database. As you might be able to tell, I think the next Julia release is going to be really awesome… // T On Tuesday, February 23, 2016 at 2:42:10 PM UTC+1, Milan Bouchet-Valat wrote: Le dimanche 21 février 2016 à 13:10 -0800, Shahbaz Chaudhary a écrit : > > I'm pretty new to Julia and only have marginally more experience with > > R so please excuse me if I don't understand something basic. > > > > According to Julia's website, the final api/syntax for manipulating > > data has not been finalized yet, although the momentum seems to be > > moving towards a dataframe style api. > I'm not sure what you base your impression on, but I'm not sure it's an > accurate description (see below). Anyway, two (o more) different APIs > can perfectly coexist. > > > Since Julia is still a new language, doesn't it make sense to base > > the model on something closer to the relational algebra/sql/list > > comprehensions? I realize these three are not synonyms for each > > other, but relational algebra is supposed to have a more rigorous > > mathematical foundation in building the primitives used to manipulate > > data. SQL now has decades of use and has unarguably democratized data > > manipulation (I've seen lawyers and traders use sql, who would never > > use a full blown programming language). > > > > At least R's dataframe feel extremely clunky, although I'll admit > > that I may be missing something fundamental since Julia/Spark/Pandas > > seem to be adopting this model instead of the relational model. > > > > A language, built from the ground up to process datasets should have > > a more intuitive syntax. > > > > One potential issue is that relational algebra/sql don't handle > > ordered data well. I don't know enough about recent advances but > > surely an extension to relational primitives is more sound than > > adapting dataframes. > > > > Frankly I'm curious to learn from the more experienced here what I'm > > not understanding about dataframes and why they are so popular. > I think your feeling about a high-level API inspired by SQL is shared > by many of the developers involved with data frames and data analysis > in Julia. DataFrames can be one of the multiple backends which could be > manipulated using a common API similar to C#'s LINQ or to R's dplyr. > Things are still not settled, but you can have a look at this package: > https://github.com/JuliaStats/DataFramesMeta.jl > > Also see this thread: > https://groups.google.com/d/msg/julia-dev/jL2FSL4EneE/c8vW8pdEqjYJ > > > Help is of course more than welcome in the area of database support! > > Regards >
