Re: [julia-users] Re: Re: are array slices views in 0.4?
ads-up that this change is breaking for them >>> long before 0.5.0 is released. >>> >>> Given that these changes potentially allow quite extreme performance >>> gains in many applications, I find it unlikely that package devs won’t want >>> to spend some time fixing any bugs this behavior change introduces, but for >>> those that just want to pin their package down before fixing it, they can >>> say julia 0.4 0.4 or similar in their REQUIRE.jl, and not even allow >>> installation on 0.5 (until they fix the problems, change the REQUIRE.jl and >>> tag a new version). Note that many (most?) packages won’t even need this - >>> package testing on the master branch will reveal to devs whether a package >>> even needs touch-up or not. >>> >>> I used from __future__ import print at the top of my python scripts for >>> a long time, so I’m entirely in favor of something like that. However, I >>> guess the behavior needs to be implemented and done in the master branch of >>> Julia before we know exactly how to mimic it in a package (if it’s even >>> possible). >>> >>> // T >>> >>> On Tuesday, November 3, 2015 at 7:41:17 AM UTC+1, Christoph Ortner wrote: >>> >>> Why not do this via Compat.jl ? >>>> C >>>> >>>> On Tuesday, 3 November 2015 04:28:44 UTC, David Anthoff wrote: >>>>> >>>>> No, Compat generall ports the new syntax back to old versions. What >>>>> I'm suggesting is quite different, namely that in julia 0.5 if you just >>>>> use >>>>> `[]` for slicing without anything else, you generate a warning, and then >>>>> you have to opt-in to the NEW syntax. >>>>> >>>>> > -Original Message- >>>>> > From: julia...@googlegroups.com [mailto:julia- >>>>> > us...@googlegroups.com] On Behalf Of Eric Forgy >>>>> > Sent: Monday, November 2, 2015 3:30 PM >>>>> > To: julia-users >>>>> > Subject: RE: [julia-users] Re: Re: are array slices views in 0.4? >>>>> > >>>>> > Hi David, >>>>> > >>>>> > I'm not an expert, i.e. I've never used it, but your idea "using >>>>> OldArrays" >>>>> > sounds exactly like the purpose of Compat.jl, which I believe is the >>>>> > compatibility module referred to. >>>>> > >>>>> > Have a look and good luck: >>>>> > >>>>> > https://github.com/JuliaLang/Compat.jl >>>>> >>>> >>> >> >
Re: [julia-users] Re: Re: are array slices views in 0.4?
My point is that what we *really* want here is the ability to test the new behavior while the old behavior is still the released version, or to opt-out of the change altogether - all the suggestions in this thread about how to handle this transition are trying to solve that problem. And solutions already exists - to pre-test, just test the package against the Julia master as soon as the change merges (and this will be done by Pkg evaluator, semi-automatically filing issues in case errors occur), and to opt-out, just pin your code to depend on julia 0.4. The way I see it, the need for soft transitioning is not as high in Julia as it is in Python - Julia is a fast-moving target, and currently in pre-release (think of all the 0.x releases as alpha releases of the real language, if you will). IMO, it’s reasonable to expect breaking changes now and then, even major ones, even ones that may lead to subtle and hard-to-find bugs, and it’s reasonable to expect users and package developers to either put in some extra effort to keep code up-to-date with the latest and greatest, or pin down dependencies to a version one knows works (no-one will force you to upgrade to 0.5, if you think 0.4 works better for you…). Having to put that effort in is the price you pay for using a system that hasn't reached 1.0 status yet. At least to me, Julia is awesome enough to make it more than worth it :) There were a couple of big breakages during the 0.4 cycle (this slide in Iain Dunning’s talk from JuliaCon 2015 <https://youtu.be/pYYmnBma9qk?list=PLP8iPy9hna6Sdx4soiGrSefrmOPdUWixM&t=417> gives a good overview of the big ones) and the community were quite quick to fix the major problems that occurred then, so that now when 0.4 was released, the actual breakage from those changes was relatively minor. As I see it, this transitioning model is good enough for Julia at the moment - if/when we want to make breaking changes between 1.x and 2.0, that’s a different story. // T On Tuesday, November 3, 2015 at 9:28:43 AM UTC+1, Kristoffer Carlsson wrote: I don't see how this enables "extreme performance gains". As have been > said, you can already now use slice/sub in base so the only difference is > if a slice is default or not for the [:] syntax. I do realize that syntax > matters and no one is going to write slice(A, :, 1) everywhere instead of > A[:,1]. > > However, I am a bit wary about it because the vast majority of the hard to > track down bugs I've had has been due to some unexpected shared state. > Wouldn't it be best if you could opt in to use the [:] syntax as slices > somehow and have the default stay as it is? > > > On Tuesday, November 3, 2015 at 8:30:04 AM UTC+1, Tomas Lycken wrote: >> >> There’s no way this transition is going to be smooth without work from >> package devs to ensure compatibility anyway, but recent history (e.g. the >> Tuplocalypse) shows that packages recover quite rapidly due to the >> automated testing done by PkgEval.jl. Thus, packages with decent test >> suites will probably get a heads-up that this change is breaking for them >> long before 0.5.0 is released. >> >> Given that these changes potentially allow quite extreme performance >> gains in many applications, I find it unlikely that package devs won’t want >> to spend some time fixing any bugs this behavior change introduces, but for >> those that just want to pin their package down before fixing it, they can >> say julia 0.4 0.4 or similar in their REQUIRE.jl, and not even allow >> installation on 0.5 (until they fix the problems, change the REQUIRE.jl and >> tag a new version). Note that many (most?) packages won’t even need this - >> package testing on the master branch will reveal to devs whether a package >> even needs touch-up or not. >> >> I used from __future__ import print at the top of my python scripts for >> a long time, so I’m entirely in favor of something like that. However, I >> guess the behavior needs to be implemented and done in the master branch of >> Julia before we know exactly how to mimic it in a package (if it’s even >> possible). >> >> // T >> >> On Tuesday, November 3, 2015 at 7:41:17 AM UTC+1, Christoph Ortner wrote: >> >> Why not do this via Compat.jl ? >>> C >>> >>> On Tuesday, 3 November 2015 04:28:44 UTC, David Anthoff wrote: >>>> >>>> No, Compat generall ports the new syntax back to old versions. What I'm >>>> suggesting is quite different, namely that in julia 0.5 if you just use >>>> `[]` for slicing without anything else, you generate a warning, and then >>>> you have to opt-in to the NEW syntax. >>>> &
Re: [julia-users] Re: Re: are array slices views in 0.4?
I don't see how this enables "extreme performance gains". As have been said, you can already now use slice/sub in base so the only difference is if a slice is default or not for the [:] syntax. I do realize that syntax matters and no one is going to write slice(A, :, 1) everywhere instead of A[:,1]. However, I am a bit wary about it because the vast majority of the hard to track down bugs I've had has been due to some unexpected shared state. Wouldn't it be best if you could opt in to use the [:] syntax as slices somehow and have the default stay as it is? On Tuesday, November 3, 2015 at 8:30:04 AM UTC+1, Tomas Lycken wrote: > > There’s no way this transition is going to be smooth without work from > package devs to ensure compatibility anyway, but recent history (e.g. the > Tuplocalypse) shows that packages recover quite rapidly due to the > automated testing done by PkgEval.jl. Thus, packages with decent test > suites will probably get a heads-up that this change is breaking for them > long before 0.5.0 is released. > > Given that these changes potentially allow quite extreme performance gains > in many applications, I find it unlikely that package devs won’t want to > spend some time fixing any bugs this behavior change introduces, but for > those that just want to pin their package down before fixing it, they can > say julia 0.4 0.4 or similar in their REQUIRE.jl, and not even allow > installation on 0.5 (until they fix the problems, change the REQUIRE.jl and > tag a new version). Note that many (most?) packages won’t even need this - > package testing on the master branch will reveal to devs whether a package > even needs touch-up or not. > > I used from __future__ import print at the top of my python scripts for a > long time, so I’m entirely in favor of something like that. However, I > guess the behavior needs to be implemented and done in the master branch of > Julia before we know exactly how to mimic it in a package (if it’s even > possible). > > // T > > On Tuesday, November 3, 2015 at 7:41:17 AM UTC+1, Christoph Ortner wrote: > > Why not do this via Compat.jl ? >> C >> >> On Tuesday, 3 November 2015 04:28:44 UTC, David Anthoff wrote: >>> >>> No, Compat generall ports the new syntax back to old versions. What I'm >>> suggesting is quite different, namely that in julia 0.5 if you just use >>> `[]` for slicing without anything else, you generate a warning, and then >>> you have to opt-in to the NEW syntax. >>> >>> > -Original Message- >>> > From: julia...@googlegroups.com [mailto:julia- >>> > us...@googlegroups.com] On Behalf Of Eric Forgy >>> > Sent: Monday, November 2, 2015 3:30 PM >>> > To: julia-users >>> > Subject: RE: [julia-users] Re: Re: are array slices views in 0.4? >>> > >>> > Hi David, >>> > >>> > I'm not an expert, i.e. I've never used it, but your idea "using >>> OldArrays" >>> > sounds exactly like the purpose of Compat.jl, which I believe is the >>> > compatibility module referred to. >>> > >>> > Have a look and good luck: >>> > >>> > https://github.com/JuliaLang/Compat.jl >>> >> >
Re: [julia-users] Re: Re: are array slices views in 0.4?
There’s no way this transition is going to be smooth without work from package devs to ensure compatibility anyway, but recent history (e.g. the Tuplocalypse) shows that packages recover quite rapidly due to the automated testing done by PkgEval.jl. Thus, packages with decent test suites will probably get a heads-up that this change is breaking for them long before 0.5.0 is released. Given that these changes potentially allow quite extreme performance gains in many applications, I find it unlikely that package devs won’t want to spend some time fixing any bugs this behavior change introduces, but for those that just want to pin their package down before fixing it, they can say julia 0.4 0.4 or similar in their REQUIRE.jl, and not even allow installation on 0.5 (until they fix the problems, change the REQUIRE.jl and tag a new version). Note that many (most?) packages won’t even need this - package testing on the master branch will reveal to devs whether a package even needs touch-up or not. I used from __future__ import print at the top of my python scripts for a long time, so I’m entirely in favor of something like that. However, I guess the behavior needs to be implemented and done in the master branch of Julia before we know exactly how to mimic it in a package (if it’s even possible). // T On Tuesday, November 3, 2015 at 7:41:17 AM UTC+1, Christoph Ortner wrote: Why not do this via Compat.jl ? > C > > On Tuesday, 3 November 2015 04:28:44 UTC, David Anthoff wrote: >> >> No, Compat generall ports the new syntax back to old versions. What I'm >> suggesting is quite different, namely that in julia 0.5 if you just use >> `[]` for slicing without anything else, you generate a warning, and then >> you have to opt-in to the NEW syntax. >> >> > -Original Message- >> > From: julia...@googlegroups.com [mailto:julia- >> > us...@googlegroups.com] On Behalf Of Eric Forgy >> > Sent: Monday, November 2, 2015 3:30 PM >> > To: julia-users >> > Subject: RE: [julia-users] Re: Re: are array slices views in 0.4? >> > >> > Hi David, >> > >> > I'm not an expert, i.e. I've never used it, but your idea "using >> OldArrays" >> > sounds exactly like the purpose of Compat.jl, which I believe is the >> > compatibility module referred to. >> > >> > Have a look and good luck: >> > >> > https://github.com/JuliaLang/Compat.jl >> >
Re: [julia-users] Re: Re: are array slices views in 0.4?
Why not do this via Compat.jl ? C On Tuesday, 3 November 2015 04:28:44 UTC, David Anthoff wrote: > > No, Compat generall ports the new syntax back to old versions. What I'm > suggesting is quite different, namely that in julia 0.5 if you just use > `[]` for slicing without anything else, you generate a warning, and then > you have to opt-in to the NEW syntax. > > > -Original Message- > > From: julia...@googlegroups.com [mailto:julia- > > > us...@googlegroups.com ] On Behalf Of Eric Forgy > > Sent: Monday, November 2, 2015 3:30 PM > > To: julia-users > > > Subject: RE: [julia-users] Re: Re: are array slices views in 0.4? > > > > Hi David, > > > > I'm not an expert, i.e. I've never used it, but your idea "using > OldArrays" > > sounds exactly like the purpose of Compat.jl, which I believe is the > > compatibility module referred to. > > > > Have a look and good luck: > > > > https://github.com/JuliaLang/Compat.jl >
RE: [julia-users] Re: Re: are array slices views in 0.4?
No, Compat generall ports the new syntax back to old versions. What I'm suggesting is quite different, namely that in julia 0.5 if you just use `[]` for slicing without anything else, you generate a warning, and then you have to opt-in to the NEW syntax. > -Original Message- > From: julia-users@googlegroups.com [mailto:julia- > us...@googlegroups.com] On Behalf Of Eric Forgy > Sent: Monday, November 2, 2015 3:30 PM > To: julia-users > Subject: RE: [julia-users] Re: Re: are array slices views in 0.4? > > Hi David, > > I'm not an expert, i.e. I've never used it, but your idea "using OldArrays" > sounds exactly like the purpose of Compat.jl, which I believe is the > compatibility module referred to. > > Have a look and good luck: > > https://github.com/JuliaLang/Compat.jl
RE: [julia-users] Re: Re: are array slices views in 0.4?
Hi David, I'm not an expert, i.e. I've never used it, but your idea "using OldArrays" sounds exactly like the purpose of Compat.jl, which I believe is the compatibility module referred to. Have a look and good luck: https://github.com/JuliaLang/Compat.jl
RE: [julia-users] Re: Re: are array slices views in 0.4?
I’m not sure what you refer to by “Compatability module”. From: julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On Behalf Of Christoph Ortner Sent: Monday, November 2, 2015 1:09 PM To: julia-users Subject: Re: [julia-users] Re: Re: are array slices views in 0.4? How is this different from the Compatibility module? Christoph On Monday, 2 November 2015 20:44:38 UTC, David Anthoff wrote: Yes, I like that kind of idea. How about specifically: - If a package does nothing, every use of `[]` for slicing throws a deprecation warning on julia 0.5 - If you use `using OldArrays` the deprecation goes away, and you get the julia 0.4 behavior (maybe this options is actually not even needed, or maybe it should even not exist, not sure) - If you use `using NewArrays` you get the new slicing behavior on julia 0.5 And then in julia 0.6 `[]` just gives the new behavior by default, i.e. the `using NewArrays` is no longer needed. In julia 0.6 `using NewArrays` could actually show a deprecation warning that this using statement is no longer needed. From: julia...@googlegroups.com [mailto:julia...@googlegroups.com ] On Behalf Of Cedric St-Jean Sent: Monday, November 2, 2015 12:00 PM To: julia-users > Subject: Re: [julia-users] Re: Re: are array slices views in 0.4? Python's from __future__ import division was a reasonable way of transitioning. I would use it for slicing if it was available already. On Monday, November 2, 2015 at 2:46:04 PM UTC-5, David Anthoff wrote: The `using OldArrays` approach would essentially require an opt-in from everyone before julia 0.5 is released, i.e. an opt-in action from all package devs, right? The danger there seems that I update to julia 0.5, but now I have no clue whether all the packages I rely on have either a) made sure that they are compatible with the new array handling or b) have added the `using OldArrays` approach so that they can delay dealing with the issue. The nice thing with the current deprecation handling is that as a user of a package that has failed to deal with these things, I get alerted to issues automatically and know that something might need my attention. But a `using OldArrays` approach doesn’t seem to do that, in the end it still just means that the semantics of `[]` are changed without the typical deprecation sequence of using multiple versions for such a change. From: julia...@googlegroups.com <mailto:julia...@googlegroups.com> [mailto:julia...@googlegroups.com] On Behalf Of Stefan Karpinski Sent: Tuesday, October 27, 2015 7:43 AM To: Julia Users mailto:julia...@googlegroups.com> > Subject: Re: [julia-users] Re: Re: are array slices views in 0.4? Yes, I'm concerned about this as well. I'm sure we'll come up with something. Maybe allow using the old behavior during a transitional period with something like `using OldArrays` and then when you've gotten the chance to double check your code, you can delete that line and move to the new behavior. We'll have to see. On Mon, Oct 26, 2015 at 10:20 PM, David Anthoff mailto:ant...@berkeley.edu> > wrote: Are there plans to throw deprecation warnings in julia 0.5 whenever one slices an array with [], and then reuse the [] syntax to return views in julia 0.6? That would be approach that is consistent with previous changes of functionality, right? I’m very much in favor of the new design, but I’m very worried about the transition. There seems an enormous potential for subtle bugs to go undetected for a long time… the tuple change was nicely phased in, as were any other breaking changes since I’ve used julia (since 0.2), i.e. I always had the impression that as long as I just fixed all depreciation warnings when a new version came out, I would be good. But my understanding right now for the array change is that the behavior of slicing with [] will change drastically, with essentially no indicator where in my code I might run into trouble, right? From: julia...@googlegroups.com <mailto:julia...@googlegroups.com> [mailto:julia...@googlegroups.com] On Behalf Of Stefan Karpinski Sent: Monday, October 26, 2015 12:05 PM To: Julia Users mailto:julia...@googlegroups.com> > Subject: Re: [julia-users] Re: Re: are array slices views in 0.4? On Mon, Oct 26, 2015 at 2:17 PM, Christoph Ortner mailto:christop...@gmail.com> > wrote: Fabian - Many thanks for your comments. This was very helpful. (c) if I want to write code now that shouldn't break with 0.5, what should I do? I think when you need a copy, just surround your getindex with a copy function. (e.g. copy(x[:,10]) instead of x[:,10]). But this would lead me to make two copies. I was more interested in seeing whether there is a guideline on how to write code now so it doesn't have to be rewritten for 0.5. There will be a solution in the Compat package once this chan
Re: [julia-users] Re: Re: are array slices views in 0.4?
How is this different from the Compatibility module? Christoph On Monday, 2 November 2015 20:44:38 UTC, David Anthoff wrote: > > Yes, I like that kind of idea. How about specifically: > > > > - If a package does nothing, every use of `[]` for slicing throws a > deprecation warning on julia 0.5 > > - If you use `using OldArrays` the deprecation goes away, and you get the > julia 0.4 behavior (maybe this options is actually not even needed, or > maybe it should even not exist, not sure) > > - If you use `using NewArrays` you get the new slicing behavior on julia > 0.5 > > > > And then in julia 0.6 `[]` just gives the new behavior by default, i.e. > the `using NewArrays` is no longer needed. In julia 0.6 `using NewArrays` > could actually show a deprecation warning that this using statement is no > longer needed. > > > > *From:* julia...@googlegroups.com [mailto: > julia...@googlegroups.com ] *On Behalf Of *Cedric St-Jean > *Sent:* Monday, November 2, 2015 12:00 PM > *To:* julia-users > > *Subject:* Re: [julia-users] Re: Re: are array slices views in 0.4? > > > > Python's > > from __future__ import division > > > was a reasonable way of transitioning. I would use it for slicing if it > was available already. > > On Monday, November 2, 2015 at 2:46:04 PM UTC-5, David Anthoff wrote: > > The `using OldArrays` approach would essentially require an opt-in from > everyone before julia 0.5 is released, i.e. an opt-in action from all > package devs, right? The danger there seems that I update to julia 0.5, but > now I have no clue whether all the packages I rely on have either a) made > sure that they are compatible with the new array handling or b) have added > the `using OldArrays` approach so that they can delay dealing with the > issue. The nice thing with the current deprecation handling is that as a > user of a package that has failed to deal with these things, I get alerted > to issues automatically and know that something might need my attention. > But a `using OldArrays` approach doesn’t seem to do that, in the end it > still just means that the semantics of `[]` are changed without the typical > deprecation sequence of using multiple versions for such a change. > > > > *From:* julia...@googlegroups.com [mailto:julia...@googlegroups.com] *On > Behalf Of *Stefan Karpinski > *Sent:* Tuesday, October 27, 2015 7:43 AM > *To:* Julia Users > *Subject:* Re: [julia-users] Re: Re: are array slices views in 0.4? > > > > Yes, I'm concerned about this as well. I'm sure we'll come up with > something. Maybe allow using the old behavior during a transitional period > with something like `using OldArrays` and then when you've gotten the > chance to double check your code, you can delete that line and move to the > new behavior. We'll have to see. > > > > On Mon, Oct 26, 2015 at 10:20 PM, David Anthoff > wrote: > > Are there plans to throw deprecation warnings in julia 0.5 whenever one > slices an array with [], and then reuse the [] syntax to return views in > julia 0.6? That would be approach that is consistent with previous changes > of functionality, right? > > > > I’m very much in favor of the new design, but I’m very worried about the > transition. There seems an enormous potential for subtle bugs to go > undetected for a long time… the tuple change was nicely phased in, as were > any other breaking changes since I’ve used julia (since 0.2), i.e. I always > had the impression that as long as I just fixed all depreciation warnings > when a new version came out, I would be good. But my understanding right > now for the array change is that the behavior of slicing with [] will > change drastically, with essentially no indicator where in my code I might > run into trouble, right? > > > > *From:* julia...@googlegroups.com [mailto:julia...@googlegroups.com] *On > Behalf Of *Stefan Karpinski > *Sent:* Monday, October 26, 2015 12:05 PM > *To:* Julia Users > *Subject:* Re: [julia-users] Re: Re: are array slices views in 0.4? > > > > On Mon, Oct 26, 2015 at 2:17 PM, Christoph Ortner > wrote: > > Fabian - Many thanks for your comments. This was very helpful. > > > > (c) if I want to write code now that shouldn't break with 0.5, what should > I do? > > > I think when you need a copy, just surround your getindex with a copy > function. (e.g. copy(x[:,10]) instead of x[:,10]). > > > > But this would lead me to make two copies. I was more interested in seeing > whether there is a guideline on how to write code now so it doesn't have to > be rewritten fo
RE: [julia-users] Re: Re: are array slices views in 0.4?
Yes, I like that kind of idea. How about specifically: - If a package does nothing, every use of `[]` for slicing throws a deprecation warning on julia 0.5 - If you use `using OldArrays` the deprecation goes away, and you get the julia 0.4 behavior (maybe this options is actually not even needed, or maybe it should even not exist, not sure) - If you use `using NewArrays` you get the new slicing behavior on julia 0.5 And then in julia 0.6 `[]` just gives the new behavior by default, i.e. the `using NewArrays` is no longer needed. In julia 0.6 `using NewArrays` could actually show a deprecation warning that this using statement is no longer needed. From: julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On Behalf Of Cedric St-Jean Sent: Monday, November 2, 2015 12:00 PM To: julia-users Subject: Re: [julia-users] Re: Re: are array slices views in 0.4? Python's from __future__ import division was a reasonable way of transitioning. I would use it for slicing if it was available already. On Monday, November 2, 2015 at 2:46:04 PM UTC-5, David Anthoff wrote: The `using OldArrays` approach would essentially require an opt-in from everyone before julia 0.5 is released, i.e. an opt-in action from all package devs, right? The danger there seems that I update to julia 0.5, but now I have no clue whether all the packages I rely on have either a) made sure that they are compatible with the new array handling or b) have added the `using OldArrays` approach so that they can delay dealing with the issue. The nice thing with the current deprecation handling is that as a user of a package that has failed to deal with these things, I get alerted to issues automatically and know that something might need my attention. But a `using OldArrays` approach doesn’t seem to do that, in the end it still just means that the semantics of `[]` are changed without the typical deprecation sequence of using multiple versions for such a change. From: julia...@googlegroups.com [mailto:julia...@googlegroups.com ] On Behalf Of Stefan Karpinski Sent: Tuesday, October 27, 2015 7:43 AM To: Julia Users > Subject: Re: [julia-users] Re: Re: are array slices views in 0.4? Yes, I'm concerned about this as well. I'm sure we'll come up with something. Maybe allow using the old behavior during a transitional period with something like `using OldArrays` and then when you've gotten the chance to double check your code, you can delete that line and move to the new behavior. We'll have to see. On Mon, Oct 26, 2015 at 10:20 PM, David Anthoff > wrote: Are there plans to throw deprecation warnings in julia 0.5 whenever one slices an array with [], and then reuse the [] syntax to return views in julia 0.6? That would be approach that is consistent with previous changes of functionality, right? I’m very much in favor of the new design, but I’m very worried about the transition. There seems an enormous potential for subtle bugs to go undetected for a long time… the tuple change was nicely phased in, as were any other breaking changes since I’ve used julia (since 0.2), i.e. I always had the impression that as long as I just fixed all depreciation warnings when a new version came out, I would be good. But my understanding right now for the array change is that the behavior of slicing with [] will change drastically, with essentially no indicator where in my code I might run into trouble, right? From: julia...@googlegroups.com [mailto:julia...@googlegroups.com ] On Behalf Of Stefan Karpinski Sent: Monday, October 26, 2015 12:05 PM To: Julia Users > Subject: Re: [julia-users] Re: Re: are array slices views in 0.4? On Mon, Oct 26, 2015 at 2:17 PM, Christoph Ortner > wrote: Fabian - Many thanks for your comments. This was very helpful. (c) if I want to write code now that shouldn't break with 0.5, what should I do? I think when you need a copy, just surround your getindex with a copy function. (e.g. copy(x[:,10]) instead of x[:,10]). But this would lead me to make two copies. I was more interested in seeing whether there is a guideline on how to write code now so it doesn't have to be rewritten for 0.5. There will be a solution in the Compat package once this change is made. It will probably consist of having a replacement for getindex that creates a slice rather than a copy so that calling copy won't result in two copies. I.e. it will backport the 0.5 behavior to earlier versions of Julia. Regarding this change I am also more on the sceptical side. I would very much prefer a copy-on-write like solution like Matlab and R provide, but I don't know if and how this would be possible to implement, so I don't raise my voice here. To me the main benefit of this change is that it drove the main developers to make array views much more performant and first class members
Re: [julia-users] Re: Re: are array slices views in 0.4?
Python's from __future__ import division was a reasonable way of transitioning. I would use it for slicing if it was available already. On Monday, November 2, 2015 at 2:46:04 PM UTC-5, David Anthoff wrote: > > The `using OldArrays` approach would essentially require an opt-in from > everyone before julia 0.5 is released, i.e. an opt-in action from all > package devs, right? The danger there seems that I update to julia 0.5, but > now I have no clue whether all the packages I rely on have either a) made > sure that they are compatible with the new array handling or b) have added > the `using OldArrays` approach so that they can delay dealing with the > issue. The nice thing with the current deprecation handling is that as a > user of a package that has failed to deal with these things, I get alerted > to issues automatically and know that something might need my attention. > But a `using OldArrays` approach doesn’t seem to do that, in the end it > still just means that the semantics of `[]` are changed without the typical > deprecation sequence of using multiple versions for such a change. > > > > *From:* julia...@googlegroups.com [mailto: > julia...@googlegroups.com ] *On Behalf Of *Stefan Karpinski > *Sent:* Tuesday, October 27, 2015 7:43 AM > *To:* Julia Users > > *Subject:* Re: [julia-users] Re: Re: are array slices views in 0.4? > > > > Yes, I'm concerned about this as well. I'm sure we'll come up with > something. Maybe allow using the old behavior during a transitional period > with something like `using OldArrays` and then when you've gotten the > chance to double check your code, you can delete that line and move to the > new behavior. We'll have to see. > > > > On Mon, Oct 26, 2015 at 10:20 PM, David Anthoff > wrote: > > Are there plans to throw deprecation warnings in julia 0.5 whenever one > slices an array with [], and then reuse the [] syntax to return views in > julia 0.6? That would be approach that is consistent with previous changes > of functionality, right? > > > > I’m very much in favor of the new design, but I’m very worried about the > transition. There seems an enormous potential for subtle bugs to go > undetected for a long time… the tuple change was nicely phased in, as were > any other breaking changes since I’ve used julia (since 0.2), i.e. I always > had the impression that as long as I just fixed all depreciation warnings > when a new version came out, I would be good. But my understanding right > now for the array change is that the behavior of slicing with [] will > change drastically, with essentially no indicator where in my code I might > run into trouble, right? > > > > *From:* julia...@googlegroups.com [mailto: > julia...@googlegroups.com ] *On Behalf Of *Stefan Karpinski > *Sent:* Monday, October 26, 2015 12:05 PM > *To:* Julia Users > > *Subject:* Re: [julia-users] Re: Re: are array slices views in 0.4? > > > > On Mon, Oct 26, 2015 at 2:17 PM, Christoph Ortner > wrote: > > Fabian - Many thanks for your comments. This was very helpful. > > > > (c) if I want to write code now that shouldn't break with 0.5, what should > I do? > > > I think when you need a copy, just surround your getindex with a copy > function. (e.g. copy(x[:,10]) instead of x[:,10]). > > > > But this would lead me to make two copies. I was more interested in seeing > whether there is a guideline on how to write code now so it doesn't have to > be rewritten for 0.5. > > > > There will be a solution in the Compat package once this change is made. > It will probably consist of having a replacement for getindex that creates > a slice rather than a copy so that calling copy won't result in two copies. > I.e. it will backport the 0.5 behavior to earlier versions of Julia. > > > > Regarding this change I am also more on the sceptical side. I would very > much prefer a copy-on-write like solution like Matlab and R provide, but I > don't know if and how this would be possible to implement, so I don't raise > my voice here. > To me the main benefit of this change is that it drove the main developers > to make array views much more performant and first class members of julia. > As Tim Holy mentioned, the actual change seems to be be very small, but it > needed and still needs a lot of work to make it possible. > > > > My own scepticism comes from the idea that using immutable objects > throughout prevents bugs and one should only use mutable objects sparingly > (primarily for performance - but I thought it shouldn't be the default) > > > > Copy-on-wr
RE: [julia-users] Re: Re: are array slices views in 0.4?
The `using OldArrays` approach would essentially require an opt-in from everyone before julia 0.5 is released, i.e. an opt-in action from all package devs, right? The danger there seems that I update to julia 0.5, but now I have no clue whether all the packages I rely on have either a) made sure that they are compatible with the new array handling or b) have added the `using OldArrays` approach so that they can delay dealing with the issue. The nice thing with the current deprecation handling is that as a user of a package that has failed to deal with these things, I get alerted to issues automatically and know that something might need my attention. But a `using OldArrays` approach doesn’t seem to do that, in the end it still just means that the semantics of `[]` are changed without the typical deprecation sequence of using multiple versions for such a change. From: julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On Behalf Of Stefan Karpinski Sent: Tuesday, October 27, 2015 7:43 AM To: Julia Users Subject: Re: [julia-users] Re: Re: are array slices views in 0.4? Yes, I'm concerned about this as well. I'm sure we'll come up with something. Maybe allow using the old behavior during a transitional period with something like `using OldArrays` and then when you've gotten the chance to double check your code, you can delete that line and move to the new behavior. We'll have to see. On Mon, Oct 26, 2015 at 10:20 PM, David Anthoff mailto:anth...@berkeley.edu> > wrote: Are there plans to throw deprecation warnings in julia 0.5 whenever one slices an array with [], and then reuse the [] syntax to return views in julia 0.6? That would be approach that is consistent with previous changes of functionality, right? I’m very much in favor of the new design, but I’m very worried about the transition. There seems an enormous potential for subtle bugs to go undetected for a long time… the tuple change was nicely phased in, as were any other breaking changes since I’ve used julia (since 0.2), i.e. I always had the impression that as long as I just fixed all depreciation warnings when a new version came out, I would be good. But my understanding right now for the array change is that the behavior of slicing with [] will change drastically, with essentially no indicator where in my code I might run into trouble, right? From: julia-users@googlegroups.com <mailto:julia-users@googlegroups.com> [mailto:julia-users@googlegroups.com <mailto:julia-users@googlegroups.com> ] On Behalf Of Stefan Karpinski Sent: Monday, October 26, 2015 12:05 PM To: Julia Users mailto:julia-users@googlegroups.com> > Subject: Re: [julia-users] Re: Re: are array slices views in 0.4? On Mon, Oct 26, 2015 at 2:17 PM, Christoph Ortner mailto:christophortn...@gmail.com> > wrote: Fabian - Many thanks for your comments. This was very helpful. (c) if I want to write code now that shouldn't break with 0.5, what should I do? I think when you need a copy, just surround your getindex with a copy function. (e.g. copy(x[:,10]) instead of x[:,10]). But this would lead me to make two copies. I was more interested in seeing whether there is a guideline on how to write code now so it doesn't have to be rewritten for 0.5. There will be a solution in the Compat package once this change is made. It will probably consist of having a replacement for getindex that creates a slice rather than a copy so that calling copy won't result in two copies. I.e. it will backport the 0.5 behavior to earlier versions of Julia. Regarding this change I am also more on the sceptical side. I would very much prefer a copy-on-write like solution like Matlab and R provide, but I don't know if and how this would be possible to implement, so I don't raise my voice here. To me the main benefit of this change is that it drove the main developers to make array views much more performant and first class members of julia. As Tim Holy mentioned, the actual change seems to be be very small, but it needed and still needs a lot of work to make it possible. My own scepticism comes from the idea that using immutable objects throughout prevents bugs and one should only use mutable objects sparingly (primarily for performance - but I thought it shouldn't be the default) Copy-on-write is complex and leads to brittle performance properties that cannot be reasoned about locally. The semantics of R and Matlab also notoriously make it impossible to write efficient mutating functions – people generally end up writing C extensions to do that. It remains to be seen how this pans out, but keep in mind that C, C++, Java, Fortran, Julia, Python, Ruby, etc. all use mutable non-copy-on-write arrays everywhere and the world has not ended. Slices are a bit different, but NumPy, for example, creates views for
Re: [julia-users] Re: Re: are array slices views in 0.4?
Yes, I'm concerned about this as well. I'm sure we'll come up with something. Maybe allow using the old behavior during a transitional period with something like `using OldArrays` and then when you've gotten the chance to double check your code, you can delete that line and move to the new behavior. We'll have to see. On Mon, Oct 26, 2015 at 10:20 PM, David Anthoff wrote: > Are there plans to throw deprecation warnings in julia 0.5 whenever one > slices an array with [], and then reuse the [] syntax to return views in > julia 0.6? That would be approach that is consistent with previous changes > of functionality, right? > > > > I’m very much in favor of the new design, but I’m very worried about the > transition. There seems an enormous potential for subtle bugs to go > undetected for a long time… the tuple change was nicely phased in, as were > any other breaking changes since I’ve used julia (since 0.2), i.e. I always > had the impression that as long as I just fixed all depreciation warnings > when a new version came out, I would be good. But my understanding right > now for the array change is that the behavior of slicing with [] will > change drastically, with essentially no indicator where in my code I might > run into trouble, right? > > > > *From:* julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] > *On Behalf Of *Stefan Karpinski > *Sent:* Monday, October 26, 2015 12:05 PM > *To:* Julia Users > *Subject:* Re: [julia-users] Re: Re: are array slices views in 0.4? > > > > On Mon, Oct 26, 2015 at 2:17 PM, Christoph Ortner < > christophortn...@gmail.com> wrote: > > Fabian - Many thanks for your comments. This was very helpful. > > > > (c) if I want to write code now that shouldn't break with 0.5, what should > I do? > > > I think when you need a copy, just surround your getindex with a copy > function. (e.g. copy(x[:,10]) instead of x[:,10]). > > > > But this would lead me to make two copies. I was more interested in seeing > whether there is a guideline on how to write code now so it doesn't have to > be rewritten for 0.5. > > > > There will be a solution in the Compat package once this change is made. > It will probably consist of having a replacement for getindex that creates > a slice rather than a copy so that calling copy won't result in two copies. > I.e. it will backport the 0.5 behavior to earlier versions of Julia. > > > > Regarding this change I am also more on the sceptical side. I would very > much prefer a copy-on-write like solution like Matlab and R provide, but I > don't know if and how this would be possible to implement, so I don't raise > my voice here. > To me the main benefit of this change is that it drove the main developers > to make array views much more performant and first class members of julia. > As Tim Holy mentioned, the actual change seems to be be very small, but it > needed and still needs a lot of work to make it possible. > > > > My own scepticism comes from the idea that using immutable objects > throughout prevents bugs and one should only use mutable objects sparingly > (primarily for performance - but I thought it shouldn't be the default) > > > > Copy-on-write is complex and leads to brittle performance properties that > cannot be reasoned about locally. The semantics of R and Matlab also > notoriously make it impossible to write efficient mutating functions – > people generally end up writing C extensions to do that. > > > > It remains to be seen how this pans out, but keep in mind that C, C++, > Java, Fortran, Julia, Python, Ruby, etc. all use mutable non-copy-on-write > arrays everywhere and the world has not ended. Slices are a bit different, > but NumPy, for example, creates views for slices and that works well in the > SciPy ecosystem. > > > > Philosophically, I think that returning views from operations is > problematic when the object being viewed is conceptually a single value – > strings being a good example that have gone different ways in different > languages. In C everyone thinks of strings as arrays of characters and it > works pretty well since everyone has that in mind. In higher level > languages, people stop thinking of strings this way, which means that > making strings mutable or returning slices of them as views becomes > problematic because it's at odds with how we think of strings. Arrays are > the prototypical example of a container-like thing, so I don't think that > this will be that confusing. If you "own" the array, then it's ok to make a > slice and potentially mutate it – if you don't, then it's not ok. We could > potentially add tooling to help enforce this since we know by the f! naming > convention which functions should and shouldn't mutate their arguments. >
RE: [julia-users] Re: Re: are array slices views in 0.4?
Are there plans to throw deprecation warnings in julia 0.5 whenever one slices an array with [], and then reuse the [] syntax to return views in julia 0.6? That would be approach that is consistent with previous changes of functionality, right? I’m very much in favor of the new design, but I’m very worried about the transition. There seems an enormous potential for subtle bugs to go undetected for a long time… the tuple change was nicely phased in, as were any other breaking changes since I’ve used julia (since 0.2), i.e. I always had the impression that as long as I just fixed all depreciation warnings when a new version came out, I would be good. But my understanding right now for the array change is that the behavior of slicing with [] will change drastically, with essentially no indicator where in my code I might run into trouble, right? From: julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On Behalf Of Stefan Karpinski Sent: Monday, October 26, 2015 12:05 PM To: Julia Users Subject: Re: [julia-users] Re: Re: are array slices views in 0.4? On Mon, Oct 26, 2015 at 2:17 PM, Christoph Ortner mailto:christophortn...@gmail.com> > wrote: Fabian - Many thanks for your comments. This was very helpful. (c) if I want to write code now that shouldn't break with 0.5, what should I do? I think when you need a copy, just surround your getindex with a copy function. (e.g. copy(x[:,10]) instead of x[:,10]). But this would lead me to make two copies. I was more interested in seeing whether there is a guideline on how to write code now so it doesn't have to be rewritten for 0.5. There will be a solution in the Compat package once this change is made. It will probably consist of having a replacement for getindex that creates a slice rather than a copy so that calling copy won't result in two copies. I.e. it will backport the 0.5 behavior to earlier versions of Julia. Regarding this change I am also more on the sceptical side. I would very much prefer a copy-on-write like solution like Matlab and R provide, but I don't know if and how this would be possible to implement, so I don't raise my voice here. To me the main benefit of this change is that it drove the main developers to make array views much more performant and first class members of julia. As Tim Holy mentioned, the actual change seems to be be very small, but it needed and still needs a lot of work to make it possible. My own scepticism comes from the idea that using immutable objects throughout prevents bugs and one should only use mutable objects sparingly (primarily for performance - but I thought it shouldn't be the default) Copy-on-write is complex and leads to brittle performance properties that cannot be reasoned about locally. The semantics of R and Matlab also notoriously make it impossible to write efficient mutating functions – people generally end up writing C extensions to do that. It remains to be seen how this pans out, but keep in mind that C, C++, Java, Fortran, Julia, Python, Ruby, etc. all use mutable non-copy-on-write arrays everywhere and the world has not ended. Slices are a bit different, but NumPy, for example, creates views for slices and that works well in the SciPy ecosystem. Philosophically, I think that returning views from operations is problematic when the object being viewed is conceptually a single value – strings being a good example that have gone different ways in different languages. In C everyone thinks of strings as arrays of characters and it works pretty well since everyone has that in mind. In higher level languages, people stop thinking of strings this way, which means that making strings mutable or returning slices of them as views becomes problematic because it's at odds with how we think of strings. Arrays are the prototypical example of a container-like thing, so I don't think that this will be that confusing. If you "own" the array, then it's ok to make a slice and potentially mutate it – if you don't, then it's not ok. We could potentially add tooling to help enforce this since we know by the f! naming convention which functions should and shouldn't mutate their arguments.
Re: [julia-users] Re: Re: are array slices views in 0.4?
Hi Stefan, Many thanks for the clarifications. Just for the record: I've often complained about the changes to Julia since 0.2, but this is one that I am more on the fence about. Christoph
Re: [julia-users] Re: Re: are array slices views in 0.4?
On Mon, Oct 26, 2015 at 2:17 PM, Christoph Ortner < christophortn...@gmail.com> wrote: > Fabian - Many thanks for your comments. This was very helpful. > > (c) if I want to write code now that shouldn't break with 0.5, what should >> I do? >> > > I think when you need a copy, just surround your getindex with a copy > function. (e.g. copy(x[:,10]) instead of x[:,10]). > > But this would lead me to make two copies. I was more interested in seeing > whether there is a guideline on how to write code now so it doesn't have to > be rewritten for 0.5. > There will be a solution in the Compat package once this change is made. It will probably consist of having a replacement for getindex that creates a slice rather than a copy so that calling copy won't result in two copies. I.e. it will backport the 0.5 behavior to earlier versions of Julia. > Regarding this change I am also more on the sceptical side. I would very >> much prefer a copy-on-write like solution like Matlab and R provide, but I >> don't know if and how this would be possible to implement, so I don't raise >> my voice here. >> To me the main benefit of this change is that it drove the main >> developers to make array views much more performant and first class members >> of julia. As Tim Holy mentioned, the actual change seems to be be very >> small, but it needed and still needs a lot of work to make it possible. > > > My own scepticism comes from the idea that using immutable objects > throughout prevents bugs and one should only use mutable objects sparingly > (primarily for performance - but I thought it shouldn't be the default) > Copy-on-write is complex and leads to brittle performance properties that cannot be reasoned about locally. The semantics of R and Matlab also notoriously make it impossible to write efficient mutating functions – people generally end up writing C extensions to do that. It remains to be seen how this pans out, but keep in mind that C, C++, Java, Fortran, Julia, Python, Ruby, etc. all use mutable non-copy-on-write arrays everywhere and the world has not ended. Slices are a bit different, but NumPy, for example, creates views for slices and that works well in the SciPy ecosystem. Philosophically, I think that returning views from operations is problematic when the object being viewed is conceptually a single value – strings being a good example that have gone different ways in different languages. In C everyone thinks of strings as arrays of characters and it works pretty well since everyone has that in mind. In higher level languages, people stop thinking of strings this way, which means that making strings mutable or returning slices of them as views becomes problematic because it's at odds with how we think of strings. Arrays are the prototypical example of a container-like thing, so I don't think that this will be that confusing. If you "own" the array, then it's ok to make a slice and potentially mutate it – if you don't, then it's not ok. We could potentially add tooling to help enforce this since we know by the f! naming convention which functions should and shouldn't mutate their arguments.
Re: [julia-users] Re: Re: are array slices views in 0.4?
On Monday, 26 October 2015 11:17:58 UTC-7, Christoph Ortner wrote: > > Fabian - Many thanks for your comments. This was very helpful. > > (c) if I want to write code now that shouldn't break with 0.5, what should >> I do? >> > > I think when you need a copy, just surround your getindex with a copy > function. (e.g. copy(x[:,10]) instead of x[:,10]). > > But this would lead me to make two copies. I was more interested in seeing > whether there is a guideline on how to write code now so it doesn't have to > be rewritten for 0.5. > > > you could do copy(sub(x, :, 10)) to avoid two copies. Not as pretty ;) > My own scepticism comes from the idea that using immutable objects > throughout prevents bugs and one should only use mutable objects sparingly > (primarily for performance - but I thought it shouldn't be the default) > > the all immutable idea has its merits, but it is really not how Julia works already. If I pass the name of an array to a function I get a reference not a copy. I only get a copy of a slice of that same array at the moment which feels inconsistent to me. if your function accepts an array and you are mutating it in the body of the function I really don't like that func(a) func(b[1:2, 3:4]) has different behavior if I am mutating the passed in array. This will be a source of far more bugs in my mind than simply having it so that if you mutate the passed in arguments to a function you are changing the parameters. Also I would hate to have to do func(ref(a)) or some such thing when I have no plans to change a in my function (which for me is the farr more common case). Ultimately I find that Julia treats us like consenting adults when it comes to passing objects to functions ... and the new array views behavior simply adds consistency.
Re: [julia-users] Re: Re: are array slices views in 0.4?
Fabian - Many thanks for your comments. This was very helpful. (c) if I want to write code now that shouldn't break with 0.5, what should > I do? > I think when you need a copy, just surround your getindex with a copy function. (e.g. copy(x[:,10]) instead of x[:,10]). But this would lead me to make two copies. I was more interested in seeing whether there is a guideline on how to write code now so it doesn't have to be rewritten for 0.5. Regarding this change I am also more on the sceptical side. I would very much prefer a copy-on-write like solution like Matlab and R provide, but I don't know if and how this would be possible to implement, so I don't raise my voice here. To me the main benefit of this change is that it drove the main developers to make array views much more performant and first class members of julia. As Tim Holy mentioned, the actual change seems to be be very small, but it needed and still needs a lot of work to make it possible. My own scepticism comes from the idea that using immutable objects throughout prevents bugs and one should only use mutable objects sparingly (primarily for performance - but I thought it shouldn't be the default) Christoph
Re: [julia-users] Re: Re: are array slices views in 0.4?
On Friday, October 23, 2015 at 9:08:55 PM UTC+2, Christoph Ortner wrote: > > > Apparently yes. For me it is very counterintuitive that this would be the > default behaviour. But presumably there was a lot of discussion that this > is desirable. > > (a) What are reasons, other than performance? > I think this is the issue where this change was brought up: https://github.com/JuliaLang/julia/issues/3701 and the respective pull request is here: https://github.com/JuliaLang/julia/pull/9150 The main reason seems to be that Julia generally prefers reference-passing by default so this fits the language spirit. > (b) Is this still under discussion or pretty much settled? > > >From reading the issues, the core devs agree on the change and there wasn't a single objection so far. > (c) if I want to write code now that shouldn't break with 0.5, what should > I do? > I think when you need a copy, just surround your getindex with a copy function. (e.g. copy(x[:,10]) instead of x[:,10]). Regarding this change I am also more on the sceptical side. I would very much prefer a copy-on-write like solution like Matlab and R provide, but I don't know if and how this would be possible to implement, so I don't raise my voice here. To me the main benefit of this change is that it drove the main developers to make array views much more performant and first class members of julia. As Tim Holy mentioned, the actual change seems to be be very small, but it needed and still needs a lot of work to make it possible.
Re: [julia-users] Re: Re: are array slices views in 0.4?
Apparently yes. For me it is very counterintuitive that this would be the default behaviour. But presumably there was a lot of discussion that this is desirable. (a) What are reasons, other than performance? (b) Is this still under discussion or pretty much settled? (c) if I want to write code now that shouldn't break with 0.5, what should I do? Thanks, Christoph On Friday, 23 October 2015 20:04:58 UTC+1, Christoph Ortner wrote: > > > If I write into a view, does it change the original array? > > Christoph >
Re: [julia-users] Re: Re: are array slices views in 0.4?
Yes, that's the whole premise of a view. On Fri, Oct 23, 2015 at 3:04 PM, Christoph Ortner < christophortn...@gmail.com> wrote: > > If I write into a view, does it change the original array? > > Christoph >
Re: [julia-users] Re: Re: are array slices views in 0.4?
If I write into a view, does it change the original array? Christoph
Re: [julia-users] Re: Re: are array slices views in 0.4?
On Friday, October 23, 2015 at 12:24:59 PM UTC-4, Tom Breloff wrote: > > On this note, is there a good summary somewhere of the pros/cons between > using `sub` vs `slice` vs `ArrayViews.view`? > As I understand it, slice tries to squeeze the dimensions (producing lower-dimensional arrays if possible) more than sub. Otherwise they are equivalent; both produce a SubArray view of the original array. ArrayViews is mostly obsolete.
Re: [julia-users] Re: Re: are array slices views in 0.4?
On this note, is there a good summary somewhere of the pros/cons between using `sub` vs `slice` vs `ArrayViews.view`? On Fri, Oct 23, 2015 at 11:57 AM, Steven G. Johnson wrote: > On Friday, October 23, 2015 at 11:49:21 AM UTC-4, Neal Becker wrote: >> >> To be more precise, I'm coming from python/numpy. There, a native slice: >> M[1:,2:] for example >> returns a view, not a copy. As does a strided view: >> M[::2] >> I like this design. Does julia do this now, or does it plan to in the >> future? > > > In 0.5, the "native" slice syntax like M[1:end,2:end], both strided and > non-strided, will return a view. > > In 0.4, M[:] etc returns a copy, but you can get a view by the "slice" > function as Tim mentioned. So, the functionality is there now, but the > syntax is a little less familiar. >
[julia-users] Re: Re: are array slices views in 0.4?
On Friday, October 23, 2015 at 11:49:21 AM UTC-4, Neal Becker wrote: > > To be more precise, I'm coming from python/numpy. There, a native slice: > M[1:,2:] for example > returns a view, not a copy. As does a strided view: > M[::2] > I like this design. Does julia do this now, or does it plan to in the > future? In 0.5, the "native" slice syntax like M[1:end,2:end], both strided and non-strided, will return a view. In 0.4, M[:] etc returns a copy, but you can get a view by the "slice" function as Tim mentioned. So, the functionality is there now, but the syntax is a little less familiar.
Re: [julia-users] Re: Re: are array slices views in 0.4?
My point is it's just notation. `M[:,2]` is notation for `getindex(M, :, 2)`, which returns a copy. `slice(M, :, 2)` returns a view. In julia 0.5, `M[:,2]` will effectively become notation for `slice(M, :, 2)`. So we're talking about a difference of 6 characters. I know syntax is important, and that as heavily as arrays are used, 6 characters may not be trivial. (More importantly, the lack of bounds checking is not to be dismissed.) Still, it's worth distinguishing between "how many keystrokes do I have to type?" and "what functionality exists today?"---those are two very different questions. --Tim On Friday, October 23, 2015 11:48:31 AM Neal Becker wrote: > Tim Holy wrote: > > This will be a ridiculous charicature, but this issue has come up so many > > times that I think a little charicaturization is timely. > > > > With regards to arrays and views, the "big change" in julia 0.5 will > > > > essentially be a 1-line change: > > getindex(A, indexes...) = slice(A, indexes...) > > > > Since slice is already in base, you can use it today :-). > > > > There's much more to it than that: SubArrays don't always check bounds, so > > currently this is At Your Own Risk. There are some difficulties in > > reshaping SubArrays. > > > > But if the fundamental question is, "can I have performant and very > > flexible views?" the answer is: you already have them. > > > > Best, > > --Tim > > To be more precise, I'm coming from python/numpy. There, a native slice: > M[1:,2:] for example > returns a view, not a copy. As does a strided view: > M[::2] > I like this design. Does julia do this now, or does it plan to in the > future?
[julia-users] Re: Re: are array slices views in 0.4?
Tim Holy wrote: > This will be a ridiculous charicature, but this issue has come up so many > times that I think a little charicaturization is timely. > > With regards to arrays and views, the "big change" in julia 0.5 will > essentially be a 1-line change: > > getindex(A, indexes...) = slice(A, indexes...) > > Since slice is already in base, you can use it today :-). > > There's much more to it than that: SubArrays don't always check bounds, so > currently this is At Your Own Risk. There are some difficulties in > reshaping SubArrays. > > But if the fundamental question is, "can I have performant and very > flexible views?" the answer is: you already have them. > > Best, > --Tim To be more precise, I'm coming from python/numpy. There, a native slice: M[1:,2:] for example returns a view, not a copy. As does a strided view: M[::2] I like this design. Does julia do this now, or does it plan to in the future?
