+1. That will preserve current behavior on older Spark versions too.
> On Mar 8, 2018, at 5:24 PM, Ted Yu <[email protected]> wrote:
>
> +1 on upgrading
> -------- Original message --------From: Matthias Boehm <[email protected]>
> Date: 3/8/18 5:19 PM (GMT-08:00) To: [email protected] Subject: Re:
> Release Planning
> related to Spark 2.3, we might want to update our ANTLR version because
> with Spark 2.3 every parsed DML script (i.e., multiple times with imported
> DML files) produces the following warning:
>
> "ANTLR Tool version 4.5.3 used for code generation does not match the
> current runtime version 4.7"
>
> Regards,
> Matthias
>
>> On Thu, Mar 1, 2018 at 5:22 PM, Matthias Boehm <[email protected]> wrote:
>>
>> Hi all,
>>
>> I'm sure you've seen that Spark 2.3 just got released. This lines up
>> beautifully with our own SystemML 1.1 release. Accordingly, I would
>> recommend to use Spark 2.3 for our due diligence algorithm-level Q/A. How
>> about we shoot for an RC1 by March 12? This should give enough time to run
>> over reasonably large data and fix all related issues.
>>
>> Regards,
>> Matthias
>>
>>> On Tue, Feb 6, 2018 at 12:51 PM, Matthias Boehm <[email protected]> wrote:
>>>
>>> yes, absolutely. Here is a list of new features and improvements - please
>>> feel free to extend as needed:
>>>
>>> 1) Extended Caffe2DML and Keras2DML APIs
>>> 2) Support for large-dense blocks >16GB in CP
>>> 3) New builtin functions ifelse, xor, as well as and/or/not over matrices
>>> 4) Single-precision support for native conv2d and mm operations.
>>> 5) Performance features and correctness of ultra-sparse operations
>>> 6) Codegen: new plan cache, nary cbind
>>> 7) Parfor: result merge with accumulators +=, reduced initialization
>>> overhead
>>> 8) Compiler improvements: avoid unnecessary spark instructions, corrected
>>> memory estimates
>>>
>>> Until the RC, I'd like to support all deep learning builtin functions in
>>> codegen, add a couple of pending parfor improvements, and potentially do a
>>> first cut of the compiler/runtime integration for parameter servers.
>>>
>>> Regards,
>>> Matthias
>>>
>>>
>>>
>>> On Tue, Feb 6, 2018 at 12:36 PM, Niketan Pansare <[email protected]>
>>> wrote:
>>>
>>>> +1.
>>>>
>>>> We should consider including single precision native BLAS in the release
>>>> notes as well. If possible, we should add JNI wrappers for PowerPC, Windows
>>>> and Mac too in this release.
>>>>
>>>>> On Feb 6, 2018, at 12:27 PM, Berthold Reinwald <[email protected]>
>>>> wrote:
>>>>>
>>>>> sure.
>>>>>
>>>>> Makes sense. Codegen and Keras2DML made good progress, and many other
>>>>> fixes/improvements.
>>>>>
>>>>> What else do we want time/track/highlight for it?
>>>>>
>>>>> Regards,
>>>>> Berthold Reinwald
>>>>> IBM Almaden Research Center
>>>>> office: (408) 927 2208; T/L: 457 2208
>>>>> e-mail: [email protected]
>>>>>
>>>>>
>>>>>
>>>>> From: Matthias Boehm <[email protected]>
>>>>> To: [email protected]
>>>>> Date: 02/05/2018 11:05 PM
>>>>> Subject: Release Planning
>>>>>
>>>>>
>>>>>
>>>>> Hi all,
>>>>>
>>>>> since our 1.0 release in Dec, we already got a number enhancements and
>>>> new
>>>>> features in, so I think it would be good to discuss the timeline for
>>>> our
>>>>> next SystemML 1.1 release. How about, we target mid March for a first
>>>> RC?
>>>>> Also, Berthold would you be willing to serve again as the release
>>>> manager?
>>>>>
>>>>> Regards,
>>>>> Matthias
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
