Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

2016-10-22 Thread dusenberrymw
+1 I finished running some test jobs in my large scale scenario on this release candidate, and I think it is good to go. Specifically, my scenario involved large numerical DataFrames, MLContext, matrices, DML, and multiple script invocations involving the various intermediate outputs. One opt

Re: [Discuss] String requirements for data passed to SystemML Frames.

2016-10-22 Thread Acs S
1. I don't believe we are suggesting to remove quotes. If we are following RFC 4180 based string format, then input coming from any source to be converted to RC 4180 based string format before any processing should occur. This will avoid exception if string is not compliant to RFC 4180 based for

Jenkins build is back to normal : SystemML-DailyTest #580

2016-10-22 Thread jenkins
See

Re: [Discuss] String requirements for data passed to SystemML Frames.

2016-10-22 Thread Matthias Boehm
ok let me clarify a couple of things and provide an easy solution that resolves this issue altogether. 1) Escaping: transformencode, transformdecode, and transformapply do not remove quotes to provide easy to understand semantics. If users want to match strings with different escaping policies

Re: [Discuss] String requirements for data passed to SystemML Frames.

2016-10-22 Thread Matthias Boehm
ok let me clarify a couple of things and provide an easy solution that resolves this issue altogether. 1) Escaping: transformencode, transformdecode, and transformapply do not remove quotes to provide easy to understand semantics. If users want to match strings with different escaping policies

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

2016-10-22 Thread Berthold Reinwald
-1. Transformencode throws an unnecessary error if strings to not comply with the field requirements specified in RFC 4180. Arvind has a fix on the way which should be included in the release. Regards, Berthold Reinwald IBM Almaden Research Center office: (408) 927 2208; T/L: 457 2208 e-mail:

[Discuss] String requirements for data passed to SystemML Frames.

2016-10-22 Thread Berthold Reinwald
Reading SystemML frames from CSV files, and splitting strings honoring quotes, separators, and escaping rules follows the RFC 4180 specification (https://tools.ietf.org/html/rfc4180#page-2). Populating SystemML frames from CSV files is one way, but we can also bind and pass Spark DataFrames with st