Re: Interested in contributing to SPARK-24815

[Sean Owen]

When contributing to an ASF project, it's governed by the terms of the ASF
ICLA: https://www.apache.org/licenses/icla.pdf or CCLA:
https://www.apache.org/licenses/cla-corporate.pdf

I don't believe ASF projects ever retain an original author copyright
statement, but rather source files have a statement like:

...
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
...

While it's conceivable that such a statement could live in a NOTICE file, I
don't believe that's been done for any of the thousands of other
contributors. That's really more for noting the license of
non-Apache-licensed code. Code directly contributed to the project is
assumed to have been licensed per above already.

It might be wise to review the CCLA with Twilio and consider establishing
that to govern contributions.

On Mon, Jul 24, 2023 at 6:10 PM Pavan Kotikalapudi
 wrote:

> Hi Spark Dev,
>
> My name is Pavan Kotikalapudi, I work at Twilio.
>
> I am looking to contribute to this spark issue
> https://issues.apache.org/jira/browse/SPARK-24815.
>
> There is a clause from the company's OSS saying
>
> - The proposed contribution is about 100 lines of code modification in the
> Spark project, involving two files - this is considered a large
> contribution. An appropriate Twilio copyright notice needs to be added for
> the portion of code that is newly added.
>
> Please let me know if that is acceptable?
>
> Thank you,
>
> Pavan
>
>

Fwd: Interested in contributing to SPARK-24815

[Pavan Kotikalapudi]

Hi Spark Dev,

My name is Pavan Kotikalapudi, I work at Twilio.

I am looking to contribute to this spark issue
https://issues.apache.org/jira/browse/SPARK-24815.

There is a clause from the company's OSS saying

- The proposed contribution is about 100 lines of code modification in the
Spark project, involving two files - this is considered a large
contribution. An appropriate Twilio copyright notice needs to be added for
the portion of code that is newly added.

Please let me know if that is acceptable?

Thank you,

Pavan

Re: Spark 3.3 + parquet 1.10

[Mich Talebzadeh]

personally I have not done it myself.

CCed to spark user group if some user has tried it among users.

HTH

Mich Talebzadeh,
Solutions Architect/Engineering Lead
Palantir Technologies Limited
London
United Kingdom


   view my Linkedin profile



 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Mon, 24 Jul 2023 at 10:56, Pralabh Kumar  wrote:

> Spark3.3 in OSS built with parquet 1.12.  Just compiling  with parquet
> 1.10 results in build failure , so just wondering if any one have build &
> compiled Spark 3.3 with parquet 1.10.
>
> Regards
> Pralabh Kumar
>
> On Mon, Jul 24, 2023 at 3:04 PM Mich Talebzadeh 
> wrote:
>
>> Hi,
>>
>> Where is this limitation coming from (using 1.1.0)? That is 2018 build
>>
>> Have you tried Spark 3.3 with parquet writes as is? Just a small PoC will
>> prove it.
>>
>> HTH
>> Mich Talebzadeh,
>> Solutions Architect/Engineering Lead
>> Palantir Technologies Limited
>> London
>> United Kingdom
>>
>>
>>view my Linkedin profile
>> 
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 24 Jul 2023 at 10:15, Pralabh Kumar 
>> wrote:
>>
>>> Hi Dev community.
>>>
>>> I have a quick question with respect to Spark 3.3. Currently Spark 3.3
>>> is built with parquet 1.12.
>>> However, anyone tried Spark 3.3 with parquet 1.10 .
>>>
>>> We are at Uber , planning to migrate Spark 3.3 but we have limitations
>>> of using parquet 1.10  . Has anyone tried building Spark 3.3 with parquet
>>> 1.10 ? What are the dos/ don't for it ?
>>>
>>>
>>> Regards
>>> Pralabh Kumar
>>>
>>

Spark3.3 with parquet 1.10.x

[Pralabh Kumar]

Hi Spark Users .

I have a quick question with respect to Spark 3.3. Currently Spark 3.3 is
built with parquet 1.12.
However, anyone tried Spark 3.3 with parquet 1.10 .

We are at Uber , planning to migrate Spark 3.3 but we have limitations of
using parquet 1.10  . Has anyone tried building Spark 3.3 with parquet 1.10
? What are the dos/ don't for it ?


Regards
Pralabh Kumar