Re: Spark with .NET

[Arko Provo Mukherjee]

Doesn't seem to be supported, but thanks! I will probably write some .NET
wrapper in my front end and use the java api in the backend.
Warm regards
Arko


On Tue, Feb 9, 2016 at 12:05 PM, Ted Yu  wrote:

> This thread is related:
> http://search-hadoop.com/m/q3RTtwp4nR1lugin1=+NET+on+Apache+Spark+
>
> On Tue, Feb 9, 2016 at 11:43 AM, Arko Provo Mukherjee <
> arkoprovomukher...@gmail.com> wrote:
>
>> Hello,
>>
>> I want to use Spark (preferable Spark SQL) using C#. Anyone has any
>> pointers to that?
>>
>> Thanks & regards
>> Arko
>>
>>
>

Re: Spark with .NET

[Bryan Jeffrey]

Arko,

Check this out: https://github.com/Microsoft/SparkCLR

This is a Microsoft authored C# language binding for Spark.

Regards,

Bryan Jeffrey

On Tue, Feb 9, 2016 at 3:13 PM, Arko Provo Mukherjee <
arkoprovomukher...@gmail.com> wrote:

> Doesn't seem to be supported, but thanks! I will probably write some .NET
> wrapper in my front end and use the java api in the backend.
> Warm regards
> Arko
>
>
> On Tue, Feb 9, 2016 at 12:05 PM, Ted Yu  wrote:
>
>> This thread is related:
>> http://search-hadoop.com/m/q3RTtwp4nR1lugin1=+NET+on+Apache+Spark+
>>
>> On Tue, Feb 9, 2016 at 11:43 AM, Arko Provo Mukherjee <
>> arkoprovomukher...@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I want to use Spark (preferable Spark SQL) using C#. Anyone has any
>>> pointers to that?
>>>
>>> Thanks & regards
>>> Arko
>>>
>>>
>>
>

Re: Spark with .NET

[Ted Yu]

This thread is related:
http://search-hadoop.com/m/q3RTtwp4nR1lugin1=+NET+on+Apache+Spark+

On Tue, Feb 9, 2016 at 11:43 AM, Arko Provo Mukherjee <
arkoprovomukher...@gmail.com> wrote:

> Hello,
>
> I want to use Spark (preferable Spark SQL) using C#. Anyone has any
> pointers to that?
>
> Thanks & regards
> Arko
>
>

Re: Spark with .NET

[Ted Yu]

Looks like they have some system support whose source is not in the repo:


FYI

On Tue, Feb 9, 2016 at 12:17 PM, Bryan Jeffrey 
wrote:

> Arko,
>
> Check this out: https://github.com/Microsoft/SparkCLR
>
> This is a Microsoft authored C# language binding for Spark.
>
> Regards,
>
> Bryan Jeffrey
>
> On Tue, Feb 9, 2016 at 3:13 PM, Arko Provo Mukherjee <
> arkoprovomukher...@gmail.com> wrote:
>
>> Doesn't seem to be supported, but thanks! I will probably write some .NET
>> wrapper in my front end and use the java api in the backend.
>> Warm regards
>> Arko
>>
>>
>> On Tue, Feb 9, 2016 at 12:05 PM, Ted Yu  wrote:
>>
>>> This thread is related:
>>> http://search-hadoop.com/m/q3RTtwp4nR1lugin1=+NET+on+Apache+Spark+
>>>
>>> On Tue, Feb 9, 2016 at 11:43 AM, Arko Provo Mukherjee <
>>> arkoprovomukher...@gmail.com> wrote:
>>>
 Hello,

 I want to use Spark (preferable Spark SQL) using C#. Anyone has any
 pointers to that?

 Thanks & regards
 Arko


>>>
>>
>

Re: Spark with .NET

[Silvio Fiorito]

That’s just a .NET assembly (not related to Spark DataSets) but doesn’t look 
like they’re actually using it. It’s typically a default reference pulled in by 
the project templates.

The code though is available from Mono here: 
https://github.com/mono/mono/tree/master/mcs/class/System.Data.DataSetExtensions

From: Ted Yu <yuzhih...@gmail.com<mailto:yuzhih...@gmail.com>>
Date: Tuesday, February 9, 2016 at 3:56 PM
To: Bryan Jeffrey <bryan.jeff...@gmail.com<mailto:bryan.jeff...@gmail.com>>
Cc: Arko Provo Mukherjee 
<arkoprovomukher...@gmail.com<mailto:arkoprovomukher...@gmail.com>>, user 
<user@spark.apache.org<mailto:user@spark.apache.org>>
Subject: Re: Spark with .NET

Looks like they have some system support whose source is not in the repo:


FYI

On Tue, Feb 9, 2016 at 12:17 PM, Bryan Jeffrey 
<bryan.jeff...@gmail.com<mailto:bryan.jeff...@gmail.com>> wrote:
Arko,

Check this out: https://github.com/Microsoft/SparkCLR

This is a Microsoft authored C# language binding for Spark.

Regards,

Bryan Jeffrey

On Tue, Feb 9, 2016 at 3:13 PM, Arko Provo Mukherjee 
<arkoprovomukher...@gmail.com<mailto:arkoprovomukher...@gmail.com>> wrote:
Doesn't seem to be supported, but thanks! I will probably write some .NET 
wrapper in my front end and use the java api in the backend.
Warm regards
Arko


On Tue, Feb 9, 2016 at 12:05 PM, Ted Yu 
<yuzhih...@gmail.com<mailto:yuzhih...@gmail.com>> wrote:
This thread is related:
http://search-hadoop.com/m/q3RTtwp4nR1lugin1=+NET+on+Apache+Spark+

On Tue, Feb 9, 2016 at 11:43 AM, Arko Provo Mukherjee 
<arkoprovomukher...@gmail.com<mailto:arkoprovomukher...@gmail.com>> wrote:
Hello,

I want to use Spark (preferable Spark SQL) using C#. Anyone has any pointers to 
that?

Thanks & regards
Arko

Re: Spark with .NET

[Ted Yu]

bq. it is a .NET assembly and not really used by SparkCLR

Then maybe drop the import ?

I was searching the SparkCLR repo to see whether (Spark) DataSet is
supported.

Cheer

On Tue, Feb 9, 2016 at 3:07 PM, skaarthik oss <skaarthik@gmail.com>
wrote:

> *Arko* – you could use the following links to get started with SparkCLR
> API and use C# with Spark for DataFrame processing. If you need the support
> for interactive scenario, please feel free to share your scenario and
> requirements to the SparkCLR project. Interactive scenario is one of the
> focus areas of the current milestone in SparkCLR project.
>
> ·
> https://github.com/Microsoft/SparkCLR/blob/master/examples/JdbcDataFrame/Program.cs
>
> ·
> https://github.com/Microsoft/SparkCLR/blob/master/csharp/Samples/Microsoft.Spark.CSharp/DataFrameSamples.cs
>
>
>
>
>
> *Ted* – System.Data.DataSetExtensions is a reference that is
> automatically added when a C# project is created in Visual Studio. As
> Silvio pointed out below, it is a .NET assembly and not really used by
> SparkCLR.
>
>
>
> *From:* Silvio Fiorito [mailto:silvio.fior...@granturing.com]
> *Sent:* Tuesday, February 9, 2016 1:31 PM
> *To:* Ted Yu <yuzhih...@gmail.com>; Bryan Jeffrey <bryan.jeff...@gmail.com
> >
> *Cc:* Arko Provo Mukherjee <arkoprovomukher...@gmail.com>; user <
> user@spark.apache.org>
>
> *Subject:* Re: Spark with .NET
>
>
>
> That’s just a .NET assembly (not related to Spark DataSets) but doesn’t
> look like they’re actually using it. It’s typically a default reference
> pulled in by the project templates.
>
>
>
> The code though is available from Mono here:
> https://github.com/mono/mono/tree/master/mcs/class/System.Data.DataSetExtensions
>
>
>
> *From: *Ted Yu <yuzhih...@gmail.com>
> *Date: *Tuesday, February 9, 2016 at 3:56 PM
> *To: *Bryan Jeffrey <bryan.jeff...@gmail.com>
> *Cc: *Arko Provo Mukherjee <arkoprovomukher...@gmail.com>, user <
> user@spark.apache.org>
> *Subject: *Re: Spark with .NET
>
>
>
> Looks like they have some system support whose source is not in the repo:
>
> 
>
>
>
> FYI
>
>
>
> On Tue, Feb 9, 2016 at 12:17 PM, Bryan Jeffrey <bryan.jeff...@gmail.com>
> wrote:
>
> Arko,
>
>
> Check this out: https://github.com/Microsoft/SparkCLR
>
>
>
> This is a Microsoft authored C# language binding for Spark.
>
>
>
> Regards,
>
>
>
> Bryan Jeffrey
>
>
>
> On Tue, Feb 9, 2016 at 3:13 PM, Arko Provo Mukherjee <
> arkoprovomukher...@gmail.com> wrote:
>
> Doesn't seem to be supported, but thanks! I will probably write some .NET
> wrapper in my front end and use the java api in the backend.
>
> Warm regards
>
> Arko
>
>
>
>
>
> On Tue, Feb 9, 2016 at 12:05 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>
> This thread is related:
>
> http://search-hadoop.com/m/q3RTtwp4nR1lugin1=+NET+on+Apache+Spark+
>
>
>
> On Tue, Feb 9, 2016 at 11:43 AM, Arko Provo Mukherjee <
> arkoprovomukher...@gmail.com> wrote:
>
> Hello,
>
>
>
> I want to use Spark (preferable Spark SQL) using C#. Anyone has any
> pointers to that?
>
> Thanks & regards
>
> Arko
>
>
>
>
>
>
>
>
>
>
>

RE: Spark with .NET

[skaarthik oss]

Arko – you could use the following links to get started with SparkCLR API and 
use C# with Spark for DataFrame processing. If you need the support for 
interactive scenario, please feel free to share your scenario and requirements 
to the SparkCLR project. Interactive scenario is one of the focus areas of the 
current milestone in SparkCLR project.

· 
https://github.com/Microsoft/SparkCLR/blob/master/examples/JdbcDataFrame/Program.cs

· 
https://github.com/Microsoft/SparkCLR/blob/master/csharp/Samples/Microsoft.Spark.CSharp/DataFrameSamples.cs

 

 

Ted – System.Data.DataSetExtensions is a reference that is automatically added 
when a C# project is created in Visual Studio. As Silvio pointed out below, it 
is a .NET assembly and not really used by SparkCLR.

 

From: Silvio Fiorito [mailto:silvio.fior...@granturing.com] 
Sent: Tuesday, February 9, 2016 1:31 PM
To: Ted Yu <yuzhih...@gmail.com>; Bryan Jeffrey <bryan.jeff...@gmail.com>
Cc: Arko Provo Mukherjee <arkoprovomukher...@gmail.com>; user 
<user@spark.apache.org>
Subject: Re: Spark with .NET

 

That’s just a .NET assembly (not related to Spark DataSets) but doesn’t look 
like they’re actually using it. It’s typically a default reference pulled in by 
the project templates.

 

The code though is available from Mono here: 
https://github.com/mono/mono/tree/master/mcs/class/System.Data.DataSetExtensions

 

From: Ted Yu <yuzhih...@gmail.com <mailto:yuzhih...@gmail.com> >
Date: Tuesday, February 9, 2016 at 3:56 PM
To: Bryan Jeffrey <bryan.jeff...@gmail.com <mailto:bryan.jeff...@gmail.com> >
Cc: Arko Provo Mukherjee <arkoprovomukher...@gmail.com 
<mailto:arkoprovomukher...@gmail.com> >, user <user@spark.apache.org 
<mailto:user@spark.apache.org> >
Subject: Re: Spark with .NET

 

Looks like they have some system support whose source is not in the repo:



 

FYI

 

On Tue, Feb 9, 2016 at 12:17 PM, Bryan Jeffrey <bryan.jeff...@gmail.com 
<mailto:bryan.jeff...@gmail.com> > wrote:

Arko,


Check this out: https://github.com/Microsoft/SparkCLR

 

This is a Microsoft authored C# language binding for Spark.

 

Regards,

 

Bryan Jeffrey

 

On Tue, Feb 9, 2016 at 3:13 PM, Arko Provo Mukherjee 
<arkoprovomukher...@gmail.com <mailto:arkoprovomukher...@gmail.com> > wrote:

Doesn't seem to be supported, but thanks! I will probably write some .NET 
wrapper in my front end and use the java api in the backend. 

Warm regards

Arko

 

 

On Tue, Feb 9, 2016 at 12:05 PM, Ted Yu <yuzhih...@gmail.com 
<mailto:yuzhih...@gmail.com> > wrote:

This thread is related:

http://search-hadoop.com/m/q3RTtwp4nR1lugin1 
<http://search-hadoop.com/m/q3RTtwp4nR1lugin1=+NET+on+Apache+Spark+> 
=+NET+on+Apache+Spark+

 

On Tue, Feb 9, 2016 at 11:43 AM, Arko Provo Mukherjee 
<arkoprovomukher...@gmail.com <mailto:arkoprovomukher...@gmail.com> > wrote:

Hello, 

 

I want to use Spark (preferable Spark SQL) using C#. Anyone has any pointers to 
that? 

Thanks & regards

Arko

 

 

 

 

 

Re: Spark with .NET

[Arko Provo Mukherjee]

Hello,

Thanks much for your help, much helpful! Let me explore some of the stuff
suggested :)

Thanks & regards
Arko


On Tue, Feb 9, 2016 at 3:17 PM, Ted Yu <yuzhih...@gmail.com> wrote:

> bq. it is a .NET assembly and not really used by SparkCLR
>
> Then maybe drop the import ?
>
> I was searching the SparkCLR repo to see whether (Spark) DataSet is
> supported.
>
> Cheer
>
> On Tue, Feb 9, 2016 at 3:07 PM, skaarthik oss <skaarthik@gmail.com>
> wrote:
>
>> *Arko* – you could use the following links to get started with SparkCLR
>> API and use C# with Spark for DataFrame processing. If you need the support
>> for interactive scenario, please feel free to share your scenario and
>> requirements to the SparkCLR project. Interactive scenario is one of the
>> focus areas of the current milestone in SparkCLR project.
>>
>> ·
>> https://github.com/Microsoft/SparkCLR/blob/master/examples/JdbcDataFrame/Program.cs
>>
>> ·
>> https://github.com/Microsoft/SparkCLR/blob/master/csharp/Samples/Microsoft.Spark.CSharp/DataFrameSamples.cs
>>
>>
>>
>>
>>
>> *Ted* – System.Data.DataSetExtensions is a reference that is
>> automatically added when a C# project is created in Visual Studio. As
>> Silvio pointed out below, it is a .NET assembly and not really used by
>> SparkCLR.
>>
>>
>>
>> *From:* Silvio Fiorito [mailto:silvio.fior...@granturing.com]
>> *Sent:* Tuesday, February 9, 2016 1:31 PM
>> *To:* Ted Yu <yuzhih...@gmail.com>; Bryan Jeffrey <
>> bryan.jeff...@gmail.com>
>> *Cc:* Arko Provo Mukherjee <arkoprovomukher...@gmail.com>; user <
>> user@spark.apache.org>
>>
>> *Subject:* Re: Spark with .NET
>>
>>
>>
>> That’s just a .NET assembly (not related to Spark DataSets) but doesn’t
>> look like they’re actually using it. It’s typically a default reference
>> pulled in by the project templates.
>>
>>
>>
>> The code though is available from Mono here:
>> https://github.com/mono/mono/tree/master/mcs/class/System.Data.DataSetExtensions
>>
>>
>>
>> *From: *Ted Yu <yuzhih...@gmail.com>
>> *Date: *Tuesday, February 9, 2016 at 3:56 PM
>> *To: *Bryan Jeffrey <bryan.jeff...@gmail.com>
>> *Cc: *Arko Provo Mukherjee <arkoprovomukher...@gmail.com>, user <
>> user@spark.apache.org>
>> *Subject: *Re: Spark with .NET
>>
>>
>>
>> Looks like they have some system support whose source is not in the repo:
>>
>> 
>>
>>
>>
>> FYI
>>
>>
>>
>> On Tue, Feb 9, 2016 at 12:17 PM, Bryan Jeffrey <bryan.jeff...@gmail.com>
>> wrote:
>>
>> Arko,
>>
>>
>> Check this out: https://github.com/Microsoft/SparkCLR
>>
>>
>>
>> This is a Microsoft authored C# language binding for Spark.
>>
>>
>>
>> Regards,
>>
>>
>>
>> Bryan Jeffrey
>>
>>
>>
>> On Tue, Feb 9, 2016 at 3:13 PM, Arko Provo Mukherjee <
>> arkoprovomukher...@gmail.com> wrote:
>>
>> Doesn't seem to be supported, but thanks! I will probably write some .NET
>> wrapper in my front end and use the java api in the backend.
>>
>> Warm regards
>>
>> Arko
>>
>>
>>
>>
>>
>> On Tue, Feb 9, 2016 at 12:05 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>>
>> This thread is related:
>>
>> http://search-hadoop.com/m/q3RTtwp4nR1lugin1=+NET+on+Apache+Spark+
>>
>>
>>
>> On Tue, Feb 9, 2016 at 11:43 AM, Arko Provo Mukherjee <
>> arkoprovomukher...@gmail.com> wrote:
>>
>> Hello,
>>
>>
>>
>> I want to use Spark (preferable Spark SQL) using C#. Anyone has any
>> pointers to that?
>>
>> Thanks & regards
>>
>> Arko
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>