Arko – you could use the following links to get started with SparkCLR API and use C# with Spark for DataFrame processing. If you need the support for interactive scenario, please feel free to share your scenario and requirements to the SparkCLR project. Interactive scenario is one of the focus areas of the current milestone in SparkCLR project.
· https://github.com/Microsoft/SparkCLR/blob/master/examples/JdbcDataFrame/Program.cs · https://github.com/Microsoft/SparkCLR/blob/master/csharp/Samples/Microsoft.Spark.CSharp/DataFrameSamples.cs Ted – System.Data.DataSetExtensions is a reference that is automatically added when a C# project is created in Visual Studio. As Silvio pointed out below, it is a .NET assembly and not really used by SparkCLR. From: Silvio Fiorito [mailto:[email protected]] Sent: Tuesday, February 9, 2016 1:31 PM To: Ted Yu <[email protected]>; Bryan Jeffrey <[email protected]> Cc: Arko Provo Mukherjee <[email protected]>; user <[email protected]> Subject: Re: Spark with .NET That’s just a .NET assembly (not related to Spark DataSets) but doesn’t look like they’re actually using it. It’s typically a default reference pulled in by the project templates. The code though is available from Mono here: https://github.com/mono/mono/tree/master/mcs/class/System.Data.DataSetExtensions From: Ted Yu <[email protected] <mailto:[email protected]> > Date: Tuesday, February 9, 2016 at 3:56 PM To: Bryan Jeffrey <[email protected] <mailto:[email protected]> > Cc: Arko Provo Mukherjee <[email protected] <mailto:[email protected]> >, user <[email protected] <mailto:[email protected]> > Subject: Re: Spark with .NET Looks like they have some system support whose source is not in the repo: <Reference Include="System.Data.DataSetExtensions" /> FYI On Tue, Feb 9, 2016 at 12:17 PM, Bryan Jeffrey <[email protected] <mailto:[email protected]> > wrote: Arko, Check this out: https://github.com/Microsoft/SparkCLR This is a Microsoft authored C# language binding for Spark. Regards, Bryan Jeffrey On Tue, Feb 9, 2016 at 3:13 PM, Arko Provo Mukherjee <[email protected] <mailto:[email protected]> > wrote: Doesn't seem to be supported, but thanks! I will probably write some .NET wrapper in my front end and use the java api in the backend. Warm regards Arko On Tue, Feb 9, 2016 at 12:05 PM, Ted Yu <[email protected] <mailto:[email protected]> > wrote: This thread is related: http://search-hadoop.com/m/q3RTtwp4nR1lugin1 <http://search-hadoop.com/m/q3RTtwp4nR1lugin1&subj=+NET+on+Apache+Spark+> &subj=+NET+on+Apache+Spark+ On Tue, Feb 9, 2016 at 11:43 AM, Arko Provo Mukherjee <[email protected] <mailto:[email protected]> > wrote: Hello, I want to use Spark (preferable Spark SQL) using C#. Anyone has any pointers to that? Thanks & regards Arko
