Re: Data Analysis and visualization

[Ishan Jain]

Size would be very large as stock prices would be streamed every hour

On Tue, Jun 13, 2017 at 12:05 PM, Jörn Franke  wrote:

> What is the size of the data?
> For me it looks more that orc or parquet would be enough.
>
>  I do not see here specific in-memory requirements.
>
> On 12. Jun 2017, at 09:59, Ishan Jain  wrote:
>
> I need to just get the price of a stock which is stored in hdfs with
> timestamp and make a graph with the prices of that stock over time.
>
> On Mon, Jun 12, 2017 at 1:03 PM, Jörn Franke  wrote:
>
>> First you need the user requirements - without them answering your
>> questions will be difficult
>>
>> > On 12. Jun 2017, at 07:08, ishan-jain  wrote:
>> >
>> > I am new to BIG Data .Just been working for a month.
>> > I have HDFS data of stock prices. I need to perform data analysis(maybe
>> some
>> > ML) and visualizations(Graphs and charts). For that i need Mapreduce
>> > functions. Which approach should i use?
>> > 1. Stream data from IGFS into ignite cache and work on it?
>> > 2. Use Hive with Tez and LLap function.(Should i use it with ignite or
>> > independent and directly on HDFS. No info available on the net.)
>> > 3. Use presto (Which is the better variant?(Hive or presto))
>> > 4. Some other fast way with IGFS if possible.
>> > 5. Also which open source tools should i use to accomplish this.
>> > Any help would be appreciated.
>> >
>> >
>> >
>> > --
>> > View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/Data-Analysis-and-visualization-tp13614.html
>> > Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>

Re: Data Analysis and visualization

[Jörn Franke]

What is the size of the data? 
For me it looks more that orc or parquet would be enough.

 I do not see here specific in-memory requirements.

> On 12. Jun 2017, at 09:59, Ishan Jain  wrote:
> 
> I need to just get the price of a stock which is stored in hdfs with 
> timestamp and make a graph with the prices of that stock over time.
> 
>> On Mon, Jun 12, 2017 at 1:03 PM, Jörn Franke  wrote:
>> First you need the user requirements - without them answering your questions 
>> will be difficult
>> 
>> > On 12. Jun 2017, at 07:08, ishan-jain  wrote:
>> >
>> > I am new to BIG Data .Just been working for a month.
>> > I have HDFS data of stock prices. I need to perform data analysis(maybe 
>> > some
>> > ML) and visualizations(Graphs and charts). For that i need Mapreduce
>> > functions. Which approach should i use?
>> > 1. Stream data from IGFS into ignite cache and work on it?
>> > 2. Use Hive with Tez and LLap function.(Should i use it with ignite or
>> > independent and directly on HDFS. No info available on the net.)
>> > 3. Use presto (Which is the better variant?(Hive or presto))
>> > 4. Some other fast way with IGFS if possible.
>> > 5. Also which open source tools should i use to accomplish this.
>> > Any help would be appreciated.
>> >
>> >
>> >
>> > --
>> > View this message in context: 
>> > http://apache-ignite-users.70518.x6.nabble.com/Data-Analysis-and-visualization-tp13614.html
>> > Sent from the Apache Ignite Users mailing list archive at Nabble.com.
> 

Re: Data Analysis and visualization

[ishan-jain]

I need to basically have a sql query remote access from tools like tableau or
zeppelin and have fast mapreduce funtions



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Data-Analysis-and-visualization-tp13614p13639.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Data Analysis and visualization

[Ishan Jain]

I need to just get the price of a stock which is stored in hdfs with
timestamp and make a graph with the prices of that stock over time.

On Mon, Jun 12, 2017 at 1:03 PM, Jörn Franke  wrote:

> First you need the user requirements - without them answering your
> questions will be difficult
>
> > On 12. Jun 2017, at 07:08, ishan-jain  wrote:
> >
> > I am new to BIG Data .Just been working for a month.
> > I have HDFS data of stock prices. I need to perform data analysis(maybe
> some
> > ML) and visualizations(Graphs and charts). For that i need Mapreduce
> > functions. Which approach should i use?
> > 1. Stream data from IGFS into ignite cache and work on it?
> > 2. Use Hive with Tez and LLap function.(Should i use it with ignite or
> > independent and directly on HDFS. No info available on the net.)
> > 3. Use presto (Which is the better variant?(Hive or presto))
> > 4. Some other fast way with IGFS if possible.
> > 5. Also which open source tools should i use to accomplish this.
> > Any help would be appreciated.
> >
> >
> >
> > --
> > View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Data-Analysis-and-visualization-tp13614.html
> > Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Data Analysis and visualization

[Jörn Franke]

First you need the user requirements - without them answering your questions 
will be difficult 

> On 12. Jun 2017, at 07:08, ishan-jain  wrote:
> 
> I am new to BIG Data .Just been working for a month.
> I have HDFS data of stock prices. I need to perform data analysis(maybe some
> ML) and visualizations(Graphs and charts). For that i need Mapreduce
> functions. Which approach should i use?
> 1. Stream data from IGFS into ignite cache and work on it?
> 2. Use Hive with Tez and LLap function.(Should i use it with ignite or
> independent and directly on HDFS. No info available on the net.)
> 3. Use presto (Which is the better variant?(Hive or presto))
> 4. Some other fast way with IGFS if possible.
> 5. Also which open source tools should i use to accomplish this.
> Any help would be appreciated. 
> 
> 
> 
> --
> View this message in context: 
> http://apache-ignite-users.70518.x6.nabble.com/Data-Analysis-and-visualization-tp13614.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.