There are tons of such JavaScript based applications for plots. I have not compared them but flot seems to do the job. web2py is using flot for some graphs. What I meant with Java tools is that it's quite complex to write and read hdf files in Java applications. Unfortunately I have to use Java once in a while Thomas
On Tuesday, 22 October 2013 16:06:59 UTC+2, Richard wrote: > > There is also : http://d3js.org/ > > :) > > Richard > > > On Tue, Oct 22, 2013 at 10:04 AM, Richard Vézina > <[email protected]<javascript:> > > wrote: > >> request.application ?? >> >> >> On Tue, Oct 22, 2013 at 10:01 AM, Thomas S >> <[email protected]<javascript:> >> > wrote: >> >>> Yes, I use HDF5 for more than a year now. >>> It's great. The only drawback is the lack of elegant Java tools. >>> I upload those files and store their obfuscated names in a database, but >>> the actual file in uploads. >>> Having said that it's not very elegant to construct the URL to locate >>> the file: >>> >>> def show(): >>> csv = db.csv(request.args(0, cast=int)) >>> >>> import pandas >>> import os >>> >>> fff = os.path.join("applications", "cda", "uploads", csv.csvfile) >>> >>> session.dataframe = pandas.read_csv(fff, parse_dates=True, >>> index_col=0) >>> >>> return dict(title=csv.title, body=csv.body, author=csv.author, >>> keys=session.dataframe.keys()) >>> >>> Note that sesssion,dataframe is now a global variable. And note the ugly >>> os.path.join with the name of the application hardcoded.... UGLY! >>> >>> Here's a link: >>> https://tschmelz.**pythonanywher**e.com/cda<https://tschmelz.pythonanywhere.com/cda> >>> >>> and >>> >>> https://github.com/tschm/cda >>> >>> Thomas >>> >>> >>> >>> >>> >>> On Tuesday, 22 October 2013 15:16:25 UTC+2, Richard wrote: >>> >>>> I heard a lot of good about HDF5 file format to hande important volume >>>> of data hierachical (mean you can query what ever data you need without >>>> load the full data set into a json for instance) : >>>> http://en.wikipedia.org/**wiki/Hierarchical_Data_Format<http://en.wikipedia.org/wiki/Hierarchical_Data_Format> >>>> >>>> It very much faster then postgres (sure postgres is not the faster >>>> backend but it scale gracefully)... >>>> >>>> The intend of this file format is to be used in conjonction with a DB. >>>> >>>> If I remember Pandas can write HDF5, not sure which lib it uses, there >>>> is two major lib in python which have different set of feature, one is >>>> more >>>> fancy but not support all the HDF5 feature and the other is supporting >>>> "all" the feature but is less sexy... >>>> >>>> Richard >>>> >>>> >>>> On Tue, Oct 22, 2013 at 6:54 AM, Cliff Kachinske <[email protected]>wrote: >>>> >>>>> use the rows field in auth_permission as described here. >>>>> >>>>> http://web2py.com/books/**default/chapter/29/09/access-** >>>>> control#Authorization<http://web2py.com/books/default/chapter/29/09/access-control#Authorization> >>>>> >>>>> >>>>> On Tuesday, October 22, 2013 5:40:49 AM UTC-4, Thomas S wrote: >>>>>> >>>>>> Ok, I am making progress. I guess all those JavaScript tools are not >>>>>> great when it comes to plotting millions of points but I am happy to >>>>>> downsample on the server side and send less points >>>>>> >>>>>> - I am using flot instead of highcharts >>>>>> >>>>>> - Currently, the user is uploading a csv file. I don't do any parsing >>>>>> at this stage. However, I rather keep the file (under uploads) and parse >>>>>> it >>>>>> on request. Having said that this will become a lot more slick soon. >>>>>> This >>>>>> is my first application. >>>>>> >>>>>> One thing that puzzles me for now... >>>>>> >>>>>> A user has to login to upload a file (that's good), but he can then >>>>>> also modify or delete entries in the SQL database created by others. How >>>>>> can I make sure he/she only deletes rows he/she has created in the first >>>>>> place. All users should be able to see all files though. >>>>>> >>>>>> Here's a link: >>>>>> https://tschmelz.**pythonanywher**e.com/csv<https://tschmelz.pythonanywhere.com/csv> >>>>>> >>>>>> I will soon post it to my Github (username tschm) >>>>>> thomas >>>>>> >>>>>> >>>>>> On Sunday, 20 October 2013 15:38:07 UTC+2, Niphlod wrote: >>>>>>> >>>>>>> first things first: are you sure that highcharts can handle 10*100k >>>>>>> points to draw a graph ? >>>>>>> As for the storage, you can do anything you like: if the data >>>>>>> doesn't change that much, storing into the database will be a long >>>>>>> process >>>>>>> only on the first time. >>>>>>> On the other end, if you need to fetch 100k records and transform >>>>>>> them to json, it's going to take some time. >>>>>>> Not sure on how much you'll gain from parsing i.e. a csv file >>>>>>> instead of a querying a db.... >>>>>>> if the returning json object is , let's say, 10 mb, it's always >>>>>>> gonna feel heavy. >>>>>>> >>>>>>> On Sunday, October 20, 2013 9:11:07 AM UTC+2, Thomas S wrote: >>>>>>>> >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I have created a standard application relying on Pandas and PyQt4 >>>>>>>> to browse through a Pandas Dataframe. >>>>>>>> A dataframe is essentially a dictionary of time series data. >>>>>>>> >>>>>>>> I am new to web2py but I have experience with Pandas and matplotlib. >>>>>>>> I am also tempted to embed www.highcharts.com into my application. >>>>>>>> >>>>>>>> Before I dig into web2py I would like to know which route is >>>>>>>> probably most promising. >>>>>>>> >>>>>>>> Should I parse the dataframe on the webserver and write it into a >>>>>>>> SQL database? I guess that could be slow? >>>>>>>> Such a dataframe may consist of a dictionary with 100 elements each >>>>>>>> several 100k points. >>>>>>>> >>>>>>>> Should I parse a time series onto request into a json format and >>>>>>>> export to javaScript? >>>>>>>> In this case how could I provide a way to generate a menu from the >>>>>>>> keys in the dictionary. >>>>>>>> E.g. user clicks on a key, python does all the computations for >>>>>>>> some stats. >>>>>>>> >>>>>>>> The plan is to upload the data using csv files. >>>>>>>> >>>>>>>> So, I am a bit lost by the wide range of possibilities in web2py. I >>>>>>>> would be delighted if you would like to get involved in this open >>>>>>>> source >>>>>>>> project. >>>>>>>> The main goal for now is to learn web2py :-) >>>>>>>> >>>>>>>> Please find the Github of the original application here >>>>>>>> https://github.com/tschm/**Panda**sMonitor<https://github.com/tschm/PandasMonitor> >>>>>>>> >>>>>>>> Sorry for being so unprecise in my questions but it just reflects >>>>>>>> that I don't have a very precise plan at this stage. >>>>>>>> >>>>>>>> Kind regards >>>>>>>> Thomas >>>>>>>> >>>>>>> -- >>>>> Resources: >>>>> - http://web2py.com >>>>> - http://web2py.com/book (Documentation) >>>>> - http://github.com/web2py/**web2py >>>>> <http://github.com/web2py/web2py>(Source code) >>>>> - >>>>> https://code.google.com/p/**web2py/issues/list<https://code.google.com/p/web2py/issues/list>(Report >>>>> Issues) >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "web2py-users" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to web2py+un...@**googlegroups.com. >>>>> >>>>> For more options, visit >>>>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >>>>> . >>>>> >>>> >>>> -- >>> Resources: >>> - http://web2py.com >>> - http://web2py.com/book (Documentation) >>> - http://github.com/web2py/web2py (Source code) >>> - https://code.google.com/p/web2py/issues/list (Report Issues) >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "web2py-users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected] <javascript:>. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >> >> > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
