Re: [MTT devel] MTT Database and Reporter Upgrade **Action Required**

2007-08-26 Thread Josh Hursey
Jeff and Ethan, How do I shutdown the MTT server? Do I just log into milliways as mtt, or can I disable it from my user? Cheers, -- Josh Begin forwarded message: From: Josh Hursey <jjhur...@open-mpi.org> Date: August 24, 2007 1:37:18 PM EDT To: General user list for the MPI Testin

Re: [MTT devel] --trial pruning for v3 schema?

2007-08-29 Thread Josh Hursey
yeah I think we should be ok for now leving the trial results in the db. Let's revisit this in the future if things start to slow down again. -- Josh On Aug 28, 2007, at 3:28 PM, Jeff Squyres wrote: Let's see how the speed goes as we keep accumulating trials. If it becomes a problem, we

Re: [MTT devel] Thoughts on tagging...

2007-08-29 Thread Josh Hursey
Humm. How do we deal with row # now that we don't have temporary tables? I remember having to hack around this a bit to get it to work. What I was initially thinking was that we would tag each row with it's corresponding triplet [mpi_install_id, test_build_id, test_run_id] then use the

Re: [MTT devel] reporter.php

2007-08-30 Thread Josh Hursey
I donno. But I would favor the index.php route as well. Also FYI I'm working on the submit.php script a bit this morning. I'm doing 2 things: - Cleaning up error messages - Inserting a transaction to solve a race condition that was triggered last night by an IU submit. -- Josh On Aug

Re: [MTT devel] reporter.php

2007-08-30 Thread Josh Hursey
FYI: I finished cleaning up the PHP submit script. It seems that there are some associated issues with the stats PHP script. I'm cleaning that up at the moment. -- Josh On Aug 30, 2007, at 9:47 AM, Josh Hursey wrote: I donno. But I would favor the index.php route as well. Also FYI I'm

Re: [MTT devel] reporter.php

2007-08-30 Thread Josh Hursey
-mpi.org/mtt/stats/index.php - I think this one is for Jeff. -- Josh On Aug 30, 2007, at 10:37 AM, Jeff Squyres wrote: K. I made the switch index.php <--> reporter.php. On Aug 30, 2007, at 10:33 AM, Josh Hursey wrote: FYI: I finished cleaning up the PHP submit

Re: [MTT devel] UH in summary stats?

2007-08-30 Thread Josh Hursey
I haven't updated them in a couple days (since monday actually). I was working on updateing the script so it can run once a day. Should be done later tonight sometime. So that's why UH isn't in there yet. -- josh On Aug 30, 2007, at 3:35 PM, Jeff Squyres wrote: Shouldn't UH now show up

Re: [MTT devel] No nightly summary last night?

2007-08-30 Thread Josh Hursey
Humm. I think Ethan has this in his crontab and should be able to check if it is giving errors. I'm not currently subscribed to the nightly emails, I probably should do that again. -- Josh On Aug 30, 2007, at 3:30 PM, Jeff Squyres wrote: Come to think of it, I didn't get mails this

Re: [MTT devel] UH in summary stats?

2007-08-30 Thread Josh Hursey
-- it's a hard-coded list of org's. I thought it got them from the db. No rush -- I just saw it today and was curious. On Aug 30, 2007, at 4:32 PM, Josh Hursey wrote: I haven't updated them in a couple days (since monday actually). I was working on updateing the script so it can run once a day

Re: [MTT devel] change description in db

2007-08-30 Thread Josh Hursey
Pretty trivial to do. Just let me know when you want me to do it, and I can run the update. On Aug 30, 2007, at 7:19 PM, Jeff Squyres wrote: Josh -- How hard would it be to go change all the existing bakeoff descriptions in the database to be "[testbake]" in order to distinguish them from

Re: [MTT devel] [MTT users] Database submit error

2007-08-31 Thread Josh Hursey
Sounds good. Cleaning up now. Cheers, Josh On Aug 31, 2007, at 1:38 PM, Jeff Squyres wrote: No objections. If the data is junk, just ditch it. On Aug 31, 2007, at 12:47 PM, Josh Hursey wrote: I was looking at the data from Monday Aug 27, 8 am to Tuesday Aug 28, Noonish when this problem

Re: [MTT devel] MTT server error (user: sun)

2007-09-17 Thread Josh Hursey
I've been getting quite a few errors from submit.php of the below form. It seems that the MPI Install that is being referenced is not valid. Could this is an 'already installed' issue where the user is trying to submit results to the database without having submitted an mpi_install? --

Re: [MTT devel] MTT server error (user: sun)

2007-09-19 Thread Josh Hursey
: On Mon, Sep/17/2007 10:46:45AM, Josh Hursey wrote: I've been getting quite a few errors from submit.php of the below form. It seems that the MPI Install that is being referenced is not valid. Could this is an 'already installed' issue where the user is trying to submit results

Re: [MTT devel] MTT server error (user: cisco)

2007-09-22 Thread Josh Hursey
So I got two of these emails this morning. Any thoughts on why these 2 latency/bandwidth submissions are failing? The problem is with: SQL ERROR: ERROR: malformed array literal: "{136.36,124.59,124.58,124.61,124.74,124.52,124.57,137.98,124.62,124.69, ,

Re: [MTT devel] Fwd: MTT server error (user: iu)

2007-10-04 Thread Josh Hursey
, at 5:59 PM, Josh Hursey wrote: I have recently been receiving a ton of errors like the following. I traced it as best I could to an 'Already Installed' configuration that Tim M is running on BigRed for the collectives bakeoff. Is it possible that the AlreadyInstalled is not even trying to send any

Re: [MTT devel] MTT server error (user: cisco)

2007-10-07 Thread Josh Hursey
FYI: I just got this error from MTT submit.php. It seems that the message_size array is a bit corrupted. On Oct 7, 2007, at 11:25 AM, jjhur...@open-mpi.org wrote: SQL QUERY: INSERT INTO latency_bandwidth (latency_bandwidth_id, message_size, latency_min, latency_avg, latency_max,

Re: [MTT devel] Adding libnbc performance Performance Analyze module for MTT

2007-10-08 Thread Josh Hursey
I'm inclined to say 'let's just do it right the first time' meaning let's implement the 3D table we designed a few weeks ago. This would allow for arbitrary storage of performance data. But would take a considerable amount of effort. However, it would probably be quicker to implement a NBC

Re: [MTT devel] MTT server error (user: uh)

2007-10-15 Thread Josh Hursey
I'm getting the following from 'uh'. The problem is that they supply '-np' with no argument. The submit script is rejecting the submit, so the database is fine. I think this is a user problem, but it kind of looks like a client problem. Thought I would post to see if anyone has any

[MTT devel] MTT Visualization

2008-01-10 Thread Josh Hursey
I met with Joseph Cottam (Grad student in my lab at IU) yesterday about MTT visualization. He is working on some new visualization techniques and wants to apply them to the MTT dataset. Since we are ramping up to a v1.3 release we want to visualization to support this effort. So we want to

Re: [MTT devel] [OMPI devel] Cisco MTT runs

2008-01-10 Thread Josh Hursey
: On Jan 8, 2008, at 10:45 AM, Josh Hursey wrote: (Switching to MTT devel, forgot that we were mailing to OMPI devel before :/) Ah. Yeah that makes sense we can allow folks to add some custom, static text to the reporter page. Are we talking just a header and a footer for now? I think that would

Re: [MTT devel] mtt reporter popups broken

2008-01-10 Thread Josh Hursey
Fixed. This was a latent bug that my commit highlighted, but I didn't check for (Sorry). If mtt_body_html_suffix was non-null then the popups would not work. I just fixed this in r1133. -- Josh On Jan 10, 2008, at 12:26 PM, Jeff Squyres wrote: I notice that I can no longer get the MTT

Re: [MTT devel] Database Notice

2008-01-30 Thread Josh Hursey
the database for a few hours to do so. -- Josh On Jan 9, 2008, at 4:27 PM, Ethan Mallove wrote: "pg_dump -s" seems to show that we are set until 2009? (Just put a note in my calendar about this for late December 2008 :-)) On Wed, Jan/09/2008 04:07:01PM, Josh Hursey wrote: I was s

[MTT devel] Reporter Slowness

2008-01-30 Thread Josh Hursey
It seems the reporter has gotten slower :( Now it is working in the range of 40 - 50 seconds for the 24 hour query which is not reasonable. This should be much lower. Looking at the explain of the query I have some ideas on how to make things better, but this will slow things down a for a

Re: [MTT devel] Reporter Slowness

2008-01-31 Thread Josh Hursey
patience, and let me know if it feels sluggish again. So as of this email things should be back to normal. Cheers, Josh On Jan 30, 2008, at 5:09 PM, Josh Hursey wrote: I've started the script running. Below is a short version, and a trilogy of the gory details. I wanted to write up the details

Re: [MTT devel] Extracting transparent data from OMPI

2008-02-06 Thread Josh Hursey
On Feb 6, 2008, at 11:32 AM, Ethan Mallove wrote: For the configure options we *could* parse the config.log to extract this data. The question is, if we did this, what do we want to look? And is this something we want to do? Is there another way? I think having a network-like field

[MTT devel] Weird MTT test names

2008-03-06 Thread Josh Hursey
I'm trying to cleanup the test_suite/test_name tables in the database, removing some test names that are invalid and unreferenced. In the process I found a whole series of test_names of the following form: test_suite = trivial test_name = MXIQNeaG8e-ping For this particular one it points

[MTT devel] Contributions down

2008-04-15 Thread Josh Hursey
Looking at the MTT Contribution graph it seems that the number of testing being submitted has dropped off quite significantly in the past couple weeks: http://www.open-mpi.org/mtt/stats/mtt-contrib.pdf Is this something to be concerned about? -- Josh

Re: [MTT devel] Contributions down

2008-04-15 Thread Josh Hursey
deadlines have not enabled me to fully fix them yet... On Apr 15, 2008, at 9:37 AM, Josh Hursey wrote: Looking at the MTT Contribution graph it seems that the number of testing being submitted has dropped off quite significantly in the past couple weeks: http://www.open-mpi.org/mtt/stats/mtt

Re: [MTT devel] [MTT bugs] [MTT] #355: tooltips for reporter

2008-04-21 Thread Josh Hursey
Isn't this just setting the 'alt' field of the tag? Even without javascript this is just an HTML tooltip trick. I suspect that Mail would know how to deal with this even if it has javascript turned off. Would that be enough? On Apr 21, 2008, at 2:50 PM, Jeff Squyres wrote: Ethan and I

[MTT devel] Tracking process stats

2008-04-22 Thread Josh Hursey
On the Open MPI teleconf this morning Rich mentioned that he was noticing odd memory usage. It got me thinking, would it be useful for MTT to track important aspects of the process such as memory use? Just a thought. I'm not exactly sure how we would report or record this, but if it is

Re: [MTT devel] Tracking process stats

2008-04-22 Thread Josh Hursey
I was thinking for very test as an extra bit or two of information. But we could try a test suite too, but we would have to think about what we would want to exercise in it. -josh On Apr 22, 2008, at 1:45 PM, Ethan Mallove wrote: On Tue, Apr/22/2008 01:35:06PM, Josh Hursey wrote

Re: [MTT devel] bogus timestamps in database

2008-07-21 Thread Josh Hursey
Sounds good to me. On Jul 21, 2008, at 9:52 AM, Jeff Squyres wrote: Sorry for the late reply -- how about 3pm today (Monday)? On Jul 18, 2008, at 3:55 PM, Ethan Mallove wrote: I'm basically available except for 2-4p on Friday. -Ethan On Fri, Jul/18/2008 10:02:10AM, Josh Hursey wrote

Re: [MTT devel] bogus timestamps in database

2008-07-22 Thread Josh Hursey
Mallove wrote: I'm basically available except for 2-4p on Friday. -Ethan On Fri, Jul/18/2008 10:02:10AM, Josh Hursey wrote: Yeah it might be good to touch base and see if there are any burning fires that we need to put out. For me it would have to be either next week or most likely the end

Re: [MTT devel] Time to make the 2009 mtt database partitions?

2008-12-01 Thread Josh Hursey
Good catch. I have a reminder email set to be sent out later this month regarding this so we don't hit the same problem we had last year. But we should probably do it now so we don't forget later. There is a script here: server/sql/support/create-partitions.sh This will generate SQL files

Re: [MTT devel] visual reports for mtt

2009-02-24 Thread Josh Hursey
On Feb 24, 2009, at 8:08 AM, Jeff Squyres wrote: On Feb 24, 2009, at 1:02 AM, Mike Dubman wrote: I`m looking for a way having automatic regression report at the end of mtt run which include graph+table for bw/lat/2way-bw for this specific run as well as for previous runs on the same

Re: [MTT devel] GSOC apps now open

2009-03-10 Thread Josh Hursey
Yeah I have some time to dedicate do this. We should talk about a couple of specific topics to propose from this list we posted on the wiki. I can start digging in later this evening/tomorrow morning. -- Josh On Mar 10, 2009, at 10:42 AM, Jeff Squyres wrote: Google Summer of Code

Re: [MTT devel] GSOC application

2009-03-12 Thread Josh Hursey
Jeff is going to take the first pass at the application. I am going to go through the Idea page on the wiki and polish a bit: https://svn.open-mpi.org/trac/mtt/wiki/MttNewFeaturesIdeas I'll let folks know when I'm done, and we can start iterating on drafts. Cheers, Josh On Mar 12, 2009, at

Re: [MTT devel] GSOC application

2009-03-13 Thread Josh Hursey
the text up as necessary). The application is due by 3 pm EST. So I hope to have the application ready by 2ish. I'll move onto the application itself now. -- Josh On Mar 12, 2009, at 4:43 PM, Josh Hursey wrote: Jeff is going to take the first pass at the application. I am going to go through

Re: [MTT devel] GSOC application

2009-03-13 Thread Josh Hursey
I just pushed a final draft to the repository. I'll probably plan on submitting at 2:30/2:45. Let me know if you have any edits before then either through email or IM. Cheers, Josh On Mar 13, 2009, at 12:11 PM, Josh Hursey wrote: I finished a first pass at cleaning up the Ideas page

Re: [MTT devel] GSOC application

2009-03-13 Thread Josh Hursey
The application has been submitted. We find out on March 18 (3 pm) if we have been accepted. Link to timeline below: http://socghop.appspot.com/document/show/program/google/gsoc2009/ timeline Cheers, Josh On Mar 13, 2009, at 2:19 PM, Josh Hursey wrote: I just pushed a final draft

Re: [MTT devel] GSOC application

2009-03-18 Thread Josh Hursey
will forward it along. Cheers, Josh On Mar 13, 2009, at 3:19 PM, Jeff Squyres wrote: Awesome; many thanks for carrying the baton over the finish line, Josh! On Mar 13, 2009, at 2:56 PM, Josh Hursey wrote: The application has been submitted. We find out on March 18 (3 pm) if we have been accepted

Re: [MTT devel] [MTT svn] svn:mtt-svn r1273 (Analyze/Performance plug-ins)

2009-03-20 Thread Josh Hursey
A while back we designed what needed to happen on the database side to support arbitrary performance data (we were thinking about Skampi at the time, I believe). The only documentation that I could find of this discussion is in the DB schema diagram:

Re: [MTT devel] GSOC application

2009-03-20 Thread Josh Hursey
Yeah I think this sounds like a good way to move forward with this work. The database schema is pretty complex. If you need help on the database side of things let me know. To get started, would it be useful to have a meeting over the phone/ telepresence to design the datastore layout? This

Re: [MTT devel] GSOC application

2009-03-23 Thread Josh Hursey
save, load, query http://code.google.com/appengine/docs/python/gettingstarted/usingdatastore.html please comment. Thanks Mike On Fri, Mar 20, 2009 at 5:38 PM, Jeff Squyres <jsquy...@cisco.com> wrote: On Mar 20, 2009, at 10:42 AM, Josh Hursey wrote: Yeah I think this sounds like a

Re: [MTT devel] GSOC application

2009-04-15 Thread Josh Hursey
I have been listening in on the thread, but have not had time to really look at much (which is why I have not been replying). I'm interested in listening in on the teleconf as well, though if I become a blocker for finding a time feel free to cut me out. Best, Josh On Apr 14, 2009, at

Re: [MTT devel] [MTT svn] svn:mtt-svn r1400

2011-08-18 Thread Josh Hursey
Good database side stuff coming soon. Stay tuned :) On Thu, Aug 18, 2011 at 1:21 PM, Jeff Squyres wrote: > Awesome.  :-) > > On Aug 18, 2011, at 1:20 PM, jjhur...@osl.iu.edu wrote: > >> Author: jjhursey >> Date: 2011-08-18 13:20:35 EDT (Thu, 18 Aug 2011) >> New Revision: 1400

[MTT devel] Open MPI MTT Issue

2011-08-26 Thread Josh Hursey
Due to a unforeseen problem with an upgrade to the Open MPI MTT Reporter yesterday, all testing data submitted between yesterday at 9:30 am EDT to today at 9:45 am EDT was not collected by the Open MPI MTT database. The problem was identified, and fixed this morning by the Open MPI MTT

Re: [MTT devel] duplicate results

2012-01-06 Thread Josh Hursey
Weird. I don't know what is going on here, unless the client is somehow submitting some of the results too many times. One thing to check is the debug output file that the MTT client is submitting to the server. Check that for duplicates. That will help determine whether this is a server side

Re: [MTT devel] duplicate results

2012-01-09 Thread Josh Hursey
:07 PM, Eugene Loh <eugene@oracle.com> wrote: > On 01/06/12 08:52, Josh Hursey wrote: > > Weird. I don't know what is going on here, unless the client is somehow > submitting some of the results too many times. One thing to check is the > debug output file that the MTT

Re: [MTT devel] PHP5 compatibility patch applied

2012-05-30 Thread Josh Hursey
Awesome! Thanks for the fix. -- Josh On Wed, May 30, 2012 at 12:03 AM, Christopher Yeoh wrote: > Hi, > > I have applied the PHP5 compatibility patch to trunk (from ticket #389). > Note that this means the reporter code will no longer work on PHP4. > > Regards, > > Chris > --

Re: [MTT devel] PHP5 compatibility patch applied

2012-05-30 Thread Josh Hursey
s://mtt.open-mpi.org/ rather than > http[s]://www.open-mpi.org/mtt/.  So we'll all need to update our MTT .ini > files when the time comes. > > I'll let you know the timing of the change when we figure it out; it'll > probably be in a week or two. > > > > On May 30, 2012,

Re: [MTT devel] MTT queries... problems

2012-05-30 Thread Josh Hursey
Weird. I was able to reproduce. I suspect that there is probably a bug somewhere in the uglier magic in the reporter. It is tricky the way the reporter handles multiple pages of results. Unfortunately, I don't think anyone understands that code any more (it -really- needs to be reworked). I'm not

Re: [MTT devel] Detail link error

2015-08-21 Thread Josh Hursey
Thanks for the report. This is a known problem with our aging Reporter interface and large queries. PHP has a memory limit that we set, and large queries sometimes exceed that. There is some work on a new Reporter that should avoid this issue. In the mean time, you need to reduce the size of the

[MTT devel] statement_timeout on MTT Reporter Queries

2015-11-30 Thread Josh Hursey
I just added a 60 second timeout on any query submitted to the MTT Reporter. Two commits for this (sorry I didn't go through the pull request mechanism - will do next time) [1][2] This was in response to, what we think, was a bot clicking around on the web form and submitting some very long

[MTT devel] GitHub Issue Cleanup

2016-05-13 Thread Josh Hursey
We are seeing more activity with MTT development, and there is a desire to push to a formal release at some point in the not-to-distant future. As such, I think it is time to clean up/out the issues on GitHub. Quite a number of those issues are feature ideas that we came up with that were never

Re: [MTT devel] GitHub Issue Cleanup

2016-05-18 Thread Josh Hursey
uple weeks ago we identified some development items for the short term. Can folks take point on filing those tickets so we can track their progress? I'll add the left-to-do items for the Server side. Label all those tickets that you need/want soon as v4.0. On Fri, May 13, 2016 at 1:08 PM, Josh Hur

[MTT devel] Ticket to finish the python support for the IU Database submissions

2016-06-22 Thread Josh Hursey
Per the teleconf today, I filed a ticket for the remaining todo items in the IUDatabase python module: https://github.com/open-mpi/mtt/issues/443 -- Josh

[MTT devel] Teleconf restart for 2017

2017-01-17 Thread Josh Hursey
When should we restart our weekly teleconfs for 2017? Maybe this week or the week of Jan 30? Next week is the OMPI face-to-face meeting so it might not be the best timing. -- Josh Hursey IBM Spectrum MPI Developer ___ mtt-devel mailing list mtt-devel

Re: [MTT devel] reporter error using pyclient

2016-08-23 Thread Josh Hursey
t Found > > > > Not Found > > The requested URL /submit//serial was not found on this server. > > > > Apache/2.2.15 (Red Hat) Server at mtt.open-mpi.org Port > 443 > > > > > Any ideas on how to proceed? > &g

[MTT devel] Performance Measurement

2016-09-14 Thread Josh Hursey
(next couple months). We can use the GitHub to track comments, but maybe carve out part of the weekly teleconf to talk about progress. -- Josh Hursey IBM Spectrum MPI Developer ___ mtt-devel mailing list mtt-devel@lists.open-mpi.org https

Re: [MTT devel] reporter error using pyclient

2016-09-07 Thread Josh Hursey
) > InternalError: current transaction is aborted, commands ignored until end > of transaction block > > > > > Powered by http://www.cherrypy.org;>CherryPy 5.1.0 > > > > > > <<<<<<<

Re: [MTT devel] reporter error using pyclient

2016-08-26 Thread Josh Hursey
; > It looks to me like the URL isn’t correctly set - yes? > > On Aug 26, 2016, at 7:10 AM, r...@open-mpi.org wrote: > > I checked it with my browser and got the expected return message > > > On Aug 25, 2016, at 1:46 PM, Josh Hursey <jjhur...@open-mpi.org> wrote: >

Re: [MTT devel] reporter error using pyclient

2016-08-29 Thread Josh Hursey
ting dropped > connection: mtt.open-mpi.org > DEBUG:requests.packages.urllib3.connectionpool:"POST > /submit/cpy/api/submit HTTP/1.1" 200 90 > <<<<<<< Response -->>>>>> > Result: 200: application/json > {'content-lengt

Re: [MTT devel] reporter error using pyclient

2016-08-25 Thread Josh Hursey
t=iso-8859-1', 'Connection': 'close', > 'Server': 'Apache/2.2.15 (Red Hat)'} > > Not Found > > <<<<<<<---- Raw Output (Start) >>>>>> > > > > > > 404 Not Found > > > > Not Found > &g

Re: [MTT devel] trying again to ppost results to "IU" using pyclient

2017-08-16 Thread Josh Hursey
gt;> Absolute URI not allowed if server is not a proxy. >> >> Also, what should be the format of the POST request to get a serial #? >> I'm suspicious that >> >> this area of the pyclient is buggy. >> >> Howard >> >> >> >> 2017-08-14 15:

[MTT devel] MTT / Open MPI Visibility of New Failures

2017-07-11 Thread Josh Hursey
eir "known to fail" list. If the "failed, but we expected to pass" number is >0 then this is a 'new failure' and an email to the community is generated. -- Josh Hursey IBM Spectrum MPI Developer ___ mtt-devel mailing list mtt-devel

Re: [MTT devel] MTT / Open MPI Visibility of New Failures

2017-07-19 Thread Josh Hursey
We are putting a pin in this for now. Amazon had an idea they wanted to run with and report back. Once they have more info then we'll try to setup another meeting. On Tue, Jul 11, 2017 at 3:20 PM, Josh Hursey <jjhur...@open-mpi.org> wrote: > In the Open MPI face-to-face meeting we h