Jinwei Zhu, Please lets move this discussion back to the JIRA, there is quite a bit of useful info.
In a production setup, you don't want a system to do DDL changes when it starts. Furthermore, in many setups admins create the DB offline using a DB admin user, the Oozie user does not have DDL rights. Thx PS: we use to DB creation automatically, and we still do it for testcases, the above reason is why we don't do it anymore. On Thu, Dec 20, 2012 at 7:18 AM, jinwei zhu <[email protected]> wrote: > Hi,Alejandro, > because the patch has some JPA changes, this caused "-1" in the test-patch > result , and if we submit this patch into trunk, it means we should bump up > trunk to v4, and most likely create a new dev branch-3. When oozie users > use v4, to be compatible with old workflow , they maybe need to upgrade > database manually. To avoid upgrading database manually,we can replace > this code " > > props.setProperty("openjpa.jdbc.SynchronizeMappings","buildSchema(ForeignKeys=true)");" > into "props.setProperty("openjpa.jdbc.SynchronizeMappings", > "buildSchema(ForeignKeys=true,SchemaAction=refresh)"); > "SchemaAction=refresh" means JPA will update database according to our > entity automaticlly . > > Thx! > > > > 2012/12/20 Alejandro Abdelnur <[email protected]>: > > Thanks for the clarification, still I don't understand why you want to > > modify JPA settings for this. This should be done as part of a command > > queued everything the WF state changes. > > > > Cheers > > > > > > On Thu, Dec 20, 2012 at 2:48 AM, jinwei zhu <[email protected]> > wrote: > > > >> Hi,Alejandro, > >> 1. the patch computes total Actions num that will run in one workflow > >> first, and then use "lastActionNumber * 1.0f / totalActionNumber" to > >> compute the progress , the "lastActionNumber" is the number of actions > >> that have done. "totalActionNumber" will be persisted into db. > >> 2. the patch does not compute the progress by getting job progress > >> reporting from the cluster. > >> > >> for example, > >> if the workflow has 4 actions, then the progress will be shown like " > >> 0, 25, 50, 75, 100%"; > >> if the workflow has only 1 action, then the progress will be shown > >> like " 0, 100%"; > >> > >> Thx > >> > >> > >> > >> 2012/12/20 Alejandro Abdelnur <[email protected]>: > >> > Jinwei Zhu, > >> > > >> > I think you are confused, job progress reporting from the cluster has > >> > nothing to do with JPA. > >> > > >> > Thx > >> > > >> > On Wed, Dec 19, 2012 at 8:48 AM, jinwei zhu <[email protected]> > >> wrote: > >> > > >> >> hi, Alejandro, I think we can change this code in JPAService.java: > >> >> props.setProperty("openjpa.jdbc.SynchronizeMappings", > >> >> "buildSchema(ForeignKeys=true)"); > >> >> into > >> >> props.setProperty("openjpa.jdbc.SynchronizeMappings", > >> >> "buildSchema(ForeignKeys=true,SchemaAction=refresh)"); > >> >> > >> >> after changing, oozie can update db automaticlly if our entity > changed. > >> >> > >> >> > >> >> > >> >> 2012/12/18 Alejandro Abdelnur (JIRA) <[email protected]>: > >> >> > > >> >> > [ > >> >> > >> > > https://issues.apache.org/jira/browse/OOZIE-547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534877#comment-13534877 > >> ] > >> >> > > >> >> > Alejandro Abdelnur commented on OOZIE-547: > >> >> > ------------------------------------------ > >> >> > > >> >> > If we commit this patch to trunk, it means we should bump up trunk > to > >> >> v4, and most likely create a new dev branch-3 to introduce new > features > >> in > >> >> v3.4 and onwards. Would you mind starting the discussion in > >> [email protected]? > >> >> > > >> >> >> build workflow progress information in Oozie > >> >> >> -------------------------------------------- > >> >> >> > >> >> >> Key: OOZIE-547 > >> >> >> URL: > https://issues.apache.org/jira/browse/OOZIE-547 > >> >> >> Project: Oozie > >> >> >> Issue Type: New Feature > >> >> >> Reporter: Hadoop QA > >> >> >> Assignee: zhu jin wei > >> >> >> Attachments: oozie-547.patch > >> >> >> > >> >> >> > >> >> >> For a user, knowing progress of her workflow is always desirable. > >> This > >> >> ticket is to introduce this support to Oozie. > >> >> >> I know it's a hard problem. For my initial effort, I plan to start > >> with > >> >> simple workflows that do not contain decision nodes or fork/join > nodes, > >> >> i.e., chain type workflows. I plan to use percentage of finished > >> actions as > >> >> the overall wf progress estimate. > >> >> >> Going forward we can improve the estimation by: > >> >> >> 1) handle general workflows that contain decision, fork/join > nodes; > >> >> >> 2) incorporate the action level progress into wf level progress > >> >> estimation to make the estimate better. To be more specific: > >> >> >> In the case of "opaque" actions like pig/hive/jaql where the > status > >> can > >> >> only be 0% or 100% (or failure) we plug that value into the overall > DAG > >> >> status of 0-100%. If a DAG had say 4 opaque actions, the progress > would > >> >> move in discrete steps 0, 25, 50, 75, 100%. For the m/r actions > where > >> the > >> >> JobTracker > >> >> >> gives values between 0-100% for an action then the overall > progress > >> >> will be smoother. We can do same thing for pig/hive/jaql actions as > >> well if > >> >> they expose their own progress info. > >> >> > > >> >> > -- > >> >> > This message is automatically generated by JIRA. > >> >> > If you think it was sent incorrectly, please contact your JIRA > >> >> administrators > >> >> > For more information on JIRA, see: > >> >> http://www.atlassian.com/software/jira > >> >> > >> > > > > > > > > -- > > Alejandro >
