Hi, Alejandro, Yes, it is not a good idea that we want a system to do DDL changes once the systm starts. And meanwhile if oozie can not get the DDL rights, oozie can not do DDL changes. Since the patch has some JPA changes, is there any possibility for this patch to commit into trunk? If yes, when this patch could be commited ?
Thx! 2012/12/20 Alejandro Abdelnur <[email protected]> > Jinwei Zhu, > > Please lets move this discussion back to the JIRA, there is quite a bit of > useful info. > > In a production setup, you don't want a system to do DDL changes when it > starts. Furthermore, in many setups admins create the DB offline using a DB > admin user, the Oozie user does not have DDL rights. > > Thx > > PS: we use to DB creation automatically, and we still do it for testcases, > the above reason is why we don't do it anymore. > > On Thu, Dec 20, 2012 at 7:18 AM, jinwei zhu <[email protected]> wrote: > > > Hi,Alejandro, > > because the patch has some JPA changes, this caused "-1" in the > test-patch > > result , and if we submit this patch into trunk, it means we should bump > up > > trunk to v4, and most likely create a new dev branch-3. When oozie users > > use v4, to be compatible with old workflow , they maybe need to upgrade > > database manually. To avoid upgrading database manually,we can replace > > this code " > > > > > props.setProperty("openjpa.jdbc.SynchronizeMappings","buildSchema(ForeignKeys=true)");" > > into "props.setProperty("openjpa.jdbc.SynchronizeMappings", > > "buildSchema(ForeignKeys=true,SchemaAction=refresh)"); > > "SchemaAction=refresh" means JPA will update database according to our > > entity automaticlly . > > > > Thx! > > > > > > > > 2012/12/20 Alejandro Abdelnur <[email protected]>: > > > Thanks for the clarification, still I don't understand why you want to > > > modify JPA settings for this. This should be done as part of a command > > > queued everything the WF state changes. > > > > > > Cheers > > > > > > > > > On Thu, Dec 20, 2012 at 2:48 AM, jinwei zhu <[email protected]> > > wrote: > > > > > >> Hi,Alejandro, > > >> 1. the patch computes total Actions num that will run in one workflow > > >> first, and then use "lastActionNumber * 1.0f / totalActionNumber" to > > >> compute the progress , the "lastActionNumber" is the number of actions > > >> that have done. "totalActionNumber" will be persisted into db. > > >> 2. the patch does not compute the progress by getting job progress > > >> reporting from the cluster. > > >> > > >> for example, > > >> if the workflow has 4 actions, then the progress will be shown like " > > >> 0, 25, 50, 75, 100%"; > > >> if the workflow has only 1 action, then the progress will be shown > > >> like " 0, 100%"; > > >> > > >> Thx > > >> > > >> > > >> > > >> 2012/12/20 Alejandro Abdelnur <[email protected]>: > > >> > Jinwei Zhu, > > >> > > > >> > I think you are confused, job progress reporting from the cluster > has > > >> > nothing to do with JPA. > > >> > > > >> > Thx > > >> > > > >> > On Wed, Dec 19, 2012 at 8:48 AM, jinwei zhu <[email protected]> > > >> wrote: > > >> > > > >> >> hi, Alejandro, I think we can change this code in JPAService.java: > > >> >> props.setProperty("openjpa.jdbc.SynchronizeMappings", > > >> >> "buildSchema(ForeignKeys=true)"); > > >> >> into > > >> >> props.setProperty("openjpa.jdbc.SynchronizeMappings", > > >> >> "buildSchema(ForeignKeys=true,SchemaAction=refresh)"); > > >> >> > > >> >> after changing, oozie can update db automaticlly if our entity > > changed. > > >> >> > > >> >> > > >> >> > > >> >> 2012/12/18 Alejandro Abdelnur (JIRA) <[email protected]>: > > >> >> > > > >> >> > [ > > >> >> > > >> > > > > > https://issues.apache.org/jira/browse/OOZIE-547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534877#comment-13534877 > > >> ] > > >> >> > > > >> >> > Alejandro Abdelnur commented on OOZIE-547: > > >> >> > ------------------------------------------ > > >> >> > > > >> >> > If we commit this patch to trunk, it means we should bump up > trunk > > to > > >> >> v4, and most likely create a new dev branch-3 to introduce new > > features > > >> in > > >> >> v3.4 and onwards. Would you mind starting the discussion in > > >> [email protected]? > > >> >> > > > >> >> >> build workflow progress information in Oozie > > >> >> >> -------------------------------------------- > > >> >> >> > > >> >> >> Key: OOZIE-547 > > >> >> >> URL: > > https://issues.apache.org/jira/browse/OOZIE-547 > > >> >> >> Project: Oozie > > >> >> >> Issue Type: New Feature > > >> >> >> Reporter: Hadoop QA > > >> >> >> Assignee: zhu jin wei > > >> >> >> Attachments: oozie-547.patch > > >> >> >> > > >> >> >> > > >> >> >> For a user, knowing progress of her workflow is always > desirable. > > >> This > > >> >> ticket is to introduce this support to Oozie. > > >> >> >> I know it's a hard problem. For my initial effort, I plan to > start > > >> with > > >> >> simple workflows that do not contain decision nodes or fork/join > > nodes, > > >> >> i.e., chain type workflows. I plan to use percentage of finished > > >> actions as > > >> >> the overall wf progress estimate. > > >> >> >> Going forward we can improve the estimation by: > > >> >> >> 1) handle general workflows that contain decision, fork/join > > nodes; > > >> >> >> 2) incorporate the action level progress into wf level progress > > >> >> estimation to make the estimate better. To be more specific: > > >> >> >> In the case of "opaque" actions like pig/hive/jaql where the > > status > > >> can > > >> >> only be 0% or 100% (or failure) we plug that value into the overall > > DAG > > >> >> status of 0-100%. If a DAG had say 4 opaque actions, the progress > > would > > >> >> move in discrete steps 0, 25, 50, 75, 100%. For the m/r actions > > where > > >> the > > >> >> JobTracker > > >> >> >> gives values between 0-100% for an action then the overall > > progress > > >> >> will be smoother. We can do same thing for pig/hive/jaql actions as > > >> well if > > >> >> they expose their own progress info. > > >> >> > > > >> >> > -- > > >> >> > This message is automatically generated by JIRA. > > >> >> > If you think it was sent incorrectly, please contact your JIRA > > >> >> administrators > > >> >> > For more information on JIRA, see: > > >> >> http://www.atlassian.com/software/jira > > >> >> > > >> > > > > > > > > > > > > -- > > > Alejandro > > >
