> On June 9, 2013, 11:11 p.m., Rohini Palaniswamy wrote: > > trunk/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java, line 309 > > <https://reviews.apache.org/r/11772/diff/1/?file=302439#file302439line309> > > > > What is the purpose of post upgrade? Why is it called in both upgradeDB > > ( via upgradeDBTo40) and also in postUpgradeDB ( via postUpgradeDBTo40). It > > also has code previous to 4.0. Can you check with Tucu as to what should be > > in upgrade and what goes in post upgrade. Do we need postUpgrade at all?
my understanding is that, postUpgradeTask/ddltweaks exists to do some changes which JPA mappingTool doesn't take care of in createUpgradeDB. so far my findings after couple of tests are - in fresh install, mappingTool automatically map string to varchar properly when creating db. - but in upgrade (LOb->varchar), when column has been text/mediumtext (in mysql), clob(derby), mappingTool does NOT automatically change it to varchar, and leave as it is (and the conservative way makes sense to avoid silently losing some of data by bigger->small column conversion). the same applies to dropping column, hence 'alter table' SQL need to be added to oozieDBCLI's postupgrade might be good to merge postupgradetask/ddlTweaks. > On June 9, 2013, 11:11 p.m., Rohini Palaniswamy wrote: > > trunk/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java, line 440 > > <https://reviews.apache.org/r/11772/diff/1/?file=302439#file302439line440> > > > > When you change from CLOB to varchar2 the column length should be 4000. > > It will not be possible to store the sla_xml with 255 bytes. In case of > > Oracle, you do not have to specify the length. The default will be 4K. But > > anyways to be consistent among all vendors we can specify 4K length. Please > > keep it same in the upgrade script as what @Column(length=4000) produces > > during fresh install for different databases. yes, that's what I thought too and put 255 temporarily, now uploading patch with 4000, (wf_actions.action, error_message also 4000, 2k chars check in setData/setErrorinfo changed to 4k, correct?) - Ryota ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11772/#review21630 ----------------------------------------------------------- On June 9, 2013, 9:56 p.m., Ryota Egashira wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/11772/ > ----------------------------------------------------------- > > (Updated June 9, 2013, 9:56 p.m.) > > > Review request for oozie. > > > Description > ------- > > https://issues.apache.org/jira/browse/OOZIE-1398 > > trying to solve following items > > 1 convert following to varchar/varchar2 on DB > > COORD_JOBS.SLA_XML > COORD_ACTIONS.SLA_XML > WF_JOBS.SLA_XML > WF_ACTIONS.SLA_XML > > WF_ACTIONS.ERROR_MESSAGE (Can add logic to truncate and store) > WF_ACTIONS.DATA (Max length for storing to this column is set to 2K in code > in places where it is populated with action data) > > 2. remove AUTH_token from DB and codebase, since not used. > > Candidates for removal: > BUNDLE_JOBS.AUTH_TOKEN > COORD_JOBS.AUTH_TOKEN > WORKFLOW_JOBS.AUTH_TOKEN > > > This addresses bug OOZIE-1398. > https://issues.apache.org/jira/browse/OOZIE-1398 > > > Diffs > ----- > > trunk/core/src/main/java/org/apache/oozie/BaseEngine.java 1490887 > trunk/core/src/main/java/org/apache/oozie/BundleEngine.java 1490887 > trunk/core/src/main/java/org/apache/oozie/BundleJobBean.java 1490887 > trunk/core/src/main/java/org/apache/oozie/CoordinatorActionBean.java > 1490887 > trunk/core/src/main/java/org/apache/oozie/CoordinatorEngine.java 1490887 > trunk/core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 1490887 > trunk/core/src/main/java/org/apache/oozie/DagEngine.java 1490887 > trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1490887 > trunk/core/src/main/java/org/apache/oozie/WorkflowJobBean.java 1490887 > > trunk/core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java > 1490887 > > trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java > 1490887 > > trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleStartXCommand.java > 1490887 > > trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleSubmitXCommand.java > 1490887 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java > 1490887 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java > 1490887 > > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java > 1490887 > trunk/core/src/main/java/org/apache/oozie/command/wf/ReRunXCommand.java > 1490887 > > trunk/core/src/main/java/org/apache/oozie/command/wf/SubmitHiveXCommand.java > 1490887 > > trunk/core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java > 1490887 > trunk/core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java > 1490887 > trunk/core/src/main/java/org/apache/oozie/command/wf/SubmitPigXCommand.java > 1490887 > > trunk/core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java > 1490887 > trunk/core/src/main/java/org/apache/oozie/command/wf/SubmitXCommand.java > 1490887 > trunk/core/src/main/java/org/apache/oozie/local/LocalOozie.java 1490887 > trunk/core/src/main/java/org/apache/oozie/service/BundleEngineService.java > 1490887 > > trunk/core/src/main/java/org/apache/oozie/service/CoordinatorEngineService.java > 1490887 > trunk/core/src/main/java/org/apache/oozie/service/DagEngineService.java > 1490887 > > trunk/core/src/main/java/org/apache/oozie/service/LiteWorkflowAppService.java > 1490887 > trunk/core/src/main/java/org/apache/oozie/service/RecoveryService.java > 1490887 > trunk/core/src/main/java/org/apache/oozie/service/WorkflowAppService.java > 1490887 > trunk/core/src/main/java/org/apache/oozie/servlet/BaseJobsServlet.java > 1490887 > trunk/core/src/main/java/org/apache/oozie/servlet/JsonRestServlet.java > 1490887 > trunk/core/src/main/java/org/apache/oozie/servlet/V0JobServlet.java 1490887 > trunk/core/src/main/java/org/apache/oozie/servlet/V0JobsServlet.java > 1490887 > trunk/core/src/main/java/org/apache/oozie/servlet/V1JobServlet.java 1490887 > trunk/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java > 1490887 > trunk/core/src/main/java/org/apache/oozie/servlet/V2JobServlet.java 1490887 > trunk/core/src/main/java/org/apache/oozie/store/CoordinatorStore.java > 1490887 > trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1490887 > trunk/core/src/test/java/org/apache/oozie/TestBundleJobBean.java 1490887 > trunk/core/src/test/java/org/apache/oozie/TestCoordinatorEngine.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java > 1490887 > trunk/core/src/test/java/org/apache/oozie/TestCoordinatorJobBean.java > 1490887 > trunk/core/src/test/java/org/apache/oozie/TestDagEngine.java 1490887 > trunk/core/src/test/java/org/apache/oozie/TestWorkflowBean.java 1490887 > > trunk/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java > 1490887 > trunk/core/src/test/java/org/apache/oozie/command/TestPurgeXCommand.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/command/bundle/TestBundleJobSuspendXCommand.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/command/bundle/TestBundleKillXCommand.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/command/bundle/TestBundleStartXCommand.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/command/bundle/TestBundleSubmitXCommand.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordActionStartXCommand.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/command/coord/TestFutureActionsTimeOut.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/command/coord/TestPastActionsTimeOut.java > 1490887 > trunk/core/src/test/java/org/apache/oozie/command/wf/TestActionErrors.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/command/wf/TestSubmitXCommand.java > 1490887 > trunk/core/src/test/java/org/apache/oozie/event/TestEventGeneration.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestBulkUpdateInsertForCoordActionStartJPAExecutor.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestBulkUpdateInsertForCoordActionStatusJPAExecutor.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestBulkUpdateInsertJPAExecutor.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobsGetForPurgeJPAExecutor.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobsToBeMaterializedJPAExecutor.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowIdGetForExternalIdJPAExecutor.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowJobsGetForPurgeJPAExecutor.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/service/TestActionCheckerService.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/service/TestAuthorizationService.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/service/TestCoordMaterializeTriggerService.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/service/TestLiteWorkflowAppService.java > 1490887 > trunk/core/src/test/java/org/apache/oozie/service/TestPurgeService.java > 1490887 > trunk/core/src/test/java/org/apache/oozie/service/TestRecoveryService.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/service/TestStatusTransitService.java > 1490887 > > trunk/core/src/test/java/org/apache/oozie/servlet/MockCoordinatorEngineService.java > 1490887 > trunk/core/src/test/java/org/apache/oozie/servlet/MockDagEngineService.java > 1490887 > trunk/core/src/test/java/org/apache/oozie/servlet/TestJobsServlet.java > 1490887 > trunk/core/src/test/java/org/apache/oozie/servlet/TestV1JobsServlet.java > 1490887 > trunk/core/src/test/java/org/apache/oozie/sla/TestSLAEventGeneration.java > 1490887 > trunk/core/src/test/java/org/apache/oozie/store/TestDBWorkflowStore.java > 1490887 > trunk/core/src/test/java/org/apache/oozie/test/XDataTestCase.java 1490887 > trunk/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java 1490887 > > Diff: https://reviews.apache.org/r/11772/diff/ > > > Testing > ------- > > Still WIP in terms of test/small bug fix, > but most of code change done, uploading initial patch. > > I did following tests so far > - run example workflow/coordinator, which works fine(tested because lots of > code changes to remove auth_token) > - DB upgrade test (ooziedb.sh upgrade) from 3.1 to 3.3 to 4.1 > - derby (hit one issue) > - mysql (works fine) > > currently doing oracle test. > > derby issue is that, it doesn't allow change column type from CLOB to VARCHAR > by 'ALTER TABLE COORD_JOBS ALTER COLUMN sla_xml SET DATA TYPE VARCHAR(255)" > > > Thanks, > > Ryota Egashira > >
