[jira] [Work logged] (TRAFODION-3314) OSIM generates redundant DDL for unique constraints
[
https://issues.apache.org/jira/browse/TRAFODION-3314?focusedWorklogId=271273=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-271273
]
ASF GitHub Bot logged work on TRAFODION-3314:
-
Author: ASF GitHub Bot
Created on: 02/Jul/19 23:12
Start Date: 02/Jul/19 23:12
Worklog Time Spent: 10m
Work Description: Traf-Jenkins commented on issue #1846: [TRAFODION-3314]
Avoid generating redundant DDL in OSIM for unique constraints
URL: https://github.com/apache/trafodion/pull/1846#issuecomment-507877640
Test Passed. https://jenkins.esgyn.com/job/Check-PR-master/3238/
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 271273)
Time Spent: 40m (was: 0.5h)
> OSIM generates redundant DDL for unique constraints
> ---
>
> Key: TRAFODION-3314
> URL: https://issues.apache.org/jira/browse/TRAFODION-3314
> Project: Apache Trafodion
> Issue Type: Improvement
> Components: sql-cmp
>Affects Versions: 2.4
>Reporter: David Wayne Birdsall
>Assignee: David Wayne Birdsall
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Unique constraints are usually implemented by creation of a unique index
> under the covers. When doing an OSIM CAPTURE of a query on a table that has a
> unique constraint, OSIM generates both an ALTER TABLE ADD CONSTRAINT
> statement and a CREATE UNIQUE INDEX statement in the CREATE_TABLE_DDLS.txt
> file. When this is replayed in OSIM LOAD, we get an error when creating the
> unique index.
> To fix this, perhaps OSIM should generate the CREATE UNIQUE INDEX statement
> only in commented-out form.
> {{To reproduce, run the following script in sqlci:}}
> {{?section setup}}
> {{drop table if exists t1 cascade;}}
> {{create table t1 (a int not null, b int not null, c int not null, primary
> key (a));}}
> {{alter table t1 add constraint uniqueb unique(b);}}
> {{?section osimgen}}
> {{control osim capture location 'osimgen';}}
> {{prepare xx from select b,count(*) from t1 group by b;}}
> {{control osim capture stop;}}
> {{Then in a separate session, do:}}
> {{control osim load from 'osimgen';}}
> {{This will fail with the following output:}}
> {{>>control osim load from '.';}}
> {{[OSIM]loading tables and views ...}}
> {{[OSIM]DROP TABLE IF EXISTS TRAFODION.SCH.T1 CASCADE;}}
> {{[OSIM]Step 1 Create Schemas:}}
> {{[OSIM]CREATE SCHEMA IF NOT EXISTS TRAFODION.SCH}}
> {{[OSIM]Step 2 Create Tables:}}
> {{[OSIM]CREATE TABLE TRAFODION.SCH.T1}}
> {{[OSIM] The following index is a system created index --CREATE UNIQUE INDEX
> UNIQUEB ON TRAFODION.SCH.T1}}
> {{[OSIM]ALTER TABLE TRAFODION.SCH.T1 ADD CONSTRAINT TRAFODION.SCH.UNIQUEB
> UNIQUE}}
> {{*** ERROR[1043] Constraint TRAFODION.SCH.UNIQUEB already exists.}}
> {{*** ERROR[6009] The Optimizer Simulator (OSIM): Create Table Error: -1043}}
> {{*** ERROR[8822] The statement was not prepared.}}
> {{>>exit;}}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (TRAFODION-3263) Disable LOB locking and refactor order of LOB iud expression evaluation
[ https://issues.apache.org/jira/browse/TRAFODION-3263?focusedWorklogId=271235=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-271235 ] ASF GitHub Bot logged work on TRAFODION-3263: - Author: ASF GitHub Bot Created on: 02/Jul/19 21:33 Start Date: 02/Jul/19 21:33 Worklog Time Spent: 10m Work Description: Traf-Jenkins commented on issue #1845: [TRAFODION-3263] and other misc fixes for lob locking and refactoring URL: https://github.com/apache/trafodion/pull/1845#issuecomment-507852837 Test Passed. https://jenkins.esgyn.com/job/Check-PR-master/3237/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 271235) Time Spent: 2h 10m (was: 2h) > Disable LOB locking and refactor order of LOB iud expression evaluation > > > Key: TRAFODION-3263 > URL: https://issues.apache.org/jira/browse/TRAFODION-3263 > Project: Apache Trafodion > Issue Type: Improvement > Components: sql-general >Affects Versions: 2.2.0 >Reporter: Sandhya Sundaresan >Assignee: Sandhya Sundaresan >Priority: Major > Time Spent: 2h 10m > Remaining Estimate: 0h > > The change to use JNI to do HDFS writes improved the interface by returning > more useful infomration to the caller. In TRAFODION-2946, we ddescribe the > need for LOB locking because of a condition where multiple threads writing to > the same LOB column could interleave and cause problems. TWith the new JNI > interface and HDFS write will now return the offset where the data was > written. So we can use this return offset to store in the descriptor tables. > Prior to this while using the libhdfs API, we would not get back the "written > offset". > > So the order of operations before this change used to be : > # Get the EOD for the LOB data file in HDFS > # Store this offset into the LOB descriptor tables so we know where to > retrieve the data from during a read. > # call hdfsWrite to write to the LOB data file. And hope that the offset > where the hdfsWrite writes is the same as the EOD calculated in 1. hdfs being > an "append only"file system, this is usually how it works. But if another > process comes in and does an insert into the LOB column between 2 and 3, then > we have an incorrect offset stored int he descriptor tables. Hence we added a > Lob Lock to make steps 1,2 and 3 atomic as part of Trafodion-2946 to address > this issue. > The order of operations with this change is as follows : > # Call JNI hdfs Write API to write the lob data to hdfs. > # Use return data offset from JNI hdfswrite API in 1. as the offset to store > in the LOB descriptor tables. > # If there are multiple chunks to write, do it in a loop and append to the > first chunk. This way each chunk can be anywhere in hdfs and not necessarily > continguous. But we are guaranteed that whatever we wrote will be stored in > our interna lLOB descriptor files. > # If any failure or TM error occurs while writing to the LOB descriptor > tables, transaction gets rolled back and the chunk of hdfs data written > becomes "dead data". It doesn't harm the next operation. > # GC check is now done before an update or insert. Earlier it was done as > part of the ::allocateDesc operation to get the EOD of the file. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (TRAFODION-3314) OSIM generates redundant DDL for unique constraints
[
https://issues.apache.org/jira/browse/TRAFODION-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on TRAFODION-3314 started by David Wayne Birdsall.
---
> OSIM generates redundant DDL for unique constraints
> ---
>
> Key: TRAFODION-3314
> URL: https://issues.apache.org/jira/browse/TRAFODION-3314
> Project: Apache Trafodion
> Issue Type: Improvement
> Components: sql-cmp
>Affects Versions: 2.4
>Reporter: David Wayne Birdsall
>Assignee: David Wayne Birdsall
>Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Unique constraints are usually implemented by creation of a unique index
> under the covers. When doing an OSIM CAPTURE of a query on a table that has a
> unique constraint, OSIM generates both an ALTER TABLE ADD CONSTRAINT
> statement and a CREATE UNIQUE INDEX statement in the CREATE_TABLE_DDLS.txt
> file. When this is replayed in OSIM LOAD, we get an error when creating the
> unique index.
> To fix this, perhaps OSIM should generate the CREATE UNIQUE INDEX statement
> only in commented-out form.
> {{To reproduce, run the following script in sqlci:}}
> {{?section setup}}
> {{drop table if exists t1 cascade;}}
> {{create table t1 (a int not null, b int not null, c int not null, primary
> key (a));}}
> {{alter table t1 add constraint uniqueb unique(b);}}
> {{?section osimgen}}
> {{control osim capture location 'osimgen';}}
> {{prepare xx from select b,count(*) from t1 group by b;}}
> {{control osim capture stop;}}
> {{Then in a separate session, do:}}
> {{control osim load from 'osimgen';}}
> {{This will fail with the following output:}}
> {{>>control osim load from '.';}}
> {{[OSIM]loading tables and views ...}}
> {{[OSIM]DROP TABLE IF EXISTS TRAFODION.SCH.T1 CASCADE;}}
> {{[OSIM]Step 1 Create Schemas:}}
> {{[OSIM]CREATE SCHEMA IF NOT EXISTS TRAFODION.SCH}}
> {{[OSIM]Step 2 Create Tables:}}
> {{[OSIM]CREATE TABLE TRAFODION.SCH.T1}}
> {{[OSIM] The following index is a system created index --CREATE UNIQUE INDEX
> UNIQUEB ON TRAFODION.SCH.T1}}
> {{[OSIM]ALTER TABLE TRAFODION.SCH.T1 ADD CONSTRAINT TRAFODION.SCH.UNIQUEB
> UNIQUE}}
> {{*** ERROR[1043] Constraint TRAFODION.SCH.UNIQUEB already exists.}}
> {{*** ERROR[6009] The Optimizer Simulator (OSIM): Create Table Error: -1043}}
> {{*** ERROR[8822] The statement was not prepared.}}
> {{>>exit;}}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (TRAFODION-3314) OSIM generates redundant DDL for unique constraints
[
https://issues.apache.org/jira/browse/TRAFODION-3314?focusedWorklogId=271199=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-271199
]
ASF GitHub Bot logged work on TRAFODION-3314:
-
Author: ASF GitHub Bot
Created on: 02/Jul/19 20:51
Start Date: 02/Jul/19 20:51
Worklog Time Spent: 10m
Work Description: Traf-Jenkins commented on issue #1846: [TRAFODION-3314]
Avoid generating redundant DDL in OSIM for unique constraints
URL: https://github.com/apache/trafodion/pull/1846#issuecomment-507839757
Check Test Started: https://jenkins.esgyn.com/job/Check-PR-master/3238/
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 271199)
Time Spent: 0.5h (was: 20m)
> OSIM generates redundant DDL for unique constraints
> ---
>
> Key: TRAFODION-3314
> URL: https://issues.apache.org/jira/browse/TRAFODION-3314
> Project: Apache Trafodion
> Issue Type: Improvement
> Components: sql-cmp
>Affects Versions: 2.4
>Reporter: David Wayne Birdsall
>Assignee: David Wayne Birdsall
>Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Unique constraints are usually implemented by creation of a unique index
> under the covers. When doing an OSIM CAPTURE of a query on a table that has a
> unique constraint, OSIM generates both an ALTER TABLE ADD CONSTRAINT
> statement and a CREATE UNIQUE INDEX statement in the CREATE_TABLE_DDLS.txt
> file. When this is replayed in OSIM LOAD, we get an error when creating the
> unique index.
> To fix this, perhaps OSIM should generate the CREATE UNIQUE INDEX statement
> only in commented-out form.
> {{To reproduce, run the following script in sqlci:}}
> {{?section setup}}
> {{drop table if exists t1 cascade;}}
> {{create table t1 (a int not null, b int not null, c int not null, primary
> key (a));}}
> {{alter table t1 add constraint uniqueb unique(b);}}
> {{?section osimgen}}
> {{control osim capture location 'osimgen';}}
> {{prepare xx from select b,count(*) from t1 group by b;}}
> {{control osim capture stop;}}
> {{Then in a separate session, do:}}
> {{control osim load from 'osimgen';}}
> {{This will fail with the following output:}}
> {{>>control osim load from '.';}}
> {{[OSIM]loading tables and views ...}}
> {{[OSIM]DROP TABLE IF EXISTS TRAFODION.SCH.T1 CASCADE;}}
> {{[OSIM]Step 1 Create Schemas:}}
> {{[OSIM]CREATE SCHEMA IF NOT EXISTS TRAFODION.SCH}}
> {{[OSIM]Step 2 Create Tables:}}
> {{[OSIM]CREATE TABLE TRAFODION.SCH.T1}}
> {{[OSIM] The following index is a system created index --CREATE UNIQUE INDEX
> UNIQUEB ON TRAFODION.SCH.T1}}
> {{[OSIM]ALTER TABLE TRAFODION.SCH.T1 ADD CONSTRAINT TRAFODION.SCH.UNIQUEB
> UNIQUE}}
> {{*** ERROR[1043] Constraint TRAFODION.SCH.UNIQUEB already exists.}}
> {{*** ERROR[6009] The Optimizer Simulator (OSIM): Create Table Error: -1043}}
> {{*** ERROR[8822] The statement was not prepared.}}
> {{>>exit;}}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (TRAFODION-3314) OSIM generates redundant DDL for unique constraints
[
https://issues.apache.org/jira/browse/TRAFODION-3314?focusedWorklogId=271196=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-271196
]
ASF GitHub Bot logged work on TRAFODION-3314:
-
Author: ASF GitHub Bot
Created on: 02/Jul/19 20:47
Start Date: 02/Jul/19 20:47
Worklog Time Spent: 10m
Work Description: DaveBirdsall commented on pull request #1846:
[TRAFODION-3314] Avoid generating redundant DDL in OSIM for unique constraints
URL: https://github.com/apache/trafodion/pull/1846#discussion_r299677737
##
File path: core/sql/optimizer/OptimizerSimulator.cpp
##
@@ -541,31 +541,43 @@ void OptimizerSimulator::dumpDDLs(const QualifiedName &
qualifiedName)
<<"."<< qualifiedName.getSchemaName()
<< ";" << endl;
+// skippingSystemGeneratedIndex is set to TRUE to avoid generating
redundant
+// DDL for system-generated indexes
+NABoolean skippingSystemGeneratedIndex = FALSE;
outQueue->position();//rewind
for (int i = 0; i < outQueue->numEntries(); i++) {
OutputInfo * vi = (OutputInfo*)outQueue->getNext();
char * ptr = vi->get(0);
-// skip heading newline, and add a comment line
-// for the DDL text upto the first trailing '\n'
-Int32 ix = 0;
-for(; ptr[ix]=='\n'; ix++);
-if( strstr(ptr, "CREATE TABLE") ||
-strstr(ptr, "CREATE INDEX") ||
-strstr(ptr, "CREATE UNIQUE INDEX") ||
-strstr(ptr, "ALTER TABLE") )
+if (strcmp(ptr,"\n-- The following index is a system created index
--") == 0)
+ skippingSystemGeneratedIndex = TRUE;
+if (!skippingSystemGeneratedIndex)
{
- (*createTable) << "--";
- char* x = ptr+ix;
- while ( (*x) && *x != '\n' ) {
-(*createTable) << *x;
-x++;
- }
- (*createTable) << endl;
+// skip heading newline, and add a comment line
+// for the DDL text upto the first trailing '\n'
+Int32 ix = 0;
+for(; ptr[ix]=='\n'; ix++);
+if( strstr(ptr, "CREATE TABLE") ||
+strstr(ptr, "CREATE INDEX") ||
+strstr(ptr, "CREATE UNIQUE INDEX") ||
+strstr(ptr, "ALTER TABLE") )
+
+{
+ (*createTable) << "--";
+ char* x = ptr+ix;
+ while ( (*x) && *x != '\n' ) {
+(*createTable) << *x;
+x++;
+ }
+ (*createTable) << endl;
+}
+
+//output ddl
+(*createTable) << ptr << endl;
}
-//output ddl
-(*createTable) << ptr << endl;
+if (skippingSystemGeneratedIndex && (strcmp(ptr,";") == 0)) // at
end of DDL to be skipped?
+ skippingSystemGeneratedIndex = FALSE;
Review comment:
Note to reviewers: The code from lines 556 through 576 in the new file is
merely re-indented lines 548 through 568 from the old file, with an "if
(!skippingSystemGenerated) { ... }" around it. Unfortunately the comparison
tool doesn't render this difference nicely.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 271196)
Time Spent: 20m (was: 10m)
> OSIM generates redundant DDL for unique constraints
> ---
>
> Key: TRAFODION-3314
> URL: https://issues.apache.org/jira/browse/TRAFODION-3314
> Project: Apache Trafodion
> Issue Type: Improvement
> Components: sql-cmp
>Affects Versions: 2.4
>Reporter: David Wayne Birdsall
>Assignee: David Wayne Birdsall
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Unique constraints are usually implemented by creation of a unique index
> under the covers. When doing an OSIM CAPTURE of a query on a table that has a
> unique constraint, OSIM generates both an ALTER TABLE ADD CONSTRAINT
> statement and a CREATE UNIQUE INDEX statement in the CREATE_TABLE_DDLS.txt
> file. When this is replayed in OSIM LOAD, we get an error when creating the
> unique index.
> To fix this, perhaps OSIM should generate the CREATE UNIQUE INDEX statement
> only in commented-out form.
>
[jira] [Work logged] (TRAFODION-3314) OSIM generates redundant DDL for unique constraints
[
https://issues.apache.org/jira/browse/TRAFODION-3314?focusedWorklogId=271194=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-271194
]
ASF GitHub Bot logged work on TRAFODION-3314:
-
Author: ASF GitHub Bot
Created on: 02/Jul/19 20:45
Start Date: 02/Jul/19 20:45
Worklog Time Spent: 10m
Work Description: DaveBirdsall commented on pull request #1846:
[TRAFODION-3314] Avoid generating redundant DDL in OSIM for unique constraints
URL: https://github.com/apache/trafodion/pull/1846
The problem: When a table has a unique constraint, OSIM generates both the
ALTER TABLE statement to create the constraint and the CREATE INDEX statement
for the system-generated index into CREATE_TABLES_DDL.txt. When we then do OSIM
LOAD, the CREATE INDEX is executed, then the ALTER TABLE. The ALTER TABLE fails
because the index already exists.
The fix: Don't write the CREATE INDEX to the CREATE_TABLES_DDL.txt file.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 271194)
Time Spent: 10m
Remaining Estimate: 0h
> OSIM generates redundant DDL for unique constraints
> ---
>
> Key: TRAFODION-3314
> URL: https://issues.apache.org/jira/browse/TRAFODION-3314
> Project: Apache Trafodion
> Issue Type: Improvement
> Components: sql-cmp
>Affects Versions: 2.4
>Reporter: David Wayne Birdsall
>Assignee: David Wayne Birdsall
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Unique constraints are usually implemented by creation of a unique index
> under the covers. When doing an OSIM CAPTURE of a query on a table that has a
> unique constraint, OSIM generates both an ALTER TABLE ADD CONSTRAINT
> statement and a CREATE UNIQUE INDEX statement in the CREATE_TABLE_DDLS.txt
> file. When this is replayed in OSIM LOAD, we get an error when creating the
> unique index.
> To fix this, perhaps OSIM should generate the CREATE UNIQUE INDEX statement
> only in commented-out form.
> {{To reproduce, run the following script in sqlci:}}
> {{?section setup}}
> {{drop table if exists t1 cascade;}}
> {{create table t1 (a int not null, b int not null, c int not null, primary
> key (a));}}
> {{alter table t1 add constraint uniqueb unique(b);}}
> {{?section osimgen}}
> {{control osim capture location 'osimgen';}}
> {{prepare xx from select b,count(*) from t1 group by b;}}
> {{control osim capture stop;}}
> {{Then in a separate session, do:}}
> {{control osim load from 'osimgen';}}
> {{This will fail with the following output:}}
> {{>>control osim load from '.';}}
> {{[OSIM]loading tables and views ...}}
> {{[OSIM]DROP TABLE IF EXISTS TRAFODION.SCH.T1 CASCADE;}}
> {{[OSIM]Step 1 Create Schemas:}}
> {{[OSIM]CREATE SCHEMA IF NOT EXISTS TRAFODION.SCH}}
> {{[OSIM]Step 2 Create Tables:}}
> {{[OSIM]CREATE TABLE TRAFODION.SCH.T1}}
> {{[OSIM] The following index is a system created index --CREATE UNIQUE INDEX
> UNIQUEB ON TRAFODION.SCH.T1}}
> {{[OSIM]ALTER TABLE TRAFODION.SCH.T1 ADD CONSTRAINT TRAFODION.SCH.UNIQUEB
> UNIQUE}}
> {{*** ERROR[1043] Constraint TRAFODION.SCH.UNIQUEB already exists.}}
> {{*** ERROR[6009] The Optimizer Simulator (OSIM): Create Table Error: -1043}}
> {{*** ERROR[8822] The statement was not prepared.}}
> {{>>exit;}}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (TRAFODION-3263) Disable LOB locking and refactor order of LOB iud expression evaluation
[ https://issues.apache.org/jira/browse/TRAFODION-3263?focusedWorklogId=271135=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-271135 ] ASF GitHub Bot logged work on TRAFODION-3263: - Author: ASF GitHub Bot Created on: 02/Jul/19 19:15 Start Date: 02/Jul/19 19:15 Worklog Time Spent: 10m Work Description: Traf-Jenkins commented on issue #1845: [TRAFODION-3263] and other misc fixes for lob locking and refactoring URL: https://github.com/apache/trafodion/pull/1845#issuecomment-507809369 New Check Test Started: https://jenkins.esgyn.com/job/Check-PR-master/3237/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 271135) Time Spent: 2h (was: 1h 50m) > Disable LOB locking and refactor order of LOB iud expression evaluation > > > Key: TRAFODION-3263 > URL: https://issues.apache.org/jira/browse/TRAFODION-3263 > Project: Apache Trafodion > Issue Type: Improvement > Components: sql-general >Affects Versions: 2.2.0 >Reporter: Sandhya Sundaresan >Assignee: Sandhya Sundaresan >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > The change to use JNI to do HDFS writes improved the interface by returning > more useful infomration to the caller. In TRAFODION-2946, we ddescribe the > need for LOB locking because of a condition where multiple threads writing to > the same LOB column could interleave and cause problems. TWith the new JNI > interface and HDFS write will now return the offset where the data was > written. So we can use this return offset to store in the descriptor tables. > Prior to this while using the libhdfs API, we would not get back the "written > offset". > > So the order of operations before this change used to be : > # Get the EOD for the LOB data file in HDFS > # Store this offset into the LOB descriptor tables so we know where to > retrieve the data from during a read. > # call hdfsWrite to write to the LOB data file. And hope that the offset > where the hdfsWrite writes is the same as the EOD calculated in 1. hdfs being > an "append only"file system, this is usually how it works. But if another > process comes in and does an insert into the LOB column between 2 and 3, then > we have an incorrect offset stored int he descriptor tables. Hence we added a > Lob Lock to make steps 1,2 and 3 atomic as part of Trafodion-2946 to address > this issue. > The order of operations with this change is as follows : > # Call JNI hdfs Write API to write the lob data to hdfs. > # Use return data offset from JNI hdfswrite API in 1. as the offset to store > in the LOB descriptor tables. > # If there are multiple chunks to write, do it in a loop and append to the > first chunk. This way each chunk can be anywhere in hdfs and not necessarily > continguous. But we are guaranteed that whatever we wrote will be stored in > our interna lLOB descriptor files. > # If any failure or TM error occurs while writing to the LOB descriptor > tables, transaction gets rolled back and the chunk of hdfs data written > becomes "dead data". It doesn't harm the next operation. > # GC check is now done before an update or insert. Earlier it was done as > part of the ::allocateDesc operation to get the EOD of the file. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
