[jira] [Comment Edited] (SQOOP-3082) Sqoop import fails after TCP connection reset if split by datetime column
[
https://issues.apache.org/jira/browse/SQOOP-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479374#comment-16479374
]
Fero Szabo edited comment on SQOOP-3082 at 5/17/18 5:07 PM:
Hi [~vaifer],
This came up recently again, so I had a look at your patch. I had to rebase it
to the current version, please find this version attached.
I've tested it manually with an Integer and a Date column in the split-by
option.
The former to ensure that it doesn't alter current behavior, the latter to
check if the fix actually works. [^SQOOP-3082-1.patch]
*I can confirm that the current behavior of Sqoop is not altered and the patch
fixes the issue.*
I also checked the relevant parts of the documentation of SQL Server (1, 2) and
found that the data type precedence will ensure the correct behavior of Sqoop.
For example, if the lastRecordValue field contains a number, it will be
"encoded" as a String because of the apostrophes in the resulting statement,
however, since the column's type is still INT, the INT will take precedence and
the criteria will be evaluated correctly.
{quote}When an operator combines two expressions of different data types, the
rules for data type precedence specify that the data type with the lower
precedence is converted to the data type with the higher precedence. If the
conversion is not a supported implicit conversion, an error is returned. When
both operand expressions have the same data type, the result of the operation
has that data type.
{quote}
(1) SQL Server 2000:
[https://www.microsoft.com/en-us/download/details.aspx?id=51958],
(2) current documentation:
[https://docs.microsoft.com/en-us/sql/t-sql/data-types/data-type-precedence-transact-sql?view=sql-server-2017]
I believe we should get this committed now, since it adds a real value for
sqoop users, even without tests.
Since testing a connection reset is not a trivial issue, I've opened
SQOOP-3325, to track the implementation of the tests.
was (Author: fero):
Hi [~vaifer],
This came up recently again, so I had a look at your patch. I had to rebase it
to the current version, please find this version attached.
I've tested it manually with an Integer and a Date column in the split-by
option.
The former to ensure that it doesn't alter current behavior, the latter to
check if the fix actually works. [^SQOOP-3082-1.patch]
*I can confirm that the current behavior of Sqoop is not altered and the patch
fixes the issue.*
I also checked the relevant parts of the documentation of SQL Server (1, 2) and
found that the data type precedence will ensure the correct behavior of Sqoop.
For example, if the lastRecordValue field contains a number, it will be
"encoded" as a String because of the apostrophes in the resulting statement,
however, since the column's type is still INT, the INT will take precedence and
the criteria will be evaluated correctly.
{quote}When an operator combines two expressions of different data types, the
rules for data type precedence specify that the data type with the lower
precedence is converted to the data type with the higher precedence. If the
conversion is not a supported implicit conversion, an error is returned. When
both operand expressions have the same data type, the result of the operation
has that data type.
{quote}
(1) SQL Server 2000:
[https://www.microsoft.com/en-us/download/details.aspx?id=51958],
(2) current documentation:
[https://docs.microsoft.com/en-us/sql/t-sql/data-types/data-type-precedence-transact-sql?view=sql-server-2017)
I believe we should get this committed now, since it adds a real value for
sqoop users, even without tests.
Since testing a connection reset is not a trivial issue, I've opened
SQOOP-3325, to track the implementation of the tests.
> Sqoop import fails after TCP connection reset if split by datetime column
> -
>
> Key: SQOOP-3082
> URL: https://issues.apache.org/jira/browse/SQOOP-3082
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Sergey Svynarchuk
>Priority: Major
> Attachments: SQOOP-3082-1.patch, SQOOP-3082.patch
>
>
> If sqoop-to-mssqlserver connection reset, the whole command fails with
> "Connection reset with com.microsoft.sqlserver.jdbc.SQLServerException:
> Incorrect syntax near '00'" . On reestablishing connection, Sqoop tries to
> resume import from the last record that was successfully read by :
> {code}
> 2016-12-10 15:18:54,523 INFO [main]
> org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: select * from
> test.dbo.test1 WITH (nolock) where Date >= '2015-01-10' and Date <=
> '2016-11-24' and ( Date > 2015-09-18 00:00:00.0 ) AND ( Date < '2015-09-23
> 11:48:00.0' )
> {code}
> Not quoted 2015-09-18 00:00:00.0
[jira] [Comment Edited] (SQOOP-3082) Sqoop import fails after TCP connection reset if split by datetime column
[
https://issues.apache.org/jira/browse/SQOOP-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479374#comment-16479374
]
Fero Szabo edited comment on SQOOP-3082 at 5/17/18 5:06 PM:
Hi [~vaifer],
This came up recently again, so I had a look at your patch. I had to rebase it
to the current version, please find this version attached.
I've tested it manually with an Integer and a Date column in the split-by
option.
The former to ensure that it doesn't alter current behavior, the latter to
check if the fix actually works. [^SQOOP-3082-1.patch]
*I can confirm that the current behavior of Sqoop is not altered and the patch
fixes the issue.*
I also checked the relevant parts of the documentation of SQL Server (1, 2) and
found that the data type precedence will ensure the correct behavior of Sqoop.
For example, if the lastRecordValue field contains a number, it will be
"encoded" as a String because of the apostrophes in the resulting statement,
however, since the column's type is still INT, the INT will take precedence and
the criteria will be evaluated correctly.
{quote}When an operator combines two expressions of different data types, the
rules for data type precedence specify that the data type with the lower
precedence is converted to the data type with the higher precedence. If the
conversion is not a supported implicit conversion, an error is returned. When
both operand expressions have the same data type, the result of the operation
has that data type.
{quote}
(1) SQL Server 2000:
[https://www.microsoft.com/en-us/download/details.aspx?id=51958],
(2) current documentation:
[https://docs.microsoft.com/en-us/sql/t-sql/data-types/data-type-precedence-transact-sql?view=sql-server-2017)
I believe we should get this committed now, since it adds a real value for
sqoop users, even without tests.
Since testing a connection reset is not a trivial issue, I've opened
SQOOP-3325, to track the implementation of the tests.
was (Author: fero):
Hi [~vaifer],
This came up recently again, so I had a look at your patch. I had to rebase it
to the current version, please find this version attached.
I've tested it manually with an Integer and a Date column in the split-by
option.
The former to ensure that it doesn't alter current behavior, the latter to
check if the fix actually works. [^SQOOP-3082-1.patch]
*I can confirm that the current behavior of Sqoop is not altered and the patch
fixes the issue.*
I also checked the relevant parts of the documentation of SQL Server (1, 2) and
found that the data type precedence will ensure the correct behavior of Sqoop.
For example, if the lastRecordValue field contains a number, it will be
"encoded" as a String because of the apostrophes in the resulting statement,
however, since the column's type is still INT, the INT will take precedence and
the criteria will be evaluated correctly.
{quote}When an operator combines two expressions of different data types, the
rules for data type precedence specify that the data type with the lower
precedence is converted to the data type with the higher precedence. If the
conversion is not a supported implicit conversion, an error is returned. When
both operand expressions have the same data type, the result of the operation
has that data type.
{quote}
(1) SQL Server 2000:
[https://www.microsoft.com/en-us/download/details.aspx?id=51958],
(2) current documentation:
[https://docs.microsoft.com/en-us/sql/t-sql/data-types/data-type-precedence-transact-sql?view=sql-server-2017)
]
I believe we should get this committed now, since it adds a real value for
sqoop users, even without tests.
Since testing a connection reset is not a trivial issue, I've opened
SQOOP-3325, to track the implementation of the tests.
> Sqoop import fails after TCP connection reset if split by datetime column
> -
>
> Key: SQOOP-3082
> URL: https://issues.apache.org/jira/browse/SQOOP-3082
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Sergey Svynarchuk
>Priority: Major
> Attachments: SQOOP-3082-1.patch, SQOOP-3082.patch
>
>
> If sqoop-to-mssqlserver connection reset, the whole command fails with
> "Connection reset with com.microsoft.sqlserver.jdbc.SQLServerException:
> Incorrect syntax near '00'" . On reestablishing connection, Sqoop tries to
> resume import from the last record that was successfully read by :
> {code}
> 2016-12-10 15:18:54,523 INFO [main]
> org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: select * from
> test.dbo.test1 WITH (nolock) where Date >= '2015-01-10' and Date <=
> '2016-11-24' and ( Date > 2015-09-18 00:00:00.0 ) AND ( Date < '2015-09-23
> 11:48:00.0' )
> {code}
> Not quoted 2015-09-18
[jira] [Comment Edited] (SQOOP-3082) Sqoop import fails after TCP connection reset if split by datetime column
[
https://issues.apache.org/jira/browse/SQOOP-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479374#comment-16479374
]
Fero Szabo edited comment on SQOOP-3082 at 5/17/18 5:05 PM:
Hi [~vaifer],
This came up recently again, so I had a look at your patch. I had to rebase it
to the current version, please find this version attached.
I've tested it manually with an Integer and a Date column in the split-by
option.
The former to ensure that it doesn't alter current behavior, the latter to
check if the fix actually works. [^SQOOP-3082-1.patch]
*I can confirm that the current behavior of Sqoop is not altered and the patch
fixes the issue.*
I also checked the relevant parts of the documentation of SQL Server (1, 2) and
found that the data type precedence will ensure the correct behavior of Sqoop.
For example, if the lastRecordValue field contains a number, it will be
"encoded" as a String because of the apostrophes in the resulting statement,
however, since the column's type is still INT, the INT will take precedence and
the criteria will be evaluated correctly.
{quote}When an operator combines two expressions of different data types, the
rules for data type precedence specify that the data type with the lower
precedence is converted to the data type with the higher precedence. If the
conversion is not a supported implicit conversion, an error is returned. When
both operand expressions have the same data type, the result of the operation
has that data type.
{quote}
(1) SQL Server 2000:
[https://www.microsoft.com/en-us/download/details.aspx?id=51958],
(2) current documentation:
[https://docs.microsoft.com/en-us/sql/t-sql/data-types/data-type-precedence-transact-sql?view=sql-server-2017)
]
I believe we should get this committed now, since it adds a real value for
sqoop users, even without tests.
Since testing a connection reset is not a trivial issue, I've opened
SQOOP-3325, to track the implementation of the tests.
was (Author: fero):
Hi [~vaifer],
This came up recently again, so I had a look at your patch. I had to rebase it
to the current version, please find this version attached.
I've tested it manually with an Integer and a Date column in the split-by
option.
The former to ensure that it doesn't alter current behavior, the latter to
check if the fix actually works. [^SQOOP-3082-1.patch]
*I can confirm that the current behavior of Sqoop is not altered and the patch
fixes the issue.*
I also checked the relevant parts of the documentation of SQL Server (1, 2) and
found that the data type precedence will ensure the correct behavior of Sqoop.
For example, if the lastRecordValue field contains a number, it will be
"encoded" as a String because of the apostrophes in the resulting statement,
however, since the column's type is still INT, the INT will take precedence and
the criteria will be evaluated correctly.
{quote}When an operator combines two expressions of different data types, the
rules for data type precedence specify that the data type with the lower
precedence is converted to the data type with the higher precedence. If the
conversion is not a supported implicit conversion, an error is returned. When
both operand expressions have the same data type, the result of the operation
has that data type.
{quote}
(1) SQL Server 2000:
[https://www.microsoft.com/en-us/download/details.aspx?id=51958],
(2) current documentation:
[https://docs.microsoft.com/en-us/sql/t-sql/data-types/data-type-precedence-transact-sql?view=sql-server-2017)
]
I believe we should get this committed now, since it adds a real value for
sqoop users, even without tests.[
|https://docs.microsoft.com/en-us/sql/t-sql/data-types/data-type-precedence-transact-sql?view=sql-server-2017)]Since
testing a connection reset is not a trivial issue, I've opened SQOOP-3325, to
track the implementation of the tests.
> Sqoop import fails after TCP connection reset if split by datetime column
> -
>
> Key: SQOOP-3082
> URL: https://issues.apache.org/jira/browse/SQOOP-3082
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Sergey Svynarchuk
>Priority: Major
> Attachments: SQOOP-3082-1.patch, SQOOP-3082.patch
>
>
> If sqoop-to-mssqlserver connection reset, the whole command fails with
> "Connection reset with com.microsoft.sqlserver.jdbc.SQLServerException:
> Incorrect syntax near '00'" . On reestablishing connection, Sqoop tries to
> resume import from the last record that was successfully read by :
> {code}
> 2016-12-10 15:18:54,523 INFO [main]
> org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: select * from
> test.dbo.test1 WITH (nolock) where Date >= '2015-01-10' and Date <=
> '2016-11-24' and (
