[
https://issues.apache.org/jira/browse/BEAM-13242?focusedWorklogId=682967&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-682967
]
ASF GitHub Bot logged work on BEAM-13242:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 17/Nov/21 22:26
Start Date: 17/Nov/21 22:26
Worklog Time Spent: 10m
Work Description: borovikovd commented on a change in pull request #15987:
URL: https://github.com/apache/beam/pull/15987#discussion_r751686195
##########
File path:
sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/LogicalTypes.java
##########
@@ -266,8 +266,8 @@ private FixedPrecisionNumeric(
@Override
public BigDecimal toInputType(BigDecimal base) {
checkArgument(
- base == null || (base.precision() == precision && base.scale() ==
scale),
- "Expected BigDecimal base to be null or have precision = %s (was
%s), scale = %s (was %s)",
+ base == null || (base.precision() <= precision && base.scale() <=
scale),
Review comment:
@kennknowles would
`(base.precision() - base.scale()) <= (precision - scale) && (base.scale()
<= scale)`
work?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 682967)
Time Spent: 1h 20m (was: 1h 10m)
> IllegalArgumentException while reading a Numeric column with a fixed precision
> ------------------------------------------------------------------------------
>
> Key: BEAM-13242
> URL: https://issues.apache.org/jira/browse/BEAM-13242
> Project: Beam
> Issue Type: Bug
> Components: io-java-jdbc
> Affects Versions: 2.33.0
> Environment: openjdk version "11.0.10" 2021-01-19
> macOS 11.1
> Reporter: Denis
> Assignee: Vitaly Ivanov
> Priority: P2
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> *Error*
>
> I am trying to read a table in a Postgresql database. One of columns has a
> type definition
> {code:java}
> Numeric(15, 2){code}
>
> At runtime I am getting:
> {code:java}
> Caused by: java.lang.IllegalArgumentException: Expected BigDecimal base to be
> null or have precision = 15 (was 6), scale = 2 (was 2) at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument(Preconditions.java:477)
> at
> org.apache.beam.sdk.io.jdbc.LogicalTypes$FixedPrecisionNumeric.toInputType(LogicalTypes.java:268)
> at
> org.apache.beam.sdk.io.jdbc.LogicalTypes$FixedPrecisionNumeric.toInputType(LogicalTypes.java:246)
> at
> org.apache.beam.sdk.io.jdbc.SchemaUtil.lambda$createLogicalTypeExtractor$ca0ab2ec$1(SchemaUtil.java:289)
> at
> org.apache.beam.sdk.io.jdbc.SchemaUtil$BeamRowMapper.mapRow(SchemaUtil.java:354)
> at
> org.apache.beam.sdk.io.jdbc.SchemaUtil$BeamRowMapper.mapRow(SchemaUtil.java:332)
> at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn.processElement(JdbcIO.java:1172)
> {code}
>
> It seems that when JdbcIO reads a numeric column it might have smaller
> precision/scale than defined in DDL.
>
> *How to reproduce*
>
> {code:java}
> import org.apache.beam.sdk.io.jdbc.JdbcIO
> import org.apache.beam.sdk.testing.PAssert
> import org.apache.beam.sdk.testing.TestPipeline
> import org.apache.beam.sdk.transforms.MapElements
> import org.apache.beam.sdk.transforms.ProcessFunction
> import org.apache.beam.sdk.values.TypeDescriptors
> import org.junit.AfterClass
> import org.junit.BeforeClass
> import org.junit.Test
> import org.testcontainers.containers.PostgreSQLContainer
> import java.math.BigDecimal
> import java.sql.Connection
> import java.sql.DriverManager
> class PostgresNumericBugTest {
> companion object {
> val pgContainer = PostgreSQLContainer<Nothing>("postgres:14.0")
> @BeforeClass
> @JvmStatic
> fun start() {
> pgContainer.start()
> getConnection().use {
> val stmt = it.createStatement()
> stmt.execute(
> """create table account (
> id integer not null primary key,
> balance numeric(15, 2)
> )
> """.trimIndent()
> )
> stmt.execute("insert into account(id, balance) values(1,
> 5755.94)")
> }
> }
> @AfterClass
> @JvmStatic
> fun stop() {
> pgContainer.stop()
> }
> private fun getConnection(): Connection {
> return DriverManager.getConnection(
> pgContainer.jdbcUrl, pgContainer.username,
> pgContainer.password
> )
> }
> private fun getDataSourceConfiguration():
> JdbcIO.DataSourceConfiguration =
> JdbcIO.DataSourceConfiguration.create(
> pgContainer.driverClassName,
> pgContainer.jdbcUrl
> )
> .withUsername(pgContainer.username)
> .withPassword(pgContainer.password)
> }
> @Test
> fun readNumeric() {
> val pipeline =
> TestPipeline.create().enableAbandonedNodeEnforcement(false)
> val pBalance = pipeline
> .apply(
> "Read Account from DB",
> JdbcIO
> .readRows()
> .withQuery("select balance from account")
> .withDataSourceConfiguration(getDataSourceConfiguration())
> )
> .apply(
> "Get Balance",
>
> MapElements.into(TypeDescriptors.bigdecimals()).via(ProcessFunction {
> it.getDecimal(0) })
> )
> PAssert.that(pBalance).containsInAnyOrder(BigDecimal("5755.94"))
> pipeline.run()
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
