[
https://issues.apache.org/jira/browse/BEAM-13242?focusedWorklogId=693204&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-693204
]
ASF GitHub Bot logged work on BEAM-13242:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 09/Dec/21 12:22
Start Date: 09/Dec/21 12:22
Worklog Time Spent: 10m
Work Description: timrobertson100 edited a comment on pull request #15987:
URL: https://github.com/apache/beam/pull/15987#issuecomment-989802457
Hi @Pavlmits
Beam releases every ~6 weeks, so there should be a release in January. In
the meantime, perhaps you could apply the changes of this PR into your own
project by creating the`org/apache/beam/sdk/io/jdbc/LogicalTypes.java` class
and copying the contents of [this
file](https://github.com/vitaly-ivanov/beam/blob/9fa3bcaf724a9be53775af3d838154228022c9e2/sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/LogicalTypes.java)
in?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 693204)
Time Spent: 3h 50m (was: 3h 40m)
> IllegalArgumentException while reading a Numeric column with a fixed precision
> ------------------------------------------------------------------------------
>
> Key: BEAM-13242
> URL: https://issues.apache.org/jira/browse/BEAM-13242
> Project: Beam
> Issue Type: Bug
> Components: io-java-jdbc
> Affects Versions: 2.33.0
> Environment: openjdk version "11.0.10" 2021-01-19
> macOS 11.1
> Reporter: Denis
> Assignee: Vitaly Ivanov
> Priority: P2
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> *Error*
>
> I am trying to read a table in a Postgresql database. One of columns has a
> type definition
> {code:java}
> Numeric(15, 2){code}
>
> At runtime I am getting:
> {code:java}
> Caused by: java.lang.IllegalArgumentException: Expected BigDecimal base to be
> null or have precision = 15 (was 6), scale = 2 (was 2) at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument(Preconditions.java:477)
> at
> org.apache.beam.sdk.io.jdbc.LogicalTypes$FixedPrecisionNumeric.toInputType(LogicalTypes.java:268)
> at
> org.apache.beam.sdk.io.jdbc.LogicalTypes$FixedPrecisionNumeric.toInputType(LogicalTypes.java:246)
> at
> org.apache.beam.sdk.io.jdbc.SchemaUtil.lambda$createLogicalTypeExtractor$ca0ab2ec$1(SchemaUtil.java:289)
> at
> org.apache.beam.sdk.io.jdbc.SchemaUtil$BeamRowMapper.mapRow(SchemaUtil.java:354)
> at
> org.apache.beam.sdk.io.jdbc.SchemaUtil$BeamRowMapper.mapRow(SchemaUtil.java:332)
> at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn.processElement(JdbcIO.java:1172)
> {code}
>
> It seems that when JdbcIO reads a numeric column it might have smaller
> precision/scale than defined in DDL.
>
> *How to reproduce*
>
> {code:java}
> import org.apache.beam.sdk.io.jdbc.JdbcIO
> import org.apache.beam.sdk.testing.PAssert
> import org.apache.beam.sdk.testing.TestPipeline
> import org.apache.beam.sdk.transforms.MapElements
> import org.apache.beam.sdk.transforms.ProcessFunction
> import org.apache.beam.sdk.values.TypeDescriptors
> import org.junit.AfterClass
> import org.junit.BeforeClass
> import org.junit.Test
> import org.testcontainers.containers.PostgreSQLContainer
> import java.math.BigDecimal
> import java.sql.Connection
> import java.sql.DriverManager
> class PostgresNumericBugTest {
> companion object {
> val pgContainer = PostgreSQLContainer<Nothing>("postgres:14.0")
> @BeforeClass
> @JvmStatic
> fun start() {
> pgContainer.start()
> getConnection().use {
> val stmt = it.createStatement()
> stmt.execute(
> """create table account (
> id integer not null primary key,
> balance numeric(15, 2)
> )
> """.trimIndent()
> )
> stmt.execute("insert into account(id, balance) values(1,
> 5755.94)")
> }
> }
> @AfterClass
> @JvmStatic
> fun stop() {
> pgContainer.stop()
> }
> private fun getConnection(): Connection {
> return DriverManager.getConnection(
> pgContainer.jdbcUrl, pgContainer.username,
> pgContainer.password
> )
> }
> private fun getDataSourceConfiguration():
> JdbcIO.DataSourceConfiguration =
> JdbcIO.DataSourceConfiguration.create(
> pgContainer.driverClassName,
> pgContainer.jdbcUrl
> )
> .withUsername(pgContainer.username)
> .withPassword(pgContainer.password)
> }
> @Test
> fun readNumeric() {
> val pipeline =
> TestPipeline.create().enableAbandonedNodeEnforcement(false)
> val pBalance = pipeline
> .apply(
> "Read Account from DB",
> JdbcIO
> .readRows()
> .withQuery("select balance from account")
> .withDataSourceConfiguration(getDataSourceConfiguration())
> )
> .apply(
> "Get Balance",
>
> MapElements.into(TypeDescriptors.bigdecimals()).via(ProcessFunction {
> it.getDecimal(0) })
> )
> PAssert.that(pBalance).containsInAnyOrder(BigDecimal("5755.94"))
> pipeline.run()
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
