[
https://issues.apache.org/jira/browse/PHOENIX-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15392290#comment-15392290
]
Pierre Lacave commented on PHOENIX-3112:
----------------------------------------
I have simplified the test case and made it programatically reproducable.
The bug can be seen in 4.7 and 4.8/master.
[~jamestaylor] I see you are planning to cut a 4.8 RC tomorrow, would you mind
assessing the importance of this bug?
I could use some guidance to continue troubleshooting as well.
Thanks
{noformat}
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.phoenix.end2end;
import org.apache.commons.lang.StringUtils;
import org.apache.phoenix.util.PropertiesUtil;
import org.junit.Test;
import java.sql.*;
import java.util.Properties;
import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
import static org.junit.Assert.*;
public class LargeTableIT extends BaseHBaseManagedTimeIT {
private static void initTableValues() throws SQLException {
Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
Connection conn = DriverManager.getConnection(getUrl(), props);
conn.setAutoCommit(false);
try {
String ddl = "CREATE TABLE IF NOT EXISTS test_table (\n" +
" TIMESTAMP UNSIGNED_LONG NOT NULL,\n" +
" MI UNSIGNED_LONG,\n" +
" MB VARBINARY\n" +
" CONSTRAINT pk PRIMARY KEY (TIMESTAMP))\n";
createTestTable(getUrl(), ddl);
String query;
PreparedStatement stmt;
query = "UPSERT INTO test_table"
+ "(TIMESTAMP, MI, MB) "
+ "VALUES(?,?,?)";
stmt = conn.prepareStatement(query);
for (int i=0; i<100; i++) {
stmt.setLong(1, 1000000L+i);
stmt.setLong(2, 2000000L+i);
stmt.setBytes(3, StringUtils.repeat("E", 30000).getBytes());
stmt.execute();
}
conn.commit();
} finally {
conn.close();
}
}
@Test
public void testSelectValues() throws SQLException {
Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
Connection conn = DriverManager.getConnection(getUrl(), props);
try {
initTableValues();
String query = "SELECT * FROM test_table";
PreparedStatement stmt = conn.prepareStatement(query);
ResultSet rs = stmt.executeQuery();
for (int i=0; i<100; i++) {
assertTrue(rs.next());
assertEquals(1000000L+i, rs.getLong(1));
assertEquals(2000000L+i, rs.getLong(2));
}
assertFalse(rs.next());
} finally {
conn.close();
}
}
}
{noformat}
{noformat}
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m;
support was removed in 8.0
Running org.apache.phoenix.end2end.LargeTableIT
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 29.147 sec <<<
FAILURE! - in org.apache.phoenix.end2end.LargeTableIT
testSelectValues(org.apache.phoenix.end2end.LargeTableIT) Time elapsed: 2.751
sec <<< FAILURE!
java.lang.AssertionError: expected:<2000069> but was:<0>
at
org.apache.phoenix.end2end.LargeTableIT.testSelectValues(LargeTableIT.java:84)
Results :
Failed tests:
LargeTableIT.testSelectValues:84 expected:<2000069> but was:<0>
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
{noformat}
> Rows randomly missing values with large select
> ----------------------------------------------
>
> Key: PHOENIX-3112
> URL: https://issues.apache.org/jira/browse/PHOENIX-3112
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.7.0
> Reporter: Pierre Lacave
>
> When doing a select of a relatively large table (a few touthands rows) some
> rows return partially missing.
> When increasing the fitler to return those specific rows, the values appear
> as expected
> {noformat}
> CREATE TABLE IF NOT EXISTS TEST (
> BUCKET VARCHAR,
> TIMESTAMP_DATE TIMESTAMP,
> TIMESTAMP UNSIGNED_LONG NOT NULL,
> SRC VARCHAR,
> DST VARCHAR,
> ID VARCHAR,
> ION VARCHAR,
> IC BOOLEAN NOT NULL,
> MI UNSIGNED_LONG,
> AV UNSIGNED_LONG,
> MA UNSIGNED_LONG,
> CNT UNSIGNED_LONG,
> DUMMY VARCHAR
> CONSTRAINT pk PRIMARY KEY (BUCKET, TIMESTAMP DESC, SRC, DST, ID, ION, IC)
> );{noformat}
> using a python script to generate a CSV with 5000 rows
> {noformat}
> for i in xrange(5000):
> print "5SEC,2016-07-21
> 07:25:35.{i},146908593500{i},WWWWWWWW,AAA,BBBB,CCCCCCCC,false,{i}1181000,1788000{i},2497001{i},{i},aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa{i}".format(i=i)
> {noformat}
> bulk inserting the csv in the table
> {noformat}
> phoenix/bin/psql.py localhost -t TEST large.csv
> {noformat}
> here we can see one row that contains no TIMESTAMP_DATE and null values in MI
> and MA
> {noformat}
> 0: jdbc:phoenix:localhost:2181> select * from TEST
> ....
> +---------+--------------------------+-------------------+-----------+------+-------+-----------+--------+--------------+--------------+--------------+-------+----------------------------------------------------------------------------+
> | BUCKET | TIMESTAMP_DATE | TIMESTAMP | SRC | DST |
> ID | ION | IC | MI | AV | MA |
> CNT | DUMMY
> |
> +---------+--------------------------+-------------------+-----------+------+-------+-----------+--------+--------------+--------------+--------------+-------+----------------------------------------------------------------------------+
> | 5SEC | 2016-07-21 07:25:35.100 | 1469085935001000 | WWWWWWWW | AAA |
> BBBB | CCCCCCCC | false | 10001181000 | 17880001000 | 24970011000 |
> 1000 |
> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa1000 |
> | 5SEC | 2016-07-21 07:25:35.999 | 146908593500999 | WWWWWWWW | AAA |
> BBBB | CCCCCCCC | false | 9991181000 | 1788000999 | 2497001999 | 999
> | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa999
> |
> | 5SEC | 2016-07-21 07:25:35.998 | 146908593500998 | WWWWWWWW | AAA |
> BBBB | CCCCCCCC | false | 9981181000 | 1788000998 | 2497001998 | 998
> | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa998
> |
> | 5SEC | | 146908593500997 | WWWWWWWW | AAA |
> BBBB | CCCCCCCC | false | null | 1788000997 | null | 997
> |
> |
> | 5SEC | 2016-07-21 07:25:35.996 | 146908593500996 | WWWWWWWW | AAA |
> BBBB | CCCCCCCC | false | 9961181000 | 1788000996 | 2497001996 | 996
> | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa996
> |
> | 5SEC | 2016-07-21 07:25:35.995 | 146908593500995 | WWWWWWWW | AAA |
> BBBB | CCCCCCCC | false | 9951181000 | 1788000995 | 2497001995 | 995
> | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa995
> |
> | 5SEC | 2016-07-21 07:25:35.994 | 146908593500994 | WWWWWWWW | AAA |
> BBBB | CCCCCCCC | false | 9941181000 | 1788000994 | 2497001994 | 994
> | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa994
> |
> ....
> {noformat}
> but when selecting that row specifically the values are correct
> {noformat}
> 0: jdbc:phoenix:localhost:2181> select * from TEST where timestamp =
> 146908593500997;
> +---------+--------------------------+------------------+-----------+------+-------+-----------+--------+-------------+-------------+-------------+------+---------------------------------------------------------------------------+
> | BUCKET | TIMESTAMP_DATE | TIMESTAMP | SRC | DST |
> ID | ION | IC | MI | AV | MA | CNT |
> DUMMY |
> +---------+--------------------------+------------------+-----------+------+-------+-----------+--------+-------------+-------------+-------------+------+---------------------------------------------------------------------------+
> | 5SEC | 2016-07-21 07:25:35.997 | 146908593500997 | WWWWWWWW | AAA |
> BBBB | CCCCCCCC | false | 9971181000 | 1788000997 | 2497001997 | 997 |
> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa997 |
> +---------+--------------------------+------------------+-----------+------+-------+-----------+--------+-------------+-------------+-------------+------+---------------------------------------------------------------------------+
> 1 row selected (0.159 seconds){noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
