Lili Ma created HAWQ-1144:
-----------------------------
Summary: Register into a 2-level partition table, hawq register
didn't throw error, and indicates that hawq register succeed, but no data can
be selected out.
Key: HAWQ-1144
URL: https://issues.apache.org/jira/browse/HAWQ-1144
Project: Apache HAWQ
Issue Type: Bug
Components: Command Line Tools
Reporter: Lili Ma
Assignee: Lei Chang
Fix For: 2.0.1.0-incubating
Register into a 2-level partition table, hawq register didn't throw error, and
indicates that hawq register succeed, but no data can be selected out.
Reproduce Steps:
1. Create a one-level partition table
```
create table parquet_wt (id SERIAL,a1 int,a2 char(5),a3 numeric,a4 boolean
DEFAULT false ,a5 char DEFAULT 'd',a6 text,a7 timestamp,a8 character
varying(705),a9 bigint,a10 date,a11 varchar(600),a12 text,a13 decimal,a14
real,a15 bigint,a16 int4 ,a17 bytea,a18 timestamp with time zone,a19 timetz,a20
path,a21 box,a22 macaddr,a23 interval,a24 character varying(800),a25 lseg,a26
point,a27 double precision,a28 circle,a29 int4,a30 numeric(8),a31 polygon,a32
date,a33 real,a34 money,a35 cidr,a36 inet,a37 time,a38 text,a39 bit,a40 bit
varying(5),a41 smallint,a42 int ) WITH (appendonly=true, orientation=parquet)
distributed randomly Partition by range(a1) (start(1) end(5000) every(1000) );
```
2. insert some data into this table
```
insert into parquet_wt
(a1,a2,a3,a4,a5,a6,a7,a8,a9,a10,a11,a12,a13,a14,a15,a16,a17,a18,a19,a20,a21,a22,a23,a24,a25,a26,a27,a28,a29,a30,a31,a32,a33,a34,a35,a36,a37,a38,a39,a40,a41,a42)
values(generate_series(1,20),'M',2011,'t','a','This is news of today: Deadlock
between Republicans and Democrats over how best to reduce the U.S. deficit, and
over what period, has blocked an agreement to allow the raising of the $14.3
trillion debt ceiling','2001-12-24 02:26:11','U.S. House of Representatives
Speaker John Boehner, the top Republican in Congress who has put forward a
deficit reduction plan to be voted on later on Thursday said he had no control
over whether his bill would avert a credit
downgrade.',generate_series(2490,2505),'2011-10-11','The Republican-controlled
House is tentatively scheduled to vote on Boehner proposal this afternoon at
around 6 p.m. EDT (2200 GMT). The main Republican vote counter in the House,
Kevin McCarthy, would not say if there were enough votes to pass the
bill.','WASHINGTON:House Speaker John Boehner says his plan mixing spending
cuts in exchange for raising the nations $14.3 trillion debt limit is not
perfect but is as large a step that a divided government can take that is
doable and signable by President Barack Obama.The Ohio Republican says the
measure is an honest and sincere attempt at compromise and was negotiated with
Democrats last weekend and that passing it would end the ongoing debt crisis.
The plan blends $900 billion-plus in spending cuts with a companion increase in
the nations borrowing
cap.','1234.56',323453,generate_series(3452,3462),7845,'0011','2005-07-16
01:51:15+1359','2001-12-13
01:51:15','((1,2),(0,3),(2,1))','((2,3)(4,5))','08:00:2b:01:02:03','1-2','Republicans
had been working throughout the day Thursday to lock down support for their
plan to raise the nations debt ceiling, even as Senate Democrats vowed to
swiftly kill it if
passed.','((2,3)(4,5))','(6,7)',11.222,'((4,5),7)',32,3214,'(1,0,2,3)','2010-02-21',43564,'$1,000.00','192.168.1','126.1.3.4','12:30:45','Johnson
& Johnsons McNeil Consumer Healthcare announced the voluntary dosage reduction
today. Labels will carry new dosing instructions this fall.The company says it
will cut the maximum dosage of Regular Strength Tylenol and other
acetaminophen-containing products in 2012.Acetaminophen is safe when used as
directed, says Edwin Kuffner, MD, McNeil vice president of over-the-counter
medical affairs. But, when too much is taken, it can cause liver damage.The
action is intended to cut the risk of such accidental overdoses, the company
says in a news release.','1','0',12,23);
```
3. extract the metadata out for the table
```
hawq extract -d postgres -o ~/parquet.yaml parquet_wt
```
4. create a two-level partition table
```
CREATE TABLE parquet_wt_subpartgzip2
(id SERIAL,a1 int,a2
char(5),a3 numeric,a4 boolean DEFAULT false ,a5 char DEFAULT 'd',a6 text,a7
timestamp,a8 character varying(705),a9 bigint,a10 date,a11 varchar(600),a12
text,a13 decimal,a14 real,a15 bigint,a16 int4 ,a17 bytea,a18 timestamp with
time zone,a19 timetz,a20 path,a21 box,a22 macaddr,a23 interval,a24 character
varying(800),a25 lseg,a26 point,a27 double precision,a28 circle,a29 int4,a30
numeric(8),a31 polygon,a32 date,a33 real,a34 money,a35 cidr,a36 inet,a37
time,a38 text,a39 bit,a40 bit varying(5),a41 smallint,a42 int )
WITH (appendonly=true, orientation=parquet) distributed
randomly Partition by range(a1) Subpartition by list(a2) subpartition template
( default subpartition df_sp, subpartition sp1 values('M') , subpartition sp2
values('F') WITH
(appendonly=true, orientation=parquet,compresstype=gzip,compresslevel=2))
(start(1) end(5000) every(1000) );
```
5. call register
```
hawq register -d postgres -c ~/parquet.yaml parquet_wt_subpartgzip2
```
6. It reflects register succeed.
```
malilis-MacBook-Pro:tpch malili$ hawq register -d postgres -c ~/parquet.yaml
parquet_wt_subpartgzip2
20161103:15:58:10:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-try to
connect database localhost:5432 postgres
20161103:15:58:10:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files
check...
20161103:15:58:11:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files
check done...
20161103:15:58:11:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files
check...
20161103:15:58:13:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files
check done...
20161103:15:58:13:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files
check...
20161103:15:58:14:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files
check done...
20161103:15:58:14:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files
check...
20161103:15:58:16:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files
check done...
20161103:15:58:16:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files
check...
20161103:15:58:17:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files
check done...
20161103:15:58:20:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-New
file(s) to be registered:
['hdfs://localhost:8020/hawq_default/16385/16387/17065/1']
20161103:15:58:29:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-New
file(s) to be registered:
['hdfs://localhost:8020/hawq_default/16385/16387/17074/1']
20161103:15:58:35:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-New
file(s) to be registered:
['hdfs://localhost:8020/hawq_default/16385/16387/17083/1']
20161103:15:58:41:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-New
file(s) to be registered:
['hdfs://localhost:8020/hawq_default/16385/16387/17092/1']
20161103:15:58:47:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-New
file(s) to be registered:
['hdfs://localhost:8020/hawq_default/16385/16387/17101/1']
hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/17065/1
hdfs://localhost:8020/hawq_default/16385/16387/16784/1"
hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/17074/1
hdfs://localhost:8020/hawq_default/16385/16387/16822/1"
hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/17083/1
hdfs://localhost:8020/hawq_default/16385/16387/16860/1"
hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/17092/1
hdfs://localhost:8020/hawq_default/16385/16387/16898/1"
hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/17101/1
hdfs://localhost:8020/hawq_default/16385/16387/16936/1"
20161103:15:58:58:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Hawq
Register Succeed.
```
7. But when we select the table, no data can be selected out.
```
postgres=# select count(*) from parquet_wt_subpartgzip2;
count
-------
0
(1 row)
```
Actually we should throw error if hawq register wants to register into a 2 or
more -level partition table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
