git commit: Prepare for 3.1.0 release

2014-08-11 Thread mujtaba
Repository: phoenix
Updated Branches:
  refs/heads/3.0 ece6aaf06 - aa46fbd01


Prepare for 3.1.0 release


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/aa46fbd0
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/aa46fbd0
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/aa46fbd0

Branch: refs/heads/3.0
Commit: aa46fbd012f712212bfa6f51baddc2b9cae44e40
Parents: ece6aaf
Author: Mujtaba mujt...@apache.org
Authored: Mon Aug 11 11:13:11 2014 -0700
Committer: Mujtaba mujt...@apache.org
Committed: Mon Aug 11 11:13:11 2014 -0700

--
 CHANGES| 121 +++-
 README |  54 
 dev/make_rc.sh |   4 +-
 phoenix-assembly/pom.xml   |   2 +-
 phoenix-core/pom.xml   |   2 +-
 phoenix-flume/pom.xml  |   2 +-
 phoenix-hadoop-compat/pom.xml  |   2 +-
 phoenix-hadoop1-compat/pom.xml |   2 +-
 phoenix-hadoop2-compat/pom.xml |   2 +-
 phoenix-pig/pom.xml|   2 +-
 pom.xml|   2 +-
 11 files changed, 130 insertions(+), 65 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/aa46fbd0/CHANGES
--
diff --git a/CHANGES b/CHANGES
index 3054940..d1d4c97 100644
--- a/CHANGES
+++ b/CHANGES
@@ -1,5 +1,124 @@
+Release Notes - Phoenix - Version 3.1
 
-Release Notes - Phoenix - Version 3.0.0-incubating
+** New Feature
+* [PHOENIX-71] - Support sub-joins
+* [PHOENIX-983] - Add a function for getting the GMT offset for a given 
timezone and time
+* [PHOENIX-984] - Add support for DECODE built-in function
+* [PHOENIX-995] - ADD ENCODE AND LPAD functions 
+* [PHOENIX-1000] - Support FIRST_VALUE, LAST_VALUE, and NTH_VALUE 
aggregate functions
+* [PHOENIX-1002] - Add support for % operator
+* [PHOENIX-1043] - Add function for convert date between timezones
+* [PHOENIX-1062] - A SQL Trimmer for log sql execute times
+
+** Improvement
+* [PHOENIX-19] - Enhance JDBC connection of Phoenix to support connecting 
to a Secure HBase cluster.
+* [PHOENIX-890] - Create dev-support script for generating binary and 
source release tars with hadoop1 and hadoop2 profiles
+* [PHOENIX-898] - Extend PhoenixHBaseStorage to specify upsert columns
+* [PHOENIX-939] - Generalize SELECT expressions for Pig Loader
+* [PHOENIX-950] - Improve Secondary Index Update Failure Handling
+* [PHOENIX-982] - Avoid spinning up and tearing down mini cluster in tests
+* [PHOENIX-1069] - Improve CsvBulkLoadTool to build indexes when loading 
data.
+* [PHOENIX-1072] - Fast fail sqlline.py when pass wrong quorum string or 
hbase cluster hasnt' started yet 
+* [PHOENIX-1101] - Provide a way to create Phoenix Statements on the 
server side inside a CP
+* [PHOENIX-1103] - Remove hash join special case for ChunkedResultIterator
+
+** Bug
+* [PHOENIX-3] - TRUNC method doesn't work on column of datatype timestamp
+* [PHOENIX-11] - Create Pig Loader
+* [PHOENIX-41] - HBase restart ConnectionLoss
+* [PHOENIX-63] - Can't add null value to a secondary indexed table
+* [PHOENIX-831] - First Connection to Phoenix blows up if it is tenant 
specific
+* [PHOENIX-902] - Allow family qualified names in CSVCommonsLoader
+* [PHOENIX-908] - At MetaDataEndpointImpl createTable, print large of 
Release unknown lockId: 5
+* [PHOENIX-909] - Handle scan time range for low time resolution OS
+* [PHOENIX-911] - DDL containing only PK cols in the column list doesn't 
respect HBase specific attributes
+* [PHOENIX-915] - Numeric functions ROUND, CEIL, FLOOR return incorrect 
results for DOUBLE
+* [PHOENIX-916] - Time clock skew issue when SYSTEM.CATALOG moves between 
region servers
+* [PHOENIX-920] - SqlLine shows funky characters when running on Windows
+* [PHOENIX-924] - Classify PhoenixHBaseStorageTest as an integration test
+* [PHOENIX-948] - Pig integration tests don't get run via Maven
+* [PHOENIX-951] - Don't push LIMIT as PageFilter for joins
+* [PHOENIX-952] - Support ANY and ALL built-ins for ARRAYs
+* [PHOENIX-954] - View index sequences of a non multi tenant table are not 
getting deleted after main table drop
+* [PHOENIX-956] - Connection properties ignored
+* [PHOENIX-962] - Basic querying fails if the current thread's context 
classloader is changed
+* [PHOENIX-966] - Phoenix needs to be on the main classpath of SQL tools
+* [PHOENIX-969] - java.util.concurrent.TimeoutException while reading the 
result set
+* [PHOENIX-979] - DELETE deletes row when WHERE contains PK fields but 
other than PK conditions are false
+* [PHOENIX-981] - Joins using Phoenix client on v.3.0.0 and 

svn commit: r6132 - in /release/phoenix/phoenix-3.1.0-rc0: ./ bin/ src/

2014-08-11 Thread mujtaba
Author: mujtaba
Date: Mon Aug 11 18:22:57 2014
New Revision: 6132

Log:
Phoenix 3.1.0 RC0

Added:
release/phoenix/phoenix-3.1.0-rc0/
release/phoenix/phoenix-3.1.0-rc0/bin/
release/phoenix/phoenix-3.1.0-rc0/bin/phoenix-3.1.0-bin.tar.gz   (with 
props)
release/phoenix/phoenix-3.1.0-rc0/bin/phoenix-3.1.0-bin.tar.gz.asc
release/phoenix/phoenix-3.1.0-rc0/bin/phoenix-3.1.0-bin.tar.gz.md5
release/phoenix/phoenix-3.1.0-rc0/bin/phoenix-3.1.0-bin.tar.gz.sha
release/phoenix/phoenix-3.1.0-rc0/src/
release/phoenix/phoenix-3.1.0-rc0/src/phoenix-3.1.0-src.tar.gz   (with 
props)
release/phoenix/phoenix-3.1.0-rc0/src/phoenix-3.1.0-src.tar.gz.asc
release/phoenix/phoenix-3.1.0-rc0/src/phoenix-3.1.0-src.tar.gz.md5
release/phoenix/phoenix-3.1.0-rc0/src/phoenix-3.1.0-src.tar.gz.sha

Added: release/phoenix/phoenix-3.1.0-rc0/bin/phoenix-3.1.0-bin.tar.gz
==
Binary file - no diff available.

Propchange: release/phoenix/phoenix-3.1.0-rc0/bin/phoenix-3.1.0-bin.tar.gz
--
svn:mime-type = application/octet-stream

Added: release/phoenix/phoenix-3.1.0-rc0/bin/phoenix-3.1.0-bin.tar.gz.asc
==
--- release/phoenix/phoenix-3.1.0-rc0/bin/phoenix-3.1.0-bin.tar.gz.asc (added)
+++ release/phoenix/phoenix-3.1.0-rc0/bin/phoenix-3.1.0-bin.tar.gz.asc Mon Aug 
11 18:22:57 2014
@@ -0,0 +1,17 @@
+-BEGIN PGP SIGNATURE-
+Version: GnuPG v1.4.10 (GNU/Linux)
+
+iQIcBAABCgAGBQJT6Qh8AAoJEC1+sLgXr24/pYEP/24GA/A6bFNFVLHZpSmC88db
+TadMIV+KifPFCz56Vncl+aqvgGqinLuprpQg2N9NtXePVmWcm97kZZRE4DEFteCz
+Ie4mlZtBrAvS9q7X7qITCIMHUnpw0Il2qek+xtGGz0+QEQ9MM05f9olfvIxfr9Uy
+Ij1iFvSR1CXaqTzpTfCxBM/oAuFhXNPk+CrhBEaQIlYmlb8fW4E8n0Ssb6o3/uOt
+AooZAWDzJpiMaNGEROWaEiRUH4FxYU+QpjZ/u0L8wydwRKlmWntiI91NsfqT90eR
+m9dymzpgg2FGKkGjtB5xjDEZ6Rsc2krnDV8v6namRacJoSz+CYwNluMaICAymoR2
+X2DOBAtRuq9sNTvq1May6qmNK3h50EK4/0EZEbsYfWFltnUoFFP7GLsY8R2f2IpV
+741i7yOPGDA7U7QRppZCMLQQ7j0YjsBcrYh5InTjsucyYomuxeIoG2wqbgOMdQSp
+XLA2Kk7rXFXQ7JwRVb/Pb2wrVQtV8Jp1K+CYrj1w7YNJdw3WFF2SMLV/Kc4Vtj1A
+4uUgEs6my7+mY0VsQDgHoljzHY3hHlz/eVdpMc4yFaWBtecLO4dBtYicwsqJa+O1
+OIlyAdvc1zAVdIudqgLLQgonxuQUzBbC03U5hzhzpsX710HnVg/49VPmz26UeXUj
+Vu+bMlz2zcyZtQl25su0
+=1eWX
+-END PGP SIGNATURE-

Added: release/phoenix/phoenix-3.1.0-rc0/bin/phoenix-3.1.0-bin.tar.gz.md5
==
--- release/phoenix/phoenix-3.1.0-rc0/bin/phoenix-3.1.0-bin.tar.gz.md5 (added)
+++ release/phoenix/phoenix-3.1.0-rc0/bin/phoenix-3.1.0-bin.tar.gz.md5 Mon Aug 
11 18:22:57 2014
@@ -0,0 +1 @@
+ecbe2957311302a0f21e3722a8384bcc *phoenix-3.1.0-bin.tar.gz

Added: release/phoenix/phoenix-3.1.0-rc0/bin/phoenix-3.1.0-bin.tar.gz.sha
==
--- release/phoenix/phoenix-3.1.0-rc0/bin/phoenix-3.1.0-bin.tar.gz.sha (added)
+++ release/phoenix/phoenix-3.1.0-rc0/bin/phoenix-3.1.0-bin.tar.gz.sha Mon Aug 
11 18:22:57 2014
@@ -0,0 +1,2 @@
+7fb2e24bae98b03754b9edee1530c5ca7670e3600a2d33ef9a7014dc86edcea381ee87481442eac593bcefd6b6ab3d31eda9d0a370c94cf7a6b1c320772124cf
 *phoenix-3.1.0-bin.tar.gz
+390197cbcb00d8ec1c70a4b76342505156b43212e7801e861ffe70014e6f4102 
*phoenix-3.1.0-bin.tar.gz

Added: release/phoenix/phoenix-3.1.0-rc0/src/phoenix-3.1.0-src.tar.gz
==
Binary file - no diff available.

Propchange: release/phoenix/phoenix-3.1.0-rc0/src/phoenix-3.1.0-src.tar.gz
--
svn:mime-type = application/octet-stream

Added: release/phoenix/phoenix-3.1.0-rc0/src/phoenix-3.1.0-src.tar.gz.asc
==
--- release/phoenix/phoenix-3.1.0-rc0/src/phoenix-3.1.0-src.tar.gz.asc (added)
+++ release/phoenix/phoenix-3.1.0-rc0/src/phoenix-3.1.0-src.tar.gz.asc Mon Aug 
11 18:22:57 2014
@@ -0,0 +1,17 @@
+-BEGIN PGP SIGNATURE-
+Version: GnuPG v1.4.10 (GNU/Linux)
+
+iQIcBAABCgAGBQJT6QiGAAoJEC1+sLgXr24/L9wP/iVKd3tav8Ik9hfCwjXFkJjc
+cRJsse3Qir3O948EGBugWu/VpS+DGDDWNmXwCS9qKb+cZRgf2lsTzgKwVoXh+UjZ
+bYEZCNhstVwjp7K6mOtWyEd5E/go1B00pcl4Y5MTXOoLXkg8N2oyRztevJrrFYFL
+Gdzje7HNM3spidd/2LqsINC9ORTZ52m3QS3u7MKEgIhGIcdzjqDdGh9XoftpmrI9
+hR2kzVO+R5VJ1la44SE/C7hZLcspndH6S8Emn4w2h7rOXVyYNTC3cP7BJz7nWUXq
+nhQaACzo1+i8W6GLXEc9u6FXNgpcFpRrc57+GfO8jXXUdqGDlyRd0Fj9gfln/r/J
+/61vq60RxkMUCyvSMTkSQ9OC96+jzfj3aVzoik9UnAfV9c/2F7ol8YaQD+UkFMeQ
+k+fhW138/bsxMWsr3Z+aeZ6rwyp8rXRGevZNW48ZNGg/A2q62vTBjVtNdfPznwWB
+qUKgL5UhyU1E3t3kKQVhN295d0Xj1cqKAzXrUskOY48fm6jTlEq0E3EcFRVt71X2
+JqKrs9z8WXNUglSqX5aEaJwz3E9DkyObFSE25XErld+kmyWKH5oEEHBjU0mUOnUt
+jJksZ5EAIrOO262RKUw4bhAA4aQRY+2tV/c5OZZuZVA803kzxfnKOlbYZb6StgcB
+mrEIaC3asKhyNrVg5QrT
+=4TPu
+-END PGP SIGNATURE-

Added: 

Git Push Summary

2014-08-11 Thread mujtaba
Repository: phoenix
Updated Tags:  refs/tags/v3.1.0-rc0 [created] 9573dbfb7


svn commit: r6135 - /release/phoenix/KEYS

2014-08-11 Thread mujtaba
Author: mujtaba
Date: Mon Aug 11 18:32:36 2014
New Revision: 6135

Log:
Add KEYS file

Added:
release/phoenix/KEYS

Added: release/phoenix/KEYS
==
--- release/phoenix/KEYS (added)
+++ release/phoenix/KEYS Mon Aug 11 18:32:36 2014
@@ -0,0 +1,73 @@
+This file contains the PGP keys of various developers.
+Please don't use them for email unless you have to. Their main
+purpose is code signing.
+
+Examples of importing this file in your keystore:
+ gpg --import KEYS.txt
+
+Examples of adding your key to this file:
+ pgp -kxa your name and append it to this file.
+ (pgpk -ll your name  pgpk -xa your name)  this file.
+ (gpg --list-sigs your name
+  gpg --armor --export your name)  this file.
+
+
+
+pub   4096R/17AF6E3F 2014-02-07
+uid  Mujtaba Chohan (CODE SIGNING KEY) mujt...@apache.org
+sig 317AF6E3F 2014-02-07  Mujtaba Chohan (CODE SIGNING KEY) 
mujt...@apache.org
+sub   4096R/7E67B00D 2014-02-07
+sig  17AF6E3F 2014-02-07  Mujtaba Chohan (CODE SIGNING KEY) 
mujt...@apache.org
+
+-BEGIN PGP PUBLIC KEY BLOCK-
+Version: GnuPG v1.4.10 (GNU/Linux)
+
+mQINBFL1UAsBEACcbU3ztdV/TAzUUND9X3JMsDRd2uGs8YLawwM2k1IrYMb2kXXh
+2x9U+WiSf2zGWNIgQ+tB5wp1d2SdCblqOTfYUPeuEeGK1y77KZgbar1UsF+zU2ny
+wwpXH1wUIXoGswcI1WN6EkKiciQUNdKUJi9D3tkuNEc5usutdPC4rjJb3CLq2DTw
+PKID8cuTyM645K7+uKElDrGOz24PpFIYDiFyCL6GZOOLyY4nkU2kcybCB42FHD64
+w+fMEcLXp5rmNIKv/W6r4j6kL54HZO8xsLkBtbaj/dz4RXmNrVjvsTuwTd35Jcnl
+CLIKxMIC/bj3TkIHE9HFAbR7QuIPnXDR3jYL0pJbi0KOqd7klZ+qoh2QtMZrVuvQ
+LR4ALf8oJNjd9zuXWnB+e3K6g7t1ainPF3ELiOykMzrBZ9SmZSv5pW6nosZht6Tq
+D8spRM5C+Mk0ZPkCuVWOSJXmxaoU6lRenENfQuA8GruQZGTxWE5rTngd3+1wrZZu
+jQRTohtFIIWa9cAvyF7Dj/yXnSBUK/7N2z9eWglHIu64qkbQuDBIMDjb96a9FV9h
+CsR+aZqxM/dE/LUavTWyTV0NKWWG/1cPqb1/SnkKt/xMw6y4+lDp8d1C8BLWehUD
+FPXOIHubJq+UoR+zZlI7HrdCH87vKsvsP8Ayr2A7vHF/oXjg4vFS0YqvSQARAQAB
+tDZNdWp0YWJhIENob2hhbiAoQ09ERSBTSUdOSU5HIEtFWSkgPG11anRhYmFAYXBh
+Y2hlLm9yZz6JAjcEEwEKACEFAlL1UAsCGwMFCwkIBwMFFQoJCAsFFgIDAQACHgEC
+F4AACgkQLX6wuBevbj+SIw/+I9ryZVKuC5qjEOGRkCunWH7kLOMFVb2OMiQEKF7F
+SC7a4cmo9AzRZZmI/FKp3zRS0HK79HuM2CLzvMZUVAOKqZK2YDzEfwyO0pYaA+n3
+LMgdquVOvELPb6M1WSom1pbx3iX51tWpxFjodo61GCYr0xTJnoVqgiDpVCOQNLE4
+PrF1nvWaNTZ8/xEDSdPFbgP5gMLDsvhfM2P9QNOklFzlhjgMiMdz2qRUjMTNB+zo
+JA51vvr5bAs25w1y62HIbNTFuHzVrRACyDL/DfmmOB3u6cXC2cZIpWGPWwzRrmV7
+IMDvu0xKnrk6ug+63iwUlDeY6Gbc5QZjmbuGYsYTC4QhINv3iLhyFCkYRkLq82bw
+1x1ahLYtaZPlS8ulcM3g1QSBsw4u4Iv20/uxwJOdlqgMwvuK+U7EnQ1CjwofGPd4
+To47x4C5s4Pu1/MJVWuJ8ZYjQu5+nltz0LVxilZM8xg9L03++0OBOqI7lWPjkg3+
+qWAaj9lTQDHio1D88TGxih+i9cVV0fweGWm7iRPhvW4Iu2mYhairIkHIO1RVoGVM
+2uKYxr/jHR3zmK4ytn7Bfp3KEGru5gFVfDg1q0Dy3Xhud5AWQGti0Zvqel8KoGDO
+iGTJFLby1XWb2kxUEJdQsakrhoeCnVCjf4J+FUc+6zzyCBnmLqSO+lCB+GhT46XF
+hmO5Ag0EUvVQCwEQANNN8quSUuZH68Gakjt/Pmc1WZjy0sb87mkGyfmAVCvE1Nyn
+RzGfkeGn2J29+wPmn3bvPhfHa25xAgBBLXfUOh5JKbsMmfpgWr71+An4h5Eehjat
+czR5EZ2r50hqdDDvcO4jB/oXU8cWxVNZztHkqtSlltwdgwNbYp4wNNjLFqYbNP6I
+QmH6ZQYV07aN9GbCqd8WsypFq9SCRNYapiKWXNSe9zLE9no8rNG38uOmGN7zQvMA
+3lmrYZRTY/VsIeel6wyCplWDydsqJJfqe7SHXCwaGTejyScPe7brLWQRAcBJCuL/
+0ujCQCP5MEZXvINsCCHr4o1K9e4gVz45aMKf6TWQjYkV9Jz1Lp4NaM14cJzANhco
+MsqDQkz0VFHadKZltrCVEki8bA5VtYlMCDVRZMz9kfsVMcWLgXjoQJ1SMOCg9Go3
+EoQVhBvUUARl4SOqGOz6359tSrvy2jAmtomD9SeRZu0bWfMiW0nHp8/bLFYU/+tY
+4u23eFdNedCSp86FGQ072F3rdcE9OtMqca2wD4DT+hA8O9VETNCHCzAhIJEDKDud
+hc8pWwGA7jbxiV4rDPrZ7+7z9bxTpVrJfQDmruARrilxOStoMKrwV2a61NemNYjZ
+eaz+76ey/msTnUGPIxsHWByDm5m9nVXteCkM5DcejBms1mCisxplktjPcPElABEB
+AAGJAh8EGAEKAAkFAlL1UAsCGwwACgkQLX6wuBevbj9Lvw//QIKt4/uc0u35kaPG
+6DfAaQCq3G5HdyURU2OGpwID8VjSE0fClTsDUfzPheBClvvRN+tOteCrd3DXUv++
+iFprrP+Aq40zqPUx2sEc9l8D4gckXGEAp6mchpA8Rp5g4LRG7Bpon41icKBoZNpt
+/qEWLiAGJ2YTi9tApJHtobcSzjqs+tuF9CC8YMUpL5j7NjE+wTEkhXnR9kAvCok0
+R6bqCKggubu9qHLCSH+vgbEfAE+Nsd3Z6UjPoyVcES6eW9TxFneBtXU2qxDZKwRJ
+dPllgjJZj6b8NtVsMZq7PMQHYQ+dMB+t/wPvu4W0V8JbgR5GKoUMsuu1A7nUfMpX
+QdlfAhK2EKpW8zRhlGrvcgw3zTEzJlUBpxCWv2T1w9dU2Ityhbkbx+KdbVZwH261
+d7XcJnw2IkCFG+vct4h2rfVwGXqoZyYb+d2hMsPtyQjOKnZ3srzohxepH7n7e/na
+4xw9rjGvGRMWsgrbhTXjWbw1HCQG6Tzs+xGT9PbkHuGa2bba0Kq83nFqfESQ0NCA
+fMFOWYmlNK8cL+Z7zaEOFYF5ug/i0hZhhzKZc03ryhEaIEVB9Un62rLnbmgIQxF6
+8tc9Ra+TfPKIXgIY0QdjgBN5Yy6B2aBWHx2E3AkTLGyWgv/8BERDC+xCWCdORdIs
+GR2wPntvTGNAWEcuwD1/D1gws5A=
+=nO9e
+-END PGP PUBLIC KEY BLOCK-




[1/2] PHOENIX-1058 Support index region split on it's corresponding data region split (Rajeshbabu)

2014-08-11 Thread jamestaylor
Repository: phoenix
Updated Branches:
  refs/heads/4.0 2ed929a75 - 4fa6146b3


http://git-wip-us.apache.org/repos/asf/phoenix/blob/4fa6146b/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMaintainer.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMaintainer.java 
b/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMaintainer.java
index c8c031b..68cdb26 100644
--- a/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMaintainer.java
+++ b/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMaintainer.java
@@ -525,6 +525,38 @@ public class IndexMaintainer implements Writable, 
IterableColumnReference {
 }
 }
 
+/*
+ * return the view index id from the index row key
+ */
+public byte[] getViewIndexIdFromIndexRowKey(ImmutableBytesWritable 
indexRowKeyPtr) {
+assert(isLocalIndex);
+RowKeySchema indexRowKeySchema = getIndexRowKeySchema();
+// TODO add logic to skip region start key as well because we cannot 
find the region startkey in indexhalfstorefilereader.
+ImmutableBytesWritable ptr = new ImmutableBytesWritable();
+TrustedByteArrayOutputStream stream =
+new TrustedByteArrayOutputStream(estimatedIndexRowKeyBytes);
+DataOutput output = new DataOutputStream(stream);
+try {
+int indexPosOffset = (!isLocalIndex  nIndexSaltBuckets  0 ? 1 : 
0) + (isMultiTenant ? 1 : 0) + (viewIndexId == null ? 0 : 1);
+Boolean hasValue =
+indexRowKeySchema.iterator(indexRowKeyPtr, ptr, 
indexPosOffset);
+if (Boolean.TRUE.equals(hasValue)) {
+output.write(ptr.get(), ptr.getOffset(), ptr.getLength());
+}
+int length = stream.size();
+byte[] dataRowKey = stream.getBuffer();
+return dataRowKey.length == length ? dataRowKey : 
Arrays.copyOf(dataRowKey, length);
+} catch (IOException e) {
+throw new RuntimeException(e); // Impossible
+} finally {
+try {
+stream.close();
+} catch (IOException e) {
+throw new RuntimeException(e); // Impossible
+}
+}
+}
+
 private volatile RowKeySchema indexRowKeySchema;
 
 // We have enough information to generate the index row key schema

http://git-wip-us.apache.org/repos/asf/phoenix/blob/4fa6146b/phoenix-core/src/main/java/org/apache/phoenix/iterate/LocalIndexParallelIteratorRegionSplitter.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/iterate/LocalIndexParallelIteratorRegionSplitter.java
 
b/phoenix-core/src/main/java/org/apache/phoenix/iterate/LocalIndexParallelIteratorRegionSplitter.java
index c3a38d5..0ccd738 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/iterate/LocalIndexParallelIteratorRegionSplitter.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/iterate/LocalIndexParallelIteratorRegionSplitter.java
@@ -37,6 +37,7 @@ public class LocalIndexParallelIteratorRegionSplitter extends 
DefaultParallelIte
 
 @Override
 protected ListHRegionLocation getAllRegions() throws SQLException {
+
context.getConnection().getQueryServices().clearTableRegionCache(tableRef.getTable().getPhysicalName().getBytes());
 return 
context.getConnection().getQueryServices().getAllTableRegions(tableRef.getTable().getPhysicalName().getBytes());
 }
 

http://git-wip-us.apache.org/repos/asf/phoenix/blob/4fa6146b/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
 
b/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
index ee0be95..4965813 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
@@ -59,6 +59,8 @@ import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
 import org.apache.hadoop.hbase.ipc.BlockingRpcCallback;
 import org.apache.hadoop.hbase.ipc.ServerRpcController;
 import org.apache.hadoop.hbase.protobuf.generated.ClientProtos.MutationProto;
+import org.apache.hadoop.hbase.regionserver.IndexHalfStoreFileReaderGenerator;
+import org.apache.hadoop.hbase.regionserver.LocalIndexSplitter;
 import org.apache.hadoop.hbase.security.User;
 import org.apache.hadoop.hbase.util.Bytes;
 import org.apache.hadoop.hbase.util.Pair;
@@ -620,6 +622,21 @@ public class ConnectionQueryServicesImpl extends 
DelegateQueryServices implement
 Indexer.enableIndexing(descriptor, PhoenixIndexBuilder.class, 
opts);
 }
 
+

[2/2] git commit: PHOENIX-1058 Support index region split on it's corresponding data region split (Rajeshbabu)

2014-08-11 Thread jamestaylor
PHOENIX-1058 Support index region split on it's corresponding data region split 
(Rajeshbabu)


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/4fa6146b
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/4fa6146b
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/4fa6146b

Branch: refs/heads/4.0
Commit: 4fa6146b3a2b0352f2b71e176a36d08499307438
Parents: 2ed929a
Author: Rajeshbabu Chintaguntla rajeshbabu.chintagun...@huawei.com
Authored: Mon Aug 11 23:53:34 2014 +0530
Committer: James Taylor jtay...@salesforce.com
Committed: Mon Aug 11 16:26:01 2014 -0700

--
 .../org/apache/phoenix/end2end/BaseQueryIT.java |   6 +
 .../org/apache/phoenix/end2end/QueryIT.java |  24 +-
 .../phoenix/end2end/index/LocalIndexIT.java | 110 +++
 .../regionserver/IndexHalfStoreFileReader.java  | 458 +
 .../IndexHalfStoreFileReaderGenerator.java  | 159 +++
 .../regionserver/IndexSplitTransaction.java | 974 +++
 .../hbase/regionserver/LocalIndexSplitter.java  | 101 ++
 .../apache/phoenix/index/IndexMaintainer.java   |  32 +
 ...ocalIndexParallelIteratorRegionSplitter.java |   1 +
 .../query/ConnectionQueryServicesImpl.java  |  17 +
 .../java/org/apache/phoenix/query/BaseTest.java |   9 +-
 11 files changed, 1876 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/4fa6146b/phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseQueryIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseQueryIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseQueryIT.java
index d736612..4263dd2 100644
--- a/phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseQueryIT.java
+++ b/phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseQueryIT.java
@@ -113,6 +113,12 @@ public abstract class BaseQueryIT extends 
BaseClientManagedTimeIT {
 + B_STRING,  + A_DATE) });
 testCases.add(new String[] { CREATE INDEX  + ATABLE_INDEX_NAME +  
ON aTable (a_integer) INCLUDE (
 + A_STRING,  + B_STRING,  + A_DATE) });
+testCases.add(new String[] { CREATE LOCAL INDEX  + ATABLE_INDEX_NAME 
+  ON aTable (a_integer DESC) INCLUDE (
++ A_STRING,  + B_STRING,  + A_DATE) });
+testCases.add(new String[] { CREATE LOCAL INDEX  + ATABLE_INDEX_NAME 
+  ON aTable (a_integer, a_string) INCLUDE (
++ B_STRING,  + A_DATE) });
+testCases.add(new String[] { CREATE LOCAL INDEX  + ATABLE_INDEX_NAME 
+  ON aTable (a_integer) INCLUDE (
++ A_STRING,  + B_STRING,  + A_DATE) });
 testCases.add(new String[] {  });
 return testCases;
 }

http://git-wip-us.apache.org/repos/asf/phoenix/blob/4fa6146b/phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryIT.java
--
diff --git a/phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryIT.java
index d9c3862..35140f4 100644
--- a/phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryIT.java
+++ b/phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryIT.java
@@ -739,19 +739,17 @@ public class QueryIT extends BaseQueryIT {
 
 byte[] tableName = Bytes.toBytes(ATABLE_NAME);
 admin = 
conn.unwrap(PhoenixConnection.class).getQueryServices().getAdmin();
-if 
(admin.tableExists(TableName.valueOf(MetaDataUtil.getLocalIndexTableName(atable
 {
-HTable htable = (HTable) 
conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(tableName);
-htable.clearRegionCache();
-int nRegions = htable.getRegionLocations().size();
-admin.split(tableName, 
ByteUtil.concat(Bytes.toBytes(tenantId), Bytes.toBytes(00A + 
Character.valueOf((char) ('3' + nextRunCount())) + ts))); // vary split point 
with test run
-int retryCount = 0;
-do {
-Thread.sleep(2000);
-retryCount++;
-//htable.clearRegionCache();
-} while (retryCount  10  htable.getRegionLocations().size() 
== nRegions);
-assertNotEquals(nRegions, htable.getRegionLocations().size());
-}
+HTable htable = (HTable) 
conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(tableName);
+htable.clearRegionCache();
+int nRegions = htable.getRegionLocations().size();
+admin.split(tableName, ByteUtil.concat(Bytes.toBytes(tenantId), 
Bytes.toBytes(00A + Character.valueOf((char) ('3' + 

[1/2] PHOENIX-1058 Support index region split on it's corresponding data region split (Rajeshbabu)

2014-08-11 Thread jamestaylor
Repository: phoenix
Updated Branches:
  refs/heads/master 416b8a67a - bdf873f5a


http://git-wip-us.apache.org/repos/asf/phoenix/blob/bdf873f5/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMaintainer.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMaintainer.java 
b/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMaintainer.java
index c8c031b..68cdb26 100644
--- a/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMaintainer.java
+++ b/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMaintainer.java
@@ -525,6 +525,38 @@ public class IndexMaintainer implements Writable, 
IterableColumnReference {
 }
 }
 
+/*
+ * return the view index id from the index row key
+ */
+public byte[] getViewIndexIdFromIndexRowKey(ImmutableBytesWritable 
indexRowKeyPtr) {
+assert(isLocalIndex);
+RowKeySchema indexRowKeySchema = getIndexRowKeySchema();
+// TODO add logic to skip region start key as well because we cannot 
find the region startkey in indexhalfstorefilereader.
+ImmutableBytesWritable ptr = new ImmutableBytesWritable();
+TrustedByteArrayOutputStream stream =
+new TrustedByteArrayOutputStream(estimatedIndexRowKeyBytes);
+DataOutput output = new DataOutputStream(stream);
+try {
+int indexPosOffset = (!isLocalIndex  nIndexSaltBuckets  0 ? 1 : 
0) + (isMultiTenant ? 1 : 0) + (viewIndexId == null ? 0 : 1);
+Boolean hasValue =
+indexRowKeySchema.iterator(indexRowKeyPtr, ptr, 
indexPosOffset);
+if (Boolean.TRUE.equals(hasValue)) {
+output.write(ptr.get(), ptr.getOffset(), ptr.getLength());
+}
+int length = stream.size();
+byte[] dataRowKey = stream.getBuffer();
+return dataRowKey.length == length ? dataRowKey : 
Arrays.copyOf(dataRowKey, length);
+} catch (IOException e) {
+throw new RuntimeException(e); // Impossible
+} finally {
+try {
+stream.close();
+} catch (IOException e) {
+throw new RuntimeException(e); // Impossible
+}
+}
+}
+
 private volatile RowKeySchema indexRowKeySchema;
 
 // We have enough information to generate the index row key schema

http://git-wip-us.apache.org/repos/asf/phoenix/blob/bdf873f5/phoenix-core/src/main/java/org/apache/phoenix/iterate/LocalIndexParallelIteratorRegionSplitter.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/iterate/LocalIndexParallelIteratorRegionSplitter.java
 
b/phoenix-core/src/main/java/org/apache/phoenix/iterate/LocalIndexParallelIteratorRegionSplitter.java
index c3a38d5..0ccd738 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/iterate/LocalIndexParallelIteratorRegionSplitter.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/iterate/LocalIndexParallelIteratorRegionSplitter.java
@@ -37,6 +37,7 @@ public class LocalIndexParallelIteratorRegionSplitter extends 
DefaultParallelIte
 
 @Override
 protected ListHRegionLocation getAllRegions() throws SQLException {
+
context.getConnection().getQueryServices().clearTableRegionCache(tableRef.getTable().getPhysicalName().getBytes());
 return 
context.getConnection().getQueryServices().getAllTableRegions(tableRef.getTable().getPhysicalName().getBytes());
 }
 

http://git-wip-us.apache.org/repos/asf/phoenix/blob/bdf873f5/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
 
b/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
index 9bfd0df..94a6b12 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
@@ -59,6 +59,8 @@ import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
 import org.apache.hadoop.hbase.ipc.BlockingRpcCallback;
 import org.apache.hadoop.hbase.ipc.ServerRpcController;
 import org.apache.hadoop.hbase.protobuf.generated.ClientProtos.MutationProto;
+import org.apache.hadoop.hbase.regionserver.IndexHalfStoreFileReaderGenerator;
+import org.apache.hadoop.hbase.regionserver.LocalIndexSplitter;
 import org.apache.hadoop.hbase.security.User;
 import org.apache.hadoop.hbase.util.Bytes;
 import org.apache.hadoop.hbase.util.Pair;
@@ -620,6 +622,21 @@ public class ConnectionQueryServicesImpl extends 
DelegateQueryServices implement
 Indexer.enableIndexing(descriptor, PhoenixIndexBuilder.class, 
opts);
 }
 
+ 

[2/2] git commit: PHOENIX-1058 Support index region split on it's corresponding data region split (Rajeshbabu)

2014-08-11 Thread jamestaylor
PHOENIX-1058 Support index region split on it's corresponding data region split 
(Rajeshbabu)


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/bdf873f5
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/bdf873f5
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/bdf873f5

Branch: refs/heads/master
Commit: bdf873f5a0789420f02f9f0a303b91a36ce85d39
Parents: 416b8a6
Author: Rajeshbabu Chintaguntla rajeshbabu.chintagun...@huawei.com
Authored: Mon Aug 11 23:53:34 2014 +0530
Committer: James Taylor jtay...@salesforce.com
Committed: Mon Aug 11 16:27:44 2014 -0700

--
 .../org/apache/phoenix/end2end/BaseQueryIT.java |   6 +
 .../org/apache/phoenix/end2end/QueryIT.java |  24 +-
 .../phoenix/end2end/index/LocalIndexIT.java | 110 +++
 .../regionserver/IndexHalfStoreFileReader.java  | 458 +
 .../IndexHalfStoreFileReaderGenerator.java  | 159 +++
 .../regionserver/IndexSplitTransaction.java | 974 +++
 .../hbase/regionserver/LocalIndexSplitter.java  | 101 ++
 .../apache/phoenix/index/IndexMaintainer.java   |  32 +
 ...ocalIndexParallelIteratorRegionSplitter.java |   1 +
 .../query/ConnectionQueryServicesImpl.java  |  17 +
 .../java/org/apache/phoenix/query/BaseTest.java |   9 +-
 11 files changed, 1876 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/bdf873f5/phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseQueryIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseQueryIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseQueryIT.java
index d736612..4263dd2 100644
--- a/phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseQueryIT.java
+++ b/phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseQueryIT.java
@@ -113,6 +113,12 @@ public abstract class BaseQueryIT extends 
BaseClientManagedTimeIT {
 + B_STRING,  + A_DATE) });
 testCases.add(new String[] { CREATE INDEX  + ATABLE_INDEX_NAME +  
ON aTable (a_integer) INCLUDE (
 + A_STRING,  + B_STRING,  + A_DATE) });
+testCases.add(new String[] { CREATE LOCAL INDEX  + ATABLE_INDEX_NAME 
+  ON aTable (a_integer DESC) INCLUDE (
++ A_STRING,  + B_STRING,  + A_DATE) });
+testCases.add(new String[] { CREATE LOCAL INDEX  + ATABLE_INDEX_NAME 
+  ON aTable (a_integer, a_string) INCLUDE (
++ B_STRING,  + A_DATE) });
+testCases.add(new String[] { CREATE LOCAL INDEX  + ATABLE_INDEX_NAME 
+  ON aTable (a_integer) INCLUDE (
++ A_STRING,  + B_STRING,  + A_DATE) });
 testCases.add(new String[] {  });
 return testCases;
 }

http://git-wip-us.apache.org/repos/asf/phoenix/blob/bdf873f5/phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryIT.java
--
diff --git a/phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryIT.java
index d9c3862..35140f4 100644
--- a/phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryIT.java
+++ b/phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryIT.java
@@ -739,19 +739,17 @@ public class QueryIT extends BaseQueryIT {
 
 byte[] tableName = Bytes.toBytes(ATABLE_NAME);
 admin = 
conn.unwrap(PhoenixConnection.class).getQueryServices().getAdmin();
-if 
(admin.tableExists(TableName.valueOf(MetaDataUtil.getLocalIndexTableName(atable
 {
-HTable htable = (HTable) 
conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(tableName);
-htable.clearRegionCache();
-int nRegions = htable.getRegionLocations().size();
-admin.split(tableName, 
ByteUtil.concat(Bytes.toBytes(tenantId), Bytes.toBytes(00A + 
Character.valueOf((char) ('3' + nextRunCount())) + ts))); // vary split point 
with test run
-int retryCount = 0;
-do {
-Thread.sleep(2000);
-retryCount++;
-//htable.clearRegionCache();
-} while (retryCount  10  htable.getRegionLocations().size() 
== nRegions);
-assertNotEquals(nRegions, htable.getRegionLocations().size());
-}
+HTable htable = (HTable) 
conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(tableName);
+htable.clearRegionCache();
+int nRegions = htable.getRegionLocations().size();
+admin.split(tableName, ByteUtil.concat(Bytes.toBytes(tenantId), 
Bytes.toBytes(00A + Character.valueOf((char) ('3' + 

Apache-Phoenix | Master | Hadoop1 | Build Successful

2014-08-11 Thread Apache Jenkins Server
Master branch build status Successful
Source repository https://git-wip-us.apache.org/repos/asf/incubator-phoenix.git

Last Successful Compiled Artifacts https://builds.apache.org/job/Phoenix-master-hadoop1/lastSuccessfulBuild/artifact/

Last Complete Test Report https://builds.apache.org/job/Phoenix-master-hadoop1/lastCompletedBuild/testReport/

Changes
[jtaylor] PHOENIX-1058 Support index region split on it's corresponding data region split (Rajeshbabu)



Jenkins build is back to stable : Phoenix | 4.0 | Hadoop2 #24

2014-08-11 Thread Apache Jenkins Server
See https://builds.apache.org/job/Phoenix-4.0-hadoop2/24/changes