Re: [ANNOUNCE] New Hive Committer - John Pullokkaran

2014-10-01 Thread Szehon Ho
Congratulations John! On Wed, Oct 1, 2014 at 10:57 AM, Gunther Hagleitner < ghagleit...@hortonworks.com> wrote: > Congrats John! > > Thanks, > Gunther. > > On Wed, Oct 1, 2014 at 10:52 AM, Eugene Koifman > wrote: > > > Congrats! > > > > On Wed, Oct 1, 2014 at 9:35 AM, Vikram Dixit K > > wrote:

Re: Number of unit tests

2014-10-10 Thread Szehon Ho
I see vector and some other tests have stopped running now. Couldn't find any reason from a quick glance. Thanks Szehon On Fri, Oct 10, 2014 at 10:30 AM, Gunther Hagleitner < gunther.hagleit...@gmail.com> wrote: > Does anyone know why our test count has dropped recently from 6k to 4k? > > This

Re: Number of unit tests

2014-10-11 Thread Szehon Ho
.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.tool/ > > > > 5 > > org.apache.hadoop.hive.ql.udaf > > < > http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apach

Enabling hadoop-1 in precommit builds

2014-10-14 Thread Szehon Ho
Hi all, After we enabled hadoop-1 builds and after compilation errors are fixed, looks like there are 198 test failures: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-TRUNK-HADOOP-1/20/ By switching builds to hadoop-2, there's a pingpong so that they aren't catching hadoop-1

Re: Build failure on trunk

2014-10-24 Thread Szehon Ho
I think it got stuck behind the daily hadoop-1 build. Those are failing and taking a long time (some groups of Minimr tests take over an hour). On Fri, Oct 24, 2014 at 2:29 PM, Prasanth Jayachandran < pjayachand...@hortonworks.com> wrote: > Looks like the patch did not apply cleanly on trunk. St

Re: Review Request 27247: convert joinOp to MapJoinOp and generate MapWorks only

2014-10-27 Thread Szehon Ho
ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkCompiler.java <https://reviews.apache.org/r/27247/#comment99836> I don't see this class attached ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkCompiler.java <https://reviews.apache.org/r/27247/#comment99837> Let&#

Re: [ANNOUNCE] New Hive PMC Member - Alan Gates

2014-10-27 Thread Szehon Ho
Congratulations, Alan! On Mon, Oct 27, 2014 at 4:17 PM, Hari Subramaniyan < hsubramani...@hortonworks.com> wrote: > Congrats Alan! > > Thanks > Hari > > On Mon, Oct 27, 2014 at 4:08 PM, Vaibhav Gumashta < > vgumas...@hortonworks.com > > wrote: > > > Congratulations Alan! > > > > On Mon, Oct 27, 2

Review Request 27265: Support SMB Join for Hive on Spark [Spark Branch]

2014-10-27 Thread Szehon Ho
--- Ran existing auto_sortmerge_* tests. Thanks, Szehon Ho

Re: pre-commit test scripts

2014-10-27 Thread Szehon Ho
The test-run script of precommit is here : https://github.com/apache/hive/blob/trunk/testutils/ptest2/src/main/resources/batch-exec.vm It references some properties that are set in properties file. Right now those prop files live on build-machine, but we are planning eventually to source-control

Re: Review Request 27265: Support SMB Join for Hive on Spark [Spark Branch]

2014-10-28 Thread Szehon Ho
sting --- Ran existing auto_sortmerge_* tests. Thanks, Szehon Ho

Re: Review Request 27265: Support SMB Join for Hive on Spark [Spark Branch]

2014-10-28 Thread Szehon Ho
ork get > > deferred to? It is deferred to SparkSortMergeJoinFactory - Szehon --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27265/#review58769 --

Re: Review Request 27265: Support SMB Join for Hive on Spark [Spark Branch]

2014-10-28 Thread Szehon Ho
/r/27265/#review58809 --- On Oct. 28, 2014, 10:32 p.m., Szehon Ho wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/2

Re: Review Request 27265: Support SMB Join for Hive on Spark [Spark Branch]

2014-10-29 Thread Szehon Ho
-- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27265/#review58958 ----------- On Oct. 28, 2014, 10:32 p.m., Szehon Ho wrote: > > ---

Re: Builds on hive 0.14 branch

2014-10-30 Thread Szehon Ho
Sure, I'll try to set this up. There are some (meager) instructions at https://cwiki.apache.org/confluence/display/Hive/Hive+PTest2+Infrastructure. It will time-share with the precommit tests. I'll try to set this to auto-build every other day (and set hadoop-1 to also every other day), and allo

Re: Review Request 27265: Support SMB Join for Hive on Spark [Spark Branch]

2014-10-30 Thread Szehon Ho
/clientpositive/spark/smb_mapjoin_25.q.out efa38d4 ql/src/test/results/clientpositive/spark/tez_join_tests.q.out 9254944 Diff: https://reviews.apache.org/r/27265/diff/ Testing --- Ran existing auto_sortmerge_* tests. Thanks, Szehon Ho

Re: Review Request 24962: HIVE-7730: Extend ReadEntity to add accessed columns from query

2014-08-22 Thread Szehon Ho
> On Aug. 22, 2014, 6:14 a.m., Szehon Ho wrote: > > ql/src/java/org/apache/hadoop/hive/ql/hooks/ReadEntity.java, line 54 > > <https://reviews.apache.org/r/24962/diff/1/?file=666753#file666753line54> > > > > Can we make this final, and not have a setter? Th

Review Request 25001: HIVE-7861 : Enable reduce-side join tests (1) [Spark Branch]

2014-08-22 Thread Szehon Ho
/join9.q.out PRE-CREATION ql/src/test/results/clientpositive/spark/join_1to1.q.out PRE-CREATION Diff: https://reviews.apache.org/r/25001/diff/ Testing --- Verified against MR results using Brock's script. Thanks, Szehon Ho

Re: Review Request 24962: HIVE-7730: Extend ReadEntity to add accessed columns from query

2014-08-25 Thread Szehon Ho
> On Aug. 22, 2014, 6:14 a.m., Szehon Ho wrote: > > ql/src/java/org/apache/hadoop/hive/ql/hooks/ReadEntity.java, line 54 > > <https://reviews.apache.org/r/24962/diff/1/?file=666753#file666753line54> > > > > Can we make this final, and not have a setter? Th

Re: Review Request 24688: parallel order by clause on a string column fails with IOException: Split points are out of order

2014-08-25 Thread Szehon Ho
688/#comment89742> Please add some more context to this debug statement. ql/src/java/org/apache/hadoop/hive/ql/exec/PartitionKeySampler.java <https://reviews.apache.org/r/24688/#comment89743> If needs to be exposed, message can be "Sampling generated x number of reducers

Re: Review Request 24962: HIVE-7730: Extend ReadEntity to add accessed columns from query

2014-08-26 Thread Szehon Ho
Please indent 2 spaces. - Szehon Ho On Aug. 26, 2014, 2:22 a.m., Xiaomeng Huang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > http

Re: Review Request 23320: HiveServer2 using embedded MetaStore leaks JDOPersistanceManager

2014-08-26 Thread Szehon Ho
re they are added, etc.. Also maybe we can remove Hive.java cleanup from this patch as well? - Szehon Ho On Aug. 26, 2014, 5:10 p.m., Vaibhav Gumashta wrote: > > --- > This is an automatically generated e-mail. To r

Re: Review Request 24688: parallel order by clause on a string column fails with IOException: Split points are out of order

2014-08-26 Thread Szehon Ho
me context, like "Sampled partition key: current..."? - Szehon Ho On Aug. 26, 2014, 3:51 a.m., Navis Ryu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: &

Re: Review Request 25086: HIVE-7889 : Query fails with char partition column

2014-08-26 Thread Szehon Ho
/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveCharObjectInspector.java <https://reviews.apache.org/r/25086/#comment90120> Please put a space after the cast, and let's get rid of the useless assignment in the below line, even though its like that in other inspectors.

Re: Review Request 24688: parallel order by clause on a string column fails with IOException: Split points are out of order

2014-08-27 Thread Szehon Ho
eySampler.java <https://reviews.apache.org/r/24688/#comment90312> Please add license header. - Szehon Ho On Aug. 27, 2014, 2:18 a.m., Navis Ryu wrote: > > --- > This is an automatically generated e-mail. To reply, visit:

Re: Review Request 24688: parallel order by clause on a string column fails with IOException: Split points are out of order

2014-08-27 Thread Szehon Ho
tps://reviews.apache.org/r/24688/#comment90321> Yep, thats what I meant. - Szehon Ho On Aug. 27, 2014, 2:18 a.m., Navis Ryu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.

Re: Review Request 24688: parallel order by clause on a string column fails with IOException: Split points are out of order

2014-08-28 Thread Szehon Ho
> On Aug. 28, 2014, 6:05 a.m., Szehon Ho wrote: > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 1040 > > <https://reviews.apache.org/r/24688/diff/3/?file=669965#file669965line1040> > > > > Yep, thats what I meant. > > Navis Ryu wrot

Review Request 25906: HIVE-7856 : Enable parallelism in Reduce Side Join [Spark Branch]

2014-09-22 Thread Szehon Ho
iff: https://reviews.apache.org/r/25906/diff/ Testing --- Adding a few tests that force reducers > 1, manually verified results. Thanks, Szehon Ho

Re: Review Request 25906: HIVE-7856 : Enable parallelism in Reduce Side Join [Spark Branch]

2014-09-22 Thread Szehon Ho
st/results/clientpositive/spark/parallel_join1.q.out PRE-CREATION Diff: https://reviews.apache.org/r/25906/diff/ Testing --- Adding a few tests that force reducers > 1, manually verified results. Thanks, Szehon Ho

Re: Review Request 25906: HIVE-7856 : Enable parallelism in Reduce Side Join [Spark Branch]

2014-09-24 Thread Szehon Ho
ql/src/test/results/clientpositive/spark/parallel_join1.q.out PRE-CREATION Diff: https://reviews.apache.org/r/25906/diff/ Testing --- Adding a few tests that force reducers > 1, manually verified results. Thanks, Szehon Ho

Re: Review Request 25906: HIVE-7856 : Enable parallelism in Reduce Side Join [Spark Branch]

2014-09-25 Thread Szehon Ho
t; 1, manually verified results. Thanks, Szehon Ho

Re: Review Request 25906: HIVE-7856 : Enable parallelism in Reduce Side Join [Spark Branch]

2014-09-25 Thread Szehon Ho
tive/spark/vectorized_shufflejoin.q.out bc4262b Diff: https://reviews.apache.org/r/25906/diff/ Testing --- Adding a few tests that force reducers > 1, manually verified results. Thanks, Szehon Ho

Restarting hadoop-1 builds

2014-09-25 Thread Szehon Ho
Hi all, There's been no build coverage of hadoop-1 on hive-trunk since 0.13 release. I'm planning to restart periodic hadoop-1 builds at http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-TRUNK-HADOOP-1, so we know the current state. I hope this helps for the 0.14 release and bey

Re: Start releasing the master branch

2022-02-09 Thread Szehon Ho
+1 that would be awesome to see Hive master released after so long. Either 4.0 or 4.0.0-alpha-1 makes sense to me, not sure how we would pick any 3.x or calendar date (which could tend to slip and be more confusing?). Thanks in any case to get the ball rolling. Szehon On Wed, Feb 9, 2022 at 4:55

Re: [VOTE] Apache Hive 3.1.3 Release Candidate 3

2022-04-06 Thread Szehon Ho
+1 (binding) Downloaded and ran create , insert, simple query on postgres. Verified checksums. Built from source. Thanks, Szehon On Mon, Apr 4, 2022 at 7:56 AM Naveen Gangam wrote: > *[No new commits from RC2]. Just cleaned up **apache-hive-3.1.3-src.tar.gz* > *archive* > > > Apache Hive 3.1.3

Re: [VOTE] Apache Hive 2.3.10 Release Candidate 1

2024-05-07 Thread Szehon Ho
+1 (binding) - Checked checksums - Built from source - Create and query simple table - It seems some basic query print an ugly warning, but still pass in the end. Not sure if need to change some config. 0: jdbc:hive2://> insert into test select 1; 24/05/07 12:40:34 [537b5f20-dd1a-4be6-af09-effa6

Re: [VOTE] Apache Hive 2.3.8 Release Candidate 3

2021-01-12 Thread Szehon Ho
+1 Verified signatures Created a table, insert data, ran simple queries. Thanks Chao, Szehon On Mon, Jan 11, 2021 at 5:43 AM Xuefu Zhang wrote: > +1. I performed the following: > > 1. Downloaded bin and src tarball and verified signature and checksum > 2. Freshly initiated a mysql based metast

Re: [VOTE] Apache Hive 2.3.9 Release Candidate 0

2021-06-07 Thread Szehon Ho
+1 Verified checksums, build from source, create db/table, populate data, ran some queries. Thanks Szehon On Fri, Jun 4, 2021 at 10:07 AM Xuefu Zhang wrote: > +1. Thank Chao for doing this. I performed the following actions: > > 1. Downloaded the release candidate artifacts and verified the si

Re: [VOTE] Should we release Hive Storage API 2.8.0-rc0 ?

2021-07-21 Thread Szehon Ho
+1 (binding) * Built module * Ran tests * Checked artifact checksum and signature Thanks Szehon On Tue, Jul 20, 2021 at 2:11 PM Owen O'Malley wrote: > I think we should go ahead and release storage-api 2.8.0 and catch it on > the next cycle. HIVE-25190 is a long standing bug that rarely affect

Re: Review Request 23320: HiveServer2 using embedded MetaStore leaks JDOPersistanceManager

2014-07-09 Thread Szehon Ho
his wouldn't make sense then? - Szehon Ho On July 7, 2014, 8:29 p.m., Vaibhav Gumashta wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https

Re: MiniTezCliDriver pre-commit tests are running

2014-07-13 Thread Szehon Ho
retire that wiki page). There are just 1-2 failing MiniXCliDriver tests that hasn't been run as part of pre-commit suite until this, that may show up in the failures now. Thanks Szehon On Thu, Jun 19, 2014 at 7:09 AM, Szehon Ho wrote: > (changing subject) > > The MiniTezCliDri

Re: MiniTezCliDriver pre-commit tests are running

2014-07-14 Thread Szehon Ho
< > https://cwiki.apache.org/confluence/display/Hive/Hive+PreCommit+Patch+Testing > > > > > > Page: DeveloperDocs > > <https://cwiki.apache.org/confluence/display/Hive/DeveloperDocs> > > > > -- Lefty > > > > > > On Mon, Jul 14

Re: Newbie question from an oldbie -- "local"

2014-07-18 Thread Szehon Ho
Yea they are orthogonal to me. Local mode (local MR) means execution is co-located in Hive, local metastore means the HMS (metadata service) is co-located with Hive. Thanks Szehon On Fri, Jul 18, 2014 at 12:57 PM, Lefty Leverenz wrote: > How is local mode (for execution of queries) related to

Re: Review Request 23738: HIVE-5160: HS2 should support .hiverc

2014-07-21 Thread Szehon Ho
ession/TestSessionGlobalInitFile.java <https://reviews.apache.org/r/23738/#comment84788> Small issue, can we fix the spelling for password? - Szehon Ho On July 21, 2014, 1:53 p.m., Dong Chen wrote: > > --- > This is an automatically gener

Re: Review Request 23738: HIVE-5160: HS2 should support .hiverc

2014-07-22 Thread Szehon Ho
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23738/#review48439 --- Ship it! Thanks Dong! - Szehon Ho On July 22, 2014, 8:24 a.m

Re: Review Request 23820: HIVE-7445:Improve LOGS for Hive when a query is not able to acquire locks

2014-07-22 Thread Szehon Ho
the red-space? It's two lines per indent in hive, thanks. ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java <https://reviews.apache.org/r/23820/#comment85045> Why was this necessary to change? - Szehon Ho On July 22, 2014, 7:21 p.m., Chaoyu

Re: Review Request 23820: HIVE-7445:Improve LOGS for Hive when a query is not able to acquire locks

2014-07-22 Thread Szehon Ho
> On July 22, 2014, 10:33 p.m., Szehon Ho wrote: > > Hey Chaoyu thanks, the logic looks good now. Can you also fix all the > > red-space? It's two lines per indent in hive, thanks. Just realized I had a typo, its two spaces pe

Re: Review Request 23820: HIVE-7445:Improve LOGS for Hive when a query is not able to acquire locks

2014-07-23 Thread Szehon Ho
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23820/#review48601 --- Ship it! - Szehon Ho On July 23, 2014, 12:38 p.m., Chaoyu Tang

Re: Review Request 23953: HIVE-7519: Refactor QTestUtil to remove its duplication with QFileClient for qtest setup and teardown

2014-07-25 Thread Szehon Ho
g/apache/hadoop/hive/ql/QTestUtil.java <https://reviews.apache.org/r/23953/#comment85539> Sorry as I'm not that familiar , can you explain what these codes are for? - Szehon Ho On July 25, 2014, 10:49 p.m., Ashish Singh wrote: > >

Re: Review Request 22946: HIVE-7286: Parameterize HCatMapReduceTest for testing against all Hive storage formats.

2014-07-25 Thread Szehon Ho
. ql/pom.xml <https://reviews.apache.org/r/22946/#comment85541> Why disable tests here? serde/src/test/resources/alltypes.avsc <https://reviews.apache.org/r/22946/#comment85540> Do we still need this file? - Szehon Ho On July 25, 2014, 9:25 p.m., Davi

Re: Review Request 23722: HIVE-7434: refactor csv output format with in RFC mode

2014-07-28 Thread Szehon Ho
r beeline to depend on zookeeper class, can we use another common-util? - Szehon Ho On July 22, 2014, 1:16 a.m., cheng xu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://re

Re: Review Request 23953: HIVE-7519: Refactor QTestUtil to remove its duplication with QFileClient for qtest setup and teardown

2014-07-28 Thread Szehon Ho
change there if we ever refactor, in addition to being more consistent with the other vars. - Szehon Ho On July 28, 2014, 11:16 p.m., Ashish Singh wrote: > > --- > This is an automatically generated e-mail. To reply,

Review Request 24084: HIVE-7547 - Add ipAddress and userName to ExecHook

2014-07-29 Thread Szehon Ho
non-kerberos mode. Thanks, Szehon Ho

Re: Review Request 24084: HIVE-7547 - Add ipAddress and userName to ExecHook

2014-07-29 Thread Szehon Ho
/service/cli/thrift/ThriftCLIService.java 5c87bcb Diff: https://reviews.apache.org/r/24084/diff/ Testing --- Added tests in both kerberos and non-kerberos mode. Thanks, Szehon Ho

Re: Review Request 24084: HIVE-7547 - Add ipAddress and userName to ExecHook

2014-07-29 Thread Szehon Ho
/hive/service/cli/thrift/ThriftCLIService.java 5c87bcb Diff: https://reviews.apache.org/r/24084/diff/ Testing --- Added tests in both kerberos and non-kerberos mode. Thanks, Szehon Ho

Re: Review Request 23953: HIVE-7519: Refactor QTestUtil to remove its duplication with QFileClient for qtest setup and teardown

2014-07-30 Thread Szehon Ho
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23953/#review49132 --- Ship it! Looks good to me, pending test fixes. - Szehon Ho On

Re: Review Request 24084: HIVE-7547 - Add ipAddress and userName to ExecHook

2014-07-30 Thread Szehon Ho
/service/cli/thrift/ThriftCLIService.java 5c87bcb Diff: https://reviews.apache.org/r/24084/diff/ Testing --- Added tests in both kerberos and non-kerberos mode. Thanks, Szehon Ho

Re: Review Request 24084: HIVE-7547 - Add ipAddress and userName to ExecHook

2014-07-30 Thread Szehon Ho
/ThriftCLIService.java 5c87bcb Diff: https://reviews.apache.org/r/24084/diff/ Testing --- Added tests in both kerberos and non-kerberos mode. Thanks, Szehon Ho

Re: Review Request 24086: HIVE-7348: Beeline could not parse ; separated queries provided with -e option

2014-07-31 Thread Szehon Ho
. itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java <https://reviews.apache.org/r/24086/#comment86283> We don't need 'copy' and can just pass in arglist right? And while we're at it, can we also remove the inner parens? - Szehon Ho

Re: Review Request 24211: HIVE-7554 - Parquet Hive should resolve column names in case insensitive manner

2014-08-04 Thread Szehon Ho
in the next line to again use the lower-case version to lookup from the map, or can we just do tableSchema.getType on the original col as in the earlier case? - Szehon Ho On Aug. 4, 2014, 1:27 a.m., Brock Noland wrote: > > ---

Re: Review Request 24284: HIVE-7441:Custom partition scheme gets rewritten with hive scheme upon concatenate

2014-08-05 Thread Szehon Ho
/HiveAlterHandler.java <https://reviews.apache.org/r/24284/#comment86881> Can we get rid of the space after the '!'? - Szehon Ho On Aug. 5, 2014, 1:12 a.m., Chaoyu Tang wrote: > > --- > This is an automatically generate

Re: Review Request 23799: HIVE-7390: refactor csv output format with in RFC mode and add one more option to support formatting as the csv format in hive cli

2014-08-05 Thread Szehon Ho
/BeeLine.properties <https://reviews.apache.org/r/23799/#comment86956> Do we need to put a help for 'delimiterForDSV'? - Szehon Ho On Aug. 1, 2014, 4:15 p.m., cheng xu wrote: > > --- > This is an automatically generate

Re: Review Request 24363: HIVE-7597 Support analyze table

2014-08-05 Thread Szehon Ho
/SparkCompiler.java <https://reviews.apache.org/r/24363/#comment86984> Not sure if I understand the name right, but we don't need to say 'potential' anymore? - Szehon Ho On Aug. 6, 2014, 4:53 a.m., chengxiang li wrote: > > -

Re: Review Request 23320: HiveServer2 using embedded MetaStore leaks JDOPersistanceManager

2014-08-06 Thread Szehon Ho
we move the javadoc to before annotation? service/src/java/org/apache/hive/service/server/ThreadWithGarbageCleanup.java <https://reviews.apache.org/r/23320/#comment87218> We can refer to the map variable here. - Szehon Ho On A

Review Request 24520: HIVE-7665 - Create TestSparkCliDriver to run test in spark local mode [Spark Branch]

2014-08-08 Thread Szehon Ho
rkpositive/spark_test.q.out PRE-CREATION Diff: https://reviews.apache.org/r/24520/diff/ Testing --- Ran -Dtest=TestSparkCliDriver. Added one very basic test that exercises spark as an example. Thanks, Szehon Ho

Re: Review Request 24520: HIVE-7665 - Create TestSparkCliDriver to run test in spark local mode [Spark Branch]

2014-08-08 Thread Szehon Ho
iff: https://reviews.apache.org/r/24520/diff/ Testing --- Ran -Dtest=TestSparkCliDriver. Added one very basic test that exercises spark as an example. Thanks, Szehon Ho

Re: Review Request 24520: HIVE-7665 - Create TestSparkCliDriver to run test in spark local mode [Spark Branch]

2014-08-11 Thread Szehon Ho
kCliDriver. Added one very basic test that exercises spark as an example. Thanks, Szehon Ho

Re: Review Request 24608: HIVE-7691 - Improve maintainability of testconfiguration.properties

2014-08-12 Thread Szehon Ho
. - Szehon Ho On Aug. 12, 2014, 4:25 p.m., Brock Noland wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache

Re: Review Request 24689: HIVE-7707 Optimize SparkMapRecordHandler implementation

2014-08-13 Thread Szehon Ho
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24689/#review50553 --- Ship it! Looks fine to me. - Szehon Ho On Aug. 14, 2014, 2:43

Re: Review Request 24713: HIVE-7735 : Implement Char, Varchar in ParquetSerDe

2014-08-15 Thread Szehon Ho
/org/apache/hadoop/hive/ql/metadata/VirtualColumn.java <https://reviews.apache.org/r/24713/#comment88575> Can we put VCols in a set for more efficiency, and also can we use Guava's Iterables to make this logic cleaner? - Szehon Ho On Aug. 14, 2014, 10:53 p.m., Mohit Sabh

Re: HiveQA fails due to some spark qfiles issue

2014-08-15 Thread Szehon Ho
Yea I saw that too, and it should be fixed it as of build 336. Brock and I were doing some work on spark branch test, and I guess wrong branch build properties got changed at some point? Patch in that window needs to be uploaded again though unfortunately.. On Fri, Aug 15, 2014 at 11:14 AM, Ser

Review Request 24919: HIVE-7815 : Reduce Side Join with single reducer [Spark Branch]

2014-08-20 Thread Szehon Ho
t PRE-CREATION ql/src/test/results/clientpositive/spark/join_casesensitive.q.out PRE-CREATION Diff: https://reviews.apache.org/r/24919/diff/ Testing --- Added three join tests to the TestSparkCliDriver suite. Thanks, Szehon Ho

Re: Review Request 24918: HIVE-7791 - Enable tests on Spark branch (1) [Sparch Branch]

2014-08-21 Thread Szehon Ho
parse/spark/SparkCompiler.java <https://reviews.apache.org/r/24918/#comment89216> This is a bit strange, as the resolve() method returns null. Not sure if we should assign to a variable as of now? - Szehon Ho On Aug. 21, 2014, 12:26 a.m., B

Re: Review Request 24934: HIVE-7829 - Entity.getLocation can throw an NPE

2014-08-21 Thread Szehon Ho
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24934/#review51189 --- Ship it! Ship It! - Szehon Ho On Aug. 21, 2014, 3:47 p.m

Re: Review Request 24919: HIVE-7815 : Reduce Side Join with single reducer [Spark Branch]

2014-08-21 Thread Szehon Ho
24919/diff/ Testing --- Added three join tests to the TestSparkCliDriver suite. Thanks, Szehon Ho

Re: Review Request 24919: HIVE-7815 : Reduce Side Join with single reducer [Spark Branch]

2014-08-21 Thread Szehon Ho
ps://reviews.apache.org/r/24919/#review51222 ------- On Aug. 21, 2014, 10:44 p.m., Szehon Ho wrote: > > --- > This is an automatically generated e-mail.

Re: Review Request 24919: HIVE-7815 : Reduce Side Join with single reducer [Spark Branch]

2014-08-21 Thread Szehon Ho
x27;t write this method but can we remove it? > > Szehon Ho wrote: > For this one, its actually used so cant be removed, not sure if you meant > something else? > > Brock Noland wrote: > It's a private method which only calls "new Object()" so I was thin

Re: Review Request 24962: HIVE-7730: Extend ReadEntity to add accessed columns from query

2014-08-21 Thread Szehon Ho
ache/hadoop/hive/ql/parse/SemanticAnalyzer.java <https://reviews.apache.org/r/24962/#comment89360> No need for '==true' part. ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java <https://reviews.apache.org/r/24962/#comment89362> Can we indent this co

Re: Review Request 27265: Support SMB Join for Hive on Spark [Spark Branch]

2014-10-30 Thread Szehon Ho
mb_mapjoin_25.q.out 131f72d ql/src/test/results/clientpositive/spark/tez_join_tests.q.out 9254944 Diff: https://reviews.apache.org/r/27265/diff/ Testing --- Ran existing auto_sortmerge_* tests. Thanks, Szehon Ho

Re: Builds on hive 0.14 branch

2014-10-31 Thread Szehon Ho
this helps, Szehon On Thu, Oct 30, 2014 at 11:49 AM, Szehon Ho wrote: > Sure, I'll try to set this up. There are some (meager) instructions at > https://cwiki.apache.org/confluence/display/Hive/Hive+PTest2+Infrastructure > . > > It will time-share with the precommit tests. I&#

Re: Review Request 26968: HIVE-8122: convert ExprNode to Parquet supported FilterPredict

2014-11-03 Thread Szehon Ho
capitalize this? Some minor formatting comments only. - Szehon Ho On Oct. 21, 2014, 8:13 a.m., cheng xu wrote: > > --- > This is an automatically generated e-mail. To reply, visi

Re: Review Request 27632: Enable q-tests for ANALYZE TABLE feature [Spark Branch]

2014-11-05 Thread Szehon Ho
test-driver is setup to use "fs", I dont think we can change the qtest to all use fs, as different test drivers might want to test with different configurations. Hope that makes sense, thanks. - Szehon Ho On Nov. 5, 2014, 6:27 p.m.

Re: Review Request 27640: HIVE-8700 Replace ReduceSink to HashTableSink (or equi.) for small tables [Spark Branch]

2014-11-05 Thread Szehon Ho
alize the HTSOperator's values like it does in LocalMapJoinProcFactory? ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java <https://reviews.apache.org/r/27640/#comment101367> Nit: 'parentOps' is probably a better

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-05 Thread Szehon Ho
://reviews.apache.org/r/27640/, as Suhas put it up for a separate review in combination with his patch. - Szehon Ho On Nov. 5, 2014, 5:51 p.m., Chao Sun wrote: > > --- > This is an automatically generated e-mail. To reply, visit

Re: Review Request 27640: HIVE-8700 Replace ReduceSink to HashTableSink (or equi.) for small tables [Spark Branch]

2014-11-05 Thread Szehon Ho
s some basic information. Maybe you know more about it, but was wondering do we need to save the info to a context when we cut the small-table RS from MapJoin in ReduceSinkMapJoinProc? Thanks. - Szehon Ho On Nov. 5, 2014, 8:29 p.m., Suhas Satish

Re: Review Request 27640: HIVE-8700 Replace ReduceSink to HashTableSink (or equi.) for small tables [Spark Branch]

2014-11-05 Thread Szehon Ho
> On Nov. 5, 2014, 10:41 p.m., Szehon Ho wrote: > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java, > > line 201 > > <https://reviews.apache.org/r/27640/diff/1/?file=750693#file750693line201> > > > > Hi Suhas, I was

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-07 Thread Szehon Ho
another(!) mapjoin. I dont know if this is still a valid case , but do you think this handles it, as it just indisciriminately adds it to the parent map if it has HTS? - Szehon Ho On Nov. 7, 2014, 6:07 p.m., Chao Sun wrote: > > -

Re: Pre-commit builds

2014-11-10 Thread Szehon Ho
Dug a bit on the logs on the server, saw the following message: 2014-11-10 20:10:52 ERROR TestExecutor:132 - Unxpected Error org.jclouds.aws.AWSResponseException: request POST https://ec2.us-east-1.amazonaws.com/ HTTP/1.1 failed with code 400, error: AWSError{requestId='afc1d9aa-0a43-4024-b2a9-1a

Re: [VOTE] Apache Hive 0.14.0 Release Candidate 0

2014-11-10 Thread Szehon Ho
+1, built src with hadoop-1, hadoop-2, tried binary and ran a few DDL and queries. Observation: new beeline output has an addition '--+' after every header row, different than 0.13.1. A cosmetic issue, not sure if it needs to be addressed before the release. Thanks, Szehon [image: Inline image

Re: Review Request 24609: Hive-7653: AvroSerDe does not support circular references in Schema

2014-11-10 Thread Szehon Ho
pe?) , then can't we just use a Set instead of map? If its in the set, return VOID. serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerializer.java <https://reviews.apache.org/r/24609/#comment102157> Formatting is wrong for these classes, they need to be properly ind

Re: Review Request 27917: HIVE-8839: Support "alter table .. add/replace columns cascade"

2014-11-13 Thread Szehon Ho
a/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java <https://reviews.apache.org/r/27917/#comment102982> Please fix the indent here of inside the switch statement. ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java <https://reviews.apache.org/r/27917/#comment102983> Do we

Re: Review Request 28064: HIVE-8844 Choose a persisent policy for RDD caching [Spark Branch]

2014-11-14 Thread Szehon Ho
f hardcoding, can we just use the StorageHandler.MEMORY_AND_DISK if the string is null? ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkPlanGenerator.java <https://reviews.apache.org/r/28064/#comment103339> Exception handling? - Szehon Ho On Nov. 15, 2014, 12:32 a.m.,

Re: Review Request 28064: HIVE-8844 Choose a persisent policy for RDD caching [Spark Branch]

2014-11-14 Thread Szehon Ho
> On Nov. 15, 2014, 2:08 a.m., Szehon Ho wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkPlanGenerator.java, > > line 155 > > <https://reviews.apache.org/r/28064/diff/1/?file=764643#file764643line155> > > > > Ins

Re: Review Request 28064: HIVE-8844 Choose a persisent policy for RDD caching [Spark Branch]

2014-11-14 Thread Szehon Ho
OK ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkPlanGenerator.java <https://reviews.apache.org/r/28064/#comment103350> Sounds good, but I think we can log it. - Szehon Ho On Nov. 15, 2014, 12:32 a.m., Jimmy Xiang wrote: > > --

Re: Review Request 27987: HIVE-8833 implement remote spark client

2014-11-17 Thread Szehon Ho
ark' as spark.master signify? - Szehon Ho On Nov. 17, 2014, 3:47 a.m., chengxiang li wrote: > > --- > This is an automatically generated e-mail. To reply, v

Re: Review Request 28255: HIVE-8916 : Handle user@domain username under LDAP authentication

2014-11-19 Thread Szehon Ho
consideration. service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java <https://reviews.apache.org/r/28255/#comment104245> Will be it simpler to use a regex like [^\@]+ to find this? - Szehon Ho On Nov. 19, 2014, 8:49 p.m., Mohit Sabharwal

Re: Review Request 28145: HIVE-8883 - Investigate test failures on auto_join30.q [Spark Branch]

2014-11-19 Thread Szehon Ho
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28145/#review62295 --- Ship it! Ship It! - Szehon Ho On Nov. 19, 2014, 11:57 p.m

Re: Review Request 28299: HIVE-8921 - Investigate test failure on auto_join2.q [Spark Branch]

2014-11-20 Thread Szehon Ho
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28299/#review62490 --- Ship it! Ship It! - Szehon Ho On Nov. 21, 2014, 1:28 a.m., Chao

Re: Propose to put JIRA traffic on separate hive list

2014-11-21 Thread Szehon Ho
+1. Create + resolve might be a good idea too, as I imagine contributors might want to see what is changing in Hive, and not just issues created. Thanks, Szehon On Fri, Nov 21, 2014 at 1:15 PM, Prasanth Jayachandran < pjayachand...@hortonworks.com> wrote: > +1 > > > - Prasanth > > On Fri, Nov 2

Review Request 28500: HIVE-8943 : Fix memory limit check for combine nested mapjoins [Spark Branch]

2014-11-26 Thread Szehon Ho
checks that algorithm does not put more than 1 mapjoin in one BaseWork 2. auto_join_stats2, which is the same query without memory limit, and check that algorithm puts all mapjoin in one BaseWork because it can. Thanks, Szehon Ho

Re: Review Request 28500: HIVE-8943 : Fix memory limit check for combine nested mapjoins [Spark Branch]

2014-12-01 Thread Szehon Ho
rithm does not put more than 1 mapjoin in one BaseWork 2. auto_join_stats2, which is the same query without memory limit, and check that algorithm puts all mapjoin in one BaseWork because it can. Thanks, Szehon Ho

  1   2   3   4   5   6   7   8   9   10   >