[jira] [Created] (HAWQ-845) Parameter kerberos principal name for HAWQ

[bhuvnesh chaudhary (JIRA)]

bhuvnesh chaudhary created HAWQ-845:
---

 Summary: Parameter kerberos principal name for HAWQ
 Key: HAWQ-845
 URL: https://issues.apache.org/jira/browse/HAWQ-845
 Project: Apache HAWQ
  Issue Type: Bug
Reporter: bhuvnesh chaudhary
Assignee: Lei Chang


Currently HAWQ only accept the principle 'postgres' for kerberos settings.
This is because there its hardcoded in gpcheckhdfs, we should ensure that it 
can be parameterized.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-844) Please remove your private branch from apache hawq project on githup

[Ming LI (JIRA)]

Ming LI created HAWQ-844:


 Summary: Please remove your private branch from apache hawq 
project on githup
 Key: HAWQ-844
 URL: https://issues.apache.org/jira/browse/HAWQ-844
 Project: Apache HAWQ
  Issue Type: Task
Reporter: Ming LI
Assignee: Lei Chang


We are planning the new release on apache hawq, it is better to keep the git 
branch list clean, however there are a lot private branch on 
https://github.com/apache/incubator-hawq/branches, which make user/developer 
confusing. 

Could you please remove those private branch from the public hawq repo?  I 
don't know who have privilege to remove it, maybe the branch owner can have.

FYI, All developer who want to checkin your code change, the better way is as 
below: 
1) Login your githup account, and clone hawq repo to your private repo. 
2) Create a git branch locally, commit your code changes in this new branch
3) Push this new branch to your private repo on githup
4) Go to  https://github.com/apache/incubator-hawq to create pull request.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: First HAWQ release

[Radar Da lei]

Hi Vineet,

Quicklz already been removed. I will cut a branch now.

Thanks.

Regards,
Radar

On Tue, Jun 21, 2016 at 9:27 AM, Vineet Goel  wrote:

> +1 to the proposed release name and cutting a release branch asap.
> Sounds like we need to find a good commit point right after the Apache
> release requirements were met (quicklz removal etc).
>
> -Vineet
>
>
>
> On Mon, Jun 20, 2016 at 2:13 PM, Ting(Goden) Yao  wrote:
>
> > I can start the voting process - Also I'd like to change the release name
> > to "2.0.0-incubating" , removing the beta tag given we've passed that
> point
> > in the timeline.
> > I also need help to cut a clean release branch as there's been so many
> > commits coming in which is not in the original scope - can anyone who
> made
> > the commits help me on that?
> >
> > -Goden
> >
> >
> > On Fri, Jun 17, 2016 at 9:37 PM Lei Chang  wrote:
> >
> > > A summary on the JIRAs left:
> > >
> > > HAWQ-783: it needs catalog changes, so I think it is better to put it
> > into
> > > following release and the impact is small.
> > > HAWQ-460: resolved since thrift library is in.
> > >
> > > Looks we are ready to go for voting if no other issues left or
> concerns!
> > >
> > > Cheers
> > > Lei
> > >
> > >
> > > On Sat, Jun 18, 2016 at 7:47 AM, Ting(Goden) Yao 
> > wrote:
> > >
> > > > I still have concerns that left out HAWQ-783 is not a clean cut for
> > this
> > > > release.
> > > > Guo - can you shed some light on what's the impact of leaving this to
> > the
> > > > future?
> > > > I may also go through the previous commits and understand the
> changes.
> > > >
> > > > On Fri, Jun 17, 2016 at 4:26 PM Roman Shaposhnik <
> ro...@shaposhnik.org
> > >
> > > > wrote:
> > > >
> > > > > On Thu, Jun 16, 2016 at 6:58 PM, Guo Gang 
> wrote:
> > > > > > HAWQ-783 (Remove quickly in metadata) is a JIRA for future work.
> > This
> > > > is
> > > > > not
> > > > > > for the shortcoming next release. Please remove the jira from the
> > > > > blocking
> > > > > > lists.
> > > > >
> > > > > Ok, if we're down to HAWQ-460 as the only release blocker, can
> > somebody
> > > > > please
> > > > > provide an update? The last comment on the JIRA was more than a
> month
> > > > ago.
> > > > >
> > > > > Thanks,
> > > > > Roman.
> > > > >
> > > >
> > >
> >
>

[jira] [Created] (HAWQ-843) HAWQ 2.0 new error handling mechanism implementation

[Lili Ma (JIRA)]

Lili Ma created HAWQ-843:


 Summary: HAWQ 2.0 new error handling mechanism implementation
 Key: HAWQ-843
 URL: https://issues.apache.org/jira/browse/HAWQ-843
 Project: Apache HAWQ
  Issue Type: Wish
Reporter: Lili Ma
Assignee: Lei Chang


As a HAWQ user, I want other QEs of the same query still keep alive when one QE 
fails, so that I can reuse the alive QEs to execute the following queries.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-842) Failed to acquire resource from resource manager

[Bill Wailliam (JIRA)]

Bill Wailliam created HAWQ-842:
--

 Summary: Failed to acquire resource from resource manager
 Key: HAWQ-842
 URL: https://issues.apache.org/jira/browse/HAWQ-842
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Resource Manager
Reporter: Bill Wailliam
Assignee: Lei Chang


This is the pg_log:
2016-06-20 17:56:03.864644 
CST,,,p526096,th5032941760,,,seg-1,"LOG","0","database system 
was shut down at 2016-06-20 17:54:32 CST",,,0,,"xlog.c",6205,
2016-06-20 17:56:03.864908 
CST,,,p526096,th5032941760,,,seg-1,"LOG","0","checkpoint record 
is at 0/2672EF8",,,0,,"xlog.c",6304,
2016-06-20 17:56:03.864923 
CST,,,p526096,th5032941760,,,seg-1,"LOG","0","redo record is at 
0/2672EF8; undo record is at 0/0; shutdown TRUE",,,0,,"xlog.c",6338,
2016-06-20 17:56:03.864933 
CST,,,p526096,th5032941760,,,seg-1,"LOG","0","next transaction 
ID: 0/1284; next OID: 16514",,,0,,"xlog.c",6342,
2016-06-20 17:56:03.864942 
CST,,,p526096,th5032941760,,,seg-1,"LOG","0","next MultiXactId: 
1; next MultiXactOffset: 0",,,0,,"xlog.c",6345,
2016-06-20 17:56:03.864951 
CST,,,p526096,th5032941760,,,seg-1,"LOG","0","end of 
transaction log location is 0/2672F48",,,0,,"xlog.c",6582,
2016-06-20 17:56:03.865750 
CST,,,p526096,th5032941760,,,seg-1,"LOG","0","Oldest active 
transaction from prepared transactions 1284",,,0,,"xlog.c",5996,
2016-06-20 17:56:03.867372 
CST,,,p526096,th5032941760,,,seg-1,"LOG","0","database system 
is ready",,,0,,"xlog.c",6022,
2016-06-20 17:56:03.867394 
CST,,,p526096,th5032941760,,,seg-1,"LOG","0","PostgreSQL 8.2.15 
(Greenplum Database 4.2.0 build 1) (HAWQ 2.0.0.0 build dev) on 
x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.8.0 compiled on Jun 19 
2016 03:02:01",,,0,,"xlog.c",6032,
2016-06-20 17:56:03.868503 
CST,,,p526096,th5032941760,,,seg-1,"LOG","0","Finished normal 
startup for clean shutdown case",,,0,,"xlog.c",6810,
2016-06-20 17:56:03.876213 
CST,,,p526097,th5032941760,,,seg-1,"LOG","0","Finished startup 
integrity checking",,,0,,"xlog.c",7159,
2016-06-20 17:56:03.879998 
CST,,,p526104,th5032941760,,,seg-1,"LOG","0","HAWQ Segment RM 
:: Temporary directory /data1/hawq/tmp",,,0,,"resourcemanager.c",1055,
2016-06-20 17:56:03.880039 
CST,,,p526104,th5032941760,,,seg-1,"LOG","0","checkAndBuildFailedTmpDirList
 finished checking temporary directory, which costs 41 
us",,,0,,"resourcemanager_RMSEG.c",274,
2016-06-20 17:56:03.883958 
CST,,,p526104,th5032941760,con4,,seg-1,"LOG","0","YARN mode 
resource broker created resource broker process 
PID=526105.",,,0,,"resourcebroker_LIBYARN.c",158,
2016-06-20 17:56:03.884155 
CST,,,p526105,th5032941760,con4,,seg-1,"LOG","0","YARN mode 
resource broker accepted YARN connection arguments : YARN Server 
RM_IP_XX:8032 Scheduler server RM_IP_XX:8030 Queue hawq Application 
name hawq, by user:postgres",,,0,,"resourcebroker_LIBYARN_proc.c",501,
2016-06-20 17:56:03.884283 
CST,,,p526104,th5032941760,con4,,seg-1,"LOG","0","Resource 
manager starts accepting resource request. Listening normal socket port 5437. 
Total listened 1 FDs.",,,0,,"resourcemanager.c",2492,
2016-06-20 17:56:03.884378 
CST,,,p526094,th5032941760,,,seg-1,"LOG","0","Wait for HAWQ RM 
-1",,,0,,"resourcemanager.c",421,
2016-06-20 17:56:03.884409 
CST,,,p526094,th5032941760,,,seg-1,"LOG","0","HAWQ :: Received 
signal notification that HAWQ RM works now.",,,0,,"resourcemanager.c",429,
2016-06-20 17:56:03.884424 
CST,,,p526094,th5032941760,,,seg-1,"LOG","0","PostgreSQL 8.2.15 
(Greenplum Database 4.2.0 build 1) (HAWQ 2.0.0.0 build dev) on 
x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.8.0 compiled on Jun 19 
2016 03:02:03",,,0,,"postmaster.c",3694,
2016-06-20 17:56:03.884441 
CST,,,p526094,th5032941760,,,seg-1,"LOG","0","database system 
is ready to accept connections","PostgreSQL 8.2.15 (Greenplum Database 4.2.0 
build 1) (HAWQ 2.0.0.0 build dev) on x86_64-unknown-linux-gnu, compiled by GCC 
gcc (GCC) 4.8.0 compiled on Jun 19 2016 03:02:03",,0,,"postmaster.c",3701,
2016-06-20 17:56:03.885673 
CST,,,p526095,th5032941760,,,seg-1,"LOG","0","3rd party error 
log:
2016-06-20 17:56:03.885486, p526105, th140510358382816, INFO ApplicationClient 
session auth method : simple""SysLoggerMain","syslogger.c",518,
2016-06-20 17:56:03.892215 
CST,,,p526104,th5032941760,con4,,seg-1,"LOG","0","Cleanup 
segment configuration catalog table 
successfully!",,,0,,"resourcepool.c",460,
2016-06-20 17:56:03.892604 

Re: First HAWQ release

[Vineet Goel]

+1 to the proposed release name and cutting a release branch asap.
Sounds like we need to find a good commit point right after the Apache
release requirements were met (quicklz removal etc).

-Vineet



On Mon, Jun 20, 2016 at 2:13 PM, Ting(Goden) Yao  wrote:

> I can start the voting process - Also I'd like to change the release name
> to "2.0.0-incubating" , removing the beta tag given we've passed that point
> in the timeline.
> I also need help to cut a clean release branch as there's been so many
> commits coming in which is not in the original scope - can anyone who made
> the commits help me on that?
>
> -Goden
>
>
> On Fri, Jun 17, 2016 at 9:37 PM Lei Chang  wrote:
>
> > A summary on the JIRAs left:
> >
> > HAWQ-783: it needs catalog changes, so I think it is better to put it
> into
> > following release and the impact is small.
> > HAWQ-460: resolved since thrift library is in.
> >
> > Looks we are ready to go for voting if no other issues left or concerns!
> >
> > Cheers
> > Lei
> >
> >
> > On Sat, Jun 18, 2016 at 7:47 AM, Ting(Goden) Yao 
> wrote:
> >
> > > I still have concerns that left out HAWQ-783 is not a clean cut for
> this
> > > release.
> > > Guo - can you shed some light on what's the impact of leaving this to
> the
> > > future?
> > > I may also go through the previous commits and understand the changes.
> > >
> > > On Fri, Jun 17, 2016 at 4:26 PM Roman Shaposhnik  >
> > > wrote:
> > >
> > > > On Thu, Jun 16, 2016 at 6:58 PM, Guo Gang  wrote:
> > > > > HAWQ-783 (Remove quickly in metadata) is a JIRA for future work.
> This
> > > is
> > > > not
> > > > > for the shortcoming next release. Please remove the jira from the
> > > > blocking
> > > > > lists.
> > > >
> > > > Ok, if we're down to HAWQ-460 as the only release blocker, can
> somebody
> > > > please
> > > > provide an update? The last comment on the JIRA was more than a month
> > > ago.
> > > >
> > > > Thanks,
> > > > Roman.
> > > >
> > >
> >
>

Re: HAWQ 795,796,797

[Lei Chang]

sounds great! +1

Cheers
Lei

On Tue, Jun 21, 2016 at 8:16 AM, Roman Shaposhnik 
wrote:

> On Fri, Jun 17, 2016 at 9:50 PM, Lei Chang  wrote:
> > My understanding of the intention here is not to maintain our own copy
> and
> > and the target is to contribute it back to ORC project. It is just for
> > improving developer efficiency that might be introduced by delay from
> > acceptance from another project.
> >
> > Hong and I had a offline discussion, I think we can have a better way for
> > this. From initial development, we even do not need to change the library
> > and if changes are needed, the proposal is to start JIRAs and submit pull
> > requests on Apache ORC.
>
> That actually would be an ideal choice. Another choice, of course, would
> be to work with ORC community to enable plug-points that would then
> enable you to replace/augment parts of ORC library functionality with
> your own code (C++ OOP is supposed to be good for that ;-)).
>
> Finally, if all else fails you can always maintain a copy of the
> library in a branch
> waiting for all your required changes to find their way into the ORC
> release
> propper.
>
> And speaking of branches: at the end of the day our #1 priority should
> be unblocking the upcoming HAWQ release. While we can keep the conversation
> going on what to do with the ORC, we need to get a release branch without
> that commit out so that we can have a release done.
>
> Makes sense?
>
> Thanks,
> Roman.
>

Re: Support orc format

[Lei Chang]

On Tue, Jun 21, 2016 at 8:38 AM, Roman Shaposhnik 
wrote:

> On Fri, Jun 17, 2016 at 3:02 AM, Ming Li  wrote:
> > Hi Guys,
> >
> > ORC (Optimized Row Columnar) is a very popular open source format adopted
> > in some major components in Hadoop eco-system. It is also used by a lot
> of
> > users. The advantages of supporting ORC storage in HAWQ are in two folds:
> > firstly, it makes HAWQ more Hadoop native which interacts with other
> > components more easily; secondly, ORC stores some meta info for query
> > optimization, thus, it might potentially outperform two native formats
> > (i.e., AO, Parquet) if it is available.
> >
> > Since there are lots of popular formats available in HDFS community, and
> > more advanced formats are emerging frequently. It is good option for HAWQ
> > to design a general framework that supports pluggable c/c++ formats such
> as
> > ORC, as well as native format such as AO and Parquet. In designing this
> > framework, we also need to support data stored in different file systems:
> > HDFS, local disk, amazon S3, etc. Thus, it is better to offer a framework
> > to support pluggable formats and pluggable file systems.
> >
> > We are proposing support ORC in JIRA (
> > https://issues.apache.org/jira/browse/HAWQ-786). Please see the design
> spec
> > in the JIRA.
> >
> > Your comments are appreciated!
>
> This sounds reasonable, but I'd like to understand the trade-offs
> between supporting
> something like ORC in PXF vs. implementing it natively in C/C++.
>
> Is there any hard performance/etc. data that you could share to
> illuminated the
> tradeoffs between these two approaches?
>

Implementing it natively in C/C++ will get at least comparable performance
with current native AO and parquet format.

And we know that ao and parquet is faster than pxf, so we are expecting
better performance here.

Cheers
Lei

Re: Support orc format

[Roman Shaposhnik]

On Fri, Jun 17, 2016 at 3:02 AM, Ming Li  wrote:
> Hi Guys,
>
> ORC (Optimized Row Columnar) is a very popular open source format adopted
> in some major components in Hadoop eco-system. It is also used by a lot of
> users. The advantages of supporting ORC storage in HAWQ are in two folds:
> firstly, it makes HAWQ more Hadoop native which interacts with other
> components more easily; secondly, ORC stores some meta info for query
> optimization, thus, it might potentially outperform two native formats
> (i.e., AO, Parquet) if it is available.
>
> Since there are lots of popular formats available in HDFS community, and
> more advanced formats are emerging frequently. It is good option for HAWQ
> to design a general framework that supports pluggable c/c++ formats such as
> ORC, as well as native format such as AO and Parquet. In designing this
> framework, we also need to support data stored in different file systems:
> HDFS, local disk, amazon S3, etc. Thus, it is better to offer a framework
> to support pluggable formats and pluggable file systems.
>
> We are proposing support ORC in JIRA (
> https://issues.apache.org/jira/browse/HAWQ-786). Please see the design spec
> in the JIRA.
>
> Your comments are appreciated!

This sounds reasonable, but I'd like to understand the trade-offs
between supporting
something like ORC in PXF vs. implementing it natively in C/C++.

Is there any hard performance/etc. data that you could share to illuminated the
tradeoffs between these two approaches?

Thanks,
Roman.

Re: HAWQ 795,796,797

[Roman Shaposhnik]

On Fri, Jun 17, 2016 at 9:50 PM, Lei Chang  wrote:
> My understanding of the intention here is not to maintain our own copy and
> and the target is to contribute it back to ORC project. It is just for
> improving developer efficiency that might be introduced by delay from
> acceptance from another project.
>
> Hong and I had a offline discussion, I think we can have a better way for
> this. From initial development, we even do not need to change the library
> and if changes are needed, the proposal is to start JIRAs and submit pull
> requests on Apache ORC.

That actually would be an ideal choice. Another choice, of course, would
be to work with ORC community to enable plug-points that would then
enable you to replace/augment parts of ORC library functionality with
your own code (C++ OOP is supposed to be good for that ;-)).

Finally, if all else fails you can always maintain a copy of the
library in a branch
waiting for all your required changes to find their way into the ORC release
propper.

And speaking of branches: at the end of the day our #1 priority should
be unblocking the upcoming HAWQ release. While we can keep the conversation
going on what to do with the ORC, we need to get a release branch without
that commit out so that we can have a release done.

Makes sense?

Thanks,
Roman.

Re: HAWQ 795,796,797

[Ting(Goden) Yao]

If there's no intention to maintain our own copy, why would we have source
code incorporated in the code base?
Can we just refer to a specific version of Apache ORC?


On Fri, Jun 17, 2016 at 9:50 PM Lei Chang  wrote:

> My understanding of the intention here is not to maintain our own copy and
> and the target is to contribute it back to ORC project. It is just for
> improving developer efficiency that might be introduced by delay from
> acceptance from another project.
>
> Hong and I had a offline discussion, I think we can have a better way for
> this. From initial development, we even do not need to change the library
> and if changes are needed, the proposal is to start JIRAs and submit pull
> requests on Apache ORC.
>
> Cheers
> Lei
>
>
> On Fri, Jun 17, 2016 at 8:32 AM, Roman Shaposhnik 
> wrote:
>
> > Guys,
> >
> > I'm looking at:
> > https://github.com/apache/incubator-hawq/pull/701/
> >
> > And I've got to say -- this gives me a bit of a concern.
> >
> > Unless I'm missing something it looks like you're planning
> > on forking the entire Apache ORC codebase and maintaining
> > it inside of the Apache HAWQ codebase. If that's the case
> > I have to say with my mentor (and frankly just a developers')
> > hat on -- I'm a strong -1 on the approach.
> >
> > Can somebody please comment?
> >
> > Thanks,
> > Roman.
> >
>

Re: First HAWQ release

[Ting(Goden) Yao]

I can start the voting process - Also I'd like to change the release name
to "2.0.0-incubating" , removing the beta tag given we've passed that point
in the timeline.
I also need help to cut a clean release branch as there's been so many
commits coming in which is not in the original scope - can anyone who made
the commits help me on that?

-Goden


On Fri, Jun 17, 2016 at 9:37 PM Lei Chang  wrote:

> A summary on the JIRAs left:
>
> HAWQ-783: it needs catalog changes, so I think it is better to put it into
> following release and the impact is small.
> HAWQ-460: resolved since thrift library is in.
>
> Looks we are ready to go for voting if no other issues left or concerns!
>
> Cheers
> Lei
>
>
> On Sat, Jun 18, 2016 at 7:47 AM, Ting(Goden) Yao  wrote:
>
> > I still have concerns that left out HAWQ-783 is not a clean cut for this
> > release.
> > Guo - can you shed some light on what's the impact of leaving this to the
> > future?
> > I may also go through the previous commits and understand the changes.
> >
> > On Fri, Jun 17, 2016 at 4:26 PM Roman Shaposhnik 
> > wrote:
> >
> > > On Thu, Jun 16, 2016 at 6:58 PM, Guo Gang  wrote:
> > > > HAWQ-783 (Remove quickly in metadata) is a JIRA for future work. This
> > is
> > > not
> > > > for the shortcoming next release. Please remove the jira from the
> > > blocking
> > > > lists.
> > >
> > > Ok, if we're down to HAWQ-460 as the only release blocker, can somebody
> > > please
> > > provide an update? The last comment on the JIRA was more than a month
> > ago.
> > >
> > > Thanks,
> > > Roman.
> > >
> >
>

Re: [VIRTUAL] TOMORROW - Meetup for Data Scientists Tuesday 6/21 - 9AM Pacific : Zeppelin meets MADlib & HAWQ

[Greg Chase]

This is a reminder about the virtual meeting tomorrow at 9AM Pacific
discussing running Apache Zeppelin with Apache MADlib and Apache HAWQ.
Join at this URL: 
https://pivotalcommunity.adobeconnect.com/madlib/

On Wed, Jun 15, 2016 at 12:20 PM, Greg Chase  wrote:

> Dear members of the Apache Zeppelin, Apache MADlib, and Apache HAWQ
> communities,
>
> We are hosting a cross-community virtual meeting for data science users
> this next Tuesday, June 21, 9AM Pacific.  No sign up is necessary, just
> join the event here .
> This meeting will also be recorded and posted.
>
> We'll be introducing users of Zeppelin to the capabilities of the MADlib
> SQL machine learning library.  We'll be showing users of MADlib how they
> can visualize their investigations and publish their work using the
> Zeppelin notebook.
>
> Agenda:
> * What is Apache Zeppelin?
> * What is Apache HAWQ (incubating) and Apache MADlib (incubating)? - 5
> min/Frank
> * Demo of Zeppelin and MADlib for data science running on HAWQ
>
> More about the technologies we'll be discussing:
>
> *Apache Zeppelin*  is a web-based notebook
> that enables interactive data analytics. It lets you make beautiful
> data-driven, interactive and collaborative documents with SQL, Scala and
> more.  Apache Zeppelin has plugins to many database engines including
> PostgreSQL, and massively parallel processing engines based off PostgreSQL
> including Apache HAWQ and the Greenplum Database.
>
> *Apache MADlib (incubating) * is a
> big data machine learning library in SQL for data scientists. It operates
> on data locally in PostgreSQL-compatible database engines. MADlib is
> optimized for parallel processing platforms such as Apache HAWQ and the
> Greenplum Database.
>
> *Apache HAWQ (incubating) * is an
> Apache Hadoop-Native SQL query engine that operates directly on data in a
> Hadoop cluster. It provides the highest degree of SQL completeness of any
> SQL on Hadoop engine. It scales elastically, is parallel processing, and
> integrates with MADlib and Zeppelin.
>
> Speakers will be:
>
> *Moon soo Lee -  CTO, NFLabs*
> LeeMoonSoo is a creator for Apache Zeppelin (incubating) and a Co-Founder,
> CTO at NFLabs.
>
> *Frank McQuillan - Director of Product Management, Pivotal Software*
> Frank McQuillan focuses on analytics and machine learning for large data
> sets.
>
> *Rahul Iyer - R Manager, Pivotal Software*
> Rahul Iyer leads the development team at Pivotal Software that contributes
> to the Apache MADlib project.
>
> See you this next Tuesday, June 21, 9AM Pacific.  No sign up is necessary,
> just join the event here
> .
>
>
>

[jira] [Created] (HAWQ-841) Running "make -j8 unittest-check" under src/backedn will throw "File exists error"

[Paul Guo (JIRA)]

Paul Guo created HAWQ-841:
-

 Summary: Running "make -j8 unittest-check" under src/backedn will 
throw "File exists error"
 Key: HAWQ-841
 URL: https://issues.apache.org/jira/browse/HAWQ-841
 Project: Apache HAWQ
  Issue Type: Bug
  Components: Tests
Reporter: Paul Guo
Assignee: Jiali Yao


ERROR:root:Error has occurred during parsing 
/home/build/incubator-hawq/src/backend/executor/execAmi.c: [Errno 17] File 
exists: '/home/build/incubator-hawq/src/test/unit/mock/backend/executor'
Traceback (most recent call last):
  File "mocker.py", line 307, in 
main()
  File "mocker.py", line 300, in main
mock = MockFile(cfile, options)
  File "mocker.py", line 121, in __init__
self.outname = self.output_filename()
  File "mocker.py", line 132, in output_filename
os.makedirs(out_dir)
  File "/usr/lib64/python2.6/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 17] File exists: 
'/home/build/incubator-hawq/src/test/unit/mock/backend/executor'
mocking /home/build/incubator-hawq/src/backend/executor/execQual.c
make[4]: *** 
[../../../../../src/test/unit/mock/backend/executor/execAmi_mock.c] Error 1
make[4]: *** Waiting for unfinished jobs
make[4]: Leaving directory 
`/tmp/build/78017950/incubator-hawq/src/backend/access/common/test'
make[3]: *** [mockup-phony] Error 2
make[3]: Leaving directory 
`/tmp/build/78017950/incubator-hawq/src/backend/access/common/test'
make[2]: *** [unittest-check] Error 2
make[2]: Leaving directory 
`/tmp/build/78017950/incubator-hawq/src/backend/access/common'
make[1]: *** [unittest-check] Error 2
make[1]: Leaving directory 
`/tmp/build/78017950/incubator-hawq/src/backend/access'
make: *** [unittest-check] Error 2





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-840) Partition Sort Support

[Lei Chang (JIRA)]

Lei Chang created HAWQ-840:
--

 Summary: Partition Sort Support
 Key: HAWQ-840
 URL: https://issues.apache.org/jira/browse/HAWQ-840
 Project: Apache HAWQ
  Issue Type: Wish
Reporter: Lei Chang
Assignee: Lei Chang


Support the partition sort as a new method for sorting.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-839) Libyarn coredump when failover to standby RM

[Lin Wen (JIRA)]

Lin Wen created HAWQ-839:


 Summary: Libyarn coredump when failover to standby RM
 Key: HAWQ-839
 URL: https://issues.apache.org/jira/browse/HAWQ-839
 Project: Apache HAWQ
  Issue Type: Bug
  Components: libyarn
Reporter: Lin Wen
Assignee: Lei Chang


Start hawq with yarn mode and kill Hadoop Yarn resource manager, coredump 
happens, the stack is below: 
#0  0x003e054325e5 in raise () from /lib64/libc.so.6
#1  0x003e05433dc5 in abort () from /lib64/libc.so.6
#2  0x7f04980b1109 in libyarn::HandleYarnFailoverException (e=...)
at 
/home/gpadmin/workspace/hawq/incubator-hawq/depends/libyarn/src/libyarnclient/ApplicationClient.cpp:170
#3  0x7f04980b3211 in libyarn::ApplicationClient::getNewApplication 
(this=0x1f17cd0)
at 
/home/gpadmin/workspace/hawq/incubator-hawq/depends/libyarn/src/libyarnclient/ApplicationClient.cpp:215
#4  0x7f049809d639 in libyarn::LibYarnClient::createJob (this=0x1f1e500, 
jobName="hawq", queue="default",
jobId="")
at 
/home/gpadmin/workspace/hawq/incubator-hawq/depends/libyarn/src/libyarnclient/LibYarnClient.cpp:163
#5  0x7f04980987b8 in createJob (client=0x1f25950, jobName=Unhandled dwarf 
expression opcode 0xf3
)
at 
/home/gpadmin/workspace/hawq/incubator-hawq/depends/libyarn/src/libyarnclient/LibYarnClientC.cpp:61
#6  createJob (client=0x1f25950, jobName=Unhandled dwarf expression opcode 0xf3
)
at 
/home/gpadmin/workspace/hawq/incubator-hawq/depends/libyarn/src/libyarnclient/LibYarnClientC.cpp:180
#7  0x008e1117 in RB2YARN_registerYARNApplication ()
#8  0x008e31ad in RB2YARN_initializeConnection ()
#9  0x008e358b in ResBrokerMainInternal ()
#10 0x008e38e8 in ResBrokerMain ()
#11 0x008dfb66 in RB_LIBYARN_start ()
#12 0x0090ae5e in MainHandlerLoop ()
#13 0x0090b46a in ResManagerMainServer2ndPhase ()
#14 0x0090ba14 in ResManagerMain ()
#15 0x0090bd71 in ResManagerProcessStartup ()
#16 0x00767f98 in CommenceNormalOperations ()
#17 0x00768d44 in do_reaper ()
#18 0x0076dbed in ServerLoop ()
#19 0x0076f73e in PostmasterMain ()
#20 0x006c828a in main ()



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)