Remove the apache.org filter when using YourKit

2016-06-01 Thread Jianfeng Jia
. It is because the default filter in Yourkit skip the “org.apache” packages :-) To change the filter: click “settings” -> “filters” and click out the “org.apache”. Enjoy profiling apache code then! Best, Jianfeng Jia PhD Candidate of Computer Science University of California, Irvine

Re: Wishlist

2016-06-18 Thread Jianfeng Jia
el? > > Cheers, > Till > > [1] > https://cwiki.apache.org/confluence/display/ASTERIXDB/Things+will+be+easier+if+AsterixDB+have+these+features Best, Jianfeng Jia PhD Candidate of Computer Science University of California, Irvine

Re: Creating RTree: no space left

2016-08-26 Thread Jianfeng Jia
t;> The data contains: >>>>>>>> 1- 2887453794 records. >>>>>>>> 2- Schema: >>>>>>>> >>>>>>>> create type CDRType as { >>>>>>>> >>>>>>>> id:uuid, >>>>>>>> >>>>>>>> 'date':string, >>>>>>>> >>>>>>>> 'time':string, >>>>>>>> >>>>>>>> 'duration':int64, >>>>>>>> >>>>>>>> 'caller':int64, >>>>>>>> >>>>>>>> 'callee':int64, >>>>>>>> >>>>>>>> location:point? >>>>>>>> >>>>>>>> } >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Aug 23, 2016 at 9:06 AM, Wail Alkowaileet < >>> wael@gmail.com >>>>> >>>>>>>> wrote: >>>>>>>> >>>>>>>> Dears, >>>>>>>>> >>>>>>>>> I have a dataset of size 290GB loaded in a 3 NCs each of which >> has >>>>>>>>> >>>>>>>> 2x500GB >>>>>>>> >>>>>>>>> SSD. >>>>>>>>> >>>>>>>>> Each of NC has two IODevices (partitions) in each hard drive (i.e >>> the >>>>>>>>> total is 4 iodevices per NC). After loading the data, each >> Asterix >>>>>>>>> partition occupied 31GB. >>>>>>>>> >>>>>>>>> The cluster has about 50% free space in each hard drive >>>> (approximately >>>>>>>>> about 250GB free space in each hard drive). However, when I tried >>> to >>>>>>>>> >>>>>>>> create >>>>>>>> >>>>>>>>> an index of type RTree, I got an exception that no space left in >>> the >>>>>>>>> hard >>>>>>>>> drive during the External Sort phase. >>>>>>>>> >>>>>>>>> Is that normal ? >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> *Regards,* >>>>>>>>> Wail Alkowaileet >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> *Regards,* >>>>>>>> Wail Alkowaileet >>>>>>>> >>>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> *Regards,* >>>>> Wail Alkowaileet >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> *Regards,* >>>> Wail Alkowaileet >>>> >>> >> > > > > -- > > *Regards,* > Wail Alkowaileet Best, Jianfeng Jia PhD Candidate of Computer Science University of California, Irvine

Re: Creating RTree: no space left

2016-08-27 Thread Jianfeng Jia
ck >> what SORT process is required by R-Tree index. >> >> Best, >> Taewoo >> >> On Fri, Aug 26, 2016 at 7:52 AM, Jianfeng Jia >> wrote: >> >>> If all of the file names start with “ExternalSortRunGenerator”, then they >>> are the f

How to set the lsm component size?

2016-09-12 Thread Jianfeng Jia
increase the component size? Thanks! Best, Jianfeng Jia PhD Candidate of Computer Science University of California, Irvine

Re: How to set the lsm component size?

2016-09-12 Thread Jianfeng Jia
e size that you have set 128KB looks reasonable for most cases. Your > best bet is to increase the value of storage.memorycomponent.numpage to a > higher number. > > Sattam > > > On Mon, Sep 12, 2016 at 11:33 AM, Jianfeng Jia > wrote: > >> Dear devs, >>

Re: How to set the lsm component size?

2016-09-12 Thread Jianfeng Jia
exes in that machine. When > it is exhausted, a victim dataset must be evicted and closed to have a > space for another dataset. > > On Mon, Sep 12, 2016 at 12:29 PM, Jianfeng Jia > wrote: > >> I was a little confused, there is another configuration: >> >> stora

Re: Let one Operator finished the job before another one begin in Hyracks

2016-10-11 Thread Jianfeng Jia
Based on the described example, it seems possible to implement it in one job by using MToNPartitioningConnectorDescriptor. You can force that merge-BF-operator only runs in one partition by using PartitionConstraintHelper.addAbsoluteLocationConstraint() function. > On Oct 10, 2016, at 11:43 PM,

Re: How to set the lsm component size?

2016-10-13 Thread Jianfeng Jia
> Taewoo >> >> On Mon, Sep 12, 2016 at 4:10 PM, Mike Carey wrote: >> >>> +1 >>> >>> >>> >>> On 9/12/16 3:42 PM, Taewoo Kim wrote: >>> >>>> It would be really helpful this conversation can be applied in the >>

Re: [VOTE] Release Apache AsterixDB 0.9.0 and Hyracks 0.3.0 (RC2)

2017-01-21 Thread Jianfeng Jia
+1 - signatures and hash checks - source compilation works - nc service works > On Jan 21, 2017, at 6:03 PM, Yingyi Bu wrote: > > +1 > > - signatures and hashes of all 5 archives ok > - nc service binary works > - version api agrees with the commit id on ASF repo > - source compilation works >

Re: Choosing defaults for AsterixDB

2017-02-03 Thread Jianfeng Jia
Hi, I want to pick up this thread to verify if the AQL will still be supported in the future? Currently, Cloudberry automatically translates the JSON request to AQL statements. It will be a hard work to switch to SQL++. I’m not object to set the default option to SQL++. However, we will keep t

Re: Choosing defaults for AsterixDB

2017-02-03 Thread Jianfeng Jia
@Yingyi, I’m not saying learning SQL++ is difficult. Currently, we have a class called AQLGenerator that can translate the Cloudberry request syntax to AQL. It took us several weeks finishing it. I guess it will take similar time to write a SQLPPGenerator to achieve the same goal. As long as

Re: Add the Java driver for Asterix

2017-03-03 Thread Jianfeng Jia
expert on JDBC. Do you think it’s a good project? and will anyone interested in give the detailed instruction on it? Otherwise, I will close this issue to give it a very low priority. > On Mar 3, 2017, at 10:19 AM, jabbar memon wrote: > > Hi Jianfeng Jia, > i am postgraduate a

What is the new path to check Hyracks jobs status in AsterixDB?

2017-04-06 Thread Jianfeng Jia
current system workload (e.g., how many queries are running, when did they submit, how long it has ran …). Right now, the same link returns a following error. page can’t be found I’m wondering what is the new path to get the same information? Thanks! Best, Jianfeng Jia PhD Candidate of Computer

Re: What is the new path to check Hyracks jobs status in AsterixDB?

2017-04-07 Thread Jianfeng Jia
JID:0/job-run > > Best, > Yingyi > > > On Thu, Apr 6, 2017 at 5:18 PM, Jianfeng Jia wrote: > >> Dear Devs, >> >> We used to have a Hyracks adminconsole web page xxx:/adminconsole (or >> on 16001 port if not using managix) which can watc

Re: What is the new path to check Hyracks jobs status in AsterixDB?

2017-04-07 Thread Jianfeng Jia
e a page around it (which is > clearly feasible as well). > > On 7 Apr 2017, at 16:21, Mike Carey wrote: > >> Could we use the same library that Xikui used for JSON (formatted) as a baby >> step? >> >> >> On 4/7/17 10:04 AM, Jianfeng Jia w

Re: Searching for duplicates during feed ingestion.

2017-05-08 Thread Jianfeng Jia
I also observe this getting slower problem every-time when we re-ingest the twitter data. One difference is that the duplicate key could happen, and we know that is indeed duplicate record. To skip the search, we would expect an “upsert” logic ( just replace the old one :-) ) instead of an inse

Re: Searching for duplicates during feed ingestion.

2017-05-08 Thread Jianfeng Jia
ix-app/src/test/resources/runtimets/queries/feeds/upsert-feed/upsert-feed.1.ddl.aql > > <https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/upsert-feed/upsert-feed.1.ddl.aql> >> On May 8, 2017, at 12:04, Jianfeng Jia

Re: Searching for duplicates during feed ingestion.

2017-05-08 Thread Jianfeng Jia
to get the old record > to update secondary indexes from.) > > > On 5/8/17 12:10 PM, Jianfeng Jia wrote: >> Aha, never knew that before. We will definitely try upsert feed next time! >> Thanks for pointing it out! >> >>> On May 8, 2017, at 12:07 PM, Ildar

Re: /rest/jobs visualization

2018-10-23 Thread Jianfeng Jia
ng the output of the > JSON form of the job-run and job-activity-graph ? I know we used to > have a servlet for it, but since we got rid of that, I know it's been > asked about so I assume it must be used somehow at the moment... > Thanks, > - Ian > -- --------- Be