John,
Attached are files that demonstrate the problem.
zip.csv and country.csv each have 3 columns, and 2 in common. So, when I
append them to an (initially empty) partition, I would expect the result to
have 4 columns (the union). However, the -part.txt file shows only the 3
columns of the last one appended.
load-commands.txt contains the shell commands used to load the original
CSVs and combine them using append.
Sorry for the delay getting this to you.
-- Mark
On Wed, Jun 12, 2013 at 1:47 AM, K. John Wu <[email protected]> wrote:
> Hi, Mark,
>
> Sorry for the slow response.
>
> I have not seen an sample data from you yet. Have you resolved the
> problem with appending data partitions?
>
> There are a couple of examples of using ibis -append option in
> tests/Makefile.am for targets do-small-append and do-large-append.
> Please take a look and see if something similar to that works for you.
>
> John
>
>
>
> On 6/8/13 4:41 AM, Mark Hansen wrote:
> > So, I should be able to achieve such a union of columns by writing some
> C++ code, right?
> >
> > I'm building a wrapper to do some specialized loading and queries.
> >
> > I will send you an example later today.
> >
> >
> > Mark Hansen
> > President & Founder
> > Digital Brand Mine - developers of Megalytic
> > http://megalytic.com
> > http://digitalbrandmine.com
> >
> > On Jun 7, 2013, at 10:40 PM, "K. John Wu" <[email protected]> wrote:
> >
> >> Hi, Mark,
> >>
> >> We did intent -a option to create a union of the columns. Note that
> >> this operation will assume the columns did not exist in the first
> >> table to be NULLs.
> >>
> >> Anyway, if you have a small example you can share with us, I will take
> >> a look into this problem over the weekend.
> >>
> >> John
> >>
> >>
> >> On 6/3/13 8:17 PM, Mark Hansen wrote:
> >>> I have 2 tables: X and Y. I want to append the into a new table Z.
> >>> So, I do this:
> >>>
> >>> ibis -a X to ./Z
> >>>
> >>> That works fine, and Z is now identical to X. Then, I do this:
> >>>
> >>> ibis -a Y to ./Z
> >>>
> >>> It seems to append the rows, but the ./Z/-part.txt file is not the
> >>> UNION of X and Y as I expected. Instead, it is identical to
> >>> ./Y/-part.txt. The columns from both X and Y are there as files under
> >>> ./Z, it is just the ./Z/-part.txt which does not have all the columns.
> >>>
> >>> What am I missing?
> >>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> FastBit-users mailing list
> >>> [email protected]
> >>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
> >>>
>
--
Mark Hansen
Founder & President
Digital Brand Mine | 708 3rd Ave | New York, New York 10017
office: 212-961-7250
http://digitalbrandmine.com/ | email: [email protected]
ardea -d zip -b "," -m "date:int,zip:text,visits:int" -t ./zip.csv
ardea -d country -b "," -m "date:int,country:text,visits:int" -t ./country.csv
ibis -a zip ./combined
ibis -a country ./combined
20110501,"10583",3456
20110502,"10017",6786
20110501,"USA",1923
20110501,"CA",1543
20110502,"USA",3245
20110502,"CA",3541
_______________________________________________
FastBit-users mailing list
[email protected]
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
