[
https://issues.apache.org/jira/browse/PIG-3879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Taylor Finnell updated PIG-3879:
--------------------------------
Description:
We are having an issue with multiple store statements in the same script. The
STORE fails to insert the records into the database. The script is roughly as
follows...
{code}
A = LOAD '...' USING CSVLoader ...;
STORE A INTO '/tmp/A-unused' USING DBStorage (org.postgresql.Driver, ...,
INSERT INTO ....);
B = FOREACH A GENERATE X, Y, CONCAT(X, Y) as Z;
STORE B INTO '/tmp/B-unused' USING DBStorage (org.postgresql.Driver, ...,
INSERT INTO ....);
{code}
Both DBStorage calls insert into different tables in the same database.
When the script is run both A, B are stored into their /tmp/ locations.
However, the data never makes it into the database. We found two ways to get
the data to make it into the database. The first, was to add a DUMP B command
after the assignment of B. The second was to execute the script with the -M
flag.
was:
We are having an issue with multiple store statements in the same script. The
STORE fails to insert the records into the database. The script is roughly as
follows...
{code}
A = LOAD '...' USING CSVLoader ...;
STORE A INTO '/tmp/A-unused' USING DBStorage (org.postgresql.Driver, ...,
INSERT INTO ....);
B = FOREACH A GENERATE X, Y, CONCAT(X, Y) as Z;
STORE B INTO '/tmp/B-unused' USING DBStorage (org.postgresql.Driver, ...,
INSERT INTO ....);
{code}
Both DBStorage calls insert into different tables in the same database.
When the script is run both A, B are stored into their /tmp/ locations.
However, the data never makes it into the database. We found two ways to get
the data to make it into the database. The first, was to add a DUMP B command
after the assignment of B. The second was to execute the script with the -M
flag.
Some previous discussion here: https://issues.apache.org/jira/browse/PIG-2872
> Script fails with multiple STORE statements unless multiquery is off
> --------------------------------------------------------------------
>
> Key: PIG-3879
> URL: https://issues.apache.org/jira/browse/PIG-3879
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.12.0
> Reporter: Taylor Finnell
>
> We are having an issue with multiple store statements in the same script. The
> STORE fails to insert the records into the database. The script is roughly as
> follows...
> {code}
> A = LOAD '...' USING CSVLoader ...;
> STORE A INTO '/tmp/A-unused' USING DBStorage (org.postgresql.Driver, ...,
> INSERT INTO ....);
> B = FOREACH A GENERATE X, Y, CONCAT(X, Y) as Z;
> STORE B INTO '/tmp/B-unused' USING DBStorage (org.postgresql.Driver, ...,
> INSERT INTO ....);
> {code}
> Both DBStorage calls insert into different tables in the same database.
> When the script is run both A, B are stored into their /tmp/ locations.
> However, the data never makes it into the database. We found two ways to get
> the data to make it into the database. The first, was to add a DUMP B command
> after the assignment of B. The second was to execute the script with the -M
> flag.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
