Re: can't ush data to bigQuery

2019-09-26 Thread dan young
yea, the arrayswe use jq to flatten and format all our JSON glad to see you got it! dano On Thu, Sep 26, 2019 at 6:54 AM Nicolas Delsaux wrote: > Oh well, i've understood my last error : incorrect flow files (with JSOn > arrays) were stuck in the queue. > > I removed them and ... to

Re: can't ush data to bigQuery

2019-09-26 Thread Pierre Villard
Great! Glad you got it working! Le jeu. 26 sept. 2019 à 14:54, Nicolas Delsaux a écrit : > Oh well, i've understood my last error : incorrect flow files (with JSOn > arrays) were stuck in the queue. > > I removed them and ... to my delight, data seems to come in BigQuery ! > Le 26/09/2019 à

Re: can't ush data to bigQuery

2019-09-26 Thread Nicolas Delsaux
Oh well, i've understood my last error : incorrect flow files (with JSOn arrays) were stuck in the queue. I removed them and ... to my delight, data seems to come in BigQuery ! Le 26/09/2019 à 14:45, Nicolas Delsaux a écrit : I didn't knew that command ... i've edited some confidential

Re: can't ush data to bigQuery

2019-09-26 Thread Nicolas Delsaux
I didn't knew that command ... i've edited some confidential values in the result, but here it is $ bq --project_id={{PROJECT_ID}} --format=prettyjson show -j 9e790299-dc77-46f4-8978-476f284fe5b5 {   "configuration": {     "jobType": "LOAD",     "load": {   "createDisposition":

Re: can't ush data to bigQuery

2019-09-26 Thread Pierre Villard
What if you run the below command in Cloud Shell: bq --format=prettyjson show -j In your case (with your last email): bq --format=prettyjson show -j 9e790299-dc77-46f4-8978-476f284fe5b5 Does it give you more details? Le jeu. 26 sept. 2019 à 12:13, Nicolas Delsaux a écrit : > Sorry for the

Re: can't ush data to bigQuery

2019-09-26 Thread Nicolas Delsaux
Sorry for the late reply. As of today, the issue is still present. Nifi Web UI just shows the message "Error while reading data, error message: JSON table encountered too many errors, giving up. Rows: 1; errors: 1. Please look into the errors[] collection for more details." But the log is

Re: can't ush data to bigQuery

2019-09-24 Thread Pierre Villard
Hey Nicolas, Did you manage to solve your issue? Happy to help on this one. Thanks, Pierre Le ven. 20 sept. 2019 à 16:42, Nicolas Delsaux a écrit : > Hello > > I'm using PutBigQueryBash and having weird auth issues. > > I have set the GCP Credentials Controller Service to use Service Account