Re: scraping aspx web

2020-09-02 Thread Mike Thomsen 
You're better off with a tool like Scrapy for something like this:
https://scrapy.org/

On Wed, Sep 2, 2020 at 2:07 PM tkg_cangkul  wrote:

> Dear All,
>
> I wanna try to scrapping aspx web with nifi. is there any suggestion to
> convert aspx grid into html table or csv file ?
>
> Below is the sample aspx grid view format that i've got
>
>
>
> Is this possible to do with nifi?
> Need advice.
>
>
> Best Regards,
>

scraping aspx web

2020-09-02 Thread tkg_cangkul 

Dear All,

I wanna try to scrapping aspx web with nifi. is there any suggestion to 
convert aspx grid into html table or csv file ?


Below is the sample aspx grid view format that i've got



Is this possible to do with nifi?
Need advice.


Best Regards,

Re: Help - CSV Parser - Invalid char

2020-09-02 Thread KhajaAsmath Mohammed 
Hello Mark,

Removing " manually from the string resolved the issue. is there a easy way
by parser? somehow quote character is being ignored in parser.

Thanks,
Asmath


Virus-free.
www.avg.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Wed, Sep 2, 2020 at 9:38 AM Mark Payne  wrote:

> Asmanth,
>
> Can you check the logs and provide the full stack trace?
>
> On Sep 2, 2020, at 10:35 AM, KhajaAsmath Mohammed 
> wrote:
>
> Here is the error that I am getting.
>
> 
>
>
> 
>  Virus-free.
> www.avg.com
> 
>
> On Wed, Sep 2, 2020 at 9:33 AM Mark Payne  wrote:
>
>> Asmanth,
>>
>> What is the exception that you’re seeing?
>>
>> On Sep 2, 2020, at 10:28 AM, KhajaAsmath Mohammed <
>> mdkhajaasm...@gmail.com> wrote:
>>
>> Hi,
>>
>> I am having an issue using the Tab delimited CSV parser in NIFI. I need
>> to convert this csv to json before loading into queue.  I am getting an
>> exception in this data. any help on how to resolve this?
>>
>> *Sample Data:*
>>
>> SUBMISSION_ID ASSIGNMENT_ID COURSE_ID ENROLLMENT_TERM_ID USER_ID
>> GRADER_ID COURSE_ACCOUNT_ID ENROLLMENT_ROLLUP_ID SCORE PUBLISHED_SCORE
>> WHAT_IF_SCORE SUBMISSION_COMMENTS_COUNT ACCOUNT_ID ASSIGNMENT_GROUP_ID
>> GROUP_ID QUIZ_ID QUIZ_SUBMISSION_ID WIKI_ID
>> 13171007638523 7638523 "I will take this exam with integrity and
>> without using any help or materials, according to the expectations of the
>> Instructor and of UCSD's guidelines for academic integrity." Tushita Tandon
>> complete 2020-04-30 18:18:54.235 online_text_entry graded 2020-04-30
>> 18:00:22.073 2020-05-11 17:03:41.972 false true complete 2020-05-05
>> 02:26:16.152 1 13171000124204 regular_submission not_graded_anonymously
>> 470563165038122618 -480988430103600455 human_graded 2020-05-11 17:03:41.972
>>
>> *Tab Delimited CSV reader with header as schema:*
>>
>> 
>> 
>>
>> Thanks,
>> Asmath
>>
>>
>>
>>
>>
>> 
>>  Virus-free.
>> www.avg.com
>> 
>>
>>
>>
>

Re: Help - CSV Parser - Invalid char

2020-09-02 Thread Mark Payne 
Asmanth,

Can you check the logs and provide the full stack trace?

On Sep 2, 2020, at 10:35 AM, KhajaAsmath Mohammed 
mailto:mdkhajaasm...@gmail.com>> wrote:

Here is the error that I am getting.



[https://ipmcdn.avast.com/images/icons/icon-envelope-tick-green-avg-v1.png]
 Virus-free. 
www.avg.com


On Wed, Sep 2, 2020 at 9:33 AM Mark Payne 
mailto:marka...@hotmail.com>> wrote:
Asmanth,

What is the exception that you’re seeing?

On Sep 2, 2020, at 10:28 AM, KhajaAsmath Mohammed 
mailto:mdkhajaasm...@gmail.com>> wrote:

Hi,

I am having an issue using the Tab delimited CSV parser in NIFI. I need to 
convert this csv to json before loading into queue.  I am getting an exception 
in this data. any help on how to resolve this?

Sample Data:

SUBMISSION_ID ASSIGNMENT_ID COURSE_ID ENROLLMENT_TERM_ID USER_ID GRADER_ID 
COURSE_ACCOUNT_ID ENROLLMENT_ROLLUP_ID SCORE PUBLISHED_SCORE WHAT_IF_SCORE 
SUBMISSION_COMMENTS_COUNT ACCOUNT_ID ASSIGNMENT_GROUP_ID GROUP_ID QUIZ_ID 
QUIZ_SUBMISSION_ID WIKI_ID
13171007638523 7638523 "I will take this exam with integrity and without 
using any help or materials, according to the expectations of the Instructor 
and of UCSD's guidelines for academic integrity." Tushita Tandon complete 
2020-04-30 18:18:54.235 online_text_entry graded 2020-04-30 18:00:22.073 
2020-05-11 17:03:41.972 false true complete 2020-05-05 02:26:16.152 1 
13171000124204 regular_submission not_graded_anonymously 470563165038122618 
-480988430103600455 human_graded 2020-05-11 17:03:41.972

Tab Delimited CSV reader with header as schema:




Thanks,
Asmath




[https://ipmcdn.avast.com/images/icons/icon-envelope-tick-green-avg-v1.png]
 Virus-free. 
www.avg.com

Re: Help - CSV Parser - Invalid char

2020-09-02 Thread KhajaAsmath Mohammed 
Here is the error that I am getting.

[image: image.png]


Virus-free.
www.avg.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Wed, Sep 2, 2020 at 9:33 AM Mark Payne  wrote:

> Asmanth,
>
> What is the exception that you’re seeing?
>
> On Sep 2, 2020, at 10:28 AM, KhajaAsmath Mohammed 
> wrote:
>
> Hi,
>
> I am having an issue using the Tab delimited CSV parser in NIFI. I need to
> convert this csv to json before loading into queue.  I am getting an
> exception in this data. any help on how to resolve this?
>
> *Sample Data:*
>
> SUBMISSION_ID ASSIGNMENT_ID COURSE_ID ENROLLMENT_TERM_ID USER_ID GRADER_ID
> COURSE_ACCOUNT_ID ENROLLMENT_ROLLUP_ID SCORE PUBLISHED_SCORE WHAT_IF_SCORE
> SUBMISSION_COMMENTS_COUNT ACCOUNT_ID ASSIGNMENT_GROUP_ID GROUP_ID QUIZ_ID
> QUIZ_SUBMISSION_ID WIKI_ID
> 13171007638523 7638523 "I will take this exam with integrity and
> without using any help or materials, according to the expectations of the
> Instructor and of UCSD's guidelines for academic integrity." Tushita Tandon
> complete 2020-04-30 18:18:54.235 online_text_entry graded 2020-04-30
> 18:00:22.073 2020-05-11 17:03:41.972 false true complete 2020-05-05
> 02:26:16.152 1 13171000124204 regular_submission not_graded_anonymously
> 470563165038122618 -480988430103600455 human_graded 2020-05-11 17:03:41.972
>
> *Tab Delimited CSV reader with header as schema:*
>
> 
> 
>
> Thanks,
> Asmath
>
>
>
>
>
> 
>  Virus-free.
> www.avg.com
> 
>
>
>

Re: Help - CSV Parser - Invalid char

2020-09-02 Thread Mark Payne 
Asmanth,

What is the exception that you’re seeing?

On Sep 2, 2020, at 10:28 AM, KhajaAsmath Mohammed 
mailto:mdkhajaasm...@gmail.com>> wrote:

Hi,

I am having an issue using the Tab delimited CSV parser in NIFI. I need to 
convert this csv to json before loading into queue.  I am getting an exception 
in this data. any help on how to resolve this?

Sample Data:

SUBMISSION_ID ASSIGNMENT_ID COURSE_ID ENROLLMENT_TERM_ID USER_ID GRADER_ID 
COURSE_ACCOUNT_ID ENROLLMENT_ROLLUP_ID SCORE PUBLISHED_SCORE WHAT_IF_SCORE 
SUBMISSION_COMMENTS_COUNT ACCOUNT_ID ASSIGNMENT_GROUP_ID GROUP_ID QUIZ_ID 
QUIZ_SUBMISSION_ID WIKI_ID
13171007638523 7638523 "I will take this exam with integrity and without 
using any help or materials, according to the expectations of the Instructor 
and of UCSD's guidelines for academic integrity." Tushita Tandon complete 
2020-04-30 18:18:54.235 online_text_entry graded 2020-04-30 18:00:22.073 
2020-05-11 17:03:41.972 false true complete 2020-05-05 02:26:16.152 1 
13171000124204 regular_submission not_graded_anonymously 470563165038122618 
-480988430103600455 human_graded 2020-05-11 17:03:41.972

Tab Delimited CSV reader with header as schema:




Thanks,
Asmath




[https://ipmcdn.avast.com/images/icons/icon-envelope-tick-green-avg-v1.png]
 Virus-free. 
www.avg.com

Help - CSV Parser - Invalid char

2020-09-02 Thread KhajaAsmath Mohammed 
Hi,

I am having an issue using the Tab delimited CSV parser in NIFI. I need to
convert this csv to json before loading into queue.  I am getting an
exception in this data. any help on how to resolve this?

*Sample Data:*

SUBMISSION_ID ASSIGNMENT_ID COURSE_ID ENROLLMENT_TERM_ID USER_ID GRADER_ID
COURSE_ACCOUNT_ID ENROLLMENT_ROLLUP_ID SCORE PUBLISHED_SCORE WHAT_IF_SCORE
SUBMISSION_COMMENTS_COUNT ACCOUNT_ID ASSIGNMENT_GROUP_ID GROUP_ID QUIZ_ID
QUIZ_SUBMISSION_ID WIKI_ID
13171007638523 7638523 "I will take this exam with integrity and
without using any help or materials, according to the expectations of the
Instructor and of UCSD's guidelines for academic integrity." Tushita Tandon
complete 2020-04-30 18:18:54.235 online_text_entry graded 2020-04-30
18:00:22.073 2020-05-11 17:03:41.972 false true complete 2020-05-05
02:26:16.152 1 13171000124204 regular_submission not_graded_anonymously
470563165038122618 -480988430103600455 human_graded 2020-05-11 17:03:41.972

*Tab Delimited CSV reader with header as schema:*

[image: image.png]
[image: image.png]

Thanks,
Asmath





Virus-free.
www.avg.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>