[jira] [Commented] (CARBONDATA-417) [Bad Records] Not created and not writen log file when logger is True and action as Fail
[
https://issues.apache.org/jira/browse/CARBONDATA-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15714728#comment-15714728
]
MAKAMRAGHUVARDHAN commented on CARBONDATA-417:
--
As per current behavior, the issue is invalid,So closed this issue
> [Bad Records] Not created and not writen log file when logger is True and
> action as Fail
>
>
> Key: CARBONDATA-417
> URL: https://issues.apache.org/jira/browse/CARBONDATA-417
> Project: CarbonData
> Issue Type: Bug
> Components: data-load
>Affects Versions: 0.1.1-incubating
> Environment: 3 node Cluster
>Reporter: MAKAMRAGHUVARDHAN
>Assignee: Akash R Nilugal
>Priority: Minor
>
> Steps:
> 1. Create Table:
> CREATE TABLE truefail (ID int,CUST_ID int,cust_name string) STORED BY
> 'org.apache.carbondata.format';
> 2. Load Data having Logger as True and Action as False
> LOAD DATA INPATH 'hdfs://hacluster/Raghu/test2.csv' into table truefail
> OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
> 'BAD_RECORDS_ACTION'='FAIL','FILEHEADER'='ID,CUST_ID,cust_name');
> 0: jdbc:hive2://ha-cluster/default> CREATE TABLE truefail (ID int,CUST_ID
> int,cust_name string) STORED BY 'org.apache.carbondata.format';
> +-+--+
> | result |
> +-+--+
> +-+--+
> No rows selected (0.679 seconds)
> 0: jdbc:hive2://ha-cluster/default> LOAD DATA INPATH
> 'hdfs://hacluster/Raghu/test2.csv' into table truefail
> OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
> 'BAD_RECORDS_ACTION'='FAIL','FILEHEADER'='ID,CUST_ID,cust_name');
> Error: java.lang.Exception: DataLoad failure: Data load failed due to bad
> record ,The value
> "987654321010111213141516171819101122334455667788990012131415161718191909192939495969798"
> with column name CUST_ID and column data type INT is not a valid Record
> (state=,code=0)
> 0: jdbc:hive2://ha-cluster/default>
> Actual Result: Not Creating and not written log file for bad records when
> BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='FAIL'
> Expected Result: Should create and write log file when
> BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='FAIL' for Bad records
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CARBONDATA-417) [Bad Records] Not created and not writen log file when logger is True and action as Fail
[
https://issues.apache.org/jira/browse/CARBONDATA-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
MAKAMRAGHUVARDHAN updated CARBONDATA-417:
-
Description:
Steps:
1. Create Table:
CREATE TABLE truefail (ID int,CUST_ID int,cust_name string) STORED BY
'org.apache.carbondata.format';
2. Load Data having Logger as True and Action as False
LOAD DATA INPATH 'hdfs://hacluster/Raghu/test2.csv' into table truefail
OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
'BAD_RECORDS_ACTION'='FAIL','FILEHEADER'='ID,CUST_ID,cust_name');
0: jdbc:hive2://ha-cluster/default> CREATE TABLE truefail (ID int,CUST_ID
int,cust_name string) STORED BY 'org.apache.carbondata.format';
+-+--+
| result |
+-+--+
+-+--+
No rows selected (0.679 seconds)
0: jdbc:hive2://ha-cluster/default> LOAD DATA INPATH
'hdfs://hacluster/Raghu/test2.csv' into table truefail OPTIONS('DELIMITER'=','
, 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
'BAD_RECORDS_ACTION'='FAIL','FILEHEADER'='ID,CUST_ID,cust_name');
Error: java.lang.Exception: DataLoad failure: Data load failed due to bad
record ,The value
"987654321010111213141516171819101122334455667788990012131415161718191909192939495969798"
with column name CUST_ID and column data type INT is not a valid Record
(state=,code=0)
0: jdbc:hive2://ha-cluster/default>
Actual Result: Not Creating and not written log file for bad records when
BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='FAIL'
Expected Result: Should create and write log file when
BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='FAIL' for Bad records
was:
Steps:
1. Create Table:
CREATE TABLE truefail (ID int,CUST_ID int,cust_name string) STORED BY
'org.apache.carbondata.format';
2. Load Data having Logger as True and Action as False
LOAD DATA INPATH 'hdfs://hacluster/Raghu/test2.csv' into table truefail
OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
'BAD_RECORDS_ACTION'='FAIL','FILEHEADER'='ID,CUST_ID,cust_name');
Actual Result: Not Creating and not written log file for bad records when
BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='FAIL'
Expected Result: Should create and write log file when
BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='FAIL' for Bad records
> [Bad Records] Not created and not writen log file when logger is True and
> action as Fail
>
>
> Key: CARBONDATA-417
> URL: https://issues.apache.org/jira/browse/CARBONDATA-417
> Project: CarbonData
> Issue Type: Bug
> Components: data-load
>Affects Versions: 0.1.1-incubating
> Environment: 3 node Cluster
>Reporter: MAKAMRAGHUVARDHAN
>Priority: Minor
>
> Steps:
> 1. Create Table:
> CREATE TABLE truefail (ID int,CUST_ID int,cust_name string) STORED BY
> 'org.apache.carbondata.format';
> 2. Load Data having Logger as True and Action as False
> LOAD DATA INPATH 'hdfs://hacluster/Raghu/test2.csv' into table truefail
> OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
> 'BAD_RECORDS_ACTION'='FAIL','FILEHEADER'='ID,CUST_ID,cust_name');
> 0: jdbc:hive2://ha-cluster/default> CREATE TABLE truefail (ID int,CUST_ID
> int,cust_name string) STORED BY 'org.apache.carbondata.format';
> +-+--+
> | result |
> +-+--+
> +-+--+
> No rows selected (0.679 seconds)
> 0: jdbc:hive2://ha-cluster/default> LOAD DATA INPATH
> 'hdfs://hacluster/Raghu/test2.csv' into table truefail
> OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
> 'BAD_RECORDS_ACTION'='FAIL','FILEHEADER'='ID,CUST_ID,cust_name');
> Error: java.lang.Exception: DataLoad failure: Data load failed due to bad
> record ,The value
> "987654321010111213141516171819101122334455667788990012131415161718191909192939495969798"
> with column name CUST_ID and column data type INT is not a valid Record
> (state=,code=0)
> 0: jdbc:hive2://ha-cluster/default>
> Actual Result: Not Creating and not written log file for bad records when
> BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='FAIL'
> Expected Result: Should create and write log file when
> BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='FAIL' for Bad records
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CARBONDATA-417) [Bad Records] Not created and not writen log file when logger is True and action as Fail
MAKAMRAGHUVARDHAN created CARBONDATA-417:
Summary: [Bad Records] Not created and not writen log file when
logger is True and action as Fail
Key: CARBONDATA-417
URL: https://issues.apache.org/jira/browse/CARBONDATA-417
Project: CarbonData
Issue Type: Bug
Components: data-load
Affects Versions: 0.1.1-incubating
Environment: 3 node Cluster
Reporter: MAKAMRAGHUVARDHAN
Priority: Minor
Steps:
1. Create Table:
CREATE TABLE truefail (ID int,CUST_ID int,cust_name string) STORED BY
'org.apache.carbondata.format';
2. Load Data having Logger as True and Action as False
LOAD DATA INPATH 'hdfs://hacluster/Raghu/test2.csv' into table truefail
OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
'BAD_RECORDS_ACTION'='FAIL','FILEHEADER'='ID,CUST_ID,cust_name');
Actual Result: Not Creating and not written log file for bad records when
BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='FAIL'
Expected Result: Should create and write log file when
BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='FAIL' for Bad records
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CARBONDATA-400) [Bad Records] Load data is fail and displaying the string value in beeline as exception
[
https://issues.apache.org/jira/browse/CARBONDATA-400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
MAKAMRAGHUVARDHAN updated CARBONDATA-400:
-
Description:
Steps
1. Create table
CREATE TABLE String_test2 (string_col string) STORED BY
'org.apache.carbondata.format';
2. Load the data with parameter 'BAD_RECORDS_ACTION'='FORCE' and csv contains a
string value that is out of boundary.
LOAD DATA INPATH 'hdfs://hacluster/Carbon/Priyal/string5.csv' into table
String_test2 OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='string_col');
Actual Result: Load data is failed and displaying the string value in beeline
as exception trace.
Expected Result:Should display a correct error message and should not print
the exception trace on the console.
Exception thrown on console is as shown below.
Error: com.univocity.parsers.common.TextParsingException: Error processing
input: Length of parsed input (11) exceeds the maximum number of characters
defined in your parser settings (10).
Hint: Number of characters processed may have exceeded limit of 10
characters per column. Use settings.setMaxCharsPerColumn(int) to define the
maximum number of characters a column can have
Ensure your configuration is correct, with delimiters, quotes and escape
sequences that match the input format you are trying to parse
Parser Configuration: CsvParserSettings:
Column reordering enabled=true
Empty value=null
Header extraction enabled=false
Headers=null
Ignore leading whitespaces=true
Ignore trailing whitespaces=true
Input buffer size=128
Input reading on separate thread=false
Line separator detection enabled=false
Maximum number of characters per column=10
Maximum number of columns=20480
Null value=
Number of records to read=all
Parse unescaped quotes=true
Row processor=none
Selected fields=none
Skip empty lines=trueFormat configuration:
CsvFormat:
Comment character=#
Field delimiter=,
Line separator (normalized)=\n
Line separator sequence=\n
Quote character="
Quote escape character=quote escape
Quote escape escape character=\0, line=0, char=12. Content
parsed:
[hellohowareyouwelcomehellohellohellohellohellohellohellohelloheellooabcdefghijklmnopqrstuvwxyzabcqwertuyioplkjhgfdsazxcvbnmpoiuytrewqasdfghjklmnbvcxzasdghskhdgkhdbkshkjchskdhfssudkdjdudusdjhdshdshsjddshjdkdhgdhdshdhdududushdudududududududududududududududududuudududududududuudududududududududududududududududududududududududududuhellohowareyouwelcomehellohellohellohellohellohellohellohelloheellooabcdefghijklmnopqrstuvwxyzabcqwertuyioplkjhgfdsazxcvbnmpoiuytrewqasdfghjklmnbvcxzasdghskhdgkhdbkshkjchskdhfssudkdjdudusdjhdshdshsjddshjdkdhgdhdshdhdududushdudududududududududududududududududuudududududududuududududududududuu
was:
Steps
1. Create table
CREATE TABLE String_test2 (string_col string) STORED BY
'org.apache.carbondata.format';
2. Load the data with parameter 'BAD_RECORDS_ACTION'='FORCE' and csv contains a
string value that is out of boundary.
LOAD DATA INPATH 'hdfs://hacluster/Carbon/Priyal/string5.csv' into table
String_test2 OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='string_col');
Actual Result: Load data is failed and displaying the string value in beeline
as exception trace.
Expected Result:Should display a valid exception.
> [Bad Records] Load data is fail and displaying the string value in beeline as
> exception
> ---
>
> Key: CARBONDATA-400
> URL: https://issues.apache.org/jira/browse/CARBONDATA-400
> Project: CarbonData
> Issue Type: Bug
> Components: data-load
>Affects Versions: 0.1.0-incubating
> Environment: 3node cluster
>Reporter: MAKAMRAGHUVARDHAN
>Priority: Minor
>
> Steps
> 1. Create table
> CREATE TABLE String_test2 (string_col string) STORED BY
> 'org.apache.carbondata.format';
> 2. Load the data with parameter 'BAD_RECORDS_ACTION'='FORCE' and csv contains
> a string value that is out of boundary.
> LOAD DATA INPATH 'hdfs://hacluster/Carbon/Priyal/string5.csv' into table
> String_test2 OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
> 'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='string_col');
> Actual Result: Load data is failed and displaying the string value in beeline
> as exception trace.
> Expected Result:Should display a correct error message and should not print
> the exception trace on the console.
> Exception thrown
[jira] [Created] (CARBONDATA-400) [Bad Records] Load data is fail and displaying the string value in beeline as exception
MAKAMRAGHUVARDHAN created CARBONDATA-400:
Summary: [Bad Records] Load data is fail and displaying the string
value in beeline as exception
Key: CARBONDATA-400
URL: https://issues.apache.org/jira/browse/CARBONDATA-400
Project: CarbonData
Issue Type: Bug
Components: data-load
Affects Versions: 0.1.0-incubating
Environment: 3node cluster
Reporter: MAKAMRAGHUVARDHAN
Priority: Minor
Steps
1. Create table
CREATE TABLE String_test2 (string_col string) STORED BY
'org.apache.carbondata.format';
2. Load the data with parameter 'BAD_RECORDS_ACTION'='FORCE' and csv contains a
string value that is out of boundary.
LOAD DATA INPATH 'hdfs://hacluster/Carbon/Priyal/string5.csv' into table
String_test2 OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='string_col');
Actual Result: Load data is failed and displaying the string value in beeline
as exception trace.
Expected Result:Should display a valid exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
