[jira] [Commented] (FLINK-8008) PojoTypeInfo should sort fields fields based on boolean
[ https://issues.apache.org/jira/browse/FLINK-8008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16243767#comment-16243767 ] Muhammad Imran Tariq commented on FLINK-8008: - It resolves datatype application issue i.e. "datatype of Age field(which is Double) get applied on ID field". However sequence of column is changed and it is printed as: 25.0,1 33.0,2 So sequence is still in sorted order which I don't want :) > PojoTypeInfo should sort fields fields based on boolean > --- > > Key: FLINK-8008 > URL: https://issues.apache.org/jira/browse/FLINK-8008 > Project: Flink > Issue Type: Improvement > Components: DataSet API >Affects Versions: 1.3.2 >Reporter: Muhammad Imran Tariq >Priority: Minor > Original Estimate: 2h > Remaining Estimate: 2h > > Flink PojoTypeInfo sorts fields array that are passed into constructor > arguments. I want to create another constructor that takes boolean parameter > to sort field or not. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-8008) PojoTypeInfo should sort fields fields based on boolean
[
https://issues.apache.org/jira/browse/FLINK-8008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16243362#comment-16243362
]
Muhammad Imran Tariq commented on FLINK-8008:
-
I am calling
_public PojoTypeInfo(Class typeClass, List fields)_ constructor
of class _PojoTypeInfo_.
Below is my code.
_//create a PojoTupeInfo
PojoTypeInfo sourceType = new PojoTypeInfo(Person.class,
fieldList);
//create a dataset
DataSet data= env.createInput(new PojoCsvInputFormat(new
Path(textPath),
CsvInputFormat.DEFAULT_LINE_DELIMITER,
CsvInputFormat.DEFAULT_FIELD_DELIMITER, sourceType),
sourceType);
//create a table of this dataset
Table newT = tableEnv.fromDataSet(text);
//sink table
TableSink sink = new CsvTableSink("D:\\invesco\\POC\\Flink\\rules
implementation\\data3.csv", "|", 1,
WriteMode.OVERWRITE);
newT.writeToSink(sink);_
So as I said earlier that there are two fields in my POJO class. first is
ID(Integer), second one is Age(Double). PojoTypeInfo info sorts fields in
alphabetical order. But CSVReader reads file and did not sort columns. When I
sink my table, then datatype of Age field(which is Double) get applied on ID
field. So initially my data in CSV was:
1,25
2,33
After sink it becomes
1.0,25
2.0,33
To avoid this I want PojoTypeInfo class not to sort fields inside its
constructor.
> PojoTypeInfo should sort fields fields based on boolean
> ---
>
> Key: FLINK-8008
> URL: https://issues.apache.org/jira/browse/FLINK-8008
> Project: Flink
> Issue Type: Improvement
> Components: DataSet API
>Affects Versions: 1.3.2
>Reporter: Muhammad Imran Tariq
>Priority: Minor
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Flink PojoTypeInfo sorts fields array that are passed into constructor
> arguments. I want to create another constructor that takes boolean parameter
> to sort field or not.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (FLINK-8008) PojoTypeInfo should sort fields fields based on boolean
[
https://issues.apache.org/jira/browse/FLINK-8008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16243362#comment-16243362
]
Muhammad Imran Tariq edited comment on FLINK-8008 at 11/8/17 4:40 AM:
--
I am calling
_public PojoTypeInfo(Class typeClass, List fields)_ constructor
of class _PojoTypeInfo_.
Below is my code.
_//create a PojoTypeInfo
PojoTypeInfo sourceType = new PojoTypeInfo(Person.class,
fieldList);
//create a dataset
DataSet data= env.createInput(new PojoCsvInputFormat(new
Path(textPath),
CsvInputFormat.DEFAULT_LINE_DELIMITER,
CsvInputFormat.DEFAULT_FIELD_DELIMITER, sourceType),
sourceType);
//create a table of this dataset
Table newT = tableEnv.fromDataSet(text);
//sink table
TableSink sink = new CsvTableSink("D:\\invesco\\POC\\Flink\\rules
implementation\\data3.csv", "|", 1,
WriteMode.OVERWRITE);
newT.writeToSink(sink);_
So as I said earlier that there are two fields in my POJO class. first is
ID(Integer), second one is Age(Double). PojoTypeInfo info sorts fields in
alphabetical order. But CSVReader reads file and did not sort columns. When I
sink my table, then datatype of Age field(which is Double) get applied on ID
field. So initially my data in CSV was:
1,25
2,33
After sink it becomes
1.0,25
2.0,33
To avoid this I want PojoTypeInfo class not to sort fields inside its
constructor.
was (Author: imran.tariq):
I am calling
_public PojoTypeInfo(Class typeClass, List fields)_ constructor
of class _PojoTypeInfo_.
Below is my code.
_//create a PojoTupeInfo
PojoTypeInfo sourceType = new PojoTypeInfo(Person.class,
fieldList);
//create a dataset
DataSet data= env.createInput(new PojoCsvInputFormat(new
Path(textPath),
CsvInputFormat.DEFAULT_LINE_DELIMITER,
CsvInputFormat.DEFAULT_FIELD_DELIMITER, sourceType),
sourceType);
//create a table of this dataset
Table newT = tableEnv.fromDataSet(text);
//sink table
TableSink sink = new CsvTableSink("D:\\invesco\\POC\\Flink\\rules
implementation\\data3.csv", "|", 1,
WriteMode.OVERWRITE);
newT.writeToSink(sink);_
So as I said earlier that there are two fields in my POJO class. first is
ID(Integer), second one is Age(Double). PojoTypeInfo info sorts fields in
alphabetical order. But CSVReader reads file and did not sort columns. When I
sink my table, then datatype of Age field(which is Double) get applied on ID
field. So initially my data in CSV was:
1,25
2,33
After sink it becomes
1.0,25
2.0,33
To avoid this I want PojoTypeInfo class not to sort fields inside its
constructor.
> PojoTypeInfo should sort fields fields based on boolean
> ---
>
> Key: FLINK-8008
> URL: https://issues.apache.org/jira/browse/FLINK-8008
> Project: Flink
> Issue Type: Improvement
> Components: DataSet API
>Affects Versions: 1.3.2
>Reporter: Muhammad Imran Tariq
>Priority: Minor
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Flink PojoTypeInfo sorts fields array that are passed into constructor
> arguments. I want to create another constructor that takes boolean parameter
> to sort field or not.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (FLINK-8008) PojoTypeInfo should sort fields fields based on boolean
[
https://issues.apache.org/jira/browse/FLINK-8008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16243362#comment-16243362
]
Muhammad Imran Tariq edited comment on FLINK-8008 at 11/8/17 4:40 AM:
--
I am calling
_public PojoTypeInfo(Class typeClass, List fields)_ constructor
of class _PojoTypeInfo_.
Below is my code.
_//create a PojoTypeInfo
PojoTypeInfo sourceType = new PojoTypeInfo(Person.class,
fieldList);
//create a dataset
DataSet data= env.createInput(new PojoCsvInputFormat(new
Path(textPath),
CsvInputFormat.DEFAULT_LINE_DELIMITER,
CsvInputFormat.DEFAULT_FIELD_DELIMITER, sourceType),
sourceType);
//create a table of this dataset
Table newT = tableEnv.fromDataSet(text);
//sink table
TableSink sink = new CsvTableSink("fielpath.csv", "|", 1,WriteMode.OVERWRITE);
newT.writeToSink(sink);_
So as I said earlier that there are two fields in my POJO class. first is
ID(Integer), second one is Age(Double). PojoTypeInfo info sorts fields in
alphabetical order. But CSVReader reads file and did not sort columns. When I
sink my table, then datatype of Age field(which is Double) get applied on ID
field. So initially my data in CSV was:
1,25
2,33
After sink it becomes
1.0,25
2.0,33
To avoid this I want PojoTypeInfo class not to sort fields inside its
constructor.
was (Author: imran.tariq):
I am calling
_public PojoTypeInfo(Class typeClass, List fields)_ constructor
of class _PojoTypeInfo_.
Below is my code.
_//create a PojoTypeInfo
PojoTypeInfo sourceType = new PojoTypeInfo(Person.class,
fieldList);
//create a dataset
DataSet data= env.createInput(new PojoCsvInputFormat(new
Path(textPath),
CsvInputFormat.DEFAULT_LINE_DELIMITER,
CsvInputFormat.DEFAULT_FIELD_DELIMITER, sourceType),
sourceType);
//create a table of this dataset
Table newT = tableEnv.fromDataSet(text);
//sink table
TableSink sink = new CsvTableSink("D:\\invesco\\POC\\Flink\\rules
implementation\\data3.csv", "|", 1,
WriteMode.OVERWRITE);
newT.writeToSink(sink);_
So as I said earlier that there are two fields in my POJO class. first is
ID(Integer), second one is Age(Double). PojoTypeInfo info sorts fields in
alphabetical order. But CSVReader reads file and did not sort columns. When I
sink my table, then datatype of Age field(which is Double) get applied on ID
field. So initially my data in CSV was:
1,25
2,33
After sink it becomes
1.0,25
2.0,33
To avoid this I want PojoTypeInfo class not to sort fields inside its
constructor.
> PojoTypeInfo should sort fields fields based on boolean
> ---
>
> Key: FLINK-8008
> URL: https://issues.apache.org/jira/browse/FLINK-8008
> Project: Flink
> Issue Type: Improvement
> Components: DataSet API
>Affects Versions: 1.3.2
>Reporter: Muhammad Imran Tariq
>Priority: Minor
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Flink PojoTypeInfo sorts fields array that are passed into constructor
> arguments. I want to create another constructor that takes boolean parameter
> to sort field or not.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-8008) PojoTypeInfo should sort fields fields based on boolean
[ https://issues.apache.org/jira/browse/FLINK-8008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16242255#comment-16242255 ] Muhammad Imran Tariq commented on FLINK-8008: - I am using POJO way to read CSV file. Say a csv file have 2 columns. ID - Integer | AGE - Double When a csv file is read, DataSet will make id first column but PojoTypeInfo read fields in sorted way and place 'age' column at 0 index. So Double datatype is being applied to ID column. That's why I don't want to sort fields. > PojoTypeInfo should sort fields fields based on boolean > --- > > Key: FLINK-8008 > URL: https://issues.apache.org/jira/browse/FLINK-8008 > Project: Flink > Issue Type: Improvement > Components: DataSet API >Affects Versions: 1.3.2 >Reporter: Muhammad Imran Tariq >Priority: Minor > Original Estimate: 2h > Remaining Estimate: 2h > > Flink PojoTypeInfo sorts fields array that are passed into constructor > arguments. I want to create another constructor that takes boolean parameter > to sort field or not. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (FLINK-8008) PojoTypeInfo should sort fields fields based on boolean
[ https://issues.apache.org/jira/browse/FLINK-8008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Muhammad Imran Tariq updated FLINK-8008: Summary: PojoTypeInfo should sort fields fields based on boolean (was: PojoTypeInfo should not sort fields) > PojoTypeInfo should sort fields fields based on boolean > --- > > Key: FLINK-8008 > URL: https://issues.apache.org/jira/browse/FLINK-8008 > Project: Flink > Issue Type: Improvement > Components: DataSet API >Affects Versions: 1.3.2 >Reporter: Muhammad Imran Tariq >Priority: Minor > Original Estimate: 2h > Remaining Estimate: 2h > > Flink PojoTypeInfo sorts fields array that are passed into constructor > arguments. I want to create another constructor that takes boolean parameter > to sort field or not. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-8008) PojoTypeInfo should not sort fields
Muhammad Imran Tariq created FLINK-8008: --- Summary: PojoTypeInfo should not sort fields Key: FLINK-8008 URL: https://issues.apache.org/jira/browse/FLINK-8008 Project: Flink Issue Type: Improvement Components: DataSet API Affects Versions: 1.3.2 Reporter: Muhammad Imran Tariq Priority: Minor Flink PojoTypeInfo sorts fields array that are passed into constructor arguments. I want to create another constructor that takes boolean parameter to sort field or not. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
