[jira] [Commented] (CASSANDRA-8970) Allow custom time_format on cqlsh COPY TO
[
https://issues.apache.org/jira/browse/CASSANDRA-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14985379#comment-14985379
]
Aaron Ploetz commented on CASSANDRA-8970:
-
Not sure what happened here, but although this patch has not yet made it into
the community version, it *IS* out there in DSE 4.8.1. I ran across this
StackOverflow question on Thursday (and recognized the variable name):
http://stackoverflow.com/questions/33424739/display-timestamp-format-issue-on-cql
As this *did* work at one point, this issue would seem to suggest that some
kind of merge error took place. Especially since patches are supposed to
"bake" in DSC for a bit before making their way into DSE.
Basically, it looks like {{display_timestamp_format}} is not being defined. It
was previously defined where the {{shell}} was instantiated ({{shell =
Shell(}}) in the "main" method, but I don't see it in the version of cqlsh that
ships with DSE 4.8.1. Not sure where that got lost, but that definition needs
to be accounted for.
> Allow custom time_format on cqlsh COPY TO
> -
>
> Key: CASSANDRA-8970
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8970
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Aaron Ploetz
>Assignee: Aaron Ploetz
>Priority: Trivial
> Labels: cqlsh
> Fix For: 3.0.0 rc2, 2.2.4, 2.1.12
>
> Attachments: CASSANDRA-8970.patch
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> When executing a COPY TO from cqlsh, the user is currently has no control
> over the format of exported timestamp columns. If the user has indicated a
> {{time_format}} in their cqlshrc file, that format will be used. Otherwise,
> the system default format will be used.
> The problem comes into play when the timestamp format used on a COPY TO, is
> not valid when the data is sent back into Cassandra with a COPY FROM.
> For instance, if a user has {{time_format = %Y-%m-%d %H:%M:%S%Z}} specified
> in their cqlshrc, COPY TO will format timestamp columns like this:
> {{userid|posttime|postcontent}}
> {{0|2015-03-14 14:59:00CDT|rtyeryerweh}}
> {{0|2015-03-14 14:58:00CDT|sdfsdfsdgfjdsgojr}}
> {{0|2015-03-12 14:27:00CDT|sdgfjdsgojr}}
> Executing a COPY FROM on that same file will produce an "unable to coerce to
> formatted date(long)" error.
> Right now, the only way to change the way timestamps are formatted is to exit
> cqlsh, modify the {{time_format}} property in cqlshrc, and restart cqlsh.
> The ability to specify a COPY option of TIME_FORMAT with a Python strftime
> format, would allow the user to quickly alter the timestamp format for
> export, without reconfiguring cqlsh.
> {{aploetz@cqlsh:stackoverflow> COPY posts1 TO '/home/aploetz/posts1.csv' WITH
> DELIMITER='|' AND HEADER=true AND TIME_FORMAT='%Y-%m-%d %H:%M:%S%z;}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8970) Allow custom time_format on cqlsh COPY TO
[
https://issues.apache.org/jira/browse/CASSANDRA-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14957108#comment-14957108
]
Aleksey Yeschenko commented on CASSANDRA-8970:
--
Thanks. Committed to 2.1 as
[3c0a1a5c1a77ca1148bef1712a2859400d85ea8c|https://github.com/apache/cassandra/commit/3c0a1a5c1a77ca1148bef1712a2859400d85ea8c]
and merged upwards.
> Allow custom time_format on cqlsh COPY TO
> -
>
> Key: CASSANDRA-8970
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8970
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Aaron Ploetz
>Assignee: Aaron Ploetz
>Priority: Trivial
> Labels: cqlsh
> Fix For: 3.0.0 rc2, 2.2.4, 2.1.12
>
> Attachments: CASSANDRA-8970.patch
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> When executing a COPY TO from cqlsh, the user is currently has no control
> over the format of exported timestamp columns. If the user has indicated a
> {{time_format}} in their cqlshrc file, that format will be used. Otherwise,
> the system default format will be used.
> The problem comes into play when the timestamp format used on a COPY TO, is
> not valid when the data is sent back into Cassandra with a COPY FROM.
> For instance, if a user has {{time_format = %Y-%m-%d %H:%M:%S%Z}} specified
> in their cqlshrc, COPY TO will format timestamp columns like this:
> {{userid|posttime|postcontent}}
> {{0|2015-03-14 14:59:00CDT|rtyeryerweh}}
> {{0|2015-03-14 14:58:00CDT|sdfsdfsdgfjdsgojr}}
> {{0|2015-03-12 14:27:00CDT|sdgfjdsgojr}}
> Executing a COPY FROM on that same file will produce an "unable to coerce to
> formatted date(long)" error.
> Right now, the only way to change the way timestamps are formatted is to exit
> cqlsh, modify the {{time_format}} property in cqlshrc, and restart cqlsh.
> The ability to specify a COPY option of TIME_FORMAT with a Python strftime
> format, would allow the user to quickly alter the timestamp format for
> export, without reconfiguring cqlsh.
> {{aploetz@cqlsh:stackoverflow> COPY posts1 TO '/home/aploetz/posts1.csv' WITH
> DELIMITER='|' AND HEADER=true AND TIME_FORMAT='%Y-%m-%d %H:%M:%S%z;}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8970) Allow custom time_format on cqlsh COPY TO
[
https://issues.apache.org/jira/browse/CASSANDRA-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949236#comment-14949236
]
Carl Yeksigian commented on CASSANDRA-8970:
---
[2.1 branch|https://github.com/carlyeks/cassandra/tree/review/8970/2.1] [2.2
branch|https://github.com/carlyeks/cassandra/tree/review/8970/2.2]
The 2.2 branch applies cleanly up to trunk.
> Allow custom time_format on cqlsh COPY TO
> -
>
> Key: CASSANDRA-8970
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8970
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Aaron Ploetz
>Priority: Trivial
> Labels: cqlsh
> Fix For: 2.1.x
>
> Attachments: CASSANDRA-8970.patch
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> When executing a COPY TO from cqlsh, the user is currently has no control
> over the format of exported timestamp columns. If the user has indicated a
> {{time_format}} in their cqlshrc file, that format will be used. Otherwise,
> the system default format will be used.
> The problem comes into play when the timestamp format used on a COPY TO, is
> not valid when the data is sent back into Cassandra with a COPY FROM.
> For instance, if a user has {{time_format = %Y-%m-%d %H:%M:%S%Z}} specified
> in their cqlshrc, COPY TO will format timestamp columns like this:
> {{userid|posttime|postcontent}}
> {{0|2015-03-14 14:59:00CDT|rtyeryerweh}}
> {{0|2015-03-14 14:58:00CDT|sdfsdfsdgfjdsgojr}}
> {{0|2015-03-12 14:27:00CDT|sdgfjdsgojr}}
> Executing a COPY FROM on that same file will produce an "unable to coerce to
> formatted date(long)" error.
> Right now, the only way to change the way timestamps are formatted is to exit
> cqlsh, modify the {{time_format}} property in cqlshrc, and restart cqlsh.
> The ability to specify a COPY option of TIME_FORMAT with a Python strftime
> format, would allow the user to quickly alter the timestamp format for
> export, without reconfiguring cqlsh.
> {{aploetz@cqlsh:stackoverflow> COPY posts1 TO '/home/aploetz/posts1.csv' WITH
> DELIMITER='|' AND HEADER=true AND TIME_FORMAT='%Y-%m-%d %H:%M:%S%z;}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8970) Allow custom time_format on cqlsh COPY TO
[
https://issues.apache.org/jira/browse/CASSANDRA-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14941828#comment-14941828
]
Aleksey Yeschenko commented on CASSANDRA-8970:
--
[~thobbs] [~carlyeks] can either of you have a look at rebasing the
patch/seeing if it's still relevant? Thank you.
> Allow custom time_format on cqlsh COPY TO
> -
>
> Key: CASSANDRA-8970
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8970
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Aaron Ploetz
>Priority: Trivial
> Labels: cqlsh
> Fix For: 2.1.x
>
> Attachments: CASSANDRA-8970.patch
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> When executing a COPY TO from cqlsh, the user is currently has no control
> over the format of exported timestamp columns. If the user has indicated a
> {{time_format}} in their cqlshrc file, that format will be used. Otherwise,
> the system default format will be used.
> The problem comes into play when the timestamp format used on a COPY TO, is
> not valid when the data is sent back into Cassandra with a COPY FROM.
> For instance, if a user has {{time_format = %Y-%m-%d %H:%M:%S%Z}} specified
> in their cqlshrc, COPY TO will format timestamp columns like this:
> {{userid|posttime|postcontent}}
> {{0|2015-03-14 14:59:00CDT|rtyeryerweh}}
> {{0|2015-03-14 14:58:00CDT|sdfsdfsdgfjdsgojr}}
> {{0|2015-03-12 14:27:00CDT|sdgfjdsgojr}}
> Executing a COPY FROM on that same file will produce an "unable to coerce to
> formatted date(long)" error.
> Right now, the only way to change the way timestamps are formatted is to exit
> cqlsh, modify the {{time_format}} property in cqlshrc, and restart cqlsh.
> The ability to specify a COPY option of TIME_FORMAT with a Python strftime
> format, would allow the user to quickly alter the timestamp format for
> export, without reconfiguring cqlsh.
> {{aploetz@cqlsh:stackoverflow> COPY posts1 TO '/home/aploetz/posts1.csv' WITH
> DELIMITER='|' AND HEADER=true AND TIME_FORMAT='%Y-%m-%d %H:%M:%S%z;}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8970) Allow custom time_format on cqlsh COPY TO
[
https://issues.apache.org/jira/browse/CASSANDRA-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14790828#comment-14790828
]
Brandon Williams commented on CASSANDRA-8970:
-
Looks like this needs a rebase now.
> Allow custom time_format on cqlsh COPY TO
> -
>
> Key: CASSANDRA-8970
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8970
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Aaron Ploetz
>Priority: Trivial
> Labels: cqlsh
> Fix For: 2.1.x
>
> Attachments: CASSANDRA-8970.patch
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> When executing a COPY TO from cqlsh, the user is currently has no control
> over the format of exported timestamp columns. If the user has indicated a
> {{time_format}} in their cqlshrc file, that format will be used. Otherwise,
> the system default format will be used.
> The problem comes into play when the timestamp format used on a COPY TO, is
> not valid when the data is sent back into Cassandra with a COPY FROM.
> For instance, if a user has {{time_format = %Y-%m-%d %H:%M:%S%Z}} specified
> in their cqlshrc, COPY TO will format timestamp columns like this:
> {{userid|posttime|postcontent}}
> {{0|2015-03-14 14:59:00CDT|rtyeryerweh}}
> {{0|2015-03-14 14:58:00CDT|sdfsdfsdgfjdsgojr}}
> {{0|2015-03-12 14:27:00CDT|sdgfjdsgojr}}
> Executing a COPY FROM on that same file will produce an "unable to coerce to
> formatted date(long)" error.
> Right now, the only way to change the way timestamps are formatted is to exit
> cqlsh, modify the {{time_format}} property in cqlshrc, and restart cqlsh.
> The ability to specify a COPY option of TIME_FORMAT with a Python strftime
> format, would allow the user to quickly alter the timestamp format for
> export, without reconfiguring cqlsh.
> {{aploetz@cqlsh:stackoverflow> COPY posts1 TO '/home/aploetz/posts1.csv' WITH
> DELIMITER='|' AND HEADER=true AND TIME_FORMAT='%Y-%m-%d %H:%M:%S%z;}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8970) Allow custom time_format on cqlsh COPY TO
[
https://issues.apache.org/jira/browse/CASSANDRA-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14727468#comment-14727468
]
Carl Yeksigian commented on CASSANDRA-8970:
---
Sorry I missed this ticket. This looks good, I created CASSANDRA-10252 as the
follow up to use the same property for both {{COPY TO}} and {{COPY FROM}}.
> Allow custom time_format on cqlsh COPY TO
> -
>
> Key: CASSANDRA-8970
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8970
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Aaron Ploetz
>Priority: Trivial
> Labels: cqlsh
> Fix For: 2.1.x
>
> Attachments: CASSANDRA-8970.patch
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> When executing a COPY TO from cqlsh, the user is currently has no control
> over the format of exported timestamp columns. If the user has indicated a
> {{time_format}} in their cqlshrc file, that format will be used. Otherwise,
> the system default format will be used.
> The problem comes into play when the timestamp format used on a COPY TO, is
> not valid when the data is sent back into Cassandra with a COPY FROM.
> For instance, if a user has {{time_format = %Y-%m-%d %H:%M:%S%Z}} specified
> in their cqlshrc, COPY TO will format timestamp columns like this:
> {{userid|posttime|postcontent}}
> {{0|2015-03-14 14:59:00CDT|rtyeryerweh}}
> {{0|2015-03-14 14:58:00CDT|sdfsdfsdgfjdsgojr}}
> {{0|2015-03-12 14:27:00CDT|sdgfjdsgojr}}
> Executing a COPY FROM on that same file will produce an "unable to coerce to
> formatted date(long)" error.
> Right now, the only way to change the way timestamps are formatted is to exit
> cqlsh, modify the {{time_format}} property in cqlshrc, and restart cqlsh.
> The ability to specify a COPY option of TIME_FORMAT with a Python strftime
> format, would allow the user to quickly alter the timestamp format for
> export, without reconfiguring cqlsh.
> {{aploetz@cqlsh:stackoverflow> COPY posts1 TO '/home/aploetz/posts1.csv' WITH
> DELIMITER='|' AND HEADER=true AND TIME_FORMAT='%Y-%m-%d %H:%M:%S%z;}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8970) Allow custom time_format on cqlsh COPY TO
[
https://issues.apache.org/jira/browse/CASSANDRA-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14701683#comment-14701683
]
Tyler Hobbs commented on CASSANDRA-8970:
Regarding the timezone issue with Windows, the recently committed
CASSANDRA-1 makes cqlsh always use UTC for the timezone, so I don't think
that will be an issue any more.
Allow custom time_format on cqlsh COPY TO
-
Key: CASSANDRA-8970
URL: https://issues.apache.org/jira/browse/CASSANDRA-8970
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Aaron Ploetz
Priority: Trivial
Labels: cqlsh
Fix For: 2.1.x
Attachments: CASSANDRA-8970.patch
Original Estimate: 4h
Remaining Estimate: 4h
When executing a COPY TO from cqlsh, the user is currently has no control
over the format of exported timestamp columns. If the user has indicated a
{{time_format}} in their cqlshrc file, that format will be used. Otherwise,
the system default format will be used.
The problem comes into play when the timestamp format used on a COPY TO, is
not valid when the data is sent back into Cassandra with a COPY FROM.
For instance, if a user has {{time_format = %Y-%m-%d %H:%M:%S%Z}} specified
in their cqlshrc, COPY TO will format timestamp columns like this:
{{userid|posttime|postcontent}}
{{0|2015-03-14 14:59:00CDT|rtyeryerweh}}
{{0|2015-03-14 14:58:00CDT|sdfsdfsdgfjdsgojr}}
{{0|2015-03-12 14:27:00CDT|sdgfjdsgojr}}
Executing a COPY FROM on that same file will produce an unable to coerce to
formatted date(long) error.
Right now, the only way to change the way timestamps are formatted is to exit
cqlsh, modify the {{time_format}} property in cqlshrc, and restart cqlsh.
The ability to specify a COPY option of TIME_FORMAT with a Python strftime
format, would allow the user to quickly alter the timestamp format for
export, without reconfiguring cqlsh.
{{aploetz@cqlsh:stackoverflow COPY posts1 TO '/home/aploetz/posts1.csv' WITH
DELIMITER='|' AND HEADER=true AND TIME_FORMAT='%Y-%m-%d %H:%M:%S%z;}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8970) Allow custom time_format on cqlsh COPY TO
[
https://issues.apache.org/jira/browse/CASSANDRA-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14700067#comment-14700067
]
Jonathan Ellis commented on CASSANDRA-8970:
---
bq. it becomes a nice to have if COPY FROM could interpret the default
exported timestamps correctly
Did anyone create a ticket for that?
Allow custom time_format on cqlsh COPY TO
-
Key: CASSANDRA-8970
URL: https://issues.apache.org/jira/browse/CASSANDRA-8970
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Aaron Ploetz
Priority: Trivial
Labels: cqlsh
Fix For: 2.1.x
Attachments: CASSANDRA-8970.patch
Original Estimate: 4h
Remaining Estimate: 4h
When executing a COPY TO from cqlsh, the user is currently has no control
over the format of exported timestamp columns. If the user has indicated a
{{time_format}} in their cqlshrc file, that format will be used. Otherwise,
the system default format will be used.
The problem comes into play when the timestamp format used on a COPY TO, is
not valid when the data is sent back into Cassandra with a COPY FROM.
For instance, if a user has {{time_format = %Y-%m-%d %H:%M:%S%Z}} specified
in their cqlshrc, COPY TO will format timestamp columns like this:
{{userid|posttime|postcontent}}
{{0|2015-03-14 14:59:00CDT|rtyeryerweh}}
{{0|2015-03-14 14:58:00CDT|sdfsdfsdgfjdsgojr}}
{{0|2015-03-12 14:27:00CDT|sdgfjdsgojr}}
Executing a COPY FROM on that same file will produce an unable to coerce to
formatted date(long) error.
Right now, the only way to change the way timestamps are formatted is to exit
cqlsh, modify the {{time_format}} property in cqlshrc, and restart cqlsh.
The ability to specify a COPY option of TIME_FORMAT with a Python strftime
format, would allow the user to quickly alter the timestamp format for
export, without reconfiguring cqlsh.
{{aploetz@cqlsh:stackoverflow COPY posts1 TO '/home/aploetz/posts1.csv' WITH
DELIMITER='|' AND HEADER=true AND TIME_FORMAT='%Y-%m-%d %H:%M:%S%z;}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8970) Allow custom time_format on cqlsh COPY TO
[
https://issues.apache.org/jira/browse/CASSANDRA-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14699174#comment-14699174
]
Yariv Amar commented on CASSANDRA-8970:
---
Hi
I'd like to point out another problem with timestamps and COPY TO / FROM.
When using {{COPY TO}} from a cqlsh that is on a *windows* machine, i get the
following csv:{{some_data,2015-08-16 18:07:42Jerusalem Daylight Time}}
When using {{COPY TO}} from a cqlsh that is on a *Unix* machine i get the
following csv:{{some_data,2015-08-16 18:07:42+0300}}
As you can see the time zones presented differently. Unfortunately, the export
from windows can't be imported back with {{COPY FROM}}.
It produce the error: {{bad Request: unable to coerce '2015-08-16
18:07:42Jerusalem Standard Time' to a formatted date (long)}}
The interesting part is that even when I change the {{time_format}} in the
{{cqlshrc}} on the *windows* machine, I still get the same output string:
{{time_format=%Y-%m-%d %H:%M:%S%z}} or {{time_format=%Y-%m-%d %H:%M:%S%Z}}
still writes the timestamp as ,2015-08-16 18:07:42Jerusalem Daylight Time
Besides that, I like the idea to add a {{TIME_FORMAT}} option to {{COPY TO}}.
Allow custom time_format on cqlsh COPY TO
-
Key: CASSANDRA-8970
URL: https://issues.apache.org/jira/browse/CASSANDRA-8970
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Aaron Ploetz
Priority: Trivial
Labels: cqlsh
Fix For: 2.1.x
Attachments: CASSANDRA-8970.patch
Original Estimate: 4h
Remaining Estimate: 4h
When executing a COPY TO from cqlsh, the user is currently has no control
over the format of exported timestamp columns. If the user has indicated a
{{time_format}} in their cqlshrc file, that format will be used. Otherwise,
the system default format will be used.
The problem comes into play when the timestamp format used on a COPY TO, is
not valid when the data is sent back into Cassandra with a COPY FROM.
For instance, if a user has {{time_format = %Y-%m-%d %H:%M:%S%Z}} specified
in their cqlshrc, COPY TO will format timestamp columns like this:
{{userid|posttime|postcontent}}
{{0|2015-03-14 14:59:00CDT|rtyeryerweh}}
{{0|2015-03-14 14:58:00CDT|sdfsdfsdgfjdsgojr}}
{{0|2015-03-12 14:27:00CDT|sdgfjdsgojr}}
Executing a COPY FROM on that same file will produce an unable to coerce to
formatted date(long) error.
Right now, the only way to change the way timestamps are formatted is to exit
cqlsh, modify the {{time_format}} property in cqlshrc, and restart cqlsh.
The ability to specify a COPY option of TIME_FORMAT with a Python strftime
format, would allow the user to quickly alter the timestamp format for
export, without reconfiguring cqlsh.
{{aploetz@cqlsh:stackoverflow COPY posts1 TO '/home/aploetz/posts1.csv' WITH
DELIMITER='|' AND HEADER=true AND TIME_FORMAT='%Y-%m-%d %H:%M:%S%z;}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8970) Allow custom time_format on cqlsh COPY TO
[
https://issues.apache.org/jira/browse/CASSANDRA-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379863#comment-14379863
]
Aaron Ploetz commented on CASSANDRA-8970:
-
[~thobbs] you're absolutely right. Solving the interpretation of timestamps
with {{COPY FROM}} is definitely the larger issue here. My patch helps as a
workaround to allow timestamps to be exported in a format that the current
{{COPY FROM}} can understand. But it becomes a nice to have if {{COPY FROM}}
could interpret the default exported timestamps correctly.
Allow custom time_format on cqlsh COPY TO
-
Key: CASSANDRA-8970
URL: https://issues.apache.org/jira/browse/CASSANDRA-8970
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Aaron Ploetz
Priority: Trivial
Labels: cqlsh
Fix For: 2.1.4
Attachments: CASSANDRA-8970.patch
Original Estimate: 4h
Remaining Estimate: 4h
When executing a COPY TO from cqlsh, the user is currently has no control
over the format of exported timestamp columns. If the user has indicated a
{{time_format}} in their cqlshrc file, that format will be used. Otherwise,
the system default format will be used.
The problem comes into play when the timestamp format used on a COPY TO, is
not valid when the data is sent back into Cassandra with a COPY FROM.
For instance, if a user has {{time_format = %Y-%m-%d %H:%M:%S%Z}} specified
in their cqlshrc, COPY TO will format timestamp columns like this:
{{userid|posttime|postcontent}}
{{0|2015-03-14 14:59:00CDT|rtyeryerweh}}
{{0|2015-03-14 14:58:00CDT|sdfsdfsdgfjdsgojr}}
{{0|2015-03-12 14:27:00CDT|sdgfjdsgojr}}
Executing a COPY FROM on that same file will produce an unable to coerce to
formatted date(long) error.
Right now, the only way to change the way timestamps are formatted is to exit
cqlsh, modify the {{time_format}} property in cqlshrc, and restart cqlsh.
The ability to specify a COPY option of TIME_FORMAT with a Python strftime
format, would allow the user to quickly alter the timestamp format for
export, without reconfiguring cqlsh.
{{aploetz@cqlsh:stackoverflow COPY posts1 TO '/home/aploetz/posts1.csv' WITH
DELIMITER='|' AND HEADER=true AND TIME_FORMAT='%Y-%m-%d %H:%M:%S%z;}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8970) Allow custom time_format on cqlsh COPY TO
[
https://issues.apache.org/jira/browse/CASSANDRA-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378155#comment-14378155
]
Tyler Hobbs commented on CASSANDRA-8970:
Hi [~aploetz], thanks for the patch!
It seems to me that the biggest problem raised in this ticket is that while
{{COPY TO}} uses the {{time_format}} property to encode timestamps when writing
to a CSV, {{COPY FROM}} does _not_ use that property. It seems like {{COPY
FROM}} should use {{time_format}} to convert the date/timestamp to a long
numeric timestamp (which Cassandra can interpret properly). Would you agree
with that? (I could open a separate ticket for that.)
Besides that, your patch to add a {{TIME_FORMAT}} option to {{COPY TO}} looks
fine to me.
Allow custom time_format on cqlsh COPY TO
-
Key: CASSANDRA-8970
URL: https://issues.apache.org/jira/browse/CASSANDRA-8970
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Aaron Ploetz
Priority: Trivial
Labels: cqlsh
Fix For: 2.1.4
Attachments: CASSANDRA-8970.patch
Original Estimate: 4h
Remaining Estimate: 4h
When executing a COPY TO from cqlsh, the user is currently has no control
over the format of exported timestamp columns. If the user has indicated a
{{time_format}} in their cqlshrc file, that format will be used. Otherwise,
the system default format will be used.
The problem comes into play when the timestamp format used on a COPY TO, is
not valid when the data is sent back into Cassandra with a COPY FROM.
For instance, if a user has {{time_format = %Y-%m-%d %H:%M:%S%Z}} specified
in their cqlshrc, COPY TO will format timestamp columns like this:
{{userid|posttime|postcontent}}
{{0|2015-03-14 14:59:00CDT|rtyeryerweh}}
{{0|2015-03-14 14:58:00CDT|sdfsdfsdgfjdsgojr}}
{{0|2015-03-12 14:27:00CDT|sdgfjdsgojr}}
Executing a COPY FROM on that same file will produce an unable to coerce to
formatted date(long) error.
Right now, the only way to change the way timestamps are formatted is to exit
cqlsh, modify the {{time_format}} property in cqlshrc, and restart cqlsh.
The ability to specify a COPY option of TIME_FORMAT with a Python strftime
format, would allow the user to quickly alter the timestamp format for
export, without reconfiguring cqlsh.
{{aploetz@cqlsh:stackoverflow COPY posts1 TO '/home/aploetz/posts1.csv' WITH
DELIMITER='|' AND HEADER=true AND TIME_FORMAT='%Y-%m-%d %H:%M:%S%z;}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
