matriv commented on a change in pull request #17878:
URL: https://github.com/apache/flink/pull/17878#discussion_r754928044
##########
File path:
flink-connectors/flink-connector-hbase-2.2/src/test/java/org/apache/flink/connector/hbase2/util/HBaseTestBase.java
##########
@@ -263,11 +262,13 @@ private static Put putRow(
put.addColumn(
Bytes.toBytes(FAMILY4),
Bytes.toBytes(F4COL1),
- Bytes.toBytes(timestampToInternal(f4c1)));
+ Bytes.toBytes(DateTimeUtils.toInternal(f4c1)));
put.addColumn(
- Bytes.toBytes(FAMILY4), Bytes.toBytes(F4COL2),
Bytes.toBytes(dateToInternal(f4c2)));
+ Bytes.toBytes(FAMILY4), Bytes.toBytes(F4COL2),
Bytes.toBytes(toInternal(f4c2)));
Review comment:
```suggestion
Bytes.toBytes(FAMILY4), Bytes.toBytes(F4COL2),
Bytes.toBytes(DateTimUtils.toInternal(f4c2)));
```
or above use also the static import
##########
File path:
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/expressions/converter/ExpressionConverter.java
##########
@@ -64,8 +64,8 @@
import java.util.Optional;
import java.util.stream.Collectors;
+import static
org.apache.flink.table.planner.utils.TimestampStringUtils.fromLocalDateTime;
Review comment:
maybe hotfix commit, since nothing else has changed in this file.
##########
File path:
flink-connectors/flink-connector-hbase-1.4/src/test/java/org/apache/flink/connector/hbase1/util/HBaseTestBase.java
##########
@@ -263,11 +262,13 @@ private static Put putRow(
put.addColumn(
Bytes.toBytes(FAMILY4),
Bytes.toBytes(F4COL1),
- Bytes.toBytes(timestampToInternal(f4c1)));
+ Bytes.toBytes(DateTimeUtils.toInternal(f4c1)));
put.addColumn(
- Bytes.toBytes(FAMILY4), Bytes.toBytes(F4COL2),
Bytes.toBytes(dateToInternal(f4c2)));
+ Bytes.toBytes(FAMILY4), Bytes.toBytes(F4COL2),
Bytes.toBytes(toInternal(f4c2)));
Review comment:
```suggestion
Bytes.toBytes(FAMILY4), Bytes.toBytes(F4COL2),
Bytes.toBytes(DateTimeUtils.toInternal(f4c2)));
```
or use the static import everywhere
##########
File path:
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/functions/casting/AbstractCodeGeneratorCastRule.java
##########
@@ -102,12 +105,32 @@ protected AbstractCodeGeneratorCastRule(CastRulePredicate
predicate) {
final String functionSignature =
"@Override public Object cast(Object _myInputObj) throws "
+ className(TableException.class);
- final String inputVarDecl =
- inputTypeTerm + " " + inputTerm + " = (" + inputTypeTerm + ")
_myInputObj;\n";
- final String inputIsNullVarDecl =
- "boolean " + inputIsNullTerm + " = _myInputObj == null;\n";
- final String returnStmt = "return " + codeBlock.getReturnTerm() +
";\n";
+ // Write the function body
+ final CastRuleUtils.CodeWriter bodyWriter = new
CastRuleUtils.CodeWriter();
+ bodyWriter.declStmt(inputTypeTerm, inputTerm, cast(inputTypeTerm,
"_myInputObj"));
+ bodyWriter.declStmt("boolean", inputIsNullTerm, "_myInputObj == null");
+ ctx.variableDeclarationStatements.forEach(decl ->
bodyWriter.appendBlock(decl + "\n"));
+
+ if (this.canFail()) {
+ bodyWriter.tryCatchStmt(
+ tryWriter ->
+ tryWriter.append(codeBlock).stmt("return " +
codeBlock.getReturnTerm()),
+ Throwable.class,
+ (exceptionTerm, catchWriter) ->
+ catchWriter.throwStmt(
+ constructorCall(
Review comment:
Maybe it makes sense to also provide the exception instantiation also as
a utility method.
##########
File path:
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/codegen/calls/BuiltInMethodsTest.java
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.codegen.calls;
+
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.lang.reflect.Method;
+import java.lang.reflect.Modifier;
+import java.util.Arrays;
+import java.util.stream.Stream;
+
+import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+
+/** This test class is checking if methods defined in {@link BuiltInMethods}
are valid. */
+class BuiltInMethodsTest {
+
+ private static Stream<Method> testMethodsAreAvailable() {
+ return Arrays.stream(BuiltInMethods.class.getMethods())
+ .filter(
+ m ->
+ Modifier.isStatic(m.getModifiers())
+ &&
Modifier.isPublic(m.getModifiers()));
+ }
+
+ @ParameterizedTest
+ @MethodSource
+ void testMethodsAreAvailable(Method m) throws Exception {
+ assertDoesNotThrow(
Review comment:
Maybe add a comment regarding this invocation with `null` and the
specific use case on static methods.
##########
File path:
flink-table/flink-table-common/src/main/java/org/apache/flink/table/utils/DateTimeUtils.java
##########
@@ -1251,133 +1356,24 @@ private static long julianDateFloor(TimeUnitRange
range, int julian, boolean flo
}
}
- //
--------------------------------------------------------------------------------------------
- // DATE DIFF/ADD
- //
--------------------------------------------------------------------------------------------
-
- /**
- * NOTE: (1). JDK relies on the operating system clock for time. Each
operating system has its
- * own method of handling date changes such as leap seconds(e.g. OS will
slow down the clock to
- * accommodate for this). (2). DST(Daylight Saving Time) is a legal issue,
governments changed
- * it over time. Some days are NOT exactly 24 hours long, it could be
23/25 hours long on the
- * first or last day of daylight saving time. JDK can handle DST
correctly. TODO: carefully
- * written algorithm can improve the performance
- */
- public static int dateDiff(long t1, long t2, TimeZone tz) {
- ZoneId zoneId = tz.toZoneId();
- LocalDate ld1 = Instant.ofEpochMilli(t1).atZone(zoneId).toLocalDate();
- LocalDate ld2 = Instant.ofEpochMilli(t2).atZone(zoneId).toLocalDate();
- return (int) ChronoUnit.DAYS.between(ld2, ld1);
- }
-
- public static int dateDiff(String t1Str, long t2, TimeZone tz) {
- long t1 = parseToTimeMillis(t1Str, tz);
- return dateDiff(t1, t2, tz);
- }
-
- public static int dateDiff(long t1, String t2Str, TimeZone tz) {
- long t2 = parseToTimeMillis(t2Str, tz);
- return dateDiff(t1, t2, tz);
- }
-
- public static int dateDiff(String t1Str, String t2Str, TimeZone tz) {
- long t1 = parseToTimeMillis(t1Str, tz);
- long t2 = parseToTimeMillis(t2Str, tz);
- return dateDiff(t1, t2, tz);
- }
-
- public static int dateDiff(long t1, long t2) {
- return dateDiff(t1, t2, UTC_ZONE);
- }
-
- public static int dateDiff(String t1Str, long t2) {
- return dateDiff(t1Str, t2, UTC_ZONE);
- }
-
- public static int dateDiff(long t1, String t2Str) {
- return dateDiff(t1, t2Str, UTC_ZONE);
- }
-
- public static int dateDiff(String t1Str, String t2Str) {
- return dateDiff(t1Str, t2Str, UTC_ZONE);
- }
-
- /**
- * Do subtraction on date string.
- *
- * @param dateStr formatted date string.
- * @param days days count you want to subtract.
- * @param tz time zone of the date time string
- * @return datetime string.
- */
- public static String dateSub(String dateStr, int days, TimeZone tz) {
- long ts = parseToTimeMillis(dateStr, tz);
- if (ts == Long.MIN_VALUE) {
- return null;
- }
- return dateSub(ts, days, tz);
- }
-
- /**
- * Do subtraction on date string.
- *
- * @param ts the timestamp.
- * @param days days count you want to subtract.
- * @param tz time zone of the date time string
- * @return datetime string.
- */
- public static String dateSub(long ts, int days, TimeZone tz) {
- ZoneId zoneId = tz.toZoneId();
- Instant instant = Instant.ofEpochMilli(ts);
- ZonedDateTime zdt = ZonedDateTime.ofInstant(instant, zoneId);
- long resultTs = zdt.minusDays(days).toInstant().toEpochMilli();
- return dateFormat(resultTs, DATE_FORMAT_STRING, tz);
- }
-
- public static String dateSub(String dateStr, int days) {
- return dateSub(dateStr, days, UTC_ZONE);
- }
-
- public static String dateSub(long ts, int days) {
- return dateSub(ts, days, UTC_ZONE);
- }
-
/**
- * Do addition on date string.
+ * Convert datetime string from a time zone to another time zone.
*
- * @param dateStr formatted date string.
- * @param days days count you want to add.
- * @return datetime string.
+ * @param dateStr the date time string
+ * @param tzFrom the original time zone
+ * @param tzTo the target time zone
*/
- public static String dateAdd(String dateStr, int days, TimeZone tz) {
- long ts = parseToTimeMillis(dateStr, tz);
- if (ts == Long.MIN_VALUE) {
+ public static String convertTz(String dateStr, String tzFrom, String tzTo)
{
+ try {
+ return dateFormatTz(parseTimestampTz(dateStr, tzFrom), tzTo);
+ } catch (ParseException e) {
return null;
}
- return dateAdd(ts, days, tz);
- }
-
- /**
- * Do addition on timestamp.
- *
- * @param ts the timestamp.
- * @param days days count you want to add.
- * @return datetime string.
- */
- public static String dateAdd(long ts, int days, TimeZone tz) {
- ZoneId zoneId = tz.toZoneId();
- Instant instant = Instant.ofEpochMilli(ts);
- ZonedDateTime zdt = ZonedDateTime.ofInstant(instant, zoneId);
- long resultTs = zdt.plusDays(days).toInstant().toEpochMilli();
- return dateFormat(resultTs, DATE_FORMAT_STRING, tz);
- }
-
- public static String dateAdd(String dateStr, int days) {
- return dateAdd(dateStr, days, UTC_ZONE);
}
- public static String dateAdd(long ts, int days) {
- return dateAdd(ts, days, UTC_ZONE);
+ private static String dateFormatTz(long ts, String tzStr) {
Review comment:
Wouldn't `timestampFormatTz` be more appropriate name? `date` for me is
confusing.
##########
File path:
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/codegen/calls/BuiltInMethodsTest.java
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.codegen.calls;
+
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.lang.reflect.Method;
+import java.lang.reflect.Modifier;
+import java.util.Arrays;
+import java.util.stream.Stream;
+
+import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+
+/** This test class is checking if methods defined in {@link BuiltInMethods}
are valid. */
Review comment:
```suggestion
/** Check if the static methods defined in {@link BuiltInMethods} are valid.
*/
```
##########
File path:
flink-table/flink-table-common/src/main/java/org/apache/flink/table/utils/DateTimeUtils.java
##########
@@ -412,92 +492,267 @@ public static Long toTimestamp(String dateStr, TimeZone
tz) {
* @param format date time string format
* @param tz the time zone
*/
- private static Long toTimestamp(String dateStr, String format, TimeZone
tz) {
+ private static long parseTimestampMillis(String dateStr, String format,
TimeZone tz)
+ throws ParseException {
SimpleDateFormat formatter = FORMATTER_CACHE.get(format);
formatter.setTimeZone(tz);
- try {
- return formatter.parse(dateStr).getTime();
- } catch (ParseException e) {
- return null;
- }
- }
-
- public static Long toTimestamp(String dateStr, String format) {
- return toTimestamp(dateStr, format, UTC_ZONE);
+ return formatter.parse(dateStr).getTime();
}
/**
* Parse date time string to timestamp based on the given time zone string
and format. Returns
* null if parsing failed.
*
* @param dateStr the date time string
- * @param format the date time string format
* @param tzStr the time zone id string
*/
- public static Long toTimestampTz(String dateStr, String format, String
tzStr) {
+ private static long parseTimestampTz(String dateStr, String tzStr) throws
ParseException {
TimeZone tz = TIMEZONE_CACHE.get(tzStr);
- return toTimestamp(dateStr, format, tz);
- }
-
- public static Long toTimestampTz(String dateStr, String tzStr) {
- // use "yyyy-MM-dd HH:mm:ss" as default format
- return toTimestampTz(dateStr, TIMESTAMP_FORMAT_STRING, tzStr);
+ return parseTimestampMillis(dateStr,
DateTimeUtils.TIMESTAMP_FORMAT_STRING, tz);
}
- //
--------------------------------------------------------------------------------------------
- // String --> Date conversion
- //
--------------------------------------------------------------------------------------------
-
/** Returns the epoch days since 1970-01-01. */
- public static int strToDate(String dateStr, String fromFormat) {
+ public static int parseDate(String dateStr, String fromFormat) {
// It is OK to use UTC, we just want get the epoch days
// TODO use offset, better performance
- long ts = parseToTimeMillis(dateStr, fromFormat,
TimeZone.getTimeZone("UTC"));
+ long ts = internalParseTimestampMillis(dateStr, fromFormat,
TimeZone.getTimeZone("UTC"));
ZoneId zoneId = ZoneId.of("UTC");
Instant instant = Instant.ofEpochMilli(ts);
ZonedDateTime zdt = ZonedDateTime.ofInstant(instant, zoneId);
return ymdToUnixDate(zdt.getYear(), zdt.getMonthValue(),
zdt.getDayOfMonth());
}
- //
--------------------------------------------------------------------------------------------
- // DATE_FORMAT
- //
--------------------------------------------------------------------------------------------
-
- /**
- * Format a timestamp as specific.
- *
- * @param ts the {@link TimestampData} to format.
- * @param format the string formatter.
- * @param zoneId the ZoneId.
- */
- public static String dateFormat(TimestampData ts, String format, ZoneId
zoneId) {
- DateTimeFormatter formatter = DATETIME_FORMATTER_CACHE.get(format);
- Instant instant = ts.toInstant();
- return LocalDateTime.ofInstant(instant, zoneId).format(formatter);
- }
-
- public static String dateFormat(TimestampData ts, String format) {
- return dateFormat(ts, format, ZoneId.of("UTC"));
+ public static Integer parseDate(String s) {
+ // allow timestamp str to date, e.g. 2017-12-12 09:30:00.0
+ int ws1 = s.indexOf(" ");
+ if (ws1 > 0) {
+ s = s.substring(0, ws1);
+ }
+ int hyphen1 = s.indexOf('-');
+ int y;
+ int m;
+ int d;
+ if (hyphen1 < 0) {
+ if (!isInteger(s.trim())) {
+ return null;
+ }
+ y = Integer.parseInt(s.trim());
+ m = 1;
+ d = 1;
+ } else {
+ if (!isInteger(s.substring(0, hyphen1).trim())) {
+ return null;
+ }
+ y = Integer.parseInt(s.substring(0, hyphen1).trim());
+ final int hyphen2 = s.indexOf('-', hyphen1 + 1);
+ if (hyphen2 < 0) {
+ if (!isInteger(s.substring(hyphen1 + 1).trim())) {
+ return null;
+ }
+ m = Integer.parseInt(s.substring(hyphen1 + 1).trim());
+ d = 1;
+ } else {
+ if (!isInteger(s.substring(hyphen1 + 1, hyphen2).trim())) {
+ return null;
+ }
+ m = Integer.parseInt(s.substring(hyphen1 + 1, hyphen2).trim());
+ if (!isInteger(s.substring(hyphen2 + 1).trim())) {
+ return null;
+ }
+ d = Integer.parseInt(s.substring(hyphen2 + 1).trim());
+ }
+ }
+ if (!isIllegalDate(y, m, d)) {
+ return null;
+ }
+ return ymdToUnixDate(y, m, d);
}
- public static String dateFormat(TimestampData ts, String format, TimeZone
zone) {
- return dateFormat(ts, format, zone.toZoneId());
+ public static Integer parseTime(String v) {
+ final int start = 0;
+ final int colon1 = v.indexOf(':', start);
+ // timezone hh:mm:ss[.ssssss][[+|-]hh:mm:ss]
+ // refer https://www.w3.org/TR/NOTE-datetime
+ int timezoneHour;
+ int timezoneMinute;
+ int hour;
+ int minute;
+ int second;
+ int milli;
+ int operator = -1;
+ int end = v.length();
+ int timezone = v.indexOf('-', start);
+ if (timezone < 0) {
+ timezone = v.indexOf('+', start);
+ operator = 1;
+ }
+ if (timezone < 0) {
+ timezoneHour = 0;
+ timezoneMinute = 0;
+ } else {
+ end = timezone;
+ final int colon3 = v.indexOf(':', timezone);
+ if (colon3 < 0) {
+ if (!isInteger(v.substring(timezone + 1).trim())) {
+ return null;
+ }
+ timezoneHour = Integer.parseInt(v.substring(timezone +
1).trim());
+ timezoneMinute = 0;
+ } else {
+ if (!isInteger(v.substring(timezone + 1, colon3).trim())) {
+ return null;
+ }
+ timezoneHour = Integer.parseInt(v.substring(timezone + 1,
colon3).trim());
+ if (!isInteger(v.substring(colon3 + 1).trim())) {
+ return null;
+ }
+ timezoneMinute = Integer.parseInt(v.substring(colon3 +
1).trim());
+ }
+ }
+ if (colon1 < 0) {
+ if (!isInteger(v.substring(start, end).trim())) {
+ return null;
+ }
+ hour = Integer.parseInt(v.substring(start, end).trim());
+ minute = 0;
+ second = 0;
+ milli = 0;
+ } else {
+ if (!isInteger(v.substring(start, colon1).trim())) {
+ return null;
+ }
+ hour = Integer.parseInt(v.substring(start, colon1).trim());
+ final int colon2 = v.indexOf(':', colon1 + 1);
+ if (colon2 < 0) {
+ if (!isInteger(v.substring(colon1 + 1, end).trim())) {
+ return null;
+ }
+ minute = Integer.parseInt(v.substring(colon1 + 1, end).trim());
+ second = 0;
+ milli = 0;
+ } else {
+ if (!isInteger(v.substring(colon1 + 1, colon2).trim())) {
+ return null;
+ }
+ minute = Integer.parseInt(v.substring(colon1 + 1,
colon2).trim());
+ int dot = v.indexOf('.', colon2);
+ if (dot < 0) {
+ if (!isInteger(v.substring(colon2 + 1, end).trim())) {
+ return null;
+ }
+ second = Integer.parseInt(v.substring(colon2 + 1,
end).trim());
+ milli = 0;
+ } else {
+ if (!isInteger(v.substring(colon2 + 1, dot).trim())) {
+ return null;
+ }
+ second = Integer.parseInt(v.substring(colon2 + 1,
dot).trim());
+ milli = parseFraction(v.substring(dot + 1, end).trim());
+ }
+ }
+ }
+ hour += operator * timezoneHour;
+ minute += operator * timezoneMinute;
+ return hour * (int) MILLIS_PER_HOUR
+ + minute * (int) MILLIS_PER_MINUTE
+ + second * (int) MILLIS_PER_SECOND
+ + milli;
}
/**
- * Format a timestamp as specific.
+ * Parses a fraction, multiplying the first character by {@code
multiplier}, the second
+ * character by {@code multiplier / 10}, the third character by {@code
multiplier / 100}, and so
+ * forth.
*
- * @param ts the timestamp to format.
- * @param format the string formatter.
- * @param tz the time zone
+ * <p>For example, {@code parseFraction("1234", 100)} yields {@code 123}.
*/
- public static String dateFormat(long ts, String format, TimeZone tz) {
- SimpleDateFormat formatter = FORMATTER_CACHE.get(format);
- formatter.setTimeZone(tz);
- Date dateTime = new Date(ts);
- return formatter.format(dateTime);
- }
-
+ private static int parseFraction(String v) {
+ int multiplier = 100;
+ int r = 0;
+ for (int i = 0; i < v.length(); i++) {
+ char c = v.charAt(i);
+ int x = c < '0' || c > '9' ? 0 : (c - '0');
+ r += multiplier * x;
+ if (multiplier < 10) {
+ // We're at the last digit. Check for rounding.
+ if (i + 1 < v.length() && v.charAt(i + 1) >= '5') {
+ ++r;
+ }
+ break;
+ }
+ multiplier /= 10;
+ }
+ return r;
+ }
+
+ //
--------------------------------------------------------------------------------------------
+ // Format
+ //
--------------------------------------------------------------------------------------------
+
+ public static String formatTimestamp(TimestampData ts, String format) {
+ return formatTimestamp(ts, format, ZoneId.of("UTC"));
+ }
+
+ public static String formatTimestamp(TimestampData ts, String format,
TimeZone zone) {
+ return formatTimestamp(ts, format, zone.toZoneId());
+ }
+
+ private static String formatTimestamp(TimestampData ts, int precision) {
+ LocalDateTime ldt = ts.toLocalDateTime();
+
+ String fraction = pad(9, ldt.getNano());
+ while (fraction.length() > precision && fraction.endsWith("0")) {
+ fraction = fraction.substring(0, fraction.length() - 1);
+ }
+
+ StringBuilder ymdhms =
+ ymdhms(
+ new StringBuilder(),
+ ldt.getYear(),
+ ldt.getMonthValue(),
+ ldt.getDayOfMonth(),
+ ldt.getHour(),
+ ldt.getMinute(),
+ ldt.getSecond());
+
+ if (fraction.length() > 0) {
+ ymdhms.append(".").append(fraction);
+ }
+
+ return ymdhms.toString();
+ }
+
+ public static String formatTimestamp(TimestampData ts, TimeZone tz, int
precision) {
+ return formatTimestamp(timestampWithLocalZoneToTimestamp(ts, tz),
precision);
+ }
+
+ /**
+ * Format a timestamp as specific.
Review comment:
same here, I think the javadoc is superfluous.
##########
File path:
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/functions/casting/CastRuleUtils.java
##########
@@ -233,11 +233,39 @@ public CodeWriter ifStmt(
return this;
}
+ public CodeWriter tryCatchStmt(
+ Consumer<CodeWriter> bodyWriterConsumer,
+ Class<? extends Throwable> catchClass,
Review comment:
Maybe you can have another method without `catchClass` arg which just
calls this with `Throwable.class`, as I'm guessing that's the most common usage.
##########
File path:
flink-table/flink-table-common/src/main/java/org/apache/flink/table/utils/DateTimeUtils.java
##########
@@ -412,92 +492,267 @@ public static Long toTimestamp(String dateStr, TimeZone
tz) {
* @param format date time string format
* @param tz the time zone
*/
- private static Long toTimestamp(String dateStr, String format, TimeZone
tz) {
+ private static long parseTimestampMillis(String dateStr, String format,
TimeZone tz)
+ throws ParseException {
SimpleDateFormat formatter = FORMATTER_CACHE.get(format);
formatter.setTimeZone(tz);
- try {
- return formatter.parse(dateStr).getTime();
- } catch (ParseException e) {
- return null;
- }
- }
-
- public static Long toTimestamp(String dateStr, String format) {
- return toTimestamp(dateStr, format, UTC_ZONE);
+ return formatter.parse(dateStr).getTime();
}
/**
* Parse date time string to timestamp based on the given time zone string
and format. Returns
* null if parsing failed.
*
* @param dateStr the date time string
- * @param format the date time string format
* @param tzStr the time zone id string
*/
- public static Long toTimestampTz(String dateStr, String format, String
tzStr) {
+ private static long parseTimestampTz(String dateStr, String tzStr) throws
ParseException {
TimeZone tz = TIMEZONE_CACHE.get(tzStr);
- return toTimestamp(dateStr, format, tz);
- }
-
- public static Long toTimestampTz(String dateStr, String tzStr) {
- // use "yyyy-MM-dd HH:mm:ss" as default format
- return toTimestampTz(dateStr, TIMESTAMP_FORMAT_STRING, tzStr);
+ return parseTimestampMillis(dateStr,
DateTimeUtils.TIMESTAMP_FORMAT_STRING, tz);
}
- //
--------------------------------------------------------------------------------------------
- // String --> Date conversion
- //
--------------------------------------------------------------------------------------------
-
/** Returns the epoch days since 1970-01-01. */
- public static int strToDate(String dateStr, String fromFormat) {
+ public static int parseDate(String dateStr, String fromFormat) {
// It is OK to use UTC, we just want get the epoch days
// TODO use offset, better performance
- long ts = parseToTimeMillis(dateStr, fromFormat,
TimeZone.getTimeZone("UTC"));
+ long ts = internalParseTimestampMillis(dateStr, fromFormat,
TimeZone.getTimeZone("UTC"));
ZoneId zoneId = ZoneId.of("UTC");
Instant instant = Instant.ofEpochMilli(ts);
ZonedDateTime zdt = ZonedDateTime.ofInstant(instant, zoneId);
return ymdToUnixDate(zdt.getYear(), zdt.getMonthValue(),
zdt.getDayOfMonth());
}
- //
--------------------------------------------------------------------------------------------
- // DATE_FORMAT
- //
--------------------------------------------------------------------------------------------
-
- /**
- * Format a timestamp as specific.
- *
- * @param ts the {@link TimestampData} to format.
- * @param format the string formatter.
- * @param zoneId the ZoneId.
- */
- public static String dateFormat(TimestampData ts, String format, ZoneId
zoneId) {
- DateTimeFormatter formatter = DATETIME_FORMATTER_CACHE.get(format);
- Instant instant = ts.toInstant();
- return LocalDateTime.ofInstant(instant, zoneId).format(formatter);
- }
-
- public static String dateFormat(TimestampData ts, String format) {
- return dateFormat(ts, format, ZoneId.of("UTC"));
+ public static Integer parseDate(String s) {
+ // allow timestamp str to date, e.g. 2017-12-12 09:30:00.0
+ int ws1 = s.indexOf(" ");
+ if (ws1 > 0) {
+ s = s.substring(0, ws1);
+ }
+ int hyphen1 = s.indexOf('-');
+ int y;
+ int m;
+ int d;
+ if (hyphen1 < 0) {
+ if (!isInteger(s.trim())) {
+ return null;
+ }
+ y = Integer.parseInt(s.trim());
+ m = 1;
+ d = 1;
+ } else {
+ if (!isInteger(s.substring(0, hyphen1).trim())) {
+ return null;
+ }
+ y = Integer.parseInt(s.substring(0, hyphen1).trim());
+ final int hyphen2 = s.indexOf('-', hyphen1 + 1);
+ if (hyphen2 < 0) {
+ if (!isInteger(s.substring(hyphen1 + 1).trim())) {
+ return null;
+ }
+ m = Integer.parseInt(s.substring(hyphen1 + 1).trim());
+ d = 1;
+ } else {
+ if (!isInteger(s.substring(hyphen1 + 1, hyphen2).trim())) {
+ return null;
+ }
+ m = Integer.parseInt(s.substring(hyphen1 + 1, hyphen2).trim());
+ if (!isInteger(s.substring(hyphen2 + 1).trim())) {
+ return null;
+ }
+ d = Integer.parseInt(s.substring(hyphen2 + 1).trim());
+ }
+ }
+ if (!isIllegalDate(y, m, d)) {
+ return null;
+ }
+ return ymdToUnixDate(y, m, d);
}
- public static String dateFormat(TimestampData ts, String format, TimeZone
zone) {
- return dateFormat(ts, format, zone.toZoneId());
+ public static Integer parseTime(String v) {
+ final int start = 0;
+ final int colon1 = v.indexOf(':', start);
+ // timezone hh:mm:ss[.ssssss][[+|-]hh:mm:ss]
+ // refer https://www.w3.org/TR/NOTE-datetime
+ int timezoneHour;
+ int timezoneMinute;
+ int hour;
+ int minute;
+ int second;
+ int milli;
+ int operator = -1;
+ int end = v.length();
+ int timezone = v.indexOf('-', start);
+ if (timezone < 0) {
+ timezone = v.indexOf('+', start);
+ operator = 1;
+ }
+ if (timezone < 0) {
+ timezoneHour = 0;
+ timezoneMinute = 0;
+ } else {
+ end = timezone;
+ final int colon3 = v.indexOf(':', timezone);
+ if (colon3 < 0) {
+ if (!isInteger(v.substring(timezone + 1).trim())) {
+ return null;
+ }
+ timezoneHour = Integer.parseInt(v.substring(timezone +
1).trim());
+ timezoneMinute = 0;
+ } else {
+ if (!isInteger(v.substring(timezone + 1, colon3).trim())) {
+ return null;
+ }
+ timezoneHour = Integer.parseInt(v.substring(timezone + 1,
colon3).trim());
+ if (!isInteger(v.substring(colon3 + 1).trim())) {
+ return null;
+ }
+ timezoneMinute = Integer.parseInt(v.substring(colon3 +
1).trim());
+ }
+ }
+ if (colon1 < 0) {
+ if (!isInteger(v.substring(start, end).trim())) {
+ return null;
+ }
+ hour = Integer.parseInt(v.substring(start, end).trim());
+ minute = 0;
+ second = 0;
+ milli = 0;
+ } else {
+ if (!isInteger(v.substring(start, colon1).trim())) {
+ return null;
+ }
+ hour = Integer.parseInt(v.substring(start, colon1).trim());
+ final int colon2 = v.indexOf(':', colon1 + 1);
+ if (colon2 < 0) {
+ if (!isInteger(v.substring(colon1 + 1, end).trim())) {
+ return null;
+ }
+ minute = Integer.parseInt(v.substring(colon1 + 1, end).trim());
+ second = 0;
+ milli = 0;
+ } else {
+ if (!isInteger(v.substring(colon1 + 1, colon2).trim())) {
+ return null;
+ }
+ minute = Integer.parseInt(v.substring(colon1 + 1,
colon2).trim());
+ int dot = v.indexOf('.', colon2);
+ if (dot < 0) {
+ if (!isInteger(v.substring(colon2 + 1, end).trim())) {
+ return null;
+ }
+ second = Integer.parseInt(v.substring(colon2 + 1,
end).trim());
+ milli = 0;
+ } else {
+ if (!isInteger(v.substring(colon2 + 1, dot).trim())) {
+ return null;
+ }
+ second = Integer.parseInt(v.substring(colon2 + 1,
dot).trim());
+ milli = parseFraction(v.substring(dot + 1, end).trim());
+ }
+ }
+ }
+ hour += operator * timezoneHour;
+ minute += operator * timezoneMinute;
+ return hour * (int) MILLIS_PER_HOUR
+ + minute * (int) MILLIS_PER_MINUTE
+ + second * (int) MILLIS_PER_SECOND
+ + milli;
}
/**
- * Format a timestamp as specific.
+ * Parses a fraction, multiplying the first character by {@code
multiplier}, the second
+ * character by {@code multiplier / 10}, the third character by {@code
multiplier / 100}, and so
+ * forth.
*
- * @param ts the timestamp to format.
- * @param format the string formatter.
- * @param tz the time zone
+ * <p>For example, {@code parseFraction("1234", 100)} yields {@code 123}.
*/
- public static String dateFormat(long ts, String format, TimeZone tz) {
- SimpleDateFormat formatter = FORMATTER_CACHE.get(format);
- formatter.setTimeZone(tz);
- Date dateTime = new Date(ts);
- return formatter.format(dateTime);
- }
-
+ private static int parseFraction(String v) {
+ int multiplier = 100;
+ int r = 0;
+ for (int i = 0; i < v.length(); i++) {
+ char c = v.charAt(i);
+ int x = c < '0' || c > '9' ? 0 : (c - '0');
+ r += multiplier * x;
+ if (multiplier < 10) {
+ // We're at the last digit. Check for rounding.
+ if (i + 1 < v.length() && v.charAt(i + 1) >= '5') {
+ ++r;
+ }
+ break;
+ }
+ multiplier /= 10;
+ }
+ return r;
+ }
+
+ //
--------------------------------------------------------------------------------------------
+ // Format
+ //
--------------------------------------------------------------------------------------------
+
+ public static String formatTimestamp(TimestampData ts, String format) {
+ return formatTimestamp(ts, format, ZoneId.of("UTC"));
+ }
+
+ public static String formatTimestamp(TimestampData ts, String format,
TimeZone zone) {
+ return formatTimestamp(ts, format, zone.toZoneId());
+ }
+
+ private static String formatTimestamp(TimestampData ts, int precision) {
+ LocalDateTime ldt = ts.toLocalDateTime();
+
+ String fraction = pad(9, ldt.getNano());
+ while (fraction.length() > precision && fraction.endsWith("0")) {
+ fraction = fraction.substring(0, fraction.length() - 1);
+ }
+
+ StringBuilder ymdhms =
+ ymdhms(
+ new StringBuilder(),
+ ldt.getYear(),
+ ldt.getMonthValue(),
+ ldt.getDayOfMonth(),
+ ldt.getHour(),
+ ldt.getMinute(),
+ ldt.getSecond());
+
+ if (fraction.length() > 0) {
+ ymdhms.append(".").append(fraction);
+ }
+
+ return ymdhms.toString();
+ }
+
+ public static String formatTimestamp(TimestampData ts, TimeZone tz, int
precision) {
+ return formatTimestamp(timestampWithLocalZoneToTimestamp(ts, tz),
precision);
+ }
+
+ /**
+ * Format a timestamp as specific.
+ *
+ * @param ts the {@link TimestampData} to format.
+ * @param format the string formatter.
+ * @param zoneId the ZoneId.
+ */
+ private static String formatTimestamp(TimestampData ts, String format,
ZoneId zoneId) {
+ DateTimeFormatter formatter = DATETIME_FORMATTER_CACHE.get(format);
+ Instant instant = ts.toInstant();
+ return LocalDateTime.ofInstant(instant, zoneId).format(formatter);
+ }
+
+ /**
+ * Format a timestamp as specific.
Review comment:
```suggestion
* Format a timestamp as string using a custom format.
```
Although I don't think that javadoc is usuful here, the method is quite
obvious by its name and args, and it's a private one.
##########
File path:
flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/data/vector/VectorizedColumnBatchTest.java
##########
@@ -176,8 +176,7 @@ public TimestampData getTimestamp(int colId, int precision)
{
julianDay |= (bytes[i] & (0xff));
}
long millisecond =
- (julianDay - DateTimeUtils.EPOCH_JULIAN)
- * DateTimeUtils.MILLIS_PER_DAY
+ (julianDay - 2440588) *
DateTimeUtils.MILLIS_PER_DAY
Review comment:
Why replace the constant with a literal? If you make the constant
`private` I'd suggest to extract this computation in a utility method, even if
it's only for testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
