This is an automated email from the ASF dual-hosted git repository.
nehapawar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
The following commit(s) were added to refs/heads/master by this push:
new c42af32 Change the realtime stats file to a new name (#3685)
c42af32 is described below
commit c42af32b2fae92c741a68c59ae90a1c19061887e
Author: Neha Pawar <[email protected]>
AuthorDate: Fri Jan 11 18:27:34 2019 -0800
Change the realtime stats file to a new name (#3685)
The stats files generated in realtime servers will no longer be
deserializable. This is because when serializing, the older objects were used,
the ones with "com.linkedin". After the package name change, the newer objects
will be used to deserialize, the ones with "org.apache", and the
deserialization will fail.
Changing the stats file name from "stats.ser" to "segment-stats.ser". We
will start populating stats afresh in all realtime tables.
Please note: stats from stats.ser will not be considered once the new
server is started.
---
.../data/manager/realtime/RealtimeTableDataManager.java | 4 +++-
.../realtime/impl/RealtimeSegmentStatsHistoryTest.java | 2 +-
.../data/realtime-segment-stats-history-v1.ser | Bin 822 -> 832 bytes
3 files changed, 4 insertions(+), 2 deletions(-)
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java
b/pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java
index ff8a8b8..d24902b 100644
---
a/pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java
+++
b/pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java
@@ -65,7 +65,9 @@ public class RealtimeTableDataManager extends
BaseTableDataManager {
private RealtimeSegmentStatsHistory _statsHistory;
private final Semaphore _segmentBuildSemaphore;
- private static final String STATS_FILE_NAME = "stats.ser";
+ // The old name of the stats file used to be stats.ser which we changed when
we moved all packages
+ // from com.linkedin to org.apache because of not being able to deserialize
the old files using the newer classes
+ private static final String STATS_FILE_NAME = "segment-stats.ser";
private static final String CONSUMERS_DIR = "consumers";
// Topics tend to have similar cardinality for values across partitions
consumed during the same time.
diff --git
a/pinot-core/src/test/java/org/apache/pinot/core/realtime/impl/RealtimeSegmentStatsHistoryTest.java
b/pinot-core/src/test/java/org/apache/pinot/core/realtime/impl/RealtimeSegmentStatsHistoryTest.java
index a1a3c79..3b4a418 100644
---
a/pinot-core/src/test/java/org/apache/pinot/core/realtime/impl/RealtimeSegmentStatsHistoryTest.java
+++
b/pinot-core/src/test/java/org/apache/pinot/core/realtime/impl/RealtimeSegmentStatsHistoryTest.java
@@ -222,7 +222,7 @@ public class RealtimeSegmentStatsHistoryTest {
// This test attempts to ensure that future modifications to
RealtimeSegmentStatsHistory does not prevent the software
// from reading data serialized by earlier versions. The serialized data has
one segment, with 2 columns -- "v1col1" and
// "v1col2".
- //@Test
+ @Test
public void testVersion1() throws Exception {
final String fileName = "realtime-segment-stats-history-v1.ser";
File v1StatsFile = new
File(TestUtils.getFileFromResourceUrl(RealtimeSegmentStatsHistoryTest.class.getClassLoader().getResource("data")),
fileName);
diff --git
a/pinot-core/src/test/resources/data/realtime-segment-stats-history-v1.ser
b/pinot-core/src/test/resources/data/realtime-segment-stats-history-v1.ser
index e0f09dc..06519f8 100644
Binary files
a/pinot-core/src/test/resources/data/realtime-segment-stats-history-v1.ser and
b/pinot-core/src/test/resources/data/realtime-segment-stats-history-v1.ser
differ
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
