[GitHub] [zeppelin] zjffdu commented on a change in pull request #3320: [ZEPPELIN-3977]. Create shell script for converting old note file to new file
zjffdu commented on a change in pull request #3320: [ZEPPELIN-3977]. Create shell script for converting old note file to new file URL: https://github.com/apache/zeppelin/pull/3320#discussion_r265440505 ## File path: docs/setup/operation/upgrading.md ## @@ -35,6 +35,12 @@ So, copying `notebook` and `conf` directory should be enough. ## Migration Guide +### Upgrading from Zeppelin 0.8 to 0.9 + + - From 0.9, we change the notes file name structure ([ZEPPELIN-2619](https://issues.apache.org/jira/browse/ZEPPELIN-2619)). So when you upgrading zeppelin to 0.9, you need to upgrade note file. Here's steps you need to follow: Review comment: I have added the version in note file and also update the commit message to include version. ``` commit b5ed201cab6cab13eed59cc1330b29937ae55889 (HEAD -> master) Author: Jeff Zhang Date: Thu Mar 14 15:03:58 2019 +0800 Upgrade note 'Untitled Note 1' to 0.9.0-SNAPSHOT commit f4d08fe5043eb6e905dea5ade45b6ccd2458e8c1 Author: Jeff Zhang Date: Thu Mar 14 15:03:58 2019 +0800 Upgrade note 'Basic Features (Spark)' to 0.9.0-SNAPSHOT ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] zjffdu commented on a change in pull request #3320: [ZEPPELIN-3977]. Create shell script for converting old note file to new file
zjffdu commented on a change in pull request #3320: [ZEPPELIN-3977]. Create
shell script for converting old note file to new file
URL: https://github.com/apache/zeppelin/pull/3320#discussion_r265439934
##
File path:
zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepoSync.java
##
@@ -62,59 +63,84 @@ public void init(ZeppelinConfiguration conf) throws
IOException {
oneWaySync = conf.getBoolean(ConfVars.ZEPPELIN_NOTEBOOK_ONE_WAY_SYNC);
String allStorageClassNames = conf.getNotebookStorageClass().trim();
if (allStorageClassNames.isEmpty()) {
- allStorageClassNames = defaultStorage;
+ allStorageClassNames = DEFAULT_STORAGE;
LOGGER.warn("Empty ZEPPELIN_NOTEBOOK_STORAGE conf parameter, using
default {}",
- defaultStorage);
+ DEFAULT_STORAGE);
}
String[] storageClassNames = allStorageClassNames.split(",");
if (storageClassNames.length > getMaxRepoNum()) {
LOGGER.warn("Unsupported number {} of storage classes in
ZEPPELIN_NOTEBOOK_STORAGE : {}\n" +
"first {} will be used", storageClassNames.length,
allStorageClassNames, getMaxRepoNum());
}
+// init the underlying NotebookRepo
for (int i = 0; i < Math.min(storageClassNames.length, getMaxRepoNum());
i++) {
NotebookRepo notebookRepo =
PluginManager.get().loadNotebookRepo(storageClassNames[i].trim());
if (notebookRepo != null) {
notebookRepo.init(conf);
repos.add(notebookRepo);
}
}
+
// couldn't initialize any storage, use default
if (getRepoCount() == 0) {
- LOGGER.info("No storage could be initialized, using default {} storage",
defaultStorage);
- NotebookRepo defaultNotebookRepo =
PluginManager.get().loadNotebookRepo(defaultStorage);
+ LOGGER.info("No storage could be initialized, using default {} storage",
DEFAULT_STORAGE);
+ NotebookRepo defaultNotebookRepo =
PluginManager.get().loadNotebookRepo(DEFAULT_STORAGE);
defaultNotebookRepo.init(conf);
repos.add(defaultNotebookRepo);
}
+// sync for anonymous mode on start
+if (getRepoCount() > 1 &&
conf.getBoolean(ConfVars.ZEPPELIN_ANONYMOUS_ALLOWED)) {
+ try {
+sync(AuthenticationInfo.ANONYMOUS);
+ } catch (IOException e) {
+LOGGER.error("Couldn't sync anonymous mode on start ", e);
+ }
+}
+ }
+
+ // Zeppelin change its note file name structure in 0.9.0, this is called
when upgrading
+ // from 0.9.0 before to 0.9.0 after
+ public void convertNoteFiles(ZeppelinConfiguration conf, boolean deleteOld)
throws IOException {
// convert old note file (noteId/note.json) to new note file
(note_name_note_id.zpln)
-boolean convertToNew =
conf.getBoolean(ConfVars.ZEPPELIN_NOTEBOOK_NEW_FORMAT_CONVERT);
-boolean deleteOld =
conf.getBoolean(ConfVars.ZEPPELIN_NOTEBOOK_NEW_FORMAT_DELETE_OLD);
-if (convertToNew) {
- NotebookRepo newNotebookRepo = repos.get(0);
+for (int i = 0; i < repos.size(); ++i) {
+ NotebookRepo newNotebookRepo = repos.get(i);
OldNotebookRepo oldNotebookRepo =
-
PluginManager.get().loadOldNotebookRepo(newNotebookRepo.getClass().getCanonicalName());
+
PluginManager.get().loadOldNotebookRepo(newNotebookRepo.getClass().getCanonicalName());
oldNotebookRepo.init(conf);
List oldNotesInfo =
oldNotebookRepo.list(AuthenticationInfo.ANONYMOUS);
LOGGER.info("Convert old note file to new style, note count: " +
oldNotesInfo.size());
+ LOGGER.info("Delete old note: " + deleteOld);
Review comment:
It must be old format, because OldNotebookRepo can only read old format note
file.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zeppelin] zjffdu commented on a change in pull request #3320: [ZEPPELIN-3977]. Create shell script for converting old note file to new file
zjffdu commented on a change in pull request #3320: [ZEPPELIN-3977]. Create shell script for converting old note file to new file URL: https://github.com/apache/zeppelin/pull/3320#discussion_r264999491 ## File path: docs/setup/operation/upgrading.md ## @@ -35,6 +35,12 @@ So, copying `notebook` and `conf` directory should be enough. ## Migration Guide +### Upgrading from Zeppelin 0.8 to 0.9 + + - From 0.9, we change the notes file name structure ([ZEPPELIN-2619](https://issues.apache.org/jira/browse/ZEPPELIN-2619)). So when you upgrading zeppelin to 0.9, you need to upgrade note file. Here's steps you need to follow: Review comment: ping @felixcheung This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] zjffdu commented on a change in pull request #3320: [ZEPPELIN-3977]. Create shell script for converting old note file to new file
zjffdu commented on a change in pull request #3320: [ZEPPELIN-3977]. Create shell script for converting old note file to new file URL: https://github.com/apache/zeppelin/pull/3320#discussion_r264550663 ## File path: docs/setup/operation/upgrading.md ## @@ -35,6 +35,12 @@ So, copying `notebook` and `conf` directory should be enough. ## Migration Guide +### Upgrading from Zeppelin 0.8 to 0.9 + + - From 0.9, we change the notes file name structure ([ZEPPELIN-2619](https://issues.apache.org/jira/browse/ZEPPELIN-2619)). So when you upgrading zeppelin to 0.9, you need to upgrade note file. Here's steps you need to follow: Review comment: @felixcheung I have fixed by adding commit message during note message. Here's the what I see when I upgrade the built-in zeppelin tutorial notes. But there's no commit message when deleting note. This is a bug of `GitNotebookRepo` ``` commit 7b1e12fe720fcaee916b7be7494e587d766703cd (HEAD -> master) Author: Jeff Zhang Date: Tue Mar 12 15:36:46 2019 +0800 Upgrade note 'Untitled Note 1' from old format to new format commit db9de85f65896f02f8ed112d84ca0ba89d09625b Author: Jeff Zhang Date: Tue Mar 12 15:36:46 2019 +0800 Upgrade note 'Basic Features (Spark)' from old format to new format commit cf0c44f3c6ecf17ed846ce953c4d281a9623b4b5 Author: Jeff Zhang Date: Tue Mar 12 15:36:45 2019 +0800 Upgrade note 'Using Pig for querying data' from old format to new format commit b4fb0f512b0f5862983a308c792ea1ab6bd2351f Author: Jeff Zhang Date: Tue Mar 12 15:36:45 2019 +0800 Upgrade note 'Matplotlib (Python • PySpark)' from old format to new format commit 6f1d7608adb793f5efa58b9916eb5ff36002f236 Author: Jeff Zhang Date: Tue Mar 12 15:36:45 2019 +0800 Upgrade note 'R (SparkR)' from old format to new format ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] zjffdu commented on a change in pull request #3320: [ZEPPELIN-3977]. Create shell script for converting old note file to new file
zjffdu commented on a change in pull request #3320: [ZEPPELIN-3977]. Create shell script for converting old note file to new file URL: https://github.com/apache/zeppelin/pull/3320#discussion_r263989619 ## File path: docs/setup/operation/upgrading.md ## @@ -35,6 +35,12 @@ So, copying `notebook` and `conf` directory should be enough. ## Migration Guide +### Upgrading from Zeppelin 0.8 to 0.9 + + - From 0.9, we change the notes file name structure ([ZEPPELIN-2619](https://issues.apache.org/jira/browse/ZEPPELIN-2619)). So when you upgrading zeppelin to 0.9, you need to upgrade note file. Here's steps you need to follow: Review comment: hmm, good point, it would be an issue if user is using git. Let me think about it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] zjffdu commented on a change in pull request #3320: [ZEPPELIN-3977]. Create shell script for converting old note file to new file
zjffdu commented on a change in pull request #3320: [ZEPPELIN-3977]. Create shell script for converting old note file to new file URL: https://github.com/apache/zeppelin/pull/3320#discussion_r263687139 ## File path: docs/setup/operation/upgrading.md ## @@ -35,6 +35,12 @@ So, copying `notebook` and `conf` directory should be enough. ## Migration Guide +### Upgrading from Zeppelin 0.8 to 0.9 + + - From 0.9, we change the notes file name structure ([ZEPPELIN-2619](https://issues.apache.org/jira/browse/ZEPPELIN-2619)). So when you upgrading zeppelin to 0.9, you need to upgrade note file. Here's steps you need to follow: Review comment: I am afraid not, the only way to via zeppelin version. Before 0.9.0, it is old format. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
