This is an automated email from the ASF dual-hosted git repository.
msciabarra pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/openserverless-site.git
The following commit(s) were added to refs/heads/main by this push:
new 770dba97 Feature/72-trascribe all the text (#76)
770dba97 is described below
commit 770dba97791063dd3750b78948cf0c78162194e5
Author: Hyanaru <[email protected]>
AuthorDate: Fri Aug 1 15:02:05 2025 +0200
Feature/72-trascribe all the text (#76)
* Feature/72-trascribe all the text
* Transcriptions added for Lesson1 and Lesson2
* Refctor th structure addressing d4rkstar comments
---
content/en/docs/Ai Tutorial/Lesson0/_index.md | 277 +++++++++++
content/en/docs/Ai Tutorial/Lesson1/_index.md | 421 +++++++++++++++++
content/en/docs/Ai Tutorial/Lesson2/_index.md | 656 ++++++++++++++++++++++++++
content/en/docs/Ai Tutorial/_index.md | 13 +
4 files changed, 1367 insertions(+)
diff --git a/content/en/docs/Ai Tutorial/Lesson0/_index.md b/content/en/docs/Ai
Tutorial/Lesson0/_index.md
new file mode 100644
index 00000000..0a83c28f
--- /dev/null
+++ b/content/en/docs/Ai Tutorial/Lesson0/_index.md
@@ -0,0 +1,277 @@
+---
+title: Lesson 0
+weight: 10
+draft: true
+---
+## Lesson 0
+transcript here:
+0:00
+hello everyone and welcome to the course application development with LLM open
+0:06
+using Apache open serverless a mileage title worthy of Wart Muller but it's the
+0:11
+Mastro GPT course for friends the first step is: start the environment.
+0:18
+environment that at this moment I have already started you can do it by simply
+0:23
+going here on GitHub/mastrogpt and you will find here
+0:29
+the slide shows it. it says to go to GitHub master GPT you will find the
+0:34
+starter and from here you have to press the code button and launch the code
+0:39
+space so you see go to master GPT launch the starter here is this code button
and
+0:47
+here it appears to make a button this one create code space on main I have
+0:52
+already launched it because otherwise it takes a while to start so you see that
+0:57
+this is already active it's what I'm working with it's not the only job
+1:01
+option so you don't have to you are not obliged to use the code space on the
web
+1:07
+because not everyone likes to work in the cloud like I'm doing however but it
+1:12
+is convenient than both for training and for other reasons for this reason for
+1:19
+this reason I use it and I recommend you to use it too because it saves you a
lot
+1:24
+a lot of time to configure the things but if you want you can configure yours
+1:28
+as you want so at this point once launched all this will start and above
+1:33
+all you will have to have these icons which I'm matting here at the beginning
+1:39
+the icon icon with the little cloud which actually starts this this
+1:44
+extension which has the main functions of the and the first button is the login
+1:48
+button so the first step is to log in to open serverless so the first step is
to
+1:54
+write your name your username and password and log in and then you should
+1:59
+see this message you successfully logged in you can now use open serless here
is
+2:06
+the message and all therefore first step actually this this I am paying for is
+2:12
+not the first lesson it is the pre-lessson the zero module to
+2:16
+initialize to verify that everything works and understand if everything is
+2:20
+okay in short so if everything is fine you will see the extension you will see
+2:25
+the login and you can log in and start working let's take a tour of the
+2:30
+environment so it is important in addition to the little cloud this other
+2:35
+icon the test tube who it must also show the tests therefore the tests that are
+2:42
+there however for everything so you should see by clicking on the cloud icon
+2:47
+the extension clicking on the icon of the test tube you will see the tests and
+2:53
+then there is one more thing to see this lesson here you will have to go to the
+2:57
+docs icon on the documents icon you will see the list of all the documents
under
+3:03
+lessons you can find this text which is the slide you only see the source but
+3:09
+actually just click on preview and you can see the slide real already rendering
+3:15
+and we also use the source because sometimes we take and use the sources to
+3:21
+take the exercises etc so it's convenient to have the slide in markdown
+3:26
+in source so this slide it recapitulates the various parts document test
research
+3:34
+research is useful for example to find points in the source where you need to
+3:40
+intervene to do the exercises okay then so recapping documents search testing
+3:47
+the open serverless icon and here are the buttons to perform the tests the
+3:52
+first step is to deploy then go to the deploy icon that if you don't deploy you
+3:59
+can't run the tests so now he's taking the initial code and is installing it
+4:06
+here now you can go here and run the test and therefore everything works
+4:11
+because the test passes okay now when you work you don't deploy but go into
+4:18
+development mode which also allows you to see the user interface so now let's
+4:23
+also try this other feature which is on "devel" will open here too the
deployment
+4:30
+window and now some of you will even see an open button in browser if you don't
+4:35
+see it this is a bit hidden it is explained in the slide in the slide this
+4:40
+small detail is explained which is a bit hidden there is this antenna below
this
+4:46
+antenna below simply shows the ports the ports of the services launched then
+4:52
+devel launches a web server to run the user interface so you simply have to
+4:57
+click here it opens and to see it this icon of the globe also hidden but there
+5:02
+is the slide that explains it and here you can log in then you can log in and
+5:07
+since you are master GPT reference to master Geppetto we have called the user
+5:13
+interface with great effort of imagination Pinocchio so you can log in
+5:18
+Pinocchio the default password is already Geppetto the initial password is
+5:24
+Pinocchio or Geppetto anyway we change it immediately so no problem obviously
she
+5:30
+immediately tells me that she is not sure in fact now I'll show you how to
+5:35
+change it here in part of the user interface so here I am going to give you
+5:40
+a tour then this is user interface so all the resort will use Python as a
+5:46
+programming language but we have developed a user interface that serves
+5:50
+as a front end for the applications that we will use to make the course but
then
+5:55
+you don't have to modify it because this will remain more or less fixed you
will
+6:00
+simply develop chats so this is the multi-hat application so the first chat
+6:07
+just to see if everything works is called hello and does nothing but say
+6:12
+hello to whatever you write to him then hello Michelle Mela hello Mela etc and
+6:19
+then here there is a chat a little more interesting which is instead the demo
of
+6:24
+the characteristics of Pinocchio of this interface so basically that this
+6:30
+interface can display code see you can also close this side view so when
+6:37
+something is produced that it is interesting part of the side view to
+6:41
+view it or you can view HTML or a chessboard here therefore the support
+6:49
+for so if I ask form shows me a form and I write on it because it's good we use
+6:58
+this in the exercises CEO etc and here it gives me the result of the form this
+7:05
+is fine so you understand that here there is an extendable menu this is the
+7:10
+menu and there is another thing which is the upload of documents so here you
can
+7:15
+choose a document to upload it i'll show you this later how to use it the side
+7:21
+display is customizable so you can powerful extend it with your own
+7:25
+displays and the side view that can be open and closed and with this we have
+7:31
+seen this sums up the features of our interface last step the terminal because
+7:37
+we will use a lot of terminal because many many commands are command line and
+7:43
+therefore for this reason let's start immediately to show the command line
+7:48
+commands and as a command I show you immediately the terminal opens in
+7:53
+various ways the easiest way is to go here terminal new terminal and this
+7:59
+opens a terminal okay here let's do a command right away to change the
+8:05
+password then the password is actually stored here master GPT packages master
+8:13
+GPT login users but as you can see it is encrypted so it is not easy to change
so
+8:19
+if I now decide to change the password therefore I give this command
+8:25
+ops ai user update Pinocchio and I put another password here I changed it here
+8:33
+I changed my password here the login is customizable so instead of using the
+8:39
+file you can use anything a database an external system whatever you want and
of
+8:45
+course the easy implementation is that si increase users of the interface are
+8:50
+in the file that we can change which we have changed but when we change the
+8:55
+users we also have to redeploy so second command is ops ai deploy master gpt
+9:02
+login so now he deploys an action to me this is to show that you can otherwise
+9:08
+you could have gone here do deploy and we deployed them all this serves to make
+9:14
+it clear which also exist many command line commands that are used to do the
+9:20
+most advanced things when you work a lot with the command line then for those
who
+9:24
+use code space let me give you a note here on GitHub a piece of advice go here
+9:31
+go to settings here are code space and here I recommend you to change
+9:37
+eventually the timeout because there are 120 hours free so you really do a lot
+9:43
+but it puts you the auto off time at 5 or 10 minutes maximum because otherwise
+9:49
+it consumes 30 minutes each time and if you forget it you quickly run out of
+9:53
+credit you can also decide to use and instead of Visual Studio on the web as I
+10:00
+did you can use the local one so you can change this option in reality it is
+10:05
+possible also do not use the environment in the cloud but use everything
locally
+10:12
+with docker so you can also install nuvolaris locally instead of use what we
+10:17
+offer regarding all these things for those who are interested we can do an ad
+10:22
+hoc lesson in which I show how to install in a different way however for
+10:28
+the course we have provided this environment ultra simplified which
+10:32
+should allow everyone to start with practically zero effort then course
+10:38
+support what is available first of all as a support to the nuvolaris course
+10:44
+enterprise available to everyone with a free account which as you can see
+10:48
+includes several services so it includes Redis understands Mino includes
postgres
+10:55
+SQL although in this course I don't think I'll use it you can deploy on AWS
+11:01
+G-Cloud Azure Akamai on physical servers we for example use Hezner on Ubuntu
on Red
+11:08
+Hat Open Shift so you to get an account on the our open serverless environment
+11:14
+where you can practice ask now so if you have already written account we
+11:20
+are already preparing it for you or you can go to the
+11:24
+mastrogpt.com website where there is a chatbot and you can ask for it here or
+11:29
+you can contact us on LinkedIn and there you send a message on LinkedIn or you
+11:34
+can install it yourself on the open serverless site where there is all the
+11:39
+documentation for the installation so how to install it etc as for support
+11:44
+there is Discord i ask you and I recommend you to use that so go to
+11:49
+Discord and here we are there are already the first people this is a brand
+11:54
+new server this is the second lesson so come and this is where we offer support
+12:00
+possibly use the Italian channel because the language and then there's always
+12:06
+Reddit there is also Reddit which is a forum and here you can ask your
+12:11
+questions and discuss etc so these are the course support options we offer the
+12:17
+next step and with this I conclude the first module even in reality this as
+12:22
+mentioned is lesson zero then we will do the first real lesson go still on the
+12:28
+extension then therefore here you see click on the lessons and here simply
+12:35
+click on the first lesson so here is downloading all the lessons downloaded
+12:41
+the lessons and so we are ready to start and move on to the next lesson
\ No newline at end of file
diff --git a/content/en/docs/Ai Tutorial/Lesson1/_index.md b/content/en/docs/Ai
Tutorial/Lesson1/_index.md
new file mode 100644
index 00000000..8c680395
--- /dev/null
+++ b/content/en/docs/Ai Tutorial/Lesson1/_index.md
@@ -0,0 +1,421 @@
+---
+title: Lesson 1
+weight: 10
+draft: true
+---
+## Lesson 1
+transcript here:
+0:00
+hello everyone to access the lesson after starting the environment and after
+0:06
+configuring it you must be logged in the first step is to select here the first
+0:12
+lesson which is the one that we are about to see and here all the files of
+0:17
+the lesson are downloaded if we go and see now the files you see that there is
+0:22
+a new lesson that we can open and we can do the preview of this and this is the
+0:28
+new lesson here then this is the first real lesson of the course while what you
+0:35
+have already seen was lesson zero which explains how to configure the
+0:39
+environment now let's talk about the integrated services who are supplied
+0:46
+with the system and we show you some examples which we call hello exercising
+0:53
+the various services available and these are useful both as a code example that
+0:58
+one can look at to try out the various services are useful in themselves
+1:02
+because they are used for example for debugging for interact with the cache
+1:08
+with the database and to do tests so the first step is to see these
+1:12
+examples then I'll show you the tool command line that are even more useful
+1:18
+because when you work you use them extensively and so we'll end the second
+1:23
+module then the third module is that of the exercise in which I show you step
by
+1:30
+step how implement a simple service which in this case will not use LLMs but
+1:37
+it is still used for variations with the environment and it is the exercise
that
+1:43
+we will ask you to do even with some modifications in the exercise and so we
+1:49
+complete the lesson well let's proceed downloading the lesson a hello package
+1:55
+was downloaded this package contains all the various examples so an example to
+2:02
+exercise the cache that is Redis an example to exercise the store that is
+2:08
+Milvus the database an example to exercise streaming and an example to
exercise the
+2:15
+LLM which is Ollama etc and there are also two tests trivial ones that fail
that
+2:21
+serve to try to correct them and check if the testing system works therefore
+2:28
+after downloading if you go to see the tests and eventually it may take a
+2:34
+refresh to see them if you don't see them here they are you see these are all
+2:38
+the tests we launch them and verify that all services work so now let's check
+2:44
+that everything goes the two tests deliberately fail precisely to verify
+2:49
+that everything works then check that everything works by failing some tests
+2:54
+and then correcting it okay here all the tests work except the two that we have
+3:00
+deliberately left broken so the first exercise you should do too we'll correct
+3:05
+the tests then let's go to look for the string to do that says where and we
find
+3:12
+here the wrong test the output should be hello and the string hi so I suppose
it
+3:20
+is understood that it is not this so we change the string and save it now
notice
+3:26
+this if I now perform the test happen that one passes by and is the test
+3:34
+unitary because the unitary test it only requires the code but the integration
+3:40
+test that requires deployment on the system does not pass because we have not
+3:45
+deployed it so what we have to do is go here and do the
+3:52
+deploy and at this point the integration test should also pass so we deploy
what
+3:58
+now the integration test also passes so so there is practice with the test
+4:04
+performer in general it is better put develop mode devel so the development
+4:11
+mode has the advantage that it immediately deploys the code as soon as
+4:16
+you change it and we will see this when we do the actual exercise so now let's
+4:22
+move on let's see the examples we deployed together with the test also all
+4:28
+the examples that we can go to see so when you do the deploy it also tells you
+4:33
+the URL of where the environment is in production so each of you will have a
+4:39
+different access so he deployed on the environment and you will have to use the
+4:45
+password that you have put before and that I changed that's it good now there
+4:52
+are new services stream Redis and now we see them one by one so
+4:59
+we realize what is available in the environment so first of all there is
+5:06
+obviously the LLM since this is a private AI development course we will
+5:12
+use Ollama where we deployed the llama model 3.18 billions but you will find
+5:19
+that it is very powerful in fact it is remarkable what it can do and is
+5:24
+sufficient for for the course objectives we'll actually also use a vision model
+5:29
+for image recognition and an embed model to do the ending that is used for the
ra
+5:35
+these are things that we will address during the course so if we click on
Ollama
+5:41
+here gives us a welcome message i ask him who are you here's his answer he
+5:48
+says he's an artificial intelligence i can also ask questions in Italian what
+5:54
+is the capital of Italy it is an absolutely remarkable model for
+5:59
+what it manages to do however small because it is performed on a small GPU
+6:05
+that costs quite little yet it really does so much in fact the course will
+6:11
+purely use this model although nothing prevents you from using other larger
+6:16
+ones and even connect to AI such as OpenAI Claude etc if you go and see the
+6:22
+code you will also see the code of the LLM which is of devastating simplicity
+6:27
+so this is all you need to connect moving on then the next thing to test is
+6:33
+the streamer here if I write to him hi how are you it does the streaming that
+6:41
+is it basically takes the string it sends and it tells you the ASCI
+6:46
+characters separating them 1 second from each other to demonstrate how
streaming
+6:51
+works so we also have streaming support we will see it this one too in the next
+6:58
+lessons here too you can look at the code to understand how streaming works
+7:04
+now let's see the cache redis works with commands so the logical thing was
+7:10
+to select redis and be able to talk to him so for example I can ask info and
+7:17
+gives me all the info or I can ask for the keys that are available and shows
+7:21
+them to me etc it's okay then moving on the next step is file storage then so
+7:30
+this is called S3 but this is not Amazon's S3 that's embedded in
+7:36
+Nuvolaris so this is to list the files that are in the storage i can also
+7:41
+create one plus hello equal world here you see now I can also list by substring
+7:48
+and I can also delete them or upload like this i now take a file and I upload
+7:55
+it and so if I now make star here it is it has arrived i did the upload then of
+8:02
+course once uploaded move on to the LLM to process it then last but not least
is
+8:08
+the vector database the vector database is what you need to insert the
documents
+8:15
+in our case Milvus therefore I simply write a few things for example asterisk
+8:21
+test and I look for everything that uses test obviously first I have to insert
+8:27
+some texts so this is a test test this is another test i am testing the system
+8:37
+okay now if I do the search with an asterisk test will look for me tests in
+8:43
+order of distance which is an essential concept for research vector which is
+8:48
+practically used to find in documents of the phrases relevant to what you are
+8:53
+asking so this concept of vector search is essential this allows you to test so
+9:00
+you are telling me that if I search for the word test I found some phrases of
+9:05
+which the first is the closest to all so it's a kind of search for relevance
that
+9:10
+however with LLMs is done in a particular way using a technique called
+9:15
+embedding which we will discuss in detail in the lesson relevant hour so
+9:22
+far we have seen the user interface and now let's talk about the command line
+9:26
+tools so there are many command line tools they are all documented and the
+9:32
+tool is called ops here there is explained everything he does there is
+9:38
+also the reference of all the components that there are which are available
+9:44
+therefore our divisions into tools and tasks the tools begin with the hyphen
+9:50
+and the tasks on the other hand are the things that are normally used of the
+9:55
+many available we will basically focus on IDE tasks and on AI tasks basically a
+10:03
+subcommand that has many subcomands therefore here we take an
+10:08
+example how ops works creating an action and then and
+10:14
+then removing it so practice I repeat by hand what we did with the extension
and
+10:20
+so you better understand how it works take lesson one and I perform this piece
+10:26
+of code instead of copying it I run it directly from the example now I'm going
+10:32
+to give you an introduction to the main commands of ops the first ops action
+10:37
+list is used to list the actions which are the functions the components of our
+10:43
+applications then we can create an action i have here added a simple
+10:48
+example action of a single file usually the actions we use are many when it is
a
+10:54
+single file you have to package it but this is done by the other tools when you
+10:59
+have an action of only one file you can create it on the fly you can invoke it
+11:04
+with an input you can get a URL by default the action it is so if you try
+11:11
+to use it you will get an error requires permission but you can make it public
+11:17
+then you can make an update and put a parameter on it so this made it public
+11:22
+now we get a URL now we can invoke it directly and pass parameters directly to
+11:28
+it using curl etc and finally the possible delete delete and now we see
+11:34
+that it is no longer there okay all right this is to give you an idea of how
+11:40
+it works all the other things you can simply see
+11:44
+by doing ops minus t and lists all the various action invokers URL logs and
+11:50
+then the plugins AI IDE admin configuration setup all this serves to
+11:57
+install it there is really a lot of stuff so you should go and see the
+12:02
+documentation for details normally however this is what you need and these
+12:08
+are a bit low-level because almost everything is done using IDE actions
+12:14
+instead that are the ones that are here so what does the login that you have
+12:20
+already seen so I can redo the login so it's HTTP open serverless point dev so
+12:27
+here I can redo the login instead of using this can I deploy which is the
+12:32
+same thing as pressing here i can deploy only one action so if you don't want
to
+12:38
+redeploy everything every time and you want to deploy one just write ops IDE
+12:42
+deploy hello llm the difference that there is between the deployment with
+12:48
+this tool and the creation of the action is that this manages the packaging of
+12:53
+shares and a whole host of extra things that you would otherwise have to do by
+12:57
+hand it is explained in the documentation how it is done but this
+13:02
+automates it so basically using these tools do now and do it even sooner using
+13:08
+devel that pretty much throws everything pack it up and you can write the code
+13:12
+and deploy it on the fly well last thing the clean which is useful because
there
+13:19
+are temporary files and sometimes pieces remain so it's better to do a clean
+13:26
+because it is incremental so if an old file remains then you may have some
+13:30
+problems and then the clean comes to your rescue okay so these are the
+13:36
+development tools and finally there's a specific plug-in for the course that
+13:40
+ops ai which provides a number of python specific tools so ops ai lesson is
used
+13:47
+to download the lessons of the course then ops ai user you saw it it is used
+13:53
+to add and create user then there's ops ai chat which is very nice because
+13:58
+it's a command line version of the chat you saw here so for example I can do
+14:02
+opsai chat hello llm and I can talk with AI as I did before who are you from
the
+14:10
+command line so now I'm talking to llama or I can call the store okay so
asterisk
+14:18
+that lists the files i can also upload them like this at tests mastro GPT test
+14:25
+hello here and then AI CLI this is immensely useful when you do development
+14:32
+in fact I will use it because it is a Python interpreter and finally ops ai new
+14:39
+that creates you a new service you have to give it a parameter in fact now we
+14:45
+will use it in the exercise all right so here I am summarized the things I have
+14:51
+said so then exercise now let's put together everything we have seen doing
+14:58
+an exercise therefore preview I take the
+15:06
+exercise the exercise consists of implementing a simple chat that in this
+15:11
+serves to reverse the text then so let's go here let's go to Ollama and write
how
+15:17
+do I invert the string s in Python and told us code to reverse the
+15:23
+string they're going to use this thing to invert the string then the goal is to
+15:29
+create a new service and so here we will execute this command which I can do
+15:34
+immediately because as I told you ops AI to a wizard new reverse and usually
+15:41
+makes a package that is called as the action in this case I give him the
+15:45
+package mine and I call it reverse "msciab" so I do mine lapels in my packag
+15:52
+so if I now go to the packages you see that there is "msciab" so there is a
+15:57
+Msciab package there is a reverse service that ready to use for deployment
using
+16:03
+all the conventions of the environment and there are also two tests therefore
+16:09
+you see that under msciab there is unit reverse testing and integration testing
+16:15
+so if you try to run the tests you will find out right away that the unit
+16:20
+integration test passes so now I'm going to refresh here you are if I do the
+16:26
+integration test passes but I have the unitary test no indeed it had passed
+16:33
+because had already been filed before i try to remove this problem so we send
it
+16:40
+back to the terminal and I do ops action delete msciab reverse he was not
+16:47
+supposed to pass but unfortunately he remained from the previous time then as
+16:53
+soon as you create the integration test it should fail because has not been
+16:58
+deployed therefore or deploy it everything or just that service the best
+17:05
+thing is to deploy everything deploy and then passes the test integration of a
+17:11
+service empty practically which simply it echoes what it writes so that's it
+17:17
+done the deploy let's rerun the integration test and default pass okay
+17:22
+so now we have a new service that we can use the nice thing is that we can
+17:27
+immediately add this service to the other services simply going here as the
+17:33
+slide says add it to the index so there is an index under packages mastro GPT
+17:40
+index there are two files you can create a third file or just edit this one and
+17:46
+put the new service on it then reverse msciab
+17:52
+reverse okay good and now if we go to the user interfaces in msciab we recharge
+18:01
+it pinocchio Geppetto login I realized that I had to deploy also in the
+18:09
+index so ops ai deploy mastro GPT index here this here I had to deploy
+18:19
+it here is the reverse which is trivial because whatever we say to him always
+18:25
+tells us the reverse but it is a starting point right now let's implement
+18:30
+the actual reverse and go to do the code now here I use Python programmer
+18:35
+techniques so I put the command line here i take the code of reverse then
+18:42
+packages msciab of reverse here here you see that there is always a main that
+18:48
+calls a function and the function is this so here we need to implement a I
+18:53
+will explain in the next lessons how it works the mechanism however sustainably
+18:59
+the functionals take an input and return an output so the thing I do is in
equals
+19:04
+args get input and it is better to put this one here which also gives a default
+19:09
+then I write if int equals zero because the first time he start the
conversation
+19:16
+with a message blank then please provide an input so I the answer I will return
+19:23
+without and only if inp will out equal to the inverse in
+19:31
+practice here is a very simple exercise it's interesting when you develop it
put
+19:37
+it in devel mode so as you write the code he deploys it on the fly so
+19:44
+now okay if I try to ask him now to redo reverse here you see that the first
time
+19:51
+I activate a chat he sends an empty input and he replied if I now say pipo
+19:57
+he gives it to me inverted adapting it to show you how you can edit i now give
+20:02
+it an output with an asterisk at the beginning and in the end just to show
+20:08
+you see that he immediately updated everything so if I now I try it again
+20:15
+instead he interprets it as italics if I put the double asterisk instead as
bold
+20:21
+here this is a simple exercise but it allows you to familiarize yourself with
+20:26
+the system and to understand how easy it is to create new actions to extend it
+20:32
+since we will create many of them so I recommend you do it with this I am
+20:37
+finished goodbye
\ No newline at end of file
diff --git a/content/en/docs/Ai Tutorial/Lesson2/_index.md b/content/en/docs/Ai
Tutorial/Lesson2/_index.md
new file mode 100644
index 00000000..a77b93d0
--- /dev/null
+++ b/content/en/docs/Ai Tutorial/Lesson2/_index.md
@@ -0,0 +1,656 @@
+---
+title: Lesson 2
+weight: 10
+draft: true
+---
+## Lesson 2
+transcript here:
+
+0:00
+welcome to the second lesson of the Open LLM course with Apache Open Serverless
+0:05
+for friends the course of Mastri GPT for private AI we start immediately by
going
+0:11
+to the Mastro GPT site then GitHub Mastro GPT here you will find the
+0:17
+starter as explained in the first lesson which we can launch this way you will
+0:23
+have the environment pointing to our Mastro GPT sta repository on which you
+0:28
+obviously do not have right permissions so you should fork that is create a
copy
+0:33
+of the environment and choose your account this way you can write all your
+0:38
+changes and save them without depending on the original so you can save
+0:42
+everything using the git version control tool here I have already made a fork
as
+0:47
+you can see and I have launched the code space i repeat that you can also use
the
+0:52
+local environment but simply for this course we will use the code space that
+0:57
+allows you to work quickly without too much effort and without having to
+1:01
+configure by hand then click on create code space and you will see this screen
+1:06
+you should see the speech bubble which is our extension that allows you to log
+1:10
+in using the parameters we gave you in the first lesson you can log into the
+1:15
+open serverless environment that we provide at this point you see the
+1:19
+starter files basically empty by selecting the lessons you can select the
+1:24
+files of the second lesson in this way we are downloading all the files of the
+1:29
+second lesson each lesson is independent of the others so you don't have to
have
+1:34
+the files of the first lesson to be able to use those of the second in the
second
+1:39
+lesson you will find the lesson both in markdown format and in PDF format for
+1:44
+ease of reading so that you can also consult it by downloading the local so
+1:49
+this is the second lesson and we are going to talk about how to start
+1:52
+actually talking to LLMs by creating an LLM chat in which we will implement an
+1:58
+important feature that is streaming which is the ability to see what the LLM
+2:03
+produces along the way without having to wait until everything has been
+2:06
+completely generated and this makes interaction with these systems much more
+2:11
+immediate instead of waiting until the end for everything to come we will also
+2:16
+address a series of related topics so the plan of this lesson will be first of
+2:21
+all how to access the LLM and then we will learn how to manage the secrets
+2:26
+that allow you to access it then we will embark on streaming then we will put
+2:31
+everything together by accessing the LLM in streaming mode basically having you
+2:36
+implement it with an exercise before starting since I will use a lot of a
+2:41
+trick which is to copy lines and text from a file opened in the editor
+2:45
+directly from the terminal to repeat the experience I recommend you to
configure
+2:50
+a key in my case control enter which allows you to copy the text from a file
+2:55
+to the terminal this is a feature present in studio code only that it is
+3:00
+not normally enabled by default so to enable it you have to go to settings i.e
+3:05
+the wheel in the bottom left select the keyboard shortcuts and we'll try run
+3:12
+selected text in active terminal this one you click on it and set your
+3:17
+preferred sequence for example I settl enter which allows me to copy the text
+3:23
+of the file into the terminal it is truly extremely comfortable and I
+3:28
+recommend it now let's start talking about how to access the LLM to access
+3:33
+the LLM we need the credentials the credentials we need are stored in
+3:38
+environment variables then I will explain exactly how they get there and
+3:42
+how to use them however these credentials are in the environment
+3:46
+variables Olma host which is the URL and for AUTH we use the credentials in
+3:52
+reality the credentials are the same ones that are used to log in to the
+3:58
+environment open serverless so the LLM is protected with the same credentials
+4:03
+as open serverless let's see it right away we do it by opening a terminal and
+4:09
+at the terminal we type this command ops ai CLI which launches an interactive
+4:15
+phyon environment and now I use the trick I was talking about i copy these
+4:19
+commands which are used to read the environment variables so as a first step
+4:24
+I import the O s module i tie the Ollama host variable and a UT this one you
see
+4:31
+that we will use to access Ollama host now using these credentials you can get
a
+4:37
+URL to access the llama server where we can invoke the LLMs in particular we
+4:43
+will use these commands i.e host note that here I am doing a technique which
+4:48
+is the one you have to follow in the code i.e every definitive function in
+4:52
+open serverless receives a diction called args i will always look for the
+4:57
+secret parameters in args and if I don't find them I search the environment
this
+5:02
+is a technique that is used to be able to write code that takes secrets both in
+5:07
+the test environment and in the production environment so these are the
+5:11
+three commands to give and I'm going to give them like this so let's set the
+5:15
+empty yards to simulate in the command line what is done in test i take the
+5:20
+host I take the off and I compose a URL simply by putting the https prefix on
it
+5:27
+using the key as username and password and at host our this URL that we have
+5:33
+called base is the URL we need to access to Llama password protected so if I
+5:40
+invoke beware of this exclamation mark which allows me to run while I'm in the
+5:45
+shell command editor without having to exit the shell and re-enter and I can
+5:50
+use variables so I basically run curl and in this way I invoked the URL to
+5:56
+access Llama and you can see that I got this message Llama is running now we
see
+6:01
+that the credentials are correct now let's always see at the command line how
+6:06
+to access Llama therefore we have to use a simple API that basically uses JSON
+6:13
+messages that specify the template the prompt the stream false because we start
+6:18
+immediately by disabling the streaming which is easier and we get the whole
+6:22
+complete answer so let's set the llama as a model llama 3.1.8 billions which is
+6:29
+a model made by meta small enough to fit into the GPT we have as input we ask
you
+6:35
+who are you and let's compose this message at this point using that URL the
+6:41
+base API/generate we can invoke it we invoke
+6:46
+it by executing a post therefore here now I import the request another Python
+6:53
+library which allows you to make HTTP calls and I make a post to the riquest
+6:58
+using the URL I composed and the message I composed and extract the result in
+7:03
+JSON on now from here the result will be the answer to my request who are you
so
+7:09
+this is basically the generic way you interact with Llama with its API there
+7:15
+are also libraries that allow you to avoid composing messages but generally
+7:20
+it is convenient to know how to do it and in many cases it is not needed you
+7:24
+just need when the requests become complex then I always teach the
+7:29
+essential information and then complicate it with library when it
+7:32
+becomes necessary so we can try running this code right away let's set the
model
+7:38
+first let's set the input let's compose a message let's compose the URL using
+7:43
+the base we prepared before and so now we have a full URL and a full message to
+7:49
+make the request to Llama asking him to use the llama 3.1.8b 8B template with
+7:54
+this prompt without streaming so now I do a post then we import requests we
+8:01
+make a request he performed all the result without streaming so now I get
+8:06
+this item here res which is all the complete answer that has various
+8:10
+information of this one I am only interested in extracting the answer that
+8:14
+I can print that's it like this I was able to quickly create a call and then
+8:20
+we can write our first function now I'll show you the code I made put inside
the
+8:25
+code of a function so we simply open it the code is what we saw before let's
+8:30
+take the secrets then we compose a message being careful that the input is
+8:35
+not empty and we invoke it at this point we make the request we return the
result
+8:41
+with boy and output as we have seen the previous time now we will do another
+8:46
+somewhat magical thing we'll expressly create the action from the file and pass
+8:51
+the parameters directly to it so one and two don't do that because this is to
+8:57
+understand how it works below because there is actually a feature that is
+9:01
+there in the deploy that automatically implements this feature but as you see
+9:05
+when you create a function or update so for this you can use update even if the
+9:10
+function does not exist it creates it in this package using a single file and
+9:15
+these parameters are enough for him obviously instead of doing this directly
+9:20
+we will use the deploy command which does a series of more complex actions
+9:24
+that I will now explain to you this is to understand what are the steps
+9:29
+necessary to do the deployment now this can be tried actually now since you
need
+9:34
+to deploy everything else you also need the front end and so on i now perform a
+9:40
+complete deployment and go to C here we have the simple action that allows you
+9:46
+to converse without streaming so now it takes a while with Llama so we learned
+9:52
+how to create a simple LLM without streaming at the moment but we have
+9:56
+already made some steps forward by learning all the steps necessary to
+10:01
+build our system that will become seems more complex now we have stumbled over
+10:06
+the secrets which are necessary to be able to pass the information to Llama and
+10:12
+also to the other services for this modification we would have to answer
+10:16
+some questions where do the Llama host and O variables come from in reality
+10:22
+they come from many places the main source is the configuration that is
+10:27
+loaded when you log in here comes a whole series of information that is made
+10:32
+available both to the test environment and to the deployment environment they
+10:37
+are not available in shell for this reason I use it directly from the CLI
+10:42
+which instead sees them this information can be overwritten i.e you can add
your
+10:47
+own by creating an env file or you can further create a packages env file that
+10:54
+puts additional variables but valid only for deployment and an additional env
+11:01
+which instead applies only to the if we go to check we see that there are
values
+11:06
+I try to read these two variables then I try to look for them I make grep and
you
+11:11
+see that I find llama new valeris.io IO if instead I do the GP of the test I
see
+11:17
+that different words in general these values are different because the
+11:23
+variables for the test they are used to perform tests in a test environment
+11:29
+while the default variables are used for production therefore in the newis
+11:33
+deployment environment so the CLI looks at the test ones not those of
production
+11:40
+but if someone goes to look at these variables which are available he
+11:44
+realizes that O is not in any of the env so if one tries to make this command
by
+11:50
+looking for it in this case there is no output because because the information
+11:54
+is actually in the config that we can read with the command ops config dump
+12:00
+which lists them all let's make a nice GP and we see only the ones we want to
+12:06
+look for and in fact there is off so off we have it and we can pass it this one
+12:12
+is actually generally used to access all the features of your open serverless
in
+12:17
+particular it is used to access Llama how are secrets propagated to actions the
+12:23
+way in which secrets propagate to actions is simply to pass on a command
+12:27
+line as I did before but it is quite an inconvenient thing because in practice
+12:32
+you would have to write a rather long command line each time to pass these
+12:36
+operations by the way this command can only be done from the CLI because these
+12:42
+variables are not visible from the shell then the problem is solved simply with
+12:46
+these annotations that begin with a sharp dash dash which basically allow
+12:51
+you to specify in the action the arguments to pass to the command line so
+12:57
+if I go and see the simple code you can see that above I have these three
+13:01
+annotations that are used to specify the parameters and these annotations are
+13:06
+able to read the environment variables of the configuration as well to use just
+13:11
+use ops IDE deploy all or just one function note that here I deploy a
+13:16
+function that is composed of its own file and here I have to specify the
+13:20
+extension alternatively if I have a directory I specify the directory and it
+13:25
+is zipped so in this case I do the deploy and you see that the deploy has
+13:30
+composed the action by passing all the parameters that are in the config in
+13:36
+this way from the login the secrets are prepaid to the actions in practice I
+13:41
+have revealed almost all the tricks of the secrets keep in mind that the tests
+13:45
+and CLI see the configuration on ENV and the test
+13:51
+envirment sees configuration environ that the CLI only sees test env
+14:00
+it makes sense because it is normally used to develop tests and not production
+14:05
+if you want to see what happens in production write an integration test
+14:09
+that doesn't need access to the variables as a rule to follow always use
+14:15
+secrets from the args and get a default by taking the equivalent value on the
+14:19
+environment this way you are able to have code that works both in production
+14:23
+and in testing note that you need to add the pums in the main file of an action
+14:29
+when it is a single file or in underscore main when an action is
+14:36
+multifile then deploy with ops IDE deploy and you will have all the
+14:41
+parameters passed for this reason it is never advisable to use the creation of
+14:46
+actions directly except for special cases when you have to debug here's a
+14:51
+few things about what ops IDE deploy does it's built on top of ops actions
+14:56
+and packages it currently supports Python Node and PHP we will also support
+15:03
+other create packages for the actions create zip for multifile actions
+15:09
+resolves dependencies so a requirement a packages JSON or the PHP composer it
+15:17
+extracts annotations and is used to propagate secrets and also integrates
+15:22
+with the upside devel by doing an interactive incremental deploy every
+15:26
+time you change something it only redeploys the function you changed with
+15:31
+this we have finished the discussion of secrets now let's focus on the other
+15:35
+topic of this lesson which is streaming how do you do streaming let's try
+15:40
+streaming from the command line right away let's import the libraries to
+15:44
+invoke now here I have created a URL in a quick way because I am in a test
+15:49
+environment don't do this do as I told you in general i create a message and
+15:55
+pass it to him note that this time the stream is true for this reason if I now
+16:01
+execute the result I will be given a streamed answer what is a stream in
+16:06
+Python it's an iterator so now I get an iterator from the answer this is a
+16:12
+particular object called a generator that can be executed with a four cycle
+16:17
+if I now execute the four you see that it produces not one but several answers
+16:23
+so the iterator is iterated each piece of response that the streamer is
+16:27
+returning is extracted so the requests from Llama are streamed note that the
+16:33
+format of each request has this format remember this because we will use it in
+16:38
+the exercise when you stream an Llama request he returns a sequence of Jun
+16:43
+that have the model response and the done flag until the last one says done
+16:49
+equals true and puts a series of additional information including context
+16:54
+duration and other information so here we have seen the iteration now let's
+16:59
+make a function that whole let's create for simplicity an iterator that counts
+17:04
+to zero by waiting for a second then count to zero i give it n puts a second
+17:10
+pause between one request and another this is to show the streaming that I can
+17:15
+try so if I now simply copy this code in the environment here I can test it and
+17:21
+see that it counts down to zero c is a dummy iterator that we can use to test
+17:27
+streaming and since we write in tests for everything we do then we will use a
+17:32
+socket to do the streaming the problem is this the actions when they are
+17:37
+performed are asynchronous so they are managed in a rather complex serverless
+17:42
+environment so they lose contact with the web server because the serverless
+17:46
+system is quite complex so to do the steaming that is only useful in special
+17:51
+cases there is a component called streamer that invokes the action and
+17:55
+passes it a socket to receive the intermediate results the socket is
+18:00
+passed in two parameters stream host and stream port by the way the door is
+18:06
+always changing so it is necessary so to do the streaming you will take the
+18:10
+parameters stream host and stream port you will have a socket and you will
+18:15
+connect to this socket and then along the way when you produce the output you
+18:19
+send the result to this socket this is the full streaming feature that you can
+18:24
+use with minor tweaks that depend on what you're streaming with this function
+18:28
+you receive an iterator and send it to the stream that was passed through the
+18:32
+arguments most of the time you can copy it like this i collect the result and
+18:38
+return it at the end all concatenated even though I have gradually sent the
+18:42
+intermediate steps but testing something like this is not easy that's why we
have
+18:48
+a mock something called a stream mock that's able to simulate what's happening
+18:52
+on the stream so you can write code that uses the stream and test it locally to
+18:57
+see what effect it has so that when you send it to production it's already been
+19:01
+tested so using the mock everything is prepared args is created the mock is
+19:07
+created then both executes the stream you simply need to pass the args the
+19:13
+args are located where the stream is created from the mock so the code is the
+19:18
+same both in production and in testing and then the mock stops and you see the
+19:23
+results so now I'll show you how it works in the command line amount this is
+19:29
+necessary because it is used to refund the stream because it is in the test
+19:33
+folder import a stream and I'll give you the args you see this created me a
host
+19:40
+stream is a random stream port that change every time and then using these
+19:44
+arguments I create the mock now he is practically listening something he will
+19:49
+receive okay this is normal because the mock wait 5 seconds and then and then
+19:56
+stops therefore so we have to do it quickly so now I'll show you how the
+20:02
+streamer works first of all I take this stream function and put it in the
+20:07
+command line so now I have this stream function and I can use the mock be
+20:12
+careful because the mock is a bit impatient because it waits 5 seconds and
+20:16
+then finishes this serves to prevent the texts from getting caught in fact if I
+20:21
+start and I don't do anything the mock times out and it dies because it waits 5
+20:27
+seconds at most however now I do it again because if there were no this time
+20:31
+out the mock could remain stuck indefinitely so now let's do the test we
+20:37
+have to hurry because we have 5 seconds so I now I launch the mock i take the
+20:43
+generator and launch the stream here it is it's steaming count to zero so it'll
+20:48
+be 10 seconds there there you go you're done now I can stop the mock and see
the
+20:54
+result so you see that it produced the various results 9 8 7 5 4 3 this is to
+21:03
+understand how the mock basically simulates what the streamer does in
+21:07
+production now I'll show you the actual test code and test it then let's open
+21:13
+the countdown and countdown test code this is a countdown function this is the
+21:18
+iterator this is the streaming function then here is the main which as usual
+21:23
+reads an input gives default messages if there is no inpod it does a conversion
+21:29
+of the input into a numbery and then executes take the iterator and it does
+21:34
+the streaming a very important thing is that when you write a function you have
+21:38
+to return streaming equals true in this way the first response sustainably did
+21:44
+not stream but if you return streaming equals true the client and in particular
+21:49
+Pinocchio he knows he needs to do the streaming and so he will reinvoke the
+21:54
+request to the streamer so instead of calling the normal URL it will call a
+21:59
+special URL that is used to do the streaming now let's see the test code
+22:04
+which is similar to the one before the difference is that you create the mock
+22:09
+and then run the two functions count to zero and stream in such a way as to see
+22:14
+if it works if I now go to the tests I run it you see that the test passes
+22:20
+because it is able to test a streamed code with a mock so now if I go to see
+22:24
+countdown I can now write three he does 2 1 0 we put 10 and it makes 9 8 7 it
is
+22:34
+implementing the streamed and tested feature we put everything we've learned
+22:39
+into practice and create an LLM stream we have three exercises to do for the
+22:45
+first exercise look for to-do E2 one and here you have to add the parameters to
+22:52
+access and authorize Llama so see here the point where you need to complete the
+22:56
+code and to add the secrets then the second exercise is to fix the streaming
+23:03
+because if you watch the streaming feature at the moment it is written to
+23:07
+work with the countdown and does not handle the format that instead returns
+23:12
+Llama so you have to decode the object I return extract the part one I'm
+23:18
+interested in and estimate only that part the third exercise is to modify the
+23:24
+application which will be a chat that accesses the LLM in streaming and adds a
+23:28
+model switcher that is a command that allows you to change the model from
+23:33
+llama to deepsek and there is also a correction to be made because deepseek
+23:38
+produces a think that is not readable you need to make a modification so that
+23:42
+thing is readable by writing in square brackets and instead of staples now if
+23:48
+you want to try do it otherwise I will solve the exercises in the rest of the
+23:52
+video so exercise number one let's add the secrets so I'm going here here it is
+24:00
+the solution is obviously to add perm Ollama host and then we have to add here
+24:07
+off args get Ollama host args get off here right now as exercise says the
result of
+24:17
+this change is that the test URL passes but the stream test does not so the URL
+24:24
+test before did not pass i forgot to run it it was red so much so that if we
+24:29
+deduce an error for example to see we put out one if I put this the test
+24:36
+breaks because you can't access Ollama if on the other hand it takes the right
+24:41
+parameter the test passes the streaming tests instead do not pass we have to
+24:47
+correct exercise two says to correct the streaming here it is fix streaming now
+24:54
+if you remember in this code we're accessing Ollama using the code we used
+24:58
+with an iterator what is produced at each step is no longer a simple 10 98
+25:04
+but is an object so what we have to do here is decode the object then I do deck
+25:09
+equals JSON loads which is the function that decodes a JSON object among other
+25:16
+things the streamer returns it to me like this so I have to transform it into
+25:20
+a string UTF8 decoder and I get an item that I have to extract response and by
+25:28
+default let's put error if it does not find it at this point I can return to
+25:34
+this deck and I can add to the final result this is what we have to do to
+25:41
+properly implement streaming when we access Ollama which returns to me that
+25:45
+sequence of objects that we only care about a part not everything so now the
+25:50
+streaming test should pass now I run the test stream and it passes here we have
+25:56
+corrected and we can deploy and now if we go here on the chat I say who are you
+26:02
+and here he is answering me in an exhausted way list the capitals of
+26:07
+United States here it is this is the list is giving it to me in streamed mode
+26:14
+now last finesse this is the third exercise test the model switch so now
+26:20
+simply let us to do e 2 three and let's add a if to manage the model switch so
+26:29
+if I write if in equals llama model equals llama and I change the input so
+26:36
+that it changes the model and makes it say who are you if the input is deepseek
+26:42
+the model is this and I make him say who are you so in this way I basically
added
+26:50
+the functionality of switch of the models being able to change from llama
+26:55
+to deepsek and others if we want now we deploy always deoy also that I could do
+27:01
+it incrementally with devel and let's see the result now I'm going to
+27:06
+Pinocchio again calling the chat here now I write deepsek to him it is taking
+27:12
+a while because it is carrying out the change of model and therefore it is
+27:16
+loading the model here is if I write llama here now I change to llama here
+27:23
+what about slope okay do you see this empty space this blank space is actually
+27:30
+due to the fact that it says what it thinks but you can't see it so a
+27:34
+subtlety is going to modify this code and when I reverse engineer it I do so
+27:39
+if I ask what about China deepsek this changes deepseek tell me about the
+27:46
+meaning of life universe and everything baneing of life universe epic see
that's
+27:53
+thinking sometimes it begins to think and does not come out anymore so it goes
+27:58
+into a loop however the execution it ends after a timeout that you can change
+28:04
+with the action parameters so usually it seems to me that it is 3 minutes and
+28:09
+after that he stops it when he begins to loop in his infinite thoughts and in
+28:14
+fact if he doesn't loop on a question of this kind I don't know what else can
+28:18
+send him into a loop here is his endless answer about life the universe and
+28:23
+everything else okay that's it because there is a limit that how can be changed
+28:29
+with the parameters of the duration of an execution for this reason it stopped
+28:33
+when it took too long Okay thank you for your attention and with this we finish
+28:38
+the second lesson
\ No newline at end of file
diff --git a/content/en/docs/Ai Tutorial/_index.md b/content/en/docs/Ai
Tutorial/_index.md
new file mode 100644
index 00000000..078a1b6e
--- /dev/null
+++ b/content/en/docs/Ai Tutorial/_index.md
@@ -0,0 +1,13 @@
+---
+title: Tutorial
+description: Showcase serverless development in action
+weight: 10
+draft: true
+---
+
+## AI Tutorial
+We are transcribing all the lessons done for the MastroGPT AI tutorial.
+
+The ultimate goal is to build a consistent and well-structured documentation
set based on the original course content, enabling easier access, referencing,
and further processing.
+
+The transcription process begins by leveraging the auto-generated captions
provided by Google Drive for each recorded lesson. These initial drafts are
then reviewed and refined on a per-lesson basis to ensure accuracy and clarity.
\ No newline at end of file
