[Pharo-users] Re: Backing up data

[Tim Mackinnon]

I think Voyage with its connection to Mongo is somewhat more palatable to some 
companies - however I did forget to mention Glorp of course - which does map 
directly to a SQL database. 

Not to dis Gemstone in anyway (I do wish OO db's had gained more traction in 
the past, they are still glorious) - and I do think they might have solutions 
to map to a SQL db to appease corp teams - although I'm not sure.

Again - I think its useful to understand the patterns at play and how we can 
help folks understand where/how to play.

Tim

On Sun, 18 Feb 2024, at 10:40 PM, Yanni Chiu wrote:
> Agreed that larger orgs and BI teams likely slant toward SQL. But GemStone 
> and Voyage/Mongo wouldn’t address that either. An export from a Soil, 
> GemStone or Mongo db, into a SQL db should address the BI tools.
> 
> On Sun, Feb 18, 2024 at 4:35 PM Tim Mackinnon  wrote:
>> __
>> I wasn't particularly advocating any path - but have observed that in larger 
>> orgs  its a more difficult discussion to tread a different path (rightly or 
>> wrongly) - you have to cope with BI teams, who know mainly SQL based tools 
>> and equivalently support teams who know the same - and if in that world you 
>> may have to play their game (or not - if you have traction). So I'm just 
>> observing that we can work in any of those spaces - you can take your pick 
>> and apply what makes sense in your environment.
>> 
>> Of course I love a rebel technology as much as anyone else - but sometimes 
>> their are other battles to fight - or more interesting niches to explore.
>> 
>> 
>> Tim
>> 
>> On Sun, 18 Feb 2024, at 5:06 PM, Yanni Chiu wrote:
>>> Tim,
>>> 
>>> What is the thinking behind “Finally you might need something more 
>>> enterprise and then Gemstone or Voyage…”?
>>> 
>>> Is it the maturity level of Soil codebase vs. these others? Or is it a 
>>> belief that a database has to be a complex separate piece of engineering 
>>> (therefore best outsourced).
>>> 
>>> Yanni
>>> 
>>> On Sun, Feb 18, 2024 at 11:02 AM Tim Mackinnon  wrote:
 __
 I think Ross (and what Norbert said) nicely alludes to the path people 
 follow - for really simple persistence, Fuel or simple image saving give 
 you an instant solution. The next step (assuming no real concurrency 
 issues) are what Sean has maintained - something that gives you rolling 
 snapshots and a simple UI/mechanism to recover old versions and then you 
 probably realise that you need something more transactional and Soil 
 sounds like it fits that perfectly. Finally you might need something more 
 enterprise and then Gemstone or Voyage are the direction to travel.
 
 Having so many options is terrific particularly if you can defer some of 
 the complexity the latter stages bring. I love it.
 
 Thanks for everyone for giving us so many options.
 
 Tim
>> 

[Pharo-users] Re: Backing up data

[Yanni Chiu]

Agreed that larger orgs and BI teams likely slant toward SQL. But GemStone
and Voyage/Mongo wouldn’t address that either. An export from a Soil,
GemStone or Mongo db, into a SQL db should address the BI tools.

On Sun, Feb 18, 2024 at 4:35 PM Tim Mackinnon  wrote:

> I wasn't particularly advocating any path - but have observed that in
> larger orgs  its a more difficult discussion to tread a different path
> (rightly or wrongly) - you have to cope with BI teams, who know mainly SQL
> based tools and equivalently support teams who know the same - and if in
> that world you may have to play their game (or not - if you have traction).
> So I'm just observing that we can work in any of those spaces - you can
> take your pick and apply what makes sense in your environment.
>
> Of course I love a rebel technology as much as anyone else - but sometimes
> their are other battles to fight - or more interesting niches to explore.
>
>
> Tim
>
> On Sun, 18 Feb 2024, at 5:06 PM, Yanni Chiu wrote:
>
> Tim,
>
> What is the thinking behind “Finally you might need something more
> enterprise and then Gemstone or Voyage…”?
>
> Is it the maturity level of Soil codebase vs. these others? Or is it a
> belief that a database has to be a complex separate piece of engineering
> (therefore best outsourced).
>
> Yanni
>
> On Sun, Feb 18, 2024 at 11:02 AM Tim Mackinnon  wrote:
>
>
> I think Ross (and what Norbert said) nicely alludes to the path people
> follow - for really simple persistence, Fuel or simple image saving give
> you an instant solution. The next step (assuming no real concurrency
> issues) are what Sean has maintained - something that gives you rolling
> snapshots and a simple UI/mechanism to recover old versions and then you
> probably realise that you need something more transactional and Soil sounds
> like it fits that perfectly. Finally you might need something more
> enterprise and then Gemstone or Voyage are the direction to travel.
>
> Having so many options is terrific particularly if you can defer some of
> the complexity the latter stages bring. I love it.
>
> Thanks for everyone for giving us so many options.
>
> Tim
>
>
>

[Pharo-users] Re: Backing up data

[Tim Mackinnon]

I wasn't particularly advocating any path - but have observed that in larger 
orgs  its a more difficult discussion to tread a different path (rightly or 
wrongly) - you have to cope with BI teams, who know mainly SQL based tools and 
equivalently support teams who know the same - and if in that world you may 
have to play their game (or not - if you have traction). So I'm just observing 
that we can work in any of those spaces - you can take your pick and apply what 
makes sense in your environment.

Of course I love a rebel technology as much as anyone else - but sometimes 
their are other battles to fight - or more interesting niches to explore.

Tim

On Sun, 18 Feb 2024, at 5:06 PM, Yanni Chiu wrote:
> Tim,
> 
> What is the thinking behind “Finally you might need something more enterprise 
> and then Gemstone or Voyage…”?
> 
> Is it the maturity level of Soil codebase vs. these others? Or is it a belief 
> that a database has to be a complex separate piece of engineering (therefore 
> best outsourced).
> 
> Yanni
> 
> On Sun, Feb 18, 2024 at 11:02 AM Tim Mackinnon  wrote:
>> __
>> I think Ross (and what Norbert said) nicely alludes to the path people 
>> follow - for really simple persistence, Fuel or simple image saving give you 
>> an instant solution. The next step (assuming no real concurrency issues) are 
>> what Sean has maintained - something that gives you rolling snapshots and a 
>> simple UI/mechanism to recover old versions and then you probably realise 
>> that you need something more transactional and Soil sounds like it fits that 
>> perfectly. Finally you might need something more enterprise and then 
>> Gemstone or Voyage are the direction to travel.
>> 
>> Having so many options is terrific particularly if you can defer some of the 
>> complexity the latter stages bring. I love it.
>> 
>> Thanks for everyone for giving us so many options.
>> 
>> Tim

[Pharo-users] Re: Backing up data

[Yanni Chiu]

Tim,

What is the thinking behind “Finally you might need something more
enterprise and then Gemstone or Voyage…”?

Is it the maturity level of Soil codebase vs. these others? Or is it a
belief that a database has to be a complex separate piece of engineering
(therefore best outsourced).

Yanni

On Sun, Feb 18, 2024 at 11:02 AM Tim Mackinnon  wrote:

> I think Ross (and what Norbert said) nicely alludes to the path people
> follow - for really simple persistence, Fuel or simple image saving give
> you an instant solution. The next step (assuming no real concurrency
> issues) are what Sean has maintained - something that gives you rolling
> snapshots and a simple UI/mechanism to recover old versions and then you
> probably realise that you need something more transactional and Soil sounds
> like it fits that perfectly. Finally you might need something more
> enterprise and then Gemstone or Voyage are the direction to travel.
>
> Having so many options is terrific particularly if you can defer some of
> the complexity the latter stages bring. I love it.
>
> Thanks for everyone for giving us so many options.
>
> Tim
>

[Pharo-users] Re: Backing up data

[Tim Mackinnon]

I think Ross (and what Norbert said) nicely alludes to the path people follow - 
for really simple persistence, Fuel or simple image saving give you an instant 
solution. The next step (assuming no real concurrency issues) are what Sean has 
maintained - something that gives you rolling snapshots and a simple 
UI/mechanism to recover old versions and then you probably realise that you 
need something more transactional and Soil sounds like it fits that perfectly. 
Finally you might need something more enterprise and then Gemstone or Voyage 
are the direction to travel.

Having so many options is terrific particularly if you can defer some of the 
complexity the latter stages bring. I love it.

Thanks for everyone for giving us so many options.

Tim

On Sat, 17 Feb 2024, at 6:46 PM, Russ Whaley wrote:
> Norbert,
> I'm very interested to investigate Soil.  Do you have examples on how to use 
> Soil? 
> 
> I currently use a STON persistence model. My current approach to my model is 
> structured specifically to make STON full save and restores simple...
>  
> - Wrapper Class which contains a collection of high level objects
>- high level objects contain collections of next level objects
>   - next level objects contain collections of lower level objects
>  - etc.
> 
> While the database size (7.3Mb) is not terribly large, I am writing about 50 
> of those per day as each save simply saves off the entire old STON and writes 
> a new one. Concurrency is generally not a problem as we have the application 
> automatically save itself every 10 minutes, but every once in a while we do 
> run up against a conflict that would be best handled by a concurrent capable 
> model.  
> 
> I have roughly nine applications using the same class models, so it would be 
> very helpful to adopt a more OO database approach to persistence... even 
> though we're nowhere near Google :) 
> 
> Thoughts on where to begin?  I'm on the Discord server as well, if you'd 
> prefer to discuss there.
> 
> Thanks,
> Russ 
> 
> 
> On Thu, Feb 15, 2024 at 4:49 AM Norbert Hartl  wrote:
>> The approach as I understand is simple and valuable until you face 
>> concurrency. Nothing in SimplePersistence prevents your data from becoming 
>> corrupt in a concurrent situation. I agree that when you start a project you 
>> almost never face concurrency because the odds are way too low to have two 
>> things at the same time happening. This is because computers are really fast 
>> today and the amount of activity on a service is largely overestimated.
>> 
>> But there is still a huge gap between SimplePersistence and those overkill 
>> persistence solutions. For exactly that reason I've done Soil 
>> (https://github.com/ApptiveGrid/Soil) to fill that gap. It is one project 
>> with no dependency and setting it up is a no-brainer. It is way bigger than 
>> SimplePersistence but it deals with concurrency and adds query capabilties 
>> for your objects. But you have to use transactions to do this. 
>> 
>> I started ApptiveGrid just by having the data persisted through saving the 
>> image. Next was to STON out the model. Then versioned STON files to add 
>> fallback. Then there was Soil (well, first voyage and then omnibase and then 
>> Soil) which is the next level to iterate. 
>> 
>> So ApptiveGrid is not the next google but it also wouldn't work with 
>> SimplePersistence. So good that we have enough tools to improve our projects 
>> on.
>> 
>> Norbert
>> 
>> 
>>> Am 15.02.2024 um 06:43 schrieb s...@clipperadams.com:
>>> 
>>> I’m happy to answer any questions about Simple Persistence. It is a nice 
>>> framework around (potentially any) serializer. It’s meant to be pluggable 
>>> but currently uses Fuel out of the box. You just tell it what classes to 
>>> persist and then create two methods per class to handle 
>>> materialization/serialization.
>>> 
>>> For the record, Ramon Leon created it in a blog post. I just ported it to 
>>> Pharo and have been maintaining it.
>>> 
>>> The stated reason he created it is that most solutions are complete 
>>> overkill for 99% of projects that will never “become the next google”. It’s 
>>> pretty basic, but at least two steps more sophisticated than simply saving 
>>> the image because it keeps old versions and is easily reloadable into 
>>> another image.
>>> 
>>> That said, I’ve been using it almost exclusively for my personal projects 
>>> and have yet to grow out of it.
>>> 
>> 
> 
> 
> --
> Russ Whaley
> whaley.r...@gmail.com

[Pharo-users] Re: Backing up data

[Norbert Hartl]

Hi Russ,stay tuned, we are preparing a 0.1 release for Soil before or at the time of pharo12 release. With 0.1 we will document how to use it. If you use STON it will be similar because if you do db := Soil path: ‚mydb‘.db initializeFilesystem.txn := db newTransaction.txn root: yourModelRoot.txn commit.it will save all of the 7,3 MB in the database. Every time you do txn := db newTransaction.txn rootit will read all of the model you have just write now but concurrent safe.Optimizing this is partitioning your graph and saying which of your objects are standalaone by sayingtxn makeRoot: anObject.then only the needed parts of your model are loaded. But in order to save it you have to mark the objects you made root dirty by sayingtxn markDirty: anObjectThat‘s all for basic usage. For the rest you need to wait for the documentation.NorbertAm 17.02.2024 um 19:47 schrieb Russ Whaley :Norbert,I'm very interested to investigate Soil.  Do you have examples on how to use Soil? I currently use a STON persistence model. My current approach to my model is structured specifically to make STON full save and restores simple... - Wrapper Class which contains a collection of high level objects   - high level objects contain collections of next level objects      - next level objects contain collections of lower level objects         - etc.While the database size (7.3Mb) is not terribly large, I am writing about 50 of those per day as each save simply saves off the entire old STON and writes a new one. Concurrency is generally not a problem as we have the application automatically save itself every 10 minutes, but every once in a while we do run up against a conflict that would be best handled by a concurrent capable model.  I have roughly nine applications using the same class models, so it would be very helpful to adopt a more OO database approach to persistence... even though we're nowhere near Google :) Thoughts on where to begin?  I'm on the Discord server as well, if you'd prefer to discuss there.Thanks,Russ On Thu, Feb 15, 2024 at 4:49 AM Norbert Hartl  wrote:The approach as I understand is simple and valuable until you face concurrency. Nothing in SimplePersistence prevents your data from becoming corrupt in a concurrent situation. I agree that when you start a project you almost never face concurrency because the odds are way too low to have two things at the same time happening. This is because computers are really fast today and the amount of activity on a service is largely overestimated.But there is still a huge gap between SimplePersistence and those overkill persistence solutions. For exactly that reason I've done Soil (https://github.com/ApptiveGrid/Soil) to fill that gap. It is one project with no dependency and setting it up is a no-brainer. It is way bigger than SimplePersistence but it deals with concurrency and adds query capabilties for your objects. But you have to use transactions to do this. I started ApptiveGrid just by having the data persisted through saving the image. Next was to STON out the model. Then versioned STON files to add fallback. Then there was Soil (well, first voyage and then omnibase and then Soil) which is the next level to iterate. So ApptiveGrid is not the next google but it also wouldn't work with SimplePersistence. So good that we have enough tools to improve our projects on.NorbertAm 15.02.2024 um 06:43 schrieb s...@clipperadams.com:I’m happy to answer any questions about Simple Persistence. It is a nice framework around (potentially any) serializer. It’s meant to be pluggable but currently uses Fuel out of the box. You just tell it what classes to persist and then create two methods per class to handle materialization/serialization. For the record, Ramon Leon created it in a blog post. I just ported it to Pharo and have been maintaining it.The stated reason he created it is that most solutions are complete overkill for 99% of projects that will never “become the next google”. It’s pretty basic, but at least two steps more sophisticated than simply saving the image because it keeps old versions and is easily reloadable into another image.That said, I’ve been using it almost exclusively for my personal projects and have yet to grow out of it.

-- Russ Whaleywhaley.r...@gmail.com

[Pharo-users] Re: Backing up data

[Russ Whaley]

Norbert,
I'm very interested to investigate Soil.  Do you have examples on how to
use Soil?

I currently use a STON persistence model. My current approach to my model
is structured specifically to make STON full save and restores simple...

- Wrapper Class which contains a collection of high level objects
   - high level objects contain collections of next level objects
  - next level objects contain collections of lower level objects
 - etc.

While the database size (7.3Mb) is not terribly large, I am writing about
50 of those per day as each save simply saves off the entire old STON and
writes a new one. Concurrency is generally not a problem as we have the
application automatically save itself every 10 minutes, but every once in a
while we do run up against a conflict that would be best handled by a
concurrent capable model.

I have roughly nine applications using the same class models, so it would
be very helpful to adopt a more OO database approach to persistence... even
though we're nowhere near Google :)

Thoughts on where to begin?  I'm on the Discord server as well, if you'd
prefer to discuss there.

Thanks,
Russ


On Thu, Feb 15, 2024 at 4:49 AM Norbert Hartl  wrote:

> The approach as I understand is simple and valuable until you face
> concurrency. Nothing in SimplePersistence prevents your data from becoming
> corrupt in a concurrent situation. I agree that when you start a project
> you almost never face concurrency because the odds are way too low to have
> two things at the same time happening. This is because computers are really
> fast today and the amount of activity on a service is largely overestimated.
>
> But there is still a huge gap between SimplePersistence and those overkill
> persistence solutions. For exactly that reason I've done Soil (
> https://github.com/ApptiveGrid/Soil) to fill that gap. It is one project
> with no dependency and setting it up is a no-brainer. It is way bigger than
> SimplePersistence but it deals with concurrency and adds query capabilties
> for your objects. But you have to use transactions to do this.
>
> I started ApptiveGrid just by having the data persisted through saving the
> image. Next was to STON out the model. Then versioned STON files to add
> fallback. Then there was Soil (well, first voyage and then omnibase and
> then Soil) which is the next level to iterate.
>
> So ApptiveGrid is not the next google but it also wouldn't work with
> SimplePersistence. So good that we have enough tools to improve our
> projects on.
>
> Norbert
>
>
> Am 15.02.2024 um 06:43 schrieb s...@clipperadams.com:
>
> I’m happy to answer any questions about Simple Persistence. It is a nice
> framework around (potentially any) serializer. It’s meant to be pluggable
> but currently uses Fuel out of the box. You just tell it what classes to
> persist and then create two methods per class to handle
> materialization/serialization.
>
> For the record, Ramon Leon created it in a blog post. I just ported it to
> Pharo and have been maintaining it.
>
> The stated reason he created it is that most solutions are complete
> overkill for 99% of projects that will never “become the next google”. It’s
> pretty basic, but at least two steps more sophisticated than simply saving
> the image because it keeps old versions and is easily reloadable into
> another image.
>
> That said, I’ve been using it almost exclusively for my personal projects
> and have yet to grow out of it.
>
>
>

-- 
Russ Whaley
whaley.r...@gmail.com

[Pharo-users] Re: Backing up data

[Norbert Hartl]

The approach as I understand is simple and valuable until you face concurrency. 
Nothing in SimplePersistence prevents your data from becoming corrupt in a 
concurrent situation. I agree that when you start a project you almost never 
face concurrency because the odds are way too low to have two things at the 
same time happening. This is because computers are really fast today and the 
amount of activity on a service is largely overestimated.

But there is still a huge gap between SimplePersistence and those overkill 
persistence solutions. For exactly that reason I've done Soil 
(https://github.com/ApptiveGrid/Soil) to fill that gap. It is one project with 
no dependency and setting it up is a no-brainer. It is way bigger than 
SimplePersistence but it deals with concurrency and adds query capabilties for 
your objects. But you have to use transactions to do this. 

I started ApptiveGrid just by having the data persisted through saving the 
image. Next was to STON out the model. Then versioned STON files to add 
fallback. Then there was Soil (well, first voyage and then omnibase and then 
Soil) which is the next level to iterate. 

So ApptiveGrid is not the next google but it also wouldn't work with 
SimplePersistence. So good that we have enough tools to improve our projects on.

Norbert


> Am 15.02.2024 um 06:43 schrieb s...@clipperadams.com:
> 
> I’m happy to answer any questions about Simple Persistence. It is a nice 
> framework around (potentially any) serializer. It’s meant to be pluggable but 
> currently uses Fuel out of the box. You just tell it what classes to persist 
> and then create two methods per class to handle materialization/serialization.
> 
> For the record, Ramon Leon created it in a blog post. I just ported it to 
> Pharo and have been maintaining it.
> 
> The stated reason he created it is that most solutions are complete overkill 
> for 99% of projects that will never “become the next google”. It’s pretty 
> basic, but at least two steps more sophisticated than simply saving the image 
> because it keeps old versions and is easily reloadable into another image.
> 
> That said, I’ve been using it almost exclusively for my personal projects and 
> have yet to grow out of it.
> 

[Pharo-users] Re: Backing up data

[sean]

I’m happy to answer any questions about Simple Persistence. It is a nice 
framework around (potentially any) serializer. It’s meant to be pluggable but 
currently uses Fuel out of the box. You just tell it what classes to persist 
and then create two methods per class to handle materialization/serialization. 

For the record, Ramon Leon created it in a blog post. I just ported it to Pharo 
and have been maintaining it.

The stated reason he created it is that most solutions are complete overkill 
for 99% of projects that will never “become the next google”. It’s pretty 
basic, but at least two steps more sophisticated than simply saving the image 
because it keeps old versions and is easily reloadable into another image.

That said, I’ve been using it almost exclusively for my personal projects and 
have yet to grow out of it.

[Pharo-users] Re: Backing up data

[sergio ruiz]

Let me check it out.

It’s not too late. This is a fun “scratch my own itch” project. 

So there are no deadlines looming or anything..



> On Jan 31, 2024, at 9:20 AM, Tim Mackinnon  wrote:
> 
> Sergio - might be too late now, but I haven't seen anyone mention Sean's 
> solution - https://github.com/seandenigris/Simple-Persistence  (I think it 
> uses Fuel under the hood, but its a nice simple solution)
> 


peace,
sergio
photographer, journalist, visionary

Public Key: 
https://pgp.key-server.io/pks/lookup?op=get=0x69B08F58923AB3A2
#BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV
@sergio_101@mastodon.social
https://sergio101.com
http://www.codeandmusic.com
http://www.twitter.com/sergio_101
http://www.facebook.com/sergio101

[Pharo-users] Re: Backing up data

[Tim Mackinnon]

Sergio - might be too late now, but I haven't seen anyone mention Sean's 
solution - https://github.com/seandenigris/Simple-Persistence  (I think it uses 
Fuel under the hood, but its a nice simple solution)

Tim

On Thu, 18 Jan 2024, at 4:48 AM, sergio ruiz wrote:
> I have been in the relational database world for decades. One of the things 
> that is super simple is backing up and restoring data.
> 
> Sometimes, it makes it makes sense to pull the production data into 
> development. In the case of a catastrophe, it’s super simple to restore data 
> from a backup.
> 
> I am thinking of backup strategies for my data once again, and am wondering 
> if things have changed since I was last using smalltalk.
> 
> I am thinking of the following:
> 
> STON
> I could dump all of my data into STON, but there is one slight twist. In this 
> project, it’s just making a list of movie listings. There are three basic 
> classes: Theater, Movie, Event. Each of these classes stores an 
> OrderedCollection of items of that type on a class variable, as I’ll need to 
> access each on their own.. so, if an event has a theater, I’ll need to access 
> Theaters on their own, and Movies on their own.
> 
> So, I would have to rebuild the list of Theaters from the Theaters in each 
> event. Otherwise, I would lose the link between the Theaters on the Theater 
> class variable and the Theater in the Event instance.
> 
> Am I thinking about this incorrectly?
> 
> Gemstones
> I am totally into the idea of using gemstones. Is there a backup and restore 
> for objects here? do the relationships remain?
> 
> Voyage
> This seems like a good solution, but I would rather stay in the smalltalk 
> object store.
> 
> Ideas?
> 
> Thanks!
> 
> 
> peace,
> sergio
> photographer, journalist, visionary
> 
> Public Key: 
> https://pgp.key-server.io/pks/lookup?op=get=0x69B08F58923AB3A2
> #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV
> @sergio_101@mastodon.social
> https://sergio101.com
> http://www.codeandmusic.com
> http://www.twitter.com/sergio_101
> http://www.facebook.com/sergio101

[Pharo-users] Re: Backing up data

[Stewart MacLean]

Hi Sergio,

I have used Fuel to good effect to store a tree of objects (with all
interlinkages).

Saving:

saveTo: aFileOrName

| aFileName |

aFileName := aFileOrName fullName.
FLSerializer serialize: self toFileNamed: aFileName


Loading:

loadFrom: aFileOrFileName

| new |

new := FLMaterializer materializeFromFileNamed: aFileOrFileName fullName.

new ifNil: [^nil]. "becomeForward fails for nil!"

self becomeForward: new

Just make sure you don't have any references to objects outside of your
model tree - you'll disappear down a black hole as it tries to save the
whole image!

Cheers,

Stew

On Thu, Jan 18, 2024 at 5:48 PM sergio ruiz  wrote:

> I have been in the relational database world for decades. One of the
> things that is super simple is backing up and restoring data.
>
> Sometimes, it makes it makes sense to pull the production data into
> development. In the case of a catastrophe, it’s super simple to restore
> data from a backup.
>
> I am thinking of backup strategies for my data once again, and am
> wondering if things have changed since I was last using smalltalk.
>
> I am thinking of the following:
>
> STON
> I could dump all of my data into STON, but there is one slight twist. In
> this project, it’s just making a list of movie listings. There are three
> basic classes: Theater, Movie, Event. Each of these classes stores an
> OrderedCollection of items of that type on a class variable, as I’ll need
> to access each on their own.. so, if an event has a theater, I’ll need to
> access Theaters on their own, and Movies on their own.
>
> So, I would have to rebuild the list of Theaters from the Theaters in each
> event. Otherwise, I would lose the link between the Theaters on the Theater
> class variable and the Theater in the Event instance.
>
> Am I thinking about this incorrectly?
>
> Gemstones
> I am totally into the idea of using gemstones. Is there a backup and
> restore for objects here? do the relationships remain?
>
> Voyage
> This seems like a good solution, but I would rather stay in the smalltalk
> object store.
>
> Ideas?
>
> Thanks!
>
> 
> peace,
> sergio
> photographer, journalist, visionary
>
> Public Key:
> https://pgp.key-server.io/pks/lookup?op=get=0x69B08F58923AB3A2
> #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV
> @sergio_101@mastodon.social
> https://sergio101.com
> http://www.codeandmusic.com
> http://www.twitter.com/sergio_101
> http://www.facebook.com/sergio101
>
>

[Pharo-users] Re: Backing up data

[Yanni Chiu]

Hi Sergio,

I added a Movies demo, using a Soil database. There was a previous
Accounting demo that had two versions (in memory database and Soil
database) from which I based the new demo. I like the Movie demo because it
is simple yet complex enough. I suspect the Accounting demo is too
complicated to grasp as a first encounter. I’m going to use the Movie demo
for a presentation, so it’ll continue to be tweaked. Mainly, I’ll be adding
a way to use SoilTransaction with #commit, instead of
#checkpointAndContinue.

The code is at:

[image: Seaside-Quasar.png]

yannij/Seaside-Quasar: Pharo/Smalltalk code to use Quasar Framework in
Seaside. Also, some demo apps using Magritte are provided.

github.com



Cheers,
Yanni


On Thu, Jan 18, 2024 at 11:11 AM sergio ruiz  wrote:

>
> […]
>
> In the end, I’ll end up with:
>
> - an application of some sort that scratches my own itch
> - a pile of articles outlining how i designed the tool set for the
> application.
>
> and in reality, it’s gonna be thinly veiled Smalltalk evangelism.
>

[Pharo-users] Re: Backing up data

[Offray Vladimir Luna Cárdenas]

Sergio,

I would go with STON, as advised by Norbert (I don't have experience 
with Soil). One of the nice things of STON is that you can use whatever 
DVCS you like to versionate your data. For example, we locally use 
Fossil, while most people prefer Git.


STON is super versatile! As an example, I used it with Grafoscopio, my 
Pharo powered interactive outliner/documenter, to store notebooks with 
Markdown stored inside STON, as you can see in [1]. And now I flipped 
that idea and I'm using STON to embed notebooks metadata inside 
Markdeep. In this way anyone can read notebooks directly in the web, 
without specialized software. [1a] is just the same document in [1] but 
with this flipped format idea (if you,re even more curious, look at the 
source code of the page at [1a] and you  will find that the STON 
representation of notebook cell/node medatada is pretty light and readable).


[1] https://mutabit.com/repos.fossil/indieweb/file?name=indieweb.ston=tip
[1a] 
https://mutabit.com/repos.fossil/indieweb/doc/tip/indie-web--25zqu.md.html


This is just an example of how STON allowed us to port lessons learned 
from Grafoscopio, made in 2016, to Lepiter, released in 2021, ensuring 
longevity of our community memory and data. Hopefully you will find STON 
pretty empowering in your own experiments.


Cheers,

Offray

On 18/01/24 9:37, Norbert Hartl wrote:




Am 18.01.2024 um 14:54 schrieb sergio ruiz :

Using this Will my links remain viable using this method?

Meaning that an event has a theater. That theater is also in the 
OrderedCollection of Theaters.


If the name of he theater changed in the Theaters collection, will 
that name change occur in the event theater object?


After bringing these collections back in via STON, will all of these 
relationships remain intact?


If with relationship you mean that one object event has a reference to 
a theater object then yes. STON serializes the graph as objects and 
all references will be reestablished on materialization.


Norbert





On Jan 18, 2024, at 8:26 AM, Norbert Hartl  wrote:

If you have three classes with class side collections you can have a 
make e.g. a dictionary and put the collections from the class side 
into it like


model := {
 #theaters -> Theater instances.
 #movies -> Movie instances.
 #event -> Event instances.
} asDictionary.





peace,
sergio
photographer, journalist, visionary

Public Key: 
https://pgp.key-server.io/pks/lookup?op=get=0x69B08F58923AB3A2

#BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV
@sergio_101@mastodon.social
https://sergio101.com
http://www.codeandmusic.com
http://www.twitter.com/sergio_101
http://www.facebook.com/sergio101

[Pharo-users] Re: Backing up data

[sergio ruiz]

The idea is, I have been a software engineer for the past three decades.

I REALLY love my profession (even though now, it’s pretty much sitting in 
meetings all day), but I love it.

I have a few projects I always wanted to do, because I just want to do them 
because they are useful to me.

I have architected and coded hundreds of giant projects. At this point, 
building anything with the big web frameworks just seems exhausting.

I have decided to bring each of these pet projects to fruition and then 
document how and why i made each decision.

Every project will be using smalltalk as the data store. If the project is a 
website, I will use seaside as the web framework. Otherwise, I’ll use the 
teapot as an API interface.

The series of articles I will write will be aimed at people in my position. 
People who are familiar with software development, and would like to read about 
it at a higher than tutorial level.

In the end, I’ll end up with:

- an application of some sort that scratches my own itch
- a pile of articles outlining how i designed the tool set for the application.

and in reality, it’s gonna be thinly veiled Smalltalk evangelism.

But i think it might be worthwhile to talk about this stuff in a pubilc 
platform.



> On Jan 18, 2024, at 10:34 AM, Yanni Chiu  wrote:
> 
> Can you sketch out the complexity aspects of your five projects? Is this all 
> for learning, or are they “production” applications? Do you want to gain 
> experience with different ways to persist data, or are you just thinking that 
> you have to use different ways due to project complexity? It might be 


peace,
sergio
photographer, journalist, visionary

Public Key: 
https://pgp.key-server.io/pks/lookup?op=get=0x69B08F58923AB3A2
#BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV
@sergio_101@mastodon.social
https://sergio101.com
http://www.codeandmusic.com
http://www.twitter.com/sergio_101
http://www.facebook.com/sergio101

[Pharo-users] Re: Backing up data

[Yanni Chiu]

Can you sketch out the complexity aspects of your five projects? Is this
all for learning, or are they “production” applications? Do you want to
gain experience with different ways to persist data, or are you just
thinking that you have to use different ways due to project complexity? It
might be that Soil database could handle all your use cases, so you could
avoid working through a bunch of databases, unless you want to.

On Thu, Jan 18, 2024 at 8:56 AM sergio ruiz  wrote:

> I figured Gemstones would be SUPER heavy handed for this application, but
> I have a list of five projects, each increasing in complexity that I want
> to launch and document. So, sooner or later, I will need a more robust
> solution.
>
> Thanks!
>
>
>
> On Jan 18, 2024, at 8:26 AM, Norbert Hartl  wrote:
>
> GemStone is a huge thing to tame. I would not say the above and GemStone
> are alternatives. GemStone is a lot to install and learn but you get
> something that makes it hard to corrupt your data. So you have requirements
> that make it fit into GemStone or you have requirements that make it a fit
> for STON. Between both there is no overlap but rather a huge gap between
> the applicability of each.
>
>
> 
> peace,
> sergio
> photographer, journalist, visionary
>
> Public Key:
> https://pgp.key-server.io/pks/lookup?op=get=0x69B08F58923AB3A2
> #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV
> @sergio_101@mastodon.social
> https://sergio101.com
> http://www.codeandmusic.com
> http://www.twitter.com/sergio_101
> http://www.facebook.com/sergio101
>
>

[Pharo-users] Re: Backing up data

[Norbert Hartl]

> Am 18.01.2024 um 14:54 schrieb sergio ruiz :
> 
> Using this Will my links remain viable using this method?
> 
> Meaning that an event has a theater. That theater is also in the 
> OrderedCollection of Theaters.
> 
> If the name of he theater changed in the Theaters collection, will that name 
> change occur in the event theater object?
> 
> After bringing these collections back in via STON, will all of these 
> relationships remain intact?

If with relationship you mean that one object event has a reference to a 
theater object then yes. STON serializes the graph as objects and all 
references will be reestablished on materialization.

Norbert
> 
> 
> 
>> On Jan 18, 2024, at 8:26 AM, Norbert Hartl  wrote:
>> 
>> If you have three classes with class side collections you can have a make 
>> e.g. a dictionary and put the collections from the class side into it like 
>> 
>> model := {
>>#theaters -> Theater instances.
>>#movies -> Movie instances.
>>#event -> Event instances.
>> } asDictionary.
>> 
> 
> 
> 
> 
> peace,
> sergio
> photographer, journalist, visionary
> 
> Public Key: 
> https://pgp.key-server.io/pks/lookup?op=get=0x69B08F58923AB3A2
> #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV
> @sergio_101@mastodon.social
> https://sergio101.com
> http://www.codeandmusic.com
> http://www.twitter.com/sergio_101
> http://www.facebook.com/sergio101
> 

[Pharo-users] Re: Backing up data

[sergio ruiz]

I figured Gemstones would be SUPER heavy handed for this application, but I 
have a list of five projects, each increasing in complexity that I want to 
launch and document. So, sooner or later, I will need a more robust solution.

Thanks!



> On Jan 18, 2024, at 8:26 AM, Norbert Hartl  wrote:
> 
> GemStone is a huge thing to tame. I would not say the above and GemStone are 
> alternatives. GemStone is a lot to install and learn but you get something 
> that makes it hard to corrupt your data. So you have requirements that make 
> it fit into GemStone or you have requirements that make it a fit for STON. 
> Between both there is no overlap but rather a huge gap between the 
> applicability of each.


peace,
sergio
photographer, journalist, visionary

Public Key: 
https://pgp.key-server.io/pks/lookup?op=get=0x69B08F58923AB3A2
#BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV
@sergio_101@mastodon.social
https://sergio101.com
http://www.codeandmusic.com
http://www.twitter.com/sergio_101
http://www.facebook.com/sergio101

[Pharo-users] Re: Backing up data

[sergio ruiz]

Using this Will my links remain viable using this method?

Meaning that an event has a theater. That theater is also in the 
OrderedCollection of Theaters.

If the name of he theater changed in the Theaters collection, will that name 
change occur in the event theater object?

After bringing these collections back in via STON, will all of these 
relationships remain intact?



> On Jan 18, 2024, at 8:26 AM, Norbert Hartl  wrote:
> 
> If you have three classes with class side collections you can have a make 
> e.g. a dictionary and put the collections from the class side into it like 
> 
> model := {
>#theaters -> Theater instances.
>#movies -> Movie instances.
>#event -> Event instances.
> } asDictionary.
> 




peace,
sergio
photographer, journalist, visionary

Public Key: 
https://pgp.key-server.io/pks/lookup?op=get=0x69B08F58923AB3A2
#BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV
@sergio_101@mastodon.social
https://sergio101.com
http://www.codeandmusic.com
http://www.twitter.com/sergio_101
http://www.facebook.com/sergio101

[Pharo-users] Re: Backing up data

[sergio ruiz]

OOH..  I like this idea.

I’ll do a quick test on this and see if it works as expected..

For now, I want to store the data in the image.

The trick is, I like to start with a fresh image every few days. So, I’d like 
to open a new Pharo, pull the source code, then pull the data.

Thanks!


> 
> If you are keeping data in a Pharo image then you could just save the image 
> instead of using STON. Alternatively, you could model your data in an 
> Application class with three instance variables: theaters, movies, and 
> events. Then you could dump the root application object and STON should 
> preserve the relationships (I expect). I prefer this model since it allows me 
> to create a test object that encapsulates all the data in one place (even if 
> it is referenced from a class variable).


peace,
sergio
photographer, journalist, visionary

Public Key: 
https://pgp.key-server.io/pks/lookup?op=get=0x69B08F58923AB3A2
#BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV
@sergio_101@mastodon.social
https://sergio101.com
http://www.codeandmusic.com
http://www.twitter.com/sergio_101
http://www.facebook.com/sergio101

[Pharo-users] Re: Backing up data

[Norbert Hartl]

I think it depends how many objects you have and what your consistency 
requirements are.

> Am 18.01.2024 um 05:48 schrieb sergio ruiz :
> 
> I have been in the relational database world for decades. One of the things 
> that is super simple is backing up and restoring data.
> 
> Sometimes, it makes it makes sense to pull the production data into 
> development. In the case of a catastrophe, it’s super simple to restore data 
> from a backup.
> 
> I am thinking of backup strategies for my data once again, and am wondering 
> if things have changed since I was last using smalltalk.
> 
> I am thinking of the following:
> 
> STON
> I could dump all of my data into STON, but there is one slight twist. In this 
> project, it’s just making a list of movie listings. There are three basic 
> classes: Theater, Movie, Event. Each of these classes stores an 
> OrderedCollection of items of that type on a class variable, as I’ll need to 
> access each on their own.. so, if an event has a theater, I’ll need to access 
> Theaters on their own, and Movies on their own.
> 
> So, I would have to rebuild the list of Theaters from the Theaters in each 
> event. Otherwise, I would lose the link between the Theaters on the Theater 
> class variable and the Theater in the Event instance.

> Am I thinking about this incorrectly?

If you have three classes with class side collections you can have a make e.g. 
a dictionary and put the collections from the class side into it like 

model := {
   #theaters -> Theater instances.
   #movies -> Movie instances.
   #event -> Event instances.
} asDictionary.

The model object you can use with STON or fuel to store it and on reading back 
you store each of the values back on the class side variable. Simple and nice. 
I would use STON for that because it is equivalent to fuel but fuel has 
migration problems. Or better you need to load all your data, update fuel to a 
newer version and write it again with the new fuel version. STON is more 
reliable in that regard. Both have the problem that when objects change during 
serialization you might corrupt the structure of your data,

> 
> Gemstones
> I am totally into the idea of using gemstones. Is there a backup and restore 
> for objects here? do the relationships remain?
> 
GemStone is a huge thing to tame. I would not say the above and GemStone are 
alternatives. GemStone is a lot to install and learn but you get something that 
makes it hard to corrupt your data. So you have requirements that make it fit 
into GemStone or you have requirements that make it a fit for STON. Between 
both there is no overlap but rather a huge gap between the applicability of 
each.

> Voyage
> This seems like a good solution, but I would rather stay in the smalltalk 
> object store.
> 
Absolutely. Voyage is good for some use cases but you have to deal with mongoDB 
(not sure the small libraries work right now)

You could also consider using soil (https://github.com/ApptiveGrid/Soil) 
I might be a bit biased here :) but for most use cases this is the best of all 
the worlds described aboved. Even if there it has no release yet (coming soon) 
it is quite stable.

Hope this helps,

Norbert

> Ideas?
> 
> Thanks!
> 
> 
> peace,
> sergio
> photographer, journalist, visionary
> 
> Public Key: 
> https://pgp.key-server.io/pks/lookup?op=get=0x69B08F58923AB3A2
> #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV
> @sergio_101@mastodon.social
> https://sergio101.com
> http://www.codeandmusic.com
> http://www.twitter.com/sergio_101
> http://www.facebook.com/sergio101
> 

[Pharo-users] Re: Backing up data

[Davide Varvello via Pharo-users]

 +1 for Fuel, see here https://github.com/theseion/Fuel
CheersDavide



On Thursday, January 18, 2024 at 09:37:21 AM GMT+1, Todd Blanchard via 
Pharo-users  wrote:  
 
 You want to look at Fuel, a serialization library.

Sent from my iPhone

On Jan 17, 2024, at 9:49 PM, sergio ruiz  wrote:




I have been in the relational database world for decades. One of the things 
that is super simple is backing up and restoring data.
Sometimes, it makes it makes sense to pull the production data into 
development. In the case of a catastrophe, it’s super simple to restore data 
from a backup.
I am thinking of backup strategies for my data once again, and am wondering if 
things have changed since I was last using smalltalk.
I am thinking of the following:
STONI could dump all of my data into STON, but there is one slight twist. In 
this project, it’s just making a list of movie listings. There are three basic 
classes: Theater, Movie, Event. Each of these classes stores an 
OrderedCollection of items of that type on a class variable, as I’ll need to 
access each on their own.. so, if an event has a theater, I’ll need to access 
Theaters on their own, and Movies on their own.
So, I would have to rebuild the list of Theaters from the Theaters in each 
event. Otherwise, I would lose the link between the Theaters on the Theater 
class variable and the Theater in the Event instance.
Am I thinking about this incorrectly?
GemstonesI am totally into the idea of using gemstones. Is there a backup and 
restore for objects here? do the relationships remain?
VoyageThis seems like a good solution, but I would rather stay in the smalltalk 
object store.
Ideas?
Thanks!

peace,
sergio
photographer, journalist, visionary

Public Key: 
https://pgp.key-server.io/pks/lookup?op=get=0x69B08F58923AB3A2
#BitMessage 
BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV@sergio_101@mastodon.socialhttps://sergio101.com
http://www.codeandmusic.com
http://www.twitter.com/sergio_101
http://www.facebook.com/sergio101
  

[Pharo-users] Re: Backing up data

[Todd Blanchard via Pharo-users]

You want to look at Fuel, a serialization library.

Sent from my iPhone

> On Jan 17, 2024, at 9:49 PM, sergio ruiz  wrote:
> 
> I have been in the relational database world for decades. One of the things 
> that is super simple is backing up and restoring data.
> 
> Sometimes, it makes it makes sense to pull the production data into 
> development. In the case of a catastrophe, it’s super simple to restore data 
> from a backup.
> 
> I am thinking of backup strategies for my data once again, and am wondering 
> if things have changed since I was last using smalltalk.
> 
> I am thinking of the following:
> 
> STON
> I could dump all of my data into STON, but there is one slight twist. In this 
> project, it’s just making a list of movie listings. There are three basic 
> classes: Theater, Movie, Event. Each of these classes stores an 
> OrderedCollection of items of that type on a class variable, as I’ll need to 
> access each on their own.. so, if an event has a theater, I’ll need to access 
> Theaters on their own, and Movies on their own.
> 
> So, I would have to rebuild the list of Theaters from the Theaters in each 
> event. Otherwise, I would lose the link between the Theaters on the Theater 
> class variable and the Theater in the Event instance.
> 
> Am I thinking about this incorrectly?
> 
> Gemstones
> I am totally into the idea of using gemstones. Is there a backup and restore 
> for objects here? do the relationships remain?
> 
> Voyage
> This seems like a good solution, but I would rather stay in the smalltalk 
> object store.
> 
> Ideas?
> 
> Thanks!
> 
> 
> peace,
> sergio
> photographer, journalist, visionary
> 
> Public Key: 
> https://pgp.key-server.io/pks/lookup?op=get=0x69B08F58923AB3A2
> #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV
> @sergio_101@mastodon.social
> https://sergio101.com
> http://www.codeandmusic.com
> http://www.twitter.com/sergio_101
> http://www.facebook.com/sergio101
> 

[Pharo-users] Re: Backing up data

[James Foster via Pharo-users]

Sergio,

If you are keeping data in a Pharo image then you could just save the image 
instead of using STON. Alternatively, you could model your data in an 
Application class with three instance variables: theaters, movies, and events. 
Then you could dump the root application object and STON should preserve the 
relationships (I expect). I prefer this model since it allows me to create a 
test object that encapsulates all the data in one place (even if it is 
referenced from a class variable).

For GemStone, execute `SystemRepository fullBackupGzCompressedTo: aFilePath` 
(see details at 
https://downloads.gemtalksystems.com/docs/GemStone64/3.7.x/GS64-SysAdminGuide-3.7/11-BackupAndRestore.htm).
 This does a full object dump and preserves the relationship between objects. 
Note that GemStone also creates a transaction log so once a transaction 
commits, you can restore it in event of a system crash.

James Foster

> On Jan 17, 2024, at 8:48 PM, sergio ruiz  wrote:
> 
> I have been in the relational database world for decades. One of the things 
> that is super simple is backing up and restoring data.
> 
> Sometimes, it makes it makes sense to pull the production data into 
> development. In the case of a catastrophe, it’s super simple to restore data 
> from a backup.
> 
> I am thinking of backup strategies for my data once again, and am wondering 
> if things have changed since I was last using smalltalk.
> 
> I am thinking of the following:
> 
> STON
> I could dump all of my data into STON, but there is one slight twist. In this 
> project, it’s just making a list of movie listings. There are three basic 
> classes: Theater, Movie, Event. Each of these classes stores an 
> OrderedCollection of items of that type on a class variable, as I’ll need to 
> access each on their own.. so, if an event has a theater, I’ll need to access 
> Theaters on their own, and Movies on their own.
> 
> So, I would have to rebuild the list of Theaters from the Theaters in each 
> event. Otherwise, I would lose the link between the Theaters on the Theater 
> class variable and the Theater in the Event instance.
> 
> Am I thinking about this incorrectly?
> 
> Gemstones
> I am totally into the idea of using gemstones. Is there a backup and restore 
> for objects here? do the relationships remain?
> 
> Voyage
> This seems like a good solution, but I would rather stay in the smalltalk 
> object store.
> 
> Ideas?
> 
> Thanks!
> 
> 
> peace,
> sergio
> photographer, journalist, visionary
> 
> Public Key: 
> https://pgp.key-server.io/pks/lookup?op=get=0x69B08F58923AB3A2
> #BitMessage BM-NBaswViL21xqgg9STRJjaJaUoyiNe2dV
> @sergio_101@mastodon.social
> https://sergio101.com
> http://www.codeandmusic.com
> http://www.twitter.com/sergio_101
> http://www.facebook.com/sergio101
>