Re: Compuserve

2021-01-12 Thread James B DiGriz via cctalk
On Tue, 12 Jan 2021 10:44:45 -0600
John Foust via cctalk  wrote:

> At 09:56 AM 1/12/2021, James B DiGriz via cctalk wrote:
> >I'm kicking myself right now because I had forgotten that archive.org
> >hosts a web interface to an (incomplete) archive of the Compuserve
> >web forums. I think this is based on the archiveteam emergency
> >scrape of 2017.  
> 
> Do you have a link to it?

https://web.archive.org/web/sitemap/forums.compuserve.com

> 
> > Somebody correct me if I'm wrong. There is also a 64GB tranche of
> >raw SC-40 and SCSI FE disk images on IA.   
> 
> Disk images of what, from where?  CompuServe?

Yes, Compuserve. 34GB, though, not 64. My bad.

https://archive.org/details/2015-05-compuserve-raw-disks

An attempt by Christian Mund to use SIMH to boot and inspect these
disks is described here:

https://medium.com/@mpnet/trying-to-make-sense-of-compuserve-server-hard-disk-images-posted-on-archive-org-b1c62ce6012b

jbdigriz



jbdigriz



Re: Compuserve

2021-01-12 Thread John Foust via cctalk
At 09:56 AM 1/12/2021, James B DiGriz via cctalk wrote:
>I'm kicking myself right now because I had forgotten that archive.org
>hosts a web interface to an (incomplete) archive of the Compuserve web
>forums. I think this is based on the archiveteam emergency scrape of
>2017.

Do you have a link to it?

> Somebody correct me if I'm wrong. There is also a 64GB tranche of
>raw SC-40 and SCSI FE disk images on IA. 

Disk images of what, from where?  CompuServe?

- John



Re: Compuserve

2021-01-12 Thread James B DiGriz via cctalk
On Tue, 12 Jan 2021 00:25:30 -0600
Jason T via cctalk  wrote:

> On Sun, Jan 10, 2021 at 6:46 PM John Foust via cctalk
>  wrote
> >
> >  From '85 until '94 or so, I saved my transcripts from visiting
> > CompuServe, PeopleLink, Delphi, Genie, BIX, the Well, etc.
> >  
> 
> This is an incredible archive on its own.  Are you willing to share
> the source files?  I'd enjoy just wandering through the raw data.
> It's probably been over 30 years since I've even seen the CIS forum
> message format (not counting a few moments ago, of course.)
> 
> -j
> 

I agree with John that some curation will be in order here before
making stuff public accessible. And he's right that building the
database would be the first order of business. Reasonably
straightforward to bolt different interfaces to that. 

I'm kicking myself right now because I had forgotten that archive.org
hosts a web interface to an (incomplete) archive of the Compuserve web
forums. I think this is based on the archiveteam emergency scrape of
2017. Somebody correct me if I'm wrong. There is also a 64GB tranche of
raw SC-40 and SCSI FE disk images on IA. And like I said there
are bits and pieces here and there. Some of the CIS TI file libraries
are on whtech, for instance. 

Something people could easily add to would be great. 

jbdigriz


Re: Compuserve

2021-01-11 Thread Jason T via cctalk
On Sun, Jan 10, 2021 at 6:46 PM John Foust via cctalk
 wrote
>
>  From '85 until '94 or so, I saved my transcripts from visiting
> CompuServe, PeopleLink, Delphi, Genie, BIX, the Well, etc.
>

This is an incredible archive on its own.  Are you willing to share
the source files?  I'd enjoy just wandering through the raw data.
It's probably been over 30 years since I've even seen the CIS forum
message format (not counting a few moments ago, of course.)

-j


Re: Compuserve

2021-01-11 Thread John Foust via cctalk
At 09:57 AM 1/11/2021, Ethan O'Toole wrote:
>>So how could you organize all that into a web database and interface?
>
>Could you put a text front end on it that emulates the original CIS?

I'd say the first step is defining a SQL database structure to hold
it all, then the code to display it, and sure, maybe an interactive
emulation would be nifty...  but note that CompuServe messages 
disappeared on their own.  A forum could only hold so many, and
the oldest were flushed.  So an emulation would need to pick a
moment in time to emulate.

In today's interface style, I'd say you'd want to be able to search 
it all, then view a message in context with the rest of its thread.

>>Am I going to trust my parser to have never made a mistake?
>>Or do I need to read a half-million posts to confirm it didn't?
>
>Search for your name?

Not that simple.  After you'd navigated to the email area and
asked to display a particular waiting message, it only showed
date / from / subject at the top, and at least in 1995, it
actually put the "Distribution: To: John Foust - Syndesis 
Corporation > [76004,1763]" at the bottom after the message!

For Internet gateway'd email, it showed the full message as is, 
with the raw headers and/or "Content-type:" and left you to
figure it out on your own.

- John



Re: Compuserve

2021-01-11 Thread Ethan O'Toole via cctalk

I was often free-flagged on CIS and many other services so surfing was free.
For the kids out there, in today's dollars CompuServe was about $20 an hour.


As a young lad, the high costs of things like CompuServe is why I stuck to 
BBS systems, until the internet came along.



Perl parser to separate individual forum posts.  I have about 400 megs of
messages, on the order of 525,000 messages.  I split each post
into its own XML file, tagging the pieces (from, to, subject, etc.)


That is awesome! You should definitely get that stuff on the web!!


I've left the hard part of constructing the web software to make it all
readable and searchable.



 #: 126608 S0/Sysop's Corner
 29-Oct-95  00:20:12
 Sb: #126546-#Problem Uploading Files
 Fm: Betty Clay 76702,337
 To: Shawn/Silent Paw 74777,2602 (X)

 This would be caused by the entire forum running out of blocks in the library.
 Steve handles that, and I'm sure he has noticed and requested more by now.  It
 is quite likely that we won't get them before Monday, though. Sometimes there's
 no one around to handle the request on weekends.

Like this message, some sections were "sysop" and perhaps should be
considered non-public.

My transcripts also have stuff from before and after the forum posts.
Who knows, maybe someone would find interesting the "What's new this week"
intro menu you'd see at login.

Or better yet, the file libraries.  Each section had a "data library".
Each file showed a filename, timestamp and size, uploader PPN and
name, and title line and keywords and a paragraph of description.
I have some listings of the file areas. I may also have the original
files I downloaded.  All that could be linked into a new web version, too.

Maybe there are some text-based public live group chats (conferences), too.

So how could you organize all that into a web database and interface?


Could you put a text front end on it that emulates the original CIS?

A modern emulator that emulated the UI from the original on the internet..


It would also make sense for me to improve my parser to separate
out each email I sent or received.  I'd be eager to release the public
forum posts but I also don't want to release my private emails.


Makes sense


Am I going to trust my parser to have never made a mistake?
Or do I need to read a half-million posts to confirm it didn't?


Search for your name?


I have to believe that other people saved transcripts of CompuServe
forums if not other services, too.  Ideally there could be a way to add
everyone's messages to the reconstruction.

Indeed, google ' "Sb:" "Fm:" "To:" sysop cis ' and you'll see
other bits and pieces of CompuServe transcripts.

https://www.google.com/search?q=%22Sb%3A%22+%22Fm%3A%22+%22To%3A%22+sysop+cis

I don't know what it was like inside CIS back then, but I can't help
but wonder if any magtape archives left the building.  I asked
an Amiga forum owner if he saved anything...  he said "no" but
maybe other forum owners did.

As an aside, keep in mind that the people running CIS forums held a
contract with CIS and they were paid.  For many, it was their sole income.
Some ran several forums and made quite a bit of money at it.

I was very surprised to learn the web-based http://forums.compuserve.com/
existed until three-four years ago.
- John





Re: Compuserve

2021-01-11 Thread geneb via cctalk

On Sun, 10 Jan 2021, Grant Taylor via cctalk wrote:


On 1/10/21 7:16 PM, James B DiGriz wrote:

Synchronet is still very much "a thing".


Yes, and no.

Yes, there is /a/ product named Synchronet, probably from the same vendor or 
subsidiary.  However, I don't think that it is the /same/ product.


It's the same package, by the same author.  Rob Swindell(author) released 
Synchronet as open source a number of years ago.  I currently run v3.18c, 
which is the current release version. You can run it in Windows, Linux, or 
*BSD.  MS-DOS and OS/2 versions are also available under the "legacy" 
branch.  Downloads: http://www.synchro.net/download.html


If v3 won't import the DOS message bases automagically, I'm quite sure 
that if you email Rob, he can help make this happen.


g.


--
Proud owner of F-15C 80-0007
http://www.f15sim.com - The only one of its kind.
http://www.diy-cockpits.org/coll - Go Collimated or Go Home.
Some people collect things for a hobby.  Geeks collect hobbies.

ScarletDME - The red hot Data Management Environment
A Multi-Value database for the masses, not the classes.
http://scarlet.deltasoft.com - Get it _today_!


Re: Compuserve

2021-01-10 Thread Grant Taylor via cctalk

On 1/10/21 7:16 PM, James B DiGriz wrote:

Synchronet is still very much "a thing".


Yes, and no.

Yes, there is /a/ product named Synchronet, probably from the same 
vendor or subsidiary.  However, I don't think that it is the /same/ product.


More specifically, I believe that contemporary Synchronet is more of a 
suite of packages.  And those packages are traditional Linux packages.


I believe the old DOS / Windows (?) Synchronet package is LONG since dead.


I'm not familiar with the message base format it uses,


"JAM" message format comes to mind from conversations with the original 
SYSOP.  Though I don't know if that's what Synchronet is using or 
something else from conversstion collisions.



and a lot will depend on the version you have,


I'm guessing it's from '95 - '99.  2.x or 3.x comes to mind.

but there should be utilities available for exporting messages directly 
from a mounted disk image, or at least a running instance.


I suspect the best thing to do would be to restore the VM, boot it, and 
see what I can find.  MS-DOS 5.x or 6.x don't need many resources.  I 
think the machine had a ~300 MB hard drive and ~32 MB of memory (if that).


Should something be available on the Synchronet website or Vertrauen 
BBS. There is a support echo still on Fidonet, too, iirc.


Is the echo /active/?  I know that there are LOTS of newsgroups still in 
active files on news servers, even though the the newsgroup hasn't seen 
traffic in years.


Worst case you can read messages to a capture log, or export .QWK to 
an offline reader with any version since the early '90s.


I'm sure there's something clever that we can do to get data out.  }:-)



--
Grant. . . .
unix || die


Re: Compuserve

2021-01-10 Thread James B DiGriz via cctalk
On Sun, 10 Jan 2021 17:12:04 -0700
Grant Taylor via cctalk  wrote:

> On 1/10/21 3:03 PM, James B DiGriz via cctalk wrote:
> > Fidonet echos.  
> 
> I have a copy of a friend's BBS from back in the early 2000's.
> 
> I'm sure that there is a way to extract messages from the echos.
> 
> It's Synchronet if anyone wants to direct me how to extract things.
> 
> 
> 
Synchronet is still very much "a thing". I'm not familiar with the
message base format it uses, and a lot will depend on the version you
have, but there should be utilities available for exporting
messages directly from a mounted disk image, or at least a running
instance. Should something be available  on the Synchronet website or
Vertrauen BBS. There is a support echo still on Fidonet, too, iirc. 

Worst case you can read messages to a capture log, or export .QWK to an
offline reader with any version since the early '90s. 


Re: Compuserve

2021-01-10 Thread Warner Losh via cctalk
On Sun, Jan 10, 2021, 3:03 PM James B DiGriz via cctalk <
cctalk@classiccmp.org> wrote:

> On Sun, 10 Jan 2021 13:57:17 -0600
> Jason T via cctalk  wrote:
>
> > On Sun, Jan 10, 2021, 05:41 Tomas By via cctalk
> >  wrote:
> >
> > > Hi all,
> > >
> > > Does anybody know if the old Compuserve discussion forums are
> > > available anywhere now?
> > >
> > > I'm specifically interested in the ones about HP Palmtops and
> > > cc:Mail.
> >
> > As far as I know, nothing was saved. It was a massive loss for early
> > online history.
> >
> > >
> >
>
> Ditto for Delphi, The Source, etc. And Fidonet echos. Some of us tried
> to collect and preserve the Delphi TI Forum about 20 years ago when it
> was known to still exist on tapes, but nothing ever came of it that I'm
> aware of.
>


I have some fidonet echos for the DEC Rainbow I downloaded from one of the
archives...

There will have been pieces saved here and there, but mostly on floppies or
> old,
> failing hard disks or rotting tapes not stored safely, so if anyone
> wants to archive them, they need to do it ASAP.
>

Yes. Indeed.

Warner

>
>


Re: Compuserve

2021-01-10 Thread Chuck Guzis via cctalk
On 1/10/21 4:45 PM, John Foust via cctalk wrote:
> At 04:03 PM 1/10/2021, James B DiGriz via cctalk wrote:
>> There will have been pieces saved here and there, but mostly on floppies or 
>> old,
>> failing hard disks or rotting tapes not stored safely, so if anyone
>> wants to archive them, they need to do it ASAP.  

I wonder if MS archived their areas from CIS--at one point, it was the
primary avenue for MS OEM support--for example, you requested a code for
a vendor-specific 16-bit DLL on Windows 3.

--Chuck



Re: Compuserve

2021-01-10 Thread John Foust via cctalk
At 04:03 PM 1/10/2021, James B DiGriz via cctalk wrote:
>There will have been pieces saved here and there, but mostly on floppies or 
>old,
>failing hard disks or rotting tapes not stored safely, so if anyone
>wants to archive them, they need to do it ASAP.  

 From '85 until '94 or so, I saved my transcripts from visiting 
CompuServe, PeopleLink, Delphi, Genie, BIX, the Well, etc.  

I probably focused on Amiga forums.  There are some CAD and 
3D animation and programmer and Atari and Tandy M-100 forums, too.

I was often free-flagged on CIS and many other services so surfing was free.  
For the kids out there, in today's dollars CompuServe was about $20 an hour.

I had largely automated my visits with Pro-YAM on a PC.  Log in, grab
my personal email, upload the outgoing mail, scrape the forums I wanted 
to read, save it all to a text file, and then I'd read and edit in Brief 
at my leisure.  Some of the time, my visits were not scripted.

Then I'd squeeze them all into floppy-sized ZIPs for archiving.  At 
some point I copied all those floppies to the active storage I use and
backup now.

My archives aren't pristine backups.  I erased many posts that
I thought were uninteresting at the time.  As I read, I compulsively 
reformatted paragraphs with Brief's word-wrap macro.  

The largest part of my archive was CompuServe.  In 2015 I wrote a little 
Perl parser to separate individual forum posts.  I have about 400 megs of 
messages, on the order of 525,000 messages.  I split each post 
into its own XML file, tagging the pieces (from, to, subject, etc.) 

I've left the hard part of constructing the web software to make it all
readable and searchable.  

There are a few problems to solve.  You'd want to impose some structure
on it all.  A topic such as "Amiga" had several forums...  user, programmer,
arts, vendors.  

Enter a forum and it would tell you the range of message numbers 
available and the number of the last one you've read.  At that
point I'd generally say "read all new."  Each forum had a dozen or so 
numbered sections.  Sections had names and they changed over time.  

Posts have a number and replies mention the parent message in the subject
line, so they're kind of threaded.  An example:

  #: 126608 S0/Sysop's Corner
  29-Oct-95  00:20:12
  Sb: #126546-#Problem Uploading Files
  Fm: Betty Clay 76702,337
  To: Shawn/Silent Paw 74777,2602 (X)

  This would be caused by the entire forum running out of blocks in the library.
  Steve handles that, and I'm sure he has noticed and requested more by now.  It
  is quite likely that we won't get them before Monday, though. Sometimes 
there's
  no one around to handle the request on weekends.

Like this message, some sections were "sysop" and perhaps should be 
considered non-public.

My transcripts also have stuff from before and after the forum posts.  
Who knows, maybe someone would find interesting the "What's new this week" 
intro menu you'd see at login.  

Or better yet, the file libraries.  Each section had a "data library".  
Each file showed a filename, timestamp and size, uploader PPN and
name, and title line and keywords and a paragraph of description.
I have some listings of the file areas. I may also have the original
files I downloaded.  All that could be linked into a new web version, too.

Maybe there are some text-based public live group chats (conferences), too.

So how could you organize all that into a web database and interface?

It has some similarities to Usenet posts.  But not quite.  The sections
change over time.

It would also make sense for me to improve my parser to separate 
out each email I sent or received.  I'd be eager to release the public
forum posts but I also don't want to release my private emails.

Am I going to trust my parser to have never made a mistake?
Or do I need to read a half-million posts to confirm it didn't?

I have to believe that other people saved transcripts of CompuServe 
forums if not other services, too.  Ideally there could be a way to add
everyone's messages to the reconstruction.

Indeed, google ' "Sb:" "Fm:" "To:" sysop cis ' and you'll see
other bits and pieces of CompuServe transcripts.

https://www.google.com/search?q=%22Sb%3A%22+%22Fm%3A%22+%22To%3A%22+sysop+cis

I don't know what it was like inside CIS back then, but I can't help
but wonder if any magtape archives left the building.  I asked 
an Amiga forum owner if he saved anything...  he said "no" but
maybe other forum owners did.

As an aside, keep in mind that the people running CIS forums held a 
contract with CIS and they were paid.  For many, it was their sole income.
Some ran several forums and made quite a bit of money at it.

I was very surprised to learn the web-based http://forums.compuserve.com/ 
existed until three-four years ago.

- John



Re: Compuserve

2021-01-10 Thread Grant Taylor via cctalk

On 1/10/21 3:03 PM, James B DiGriz via cctalk wrote:

Fidonet echos.


I have a copy of a friend's BBS from back in the early 2000's.

I'm sure that there is a way to extract messages from the echos.

It's Synchronet if anyone wants to direct me how to extract things.



--
Grant. . . .
unix || die


Re: Compuserve

2021-01-10 Thread James B DiGriz via cctalk
On Sun, 10 Jan 2021 13:57:17 -0600
Jason T via cctalk  wrote:

> On Sun, Jan 10, 2021, 05:41 Tomas By via cctalk
>  wrote:
> 
> > Hi all,
> >
> > Does anybody know if the old Compuserve discussion forums are
> > available anywhere now?
> >
> > I'm specifically interested in the ones about HP Palmtops and
> > cc:Mail. 
> 
> As far as I know, nothing was saved. It was a massive loss for early
> online history.
> 
> >  
> 

Ditto for Delphi, The Source, etc. And Fidonet echos. Some of us tried
to collect and preserve the Delphi TI Forum about 20 years ago when it
was known to still exist on tapes, but nothing ever came of it that I'm
aware of.  

There will have been pieces saved here and there, but mostly on floppies or old,
failing hard disks or rotting tapes not stored safely, so if anyone
wants to archive them, they need to do it ASAP.  




Re: Compuserve

2021-01-10 Thread Jason T via cctalk
On Sun, Jan 10, 2021, 05:41 Tomas By via cctalk 
wrote:

> Hi all,
>
> Does anybody know if the old Compuserve discussion forums are
> available anywhere now?
>
> I'm specifically interested in the ones about HP Palmtops and cc:Mail.
>

As far as I know, nothing was saved. It was a massive loss for early online
history.

>


Compuserve

2021-01-10 Thread Tomas By via cctalk
Hi all,

Does anybody know if the old Compuserve discussion forums are
available anywhere now?

I'm specifically interested in the ones about HP Palmtops and cc:Mail.

/Tomas