Operation Copy Pasta (or how to preserve a forum)

[misterprofessionality]

So some of you may or may not be aware that I've been poking around behind the scenes for a way to ensure that the CB1100 Forum keeps humming along for the foreseeable future. I started researching in my spare time the moment Guth made his announcement and let us all know that he needs to step away. Keep in mind as you read this that it may look like I don't have a lot to show for it but web administration is neither a hobby nor an occupation for me, so I'm learning as I go.

So far I've discovered that I can mirror the site using a bit of software called HTTrack, which will slowly run a deep dive down every single link on the forum to create a copy of every single page. This should manage to fully copy all the threads, posts, pictures, links, ect. that exist on the forum. My understanding from this point is that I should be able to purchase a web domain and upload the site as if it were my own. Putting aside that this is essentially an act of plagiarism, It may be the only way to preserve the forum in its current format with as little transitional pain as possible. Once an operational "mirror" of the CB1100 forum is up and running, the work begins to re-register all the accounts and get every user operating on the new site under their preferred username. This process is beyond my ken at this point, but my goal is to get there.

The first hiccup I encountered was the sheer size of the forum itself, which I underestimated when I began trying to mirror it with HTTrack. I just placed an order for a 2tb hard drive today, so as soon as that arrives I should I have all the space I need to 100% complete the mirror process and start experimenting with bringing the doppleganger site online.

Anyway, I'm leaving this information here so that people know I'm not resting on my laurels about this, and to create a space for anybody with Web Admin experience to drop their advice if they're feeling kind enough to help. As long as you keep it simple you shouldn't go over my head too far :rofl:

 

[pdedse]

Very good to hear MP...I certainly hope this works for all of us interested. I'll support anyway I can.

 

[GoldOxide_imp]

Subject: "Operation Copy Pasta (or how to preserve a forum)"

"Mmmm, pasta, uuuuuuuh."

Would the Forum soul survive a site cloning process?

 

[misterprofessionality]

Subject: "Operation Copy Pasta (or how to preserve a forum)"

"Mmmm, pasta, uuuuuuuh."

Would the Forum soul survive a site cloning process?

Definitely not. The new one will likely come out with a goatee and a tendancy to eat tacos and hot dogs from the top down.

 

[j3gq_imp]

Hello misterP,
What you do sounds great - thank you for trying !!! Let me add two trains of thought, for what it's worth:
1) your approach comes could (potentially) hit a snag, which may not surface until almost all the effort is spent. Databases like this have permissions at every "leave and branch of the tree". Just think of permissions on the files of your harddisk, admin can open a file, you cannot, etc. While the harvester (HTTrack etc.) may be able to download the whole forum database and all its files, there is no guaranty that on the new website/server one can get all these permissions right as needed. A skilled programmer may be able to fix that, but it may involve a lot of time and trouble-shooting (and money). I am not trying to discourage you, but this is not like uploading a file to a different file server (where you could fix the permissions).
2) if we could get access to Guth's user account information (operator/server, user name, passwords, payment details ...) a "new administrator" could, on behalf of all of us, keep running the forum as it is. Whoever is at the receiving end of Guth's payments will happily continue the contract. The money we could collect from "the willing coalition of prime members". Of course this approach does not cover all possible risks (the server operator could go bust), but at least we would have a hard cut-off date to face.

 

[misterprofessionality]

Hello misterP,
What you do sounds great - thank you for trying !!! Let me add two trains of thought, for what it's worth:
1) your approach comes could (potentially) hit a snag, which may not surface until almost all the effort is spent. Databases like this have permissions at every "leave and branch of the tree". Just think of permissions on the files of your harddisk, admin can open a file, you cannot, etc. While the harvester (HTTrack etc.) may be able to download the whole forum database and all its files, there is no guaranty that on the new website/server one can get all these permissions right as needed. A skilled programmer may be able to fix that, but it may involve a lot of time and trouble-shooting (and money). I am not trying to discourage you, but this is not like uploading a file to a different file server (where you could fix the permissions).
2) if we could get access to Guth's user account information (operator/server, user name, passwords, payment details ...) a "new administrator" could, on behalf of all of us, keep running the forum as it is. Whoever is at the receiving end of Guth's payments will happily continue the contract. The money we could collect from "the willing coalition of prime members". Of course this approach does not cover all possible risks (the server operator could go bust), but at least we would have a hard cut-off date to face.

You are 100% correct, and that was my first approach, but it's starting to look like that won't happen, so I'm preparing alternative ideas (like what you see here). I'm not opposed to re-developing the website entirely from scratch but backing up the entire archive of posts and data is still an important goal, even if it just takes the form of a massive archive rather than the forum in its current shape.

 

[PowerDubs_imp]

Silly question-

He isn't gone yet- why not make him an offer, throw him an agreed payment and take over the account login credentials- then change the password.

I'm sure many of us would chip in.

To just shut the site down is.... silly. No reason to not pass the keys on to someone else who wants to drive.

 

[Inhouse Bob]

Silly question-

He isn't gone yet- why not make him an offer, throw him an agreed payment and take over the account login credentials- then change the password.

I'm sure many of us would chip in.

To just shut the site down is.... silly. No reason to not pass the keys on to someone else who wants to drive.

I have no other friends, so this is an important issue to me.
I'm sure many of us would be willing to chip in. I myself have a jar of change that is just sitting there on my desk.
I don't know how much these things cost or how much Mr. Guth has invested, but we should be able to help him in his retirement. I could pass a hat at the Rally

 

[misterprofessionality]

Silly question-

He isn't gone yet- why not make him an offer, throw him an agreed payment and take over the account login credentials- then change the password.

I'm sure many of us would chip in.

To just shut the site down is.... silly. No reason to not pass the keys on to someone else who wants to drive.

Ive actually put forward that very same approach but Guth seems understandably buried and unable to answer. I wanted to be prepared in the event i have to carry forward without that support, thats why im pursuing the mirror idea.

Ive made my offer to help or take over or even just host it and let somebody with more experience run it. The last thing i want to do is harrass the poor guy about it.

 

[GoldOxide_imp]

Subject: "Operation Copy Pasta (or how to preserve a forum)"

"Mmmm, pasta, uuuuuuuh."

Would the Forum soul survive a site cloning process?

Definitely not. The new one will likely come out with a goatee and a tendancy to eat tacos and hot dogs from the top down.

Definitely not. The new one will likely come out with a goatee and a tendancy to eat tacos and hot dogs from the top down.
lol - maybe so.

 

[Cormanus]

Interesting, misterprofessionality, I looked at doing the same thing recently. Now I won't as you appear to have it in the bag. I think its worth pursuing, even as a just-in-case strategy.

However, to add a little to the conversation. Recently—and reluctantly, as I'm loathe to disturb him—I contacted Guth. His struggle with the health issues outlined in his announcement continues. Poor fella. He did tell me, though, that he intends for the forum to survive and has been in contact with a member with forum admin experience who had expressed interest in taking it over. I've no idea who that member is.

 

[misterprofessionality]

Its far from in the bag Cormanus. Please pursue it if you have the inclination. Multiple people working to preserve things can only be a big help

 

[Cormanus]

Ok. Sure. Not right at the moment, but later in the year I'm happy to have a crack at it. Do you have any idea how large the site is?

 

[misterprofessionality]

My nearest estimate from partial attempts to mirror it places it in the 1tb range. I just ordered a 2tb hard drive.

 

[Cormanus]

Thanks.

 

[PowerDubs_imp]

In other words- less than a decent night out at dinner with my wife.


Sent from my iPhone using Tapatalk

 

[Yata-Garasu_imp]

It would be sad if this forum didn't survive.

Let us know what we can do to help.

 

[j3gq_imp]

Ok. Sure. Not right at the moment, but later in the year I'm happy to have a crack at it. Do you have any idea how large the site is?

Not sure why the size matters, but let me give you a better estimate based on numbers we do already know:
- from forum statistics : 300,000 posts
- bare size of one average post ~ 2k of text
- database overheads per post up to 20k of text
= this turns it into 300,000 * 22kB ~ 6.5GB

- come on top pictures, but only those pictures uploaded as a real attachment (links through imgur etc. do not take up space)
- my guess only on in a 50 posts uses this feature
- pic size is limited to 250 kB
- 300,000 / 50 * 250kB ~ 1.5 GB

- finally space to expand in the future ...

All in all the forum database today, I cannot imagine is more than a total of 10 GB, a far cry from the 1TB mentioned before. I haven't seen a cloud hosting service lately, not offering this amount or much larger. In other words, our 10 (or future 20) GB can be hosted anywhere in the world at very low cost.

Based on the idea that even a bad guess is always better than no idea at all ...I hope this helps.

 

[Yata-Garasu_imp]

I'm surprised that Honda doesn't subsidize a motorcycle forum that includes all their models, vintage, contemporary, and in between.

 

[peterbaron]

I'm surprised that Honda doesn't subsidize a motorcycle forum that includes all their models, vintage, contemporary, and in between.

HONDA Co does not, but the Forum members subsidize Honda by writing interesting threads, posts, articles, issue technical notes, place photos so the guest visiting this Forum are encouraged to buy the CB1100 or other Honda motorcycle models......