Fusor site operation advisor board.

Announcements and items of immediate importance.
Post Reply
Frank Sanns
Site Admin
Posts: 2119
Joined: Fri Jun 14, 2002 2:26 pm
Real name: Frank Sanns

Fusor site operation advisor board.

Post by Frank Sanns »

The recent backup and restore problem with the site has been a many pronged issue. It seems totally preventable yet here we are again. For this reason, I would like to solicit input and assistance with the running of this site.

There are many things that all of you are not aware of going on behind the scenes. We have limited storage space, backups take multiple hours, there are several full access passwords and accounts out there that can take down the site in one stroke, there is a conflict between this site and Paul's other sites on the hosting service, and a few other gems. It was actually a conflict of differing versions of platforms that precipitated this mess because I had professed to never allow another update because of the complications. Paul has now had to roll back the PHP version from 8 to 7.4 to keep his end of things working. It appears the fusor site is ok. I think. But what happens when we must update again? I shudder to think that we might again be in some thin line where one version is maxed out and the other is just starting by a thread.

Many people offer some level of help and it is greatly appreciated. The full help of thecoalman has been invaluable to restore our site again.

I still see some issues going forward and one of them is the complete lack of full redundancy. There is one person, one backup, one stroke away from success and failure. I just don't like it and never have.

The solution is not more cooks in the kitchen because then nobody ultimately knows the big picture and everything that is going on. If people are making changes and it is not known by Paul and me and something goes down, we do not know where to start. It is really a dilemma.

Then there is the time factor. If you wondered why it took us so many days to be able to try workarounds to fix the site? Well, the transfer speed of ftp and SSH and such are slow. Sometimes as slow as a few times faster than an old dial up modem. The site is roughly 18GB so do the math. It can take many hours just to upload the files (picture directory) which is around one third of the site.

To figure out if the change took you have a very long upload and download delay. It is a little like communicating with Voyager 1 at the edge of the solar system. Multiple hours before you get a communication. What should just be a keystroke and move on can have a multiple hour delay before the next keystroke can be input.

There is also no undo button. Once done, it is done. If this is on a critical file, well, it can have wide spread ramifications.

At the moment, I am doing a file synchronization download as thecoalman has laid out. It seems around 8gb has changed since our January backup. It will be interesting to see what future synchronizations look like. Hopefully they will not be so long.

So maybe some people can volunteer to run these synchronization to so there are multiple versions of the site around. One complication is that the synch has frozen some elements of administrator control because the cache is captured during the operation.

It is not good enough to just hand out full site or ftp access. There must be a plan and a schedule and a backup to the backup and a check and balance to be sure nothing goes wrong with any of these updates, backups, synchs, edits, photos, etc.

Let the discussion begin.
Achiever's madness; when enough is still not enough. ---FS
We have to stop looking at the world through our physical eyes. The universe is NOT what we see. It is the quantum world that is real. The rest is just an electron illusion. ---FS
Frank Sanns
Site Admin
Posts: 2119
Joined: Fri Jun 14, 2002 2:26 pm
Real name: Frank Sanns

Re: Fusor site operation advisor board.

Post by Frank Sanns »

Rex Allers, Here is your chance to have control over the site. You have had much to say about this for quite a while. Here is your chance. You can have ultimate responsibility for backups and proper restoration of files and to be sure everything is running and updated properly.
Achiever's madness; when enough is still not enough. ---FS
We have to stop looking at the world through our physical eyes. The universe is NOT what we see. It is the quantum world that is real. The rest is just an electron illusion. ---FS
User avatar
Bob Reite
Posts: 576
Joined: Sun Aug 25, 2013 9:03 pm
Real name: Bob Reite
Location: Wilkes Barre/Scranton area

Re: Fusor site operation advisor board.

Post by Bob Reite »

Isn't there some way to mirror the site?
The more reactive the materials, the more spectacular the failures.
The testing isn't over until the prototype is destroyed.
Frank Sanns
Site Admin
Posts: 2119
Joined: Fri Jun 14, 2002 2:26 pm
Real name: Frank Sanns

Re: Fusor site operation advisor board.

Post by Frank Sanns »

Bob Reite wrote: Thu Feb 17, 2022 10:43 pm Isn't there some way to mirror the site?
There is but we are out of memory to have an online backup and the site. Also if we actually mirror (it is possible), anything that corrupts the main site will corrupt the mirror. There is no undo button. That is why the online backups and restore procedures have to be full proof and long full backups weekly or every other week of each component needs to continue. I am tired.
Achiever's madness; when enough is still not enough. ---FS
We have to stop looking at the world through our physical eyes. The universe is NOT what we see. It is the quantum world that is real. The rest is just an electron illusion. ---FS
Frank Sanns
Site Admin
Posts: 2119
Joined: Fri Jun 14, 2002 2:26 pm
Real name: Frank Sanns

Re: Fusor site operation advisor board.

Post by Frank Sanns »

And we are partially down again. Richard just messaged me. Some functions for posting are not working. I can see some things too.

Paul is still having trouble with his other sites and apparently the PHP version got rolled back so those sites work. Ours however, has issues again.

No comment............but this is all getting old.
Achiever's madness; when enough is still not enough. ---FS
We have to stop looking at the world through our physical eyes. The universe is NOT what we see. It is the quantum world that is real. The rest is just an electron illusion. ---FS
Matt_Gibson
Posts: 505
Joined: Thu Sep 23, 2021 10:36 am
Real name: Matt Gibson

Re: Fusor site operation advisor board.

Post by Matt_Gibson »

Could any of these problems be solved by a paid upgrade of some sort? If so, I’m sure that many of us would kick in to help…

-Matt
thecoalman
Posts: 189
Joined: Sun Feb 06, 2022 1:26 pm
Real name:

Re: Fusor site operation advisor board.

Post by thecoalman »

Good morning<very early morning here>. Still here but I promise I'm leaving soon, as soon as I get that script finsihed. It works fine on my local computer but needs some tweaking for live environment where resources are not unlimited.

php version running here is 7.4.X which should not be an issue at all. I'm not seeing errors in server error log or anywhere else. Any specifics?

If Paul needs to run even lower versions of php on his other sites you can just set a specific version for fusor.net to any version you want, preferably php8.
Frank Sanns wrote: Thu Feb 17, 2022 6:57 pm It appears the fusor site is ok.
I'm not sure exactly why the error when you switched back to php5.6 but the obvious culprit is phpBB's cache. Anytime you are encountering errors with phpBB for a quick fix you can try to see if cleaing the cache from ACP helps, if you don't have access to the ACP because of the error you can manually delete all the files and folders in the cache folder except index.htm and .htaccess(*see below). If problem still persists the next step is to look at server error log, you'll find this in the domain root and I believe you can access truncated version in Cpanel with the last hundred lines or whatever amount it is.

Generally speaking you do not need to clear the cache unless there is something wrong. The only other reason is if you make an edit to a template file.

phpBB typically supports the latest versions of php shortly after it's release and older ones will fall out naturally because of implementation of features in new versions. The latest release of phpBB still supports down to 7.1.3 but that's already EOL and won't be supported in future releases at some point. Just an FYI but with Cpanel you can set any version of php that is installed on the server on a per directory basis if the need arises. It's just a couple of lines to add to the .htaaceess file and will affect every directory below it. php information is available in the ACP >> System tab >> php information link on the left.

Sometimes as slow as a few times faster than an old dial up modem.
I'd check to see if it's problem on your end, I wasn't getting lightning fast speeds but they were adequate.

One thing to remember is there is a lot of overhead per file and it's a lot of files, the overhead on files a few KB's can be more than that actual data transfer. This applies to anything whether you are copying file on your computer to USB stick or it's transferring files on server. This overhead is even higher with SFTP. If you are going to transfer a lot data it's usually much quicker to compress the files into one archive before transfer, even if it were the same file size it's still going to be faster because it's one file.
At the moment, I am doing a file synchronization download as thecoalman has laid out
The script I gave for WinSCP only downloads the absolute necessary files to restore phpBB forum. If you create database backup in ACP it will download that too.** It only downloads new or changed files on the remote machine and will not delete files on local computer that have gone missing on remote machine. Additional steps should be taken to backup your backup and the reason for that is does not account for versioning. I'd suggest looking into Windows backup and Shadow copy features which can fully account for versioning.

*.htaccess is system file, make sure they exist in backup. Regardless of whether it is there or not find the option for Hide system files in WinSCP and make sure it's set to no. This file is not necessarily that important to backup as you can just use copies included with phpBB but it will help prevent accidently uploading a backup without it.

**The credentials you are using for WinSCP should work for SSH console, Downlead Putty and try to login. I know you mentioned using computers in the early years so you should feel right at home when nothing but a command prompt appears.

Export entire database, output file will not have commands for creation of the database itself:

Code: Select all

mysqldump --no-create-db -u Username -p Databasename > data.sql
Might take a bit but once the command prompt returns you should have file in the home directory, in this case should be the upper most directory you can navigate too with SFTP but I may be wrong.

Import into a database, assumes the database exists.

Code: Select all

mysql -u Username  -p -h localhost Databasename < data.sql
In your case the databasename and username are the same, you'll also need the password. It can be included in the command but I keep these commands on cheatsheet and just enter the password when prompted. The information for the database credentials can be found in phpBB's config.php file if you don't want to go digging for it in Cpanel

You can automate this by creating a cron task in cpanel to execute the mysql command but you will need to include the password and look up wht the exact syntax is, set it to run every night. Adjust the path so it's being written to specific folder. Add a new line in your Wiscp script for his folder. Create another cron task to delete files on server older than 7 days or whatever floats your boat..... automated database backups.

One complication is that the synch has frozen some elements of administrator control because the cache is captured during the operation.


The script I gave should not be downloading cache folder and not sure that would have any affect on anything anyway, but I may be wrong. If you are going to point sync at larger directories look into the file mask options. While you need that directory and the files named .htaccess and index.htm in it all other files in it should be deleted before upload if you are using it to restore. There is always copy of these in phpBB downlaod.
User avatar
Paul_Schatzkin
Site Admin
Posts: 993
Joined: Thu Jun 14, 2001 12:49 pm
Real name: aka The Perfesser
Contact:

What Happened Last Night?

Post by Paul_Schatzkin »

I'll come back later to address the question of an advisory board and a clear and reliable path to a sustainable future.

It struck me while I was reflecting on all this on my 'morning constitutional' that we are doing something quite unique in the Annals of Internet History here: I can't think of another situation where a knowledge base has been accumulated over more than two decades by an ad-hoc group with extremely limited resources at its disposal.

More typically, I imagine, those sites and operations that have been around and long as Fusor.net have considerable capital at their command. We get by on the occasional contributions of less than $1,000 year.

So it is not surprising that we would run into the kind of issues we've encountered over the past two years. We are motorcycles towing a Mack Truck.

Re: the problems encountered last night, which Frank alludes to above: yesterday I re-activated my ttbrown.com Wordpress installation - you all remember that fiasco, right? — and needed to cleanup the Wordpress installation. Before I started deleting stuff I tried to run a backup of that site. That's when we started having more issues.

First, I could not complete that backup because of (what turned out to be) available storage issues. Even though the backup would have been ~500MB and the cPanel for my GoDaddy account says we still have like 7GB available, we have since learned that, in fact, our disk is full.

And then last night my sites started going off line again, and Frank encountered issues with the operability here. I filed another Help! ticket with GoDaddy, and this morning woke up to an email basically telling us that 94% disk usage = full. They cleared some redundant files in order to restore operability, but I am going to have to spring for 'more resources' before I can do any more of what I'm wanting to do these days.

All of which underscores the opening theme here: we are trying to do something unique and valuable, and some provision must be made for it to continue in a more robust manner than has been the case for the past two years. I'm not saying we're in over our heads, but I am saying that the water is deep and we are treading on the surface, and too often we find out what sort of demons are lurking in the water below.

We need clearer pond to swim in.

--P
Paul Schatzkin, aka "The Perfesser" – Founder and Host of Fusor.net
Author of The Boy Who Invented Television: 2023 Edition – https://amz.run/6ag1
"Fusion is not 20 years in the future; it is 60 years in the past and we missed it."
User avatar
Dennis P Brown
Posts: 3159
Joined: Sun May 20, 2012 10:46 am
Real name: Dennis Brown

Re: Fusor site operation advisor board.

Post by Dennis P Brown »

When did you guys run a request for donations? Seems like a long time since I last remembered seeing one - or did I miss any recent requests?
User avatar
Paul_Schatzkin
Site Admin
Posts: 993
Joined: Thu Jun 14, 2001 12:49 pm
Real name: aka The Perfesser
Contact:

Re: Fusor site operation advisor board.

Post by Paul_Schatzkin »

Matt_Gibson wrote: Fri Feb 18, 2022 3:11 am Could any of these problems be solved by a paid upgrade of some sort? If so, I’m sure that many of us would kick in to help…

-Matt
The simple answer is: yes.

The more difficult part is that a lot of these components are interwoven on the server, as The Coalman alludes to above. It's like having different houses built on one foundation.

What would have been nice to know a month ago is GoDaddy in fact keeps backups for 30 days. About 2-1/2 weeks ago an officious GoDaddy rep in the Phillipines (yes, I am being Xenophobic here) told us they only have backups for like two days - but that didn't matter because whatever level of service we're paying for didn't entitle us to a server-level backup and or restore.

Wednesday I spent 2-1/2 hours on the horn with a GoDaddy rep in Iowa who informed me that "we have backups going back 30 days." That was very useful information to learn on what amounted to... wait for it... day 31.

So to say that our host has not served us admirably would be a bit of an understatement at this point.

But I digress (because I. Am. Pissed. Off....)

During that 2-1/2 hour support call on Wednesday I discussed two options.

I would like it a lot if Fusor.net could be all alone in its own hosting account, with none of my other activities under the same umbrella / in the same account / occupying the same server space. This is probably the least practical option because it would involve duplicate expense and, well, something would have to be moved and Lord Knows we are NOT 'moving' this site again.

The second option, as Matt suggests, is buying 'more resources' for the current account. We should really need more resources. We have ~100GB of disk storage in this account, but right now we have THREE iterations of fusor.net in the account, and accounts for 60-70GB of our assigned storage. Last night we hit the wall when I just tried to do a 500MB backup of one of my other sites.

I am trying to reach the woman in Iowa that I spoke to on Wednesday about upgrading the account, but it's frustrating when you get a good agent, she gives you her direct contact info... and then doesn't respond. So let's see what happens today.

At least as near as Frank or I can tell, what vestige of full operability that got compromised last night has been restored by clearing some disk space. Now some provision must be made to obtain still more.

Does that answer your question, Matt?

--PS
Paul Schatzkin, aka "The Perfesser" – Founder and Host of Fusor.net
Author of The Boy Who Invented Television: 2023 Edition – https://amz.run/6ag1
"Fusion is not 20 years in the future; it is 60 years in the past and we missed it."
User avatar
Paul_Schatzkin
Site Admin
Posts: 993
Joined: Thu Jun 14, 2001 12:49 pm
Real name: aka The Perfesser
Contact:

Re: Fusor site operation advisor board.

Post by Paul_Schatzkin »

Dennis P Brown wrote: Fri Feb 18, 2022 10:35 am When did you guys run a request for donations? Seems like a long time since I last remembered seeing one - or did I miss any recent requests?
The last time we solicited funds was two years ago when we moved from Media Temple to GoDaddy.

That time is coming again soon.

I went to the bank yesterday and set up a new account so that funds for Fusor.net will not be co-mingled with my other personal and business accounts. I am going to set up a new PayPal account JUST for Fusor.net so that contributions will go into that account.

Or, I dunno, maybe I could set up a 'GoFundMe'? I'll have to look into that option, too.

Thank of course, there's that $150-million I'm trying to raise via Kickstarter.... 🤪

--P
Paul Schatzkin, aka "The Perfesser" – Founder and Host of Fusor.net
Author of The Boy Who Invented Television: 2023 Edition – https://amz.run/6ag1
"Fusion is not 20 years in the future; it is 60 years in the past and we missed it."
Matt_Gibson
Posts: 505
Joined: Thu Sep 23, 2021 10:36 am
Real name: Matt Gibson

Re: Fusor site operation advisor board.

Post by Matt_Gibson »

It definitely sounds like it’s upgrade time then. Anyone know anyone from ITER? :-)

Let us know once something is setup, I’d think there would be a lot of support.

-Matt
Jon Rosenstiel
Posts: 1494
Joined: Thu Jun 28, 2001 1:30 am
Real name: Jon Rosenstiel
Location: Southern California

Re: Fusor site operation advisor board.

Post by Jon Rosenstiel »

FWIW… Last night I was attempting to restore the missing images in my bio. After successfully uploading two images I realized I’d done it in the wrong order so I deleted the images and started over. Then things went south and all I got were the following error messages.

Jon Rosenstiel
Attachments
Error msg 1.jpg
Error msg 2.jpg
Frank Sanns
Site Admin
Posts: 2119
Joined: Fri Jun 14, 2002 2:26 pm
Real name: Frank Sanns

Re: Fusor site operation advisor board.

Post by Frank Sanns »

Jon Rosenstiel wrote: Fri Feb 18, 2022 12:58 pm FWIW… Last night I was attempting to restore the missing images in my bio. After successfully uploading two images I realized I’d done it in the wrong order so I deleted the images and started over. Then things went south and all I got were the following error messages.

Jon Rosenstiel
Attachments
TascamFront.jpg
Achiever's madness; when enough is still not enough. ---FS
We have to stop looking at the world through our physical eyes. The universe is NOT what we see. It is the quantum world that is real. The rest is just an electron illusion. ---FS
User avatar
Richard Hull
Moderator
Posts: 14991
Joined: Fri Jun 15, 2001 9:44 am
Real name: Richard Hull

Re: Fusor site operation advisor board.

Post by Richard Hull »

Jon, this happened to me last night while restoring images in My FAQs. I contacted Frank and he noted this issue is due to updating off fusor.net. We will just have to wait until the image editing/restoration in old post works again. Oh well.

Richard Hull
Progress may have been a good thing once, but it just went on too long. - Yogi Berra
Fusion is the energy of the future....and it always will be
The more complex the idea put forward by the poor amateur, the more likely it will never see embodiment
Rex Allers
Posts: 570
Joined: Sun Dec 30, 2012 3:39 am
Real name:
Location: San Jose CA

Re: Fusor site operation advisor board.

Post by Rex Allers »

Frank,

I just wanted to let you know that I have been, in the background, reading this thread.

I saw your statement,
"Rex Allers, Here is your chance to have control over the site."

I have been thinking about how to best reply, but haven't got there yet.
Rex Allers
User avatar
Paul_Schatzkin
Site Admin
Posts: 993
Joined: Thu Jun 14, 2001 12:49 pm
Real name: aka The Perfesser
Contact:

Re: Fusor site operation advisor board.

Post by Paul_Schatzkin »

Richard Hull wrote: Fri Feb 18, 2022 3:41 pm Jon, this happened to me last night while restoring images in My FAQs. I contacted Frank and he noted this issue is due to updating off fusor.net. We will just have to wait until the image editing/restoration in old post works again. Oh well.

Richard Hull
Richard, last night's disruption was on me, though, again, unknowingly.

I'd tried to make a backup of one of my other sites (ttbrown may yet live again...) — at which point we learned that 85GB of 95GB actually equals 100% of our disk allowance. Again, who knew?

GoDaddy cleared out enough of the detritus to get everything here back to what passes for normal these days. This afternoon I purchased another 100GB of storage so that we can have the overhead we need to continue.

I know this has all been beyond frustrating. I about blew a gasket on Wednesday when I learned that what we'd been told back in January re: a useful backup was false, that GoDaddy has backups going back 30 days and... I learned all this ONE F'ING DAY too late.

Which is just how it's been this year. Weird f'ing sh*t.

Today I learned what it will take to firm our foundation. It's going to get pricey. I'll have more to say soon.

I'd say "hang in there" but, jeezus, everybody has already done enough of that.

--PS
Paul Schatzkin, aka "The Perfesser" – Founder and Host of Fusor.net
Author of The Boy Who Invented Television: 2023 Edition – https://amz.run/6ag1
"Fusion is not 20 years in the future; it is 60 years in the past and we missed it."
User avatar
Richard Hull
Moderator
Posts: 14991
Joined: Fri Jun 15, 2001 9:44 am
Real name: Richard Hull

Re: Fusor site operation advisor board.

Post by Richard Hull »

I've been here since the beginning of songs, so I am willing to hang in there.

Richard Hull
Progress may have been a good thing once, but it just went on too long. - Yogi Berra
Fusion is the energy of the future....and it always will be
The more complex the idea put forward by the poor amateur, the more likely it will never see embodiment
thecoalman
Posts: 189
Joined: Sun Feb 06, 2022 1:26 pm
Real name:

Re: Fusor site operation advisor board.

Post by thecoalman »

Paul, thanks for the book, I'll be sure to read it and I'm not just saying that either. I used to devour books when I was younger into my twenties, haven't read an actual book in while.

A managed VPS starts in the $50/month range including WHM/Cpanel, WHM is the panel above Cpanel. This is limited to 5 Cpanel accounts for that price but I believe each Cpanel account can have unlimited domains. It's a step up over Cpanel but the benefit is you get the the keys to the kingdom, the downside is you have the leys to the kingdom. That 1 hour conversation with tech support for example could of been solved in about 5 minutes with my directions if you had access to the WHM panel.

The host and WHM can pretty much handle itself as far as upgrading software, that's the purpose along with many configurations. You are not bound to it or the host, you have complete root access. I haven't used it but there is backup feature that would include backing up the WHM panel itself which makes it portable between hosts. These backups can be stored on AWS or other services pretty cheaply.

My suggestion is to look into it, move one simple domain for a month and see if it works out before jumping into anything.
User avatar
Richard Hull
Moderator
Posts: 14991
Joined: Fri Jun 15, 2001 9:44 am
Real name: Richard Hull

Re: Fusor site operation advisor board.

Post by Richard Hull »

The edit and restore image function is working again. Thanks to whoever got it going again. I have finished the first page of many lost images in the neutron Radiation FAQs forum.

Richard Hull
Progress may have been a good thing once, but it just went on too long. - Yogi Berra
Fusion is the energy of the future....and it always will be
The more complex the idea put forward by the poor amateur, the more likely it will never see embodiment
Post Reply

Return to “Announcements and Site Administration Topics”