I want to help fix fusor.net!! (join up listing)
Posted: Fri Nov 08, 2019 11:44 am
This is a repository of the names of those willing to help repair their old posts by replacing or updating their missing attachments.
Here is how it is done once you have privileges granted.
Go to your damaged report. click on the "pencil icon" to edit your post. Scroll all the way down to the Attachments tab and click on "add files". Have you new or original attachments in mind. now, delete all the old files. Now re-click on "add files" a blank line will open and now start to re-insert you new images and attachments one by one. once done, click submit above and your old post will have the new images attached.
If you have the guts and verve to wish to repair some of you old post's attachments that have gone missing just reply and simple state "Billy Jones wants editing privileges" and close it out.
In this matter, we will have a list of folks who stand ready to help out should the admins desire to give you the editing privilege.
NOTE: All of the powers that be here take the editing privilege as a sacred gift.
If you should be granted this gift, do not abuse it! Never ever delete one of your own posts. Feel free to correct spelling, punctuation or to add text to make the post more clear in your second pass.
Remember, your main mission is to delete the old, now failed, attachments and re-install your images or any new ones that you feel would further enhance your post.
To join this group of helpers, Just reply to this message and type one line.....
I, John Smith want my personal editing privileges restored
Re: I want to help fix fusor.net!! (join up listing)
Posted: Fri Nov 08, 2019 8:16 pm
If anybody wants to sign up for this mission, great, we might still need some help but first...
For once we've got some good news about all this: after about 2 hours of digital spelunking this afternoon, we have finally found a trove of image files from one of the earlier iterations of the fusor.net site.
We still haven't figured out precisely how we are going to reconcile what we've found with the database that delivers images to the posts, but at least we have a lot of previously-thought-to-be-lost images to work with.
If it does come down to the kind of manual, post-by-post effort that Richard is anticipating, we'll need all the help we can get.
Re: I want to help fix fusor.net!! (join up listing)
Posted: Sun Dec 08, 2019 3:35 am
I was worried too about all the random repair of missing files. Having only normal user privileges, I have been browsing around the forum and as of yesterday I think I understand what the problem is. I no longer have very much hope of an automated fix, although it would be possible with a good backup from before the site move.
A while back I had noticed that it seemed to be related to certain users. After finding a post with a missing attachment file, I could look at the profile for that user and get the list of all their posts. Randomly browsing through their messages, I never found any post they made that had an intact attachment; any attachment files they had uploaded were all missing.
So I concluded that the UserID number of the poster was a key factor. My instincts had me thinking that there should be a consecutive block of UserID numbers with the problem. What I found didn't seem to fit that conjecture. One member with the problem had recently joined, Joe Gayo (4450). Another user was a very early member, Richard Hull (58). Many users with ID numbers between had posts with no problems. I couldn't see a pattern.
Yesterday, Andrew Robinson provided a post where he gave a spreadsheet list of the files that were missing and some other details.
Re: Senior users, Admins and Moderator - help us make repairs
Andrew Robinson » Thu Dec 05, 2019 7:12 pm
I sorted using column A of that list (the attachment filenames as stored for the BB use) and looking at that, finally the pattern hit me. The current user IDs can be 2, 3, or 4 digits long. The members with the problem have UserIDs beginning with 4 or greater. So, UserIDs in these ranges:
40 to 99,
400 to 999,
4000 to 9999 (but the current highest ID is in the 6000s.)
I'm not sure if 40 or 400 or 4000 is the exact crossover boundary for UserIDs with the problem, but it should be very close. (Maybe it could be something like 41, 410, 4100.)
I'll follow this post with another giving the list of UserIDs having missing files that I have extracted from Andrew R's list of missing files. Although maybe not exactly right as things get fixed, there are 167 members on that list with 3595 missing files.
For some of the UserIDs in my list I looked up and added the associated username. For that I was using a manual method and gave up on trying to do most of the list. The are ways a normal user can find their own UserID or look up the profile of a member using their UserID number. I'll try to add another post describing that.
--- What happened and how did it happen? ---
I don't know if we will ever be sure but I have a plausible explanation. First I'll provide some descriptions of how attachments work on a phpBB forum. I'm not an expert but after some research, I think this is accurate enough for our purposes.
Attachments in a phpBB forum
Each phpBB setup has an associated database. The database contains some configuration info and also holds the text content and associated information for each post made on the forum (poster, text of the message, location of the post in the site structure, time/date, etc.)
When a member makes a post they have the option to upload one or more attachment files to include in the message. Usually this file will be a picture (jpg, png, ...) but could also be something else like a document (pdf, doc, ...). When a file is uploaded, the post part of the database records that there is an attachment. Other parts of the database record info about the uploaded file (original filename, poster, date/time, size, ...).
All uploaded files for all users are saved in a directory. The default for this directory's name is '/files'. Before a new file is saved, its filename is changed. The new filename has two parts separated by an underscore character. The beginning of the new name is the numeric UserID of the poster, then the underscore, then a long generated hexadecimal number to make this name unique on the site.
Here's an example of one generated filename:
This was uploaded as an attachment by Richard Hull (UserID = 58). From Andrew R's spreadsheet list, it originally was the file '13.jpg'.
Here's another one uploaded by Tyler Meagher:
Tyler has a much higher UserID starting this filename (4251).
These renamed attachment files are all stored in the '/files' directory. If you sort their names in alpha-numeric order the files beginning with 2n will be close in the list to 2nn or 2nnn.
Who can say for sure? My speculation is that the backup of the files directory from the old site when preparing for the move, was not complete. For whatever reason, the symptoms of the problem seem to indicate that all the filenames beginning with 4 or greater did not make it into the /files directory on the new site. Because the beginning of the filenames is the UserID of the poster, it has caused all the attachments by certain users to be lost.
After the site move, any user making a new post with an attachment will have the process work properly with a new attachment file added in the /file directory. If an old message is edited with a new uploaded file to replace a missing one, that should (and seems to) work correctly to add a new file in the /files directory.
Could there be a bulk fix for this problem? I think yes, if there is a backup saved somewhere of the phpBB forum on the old site that contains the /files directory. The backed up files could be sorted alpha-numerically and the files beginning with 4 or greater copied into the present site /files directory.
Would that bulk copy hurt any of the repair work that has already been done? I doubt it. Each of the new files from making changes since the move would have a new unique filename generated for it. The database entries for the post should now point to this new filename. So if a group of old files was merged into the current /files dir I would expect any missing files that have already be fixed to have a name that is no longer pointed to in the database, so the added old files would just use a bit of extra space in /files without breaking any work already done to replace them.
I'm not directly involved with any of the processes related to maintaining the site, nor do I even have any privileges beyond a basic member. I would have hoped there would be multiple backups of the old site saved somewhere. From the most recent good one, many of the files could be extracted and merged. Even a very old backup should be able to provide many of the missing files. From reading the many messages related to trying to understand and repair the problem, I am somewhat shocked and disappointed that I have not heard any indication that such backups exist.
If no such backup exists to get some of the missing files, the only hope is to manually replace missing files one at a time. Some of that can be done and any repair will help. I gather that there are around 3500 attachment files that have been lost. I would expect most of them may never be recovered or replaced. Some members who posted are no longer around to even try to remember the missing content and for many attachments I'd be surprised if the original content still exists somewhere.
PS for Andrew Seltzman,
Your user ID is 344 so you shouldn't have any posts that need fixing.
Re: I want to help fix fusor.net!! (join up listing)
Posted: Sun Dec 08, 2019 3:37 am
Using the spreadsheet list that Andrew Robinson created and shared, I was able to process and create a list of the UserIDs of members who have one or more missing attachment files in their posts.
For a small number of these I looked up the user's name and added it after the ID number. I was using a slow manual process to get the names, so I gave up after a while.
I hope members will look up their own UserID number and check if they are in this list. If you are in the list it means that one or more post you made has lost an attachment file you uploaded and could benefit if you can edit and replace any missing files.
I'll add another follow-up post describing how any member can get their UserID number.