Page 1 of 2

web site problem

Posted: Fri Mar 22, 2019 6:38 pm
by Dan Knapp
Earlier today the forums site was telling me it was not working due to "too much activity." Later, I was able to get in, but only after logging in on the forum. My previous bookmark returns the message "you do not have authorization to search" and it won't let me connect to "active topics" unless I log in. Is there a problem with the site today?

Re: web site problem

Posted: Fri Mar 22, 2019 7:38 pm
by Frank Sanns
The site was offline due to high volumes being driven here by search engines. It was due to extensive promoting of a member of the forum and it cost Paul $$$$ and forced the rest of the members to be offline for a day and a half.

Paul Shatzkin said in another recent post, "The discussion reminded me of something that Carrie Fisher said shortly before she died, when she was getting trolled on the Interwebs about her appearance in the recent Star Wars films:
"Youth and beauty are not accomplishments."
By a similar token, the fact that somebody in a single digit or low-double-digit age bracket has done something that many others have already done is really not so much an "accomplishment" as it is a "novelty" –– and that is not why this site was started"

This unabated pressure on our site has been going on for around a month now with spikes coming the past few days. It has forced us to make some changes with the site with more coming. The changes that you will see immediately are:

1. Not all categories are visible without being signed in.
2. Search functions and other parsing functions are not accessible without being signed it
3. Ages are being taken out of fusioneer records
4. Bots will now skip over portions of the site as well as some threads and some categories/words.
5. Member lists are no longer visible unless signed in

All of these actions have been precipitated by needs to protect the site from abuses coming from a few sources.

There will be more locking down of the site in the coming days but for the regulars, it is about keeping the site up and fully accessible to those using it. Nothing is being held back on the site for its members. It will just require a login for all features and technical information.

Re: web site problem

Posted: Fri Mar 22, 2019 11:17 pm
by Bob Reite
I guess that you had to alter robots.txt to restrict some of the site. I would not think it easy to do with phpbb

Re: web site problem

Posted: Sat Mar 23, 2019 2:21 am
by Frank Sanns
Robot.text is borderline useless for a variety of reasons. For one, search engines can ignore them and crawlers/spiders can follow links from elsewhere.

The mods are done within the phpbb framework where entire areas can be sectioned off from outside. Even login in only allows access to areas where permissions are specifically granted. This completely excludes areas of the site from any outside access.

Re: web site problem

Posted: Sat Mar 23, 2019 4:42 pm
by Andrew Seltzman
I would like to weigh in strongly against #1
"1. Not all categories are visible without being signed in."
and suggest that for #2:
"2. Search functions and other parsing functions are not accessible without being signed it"
should be modified to allow users that are not signed in to search after solving a kaptcha or other suitable anti-bot method

Throughout the history of we have always promoted the open sharing of scientific knowledge accessible to the general public, exemplified by the text "Open Source Fusor Research Consortium" in our main banner headline. I would like to emphasize that the commitment to "open source" should include maintaining open access to the forum posts and search functions for non-members who are visiting or just browsing by. When I first found, I was browsing topics as a non-member for about half a year before I began to post. Lets be honest, when you run into a paywall or "membership requirement" while browsing on the internet, you just click off and go somewhere else. Sure, joining is free and easy, but the "activation energy", so to speak, to join a new forum may be higher than we would like and discourage potential new members from browsing topics they might not even have known they were interested in, and now can not see because of the membership requirement. Certainly with changing times, more bots, and bursts of traffic, occasional traffic problems with the site occur, but these are very occasional and do not seem to be a routine occurrence. Yes, there is some media excitement that generated a burst of traffic, but this to shall pass, traffic will return to normal, and the additional interest in the forum may help bring in new members.

3 through 5 are really great ideas.

Re: web site problem

Posted: Sat Mar 23, 2019 7:31 pm
by Frank Sanns
Agreed Andrew.

For everybody's knowledge, NONE of the core technical categories or FAQ are restricted in any way. They are what this forum is about.

The categories that are not visible by not being signed in are "noise" categories that were dozens or more pages of posts in each category. While they are part of the historical association with the site, they are not where the meat is. Categories not visible to guest are archived announcements, introduction and chat area, and the advanced discussion area. In addition, these forums have been pruned as they used to be and that has always been mentioned in the headings of the categories. It was more about housekeeping and keeping a bunch of hits coming to the site which are not about the core function. Getting rid of bunches of extraneous search terms from the site was the goal, not censorship.

Lastly is the search function for guests and bots. That is a tough one from a technical perspective. How to allow a legitimate guest to search the site without allowing access to the omitted forums and archives? I will continue to look into that but with the setup the way it is, either a password is required or it is pretty much open to anybody and anybot that may happen by. The anti spambot Captcha just do not work no matter what you might read. We tried all sorts of them but had entry to the site on many, many occasions. it was only the scrambled registration phrase that was designed to misdirect any bot to give the wrong answer that has saved the site from being over run.

Thanks for your input though and hope others will chime in on how to maintain the original mission without the noise and costs.

Re: web site problem

Posted: Sat Mar 23, 2019 8:16 pm
by Harald_Consul
We had a similar "search engine problem" with the german statistics forum.

The aggressive searching has stopped, as we agreeed to the terms of the domestic intelligence service, that certain parts of the forum should not be open to the public without login. (They wanna trace certain request back.)

Do not know, if it is the same, here.

Re: web site problem

Posted: Sun Mar 24, 2019 3:13 am
by Andrew Seltzman
Could we modify the "search restrictions" to allow viewing of "active topics" without having to log in?

Re: web site problem

Posted: Sun Mar 24, 2019 3:27 am
by John Futter
that will not help the traffic problem
problem is hits on this site followed by further hits (searches and or forum reads)

Re: web site problem

Posted: Sun Mar 24, 2019 4:20 am
by Andrew Seltzman
Checking active topics via the "search.php?search_id=active_topics" link is the way I (and probably others) check for new posts on the forum. Access to this link is now disabled unless you are logged in.

Also keeps logging me out automatically now after some period of time. Do we have a setting for auto logout time?