Can we fix search so it actually works?

Announcements and items of immediate importance.
Post Reply
JoeBallantyne
Posts: 313
Joined: Tue Jul 20, 2010 4:08 pm
Real name: Joe Ballantyne
Location: Redmond, WA

Can we fix search so it actually works?

Post by JoeBallantyne »

So, I tried to do a search on grid wire diameter, and it was immediately rejected because the words grid and diameter were "too common".

What is the point of a search function on the forums site if it is useless?

It is kind of pathetic that I have to do searches of fusor.net by using google, and targeting the google search to site:fusor.net.

Please, can someone remove this "too common a word" limitation from the built in search?

Or at least fix it so the common words rejected from a search aren't actually relevant to fusors in any way?

The search also rejects names, so you can't search for Jon for example. Names should also not be in the list of too common words.

Thanks.

Joe.
Frank Sanns
Site Admin
Posts: 2117
Joined: Fri Jun 14, 2002 2:26 pm
Real name: Frank Sanns

Re: Can we fix search so it actually works?

Post by Frank Sanns »

During the migration, some settings were unintentinally altered.

For the longest time, the old forum search function did not work very well either. I made some changes that seemed to work well so I have updated the the values on this site too.

Unfortunately, short strings of 2 alpha numerics and less are no longer searchable. The search function returns too many results with very short strings.

Other than that, the search should be working much better now.
Achiever's madness; when enough is still not enough. ---FS
We have to stop looking at the world through our physical eyes. The universe is NOT what we see. It is the quantum world that is real. The rest is just an electron illusion. ---FS
vlad_klanoff
Posts: 27
Joined: Sun Nov 17, 2013 4:32 am
Real name: Vlad Klanoff

Re: Can we fix search so it actually works?

Post by vlad_klanoff »

I tried to search for a grid construction, but it doesn’t work because “grid is too common word”. I think the same happend with all other words I tried to search for, so I just gave up trying…
JoeBallantyne
Posts: 313
Joined: Tue Jul 20, 2010 4:08 pm
Real name: Joe Ballantyne
Location: Redmond, WA

Re: Can we fix search so it actually works?

Post by JoeBallantyne »

So Frank, after seeing your message that things should work better, I tried again.

grid wire diameter gives the same error. Too common a word.

grid by itself, same result

wire by itself, same result

diameter by itself, same result.

Doesn't seeem like anything has changed at all.

How is the search function deciding if a word is too common?

Joe.
Frank Sanns
Site Admin
Posts: 2117
Joined: Fri Jun 14, 2002 2:26 pm
Real name: Frank Sanns

Re: Can we fix search so it actually works?

Post by Frank Sanns »

I will solicit your help Joe and anybody else on the forum. I have totally eliminated the common words. Deleting the old index and reindexing did absolutely nothing but not allow for any words to be searched. Suggestions instead of complaints?
Achiever's madness; when enough is still not enough. ---FS
We have to stop looking at the world through our physical eyes. The universe is NOT what we see. It is the quantum world that is real. The rest is just an electron illusion. ---FS
Jon Rosenstiel
Posts: 1494
Joined: Thu Jun 28, 2001 1:30 am
Real name: Jon Rosenstiel
Location: Southern California

Re: Can we fix search so it actually works?

Post by Jon Rosenstiel »

Replace the forum’s search box with a Google search box?

Jon Rosenstiel
Frank Sanns
Site Admin
Posts: 2117
Joined: Fri Jun 14, 2002 2:26 pm
Real name: Frank Sanns

Re: Can we fix search so it actually works?

Post by Frank Sanns »

I have made some progress on this. The common word screening has not been eliminated. All alpha numerics that are size 3 to 20 can be searched.

Unfortunately, this can make a search return hundreds of pages of results. This is no more helpful than none.

The settings now will give one page of results. The string length is now only 100 characters so the page will have more results.

Use the advanced search to find what you want. That is, pick the forum that you want to search rather than tossing out a full forum search. Unless of course you use the boolean operators as they were meant to be used. This should give a meaningful result to your searches.

There is also a time window that can be used to find posts.

Using the search engine properly should give all of the results needed.

If the masses do not like this approach, I can take Jon's suggestion (thank you Jon) and put a Google search up to replace the phpbb search function.
Achiever's madness; when enough is still not enough. ---FS
We have to stop looking at the world through our physical eyes. The universe is NOT what we see. It is the quantum world that is real. The rest is just an electron illusion. ---FS
JoeBallantyne
Posts: 313
Joined: Tue Jul 20, 2010 4:08 pm
Real name: Joe Ballantyne
Location: Redmond, WA

Re: Can we fix search so it actually works?

Post by JoeBallantyne »

So now I do indeed get exactly 1 page of results for my grid wire diameter search which is infinitely better than the previous error message.

Could you perhaps up the max number of pages of results returned to 5, or maybe 10.

I fully understand that you don't want the search to give back hundreds or thousands of pages of results, but 10 would be an order of magnitude better than 1. :-)

I think there are typically 10 or so posts per page, so 10 pages means 100 posts returned max.

Which IMO is reasonable.

If one doesn't find the post they are seeking in those 10 pages one can always add more words to the search to reduce the number of results.

Thanks for making this better Frank.

Much appreciated.

Joe.
JoeBallantyne
Posts: 313
Joined: Tue Jul 20, 2010 4:08 pm
Real name: Joe Ballantyne
Location: Redmond, WA

Re: Can we fix search so it actually works?

Post by JoeBallantyne »

I think I spoke too soon.

Looks like now when I try to see active topics, or do anything else that I guess was using the search function, it only puts 1 post per result.

Before active topics - which I use all the time, would have a whole list of posts. Not 1 per page of results.

So I think your change Frank, broke how the Active Topics search works.

Joe.
JoeBallantyne
Posts: 313
Joined: Tue Jul 20, 2010 4:08 pm
Real name: Joe Ballantyne
Location: Redmond, WA

Re: Can we fix search so it actually works?

Post by JoeBallantyne »

After I logged out and logged back in, and made the previous post, now Active Topics is working again.

It was almost as if doing an active topic search after searching for particular words, was remembering the words from the previous search, and only showing me active topics with those words in it.

It was very strange.

Joe.
User avatar
Richard Hull
Moderator
Posts: 14976
Joined: Fri Jun 15, 2001 9:44 am
Real name: Richard Hull

Re: Can we fix search so it actually works?

Post by Richard Hull »

I called Frank related to the above issues. He has been working on the search engine and occasionally has to monkey with the site in a manner that might tend to freak some of us out. Never fear, he is just working to try and make folks happy on this issue. Note he may not be so successful that it will be a marvel or even be pleasing to some. Just be aware there is a man at work on the search engine and that sometimes re-indexing takes time.
Give the poor guy a break. Remember, he is getting no money for all the heartache and heartburn in doing what he does for us here.

Honestly, for me, the search engine was crap, is crap and might always be crap. I would have said...live with it as is or "let them eat cake". Now, aren't you glad I am not the lord and master here ,and am instead, just the garbage man?

Hey, that is what 76 years of listening to what I now view as whining does. I might use this search feature once a year and it never fails to disappoint. I am blessedly spared worrying about it as life is getting shorter and I got stuff to do that is easy and fun. Farting with a nasty search engine is not one of them.

Richard Hull
Progress may have been a good thing once, but it just went on too long. - Yogi Berra
Fusion is the energy of the future....and it always will be
The more complex the idea put forward by the poor amateur, the more likely it will never see embodiment
Frank Sanns
Site Admin
Posts: 2117
Joined: Fri Jun 14, 2002 2:26 pm
Real name: Frank Sanns

Re: Can we fix search so it actually works?

Post by Frank Sanns »

Thanks for the call and the post Richard.

Yes, I was working on the site. I hate to take it off line but I hate not too. Either way, emails flood my inbox. While this is good that people are responsive to problems on the site, it can be frustrating when I am just trying to get something done without disrupting things. I apologize for the short interruptions during one of these changes.

Sooooo, here is the report. The current version of phpbb allows for global format of the number of topics, currently set to 25, and posts per page, also previously set at 25. Looking at the board then shows the top 25 topics once you are in a forum heading like Announcements and Admin section or any other. At the bottom will be additional pages for previous topics. No problem.

A similar result occurs when a topic is clicked on. This is seen with posts that have 25 or more responses. There will be numbered pages to see additional responses. There is no number or time limit for this to occur and it is the way the site has been for decades.

Changing either the number of Topics per page or Posts per page will change it globally for all forums and all posts. All is good in the world. Well not quite.

It seems that this version of phpbb does this consistently and globally very well except for one thing. IT DOESN'T ALLOW ADDITIONAL PAGES when using the search function! That is right. Shrinking the number of posts or topics IS applied to the search results so phpbb is going it globally and applying it also to the search function HOWEVER, when increasing the number of posts, it truncates to only what fits on a page. No additional page numbers appear like in the Topics and Posts. This is very strange behavior but I looked in on one other site that is not related to ours and he exact same thing is happening so it is not just us or something that I have missed.

What I have done is to increase the number of posts allowed on a page to 150 and I have reduced the word count of the search finding down to get as many on a page in a small area as possible. This will require some more scrolling on ALL of the posts. I am not sure if this is a problem or not since all of the information is there but there will often not be additional pages to click on. If 150 is causing endless scrolling for everybody, I can reduce this number so it is tolerable day to day but still gives many search results.

On the search results, the limit is the NUMBER of posts listed and not the date. I just did some boolean searches that go all of the way to day one of the forum so all posts are being picked up. The common words are now essentially eliminated because we have some words that are short and common but are pertinent not to a stand alone search but to a boolean search.

I cannot stress enough the value in a boolean search. Gone are the ampersands but they are replaced with the plus sign now. Ok, progress? Syntax but important to find what you want. USE THEM. If you are searching, add words to limit your searches if more than the page limit of 150 come up. If you are searching that deep and overflowing, you probably should stop and go read the FAQ as your search is too general.

Searches can also be limited to a particular forum. This greatly and appropriately reduces the search results. Looking for ion gun information in the Admin area or the trading post or new user chat does not make much sense. There is much noise in some of these areas so focus your search.

You can also search by member. Just put in their ID and you are on your way. You can also limit the time if you do not want to go back so far but it is not always useful as posts come back in ascending or depending chronological order already so you really do not gain anything.

Let's see how the forum is operating with the longer pages. Feedback is welcome complaints, not so much.
Achiever's madness; when enough is still not enough. ---FS
We have to stop looking at the world through our physical eyes. The universe is NOT what we see. It is the quantum world that is real. The rest is just an electron illusion. ---FS
Jon Rosenstiel
Posts: 1494
Joined: Thu Jun 28, 2001 1:30 am
Real name: Jon Rosenstiel
Location: Southern California

Re: Can we fix search so it actually works?

Post by Jon Rosenstiel »

Frank,

Search function works great, thank you.

Jon Rosenstiel
User avatar
Richard Hull
Moderator
Posts: 14976
Joined: Fri Jun 15, 2001 9:44 am
Real name: Richard Hull

Re: Can we fix search so it actually works?

Post by Richard Hull »

Works OK. NIce. I may start actively using it again, though I rarely ever needed it. Good Job Frank!

Richard Hull
Progress may have been a good thing once, but it just went on too long. - Yogi Berra
Fusion is the energy of the future....and it always will be
The more complex the idea put forward by the poor amateur, the more likely it will never see embodiment
JoeBallantyne
Posts: 313
Joined: Tue Jul 20, 2010 4:08 pm
Real name: Joe Ballantyne
Location: Redmond, WA

Re: Can we fix search so it actually works?

Post by JoeBallantyne »

Thanks Frank. Tried it today, and search now works well. And I can search, and then use quick links to get active topics, and get the full typical list.

Was able to use search yesterday to track down the post where Jon had pictures of his cube fusor feedthrough with the silicone o-rings on it, that allowed use of a PVC pipe w/mineral oil to raise the holdoff of a standard 30kV feedthrough up to 70kV or more.

Search is now useful.

Thanks again.

Joe.
thecoalman
Posts: 189
Joined: Sun Feb 06, 2022 1:26 pm
Real name:

Re: Can we fix search so it actually works?

Post by thecoalman »

Richard Hull wrote: Wed Jun 08, 2022 6:25 pm Honestly, for me, the search engine was crap, is crap and might always be crap.


Search on phpBB is "dumb" as it is on any PHP application, it just returns all results for what you are searching for. It can be very useful for very specific search criteria using the advanced search, e.g. by author which is one I'll use often. Another is limiting it to the first post in topic so you get a list of many topics. This is actually the default on my own forum but it requires some edits to files. Those are things Google can't do precisely.

search.php


-------------------------


The search settings in the ACP are there to limit the size of the search tables which can be as large as the posts table using the default settings. Lowering the limits can make them very large but you should have sufficient resources. Personally I never liked the "common word" setting, phpBB in te past had a stopword filter instead: "and", "to", "the" etc." phpBB is multi lingual so you would need stopwords filter for each language which is probably why it was changed.

Since you are now on VPS I could install Sphinx for you which uses stop words filter instead. I'd have to check to see if you have sufficient RAM. It is also lighting quick with results.

-----------------------------------
Just a heads up but.....

New topics - based on the sessions length which is currently set to 3600 seconds or 1 hour. See that, I can do math too. :) That list will remain active until you don't click on something for one hour. After that it resets and will only display topics/replies since your last click.

Active topics - Hardcoded to 7 days for all active topics How many will appear depends on activity over the past 7 days The hardcoding can easily be changed but it needs to be maintained through updates.

Jon Rosenstiel wrote: Wed Jun 08, 2022 10:23 am Replace the forum’s search box with a Google search box?

Jon Rosenstiel
You can do this manually with:

Code: Select all

site:fusor.net Jon Rosenstiel
https://www.google.com/search?q=site%3A ... Rosenstiel
Post Reply

Return to “Announcements and Site Administration Topics”