Allowed memory size exhausted fatal error on Low Search
Hi there,
I'm running Low search (v3.2.0) on an ee 2.7.3 site and I am getting a fatal error running out of memory. My memory limit is set to 128mb, so I would have thought this would be enough? My search term is 4 words, and this is the error I'm getting:
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 7089 bytes) in /Sites/emc/ee2/codeigniter/system/database/drivers/mysql/mysql_result.php on line 147
I also get the error with a 3 word search, but 2 words is fine. All collection indexes are up to date.
Hope you can help!
Replies
Low 2 Oct 2014 16:18
Not much LS itself can do about that, I'm afraid. It can be due to the amount of entries the keyword search returns, specifically when performing a fallback search. But since the error points to a MySQL result set, it seems it would be a query result. Can you double up that 128 to 256?
Joe Beaver 2 Oct 2014 16:29
Hi Low,
Thought this might be the case. I just upped my memory limit to 256 and no more fatal error, but now i just get a blank white screen :(
Low 2 Oct 2014 21:39
Hmm. Still not a lot LS can do about it. Maybe check with your host if the memory can be increased?
Joe Beaver 3 Oct 2014 08:35
Hi Low,
This was actually local. Upped my memory to 1G and that worked so will have to make a judgement call on how high to go in production ( not that high! ). I've got a further question, I hope you don't mind:
My search query here as I said is 4 words. This is bringing back many more results than expected. My search term is 'studying othello with dvd'. I have an entry that has a title of 'Studying Othello with DVD' and one with the title 'Studying Othello DVD'. Strangely 'Studying Othello DVD' is the first result, and 'Studying Othello with DVD' is the 11th.
Do you know what might be going on here, and how I might go about getting more relevant results?
My form tag:
{exp:low_search:form result_page="search" form_id="desktop-search" required="keywords"}
...
{/exp:low_search:form}
My results tag:
{exp:low_search:results limit="10" paginate="both"}
...
{/exp:low_search:results}
Thanks
Low 3 Oct 2014 09:02
This depends on your search mode. If you have that set to "Any", entries that have any of the given words will be returned. And since the search term contains "with", a common word, that can be a lot. Setting it to "All" can already help.
As for relevance... This is always a tricky beast. Relevance is relative. Low Search calculates relevance in 2 ways: using the full-text scoring mechanism provided by MySQL. This is the fastest method, but the method itself is a black-box, so there's no real way of telling how MySQL determines that score.
The full-text search is only performed if the search terms allow it. That is to say, the terms need to fall below the ft_min_word_len threshold, which is 4 by default. Also, the search terms shouldn't contain stop words.
So, when keywords contain words of 3 letters or less, or if it contains stop words, a fallback search is triggered. This uses LIKE and calculates the relevance score in PHP, for which it needs the index text. It it exactly this that (I think) could cause a memory limit like that. The way the fallback relevance score is calculated, depends on the amount of search terms, the amount of text in the index, and the weight of the field. That can mean that an entry that doesn't have a lot of words but is a close match is rated higher than a closer match that has lots of words.
Hope that helps.
Joe Beaver 3 Oct 2014 09:43
Hi Low,
That helps a lot, thanks. Setting the search mode to all instead of any was an instant improvement, both on speed and relevance.
One (hopefully final!) question left - is there a way to give more weighting to one collection over another?
For example, I have one collection called Video Clips and another called Publications. I would like publications to have more weight in the results than video clips so that they appear higher.
An example is, I search for "othello" which brings up 3 results in this order:
Video clips entry: "Various contributors". This has the word othello in a field with a weight of 1.
Publications entry: "Studying Othello DVD"
Publications entry: "Studying Othello with DVD"
The publications titles have a weight of 3 and the publications collection has the modifier set to 8.0 (I had hoped the modifier would have solved this but it doesn't seem to have had an effect).
Sorry, I know we're branching off to other issues here but hopefully that's OK and this is the last one :P
Joe
Low 3 Oct 2014 10:19
The modifier should have that effect, but if the scores are low overall, multiplying it with 8 will still end up with a relatively small score. You can try and set a modifier of the Video Clips collection to 0.5, halving their scores. Otherwise, there's not much you can do, really.
Joe Beaver 3 Oct 2014 10:26
Set the video clips to 0.5 and upped publications to 10, still no change unfortunately. If that's all I can do and one scores just much higher than the other then I cant do much else I suppose.
I would have thought that the score would be higher for publications as the word othello appears more than once and fields that hold it have higher weighting, but maybe that's not how it works?
Low 3 Oct 2014 10:33
This depends again on which search is triggered. For the MySQL full-text score, it's a black box. For the fallback search, scores are calculated like this: number of occurrences in text / total words * 100, again divided by the amount of keywords.
I guess this could be tweaked somehow, but it works for most cases.
Joe Beaver 3 Oct 2014 11:34
Thanks for all your help on this - I will treat this as an edge case. I actually realised that there's more references to othello in the entry that aren't displayed on the front end, so the score is correct. just a shame that the modifier wasn't powerful enough to push it down.
Thanks again :)