Case-insensitive search with all-caps and umlauts...
This is has got me puzzled. When I have search-terms in all-caps or initial caps, the search engine always returns results (if there are any).
However a search where the search terms are entered with all-caps AND contain one or several umlauts fails (no results).
My setup is:
- EE v2.9.2
- MySQL v5.1.66 (utf8_general_ci)
- PHP v5.5.17
- Low Search v3.3.3
Any ideas about this?
Regards,
Matts
Replies
Low 26 Jan 2015 14:16
Can you turn on Template Debugging and the Output Profiler and copy/paste all lines that mention low_search/Low Search, as well as any queries that target any exp_low_search_* tables here?
Matts Härsing 26 Jan 2015 14:48
There were quite a lot of sql-queries, so i chose one that contained my search term (it had been converted to lowercase):
SELECT entry_id, collection_id, index_text
FROM ('exp_low_search_indexes')
WHERE 'site_id' IN ('1')
AND ( (index_text LIKE '%städavtal%') )
#APP/third_party/low_search/filters/keywords/lsf.keywords.php L:173 Low_search_filter_keywords::filter()
0.0004 UPDATE 'exp_low_search_log'
SET num_results = IF(num_results IS NULL, 0, num_results + 0)
WHERE log_id = 350
#APP/third_party/low_search/mod.low_search.php L:1623 Low_search_log_model::add_num_results()
------------------------------------------
The profiler had this:
(0.577405 / 1.86MB) Low Search: Applying Low_search_filter_keywords
(0.580518 / 1.88MB) Low Search: Starting search (fallback)
(0.643461 / 1.88MB) Low Search: Searched but found nothing. Returning no results.
(0.643508 / 1.88MB) Low Search: Filters found no matches, returning no results
---
(0.648529 / 1.88MB) Low Search: Shortcut eyJyZXN1bHRfcGFnZSI6InNvayIsImtleXdvcmRzIjoiU1RcdTAwYzREQVZUQUwifQ not found
(0.648874 / 1.88MB) Low Search: Applying Low_search_filter_keywords
(0.649257 / 1.88MB) Low Search: Starting search (fallback)
(0.653558 / 1.88MB) Low Search: Searched but found nothing. Returning no results.
(0.653587 / 1.88MB) Low Search: Filters found no matches, returning no results
---
(0.656002 / 1.89MB) Low Search: Shortcut eyJyZXN1bHRfcGFnZSI6InNvayIsImtleXdvcmRzIjoiU1RcdTAwYzREQVZUQUwifQ not found
(0.656237 / 1.89MB) Low Search: Applying Low_search_filter_tags
(0.657878 / 1.89MB) Low Search: Applying Low_search_filter_keywords
(0.658224 / 1.89MB) Low Search: Starting search (fallback)
(0.658582 / 1.89MB) Low Search: Searched but found nothing. Returning no results.
(0.658601 / 1.89MB) Low Search: Filters found no matches, returning no results
---
(0.659967 / 1.89MB) Low Search: Shortcut eyJyZXN1bHRfcGFnZSI6InNvayIsImtleXdvcmRzIjoiU1RcdTAwYzREQVZUQUwifQ not found
(0.660171 / 1.89MB) Low Search: Applying Low_search_filter_tags
(0.661017 / 1.89MB) Low Search: Applying Low_search_filter_keywords
(0.661315 / 1.89MB) Low Search: Starting search (fallback)
(0.661620 / 1.89MB) Low Search: Searched but found nothing. Returning no results.
(0.661636 / 1.89MB) Low Search: Filters found no matches, returning no results
---
(0.662795 / 1.89MB) Low Search: Shortcut eyJyZXN1bHRfcGFnZSI6InNvayIsImtleXdvcmRzIjoiU1RcdTAwYzREQVZUQUwifQ not found
(0.663050 / 1.89MB) Low Search: Applying Low_search_filter_field_search
(0.663952 / 1.90MB) Low Search: Getting channel field info from API
(0.734898 / 2.13MB) Low Search: Applying Low_search_filter_keywords
(0.735357 / 2.13MB) Low Search: Starting search (fallback)
(0.737586 / 2.13MB) Low Search: Searched but found nothing. Returning no results.
(0.737606 / 2.13MB) Low Search: Filters found no matches, returning no results
---
(0.740363 / 2.15MB) Low Search: Shortcut eyJyZXN1bHRfcGFnZSI6InNvayIsImtleXdvcmRzIjoiU1RcdTAwYzREQVZUQUwifQ not found
(0.740604 / 2.15MB) Low Search: Applying Low_search_filter_keywords
(0.740904 / 2.15MB) Low Search: Starting search (fallback)
(0.785796 / 2.15MB) Low Search: Searched but found nothing. Returning no results.
(0.785837 / 2.15MB) Low Search: Filters found no matches, returning no results
---
(0.789408 / 2.14MB) Low Search: Shortcut eyJyZXN1bHRfcGFnZSI6InNvayIsImtleXdvcmRzIjoiU1RcdTAwYzREQVZUQUwifQ not found
(0.789677 / 2.14MB) Low Search: Applying Low_search_filter_keywords
(0.790097 / 2.14MB) Low Search: Starting search (fallback)
(0.792540 / 2.14MB) Low Search: Searched but found nothing. Returning no results.
(0.792560 / 2.14MB) Low Search: Filters found no matches, returning no results
---
(0.793674 / 2.14MB) Low Search: Shortcut eyJyZXN1bHRfcGFnZSI6InNvayIsImtleXdvcmRzIjoiU1RcdTAwYzREQVZUQUwifQ not found
(0.793906 / 2.14MB) Low Search: Applying Low_search_filter_field_search
(0.795037 / 2.14MB) Low Search: Applying Low_search_filter_keywords
(0.795423 / 2.14MB) Low Search: Starting search (fallback)
(0.796192 / 2.14MB) Low Search: Searched but found nothing. Returning no results.
(0.796209 / 2.14MB) Low Search: Filters found no matches, returning no results
---
(0.797186 / 2.14MB) Low Search: Shortcut eyJyZXN1bHRfcGFnZSI6InNvayIsImtleXdvcmRzIjoiU1RcdTAwYzREQVZUQUwifQ not found
(0.797394 / 2.14MB) Low Search: Applying Low_search_filter_field_search
(0.799526 / 2.14MB) Low Search: Applying Low_search_filter_keywords
(0.799847 / 2.14MB) Low Search: Starting search (fallback)
(0.800328 / 2.14MB) Low Search: Searched but found nothing. Returning no results.
(0.800344 / 2.14MB) Low Search: Filters found no matches, returning no results
Matts Härsing 26 Jan 2015 14:50
When I do the same with a lowercase search-term, it seems that the umlauts get converted, like so:
SELECT entry_id, collection_id, index_text
FROM ('exp_low_search_indexes')
WHERE 'site_id' IN ('1')
AND ( (index_text LIKE '%staedavtal%') )
#APP/third_party/low_search/filters/keywords/lsf.keywords.php L:173 Low_search_filter_keywords::filter()
------
So all-caps searchterms with umlauts don't get cleaned?
Low 26 Jan 2015 15:09
It depends on what's present in EE's config/foreign_characters.php file. It looks like uppercase letters aren't accounted for, there. You can add them yourself, for example: '196' => 'AE' which should transform Ä to AE.
Also, are you using a lot of Results tag on the same page? And if so, is that absolutely necessary? Looks like performance could be improved, there.
Low 26 Jan 2015 15:14
Oh, if you do edit the foreign_characters.php file, remember to rebuild your indexes to apply the changes to the collections.
Matts Härsing 26 Jan 2015 15:17
I'll give that a try. Thanks!
Yes, I'm using a lot of results tags. The site is not live yet, but I could use som performance tips.
I guess the main problem for me is to differentiate search results from different categories as well as differentiating between normal channel results and those from a collection.
I need to split the results and show them in different DIVs on the page.
Regards,
Matts
Low 26 Jan 2015 15:23
Well, you can differentiate between different channels or collections by using normal conditionals, as explained here.
If that doesn't give you the right output, you can try and use Stash to load up the search results in one go and display them in batches elsewhere. Not an expert on Stash myself, but I've heard that people had good results with it. It would definitely help with performance here instead of having multiple Results tags.
Matts Härsing 26 Jan 2015 15:26
Editing foreign_characters.php did the trick. Thanks a lot!
I'm using Stash elsewhere so I'll try that.
Once again: fantastic support of a fantastic product!
Matts Härsing 26 Jan 2015 15:33
Just one more thing. Differentiating search results by channel is one thing, but can the same be done with collections?
That is: can I tell if an entry belongs to a collection inside the results loop, and if so which collection that is?
Or am I thinking backwards?
/Matts
Low 26 Jan 2015 15:35
You can use {low_search_collection_name} in a search result for that. Works in a conditional, too.
Matts Härsing 26 Jan 2015 15:36
Well, that would work perfectly.
Thanks again!