Chinese Searching

kamho

The search function cannot normal work with Chinese characters in basic/advance search. We are using the default search engine "SQLSearch". It need to add the Wildcards(*) at suffixes if the search value is not a whole title or the record's title not contain(space character/comma). Can the problem fix by regular expression in search.conf and how?

Thanks for your kind help.

seth

I'm sorry but I don't fully understand. Is the problem the added wildcards?

kamho

Hi Seth, Thanks for your reply. I have created three testing records in demo site under "Chinese collection" http://demo.collectiveaccess.org/index.php/find/BrowseObjects/modifyCriteria/facet/collection_facet/id/98/mod_id/0 The title name are (1) "教育改革" , (2) "教育改革 2" & (3)" 教育,改革,3". If we search the "教育" single word; the result only return second(2) & third(3) records, if we add the wildcards "教育*" in the search box; the result will include all of those records. Since most of the chinese wording is not contain the space/comma. How can don't add the wildcards to search those three records?

Ton of thanks.

seth

In app/conf/search.conf you can add a regex that will append a wildcard to all searches.

The "search_suffixes" directive is a list of pairs of regular expressions (pattern matches) and suffixes. When one of the regular expressions matches the user's search the suffix is appended to the search before it is evaluated. This can be handy for all sorts of things. In your case you can use it to append a wildcard onto all searches, like so:

search_suffixes = { .* = * }

kamho

Hi seth,

It works on the suffix, however if i search the "改革" single word; it also can't return all of those results. Can the search.conf control the prefix?

Moreover, in the providence when we generate the PDF in summary page, the chinese wording cannot display, is it be possible to change the font type? (http://demo.collectiveaccess.org/index.php/editor/objects/ObjectEditor/PrintSummary/object_id/1520)

Thank you very much for your help.

seth

The PDF issue is a matter of changing the output font. The default font we use is called DejaVu. It contains many Unicode characters and glyphs, but not Chinese characters I guess.

Regarding the 改革 search, it sounds like you want to find these characters wherever they appear? Not just at the beginning of a "word"?

garytemp88

Hi @seth,

I have the same problem when I search Chinese characters.

In Chinese, 教育改革 contain 4 words, but the indexing process indexes words with spaces or "," between words. In this case, 教育改革 is one word. That's why if I search 改革, it returns no result. I need to search the whole phrase 教育改革 to return the record.

If I add a space or "," between 教育 and 改革 (i.e. 教育改革 or 教育,改革), I can search 教育 or 改革 successful. But in Chinese, we do not have spaces between characters.

English vs Chinese
I am a man. vs 我是一個男人.

I am sure if there is a configuration to handle CJK characters in CollectiveAccess.

Gary