SqlSearch index holds lots of possibly useless rows?

yelto

We find a very significant number of rows (₃₀%) in ca_sql_search_word_index [Prov 1.6.3] to be curious, in that they are either:

such that word_id=0

or duplicate rows (two or more rows where the only differing column value is index_id)

So a couple of quick questions:

Is this normal/expected or does it suggest a problem somewhere (search_indexing.conf)?

Any harm in deleting rows where word_id=0 and deduplicating? The index is very large, removing a few million rows if indeed they are useless sounds attractive.

Thanks.

seth

The ones with word_id=0 may be count rows. If you remove these count queries won't work. Removing duplicate rows will affect phrase queries. Otherwise go for it.

yelto

Thanks Seth, yes, I see now a bunch of word_id=0 that could indeed be there for relationship counts. The bulk of them though are single-table attributes or intrinsics  (word_id=0 and table_num=field_table_num and row_id=field_row_id and (field_num like 'I%' or field_num like 'A%')). These look like they correspond to attributes not set, could that be? They make tempting candidates for cost-free deletion, can you think of any count or existence predicate that relies on those rows?