Process indexing queue failure

ericthrift

I'm migrating a fairly large database (approximately 300,000 records) into CollectiveAccess using the command-line import tools. The search indexing is giving an "out of range value" database error for ca_sql_search_word_index.field_index. I'm wondering if this comes from a limit associated with the field being stored as TINYINT, and whether there might be a recommended workaround? For reference, here is the error I'm seeing when running caUtils process-indexing-queue:

PHP Fatal error:  Uncaught DatabaseException: Out of range value for column 'field_index' at row 769 in /var/www/html/ca/app/lib/Db/mysqli.php:358
Stack trace:
#0 /var/www/html/ca/app/lib/Db/DbStatement.php(150): Db_mysqli->execute()
#1 /var/www/html/ca/app/lib/Db.php(261): DbStatement->executeWithParamsAsArray()
#2 /var/www/html/ca/app/lib/Plugins/SearchEngine/SqlSearch2.php(1561): Db->query()
#3 /var/www/html/ca/app/lib/Search/SearchIndexer.php(1410): WLPlugSearchEngineSqlSearch2->updateIndexingInPlace()
#4 /var/www/html/ca/app/models/ca_search_indexing_queue.php(233): SearchIndexer->indexRow()
#5 /var/www/html/ca/app/lib/Utils/CLIUtils/Search.php(201): ca_search_indexing_queue::process()
#6 /var/www/html/ca/support/bin/caUtils(171): CLIUtils::process_indexing_queue()
#7 {main}
   thrown in /var/www/html/ca/app/lib/Db/mysqli.php on line 358

Any suggestions would be extremely welcome!

seth

There are recent indexing changes to better support positional searches (Eg. 8th word in a field) on short values. The indexer will sometimes exceed the bounds of the field as you've noticed. This is fixed in the current GitHub dev/2.0 branch code and will be included the next release.

If you can please pull down the latest dev/2.0 code. If you can't, we can roll up a new release later today or tomorrow.

Seth

ericthrift

Hi Seth, thanks for the quick response. I can certainly pull in the dev version from git and rerun the indexer, assuming the rest of the code on the dev branch is stable enough for production 🙂.

I'm running an import on a large batch of records right now, which looks like it will take until tomorrow to complete, so will wait for that to finish then back everything up before upgrading. Thanks!

Eric

seth

Yes, it the same code you're using with a handful of fixes.