Why is CollectiveAccess Searching and Upload Slow

Laith2001

I'm experiencing slow performance with CollectiveAccess, particularly with searching and media uploads. Given my server specifications, I expected better performance:

OS: CentOS Linux 7.9
CPU: Intel(R) Xeon(R) Gold 6240 @ 2.60GHz
Memory: 39GB
Swap: 7.9GB
DB size: _15GB
Redis: v3.2.12
GraphicsMagick: v1.3.38
Issues I’m noticing:

Searching: Queries take significantly longer than expected, even for relatively small datasets.
Media Uploads: Uploading and processing media files (e.g., images, TIFFs, PDFs) is slow.
Some things I’ve tried:

Ensuring Redis is running and caching properly
Checking system load and memory usage (both seem fine)
Optimizing MySQL database indexes

What could be causing this?

darrigan

When I started using CA in 2007, I noticed that everything was getting slower and slower over time. I didn't use Redis, but I noticed that by emptying the app/cache folder regularly I could get it back to normal. Maybe it's unrelated to your case, but have you checked if the providence and pawtucket app/cache folders contain a lot of files? And try emptying it? And maybe reboot your server?

seth

If disk I/O on the machine is slow everything will be slow. Perhaps there's something else running taking up bandwidth? I'd look at disk I/O and other processes to start with, as the server specifications you've posted should be more than enough.

How large are the media files you're uploading? And what media processing plugins are you using? If you're uploading large files and using one of the command-line based media processing plugins (ImageMagick or GraphicsMagick) without background processing it'll be very very slow.

Laith2001

I checked the I/O and the problem dosent seem to be from it.
the images are mostly TIFF format the largest was 20 MB.
I have installed GraphicsMagick, and unchecked this option on setup.php dont_use_imagick = 1
in version 2 there is option in app.conf background_process_mode = process, but in my version 1.8 it doesn't exist

seth

Make sure you have the Gmagick PHP extension installed. A 20mb TIFF is fairly large and will drag when processed using straight GraphicsMagick, which uses shell commands (which are slow)

Laith2001

then maybe upgrading to version 2 would give faster speed?
from php8 instead of 7, and the background_process_mode = process option