Import large media impossible

jmherp

I updated version of CollectiveAccess to version 2.0.5 and I'm no more able to import large medias. I used the frontend interface Import data to create objects with representations. It works with pictures smaller than 3 Mb but for larger, it runs a long time and finishes with nothing.
I tried also by command line but same thing : it runs a very long time and nothing at the end.
I wanted to test import-media but there, it says "Could not import media from ...: Specified import directory '...' is not valid"
I can't see if configuration is correct, page https://virtualcol.africamuseum.be/providence/index.php/administrate/setup/ConfigurationCheck/DoCheck finished with a timeout without giving any result...
Any help would be appreciated!

monica

jmherp you’ll need to look at logs for errors to know what is going on. What version did you upgrade from and how?

seth

Is your server running Windows or Linux or something else? Did you update PHP to new version as part of this update?

jmherp

I updated from version 1.7.15. I kept the MariaDB database and I used the new files for Providence and Pawtucket.
I checked in the error log but there is no error. But in debug, I see hundreds of lines only for the import of 1 picture of 13Mb. And it takes a long time.
One of these lines is
2025-07-31 12:10:10 - DEBUG --> Executing command: '/usr/bin/gm convert '/var/www/html/providence/app/tmp/tpc_layer_scale_iGfU05.jpg[0]' -quality 40 -crop 256x256+0+0 +profile "*" "/var/www/html/providence/app/tmp/tpc_tile_pdx5DI.jpg"'
Is it normal?
Is there something wrong perhaps with GMagick, IMagick,... ?

seth

Looks like you're running the command-line GraphicsMagick media processing back-end. It can be really slow. Try reinstalling the PHP gmagick extension. It'll make a huge difference in terms of performance.

jmherp

Thanks a lot Seth! I have been searching for a solution to this problem since weeks ! I could easily import 350 records with large pictures in 24 minutes!
Thanks!

jmherp

There is still a performance problem : charge on CPU is very high and there are several php lines running together. Is this normal? It seems to have also an impact on Pawtucket interface that answer very slowly.

monica

jmherp check your access logs, you most likely have bots smashing your Pawtucket and slowing everything down

jmherp

Hi, I checked the log. There were no special massive robot requests. Is it possible that CollectiveAccess launches some processes regularly, as indexing? Is it possible to disable them? You can see in that printscreen that a process is running on MariaDB during a long time

monica

jmherp looks completely normal to me. Mariadbd is the database server, you want that process to be running, or the database connection goes away. PHP-fpm is a web session open. You will get multiples of those as each one can only support a certain number of connections at a time.

seth

Mariadb is always running, as Monica mentioned. Try logging into Mariadb and run SHOW PROCESSLIST Are there queries running with long execution times?

jmherp

I don't think so but I see regularly indexing jobs running