"Image is being processed"

jmherp

We put the command process-task-queue in a cron job but since then, some images are always in "processing". I tried to launch manually the command, as well as bin/caUtils reprocess-media but more and more images are now in processing! Is there a checklist to verify what goes wrong with the media processing? Which command line can I launch, which are the good rights and user on the folders with media?

jmherp

@seth, our site is blocked because of this problem. Is it possible to get help?

monica

What version are you using? If it’s new enough, you can put the processing queue widget on your dashboard and it will tell you if any processes are stuck and be able to restart them.

Otherwise you run the process-task-queue with the -r option to restart any crashed processes

The reprocess-media command deletes all the derivatives and reprocesses everything, adding it to the queue.

jmherp

Hi Monica, I have version 1.7.7
I added the plugin you mentioned. I see this:
I ran the process-task-queue -r but it's very rapidly finished, as if it didn't find anything

jmherp

system is also frequently slow, I see that Mariadb use a lot of resources doing I don't know what :

jmherp

We have 90000 objects and a little more than 200000 images. Is it too much for collectiveaccess?

jmherp

is it normal that image seems to be deleted ??
How can I manually force to restart the queue? process-task-queue -r seems to do nothing

seth

Just to confirm you're running 1.7.7? Not the latest 1.7, or 2.0? process-task-queue does manually restart the task queue, so if it's not doing anything when you run that command something else is going on.

jmherp

Update : when I launch process-task-queue, it seems it does only 1 picture, no more

You can see a difference between this printscreen and the former one.
Is there a parameter that I have to change to process the whole queue?

jmherp

well, even if the queue is processed, it doesn't change anything, all remains in "being processed"!!

monica

jmherp with the queue that large it probably hasn’t reached the media yet. You said you triggered a reprocess-media, so that made the task queue even longer. To determine the media processing issue, you’ll need to get this task queue processed and cleared.
Do you have any task queue settings in your app.conf? In particular ‘taskqueue_max_items_processed_per_session’

jmherp

@monica, I have in app.conf:
taskqueue_max_processes = 4
taskqueue_process_timeout = 3600
taskqueue_max_items_processed_per_session = 100
When I launch caUtils process-task-queue -r, it's correct 100 records are treated.
I made printscreens of the statistics just before and after a launch of process-task-queue:

you see that there is a difference of 100 between the 2. Record 14189 is normally treated, it disappeared from the list, first record is now 14290 but when I look record 14189, I see that nothing changed, it's always in "Image is being processed" ! Or another process must be launched after that queue is empty?

monica

jmherp I would let the queue finish processing before problem solving further. If you have process-task-queue running in cron, then it should just slowly tick through and get things processed. You might need to check you have it set to run every minute however.

jmherp

Hi Monica, here is the result after queue processing

The one stuck is not important, only a wav file that wasn't treated. But as you can see, situation is the same :-(

jmherp

I also have to say that this "image is being processed" seems to be on old pictures as well as new ones, randomly

monica

Because you ran the 'reprocess-media' command, it means every single representation and media derivative on the system was deleted and reprocessed. This is why old and new reps are showing this. However in that screenshot you have 2 images showing. Are these "old" ones or newly uploaded ones? Is there anything different about the ones that are showing, say the format of the original image?

If you look in the providence/app/logs directory, are there any log files in there with today dates (or with the date from when the task queue was being processed). Look in that log file to see if there are any errors showing up for the media processing.

jmherp

the last printscreen shows pictures imported more than 2 years ago. The only difference is perhaps as you say the size. I read in another post in the forum that originals pictures should be kept be it's not the case, all the pictures disappeared from media directory. I found them back in the bin! So the command reprocess-media doesn't work correctly (also seen in https://support.collectiveaccess.org/d/301792-corrupted-images-after-reprocess-media) . If all the pictures are deleted, what's the basis to reprocess? Where are the originals?
All I see in the log is
2026-02-25 11:46:12 - DEBUG --> Executing command: '/usr/bin/ffmpeg> /dev/null 2>&1'
2026-02-25 11:46:12 - DEBUG --> Executing command: '/usr/bin/mediainfo --Help > /dev/null'
2026-02-25 11:46:12 - DEBUG --> Executing command: '/usr/bin/gs -v 2> /dev/null'
2026-02-25 11:46:12 - DEBUG --> Executing command: '/usr/bin/gm 2> /dev/null'

monica

jmherp Only the derivatives are deleted, not the originals, it's then reprocessed based on the original.
If everything disappeared from the media directory, then something went terribly wrong and you should restore from a backup.

I can't tell you if there is an issue in version 1.7.7 with the reprocessing of media, that version is before my time on this project.

The issue you link to was because they modified the original by overriding the settings in media_processing.conf, therefore it's not really an original anymore and derivatives couldn't safely be created from it.

Have you edited the settings in media_processing.conf from default?

jmherp

I didn't change settings in media_processing.conf.
How is it possible to make the link between a name like "10188_ca_object_representations_media_20047_original.jpg" and the object id - representation id ? I read it's in a blob, in field "media" of ca_object_representations but it's unreadable.

monica

jmherp The file name 10188_ca_object_representations_media_20047_original.jpg is the original derivative for the representation with representation_id = 20047. You would most likely find it in (or it should live in) the providence/media/collectiveaccess/images/2/0/0 folder