CA 2.09 Autonumbering fails

remco

Setup: Shared server Collective Access 2.09

Adding a new object record the object ID is initially showing the next value correctly (for instance 00053). After saving the object the id number is 00001 (which is a duplicate ID number) where it should be 00053.
My multipart_id_numbering.conf for objects is:
ca_objects = { __default__ = { separator = , elements = { coll_num = { type = LIST, values = [_RU, SKU, KUN, UMC, T2O], default = UMC, width = 3, description = Collectie code, editable = 1, }, item_num = { type = SERIAL, width = 5, zeropad_to_length = 5, description = Item number, editable = 1, table = ca_objects, field = idno, sort_field = idno_sort, }, item_sub = { type = FREE, width = 8, description = Item subnumber or letter, editable = 1, table = ca_objects, field = idno, sort_field = idno_sort, }, }, },

remco

If I change my code for the code in the manual the difference is that no correct suggestion is made before save, After save with this code I also get a duplicate number.
It seems to be an issue in the source code rather than in my code for multipart_id_numbering.conf.

remco

If I am right the suggested number is generated by another piece of code (js/ca.idnochecker.js) than the code that saves the number. Could that be right?

monica

remco did you already have records with idno in the system when you set your idno rules? Most of the time, the serial won’t work correctly if there are already numbers in there, or one that was manually edited with the incorrect format. Auto numbering looks at the numbers already there and picks the next, if any go against the format, it reverts back to one.

If you’ve had this rule all along and it just started not working, then check the manually edited serial number elements. Someone might have put a letter into it something.

remco

The system worked with (basically) these autonumbering rules for years. I made some adaptions about 6 months ago, but the autonumbering still worked as expected.
This was with version CA1.8.
Last week we upgraded to CA2.09.
Now after saving it saves the wrong number. The suggestion before save has the correct expected number so I assume at least part of my multipart_id_numbering.conf can generate the expected result.
I have a test system with exactly the same setup so I can test and confirm there are no typo's or things like that.

remco

I did change the file BaseModelWithAttributes.php. a few weeks ago to get Batch edititing to work. (see
https://support.collectiveaccess.org/d/302156-collective-access-209-batch-editing-error-with-source-access-checking )
I reverted that and cleared the cache to see if that had any effect, but it did not (other than batch editing not working) so I set it back to the changed file.

seth

Make sure you update all sort values. Run caUtils rebuild-sort-values

remco

I did clear the cache, update sort values, search index and hierarchy after every change.
The weird thing is that autonumbering does work on movements, loans and occurences.
They have the same structure.

remco

Are the sort values related to the search index?
In my (and in the standard) search_indexing.conf the field idno_sort is not indexed.

seth

Sort values are completely separate.

There are hidden fields for preferred labels (titles) and identifiers that have the content from their visible counterparts reformatted so they can be sorted to user expectations by the database. The sortable formatting has changed over time to reflect new requirements. If your identifiers aren't sorry correctly, the next-in-sequence numbering may be affected. Thus the suggestion to regenerate those values.

remco

I think I am on to something.
The formatting for movements, loans and occurences is missing the third element (item_sub).
I tried both (with and without) the third element for objects.
This code does not work

ca_objects = {
default = {
separator = ,
allow_extra_elements=1,
extra_element_width=8,
elements = {
coll_num = {
type = LIST,
values = [_RU, SKU, KUN, UMC, T2O],
default = UMC,
width = 3,
description = Collectie code,
editable = 1,
},
item_num = {
type = SERIAL,
width = 5,
zeropad_to_length = 5,
description = Item number,
editable = 1,
},
item_sub = {
type = FREE,
width = 8,
description = Item subnumber or letter,
editable = 1,
},
},

},

}

And this code works

},

}

Another slightly annoying issue is that the default value for the LIST items in no longer working as it did.
I have formatting for objects of different types with a different default value for the LIST. This used to work fine but stopped working now.

remco

Sorry, somehow the formatting for the code got lost.

remco

With this code the default value for the LIST items is also working.

__default__ = {
			separator = ,
                        allow_extra_elements=1,
                        extra_element_width=8,
                        sort_order = [coll_num, item_num],			
			elements = {
				coll_num = {
					type = LIST,
					values = [_RU, SKU, KUN, UMC, T2O], 
					default = UMC, 
					width = 3, 
					description = Collectie code,
                                        editable = 1,
				},
                                item_num = {
					type = SERIAL,
					width = 5,
                                        zeropad_to_length = 5,
					description = Item number,
					editable = 1,
					table = ca_objects,
					field = idno,
					sort_field = idno_sort,
				},
			},                        
		},

The difference is that I have now sort_order, table, field and sort_field added.
Adding an extra element still breaks autonumbering. I have tried this for FREE and for ALPHANUMERIC.

remco

Update: The third element breaks auto numbering in the sense that the suggested value is the highest for the default coll_num.
Saving the object the saved value is not the same as the suggested value. For instance if the suggested value is 00066 the saved value would be 00063 (the highest value at the start of this test) and for subsequent numbers stays 00063, even though the suggested value is correctly incremented. Changing the coll_num value does not recalculate the suggested value as it used to do.
I think this is a bug in CA 2.0.9.
I am not sure weather it is also a bug in earlier versions?
We came from CA 1.8 and in that version it worked.

monica

remco Might be worth while submitting an issue on github.
I am curious to know if the number the serial gives you on save changes at all depending on the last element value (Leaving it empty, compared to other values). I ask, because the sort values are most likely getting confused with that last element in the mix.

remco

monica
With the 3 elements formatting the suggested next serial is only correct for the default value in coll_num. It is correct with or without a value in the last element for the previous object.
When I save the object (with correct suggested next serial) the saved serial is the highest for the default value in coll_num at the moment the new configuration was applied and that same serial is saved for every next new object.

So somehow the mechanism that predicts the next serial is confused by the 3 elements setup in not being able to predict a next serial for any other value in coll_num than the default value.
And the mechanism that actually gets the next serial to save is unable to find the new next serial at all (apart from the highest value at the time the new configuration was applied).

Does that answer your question?

I will wait submitting the issue on github untill you have seen and responded to my other post about the database query.

monica

Could you share the output of this database query?
select idno, idno_sort, idno_sort_num from ca_objects order by idno_sort_num desc limit 10;
It should show us the highest value at the top, which it's basing your next value on.

remco

I will give you the output for two cases:
Case one:
With this formatting (3 elements):

	__default__ = {
		separator = ,
                    sort_order = [coll_num, item_num, sub_number],
		
		elements = {
			coll_num = {
				type = LIST,
				values = [_RU, SKU, KUN, UMC, T2O], 
				default = UMC, 
				width = 3, 
				description = Collectie code,
                                    editable = 1,
				table = ca_objects,
				field = idno,
				sort_field = idno_sort,
			},
                            item_num = {
				type = SERIAL,
                                    allowsuffix = 1,
				width = 16,
                                    zeropad_to_length = 5,
				description = Item number,
				editable = 1,
				table = ca_objects,
				field = idno,
				sort_field = idno_sort,
			},
			sub_number = {
				type = FREE,
				width = 30,
                                    default = "", 
				editable = 1,					
				description = _(Sub number)
				table = ca_objects,
				field = idno,
				sort_field = idno_sort,
			},
		},                        
	},

I get this output

idno idno_sort idno_sort_num
UMC 00061 Sub004 UMC 00061 Sub004 ... 311429700
UMC 00061 Sub003 UMC 00061 Sub003 ... 311429600
UMC 00061 Sub002 UMC 00061 Sub002 ... 311429500
UMC 00061 Sub001 UMC 00061 Sub001 ... 311429400
_RU 02357 SUB01 _RU 02357 SUB01 ... 31375800
_RU 02357 SUB01 SUB01 _RU 02357 SUB01 ... 31375800
_RU 00022 Sub01 _RU 00022 Sub01 ... 31142300
_RU 00051 W01 K02 _RU 00051 W01 ... 355200
_RU 00051 W01 K02 R01 _RU 00051 W01 ... 355200
_RU 00051 W01 _RU 00051 W01 ... 355200

In the UI I get right suggestion of next serial only for the default value in coll_num and it does not update on switching coll_num and saves the wrong serial. It keeps saving the formally highest serial for the default coll_num it had found first time the configuration for multipart_id_numbering.conf was changed

Case two:
With this formatting (2 elements):

	__default__ = {
		separator = ,
                    sort_order = [coll_num, item_num],			
		elements = {
			coll_num = {
				type = LIST,
				values = [_RU, SKU, KUN, UMC, T2O], 
				default = UMC, 
				width = 3, 
				description = Collectie code,
                                    editable = 1,
				table = ca_objects,
				field = idno,
				sort_field = idno_sort,
			},
                            item_num = {
				type = SERIAL,
                                    allowsuffix = 1,
				width = 16,
                                    zeropad_to_length = 5,
				description = Item number,
				editable = 1,
				table = ca_objects,
				field = idno,
				sort_field = idno_sort,
			},
		},                        
	},

I get this output

idno idno_sort idno_sort_num
UMC 00070 UMC 00070 330200
UMC 00069 UMC 00069 330100
UMC 00068 UMC 00068 330000
UMC 00067 UMC 00067 329900
UMC 00066 UMC 00066 329800
UMC 00065 UMC 00065 329700
UMC 00064 UMC 00064 329600
UMC 00063 UMC 00063 329500
UMC 00062 UMC 00062 329400
UMC 00061 UMC 00061 329300

In the UI I get correct suggestion of next serial that updates on switching coll_num and saves the correct serial.
Disadvantage: sorting of idno's with Alphanumeric suffix follows after highest idno without alphanumeric suffix

remco

Between every configuration change for multipart_id_numbering.conf I clear the caches and rebuild search and sort values.