
Migrating obsolete datatypes in Umbraco v7
In older versions of Umbraco v7, there are a bunch of data types that have become obsolete. Although they can still be used with v7, it is highly recommended to migrate them to use their newer versions, especially if you are using Umbraco Cloud. We have experienced issues deploying content between cloud environments where obsolete data types are involved, resulting in wrong values being stored inside properties that are using obsolete datatypes.
It is easy to find which of these properties have become obsolete. They are clearly marked as such in the Developers section (v7).
Starting from v7.6.0, Umbraco introduced UDI identifiers and started storing identifiers in this new format for most object types. A UDI stores all necessary metadata to retrieve an Umbraco object and is formed as a string containing the scheme, the type of object, and the GUID id (without dashes).
umb://document/4fed18d8c5e34d5e88cfff3a5b457bf2
In this way, all those properties that stored identifiers pointing to other objects (i.e. media pickers or content pickers) as their integer id 1175
, will now contain identifiers in UDI form umb://document/4fed18d8c5e34d5e88cfff3a5b457bf2
.
This is the case for data types using the following property editors:
- Umbraco.ContentPickerAlias -> new editor: Umbraco.ContentPicker2
- Umbraco.MediaPicker -> new editor: Umbraco.MediaPicker2
- Umbraco.MultipleMediaPicker -> new editor: Umbraco.MediaPicker2
- Umbraco.MultiNodeTreePicker -> new editor: Umbraco.MultiNodeTreePicker2
- Our.Umbraco.NestedContent -> new editor: Umbraco.NestedContent
- Umbraco.RelatedLinks -> new editor: Umbraco.RelatedLinks2
- RJP.MultiUrlPicker -> new editor: Umbraco.MultiUrlPicker
Change to the new property editors
When you change a data type to use the new editor, you will need to manually update the data so it stores the values in the right format. You have to do this manually for each document. If you open the node and re-save it, Umbraco will update the data in the correct format. But this won't work with a bulk save or bulk publish. This can be fine if you don't have a large number of nodes and you don't mind going through all of them, but why shouldn't we automate the process. Automation is so much better and fun, even if writing the code for it takes us double or triple the time we would need to do it manually.
We found a very helpful GitHub gist where they already created a sort of migration process for the MultiNodeTreePicker datatypes. The code hooks into a start up handler and searches for new pickers that are still using the old format and updates the values.
https://gist.github.com/kiasyn/bb067269d97a37f76e9a0f8743972837
As a matter of fact, the key part of the code is the SQL query it uses to get these properties. It takes the nodes and values using the new property editors, but still containing old format values, which are stored in the dataNvarchar or dataInt columns of the PropertyData table of the database. It then converts the values from integer ids to UDIs and saves them in the dataNtext column. After a republish and a content refresh, the data is finally properly migrated.
As helpful as it is, we extended it to do the same thing for the rest of pickers, like MediaPicker, ContentPicker and MultipleUrlPicker, with slight differences from the original code.
The problem with complex data types
But we are still missing the migration of complex datatypes. I mean, Archetype, Nested Content and Vorto properties that contain obsolete media or content pickers within them. We can still take the same principles that we have used so far and extend the code, although it grows in complexity and has some particularities. The values stored inside these properties are also complex objects and cannot just be picked as an integer or a string of integers.
As an example, we wanted to migrate a Vorto property that contained an archetype that contained an obsolete media picker. For this scenario, we used a couple of foreach to finally select the properties with obsolete values and run the migration on them.
In the first foreach, we found all the media pickers data types. Then, in a second foreach, we found all archetypes data types that contain the media pickers, by using the data type id that is stored in the archetype prevalues. Finally, we found the Vorto properties that contain any of the selected archetypes, whose id is also stored as part of the Vorto prevalues. Now that we have all Vorto properties selected, we run the migration on the ones that haven't been migrated already. To proceed with this, we have to take each of the deserialized Vorto values, deserialize them as Archetype, select the fields of the archetype that store the previously selected media picker and update its value if hasn't been migrated already.
It is a complex operation, and thus is the code. But reading through the code might be easier to understand than trying to explain it with words:
private static void MigrateVortoArchetypeDataIdsToUdis(UmbracoDatabase database)
{
// Find media picker datatypes
string contentPickerSql = @"SELECT umbracoNode.uniqueID
FROM cmsDataType
JOIN umbracoNode ON umbracoNode.id = cmsDataType.nodeId
WHERE cmsDataType.propertyEditorAlias = 'Umbraco.MediaPicker2'";
var mediaPickers = database.Query<Guid>(contentPickerSql).ToList();
foreach (var mediaPicker in mediaPickers)
{
// Find archetypes using media pickers
string archetypeSql = $@"SELECT umbracoNode.uniqueID, cmsDataTypePreValues.value
FROM cmsPropertyType
JOIN cmsDataType ON cmsDataType.nodeId = cmsPropertyType.dataTypeId
JOIN cmsDataTypePreValues ON cmsDataTypePreValues.datatypeNodeId = cmsDataType.nodeId AND cmsDataTypePreValues.alias = 'archetypeConfig'
JOIN umbracoNode ON umbracoNode.id = cmsPropertyType.dataTypeId
WHERE cmsDataType.propertyEditorAlias IN ('Imulus.Archetype')
AND cmsDataTypePreValues.value LIKE '%""dataTypeGuid"": ""{mediaPicker}""%'";
var archetypes = database.Query<ArchetypeConfigRow>(archetypeSql).ToList();
foreach (var archetype in archetypes)
{
// Find Vorto properties of archetypes using content pickers
var sql = $@"SELECT cmsPropertyData.id, cmsPropertyData.contentNodeId, cmsPropertyType.alias, dataNvarchar, dataNtext, dataInt, cmsDocument.*
FROM cmsPropertyData
JOIN cmsPropertyType ON cmsPropertyType.id = cmsPropertyData.propertytypeid
JOIN cmsDataType ON cmsDataType.nodeId = cmsPropertyType.dataTypeId
JOIN cmsDataTypePreValues ON cmsDataTypePreValues.datatypeNodeId = cmsDataType.nodeId AND cmsDataTypePreValues.alias = 'dataType'
JOIN cmsContentVersion ON cmsContentVersion.VersionId = cmsPropertyData.versionId
JOIN umbracoNode ON umbracoNode.id = cmsContentVersion.ContentId
JOIN cmsDocument ON cmsDocument.nodeId = umbracoNode.id
WHERE cmsDataType.propertyEditorAlias IN ('Our.Umbraco.Vorto')
AND cmsDataTypePreValues.value LIKE '%""propertyEditorAlias"": ""Imulus.Archetype""%'
AND cmsDataTypePreValues.value LIKE '%""guid"": ""{archetype.UniqueID}""%'
AND(dataNtext IS NOT NULL)
AND(cmsDocument.published = 1 OR cmsDocument.newest = 1 OR cmsDocument.updateDate > (SELECT updateDate FROM cmsDocument AS innerDoc WHERE innerDoc.nodeId = cmsDocument.nodeId AND innerDoc.published = 1 AND newest = 1))
ORDER BY contentNodeId, cmsDataType.propertyEditorAlias";
var vortoDataToMigrate = database.Query<Row>(sql).ToList();
var config = JsonConvert.DeserializeObject<Archetype.Models.ArchetypePreValue>(archetype.Value);
var propertyAliases = config.Fieldsets.SelectMany(fieldset => fieldset.Properties)
.Where(property => property.DataTypeGuid == mediaPicker)
.Select(property => property.Alias);
if (vortoDataToMigrate.Any())
{
foreach (var propertyData in vortoDataToMigrate)
{
string udiValue;
if (!string.IsNullOrEmpty(propertyData.dataNtext))
{
// Vorto multilingual values
var vortoValue = JsonConvert.DeserializeObject<Our.Umbraco.Vorto.Models.VortoValue>(propertyData.dataNtext);
var udiValues = new Dictionary<string, object>();
foreach (var value in vortoValue.Values)
{
// Archetype fieldsets
var archetypeValue = JsonConvert.DeserializeObject<Archetype.Models.ArchetypeModel>(value.Value.ToString());
var fieldsets = archetypeValue.Fieldsets
.Where(fieldset => fieldset.Properties.Any(property =>
propertyAliases.Contains(property.Alias)));
foreach (var fieldset in fieldsets)
{
// Properties using content picker
var properties = fieldset.Properties.Where(property =>
propertyAliases.Contains(property.Alias));
foreach (var property in properties)
{
var strValue = property.GetValue<string>();
if (!string.IsNullOrEmpty(strValue))
{
if (!strValue.StartsWith("umb://"))
{
var uniqueIds = database.Query<Guid>($"SELECT uniqueId FROM umbracoNode WHERE id IN ({strValue})").ToArray();
var uniqueIdsCsv = string.Join(",", uniqueIds.Select(id => $"umb://media/{id:N}"));
property.Value = uniqueIdsCsv;
}
}
else
{
property.Value = null;
}
}
}
udiValues[value.Key] = ToDataValue(archetypeValue);
}
vortoValue.Values = udiValues;
udiValue = ToDataValue(vortoValue);
}
else
{
LogHelper.Info(typeof(MediaPickerIdToUdiMigrator), () => $"MigrateIdsToUdis (node id: {propertyData.contentNodeId}) skipping property {propertyData.alias} - null dataNtext");
continue;
}
LogHelper.Info(typeof(MediaPickerIdToUdiMigrator), () => $"MigrateIdsToUdis (node id: {propertyData.contentNodeId}) converting property {propertyData.alias} from {propertyData.dataNtext} to {udiValue}");
database.Execute("UPDATE cmsPropertyData SET dataNtext=@0 WHERE id=@1", udiValue, propertyData.id);
}
}
}
}
}
Change the views
Please note, that the code in your views will need to be modified accordingly.
For instance, we can now get the media inside a media picker in this simple way:
var image = Model.Content.GetPropertyValue<IPublishedContent>("temelineIcon")
Or the content inside a content picker:
var nodes = Model.Content.GetPropertyValue<IEnumerable<IPublishedContent>>("reports");
This is now the way we are used to with later versions of v7 and Umbraco v8, resulting in cleaner and easier to read code.
Final thoughts
It took a while to implement the code to migrate all our data types, and the process wasn't exempt from issues when deploying to different environments. The migration was a necessary thing to do and it was finally completed and the end result was satisfactory for both the client and us.
We could have opted for a manual migration, though. It would have taken some extra time from the live site, with errors and YSOD's while the pages are being migrated, but the total time spent might have been similar or even less. Well, maybe. But now we can reuse this migration process for other sites that will need it and that for sure contain a much much larger number of nodes using obsolete datatypes.
Also, an automated process, once we know is polished and working, is less prone to make errors and it certainly won't skip or forget any step.