Migrating obsolete datatypes in Umbraco v7

Migrating obsolete datatypes in Umbraco v7

In older versions of Umbraco v7, there are a bunch of data types that have become obsolete. Although they can still be used with v7, it is highly recommended to migrate them to use their newer versions, especially if you are using Umbraco Cloud. We have experienced issues deploying content between cloud environments where obsolete data types are involved, resulting in wrong values being stored inside properties that are using obsolete datatypes.

It is easy to find which of these properties have become obsolete. They are clearly marked as such in the Developers section (v7).

obsolete media picker

Starting from v7.6.0, Umbraco introduced UDI identifiers and started storing identifiers in this new format for most object types. A UDI stores all necessary metadata to retrieve an Umbraco object and is formed as a string containing the scheme, the type of object, and the GUID id (without dashes).

umb://document/4fed18d8c5e34d5e88cfff3a5b457bf2

In this way, all those properties that stored identifiers pointing to other objects (i.e. media pickers or content pickers) as their integer id 1175, will now contain identifiers in UDI form umb://document/4fed18d8c5e34d5e88cfff3a5b457bf2.

This is the case for data types using the following property editors:

  • Umbraco.ContentPickerAlias -> new editor: Umbraco.ContentPicker2
  • Umbraco.MediaPicker -> new editor: Umbraco.MediaPicker2
  • Umbraco.MultipleMediaPicker -> new editor: Umbraco.MediaPicker2
  • Umbraco.MultiNodeTreePicker -> new editor: Umbraco.MultiNodeTreePicker2
  • Our.Umbraco.NestedContent -> new editor: Umbraco.NestedContent
  • Umbraco.RelatedLinks -> new editor: Umbraco.RelatedLinks2
  • RJP.MultiUrlPicker -> new editor: Umbraco.MultiUrlPicker

media picker

Change to the new property editors

When you change a data type to use the new editor, you will need to manually update the data so it stores the values in the right format. You have to do this manually for each document. If you open the node and re-save it, Umbraco will update the data in the correct format. But this won't work with a bulk save or bulk publish. This can be fine if you don't have a large number of nodes and you don't mind going through all of them, but why shouldn't we automate the process. Automation is so much better and fun, even if writing the code for it takes us double or triple the time we would need to do it manually.

We found a very helpful GitHub gist where they already created a sort of migration process for the MultiNodeTreePicker datatypes. The code hooks into a start up handler and searches for new pickers that are still using the old format and updates the values.

https://gist.github.com/kiasyn/bb067269d97a37f76e9a0f8743972837

As a matter of fact, the key part of the code is the SQL query it uses to get these properties. It takes the nodes and values using the new property editors, but still containing old format values, which are stored in the dataNvarchar or dataInt columns of the PropertyData table of the database. It then converts the values from integer ids to UDIs and saves them in the dataNtext column. After a republish and a content refresh, the data is finally properly migrated.

As helpful as it is, we extended it to do the same thing for the rest of pickers, like MediaPicker, ContentPicker and MultipleUrlPicker, with slight differences from the original code.

The problem with complex data types

But we are still missing the migration of complex datatypes. I mean, Archetype, Nested Content and Vorto properties that contain obsolete media or content pickers within them. We can still take the same principles that we have used so far and extend the code, although it grows in complexity and has some particularities. The values stored inside these properties are also complex objects and cannot just be picked as an integer or a string of integers.

As an example, we wanted to migrate a Vorto property that contained an archetype that contained an obsolete media picker. For this scenario, we used a couple of foreach to finally select the properties with obsolete values and run the migration on them.

In the first foreach, we found all the media pickers data types. Then, in a second foreach, we found all archetypes data types that contain the media pickers, by using the data type id that is stored in the archetype prevalues. Finally, we found the Vorto properties that contain any of the selected archetypes, whose id is also stored as part of the Vorto prevalues. Now that we have all Vorto properties selected, we run the migration on the ones that haven't been migrated already. To proceed with this, we have to take each of the deserialized Vorto values, deserialize them as Archetype, select the fields of the archetype that store the previously selected media picker and update its value if hasn't been migrated already.

It is a complex operation, and thus is the code. But reading through the code might be easier to understand than trying to explain it with words:

private static void MigrateVortoArchetypeDataIdsToUdis(UmbracoDatabase database)
{
    // Find media picker datatypes
    string contentPickerSql = @"SELECT umbracoNode.uniqueID 
        FROM cmsDataType
        JOIN umbracoNode ON umbracoNode.id = cmsDataType.nodeId
        WHERE cmsDataType.propertyEditorAlias = 'Umbraco.MediaPicker2'";

    var mediaPickers = database.Query<Guid>(contentPickerSql).ToList();

    foreach (var mediaPicker in mediaPickers)
    {
        // Find archetypes using media pickers
        string archetypeSql = $@"SELECT umbracoNode.uniqueID, cmsDataTypePreValues.value
            FROM cmsPropertyType
            JOIN cmsDataType ON cmsDataType.nodeId = cmsPropertyType.dataTypeId
            JOIN cmsDataTypePreValues ON cmsDataTypePreValues.datatypeNodeId = cmsDataType.nodeId AND cmsDataTypePreValues.alias = 'archetypeConfig'
            JOIN umbracoNode ON umbracoNode.id = cmsPropertyType.dataTypeId
            WHERE cmsDataType.propertyEditorAlias IN ('Imulus.Archetype')
            AND cmsDataTypePreValues.value LIKE '%""dataTypeGuid"": ""{mediaPicker}""%'";

        var archetypes = database.Query<ArchetypeConfigRow>(archetypeSql).ToList();

        foreach (var archetype in archetypes)
        {
            // Find Vorto properties of archetypes using content pickers
            var sql = $@"SELECT cmsPropertyData.id, cmsPropertyData.contentNodeId, cmsPropertyType.alias, dataNvarchar, dataNtext, dataInt, cmsDocument.*
                FROM cmsPropertyData
                JOIN cmsPropertyType ON cmsPropertyType.id = cmsPropertyData.propertytypeid
                JOIN cmsDataType ON cmsDataType.nodeId = cmsPropertyType.dataTypeId
                JOIN cmsDataTypePreValues ON cmsDataTypePreValues.datatypeNodeId = cmsDataType.nodeId AND cmsDataTypePreValues.alias = 'dataType'
                JOIN cmsContentVersion ON cmsContentVersion.VersionId = cmsPropertyData.versionId
                JOIN umbracoNode ON umbracoNode.id = cmsContentVersion.ContentId
                JOIN cmsDocument ON cmsDocument.nodeId = umbracoNode.id
                WHERE cmsDataType.propertyEditorAlias IN ('Our.Umbraco.Vorto')
                AND cmsDataTypePreValues.value LIKE '%""propertyEditorAlias"": ""Imulus.Archetype""%'
                AND cmsDataTypePreValues.value LIKE '%""guid"": ""{archetype.UniqueID}""%'
                AND(dataNtext IS NOT NULL)
                AND(cmsDocument.published = 1 OR cmsDocument.newest = 1 OR cmsDocument.updateDate > (SELECT updateDate FROM cmsDocument AS innerDoc WHERE innerDoc.nodeId = cmsDocument.nodeId AND innerDoc.published = 1 AND newest = 1))
                ORDER BY contentNodeId, cmsDataType.propertyEditorAlias";

            var vortoDataToMigrate = database.Query<Row>(sql).ToList();
            var config = JsonConvert.DeserializeObject<Archetype.Models.ArchetypePreValue>(archetype.Value);
            var propertyAliases = config.Fieldsets.SelectMany(fieldset => fieldset.Properties)
                .Where(property => property.DataTypeGuid == mediaPicker)
                .Select(property => property.Alias);

            if (vortoDataToMigrate.Any())
            {
                foreach (var propertyData in vortoDataToMigrate)
                {
                    string udiValue;

                    if (!string.IsNullOrEmpty(propertyData.dataNtext))
                    {
                        // Vorto multilingual values
                        var vortoValue = JsonConvert.DeserializeObject<Our.Umbraco.Vorto.Models.VortoValue>(propertyData.dataNtext);
                        var udiValues = new Dictionary<string, object>();

                        foreach (var value in vortoValue.Values)
                        {
                            // Archetype fieldsets
                            var archetypeValue = JsonConvert.DeserializeObject<Archetype.Models.ArchetypeModel>(value.Value.ToString());
                            var fieldsets = archetypeValue.Fieldsets
                                .Where(fieldset => fieldset.Properties.Any(property =>
                                    propertyAliases.Contains(property.Alias)));

                            foreach (var fieldset in fieldsets)
                            {
                                // Properties using content picker
                                var properties = fieldset.Properties.Where(property =>
                                    propertyAliases.Contains(property.Alias));

                                foreach (var property in properties)
                                {
                                    var strValue = property.GetValue<string>();

                                    if (!string.IsNullOrEmpty(strValue))
                                    {
                                        if (!strValue.StartsWith("umb://"))
                                        {
                                            var uniqueIds = database.Query<Guid>($"SELECT uniqueId FROM umbracoNode WHERE id IN ({strValue})").ToArray();
                                            var uniqueIdsCsv = string.Join(",", uniqueIds.Select(id => $"umb://media/{id:N}"));

                                            property.Value = uniqueIdsCsv;
                                        }
                                    }
                                    else
                                    {
                                        property.Value = null;
                                    }
                                }
                            }

                            udiValues[value.Key] = ToDataValue(archetypeValue);
                        }

                        vortoValue.Values = udiValues;
                        udiValue = ToDataValue(vortoValue);
                    }
                    else
                    {
                        LogHelper.Info(typeof(MediaPickerIdToUdiMigrator), () => $"MigrateIdsToUdis (node id: {propertyData.contentNodeId}) skipping property {propertyData.alias} - null dataNtext");
                        continue;
                    }

                    LogHelper.Info(typeof(MediaPickerIdToUdiMigrator), () => $"MigrateIdsToUdis (node id: {propertyData.contentNodeId}) converting property {propertyData.alias} from {propertyData.dataNtext} to {udiValue}");
                    database.Execute("UPDATE cmsPropertyData SET dataNtext=@0 WHERE id=@1", udiValue, propertyData.id);
                }
            }
        }
    }
}

Change the views

Please note, that the code in your views will need to be modified accordingly.

For instance, we can now get the media inside a media picker in this simple way:

var image = Model.Content.GetPropertyValue<IPublishedContent>("temelineIcon")

Or the content inside a content picker:

var nodes = Model.Content.GetPropertyValue<IEnumerable<IPublishedContent>>("reports");

This is now the way we are used to with later versions of v7 and Umbraco v8, resulting in cleaner and easier to read code.

Final thoughts

It took a while to implement the code to migrate all our data types, and the process wasn't exempt from issues when deploying to different environments. The migration was a necessary thing to do and it was finally completed and the end result was satisfactory for both the client and us.

We could have opted for a manual migration, though. It would have taken some extra time from the live site, with errors and YSOD's while the pages are being migrated, but the total time spent might have been similar or even less. Well, maybe. But now we can reuse this migration process for other sites that will need it and that for sure contain a much much larger number of nodes using obsolete datatypes.

Also, an automated process, once we know is polished and working, is less prone to make errors and it certainly won't skip or forget any step.


Author

Miguel Lopez

Miguel is a web developer and Umbraco Master. He enjoys both technology and a good old fashioned book. He moved to the UK from Spain in 2015 and has been fully committed to Vizioz since.


comments powered by Disqus