Integrating Merchello with an EPOS System

This blog post was actually written a couple of years ago and was never published. It seemed a shame for it to be lost, so today we are publishing it anyway :)

It was a little while ago that we built an e-commerce site for a local business. The development was split into 2 phases that spread across a noticeable span of time, plus a later third phase intended to add extra functionality and improve user experience while on site.

This is going to be a series of two articles covering the two phases in which we implemented the e-commerce functionality and the integration with their own platform. In this first article, we will treat how we dealt with the uploading of products from their store. The solution was developed using Umbraco CMS and the e-commerce plugin Merchello. Although both Umbraco and Merchello have suffered major upgrades since then, this is still a very interesting topic and the strategies and solutions used here will still be valid in most scenarios. Also, it was such an enjoyable project (through which we also learned a lot) to work on that we feel we have to share it.

This article contains code snippets intended to give a deeper explanation of some parts of the process. If you are a developer or have some coding knowledge you might find them useful (we hope so). Otherwise, you can just skip them and read through the text, as it will give you a clear idea of the initial goals and the means to achieve them.

Overview

In the client's words, the new website had to be a window shopping experience for users, where they could display all the products they had in their store, plus increase the number of online sales. Their main requisite was to integrate their EPOS system with the new website, so it would automatically update and display the products they have in their store as well as download and process the online orders.

We decided at the very beginning that the e-commerce side would be handled by Merchello.

The project was divided into two separate phases. The first phase consisted of building the e-commerce site from scratch, more in line with the current image of the store, but without the proper e-commerce functionality. This is, the site must display the products, but the actual purchasing of products was to be implemented at the later stage. It was required that this first phase include the synchronizing of the website with their store products.

Solution setup

Umbraco 7.4.1

This was the latest release when we started working on the project.

Merchello 2.2.1

Merchello is an open source e-commerce package for Umbraco that integrates seamlessly with the CMS, as well as a powerful tool that lets you fully customize workflows and gateway providers. It also allows adding extended data to products using Umbraco document types, of which we will make use.

Please note that some of the code examples in this article might differ from the classes and methods in newer versions of Merchello.

Encore

Encore is the EPOS software by Anagram Systems used by the client in their store. They have been using it for years and it integrates several workstations. A custom plugin will let us pull and push information from and to the site.

Integration

The integration between the EPOS system and the website was done through a specific module provided by Encore. This module is in charge of packing the products that need to be uploaded to the website and internally preparing them as a data set so we can then implement the methods that will do the actual requests to the site. Encore executes these methods whenever there is an update. We were responsible for writing the necessary code to communicate with the site and push the products data set.

On the other side, our API is in charge of receiving the products and convert them into Merchello products, as well as adjusting the products categories and updating the category pages. The API will work with a series of methods matching the requests done by Encore. We also set up an FTP server for the client so they can bulk upload their product images. Nevertheless, this has been rarely used since we also integrated the uploading of pictures within the Encore requests. The main methods in our API are:

• Login
• Update Web Product (repeated for every product)
• Upload Images
• Finalise
• Logout

Process

Security

We, of course, had to secure the process, and at the same time, we wanted to have the possibility of having multiple users uploading content, each from a different station at the store. Before attempting to upload any product, the specific client has to log in using their own username and password (corresponding to a specific Umbraco member) plus an authorization token only shared by the API and the store. The client is then returned a session GUID valid only for a few minutes. The client must include this GUID together with their username in all following requests during the process.

Environment

In order to make the API more scalable and make it possible to adapt to future integrations with different EPOS platforms, we included an environment variable, so the API will know to which classes it needs to delegate the request. This is implemented by making use of the factory pattern. The environment parameter is sent with the initial login request, so the following requests will be done under that particular environment.

Upload

The uploads are processed for one product at a time, as this is the way the Encore plugin works. It also allows the client to have a visual indication of the overall progress. Basically, the products are created programmatically in Merchello, so we need to go a bit under the hood to ensure the process is done all right.

The upload method receives a JSON encoded product that needs to be deserialized into our own WebProduct object. Since we are now working on a specific environment, this environment will be in charge of doing so by using a special serializer class, which will, in turn, use specific serializer settings, as well as adjust values to meet our needs. For instance, the minimum date returned by Encore will differ from the minimum date in .NET, so our serializer has to handle those differences. If you are interested, here is a little snippet of our custom serializer.

/// <summary>
/// The JSON serializer settings.
/// </summary>
private readonly JsonSerializerSettings settings;

/// <summary>
/// Initialises a new instance of the <see cref="EncoreEnvironmentSerializer"/> class.
/// </summary>
public EncoreEnvironmentSerializer()
{
    settings = new JsonSerializerSettings
    {
        ContractResolver = CustomEncoreContractResolver.Instance
    };
}

/// <summary>
/// Parses a JToken to object.
/// </summary>
/// <typeparam name="T">
/// The object type.
/// </typeparam>
/// <param name="token">
/// The token.
/// </param>
/// <returns>
/// The parse object.
/// </returns>
public T ToObject<T>(JToken token)
{
    var result = JsonConvert.DeserializeObject<T>(token.ToString(), settings);

    return result;
}

The upload process for our specific environment includes 3 types of actions; the one to be executed is indicated by a property included in the WebProduct we just deserialized.

• Normal. A regular upload/update of a product.
• Stock only. Only the stock quantity is updated.
• Remove. The product is removed.

As "Normal" is the most complex action, we will only describe this one and omit the other two. It comprises a series of steps, all of them necessary for the successful inclusion of the product to the website.

1. Create / Update

Needless to say, if the product already exists in Merchello, we don't need to create it but only update its properties.

The create method will basically create a new Merchello product, populate the main basic properties and add it to the product catalog.

// Create new product
var product = this.ProductService.CreateProductWithKey(
    encoreWebProduct.Base.Title.Trim(),
    encoreWebProduct.Base.ItemSku,
    encoreWebProduct.Base.Price ?? 0);

// Product default properties
product.Available = true;
product.Shippable = true;
product.Taxable = false;
product.Download = false;
product.OnSale = false;
product.OutOfStockPurchase = false;
product.TrackInventory = true;

// Add to the catalogue
if (this.DefaultCatalog != null)
{
    product.AddToCatalogInventory(this.DefaultCatalog);
}

If you wonder where ProductService and DefaultCatalog come from, they have been previously injected to this class. Just have a look at this other snippet:

var productService = MerchelloContext.Current.Services.ProductService;
var warehouse = MerchelloContext.Current.Services.WarehouseService.GetDefaultWarehouse();
var catalog = warehouse.WarehouseCatalogs.FirstOrDefault();

As mentioned previously, Merchello can add extended content to the products by using regular Umbraco document types, which lets us add new properties that are not part of the Merchello products by default. This is simply done by attaching a specific document type to the product. It makes the products totally customizable and adapted to our needs. They also admit multiple language versions, so the products can have different values (body text, description, etc) depending on the culture. If you have used or are using Merchello, you probably know what I am talking about and probably like this feature as much as we do. The problem is, that when we add a product programmatically the extended content is not created so we must not forget to include that in our code. After creating the extended content, you must make sure to save the product, otherwise, the extended values will not be available for you to update them.

// Extended content
var detachedContent = this.CreateProductDetachedContent(product);
product.DetachedContents.Add(detachedContent);

// Save the product so the extended content will be available
this.ProductService.Save(product);

And here are the methods used to create the detached content:

/// <summary>
/// Creates the product detached content.
/// </summary>
/// <param name="product">
/// The product.
/// </param>
/// <returns>
/// The <see cref="IProductVariantDetachedContent"/>.
/// </returns>
private IProductVariantDetachedContent CreateProductDetachedContent(IProduct product)
{
    var detachedContentType = DefaultProductUpdater.GetStandardProductDetachedContentType();

    var detachedContentTemplateId =
        ApplicationContext.Current.Services.ContentTypeService.GetContentType(ExtendedContentTypeAlias)
            .DefaultTemplate.Id;

    var detachedContent = new ProductVariantDetachedContent(
        product.ProductVariantKey,
        detachedContentType,
        _defaultCulture.Name) { TemplateId = detachedContentTemplateId, CanBeRendered = true };

    return detachedContent;
}

/// <summary>
/// Gets the detached content type for product entities.
/// </summary>
/// <returns>
/// The <see cref="IDetachedContentType"/>.
/// </returns>
private static IDetachedContentType GetStandardProductDetachedContentType()
{
    var detachedContentTypeService = MerchelloContext.Current.Services.DetachedContentTypeService;
    var contentType = ApplicationContext.Current.Services.ContentTypeService.GetContentType(ExtendedContentTypeAlias);

    var detachedContentType =
        detachedContentTypeService.GetDetachedContentTypesByContentTypeKey(contentType.Key)
            .FirstOrDefault(x => x.EntityType == EntityType.Product) ??
                detachedContentTypeService.CreateDetachedContentTypeWithKey(
                    EntityType.Product,
                    contentType.Key,
                    ExtendedContentTypeAlias);

    return detachedContentType;
}

At this point, we found a bug in Merchello, related to the product URL. The product in the back office will include a URL slug, which will be the exact address for the specific product page (not exactly, though, as we can configure it to have prefixes). This bug was fixed afterward as we added it to the issue tracker, and is now closed. The issue was related to different products with the same name when it created the same URL slugs for them. At the time that we were working on this project we had to write some extra code to override this behaviour.

After the product has been successfully created (or when it is only an update) we have to populate/update its properties. Many of them are IProduct properties (properties included in Merchello products by default) and can be accessed directly, like price, stock level, etc. But many others are part of the custom extended content. Editing these from within the back office is pretty simple and straightforward, but it is not that simple when you want to do it from the code. To overcome this, we created two extension methods that allow us to set the extended values of a product. They can be used to set the same value for all language versions of the product, or for just a specific one.

/// <summary>
/// Updates a product detached property for all cultures.
/// </summary>
/// <typeparam name="T">
/// The type.
/// </typeparam>
/// <param name="product">
/// The product.
/// </param>
/// <param name="key">
/// The key.
/// </param>
/// <param name="value">
/// The value.
/// </param>
public static void UpdateDetachedPropertyForAllCultures<T>(this IProduct product, string key, T value)
{
    var jsonValue = JsonConvert.SerializeObject(value);

    foreach (var detachedContent in product.DetachedContents)
    {
        detachedContent.DetachedDataValues.SetValue(key, jsonValue);
    }
}

/// <summary>
/// Updates a product detached property.
/// </summary>
/// <typeparam name="T">
/// The type.
/// </typeparam>
/// <param name="product">
/// The product.
/// </param>
/// <param name="key">
/// The key.
/// </param>
/// <param name="value">
/// The value.
/// </param>
/// <param name="cultureName">
/// The culture name.
/// </param>
public static void UpdateDetachedProperty<T>(this IProduct product, string key, T value, string cultureName)
{
    var jsonValue = JsonConvert.SerializeObject(value);
    var detachedContent =
        product.DetachedContents.FirstOrDefault(
            x =>
                x.DetachedContentType.Name == ExtendedContentTypeAlias &&
                x.CultureName == cultureName);

    detachedContent?.DetachedDataValues.SetValue(key, jsonValue);
}

2. Update categories

In most cases, a product will not stand by itself but will part of a product category or collection. Merchello has the ability to create product collections and add products to them. The product we received in the request contains category information, so it is just two simple things we need to do here:

a) Create the collections in Merchello (if they don't exist yet). Collections can be nested, creating a hierarchical category tree. Since this information is in the product we can create the collections as needed with the following code:

/// <summary>
/// Updates a collection in the collection tree.
/// </summary>
/// <param name="parentCollection">
/// The parent collection
/// </param>
/// <param name="categoryName">
/// The category name.
/// </param>
/// <returns>
/// The <see cref="IEntityCollection"/>.
/// </returns>
public IEntityCollection UpdateProductCollection(IEntityCollection parentCollection, string categoryName)
{
    var providerKey = Constants.ProviderKeys.EntityCollection.StaticProductCollectionProviderKey;

    // Get the root node of collections
    if (parentCollection == null)
    {
        var rootCollection =
            this.CollectionService.GetRootLevelEntityCollections(EntityType.Product)
                .FirstOrDefault(
                    x => string.Equals(x.Name.Trim(), categoryName.Trim(), StringComparison.CurrentCultureIgnoreCase)) ??
            this.CollectionService.CreateEntityCollectionWithKey(EntityType.Product, providerKey, categoryName);
        return rootCollection;
    }

    // Check whether the collection exist in the tree
    var childCollection =
        this.CollectionService.GetChildren(parentCollection.Key)
            .FirstOrDefault(x => string.Equals(x.Name.Trim(), categoryName.Trim(), StringComparison.CurrentCultureIgnoreCase));

    if (childCollection != null)
    {
        return childCollection;
    }

    // Create the collection
    childCollection = this.CollectionService.CreateEntityCollection(EntityType.Product, providerKey, categoryName);
    childCollection.ParentKey = parentCollection.Key;
    this.CollectionService.Save(childCollection);

    return childCollection;
}

b) Add products to the new categories. Once we have the proper collection tree, we just add the product to it.

product.AddToCollection(collectionId);

3. Update content tree

The process described until now will successfully create a product in Merchello. It will be available via its URL slug and will be displayed using the template associated with its document type. But we also need a category page to show the whole range of products for a collection, and that is not something that Merchello will automatically do for us. Although Merchello incorporates a couple of data types that can be used in the back office to select products or collections, we still need to create the content page itself.

So, as part of the upload process, we incorporated a method to update the content tree. Using Umbraco methods and the Umbraco ContentService we are able to programmatically create a tree structure that matches the collection tree and associate a Merchello collection to it. In this way, when a product is part of a collection that still doesn't exist in our back office, the code will create the appropriate collection content page and publish it. Of course, the content will still need some input from the editors to add custom text, images, and SEO properties. But the collection page is immediately available to potential customers. It also turns out to be quite convenient for the client since they don't have to think whether they need to add a new page to the content tree whenever a product is updated (which will occur very frequently), and they just need to keep track of their current collection trees and keep up to date with the edits.

4. Upload images

As we mentioned previously we set up an FTP server for the client to bulk upload their product images, but it turned out it was more convenient to integrate that with the product update process and let the client not worry about which images they need to upload or which are missing. The WebProduct object sent in the request contains information about images paths that can be matched against a physical path on the server. Then in the upload method, we can check which of these images are missing and return this information in the HTTP response. We added some extra lines to the Encore plugin to directly upload these images in a new request and let them be saved in the appropriate path. This proved to be way handier than the FTP, which I am not sure the client ever used.

Finalise process

After all products have been uploaded, we call a finalise method that will update and check the integrity of product relations (that is, related products, cross sales and up sales) and ensure these exist on the site.

Finally, there is a logout request which will clear the session variables.

Epilogue to the first part

The integration of an EPOS system with Merchello is a fantastic way to keep a website up to date with a business with almost no effort from the business owner.

As they already put a lot of care and hard work on their store products and keep their systems constantly updated with new products, price changes and offers, stock levels, etc, their website will be able to reflect all this internal work and changes and will make it feel it is really part of their business and not an add-on. Even when the actual online shopping and checkout was not implemented yet at this stage, the website was already a valued asset and a highly effective window shopping experience for customers.

We learned a lot on the way, not only about Merchello and e-commerce but also about how the clients feel their online and physical business have to function and integrate.

The second part of this article will treat about how the online orders are integrated into the EPOS system and the configuration of the payment providers in Merchello.