SXA and how I dealt with css/sass serialization conflicts

The Problem we were running into:

We have several front end developers working on an SXA site I'm working on. But we started encountering conflicts all over the place with the scss files that are uploaded into the cms and serialized back down to yaml files. It was taking us 15 minutes at least to merge in changes between branches. We would have to resolve the conflicts and then just take source or destination on the yaml files representing the scss and css files. Then we would have to start gulp and go and save every file under the Theme to make sure they got uploaded into Sitecore so our serialized files were correct.

Attempt 1:

The first thing I did to try and fix the problem was to add a task to gulp to upload all the scss and css files into Sitecore. This sort of worked but was causing even more conflicts because it ended up changing the BlobId on the item even if the file was the same that was being uploaded. So we limped on with this for a little because it was still faster. We would just take source and re upload.

Attempt 2:

On this attempt i decided id just directly modify the yaml files. So I started building a powershell cmdlet. I planed on using the Rainbow.Storage.Yaml library to read and write to the yaml files. But i discovered that it depended on the Sitecore.Kernal.dll. I planned to make the powershell command open source so i decided to not use the Rainbow.Storage.Yaml library as i didnt want to publish the Sitecore.Kernal.dll with my module.

Attempt 3:

I decided to use the powershell module 'powershell-yaml'. I first created the command Compare-MediaItem. It takes two arguments the yml file to compare to and the source file. It load the source file into memory and converts it to a base64 string. It then loads the yml file finds the SharedFields section. Then looks for the Item whos Hint is 'Blob' it then finds the Value and compares it to the base64 string created before. If they are equal it returns true if there not it returns false if an error accrues it returns null.

Here's the command below it can also be found on github.

<#
#>
function Compare-MediaItem {
 param (
  [parameter(mandatory=$true)]
  [string]$YamlFile,
  [parameter(mandatory=$true)]
  [string]$SourceFile
 )

 if((Test-Path $YamlFile) -ne $true)
 {
  Write-Error ("YamlFile '{0}' does not Exist" -f $YamlFile)
  return $null
 }

 if((Test-Path $SourceFile) -ne $true)
 {
  Write-Error ("SourceFile '{0}' does not Exist" -f $SourceFile)
  return $null
 }

 $content = [System.IO.File]::ReadAllText($SourceFile)
 $base64 = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($content))

 $yaml = [System.IO.File]::ReadAllText($YamlFile)
 $yamlObject = $yaml | ConvertFrom-Yaml
 if($yamlObject.SharedFields)
 {
  $blob = $yamlObject.SharedFields | Where-Object Hint -eq 'Blob'
  if($blob -and $blob.Value -eq $base64)
  {
   return $true;
  }
  else 
  {
   return $false;
  }
 }
 else
 {
  Write-Error "Yaml does not have SharedFields"
  return $null
 }
}

This worked well in fact I'm still using this command now.

Then I stared writing a command to actually update the media item Update-MediaItem. I again used the powershell module 'powershell-yaml'. Here is the first version of the command on github. It also took two arguments 'yamlFile' and 'sourceFile'. I was able to get it to update the Value of the blob and the Length shared fields. The problem came in the fact that powershell-yaml was reordering values of the yaml. Well this caused unicorn to throw a fit a nice Yellow Screen Of Death. I did some research and found that i could pass in the argument -Ordered this worked for the main yaml properties but the SharedFields were still out of order. You can find this version here on github. I think if i defined the object I may have been able to resolve this issue but didn't really want to be that restrictive in case I missed something.

Attempt 4:

So the Test-MediaItem was working great so I still used the one from Attempt 3. For the Update-MediaItem I sat and thought and came up with the idea to just treat the yaml file as a normal text file. So I loaded in all the lines from the file into memory. I went through each line looking for a line that looked like this regex

$l -match "\sHint: Blob"

Then I jump down 2 lines and see if it matches this regex

$v -match "  Value:"

If it does I then take the base64 string I created from the source file and replace that line with this format

"  Value: {0}" -f $base64

I also look for a line that mates this regex

$l -match "\sHint: Size"

Then I go down 1 line and see if it matches this regex

$v -match "  Value:"

I then update this line with the length of the base64 string.

This worked great and Unicorn didn't through a fit and would sync these item in just as if it had generated them. You can find the latest version of this command on github.

The Script:

Now that I have two working Powershell commands 'Test-MediaItem' and 'Update-MediaItem'. I was able to use these to build a script that went through our media library. It looped through all the yaml files tries to find there matching scss,css,js file tests to see if they need to be updated and if they do updates them. Here is the script:

if(Get-Command -Module Shabadu -errorAction SilentlyContinue)
{
 Import-Module Shabadu
}
else
{
 Install-Module -Scope CurrentUser Shabadu -Force
 Import-Module Shabadu
}

function ProcessFolder {
 param (
  [string]$folderName,
  [string]$ext
 )

 Write-Host $folderName

 $yamlFolder = [System.IO.Path]::GetFullPath("$PWD\..\src\Serialization\Client\Client.Project.DotCom\Theme\DotCom\usa\Client\$folderName\")
 $sourceFolder = [System.IO.Path]::GetFullPath("$PWD\..\src\Theme\Client\$folderName")

 foreach($file in Get-ChildItem -Path $yamlFolder -Recurse)
 {
  if(-Not ($file -is [System.IO.DirectoryInfo]))
  {
   [string]$path = $file.FullName
   [string]$relativePath = [System.IO.Path]::GetFileNameWithoutExtension($file.Name)
   [string]$dir = $path.Replace($yamlFolder, '').Replace($file.Name, '')
   if($dir -ne '')
   {
    $relativePath = "{0}{1}" -f $dir,$relativePath
   }

   $sourceFile = [System.IO.Path]::Combine($sourceFolder, "{0}.$ext" -f $relativePath)

   if(Test-Path $sourceFile)
   {
    if((Compare-MediaItem $path $sourceFile) -eq $false)
    {
     if((Update-MediaItem $path $sourceFile) -eq $true)
     {
      Write-Host "Updated $path"
     }
     else
     {
      Write-Host "Could not update $path"
     }
    }
   }
   
  }
 }
}

ProcessFolder -folderName "sass" -ext "scss"
ProcessFolder -folderName "styles" -ext "css"
ProcessFolder -folderName "scripts" -ext "js"

Whats Next:

It would be nice to create a script that looked at git on your local box for merge conflicts on the serialized styles/scripts file and then take one side and then run this script and commit the result back in.