Sitecore 8.2 Processing Server - Performance Tuning

-

Problem


On another Sitecore field experience episode... we performed a Sitecore Upgrade from 8.0.5 to 8.2.5, after that an issue was raised that Sitecore Analytics data and PathAnalyzer reports are not available (out of date/data missing), so we investigated and to make a long story short, based on log errors we identified some issues and after some Sitecore support tickets and research it was determined that we must Rebuild our Reporting database.

That involves a dedicated Processing Server processing xDB (MongoDB) data replicated between 2 geographically separate data centers. Server had 32GB RAM, 8 Core CPU.

Here's the problem discovered after running the RebuildReportingDB:
  • Sitecore Processing Server taking an abnormally long time to process data 
  • Projected time to process all data is 120 days (based on the History data in Analytics xDB)
  • Server not optimally using dedicated resources
Due to many reasons, we can't wait for 120 days. So... we worked with Sitecore support around performance tuning Processing Servers and this article explains the findings. 

In summary, the graph below explains the KPIs, one important one is the end result after tuning:

120 days based on default Sitecore Processing Server configuration down to 4 days with tuned configuration and optimizations!





Important Dependencies

Some things to confirm before running the ReportingDB Rebuild:
  • Ensure that all Sitecore servers have been correctly configured according to the role they are supposed to perform (e.g. CM vs CD vs ProcessingServer vs ReportingServer).

    Up to and including Sitecore 8.2 this used to be done by enabling/disabling configuration files as specified by Sitecore in an Excel spreadsheet. With Sitecore 9 changes, this can now be done by editing a configuration file(s) that specifies the server role through a setting.

    For 8.2, we automated that by re-using a Powershell approach which reads a CSV converted file based on Sitecore's Role Configuration Excel spreadsheet and the PS script enables/disables config files in the App_Config folder. Based on this article:
    [https://www.linkedin.com/pulse/processing-sitecore-config-files-powerhell-robert-senktas/] - Thanks Robert
  • Check the Sitecore application log for high error type counts. Do something about them if applicable.
  • On running each rebuild, be ready to run the Sitecore SQL Script to copy the Reporting Table data from Primary to Secondary Reporting DB (or opposite) right after you run rebuild using the Admin page: /sitecore/admin/rebuildreportingdb.aspx. By default (configurable) there's a 10 minute window to do this before the Processing Server starts its job (after clicking the Rebuild button on the admin page.).
  • Have your Processing Server close to the Primary Mongo source (i.e. within same Data Center/Network).

Optimal Performance Configuration

In our scenario this was the optimal settings:
  • Configured the Maximum Batch Size as 1024 KB
  • Configured the applicable Sitecore Aggregation Background Processes to use 8 Max Threads each
My next BLOG article will explain the tuning of configuration, what, where and explain findings.

Cheers /F

PS: Thanks to Sitecore Support for your assistance!



Comments

Post a Comment

Popular posts from this blog

Sitecore Custom Language Registration and Fallback

-
Image
On one of my recent Sitecore projects, we implemented custom languages for each country. The site supports multi-language for 100+ countries, with each country supporting between 1 and 4 languages. The requirement was that each country supported it's own country code with each language code, for example, Germany, the site supports English, German and French for Germany which looks like this: en-DE (English-Germany) de-DE (German-Germany) fr-DE (French-Germany) The site also uses language embedding in the URL and a lot of the new Languages uses Sitecore Language Fallback as Content Authors don't maintain content in each and every language variant. Problem: most of the site's Country and Language combinations are not supported out of the box by Windows Cultures and the default Sitecore Language lists which mean that we have to follow this process: Register a new Culture in Windows for each applicable Country and Language combination Add the new Language to Si...
Read more

Sitecore Salesforce Integration using Azure Microservices

-
Image
Sitecore SalesForce Integration using Azure Logic Apps During one of my projects, we had a requirement to integrate Contact Us form data on a Sitecore Website with SalesForce as new Lead objects. Our forms used to be Sitecore WFFM based but were rebuilt as custom Sitecore MVC forms. Looking at the existing Integration options available I found these two: 1        Sitecore Connect for Salesforce CRM 2.0 [https://dev.sitecore.net/Downloads/Salesforce_Connect/2x/Sitecore_Connect_for_Salesforce_CRM_20.aspx] 2)       FuseIT S4S Connector [https://www.fuseit.com/products/sitecore-salesforce-integration/] Sitecore Connect for Salesforce CRM 2.0 This Sitecore connector is developed by the “Sitecore Integrated Solutions Team” and is free and open source. It’s built on top of the Sitecore Data Exchange Framework. On a high level, the Salesforce Connect provides a connector for Salesforce.com that provides the following ca...
Read more