As developers, we often focus on functionality, ensuring that our code achieves the desired outcomes. However, how this code is written can significantly influence the speed, resource consumption, and scalability of the applications we build.
In this post, we will explore some ways in which code quality can impact system performance.
Overview and Motivation
Not knowing exactly how the framework that we use works may jeopardize our code performance and result in not efficient algorithms. As we develop on top of an existing framework that manages the application communication with the database, it is important to understand how it is done, and how it may impact the performance of our code.
This blog post focus on the performance impact between the Host application and the Database Layer. In the future, some analysis can be performed on the performance impact between UI and Host application.
The analysis presented, in this blog post, was performed using a DEE Action in local environment (application and database in the same machine).
Loading and Lazy Loading
Lazy Loading
Lazy loading is a design pattern commonly used in computer programming to defer initialization of an object until the point at which it is explicitly needed.
Below you can see a generic overview of the Lazy Loading pattern in the framework for the Product property in the Material class (not actual code of the property).
private IProduct _Product;
public IProduct Product
{
get
{
Int64 propertyId = this.GetNativeValue<Int64>("Product");
if (_Product == null)
{
_Product = _entityFactory.Create<IProduc>();
_Product.DefinitionId = propertyId;
}
if (_Product?.Id > 0 && string.IsNullOrEmpty(_Product.Name))
_Product.Load();
return _Product;
}
set
{
_Product = value;
}
}
The Lazy Loading pattern is automatically implemented in every Reference property (Product, Resource, …), even in custom entity types.
Load method and the impact of levelsToLoad
All entities have the Load method that is equally implemented, and it has one parameter called levelsToLoad that may have a tremendous impact in the system performance.
The method is responsible for loading all information of the instance, and it may retrieve it from the database, or the application cache.
Consider the following representation of the Load method for an entity instance.
public void Load(int id, int levelsToLoad) {
// Retrieves the information from the database
var dbReader = ReadFromDatabase( this, id );
// foreach property in the Entity Type that is Active and is not an Attribute
var properties = this.EntityType.Properties;
foreach (var property in properties) {
this[property.Name] = database[property.Name];
if ( property is EntityType && levelsToLoad > 0 ) // Product, Resource, etc.
{
var instance = new();
instance.Load(database[property.Name], levelsToLoad - 1);
}
}
}
Analysis
Loading one Material instance and accessing the Product property using lazy loading, took in average 5 milliseconds.
material.Load(levelsToLoad: 0);
// access the property while NULL, to force the lazy loading logic
if (material.Product != null) { }
Loading one Material instance, with levels to load as one, and accessing the Product property using lazy loading, took in average 20 milliseconds.
material.Load(levelsToLoad: 1);
// access the property while not NULL, as the references were loaded by levelsToLoad: 1
if (material.Product != null) { }
Individually, loading 500 Material instances and accessing the Product property using lazy loading, took in average 2900 milliseconds.
// rows are retrieved from a query execution
foreach(DataRow dr in rows) {
material.Load(dr.Field<string>("Name"), levelsToLoad: 0);
// access the property while NULL, to force the lazy loading logic
if (material.Product != null) { }
}
Individually, loading 500 Material instances, with levels to load as one, and accessing the Product property using lazy loading, took in average 5000 milliseconds.
// rows are retrieved from a query execution
foreach(DataRow dr in rows) {
material.Load(dr.Field<string>("Name"), levelsToLoad: 1);
// access the property while not NULL, as the references were loaded by levelsToLoad: 1
if (material.Product != null) { }
}
Loading Single Instance or Collection
Although this section focus on the loading operations, keep in mind that this can be applied in any operation (Save, Track-In, etc.).
Whenever a list of instances is important for your functionality, always keep in mind that performing actions in a collection is always better than performing them individually, to reduce the number of communications with the database.
The framework supports collection operations for almost everything, take advantage of them.
Analysis
Individually, loading 500 Material instances to have a loaded collection of materials, took in average 1400 milliseconds.
// rows are retrieved from a query execution
IMaterialCollection materials = _entityFactory.CreateCollection<IMaterialCollection>();
foreach(DataRow dr in rows) {
IMaterial material = _entityFactory.Create<IMaterial>();
material.Name = dr.Field<string>("Name");
// Load is performed individually
material.Load();
materials.Add(material);
}
Collectively, loading 500 Material instances to have a loaded collection of materials, took in average 90 milliseconds.
// rows are retrieved from a query execution
IMaterialCollection materials = _entityFactory.CreateCollection<IMaterialCollection>();
foreach(DataRow dr in rows) {
IMaterial material = _entityFactory.Create<IMaterial>();
material.Name = dr.Field<string>("Name");
materials.Add(material);
}
// Load is performed in collection using IMaterialCollection
materials.Load();
Using Native Value and Lazy Loading for comparisons
Several times we want to check if some conditions are met, in order to apply our custom logic.
In this section, I use a collection of 500 materials (previously loaded with levelsToLoad as zero), and want to validate which ones are of Product MyProduct.
Using the Lazy Loading approach, and accessing the name of the instance, the check takes in average 1600ms.
foreach (IMaterial material in materials) {
if (material.Product.Name == "MyProduct") {
// code goes here
}
}
Using the NativeValue approach, and checking using the Product DefinitionId (because it is a versionable object, otherwise it would be the Id), the check takes in average 5ms (basically the time to load the Product once).
IProduct myProduct = _entityFactory.Create<IProduct>();
myProduct.Load("MyProduct");
foreach (IMaterial material in materials) {
if (material.GetNativeValue<long>("Product") == myProduct.DefinitionId) {
// code goes here
}
}
If the NativeValue approach is not possible, and is not possible or effective to compare with Ids, it is also viable to have a local cache to reduce the call stack complexity. This approach takes around 50 milliseconds (where the number of different products is 5 or 6).
Dictionary<long, IProduct> localProductCache = new(); // holds a pointer for the IProduct reference for each different Id found
foreach (IMaterial material in materials) {
long id = material.GetNativeValue<long>("Product");
localProductCache.TryGetValue(id, out IProduct materialProduct);
if (materialProduct == null) {
value = material.Product;
localProductCache[id] = value;
} else {
material.Product = materialProduct;
}
if (material.Product.Type == "myProductType") {
// code goes here
}
}
Final Takeaway
- Collection is always better over instance operations
- Try to reduce and group database operations as much as possible
- Casually perform some performance checks on services where your customizations are running
- You can use the MES System Performance report for that, in your integration environment
- Keep in mind that everything stacks up, your code normally runs on top of the framework code that may already be performance heavy
Author
Hello, my name is César Meira 👋
I’ve been working for some years at Critical Manufacturing. I am working in Deployment Services for M1 Team. You can check me at LinkedIn
