Laravel, Programming

The Importance of Eager Loading Laravel Relationships

Problem

Page took over 30 seconds to load locally on a local Vagrant virtual machine. After investigating the results were astonishing, over 1,000 queries executed in order to load the page. Not only was there an extraordinary number of queries executing, but also no constraints on the number of records loaded from each relationship.

Test and production environments didn’t show noticeable signs of slowness, but was clear on my local virtual machine that something was wrong.

Research

My research began using the Laravel debugbar. The debugbar helped me discover the amount of queries running for each page. Laravel debugbar has a query collector that conveniently displays the query count. An additional bonus is the number of duplicate queries along with which class called the query. Using these pieces of information I began tracking down the cause.

Cause

The cause was the classic N+1 problem. Essentially a loop was calling a relationship that had not been eager loaded. Each iteration of the loop caused the app to query the database.

Solution

The problem was easily solved using Laravel’s eager loading along with eager loading constraints.  Eager loading is loading your data up front before accessing the relationship. Because the relationship data was loaded up front accessing the relationship now pulls the value from memory instead of querying the database.

Eager loading constraints tell Laravel to only load specific records within a relationship. When you don’t need every record that a relationship has. E.g. Authors have books, but you only need love and thriller book types.

Eager loading with constraints reduced:

  • Query count from over 1k queries to 50 a whopping 2000% decrease in queries.
  • Number of records loaded from 7200 to 24.
  • Memory consumed by page from 140MB to 14MB.
  • Page load from 30 seconds to 2.5 seconds.

Example

Record labels have artists and artists have songs. If you wanted to get all artists on a record label along with their songs, but only the songs that reached top 10 on the billboards

Non-eager loaded:

$artists = Artist::take(500)->get();
$artist->songs;

Eager loaded without constraints (returns all songs):

$artists = Artist::with('songs')->take(500)->get();

$artist->songs;

Eager loaded with only songs that made top 10:

$artists = Artist::with('songs' => function ($query) {
    $query->where('highest_position', '<=', 10); 
})->take(500)->get();

$artists->songs;

Consider the following loop. If we don’t eager load the songs then for each artist a query to the database will be executed. This example may be a bit over the top because in most cases paginating the data would also solve part of the issue in this specific example, but I wanted to outline the potential issue.

foreach ($artists as $artist) {
    $artist->songs->each(function ($song) use ($artist) {
        $song->fullName = $artist . ' - ' . $song->title;
    });
}

How To Bind An Interface To An Implementation In Laravel

Binding an interface to an implementation promotes good coding practices. As a result the code is less coupled, more maintainable, and testable.

Why might someone want to bind an interface using Laravel? To put an abstraction between the application and the concretion. A concretion is a class that implements the interface. It is a specific implementation, in our example it will be Amazon S3 file storage provider. The application doesn’t care which implementation it receives just that it receives an implementation with the guaranteed functions.

Lets build functionality to interact with files on a storage provider such as Amazon S3. Initially all files will be uploaded and deleted on S3 only. There are other ways to accomplish this and the main goal is to demonstrate how to bind an interface to an implementation and not so much the actual code to upload to a storage provider.

Create An Interface

The interface will determine which functions are available on the concrete implementation through the binding.

Create The Concrete Implementation

The concretion is an Amazon S3 file storage provider. This file is where Amazon specific upload/delete functionality goes.

Create The Laravel Service Provider

The Laravel service provider is the mechanism that binds the FileStorageInterface to the S3FileStorage class implementation. This means if we use dependency injection for the FileStorageInterface or use App:make it will automatically resolve to the S3FileStorage class.

Using The Interface

In the below example we are not using the implementation directly. Instead it is automatically resolved through the service provider which returns the concrete implementation. Laravel will automatically resolve dependencies if they are in the __construct() functions. We can see below that the controller doesn’t know about Amazon and nor should it. I am only using the controller in this example to provide a demonstration. There are other places where we would use the storage provider that might better organize the code.

Final Thoughts

If we wanted to swap out for Dropbox or some other provider the impact is minimal, create a Dropbox implementation and change the provider. The application would now use the Dropbox implementation instead of the Amazon S3 implementation.

Following this pattern can also help separate the application from Laravel which could make reusing this code in other projects easier than if the Amazon S3 code were included in a model directly. Definitely think about how to best structure your code and not to fall into the framework convenience trap of putting everything in a model, controller or view. In most cases there are better places for code than those 3 areas.