Configuring Drupal 8 for a simple Git development workflow

In this post we want to share our experience in configuring and managing a Drupal 8 website project for a basic and simple Git development workflow.

Content

• Installing Drupal 8 with Drush
• Configuring the Drupal 8 installation
• Development workflow
• Configuring production (and staging) server
• Deployment
• Removing user generated content from Git

Installing Drupal 8 with Drush

At first we created a new Drupal website using Drush (take a look here if you need to install Drush).

Create a new Drupal 8 project:

$ drush pm-download drupal --drupal-project-rename=[project_name]

This command will create the [project_name] folder containing a Drupal 8 website ready to be installed.

Now we must create a new empty database within MySQL, named [db_name]. You can do this with your favourite MySQL tool.

After created the database, go inside the project folder and install Drupal:

$ cd example_project
$ drush site-install standard \
    --db-url='mysql://[db_user]:[db_pass]@localhost:[db_port]/[db_name]' \
    --account-name=joe --account-pass=mom123 \
    --site-name=Example \
    --site-mail=info@example.com

Optionally you can install Drupal with a custom language adding this option in the command above:

    --locale=it

specifying the locale string for the language you want to install.

Alternatively to this command, you can install Drupal running the web wizard.

Once the installation is finished we can test it running the Drush’s built-in server from the project’s root:

$ drush runserver 8000

then access the website with a web browser at the url:

http://localhost:8000

You will see an empty Drupal website, with the default theme.

To access the admin panel go to:

http://localhost:8000/user/login

and fill the form with username and password.

Configuring the Drupal 8 installation

You can find the settings.php file within the sites/default/ directory. By default, Drupal will set this file and the whole default directory to read-only after the installation process. Sadly this could be a little annoying in a Git workflow, since if Joe change some settings and push its changes, when Bill will do a pull, its local Git will complains that it can’t modify the settings.php file.

Given that in our development workflow sometimes we needs to change some settings after the installation process, we decided to add a settings folder, writable by default, and move inside it all site settings. To preserve the security, we will set the folder to read-only in the production environment.

Configuring settings.shared.php

The file settings.shared.php will contains all the settings shared between all developers and environments (local, staging and production), and will be versioned with Git. For environment-specific settings we’ll create the settings.local.php file.

Create the sites/default/settings folder (maybe you have to make the default directory writable to do this. Drupal will restore permissions in a later moment).

$ chmod +w sites/default
$ mkdir sites/default/settings

Now copy settings.php to settings/settings.shared.php and make it writable:

$ cd sites/default
$ cp settings.php settings/settings.shared.php
$ chmod 644 settings/settings.shared.php

Open settings.php and replace all its content with:

<?php

include __DIR__ . '/settings/settings.shared.php';

If you need to add a custom services.yml file, remember to place such file within the settings folder.

Open settings.shared.php and enable a list of trusted host patterns adding this under the Trusted host configuration section:

/**
 * Trusted host configuration.
 * ...
 */
$settings['trusted_host_patterns'] = array(
  '^localhost$',
  '^127.0.0.1$',
  'example\.dev$',
  'staging\.example\.com$',
  'example\.com$',
);

Configuring settings.local.php

Create the settings.local.php file, within the settings folder, copying it from sites/example.settings.local.php:

$ cp sites/example.settings.local.php sites/default/settings/settings.local.php

Now let’s configure the settings.local.php for our local environment. Move database settings from settings.shared.php and put them at the bottom of settings.local.php:

// ...

/**
 * Database settings:
 * ...
 */
 $databases['default']['default'] = array (
  'database' => 'example_project_db',
  'username' => '[db_user]',
  'password' => '[db_pass]',
  'prefix' => '',
  'host' => 'localhost',
  'port' => '[db_port]',
  'namespace' => 'Drupal\\Core\\Database\\Driver\\mysql',
  'driver' => 'mysql',
);

Then search the settings['rebuild_access'] line within the file and comment it. Decomment all the settings['cache'] settings in order to disable the cache for the development environment. In the production server we can leave these lines commented in order to enable the caching system.

Optional: add custom 'trusted_host_patterns' for your local machine:

/**
 * Add custom 'trusted_host_patterns'
 * 
 * You can add your local IP address here.
 */
$settings['trusted_host_patterns'][] = '^192\.168\.1\.123$';

We can now clean the settings.shared.php file and include local settings. So, let’s open settings.shared.php and remove database settings (if you haven’t already done it). We keep database settings in the local settings file. Then include the settings.local.php file decommenting this block of code and moving it at the end of the file:

// ...

/**
 * Load local development override configuration, if available.
 * ...
 */
if (file_exists(__DIR__ . '/settings.local.php')) {
  include __DIR__ . '/settings.local.php';
}

Since we will ignore settings.local.php from Git, we create the file example.settings.local.php that will be used as example file for anyone will pull the code from the repository. So when Bill will pull down the project from Git, he can copy example.settings.local.php to settings.local.php within the settings folder and configure it for his local environment. So, copy your settings.local.php to sites/example.settings.local.php (overwriting it):

$ cp sites/default/settings/settings.local.php sites/example.settings.local.php

Disabling Twig’s cache from development.services.yml

The settings.local.php includes sites/development.services.yml by default.

We can disable Twig’s cache here adding this code:

# ...

# Disable Twig cache
parameters:
  twig.config:
    debug : true
    auto_reload: true
    cache: false

Rebuild the cache

After settings has been modified is a good practice to clear and rebuild the cache. We can do this with Drush:

$ drush cache-rebuild

Configuring .gitignore

Create the .gitignore file inside the project’s root, with this content:

# Ignore core when managing all of a project's dependencies with Composer
# including Drupal core.
# /core

# Core's dependencies are managed with Composer.
/vendor

# Ignore configuration files that may contain sensitive information.
/sites/*/settings/settings.local.php

# Ignore paths that contain user-generated content.
#/sites/*/files/*
#/sites/*/private/*
#!/sites/*/files/config_*

# Ignore SimpleTest multi-site environment.
/sites/simpletest

The settings.local.php is ignored, so anyone can have its own settings.local.php file, different for each developers and environments.

Directories containing user-generated content, as /sites/default/files or /sites/default/private (if you enabled private files), should be excluded from Git. But when we start to develop a website we found more convenient to share these contents within Git since usually we generate them and they are useful for the development. When the website will be deployed we’ll remove these folders from Git (see later).

If you prefer, you can start ignoring content folders right away by decommenting rows under the comment Ignore paths that contain user-generated content.

Export the database

Within the Drupal’s database are stored some website configurations, installed modules, current theme, content types, views, nodes, etc. All of this must be shared between developers. The simplest way is to export a database dump and keep it on Git. When Bill (a developer) will get the code from Git he will load the last database export on his local machine.

Our convention is to create a database/exports folder and keep database exports inside this folder.

You can create the database dump with a MySQL tool or using mysqldump:

$ mysqldump -u [db_user] -p[db_pass] [db_name] > database/exports/[db_name]--[Y-m-d]--[H:i].sql

Add an .htaccess file, with the following content, within exports in order to avoid to share your database with the world:

<Files *.sql>
  order allow,deny
  deny from all
</Files>
<Files *.gz>
  order allow,deny
  deny from all
</Files>

The biggest drawback of having configurations stored inside the database is that only one developer at a time can working on configuring the website, and more in general only one developer at time can working on the database.

In Drupal 8 you can improve this workflow managing configurations files (exporting and importing Drupal configurations). See here for more informations: https://www.drupal.org/documentation/administer/config.

First commit and push

Now that we have configured the Drupal 8 installation, we can do the first commit and push the Drupal project on the central Git repository.

$ git add .
$ git commit -am "First commit"
$ git push origin master

Development workflow

When Bill start to working on the project the first time he have to:

1. Clone the project from the central Git repository on his local PC.
2. Load the database dump from database/exports/.
3. Create the settings.local.php file, within the settings folder, copying it from example.settings.local.php and configure it with his own local settings.
4. Run composer install within the project root.
5. Rebuild the drupal cache with drush cache-rebuild.
6. Start the server with drush runserver 8000.

Commit, push, pull

Once cloned and configured the project, both Joe and Bill (and any other developer) can proceed the development workflow with a commit-push-pull routine.

Remember that with the model described in this post, only one developer at time can modify the database. After he finish to modify the database, he can export the database and share it on Git so any other developer can import the new database.

We are assuming also to working directly on the master branch, without any other branch. An improvement to this workflow could be to use a branching model with at least a develop branch other than the master.

Configuring production (and staging) server

Production and staging environment have the same configuration. So we show how to configure one of them and the other will be the same.

We configure automatic deployment with Git, so that when we do

$ git push production master

the code will be pushed and deployed on the production server.

In the following we show how to configure a bare repository and a base post-receive hook script for the automatic deployment. Depending by your hosting service, you may need to configure the server installing all what it’s needed (php, git, composer, drush) and modify the post-receive script to work with your environment. For example, here there is a good tutorial by Digital Ocean for set up automatic deploy with Git on Ubuntu VPS. Here you can find some hint about configuring Git and Composer on a GoDaddy’s shared hosting.

Let’s start creating a bare repository on the production server:

$ cd ~/git
$ git init --bare --shared [project_name].git

You can create the repository inside any directory. In the code above we created it inside a git folder within the user’s home.

The bare repository will contain the code pushed to the production remote from Git. We need to configure a post-receive hook in order to copy the code from the bare repository to the website root directory.

Enter inside the folder [project_name].git/hooks and create the post-receive file:

$ cd [project_name].git/hooks
$ vim post-receive

Write inside the file the script below, replacing the value of APP_WEB_DIR and APP_GIT_DIR with the website document root and the git repository root respectively:

#!/bin/sh

# Set up our PATH variable and export it. In this way, within the ~/bin folder,
# you can link the right php cli version to use within this script and link
# the composer executable used below
PATH="/home/netgloo/bin":$PATH
export PATH

# App directories
APP_WEB_DIR="/home/netgloo/public_html"
APP_GIT_DIR="/home/netgloo/git/[project_name].git"

# Set write permissions on sites/default/settings/*
cd ${APP_WEB_DIR}
chmod -R +w sites/default/settings
cd ${APP_GIT_DIR}

# Copy the content from the repo to the server live directory
git --work-tree=${APP_WEB_DIR} --git-dir=${APP_GIT_DIR} checkout -f

# Remove write permissions from sites/default/settings/*
cd ${APP_WEB_DIR}
chmod -R -w sites/default/settings

# Run composer install
composer install

You can use any text editor you want instead of vim to create and edit the file (via FTP or with some web interface).

After created the hook, don’t forget to make it executable:

$ chmod +x post-receive

Adding the production remote on local PC

In your local PC go inside the project folder and add the production remote to Git:

$ git remote add production ssh://joe@example.com:[ssh_port]/~/git/[project_name].git

Now you can deploy the code with:

$ git push production master

Deployment

The first time you will push the code on the production server you will have to:

1. Run composer install within the project root.
2. Create the settings.local.php, within the settings directory, copying it from example.settings.local.php and configure it for the production environment (for example, you may want to enable the cache and the aggregation of CSSs and JavaScripts).
3. Load the database dump from database/exports/.
4. Rebuild the drupal cache with drush cache-rebuild or through the Drupal Admin at Config > Development > Performance.

After the first configuration, when you want to deploy new code you have to:

1. Push the code on the production remote with git push production remote.
2. If there was changes on the database: load the last database dump.
3. Rebuild the drupal cache with drush cache-rebuild or through the Drupal Admin at Config > Development > Performance.

Once deployed the website, a common workflow is to working on the website’s code from a local development environment and then push the code on production when it’s ready. Whereas the website’s content (generated by users) is modified on the production environment and it should be downloaded if needed for development.

Production website
   ▲
   ┃        ┃
   ┃        ┃    
 Code    Content
   ┃        ┃
   ┃        ┃
            ▼
Local (development)

With Drupal, the user generated content is both inside the sites/default/files folder (and eventually inside the sites/default/private folder also) and inside the database. So you may need to download both the files folder’s content and the database.

Removing user generated content from Git

As already said, all the user generated content should not be tracked within Git. We have choosen to temporarily keep it within Git during the first development phase. Once the website is deployed, we should ignore (and remove) such content from Git.

Important: the following steps will remove such content from Git. An error performing these steps could bring to accidentally remove some content from the production server. So we strongly suggest you to carefully follow them and to make a backup before proceeding.

First we need to add folders containing user-generated content inside the .gitignore file:

# Ignore paths that contain user-generated content.
/sites/*/files/*
/sites/*/private/*
!/sites/*/files/config_*

Then commit and push the new .gitignore file:

$ git commit -am "Ignore user generated content"
$ git push origin master

Adding these folders in the .gitignore will do not delete them. So now we are going to remove ignored folders from the Git repository. But first we must prevent that such content will be deleted on the production server.

So, first push the .gitignore changes to the production remote:

$ git push production master

Then, inside the production server, update the git index respect to the new .gitignore, running this command within the bare repository:

$ git rm -r --cached sites/default/

You should run the above command also on each server (for example, the staging server) and on each local PC where you want to keep the content of the ignored folders, preventing to delete it when the Git repository will be updated.

Finally we can remove ignored files from the central Git repository, running these command on our local PC, from the project root folder:

$ git rm -r --cached sites/default/
$ git add .
$ git commit -am "Remove ignored files"
$ git push origin master

References and other reading

https://www.drupal.org/node/803746
https://www.drupal.org/documentation/install
http://www.arlocarreon.com/blog/git/untrack-files-in-git-repos-without-deleting-them/
https://api.drupal.org/api/drupal/sites!default!default.settings.php/8.2.x

Installing drupal with composer
https://github.com/drupal-composer/drupal-project
https://www.lullabot.com/articles/goodbye-drush-make-hello-composer

Advanced (and more reliable) workflows
http://nuvole.org/blog/2016/aug/19/optimal-deployment-workflow-composer-based-drupal-8-projects