Configuring Drupal 8 for a simple Git development workflow

In this post we want to share our experience in configuring and managing a Drupal 8 website project for a basic and simple Git development workflow.

Content

Installing Drupal 8 with Drush

At first we created a new Drupal website using Drush (take a look here if you need to install Drush).

Create a new Drupal 8 project:

$ drush pm-download drupal --drupal-project-rename=[project_name]

This command will create the [project_name] folder containing a Drupal 8 website ready to be installed.

Now we must create a new empty database within MySQL, named [db_name]. You can do this with your favourite MySQL tool.

After created the database, go inside the project folder and install Drupal:

$ cd example_project
$ drush site-install standard \
    --db-url='mysql://[db_user]:[db_pass]@localhost:[db_port]/[db_name]' \
    --account-name=joe --account-pass=mom123 \
    --site-name=Example \
    --site-mail=info@example.com

Optionally you can install Drupal with a custom language adding this option in the command above:

    --locale=it

specifying the locale string for the language you want to install.

Alternatively to this command, you can install Drupal running the web wizard.

Once the installation is finished we can test it running the Drush’s built-in server from the project’s root:

$ drush runserver 8000

then access the website with a web browser at the url:

http://localhost:8000

You will see an empty Drupal website, with the default theme.

To access the admin panel go to:

http://localhost:8000/user/login

and fill the form with username and password.

Configuring the Drupal 8 installation

You can find the settings.php file within the sites/default/ directory. By default, Drupal will set this file and the whole default directory to read-only after the installation process. Sadly this could be a little annoying in a Git workflow, since if Joe change some settings and push its changes, when Bill will do a pull, its local Git will complains that it can’t modify the settings.php file.

Given that in our development workflow sometimes we needs to change some settings after the installation process, we decided to add a settings folder, writable by default, and move inside it all site settings. To preserve the security, we will set the folder to read-only in the production environment.

Configuring settings.shared.php

The file settings.shared.php will contains all the settings shared between all developers and environments (local, staging and production), and will be versioned with Git. For environment-specific settings we’ll create the settings.local.php file.

Create the sites/default/settings folder (maybe you have to make the default directory writable to do this. Drupal will restore permissions in a later moment).

$ chmod +w sites/default
$ mkdir sites/default/settings

Now copy settings.php to settings/settings.shared.php and make it writable:

$ cd sites/default
$ cp settings.php settings/settings.shared.php
$ chmod 644 settings/settings.shared.php

Open settings.php and replace all its content with:

<?php

include __DIR__ . '/settings/settings.shared.php';

If you need to add a custom services.yml file, remember to place such file within the settings folder.

Open settings.shared.php and enable a list of trusted host patterns adding this under the Trusted host configuration section:

/**
 * Trusted host configuration.
 * ...
 */
$settings['trusted_host_patterns'] = array(
  '^localhost$',
  '^127.0.0.1$',
  'example\.dev$',
  'staging\.example\.com$',
  'example\.com$',
);

Configuring settings.local.php

Create the settings.local.php file, within the settings folder, copying it from sites/example.settings.local.php:

$ cp sites/example.settings.local.php sites/default/settings/settings.local.php

Now let’s configure the settings.local.php for our local environment. Move database settings from settings.shared.php and put them at the bottom of settings.local.php:

// ...

/**
 * Database settings:
 * ...
 */
 $databases['default']['default'] = array (
  'database' => 'example_project_db',
  'username' => '[db_user]',
  'password' => '[db_pass]',
  'prefix' => '',
  'host' => 'localhost',
  'port' => '[db_port]',
  'namespace' => 'Drupal\\Core\\Database\\Driver\\mysql',
  'driver' => 'mysql',
);

Then search the settings['rebuild_access'] line within the file and comment it. Decomment all the settings['cache'] settings in order to disable the cache for the development environment. In the production server we can leave these lines commented in order to enable the caching system.

Optional: add custom 'trusted_host_patterns' for your local machine:

/**
 * Add custom 'trusted_host_patterns'
 * 
 * You can add your local IP address here.
 */
$settings['trusted_host_patterns'][] = '^192\.168\.1\.123$';

We can now clean the settings.shared.php file and include local settings. So, let’s open settings.shared.php and remove database settings (if you haven’t already done it). We keep database settings in the local settings file. Then include the settings.local.php file decommenting this block of code and moving it at the end of the file:

// ...

/**
 * Load local development override configuration, if available.
 * ...
 */
if (file_exists(__DIR__ . '/settings.local.php')) {
  include __DIR__ . '/settings.local.php';
}

Since we will ignore settings.local.php from Git, we create the file example.settings.local.php that will be used as example file for anyone will pull the code from the repository. So when Bill will pull down the project from Git, he can copy example.settings.local.php to settings.local.php within the settings folder and configure it for his local environment. So, copy your settings.local.php to sites/example.settings.local.php (overwriting it):

$ cp sites/default/settings/settings.local.php sites/example.settings.local.php

Disabling Twig’s cache from development.services.yml

The settings.local.php includes sites/development.services.yml by default.

We can disable Twig’s cache here adding this code:

# ...

# Disable Twig cache
parameters:
  twig.config:
    debug : true
    auto_reload: true
    cache: false

Rebuild the cache

After settings has been modified is a good practice to clear and rebuild the cache. We can do this with Drush:

$ drush cache-rebuild

Configuring .gitignore

Create the .gitignore file inside the project’s root, with this content:

# Ignore core when managing all of a project's dependencies with Composer
# including Drupal core.
# /core

# Core's dependencies are managed with Composer.
/vendor

# Ignore configuration files that may contain sensitive information.
/sites/*/settings/settings.local.php

# Ignore paths that contain user-generated content.
#/sites/*/files/*
#/sites/*/private/*
#!/sites/*/files/config_*

# Ignore SimpleTest multi-site environment.
/sites/simpletest

The settings.local.php is ignored, so anyone can have its own settings.local.php file, different for each developers and environments.

Directories containing user-generated content, as /sites/default/files or /sites/default/private (if you enabled private files), should be excluded from Git. But when we start to develop a website we found more convenient to share these contents within Git since usually we generate them and they are useful for the development. When the website will be deployed we’ll remove these folders from Git (see later).

If you prefer, you can start ignoring content folders right away by decommenting rows under the comment Ignore paths that contain user-generated content.

Export the database

Within the Drupal’s database are stored some website configurations, installed modules, current theme, content types, views, nodes, etc. All of this must be shared between developers. The simplest way is to export a database dump and keep it on Git. When Bill (a developer) will get the code from Git he will load the last database export on his local machine.

Our convention is to create a database/exports folder and keep database exports inside this folder.

You can create the database dump with a MySQL tool or using mysqldump:

$ mysqldump -u [db_user] -p[db_pass] [db_name] > database/exports/[db_name]--[Y-m-d]--[H:i].sql

Add an .htaccess file, with the following content, within exports in order to avoid to share your database with the world:

<Files *.sql>
  order allow,deny
  deny from all
</Files>
<Files *.gz>
  order allow,deny
  deny from all
</Files>

The biggest drawback of having configurations stored inside the database is that only one developer at a time can working on configuring the website, and more in general only one developer at time can working on the database.

In Drupal 8 you can improve this workflow managing configurations files (exporting and importing Drupal configurations). See here for more informations: https://www.drupal.org/documentation/administer/config.

First commit and push

Now that we have configured the Drupal 8 installation, we can do the first commit and push the Drupal project on the central Git repository.

$ git add .
$ git commit -am "First commit"
$ git push origin master

Development workflow

When Bill start to working on the project the first time he have to:

  1. Clone the project from the central Git repository on his local PC.
  2. Load the database dump from database/exports/.
  3. Create the settings.local.php file, within the settings folder, copying it from example.settings.local.php and configure it with his own local settings.
  4. Run composer install within the project root.
  5. Rebuild the drupal cache with drush cache-rebuild.
  6. Start the server with drush runserver 8000.

Commit, push, pull

Once cloned and configured the project, both Joe and Bill (and any other developer) can proceed the development workflow with a commit-push-pull routine.

Remember that with the model described in this post, only one developer at time can modify the database. After he finish to modify the database, he can export the database and share it on Git so any other developer can import the new database.

We are assuming also to working directly on the master branch, without any other branch. An improvement to this workflow could be to use a branching model with at least a develop branch other than the master.

Configuring production (and staging) server

Production and staging environment have the same configuration. So we show how to configure one of them and the other will be the same.

We configure automatic deployment with Git, so that when we do

$ git push production master

the code will be pushed and deployed on the production server.

In the following we show how to configure a bare repository and a base post-receive hook script for the automatic deployment. Depending by your hosting service, you may need to configure the server installing all what it’s needed (php, git, composer, drush) and modify the post-receive script to work with your environment. For example, here there is a good tutorial by Digital Ocean for set up automatic deploy with Git on Ubuntu VPS. Here you can find some hint about configuring Git and Composer on a GoDaddy’s shared hosting.

Let’s start creating a bare repository on the production server:

$ cd ~/git
$ git init --bare --shared [project_name].git

You can create the repository inside any directory. In the code above we created it inside a git folder within the user’s home.

The bare repository will contain the code pushed to the production remote from Git. We need to configure a post-receive hook in order to copy the code from the bare repository to the website root directory.

Enter inside the folder [project_name].git/hooks and create the post-receive file:

$ cd [project_name].git/hooks
$ vim post-receive

Write inside the file the script below, replacing the value of APP_WEB_DIR and APP_GIT_DIR with the website document root and the git repository root respectively:

#!/bin/sh

# Set up our PATH variable and export it. In this way, within the ~/bin folder,
# you can link the right php cli version to use within this script and link
# the composer executable used below
PATH="/home/netgloo/bin":$PATH
export PATH

# App directories
APP_WEB_DIR="/home/netgloo/public_html"
APP_GIT_DIR="/home/netgloo/git/[project_name].git"

# Set write permissions on sites/default/settings/*
cd ${APP_WEB_DIR}
chmod -R +w sites/default/settings
cd ${APP_GIT_DIR}

# Copy the content from the repo to the server live directory
git --work-tree=${APP_WEB_DIR} --git-dir=${APP_GIT_DIR} checkout -f

# Remove write permissions from sites/default/settings/*
cd ${APP_WEB_DIR}
chmod -R -w sites/default/settings

# Run composer install
composer install

You can use any text editor you want instead of vim to create and edit the file (via FTP or with some web interface).

After created the hook, don’t forget to make it executable:

$ chmod +x post-receive

Adding the production remote on local PC

In your local PC go inside the project folder and add the production remote to Git:

$ git remote add production ssh://joe@example.com:[ssh_port]/~/git/[project_name].git

Now you can deploy the code with:

$ git push production master

Deployment

The first time you will push the code on the production server you will have to:

  1. Run composer install within the project root.
  2. Create the settings.local.php, within the settings directory, copying it from example.settings.local.php and configure it for the production environment (for example, you may want to enable the cache and the aggregation of CSSs and JavaScripts).
  3. Load the database dump from database/exports/.
  4. Rebuild the drupal cache with drush cache-rebuild or through the Drupal Admin at Config > Development > Performance.

After the first configuration, when you want to deploy new code you have to:

  1. Push the code on the production remote with git push production remote.
  2. If there was changes on the database: load the last database dump.
  3. Rebuild the drupal cache with drush cache-rebuild or through the Drupal Admin at Config > Development > Performance.

Once deployed the website, a common workflow is to working on the website’s code from a local development environment and then push the code on production when it’s ready. Whereas the website’s content (generated by users) is modified on the production environment and it should be downloaded if needed for development.

Production website
   ▲
   ┃        ┃
   ┃        ┃    
 Code    Content
   ┃        ┃
   ┃        ┃
            ▼
Local (development)

With Drupal, the user generated content is both inside the sites/default/files folder (and eventually inside the sites/default/private folder also) and inside the database. So you may need to download both the files folder’s content and the database.

Removing user generated content from Git

As already said, all the user generated content should not be tracked within Git. We have choosen to temporarily keep it within Git during the first development phase. Once the website is deployed, we should ignore (and remove) such content from Git.

Important: the following steps will remove such content from Git. An error performing these steps could bring to accidentally remove some content from the production server. So we strongly suggest you to carefully follow them and to make a backup before proceeding.

First we need to add folders containing user-generated content inside the .gitignore file:

# Ignore paths that contain user-generated content.
/sites/*/files/*
/sites/*/private/*
!/sites/*/files/config_*

Then commit and push the new .gitignore file:

$ git commit -am "Ignore user generated content"
$ git push origin master

Adding these folders in the .gitignore will do not delete them. So now we are going to remove ignored folders from the Git repository. But first we must prevent that such content will be deleted on the production server.

So, first push the .gitignore changes to the production remote:

$ git push production master

Then, inside the production server, update the git index respect to the new .gitignore, running this command within the bare repository:

$ git rm -r --cached sites/default/

You should run the above command also on each server (for example, the staging server) and on each local PC where you want to keep the content of the ignored folders, preventing to delete it when the Git repository will be updated.

Finally we can remove ignored files from the central Git repository, running these command on our local PC, from the project root folder:

$ git rm -r --cached sites/default/
$ git add .
$ git commit -am "Remove ignored files"
$ git push origin master

References and other reading

https://www.drupal.org/node/803746
https://www.drupal.org/documentation/install
http://www.arlocarreon.com/blog/git/untrack-files-in-git-repos-without-deleting-them/
https://api.drupal.org/api/drupal/sites!default!default.settings.php/8.2.x

Installing drupal with composer
https://github.com/drupal-composer/drupal-project
https://www.lullabot.com/articles/goodbye-drush-make-hello-composer

Advanced (and more reliable) workflows
http://nuvole.org/blog/2016/aug/19/optimal-deployment-workflow-composer-based-drupal-8-projects

  • Johny Varsami

    This is really useful and one of the few elaborated articles on this topic! Why are you not excluding core, modules, etc. from git if you are using Composer?

    • Thank you Johny. We installed Drupal with drush. Composer manages core’s dependency only. We can’t exclude from git core and modules directories because are not managed by composer.
      There should be a way to install Drupal with composer and then I guess you can exclude core and modules from git also.

      • Johny Varsami

        Ah, ok… Yes, that’s what I am doing, the whole installation is always updated with Composer. But I’m searchig for a way to not use Composer on production, as many people don’t recommend it (http://nuvole.org/blog/2016/aug/19/optimal-deployment-workflow-composer-based-drupal-8-projects ).

        • Yes, if you want a workflow more reliable as possible I guess you don’t have to run composer in production.
          In our experience, we had the staging environment on the same server of the production environment. So deploying first on the staging env give us enough confidence that the code works on production also. But yes, composer can fail or take a long time just when you are deploying in production.
          We had a fairly small project that didn’t requires the most reliable workflows, a simpler workflow (faster to set up) it was better and good enough ;).

          • Johny Varsami

            That’s my problem, the deploying from staging to production. The gitignore in a fully Composer managed workflow prevents pulling all the files. I opened a question in Stackoverflow and there are already some possible solutions: http://stackoverflow.com/questions/42089153/how-can-i-git-pull-ignored-files/

          • Frederick Henderson

            @johnyvarsami:disqus You could try drush deploy. It allow you to use drush to deploy sites. It allows the creation of tasks (PHP) that need to be run to deploy. It also allows you to roll back to a previous deployment. It will do this automatically if the deployment errors out. I have created tasks in my deploy.drushrc.php for a D8 deployment in addition to the D7 deployment tasks but have not added and example deploy.drushrc.php these to my github fork of drush deploy. The fork was mostly to add documentation to the Amazee Labs fork of the drush deploy project from Drupal.org.

            Bastian Widmer from Amazee Labs gave a presentation about Drush Deploy at Drupalcon Amsterdam. There is a youtube video of his presentation and a PDF of the sides is also available.

            I do use composer and have a composer install task that runs during deployment. I have just setup the tasks for Drupal 8 within the last 2 weeks. If you are interesting in using it and would like that I add the example deploy.drushrc.php sooner rather than later just let me know.

          • Johny Varsami

            Yes, would be cool to see it! I also wonder how you did this for D8 as there is only a D7 release of the module…

          • Frederick Henderson

            I will try to get an example deploy.drushrc.php along with an example Drush Site Aliases file added to my Github fork of Drush Deploy this week. You will need a Drush Site Aliases file for sites that you will be deploying to.

            Drush Deploy is not a Drupal module. It is a Drush extension and as such it is not Drupal 7 or Drupal 8 specific. I install it in my local drush installation and not on the sites.

            The following should install it, but until you setup your Drush Site Aliases and the Deployment Tasks it is pretty useless.

            cd ~/.drush
            git clone https://github.com/meosch/drush_deploy
            drush cc all

          • Frederick Henderson

            @johnyvarsami:disqus OK, I think I have it ready. Maybe some bugs, so the new stuff is only on the **dev** branch. I added the **examples/** directory to the project. Other than that I only updated the main README.md.
            To install:
            cd ~/.drush
            git clone -b dev https://github.com/meosch/drush_deploy
            drush cc all

            Then follow the documentation and setup your ~/.drush/deploy.drushrc.php and your Drush Site Aliases.
            Run drush deploy-setup @mywebsite.test to setup the need folders on your deployment site.
            Not sure if this in the documentation but before running drush deploy-deploy @mywebsite.test you need to create your settings.local.php on the server with the DB credentials. One of the deploy tasks will copy it into the deployment. Without it the deployment will fail. You also need to have the database setup on the server you are deploying to or the deployment will fail.
            If you are not currently using a settings.local.php file let me know and I can point you to the documentation. I think in the Drupal 8 settings.php it has commented out code and documentation on why and how to use one.
            If you find anything that needs changed fork the repo and make a pull request.

Categories

Category BootstrapCategory CoffeescriptCategory DrupalCategory GravCategory HTMLCategory JavascriptCategory JoomlaCategory jQueryCategory LaravelCategory MagentoCategory PHPCategory SharePointCategory SpringCategory ThymeleafCategory WordPressCategory Workflow

Comments

Developed and designed by Netgloo
© 2017 Netgloo