Composer

This chapter provides some notes on using composer (a free software dependency manager) on a Drupal 9 or later website.

Table of contents

• Introduction
• Key concepts
• Installing composer
• Useful commands
• Installing Drupal
• Managing extensions
• Declaring a dependency
  • Adding dependencies manually
• Best practices
  • Staging and production
  • PHP version
• Migrating a site
• Composerise an existing website
  • Prepare
  • Set up
  • Restore the codebase
  • Restore the assets
  • Restore the content
  • Finish
• Troubleshooting
  • Killed
  • Abandoned packages
  • Missing packages
  • Template drupal-project dependencies
  • General debugging tips
• Final word

Introduction

Composer (Wikipedia) is a dependency manager for PHP designed and developed by Nils Aderman and Jordi Boggiano and first released in 2012. It supports Symfony (used by Drupal sinnce version 8) and several other popular PHP-based frameworks.

It keeps track of dependencies between the modules and libraries that make up the codebase of a project (in the context of Drupal, a project is typically a single website). It may be used to manage the Drupal site itself as well a to manage inclusion of 3rd party libraries. It replaces the Libraries API module for Drupal 7 for library (package) management.

For this to work, composer must be used to download the Drupal core as well as extensions.

There exists a GUI-based alternative to composer. It is targeted towards site-builders that does want to use the CLI, and is known as Ludwig. It is linked here, in case somebody may want to try ot out. I have not evaluated it, as I prefer very much the CLI.

Old school site-building and dependency management, i.e. using drush or simply manually unpacking a tarball to install a Drupal extension, is no longer practical with Drupal 9 or later. To quote Ryan Szrama: “If you're not using a dependency manager, then you are the dependency manager, and you are unreliable.”

The links below are to sites and pages I found useful when learning about composer. Both Drupal and composer is a ecolving so some of materials may be outdated. Use with care.
- getcomposer.org: Composer home page,
- getcomposer.org: The composer.json Schema,
- getcomposer.org: Versions and constraints,
- Code Artisan: Composer – all you need to know,
- ModulesUnraveled.com: Using Composer to Manage Project Dependencies,
- Jaypan 1: Understanding Composer,
- Jaypan 2: Managing a Drupal 8 site with Composer,
- Jaypan 3: Converting Management of an Existing Drupal 8 site to Composer,
- Jaypan 4: Composer for Drupal Developers,
- Drupal.org: Download Drupal core using Composer,
- Drupal.org: Using Composer to Install Drupal and Manage Dependencies,
- Drupal.org: Add a composer.json file,
- Martin Hujer: 24 Tips for Using Composer Efficiently.
- mmjvb: Migrating a website to a server that does not have composer.
- lullabot: Drupal 8 Composer Best Practices.
Some of the text below are derived from from these source. I believe this constitutes fair use. The insight gained from these sources is hereby acknowledged.

This documentation will use this terminology:

• siteroot: The directory above the webroot. This is the canonical place for the project's composer.json file and the vendor directory.
• webroot: The directory pointed to by Apache's DocumentRoot. This is where the website's index.php lives.

Key concepts

The main rationale behind composer is to make code simpler to reuse. Rather than putting the codebase that constitute an application into a single repository, composer let the developer declare that externally hosted packages may be part of the application's codebase.

The word project is used for the codebase, generally for a website or another application, that is managed by composer. Best practice is to keep the project directory, containing the root composer.json, in the siteroot.

The root composer.json contains a json-structure that describes the codebase of a project, where to get the code, and how to assemble it. It con­tains enough information to let the codebase be reconstructed on another location without moving the actual code.

Individaul packages to is part of the project may declare their dependencies in a package-specific compsoer.json in their top directory.

Composer requires dependencies to be declared in a specific namespace. The Drupal core and contributed modules lives in the drupal/* namespace in the Drupal.org repo (https://packages.dru­pal.org/8). (You may not need to declare these in a package specific composer.json-file. See the section Declaring a dependency below for details.)

If a contributed extension (module or theme) has a dependency on third party libraries hosted on Packagist, the namespace is different. For instance, the guzzlehttp/guzzle package is in the guzzlehttp/* namespace hosted in the Packagist repo. Searching for packages in the Packagist repo is hardcoded into composer. The Drupal.org repo should be added in the “repositories” key of the root composer.json as described in the section Declaring a dependency below.

Installing composer

This is covered in other chapters:

• RHEL7/8
• Ubuntu

Useful commands

You must use composer directly from the CLI. You should be in the siteroot (the directory where the site's composer.json is located) when using these commands.

This is a command that may be used to diagnoses the system for connectivity and common errors:

$ composer diagnose

Here is a list of some useful commands:

$ composer config # To edit config settings and repositories in composer.json.
$ composer create-project # Creates a new project and install packages.
$ composer licenses # Show license identifiers.
$ composer outdated # To see list of packages that can be updated.
$ composer show # To see list of packages or information about a package.

For a longer description, and to see the arguments and options that may be used with these commands, use:

$ composer -h topic

where topic is the command you're interested in.

Installing Drupal

It is recommended that composer is used to download the Drupal core for a new Drupal 10 website. The procedure for doing this is described in a previous chapter in this ebook. This ensures that the root composer.json and the vendor directory are created correctly and in the right place (i.e. the siteroot).

See this link to see how to use composer to download and install a legacy version of the Drupal core.

Managing extensions

The four main commands used to manage the codebase with composer are:

1. require: Add the package to the root composer.json, and download it, or whitelisting.
2. update: Update packages as specified in composer.json, and update composer.lock.
3. install: Install all packages as specified in composer.lock.
4. remove: Delete the package in the file system and remove it from the root composer.json.

The “update” command will not update a package beyond the requirements set in composer.json. To adjust the requirement, use:

$ composer require package requirement --no-update
$ composer update package scope

This is known as whitelisting. It makes an explicit request for a package, overriding any restrictions stipulated in composer.json. There are two options that may be used with “update” to extend the scope:

• -w --with-dependencies: extend the scope with dependencies, except root requirements.
• -W --with-all-dependencies: extend the scope with dependencies, including root requirements.

Below are some examples of use of these commands.

To download a Drupal contributed project from the Drupal.org repo with composer, use this command in the siteroot:

$ composer require 'drupal/projectname:version'

Do not use the require command in another directory than the siteroot. You may get a warning, but if that directory already contains a file named composer.json, the wrong composer.json will be altered without any error message or warning.

The value for the version constraint can be found under "Releases" on the project page. Look for the text "Install:" and you'll see the exact command to use on the CLI to install. See example screenshot on the left.

For instance, if your project uses the extension Pathauto, use the command shown in the screenshot to download.

Composer will download the extension, and recurse through all dependencies defined in their composer.json files, and download those as well. Those dowloaded from Packagist will be placed below the vendor directory. Those from Drupal.org in the right place below the webroot. It will also merge those dependencies into the root composer.json. After downloading, you may use drush to enable the extension. Extension may also be enabled using the GUI.

For example, to download and enable Pathauto, type:

$ composer require 'drupal/pathauto:^1.9'
$ drush en pathauto -y

The string "^1.9" means version 1.9.x, or higher, but not version 2.0.0 or higher.

If you leave out the version, the most recent stable version that matches all the constrains will be downloaded.

If you get an error message like this: “Could not find a matching version of package drupal/packagename. Check the package spelling, your version constraint and that the package is available in a stability which matches your minimum-stability (stable).”, in addition to the causes listed, an additional cause may be that you are not placed in the siteroot, so the “repositories” key for the “drupal” repo is unknown.

When building a Drupal website, you shall, as a rule, only require and remove extensions in the drupal/* namespace. Third part dependencies will be declared in the individual composer.json-files that is part of the Drupal package and its dependencies, and this will be sorted out by composer. There may be exceptions to this rule, if you need to install a library not hosted on packagist.

You may specify a stability constraint when you require an extension. By adding a stability flag (e.g. @RC), you may override the default setting for "minimum-stability" in composer.json. Example:

$ composer require 'drupal/entity_pager:^1.0@RC'

To update the libraries managed by composer go to the directory with the root composer.json, and use the following command:

$ composer update

This command will ignore composer.lock and update all dependencies in composer.json according to your requirements. Please see the section Updating projects and themes in this ebook for a more detailed description of the update process.

If there is no composer.lock, but there is a composer.json (i.e. a fresh project), the following command will do the same thing:

$ composer install

However, if composer.lock exists, the install command will use that. This file has a “lock” on all packages you have installed on the project. So if a site is working well, and install is run, you can be sure that no dependency will break, because the versions have been locked to what you have in the composer.lock file.

The install and update commands has a --no-dev option. This option will prevent packages listed under the “require-dev” parameter from being installed. However, composer will not ignore these packages. An install or update will fail if not all the requirements of these packages are met. If such requirements are blocking an operation, the problems must be resolved to proceed.

If the extension is not hosted on a cloud repository, but is a custom project that only exists in the local code base, see Managing dependencies for a custom project.

Removing and uninstalling an extension using composer is almost the same as installing, but you perform the steps in opposite order.

First use the GUI or drush (pm-uninstall) to uninstall the extension and to make sure any tables and other content is removed from the project. Then use composer to remove the extension. There is no need to specify a version.

For example, to uninstall and remove Pathauto, type:

$ drush pm-uninstall pathauto
$ composer remove drupal/pathauto

This will remove the entry from the root composer.json as well as dependencies. If the Drupal project given as argument is not required by any other installed project, the project and its files will be deleted as well.

Never just remove the extension by using the CLI comamnd rm, or use composer to remove the files without first uninstalling the extension. Both actions will leave gunk behind in the site's database that may, in worst case, break the site.

Declaring a dependency

If you are developing a project (module or theme), and it has a dependency some third party libraries hosted on Packagist, you must define these dependencies in file named composer.json located in the top dirctory of your project.

If there are no such dependencies, or all your dependencies are Drupal modules declared in your module's .info.yml, there is no need to create a composer.json file. It is probably better not to create one.

If you need to declare a dependecy, You can create this file manually, but you may also use composer. To declare a new dendency of a project hosted at Packagist, use the subcommand “require” in the project's root directory.

For instance, to declare guzzlehttp/guzzle, as required by the project, use the following command:

$ composer require guzzlehttp/guzzle
No composer.json in current directory, \
 do you want to use the one at /var/www/example.com? [Y,n]? n
Using version ^6.3 for guzzlehttp/guzzle
./composer.json has been created
…

This will generate or add to the project's composer.json, and add the dependecy to a vedor subdirectory. We don't want the latter, so just delete the vendor subdirectory:

$ rm -rf vendor/

Keep the composer.json. This file is now part of your project. For guzzlehttp/guzzle, it will look something like this:

{
    "require": {
        "guzzlehttp/guzzle": "^6.3"
    }
}

While this defines the requirements, it will not validate. To make it validate, add the metadata for “name” and “description”. It is also considered good practice to add metadata for “type” and “license”. Edit the file to add these. Example:

{
    "name": "janedoe/mymodule",
    "description": "This is a description of Mymodule.",
    "type": "drupal-module",
    "license": "GPL-2.0-or-later",
    "require": {
        "guzzlehttp/guzzle": "^6.3"
    }
}

Finally, check that the file validates.

$ composer validate --no-check-all --strict
./composer.json is valid

In Drupal projects people tend to add the minimum-stability key to a project's composer.json. This do nothing. This key should only be used at in the root composer.json. Source: getcomposer.org.

For dependencies on Drupal contributed modules, there is no need for a composer.json. Drupal.org will derive the required information from the dependencies specified in the .info.yml and will sort out the depencies and make sure the required Drupal projects a project depends on is being downloaded as described above.

For instance, in an example project named Mymodule, there is an dependency on the token located in the token module. This dependency is declared under the “dependencies” key of mymodule.info.yml:

name: 'Mymodule'
description: 'This is a description of Mymodule.'
core: 8.x
type: module
dependencies:
  - token:token

If there also is a composer.json in the project, this information will be merged with the dependency information from any composer.json.

However, if there are Drupal dependencies as well dependencies on a library hosted on Packagist, it is considered best practice to also include your Drupal dependencies in case you need an explicit composer.json for non-Drupal packages. To do this, you can simply add “require” statements to your composer.json to specify dependencies on Drupal modules or themes. Note that Drupal.org translates the Drupal versioning scheme into a semantic versioning format for composer, by adding a .0 for the patch version.

{
    "name": "janedoe/mymodule",
    "description": "This is a description of Mymodule.",
    "type": "drupal-module",
    "license": "GPL-2.0-or-later",
    "require": {
        "drupal/token": "^1.5",
        "guzzlehttp/guzzle": "^6.3"
    }
}

The syntax for declaring a dependency is “project/module”. The “project” is machine name of the project as it appears in the URL pointing to the project page on Drupal.org and “module” is the main module is the main module or one of the submodules of that project.

By default composer only will look at packages that are published on Packagist when it is resolving dependencies. Drupal has its own repository. To instruct composer to look for Drupal modules in the packages.drupal.org repository by executing the following command:

$ composer config repositories.drupal composer https://packages.drupal.org/8

This command will add a “repositories” key to your root composer.json. If you've used this method for installing Drupal, the “repositories” key should already be correctly set up.

The “repositories” key is declared as “root-only” in the composer.json schema and must not be included in any module's composer.json.
When dependencies are declared in composer.json, you still need to add your Drupal dependencies to your .info.yml file.

Adding dependencies manually

If you are developing a custom module project or custom theme that will not be contributed on Drupal.org, it will not be in the Drupal.org repository.

Instead take the dependencies from your module in development, and add them to the composer.json file in the siteroot.

First, add the "path" to the project to the root composer.json. You may use a relative or absolute path. Leading tildes are expanded to the current user's home folder. Example:

"repositories": [
    …
    {
        "type": "path",
        "url": "/home/janedoe//src/gitrepo/custom/mymodule"
    }
]

Then declare the vendor in the project's composer.json (use the "name" key):

{
  "name": "janedoe/mymodule",
  "description": "This is a description of Mymodule.",
  "type": "drupal-module",
  "license": "GPL-2.0-or-later",
}

Require the project:

"require": {
  …
  "janedoe/mymodule": "*"
},

Then update your project to pull in the library.

$ composer update

This will create a symlink from the project to the repo.

Source DO: Managing dependencies for a custom project and this DO forum post.

Best practices

Below are notes on best practices when using composer to manage dependencies. It is pretty incomplete at the moment.

Staging and production

When working with composer, you should set up your environment to have a staging (or development) website and a production website. Lean Drupal websites, where the site is built directly on production using whatever tools the hosting provider has made available, is not the best practice when working with compsoer.

Only install composer on the staging server. You do not want to use it on a production server.

The staging server is usually physically located in the developers' shop. It can be a personal computer running Gnu/Linux, MS Windows 10 (with WAMP or WSL), or Apple OsX (with MAMP). The production server is usually hosted on a colocation space, on a VM hosted by a company providing as IaaS, or on a vhost hosted by a company providing shared web hosting

You do all your developement on the staging server. To update the codepase on a production server, the best practice is to use a suitable file system syncing tool (e.g. rsync to sync between servers running Gnu/Linux, or Unison to sync between different plaforms).

Some configuration settings will be different between staging and production. Most of those are located in settings.php. I have set up my syncing so that this file is not overwritten. The setting for “Logging and errors” should be different. On the staging server, this should be “Show all errors”, on the production server, it should be “None”. I just change the setting manually, but should find a way to automate this.

As for the database content and files, I always keep those up to date on the migrtation server and export back and sync back to the staging server when necessary. Merging content is nightmare and rsync is unidirectional and don't know about merging conflicts.

Whenever a daatabse is imported, it may contain a legacy schema. Best practice to always check if a database update is necessary after an database import.

PHP version

Best practice is to make sure your staging server is using the same version of PHP that is on the production server. use. Otherwise composer may select packages on your staging server that will not work in production.

However, it is not uncommon that your staging server is running a bleeding edge version like PHP 8.2, while the production server runs an older version. composer let you configure a specific PHP version by adding it to your project's configuration:

$ composer config platform.php 8.1
$ composer update --lock

Without this in place, composer will add libraries to your staging server that are incompatible with newer versions than PHP than 8.1.

Note that this contraint may not work. If the library in question uses PHP constructs that are deprecated in the version of PHP your staging server is actually running, the library will not work on the staging server. Best parctice is to use the same version of PHP on both staging and the production server.

Migrating a site

To migrate an existing Drupal website managed by composer to a new website, first clean up the configuration: Remove anything not used, and update all extensions to their stable versions.

First create the siteroot directory on the target server. Then copy the following components into and below the target siteroot:

• The file composer.json.
• The file web/web.config.
• The web/libraries directory (if it exists).
• The web/sites directory, which usually includes the public:// file system.
• The private:// file system (if it is populated).
• All custom modules and themes (i.e. everything in the code base not managed by composer).

If the config_sync_directory (its location is set in settings.php) is located outside siteroot, it should be copied as well. Make it writable by the web server.

There is no need to copy the vendor directory, the Drupal core, or any contributed extensions managed by composer.

Make a dump of the site's database, and copy that to the target's backup directory.

In the target's siteroot directory run:

$ composer install

Create a database for Drupal and restore the database.

Edit the site's settings.php:

• Make sure the setting trusted_host_patterns match the domain.
• Make sure the database name and credentials are correct.

Finally, fix permissions to protect the settings, make the public file system writable by the web server and rebuild the cache. Now inspect the migrated website.

See also DSE: Copying prod multi-site installation to dev multi-site installation.

Composerise an existing website

The procedure below describes how to take an existing website, and set it up to use composer as its dependency manager. It is typically used one of in these three situations:

1. The site was set up to not use composer. You've decided to convert it to use composer for dependency management.
2. The site was set up to use composer for dependency management, but it was set up prior to drupal/recommended-project became the the standard way to install Drupal
3. The site can no longer be updated (e.g. updates no longer work). All options for fixing it has been exhausted. What remains is to nuke the composer configuration and start from scratch. This is also known as the “thermonuclear option”.

In both cases, the steps are as follows:

1. Prepare: Update, clean up and create an inventory of extensions and content.
2. Set up: Create a clean install of Drupal that has its dependencies managed by composer
   (composer create-project drupal/recommended-project my_site_name_dir).
3. Restore the codebase: Tell composer what extensions you require.
4. Restore the assets: Place the public and private file assets in the right places.
5. Restore the content: Import the database.
6. Finish: Put everything else back to the way it was.

There exists a lot of other guidelines about adding composer to an existing site, including the official one at Drupal.org. I wrote these notes before the documentation page linked above existed. You may want to check that out instead. YMMV.

Prepare

This is what you should to prepare your site before setting it up to have its dependencies managed by composer.

First make sure that you have not made any local modifications to the Drupal core or any contributed extensions (i.e. make sure that you have not hacked core or any extension). If you have, your site can not be converted to use composer for dependency management by the procedure described below. If your existing site is pristine, proceed as follows:

1. To see run-time errors, navigate to: Configuration» Development » Logging and errors. For a staging site select “All messages”.
2. Revert the site to the default theme (i.e. Oliviero).
3. If it is multilingual site, revert the site to its default language.
4. Review all the extensions you have installed. Uninstall and remove all extensions that you do not need or use.
5. Use “old school” procedure (i.e. download and unpack a tarball) to make sure your version of Drupal 10 is updated to the latest stable version of its branch (currently 10.1.5).
6. Make sure that all extensions are updated to their latest version that is recommended by the maintainer.
7. Make an inventory of all extensions in use. Take special note of any sandboxes and custom extensions.
8. Make a note of all file assets, and the file system paths of the public and private file systems.
9. Make a full backup of the database.

After going through the checklist above, you should have a populated website running the most recent version of Drupal 10 that is not managed by composer. The next step is setting up a clean installation of Drupal 10 that shall use composer for dependency management.

Set up

Navigate to the direcory above where you want to create the siteroot.

Provided you want your siteroot to be a directory named “example.com” and you want to install the most recent version of the Drupal 10 branch, in the CLI, type the following command.

$ composer create-project drupal/recommended-project example.com

The create-project-command will place an inital root (global) composer.json in the siteroot directory. It will also create a subdirectory for Drupal named “web”.

If you look inside composer.json, you will that it will have a “repositories” key like this, telling composer to also search the Drupal.org repostory for packages:

"repositories": [
    {
        "type": "composer",
        "url": "https://packages.drupal.org/8"
    }
],

Composer will, by default, search the Packagist repository. There is no “repositories” key for the Packagist repository.

As already mentioned, in the siteroot, there will be a subdirectory named “web”. This will become the webroot and if the you used the exact same create-project command as the one in the example above. The recommended version of Drupal and all its dependencies will then be downloaded and installed in the webroot.

The create-project command will also create a file named composer.lock in the siteroot. This file will lock in your configuration so it can be recreated in its present state just by rerunning the composer install command.

The composer project directory in the siteroot will contain some files and directories. The most important are:

• composer.json: The root composer.json used to manage the project.
• composer.lock: The configuration file that locks the project.
• vendor: A directory that contains most of the libraries managed by composer.
• web: The webroot. This is where Drupal is installed.

Now, proceed to install the site as you normally would install a Drupal 10 website. If you are not familiar with the procedure, you shall find it described here

Restore the codebase

Restore the code base using composer. This is the crucial step. This will register the codebase in this project in composer.json and lock it in composer.lock.

So look at the inventory of the codebase you have made (step #7 under “Prepare”) and go through all the items, one by one. For contributed extensions look up the command you should use to require it on the extensions project page.

Repeat this procedure for all the contributed extensions you require.

Install all the sandbozes. See How to install a drupal.org sandbox module using composer for details (may need a tweak).

Install all the custom extensions. If you don't have them in sandboxed (see above) in the Drupal.org git repo, you can have them on GitHub. See Creating your very own Composer Package for details.

Restore the assets

First set up the public and private path for the file assets. To set the paths, edit the following entries in settings.php (the values are just examples and may be different on your server):

$settings['file_public_path'] = 'sites/default/files';
$settings['file_private_path'] = '/var/private';

After setting the paths, restore the public and private file assets that you made a note of (in step #8 under “Prepare”) to their respective locations.

Restore the content

Import the database that you backed up (step #9 under “Prepare”) into the database of the site that have its codebase dependencies managed by composer.

Finally, run any pending database updates. If they exist, the administrative GUI will nag you about it if you visit the status report. To make sure, you can force this by visiting the path update.php. I.e.: If your website is example.com, visit this URL: https://example.com/update.php.

Finish

1. If it is multilingual site, restore the languages.
2. Visit “Appearance” to activate the theme you want to use on the site.

After you've set up your site to use composer to manage dependencies, you should not add extensions without using composer to install them. Doing so may result in composer getting stuck and refuse to update your site.

Troubleshooting

See also getcomposer.org: Troubleshooting.

Killed

Composer is killed without explanation – example:

$ composer require drupal/bootstrap_barrio
Using version ^5.0 for drupal/bootstrap_barrio
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Killed

The message “Killed” usually means your process consumed too much memory, so you may simply need to add more memory to your system by adding swap or (on a vhost) expanding RAM.

Background: SO.com.

Requirement conflicts

You are unable to update because doing so creates a conflict with another locked-in requirement. Examples:

“Your requirements could not be resolved to an installable set of packages.
- drush/drush 10.3.5 requires symfony/finder ^3.4 || ^4.0 -> found symfony/finder[…], but it conflicts with another require.”

First try to whitelist the projects that need to be upgraded.

To reduce the need for whitelisting, you may use the update with the options --with-dependencies (-w) (exclude root requirements) and --with-all-dependencies (-W) (include root requirements. Use with caution. You may end up updating more than you first thought. Use option --dry-run to see what it will do without doing it. When you're happy with the outcome, go for it.

The dialogue below shows a typical example where the initial update, trying to update only the core with dependencies fails. We then do a dry run with all dependencies. It produces an acceptable outcome (output is not shown here), so we proceed to update with all dependencies.

$ composer update 'core/*' -w
Loading composer repositories with package information
Updating dependencies
Dependency "composer/installers" is also a root requirement. Package has not been
listed as an update argument, so keeping locked at old version.
Use --with-all-dependencies (-W) to include root dependencies.
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - …

Use the option --with-all-dependencies (-W) to allow upgrades,
downgrades and removals for packages currently locked to specific
versions.

$ composer update -W --dry-run
…
$ composer update -W
Loading composer repositories with package information
Updating dependencies
Lock file operations: 0 installs, 5 updates, 0 removals
  - Upgrading drupal/core (9.1.4 => 9.2.8)
…

If you're unable to resolve the conflicts by other means, you may try to moving the composer.lock file and the vendor directory out of the way. Then install from scratch:

$ mv composer.lock composer.lock.bu
$ mv vendor vendor.bu
$ composer install

This may break stuff, so don't delete these files just move them out of the way as shown. You may also want to back up your code base and your database before trying this.

Abandoned packages

Composer may warn about abandoned packages. Example:

“Package doctrine/reflection is abandoned, you should avoid using it. Use roave/better-reflection instead.”

(This particular problem is fixed in Drupal 10, but I use it as an example for how to deal with abandoned packages.)

If you have directly requested those libraries, you should update them according to the directions provided.

However, that may not be the case.

The next step is to find out why the package is required. In this case:

$ composer why doctrine/reflection
doctrine/common          2.13.3  requires  doctrine/reflection (^1.0)   
doctrine/persistence     1.3.8   requires  doctrine/reflection (^1.2)   
drupal/core              9.1.7   requires  doctrine/reflection (^1.1)   
drupal/core-recommended  9.1.7   requires  doctrine/reflection (1.2.2)  

In many cases, like the present one, the underlying problem is an unresolved core issue (e.g. Drupal.org: doctrine/reflection is abandoned).

If the package is required by the core, or required by a library your project depends on, there is usually nothing you can do about it. It is up to the maintainers of the component requiring the package to update their packages to not use abandoned packages. However, even when a replacement is suggested, it may not be a drop-in replacement, so some refactoring is necessary in order to use the replacement.

However, each case needs to be analysed on its own merits.

To dig deeper, you may consider copying the root composer.json to another directory and add the abandoned package to the conflict section. Example:

"conflict": {
   "drupal/drupal": "*",
   "container-interop/container-interop": "*"
},

Then, in that directory, do:

$ composer install
Your requirements could not be resolved to an installable set of packages

  Problem 1
    - …

This may provide some insight into the problem, but in my experience, it only confirms that you shall not be able to have a working project without the abandoned package.

Below are links to some pages about this topic. You may want to read them for reference:
SE.DA: What is the best practice for handling abandoned packages required by the drupal core?
SE: What is the best practice for handling composer abandoned packages?
Drupal.org forums: Various Packages abandoned, What to do about abandoned packages?.

Missing packages

If you're getting warnings about missing packages in the drupal/* namespace when trying to install or update – example:

“The requested package drupal/ctools could not be found in any version, there may be a typo in the package name.”

This is usually caused by one one of projects on your site has the property require-dev with some special versions of Drupal packages that are required for development, and you want those installed, but the root composer does not know about the Drupal repository. To fix this use composer config to configure the Drupal repository as described in a previous section.

Template drupal-project dependencies

Prior to Drupal 8.8.0 – the canonical procedure for installing Drupaø was to use a community-contributed composer template named “drupal-composer/drupal-project”. In release 8.8.0, this changed to the new official template “drupal/recommended-project”. The plugins left behind made upgrading to version 8.8 (and higher) hard. For how to get rid of these, please see Drupal.org: Migrating the Composer project for Drupal earlier than 8.8.0,

Many of these legacy plugins only do something useful the first time. You may consider removing them for prucuction sites.

My own notes

Prior to the official documentation linked above existed, I created these notes. I reatin them until Titan 2 is upgraded.

As the legacy template was created for a development project, some libraries only needed for development are included by default. They tend to block updates to a new minor version. You can get rid of these using one of these commands:

$ composer remove webflo/drupal-core-require-dev
$ composer update --no-dev

The problem with unmet dev dependencies went away when the canonical procedure to install Drupal became to use drupal/recommended-project. It should only bite you if you're upgrading from a Drupal version prior to 8.8.0.

Legacy drupal-composer/* plugins may also prevent you from using the latest version of composer. Example:

“The "drupal-composer/drupal-scaffold" plugin was skipped because it requires a Plugin API version ("^1.0.0") that does not match your Composer installation ("2.0.0").”

First figure out the underlaying cause. In this case, it turns out that drupal-composer/drupal-scaffold is deprecated and replaced by drupal/core-composer-scaffold. (Source GitHub: Compatibility with Composer 2.) The solution is downgrade to composer to a version compatible with the plugin, remove the deprecated plugin, require the correct version, and then upgrade to the latest version again.

$ composer selfupdate --1
$ composer remove drupal-composer/drupal-scaffold
$ composer require drupal/core-composer-scaffold
$ composer selfupdate --2

Sometimes, you need to force the update of a specific plugin. For example the installed version of oomphinc/composer-installers-extender was not comptible with the latest version of composer:

“The "oomphinc/composer-installers-extender" plugin was skipped because it requires a Plugin API version ("^1.0") that does not match your Composer.
…
Your requirements could not be resolved to an installable set of packages.
- oomphinc/composer-installers-extender is locked to version v1.1.2 and an update of this package was not requested.”

I removed this by editing composer.json to remove the requirement for this plugin, the installing to re-create composer.lock and vendor. Then reintroduving the requirement:

$ composer require oomphinc/composer-installers-extender

This may be a bit heavy-handed, and removing and then requiring (as in the previous example) may be suffiscient.

General debugging tips

Composer is very fastidious about syntax. Stuff will break if the syntax is wrong. Use this command in the directory that contains a composer.json to validate:

$ composer validate --no-check-all --strict

To see a list of dependencies that have been modified locally, do:

$ composer status
No local changes

Here is a link to the official documentation. It may take some time to run this. Be patient. (Tested with version 2.0.2.)

If you install something from the source (using the --prefer-source option), you will end up with a clone of that package in the /vendor folder. If you make some changes to that package, this command will show you the git status for those changes.

To discover where a package is referenced and whether it is required by other packages (i.e. why it is installed), use the following command:

$ composer why drupal/fixteaserlinks
drupal/recommended-project  -  requires  drupal/fixteaserlinks (^1.0)  

The above shows that the contributed extension named Fix Teaserlinks was required by the root project (i.e. “drupal/recommended-project”).

$ composer why drupal/coder
drupal/core-dev  8.8.6  requires  drupal/coder (^8.3.2)  

The following commands are useful when troubleshooting why composer does not update when it is expected to do that.

$ composer why-not drupal/core
$ composer update --dry-run

If a composer-managed project has gone really bad, you may need to do the following:

$ rm -rf vendor/
$ rm composer.lock

… then hand-edit composer.json to fix all errors.

If that does not fix it, the so-called thermonuclear option is to re-install the Drupal core on a fresh website, then migrate the old site project by project into the new one, and finally migrate contents and assets. See this section for more details.

If that doesn't work, try the following:

1. Delete everything related to composer from the computer.
2. Buy a new computer and start from scratch
3. Bury the old computer at the bottom of the garden and place a ring of salt around it.

It will probably not fix the problem, but it may make you feel better.

Final word

It is possible to only use composer on a staging site. To deploy, simply sync your staging codebase to your production environment. I use recursive cp if both live on the same Unix host, rsync if they live on different Unix hosts, and Unison if the staging codebase is on a PC running MS Windows or Apple OsX. If you need to migrate the database, use Backup & Migrate and if necessary adjust settings.php (user, pwd, host, port).

If done correctly, there should be no need to do anything more.

---

Last update: 2023-10-04 [gh].