playbook-infrastructure-company/ overview¶
playbook-infrastructure-company is a git repository hosted on your companies Gitlab server. It contains all configuration management code that provides management of all Debian Linux servers of your company.
Please refer to the documentation on how to setup your workstation to learn how to install the playbook-infrastructure-compnay on your workstation.
terraform/¶
The tool terraform is used to create and make changes like add RAM or disk space to the servers hosted with your companies cloud providers.
This information is stored below the directory playbook-infrastructure-company/terraform/ inside main.tf files.
Each group of servers is commonly created within a virtual datacenter of your cloud provider, like so:
Group: cus_www_prod
Instances:
- cus-www-prod-lb-1
- cus-www-prod-lb-2
- cus-www-prod-web-1
- cus-www-prod-web-2
- cus-www-prod-db-1
- cus-www-prod-db-2
This results in a terraform path like playbook-infrastructure-company/terraform/cus_www_prod/main.tf - this file is responsible of managing all instances in the group cus_www_prod.
The terraform documentation describes the syntax of the main.tf files.
inventory/¶
Ansible inventories are used to:
- specify all servers managed by Ansible
- categorize those servers into groups
- assign specific variables to groups or individual servers
.yml files¶
Inventory files are configuration files in YAML syntax below playbook-infrastructure-company/inventory/
| File | Function |
|---|---|
| playbook-infrastructure-company/inventory/hosts | List of all servers as well as grouping |
| playbook-infrastructure-company/inventory/pub.yml | Public IP to server assignment |
| playbook-infrastructure-company/inventory/vpn.yml | Wireguard server-to-server VPN IP to server assignment |
| playbook-infrastructure-company/inventory/host_vars/{{ inventory_hostname }}.yml | Assignment of variables specific to a given host |
| playbook-infrastructure-company/inventory/group_vars/{{ group_name }}.yml | Assignment of variables specific to a given group |
Dynamic inventory scripts¶
Inventory data can also be generated upon ansible or ansible-playbook execution by python scripts. These could for example call cloud provider APIs or run other programs, which return data that is then usable by Ansible.
Blunix Stack currently only uses one dynamic inventory to convert pass password-store data to Ansible variables:
| File | Function |
|---|---|
| playbook-infrastructure-company/inventory/dynamic/password-store/inventory.py | Converts pass password-store passwords into variables (described below) |
plays/¶
This directory contains collections of Ansible plays used to manage the servers. Examples:
| Directory | Function |
|---|---|
| playbook-infrastructure-company/plays/provisioning | Contains plays for provisioning freshly set up servers |
| playbook-infrastructure-company/plays/baseline/ | Contains plays for tools and programms installed on all servers by default |
| playbook-infrastructure-company/plays/utilities/ | Contains plays for the utility stack (monitoring, backups, ...) |
| playbook-infrastructure-company/plays/customers/my-company/ | Contains plays for the hosting environment of "my-company" |
| playbook-infrastructure-company/plays/customers/another-company/ | Contains plays for the hosting environment of a second company called "another-company" |
.pass/¶
playbook-infrastructure-company/.pass/password-store/ contains GnuPG encrypted passwords which can be edited using the pass password-store tool. This is used to store WebUI or database passwords, API secret keys, SSL certificate keys and alike.
This is used by the Ansible password-store dynamic inventory to make these secrets usable with Ansible.
.pass/.gpg-ids/*asc¶
playbook-infrastructure-company/.pass/gpg-ids/ contains the GPG public keys (.asc files) of all employees working with the playbook.
The playbook-infrastructure-company/.pass/.gpg-id is simply a list of all GPG public keys below .pass/gpg-ids/. This list is generated by running cake -f pass_reencrypt, which will reencrypt the password-store for all employees whos keys are below .pass/gpg-ids/.
Editing passwords¶
To show all passwords simply type pass:
CAKE master * pass
Password Store
├── backuppc_admin_password
├── env_cloud_provider_api_key
├── multiline_ssl_key_www_example_com
[...]
To edit or add a password:
CAKE master * pass edit <password_name>
The dynamic inventory will only read the first line of the password! All following lines are ignored unless you use multiline_ (see Password naming schemes below).
Run this command every time you add, edit or remove a password to make the change available to the password-store dynamic inventory:
CAKE master * compile-passwords.py
Password naming schemes¶
The name of the password can unlock different functionality with in Ansible Cake.
Regular names¶
A password created via pass edit website_secret_password will result in the Ansible variable website_secret_password.
Starting with multiline_¶
Password names staring with multiline_ will trigger the Ansible dynamic inventory password-store to read in all lines of the password file except for just the first line, which is the default. This is, for example, helpful for SSL private keys.
Starting with personal_¶
Password names starting with personal_ are not pushed to the git repository due to a rule in the playbook-infrastructure-company/.gitignore file. They contain human specific secrets, like cloud provider API credentials which are generated for a specific employee. These passwords are made available to Ansible as well, while the personal_ part is cut from the variable name (personal_cloud_provider_api_key becomes cloud_provider_api_key in Ansible).
Starting with env_¶
Password names starting with env_ are exported as environment variables upon sourcing the playbook-infrastructure-company/.env file. This is commonly used for terraform, which reads the cloud providers API credentials from environment variables. These passwords are made available to Ansible as well, while the env_ part is cut from the variable name (env_cloud_provider_api_key becomes cloud_provider_api_key in Ansible).
Starting with personal_env_¶
Combination of personal_ and env_ functionality. The password files are not pushed to the git repository, are made available to Ansible and are also available as environment variables (Example: personal_env_cloud_provider_api_key becomes cloud_prvoider_api_key).
Access management¶
To add or remove an employee to the password-store run the following commands:
# Run these inside cake
cd playbook-infrastructure-company
./start-cake.sh
# Create a new branch for adding the user
git checkout -b update/adding-user.name
# Paste the employees GPG public key to the directory
nano .pass/gpg-ids/new.user.email@company.com.asc
# Reencrypt the secrets below .password-store/passwords/
cake -f pass_reencrypt
# Push your change to git and create a git merge request by following the link in the output
git commit -m 'Reencrypted password-store for user.name'
git push -u update/update/adding-user.name
.env¶
This shell file contains environment variables required by various tools inside the Ansible Cake Docker container. It is sourced automatically by Ansible Cake when starting the docker container.
If you add new env_ passwords using pass you have to re-load this script:
CAKE master * pass edit env_cloud_provider_secret_key
CAKE master * compile-passwords.py
CAKE master * source .env
Exporting decrypted variable from password store: CLOUD_PROVIDER_SECRET_KEY
CAKE master * echo $CLOUD_PROVIDER_SECRET_KEY
secret-cloud-provider-api-key-here
cake.conf.sh¶
This shell file contains commands for each functionality of Ansible Cake which is custom to your company, for example:
# Defines ansible-playbook command to manage php-fpm on the servers in the cus_www_stag group
cus_www_stag_php_fpm() { ansible-playbook plays/www/stag/php-fpm.yml $ANSI_ARGS; }
The $ANSI_ARGS is defined by Ansible Cake during execution and usually contains Ansible arguments like --limit, --check or --inventory.
Please refer to the Ansible Cake documentation for more information.