Vault & Mongodb

Vault Mongodb

Getting Started¶

Testing Vault & Mongodb locally

While Vault has excellent documentation (here) this doc page should help you to get started more quickly with mongodb.

The goal in this doc is to demonstrate how to integrate a local mongodb with a local vault server in dev mode.

This should not be used in production.

Follow the steps in this page, If you need a simple guide to easily experiment connecting to Databases (in this case mongodb) with Vault locally, by using docker in a few minutes.

You will create everything needed in order to be able to develop your application with Vault Authentication to a mongodb.

Docker network¶

Create docker network

We are creating a docker network to prevent any network issues and make sure all containers will be able to speak to each other locally on your OS.

docker network create test-network

Running Vault¶

Here is how we can run vault in dev mode.

In this mode, Vault runs entirely in-memory.

docker run -d --name vault --net=test-network -p1234:1234 --cap-add=IPC_LOCK \
    -e 'VAULT_DEV_ROOT_TOKEN_ID=myroot' \
    -e 'VAULT_DEV_LISTEN_ADDRESS=0.0.0.0:1234' \
    vault

dev mode

As you can see, our root token would be myroot.

In dev mode, vault is automatically unsealed, and we can access it right away with curl/cli/browser UI

We will get to that in a moment.

Run mongodb¶

Run a local mongodb server with Authentication enabled

For example to run mongodb version 3.6 Run the following:

docker run -d -p27017:27017 \
    --net=test-network --name vault-mongo \
    --volume=/YOUR_LOCAL_PATH/mongo:/data/db \
    --env=MONGO_INITDB_ROOT_USERNAME=admin \
    --env=MONGO_INITDB_ROOT_PASSWORD=XXXZZZ \
    mongo:3.6 mongod --auth

Make sure to set YOUR_LOCAL_PATH

Connect to mongodb¶

Verify Authentication

docker run -it --rm --net=test-network \
    mongo:3.6 mongo  --host vault-mongo \
    -u admin -p XXXZZZ \
    --authenticationDatabase admin \
    vault-test

Check the current db name:

> db.getName();
vault-test

Login to vault, to enable the Database Secrets Engine

Login

docker exec -it vault /bin/sh

Set the vault URL:

export VAULT_ADDR=http://127.0.0.1:1234

Login (use the TOKEN set when you started vault)

hint: it's myroot if you followed the instructions.

vault login

You should see this output:

Token (will be hidden):
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                myroot
token_accessor       G92enfIN5Br7OhzB3v6KnViS
token_duration       ∞
token_renewable      false
token_policies       ["root"]
identity_policies    []
policies             ["root"]

Database Secrets Engine¶

Enable the Database Secrets Engine

vault secrets enable database

Expected output:

Success! Enabled the database secrets engine at: database/

MongoDB plugin¶

Configure the Vault mongodb Plugin

vault write database/config/vault-mongodb-database \
    plugin_name="mongodb-database-plugin" \
    allowed_roles="app-role" \
    connection_url="mongodb://{{username}}:{{password}}@vault-mongo:27017/admin" \
    username="admin" \
    password="XXXZZZ"

Configure Role¶

Configure a Role named app-role

The Role maps a name in Vault to a MongoDB command that executes and creates the database credential.

In this case we named it app-role

vault write database/roles/app-role \
    db_name="vault-mongodb-database" \
    creation_statements='{ "db": "admin", "roles": [{ "role": "readWriteAnyDatabase" }] }' \
    default_ttl="10s" \
    max_ttl="24h"

You should see this output:

Success! Data written to: database/roles/app-role

Notice

For testing we are setting the expiration (TTL) to 10s

When you will develop your app client, this can help you to quickly check renew of expired secrets.

Test the role¶

You can manually test the role

vault read database/creds/app-role

Expected output:

Key                Value
---                -----
lease_id           database/creds/app-role/UimmubtwwKvmDUb8nl9j0PmR
lease_duration     10s
lease_renewable    true
password           kWlWBuTgZBjz-Rqkr4q5
username           v-token-app-role-djsmjPAvH51x2Bn46xpM-1613489641

Use Vault API:¶

Example of how to use Vault API

From your Terminal/Postman, etc...

curl --header "X-Vault-Token: myroot" \
    http://127.0.0.1:1234/v1/database/creds/app-role

Expected Response:

{"request_id":"5abf9193-5c4c-377a-e370-9b0d9da23a97",
"lease_id":"database/creds/app-role/ZiECNiGpbd3XWJfnGfktLLEc",
"renewable":true,
"lease_duration":10,
"data":{"password":"Qg59PaTFK-hBWw3GpNy3","username":"v-token-app-role-fAD64LtblhPKmwK8af91-1613489864"},"wrap_info":null,"warnings":null,"auth":null}

Check the user in mongodb

Connect to the mongodb server and run show users right after running the above curl You should see that after 10 seconds the user will be removed from mongodb.

Python app example¶

Here is an example with python

response = requests.get(
'http://127.0.0.1:1234/v1/database/creds/app-role',
params={'q': 'requests+language:python'},
headers={'X-Vault-Token': 'myroot'},
)
json_response = response.json()
Database.USER = json_response['data']['username']
Database.PASSWORD = json_response['data']['password']
Database.URI = f'mongodb://{Database.USER}:{Database.PASSWORD}@{Database.SERVER}:{Database.PORT}'

Rotation¶

Password rotation

So far we used static role and it is set to expire (Delete) the db user and password every N seconds.

New pair will be created only if the application will initiate a request using the API.

We can also create a Static Pair, which will keep the same UserName and automatically rotate a new password every N hours/Days.

Login to mongodb and create a dedicated User:

docker run -it --rm --net=test-network \
    mongo:3.6 mongo  --host vault-mongo \
    -u admin -p XXXZZZ \
    --authenticationDatabase admin \
    admin

Create a user:

db.createUser(

    {
        user: "app-role",
        pwd: "YYYY",
        roles: [ { role: "userAdminAnyDatabase", db: "admin" }, "readWriteAnyDatabase" ]
    }
)

Create the plugin with the new Mongodb User

vault write database/config/vault-static-mongodb-database \
    plugin_name="mongodb-database-plugin" \
    allowed_roles="static-mongo-app-role" \
    connection_url="mongodb://{{username}}:{{password}}@vault-mongo:27017/admin" \
    username="app-role" \
    password="YYYY"

Create a json file with the rotation statement:

{ 
    "db_name": "vault-static-mongodb-database", 
    "username": "app-role", 
    "rotation_statements": "[\"ALTER USER \"{{name}}\" WITH PASSWORD '{{password}}';\"]", 
}

Now Create the Role:

vault write database/static-roles/static-mongo-app-role \
    db_name="vault-static-mongodb-database" \
    rotation_statements="@rotation.json" \
    username="app-role" \
    rotation_period=30

Expected output

Success! Data written to: database/static-roles/static-mongo-app-role

You can see the created role and statement by running:

vault read database/static-roles/static-mongo-app-role

Expected output:

Key                    Value
---                    -----
db_name                vault-static-mongodb-database
last_vault_rotation    2021-02-17T10:07:25.624867008Z
rotation_period        30s
rotation_statements    [{
    "db_name": "vault-static-mongodb-database",
    "username": "app-role",
    "rotation_statements": "[\"ALTER USER \"{{name}}\" WITH PASSWORD '{{password}}';\"]"

}]
username               app-role

Check the created credentials:

curl  --header "X-Vault-Token: myroot"  http://127.0.0.1:1234/v1/database/static-creds/static-mongo-app-role

{"request_id":"c641bb19-63ac-5920-578e-3decd537ff31",
"lease_id":"","renewable":false,"lease_duration":0,
"data":{"last_vault_rotation":"2021-02-17T09:43:00.57260073Z","password":"kY05UDN4-3YdA8PtqCSG","rotation_period":30,"ttl":5,"username":"app-role"},"wrap_info":null,"warnings":null,"auth":null}

Now you can see the rotation_period set to 30 and ttl is 5 seconds.

Run the command again, after expiration to see that new password was created, and ttl started countdown from 30 seconds again.

curl  --header "X-Vault-Token: myroot"  http://127.0.0.1:1234/v1/database/static-creds/static-mongo-app-role

{"request_id":"4f36c665-b1c7-1b46-5434-6343eafccfa7",
"lease_id":"","renewable":false,"lease_duration":0,
"data":{"last_vault_rotation":"2021-02-17T09:43:30.56875418Z","password":"oGvVoGGb3-u74fdtBhLC","rotation_period":30,"ttl":27,"username":"app-role"},"wrap_info":null,"warnings":null,"auth":null}

Try to connect to mongodb with the password from the output.

(increase TTL and write the role again with rotation_period=600 if you need more time )

docker run -it --rm --net=test-network \
    mongo:3.6 mongo  --host vault-mongo \
    -u app-role -p oGvVoGGb3-u74fdtBhLC \
    --authenticationDatabase admin \
    admin

Notice about db

In this example we used admin db, When in production, we will use only specific database/s which the app needs access to & we will use AppRole with a static-role credentials. The app will be granted to create mongodb credentials.

We will do that with Policy as mentioned in Vault documentation

This is explained below.

Production configuration¶

Creating AppRole

As mentioned at Vault documentation:

NOTE: For the purpose of this tutorial, you can use root token to work with Vault. However, it is recommended that root tokens are only used for just enough initial setup or in emergencies. As a best practice, use tokens with appropriate set of policies based on your role in the organization.

vault write database/config/vault-app-mongodb-database \
    plugin_name="mongodb-database-plugin" \
    allowed_roles="mongo-app-role" \
    connection_url="mongodb://{{username}}:{{password}}@vault-mongo:27017/admin" \
    username="admin" \
    password="XXXZZZ"

Create the role

vault write database/roles/mongo-app-role \
db_name="vault-app-mongodb-database" \
creation_statements='{ "db": "admin", "roles": [{ "role": "readWriteAnyDatabase" }] }' \
default_ttl="1h" \
max_ttl="24h"

Expected output:

Success! Data written to: database/roles/mongo-app-role

Check the created role:

vault read database/config/vault-app-mongodb-database

Key                                   Value
---                                   -----
allowed_roles                         [mongo-app-role]
connection_details                    map[connection_url:mongodb://{{username}}:{{password}}@vault-mongo:27017/admin username:admin]
password_policy                       n/a
plugin_name                           mongodb-database-plugin
root_credentials_rotate_statements    []

Check credentials:

curl  --header "X-Vault-Token: myroot"  http://127.0.0.1:1234/v1/database/creds/mongo-app-role

Expected output:

{"request_id":"fa3e2aae-9604-dbdc-ab57-00b8bb8be646",
"lease_id":"database/creds/mongo-app-role/WfqwEfk8NipOgXZvlkspIh6A",
"renewable":true,"lease_duration":3600,
"data":{"password":"QMbr5vsyJRtziVebeJp-","username":"v-token-mongo-app-role-Cy8GWTHPDMdBRef2JE5B-1613563679"},"wrap_info":null,"warnings":null,"auth":null}

Test the credentials from the output:

docker run -it --rm --net=test-network \
    mongo:3.6 mongo --host vault-mongo \
    -u v-token-mongo-app-role-dyZW9BQ0dTGD25gGcnLh-1613564363 -p rVkxH-rt-YG-g7hE3Xvn \
    --authenticationDatabase admin \
    admin

Create a Policy¶

Create db-policy

Go to the vault UI and create a Policy named db-policy

path "sys/mounts/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}

path "database/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}

path "sys/policies/acl/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}

path "auth/token/create" {
capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

Enable approle engine

vault auth enable approle

Create the AppRole db-app-role

curl  --header "X-Vault-Token: myroot"  --request POST  --data '{"policies": "db-policy","token_ttl": "60m"}'  http://127.0.0.1:1234/v1/auth/approle/role/db-app-role

Get the AppRole ID

curl  --header "X-Vault-Token: myroot"  http://127.0.0.1:1234/v1/auth/approle/role/db-app-role/role-id

Expected output:

{"request_id":"e1d0e823-a850-6844-5355-6fb2471f61a2","lease_id":"","renewable":false,"lease_duration":0,"data":{"role_id":"73e02815-2cd4-67ce-e1e8-9e3eaa3db2d6"},"wrap_info":null,"warnings":null,"auth":null}

Get the Secret

vault write -f auth/approle/role/db-app-role/secret-id

Expected output:

Key                   Value
---                   -----
secret_id             1189f37d-5b91-ecf5-3ed3-3b9ff67eed53
secret_id_accessor    b509e354-2df9-caed-0944-c31a3e9b80fe

App Authentication for AppRole ID

Now Applications can use the AppRole ID and Secret to read creds path on Vault, and get the credentials (with rotating password) for mongodb.

Let's test this in Docker as well

Create an empty folder and create the following files in it:

Create Dockerfile

FROM python:3.8-alpine

RUN mkdir /usr/src/app
WORKDIR /usr/src/app

COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

COPY app.py  /usr/src/app/app.py
CMD exec sh -c "python /usr/src/app/app.py; trap : TERM INT;"

Create requirements.txt file:

hvac==0.10.8

Create app.py file:

import hvac

client = hvac.Client(url="http://127.0.0.1:1234")
client.auth.approle.login('ROLE_ID','SECRET_ID')
result = client.read('database/creds/mongo-app-role')
print(result)

Replace the ROLE_ID & SECRET_ID with your outputs.

Build the container image:

docker build -t vault-test .

Run the container:

docker run  --net=host vault-test

Expected output:

{"request_id": "fccb225a-2bc8-9718-20b2-5b11a2d29fed", 
"lease_id": "database/creds/mongo-app-role/Lj6sTK04Jwd30HhGPtSZJYBl", 
"renewable": "True", "lease_duration": 3600, 
"data": {"password": "Pn-m80WNPdmzAx32rdgf", "username": "v-approle-mongo-app-role-8qZO0k6hMQ6VCsWqjU8C-1613569689"}, "wrap_info": "None", "warnings": "None", "auth": "None"}

Summary¶

That's it.

Hopefully you now understand how this works, and able to develop your application easily.

Some things are different when running in production, however the code implementation in your app should be pretty much (exactly) identical to what we have done here.

Vault documentation can be found Here

Vault & Mongodb

Getting Started¶

Docker network¶

Running Vault¶

Run mongodb¶

Connect to mongodb¶

Login to Vault¶

Database Secrets Engine¶

MongoDB plugin¶

Configure Role¶

Test the role¶

Use Vault API:¶

Python app example¶

Rotation¶

Production configuration¶

Create a Policy¶

Summary¶

Comments