Vault + Golang 安全管理代码配置中的各种密钥
这篇文章旨在介绍 HashiCorp Vault 官方推荐通用的架设实践指南，在实际生产环境中根据实际生产环境中的需求做对应适配更改。
该指南使用生产环境部署推荐把 Consul 作为 Vault 的后端存储
单数据中心部署拓扑结构
这部分解释如何部署单数据中心开源版 Vault 集群。Vault 企业版的集群复制特性支持多数据中心拓扑结构。
拓扑结构示意图
基于consul 后端存储的八个节点拓扑图
设计概述
使用开源版本的情况下，推荐按照该模式架设 Vault，具备灵活性和弹性。Consul 服务端和 Vault 服务端分离，方便软件升级管理，另外根据 consul 和 vault 对服务器不同的性能需求，可灵活选用服务器配置。Vault 和 consul 通过 TSL 安全加密和的 HTTP 协议和携带 consul 令牌两者之间进行通信，建立一个加密安全的交互通道。
分布式容错
高带宽、低延迟的跨地区云服务部署是比较典型的分布式架设方式，同样也适用于做 Consul/Vaule 分布式集群架设。接下来的方案通过 Consul 服务配置跨地区冗余的方法实现 Vaule 和 Consule 跨地区集群架设。每个可用区 (Availability Zones) 都选举出一个 consul 成员实现跨地区集群，跨地区和单节点级别的容错集群。
网络连接详情
部署对操作系统和服务器硬件的需求
不同规模的 Vaule 对服务器硬件规格的需求表，特别强调不要使用不固定性能的 CPU 类型
部署 Vault 服务端的服务器规格
| Size | CPU | Memory | Disk | Typical Cloud Instance Types |
|---|---|---|---|---|
| Small | 2 core | 4-8 GB RAM | 25 GB | AWS: m5.large |
| Azure: Standard_D2_v3 | ||||
| GCE: n1-standard-2, n1-standard-4 | ||||
| Large | 4-8 core | 16-32 GB RAM | 50 GB | AWS: m5.xlarge, m5.2xlarge |
| Azure: Standard_D4_v3, Standard_D8_v3 | ||||
| GCE: n1-standard-8, n1-standard-16 |
部署 Consul 服务端的服务器规格
| Size | CPU | Memory | Disk | Typical Cloud Instance Types |
|---|---|---|---|---|
| Small | 2 core | 8-16 GB RAM | 50 GB | AWS: m5.large, m5.xlarge |
| Azure: Standard_D2_v3, Standard_D4_v3 | ||||
| GCE: n1-standard-4, n1-standard-8 | ||||
| Large | 4-8 core | 32-64+ GB RAM | 100 GB | AWS: m5.2xlarge, m5.4xlarge |
| Azure: Standard_D4_v3, Standard_D8_v3 | ||||
| GCE: n1-standard-16, n1-standard-32 |
上表中小规格的服务器基本上满足大部分部署要求，包括开发、测试或生产环境部署。如果生产环境面临持续高负载，也就是你的系统面临大量的数据处理、大量密钥读写的场景，则需选用上表中列出的高规格服务器。
另外这里有一份生产环境安全加固要素 check list
安装和配置详情
为了提供单数据中心的高可用架设，建议在多台服务器上实现 Consul 集群来保持数据一致性。我在开发环境，使用 Docker 一键安装非常方便。默认你的服务器上已经安装并正常启动 docker 服务。
新建 docker-compose.yml 文件及相关配置文件
cd ~
mkdir docker_vault && cd docker_vault && mkdir config
touch docker-compose.yml && touch config/vault.hcl config/consul.json config/admin.hcl
新建两个文件的内容如下：
// docker-compose.yml
version: '2'
services:
consul:
container_name: consul
image: consul:latest
ports:
- "8500:8500"
- "8300:8300"
volumes:
- ./config:/config
- ./_data/consul:/data
command: agent -server -bind 0.0.0.0 -client 0.0.0.0 -bootstrap-expect=1 -config-file=/config/consul.json
vault:
container_name: vault
image: vault
links:
- consul:consul
depends_on:
- consul
ports:
- "8200:8200"
volumes_from:
- consul
cap_add:
- IPC_LOCK
command: server -config=/config/vault.hcl
webui:
container_name: webui
image: djenriquez/vault-ui
ports:
- "8000:8000"
links:
- vault:vault
environment:
NODE_TLS_REJECT_UNAUTHORIZED: 0
VAULT_URL_DEFAULT: http://vault:8200
vault & consul 的配置文件：
// config/vault.hcl
backend "consul" {
address = "consul:8500"
advertise_addr = "http://consul:8300"
scheme = "http"
}
listener "tcp" {
address = "0.0.0.0:8200"
#tls_cert_file = "/config/server.crt"
#tls_key_file = "/config/server.key"
tls_disable = 1
}
disable_mlock = true
// config/consul.json
{
"datacenter": "localhost",
"data_dir": "/_data/consul",
"log_level": "INFO",
"server": true,
"ui": true
}
// config/admin.hcl
{
"path": {
"*": {
"capabilities": [
"create",
"read",
"update",
"delete",
"list",
"sudo"
]
},
"ultrasecret/admincantlistthis/": {
"capabilities": [
"deny"
]
},
"ultrasecret/admincantreadthis": {
"capabilities": [
"deny"
]
}
}
}
接下来安装并启动 Vault, Consul 及 Vaule webui 服务
ubuntu@10-8-36-35:~/docker_vault$ docker-compose up
Creating network "docker_vault_default" with the default driver
Pulling consul (consul:latest)...
latest: Pulling from library/consul
bdf0201b3a05: Pull complete
af3d1f90fc60: Pull complete
d3a756372895: Pull complete
54efc599d7c7: Pull complete
73d2c234fe14: Pull complete
cbf8018e609a: Pull complete
Pulling vault (vault:)...
latest: Pulling from library/vault
c87736221ed0: Pull complete
6a59b449ff8b: Pull complete
a69031de877f: Pull complete
0b300d1134aa: Pull complete
12abd110ba3b: Pull complete
Creating consul ... done
Creating vault ... done
Creating webui ... done
Attaching to consul, vault, webui
consul | BootstrapExpect is set to 1; this is the same as Bootstrap mode.
consul | bootstrap = true: do not enable unless necessary
consul | ==> Starting Consul agent...
consul | ==> Consul agent running!
consul | Version: 'v1.4.4'
consul | Node ID: 'd82088fc-1bef-ebfb-9fb6-edb48121d156'
consul | Node name: 'ea610565d56d'
consul | Datacenter: 'dc1' (Segment: '<all>')
consul | Server: true (Bootstrap: true)
consul | Client Addr: [0.0.0.0] (HTTP: 8500, HTTPS: -1, gRPC: -1, DNS: 8600)
consul | Cluster Addr: 172.19.0.2 (LAN: 8301, WAN: 8302)
consul | Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false
consul |
consul | ==> Log data will now stream in as it occurs:
consul |
consul | 2019/05/02 16:47:50 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:d82088fc-1bef-ebfb-9fb6-edb48121d156 Address:172.19.0.2:8300}]
consul | 2019/05/02 16:47:50 [INFO] raft: Node at 172.19.0.2:8300 [Follower] entering Follower state (Leader: "")
consul | 2019/05/02 16:47:50 [INFO] serf: EventMemberJoin: ea610565d56d.dc1 172.19.0.2
consul | 2019/05/02 16:47:50 [INFO] serf: EventMemberJoin: ea610565d56d 172.19.0.2
consul | 2019/05/02 16:47:50 [INFO] agent: Started DNS server 0.0.0.0:8600 (udp)
consul | 2019/05/02 16:47:50 [INFO] consul: Handled member-join event for server "ea610565d56d.dc1" in area "wan"
consul | 2019/05/02 16:47:50 [INFO] consul: Adding LAN server ea610565d56d (Addr: tcp/172.19.0.2:8300) (DC: dc1)
consul | 2019/05/02 16:47:50 [INFO] agent: Started DNS server 0.0.0.0:8600 (tcp)
consul | 2019/05/02 16:47:50 [INFO] agent: Started HTTP server on [::]:8500 (tcp)
consul | 2019/05/02 16:47:50 [INFO] agent: started state syncer
vault | A storage backend must be specified
vault exited with code 1
webui | yarn run v1.6.0
webui | $ node ./server.js start_app
webui | Vault UI listening on: 8000
consul | 2019/05/02 16:47:57 [ERR] agent: failed to sync remote state: No cluster leader
consul | 2019/05/02 16:47:58 [WARN] raft: Heartbeat timeout from "" reached, starting election
consul | 2019/05/02 16:47:58 [INFO] raft: Node at 172.19.0.2:8300 [Candidate] entering Candidate state in term 2
consul | 2019/05/02 16:47:58 [INFO] raft: Election won. Tally: 1
consul | 2019/05/02 16:47:58 [INFO] raft: Node at 172.19.0.2:8300 [Leader] entering Leader state
consul | 2019/05/02 16:47:58 [INFO] consul: cluster leadership acquired
consul | 2019/05/02 16:47:58 [INFO] consul: New leader elected: ea610565d56d
consul | 2019/05/02 16:47:58 [INFO] consul: member 'ea610565d56d' joined, marking health alive
consul | 2019/05/02 16:47:59 [INFO] agent: Synced node info
容器服务正常启动，如果你的服务器 8000 端口对外网开放的话，就可以访问 Vault 图形化页面了 http://{server_ip}:8000
最后，如果你想通过 Web UI 登录，还需要 Unseal vault，方法如下：
// 在 docker_vault 目录下运行
docker-compose exec vault sh // 进入 容器的 terminal
export VAULT_ADDR=http://127.0.0.1:8200
/* 务必要记下这五个 key, 每次登录之前都需要使用其中三个 key unseal */
vault operator init -address=${VAULT_ADDR}
Unseal Key 1: aaaaa
Unseal Key 2: bbbbb
Unseal Key 3: ccccc
Unseal Key 4: ddddd
Unseal Key 5: eeeee
Initial Root Token: s.xxxx
/* 然后使用其中三个 Unseal Key 执行三次 */
vault operator unseal -address=${VAULT_ADDR} Unseal_keyxxx
/* unseal 成功之后会显示，其中 Sealed 为 false */
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed false
Total Shares 5
Threshold 3
Version 1.1.2
Cluster Name vault-cluster-8cf5c69a
Cluster ID 50ebdb22-e4c2-e070-bda7-677a7b1786b7
HA Enabled true
HA Cluster n/a
HA Mode standby
Active Node Address <none>
命令行新增 token
//https://www.vaultproject.io/docs/commands/index.html
vault login {root_token}
vault token create -policy=my-policy -period=720h
unseal 之后，即可使用 Root Token 登录 webui 添加新的 Policy 用来新建用户。在添加用户之前需要点击侧边栏新增 userpass 类型的 Auth backend
Policy 规则复制 config/admin.hcl 填充，并命名为 admin
需要强调的是，unseal 有时效，过期了之后如果你想登录，仍需使用其中三把 Unseal key 进行 Unseal。
Golang 读写例子
// vault.go
package vault
import (
"log"
"github.com/caarlos0/env"
"github.com/hashicorp/vault/api"
)
var (
cfg config
)
// Config vault server info
type config struct {
Address string `env:"address"`
Token string `env:"token"`
}
// Client vault client
type Client struct {
*api.Client
}
// NewVault new vault client
func NewVault() (*Client, error) {
client, err := api.NewClient(&api.Config{
Address: cfg.Address,
})
if err != nil {
return nil, err
}
client.SetToken(cfg.Token)
return &Client{client}, nil
}
func init() {
cfg = config{}
if err := env.Parse(&cfg); err != nil {
log.Fatalln(err.Error())
}
}
测试代码：
package vault
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestNewVault(t *testing.T) {
c, e := NewVault()
assert.NotNil(t, c)
assert.Nil(t, e)
assert.NotEmpty(t, c.Token())
}
func TestReadWrite(t *testing.T) {
c, _ := NewVault()
secretData := map[string]interface{}{
"foo": "bar",
"age": "-1",
}
_, err := c.Logical().Write("kv1/my-secret", secretData)
assert.Nil(t, err)
s, err := c.Logical().Read("kv1/my-secret")
assert.Nil(t, err)
if assert.NotNil(t, s) {
assert.Equal(t, "bar", s.Data["foo"], )
}
}
跑测试：
$ address=http://127.0.0.1:8200 token=xxx go test -run TestReadWrite ./library/vaul
ok trustkeeper-go/library/vault