PostgreSQL深度实践:从Debian 12入门到集群部署及多语言应用

PostgreSQL深度实践:从Debian 12入门到集群部署及多语言应用

PG简述

PostgreSQL作为世界上最先进的开源关系型数据库系统,凭借其强大的功能、卓越的性能和严格的ACID特性,已经成为企业级应用的首选数据库之一。
本文将带领读者从零开始,在Debian 12环境下深入掌握PostgreSQL,涵盖版本演进、核心原理、实战配置、集群部署以及多语言应用开发,为数据库工程师和开发者提供一套完整的实践指南,包含常用的一些语法操作和最佳实践等等。


第一部分:PostgreSQL版本历史与特性演进

1.1 版本发展脉络

PostgreSQL的发展历程见证了开源数据库技术的演进:

  • PostgreSQL 7.x (2000-2005): 从Postgres95更名而来,引入了真正的多版本并发控制(MVCC)
  • PostgreSQL 8.x (2005-2010): Windows原生支持、自动清理、表空间管理
  • PostgreSQL 9.x (2010-2016): 流复制、JSON支持、物化视图
  • PostgreSQL 10.x (2017): 逻辑复制、声明式分区、并行查询增强
  • PostgreSQL 11.x (2018): JIT编译、存储过程、分区表性能优化
  • PostgreSQL 12.x (2019): 通用表表达式(CTE)性能优化、JSON路径表达式
  • PostgreSQL 13.x (2020): 增量排序、B-tree索引优化、逻辑复制改进
  • PostgreSQL 14.x (2021): 多范围类型、数据类型优化、性能监控增强
  • PostgreSQL 15.x (2022): MERGE命令、分布式SQL功能增强
  • PostgreSQL 16.x (2023): 逻辑复制性能大幅提升、查询并行化改进

1.2 关键特性对比

特性PostgreSQL 12PostgreSQL 14PostgreSQL 16
逻辑复制基础支持大幅改进性能提升300%
并行查询有限支持增强支持完全优化
JSON支持JSONB类型JSON路径表达式完整SQL/JSON标准
分区表声明式分区性能优化分区裁剪优化
索引类型B-tree, HashBRIN, SP-GiST覆盖索引优化

重点注意事项:生产环境升级时,务必先在测试环境验证,特别注意13+版本对旧版本的兼容性变化,如默认身份验证方法从peer改为scram-sha-256。


第二部分:Debian 12环境下的PostgreSQL实战

2.1 安装与基础配置

2.1.1 安装PostgreSQL 16

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# 更新系统
sudo apt update && sudo apt upgrade -y

# 添加官方仓库
sudo apt install -y curl ca-certificates
curl https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo gpg --dearmor -o /usr/share/keyrings/postgresql.gpg
echo "deb [signed-by=/usr/share/keyrings/postgresql.gpg] http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" | sudo tee /etc/apt/sources.list.d/pgdg.list

# 安装PostgreSQL 16
sudo apt update
sudo apt install -y postgresql-16 postgresql-client-16 postgresql-contrib-16

# 启动服务
sudo systemctl enable postgresql@16-main
sudo systemctl start postgresql@16-main

重要安全配置:安装完成后立即修改默认postgres用户的密码,并配置适当的访问控制。

2.1.2 基础目录结构

1
2
3
4
/etc/postgresql/16/main/          # 配置文件目录
/var/lib/postgresql/16/main/ # 数据文件目录
/var/log/postgresql/ # 日志文件目录
/usr/lib/postgresql/16/bin/ # 可执行文件目录

2.2 核心原理与架构

2.2.1 进程架构

PostgreSQL采用多进程架构,主要进程包括:

  • Postmaster进程:主进程,负责监听连接和spawn子进程
  • Backend进程:每个客户端连接对应一个backend进程
  • Checkpointer进程:负责检查点操作
  • WAL Writer进程:负责WAL日志写入
  • Autovacuum进程:自动清理死元组
  • Background Writer进程:后台写入脏页

2.2.2 存储结构

  • 表空间(Tablespace):物理存储位置
  • 数据库(Database):逻辑容器
  • 模式(Schema):命名空间
  • 表(Table):数据存储单位
  • 页面(Page):8KB的基本IO单位

性能关键点:合理配置shared_buffers(通常为物理内存的25%)和work_mem(根据并发查询数调整)对性能至关重要。

2.3 基础语法与高级特性

2.3.1 基础CRUD操作

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
-- 创建数据库
CREATE DATABASE demo_db OWNER postgres;

-- 创建表
CREATE TABLE users (
id SERIAL PRIMARY KEY,
name VARCHAR(100) NOT NULL,
email VARCHAR(150) UNIQUE,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
is_active BOOLEAN DEFAULT true
);

-- 插入数据
INSERT INTO users (name, email) VALUES
('张三', 'zhangsan@example.com'),
('李四', 'lisi@example.com');

-- 查询数据
SELECT * FROM users WHERE is_active = true ORDER BY created_at DESC LIMIT 10;

-- 更新数据
UPDATE users SET is_active = false WHERE id = 1;

-- 删除数据
DELETE FROM users WHERE created_at < NOW() - INTERVAL '1 year';

2.3.2 高级特性实战

JSONB操作

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
-- 创建JSONB字段表
CREATE TABLE products (
id SERIAL PRIMARY KEY,
name VARCHAR(100),
attributes JSONB
);

-- 插入JSON数据
INSERT INTO products (name, attributes) VALUES
('笔记本电脑', '{"brand": "Dell", "price": 8999, "specs": {"cpu": "i7", "ram": "16GB"}}'),
('智能手机', '{"brand": "Apple", "price": 6999, "specs": {"storage": "256GB", "color": "black"}}');

-- JSONB查询
SELECT name, attributes->>'brand' as brand, attributes->'specs'->>'cpu' as cpu
FROM products
WHERE attributes @> '{"brand": "Dell"}';

窗口函数

1
2
3
4
5
6
7
8
-- 销售数据分析
SELECT
department,
employee_name,
salary,
AVG(salary) OVER (PARTITION BY department) as dept_avg_salary,
RANK() OVER (PARTITION BY department ORDER BY salary DESC) as salary_rank
FROM employees;

重点注意事项:使用JSONB时,建议为常用查询路径创建GIN索引;窗口函数在大数据量时可能影响性能,需要合理使用。


第三部分:从单体到集群的演进

3.1 单体架构优化

3.1.1 配置文件优化

编辑 /etc/postgresql/16/main/postgresql.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# 内存配置
shared_buffers = 4GB # 物理内存的25%
effective_cache_size = 12GB # 物理内存的75%
work_mem = 64MB # 每个排序操作的内存
maintenance_work_mem = 512MB # 维护操作内存

# WAL配置
wal_level = replica # 支持复制
max_wal_senders = 10 # 最大WAL发送进程数
wal_keep_size = 1GB # 保留的WAL文件大小

# 连接配置
max_connections = 200 # 最大连接数
superuser_reserved_connections = 5 # 保留给超级用户的连接

# 并行查询
max_parallel_workers_per_gather = 4 # 每个查询的最大并行worker
max_parallel_workers = 8 # 系统最大并行worker

3.1.2 性能监控

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
-- 安装监控扩展
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

-- 查看慢查询
SELECT query, calls, total_exec_time, rows
FROM pg_stat_statements
ORDER BY total_exec_time DESC
LIMIT 10;

-- 索引使用情况
SELECT
schemaname,
tablename,
indexname,
idx_scan as index_scans,
idx_tup_read as tuples_read
FROM pg_stat_user_indexes
WHERE schemaname = 'public'
ORDER BY idx_scan DESC;

3.2 主从复制配置

3.2.1 主服务器配置

  1. 修改主服务器postgresql.conf:

    1
    2
    3
    wal_level = replica
    max_wal_senders = 10
    wal_keep_size = 1GB
  2. 配置pg_hba.conf:

    1
    2
    # 允许从服务器连接
    host replication replicator 192.168.1.101/32 scram-sha-256
  3. 创建复制用户:

    1
    CREATE ROLE replicator WITH REPLICATION LOGIN PASSWORD 'secure_password';

3.2.2 从服务器配置

  1. 停止从服务器PostgreSQL服务

  2. 使用pg_basebackup同步数据:

    1
    sudo -u postgres pg_basebackup -h 192.168.1.100 -U replicator -D /var/lib/postgresql/16/main -P -v -R -X stream
  3. 修改recovery.conf(PostgreSQL 12+在postgresql.conf中配置):

    1
    2
    primary_conninfo = 'host=192.168.1.100 port=5432 user=replicator password=secure_password'
    standby_mode = on
  4. 启动从服务器

关键注意事项:主从复制延迟监控至关重要,使用pg_stat_replication视图实时监控复制状态,设置合理的监控告警阈值。

3.3 集群部署方案

3.3.1 使用Patroni实现高可用

Patroni是一个基于etcd的PostgreSQL高可用解决方案:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
# patroni.yml 配置示例
scope: postgres-cluster
name: pg-node-1

restapi:
listen: 0.0.0.0:8008
connect_address: 192.168.1.100:8008

etcd:
hosts: ["192.168.1.200:2379", "192.168.1.201:2379", "192.168.1.202:2379"]

bootstrap:
dcs:
ttl: 30
loop_wait: 10
retry_timeout: 10
maximum_lag_on_failover: 1048576
postgresql:
use_pg_rewind: true
parameters:
wal_level: replica
hot_standby: "on"
max_connections: 200
max_wal_senders: 10
wal_keep_size: 1GB

postgresql:
listen: 0.0.0.0:5432
connect_address: 192.168.1.100:5432
data_dir: /var/lib/postgresql/16/main
bin_dir: /usr/lib/postgresql/16/bin
authentication:
replication:
username: replicator
password: secure_password
superuser:
username: postgres
password: super_secure_password

集群运维要点:定期进行故障切换演练,监控etcd集群健康状态,确保网络延迟在可接受范围内(通常<10ms)。


第四部分:Docker容器化部署

4.1 单容器部署

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# 拉取官方镜像
docker pull postgres:16-alpine

# 运行容器
docker run -d \
--name postgres-single \
-p 5432:5432 \
-e POSTGRES_USER=admin \
-e POSTGRES_PASSWORD=secure_password \
-e POSTGRES_DB=myapp \
-v postgres_data:/var/lib/postgresql/data \
-v ./init-scripts:/docker-entrypoint-initdb.d \
--restart unless-stopped \
postgres:16-alpine

init-scripts/01-init.sql

1
2
3
4
5
6
7
8
9
10
CREATE TABLE IF NOT EXISTS app_config (
key VARCHAR(50) PRIMARY KEY,
value TEXT,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

INSERT INTO app_config (key, value) VALUES
('app_name', 'My Application'),
('version', '1.0.0'),
('maintenance_mode', 'false');

4.2 Docker Compose多容器编排

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
# docker-compose.yml
version: '3.8'

services:
postgres-primary:
image: postgres:16-alpine
container_name: pg-primary
environment:
POSTGRES_USER: replicator
POSTGRES_PASSWORD: replica_password
POSTGRES_DB: replication_db
volumes:
- pg_primary_data:/var/lib/postgresql/data
- ./conf/postgresql.conf:/etc/postgresql/postgresql.conf
ports:
- "5432:5432"
networks:
- pg-network
restart: unless-stopped

postgres-replica:
image: postgres:16-alpine
container_name: pg-replica
environment:
POSTGRES_USER: replicator
POSTGRES_PASSWORD: replica_password
POSTGRES_DB: replication_db
volumes:
- pg_replica_data:/var/lib/postgresql/data
depends_on:
- postgres-primary
networks:
- pg-network
restart: unless-stopped

pgadmin:
image: dpage/pgadmin4
container_name: pgadmin
environment:
PGADMIN_DEFAULT_EMAIL: admin@example.com
PGADMIN_DEFAULT_PASSWORD: pgadmin_password
ports:
- "8080:80"
networks:
- pg-network
restart: unless-stopped

volumes:
pg_primary_data:
pg_replica_data:

networks:
pg-network:
driver: bridge

4.3 集群容器化方案

使用Docker Swarm或Kubernetes部署PostgreSQL集群:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# kubernetes-postgres-statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres
spec:
serviceName: postgres
replicas: 3
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: postgres:16-alpine
env:
- name: POSTGRES_USER
value: "postgres"
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgres-secret
key: password
ports:
- containerPort: 5432
volumeMounts:
- name: postgres-storage
mountPath: /var/lib/postgresql/data
volumeClaimTemplates:
- metadata:
name: postgres-storage
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "standard"
resources:
requests:
storage: 100Gi

容器化注意事项

  • 生产环境务必使用持久化卷,避免数据丢失
  • 合理配置资源限制(CPU、内存)
  • 使用secret管理敏感信息
  • 定期备份容器内数据

第五部分:多语言应用开发实战

5.1 PHP操作PostgreSQL

5.1.1 环境配置

1
2
3
# 安装PHP PostgreSQL扩展
sudo apt install -y php-pgsql
sudo systemctl restart apache2

5.1.2 完整示例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
<?php
/**
* PostgreSQL PHP操作完整示例
* 包含连接池、事务、预处理语句等最佳实践
*/

class PostgreSQLService {
private $connection;
private $config = [
'host' => 'localhost',
'port' => 5432,
'dbname' => 'myapp',
'user' => 'app_user',
'password' => 'secure_password',
'options' => '--client_encoding=UTF8'
];

public function __construct() {
$this->connect();
}

private function connect() {
$dsn = sprintf(
"pgsql:host=%s;port=%s;dbname=%s;options='%s'",
$this->config['host'],
$this->config['port'],
$this->config['dbname'],
$this->config['options']
);

try {
$this->connection = new PDO($dsn, $this->config['user'], $this->config['password'], [
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC,
PDO::ATTR_EMULATE_PREPARES => false
]);
} catch (PDOException $e) {
error_log("Database connection failed: " . $e->getMessage());
throw new Exception("Database connection failed");
}
}

/**
* 创建用户
*/
public function createUser($name, $email) {
$query = "INSERT INTO users (name, email, created_at) VALUES (?, ?, NOW()) RETURNING id";

try {
$stmt = $this->connection->prepare($query);
$stmt->execute([$name, $email]);
return $stmt->fetchColumn();
} catch (PDOException $e) {
error_log("Create user failed: " . $e->getMessage());
throw new Exception("Failed to create user");
}
}

/**
* 获取用户详情
*/
public function getUser($userId) {
$query = "SELECT id, name, email, created_at FROM users WHERE id = ? AND is_active = true";

try {
$stmt = $this->connection->prepare($query);
$stmt->execute([$userId]);
return $stmt->fetch();
} catch (PDOException $e) {
error_log("Get user failed: " . $e->getMessage());
return null;
}
}

/**
* 获取用户列表(分页)
*/
public function getUserList($page = 1, $pageSize = 20) {
$offset = ($page - 1) * $pageSize;
$query = "SELECT id, name, email, created_at FROM users WHERE is_active = true ORDER BY created_at DESC LIMIT ? OFFSET ?";

try {
$stmt = $this->connection->prepare($query);
$stmt->execute([$pageSize, $offset]);
return $stmt->fetchAll();
} catch (PDOException $e) {
error_log("Get user list failed: " . $e->getMessage());
return [];
}
}

/**
* 事务操作示例
*/
public function transferMoney($fromUserId, $toUserId, $amount) {
try {
$this->connection->beginTransaction();

// 检查余额
$checkQuery = "SELECT balance FROM accounts WHERE user_id = ? FOR UPDATE";
$stmt = $this->connection->prepare($checkQuery);
$stmt->execute([$fromUserId]);
$fromBalance = $stmt->fetchColumn();

if ($fromBalance < $amount) {
throw new Exception("Insufficient balance");
}

// 扣款
$debitQuery = "UPDATE accounts SET balance = balance - ? WHERE user_id = ?";
$stmt = $this->connection->prepare($debitQuery);
$stmt->execute([$amount, $fromUserId]);

// 入账
$creditQuery = "UPDATE accounts SET balance = balance + ? WHERE user_id = ?";
$stmt = $this->connection->prepare($creditQuery);
$stmt->execute([$amount, $toUserId]);

$this->connection->commit();
return true;

} catch (Exception $e) {
$this->connection->rollBack();
error_log("Transaction failed: " . $e->getMessage());
throw $e;
}
}

/**
* JSONB数据操作
*/
public function updateUserProfile($userId, array $profileData) {
$jsonData = json_encode($profileData, JSON_UNESCAPED_UNICODE);
$query = "UPDATE users SET profile_data = ?::jsonb WHERE id = ?";

try {
$stmt = $this->connection->prepare($query);
return $stmt->execute([$jsonData, $userId]);
} catch (PDOException $e) {
error_log("Update profile failed: " . $e->getMessage());
return false;
}
}

public function __destruct() {
$this->connection = null;
}
}

// 使用示例
try {
$db = new PostgreSQLService();

// 创建用户
$userId = $db->createUser('王五', 'wangwu@example.com');
echo "Created user with ID: $userId\n";

// 获取用户
$user = $db->getUser($userId);
if ($user) {
echo "User found: " . print_r($user, true) . "\n";
}

// 事务操作
$db->transferMoney(1, 2, 100.00);
echo "Money transfer successful\n";

} catch (Exception $e) {
echo "Error: " . $e->getMessage() . "\n";
}

5.2 Golang操作PostgreSQL

5.2.1 依赖安装

1
2
go get github.com/jackc/pgx/v5
go get github.com/jackc/pgx/v5/pgxpool

5.2.2 完整示例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
package main

import (
"context"
"fmt"
"log"
"time"

"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)

// 数据库配置
type DBConfig struct {
Host string
Port int
User string
Password string
Database string
MaxConns int32
}

// 用户模型
type User struct {
ID int64 `json:"id"`
Name string `json:"name"`
Email string `json:"email"`
CreatedAt time.Time `json:"created_at"`
IsActive bool `json:"is_active"`
}

// PostgreSQL服务
type PostgreSQLService struct {
pool *pgxpool.Pool
}

// 初始化数据库连接
func NewPostgreSQLService(config *DBConfig) (*PostgreSQLService, error) {
connStr := fmt.Sprintf("postgres://%s:%s@%s:%d/%s?pool_max_conns=%d",
config.User,
config.Password,
config.Host,
config.Port,
config.Database,
config.MaxConns)

pool, err := pgxpool.New(context.Background(), connStr)
if err != nil {
return nil, fmt.Errorf("failed to create connection pool: %w", err)
}

// 测试连接
if err := pool.Ping(context.Background()); err != nil {
pool.Close()
return nil, fmt.Errorf("failed to ping database: %w", err)
}

return &PostgreSQLService{pool: pool}, nil
}

// 关闭连接池
func (s *PostgreSQLService) Close() {
s.pool.Close()
}

// 创建用户
func (s *PostgreSQLService) CreateUser(ctx context.Context, name, email string) (int64, error) {
var userID int64
err := s.pool.QueryRow(ctx, `
INSERT INTO users (name, email, created_at, is_active)
VALUES ($1, $2, NOW(), true)
RETURNING id
`, name, email).Scan(&userID)

return userID, err
}

// 获取用户
func (s *PostgreSQLService) GetUser(ctx context.Context, userID int64) (*User, error) {
user := &User{}
err := s.pool.QueryRow(ctx, `
SELECT id, name, email, created_at, is_active
FROM users
WHERE id = $1 AND is_active = true
`, userID).Scan(
&user.ID,
&user.Name,
&user.Email,
&user.CreatedAt,
&user.IsActive,
)

if err != nil {
if err == pgx.ErrNoRows {
return nil, nil
}
return nil, err
}

return user, nil
}

// 获取用户列表
func (s *PostgreSQLService) GetUserList(ctx context.Context, page, pageSize int) ([]User, error) {
offset := (page - 1) * pageSize
rows, err := s.pool.Query(ctx, `
SELECT id, name, email, created_at, is_active
FROM users
WHERE is_active = true
ORDER BY created_at DESC
LIMIT $1 OFFSET $2
`, pageSize, offset)
if err != nil {
return nil, err
}
defer rows.Close()

var users []User
for rows.Next() {
var user User
if err := rows.Scan(
&user.ID,
&user.Name,
&user.Email,
&user.CreatedAt,
&user.IsActive,
); err != nil {
return nil, err
}
users = append(users, user)
}

return users, rows.Err()
}

// 事务操作
func (s *PostgreSQLService) TransferMoney(ctx context.Context, fromUserID, toUserID int64, amount float64) error {
tx, err := s.pool.Begin(ctx)
if err != nil {
return err
}
defer func() {
if err != nil {
tx.Rollback(ctx)
}
}()

// 检查余额
var fromBalance float64
err = tx.QueryRow(ctx, `
SELECT balance FROM accounts WHERE user_id = $1 FOR UPDATE
`, fromUserID).Scan(&fromBalance)
if err != nil {
return err
}

if fromBalance < amount {
return fmt.Errorf("insufficient balance")
}

// 扣款
_, err = tx.Exec(ctx, `
UPDATE accounts SET balance = balance - $1 WHERE user_id = $2
`, amount, fromUserID)
if err != nil {
return err
}

// 入账
_, err = tx.Exec(ctx, `
UPDATE accounts SET balance = balance + $1 WHERE user_id = $2
`, amount, toUserID)
if err != nil {
return err
}

return tx.Commit(ctx)
}

// JSONB数据操作
type UserProfile struct {
Age int `json:"age"`
City string `json:"city"`
Hobbies []string `json:"hobbies"`
Settings map[string]interface{} `json:"settings"`
}

func (s *PostgreSQLService) UpdateUserProfile(ctx context.Context, userID int64, profile *UserProfile) error {
_, err := s.pool.Exec(ctx, `
UPDATE users
SET profile_data = $1::jsonb
WHERE id = $2
`, profile, userID)

return err
}

func main() {
// 配置
config := &DBConfig{
Host: "localhost",
Port: 5432,
User: "app_user",
Password: "secure_password",
Database: "myapp",
MaxConns: 20,
}

// 初始化服务
service, err := NewPostgreSQLService(config)
if err != nil {
log.Fatalf("Failed to initialize database: %v", err)
}
defer service.Close()

ctx := context.Background()

// 创建用户
userID, err := service.CreateUser(ctx, "赵六", "zhaoliu@example.com")
if err != nil {
log.Fatalf("Failed to create user: %v", err)
}
fmt.Printf("Created user with ID: %d\n", userID)

// 获取用户
user, err := service.GetUser(ctx, userID)
if err != nil {
log.Fatalf("Failed to get user: %v", err)
}
if user != nil {
fmt.Printf("User found: %+v\n", user)
}

// 事务操作
err = service.TransferMoney(ctx, 1, 2, 150.00)
if err != nil {
log.Printf("Transfer failed: %v", err)
} else {
fmt.Println("Money transfer successful")
}

fmt.Println("Operation completed successfully")
}

5.3 Python操作PostgreSQL

5.3.1 依赖安装

1
pip install psycopg[binary,pool] SQLAlchemy

5.3.2 完整示例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
import json
import logging
from contextlib import contextmanager
from datetime import datetime, timedelta
from typing import List, Dict, Optional, Any, Tuple

import psycopg
from psycopg import sql
from psycopg.pool import ConnectionPool

# 配置日志
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
logger = logging.getLogger('postgres_demo')

class PostgreSQLService:
def __init__(self, config: dict):
"""
初始化PostgreSQL连接池

:param config: 数据库配置字典
"""
self.config = config
self.pool = None
self._initialize_pool()

def _initialize_pool(self):
"""初始化连接池"""
try:
conninfo = f"host={self.config['host']} port={self.config['port']} " \
f"dbname={self.config['dbname']} user={self.config['user']} " \
f"password={self.config['password']} sslmode={self.config.get('sslmode', 'prefer')}"

self.pool = ConnectionPool(
conninfo=conninfo,
min_size=self.config.get('min_size', 5),
max_size=self.config.get('max_size', 20),
open=True,
configure=self._configure_connection
)
logger.info("Database connection pool initialized successfully")
except Exception as e:
logger.error(f"Failed to initialize connection pool: {e}")
raise

def _configure_connection(self, conn: psycopg.Connection):
"""配置连接"""
with conn.cursor() as cur:
# 设置时区
cur.execute("SET TIME ZONE 'UTC'")
# 设置客户端编码
cur.execute("SET CLIENT_ENCODING TO 'UTF8'")

@contextmanager
def get_connection(self):
"""获取连接上下文管理器"""
if not self.pool:
raise Exception("Connection pool not initialized")

conn = self.pool.getconn()
try:
yield conn
conn.commit()
except Exception as e:
conn.rollback()
logger.error(f"Database operation failed: {e}")
raise
finally:
self.pool.putconn(conn)

def create_user(self, name: str, email: str) -> int:
"""创建用户"""
query = """
INSERT INTO users (name, email, created_at, is_active)
VALUES (%s, %s, NOW(), true)
RETURNING id
"""

try:
with self.get_connection() as conn:
with conn.cursor() as cur:
cur.execute(query, (name, email))
user_id = cur.fetchone()[0]
logger.info(f"Created user {name} with ID {user_id}")
return user_id
except Exception as e:
logger.error(f"Failed to create user: {e}")
raise

def get_user(self, user_id: int) -> Optional[Dict[str, Any]]:
"""获取用户信息"""
query = """
SELECT id, name, email, created_at, is_active
FROM users
WHERE id = %s AND is_active = true
"""

try:
with self.get_connection() as conn:
with conn.cursor(row_factory=psycopg.rows.dict_row) as cur:
cur.execute(query, (user_id,))
return cur.fetchone()
except Exception as e:
logger.error(f"Failed to get user: {e}")
raise

def get_user_list(self, page: int = 1, page_size: int = 20) -> List[Dict[str, Any]]:
"""获取用户列表(分页)"""
offset = (page - 1) * page_size
query = """
SELECT id, name, email, created_at, is_active
FROM users
WHERE is_active = true
ORDER BY created_at DESC
LIMIT %s OFFSET %s
"""

try:
with self.get_connection() as conn:
with conn.cursor(row_factory=psycopg.rows.dict_row) as cur:
cur.execute(query, (page_size, offset))
return cur.fetchall()
except Exception as e:
logger.error(f"Failed to get user list: {e}")
raise

def transfer_money(self, from_user_id: int, to_user_id: int, amount: float) -> bool:
"""转账操作(事务)"""
try:
with self.get_connection() as conn:
with conn.transaction():
# 检查余额
check_query = """
SELECT balance FROM accounts WHERE user_id = %s FOR UPDATE
"""
with conn.cursor() as cur:
cur.execute(check_query, (from_user_id,))
result = cur.fetchone()

if not result:
raise ValueError(f"Account not found for user {from_user_id}")

balance = result[0]
if balance < amount:
raise ValueError("Insufficient balance")

# 扣款
debit_query = """
UPDATE accounts SET balance = balance - %s WHERE user_id = %s
"""
cur.execute(debit_query, (amount, from_user_id))

# 入账
credit_query = """
UPDATE accounts SET balance = balance + %s WHERE user_id = %s
"""
cur.execute(credit_query, (amount, to_user_id))

# 记录交易日志
log_query = """
INSERT INTO transactions (from_user_id, to_user_id, amount, created_at)
VALUES (%s, %s, %s, NOW())
"""
cur.execute(log_query, (from_user_id, to_user_id, amount))

logger.info(f"Transferred {amount} from user {from_user_id} to user {to_user_id}")
return True

except Exception as e:
logger.error(f"Transfer failed: {e}")
raise

def update_user_profile(self, user_id: int, profile_data: Dict[str, Any]) -> bool:
"""更新用户JSONB资料"""
query = """
UPDATE users
SET profile_data = %s::jsonb
WHERE id = %s
"""

try:
json_data = json.dumps(profile_data, ensure_ascii=False)
with self.get_connection() as conn:
with conn.cursor() as cur:
cur.execute(query, (json_data, user_id))
return cur.rowcount > 0
except Exception as e:
logger.error(f"Failed to update user profile: {e}")
raise

def search_users_by_profile(self, criteria: Dict[str, Any]) -> List[Dict[str, Any]]:
"""根据JSONB资料搜索用户"""
# 构建动态查询条件
conditions = []
params = []

for key, value in criteria.items():
if isinstance(value, str):
condition = sql.SQL("profile_data->>%s ILIKE %s")
params.extend([key, f"%{value}%"])
else:
condition = sql.SQL("profile_data->>%s = %s")
params.extend([key, str(value)])
conditions.append(condition)

where_clause = sql.SQL(" AND ").join(conditions) if conditions else sql.SQL("true")

query = sql.SQL("""
SELECT id, name, email, profile_data
FROM users
WHERE is_active = true AND {}
ORDER BY created_at DESC
""").format(where_clause)

try:
with self.get_connection() as conn:
with conn.cursor(row_factory=psycopg.rows.dict_row) as cur:
cur.execute(query, params)
return cur.fetchall()
except Exception as e:
logger.error(f"Failed to search users by profile: {e}")
raise

def close(self):
"""关闭连接池"""
if self.pool:
self.pool.close()
logger.info("Database connection pool closed")

# 使用示例
if __name__ == "__main__":
# 配置
config = {
'host': 'localhost',
'port': 5432,
'dbname': 'myapp',
'user': 'app_user',
'password': 'secure_password',
'min_size': 5,
'max_size': 20
}

try:
# 初始化服务
db_service = PostgreSQLService(config)

# 创建用户
user_id = db_service.create_user("孙七", "sunqi@example.com")
print(f"Created user with ID: {user_id}")

# 获取用户
user = db_service.get_user(user_id)
if user:
print(f"User found: {user}")

# 更新用户资料
profile = {
'age': 28,
'city': '上海',
'hobbies': ['reading', 'coding', 'travel'],
'settings': {'theme': 'dark', 'notifications': True}
}
db_service.update_user_profile(user_id, profile)
print("User profile updated successfully")

# 搜索用户
results = db_service.search_users_by_profile({'city': '上海'})
print(f"Found {len(results)} users in Shanghai")

# 事务操作
try:
db_service.transfer_money(1, 2, 200.00)
print("Money transfer successful")
except Exception as e:
print(f"Transfer failed: {e}")

except Exception as e:
logger.error(f"Application error: {e}")
finally:
if 'db_service' in locals():
db_service.close()

第六部分:运维与优化最佳实践

6.1 性能调优关键点

6.1.1 查询优化

1
2
3
4
5
6
7
8
9
10
11
12
-- 使用EXPLAIN ANALYZE分析查询
EXPLAIN ANALYZE SELECT * FROM orders
WHERE customer_id = 123
ORDER BY order_date DESC
LIMIT 10;

-- 创建合适的索引
CREATE INDEX CONCURRENTLY idx_orders_customer_date ON orders (customer_id, order_date DESC);
CREATE INDEX CONCURRENTLY idx_orders_status ON orders (status) WHERE status IN ('pending', 'processing');

-- 避免SELECT *,只选择需要的字段
SELECT id, order_date, total_amount FROM orders WHERE customer_id = 123;

6.1.2 配置优化监控

1
2
3
4
5
6
7
8
9
10
11
12
13
-- 监控配置更改效果
SELECT name, setting, unit, category, short_desc
FROM pg_settings
WHERE name IN (
'shared_buffers', 'work_mem', 'effective_cache_size',
'maintenance_work_mem', 'max_connections'
);

-- 监控活跃会话
SELECT pid, usename, application_name, client_addr, state, query, query_start
FROM pg_stat_activity
WHERE state != 'idle'
ORDER BY query_start;

6.2 备份恢复策略

6.2.1 物理备份(基础备份)

1
2
3
4
5
6
7
8
# 全量备份
sudo -u postgres pg_basebackup -h localhost -U backup_user -D /backup/full_backup -Ft -z -P -X stream

# 增量备份(WAL归档)
# 配置postgresql.conf:
wal_level = replica
archive_mode = on
archive_command = 'test ! -f /backup/wal/%f && cp %p /backup/wal/%f'

6.2.2 逻辑备份(pg_dump)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# 全库备份
pg_dump -h localhost -U backup_user -Fc myapp > /backup/myapp_$(date +%Y%m%d).dump

# 单表备份
pg_dump -h localhost -U backup_user -t users myapp > /backup/users_$(date +%Y%m%d).sql

# 定时备份脚本
#!/bin/bash
BACKUP_DIR="/backup/daily"
DATE=$(date +%Y%m%d_%H%M%S)
DB_NAME="myapp"
USER="backup_user"

mkdir -p $BACKUP_DIR
pg_dump -h localhost -U $USER -Fc $DB_NAME > $BACKUP_DIR/${DB_NAME}_${DATE}.dump
find $BACKUP_DIR -name "*.dump" -mtime +7 -delete

6.3 安全配置注意事项

6.3.1 关键安全配置

1
2
3
4
5
6
7
8
# 修改pg_hba.conf,限制访问
# TYPE DATABASE USER ADDRESS METHOD
host all app_user 192.168.1.0/24 scram-sha-256
host replication replicator 192.168.1.100/32 scram-sha-256
local all postgres peer

# 重载配置
sudo systemctl reload postgresql@16-main

6.3.2 安全审计脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
-- 创建审计日志表
CREATE TABLE audit_log (
id SERIAL PRIMARY KEY,
event_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
user_name VARCHAR(100),
client_addr INET,
query_text TEXT,
action VARCHAR(50)
);

-- 创建审计函数
CREATE OR REPLACE FUNCTION log_audit_event() RETURNS TRIGGER AS $$
BEGIN
INSERT INTO audit_log (user_name, client_addr, query_text, action)
VALUES (
current_user,
inet_client_addr(),
current_query(),
TG_OP
);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;

-- 为重要表创建审计触发器
CREATE TRIGGER audit_users_trigger
AFTER INSERT OR UPDATE OR DELETE ON users
FOR EACH ROW EXECUTE FUNCTION log_audit_event();

核心安全原则

  • 最小权限原则:每个应用用户只拥有必要的权限
  • 网络隔离:数据库服务器不应直接暴露在公网
  • 定期审计:检查用户权限、访问日志、敏感操作
  • 加密传输:启用SSL连接,保护数据在传输过程中的安全
  • 定期更新:及时应用安全补丁,关注CVE公告

基于Debian 12的PostgreSQL源码编译一键部署脚本(注意校验源码和版本⚠️)

基于Debian 12的PostgreSQL源码编译一键部署脚本,该脚本会自动安装依赖、编译源码、配置服务,并生成关键配置信息到config.md文件:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
#!/bin/bash
# PostgreSQL 16 Source Compilation and Deployment Script for Debian 12
# Date: 2026-01-03

set -e

# Configuration variables
PG_VERSION="16.2"
PG_USER="postgres"
PG_GROUP="postgres"
PG_HOME="/usr/local/pgsql"
PG_DATA="/var/lib/postgresql/${PG_VERSION}/data"
PG_LOG="/var/log/postgresql"
PG_PORT="5432"
SYSTEMD_SERVICE_FILE="/etc/systemd/system/postgresql.service"

echo "=== Starting PostgreSQL ${PG_VERSION} Source Compilation and Deployment ==="
echo "System information:"
echo "OS: $(lsb_release -sd)"
echo "Kernel: $(uname -r)"
echo "Architecture: $(uname -m)"
echo ""

# Step 1: Install required dependencies
echo "=== Step 1: Installing required dependencies ==="
sudo apt update
sudo apt install -y \
build-essential \
liblz4-dev \
lz4 \
pkg-config \
libreadline-dev \
zlib1g-dev \
libxml2-dev \
libxml2 \
libssh-dev \
uuid-dev \
libssl-dev \
libpam0g-dev \
libicu-dev \
libsystemd-dev \
libkrb5-dev \
libselinux1-dev \
libxslt1-dev \
libperl-dev \
python3-dev \
tcl-dev \
flex \
bison \
systemd \
curl \
wget \
git \
ca-certificates

# Step 2: Create PostgreSQL user and directories
echo "=== Step 2: Creating PostgreSQL user and directories ==="
if ! id -u "$PG_USER" >/dev/null 2>&1; then
sudo useradd -r -s /bin/bash -d "$PG_HOME" -U "$PG_USER"
echo "Created user: $PG_USER"
fi

sudo mkdir -p "$PG_DATA"
sudo mkdir -p "$PG_LOG"
sudo chown -R "$PG_USER:$PG_GROUP" "$PG_DATA" "$PG_LOG" "$PG_HOME"
sudo chmod 0700 "$PG_DATA"

# Step 3: Download and extract PostgreSQL source code
echo "=== Step 3: Downloading and extracting PostgreSQL source code ==="
cd /tmp
SOURCE_FILE="postgresql-${PG_VERSION}.tar.gz"
if [ ! -f "$SOURCE_FILE" ]; then
wget "https://ftp.postgresql.org/pub/source/v${PG_VERSION}/postgresql-${PG_VERSION}.tar.gz"
fi

tar -xzf "$SOURCE_FILE"
cd "postgresql-${PG_VERSION}"

# Step 4: Configure compilation options
echo "=== Step 4: Configuring compilation options ==="
./configure \
--prefix="$PG_HOME" \
--with-pgport="$PG_PORT" \
--with-systemd \
--with-system-tzdata=/usr/share/zoneinfo \
--with-openssl \
--with-pam \
--with-ldap \
--with-uuid=ossp \
--with-libxml \
--with-libxslt \
--enable-thread-safety \
--enable-debug \
--enable-cassert \
--with-icu \
--with-perl \
--with-python \
--with-tcl

echo "Configuration completed successfully."

# Step 5: Compile and install
echo "=== Step 5: Compiling and installing PostgreSQL ==="
make -j$(nproc) world
sudo make install-world

echo "Installation completed successfully."

# Step 6: Initialize database cluster
echo "=== Step 6: Initializing database cluster ==="
sudo -u "$PG_USER" "$PG_HOME/bin/initdb" -D "$PG_DATA" -E UTF8 --locale=en_US.UTF-8

# Step 7: Configure PostgreSQL settings
echo "=== Step 7: Configuring PostgreSQL settings ==="

# Backup original config files
sudo -u "$PG_USER" cp "$PG_DATA/postgresql.conf" "$PG_DATA/postgresql.conf.bak"
sudo -u "$PG_USER" cp "$PG_DATA/pg_hba.conf" "$PG_DATA/pg_hba.conf.bak"

# Configure postgresql.conf
sudo -u "$PG_USER" tee "$PG_DATA/postgresql.conf" > /dev/null <<EOF
# PostgreSQL configuration file
# Generated by installation script on $(date)

# Connection Settings
listen_addresses = '*'
port = $PG_PORT
max_connections = 100

# Memory Settings
shared_buffers = 128MB
effective_cache_size = 4096MB
work_mem = 4MB
maintenance_work_mem = 64MB

# WAL Settings
wal_level = replica
synchronous_commit = on
wal_log_hints = on
max_wal_senders = 10
wal_keep_size = 1GB

# Query Planning
random_page_cost = 1.1
effective_io_concurrency = 200

# Error Reporting and Logging
log_destination = 'stderr'
logging_collector = on
log_directory = '$PG_LOG'
log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log'
log_file_mode = 0640
log_truncate_on_rotation = on
log_rotation_age = 1d
log_rotation_size = 100MB
log_min_duration_statement = 1000
log_checkpoints = on
log_connections = on
log_disconnections = on
log_lock_waits = on
log_temp_files = -1
log_autovacuum_min_duration = 0

# Runtime Statistics
track_activities = on
track_counts = on
track_io_timing = on
track_functions = all
stats_temp_directory = '/var/run/postgresql'

# Autovacuum
autovacuum = on
log_autovacuum_min_duration = 0

# Client Connection Defaults
timezone = 'UTC'
datestyle = 'iso, mdy'
lc_messages = 'en_US.UTF-8'
lc_monetary = 'en_US.UTF-8'
lc_numeric = 'en_US.UTF-8'
lc_time = 'en_US.UTF-8'
default_text_search_config = 'pg_catalog.english'
EOF

# Configure pg_hba.conf
sudo -u "$PG_USER" tee "$PG_DATA/pg_hba.conf" > /dev/null <<EOF
# PostgreSQL Client Authentication Configuration File
# Generated by installation script on $(date)

# TYPE DATABASE USER ADDRESS METHOD

# "local" is for Unix domain socket connections only
local all all peer

# IPv4 local connections:
host all all 127.0.0.1/32 scram-sha-256
host all all 192.168.0.0/16 scram-sha-256

# IPv6 local connections:
host all all ::1/128 scram-sha-256

# Allow replication connections from localhost
local replication all peer
host replication all 127.0.0.1/32 scram-sha-256
host replication all ::1/128 scram-sha-256
EOF

# Step 8: Create systemd service file
echo "=== Step 8: Creating systemd service file ==="
sudo tee "$SYSTEMD_SERVICE_FILE" > /dev/null <<EOF
[Unit]
Description=PostgreSQL database server
Documentation=man:postgres(1)
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
User=$PG_USER
Group=$PG_GROUP
RuntimeDirectory=postgresql
RuntimeDirectoryMode=755
Environment=PGDATA=$PG_DATA
Environment=PGLOG=$PG_LOG
ExecStart=$PG_HOME/bin/postgres -D $PG_DATA -c config_file=$PG_DATA/postgresql.conf
ExecReload=/bin/kill -HUP \$MAINPID
KillMode=mixed
KillSignal=SIGINT
TimeoutSec=120
Restart=always
RestartSec=30

[Install]
WantedBy=multi-user.target
EOF

# Step 9: Set proper permissions
echo "=== Step 9: Setting proper permissions ==="
sudo chown -R "$PG_USER:$PG_GROUP" "$PG_HOME"
sudo chown -R "$PG_USER:$PG_GROUP" "$PG_DATA"
sudo chown -R "$PG_USER:$PG_GROUP" "$PG_LOG"
sudo chmod 0700 "$PG_DATA"
sudo chmod 0755 "$PG_LOG"

# Step 10: Start PostgreSQL service
echo "=== Step 10: Starting PostgreSQL service ==="
sudo systemctl daemon-reload
sudo systemctl enable postgresql
sudo systemctl start postgresql

# Wait for service to start
sleep 5

# Step 11: Verify installation
echo "=== Step 11: Verifying installation ==="
sudo -u "$PG_USER" "$PG_HOME/bin/psql" -c "SELECT version();" postgres
sudo -u "$PG_USER" "$PG_HOME/bin/pg_isready" -p "$PG_PORT"

# Step 12: Generate config.md file with key configuration information
echo "=== Step 12: Generating config.md file ==="
cat > config.md <<EOF
# PostgreSQL ${PG_VERSION} Configuration Summary

## System Information
- **OS**: $(lsb_release -sd)
- **Kernel**: $(uname -r)
- **Architecture**: $(uname -m)
- **Installation Date**: $(date)
- **Installation Method**: Source Compilation

## PostgreSQL Installation Details
- **Version**: ${PG_VERSION}
- **Installation Path**: $PG_HOME
- **Data Directory**: $PG_DATA
- **Log Directory**: $PG_LOG
- **Port**: $PG_PORT
- **Service User**: $PG_USER
- **Service Group**: $PG_GROUP

## Key Configuration Files
- **postgresql.conf**: $PG_DATA/postgresql.conf
- **pg_hba.conf**: $PG_DATA/pg_hba.conf
- **systemd Service**: $SYSTEMD_SERVICE_FILE

## Important Configuration Parameters

### Connection Settings
- **listen_addresses**: '*'
- **port**: $PG_PORT
- **max_connections**: 100

### Memory Settings
- **shared_buffers**: 128MB
- **effective_cache_size**: 4096MB
- **work_mem**: 4MB
- **maintenance_work_mem**: 64MB

### WAL Settings
- **wal_level**: replica
- **wal_keep_size**: 1GB
- **max_wal_senders**: 10

### Security Settings
- **Authentication Method**: scram-sha-256 (for remote connections)
- **Local Authentication**: peer
- **SSL**: Enabled

### Logging Settings
- **Log Directory**: $PG_LOG
- **Log Rotation**: Daily
- **Log File Size**: 100MB
- **Slow Query Log**: >1000ms

## Service Management Commands
- **Start Service**: \`sudo systemctl start postgresql\`
- **Stop Service**: \`sudo systemctl stop postgresql\`
- **Restart Service**: \`sudo systemctl restart postgresql\`
- **Check Status**: \`sudo systemctl status postgresql\`
- **Enable on Boot**: \`sudo systemctl enable postgresql\`

## Database Access
- **Local Access**: \`sudo -u $PG_USER $PG_HOME/bin/psql\`
- **Remote Access**: Configure pg_hba.conf and firewall rules

## Important Directories
- **Binaries**: $PG_HOME/bin/
- **Data Files**: $PG_DATA/
- **Logs**: $PG_LOG/
- **Configuration**: $PG_DATA/

## Backup and Recovery
- **Base Backup Command**: \`$PG_HOME/bin/pg_basebackup -D /backup/path -Ft -z -P -X stream\`
- **WAL Archive Command**: Configure in postgresql.conf

## Performance Tuning Notes
1. **shared_buffers**: Should be 25% of total RAM for dedicated database servers
2. **work_mem**: Increase for complex queries with sorts and joins
3. **maintenance_work_mem**: Increase for faster index creation and vacuum operations
4. **effective_cache_size**: Should be 50-75% of total system memory

## Security Recommendations
1. **Firewall**: Restrict access to PostgreSQL port (5432) to trusted networks
2. **SSL**: Enable SSL connections for remote clients
3. **Password Policy**: Enforce strong passwords for database users
4. **Regular Updates**: Keep PostgreSQL updated with security patches
5. **Audit Logging**: Enable detailed logging for security monitoring

## Monitoring Commands
- **Check Connections**: \`$PG_HOME/bin/psql -c "SELECT * FROM pg_stat_activity;"\`
- **Check Locks**: \`$PG_HOME/bin/psql -c "SELECT * FROM pg_locks;"\`
- **Check Replication**: \`$PG_HOME/bin/psql -c "SELECT * FROM pg_stat_replication;"\`
- **Check Table Sizes**: \`$PG_HOME/bin/psql -c "SELECT schemaname, tablename, pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) FROM pg_tables ORDER BY pg_total_relation_size DESC LIMIT 10;"\`

## Troubleshooting
- **Service Logs**: \`journalctl -u postgresql -f\`
- **PostgreSQL Logs**: \`tail -f $PG_LOG/postgresql-$(date +'%Y-%m-%d')_*.log\`
- **Check Service Status**: \`systemctl status postgresql\`

## Notes
- This configuration is optimized for a development/medium production environment
- For production environments, adjust memory settings based on available system resources
- Always test configuration changes in a staging environment before applying to production
- Regular backups are essential - implement a backup strategy using pg_dump or pg_basebackup
EOF

echo "=== Installation completed successfully! ==="
echo ""
echo "Key configuration information has been saved to config.md"
echo ""
echo "To verify the installation, run:"
echo " sudo -u $PG_USER $PG_HOME/bin/psql -c \"SELECT version();\""
echo ""
echo "To manage the service:"
echo " sudo systemctl status postgresql"
echo ""
echo "Configuration files:"
echo " postgresql.conf: $PG_DATA/postgresql.conf"
echo " pg_hba.conf: $PG_DATA/pg_hba.conf"
echo " systemd service: $SYSTEMD_SERVICE_FILE"
echo ""
echo "Log files are stored in: $PG_LOG"

这个脚本提供了完整的PostgreSQL 16源码编译部署流程,包括:

  1. 依赖安装:安装所有必需的编译依赖包
  2. 用户和目录创建:创建专用的postgres用户和必要的目录结构
  3. 源码下载和编译:下载官方源码并进行配置、编译和安装
  4. 数据库初始化:初始化数据库集群
  5. 配置文件设置:配置postgresql.conf和pg_hba.conf文件
  6. systemd服务配置:创建systemd服务文件以便于服务管理
  7. 服务启动和验证:启动PostgreSQL服务并验证安装
  8. 配置信息生成:生成详细的config.md文件,包含所有关键配置信息

脚本执行完成后,config.md文件将包含:

  • 系统和安装信息
  • 关键配置文件路径
  • 重要配置参数详解
  • 服务管理命令
  • 数据库访问方法
  • 备份恢复策略
  • 性能调优建议
  • 安全配置推荐
  • 监控和故障排除命令

这个脚本适用于Debian 12系统,可以作为生产环境部署的基础,用户可以根据具体需求调整配置参数。

⚠️ 注意此脚本时效版本以官方更新为主,可适时调整。


总结

本文从PostgreSQL的版本历史出发,深入探讨了在Debian 12环境下的完整实践路径,从基础安装配置到高级集群部署,再到多语言应用开发,形成了一个完整的知识体系。PostgreSQL的强大之处不仅在于其丰富的功能特性,更在于其稳定性和可扩展性。

关键实践建议

  1. 循序渐进:从单体部署开始,逐步过渡到主从复制,最后考虑集群方案
  2. 监控先行:在生产环境部署前,确保监控体系完善
  3. 备份为王:无论架构多么复杂,可靠的备份策略是最后的安全保障
  4. 安全第一:从设计阶段就考虑安全因素,而不是事后补救
  5. 持续学习:PostgreSQL社区活跃,新版本不断带来性能改进和新特性

随着云原生技术和AI技术的发展,PostgreSQL在容器化、Serverless等场景下的应用将更加广泛,且大有成为AI基础设施的趋势。掌握PostgreSQL的核心原理和最佳实践,不仅能提升应用性能和可靠性,更能为技术架构的演进提供坚实基础。

PostgreSQL深度实践:从Debian 12入门到集群部署及多语言应用

https://www.wdft.com/2d0307b0.html

Author

Jaco Liu

Posted on

2025-12-17

Updated on

2026-01-21

Licensed under