Redis 持久化:RDB vs AOF vs 混合持久化
一句话结论(30s)
Redis 持久化的本质是在「恢复速度」与「数据安全」之间做取舍:RDB 二进制快照恢复最快但窗口期全丢,AOF 写后日志最安全但逐条回放慢。关键设计:因为 bgrewriteaof 子进程重写期间父进程仍会产生新写入,Redis 用「双缓冲区」——父进程既继续写旧 AOF 兜底、又写 aof_rewrite_buf 收集增量,子进程完成后父进程把增量追加到新文件再原子 rename。权衡:4.0 起的混合持久化(RDB 全量快照 + AOF 增量命令)把恢复性能从分钟级降到秒级、同时保留 AOF 级的数据安全,是生产推荐方案。
核心原理(2min)
RDB 通过 bgsave fork 子进程,利用 COW 共享内存生成二进制快照,完成后 rename 原子替换;AOF 记录每条写命令,按 appendfsync(always/everysec/no)刷盘,everysec 最多丢 1 秒。AOF 膨胀后用 bgrewriteaof 生成最小化文件:子进程按 fork 时快照重写,期间父进程的新写同时进入旧 AOF 和 aof_rewrite_buf,子进程完成后父进程把重写缓冲区追加到新文件再 rename。大实例 fork 本身拷贝页表可能阻塞 200-500ms,需关闭 THP 避免 COW 粒度从 4KB 放大到 2MB。混合持久化文件 = RDB 全量快照 + AOF 增量命令,加载先快后少。
底层深入(5-10min)
RDB:快照式持久化
Redis 执行 bgsave,fork 出一个子进程。子进程利用 COW(写时复制)共享父进程的全部内存数据,将内存快照序列化为一个二进制的 RDB 文件。
bgsave 流程:
1. Redis 调用 fork()
2. 子进程: 遍历所有 key,序列化为二进制 → 写入临时 RDB 文件
3. 父进程: 继续处理客户端请求,修改内存数据触发 COW 复制新页
4. 子进程写完 → rename(临时文件, dump.rdb) 原子替换
RDB 的 fork 入口 —— rdbSaveBackground:
int rdbSaveBackground(int req, char *filename, rdbSaveInfo *rsi, int rdbflags) {
pid_t childpid;
if (hasActiveChildProcess()) return C_ERR;
server.stat_rdb_saves++;
server.dirty_before_bgsave = server.dirty;
server.lastbgsave_try = time(NULL);
if ((childpid = redisFork(CHILD_TYPE_RDB)) == 0) {
int retval;
/* Child */
redisSetProcTitle("redis-rdb-bgsave");
redisSetCpuAffinity(server.bgsave_cpulist);
retval = rdbSave(req, filename,rsi,rdbflags);
if (retval == C_OK) {
sendChildCowInfo(CHILD_INFO_TYPE_RDB_COW_SIZE, "RDB");
}
exitFromChild((retval == C_OK) ? 0 : 1, 0);
} else {
/* Parent */
if (childpid == -1) {
server.lastbgsave_status = C_ERR;
serverLog(LL_WARNING,"Can't save in background: fork: %s",
strerror(errno));
return C_ERR;
}
serverLog(LL_NOTICE,"Background saving started by pid %ld",(long) childpid);
server.rdb_save_time_start = time(NULL);
server.rdb_child_type = RDB_CHILD_TYPE_DISK;
return C_OK;
}
return C_OK; /* unreached */
}
bgsave 的真正入口是 rdbSaveBackground:先用 hasActiveChildProcess() 保证同一时刻只 fork 一个子进程(避免多个子进程同时复制内存),再 redisFork() 复制一份当前进程。fork 返回 0 的是子进程,它直接调 rdbSave() 把数据落盘后 exitFromChild() 退出;返回子进程 pid 的是父进程,记个日志立刻返回继续服务客户端。父进程全程不阻塞,落盘都在子进程里做,这就是「后台保存」。
子进程落盘 —— rdbSave(临时文件 + 原子 rename):
int rdbSave(int req, char *filename, rdbSaveInfo *rsi, int rdbflags) {
char tmpfile[256];
char cwd[MAXPATHLEN]; /* Current working dir path for error messages. */
startSaving(rdbflags);
snprintf(tmpfile,256,"temp-%d.rdb", (int) getpid());
if (rdbSaveInternal(req,tmpfile,rsi,rdbflags) != C_OK) {
stopSaving(0);
return C_ERR;
}
/* Use RENAME to make sure the DB file is changed atomically only
* if the generate DB file is ok. */
if (rename(tmpfile,filename) == -1) {
char *str_err = strerror(errno);
char *cwdp = getcwd(cwd,MAXPATHLEN);
serverLog(LL_WARNING,
"Error moving temp DB file %s on the final "
"destination %s (in server root dir %s): %s",
tmpfile,
filename,
cwdp ? cwdp : "unknown",
str_err);
unlink(tmpfile);
stopSaving(0);
return C_ERR;
}
if (fsyncFileDir(filename) != 0) {
serverLog(LL_WARNING,
"Failed to fsync directory while saving DB: %s", strerror(errno));
stopSaving(0);
return C_ERR;
}
serverLog(LL_NOTICE,"DB saved on disk");
server.dirty = 0;
server.lastsave = time(NULL);
server.lastbgsave_status = C_OK;
stopSaving(1);
return C_OK;
}
子进程把快照写进临时文件 temp-<pid>.rdb,全部写成功后才 rename() 原子替换成正式的 dump.rdb,随后 fsyncFileDir() 刷一下目录项防止 rename 本身在断电时丢失。写临时文件再改名,保证 dump.rdb 要么是完整旧版本、要么是完整新版本,绝不会出现写一半的脏文件。这也是 RDB 恢复「快」的来源——直接反序列化二进制,不用逐条回放命令。
快照序列化 —— rdbSaveRio(COW 读内存):
int rdbSaveRio(int req, rio *rdb, int *error, int rdbflags, rdbSaveInfo *rsi) {
char magic[10];
uint64_t cksum;
long key_counter = 0;
unsigned long long skipped = 0;
int j;
if (server.rdb_checksum)
rdb->update_cksum = rioGenericUpdateChecksum;
snprintf(magic,sizeof(magic),"REDIS%04d",RDB_VERSION);
if (rdbWriteRaw(rdb,magic,9) == -1) goto werr;
if (rdbSaveInfoAuxFields(rdb,rdbflags,rsi) == -1) goto werr;
if (!(req & SLAVE_REQ_RDB_EXCLUDE_DATA) && rdbSaveModulesAux(rdb, REDISMODULE_AUX_BEFORE_RDB) == -1) goto werr;
/* save functions */
if (!(req & SLAVE_REQ_RDB_EXCLUDE_FUNCTIONS) && rdbSaveFunctions(rdb) == -1) goto werr;
/* Save the hash template registry (id -> field names). Template encoded
* hashes are then written in compact REF form (id + values) by rdbSaveKeyValuePair. */
if (!(req & SLAVE_REQ_RDB_EXCLUDE_DATA)) {
if (rdbSaveHashTemplates(rdb) == -1) goto werr;
}
/* save all databases, skip this if we're in functions-only mode */
if (!(req & SLAVE_REQ_RDB_EXCLUDE_DATA)) {
for (j = 0; j < server.dbnum; j++) {
if (rdbSaveDb(rdb, j, rdbflags, &key_counter, &skipped) == -1) goto werr;
/* In standalone mode, dismiss bucket arrays of the saved DB's
* kvstore to reduce CoW. In cluster mode this is done per-slot. */
if (server.in_fork_child && !server.cluster_enabled)
dismissKvstoreBucketsMemory(server.db[j].keys);
}
}
if (!(req & SLAVE_REQ_RDB_EXCLUDE_DATA) && rdbSaveModulesAux(rdb, REDISMODULE_AUX_AFTER_RDB) == -1) goto werr;
/* EOF opcode */
if (rdbSaveType(rdb,RDB_OPCODE_EOF) == -1) goto werr;
/* CRC64 checksum. It will be zero if checksum computation is disabled, the
* loading code skips the check in this case. */
cksum = rdb->cksum;
memrev64ifbe(&cksum);
if (rioWrite(rdb,&cksum,8) == 0) goto werr;
serverLog(LL_NOTICE, "BGSAVE done, %ld keys saved, %llu keys skipped, %zu bytes written.", key_counter, skipped, rdb->processed_bytes);
return C_OK;
werr:
if (error) *error = errno;
return C_ERR;
}
rdbSaveRio 才是真正的快照序列化:先写 REDIS 魔数 + 版本号,再写辅助字段、函数和 hash 模板,然后 for 循环遍历所有 db 调 rdbSaveDb() 把每个 key/value 变成二进制,最后写 EOF 操作码和 CRC64 校验和。整个过程跑在子进程里,靠 COW 与父进程共享内存页——父进程此刻的写入只会复制被改动的那几页,子进程读到的始终是 fork 瞬间的静态快照。代码里 dismissKvstoreBucketsMemory() 就是为了减少 COW 复制量做的优化。
RDB 的优点:二进制紧凑,恢复速度极快(直接反序列化到内存),适合冷备份和主从全量同步。
RDB 的缺点:快照间隔期间的写入在宕机时全丢。两次 bgsave 之间的窗口期(如半小时)内所有写入都消失。
AOF:写后日志
AOF 记录每一条修改 Redis 的写命令。先执行命令,再写入 AOF 缓冲区,最后刷到磁盘:
AOF 流程:
客户端: SET key value
Redis: 执行命令 → 写入 aof_buf
→ 根据 appendfsync 策略刷盘:
always: 每条命令都 fsync — 最安全,性能最差
everysec: 每秒 fsync — 丢最多 1 秒数据,性能折中
no: 交给 OS — 最快,丢多少看 OS 心情
命令追加 —— feedAppendOnlyFile:
void feedAppendOnlyFile(int dictid, robj **argv, int argc) {
sds buf = sdsempty();
serverAssert(dictid == -1 || (dictid >= 0 && dictid < server.dbnum));
/* Feed timestamp if needed */
if (server.aof_timestamp_enabled) {
sds ts = genAofTimestampAnnotationIfNeeded(0);
if (ts != NULL) {
buf = sdscatsds(buf, ts);
sdsfree(ts);
}
}
/* The DB this command was targeting is not the same as the last command
* we appended. To issue a SELECT command is needed. */
if (dictid != -1 && dictid != server.aof_selected_db) {
char seldb[64];
snprintf(seldb,sizeof(seldb),"%d",dictid);
buf = sdscatprintf(buf,"*2\r\n$6\r\nSELECT\r\n$%lu\r\n%s\r\n",
(unsigned long)strlen(seldb),seldb);
server.aof_selected_db = dictid;
}
/* All commands should be propagated the same way in AOF as in replication.
* No need for AOF-specific translation. */
buf = catAppendOnlyGenericCommand(buf,argc,argv);
/* Append to the AOF buffer. This will be flushed on disk just before
* of re-entering the event loop, so before the client will get a
* positive reply about the operation performed. */
if (server.aof_state == AOF_ON ||
(server.aof_state == AOF_WAIT_REWRITE && server.child_type == CHILD_TYPE_AOF))
{
server.aof_buf = sdscatlen(server.aof_buf, buf, sdslen(buf));
}
sdsfree(buf);
}
每条写命令执行完都会走到 feedAppendOnlyFile:先按需补一条 SELECT 切换 db,再用 catAppendOnlyGenericCommand 把命令编码成 RESP 协议文本,最后追加到内存缓冲区 server.aof_buf。注意这里只写内存,注释写得很清楚——真正的磁盘写入要等「重新进入事件循环前」由 flushAppendOnlyFile 统一刷。这就是写后日志:命令先执行成功、再记到 AOF,与 WAL(先记日志再执行)正好相反。
刷盘策略 —— flushAppendOnlyFile(always / everysec / no):
void flushAppendOnlyFile(int force) {
ssize_t nwritten;
int sync_in_progress = 0;
mstime_t latency;
if (sdslen(server.aof_buf) == 0) {
if (server.aof_last_incr_fsync_offset == server.aof_last_incr_size) {
/* All data is fsync'd already: Update fsynced_reploff_pending just in case.
* This is needed to avoid a WAITAOF hang in case a module used RM_Call
* with the NO_AOF flag, in which case master_repl_offset will increase but
* fsynced_reploff_pending won't be updated (because there's no reason, from
* the AOF POV, to call fsync) and then WAITAOF may wait on the higher offset
* (which contains data that was only propagated to replicas, and not to AOF) */
if (!aofFsyncInProgress())
atomicSet(server.fsynced_reploff_pending, server.master_repl_offset);
} else {
/* Check if we need to do fsync even the aof buffer is empty,
* because previously in AOF_FSYNC_EVERYSEC mode, fsync is
* called only when aof buffer is not empty, so if users
* stop write commands before fsync called in one second,
* the data in page cache cannot be flushed in time. */
if (server.aof_fsync == AOF_FSYNC_EVERYSEC &&
server.mstime - server.aof_last_fsync >= 1000 &&
!(sync_in_progress = aofFsyncInProgress()))
goto try_fsync;
/* Check if we need to do fsync even the aof buffer is empty,
* the reason is described in the previous AOF_FSYNC_EVERYSEC block,
* and AOF_FSYNC_ALWAYS is also checked here to handle a case where
* aof_fsync is changed from everysec to always. */
if (server.aof_fsync == AOF_FSYNC_ALWAYS)
goto try_fsync;
}
return;
}
if (server.aof_fsync == AOF_FSYNC_EVERYSEC)
sync_in_progress = aofFsyncInProgress();
if (server.aof_fsync == AOF_FSYNC_EVERYSEC && !force) {
/* With this append fsync policy we do background fsyncing.
* If the fsync is still in progress we can try to delay
* the write for a couple of seconds. */
if (sync_in_progress) {
if (server.aof_flush_postponed_start == 0) {
/* No previous write postponing, remember that we are
* postponing the flush and return. */
server.aof_flush_postponed_start = server.mstime;
return;
} else if (server.mstime - server.aof_flush_postponed_start < 2000) {
/* We were already waiting for fsync to finish, but for less
* than two seconds this is still ok. Postpone again. */
return;
}
/* Otherwise fall through, and go write since we can't wait
* over two seconds. */
server.aof_delayed_fsync++;
serverLog(LL_NOTICE,"Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down Redis.");
}
}
/* We want to perform a single write. This should be guaranteed atomic
* at least if the filesystem we are writing is a real physical one.
* While this will save us against the server being killed I don't think
* there is much to do about the whole server stopping for power problems
* or alike */
if (server.aof_flush_sleep && sdslen(server.aof_buf)) {
usleep(server.aof_flush_sleep);
}
latencyStartMonitor(latency);
nwritten = aofWrite(server.aof_fd,server.aof_buf,sdslen(server.aof_buf));
latencyEndMonitor(latency);
/* We want to capture different events for delayed writes:
* when the delay happens with a pending fsync, or with a saving child
* active, and when the above two conditions are missing.
* We also use an additional event name to save all samples which is
* useful for graphing / monitoring purposes. */
if (sync_in_progress) {
latencyAddSampleIfNeeded("aof-write-pending-fsync",latency);
} else if (hasActiveChildProcess()) {
latencyAddSampleIfNeeded("aof-write-active-child",latency);
} else {
latencyAddSampleIfNeeded("aof-write-alone",latency);
}
latencyAddSampleIfNeeded("aof-write",latency);
/* We performed the write so reset the postponed flush sentinel to zero. */
server.aof_flush_postponed_start = 0;
if (nwritten != (ssize_t)sdslen(server.aof_buf)) {
static time_t last_write_error_log = 0;
int can_log = 0;
/* Limit logging rate to 1 line per AOF_WRITE_LOG_ERROR_RATE seconds. */
if ((server.unixtime - last_write_error_log) > AOF_WRITE_LOG_ERROR_RATE) {
can_log = 1;
last_write_error_log = server.unixtime;
}
/* Log the AOF write error and record the error code. */
if (nwritten == -1) {
if (can_log) {
serverLog(LL_WARNING,"Error writing to the AOF file: %s",
strerror(errno));
}
server.aof_last_write_errno = errno;
} else {
if (can_log) {
serverLog(LL_WARNING,"Short write while writing to "
"the AOF file: (nwritten=%lld, "
"expected=%lld)",
(long long)nwritten,
(long long)sdslen(server.aof_buf));
}
if (ftruncate(server.aof_fd, server.aof_last_incr_size) == -1) {
if (can_log) {
serverLog(LL_WARNING, "Could not remove short write "
"from the append-only file. Redis may refuse "
"to load the AOF the next time it starts. "
"ftruncate: %s", strerror(errno));
}
} else {
/* If the ftruncate() succeeded we can set nwritten to
* -1 since there is no longer partial data into the AOF. */
nwritten = -1;
}
server.aof_last_write_errno = ENOSPC;
}
/* Handle the AOF write error. */
if (server.aof_fsync == AOF_FSYNC_ALWAYS) {
/* We can't recover when the fsync policy is ALWAYS since the reply
* for the client is already in the output buffers (both writes and
* reads), and the changes to the db can't be rolled back. Since we
* have a contract with the user that on acknowledged or observed
* writes are is synced on disk, we must exit. */
serverLog(LL_WARNING,"Can't recover from AOF write error when the AOF fsync policy is 'always'. Exiting...");
exit(1);
} else {
/* Recover from failed write leaving data into the buffer. However
* set an error to stop accepting writes as long as the error
* condition is not cleared. */
server.aof_last_write_status = C_ERR;
/* Trim the sds buffer if there was a partial write, and there
* was no way to undo it with ftruncate(2). */
if (nwritten > 0) {
server.aof_current_size += nwritten;
server.aof_last_incr_size += nwritten;
sdsrange(server.aof_buf,nwritten,-1);
}
return; /* We'll try again on the next call... */
}
} else {
/* Successful write(2). If AOF was in error state, restore the
* OK state and log the event. */
if (server.aof_last_write_status == C_ERR) {
serverLog(LL_NOTICE,
"AOF write error looks solved, Redis can write again.");
server.aof_last_write_status = C_OK;
}
}
server.aof_current_size += nwritten;
server.aof_last_incr_size += nwritten;
/* Re-use AOF buffer when it is small enough. The maximum comes from the
* arena size of 4k minus some overhead (but is otherwise arbitrary). */
if ((sdslen(server.aof_buf)+sdsavail(server.aof_buf)) < 4000) {
sdsclear(server.aof_buf);
} else {
sdsfree(server.aof_buf);
server.aof_buf = sdsempty();
}
try_fsync:
/* Don't fsync if no-appendfsync-on-rewrite is set to yes and there are
* children doing I/O in the background. */
if (server.aof_no_fsync_on_rewrite && hasActiveChildProcess())
return;
/* Perform the fsync if needed. */
if (server.aof_fsync == AOF_FSYNC_ALWAYS) {
/* redis_fsync is defined as fdatasync() for Linux in order to avoid
* flushing metadata. */
latencyStartMonitor(latency);
/* Let's try to get this data on the disk. To guarantee data safe when
* the AOF fsync policy is 'always', we should exit if failed to fsync
* AOF (see comment next to the exit(1) after write error above). */
if (redis_fsync(server.aof_fd) == -1) {
serverLog(LL_WARNING,"Can't persist AOF for fsync error when the "
"AOF fsync policy is 'always': %s. Exiting...", strerror(errno));
exit(1);
}
latencyEndMonitor(latency);
latencyAddSampleIfNeeded("aof-fsync-always",latency);
server.aof_last_incr_fsync_offset = server.aof_last_incr_size;
server.aof_last_fsync = server.mstime;
atomicSet(server.fsynced_reploff_pending, server.master_repl_offset);
} else if (server.aof_fsync == AOF_FSYNC_EVERYSEC &&
server.mstime - server.aof_last_fsync >= 1000) {
if (!sync_in_progress) {
aof_background_fsync(server.aof_fd);
server.aof_last_incr_fsync_offset = server.aof_last_incr_size;
}
server.aof_last_fsync = server.mstime;
}
}
这段是三种 appendfsync 策略的实现:ALWAYS 每次 write() 后同步 redis_fsync()(Linux 上是 fdatasync,不刷元数据),失败直接 exit(1),最安全但每条命令都要等一次磁盘。EVERYSEC 只在距上次 fsync 超过 1000ms 时触发一次后台 fsync(aof_background_fsync),最多丢 1 秒、性能折中。NO 策略在 try_fsync 段根本没有分支,write() 完直接返回,把刷盘彻底交给 OS page cache,最快但丢多少全看 OS 何时刷。
AOF 的优点:数据安全性高(everysec 最多丢 1 秒),文件可读可编辑。
AOF 的缺点:恢复慢(逐条回放命令),文件持续膨胀。
停下来想一想:RDB 和 AOF 各自会丢多少数据?RDB 丢的是两次
bgsave之间的全部写入——若每半小时快照一次,最坏就是半小时的数据全没;AOF 用 everysec 时最多丢最近 1 秒的缓冲。同样是「丢数据」,量级为何差这么多?因为 RDB 是「定时拍快照」,快照之间没有任何日志兜底;而 AOF 是「每条命令都记日志、每秒刷一次盘」,未落盘窗口被 fsync 频率死死框在 1 秒内。这就是为什么对数据安全敏感的场景不能只开 RDB。
bgrewriteaof:AOF 重写
AOF 文件越来越大(同一条 key 反复修改就有多条命令),需要压缩。bgrewriteaof fork 子进程,按当前内存状态生成一个最小化的等效 AOF 文件:
重写前 AOF(膨胀版):
SET counter 1
SET counter 2
SET counter 3
重写后 AOF(精简版):
SET counter 3 ← 等价,但只有一条
重写期间父进程的新写入怎么办?
如果只让子进程重写,父进程的新写操作在子进程完成后才合并,那新写入就丢失了。 Redis 的做法是双管齐下:
重写期间:
父进程:
→ 继续写 AOF 旧文件(保证宕机后能从旧 AOF 恢复)
→ 同时写 aof_rewrite_buf(重写缓冲区,收集此期间的新写入)
子进程:
→ 按 fork 时的快照生成新 AOF 文件
子进程完成后通知父进程:
父进程将 aof_rewrite_buf 中的所有命令追加到新 AOF 文件末尾
→ rename(新AOF, 旧AOF) ← 原子替换
即使宕机,旧 AOF 文件始终完整有效。aof_rewrite_buf 和 aof_buf 是两个独立缓冲区——前者给重写的子进程兜底,后者给正常刷盘使用。
停下来想一想:为什么重写期间父进程要「同时写旧 AOF 和 aof_rewrite_buf」两处,而不是只写其中一处?若只写旧 AOF,子进程生成的新文件里就没有这期间的增量,rename 替换后这批写入就丢了;若只写 aof_rewrite_buf,一旦重写失败或中途宕机,旧 AOF 仍是重写前的状态、缺了兜底。两条腿走路的意义:旧 AOF 保证「任何时候宕机都能恢复」,rewrite_buf 保证「重写完成后新文件是最新的」。这正是 bgrewriteaof 不丢数据的核心。
bgrewriteaof 的底层实现 —— rewriteAppendOnlyFileBackground:
int rewriteAppendOnlyFileBackground(void) {
pid_t childpid;
if (hasActiveChildProcess()) return C_ERR;
if (server.backup_state == BACKUP_STATE_SNAPSHOTTING ||
server.backup_state == BACKUP_STATE_INCREMENTING)
{
return C_ERR;
}
if (dirCreateIfMissing(server.aof_dirname) == -1) {
serverLog(LL_WARNING, "Can't open or create append-only dir %s: %s",
server.aof_dirname, strerror(errno));
server.aof_lastbgrewrite_status = C_ERR;
return C_ERR;
}
/* We set aof_selected_db to -1 in order to force the next call to the
* feedAppendOnlyFile() to issue a SELECT command. */
server.aof_selected_db = -1;
flushAppendOnlyFile(1);
if (openNewIncrAofForAppend() != C_OK) {
server.aof_lastbgrewrite_status = C_ERR;
return C_ERR;
}
if (server.aof_state == AOF_WAIT_REWRITE) {
/* Wait for all bio jobs related to AOF to drain. This prevents a race
* between updates to `fsynced_reploff_pending` of the worker thread, belonging
* to the previous AOF, and the new one. This concern is specific for a full
* sync scenario where we don't wanna risk the ACKed replication offset
* jumping backwards or forward when switching to a different master. */
bioDrainWorker(BIO_AOF_FSYNC);
/* Set the initial repl_offset, which will be applied to fsynced_reploff
* when AOFRW finishes (after possibly being updated by a bio thread) */
atomicSet(server.fsynced_reploff_pending, server.master_repl_offset);
server.fsynced_reploff = 0;
}
server.stat_aof_rewrites++;
if ((childpid = redisFork(CHILD_TYPE_AOF)) == 0) {
char tmpfile[256];
/* Child */
redisSetProcTitle("redis-aof-rewrite");
redisSetCpuAffinity(server.aof_rewrite_cpulist);
snprintf(tmpfile,256,"temp-rewriteaof-bg-%d.aof", (int) getpid());
if (rewriteAppendOnlyFile(tmpfile) == C_OK) {
serverLog(LL_NOTICE,
"Successfully created the temporary AOF base file %s", tmpfile);
sendChildCowInfo(CHILD_INFO_TYPE_AOF_COW_SIZE, "AOF rewrite");
exitFromChild(0, 0);
} else {
exitFromChild(1, 0);
}
} else {
/* Parent */
if (childpid == -1) {
server.aof_lastbgrewrite_status = C_ERR;
serverLog(LL_WARNING,
"Can't rewrite append only file in background: fork: %s",
strerror(errno));
return C_ERR;
}
serverLog(LL_NOTICE,
"Background append only file rewriting started by pid %ld",(long) childpid);
server.aof_rewrite_scheduled = 0;
server.aof_rewrite_time_start = time(NULL);
return C_OK;
}
return C_OK; /* unreached */
}
bgrewriteaof 的底层实现是 rewriteAppendOnlyFileBackground:先 flushAppendOnlyFile(1) 把当前 AOF 缓冲区强制刷干净,再 openNewIncrAofForAppend() 打开一个新的增量 AOF 文件,最后才 redisFork()。子进程调 rewriteAppendOnlyFile(tmpfile) 按 fork 瞬间的内存快照生成最小化 BASE 文件,父进程立即返回继续服务。重写窗口期的新写入继续走 feedAppendOnlyFile 追加到新 INCR 文件里,所以一条命令都不会丢——这就是现代 Redis(本仓库 8.9)用增量 AOF 文件取代早期 aof_rewrite_buf 双缓冲区后的实现。
混合持久化(Redis 4.0+)
混合 AOF 文件结构:
[RDB 格式的全量快照 | AOF 格式的增量命令]
← 加载时先解析 RDB 快照(快),再逐条回放增量命令(少)
相当于把 RDB 和 AOF 的优点结合起来——全量用 RDB(恢复快),增量用 AOF(数据安全)。宕机恢复性能从 AOF 的分钟级降到 RDB 的秒级,同时数据安全性和 AOF 等同。
停下来想一想:为什么需要「混合持久化」这个第三选项,而不是让用户二选一?因为纯 RDB 快但窗口期丢数据,纯 AOF 安全但恢复要逐条回放、文件还膨胀,两者优点恰好互补:RDB 擅长「全量快照加载快」,AOF 擅长「增量日志不丢数据」。混合后文件 = RDB 全量快照 + 其后一段 AOF 增量,加载时先秒级恢复快照、再回放一点点增量命令,既拿到 RDB 的恢复速度,又保留 AOF 的数据安全。这是典型的「组合优于二选一」。
大实例的 fork 阻塞
fork 操作本身需要拷贝父进程的页表(不是数据,是指针结构)。32GB 内存的实例可能有 800 万+ 页表项,fork 拷贝页表可能阻塞 200-500ms。
优化:关闭 THP(Transparent Huge Pages)——THP 将 4KB 页面合并为 2MB 大页,但 COW 的粒度也从 4KB 变为 2MB,一次修改触发 512 倍的副本复制,严重放大写操作的内存开销。生产环境务必 echo never > /sys/kernel/mm/transparent_hugepage/enabled。
停下来想一想:fork 号称「复制进程」,为什么反而会卡住主进程?因为 fork 虽然不复制数据(靠 COW),但必须复制页表——32GB 内存可能有 800 万+ 页表项,逐项拷贝本身就是纯 CPU 操作,期间主进程无法执行。而 THP 把 4KB 页合成 2MB 大页,COW 复制粒度被放大 512 倍,父进程随便改一个字节都可能复制整 2MB,内存开销与停顿都被放大。所以生产上「关 THP」不是玄学,是实打实的 COW 成本控制。
总结
| RDB | AOF | 混合 | |
|---|---|---|---|
| 数据安全性 | 低(窗口期丢失) | 高(最多 1s) | 高 |
| 恢复速度 | 快(二进制加载) | 慢(逐条回放) | 快 |
| 文件体积 | 小 | 大(自动重写压缩) | 中 |
| 写入性能 | 快 | everysec 折中 | everysec |
| 适用场景 | 冷备份、主从同步 | 数据安全性要求高 | 生产环境推荐 |
章末提问
1. RDB 和 AOF 分别会丢多少数据?
结论先行:RDB 丢的是两次快照间隔内的全部写入(取决于 bgsave 周期,可能几分钟到半小时);AOF 在 everysec 下最多丢最近 1 秒。 因为:RDB 只在快照时刻持久化,两次快照之间没有任何日志兜底;AOF 每秒 fsync 一次,最多只有最近 1 秒的缓冲尚未落盘。
2. 为什么 AOF 是「写后日志」,而不是像 WAL 那样先写日志再执行?
结论先行:先执行再记日志,可以在写入前就校验命令合法性,避免把错误命令记进 AOF。 因为:Redis 在命令执行阶段已经完成校验(比如参数个数、类型),执行成功才追加到 aof_buf;若像 WAL 先记日志,恢复时就要额外处理「可能存在的非法命令」。代价是刚执行成功但还没刷盘的命令,在极端宕机时可能丢失——这也是 everysec 最多丢 1 秒的根源。
3. bgrewriteaof 重写期间父进程的新写入为什么不会丢?
结论先行:因为父进程同时写旧 AOF 和 aof_rewrite_buf 双缓冲,子进程完成后把增量追加到新文件再原子 rename。 因为:旧 AOF 保证任何时刻宕机都能从旧日志恢复;aof_rewrite_buf 收集重写窗口期的增量,子进程完成后父进程把这批命令追加到新文件末尾再 rename,保证新文件既「最小」又「最新」。现代 Redis 进一步用 BASE + INCR 增量 AOF 文件取代双缓冲区,原理等价。
4. 为什么 4.0 要引入混合持久化?
结论先行:为了同时拿到 RDB 的恢复速度和 AOF 的数据安全,把恢复从分钟级降到秒级。 因为:纯 RDB 快但窗口期丢数据,纯 AOF 安全但恢复慢;混合文件 = RDB 全量快照 + AOF 增量,加载先秒级恢复快照、再回放少量增量命令,两者优点互补。
5. 大实例上 fork 为什么会阻塞?怎么解决?
结论先行:fork 要拷贝页表(32GB 可能有 800 万+ 项),纯 CPU 操作会阻塞 200-500ms;优化是关闭 THP。
因为:fork 不复制数据(靠 COW),但必须复制页表指针结构,量大时主进程无法执行;THP 把 4KB 页合成 2MB 大页,使 COW 复制粒度放大 512 倍,进一步放大内存开销与停顿,生产环境应 echo never 关闭。