Skip to content
Go back

Redis RDB和AOF持久化——bgrewriteaof子进程如何不丢数据

Redis 持久化:RDB vs AOF vs 混合持久化

一句话结论(30s)

Redis 持久化的本质是在「恢复速度」与「数据安全」之间做取舍:RDB 二进制快照恢复最快但窗口期全丢,AOF 写后日志最安全但逐条回放慢。关键设计:因为 bgrewriteaof 子进程重写期间父进程仍会产生新写入,Redis 用「双缓冲区」——父进程既继续写旧 AOF 兜底、又写 aof_rewrite_buf 收集增量,子进程完成后父进程把增量追加到新文件再原子 rename。权衡:4.0 起的混合持久化(RDB 全量快照 + AOF 增量命令)把恢复性能从分钟级降到秒级、同时保留 AOF 级的数据安全,是生产推荐方案。

核心原理(2min)

RDB 通过 bgsave fork 子进程,利用 COW 共享内存生成二进制快照,完成后 rename 原子替换;AOF 记录每条写命令,按 appendfsync(always/everysec/no)刷盘,everysec 最多丢 1 秒。AOF 膨胀后用 bgrewriteaof 生成最小化文件:子进程按 fork 时快照重写,期间父进程的新写同时进入旧 AOF 和 aof_rewrite_buf,子进程完成后父进程把重写缓冲区追加到新文件再 rename。大实例 fork 本身拷贝页表可能阻塞 200-500ms,需关闭 THP 避免 COW 粒度从 4KB 放大到 2MB。混合持久化文件 = RDB 全量快照 + AOF 增量命令,加载先快后少。

底层深入(5-10min)

RDB:快照式持久化

Redis 执行 bgsave,fork 出一个子进程。子进程利用 COW(写时复制)共享父进程的全部内存数据,将内存快照序列化为一个二进制的 RDB 文件。

bgsave 流程:
  1. Redis 调用 fork()
  2. 子进程: 遍历所有 key,序列化为二进制 → 写入临时 RDB 文件
  3. 父进程: 继续处理客户端请求,修改内存数据触发 COW 复制新页
  4. 子进程写完 → rename(临时文件, dump.rdb) 原子替换

RDB 的 fork 入口 —— rdbSaveBackground

int rdbSaveBackground(int req, char *filename, rdbSaveInfo *rsi, int rdbflags) {
    pid_t childpid;

    if (hasActiveChildProcess()) return C_ERR;
    server.stat_rdb_saves++;

    server.dirty_before_bgsave = server.dirty;
    server.lastbgsave_try = time(NULL);

    if ((childpid = redisFork(CHILD_TYPE_RDB)) == 0) {
        int retval;

        /* Child */
        redisSetProcTitle("redis-rdb-bgsave");
        redisSetCpuAffinity(server.bgsave_cpulist);
        retval = rdbSave(req, filename,rsi,rdbflags);
        if (retval == C_OK) {
            sendChildCowInfo(CHILD_INFO_TYPE_RDB_COW_SIZE, "RDB");
        }
        exitFromChild((retval == C_OK) ? 0 : 1, 0);
    } else {
        /* Parent */
        if (childpid == -1) {
            server.lastbgsave_status = C_ERR;
            serverLog(LL_WARNING,"Can't save in background: fork: %s",
                strerror(errno));
            return C_ERR;
        }
        serverLog(LL_NOTICE,"Background saving started by pid %ld",(long) childpid);
        server.rdb_save_time_start = time(NULL);
        server.rdb_child_type = RDB_CHILD_TYPE_DISK;
        return C_OK;
    }
    return C_OK; /* unreached */
}

bgsave 的真正入口是 rdbSaveBackground:先用 hasActiveChildProcess() 保证同一时刻只 fork 一个子进程(避免多个子进程同时复制内存),再 redisFork() 复制一份当前进程。fork 返回 0 的是子进程,它直接调 rdbSave() 把数据落盘后 exitFromChild() 退出;返回子进程 pid 的是父进程,记个日志立刻返回继续服务客户端。父进程全程不阻塞,落盘都在子进程里做,这就是「后台保存」。

子进程落盘 —— rdbSave(临时文件 + 原子 rename)

int rdbSave(int req, char *filename, rdbSaveInfo *rsi, int rdbflags) {
    char tmpfile[256];
    char cwd[MAXPATHLEN]; /* Current working dir path for error messages. */

    startSaving(rdbflags);
    snprintf(tmpfile,256,"temp-%d.rdb", (int) getpid());

    if (rdbSaveInternal(req,tmpfile,rsi,rdbflags) != C_OK) {
        stopSaving(0);
        return C_ERR;
    }

    /* Use RENAME to make sure the DB file is changed atomically only
     * if the generate DB file is ok. */
    if (rename(tmpfile,filename) == -1) {
        char *str_err = strerror(errno);
        char *cwdp = getcwd(cwd,MAXPATHLEN);
        serverLog(LL_WARNING,
            "Error moving temp DB file %s on the final "
            "destination %s (in server root dir %s): %s",
            tmpfile,
            filename,
            cwdp ? cwdp : "unknown",
            str_err);
        unlink(tmpfile);
        stopSaving(0);
        return C_ERR;
    }
    if (fsyncFileDir(filename) != 0) {
        serverLog(LL_WARNING,
            "Failed to fsync directory while saving DB: %s", strerror(errno));
        stopSaving(0);
        return C_ERR;
    }

    serverLog(LL_NOTICE,"DB saved on disk");
    server.dirty = 0;
    server.lastsave = time(NULL);
    server.lastbgsave_status = C_OK;
    stopSaving(1);
    return C_OK;
}

子进程把快照写进临时文件 temp-<pid>.rdb,全部写成功后才 rename() 原子替换成正式的 dump.rdb,随后 fsyncFileDir() 刷一下目录项防止 rename 本身在断电时丢失。写临时文件再改名,保证 dump.rdb 要么是完整旧版本、要么是完整新版本,绝不会出现写一半的脏文件。这也是 RDB 恢复「快」的来源——直接反序列化二进制,不用逐条回放命令。

快照序列化 —— rdbSaveRio(COW 读内存)

int rdbSaveRio(int req, rio *rdb, int *error, int rdbflags, rdbSaveInfo *rsi) {
    char magic[10];
    uint64_t cksum;
    long key_counter = 0;
    unsigned long long skipped = 0;
    int j;

    if (server.rdb_checksum)
        rdb->update_cksum = rioGenericUpdateChecksum;
    snprintf(magic,sizeof(magic),"REDIS%04d",RDB_VERSION);
    if (rdbWriteRaw(rdb,magic,9) == -1) goto werr;
    if (rdbSaveInfoAuxFields(rdb,rdbflags,rsi) == -1) goto werr;
    if (!(req & SLAVE_REQ_RDB_EXCLUDE_DATA) && rdbSaveModulesAux(rdb, REDISMODULE_AUX_BEFORE_RDB) == -1) goto werr;

    /* save functions */
    if (!(req & SLAVE_REQ_RDB_EXCLUDE_FUNCTIONS) && rdbSaveFunctions(rdb) == -1) goto werr;

    /* Save the hash template registry (id -> field names). Template encoded
     * hashes are then written in compact REF form (id + values) by rdbSaveKeyValuePair. */
    if (!(req & SLAVE_REQ_RDB_EXCLUDE_DATA)) {
        if (rdbSaveHashTemplates(rdb) == -1) goto werr;
    }

    /* save all databases, skip this if we're in functions-only mode */
    if (!(req & SLAVE_REQ_RDB_EXCLUDE_DATA)) {
        for (j = 0; j < server.dbnum; j++) {
            if (rdbSaveDb(rdb, j, rdbflags, &key_counter, &skipped) == -1) goto werr;
            /* In standalone mode, dismiss bucket arrays of the saved DB's
             * kvstore to reduce CoW. In cluster mode this is done per-slot. */
            if (server.in_fork_child && !server.cluster_enabled)
                dismissKvstoreBucketsMemory(server.db[j].keys);
        }
    }

    if (!(req & SLAVE_REQ_RDB_EXCLUDE_DATA) && rdbSaveModulesAux(rdb, REDISMODULE_AUX_AFTER_RDB) == -1) goto werr;

    /* EOF opcode */
    if (rdbSaveType(rdb,RDB_OPCODE_EOF) == -1) goto werr;

    /* CRC64 checksum. It will be zero if checksum computation is disabled, the
     * loading code skips the check in this case. */
    cksum = rdb->cksum;
    memrev64ifbe(&cksum);
    if (rioWrite(rdb,&cksum,8) == 0) goto werr;
    serverLog(LL_NOTICE, "BGSAVE done, %ld keys saved, %llu keys skipped, %zu bytes written.", key_counter, skipped, rdb->processed_bytes);
    return C_OK;

werr:
    if (error) *error = errno;
    return C_ERR;
}

rdbSaveRio 才是真正的快照序列化:先写 REDIS 魔数 + 版本号,再写辅助字段、函数和 hash 模板,然后 for 循环遍历所有 db 调 rdbSaveDb() 把每个 key/value 变成二进制,最后写 EOF 操作码和 CRC64 校验和。整个过程跑在子进程里,靠 COW 与父进程共享内存页——父进程此刻的写入只会复制被改动的那几页,子进程读到的始终是 fork 瞬间的静态快照。代码里 dismissKvstoreBucketsMemory() 就是为了减少 COW 复制量做的优化。

RDB 的优点:二进制紧凑,恢复速度极快(直接反序列化到内存),适合冷备份和主从全量同步。

RDB 的缺点:快照间隔期间的写入在宕机时全丢。两次 bgsave 之间的窗口期(如半小时)内所有写入都消失。

AOF:写后日志

AOF 记录每一条修改 Redis 的写命令。先执行命令,再写入 AOF 缓冲区,最后刷到磁盘:

AOF 流程:
  客户端: SET key value
  Redis: 执行命令 → 写入 aof_buf
         → 根据 appendfsync 策略刷盘:
           always:   每条命令都 fsync — 最安全,性能最差
           everysec: 每秒 fsync — 丢最多 1 秒数据,性能折中
           no:       交给 OS — 最快,丢多少看 OS 心情

命令追加 —— feedAppendOnlyFile

void feedAppendOnlyFile(int dictid, robj **argv, int argc) {
    sds buf = sdsempty();

    serverAssert(dictid == -1 || (dictid >= 0 && dictid < server.dbnum));

    /* Feed timestamp if needed */
    if (server.aof_timestamp_enabled) {
        sds ts = genAofTimestampAnnotationIfNeeded(0);
        if (ts != NULL) {
            buf = sdscatsds(buf, ts);
            sdsfree(ts);
        }
    }

    /* The DB this command was targeting is not the same as the last command
     * we appended. To issue a SELECT command is needed. */
    if (dictid != -1 && dictid != server.aof_selected_db) {
        char seldb[64];

        snprintf(seldb,sizeof(seldb),"%d",dictid);
        buf = sdscatprintf(buf,"*2\r\n$6\r\nSELECT\r\n$%lu\r\n%s\r\n",
            (unsigned long)strlen(seldb),seldb);
        server.aof_selected_db = dictid;
    }

    /* All commands should be propagated the same way in AOF as in replication.
     * No need for AOF-specific translation. */
    buf = catAppendOnlyGenericCommand(buf,argc,argv);

    /* Append to the AOF buffer. This will be flushed on disk just before
     * of re-entering the event loop, so before the client will get a
     * positive reply about the operation performed. */
    if (server.aof_state == AOF_ON ||
        (server.aof_state == AOF_WAIT_REWRITE && server.child_type == CHILD_TYPE_AOF))
    {
        server.aof_buf = sdscatlen(server.aof_buf, buf, sdslen(buf));
    }

    sdsfree(buf);
}

每条写命令执行完都会走到 feedAppendOnlyFile:先按需补一条 SELECT 切换 db,再用 catAppendOnlyGenericCommand 把命令编码成 RESP 协议文本,最后追加到内存缓冲区 server.aof_buf。注意这里只写内存,注释写得很清楚——真正的磁盘写入要等「重新进入事件循环前」由 flushAppendOnlyFile 统一刷。这就是写后日志:命令先执行成功、再记到 AOF,与 WAL(先记日志再执行)正好相反。

刷盘策略 —— flushAppendOnlyFile(always / everysec / no)

void flushAppendOnlyFile(int force) {
    ssize_t nwritten;
    int sync_in_progress = 0;
    mstime_t latency;

    if (sdslen(server.aof_buf) == 0) {
        if (server.aof_last_incr_fsync_offset == server.aof_last_incr_size) {
            /* All data is fsync'd already: Update fsynced_reploff_pending just in case.
             * This is needed to avoid a WAITAOF hang in case a module used RM_Call
             * with the NO_AOF flag, in which case master_repl_offset will increase but
             * fsynced_reploff_pending won't be updated (because there's no reason, from
             * the AOF POV, to call fsync) and then WAITAOF may wait on the higher offset
             * (which contains data that was only propagated to replicas, and not to AOF) */
            if (!aofFsyncInProgress())
                atomicSet(server.fsynced_reploff_pending, server.master_repl_offset);
        } else {
            /* Check if we need to do fsync even the aof buffer is empty,
             * because previously in AOF_FSYNC_EVERYSEC mode, fsync is
             * called only when aof buffer is not empty, so if users
             * stop write commands before fsync called in one second,
             * the data in page cache cannot be flushed in time. */
            if (server.aof_fsync == AOF_FSYNC_EVERYSEC &&
                server.mstime - server.aof_last_fsync >= 1000 &&
                !(sync_in_progress = aofFsyncInProgress()))
                goto try_fsync;

            /* Check if we need to do fsync even the aof buffer is empty,
             * the reason is described in the previous AOF_FSYNC_EVERYSEC block,
             * and AOF_FSYNC_ALWAYS is also checked here to handle a case where
             * aof_fsync is changed from everysec to always. */
            if (server.aof_fsync == AOF_FSYNC_ALWAYS)
                goto try_fsync;
        }
        return;
    }

    if (server.aof_fsync == AOF_FSYNC_EVERYSEC)
        sync_in_progress = aofFsyncInProgress();

    if (server.aof_fsync == AOF_FSYNC_EVERYSEC && !force) {
        /* With this append fsync policy we do background fsyncing.
         * If the fsync is still in progress we can try to delay
         * the write for a couple of seconds. */
        if (sync_in_progress) {
            if (server.aof_flush_postponed_start == 0) {
                /* No previous write postponing, remember that we are
                 * postponing the flush and return. */
                server.aof_flush_postponed_start = server.mstime;
                return;
            } else if (server.mstime - server.aof_flush_postponed_start < 2000) {
                /* We were already waiting for fsync to finish, but for less
                 * than two seconds this is still ok. Postpone again. */
                return;
            }
            /* Otherwise fall through, and go write since we can't wait
             * over two seconds. */
            server.aof_delayed_fsync++;
            serverLog(LL_NOTICE,"Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down Redis.");
        }
    }
    /* We want to perform a single write. This should be guaranteed atomic
     * at least if the filesystem we are writing is a real physical one.
     * While this will save us against the server being killed I don't think
     * there is much to do about the whole server stopping for power problems
     * or alike */

    if (server.aof_flush_sleep && sdslen(server.aof_buf)) {
        usleep(server.aof_flush_sleep);
    }

    latencyStartMonitor(latency);
    nwritten = aofWrite(server.aof_fd,server.aof_buf,sdslen(server.aof_buf));
    latencyEndMonitor(latency);
    /* We want to capture different events for delayed writes:
     * when the delay happens with a pending fsync, or with a saving child
     * active, and when the above two conditions are missing.
     * We also use an additional event name to save all samples which is
     * useful for graphing / monitoring purposes. */
    if (sync_in_progress) {
        latencyAddSampleIfNeeded("aof-write-pending-fsync",latency);
    } else if (hasActiveChildProcess()) {
        latencyAddSampleIfNeeded("aof-write-active-child",latency);
    } else {
        latencyAddSampleIfNeeded("aof-write-alone",latency);
    }
    latencyAddSampleIfNeeded("aof-write",latency);

    /* We performed the write so reset the postponed flush sentinel to zero. */
    server.aof_flush_postponed_start = 0;

    if (nwritten != (ssize_t)sdslen(server.aof_buf)) {
        static time_t last_write_error_log = 0;
        int can_log = 0;

        /* Limit logging rate to 1 line per AOF_WRITE_LOG_ERROR_RATE seconds. */
        if ((server.unixtime - last_write_error_log) > AOF_WRITE_LOG_ERROR_RATE) {
            can_log = 1;
            last_write_error_log = server.unixtime;
        }

        /* Log the AOF write error and record the error code. */
        if (nwritten == -1) {
            if (can_log) {
                serverLog(LL_WARNING,"Error writing to the AOF file: %s",
                    strerror(errno));
            }
            server.aof_last_write_errno = errno;
        } else {
            if (can_log) {
                serverLog(LL_WARNING,"Short write while writing to "
                                       "the AOF file: (nwritten=%lld, "
                                       "expected=%lld)",
                                       (long long)nwritten,
                                       (long long)sdslen(server.aof_buf));
            }

            if (ftruncate(server.aof_fd, server.aof_last_incr_size) == -1) {
                if (can_log) {
                    serverLog(LL_WARNING, "Could not remove short write "
                             "from the append-only file.  Redis may refuse "
                             "to load the AOF the next time it starts.  "
                             "ftruncate: %s", strerror(errno));
                }
            } else {
                /* If the ftruncate() succeeded we can set nwritten to
                 * -1 since there is no longer partial data into the AOF. */
                nwritten = -1;
            }
            server.aof_last_write_errno = ENOSPC;
        }

        /* Handle the AOF write error. */
        if (server.aof_fsync == AOF_FSYNC_ALWAYS) {
            /* We can't recover when the fsync policy is ALWAYS since the reply
             * for the client is already in the output buffers (both writes and
             * reads), and the changes to the db can't be rolled back. Since we
             * have a contract with the user that on acknowledged or observed
             * writes are is synced on disk, we must exit. */
            serverLog(LL_WARNING,"Can't recover from AOF write error when the AOF fsync policy is 'always'. Exiting...");
            exit(1);
        } else {
            /* Recover from failed write leaving data into the buffer. However
             * set an error to stop accepting writes as long as the error
             * condition is not cleared. */
            server.aof_last_write_status = C_ERR;

            /* Trim the sds buffer if there was a partial write, and there
             * was no way to undo it with ftruncate(2). */
            if (nwritten > 0) {
                server.aof_current_size += nwritten;
                server.aof_last_incr_size += nwritten;
                sdsrange(server.aof_buf,nwritten,-1);
            }
            return; /* We'll try again on the next call... */
        }
    } else {
        /* Successful write(2). If AOF was in error state, restore the
         * OK state and log the event. */
        if (server.aof_last_write_status == C_ERR) {
            serverLog(LL_NOTICE,
                "AOF write error looks solved, Redis can write again.");
            server.aof_last_write_status = C_OK;
        }
    }
    server.aof_current_size += nwritten;
    server.aof_last_incr_size += nwritten;

    /* Re-use AOF buffer when it is small enough. The maximum comes from the
     * arena size of 4k minus some overhead (but is otherwise arbitrary). */
    if ((sdslen(server.aof_buf)+sdsavail(server.aof_buf)) < 4000) {
        sdsclear(server.aof_buf);
    } else {
        sdsfree(server.aof_buf);
        server.aof_buf = sdsempty();
    }

try_fsync:
    /* Don't fsync if no-appendfsync-on-rewrite is set to yes and there are
     * children doing I/O in the background. */
    if (server.aof_no_fsync_on_rewrite && hasActiveChildProcess())
        return;

    /* Perform the fsync if needed. */
    if (server.aof_fsync == AOF_FSYNC_ALWAYS) {
        /* redis_fsync is defined as fdatasync() for Linux in order to avoid
         * flushing metadata. */
        latencyStartMonitor(latency);
        /* Let's try to get this data on the disk. To guarantee data safe when
         * the AOF fsync policy is 'always', we should exit if failed to fsync
         * AOF (see comment next to the exit(1) after write error above). */
        if (redis_fsync(server.aof_fd) == -1) {
            serverLog(LL_WARNING,"Can't persist AOF for fsync error when the "
              "AOF fsync policy is 'always': %s. Exiting...", strerror(errno));
            exit(1);
        }
        latencyEndMonitor(latency);
        latencyAddSampleIfNeeded("aof-fsync-always",latency);
        server.aof_last_incr_fsync_offset = server.aof_last_incr_size;
        server.aof_last_fsync = server.mstime;
        atomicSet(server.fsynced_reploff_pending, server.master_repl_offset);
    } else if (server.aof_fsync == AOF_FSYNC_EVERYSEC &&
               server.mstime - server.aof_last_fsync >= 1000) {
        if (!sync_in_progress) {
            aof_background_fsync(server.aof_fd);
            server.aof_last_incr_fsync_offset = server.aof_last_incr_size;
        }
        server.aof_last_fsync = server.mstime;
    }
}

这段是三种 appendfsync 策略的实现:ALWAYS 每次 write() 后同步 redis_fsync()(Linux 上是 fdatasync,不刷元数据),失败直接 exit(1),最安全但每条命令都要等一次磁盘。EVERYSEC 只在距上次 fsync 超过 1000ms 时触发一次后台 fsync(aof_background_fsync),最多丢 1 秒、性能折中。NO 策略在 try_fsync 段根本没有分支,write() 完直接返回,把刷盘彻底交给 OS page cache,最快但丢多少全看 OS 何时刷。

AOF 的优点:数据安全性高(everysec 最多丢 1 秒),文件可读可编辑。

AOF 的缺点:恢复慢(逐条回放命令),文件持续膨胀。

停下来想一想:RDB 和 AOF 各自会丢多少数据?RDB 丢的是两次 bgsave 之间的全部写入——若每半小时快照一次,最坏就是半小时的数据全没;AOF 用 everysec 时最多丢最近 1 秒的缓冲。同样是「丢数据」,量级为何差这么多?因为 RDB 是「定时拍快照」,快照之间没有任何日志兜底;而 AOF 是「每条命令都记日志、每秒刷一次盘」,未落盘窗口被 fsync 频率死死框在 1 秒内。这就是为什么对数据安全敏感的场景不能只开 RDB。

bgrewriteaof:AOF 重写

AOF 文件越来越大(同一条 key 反复修改就有多条命令),需要压缩。bgrewriteaof fork 子进程,按当前内存状态生成一个最小化的等效 AOF 文件:

重写前 AOF(膨胀版):
SET counter 1
SET counter 2
SET counter 3

重写后 AOF(精简版):
SET counter 3              ← 等价,但只有一条

重写期间父进程的新写入怎么办?

如果只让子进程重写,父进程的新写操作在子进程完成后才合并,那新写入就丢失了。 Redis 的做法是双管齐下:

重写期间:
  父进程:
    → 继续写 AOF 旧文件(保证宕机后能从旧 AOF 恢复)
    → 同时写 aof_rewrite_buf(重写缓冲区,收集此期间的新写入)

  子进程:
    → 按 fork 时的快照生成新 AOF 文件

  子进程完成后通知父进程:
    父进程将 aof_rewrite_buf 中的所有命令追加到新 AOF 文件末尾
    → rename(新AOF, 旧AOF)  ← 原子替换

即使宕机,旧 AOF 文件始终完整有效。aof_rewrite_bufaof_buf 是两个独立缓冲区——前者给重写的子进程兜底,后者给正常刷盘使用。

停下来想一想:为什么重写期间父进程要「同时写旧 AOF 和 aof_rewrite_buf」两处,而不是只写其中一处?若只写旧 AOF,子进程生成的新文件里就没有这期间的增量,rename 替换后这批写入就丢了;若只写 aof_rewrite_buf,一旦重写失败或中途宕机,旧 AOF 仍是重写前的状态、缺了兜底。两条腿走路的意义:旧 AOF 保证「任何时候宕机都能恢复」,rewrite_buf 保证「重写完成后新文件是最新的」。这正是 bgrewriteaof 不丢数据的核心。

bgrewriteaof 的底层实现 —— rewriteAppendOnlyFileBackground

int rewriteAppendOnlyFileBackground(void) {
    pid_t childpid;

    if (hasActiveChildProcess()) return C_ERR;
    if (server.backup_state == BACKUP_STATE_SNAPSHOTTING ||
        server.backup_state == BACKUP_STATE_INCREMENTING)
    {
        return C_ERR;
    }

    if (dirCreateIfMissing(server.aof_dirname) == -1) {
        serverLog(LL_WARNING, "Can't open or create append-only dir %s: %s",
            server.aof_dirname, strerror(errno));
        server.aof_lastbgrewrite_status = C_ERR;
        return C_ERR;
    }

    /* We set aof_selected_db to -1 in order to force the next call to the
     * feedAppendOnlyFile() to issue a SELECT command. */
    server.aof_selected_db = -1;
    flushAppendOnlyFile(1);
    if (openNewIncrAofForAppend() != C_OK) {
        server.aof_lastbgrewrite_status = C_ERR;
        return C_ERR;
    }

    if (server.aof_state == AOF_WAIT_REWRITE) {
        /* Wait for all bio jobs related to AOF to drain. This prevents a race
         * between updates to `fsynced_reploff_pending` of the worker thread, belonging
         * to the previous AOF, and the new one. This concern is specific for a full
         * sync scenario where we don't wanna risk the ACKed replication offset
         * jumping backwards or forward when switching to a different master. */
        bioDrainWorker(BIO_AOF_FSYNC);

        /* Set the initial repl_offset, which will be applied to fsynced_reploff
         * when AOFRW finishes (after possibly being updated by a bio thread) */
        atomicSet(server.fsynced_reploff_pending, server.master_repl_offset);
        server.fsynced_reploff = 0;
    }

    server.stat_aof_rewrites++;

    if ((childpid = redisFork(CHILD_TYPE_AOF)) == 0) {
        char tmpfile[256];

        /* Child */
        redisSetProcTitle("redis-aof-rewrite");
        redisSetCpuAffinity(server.aof_rewrite_cpulist);
        snprintf(tmpfile,256,"temp-rewriteaof-bg-%d.aof", (int) getpid());
        if (rewriteAppendOnlyFile(tmpfile) == C_OK) {
            serverLog(LL_NOTICE,
                "Successfully created the temporary AOF base file %s", tmpfile);
            sendChildCowInfo(CHILD_INFO_TYPE_AOF_COW_SIZE, "AOF rewrite");
            exitFromChild(0, 0);
        } else {
            exitFromChild(1, 0);
        }
    } else {
        /* Parent */
        if (childpid == -1) {
            server.aof_lastbgrewrite_status = C_ERR;
            serverLog(LL_WARNING,
                "Can't rewrite append only file in background: fork: %s",
                strerror(errno));
            return C_ERR;
        }
        serverLog(LL_NOTICE,
            "Background append only file rewriting started by pid %ld",(long) childpid);
        server.aof_rewrite_scheduled = 0;
        server.aof_rewrite_time_start = time(NULL);
        return C_OK;
    }
    return C_OK; /* unreached */
}

bgrewriteaof 的底层实现是 rewriteAppendOnlyFileBackground:先 flushAppendOnlyFile(1) 把当前 AOF 缓冲区强制刷干净,再 openNewIncrAofForAppend() 打开一个新的增量 AOF 文件,最后才 redisFork()。子进程调 rewriteAppendOnlyFile(tmpfile) 按 fork 瞬间的内存快照生成最小化 BASE 文件,父进程立即返回继续服务。重写窗口期的新写入继续走 feedAppendOnlyFile 追加到新 INCR 文件里,所以一条命令都不会丢——这就是现代 Redis(本仓库 8.9)用增量 AOF 文件取代早期 aof_rewrite_buf 双缓冲区后的实现。

混合持久化(Redis 4.0+)

混合 AOF 文件结构:
  [RDB 格式的全量快照 | AOF 格式的增量命令]
  ← 加载时先解析 RDB 快照(快),再逐条回放增量命令(少)

相当于把 RDB 和 AOF 的优点结合起来——全量用 RDB(恢复快),增量用 AOF(数据安全)。宕机恢复性能从 AOF 的分钟级降到 RDB 的秒级,同时数据安全性和 AOF 等同。

停下来想一想:为什么需要「混合持久化」这个第三选项,而不是让用户二选一?因为纯 RDB 快但窗口期丢数据,纯 AOF 安全但恢复要逐条回放、文件还膨胀,两者优点恰好互补:RDB 擅长「全量快照加载快」,AOF 擅长「增量日志不丢数据」。混合后文件 = RDB 全量快照 + 其后一段 AOF 增量,加载时先秒级恢复快照、再回放一点点增量命令,既拿到 RDB 的恢复速度,又保留 AOF 的数据安全。这是典型的「组合优于二选一」。

大实例的 fork 阻塞

fork 操作本身需要拷贝父进程的页表(不是数据,是指针结构)。32GB 内存的实例可能有 800 万+ 页表项,fork 拷贝页表可能阻塞 200-500ms。

优化:关闭 THP(Transparent Huge Pages)——THP 将 4KB 页面合并为 2MB 大页,但 COW 的粒度也从 4KB 变为 2MB,一次修改触发 512 倍的副本复制,严重放大写操作的内存开销。生产环境务必 echo never > /sys/kernel/mm/transparent_hugepage/enabled

停下来想一想:fork 号称「复制进程」,为什么反而会卡住主进程?因为 fork 虽然不复制数据(靠 COW),但必须复制页表——32GB 内存可能有 800 万+ 页表项,逐项拷贝本身就是纯 CPU 操作,期间主进程无法执行。而 THP 把 4KB 页合成 2MB 大页,COW 复制粒度被放大 512 倍,父进程随便改一个字节都可能复制整 2MB,内存开销与停顿都被放大。所以生产上「关 THP」不是玄学,是实打实的 COW 成本控制。

总结

RDBAOF混合
数据安全性低(窗口期丢失)高(最多 1s)
恢复速度快(二进制加载)慢(逐条回放)
文件体积大(自动重写压缩)
写入性能everysec 折中everysec
适用场景冷备份、主从同步数据安全性要求高生产环境推荐

章末提问

1. RDB 和 AOF 分别会丢多少数据?

结论先行:RDB 丢的是两次快照间隔内的全部写入(取决于 bgsave 周期,可能几分钟到半小时);AOF 在 everysec 下最多丢最近 1 秒。 因为:RDB 只在快照时刻持久化,两次快照之间没有任何日志兜底;AOF 每秒 fsync 一次,最多只有最近 1 秒的缓冲尚未落盘。

2. 为什么 AOF 是「写后日志」,而不是像 WAL 那样先写日志再执行?

结论先行:先执行再记日志,可以在写入前就校验命令合法性,避免把错误命令记进 AOF。 因为:Redis 在命令执行阶段已经完成校验(比如参数个数、类型),执行成功才追加到 aof_buf;若像 WAL 先记日志,恢复时就要额外处理「可能存在的非法命令」。代价是刚执行成功但还没刷盘的命令,在极端宕机时可能丢失——这也是 everysec 最多丢 1 秒的根源。

3. bgrewriteaof 重写期间父进程的新写入为什么不会丢?

结论先行:因为父进程同时写旧 AOF 和 aof_rewrite_buf 双缓冲,子进程完成后把增量追加到新文件再原子 rename。 因为:旧 AOF 保证任何时刻宕机都能从旧日志恢复;aof_rewrite_buf 收集重写窗口期的增量,子进程完成后父进程把这批命令追加到新文件末尾再 rename,保证新文件既「最小」又「最新」。现代 Redis 进一步用 BASE + INCR 增量 AOF 文件取代双缓冲区,原理等价。

4. 为什么 4.0 要引入混合持久化?

结论先行:为了同时拿到 RDB 的恢复速度和 AOF 的数据安全,把恢复从分钟级降到秒级。 因为:纯 RDB 快但窗口期丢数据,纯 AOF 安全但恢复慢;混合文件 = RDB 全量快照 + AOF 增量,加载先秒级恢复快照、再回放少量增量命令,两者优点互补。

5. 大实例上 fork 为什么会阻塞?怎么解决?

结论先行:fork 要拷贝页表(32GB 可能有 800 万+ 项),纯 CPU 操作会阻塞 200-500ms;优化是关闭 THP。 因为:fork 不复制数据(靠 COW),但必须复制页表指针结构,量大时主进程无法执行;THP 把 4KB 页合成 2MB 大页,使 COW 复制粒度放大 512 倍,进一步放大内存开销与停顿,生产环境应 echo never 关闭。


Share this post on:

Previous Post
Redis SDS 与 embstr 的 44 字节阈值:从 sdshdr 到 jemalloc 分配链
Next Post
Redis GEO 地理位置:GeoHash 编码与 Sorted Set 的优雅结合