分布式ID生成方案详解与实战一、分布式ID概述在分布式系统中全局唯一ID是核心需求之一用于标识业务实体。1.1 ID生成要求特性说明唯一性全局唯一永不重复递增性按时间递增便于排序高性能高并发下低延迟可扩展性支持水平扩展安全性不可预测防止枚举攻击1.2 常见ID生成方案对比方案优点缺点适用场景数据库自增简单可靠单点瓶颈中小型系统UUID无中心依赖无序、过长对ID有序性要求低Snowflake高性能、有序依赖时钟分布式高并发Redis自增高性能需要部署Redis缓存场景数据库分段高可用实现复杂大型分布式系统二、Snowflake算法2.1 Snowflake结构┌─────────────────────────────────────────────────────────────────────┐ │ 64位Snowflake ID │ ├─────────────────────────────────────────────────────────────────────┤ │ 1位 │ 41位时间戳 │ 10位机器ID │ 12位序列号 │ │ 符号 │ (毫秒级) │ (5位数据中心5位工作节点) │ │ │ │ 可表示约69年 │ 支持1024个节点 │ 每毫秒4096个ID │ └─────────────────────────────────────────────────────────────────────┘2.2 Java实现public class SnowflakeIdGenerator { private final long twepoch 1609459200000L; // 2021-01-01 00:00:00 private final long workerIdBits 5L; private final long datacenterIdBits 5L; private final long maxWorkerId -1L ^ (-1L workerIdBits); private final long maxDatacenterId -1L ^ (-1L datacenterIdBits); private final long sequenceBits 12L; private final long workerIdShift sequenceBits; private final long datacenterIdShift sequenceBits workerIdBits; private final long timestampLeftShift sequenceBits workerIdBits datacenterIdBits; private final long sequenceMask -1L ^ (-1L sequenceBits); private long workerId; private long datacenterId; private long sequence 0L; private long lastTimestamp -1L; public SnowflakeIdGenerator(long workerId, long datacenterId) { if (workerId maxWorkerId || workerId 0) { throw new IllegalArgumentException(Worker ID out of range); } if (datacenterId maxDatacenterId || datacenterId 0) { throw new IllegalArgumentException(Datacenter ID out of range); } this.workerId workerId; this.datacenterId datacenterId; } public synchronized long nextId() { long timestamp timeGen(); if (timestamp lastTimestamp) { throw new RuntimeException(Clock moved backwards); } if (timestamp lastTimestamp) { sequence (sequence 1) sequenceMask; if (sequence 0) { timestamp tilNextMillis(lastTimestamp); } } else { sequence 0L; } lastTimestamp timestamp; return ((timestamp - twepoch) timestampLeftShift) | (datacenterId datacenterIdShift) | (workerId workerIdShift) | sequence; } private long timeGen() { return System.currentTimeMillis(); } private long tilNextMillis(long lastTimestamp) { long timestamp timeGen(); while (timestamp lastTimestamp) { timestamp timeGen(); } return timestamp; } }2.3 时钟回拨处理public synchronized long nextId() { long timestamp timeGen(); if (timestamp lastTimestamp) { long offset lastTimestamp - timestamp; if (offset 5) { try { wait(offset 1); timestamp timeGen(); if (timestamp lastTimestamp) { throw new RuntimeException(Clock moved backwards); } } catch (InterruptedException e) { throw new RuntimeException(e); } } else { throw new RuntimeException(Clock moved backwards too much); } } if (timestamp lastTimestamp) { sequence (sequence 1) sequenceMask; if (sequence 0) { timestamp tilNextMillis(lastTimestamp); } } else { sequence 0L; } lastTimestamp timestamp; return ((timestamp - twepoch) timestampLeftShift) | (datacenterId datacenterIdShift) | (workerId workerIdShift) | sequence; }三、数据库自增ID3.1 单库自增CREATE TABLE sequence ( id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT, biz_type VARCHAR(64) NOT NULL COMMENT 业务类型, create_time DATETIME DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (id), UNIQUE KEY uk_biz_type (biz_type) ) ENGINEInnoDB AUTO_INCREMENT100000 DEFAULT CHARSETutf8mb4;3.2 分段ID生成public class SegmentIdGenerator { private volatile long currentId; private volatile long maxId; private final Object lock new Object(); private final JdbcTemplate jdbcTemplate; public long nextId(String bizType) { if (currentId maxId) { synchronized (lock) { if (currentId maxId) { fetchSegment(bizType); } } } return currentId; } private void fetchSegment(String bizType) { String sql UPDATE sequence SET id LAST_INSERT_ID(id ?) WHERE biz_type ?; jdbcTemplate.update(sql, 1000, bizType); String querySql SELECT LAST_INSERT_ID(); long newId jdbcTemplate.queryForObject(querySql, Long.class); this.currentId newId; this.maxId newId 1000 - 1; } }3.3 多库多表方案┌─────────────────────────────────────────────────────────────────┐ │ ID生成器集群 │ ├─────────────────────────────────────────────────────────────────┤ │ DB1: sequence_0 DB2: sequence_1 DB3: sequence_2 │ │ step3, start0 step3, start1 step3, start2 │ │ IDs: 0,3,6,9... IDs: 1,4,7,10... IDs: 2,5,8,11... │ └─────────────────────────────────────────────────────────────────┘四、Redis自增ID4.1 基本实现public class RedisIdGenerator { private final StringRedisTemplate redisTemplate; private final String keyPrefix sequence:; private final long step 100; public long nextId(String bizType) { String key keyPrefix bizType; Long current redisTemplate.opsForValue().increment(key); if (current step) { redisTemplate.opsForValue().set(key, String.valueOf(current step * 10)); } return current; } }4.2 Lua脚本保证原子性local key KEYS[1] local step tonumber(ARGV[1]) local current redis.call(INCR, key) if current step then redis.call(SET, key, current step * 10) end return current五、UUID方案5.1 UUID生成// 标准UUID String uuid UUID.randomUUID().toString(); // 去除横线 String compactUuid uuid.replace(-, ); // 简化UUID前8位 String shortUuid compactUuid.substring(0, 8);5.2 基于时间戳的UUIDpublic class TimeBasedUUIDGenerator { public String generate() { long timestamp System.currentTimeMillis(); String timestampHex Long.toHexString(timestamp); String randomPart UUID.randomUUID().toString().substring(16); return timestampHex randomPart; } }六、美团Leaf方案6.1 Leaf架构┌─────────────────────────────────────────────────────────────┐ │ Leaf ID Generator │ ├─────────────────────────────────────────────────────────────┤ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ │ Segment │ │ Snowflake │ │ DB Backup │ │ │ │ Mode │ │ Mode │ │ │ │ │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ │ │ │ │ │ │ └──────────────────┴──────────────────┘ │ │ │ │ │ ▼ │ │ ┌───────────────┐ │ │ │ REST API │ │ │ └───────────────┘ │ └─────────────────────────────────────────────────────────────┘6.2 Segment模式配置leaf: segment: enable: true datasource: url: jdbc:mysql://localhost:3306/leaf?useUnicodetruecharacterEncodingutf-8 username: root password: password6.3 Snowflake模式配置leaf: snowflake: enable: true zk: address: localhost:2181 path: /leaf/snowflake七、实战建议7.1 方案选择策略场景推荐方案原因中小型系统数据库自增简单可靠无需额外组件高并发系统Snowflake高性能毫秒级生成数万ID跨数据中心Leaf/Snowflake支持多节点部署对ID有序性要求高SegmentID连续递增7.2 性能对比方案QPS延迟(ms)优点缺点Snowflake100万1极高性能依赖时钟Redis50万1-5高性能需要RedisSegment10万5-10ID连续依赖DBUUID10万1无依赖无序、长7.3 部署建议Worker ID分配通过配置中心或ZK自动分配时钟同步使用NTP服务同步时间容灾备份部署多节点支持故障转移监控告警监控ID生成速率、异常情况八、总结分布式ID生成方案需要根据业务场景选择Snowflake适合高并发、对ID有序性有要求的场景Segment适合需要连续ID的场景UUID适合对ID无特殊要求的场景Redis适合缓存场景或已有Redis集群的系统通过合理选择和配置可以构建可靠、高性能的分布式ID生成系统。
分布式ID生成方案详解与实战
分布式ID生成方案详解与实战一、分布式ID概述在分布式系统中全局唯一ID是核心需求之一用于标识业务实体。1.1 ID生成要求特性说明唯一性全局唯一永不重复递增性按时间递增便于排序高性能高并发下低延迟可扩展性支持水平扩展安全性不可预测防止枚举攻击1.2 常见ID生成方案对比方案优点缺点适用场景数据库自增简单可靠单点瓶颈中小型系统UUID无中心依赖无序、过长对ID有序性要求低Snowflake高性能、有序依赖时钟分布式高并发Redis自增高性能需要部署Redis缓存场景数据库分段高可用实现复杂大型分布式系统二、Snowflake算法2.1 Snowflake结构┌─────────────────────────────────────────────────────────────────────┐ │ 64位Snowflake ID │ ├─────────────────────────────────────────────────────────────────────┤ │ 1位 │ 41位时间戳 │ 10位机器ID │ 12位序列号 │ │ 符号 │ (毫秒级) │ (5位数据中心5位工作节点) │ │ │ │ 可表示约69年 │ 支持1024个节点 │ 每毫秒4096个ID │ └─────────────────────────────────────────────────────────────────────┘2.2 Java实现public class SnowflakeIdGenerator { private final long twepoch 1609459200000L; // 2021-01-01 00:00:00 private final long workerIdBits 5L; private final long datacenterIdBits 5L; private final long maxWorkerId -1L ^ (-1L workerIdBits); private final long maxDatacenterId -1L ^ (-1L datacenterIdBits); private final long sequenceBits 12L; private final long workerIdShift sequenceBits; private final long datacenterIdShift sequenceBits workerIdBits; private final long timestampLeftShift sequenceBits workerIdBits datacenterIdBits; private final long sequenceMask -1L ^ (-1L sequenceBits); private long workerId; private long datacenterId; private long sequence 0L; private long lastTimestamp -1L; public SnowflakeIdGenerator(long workerId, long datacenterId) { if (workerId maxWorkerId || workerId 0) { throw new IllegalArgumentException(Worker ID out of range); } if (datacenterId maxDatacenterId || datacenterId 0) { throw new IllegalArgumentException(Datacenter ID out of range); } this.workerId workerId; this.datacenterId datacenterId; } public synchronized long nextId() { long timestamp timeGen(); if (timestamp lastTimestamp) { throw new RuntimeException(Clock moved backwards); } if (timestamp lastTimestamp) { sequence (sequence 1) sequenceMask; if (sequence 0) { timestamp tilNextMillis(lastTimestamp); } } else { sequence 0L; } lastTimestamp timestamp; return ((timestamp - twepoch) timestampLeftShift) | (datacenterId datacenterIdShift) | (workerId workerIdShift) | sequence; } private long timeGen() { return System.currentTimeMillis(); } private long tilNextMillis(long lastTimestamp) { long timestamp timeGen(); while (timestamp lastTimestamp) { timestamp timeGen(); } return timestamp; } }2.3 时钟回拨处理public synchronized long nextId() { long timestamp timeGen(); if (timestamp lastTimestamp) { long offset lastTimestamp - timestamp; if (offset 5) { try { wait(offset 1); timestamp timeGen(); if (timestamp lastTimestamp) { throw new RuntimeException(Clock moved backwards); } } catch (InterruptedException e) { throw new RuntimeException(e); } } else { throw new RuntimeException(Clock moved backwards too much); } } if (timestamp lastTimestamp) { sequence (sequence 1) sequenceMask; if (sequence 0) { timestamp tilNextMillis(lastTimestamp); } } else { sequence 0L; } lastTimestamp timestamp; return ((timestamp - twepoch) timestampLeftShift) | (datacenterId datacenterIdShift) | (workerId workerIdShift) | sequence; }三、数据库自增ID3.1 单库自增CREATE TABLE sequence ( id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT, biz_type VARCHAR(64) NOT NULL COMMENT 业务类型, create_time DATETIME DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (id), UNIQUE KEY uk_biz_type (biz_type) ) ENGINEInnoDB AUTO_INCREMENT100000 DEFAULT CHARSETutf8mb4;3.2 分段ID生成public class SegmentIdGenerator { private volatile long currentId; private volatile long maxId; private final Object lock new Object(); private final JdbcTemplate jdbcTemplate; public long nextId(String bizType) { if (currentId maxId) { synchronized (lock) { if (currentId maxId) { fetchSegment(bizType); } } } return currentId; } private void fetchSegment(String bizType) { String sql UPDATE sequence SET id LAST_INSERT_ID(id ?) WHERE biz_type ?; jdbcTemplate.update(sql, 1000, bizType); String querySql SELECT LAST_INSERT_ID(); long newId jdbcTemplate.queryForObject(querySql, Long.class); this.currentId newId; this.maxId newId 1000 - 1; } }3.3 多库多表方案┌─────────────────────────────────────────────────────────────────┐ │ ID生成器集群 │ ├─────────────────────────────────────────────────────────────────┤ │ DB1: sequence_0 DB2: sequence_1 DB3: sequence_2 │ │ step3, start0 step3, start1 step3, start2 │ │ IDs: 0,3,6,9... IDs: 1,4,7,10... IDs: 2,5,8,11... │ └─────────────────────────────────────────────────────────────────┘四、Redis自增ID4.1 基本实现public class RedisIdGenerator { private final StringRedisTemplate redisTemplate; private final String keyPrefix sequence:; private final long step 100; public long nextId(String bizType) { String key keyPrefix bizType; Long current redisTemplate.opsForValue().increment(key); if (current step) { redisTemplate.opsForValue().set(key, String.valueOf(current step * 10)); } return current; } }4.2 Lua脚本保证原子性local key KEYS[1] local step tonumber(ARGV[1]) local current redis.call(INCR, key) if current step then redis.call(SET, key, current step * 10) end return current五、UUID方案5.1 UUID生成// 标准UUID String uuid UUID.randomUUID().toString(); // 去除横线 String compactUuid uuid.replace(-, ); // 简化UUID前8位 String shortUuid compactUuid.substring(0, 8);5.2 基于时间戳的UUIDpublic class TimeBasedUUIDGenerator { public String generate() { long timestamp System.currentTimeMillis(); String timestampHex Long.toHexString(timestamp); String randomPart UUID.randomUUID().toString().substring(16); return timestampHex randomPart; } }六、美团Leaf方案6.1 Leaf架构┌─────────────────────────────────────────────────────────────┐ │ Leaf ID Generator │ ├─────────────────────────────────────────────────────────────┤ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ │ Segment │ │ Snowflake │ │ DB Backup │ │ │ │ Mode │ │ Mode │ │ │ │ │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ │ │ │ │ │ │ └──────────────────┴──────────────────┘ │ │ │ │ │ ▼ │ │ ┌───────────────┐ │ │ │ REST API │ │ │ └───────────────┘ │ └─────────────────────────────────────────────────────────────┘6.2 Segment模式配置leaf: segment: enable: true datasource: url: jdbc:mysql://localhost:3306/leaf?useUnicodetruecharacterEncodingutf-8 username: root password: password6.3 Snowflake模式配置leaf: snowflake: enable: true zk: address: localhost:2181 path: /leaf/snowflake七、实战建议7.1 方案选择策略场景推荐方案原因中小型系统数据库自增简单可靠无需额外组件高并发系统Snowflake高性能毫秒级生成数万ID跨数据中心Leaf/Snowflake支持多节点部署对ID有序性要求高SegmentID连续递增7.2 性能对比方案QPS延迟(ms)优点缺点Snowflake100万1极高性能依赖时钟Redis50万1-5高性能需要RedisSegment10万5-10ID连续依赖DBUUID10万1无依赖无序、长7.3 部署建议Worker ID分配通过配置中心或ZK自动分配时钟同步使用NTP服务同步时间容灾备份部署多节点支持故障转移监控告警监控ID生成速率、异常情况八、总结分布式ID生成方案需要根据业务场景选择Snowflake适合高并发、对ID有序性有要求的场景Segment适合需要连续ID的场景UUID适合对ID无特殊要求的场景Redis适合缓存场景或已有Redis集群的系统通过合理选择和配置可以构建可靠、高性能的分布式ID生成系统。
